fix: OpenRouter 402 — send max_tokens cost cap; make session window configurable
gates / consistency-and-conventions (push) Successful in 32s

User hit 'API error 402: ... You requested up to 131072 tokens, but can
only afford 4511' on the assist alias: no provider ever sent max_tokens,
so OpenRouter's credit pre-check billed the routed model's full
worst-case output; user also asked to bound session history to the last
5 requests/responses.

Architect decisions:
- AI_MAX_TOKENS (num, default 2048): sent as max_tokens on OpenRouter
  and generationConfig.maxOutputTokens on Gemini — a real per-request
  cost ceiling. llamacpp unchanged (local/free, no pre-check).
- AI_SESSION_TURNS (num, default 40 kept back-compat; messages, 2 per
  exchange — 10 = last 5 conversations): resolved lazily in session_push
  because config loads after the hardcoded line-25 default.
- Both registered in the bin/pos-ai POS_CONFIG @General section, so they
  appear in 'pos config ai' with num: validation.

Reviewer hardening (CHANGES_REQUIRED -> fixed): unguarded env input could
reach jq tonumber (0/-5/010/abc all savable via config-ui's ^-?[0-9]+$)
and abort the CLI; both providers and session_push now guard with
^[1-9][0-9]*$ and fall back to the default.

Verified: fake-curl shim smoke (16 provider-body + 12 session-window
checks incl. the 010-regression proof), make gen idempotent, make check
OK, make lint 0 FAIL/0 WARN, make test 17 files / 299 checks / 0 fail
(~49s), bash -n clean, git diff --check clean. Reviewer ACCEPT (twice).

Tester regression round (permanent provider-body + session-pruning
coverage) intentionally not run this cycle — user's call; remains a
documented follow-up.
This commit is contained in:
Your Name
2026-09-07 07:25:38 -04:00
parent 8ce54794ee
commit 01aa7f3e8f
12 changed files with 887 additions and 11 deletions
+7 -3
View File
@@ -3,7 +3,7 @@ set -euo pipefail
# POS: ai ask — AI assistant: ask, chat, sessions, capture, models, providers
# POS_SUBCMDS: ask chat sessions capture models providers llamacpp
# POS_FLAGS: --provider --model --session --system --full --last --trust --no-command-execution
# POS_CONFIG: ai | ai.env | AI_PROVIDER=:Provider (gemini, openrouter or llamacpp, default gemini) | @[AI_PROVIDER=gemini|] Gemini | *providers=gemini | @[AI_PROVIDER=openrouter] OpenRouter | *providers=openrouter | @[AI_PROVIDER=llamacpp] llamacpp | LLAMACPP_PORT=:Server port (default 8088) | LLAMACPP_HOST=:Bind address (default 127.0.0.1) | LLAMACPP_MODEL=:Default model path (GGUF) | LLAMACPP_CTX_SIZE=num:Context window size (default 4096) | LLAMACPP_GPU_LAYERS=num:GPU layers (-1=auto, 0=CPU, default -1) | LLAMACPP_THREADS=num:CPU threads (default: nproc) | *providers=llamacpp | @General | AI_SYSTEM_PROMPT=:Custom system prompt (overrides built-in, empty to reset)
# POS_CONFIG: ai | ai.env | AI_PROVIDER=:Provider (gemini, openrouter or llamacpp, default gemini) | @[AI_PROVIDER=gemini|] Gemini | *providers=gemini | @[AI_PROVIDER=openrouter] OpenRouter | *providers=openrouter | @[AI_PROVIDER=llamacpp] llamacpp | LLAMACPP_PORT=:Server port (default 8088) | LLAMACPP_HOST=:Bind address (default 127.0.0.1) | LLAMACPP_MODEL=:Default model path (GGUF) | LLAMACPP_CTX_SIZE=num:Context window size (default 4096) | LLAMACPP_GPU_LAYERS=num:GPU layers (-1=auto, 0=CPU, default -1) | LLAMACPP_THREADS=num:CPU threads (default: nproc) | *providers=llamacpp | @General | AI_SYSTEM_PROMPT=:Custom system prompt (overrides built-in, empty to reset) | AI_MAX_TOKENS=num:Max output tokens per request (default 2048; OpenRouter/Gemini cost cap) | AI_SESSION_TURNS=num:Session message cap — 2 per exchange (default 40 = 20 exchanges; 10 = last 5)
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
@@ -91,6 +91,8 @@ Options:
Config: $CONFIG_FILE (edit with 'pos config ai')
AI_PROVIDER Provider to use (gemini|openrouter|llamacpp, default gemini)
AI_SYSTEM_PROMPT Custom system prompt (overrides built-in; empty to reset)
AI_MAX_TOKENS Max output tokens per request (default 2048; OpenRouter/Gemini cost cap)
AI_SESSION_TURNS Session message cap — 2 per exchange (default 40 = 20 exchanges; 10 = last 5)
Provider keys: auto-discovered from lib/ai-providers/*.sh
(AI_GEMINI_API_KEY, OPENROUTER_API_KEY, etc.)
@@ -285,8 +287,10 @@ session_save() {
# Append a turn and prune to the last MAX_SESSION_TURNS entries. stdout = JSON.
session_push() {
local messages="$1" role="$2" text="$3"
printf '%s' "$messages" | jq -c --arg r "$role" --arg t "$text" \
'.messages += [{role:$r, content:$t}] | .messages |= .[-'"$MAX_SESSION_TURNS"':]'
local n="${AI_SESSION_TURNS:-$MAX_SESSION_TURNS}"
[[ "$n" =~ ^[1-9][0-9]*$ ]] || n="$MAX_SESSION_TURNS"
printf '%s' "$messages" | jq -c --arg r "$role" --arg t "$text" --argjson n "$n" \
'.messages += [{role:$r, content:$t}] | .messages |= .[-$n:]'
}
# ── Terminal markdown rendering (tty-only; raw bytes otherwise) ──