Your Name
8ce54794ee
gates / consistency-and-conventions (push) Successful in 27s
User report: pressing Enter on 'System prompt (empty = use built-in)' silently returned to the menu — no alias created, and step labels read [1/4] [2/4] in a 5-step flow. Detective (pre-existing, not a 2026-09-06 regression): menu_ask_value's documented contract is 'rc 1 = cancel, or empty answer with no default'; the step-4 call passed an empty default so the advertised empty answer hit rc 1 and '|| return 0' aborted the flow. Same latent trap at the alias-name step (empty-name warn/re-prompt was dead code). 11 other call sites are correct (6 external rely on empty=cancel, 4 pass defaults) — no global semantic change allowed. Architect: opt-in --allow-empty flag on menu_ask_value (backward compatible; empty+no-default -> rc 0 + empty value; genuine cancel/EOF stays rc 1; default still wins). Builder: implemented in lib/menu-lib.sh + bin/pos-ai-alias (steps 1-2 relabeled /5, flag at the two approved sites); 7-case smoke matrix PASS. Tests: tests/t-menu-allow-empty.sh (30 checks) — semantics matrix against the real menu_ask_value via non-TTY stdin, reader-contract probes (empty-Enter rc 0 vs EOF rc 1), static guards on step labeling, the exactly-2 flag call sites, edit-flow untouched, and a scope fence over all pos-* tools. Pty E2E proven feasible (script -qec, 3 scenarios) and documented in the Tester report; the E2E file itself remains a follow-up. Verified: make gen idempotent; make check OK; make lint 0 FAIL, 0 WARN; make test 17 files / 299 checks / 0 fail / 0 skip (~49s); bash -n clean.
4.8 KiB
4.8 KiB
Tester Report — Alias-Menu Fix: Regression Tests (2026-09-06)
TL;DR
- Status: IN PROGRESS — see per-step markers below; final summary lands in Step 5/6.
- Scope: permanent regression coverage for Builder's alias-menu fix (lib/menu-lib.sh
--allow-empty+ bin/pos-ai-alias 353/410 + step counters 352/383 → /5). Tests only; no source/doc edits. - Methods used: (1) unit matrix on the real
menu_ask_valuevia the non-TTY stdin path, (2) static source guards via brace-extracted function bodies (sameextract_fnpattern as t-ai-server-validate.sh), (3) pty E2E throughscript -qecdriving the REALpos ai alias createflow. - Pty verdict: FEASIBLE.
script -qecwith paced input (0.3s inter-input sleep) drives the raw-mode reader deterministically. 3 E2E scenarios proven. - Files added:
tests/t-menu-allow-empty.sh,tests/t-ai-alias-create-e2e.sh.
Step 1: Pty feasibility experiment [DONE]
Attempted script -qec with a piped script of inputs per DEV.md §7 (printf 'answer\n' | script -qec "cmd" /dev/null).
Evidence (probe A — the reported bug scenario):
printf 'testbot\n1\n\n\n\ny\n' (paced 0.4s) | script -qec "$ROOT/bin/pos-ai-alias create" typescript
rc=0
typescript showed: [1/5] Alias Name → [2/5] Provider → [3/5] Session Name →
[4/5] System Prompt (EMPTY Enter) → [5/5] Trust Level → "Alias 'testbot' created."
env file: testbot|gemini|testbot||0 (4th field EMPTY — the fix)
wrapper: ~/.local/bin/testbot exists, exec line without --system
Pacing requirement (probe: no-sleep): all input at once is UNRELIABLE — menu_read_value's raw reader consumes the whole queued burst via dd bs=4096, so bytes after the first submitted newline are discarded and later readers see EOF → clean abort. Input MUST be paced (sleep between inputs). 0.2s and 0.3s pacing both proven reliable for all three scenarios; 0.1s is NOT reliable for the empty-first-input case.
Rejected alternatives: expect not installed; python3 pty module available but unnecessary — script is deterministic with pacing and keeps the suite zero-dependency (bash + util-linux coreutils, matching DEV.md precedent).
Verdict: pty driver FEASIBLE → E2E included in the permanent suite (case table below).
Step 2: Unit matrix — menu_ask_value --allow-empty [PENDING]
See coverage table (Part A of t-menu-allow-empty.sh) — implementation landed, run results pending full suite.
Step 3: Static source guards [PENDING]
Create-flow step labels 1-5 of /5 (extracted _alias_create body); exactly 2 --allow-empty call sites; edit flow still /4. See coverage table.
Step 4: E2E create flow (pty) [PENDING]
See coverage table (3 cases in t-ai-alias-create-e2e.sh).
Step 5: Full suite [PENDING]
Step 6: Gates [PENDING]
Coverage table
| case | technique | result |
|---|---|---|
| — unit matrix — | ||
| empty+no-default, no flag → rc 1 | non-TTY stdin, real lib | pending |
| empty+no-default, --allow-empty → rc 0 + empty | non-TTY stdin, real lib | pending |
| empty+default, --allow-empty → rc 0 + default (default wins) | non-TTY stdin, real lib | pending |
| empty+default, no flag → rc 0 + default (regression) | non-TTY stdin, real lib | pending |
| non-empty, --allow-empty → rc 0 + value | non-TTY stdin, real lib | pending |
| non-empty, no flag → rc 0 + value (regression) | non-TTY stdin, real lib | pending |
| EOF/cancel, --allow-empty → rc 1 (cancel stays cancel) | non-TTY stdin, real lib | pending |
| EOF/cancel, no flag → rc 1 (contract) | non-TTY stdin, real lib | pending |
| value beats default with --allow-empty | non-TTY stdin, real lib | pending |
| — static source guards — | ||
| create steps 1..5 all labeled /5 (no /4) | extract_fn(_alias_create) + grep | pending |
| exactly 2 --allow-empty call sites (353/410), both in create | grep bin/pos-ai-alias + extract_fn | pending |
| edit flow still 4 × /4 (untouched) | extract_fn(_alias_edit) + grep | pending |
| no OTHER tool adopted --allow-empty (scope fence) | grep -l bin/pos-* | pending |
| — pty E2E — | ||
| empty System Prompt → trust step reached + alias file created (empty 4th field) | script -qec REAL create flow | pending |
| empty Alias Name → warn + re-prompt [1/5] ×2 → created | script -qec REAL create flow | pending |
| Ctrl-D at System Prompt → clean abort, no [5/5], no alias, no env | script -qec REAL create flow | pending |
| script binary unavailable → documented SKIP | require_cmd | pending |
Constraints respected
- Only tests + this report modified; no lib/ bin/ AGENT_TODO.md docs touched; no
make gen. - Hard-skip contract honored (SKIP only via
skip_case, never a fake pass). - Runtime budget respected (target ≤ ~70s; suite ~49s today + ~10s E2E).