diff --git a/AGENT_TODO.md b/AGENT_TODO.md index 217c3bb..9c4927e 100644 --- a/AGENT_TODO.md +++ b/AGENT_TODO.md @@ -42,6 +42,10 @@ summary (newest last). ## Done +- **2026-09-06** — Stabilization pass: 17-point code-level audit executed via Explorer(3) → Architect(decisions D-A..D-F) → Builder(ai/security/tooling/config/netprobe/f1-f3) → Tester(regression suite) → Reviewer(2 rounds). Security: Telegram sender-owner AND-gate + `TELEGRAM_OWNER_ID` registry/docs; Matrix `MATRIX_ROOM_ID` required; gpg `--passphrase-fd 3` (no argv secret); `/dev/tcp` positional-arg form (checkport/smb-client/share-lib/NET_PROBE incl. escaping `\$1`/`\$2`); eval `--no-command-execution` now carried by both chat bridges (D-B), deny-by-default `[y/N]`, tty-gated `--trust`; D-A soft-fail model ratified by Architect amendment (fail-closed either way; listeners are `Restart=always` so strict mode would crash-loop). AI: ALL ExecStart flags validated against installed llama.cpp (requested→error, default→omit+warn, `CONFIG_REQUESTED_FLAGS`), single-line ExecStart confirmed via `systemd-analyze verify`; hf single-file failure rc/exit-0 + meta-write bug fixed; `LLAMACPP_HOST` coherent; `# POS_SUBCMDS` + metadata gaps closed. Tooling: lint-conventions rewritten Bash-native (~24-30× faster, rules byte-identical, `:num` restored on 2 WARNs, planted-violation negative verified); `pos system uninstall` covers all 12 libs + scale-tail + flags dir + systemd USER units (`|| true`) + de-hardcoded plugin markers; safe anchored `.bash_completion`/`.bashrc` removals replace `sed -i '/pos/d'`. Config: canonical `load_env_file` in `lib/config-ui.sh` (CRLF strip, env-wins, XDG, `LOADED_ENV_KEYS`); 9 tools migrated; entertainment-lib collapsed to wrappers; docker-compose deliberately NOT migrated (source-semantics, documented). Tests: first committed regression suite — `tests/run-tests.sh` zero-dep runner + `make test`; 12 files / 179 checks / 0 skip / ~52s; hard skip contract (never lie); negative lint/gen-drift gates; systemd-analyze verify included. Verification: `make gen` idempotent, `make check` OK, `make lint` `0 FAIL, 0 WARN`, `make test` green, `bash -n` clean, `git diff --check` clean; `systemd-analyze verify` PASS on generated unit; CLI smokes (`pos --help`, `pos ai --help`, `pos ai hf --help`, `pos ai server --help`, `pos tree`) OK. + +- **2026-09-06** — llamacpp optional-app installer + `ai` app category + `pos ai server` install-hint wiring: new `apps/ai/llamacpp.sh` (idempotent `install_llamacpp()`/`uninstall_llamacpp()` — GitHub release archive: scans `/releases?per_page=10` for the first `-bin-ubuntu-{x64,arm64}.tar.gz` asset since `releases/latest` (v0.4.0 milestone) ships no binaries; installs to `/usr/local/lib/llama.cpp-` with `--strip-components=1`, symlinks every `llama*` binary into `/usr/local/bin`; uninstall removes the lib dir and only the symlinks whose target points into it); `apps/install.sh` CAT_NAMES += `[ai]="AI / ML"`; `templates/app.sh` categories comment += ai; DOC/APPS.md app-table row count updated (15→16 at the time; 18 after the 2026-09-06 stabilization pass) + categories line + llama.cpp catalog row; `bin/pos-ai-server` help "Requires:" + both `err "llama-server not found…"` lines (start/status) now name the app installer + invocation (`bash apps/install.sh llamacpp`, `--apps`, `--full`) and keep the GitHub URL (scrcpy phrasing pattern); DOC/POS.md ai row notes `bash apps/install.sh llamacpp`. Verified: live API probe confirmed real asset naming — `llama--bin-ubuntu-x64.tar.gz`/`-arm64.tar.gz` on nightly bNNNNN releases, top-level dir present, `.tar.gz` not `.zip` (so `tar` replaces the brief's `unzip` step — no unnecessary apt install); `bash -n` touched scripts; `bash apps/install.sh --uninstall llamacpp` resolves app + idempotent uninstall rc 0 (no network); `bash apps/ai/llamacpp.sh uninstall` rc 0; `make gen` idempotent; `make check` green; `make lint` 0 FAIL / 0 WARN. + - **2026-09-06** — Review-driven hardening of the AI tools (cycle over commits 387f23f/0856b25 + the llamacpp wiring): adversarial review of `pos-ai-hf`/`pos-ai-server` found 2 BLOCKING + 5 REQUIRED; Builder fixed F1 (`--include`/`--exclude` now bash-`case` glob filtering — array-shape-safe, no jq regex interpolation, composes gguf→filename→include→exclude), F2 (`ExecStart` rebuilt as a single-line, correctly-quoted command — `systemd_quote()` for executable + model path, `systemd-analyze verify` rc=0, dry-run byte-identical), F3 (`--branch`/`--revision` treated as aliases, last-arg-wins, dead `BRANCH` variable removed), F4 (parallel download drains ALL jobs — per-pid `wait` + failure collection, honest `Downloaded: X of Y files, N failed: …` summary, exit rc=1, no `.hf-meta` marking a half-downloaded model complete, EXIT-trap temp cleanup), F5 (`detect_llama_version` guarded (missing binary → clean error, never crash), `validate_requested_flags` errors on explicitly-requested flags the installed llama.cpp doesn't expose, version-aware message), F6 (`pos ai hf cache [status|clear]` real implementation — dir/count/size + confirm-fail-closed clear (via /dev/tty, tty-not-stdin so no INTERACTIVE_CMDS change); dead helpers removed). Maintainer convention sweep synced `llamacpp` into `bin/pos-ai` usage() lines 42/59 + `DOC/POS.md` AI_PROVIDER row `(gemini\|openrouter\|llamacpp)` + `DOC/howto/ai.md` (adapter list, `--provider` backends, backward-compat shorthand sentence, "Available providers" table row). Final Reviewer acceptance: APPROVE_WITH_NOTES, 0 REQUIRED. Verified: `bash -n` all `bin/pos*`; `make gen` idempotent; `make check` green; `make lint` 0 FAIL / 0 WARN; probe matrix — spaced-model-path unit (`systemd-analyze verify` rc=0 + 16-token word-split), forced-failure parallel download (rc=1, named failed file, no meta), cache clear deny/accept, `--slots` rejected with version-aware error, `status` without llama-server clean error. - **2026-09-06** — Convention sweep — llamacpp doc/usage sync: `bin/pos-ai` usage() provider lists (lines 42/59) now include llamacpp; `DOC/POS.md` AI_PROVIDER config row `(gemini\|openrouter\|llamacpp)` (Builder's 3 hand-edits verified consistent end-to-end); `DOC/howto/ai.md` adapter list, `--provider` backend list, backward-compat shorthand sentence, and "Available providers" table row all include llamacpp (facts from `lib/ai-providers/llamacpp.sh`). Verified: `bash -n` all `bin/pos*`; `make gen` idempotent; `make check` green; `make lint` 0 FAIL / 0 WARN. diff --git a/AgentsReport/architect/2026-09-06_da-failmode-decision.md b/AgentsReport/architect/2026-09-06_da-failmode-decision.md new file mode 100644 index 0000000..54b414f --- /dev/null +++ b/AgentsReport/architect/2026-09-06_da-failmode-decision.md @@ -0,0 +1,95 @@ +# D-A Fail-Mode Decision — Ratify Soft-Fail (2026-09-06) + +## TL;DR + +- **Decision 1:** D-A is formally **AMENDED** to the soft-fail model (option 2): unset `TELEGRAM_OWNER_ID` / `MATRIX_ROOM_ID` → daemon **starts**, logs a startup warning naming the missing var, and ignores every incoming command (no execution, no hint reply). +- **Decision 2:** No doc/template/test changes required — they already describe soft-fail (`DOC/POS.md:372,402`, `DOC/howto/communication.md`, `config/telegram.env:10`, `config/matrix.env:10-11`, `tests/t-telegram-auth.sh:76`, `tests/t-matrix-auth.sh:78`). +- **Residual caveat (explicit):** soft-fail misconfiguration is **SILENT to senders** — an admin who forgets `TELEGRAM_OWNER_ID` gets a daemon that looks alive (systemd healthy, long-polling) but ignores everyone; the only signal is the warning in the local log. Strict mode would have been LOUD at startup but would crash-loop a systemd `Restart=always` daemon for a non-fatal config gap. +- Files changed: `AgentsReport/architect/2026-09-06_stabilization-design.md` (D-A section amendment, [DECIDED]). + +--- + +## Decision 1: Ratify soft-fail as shipped (amend D-A) + +**Problem:** D-A as written required `err`+exit (fail-stop) when `TELEGRAM_OWNER_ID`/`MATRIX_ROOM_ID` are unset. The security-track Builder implemented soft-fail: daemon starts, `warn`s, ignores commands. Reviewer finding F2 (BLOCKING) confirmed fail-closed security is preserved but the approved contract was substituted silently — violating my own decision criteria 1/5. + +**Decision:** Formally amend D-A to the soft-fail model as shipped. The strict wording in `AgentsReport/architect/2026-09-06_stabilization-design.md` D-A (Telegram bullet, Matrix guard, acceptance criteria 1/5, Files-affected rows, risk note, TL;DR) is replaced and marked **"AMENDED at review (2026-09-06): soft-fail ratified as shipped"**. + +**Rationale:** +1. The entire shipped surface already encodes soft-fail: code (`bin/pos-communication-telegram-listener:758-762,789-791`, `bin/pos-communication-matrix-listener:510-512`), docs (`DOC/POS.md:372,402`), templates (`config/telegram.env:10`, `config/matrix.env:10-11`), and tests (`t-telegram-auth.sh:76`, `t-matrix-auth.sh:78`). Ratifying creates **zero** doc/template/test churn; choosing strict would require rewriting 5+ artifacts against an internally consistent implementation. +2. The security property is **identical in both models**: fail-closed — an unauthorized/unauthored command never executes. The difference is purely operational (startup refusal vs degraded liveness), so there is no security case forcing strict. +3. Strict fail-stop on a systemd-managed daemon with `Restart=always` produces a crash-loop for a non-fatal config gap; soft-fail keeps the daemon alive, diagnosable via its log, and recoverable via `pos config`. +4. Both directions are cheaply reversible (a 2-line guard flip) if future evidence favors strict. + +**Security implications of soft-fail (explicit):** misconfiguration is silent to senders — no reply, no error, no hint that commands exist; the only signal is the local-log warning. Weighing that against strict's startup crash-loop, soft-fail is the better operation for a personal toolkit where a missing optional auth key should not take down a supervised unit. + +[DECIDED] + +--- + +## Decision 2: No docs/templates/tests changes + +`DOC/POS.md` (lines 372, 402), `DOC/howto/communication.md`, `config/telegram.env` (line 10), `config/matrix.env` (lines 10-11), `tests/t-telegram-auth.sh` (line 76), and `tests/t-matrix-auth.sh` (line 78) all describe exactly the ratified behavior. They stay as-is. + +[DECIDED] + +--- + +## Handoff + +```text +Status: DECISION_READY + +Problem: +D-A unset-owner/room behavior conflicted between the written design (fail-stop) and the implemented, documented, template-encoded, and tested behavior (soft-fail). + +Decision: +Ratify soft-fail by amending D-A. Unset owner/room → daemon runs, warns (startup + per-message), ignores all commands (no execution, no hint reply). Security property unchanged: fail-closed. + +Reasoning: +Evidence (code + 4 doc/template artifacts + 2 test files) consistently describes soft-fail; no security property differs between models; strict would crash-loop systemd units. Reversible either way. + +Ownership: +Architect (amendment) — done. Implementations stay as shipped in bin/pos-communication-{telegram,matrix}-listener. + +Interfaces: +No CLI/config-facing contract change beyond the ratified D-A text. `TELEGRAM_OWNER_ID`/`MATRIX_ROOM_ID` remain optional config keys whose absence disables command execution, not daemon start. + +Approved scope: +- `AgentsReport/architect/2026-09-06_stabilization-design.md` D-A section (amended). +- Reviewer finding F2 is resolved by amendment (downgrades); no code change. + +Explicitly out of scope: +- No change to listener code, docs, templates, or tests (they already match). +- No change to F1/F3 (BLOCKING/REQUIRED D-B items) — those remain Builder work. + +Constraints: +- D-A acceptance criteria 1/5 now assert soft-fail wording (already updated in the amended doc). +- Fail-closed remains a hard invariant: never execute a command from an unauthorized sender or chat. + +Verification: +- Re-read amended D-A section for consistency (no remaining "refuses to start"/`err` strict wording for the unset case). +- Reviewer re-check of F2 against amended criteria. +- Orchestrator runs `make gen && make check && make lint && make test` before merge (Step 10 gates). + +Risks: +- Silent misconfiguration (soft-fail caveat): mitigated by actionable warning text naming the missing var and the `listener running (chat X, owner unset)` startup log line. +- Future preference for fail-stop is a 2-line code change + doc/template updates; D-A now records both models' trade-offs. + +Recommended next agent: +Builder + +Reason: +F1 (missing `--no-command-execution` in both bridge invocations) and F3 (NO_EXEC print semantics) remain BLOCKING/REQUIRED D-B fixes independent of this amendment; F2 no longer blocks. Builder should also add the D-A amendment note to AGENT_TODO.md if not already covered. + +Architect changes: +AgentsReport/architect/2026-09-06_stabilization-design.md — D-A section amended only. +``` + +--- + +## Files changed (this decision) + +| File | Change | +|------|--------| +| `AgentsReport/architect/2026-09-06_stabilization-design.md` | D-A TL;DR item, Telegram bullet, Matrix guard, Files-affected rows, acceptance criteria 1/5, risk note — amended to soft-fail and marked "AMENDED at review (2026-09-06)" | \ No newline at end of file diff --git a/AgentsReport/architect/2026-09-06_stabilization-design.md b/AgentsReport/architect/2026-09-06_stabilization-design.md new file mode 100644 index 0000000..99bb72b --- /dev/null +++ b/AgentsReport/architect/2026-09-06_stabilization-design.md @@ -0,0 +1,770 @@ +# Stabilization Pass — Architectural Design Decisions + +**Date:** 2026-09-06 +**Architect:** Evidence-driven design pass over three Explorer audit reports +**Constraint:** Preserve existing Bash architecture; no framework rewrites; smallest clean fix per issue; no commits; all gates (`make gen`, `make check`, `make lint`) must remain green at `0 FAIL, 0 WARN`. + +--- + +## TL;DR + +1. **D-A (Telegram/Matrix auth):** Introduce `TELEGRAM_OWNER_ID` as an AND-gate with `TELEGRAM_CHAT_ID`; Matrix requires `MATRIX_ROOM_ID` before any command runs. Both platforms converge on "sender AND chat authorized" contract; unset owner/room → daemon runs but ignores all commands (soft-fail, ratified at review 2026-09-06). +2. **D-B (AI eval posture):** Flip tty confirmation default to `[y/N]` (deny); keep `--trust` interactive-only; add `--no-exec` flag to `pos-ai` and enforce it in chat bridges. Document the bridge invariant. +3. **D-C (Test framework):** Create `tests/` with a minimal runner, `make test` target, stub-PATH + PTY patterns; first suite covers 14 surfaces. +4. **D-D (Config loader):** Canonical `load_env_file` in `lib/config-ui.sh`; migrate 9 hand-rolled loaders; unify CRLF + XDG; collapse entertainment-lib read/write. +5. **D-E (Install/uninstall manifest):** Extend `pos-system-uninstall` Tier 1 to cover 9 orphaned libs, ScaleTail dir, feature-flag store, and USER systemd units; de-hardcode plugin/binary lists. +6. **D-F (pos-ai-server validation):** Validate ALL flags entering ExecStart (CLI + config + defaults); unsupported DEFAULT → warn + omit; unsupported REQUESTED → hard error. + +--- + +## D-A: Chat Authorization Model (Telegram + Matrix) + +### Decision + +Adopt a unified "sender AND chat authorized" authorization contract for both Telegram and Matrix listeners. Concretely: + +**Telegram:** +- Add `TELEGRAM_OWNER_ID` to the `# POS_CONFIG:` registry in `bin/pos-communication-telegram-sender:6` and to `config/telegram.env` template. +- Replace the OR-gate at `bin/pos-communication-telegram-listener:787` with: + 1. `chat == TELEGRAM_CHAT_ID` (chat must be the configured chat) — fail-continue (skip message silently). + 2. `from_id` must be one of the comma-separated `TELEGRAM_OWNER_ID` values — fail-continue (skip message silently). +- If `TELEGRAM_OWNER_ID` is unset, the daemon **starts in fail-closed degraded mode**: `run_daemon` logs `warn "TELEGRAM_OWNER_ID unset — chat commands WILL BE IGNORED (fail-closed); set it with 'pos config telegram'"` at startup (same location as the token/chat-id guards, line ~761) and a per-message `warn "TELEGRAM_OWNER_ID unset — ignoring command (set it with 'pos config telegram')"`; every incoming command is skipped — **no execution, no hint reply**. + + > **AMENDED at review (2026-09-06): soft-fail ratified as shipped.** The original wording required the daemon to `err`+exit on unset `TELEGRAM_OWNER_ID`. The security-track Builder implemented soft-fail (daemon runs, warns, ignores commands); `DOC/POS.md`, `DOC/howto/communication.md`, `config/telegram.env` and `tests/t-telegram-auth.sh` all consistently describe soft-fail. This is a deliberate deviation ratified at review time (Reviewer finding F2, downgraded). The security property is identical in both models: **fail-closed** — an unauthenticated/unauthorized command never executes. The difference is purely operational: strict = fail-stop (loud misconfiguration, systemd restart loop); soft-fail = degraded-liveness (silent-to-senders misconfiguration, one clean warning in the local log). Soft-fail was chosen because the entire shipped surface already encoded it and it avoids a systemd `Restart=always` crash-loop for a non-fatal config gap. +- Reply on unauthorized messages: **none** (no leak that commands exist). The message is silently dropped. +- Do **not** include a "trusted group" backward-compat mode in this pass. The OR-gate was never a documented feature; it was an implementation gap. Backward compatibility for group operation can be added later via an explicit `TELEGRAM_GROUP_MODE=true` opt-in — that is a separate, larger design decision (requires per-user allowlist, rate limiting, etc.) and explicitly **out of scope** for this stabilization. + +**Matrix:** +- Require `MATRIX_ROOM_ID` for command execution. If unset, the daemon **starts in fail-closed degraded mode**: guard in `run_daemon` at `bin/pos-communication-matrix-listener:510-512` logs `warn "MATRIX_ROOM_ID unset — refusing to run commands (fail-closed); set it with 'pos config matrix'"`; the daemon long-polls but runs no commands. + + > **AMENDED at review (2026-09-06): soft-fail ratified as shipped.** The original wording required a hard `[ -n "${MATRIX_ROOM_ID:-}" ] || err "MATRIX_ROOM_ID is required — set it in pos config matrix"` guard. The security-track Builder implemented soft-fail; `DOC/POS.md`, `DOC/howto/communication.md`, `config/matrix.env` and `tests/t-matrix-auth.sh` all consistently describe soft-fail. Deliberate deviation ratified at review time (Reviewer finding F2, downgraded). Security property identical in both models: **fail-closed** — no command runs outside the configured room. Same operational trade-off as the Telegram case above (silent-to-senders vs. startup crash-loop). +- Keep the sender=owner gate (`sender == owner` at line 550) unchanged. +- This makes the Matrix behavior: sender authorized (== owner) AND chat authorized (== configured room) — identical to the Telegram contract. + +**Consistency rule:** Both platforms enforce the same conceptual model: a message must come from an authorized sender in an authorized chat. No fallback to "any chat" or "any sender." + +### Files affected + +| File | Change | +|------|--------| +| `bin/pos-communication-telegram-sender:6` | Add `TELEGRAM_OWNER_ID` to `# POS_CONFIG:` registry | +| `bin/pos-communication-telegram-listener:787-791` | Replace OR-gate with AND-gate (chat + sender) + soft-fail `TELEGRAM_OWNER_ID` unset warning in `run_daemon` and per-message skip | +| `bin/pos-communication-matrix-listener:510-512` | Soft-fail `MATRIX_ROOM_ID` unset warning in `run_daemon` (commands refused) | +| `config/telegram.env` | Add `TELEGRAM_OWNER_ID=` template line | +| `DOC/HOWTO.md` (communication section) | Document new key, explain AND-gate | +| `DOC/POS.md` (telegram listener section) | Document authorization model | + +### Acceptance criteria + +1. With `TELEGRAM_OWNER_ID` unset, `pos communication telegram listener --run` starts the daemon, logs a startup warning naming `TELEGRAM_OWNER_ID`, and ignores every incoming command (no execution, no hint reply). +2. With `TELEGRAM_OWNER_ID=123` and `TELEGRAM_CHAT_ID=456`, a message from `from_id=123` in `chat=456` is dispatched. +3. A message from `from_id=999` in `chat=456` is silently dropped (no reply, no log of "command not found"). +4. A message from `from_id=123` in `chat=789` (wrong chat) is silently dropped. +5. With `MATRIX_ROOM_ID` unset, `pos communication matrix listener --run` starts the daemon, logs a warning naming `MATRIX_ROOM_ID`, and runs no commands (fail-closed). +6. With `MATRIX_ROOM_ID` set, messages from the owner in the configured room are dispatched; messages from other senders or other rooms are dropped. +7. `make gen && make check && make lint` all pass at 0 FAIL, 0 WARN after the change. +8. `TELEGRAM_OWNER_ID` appears in `pos config telegram` output (masked if flagged as `digits` type, same as `TELEGRAM_CHAT_ID`). + +### Risks / open questions + +- Existing users who have not set `TELEGRAM_OWNER_ID` will find the daemon stays up but ignores every command after upgrade. **Mitigation:** the startup warning is actionable ("set it with 'pos config telegram'"). This is intentional — the previous behavior was a security vulnerability (V1), not a feature. Documented residual caveat (soft-fail): misconfiguration is **silent to senders** — no reply, no error, no hint that commands exist; the only signal is the startup/per-message warning in the local log. Strict mode would have been loud at startup but would crash-loop a systemd-managed daemon for a non-fatal config gap; soft-fail preserves liveness and diagnosability. (Amended 2026-09-06.) +- The `TELEGRAM_OWNER_ID` supports comma-separated values (multi-user). The `digits` flag validation in `cfg_validate` already allows negative IDs (group/supergroup IDs); for `TELEGRAM_OWNER_ID` we should use a `digits` flag that also allows comma-separated positive-only values. **Decision:** use a plain `digits` flag (no comma support) for the initial pass; each `TELEGRAM_OWNER_ID` entry is a single numeric user ID. If multi-user is needed, a future pass adds it. This keeps the validation simple and the AND-gate implementation a direct string comparison. +- The `TELEGRAM_CHAT_ID` comparison already allows negative values (group IDs). The `TELEGRAM_OWNER_ID` should always be a positive user ID. The `cfg_validate` `digits` flag allows leading `-`; for `TELEGRAM_OWNER_ID` use a new `positive-digits` flag or simply document that only positive values are valid for this key. + +--- + +## D-B: Remote AI Command Execution Posture (eval) + +### Decision + +The `pos ai` command-execution path (`_prompt_run_command`) evaluates AI-generated shell code. The security posture change addresses three concerns: the default confirmation, the `--trust` flag, and the bridge invariant. + +**1. Flip the tty confirmation default to DENY:** + +Change `bin/pos-ai:391` from `[Y/n]` to `[y/N]`. The rationale: the AI model's output is untrusted external authority. When a user asks "run X", they mean the *task* — the specific command the model proposes is the model's interpretation, and a prompt-injection or model error can produce a harmful command. Defaulting to deny means the user must explicitly opt-in (`y` or Enter is now decline). The `[y/N]` pattern is the standard bash convention for non-destructive defaults. The user's request for help does not imply authorization to execute arbitrary code. + +Current line 391: +```bash +printf 'Run this command? [Y/n] ' >&2 +``` +Change to: +```bash +printf 'Run this command? [y/N] ' >&2 +``` + +And invert the case logic at line 394: `y|Y)` executes; `*` (including Enter) declines and adds to history. + +**2. `--trust` flag: keep as-is but document scope:** + +The `--trust` flag (`TRUST_MODE=1`, `bin/pos-ai:644`) auto-executes without confirmation on a **tty only** (line 382 `[ -w /dev/tty ] || return 0` — non-tty never executes). This is correct: `--trust` is an explicit operator action on an interactive terminal. No change needed to the flag itself, but: +- Add a `--trust` warning to `usage()` if not already present: note that this bypasses confirmation and should only be used in trusted local sessions. +- Document that `--trust` has **no effect** when invoked from a chat bridge (non-tty → early return at line 382). + +Do NOT rename `--trust` to `--trust-no-confirm`; the existing name is clear enough and renaming would break alias wrappers (`pos-ai-alias:433-436`). + +**3. Chat bridge invariant: bridges NEVER execute code blocks.** + +Both Telegram (`bin/pos-communication-telegram-listener:734`) and Matrix (`bin/pos-communication-matrix-listener:472`) invoke `pos ai gemini ask ...` which runs non-interactively. Inside `_prompt_run_command` (`bin/pos-ai:382`), `[ -w /dev/tty ] || return 0` means code blocks are never executed when the tool runs without a tty — they are printed but not run. This is the correct behavior. + +To prevent future regressions if a refactor changes the tty check or adds an auto-confirm path: +- Add `--no-command-execution` as a recognized flag in `pos-ai` (`bin/pos-ai:649` parse block, `# POS_FLAGS:` header). +- When set, `_prompt_run_command` returns 0 immediately without printing or executing (same as current non-tty behavior, but explicit). +- Both chat bridges pass `--no-command-execution` when invoking `pos ai`: + - `bin/pos-communication-telegram-listener:734`: add `--no-command-execution` to the command. + - `bin/pos-communication-matrix-listener:472`: same. +- This makes the invariant **structurally enforced**: even if a future refactor removes the tty check, the bridge-parsed flag still prevents execution. + +### Files affected + +| File | Change | +|------|--------| +| `bin/pos-ai:391` | Change prompt from `[Y/n]` to `[y/N]` | +| `bin/pos-ai:394-405` | Invert case logic: `y|Y` → execute, `*` → decline+history | +| `bin/pos-ai:649-677` | Add `--no-command-execution` to flag parser | +| `bin/pos-ai:380-382` | Check `$NO_EXEC` flag before the tty check | +| `bin/pos-ai:5` (`# POS_FLAGS:`) | Add `--no-command-execution` | +| `bin/pos-communication-telegram-listener:734` | Add `--no-command-execution` to AI bridge call | +| `bin/pos-communication-matrix-listener:472` | Add `--no-command-execution` to AI bridge call | +| `DOC/POS.md` (ai section) | Document new flag and confirmation change | +| `DOC/HOWTO.md` (ai section) | Document the security posture | + +### Acceptance criteria + +1. On a tty, `pos ai` with a command block in the response shows `[y/N]` and declines on Enter. +2. On a tty, typing `y` or `Y` at the prompt executes the command. +3. `pos ai --trust` still auto-executes on a tty (no prompt). +4. `pos ai --no-command-execution` skips execution entirely (no prompt, no output of the command block). +5. `pos communication telegram listener` → AI bridge invocation includes `--no-command-execution` in the actual command string (verifiable by reading the source). +6. Same for Matrix listener. +7. `make gen && make check && make lint` all pass at 0 FAIL, 0 WARN. +8. The `# POS_FLAGS:` header includes `--no-command-execution` and `completions/pos.bash` updates accordingly after `make gen`. + +### Risks / open questions + +- Changing the default from ALLOW to DENY is a **behavioral breaking change** for users who are accustomed to pressing Enter to run. This is intentional and justified by the security audit (V2): the model's output is untrusted. Users who want the old behavior can type `y`. +- The `--no-command-execution` flag name is long. Alternatives: `--no-exec`, `--safe-mode`. **Decision:** `--no-command-execution` is preferred because it is self-documenting and unambiguous. The flag is consumed programmatically (by bridges), not by humans typing interactively. +- Non-tty paths (`[ -w /dev/tty ] || return 0`) already prevent execution. `--no-command-execution` adds defense-in-depth for the tty path in case the tty check is ever removed. + +--- + +## D-C: Test Framework Shape + +### Decision + +Create a committed `tests/` directory with a minimal runner and a `make test` target. The framework follows the repo's established stub-PATH + PTY patterns from `DOC/DEV.md:196-214`. + +**Layout:** + +``` +tests/ + run-tests.sh # runner (check helper, pass/fail counting, exit code) + ai-server-flags.sh # unit: flag validation logic + ai-hf-download.sh # unit: single-file failure path + telegram-auth.sh # unit: auth gate logic (mocked) + matrix-auth.sh # unit: auth gate logic (mocked) + config-loader.sh # unit: load_env_file precedence + systemd-unit.sh # unit: unit generation + systemd-analyze (skip if unavailable) + uninstall-manifest.sh # unit: install/uninstall symmetry + gen-docs-drift.sh # integration: make gen && git diff --check + lint-gate.sh # integration: make lint exit code + gpg-backup.sh # unit: passphrase not in argv (mocked gpg) + config-precedence.sh # unit: env-wins-over-file precedence + unsupported-flags.sh # unit: pos-ai-server unsupported flag handling + ai-llama-detect.sh # unit: llama version detection (mocked binary) +``` + +**Runner (`tests/run-tests.sh`):** + +```bash +#!/usr/bin/env bash +set -euo pipefail +# Minimal test runner — sources test functions, counts pass/fail/skip. +# Usage: tests/run-tests.sh [test-file ...] +# If no args, runs all tests/*.sh files. +PASS=0; FAIL=0; SKIP=0 +check() { local desc="$1" expected="$2" actual="$3" + if [ "$expected" = "$actual" ]; then PASS=$((PASS+1)); printf ' PASS %s\n' "$desc" + else FAIL=$((FAIL+1)); printf ' FAIL %s (expected=%s actual=%s)\n' "$desc" "$expected" "$actual"; fi } +skip() { SKIP=$((SKIP+1)); printf ' SKIP %s (%s)\n' "$1" "$2"; } +# ... file discovery, per-file sourcing, summary ... +``` + +**`make test` target (Makefile addition):** + +```makefile +test: + ./tests/run-tests.sh +``` + +**Interaction with existing gates:** +- `make check` and `make lint` remain **unchanged** — they are static gates (syntax, exec bits, doc sync, convention). +- `make test` is a **separate** target for behavioral regression tests. +- CI (`.gitea/workflows/lint.yml`) does NOT need to run `make test` in this pass — that is a future CI enhancement. The tests exist for local validation and are committed as the regression baseline. + +**Determinism and speed:** +- All tests use stub PATH (temp dir with fake binaries) and env-overridable paths per `DEV.md:198`. +- No network calls, no real systemd, no real llama-server binary. +- Target: all tests complete in < 60 seconds on a typical dev box. +- Tests that need `systemd-analyze` (for real validation) use a skip-if-unavailable pattern: + ```bash + command -v systemd-analyze &>/dev/null || { skip "systemd-analyze validation" "systemd-analyze not available"; return 0; } + ``` +- Tests that need `jq` (for JSON assertions) use the same skip pattern. +- Skip means "not applicable in this environment" — **never** "test passed." Tests never lie about pass/fail. + +**What is unit-testable with stub PATH vs integration-only:** + +| Test | Method | Skip condition | +|------|--------|---------------| +| AI server flag validation | Stub PATH with fake `llama-server` that echoes `--help` text | Never (fake binary is self-contained) | +| AI HF single-file failure | Stub PATH with fake `curl` that returns rc=1 | Never | +| Telegram auth gate | Direct function extraction (source the auth logic, call with test args) — or stub PATH with fake `jq`/`curl` | Never | +| Matrix auth gate | Same as Telegram | Never | +| Config loader | Direct sourcing of `load_env_file` | Never | +| Systemd unit generation | Stub PATH with fake `systemd-escape`, `systemd-quote` output comparison | `systemd-analyze` not available → skip validation step | +| Uninstall manifest | Direct comparison of install list vs uninstall list (grep both scripts) | Never | +| Gen-docs drift | Run `make gen` then `git diff --check` | `git` repo not available → skip | +| Lint gate | Run `make lint` and check exit code | Never | +| GPG passphrase | Stub PATH with fake `gpg` that echoes argv to a log file; assert passphrase not in log | Never | +| Config precedence | Export env var, write config file, call loader, assert env wins | Never | +| Unsupported flags | Stub PATH with fake `llama-server` that echoes specific `--help` text | Never | +| Llama version detect | Stub PATH with fake `llama-server` that echoes `--version` | Never | + +### Files affected + +| File | Change | +|------|--------| +| `tests/run-tests.sh` (new) | Test runner | +| `tests/ai-server-flags.sh` (new) | Flag validation tests | +| `tests/ai-hf-download.sh` (new) | HF single-file failure tests | +| `tests/telegram-auth.sh` (new) | Telegram auth gate tests | +| `tests/matrix-auth.sh` (new) | Matrix auth gate tests | +| `tests/config-loader.sh` (new) | Config loader precedence tests | +| `tests/systemd-unit.sh` (new) | Systemd unit generation tests | +| `tests/uninstall-manifest.sh` (new) | Install/uninstall symmetry tests | +| `tests/gen-docs-drift.sh` (new) | Gen-docs drift test | +| `tests/lint-gate.sh` (new) | Lint gate test | +| `tests/gpg-backup.sh` (new) | GPG passphrase tests | +| `tests/config-precedence.sh` (new) | Config precedence tests | +| `tests/unsupported-flags.sh` (new) | Unsupported flag handling tests | +| `tests/ai-llama-detect.sh` (new) | Llama version detection tests | +| `Makefile` | Add `test` target | +| `scripts/check-sync.sh` | (Optional) Add `tests/*.sh` to the `bash -n` scan list | + +### Acceptance criteria + +1. `make test` runs all 14 test files and reports PASS/FAIL/SKIP counts. +2. `make test` exits 0 when all tests pass (or are skipped). +3. `make test` exits non-zero when any test FAILs. +4. No test makes network calls (verified by grepping test files for `curl`/`wget`/`http` without stub wrappers). +5. All tests complete in < 60 seconds (measured on a representative dev box). +6. `make check` and `make lint` are unaffected (0 FAIL, 0 WARN). +7. `make gen` is unaffected. +8. Tests that need `systemd-analyze` skip gracefully when unavailable (exit 0, report SKIP). +9. The test runner does not leave temp files in the working tree (uses `/tmp` for all temp dirs). +10. Each test file starts with `#!/usr/bin/env bash` + `set -euo pipefail` (passes `bash -n` and lint). + +### Risks / open questions + +- The test framework is **minimal by design**. It does not use bats, shunit2, or any external framework — consistent with the repo's zero-dependency philosophy. If the test surface grows beyond ~20 files, consider bats at that point. +- Some tests (auth gate, config loader) require extracting logic from tools into testable functions or sourcing the tool and overriding variables. This is the established pattern from DEV.md's stub-PATH approach. +- The gen-docs drift test depends on `make gen` being idempotent — which is a project invariant. +- The lint gate test is inherently coupled to `scripts/lint-conventions.sh` behavior — if lint rules change, this test may need updating. Acceptable; it is a regression canary. + +--- + +## D-D: Config Loader Centralization + +### Decision + +Establish `lib/config-ui.sh` as the canonical config loader. Add a single generic `load_env_file` function and migrate all 9 hand-rolled loaders to it. + +**1. Add `load_env_file` to `lib/config-ui.sh`:** + +```bash +# Canonical env-file loader. Reads KEY=VALUE lines, strips comments and +# CRLF, applies env-wins precedence (exported env vars are never overwritten). +# Usage: load_env_file +load_env_file() { + local f="$1" k v + [ -f "$f" ] || return 0 + while IFS='=' read -r k v; do + [ -n "$k" ] || continue + case "$k" in \#*) continue ;; esac + v="${v//$'\r'/}" + v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" + if [ -z "${!k:-}" ]; then + export "$k"="$v" + fi + done < <(grep -E '^[A-Z_]+=' "$f" || true) +} +``` + +**Precedence contract (documented, matches dominant behavior):** +``` +CLI flags > exported environment > config file values > defaults +``` +This is the existing behavior in all 9 hand-rolled loaders. The `load_env_file` function implements "exported env wins over file" (the `if [ -z "${!k:-}" ]` check). CLI flags are applied later by the tool's own arg parser. Defaults are applied at variable declaration (`${VAR:-default}`). This contract is now explicit and single-sourced. + +**2. Deprecate `load_system_env` in `lib/common.sh:146-159`:** + +Make `load_system_env` delegate to `load_env_file`: +```bash +load_system_env() { + load_env_file "$HOME/.config/linux_post_install/system.env" +} +``` + +This requires `common.sh` to source `config-ui.sh`. However, `common.sh` is the base library sourced by most tools, and `config-ui.sh` is not currently sourced by `common.sh`. **Decision:** do NOT make `common.sh` source `config-ui.sh`. Instead, keep `load_system_env` as-is (it is functionally identical to `load_env_file`) but note in `config-ui.sh` that `load_env_file` supersedes it. New tools use `load_env_file`; existing `load_system_env` callers continue working. This avoids a circular dependency risk and a mass-change to `common.sh` consumers. + +The **real migration** targets are the 9 tools that hand-roll their own `load_config()`. + +**3. CRLF policy (universal strip):** + +`load_env_file` strips `\r` unconditionally (`v="${v//$'\r'/}"`). This resolves the 5-tool vs 5-tool CRLF divergence identified in the audit. All 5 tools that currently strip CRLF will continue to work; the 5 that don't will now strip it too (defensive improvement, no behavioral regression). + +**4. XDG honoring:** + +`load_env_file` takes an absolute file path — it does not resolve `CONFIG_DIR`. The caller passes the full path. For tools that currently hardcode `$HOME/.config/linux_post_install/...`, the migration replaces the hardcoded path with `${CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/linux_post_install}/`. Since `common.sh` defines `CONFIG_DIR` at line 19, tools sourcing `common.sh` already have it; the self-contained communication tools already have their own `CONFIG_DIR` guarded copy. + +**Migration list (9 tools):** + +| Tool | Current function | Current file | Notes | +|------|-----------------|-------------|-------| +| `bin/pos-communication-telegram-sender` | `load_config:61-74` | `telegram.env` | Already sources `config-ui.sh` for `cfg_value` | +| `bin/pos-communication-telegram-listener` | `load_config:86-98` | `telegram.env` | Standalone | +| `bin/pos-communication-matrix-listener` | `load_config:65-77` | `matrix.env` | Standalone | +| `bin/pos-communication-matrix-sender` | `load_config:44-57` | `matrix.env` | Standalone | +| `bin/pos-communication-scrcpy` | `load_config:14-28` | `scrcpy.env` | Standalone | +| `bin/pos-ai` | `load_config:130-160` | `ai.env` + legacy files | Has extra legacy-file loop | +| `bin/pos-ai-server` | `load_config:21-35` | `ai.env` | Standalone | +| `bin/pos-ai-hf` | `load_hf_config:30-44` | `ai.env` | Standalone | +| `bin/pos-media-grab` | `load_grab_config:10-23` | `grab.env` | Standalone | + +For each tool: source `config-ui.sh` (with the existing fallback chain pattern), replace the hand-rolled function body with `load_env_file "$CONFIG_FILE"`, preserve any tool-specific extras (like `pos-ai`'s legacy-file loop — keep that as a second `load_env_file` call on the legacy path). + +**5. Collapse entertainment-lib read/write into config-ui:** + +`entertainment-lib.sh`'s `config_value` and `write_config_key` (lines 28-54) are functionally identical to `cfg_value` and `cfg_write` in `config-ui.sh` (lines 311-344). **Decision:** add thin wrappers in `entertainment-lib.sh` that delegate to `cfg_value`/`cfg_write`: + +```bash +config_value() { + local k="$1" + cfg_value "$CONFIG_FILE" "$k" +} +write_config_key() { + local key="$1" val="$2" + cfg_write "$CONFIG_FILE" "$key" "$val" +} +``` + +This preserves the existing `config_value`/`write_config_key` API for the 5 entertainment tools that call them, while the implementation lives in one place. `entertainment-lib.sh` must source `config-ui.sh` (add to its source chain). + +### Files affected + +| File | Change | +|------|--------| +| `lib/config-ui.sh` | Add `load_env_file` function | +| `bin/pos-communication-telegram-sender` | Replace `load_config` body with `load_env_file` | +| `bin/pos-communication-telegram-listener` | Replace `load_config` body with `load_env_file` | +| `bin/pos-communication-matrix-listener` | Replace `load_config` body with `load_env_file` | +| `bin/pos-communication-matrix-sender` | Replace `load_config` body with `load_env_file` | +| `bin/pos-communication-scrcpy` | Replace `load_config` body with `load_env_file` | +| `bin/pos-ai` | Replace `load_config` body with `load_env_file` + legacy loop | +| `bin/pos-ai-server` | Replace `load_config` body with `load_env_file` | +| `bin/pos-ai-hf` | Replace `load_hf_config` body with `load_env_file` | +| `bin/pos-media-grab` | Replace `load_grab_config` body with `load_env_file` | +| `lib/entertainment-lib.sh` | Delegate `config_value`/`write_config_key` to `cfg_value`/`cfg_write` | +| `DOC/DEV.md` | Update "No shared lib? Inline fallbacks" section to reference `config-ui.sh` as the canonical loader | + +### Acceptance criteria + +1. `grep -rn 'while IFS.*read.*k.*v' bin/pos-communication-* bin/pos-ai* bin/pos-media-grab` returns **zero** hand-rolled loader matches (all replaced). +2. Each migrated tool passes its existing behavioral expectations: env-wins precedence, CRLF stripping, missing-file graceful return (rc 0). +3. `pos config telegram` and `pos config matrix` display and mask values correctly after migration. +4. `entertainment-lib.sh` `config_value` and `write_config_key` still work (entertainment tools pass their tests). +5. `make gen && make check && make lint` all pass at 0 FAIL, 0 WARN. +6. No tool that previously sourced `common.sh` now additionally sources `config-ui.sh` unless it was already doing so. Tools that were standalone (communication tools) now source `config-ui.sh` via the fallback chain, which is the same pattern used by their existing inline copies. + +### Risks / open questions + +- **Self-contained tools sourcing config-ui.sh:** The 5 communication tools currently do NOT source any shared lib (they carry inline fallbacks). After migration, they source `config-ui.sh`. This is a net improvement (shared implementation) but changes the "no shared lib" classification of these tools. **Mitigation:** the inline fallback copies of `log`/`warn`/`err` stay in place; only the `load_config` body is replaced. The tools remain self-contained for basic logging; they just share the config loader. +- **pos-ai's legacy file loop:** `pos-ai:130-160` reads `ai.env` plus legacy `gemini.env` and `openrouter.env` files. This is tool-specific logic; keep it as a second pass after `load_env_file "$CONFIG_FILE"`. Do not try to generalize the legacy loop into `load_env_file`. +- **Regression risk:** Each migration touches a working tool's config loading. The test suite (D-C) covers config precedence, which mitigates this. + +--- + +## D-E: Install/Uninstall Manifest + +### Decision + +Close the install-only gaps in `pos-system-uninstall` with the smallest targeted fixes. Do NOT introduce a full manifest abstraction (like `lib/install-manifest.sh`) — that would be a larger refactor inconsistent with the current "list-based" approach in both `install.sh` and `pos-system-uninstall`. + +**1. Add 9 orphaned libs to Tier 1 scan and removal:** + +Extend the lib list at `bin/pos-system-uninstall:62-64` (scan) and `:238-239` (remove) from: +```bash +for f in common.sh menu-lib.sh share-lib.sh; do +``` +to: +```bash +for f in common.sh menu-lib.sh share-lib.sh flags.sh notify.sh entertainment-lib.sh scheduler-lib.sh config-ui.sh user-timers-lib.sh entertainment-plugin-lib.sh usb-lib.sh registry.sh; do +``` + +Same change in both `scan_tier1` and `remove_tier1`. + +**2. Add ScaleTail directory and feature-flag store to Tier 1 removal:** + +After the lib removal block in `remove_tier1`: +```bash +# ScaleTail templates +if [ -d /usr/local/share/linux_post_install/scale-tail ]; then + rm -rf /usr/local/share/linux_post_install/scale-tail && count=$((count+1)) +fi + +# Feature-flag store +if [ -d /usr/local/share/linux_post_install/flags ]; then + rm -rf /usr/local/share/linux_post_install/flags && count=$((count+1)) +fi + +# Clean up parent dir if empty +rmdir /usr/local/share/linux_post_install 2>/dev/null || true +``` + +Add corresponding scan entries in `scan_tier1`: +```bash +[ -d /usr/local/share/linux_post_install/scale-tail ] && found+=("/usr/local/share/linux_post_install/scale-tail/") +[ -d /usr/local/share/linux_post_install/flags ] && found+=("/usr/local/share/linux_post_install/flags/") +``` + +**3. Add USER systemd unit discovery to Tier 1:** + +After the system-unit removal block (`:287-311`), add: +```bash +# USER systemd units (pos-* and pos-entertainment-* and pos-schedule-*) +local user_unit_dir="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user" +if [ -d "$user_unit_dir" ]; then + local ufile + while IFS= read -r ufile; do + local uname + uname="$(basename "$ufile" .service)" + uname="${uname%.timer}" + systemctl --user disable --now "${basename "$ufile"}" 2>/dev/null || true + rm -f "$ufile" && count=$((count+1)) + done < <(find "$user_unit_dir" -maxdepth 1 -name 'pos-*' -type f 2>/dev/null || true) + systemctl --user daemon-reload 2>/dev/null || true +fi +``` + +Add corresponding scan in `scan_tier1`: +```bash +local user_unit_dir="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user" +if [ -d "$user_unit_dir" ]; then + while IFS= read -r ufile; do + found+=("user-unit: $(basename "$ufile")") + done < <(find "$user_unit_dir" -maxdepth 1 -name 'pos-*' -type f 2>/dev/null || true) +fi +``` + +**4. De-hardcode entertainment-plugin and prebuilt-binary lists:** + +Replace the hardcoded loops for entertainment plugins (`:72-74` scan, `:248-250` remove) with directory-driven discovery (matching `install.sh`'s pattern): +```bash +# Scan: discover plugins by POS_PLUGIN marker +while IFS= read -r f; do + found+=("$f") +done < <(for ep in /usr/local/bin/*.sh; do + grep -q '^# POS_PLUGIN:' "$ep" 2>/dev/null && echo "$ep" +done | sort) +``` +```bash +# Remove: all installed entertainment plugins (discovered by POS_PLUGIN marker) +while IFS= read -r f; do + [ -f "$f" ] && { rm -f "$f" && count=$((count+1)); } +done < <(for ep in /usr/local/bin/*.sh; do + grep -q '^# POS_PLUGIN:' "$ep" 2>/dev/null && echo "$ep" +done | sort) +``` + +**5. Fix the `.bash_completion` over-broad removal (H-002):** + +Replace `sed -i '/pos/d'` at `:333` with marker-based or anchored patterns: +```bash +# Only remove lines the installer added (source pos.bash pattern) +sed -i '/source.*pos\.bash/d' "$HOME/.bash_completion" +``` + +This is conservative: only lines that `postinstall.sh` would have added are removed. If no such lines exist, nothing is touched. + +**6. Fix `.bashrc` removal anchoring (H-003):** + +Add word-boundary anchors to the sed patterns at `:320-322`: +```bash +sed -i '/source.*pos-ai-hook\.sh/d' "$HOME/.bashrc" +sed -i '/linux_post_install.*PATH.*pos/d' "$HOME/.bashrc" +sed -i '/source.*pos\.bash/d' "$HOME/.bashrc" +``` + +**7. Document deliberately surviving artifacts:** + +Add a comment block in `pos-system-uninstall`'s `usage()` and in the scan output: +```bash +# Intentionally NOT removed (user-managed): +# - apt packages (system packages) +# - /usr/local/bin/yt-dlp (manually installed) +# - ~/.config/rclone/ (rclone manages its own config) +# - ~/.ssh/authorized_keys additions (user SSH access) +# - pos-owned config files (removed by --config tier) +# - pos-owned data files (removed by --data tier) +``` + +### Files affected + +| File | Change | +|------|--------| +| `bin/pos-system-uninstall:62-64` | Add 9 libs to scan list | +| `bin/pos-system-uninstall:238-239` | Add 9 libs to remove list | +| `bin/pos-system-uninstall:248-250` | De-hardcode entertainment plugins | +| `bin/pos-system-uninstall:260-262` | Keep prebuilt binaries (already correct, keep as-is for now) | +| `bin/pos-system-uninstall` (after 311) | Add ScaleTail dir + flags dir removal | +| `bin/pos-system-uninstall` (after 311) | Add USER systemd unit scan + removal | +| `bin/pos-system-uninstall:72-74` | De-hardcode entertainment plugin scan | +| `bin/pos-system-uninstall:333` | Fix `.bash_completion` over-broad removal | +| `bin/pos-system-uninstall:320-322` | Fix `.bashrc` anchoring | +| `bin/pos-system-uninstall` (usage) | Document deliberately surviving artifacts | + +### Acceptance criteria + +1. `scan_tier1` shows all 12 libs (3 original + 9 new) when installed. +2. `remove_tier1` removes all 12 libs from `/usr/local/bin/`. +3. After `remove_tier1`, `/usr/local/share/linux_post_install/` is empty or removed. +4. After `remove_tier1`, no `pos-*` user units remain in `~/.config/systemd/user/`. +5. `.bashrc` removal does not remove lines unrelated to pos (verified by test: a `.bashrc` with `source pos.bash` in a comment is not affected). +6. `.bash_completion` removal does not remove lines unrelated to pos (the `sed '/pos/d'` is replaced with anchored patterns). +7. `make gen && make check && make lint` all pass at 0 FAIL, 0 WARN. +8. The install manifest test (D-C `tests/uninstall-manifest.sh`) confirms: every artifact in the install scan has a corresponding removal path. + +### Risks / open questions + +- **USER systemd unit removal during `remove_tier1`:** This calls `systemctl --user` which may fail if no user session is available (e.g., running uninstall as a scheduled task). **Mitigation:** wrap in `2>/dev/null || true` (idempotent, like the system-unit path). +- **Prebuilt binaries list is still hardcoded** (`wihotspot`, `wihotspot-gui`, `create_ap`). De-hardcoding this would require a marker system similar to POS_PLUGIN, which is overengineering for 3 binaries. Keep as-is for now; document as a known limitation. +- **`/usr/local/share/linux_post_install` parent directory** is only removed if empty after removing `scale-tail` and `flags/`. This is safe because the parent is also the ScaleTail clone destination. + +--- + +## D-F: pos-ai-server Validation Semantics + +### Decision + +Expand flag validation to cover **every flag** that will appear in `ExecStart`, not just CLI-explicit ones. Distinguish between "requested" (user or config intent) and "default" (tool-emitted) flags with different error behavior. + +**1. Expand validation scope to all ExecStart flags:** + +Currently, only `REQUESTED_FLAGS` (CLI-parsed at `:295-359`) is validated at `:407-409`. The always-emitted defaults (`--port`, `--host`, `--n-gpu-layers`, `--ctx-size`, `--threads`) and config-sourced optional flags (`--gpu-threads`, `--tensor-split`, `--batch-size`, etc.) are emitted into `exec_cmd` at `:445-490` without validation. + +After this change, **all flags in `exec_cmd`** are validated. The validation split: + +- **Requested flags** (CLI OR config-sourced): If unsupported, **hard error**. The user explicitly asked for something that doesn't work — fail loudly. +- **Default flags** (emitted without user intent): If unsupported, **warn + omit**. The tool chose a default that this build doesn't support — silently degrade. + +**2. How config-sourced flags get marked "requested":** + +Add config-sourced flags to `REQUESTED_FLAGS` during config loading. After `load_config` in `cmd_start` (around `:395`), read the config file for keys that map to CLI flags and add them to `REQUESTED_FLAGS`: + +```bash +# After load_config, mark config-sourced flags as requested +if [ -n "${LLAMACPP_PORT:-}" ] && [ "$PORT" != "8088" ]; then + REQUESTED_FLAGS+=("--port") +fi +if [ -n "${LLAMACPP_HOST:-}" ] && [ "$HOST" != "127.0.0.1" ]; then + REQUESTED_FLAGS+=("--host") +fi +# ... similar for other config keys that map to flags +``` + +Wait — this is too fragile (needs manual comparison with defaults). **Better approach:** introduce a `CONFIG_REQUESTED_FLAGS` array that is populated during `load_config` when a config key that maps to a flag is actually set in the file: + +```bash +CONFIG_REQUESTED_FLAGS=() + +load_config() { + local f="$CONFIG_FILE" + [ -f "$f" ] || return 0 + while IFS='=' read -r k v; do + [ -n "$k" ] || continue + case "$k" in \#*) continue ;; esac + v="${v//$'\r'/}" + v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" + if [ -z "${!k:-}" ]; then + export "$k"="$v" + # Track config keys that map to flags + case "$k" in + LLAMACPP_PORT) CONFIG_REQUESTED_FLAGS+=("--port") ;; + LLAMACPP_HOST) CONFIG_REQUESTED_FLAGS+=("--host") ;; + LLAMACPP_CTX_SIZE) CONFIG_REQUESTED_FLAGS+=("--ctx-size") ;; + LLAMACPP_GPU_LAYERS) CONFIG_REQUESTED_FLAGS+=("--n-gpu-layers") ;; + LLAMACPP_THREADS) CONFIG_REQUESTED_FLAGS+=("--threads") ;; + # ... other config-to-flag mappings + esac + fi + done < <(grep -E '^[A-Z_]+=' "$f" || true) +} +``` + +Then in `cmd_start`: +```bash +# Merge CLI-requested and config-requested flags +ALL_REQUESTED_FLAGS=("${REQUESTED_FLAGS[@]}" "${CONFIG_REQUESTED_FLAGS[@]}") +# Deduplicate +local deduped=() +for flag in "${ALL_REQUESTED_FLAGS[@]}"; do + case " ${deduped[*]:-} " in *" $flag "*) continue ;; esac + deduped+=("$flag") +done + +# Validate requested flags (hard error on unsupported) +if [ "${#deduped[@]}" -gt 0 ]; then + validate_requested_flags "$llamacpp_bin" "$version" "${deduped[@]}" +fi + +# Validate default flags (warn + omit on unsupported) +validate_default_flags "$llamacpp_bin" "$version" +``` + +**3. New `validate_default_flags` function:** + +```bash +# validate_default_flags — for every flag that will be +# emitted by default (without user intent), check it exists in --help. +# Unsupported defaults are silently omitted from exec_cmd. +# Sets global flags: DEFAULT_PORT_OK, DEFAULT_HOST_OK, etc. +DEFAULT_FLAGS_VALIDATED=0 +validate_default_flags() { + local bin="$1" version="$2" + local help_text + help_text="$("$bin" --help 2>/dev/null)" || { + warn "Cannot obtain llama-server --help output — skipping default flag validation" + return 0 + } + DEFAULT_PORT_OK=1; DEFAULT_HOST_OK=1; DEFAULT_CTX_OK=1 + DEFAULT_GPU_OK=1; DEFAULT_THREADS_OK=1 + local check_flag() { + local flag="$1" varname="$2" + if ! printf '%s' "$help_text" | grep -qF -- "$flag"; then + warn "installed llama.cpp ${version} does not support default flag ${flag} — omitting" + eval "$varname=0" + fi + } + check_flag "--port" DEFAULT_PORT_OK + check_flag "--host" DEFAULT_HOST_OK + check_flag "--ctx-size" DEFAULT_CTX_OK + check_flag "--n-gpu-layers" DEFAULT_GPU_OK + check_flag "--threads" DEFAULT_THREADS_OK + DEFAULT_FLAGS_VALIDATED=1 +} +``` + +Then in `exec_cmd` construction (`:445-490`), wrap each default flag in a validation check: +```bash +exec_cmd="$(systemd_quote "$llamacpp_full") -m $(systemd_quote "$model")" +[ "$DEFAULT_PORT_OK" -eq 1 ] && exec_cmd+=" --port $PORT" +[ "$DEFAULT_HOST_OK" -eq 1 ] && exec_cmd+=" --host $HOST" +[ "$DEFAULT_GPU_OK" -eq 1 ] && exec_cmd+=" --n-gpu-layers $gpu_layers" +[ "$DEFAULT_CTX_OK" -eq 1 ] && exec_cmd+=" --ctx-size $CTX_SIZE" +[ "$DEFAULT_THREADS_OK" -eq 1 ] && exec_cmd+=" --threads $THREADS" +``` + +**4. `--help` unreadable → single warn + proceed (unchanged):** + +The existing `help_text` fetch at `:72-75` already handles this: `warn "Cannot obtain llama-server --help output — skipping flag validation"; return 0`. This behavior is preserved for both `validate_requested_flags` and `validate_default_flags`. + +**5. Make detected version inform error messaging (no version branching):** + +The version is already interpolated into error strings (`:84`): `"installed llama.cpp ${version} does not expose ${flag}"`. This is correct and sufficient. No version-conditional logic or compat table is added. The version is informational for the user's debugging, not a branching variable. + +**6. Fix the substring matching issue (D4, low priority):** + +Replace `grep -qF -- "$flag"` at `:83` with a word-boundary match: +```bash +if ! printf '%s' "$help_text" | grep -qE -- "(^|[[:space:]])${flag}($|[[:space:]])"; then +``` + +This prevents false-positive substring matches (e.g., `--mmap` matching `--no-mmap` if such a flag existed). Low priority but trivial to fix during this pass. + +### Files affected + +| File | Change | +|------|--------| +| `bin/pos-ai-server:21-35` | Extend `load_config` to populate `CONFIG_REQUESTED_FLAGS` | +| `bin/pos-ai-server:66-87` | Fix `validate_requested_flags` substring matching (D4) | +| `bin/pos-ai-server` (new function) | Add `validate_default_flags` | +| `bin/pos-ai-server:407-409` | Extend validation to merge CLI + config requested flags | +| `bin/pos-ai-server:445-490` | Wrap default flags in `DEFAULT_*_OK` guards | +| `DOC/POS.md` (ai server section) | Document validation behavior: requested → hard error, default → warn+omit | + +### Acceptance criteria + +1. A config file with `LLAMACPP_PORT=9999` causes `--port` to be added to `REQUESTED_FLAGS` and validated against `--help`. +2. If the binary's `--help` does not list `--port`, the server refuses to start with an error naming the flag and the detected version. +3. If the binary's `--help` does not list `--threads` (a default), `--threads` is omitted from `ExecStart` with a warning. +4. If `--help` cannot be read, all flags are accepted (warn + proceed, unchanged behavior). +5. The `grep -qF` substring issue (D4) is resolved — `--mmap` no longer matches if only `--no-mmap` appears in `--help`. +6. Generated `ExecStart` contains only flags that pass validation. +7. `make gen && make check && make lint` all pass at 0 FAIL, 0 WARN. + +### Risks / open questions + +- The baseline-compatible flag set (`--port`, `--host`, `--n-gpu-layers`, `--ctx-size`, `--threads`) is assumed universal across modern llama.cpp builds. If a very old build lacks even `--port`, the server would start with `ExecStart` containing only the binary and model — functional but unusual. This is an acceptable degradation path. +- The `CONFIG_REQUESTED_FLAGS` approach requires manually maintaining the config-key-to-flag mapping in `load_config`. If a new config key is added, the mapping must be updated. This is a small maintenance burden; document the mapping clearly. +- The default-flags guard changes the ExecStart generation logic from unconditional to conditional. If `validate_default_flags` has a bug, the ExecStart could be missing expected flags. The test suite (D-C) covers this case. + +--- + +## Recommended Execution Order + +The decisions have dependencies and independent tracks. Here is the recommended order: + +### Phase 1: Infrastructure (no behavior changes, green gates) + +1. **D-C (Test framework)** — Create `tests/` + `make test` + runner. Write skeleton test files with `skip` for features not yet implemented. Verify `make test` runs and exits 0 (all skipped). This establishes the regression safety net for all subsequent changes. +2. **D-D part 1 (Config loader)** — Add `load_env_file` to `lib/config-ui.sh`. Do NOT migrate tools yet. Verify gates green. + +### Phase 2: Security fixes (highest priority, isolated files) + +3. **D-A (Telegram/Matrix auth)** — The Telegram fix is self-contained in `bin/pos-communication-telegram-listener` + `bin/pos-communication-telegram-sender` POS_CONFIG header + config template. The Matrix fix is a one-line guard in `bin/pos-communication-matrix-listener`. Both are isolated. +4. **D-B (AI eval posture)** — Flip the confirmation default and add `--no-command-execution` flag. Touches `bin/pos-ai` + both listeners' AI bridge calls. Independent of D-A (different lines in the listeners). + +### Phase 3: Validation and correctness + +5. **D-F (pos-ai-server validation)** — Extends the validation in `bin/pos-ai-server`. Depends on D-D being complete (the config loader migration should happen first so `load_config` in `pos-ai-server` is already migrated — or do D-F's config-requested-flags addition within the hand-rolled loader before migration, then adjust during D-D migration). **Decision:** do D-F first (it modifies the hand-rolled loader in `pos-ai-server`), then D-D migration replaces it. This avoids double-touching. +6. **D-E (Install/uninstall manifest)** — Self-contained in `bin/pos-system-uninstall`. Independent of all others except D-C (the uninstall test depends on the manifest being complete). + +### Phase 4: Config loader migration + +7. **D-D parts 2-5 (Migrate loaders, collapse entertainment-lib)** — Migrate the 9 tools' hand-rolled loaders to `load_env_file`. This touches many files but each change is mechanical. Do it last so all other changes (D-A, D-B, D-F) are already in place and their config keys are stable. + +### Phase 5: Validation + +8. Run full gate suite: `make gen && make check && make lint` — verify 0 FAIL, 0 WARN. +9. Run `make test` — verify all tests pass (no more skips for implemented features). + +### Builder-track split (parallel-safe groups) + +**Group A (communication tools):** D-A Telegram + D-A Matrix → single Builder, one commit +**Group B (AI tools):** D-B (pos-ai) + D-F (pos-ai-server) → single Builder, one commit +**Group C (infrastructure):** D-C (tests) + D-D (config-ui.sh + loader) → single Builder, one commit +**Group D (uninstaller):** D-E → single Builder, one commit +**Group E (migration):** D-D tool migrations (after A+B+D complete) → single Builder, one commit + +Groups A, B, C, and D are **parallel-safe** (no file overlap). Group E depends on B and D completing (to avoid merge conflicts in `pos-ai-server` and `pos-communication-*` tools). + +--- + +## Open Items (require future decisions, not in this pass) + +1. **Trusted-group mode for Telegram:** Explicit opt-in `TELEGRAM_GROUP_MODE=true` with per-user allowlist. Deferred — the current AND-gate is sufficient for the primary use case (1:1 private chat with the owner). +2. **Lint performance optimization:** The Explorer identified ~17 hotspots in `scripts/lint-conventions.sh` (Task 1). This is a performance improvement, not a correctness fix — separate from the stabilization pass. +3. **LLAMACPP_HOST coherence (D2 from AI audit):** The server honors `LLAMACPP_HOST` but the llamacpp provider adapter and probes hardcode `127.0.0.1`. This is a behavioral coherence issue, not a security issue. Deferred — it requires changes to `lib/ai-providers/llamacpp.sh` and the server's probe paths, which is a feature-level fix. +4. **pos-ai-hf single-file download failure (D1 from AI audit):** The single-file download path does not record failures in `failed_files`, so `.hf-meta` is written for a partially-downloaded model. This is a HIGH defect but is a pure bug fix (not an architectural decision) — belongs in Builder scope directly, not in this design pass. +5. **GPG passphrase in argv (V3 from Security audit):** Fixing this requires `gpg --batch --passphrase-fd` or a temp-file approach. Behavioral change to `bin/pos-system-backup` — defer to a separate fix commit. +6. **H-002/H-003 bashrc/bash_completion over-broad removal:** Partially addressed in D-E (`.bash_completion` fix) but a more thorough marker-based approach would be ideal. The D-E fix is the minimum viable improvement. diff --git a/AgentsReport/builder/2026-09-06_f1-f3-fixes.md b/AgentsReport/builder/2026-09-06_f1-f3-fixes.md new file mode 100644 index 0000000..fb8b930 --- /dev/null +++ b/AgentsReport/builder/2026-09-06_f1-f3-fixes.md @@ -0,0 +1,119 @@ +# Builder report — Reviewer F1/F3/F4 fixes — 2026-09-06 + +## TL;DR + +- **Status: IMPLEMENTED** +- **Scope:** Reviewer block-list fixes F1 (BLOCKING), F3 (REQUIRED), F4 (SUGGESTED). No D-A/decision-pending behavior touched, no `tests/`, no `bin/pos-system-uninstall`, no `bin/pos-network-download`. +- **F1:** `--no-command-execution` added to both bridge `pos ai gemini ask` invocations (telegram `bin/pos-communication-telegram-listener:731`, matrix `bin/pos-communication-matrix-listener:466`). Parse-order validated with probes. +- **F3:** `_prompt_run_command` in `bin/pos-ai` now checks `NO_EXEC` BEFORE the "Command detected:" print and returns 0 with zero output (probe-verified). Kept the existing `return 0` rc. +- **F4:** Restored `$f:$num:` format in the two lint WARN messages (byte-identical to HEAD's strings; functional probe shows correct line numbers). +- **Docs:** minimal alignment (only literal contradictions of the fixed behavior): `DOC/POS.md:103`, `DOC/howto/ai.md:261-273`, plus the `--no-command-execution` usage text in `bin/pos-ai` — "still printed" → "neither printed nor run". +- **Gates:** bash -n OK (4 files), `make gen` rc=0 + byte-identical output across runs, `make check` OK, `make lint` **0 FAIL, 0 WARN**, `git diff --check` OK. Not committed (per brief). + +## Step 1: F1 — add `--no-command-execution` to both bridge `ask` invocations + +[DONE] + +**Change (the running subprocess now carries the flag):** +- `bin/pos-communication-telegram-listener:731`: + `timeout 120 pos ai gemini ask --no-command-execution --session "$session" --system "$AI_SYSTEM" "$prompt"` +- `bin/pos-communication-matrix-listener:466`: identical change. + +**Why that position is valid** (read `bin/pos-ai` arg parsing first): +- Dispatcher (`bin/pos:271-298`) longest-prefix matches `pos ai gemini ask ...` to `bin/pos-ai-gemini` (exists, forwarder), which execs `pos ai --provider gemini ask --no-command-execution ...`. +- `bin/pos-ai` parse loop (`bin/pos-ai:647-678`) handles flags at ANY position in the `while [ $# -gt 0 ]` case (lines 668-669 set `NO_EXEC=1; TRUST_MODE=0`; non-flag tokens become `cmd`/`args`). So the flag is accepted after the subcommand. +- Last-wins semantics vs `--trust` unchanged; bridges never pass `--trust`. + +**Probes:** +- PROBE-C: `bin/pos-ai --provider gemini ask --no-command-execution --help` → rc=0, usage shows the flag. +- PROBE-D (full chain): `PATH="$PWD/bin:$PATH" bin/pos ai gemini ask --no-command-execution --help` → rc=0, usage shows the flag (2 mentions). +- PROBE-E negative control: `bin/pos-ai --provider gemini ask --definitely-not-a-flag --help` → rc=1 (parser genuinely rejects unknown options, so C/D are not vacuously passing). +- grep: both bridge `ask` invocation command strings contain `--no-command-execution` (one occurrence each, in the executed `timeout ... pos ai gemini ask ...` line). + +## Step 2: F3 — `_prompt_run_command` NO_EXEC returns before printing + +[DONE] + +**Change (`bin/pos-ai:369-384`):** the NO_EXEC check now sits immediately after the tty guard, BEFORE the print block: + +```bash + [ -w /dev/tty ] || return 0 + if [ "${NO_EXEC:-0}" -eq 1 ]; then + return 0 + fi + printf '\n%s\n' "Command detected:" >&2 + printf ' %s\n\n' "$cmd" >&2 +``` + +The old `printf 'command execution disabled (--no-command-execution)\n' >&2` + `printf '%s\n' "$cmd"` lines are removed. rc semantics preserved: current NO_EXEC path returned 0 → still returns 0. + +**Probe (function extracted from production `bin/pos-ai`, run in tty-writable environment):** +- PROBE-A (`NO_EXEC=1`): rc=0, **stdout_bytes=0, stderr_bytes=0** — `[ -n check ]` on captured output fails as required (no output). +- PROBE-B (`NO_EXEC=0`, same environment): prints `Command detected:` + the command to stderr — proves the tty guard passed and the NO_EXEC branch is what silenced PROBE-A (differential, not vacuous). +- grep: no `command execution disabled` string remains in `bin/pos-ai`. + +## Step 3: Doc alignment for the fixed NO_EXEC behavior (literal contradictions only) + +[DONE] + +The brief permits doc edits only where a doc "literally contradicts the fixed behavior". After F3 the fixed behavior is: under `--no-command-execution` nothing is printed. These claims said "still printed" — fixed with minimal wording: + +- `bin/pos-ai` usage() `--no-command-execution` block: "The proposed command **is still printed** but never executed" → "The proposed command **is neither printed nor executed**". +- `DOC/POS.md:103`: "the block **is printed** but not run" → "the block **is neither printed nor run**"; "the detected command **is still printed** but never run" → "the detected command **is neither printed nor run**". +- `DOC/howto/ai.md:261-263` and `:270-271`: same two claims → "neither printed nor run". + +The F1-related claim (bridges rely on the flag as the structural guard) was already accurate after the F1 fix and was NOT rewritten. No other doc touched (`DOC/howto/communication.md`, `config/*.env` unchanged). + +## Step 4: F4 — restore `:num` in lint WARN messages + +[DONE] + +**Change (`scripts/lint-conventions.sh`):** added a per-file line counter (`num=0` before each inner `while IFS= read -r line` loop; `num=$((num + 1))` as the FIRST statement of the loop body — before every `continue` gate, so skipped lines still get correct numbers). The two WARN messages now emit `"$f:$num: ..."`, byte-identical to HEAD's format: +- `warn_ "$f:$num: secret-like literal assignment (manual review for hardcoded credentials)"` (line 247) +- `warn_ "$f:$num: writes to a system path (verify a VAR=\"\${VAR:-path}\" test seam exists)"` (line 297) + +No other lint behavior changed (only the two emit paths and the counters). + +**Probe (real script run from a scratch tree with planted violations):** +``` +WARN lib/violation.sh:3: secret-like literal assignment (manual review for hardcoded credentials) +WARN lib/violation.sh:4: writes to a system path (verify a VAR="${VAR:-path}" test seam exists) +0 FAIL, 2 WARN (convention lint) +``` +`:3` / `:4` are the exact line numbers of the planted `SECRET_API_KEY="abc123"` and `echo "x" > /etc/example` lines. + +## Step 5: Gates & probes + +[DONE] + +- `bash -n` on the 4 touched shell scripts: OK. +- `make gen` ×2: rc=0 both runs; generated-output hash identical before/after (``711d6379...``) → **no gen drift** from my edits. +- `make check` (`scripts/check-sync.sh`): OK. +- `make lint` (`scripts/lint-conventions.sh`): **0 FAIL, 0 WARN** (rc=0). +- `git diff --check`: OK. +- Executable bits unchanged (all 4 scripts still `-rwxrwxr-x`). +- Out-of-scope D-A behavior confirmed untouched: `bin/pos-communication-telegram-listener:761,790` and `bin/pos-communication-matrix-listener:511` still warn-based soft-fail (pending Architect decision); no edits in that region. +- Not committed (per brief). + +## Files changed + +- `bin/pos-communication-telegram-listener` (F1, line 731) +- `bin/pos-communication-matrix-listener` (F1, line 466) +- `bin/pos-ai` (F3 `_prompt_run_command`; usage text for `--no-command-execution`) +- `scripts/lint-conventions.sh` (F4, two WARN emit paths + counters) +- `DOC/POS.md` (1 sentence, literal contradiction only) +- `DOC/howto/ai.md` (2 sentences, literal contradiction only) +- `AgentsReport/builder/2026-09-06_f1-f3-fixes.md` (this report) + +## Remaining risks + +- None in-scope. The two lint WARN messages now match HEAD's `:num` format; the rest of the lint rewrite (other message classes, FAIL rules) was not touched. +- Note: `DOC/howto/ai.md:262` non-tty bullet claim ("the code block is neither printed nor run") is now accurate for current code AND for the bridges with the flag. + +## Recommended next agent + +**Orchestrator** — reviewer block-list F1/F3/F4 implemented and gate-verified; reviewer should re-run `make test` (untouched, per parallel track) and confirm F2 via the parallel D-A Architect decision before merge. + +## Changes made by Builder + +In-scope implementation only (F1, F3, F4 + literal-contradiction doc alignment). No out-of-scope modifications. \ No newline at end of file diff --git a/AgentsReport/builder/2026-09-06_llamacpp-app.md b/AgentsReport/builder/2026-09-06_llamacpp-app.md new file mode 100644 index 0000000..0ad2cde --- /dev/null +++ b/AgentsReport/builder/2026-09-06_llamacpp-app.md @@ -0,0 +1,43 @@ +# Builder Report — 2026-09-06 — llamacpp app installer + `ai` category + `pos ai server` hint wiring + +## TL;DR +- **Status:** IMPLEMENTED — all steps `[DONE]`, all gates green +- **Scope:** new `apps/ai/llamacpp.sh` installer, `ai` category plumbing (CAT_NAMES, template comment, DOC/APPS.md), 3 error-hint lines in `bin/pos-ai-server`, 1 line in `DOC/POS.md`, 1 AGENT_TODO.md Done entry +- **Real asset naming VERIFIED via live API probe** (network available): `releases/latest` is the `v0.4.0` milestone with **no binary assets** (only `nightly-tag.txt`); the binaries live on nightly `bNNNNN` prereleases as `llama--bin-ubuntu-x64.tar.gz` / `llama--bin-ubuntu-arm64.tar.gz` (`.tar.gz`, **not** `.zip`), with a top-level dir `llama-/` +- **Key adaptations vs brief:** (1) fetch `/releases?per_page=10` + scan for the first release with a matching asset instead of `/releases/latest`; (2) extract with `tar xzf --strip-components=1` instead of `unzip` — the `.zip`/`unzip` assumption is obsolete (evidence: live API + archive listing); no `unzip` apt install (wrong mutation for tar.gz assets) +- **Files changed:** `apps/ai/llamacpp.sh` (new), `apps/install.sh` (CAT_NAMES), `templates/app.sh` (comment), `DOC/APPS.md`, `bin/pos-ai-server` (3 text lines), `DOC/POS.md` (1 line), `AGENT_TODO.md` (1 entry), `AgentsReport/builder/2026-09-06_llamacpp-app.md` (new report). `make gen` changed nothing. + +## Step 1: Probe real llama.cpp release assets +- [x] Probe `api.github.com/repos/ggml-org/llama.cpp/releases/latest` + `/releases?per_page=10`, inspect archive layout +- [DONE] + +## Step 2: Write `apps/ai/llamacpp.sh` +- [x] Installer with idempotent `install_llamacpp()` / `uninstall_llamacpp()` + uninstall case +- [x] Verify `bash -n`, idempotent no-op paths +- [DONE] + +## Step 3: `ai` category plumbing +- [x] `apps/install.sh` CAT_NAMES `[ai]="AI / ML"` +- [x] `templates/app.sh` categories comment adds `ai` +- [x] `DOC/APPS.md`: categories line, count 15→16, catalog row +- [DONE] + +## Step 4: Point `pos ai server` at the installer +- [x] `bin/pos-ai-server` lines ~397, ~563, ~256 → scrcpy-style hint keeping the GitHub URL +- [x] `DOC/POS.md` ~line 125 adds "(install via `bash apps/install.sh llamacpp`)" +- [DONE] + +## Step 5: AGENT_TODO.md Done entry +- [x] Insert 2026-09-06 entry before the current first Done entry +- [DONE] + +## Step 6: Verification +- [x] `bash -n apps/ai/llamacpp.sh apps/install.sh bin/pos-ai-server` → all OK +- [x] `make gen && make gen` → idempotent, **no gen output changed** (apps installers aren't scanned by gen) +- [x] `make check` → `check-sync: OK`, rc 0 +- [x] `make lint` → `0 FAIL, 0 WARN`, rc 0 +- [x] `bash apps/install.sh --uninstall llamacpp` → resolves app, "llama.cpp not installed", rc 0, no network +- [x] `bash apps/ai/llamacpp.sh uninstall` → rc 0 (no llama-server on PATH) +- [x] Stub-guard: `bash apps/ai/llamacpp.sh` with a fake `llama-server` on PATH → "llama.cpp already installed", rc 0 (no network/sudo) +- [x] Dry-run install + uninstall with a real (stubbed) binary → parse + spawn expansion + symlink-loop body all verified +- [DONE] \ No newline at end of file diff --git a/AgentsReport/builder/2026-09-06_netprobe-fix.md b/AgentsReport/builder/2026-09-06_netprobe-fix.md new file mode 100644 index 0000000..23129e8 --- /dev/null +++ b/AgentsReport/builder/2026-09-06_netprobe-fix.md @@ -0,0 +1,52 @@ +# Builder Report — 2026-09-06: NET_PROBE unbound-variable fix + +## TL;DR +- **Status:** IMPLEMENTED (one-line scope, verified) +- **Root cause:** `NET_PROBE="${NET_PROBE:-timeout 3 bash -c 'exec 3<>/dev/tcp/$1/$2' _ 8.8.8.8 53}"` — inside the double-quoted assignment the OUTER shell expanded `$1`/`$2` at assignment time. Under `set -u` with no positional args this is an unbound-variable crash on every `pos network download` run (unless `NET_PROBE` is already exported). +- **Fix:** escape the positional markers so only the inner `bash -c` sees them — `\$1`/`\$2` in the default string. Env-var override contract (`NET_PROBE` = full command string) unchanged. +- **File changed (only):** `bin/pos-network-download` — line 31 (NET_PROBE default). Nothing else. +- **Verification:** `bash -n` OK; bare `--help` and non-network `info` path reach the assignment with `NET_PROBE` UNSET → no unbound crash; `NET_PROBE='true'` override returns 0; static inspection confirms outer shell does not expand `$1/$2`; gates green (`make gen` idempotent, `make check` OK, `make lint` `0 FAIL, 0 WARN`). +- **Not committed** (per brief). + +## Step 1: Confirm bug & scope +`bin/pos-network-download` line 31 holds the double-quoted default; `net_up()` (line 379) runs `bash -c "$NET_PROBE"`, which relies on the inner bash receiving the trailing positional args `_ 8.8.8.8 53` (`$0=_, $1=8.8.8.8, $2=53`). +Reproduced: `set -u; NET_PROBE="${NET_PROBE:-...$1/$2...}"` → `/bin/bash: line 1: $1: unbound variable`. +Confirmed the pre-existing HEAD default was `''` (no `$1/$2`) — a prior security change to the probe introduced the regression. +[DONE] + +## Step 2: Apply fix (escaped literal default) +Changed only line 31: +`NET_PROBE="${NET_PROBE:-timeout 3 bash -c 'exec 3<>/dev/tcp/\$1/\$2' _ 8.8.8.8 53}"` +The `\$` escapes keep the outer assignment from expanding `$1`/`$2`; the string stored is the literal `.../dev/tcp/$1/$2...`, so the inner `bash -c` receives the proper positional args. `net_up()` unchanged — it already passes the whole command string to the inner bash. Env override contract preserved and documented in usage() ("NET_PROBE override defaults (test seams)"). +[DONE] + +## Probes (report output) +1. **`set -u; bash -n bin/pos-network-download`** → OK (no error). +2. **Bare help, NET_PROBE UNSET** (stubbed `aria2c`/`jq`/`curl`, isolated `HOME`/`CONFIG_DIR`): `--help` exits 0, no `unbound` in output → assignment reached without crash. + **Non-network path, NET_PROBE UNSET:** `info` (no gid) → graceful `ERROR: info: gid required`, no unbound-variable crash (assignment ran at source time before dispatch). +3. **`NET_PROBE='true'` env override** → `net_up()` returns 0 (override still honored). +4. **Static default string inspection** → default stored as `timeout 3 bash -c 'exec 3<>/dev/tcp/$1/$2' _ 8.8.8.8 53` (literal `$1/$2`, NOT expanded by outer shell). Simulated probe with `echo inner sees $1 $2` → inner bash prints `8.8.8.8 53`, proving positional probe still correct. +5. **`--dry-run`/usage unaffected** — neither references NET_PROBE; usage() unchanged by this fix. +[DONE] + +## Step 3: Repo gates +- `bash -n bin/pos-network-download` → OK. +- `make gen` → ran; re-ran: **idempotent** (byte-identical diff before/after), i.e. no new output. Net-probe is an env var, not a `# POS:` header, so it feeds nothing. +- `make check` (`scripts/check-sync.sh`) → OK. +- `make lint` (`scripts/lint-conventions.sh`) → `0 FAIL, 0 WARN`. +[DONE] + +## Scope compliance +- Approved scope: `bin/pos-network-download` ONLY (NET_PROBE lines + net_up). +- Change made: exactly the NET_PROBE default string (line 31). `net_up` confirmed correct, no change required. +- Out-of-scope changes: none made by Builder. (Working tree contains pre-existing unstaged changes from prior agents — not authored here.) +- Not committed (per brief). + +## Remaining risks / follow-up +- None for this fix. The env-var override string is operator-controlled; passing an invalid command there is the operator's responsibility (unchanged behavior). + +## Recommended next agent +Orchestrator — task is complete, targeted verification and all gates pass; no cross-track coordination needed. + +## Changes made by Builder +- `bin/pos-network-download`: escaped `$1`/`$2` to `\$1`/`\$2` in the `NET_PROBE` default so the inner `bash -c` (not the outer shell) performs the probe-host positional expansion. diff --git a/AgentsReport/builder/2026-09-06_stab-ai.md b/AgentsReport/builder/2026-09-06_stab-ai.md new file mode 100644 index 0000000..dd1bf7c --- /dev/null +++ b/AgentsReport/builder/2026-09-06_stab-ai.md @@ -0,0 +1,96 @@ +# Builder Report — AI Stabilization Fixes (2026-09-06) + +## TL;DR + +- **Status: IMPLEMENTED** (all steps DONE; gates green; probes pass) +- Scope: D-B (eval posture), D-F (pos-ai-server validation), D1 (hf single-file honesty), D2 (LLAMACPP_HOST coherence), M1/M2/M4 (POS metadata), docs (POS.md ai section + howto/ai.md eval section). +- Files touched: `bin/pos-ai`, `bin/pos-ai-server`, `bin/pos-ai-hf`, `bin/pos-ai-gemini`, `bin/pos-ai-openrouter`, `bin/pos-ai-llamacpp`, `lib/ai-providers/llamacpp.sh`, `DOC/POS.md`, `DOC/howto/ai.md` + `make gen` output (`completions/pos.bash`, `DOC/AGENT_Context_Project.md`). +- Verification: `bash -n` OK on all 7 scripts; `make gen` idempotent (×2); `make check` → `check-sync: OK`; `make lint` → `0 FAIL, 0 WARN (convention lint)`; 7 probe scenarios pass (4 server-validation, 3 hf download), plus 6 functional D-B prompt-posture tests under a pty. +- Residual note: `scripts/lint-conventions.sh` is being actively refactored by its owning track (uncommitted, 213-line diff). During my gate sweeps it intermittently flapped (parse error at line 117 early on; later runs briefly reported a stale lint self-test fixture `bin/pos-zz-linttest-broken` that never existed on disk at `ls` time). Final sweep: `0 FAIL, 0 WARN`. None of this track's scripts ever appeared in lint FAIL/WARN lists. + +--- + +## Step 1: D-B — AI eval posture (`bin/pos-ai` + help + docs) + +- Flipped `_prompt_run_command` confirm default to deny: prompt is `[y/N]`, only `[yY]` executes; anything else adds to history ("Command added to history — press ↑ to recall, edit, and run."). +- Non-tty fail-safe kept (`[ -w /dev/tty ] || return 0`) and verified it sits BEFORE the trusted auto-exec branch, so `--trust` cannot auto-execute without a tty. +- Added `--no-command-execution` flag (init `NO_EXEC=0`, parsed in arg loop, `# POS_FLAGS:` header, usage row, help text). When set, `_prompt_run_command` prints the block + "command execution disabled (--no-command-execution)" + the command, returns 0 — never executes. +- `--trust`/`--no-command-execution` last-one-wins in the arg loop (`--trust` sets TRUST_MODE=1 NO_EXEC=0; `--no-command-execution` sets NO_EXEC=1 TRUST_MODE=0). +- Help text for `--trust` updated: interactive-terminals-only semantics. +- Docs: `DOC/howto/ai.md` (shared-flags mention + new "Command execution posture" section), `DOC/POS.md` (command execution posture paragraph after Messaging bridges). + +[DONE] + +## Step 2: D-F — validate ALL ExecStart flags (`bin/pos-ai-server`) + +- Added `CONFIG_REQUESTED_FLAGS` tracking right after `load_config` (before CLI parse): config/env-set `LLAMACPP_PORT/LLAMACPP_HOST/LLAMACPP_CTX_SIZE/LLAMACPP_GPU_LAYERS/LLAMACPP_THREADS` map to `--port/--host/--ctx-size/--n-gpu-layers/--threads` via `requested_from_env_config` (dedupes). +- `cmd_start` merges CLI `REQUESTED_FLAGS` + `CONFIG_REQUESTED_FLAGS`, dedupes, hard-validates the requested set (unsupported → hard `err` naming flag + detected version). +- New `validate_default_flags` checks the 5 always-emitted defaults NOT in the requested set against `--help`; unsupported default → single `warn` + omit (sets `DEFAULT_PORT_OK/DEFAULT_HOST_OK/DEFAULT_GPU_OK/DEFAULT_CTX_OK/DEFAULT_THREADS_OK`); unreadable `--help` → `warn` + all OK=1. +- `exec_cmd` only emits flags whose `DEFAULT_*_OK=1`; requested (hard-validated) flags always emitted. +- D4: word-boundary match `(^|[[:space:]])flag([[:space:]]|=|$)` in both `validate_requested_flags` and `validate_default_flags`. +- Docs: POS.md server Flags paragraph rewritten (requested-vs-default validation semantics). + +[DONE] + +## Step 3: D1 — `pos-ai-hf` single-file failure honesty + +- Single-file path now records failures into `failed_files` (was `warn; continue` only); removed the dead `if [ "$file_count" -gt 1 ]` branch (multi-file no longer touches the sequential path). +- `.hf-meta` suppressed when `failed_files` non-empty (pre-existing guard, now effective for single-file too): "Not writing .hf-meta — is incomplete (K file(s) failed)". +- Summary rewritten: any failure yields honest "📥 Downloaded: (N of M files, K failed: ...)" + "📁 /"; single-file successful summary kept ("/ (size)") only when no failures. +- `return 1` now fires for single-file failures too (was parallel-only), so scripts can detect partial failures on both paths. + +[DONE] + +## Step 4: D2 — `LLAMACPP_HOST` coherence + +- `lib/ai-providers/llamacpp.sh`: `provider_default_model`, `provider_generate`, `provider_models_list` all use `host="${LLAMACPP_HOST:-127.0.0.1}"` + `port="${LLAMACPP_PORT:-8088}"` (was hardcoded 127.0.0.1). +- `bin/pos-ai-server`: `check_health` and `cmd_status` `/v1/models` probe use `$HOST`/`$PORT` (same as the bind address). + +[DONE] + +## Step 5: M1/M2/M4 — POS metadata headers + +- M1: `bin/pos-ai-hf` gained `# POS_SUBCMDS: search download list remove info files cache`. +- M2: `bin/pos-ai` `# POS_SUBCMDS:` gained `llamacpp` (real dispatched subcommand — `pos ai llamacpp`). +- M4: forwarders — openrouter `ask chat sessions capture models providers`; gemini `ask chat models sessions capture providers`; llamacpp `ask chat models sessions capture providers` (all passthrough to `pos ai` already worked; headers now advertise the full set). + +[DONE] + +--- + +## Verification + +### Gates + +- `bash -n` on all 7 touched scripts: PASS (all OK). +- `make gen` ×2: PASS, idempotent (`gen-docs: write OK` both runs). +- `make check`: PASS — `check-sync: OK`, rc 0. +- `make lint`: PASS — `0 FAIL, 0 WARN (convention lint)`, rc 0. + - Note: an earlier run failed because working-tree `scripts/lint-conventions.sh` (modified by another track, off-limits) had a `[[ =~ .*<<-?… ]]` parse error at line 117. That track has since fixed it; gate green. Head-version lint also passed this track's scripts when the issue was live. + +### Probes — pos-ai-server flag validation (stub llama-server, version 1.2.3; env seams `CONFIG_FILE`, `USER_SYSTEMD_DIR`, `DRY_RUN=1`) + +1. `--split-mode` (unknown CLI flag) → rc 1, `ERROR: Unknown option '--split-mode' (see --help)`. +2. `--tensor-split 1:2:3` with help lacking it → rc 1, `ERROR: installed llama.cpp 1.2.3 does not expose --tensor-split — remove it or upgrade llama.cpp`. +3. Config `LLAMACPP_CTX_SIZE=2048` with help lacking `--ctx-size` → rc 1, `ERROR: installed llama.cpp 1.2.3 does not expose --ctx-size — remove it or upgrade llama.cpp`. +4. Help lacking `--threads` (never requested) → rc 0, warning `installed llama.cpp 1.2.3 does not support default flag --threads — omitting it from the unit`; dry-run ExecStart contains NO `--threads` (only `--port 8088 --host 127.0.0.1 --n-gpu-layers 0 --ctx-size 4096`). +5. No llama-server on PATH → rc 1, `ERROR: llama-server not found — install llama.cpp … see 'pos help ai server'`. + +### Probes — pos-ai-hf download (stub curl; HF_DOWNLOAD_DIR temp dir) + +6. Single-file download, `/resolve/` fails → rc 1; stderr shows `Failed to download model.gguf`, `Not writing .hf-meta — ns/test-model is incomplete (1 file(s) failed)`; summary `📥 Downloaded: ns/test-model (0 of 1 files, 1 failed: model.gguf)`; NO `.hf-meta` written. +7. Single-file success control → rc 0, meta written (`{"repo_id": … ,"files":["model.gguf"]}`). +8. Parallel 2-file, one fails → rc 1, honest summary `(1 of 2 files, 1 failed: b.gguf)`, NO `.hf-meta`. + +### Probes — D-B prompt posture (real `_prompt_run_command` body extracted from `bin/pos-ai`, run under `script` pty) + +- Non-tty (plain bash, no pty): nothing executes — `/dev/tty` read fails → auto-decline, rc 0. +- `NO_EXEC=1` pty: prints "command execution disabled (--no-command-execution)", prints command, no execution. +- `--trust` pty: auto-executes without prompt. Non-tty `--trust`: no execution. +- `y`: executes. Enter / `n`: declines, adds to history, no execution. + +### Remaining risks / notes + +- `--split-mode` probe hits the arg-parser path (pre-existing behavior), not the D4 validation path; genuinely unsupported requested flags are covered by probes 2/3. +- `--model`/`-m` and `--gpu-threads` handling predates this change and is unchanged; only flags listed in `REQUIRED_FLAGS`/defaults are validated. +- `DOC/POS.md` and the gen artifacts also carry concurrent edits from the other active tracks (app installer message, listeners, etc.) — not part of this report's scope. \ No newline at end of file diff --git a/AgentsReport/builder/2026-09-06_stab-config.md b/AgentsReport/builder/2026-09-06_stab-config.md new file mode 100644 index 0000000..c598d94 --- /dev/null +++ b/AgentsReport/builder/2026-09-06_stab-config.md @@ -0,0 +1,152 @@ +# Builder Report — 2026-09-06 — D-D Config-Loader Centralization + +## TL;DR + +Status: IMPLEMENTED + +Objective: Canonical `load_env_file` in `lib/config-ui.sh`; migrate 9 hand-rolled loaders; collapse entertainment-lib read/write; decide load_system_env/download/docker-compose adoption; docs; probes. + +Files changed (this task): `lib/config-ui.sh` (+loader, +supersession note), 9 migrated tools (`pos-ai`, `pos-ai-hf`, `pos-ai-server`, `pos-communication-{telegram-sender,matrix-listener,telegram-listener,matrix-sender,scrcpy}`, `pos-media-grab`), `pos-network-download` (`load_secret` + inline duplicate), `lib/entertainment-lib.sh` (collapsed helpers + config-ui.sh source), `DOC/DEV.md` (canonical-loader paragraph), `DOC/AGENT_Context_Project.md` (hand-maintained line-count row via `make gen`), this report. + +Verification: `bash -n` all touched files OK; `make gen` idempotent (zero diff between runs); `make check` OK; `make lint` 0 FAIL, 0 WARN; behavioral probes: telegram-sender + pos-ai + pos-network-download env-wins/CRLF/comments OK; ai-server D-F tracking survives (hard error names `--ctx-size`); entertainment-lib wrapper smoke OK (read/quote-strip/CR-strip/write/delete/chmod-600); docker-compose confirmed NOT migrated (`source` semantics preserved). + +Follow-ups (out of scope, flagged): `bin/pos-network-download` NET_PROBE line (pre-existing, from a parallel uncommitted change) crashes under `set -u` — `$1`/`$2` in a single-quoted default inside `${NET_PROBE:-...}` are expanded by the outer shell; every invocation without the env var set hits "unbound variable". Owner: the parallel track that introduced `exec 3<>/dev/tcp/$1/$2`. Also residual hand-rolled loops remain ONLY in `lib/common.sh` (`load_system_env` — kept per D-D) and `lib/entertainment-plugin-lib.sh` (plugin-only lib, must stay self-contained — by design). + +--- + +## Step 0: Scope confirmation + +Read the inputs per AGENTS.md before touching code: `AgentsReport/architect/2026-09-06_stabilization-design.md` (D-D section, esp. the loader contract at ~line 310-350 and the XDG-path replacement rule at line 326-328), `AgentsReport/explorer/2026-09-06_tooling-audit.md` (Task 3) + `AgentsReport/explorer/2026-09-06_ai-audit.md` (M3), plus the current working-tree state of every target file (repo carries uncommitted changes from parallel tracks D-A/D-B/D-F/security — always read before editing). + +Rules honored: don't touch `scripts/lint-conventions.sh`, `bin/pos-system-uninstall`, `tests/`, `apps/`, `AGENT_TODO.md`, listeners' authorization logic; `DOC/AGENT_Context_Project.md` generated blocks only through `make gen`; gates = `bash -n` + `make gen`×2 idempotent + `make check` + `make lint` (0 FAIL, 0 WARN); self-contained tools get `source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh"`. + +[DONE] + +--- + +## Step 1: `load_env_file` in `lib/config-ui.sh` + +Added the canonical loader immediately before `cfg_value` (now `lib/config-ui.sh:331-356`). Contract, matching the D-D design and every migrated tool's historic behavior: + +- `load_env_file [scope]` — `[scope]` is informational/reserved. +- A bare basename (no `/`) resolves under `${CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/linux_post_install}/`; full paths used as-is. +- Reads `^[A-Z_]+=` lines; skips blank + `#` comments; strips CR from every value (CRLF-safe); trims one pair of surrounding quotes. +- Exports a key ONLY when `${!k:-}` is empty → env always wins over file (precedence: CLI flags > env > file > defaults — CLI/defaults stay in the tools). +- Appends loaded keys to global `LOADED_ENV_KEYS` (init guarded via `declare -p`, `set -u`-safe). +- Never creates files, never chmods (chmod-600 stays with `cfg_write`); missing/unreadable file = quiet no-op, rc=0. +- Doc comment notes it SUPERSEDES `lib/common.sh`'s `load_system_env()` (kept for its 3 existing callers per D-D §"Deprecate load_system_env"; `common.sh` must NOT source config-ui.sh — circular-dependency risk). + +Smoke-tested in `/tmp/opencode/cfg-probe` before wiring tools: env-wins, CRLF strip, comments, XDG basename vs absolute path, missing-file rc=0, `LOADED_ENV_KEYS` content (only actually-loaded keys). + +[DONE] + +--- + +## Step 2: Migrate the 9 hand-rolled loaders + +Per-tool migration (each = add config-ui.sh source + replace the hand-rolled while-read body with `load_env_file`, preserving tool-specific extras): + +| Tool | Before (loader) | After | Tool-specific preservation | +|---|---|---|---| +| `pos-communication-telegram-sender` | inline env-wins loop (no CR strip) | `load_config() { load_env_file "$CONFIG_FILE"; }` | — (self-contained: owns guarded CONFIG_DIR) | +| `pos-communication-matrix-listener` | inline env-wins loop (no CR strip) | `load_config() { load_env_file "$CONFIG_FILE"; }` | — | +| `pos-communication-telegram-listener` | inline env-wins loop (no CR strip) | `load_config() { load_env_file "$CONFIG_FILE"; }` | — | +| `pos-communication-matrix-sender` | inline env-wins loop + CR strip | `load_config() { load_env_file "$CONFIG_FILE"; }` | — | +| `pos-communication-scrcpy` | inline env-wins loop + CR strip; hardcoded `$HOME/.config/.../scrcpy.env` | `load_config() { load_env_file "$CONFIG_FILE"; }`; `CONFIG_FILE="$CONFIG_DIR/scrcpy.env"` | XDG path fix (design line 328) | +| `pos-ai` | `load_config` loop + legacy two-file loop | `load_env_file "$CONFIG_FILE"` + legacy loop stays as second `load_env_file` per legacy path | legacy `LEGACY_GEMINI_CONFIG`/`LEGACY_OPENROUTER_CONFIG` loop structure kept; both legacy vars now `:-`-guarded with `$CONFIG_DIR` defaults; `CONFIG_FILE` gets env-override seam (`${CONFIG_FILE:-$CONFIG_DIR/ai.env}`) | +| `pos-ai-server` | `load_config` loop + CR strip | `load_env_file "$CONFIG_FILE"` | D-F tracking untouched: `CONFIG_REQUESTED_FLAGS`/`requested_from_env_config` is VALUE-based (reads exported LLAMACPP_* after load) → unaffected by loader; `CONFIG_FILE="${CONFIG_FILE:-$CONFIG_DIR/ai.env}"` keeps the env-override seam (`tests/t-ai-server-flags.sh` passes `CONFIG_FILE=…`) and fixes XDG | +| `pos-ai-hf` | `load_hf_config` loop + CR strip | `load_hf_config() { load_env_file "$CONFIG_FILE"; }` | `CONFIG_FILE` `:-`-guarded, `$CONFIG_DIR` default | +| `pos-media-grab` | `load_grab_config` loop (no CR strip); hardcoded `$HOME/.config/.../grab.env` | `load_grab_config() { load_env_file "$CONFIG_DIR/grab.env"; }` | XDG path fix | + +Acceptance criterion 1 (D-D): `grep -rn 'while IFS.*read.*k.*v' bin/pos-communication-* bin/pos-ai* bin/pos-media-grab` → **zero matches** (only `lib/config-ui.sh:346` — the canonical loader — and `lib/common.sh:149` `load_system_env` + `lib/entertainment-plugin-lib.sh:25` remain anywhere in bin/+). + +[DONE] + +--- + +## Step 3: `pos-network-download` `load_secret` + inline duplicate + +- Added config-ui.sh source. +- `load_secret()` now `load_env_file "$CONFIG_FILE"` + `RPC_SECRET="${RPC_SECRET:-}"` normalization (kept) — env-wins identical to the old guard; CR/quote-strips are no-ops for the machine-written unquoted hex file. +- The duplicate inline `grep RPC_SECRET` in `cmd_start` replaced with a `load_secret` call (guarded on `[ -z "$RPC_SECRET" ]` like before). + +[DONE] + +--- + +## Step 4: `pos-docker-compose` — NOT migrated (deliberate) + +`load_global_config` still `source "$CONFIG_ENV"` (`bin/pos-docker-compose:90`). Converting to `load_env_file` would change behavior: the compose.env file is shell-EXECUTED (variable expansion, file-beats-defaults layering, and `config set` writes unquoted `TS_AUTHKEY=…` values that the loader's quote/CR handling + env-wins would invert). Documented as the deliberate exception in `DOC/DEV.md` (new canonical-loader paragraph). Also noted but NOT changed (out of D-D loader scope): `pos-docker-compose:10` still hardcodes `$HOME/.config/...` for `CONFIG_ENV`. + +[DONE] + +--- + +## Step 5: `lib/entertainment-lib.sh` collapse + +- `config_value`/`write_config_key` are now thin wrappers over `cfg_value`/`cfg_write` (`lib/entertainment-lib.sh:33-43`) — identical call signatures, chmod-600 preserved, `"-"` delete supported. +- entertainment-lib now sources `config-ui.sh` via a 4-way fallback chain mirroring the user-timers source block. +- Documented minor delta: `cfg_write` emits a stderr-only warning on multi-line paste and truncates to the first line (old `write_config_key` truncated silently); write RESULT identical. +- `config_value` now also strips a trailing CR (old entertainment version did not) — alignment with the other migrated tools. +- Pre-existing contract documented in the lib header: must be sourced AFTER `lib/common.sh` (defines CONFIG_DIR); verified still true. +- Verified callers: `bin/pos-entertainment-config` (1-arg `config_value`, 2-arg `write_config_key`, `"-"` delete at line 77) and `bin/pos-entertainment-status` all match the new signatures. + +[DONE] + +--- + +## Step 6: Docs + +- `DOC/DEV.md` §"Config files (if needed)": added the canonical-loader bullet (precedence contract, CR/quote handling, LOADED_ENV_KEYS, XDG basename resolution, fallback source pattern, load_system_env supersession, docker-compose exception). +- `DOC/AGENT_Context_Project.md`: hand-maintained `lib/entertainment-lib.sh` line-count row updated (311 → 300); `make gen` regenerated the filetable counts for the migrated tools (byte-order deterministic: second run produced zero diff) and `completions/pos.bash`. + +[DONE] + +--- + +## Step 7: Gates + +- `bash -n` on all 12 touched code files: OK. +- `make gen`: run 1 vs run 2 diff — zero bytes (idempotent). +- `make check` (`scripts/check-sync.sh`): OK. +- `make lint` (`scripts/lint-conventions.sh`): `0 FAIL, 0 WARN`. + +[DONE] + +--- + +## Step 8: Behavioral probes (all PASS) + +Stub harness in `/tmp/opencode/cfg-probe/` (XDG_CONFIG_HOME + stub curl/llama-server/systemctl/aria2c; throwaway per DEV.md): + +1. **telegram-sender wiring** — telegram.env with `# comment`, blank line, `TELEGRAM_BOT_TOKEN=file-token\r\n`, `TELEGRAM_CHAT_ID=1289`; exported `TELEGRAM_CHAT_ID=env-chat`. Stub curl logged: `botfile-token` in URL (CR stripped, comment skipped) and `chat_id=env-chat` (env wins over file). Zero CR anywhere. +2. **pos-ai wiring** — ai.env with `AI_PROVIDER=gemini` + `AI_GEMINI_API_KEY=file-key\r\n`; `pos ai ask hello` with stub curl → `stub-reply` printed; header `x-goog-api-key: file-key` (CR stripped from file). With `AI_GEMINI_API_KEY=env-key` exported → header `env-key` (env wins). +3. **ai-server D-F survival** — ai.env `LLAMACPP_CTX_SIZE=8192\r\n`; stub `llama-server` whose `--help` lacks `--ctx-size`; `pos-ai-server start whatever.gguf` → `ERROR: installed llama.cpp 1.2.3 does not expose --ctx-size — remove it or upgrade llama.cpp`, exit 1. D-F "hard error on config/env-requested but unsupported flag" contract intact after the loader migration. +4. **network-download load_secret** — download.env `RPC_SECRET=file-secret\r\n`; daemon faked active via stub systemctl; `status` → RPC payload `token:file-secret` (CR stripped, from file); with `RPC_SECRET=env-secret` exported → `token:env-secret` (env wins). +5. **docker-compose** — spot-confirmed `source "$CONFIG_ENV"` unchanged (deliberate). +6. **entertainment-lib wrappers** — direct smoke: `config_value` read (CR strip), quote-stripped read, `write_config_key` overwrite, `"-"` delete, chmod-600 preserved; plus `pos entertainment status` end-to-end (ENABLED `weather,1h` parsed, "every 1h" interval). +7. **residual-loader rate** — only canonical `load_env_file` + kept `load_system_env` + plugin-only `entertainment-plugin-lib` remain. + +[DONE] + +--- + +## Step 9: Discoveries (out of scope, flagged — not fixed) + +- `bin/pos-network-download` NET_PROBE default `timeout 3 bash -c 'exec 3<>/dev/tcp/$1/$2' _ 8.8.8.8 53` (introduced by a PARALLEL uncommitted change; HEAD has `''`): under `set -u`, the single quotes inside `${NET_PROBE:-…}` are literal, so the OUTER shell expands `$1`/`$2` (unset) → `line 31: $2: unbound variable` on every run without the env var set. Real blocker for the tool's runtime; owner = the parallel track (likely the retry/healer work). D-D probes bypassed it by exporting a non-empty NET_PROBE — do not fix here (unapproved component boundary, unclear ownership). + +[DONE] + +--- + +## Scope compliance + +In-scope: loader addition + 9-tool migration + network-download load_secret + entertainment-lib collapse + config-ui.sh supersession note + DOC/DEV.md canonical-loader paragraph + hand-maintained line-count row + report. Out-of-scope changes: none (NET_PROBE bug intentionally NOT fixed; docker-compose NOT migrated; `load_system_env` NOT rewritten). + +Remaining risks / follow-ups: +1. NET_PROBE `set -u` crash (see Step 9) — needs the owning parallel track / Architect decision. +2. `pos-docker-compose` `CONFIG_ENV` still hardcodes `$HOME/.config/…` (pre-existing; XDG seam gap for that tool; explorer-flagged, not in D-D's 9). +3. `lib/entertainment-plugin-lib.sh:25` keeps its own read loop (plugin contract: plugins must not source config libs) — an intentional residual, could get a documented rationale later. +4. `load_system_env` remains duplicated logic in `lib/common.sh:146-159` — supersession documented, legacy callers (`pos system health/backup`, `pos media sync`) untouched. + +Recommended next agent: **Reviewer** (independent adversarial review of the loader migration + wrapper collapse) — or **Orchestrator** if the NET_PROBE ownership question should be routed to the owning track first. Not handed to Architect: no scope/design boundary dispute remains after the decisions above were recorded. \ No newline at end of file diff --git a/AgentsReport/builder/2026-09-06_stab-security.md b/AgentsReport/builder/2026-09-06_stab-security.md new file mode 100644 index 0000000..a230555 --- /dev/null +++ b/AgentsReport/builder/2026-09-06_stab-security.md @@ -0,0 +1,70 @@ +# Builder report — 2026-09-06 stabilization security track (D-A, V3, V5, V7) + +## TL;DR + +Status: **IMPLEMENTED** — all approved changes done, gates green, 23/23 probes PASS. +Scope: architect decision D-A (telegram owner gate, matrix fail-closed) + explorer findings V3 (backup gpg argv leak), V5 (/dev/tcp host injection), V7 (documented notice); plus template-creation and two hygiene fixes listed in the brief. +Verified: `bash -n` on all touched scripts, `make gen`, `make check` (check-sync: OK), `make lint` (0 FAIL, 0 WARN), runtime registry smoke (`pos config telegram` scope shows `TELEGRAM_OWNER_ID`), and a 23-assertion probe harness (fail-closed telegram/matrix behavior, gpg fd + cleanup, host-injection attempts) — all PASS. +Files changed: 9 code/doc files in scope, 2 new config templates, plus `DOC/AGENT_Context_Project.md` + `completions/pos.bash` from the required `make gen`. + +## Step 1: D-A telegram listener owner gate — [DONE] + +- `bin/pos-communication-telegram-listener`: per-message authorization replaced by fail-closed AND-gate: message processed only when `chat == TELEGRAM_CHAT_ID` AND `from_id == TELEGRAM_OWNER_ID`; unauthorized → warn-only, skip (no reply) (`bin/pos-communication-telegram-listener:805`). +- Owner unset → daemon runs but every command is ignored with warn "TELEGRAM_OWNER_ID unset — ignoring command" (`bin/pos-communication-telegram-listener:798`); startup logs "commands authorized" or warns fail-closed; run line now logs owner (`bin/pos-communication-telegram-listener:518-530` region). +- `bin/pos-communication-telegram-sender` registry line gained `TELEGRAM_OWNER_ID=digits:Numeric Telegram user id (your account) allowed to run chat commands`; usage() config block updated. +- Probes 1a–1d prove: owner-unset → warning + no send; right chat+user → executes; wrong user → ignore; wrong chat → ignore. 8/8 PASS. + +## Step 2: D-A matrix listener fail-closed — [DONE] + +- `bin/pos-communication-matrix-listener`: with `MATRIX_ROOM_ID` unset the daemon runs but shows `room none — fail-closed` and watches NO room; the room filter is now fail-closed (`[ -z "$room_only" ] || [ "$room" != "$room_only" ]`), startup warn at `bin/pos-communication-matrix-listener:519`. +- Probe 2 (owner message, room unset) proves: warning logged, run line shows fail-closed, nothing answered. 3/3 PASS. + +## Step 3: Config templates + sender registry — [DONE] + +- `config/telegram.env` and `config/matrix.env` created as commented reference templates (config/ai.env style) since no templates existed. +- Verified `postinstall.sh:22-50` copies only explicitly named `entertainment.env/system.env/notify.env/ai.env` (no glob) — templates are NOT auto-installed; runtime provisioning remains `pos config telegram` / `pos config matrix`. Residual gap: postinstall.sh does not copy the new templates (out of scope). + +## Step 4: V3 backup gpg passphrase fd — [DONE] + +- `bin/pos-system-backup`: both gpg calls use `--passphrase-fd 3` + `3<<<"$PASS"` (`bin/pos-system-backup:197`, `:207`); failed encrypt removes the plaintext archive + err "encryption failed — plaintext archive removed, nothing left behind"; failed verify removes the corrupt `.gpg` + err; `unset PASS` retained. +- Probes: gpg argv has `--passphrase-fd 3`, no `--passphrase `, secret absent from argv/logs; success path leaves only `.gpg` (chmod 600); fail path leaves neither plaintext nor partial artifact, with honest error. 8/8 PASS. + +## Step 5: V5 /dev/tcp host injection — [DONE] + +- Positional-arg form everywhere a remote host reaches `bash -c 'exec 3<>/dev/tcp/…'`: + - `bin/pos-network-checkport:135` (check_tcp), `:168`/`:170` (banner probes) — `_ "$ip" "$port"`. + - `bin/pos-share-smb-client:94` — `_ "$host" "$SMB_PORT"`. + - `lib/share-lib.sh:61` (share_port_probe, shared core) — `_ "${1}" "${2}"`. + - `bin/pos-network-download:28` — default `NET_PROBE` now `timeout 3 bash -c 'exec 3<>/dev/tcp/$1/$2' _ 8.8.8.8 53`. +- POS.md NET_PROBE doc updated at line 188 (also fixed pre-existing missing `>` in the doc example). +- Probes: hostile host `8.8.8.8;touch …` passed as ONE literal arg, probe source uses `$1/$2`, no marker file created, hostile connect fails harmlessly; same for `share_port_probe` and smb-client `probe_server`. 8/8 PASS. + +## Step 6: V7 notice + doc/hygiene updates — [DONE] + +- `DOC/howto/communication.md`: one-time Telegram setup notes `TELEGRAM_OWNER_ID` (set via `pos config telegram`) and the inherent token-in-argv caveat of Bot API URLs (revoke if leaked); owner-only bullet for chat commands; matrix self-messaging bullet. +- POS.md: telegram-listener row (owner id), matrix-listener row (fail-closed), backup row (passphrase on internal fd — never argv), telegram/matrix paragraphs (~372/402). +- Hygiene: `apps/ai/llamacpp.sh` chmod 755 (still untracked); `DOC/APPS.md` line 3 app count 16 → 18. + +## Step 7: Gates + probes — [DONE] + +- `bash -n` on all 10 touched scripts: OK. +- `make gen` OK (required by POS_CONFIG change; also folded in parallel-track drift), then regeneration re-verified. +- `make check`: OK. `make lint`: 0 FAIL, 0 WARN. +- Probe harness `/tmp/opencode/probe-stab.sh` (23 assertions): **PASS=23 FAIL=0**. Harness uses stub `curl`/`gpg`/`sudo` (secret-free argv assertions), extracted real function bodies verbatim for checkport/smb-client/share-lib, and verifies no marker file is created by hostile hosts. + +## Residual risks / follow-up + +- `config/telegram.env` / `config/matrix.env` are reference templates only — not wired into `postinstall.sh` (out of scope; installer currently copies only 4 explicit env files). +- Existing deployments without `TELEGRAM_OWNER_ID` / `MATRIX_ROOM_ID` now ignore all chat commands (INTENDED fail-closed; startup warn tells the operator to run `pos config telegram` / `pos config matrix`). +- Matrix-sender unchanged (its registry already listed MATRIX_ROOM_ID). +- Parallel tracks still dirty in git (ai track, app templates, lint-conventions, AGENT_TODO, docs) — not touched here. + +## Files changed (this track only) + +- `bin/pos-communication-telegram-listener`, `bin/pos-communication-telegram-sender`, `bin/pos-communication-matrix-listener` (D-A) +- `bin/pos-system-backup` (V3) +- `bin/pos-network-checkport`, `bin/pos-share-smb-client`, `lib/share-lib.sh`, `bin/pos-network-download` (V5) +- `config/telegram.env`, `config/matrix.env` (new templates) +- `DOC/POS.md`, `DOC/howto/communication.md`, `DOC/APPS.md` (docs) +- `DOC/AGENT_Context_Project.md`, `completions/pos.bash` (make gen output) +- `apps/ai/llamacpp.sh` (mode 755 only) \ No newline at end of file diff --git a/AgentsReport/builder/2026-09-06_stab-tooling.md b/AgentsReport/builder/2026-09-06_stab-tooling.md new file mode 100644 index 0000000..fb0f6bc --- /dev/null +++ b/AgentsReport/builder/2026-09-06_stab-tooling.md @@ -0,0 +1,100 @@ +# Builder Report — 2026-09-06 — Stabilization: Tooling track (lint perf, D-E uninstall, H-002/H-003) + +**Scope (per brief):** `scripts/lint-conventions.sh`, `bin/pos-system-uninstall`, `DOC/POS.md` (system uninstall row only if factual); NOT touched: other files listed in the brief. `lib/install-manifest.sh` is **NOT** created — see D-E. + +**TL;DR** +- Status: **DONE** — all 3 steps implemented, probed, gated. +- Files in scope changed by me: `scripts/lint-conventions.sh` (rewrite), `bin/pos-system-uninstall` (D-E/H-002/H-003), `DOC/POS.md` (system-uninstall row only), plus gen output in `DOC/AGENT_Context_Project.md` + `completions/pos.bash` (reflecting the whole shared tree, incl. parallel tracks). +- D-E explicitly says **do NOT** create `lib/install-manifest.sh` — the brief's manifest is conditional ("if D-E specifies it"); it does not, so I implement D-E's actual decision (extend `bin/pos-system-uninstall` directly, no manifest file). +- Baseline `make lint` = **145.9 s** (measured, `time`); final = **4.38 s**, `0 FAIL, 0 WARN`, output byte-identical to original (differential test). `make check` OK; `make gen` idempotent. +- H-002/H-003 probes pass: exact-literal + anchored `awk` removal removes only pos-owned lines; comments, user PATH/MY_POS_REPO lines, `repos`/`dispose`/`compose` completion, and `unrelated.service` all survive. +- Remaining risk: real-machine removal path validated via fixtures only (no live uninstall run) — see Step 3. + +## Step plan +1. Lint performance rewrite (Task 1). +2. D-E uninstall extensions + H-002/H-003 + surviving-artifacts docs (Tasks 2, 3). +3. Probes + full gates. + +--- + +## Step 1: Lint performance rewrite + +Refactored `scripts/lint-conventions.sh` hot paths to bash-native parsing: +- Merged per-line `uses_stdin` + top-level-`local` scans (was `printf | sed | tail` × 3 forks per line) into ONE single-pass per-file read with heredoc-delimiter state via `[[ =~ ]]`. +- Folded `# POS:` header / em-dash / posline / first `-h|--help` line into the same loop. +- Shebang via `read -r first < "$f"`; POS.md coverage via one preloaded `cat` + `[[ ]]`; secret-literal + system-path scans → bash `[[ =~ ]]` with cheap string gates; deleted dead `last_line()`. +- System-path outer filter (`(\btee\b|>>?)[^#]*?(path)`) reimplemented faithfully (`_syspath_outer` — no fork, global `_SI`), preserving the no-`#`-between-operator-and-path semantics that bash `=~` cannot replicate. + +**Verification:** +- `bash -n` OK. +- Differential test vs `git show HEAD:scripts/lint-conventions.sh`: **outputs byte-identical** (both exit 0, `0 FAIL, 0 WARN`). +- Negative test (broken tool): all expected FAIL/WARNs fire; skipped per-file checks on missing POS header match original `continue`. +- Timings (`time bash scripts/lint-conventions.sh`): **before 145.9 s → after 4.79 s (~30x)**. `time make lint` before: >120 s (timed out at 145.9 s for the bare script); after: see gates below. + +[DONE] + +--- + +## Step 2: D-E install/uninstall + H-002/H-003 + +### D-E reconciliation — NO `lib/install-manifest.sh` +D-E (architect decision `stabilization-design.md:399-401`) explicitly says: *"Do NOT introduce a full manifest abstraction (like `lib/install-manifest.sh`) — that would be a larger refactor inconsistent with the current 'list-based' approach"*. The brief's manifest requirement was conditional ("If D-E specifies `lib/install-manifest.sh`"); it does not. Per the scope rule (necessary-to-complete vs better-design), I implemented D-E's actual decision: **one** source of truth inside `bin/pos-system-uninstall` (singleton `POS_LIBS` array + `PATH_LINE`/`COMPLETION_LINE`/`HOOK_PATTERN` constants + `installed_plugins()` helper) used by both scan and remove — the removal list is no longer two hardcoded places. Nothing else in the codebase was introduced. + +### Changes to `bin/pos-system-uninstall` +- **Libs (D-E 1):** `POS_LIBS` = all 12 libs from install.sh phase 2 (`common.sh flags.sh notify.sh entertainment-lib.sh scheduler-lib.sh config-ui.sh user-timers-lib.sh entertainment-plugin-lib.sh usb-lib.sh share-lib.sh menu-lib.sh registry.sh`); scan + remove loops both read the single array. Verified **byte-identical** to install.sh:143. +- **ScaleTail + flags store (D-E 2):** scan + remove for `/usr/local/share/linux_post_install/scale-tail/` and `/flags/`, then `rmdir` the parent if empty. +- **User systemd units (D-E 3):** scan + remove `$XDG_CONFIG_HOME/$HOME/.config/systemd/user/pos-*` (`disable --now` + `rm`, `|| true` wrapped; `daemon-reload` at end). `unrelated.service` probe survives. +- **De-hardcoded entertainment plugins (D-E 4):** `installed_plugins()` discovers installed plugins by `# POS_PLUGIN:` marker (matching install.sh's directory-driven install), used by scan + remove. +- **H-002 (D-E 5):** `.bash_completion` `sed -i '/pos/d'` replaced with anchored `awk '/^[[:space:]]*source[[:space:]].*pos\.bash/ { next } { print }'` + mktemp + `chmod --reference` + count. +- **H-003 (D-E 6):** `.bashrc` sed removals replaced with exact-literal `awk` removal of postinstall.sh's `PATH_LINE` + `COMPLETION_LINE` and anchored `HOOK_PATTERN='^[[:space:]]*source[[:space:]].*pos-ai-hook\.sh'` (comment-safe per D-E acceptance criterion 5). Note: the old `/linux_post_install.*PATH/d` never removed the real PATH_LINE anyway (that literal has no `linux_post_install` substring); the new code removes the installer's actual line. +- **Surviving artifacts (D-E 7):** usage() documents deliberately NOT removed: apt packages, `/usr/local/bin/yt-dlp`, `~/.config/rclone/`, `~/.ssh/authorized_keys` additions, config/data tiers. +- **DOC/POS.md:** system-uninstall row Tier 1 description updated (factual): adds user units + ScaleTail + flags store. + +### Probes (all pass) +1. **H-002/H-003 fixture `.bashrc` (15 lines incl. unrelated `pos` lines + comments) → 3 lines removed** (PATH_LINE, COMPLETION_LINE, hook source); custom `MY_POS_REPO`, user PATH, `# source pos.bash` comments survive. +2. **Fixture `.bash_completion` (7 lines incl. `repos`/`dispose`/`compose`) → 1 line removed** (source pos.bash); `repos`/`dispose`/`compose` and comment survive. +3. **scan_tier1 (HOME fixture):** finds exact bashrc/bash_completion lines + `pos-aria2.service`, `pos-entertainment-weather.timer` user units; does NOT find `unrelated.service` or unrelated completion lines. +4. **remove block replication:** REMOVED_COUNT=6 (3 bashrc + 1 completion + 2 user units); unrelated content + `unrelated.service` survive; `systemctl --user` fails gracefully (`|| true`, no session). +5. **Inventory: POS_LIBS == install.sh lib list** (12 names identical); 13th `lib/pos-ai-hook.sh` is not installed by install.sh phase 2 (correctly not in POS_LIBS). + +[DONE] + +--- + +## Step 3: Gates + hygiene + +### Gate results (final state) +- `bash -n scripts/lint-conventions.sh bin/pos-system-uninstall` → OK. +- `make gen` ×2 → idempotent (`gen-docs: write OK` both runs; second run produces the same diff, no drift). +- `make check` → `check-sync: OK` (exit 0). +- `time make lint` → `0 FAIL, 0 WARN (convention lint)`, **4.38 s** (second run 6.26 s; final measured 4.38 s). Baseline was **145.9 s** for the bare script and >120 s (timeout) for `make lint` → ~30x speedup. Rapid repeat runs no longer leave the shell spinning (no more per-line fork storms). +- **Gen output note:** `make gen` regenerated `DOC/AGENT_Context_Project.md` + `completions/pos.bash` from the **shared** working tree. The diff includes my expected `bin/pos-system-uninstall` row-count update (435→517 lines) **plus** the other parallel tracks' already-uncommitted changes (`bin/pos-ai-server`, `bin/pos-ai`, `bin/pos-system-backup`, communication listeners, etc.). No POS headers/dirs were touched by me; gen is byte-order deterministic (`LC_ALL=C`), CI's `git diff --exit-code` will see the whole tree's refresh. + +### Hybrid conflict check (shared working tree) +Final `git status` shows many files modified by **parallel Builder tracks** (stab-ai, stab-security: `bin/pos-ai*`, `bin/pos-communication-*`, `bin/pos-network-checkport`, `bin/pos-share-smb-client`, `bin/pos-system-backup`, `lib/share-lib.sh`, `lib/ai-providers/llamacpp.sh`, `apps/install.sh`, `templates/app.sh`, `DOC/howto/*`, `DOC/APPS.md`, `AGENT_TODO.md`, `config/*.env`). These were NOT edited by me — my changed-file set is exactly: +1. `scripts/lint-conventions.sh` (rewritten; 213 changed lines vs HEAD) +2. `bin/pos-system-uninstall` (D-E + H-002/H-003; 150 changed lines vs HEAD) +3. `DOC/POS.md` — only the `pos system uninstall` row (all other rows in the file diff belong to other tracks) +4. `DOC/AGENT_Context_Project.md` + `completions/pos.bash` — gen output including, among others, my uninstall row count +5. `AgentsReport/builder/2026-09-06_stab-tooling.md` (this report) + +No protected/brief-excluded file was touched by me; no out-of-scope change made. + +### DOC/SCRIPTS.md + DOC/DEV.md +Not touched: no manifest was created (D-E rejected it), so the brief's "only if the manifest needs documenting" condition does not apply. No new documentation needed. + +[DONE] + +--- + +## Final status + +- **Scope:** lint perf (Task 1) + D-E uninstall coverage (Task 2) + H-002/H-003 safe removal (Task 3) + surviving-artifacts docs (D-E 7). All implemented; no scope expansion. +- **Timings:** `make lint` 145.9 s → 4.38 s (~30x). Differential: output byte-identical to original. +- **Probes:** all pass (H-002/H-003 fixture removal, scan_tier1 with HOME override, remove-block replication, POS_LIBS==install.sh inventory). +- **Gates:** `bash -n` OK, `make gen` idempotent, `make check` OK, `make lint` `0 FAIL, 0 WARN`. +- **Deferred/risks:** + 1. The real-machine removal path (removing live `/usr/local/bin`, `/etc/systemd`, ScaleTail dirs) was NOT executed here — validated via fixture/sandbox probes only. A dry-run pass on a real install is doable via the tool's own scan display (`pos system uninstall` interactive scan shows Tier 1 entries before any removal). + 2. `lib/pos-ai-hook.sh` (13th lib file) is intentionally not in `POS_LIBS` because install.sh phase 2 doesn't install it; the uninstaller already removes `~/.local/bin/pos-ai-hook.sh` separately — no action. + 3. Other parallel tracks' edits are interleaved in the shared tree; my in-scope files are `scripts/lint-conventions.sh`, `bin/pos-system-uninstall`, DOC/POS.md (one row). +- **Recommended next agent:** Reviewer (independent adversarial review of the uninstall diff + lint rewrite), then Orchestrator to coordinate the shared-tree commit once the parallel tracks land. diff --git a/AgentsReport/explorer/2026-09-06_ai-audit.md b/AgentsReport/explorer/2026-09-06_ai-audit.md new file mode 100644 index 0000000..418e7fe --- /dev/null +++ b/AgentsReport/explorer/2026-09-06_ai-audit.md @@ -0,0 +1,98 @@ +# AI Subsystem Audit — Evidence-Based Verification + +Date: 2026-09-06 +Explorer: read-only, evidence-first. +Scope: `bin/pos-ai`, `bin/pos-ai-hf`, `bin/pos-ai-server`, forwarders (`gemini`/`openrouter`/`llamacpp`), `lib/ai-providers/*.sh`, `ai.env` config + `# POS_CONFIG:`/`pos config ai`, `# POS_FLAGS:`/`# POS_SUBCMDS:` metadata, `DOC/POS.md` ai section, GEN doc rows, `completions/pos.bash`, `bin/pos` usage(). + +--- + +## TL;DR + +- **The AI subsystem is largely real and coherent.** Providers (gemini/openrouter/llamacpp) adapters implement real API calls with key masking; `pos-ai-server` was genuinely repaired (commit `528b166`) — the old fake `validate_server_features` stub and unquoted multi-line ExecStart are gone, `systemd_quote` quoting is correct, `detect_llama_version` is guarded. +- **One HIGH defect:** `pos-ai-hf` **single-file** download failure still exits **0** and writes `.hf-meta` advertising a complete model (only the parallel/multi-file path was fixed for partial-failure rc=1). `bin/pos-ai-hf:716-720, 727-744, 773-775`. +- **One MEDIUM coherence defect:** `LLAMACPP_HOST` is honored by the server (ExecStart `--host` + display) but **ignored** by the llamacpp provider adapter and by the server's own probes, all of which hardcode `127.0.0.1` (`lib/ai-providers/llamacpp.sh:14,20,31,46-47`; `bin/pos-ai-server:131,576`). +- **pos-ai-server flag validation is NOT complete:** only CLI-explicit flags are validated; config-sourced flags and always-emitted defaults (`--threads`, `--n-gpu-layers`, `--ctx-size`, `--port`, `--host`) are never validated. "Version-aware" is cosmetic (version only interpolated into the error string, never used to branch logic). +- **Streaming is NOT implemented** in any provider (llamacpp explicitly sets `stream:false`); **no claim** of streaming exists in docs. Not a defect, just a fact to record. +- **Command-extraction** (`_extract_commands` / `_prompt_run_command`) provenance: `trusted=1` comes ONLY from the `--trust` CLI flag or alias wrappers that inject `--trust` (alias env field 5=1). Confirmation default on a tty is **ALLOW** (Enter runs); non-tty is fail-safe (never runs). Extra alarm is warranted only if the operator marks an alias trusted. + +--- + +## Evidence table + +| ID | file:line | current behavior | classification | notes | +|----|-----------|------------------|----------------|-------| +| A1 | bin/pos-ai:649-677 | Parses exactly `--provider --model --session --system --full --last --trust` + `-h` | MATCH | `# POS_FLAGS:` (line 5) matches parse exactly | +| A2 | bin/pos-ai:693-706 | Subcommands `ask capture chat models providers sessions` + `llamacpp` shorthand | MATCH for the six; **llamacpp shorthand IMPLEMENTED-BUT-UNLISTED** | `# POS_SUBCMDS:` (line 4) omits `llamacpp`; completions/pos.bash:51 does include it | +| A3 | bin/pos-ai:130-160 | `load_config` reads ai.env (env-already-exported wins) + legacy gemini/openrouter files | MATCH | duplicated loader, no shared one (see M3) | +| A4 | bin/pos-ai:162-202 | Config precedence: `--model` > AI_MODEL > provider-specific (`AI_GEMINI_MODEL`/`OPENROUTER_MODEL`/`LLAMACPP_MODEL`) > `provider_default_model` | MATCH | llamacpp `resolve_model` passes **basename** of LLAMACPP_MODEL (line 198) | +| A5 | bin/pos-ai:363-406, 531-535, 571-572 | `_extract_commands` parses ```bash/sh/shell``` fenced blocks; `_prompt_run_command` prompts/executes | MATCH | provenance of trusted traced to `--trust` flag + alias field 5 (pos-ai-alias:64,433-436,580). tty default = run; non-tty = never runs | +| A6 | lib/ai-providers/gemini.sh:24-36 | Real `generateContent` call; `-m 60` timeout; parses error.message, non-200 → rc 1 | MATCH | no streaming tokens (`?alt=sse` absent) | +| A7 | lib/ai-providers/openrouter.sh:24-37 | Real `/api/v1/chat/completions`; `-m 60`; error.message parse; rc 1 on non-200 | MATCH | | +| A8 | lib/ai-providers/llamacpp.sh:19-42 | Real `/v1/chat/completions`; `-m 120`; **hardcodes `127.0.0.1`**, only reads `LLAMACPP_PORT`; `stream:false` | MATCH for localhost mode; **ignores LLAMACPP_HOST** → coherence defect D2 | error path only prints "API error $code", no error.message | +| A9 | bin/pos-ai-server:388-392, 445, 501-508 | ExecStart single-line, binary+model `systemd_quote()`d | MATCH | repair confirmed real | +| A10 | bin/pos-ai-server:52-59 | `detect_llama_version` guarded; returns "unknown" safely; takes binary arg | MATCH | repair confirmed; no errexit | +| A11 | bin/pos-ai-server:66-87, 407-409 | `validate_requested_flags` only validates `REQUESTED_FLAGS` (CLI-explicit only) | PARTIAL (see D3) | config + defaults never validated; grep -qF substring (see D4) | +| A12 | bin/pos-ai-server:262-266, 373-380 | Defaults: PORT 8088, HOST 127.0.0.1, CTX 4096, GPU -1, THREADS nproc; CLI > env > default | MATCH | bind localhost by default ✓ | +| A13 | bin/pos-ai-server:128-135, 576 | health probe `/health` + `/v1/models` **hardcoded `127.0.0.1`** | PARTIAL (see D2) | ignores LLAMACPP_HOST for probes | +| A14 | bin/pos-ai-hf:185-251 | `hf_api` auth header, 401/403/404/429/other → `err`; JSON validity check; 429 retry-once | MATCH | base URL https://huggingface.co/api | +| A15 | bin/pos-ai-hf:254-312 | `hf_paginate` Link rel="next"; `hf_repo_files` tree endpoint with guards + fallback; `hf_search` jq urlencode | MATCH | pagination real; error-object guard via `select(type=="...")` | +| A16 | bin/pos-ai-hf:636-700 | Parallel download drain, per-pid wait, honest failure counting, `rc=1` on partial failure, no `.hf-meta` on partial | MATCH for multi-file | **single-file path defect = D1** | +| A17 | bin/pos-ai-hf:778-812, 915-992 | `list`/`cache` only list dirs with `.hf-meta`; `cache clear` fail-closed confirm | MATCH | | +| A18 | bin/pos-ai-hf:449-469 | `--branch`/`--revision` alias, last-wins; default branch from API else "main" | MATCH | | +| A19 | completions/pos.bash:6,7,26,51 | completion flags for ai/ai-server/ai-hf match `# POS_FLAGS:`; subcmds match declared | MATCH for declared | **no `_pos_subcmds[ai-hf]`** (see M1) | +| A20 | DOC/POS.md:56-125 | ai/hf/server sections describe behavior congruent with code | MATCH | no streaming claim | +| A21 | pos config ai / cfg_display masks `secret` keys | AI_GEMINI_API_KEY, OPENROUTER_API_KEY, HF_TOKEN are `secret`-flagged → masked | MATCH | lib/config-ui.sh:347-361 | + +--- + +## Findings (ranked defects) + +### AI-subystem defects + +- **D1 (HIGH)** — `pos-ai-hf` single-file download failure returns **0** and writes `.hf-meta` for a partial model. + Evidence: `bin/pos-ai-hf:716-720` — sequential path `warn "Failed to download $fname"; continue` without recording into `failed_files`; `:727-744` — `failed_files` empty ⇒ `.hf-meta` written; `:773-775` — the `return 1` clause only fires for the parallel path. Only if `file_count>1` (parallel branch, `:641`) is a failure non-zero + meta-suppressed. Contradicts commit `528b166` message "rc=1 on partial failure, no .hf-meta for half-downloaded models". A truncated `-C -` partial `.gguf` can then be handed to `pos ai server start`. + +- **D2 (MEDIUM)** — `LLAMACPP_HOST` advertised but only honored by the server, not by the client/probes. + Evidence: server emits `--host $HOST` (`bin/pos-ai-server:445`) and displays `endpoint: http://$HOST:$PORT` (`:604`), but health probe `:131` and `/v1/models` probe `:576` hardcode `127.0.0.1`; the llamacpp **adapter** (`lib/ai-providers/llamacpp.sh:14,20,31,46-47`) hardcodes `127.0.0.1` and reads only `LLAMACPP_PORT`. A non-localhost `LLAMACPP_HOST` → server binds elsewhere while `pos ai llamacpp ask/chat/models` and probes talk to loopback. Coherence break across the two tools + the advertised config key. + +- **D3 (MEDIUM)** — `validate_requested_flags` validates **only CLI-explicit** flags. + Evidence: `REQUESTED_FLAGS` is populated only in the arg-parsing `case` clauses (`bin/pos-ai-server:295-359`); the gate `if [ "${#REQUESTED_FLAGS[@]}" -gt 0 ]` (`:407-409`) + always-serialized defaults `--n-gpu-layers $gpu_layers --ctx-size $CTX_SIZE --threads $THREADS` (`:446-448`) and config-sourced `--port/--host` (`:445`) are never checked. Also: the `version` argument is used **only** in the error string (`:84`); there is no version-conditional logic — the "version-aware" phrasing is cosmetic. Practically mitigated because the always-on core flags (`--threads`, `--n-gpu-layers`, `--ctx-size`, `--port`, `--host`) are universal across llama.cpp builds, but the guarantee is broader than the implementation. + +- **D4 (LOW)** — `validate_requested_flags` uses loose substring matching `grep -qF -- "$flag"` (`bin/pos-ai-server:83`). A flag token could match inside synthesized `--help` text unrelated to actual support (e.g. a builder that documents a placeholder), giving false-passes; no word/`--flag=` boundary. Minor robustness issue. + +- **D5 (LOW / UNKNOWN-live)** — llamacpp model id: `resolve_model` passes `basename "$LLAMACPP_MODEL"` (`bin/pos-ai:198`) as the `model` field. llama.cpp `/v1/models` `.id` is not guaranteed to equal the GGUF basename (may be the model's internal name or full path). If they differ the server may reply "model not found". UNKNOWN — requires live llama-server to confirm. + +- **D6 (LOW, security-relevant)** — command-execution confirmation default is **ALLOW** on a tty: `_prompt_run_command` prompt `[Y/n]` runs on any value except `n|N` (`bin/pos-ai:391-405`); trusted=1 (`--trust` flag `:667-668`, or alias trusted field via `pos-ai-alias:64,433-436`) runs with **no** confirmation (`:385-389`) via `run eval "$cmd"` (`:388,403`). Non-tty is fail-safe (`:382` — returns without running). Not an auto-exec bug under default settings, but the confirmation default-allow + `eval` of AI-extracted shell is worth a security review's attention. + *Provenance of trusted=1 (question 11):* (1) `--trust` CLI flag, default 0 (`:644`); (2) alias wrapper inserts `--trust` when the alias's 5th env field (`name|provider|session|prompt|trusted`) = 1 (`pos-ai-alias:64`), set interactively ("Trust this alias? (y/N)", default N) at create/edit (`:433-436, 580-591`). **No config key** drives trust. Auto-execution paths: only trusted-mode `_prompt_run_command` (`:388`) — nothing else eval's model output. + +### Metadata / doc drift + +- **M1 (MEDIUM, metadata gap)** — `pos-ai-hf` implements subcommands `search/download/list/remove/info/files/cache` but declares **no `# POS_SUBCMDS:` header** (`bin/pos-ai-hf:3-4`). Consequently `_pos_subcmds[ai-hf]` is absent from `completions/pos.bash` and the gen doc tables/tree do not surface them. IMPLEMENTED-BUT-UNLISTED. + +- **M2 (LOW)** — `pos-ai` `llamacpp` shorthand subcommand (parsed, `bin/pos-ai:701-704`) absent from `# POS_SUBCMDS:` (line 4). Completions already include it (`completions/pos.bash:51`). IMPLEMENTED-BUT-UNLISTED (only in the shorthand). + +- **M3 (LOW, maintainability)** — no shared `ai.env` loader. pos-ai (`load_config`, `:130`), pos-ai-server (`load_config`, `:21`), pos-ai-hf (`load_hf_config`, `:30`) each duplicate the env-precedence loop; `lib/common.sh` only provides `load_system_env` (system.env) and `CONFIG_DIR`. All three behave identically, but this is a future-drift seam and contradicts the "shared loader" intent implied by AGENTS.md. + +- **M4 (LOW)** — forwarder `# POS_SUBCMDS:` under-list: `pos-ai-openrouter:4` = `ask chat sessions capture` (omits `models`, which works via passthrough); all three forwarders omit `providers` (`pos-ai-gemini:4`, `pos-ai-openrouter:4`, `pos-ai-llamacpp:4`), which also works. Completions inherit the under-list. + +- **M5 (INFO, not a defect)** — `QUANT_DIR` is a CLI-flag-only variable (`bin/pos-ai-hf:119,134-135`), not an env/config key; it is not in `# POS_CONFIG:` and is correctly absent from `pos config ai`. The audit task listed it among config keys to trace; it is flag-only by design. Configure `HF_TOKEN`/`HF_DOWNLOAD_DIR` instead. + +### Could NOT verify live (read-only / no runtime) + +- llama-server binary behavior: `--version`/`--help` output, whether `--threads`/`--n-gpu-layers`/`--gpu-threads`/`--kv-cache` etc. are all accepted by the installed build; the `/v1/models` id format (D5). +- Actual Hugging Face API responses (tree endpoint shape, pagination Link header, 429 Retry-After); `hf_paginate`/`hf_repo_files` correctness with live data. +- systemd unit validity: the reviewer report for `528b166` states `systemd-analyze verify` rc=0 was checked; I could not re-run it. Static reading of `systemd_quote` + single-line ExecStart is correct. +- Whether a real Gemini/OpenRouter key/reply path works end-to-end (key required). + +--- + +## Handoff + +Status: COMPLETE (objective answered to the limit of read-only evidence). +Evidence: all file:line cited above; primary source + git commit `528b166` diff. + +- **Recommended next agent: Reviewer (adversarial review)** on the single-file download path (D1) and the `LLAMACPP_HOST` coherence break (D2) — both are concrete, source-provable behavior defects ripe for a focused fix. **Builder** if a fix is decided (D1: route single-file through the failure-counting harness or check `hf_download_with_progress` rc and suppress meta + return 1; D2: thread `LLAMACPP_HOST` into the adapter and probes). **Maintainer** for the metadata gaps (M1, M2, M4) and the duplicated ai.env loader (M3). +- Affected areas: `bin/pos-ai-hf`, `bin/pos-ai-server`, `lib/ai-providers/llamacpp.sh`, `bin/pos-ai*` POS_* headers, `completions/pos.bash`, gen doc output. +- Decision boundary: adding `# POS_SUBCMDS:` to `pos-ai-hf` is a metadata change requiring `make gen` + `make check` + `make lint` (per AGENTS.md) — that is a Builder/Maintainer action, out of Explorer scope. + +Remaining uncertainty: D5 (llamacpp model-id match) and all binary/API live behavior (listed above) — needs a live llama.cpp/HF/Gemini environment. diff --git a/AgentsReport/explorer/2026-09-06_security-audit.md b/AgentsReport/explorer/2026-09-06_security-audit.md new file mode 100644 index 0000000..97862be --- /dev/null +++ b/AgentsReport/explorer/2026-09-06_security-audit.md @@ -0,0 +1,239 @@ +# Security Audit — Command-Execution Surfaces, Chat Authorization, Secret Handling + +**Date:** 2026-09-06 +**Explorer:** read-only investigation +**Scope:** `bin/`, `lib/`, `scripts/`, `install.sh`, `preinstall.sh`, `postinstall.sh`, `apps/`, `entertainment/` + +## TL;DR + +- **Telegram listener skips SENDER authorization entirely.** It filters *only on chat id* (`TELEGRAM_CHAT_ID`), and even that filter is weakened by an OR-clause that also accepts `from_id`. No `from.username`/owner-user-id allowlist exists anywhere; there is no config key for one. Anyone who can get a message into the owner chat (shared chat, forwarded/mention, or a group where the bot sees the message with `from_id != chat_id`) can execute arbitrary mapped bash **and** the Gemini AI bridge as your user. This is the single highest-severity finding. +- **Matrix listener DOES implement exactly "sender authorized AND chat authorized"**, but both are *optional-to-configure*: `MATRIX_USER_ID` defaults to a live `/whoami` resolution and `MATRIX_ROOM_ID` to "every room joined". If both are unset at runtime you get "any sender, any room" — remote code execution. Severity depends on which homeserver/rooms the account is in. +- **AI command auto-execution (`pos ai` eval path)** is a real RCE primitive. Trusted aliases pass `--trust` (no confirmation). The listener's `` bridge passes *unvalidated chat text* into a shell string that is executed by `bash -c` — a direct RCE. +- **`/dev/tcp` probes interpolate unvalidated host/port into `bash -c` strings** in 4 places. Most reachable inputs are CLI args (interactive/low risk), but `pos-network-download` (`NET_PROBE`, network-derived) and the `share_port_probe` surface deserve review. +- **Backup GPG passphrase** is passed on the gpg **command line** (`--passphrase "$PASS"`) → visible in `ps`/process args, and exported into any `notify`/log context via `set -x` if debugging. No temp files persist the secret; `mktemp` artifacts are cleaned on both success and failure, and the archive is `chmod 600`. The SMB client and Matrix login write credentials to throwaway/templated files correctly (`chmod 600`) and avoid argv exposure, but the SMB persistent credentials dir text is world-readable risk only via file perms (mode 600). + +--- + +## 1. Command-Execution Inventory + +All primitives below were located by grep across the specified paths; input provenance chain and classification are traced from source. + +### `/dev/tcp` probes — host/port interpolated into `bash -c` string + +| ID | file:line | primitive | input source | attacker-controlled? | classification | +|----|-----------|-----------|--------------|----------------------|----------------| +| C01 | `bin/pos-network-checkport:133` | `bash -c "exec 3<>/dev/tcp/$ip/$port"` | `ip` from CLI arg (`$host` from targets, validated no-spaces at :486), `port` validated `[0-9]{1,5}` 1-65535 | port no; host partially (no-space check only, IPv6 brackets stripped) | **REVIEW** — host could contain `;`/`$(...)` if crafted (only `*" "` is rejected; `;`, backticks, `$()` not blocked). Interactive CLI, but the string is unquoted. | +| C02 | `bin/pos-network-checkport:157` | `bash -c "exec 3<>/dev/udp/$ip/$port; printf 'x' >&3"` | same provenance | same | **REVIEW** | +| C03 | `bin/pos-network-checkport:166` | `bash -c ".../dev/tcp/$ip/$port; printf 'HEAD...'"` | same | same | **REVIEW** | +| C04 | `bin/pos-network-checkport:168` | `bash -c ".../dev/tcp/$ip/$port; head -c 200"` | same | same | **REVIEW** | +| C05 | `bin/pos-share-smb-client:92` | `timeout 3 bash -c "exec 3<>/dev/tcp/${host}/${SMB_PORT}"` | `host` from `//server/share` CLI arg; `split_share` extracts `SERVER` with no validation | **partially** — no filtering of `;`/`$()`/backticks in SERVER | **VULNERABLE-INPUT** (low exploitability: requires the operator to type such a host; but a malicious remote `///share` string reaches shell). | +| C06 | `lib/share-lib.sh:59` | `timeout 3 bash -c "exec 3<>/dev/tcp/${1}/${2}"` | `$1`=host, `$2`=port from callers (nfs/smb client mounts) | host unvalidated | **VULNERABLE-INPUT** (shared helper; called from CLI-only paths). | +| C07 | `bin/pos-network-download:28,377` | `NET_PROBE=...bash -c ''`; `net_up() { bash -c "$NET_PROBE" }` | **controlled by `NET_PROBE` env var** | **yes** if `NET_PROBE` is attacker-set (e.g. via schedule/daemon env) | **REVIEW** — env-derived command string evaluated; DEFAULT is constant/safe. | + +### Listener command execution (chat-driven) + +| ID | file:line | primitive | input source | attacker-controlled? | classification | +|----|-----------|-----------|--------------|----------------------|----------------| +| C08 | `bin/pos-communication-telegram-listener:349` | `timeout "$tmo" bash -c "$cmdline"` (in `run_and_reply`) | `$cmdline` = map value (constant, owner-edited) **or** prefix bridge `$cmd $qtext` where `$qtext` = `printf '%q'` of chat text | `$qtext` is the chat text, but `%q`-quoted (safe metachar-wise) | **REVIEW** — map value is constant; prefix path quotes the arg. Map value itself is operator-owned. | +| C09 | `bin/pos-communication-telegram-listener:697` | `run_and_reply "$cmd $qtext" ...` | prefix bridge: `$cmd` = map value (operator), `$qtext` = `%q` quoted chat text | chat text via `%q` (safe) | **REVIEW/SAFE** — quoting present but the whole string is one `bash -c`; the chat text is not the only component. | +| C10 | `bin/pos-communication-matrix-listener:493` | `timeout 60 bash -c "$value"` | `$value` = map value (operator-owned, /cmd) | no (constant operator string) | **SAFE** (constant owner map). | +| C11 | `bin/pos-communication-matrix-listener:214` | `timeout 60 bash -c "$value"` (ui test) | operator input in TUI | no | **SAFE/DESIGNED-INTERACTIVE**. | +| C12 | `bin/pos-communication-telegram-listener:368` | `timeout 60 bash -c "$value"` (ui test) | operator input in TUI | no | **SAFE/DESIGNED-INTERACTIVE**. | + +### AI command eval (RCE primitive) + +| ID | file:line | primitive | input source | attacker-controlled? | classification | +|----|-----------|-----------|--------------|----------------------|----------------| +| C13 | `bin/pos-ai:388,403` | `run eval "$cmd"` (in `_prompt_run_command`) | `$cmd` = `_extract_commands` of the **AI provider's raw response** (gemini/openrouter/llamacpp output) | **yes — AI-generated** | **VULNERABLE-INPUT** (eval of untrusted model output). Trusted mode (`--trust`, `TRUST_MODE=1`) at :385-388 auto-executes with no prompt; non-trusted prompts on tty. Called from `cmd_ask:535` and `cmd_chat:572`. | +| C14 | `bin/pos-communication-telegram-listener:734` | `timeout 120 pos ai gemini ask ... "$prompt"` | AI bridge: `$prompt` = chat text after ` ` | chat text — but only as an **argument** to `pos ai gemini ask` (a subprocess arg, quoted via `"$prompt"`); the AI eval inside pos-ai then acts on the model's *reply*. | **REVIEW** — the bridge feeds arbitrary chat text to the model; if the model echoes back a command, C13 fires. Indirect injection. | +| C15 | `bin/pos-communication-matrix-listener:472` | `pos ai gemini ask ... "$prompt"` | same AI bridge (Matrix `ai ` prefix) | same | **REVIEW** (indirect). | + +### `sudo` / `systemctl` / docker / ssh / curl|sh + +- **`apps/**/*.sh`** — `sudo` used for `apt install`/`systemctl`/`usermod`/`curl | sh` installers. These run **once interactively at install time**; inputs are (mostly) constant URLs. Classified **SAFE/DESIGNED-INTERACTIVE**. Notable pipe-to-shell: + - `apps/networking/zerotier.sh:9` — `curl -s https://install.zerotier.com | sudo bash` (network→shell, no verification). **Review**. + - `apps/networking/netbird.sh:9`, `tailscale.sh:9` — `curl -fsSL ... | sh` (official vendor identical). **Review** (no checksum; standard vendor practice). + - `apps/system/docker.sh:9` — `curl -fsSL https://get.docker.com | sh`. **Review**. + - `apps/development/opencode.sh:9` — `curl -fsSL https://opencode.ai/install | bash`. **Review**. + - `apps/utilities/tsui.sh:9` — `curl -fsSL https://neuralink.com/tsui/install.sh | bash`. **Review** (vendor unknown-ish host). +- **`preinstall.sh:45,48,58,65` / `postinstall.sh:101-160` / `install.sh:134-239`** — `sudo apt`, `sudo systemctl`, `sudo install`, `git clone`. Constant/static targets. **SAFE** (interactive bootstrap). +- **`bin/pos-network-scan:88,99`** — `sudo -n nmap` / `sudo nmap`; constant. **SAFE**. +- **`bin/pos-network-hotspot:60-86`** — `sudo create_ap` with `"$@"` passthrough. CLI args reach sudo. **REVIEW** (interactive tool; operator supplies args). +- **`bin/pos-system-backup:173`** — `sudo tar -czvf "$ARCHIVE" -C "$(dirname "$FOLDER")" "$NAME"`; `$FOLDER` CLI arg baked into `$ARCHIVE` name. **REVIEW** (folder name into tar; low risk, arbitrary path backup). +- **`bin/pos-share-nfs-client/smb-client`, `lib/usb-lib.sh`, `lib/share-lib.sh`** — `sudo mount`/`umount`/`mkdir` with mount paths; mountpoint paths validated to be absolute and non-system (smb-client `ask_new_mountpoint`/`menu_ask_mountpoint:579,614` deny `/etc,/boot,/bin,...`). **SAFE/REVIEW** — some paths from `findmnt`/menu. +- **`bin/pos-docker-vbox:1074,1094,1096`** — `docker exec -it ... bash` (interactive attach). **SAFE/DESIGNED-INTERACTIVE**. +- **`scripts/lint-conventions.sh`, `make gen` pipeline** — dev-time `bash -n`/`awk`/`sed`. Out of runtime scope (dev tooling, constant). **SAFE**. +- **No `sshpass`, `scp`, or `ssh` remote-command execution** anywhere (only `pos ssh load-keys` adding keys to the agent, and doc references). No remote-shell-over-ssh primitive found. **N/A**. +- **`lib/scheduler-lib.sh:180`** — `bash -c "$JOB_COMMAND"`; `JOB_COMMAND` is the literal remainder of a `schedule.d/*.env` line, operator-authored, syntax-checked (POS tool). **SAFE/DESIGNED-INTERACTIVE** (operator-owned config; would be RCE if an attacker could write `schedule.d/`). + +### Secret-in-process-argv (gpg/openssl) + +- **`bin/pos-system-backup:195`** — `gpg --passphrase "$PASS" --symmetric ...`: the backup passphrase is placed on the gpg **command line**, visible to any local user/process via `/proc//cmdline` and to `ps as`. **See Section 3.** + +--- + +## 2. Authorization Model — Telegram + Matrix + +### Telegram listener (`bin/pos-communication-telegram-listener`) + +**Config keys read (via `load_config` line 86-99, plus direct grep in `ai_bridge_prefix`:625):** +| Key | Defined in | Purpose in listener | +|-----|-----------|---------------------| +| `TELEGRAM_BOT_TOKEN` | `# POS_CONFIG:` in `telegram-sender:6`; `telegram.env` | authenticate Bot API | +| `TELEGRAM_CHAT_ID` | same | **the only owner filter** (see below) | +| `TELEGRAM_AI_PREFIX` | same | AI bridge trigger word (default `ai`) | + +There is **no** `TELEGRAM_OWNER`, `TELEGRAM_USER_ID`, `TELEGRAM_ALLOWED`, `OWNER_ID`, or any sender-identity allowlist key anywhere in the codebase, config templates, or docs. + +**Message flow & the update loop (lines 766-792):** +1. `getUpdates` with `allowed_updates=["message"]` (line 771) — filters update *type* to messages only. +2. Per update: extracts `chat` (`.message.chat.id`), `from_id` (`.message.from.id`), `text` (line 780-783). +3. **The ONLY authorization gate is line 787:** + ``` + if [ -n "$chat" ] && [ "$chat" != "$TELEGRAM_CHAT_ID" ] && [ "$from_id" != "$TELEGRAM_CHAT_ID" ]; then continue; fi + ``` + This is: **continue (drop) ONLY IF** chat is set AND chat ≠ owner-chat **AND** from_id ≠ owner-chat. +4. `handle_message` (line 674) then dispatches with **no further sender check**: `/command` map (via `map_get`, line 743) → `run_and_reply` → `bash -c` (C08); web-URL detection → `pos media grab` (line 704); prefix bridge (line 697) → `bash -c`; AI bridge (line 734) → `pos ai gemini ask`. + +**Verdict — Telegram does NOT implement "sender authorized AND chat authorized":** +- The check is **chat-or-sender OR**, not AND, and the "sender" comparison uses **`from_id` == `TELEGRAM_CHAT_ID`** — i.e. it assumes the owner's Telegram *user id* numerically equals the *chat id*. That is only true for a direct (1:1) private chat with the owner. In a **group or supergroup**, the `chat.id` is negative and differs from any `from.id`; the OR-clause then accepts **any** `from_id` that happens to equal `TELEGRAM_CHAT_ID` (unlikely) OR requires chat==owner. In a **shared/followup private chat** scenarios or when `TELEGRAM_CHAT_ID` is a forwarded context, the model breaks. +- **Concretely exploitable:** if the bot's token is added to a group, `chat.id` (group, negative) ≠ `TELEGRAM_CHAT_ID` AND `from_id` (a member) ≠ `TELEGRAM_CHAT_ID` → condition **false** → message is dropped. So plain group members are blocked **only if** `TELEGRAM_CHAT_ID` is truly the owner's 1:1 chat and the bot isn't also filtering otherwise. BUT there is **no sender allowlist**, so the guard is the sole mechanism and it mis-handles the general case. If the owner ever sets `TELEGRAM_CHAT_ID` to a group id (a plausible misconfiguration the tool doesn't prevent), **anyone in that group executes commands**. Also the model treats `from_id == TELEGRAM_CHAT_ID` as allowed even when `chat` is different — so a message where sender id coincidentally equals the configured numeric id (or a bot re-post) is accepted regardless of chat. +- **No `from.username`, no `from.first_name/last_name` allowlist, no user-id allowlist** — the requirement "is the SENDER's from.id checked ANYWHERE before a mapped command or AI bridge executes?" is answered: **yes, but only against the chat-id value via an OR with chat-id, and only for one numeric field.** This is not a proper sender authorization. +- **Missing/invalid config behavior:** `run_daemon:759-760` `err`s (exits) if token or chat-id are unset. Good fail-closed for *chat* but there is no sender config to miss. + +**Map-file editing surface:** exclusively the **interactive TUI** (`ui()` at line 440, no-arg invocation; `ui_add/ui_edit/ui_remove/ui_test`, lines 378-438) and the `prefix` verb for the prefix map (`prefix_cmd:567`). No network/API path writes `telegram_commands.env`. The map file is `chmod 600` (`map_set:137,144`). **Risk:** it is a **local** file `~/.config/linux_post_install/telegram_commands.env` editable only by the owner at the shell; if the owner account is compromised the map is trivially editable (but that's full-host compromise anyway). The `/cmd::description=` text (user-typed description) flows into `sync_bot_commands` → `setMyCommands` (line 215), so a map *description* is sent to Telegram — a low-level info leak of the operator's own design, not an attacker surface. + +### Matrix listener (`bin/pos-communication-matrix-listener`) + +**Config keys read (via `load_config`:65-78):** +| Key | Defined in | Purpose | +|-----|-----------|---------| +| `MATRIX_HOMESERVER` | `# POS_CONFIG:` `matrix-sender:5`; `matrix.env` | server URL for /sync | +| `MATRIX_ACCESS_TOKEN` | same | auth | +| `MATRIX_USER_ID` | same | **owner/sender filter** | +| `MATRIX_ROOM_ID` | same | **room filter** (optional) | + +**Message flow & /sync loop (lines 509-557):** +1. Resolves owner: `MATRIX_USER_ID` if set, else live `/account/whoami` (lines 514-523). +2. `room_only="${MATRIX_ROOM_ID:-}"` (line 515) — **empty = watch every join.** +3. Per room: skip if `room_only` set and `room != room_only` (line 539). +4. Per event: require `type==m.room.message`, `content.msgtype==m.text` (546-547); **sender filter line 550:** `[ "$sender" = "$owner" ] || continue` — message REFUSED unless the sender is the owner. +5. `handle_message` (line 447) dispatches with **no further sender check**: `/`/`!` command map (line 483) → `bash -c` (C10); `ai ` bridge (line 472) → `pos ai gemini ask`. + +**Verdict — Matrix:** +- `MATRIX_ROOM_ID` set: **sender authorized (== owner) AND chat authorized (== configured room)** — matches the requirement. ✅ +- `MATRIX_ROOM_ID` unset but `MATRIX_USER_ID` set: sender authorized, chat = **every joined room** — chat is NOT pinned. An owner tweet from any room triggers RCE. ⚠️ +- Both unset: owner auto-resolved via whoami (still "sender == owner"), room = every room. Still sender-gated, but the room is unbounded. If `whoami` *fails* (line 520-521), it `err`s out (fail-closed). So Matrix is **sender-gated always** (owner == account's own user id), chat optionally restricted. This is a materially **stronger and correct** model than Telegram's. + +**Guard lines that would need to change to satisfy "user AND chat authorized" fully:** +- **Telegram:** replace the OR at `bin/pos-communication-telegram-listener:787` with an AND requiring `chat == TELEGRAM_CHAT_ID` **and** a sender check against a new allowlist key (e.g. `TELEGRAM_OWNER_ID`). Minimal sketch: + ```bash + # 1) Chat must be the owner chat + [ "$chat" = "$TELEGRAM_CHAT_ID" ] || continue + # 2) Sender must be an allowed user id (new key, fail-closed if unset) + [ -n "${TELEGRAM_OWNER_ID:-}" ] || { warn "no TELEGRAM_OWNER_ID — refusing"; continue; } + case " $TELEGRAM_OWNER_ID " in *" $from_id "*) ;; *) continue ;; esac + ``` + plus add the `TELEGRAM_OWNER_ID` key to the `# POS_CONFIG:` registry in `telegram-sender:6` (`pos config telegram`). **Do NOT implement — this is exploration output only.** +- **Matrix:** to fully pin chat even when `MATRIX_ROOM_ID` is unset, `run_daemon` should fail-closed (refuse to start) or require a room list; currently line 515 defaults to "watch all." Add validation that `MATRIX_ROOM_ID` is set before `--run` (or an explicit allowlist). **Do NOT implement.** + +### Other listener/plugin paths that execute commands +- **Entertainment plugins** (`entertainment/{weather,joke,gold}.sh`) fetch public APIs and print text — **no command execution**; output is sent via `pos-entertainment-send` → `notify_send`. Not an RCE surface via chat; they run on a schedule or explicit `pos entertainment send`. +- **`pos-docker-vbox enter`** (`docker exec -it bash`) is operator-attach only. +- **No other chat→command executor** found besides the two listeners and the AI bridge. + +--- + +## 3. Backup / GPG Secret Handling + +File: `bin/pos-system-backup`. **PASS provenance chain:** +1. **Prompted interactively** (lines 183-186): `read -s -rp "Enter backup password:" PASS`, then `read -s -rp "Confirm..." CONFIRM`. Read from the terminal with echo suppressed — **not** from config file, env, or argv. Good. +2. Validation (187-191): non-empty and `PASS == CONFIRM`, else loops. +3. `unset CONFIRM` (192) immediately after. +4. **Every place PASS is used:** + - line 195: `gpg --batch --yes --passphrase "$PASS" --symmetric --cipher-algo AES256 "$ARCHIVE"` — **PASS on the command line** → visible in `/proc//cmdline` and `ps` output of `gpg`. **This is the exposure.** + - line 202: `gpg --batch --quiet --passphrase "$PASS" --decrypt "$ARCHIVE" | tar -tzf -` — verify/decrypt path, **same argv exposure**. + - line 204: `unset PASS`. +5. **Temp files:** none created for the secret. `mktemp` is **not** used anywhere in backup; the archive is `tar -czvf "$ARCHIVE"` (line 173), then encrypted, then `rm -f "$ARCHIVE"` (line 197) leaving `$ARCHIVE.gpg`, `chmod 600` (line 199). The plain intermediate `.tar.gz` is removed **after** encryption. On **failure** (encrypt/verify fail → `set -euo pipefail` aborts): if gpg fails at 195, the plain `$ARCHIVE.tar.gz` **remains on disk** (sleeped only after the successful encrypt at 197). No trap removes the plain intermediate. This is a **partial-failure plaintext-leak** risk (archive stays unencrypted if `gpg --symmetric` fails). Also the top trap (line 17) sends `notify_send "Backup FAILED..."` on any ERR — does not leak PASS but does announce paths. +6. **verify/decrypt path:** line 202 pipes gpg decrypt into `tar -tzf -` and discards (no plaintext written to disk) — good, verify is streaming. +7. **notify/msg exposure:** backup success sends `notify_send "Backup completed: $ARCHIVE"` (line 211) and USB copy sends `...: $dest` (line 112) — **artifact paths (not contents) are sent to Telegram/Matrix** via `lib/notify.sh`. These are local absolute paths; harmless unless they reveal structure the owner wants private. The **gpg passphrase is NOT** in any notify message. +8. **Shell-history/log exposure:** PASS is read via `read -s` (not echo'd, not in history). It appears only in the gpg argv; not logged to any file by `pos`. However if the operator runs the tool under `set -x` or with shell tracing, `PASS` is a shell variable and would be expanded into stderr; not a repo bug, but the argv placement (not env) is the primary leak vector. + +**Other secrets in process argv repo-wide:** +- `bin/pos-network-hotspot:60` — `sudo create_ap ... "$@"`; if a passphrase/SSID is passed as an arg it enters `create_ap` argv. Interactive, no secret stored. +- `matrix-sender` `login` (pos-communication-matrix-sender:167-190): password read via `read -rsp ... /...`). The token is in curl's argv → visible via `ps`. **This is a real secondary exposure**: the Telegram bot token is a local-process-argv secret. Same class as the gpg passphrase. + +--- + +## Severity-ranked concrete vulnerabilities (real, not hypothetical) + +**V1 — HIGH — Telegram listener lacks sender authorization; remote code execution as your user.** +`bin/pos-communication-telegram-listener:787` is the only gate and it is an OR over `chat`/`from_id` against the single `TELEGRAM_CHAT_ID` value, with **no sender allowlist key existing anywhere**. Any message that satisfies either `chat == TELEGRAM_CHAT_ID` OR `from_id == TELEGRAM_CHAT_ID` triggers mapped `bash -c` (C08) and the Gemini bridge. In group/shared-chat misconfiguration, any member runs arbitrary commands as the owner. Even in the "correct" 1:1 setup there is no authenticated-sender binding, so a bot-repost or replayed `from_id` is accepted regardless of source chat. Evidence: lines 787, 743, 697, 734. + +**V2 — HIGH — `pos ai` evaluates arbitrary AI-provider output; `--trust` removes confirmation.** +`bin/pos-ai:388,403` runs `eval "$cmd"` where `$cmd` is extracted from the provider's raw response (`_extract_commands`:363). The Telegram/Matrix AI bridges (listener:734/:472 / C14/C15) feed arbitrary chat text as the prompt; if the model's reply (or a prompt-injection / model misbehavior) emits a fenced `bash`/`sh` block, it is executed. Non-trust mode prompts on a tty (`[ -w /dev/tty ]`), but **trusted aliases** pass `--trust` (`TRUST_MODE=1`, :385-388) → auto-execute, no confirm. And the **AI bridges run non-interactively (no tty)**, so `_prompt_run_command`'s `[ -w /dev/tty ] || return 0` at line 382 returns 0 **without prompting** → **any** command block in a bridge-AI reply executes automatically even without trust. This makes the chat AI bridge an unconditional RCE on model output. Evidence: pos-ai:382,388,403,535,572; listeners:734/:472. + +**V3 — HIGH — backup GPG passphrase on command line (argv exposure).** +`bin/pos-system-backup:195,202` pass `--passphrase "$PASS"` to gpg → secret readable by any local user or leak to syslog/ps. Evidence: lines 195, 202. (Also the plain `.tar.gz` can remain on a failed encrypt: line 197 is only reached after a successful 195.) + +**V4 — MEDIUM — Matrix listener optional chat authorization.** +When `MATRIX_ROOM_ID` is unset (default), `run_daemon:515` watches **all joined rooms**; sender still must equal the owner (`:550`), so it's owner-only but unbounded-room. If the account is in any shared room and the owner sends a command there, it executes. Fails closed on unresolved owner (`:521`), so severity is bounded to owner-initiated events. The authorization model is *correct* conceptually but chat-scope defaults too broadly. + +**V5 — MEDIUM — unvalidated host/port interpolated into `bash -c /dev/tcp` strings.** +`bin/pos-network-checkport:133,157,166,168`, `bin/pos-share-smb-client:92`, `lib/share-lib.sh:59`. Host strings are not fully validated (no `;`/`$()`/backtick reject) before being embedded in a shell string. Reachable via CLI args (interactive) and `NET_PROBE` env (`pos-network-download:28,377`). + +**V6 — LOW/MEDIUM — `curl | sh` / `curl | sudo bash` installers without checksums.** +`apps/{zerotier,netbird,tailscale,docker,opencode,tsui}.sh`. Vendor-standard, but supply-chain risk from the remote script. Interactive install-time only. + +**V7 — LOW — Telegram bot token in curl argv.** +`bin/pos-communication-telegram-listener:636,768` and `telegram-sender:81` place `bot` in a URL passed to curl → token visible in `/proc//cmdline`. Same class as V3. + +--- + +## Uncertainties / Could-not-verify + +- **Exact live behavior of the Telegram `from_id`/`chat_id` equality in real groups** cannot be established by read-only inspection; the numeric-equality assumption is documented in the code (line 787) but its breakage requires a live group test. This is the crux of V1's real-world exploitability and needs a live check by another agent. +- **Whether `ps`/`/proc` argv is considered a real threat model** for this homelab (single-user local machine) is a policy/rationale question the Explorer can't decide — see note in V3. +- **`pos-network-download`'s `NET_PROBE`** default is constant; whether any deployment injects an attacker-controlled value is unknown — it honours an env var seam. +- **Whether any operator already sets a `MATRIX_ROOM_ID`** (affects V4) is config state not present in the repo (config files are gitignored). +- The exact content of runtime `telegram_commands.env` / `matrix_commands.env` map files (what commands are mapped) is unknown — gitignored. + +--- + +## Important Files +- `bin/pos-communication-telegram-listener` — auth gate (787), dispatch, AI/prefix bridges. +- `bin/pos-communication-matrix-listener` — owner filter (550), room gate (539). +- `bin/pos-ai` — eval path (388,403,382), trusted mode, bridge call sites. +- `bin/pos-system-backup` — passphrase argv usage (195,202). +- `bin/pos-network-checkport`, `bin/pos-share-smb-client`, `lib/share-lib.sh`, `bin/pos-network-download` — /dev/tcp interpolation. +- `lib/ai-providers/gemini.sh` — API key in HTTP header (not argv). +- `bin/pos-communication-{telegram,matrix}-sender` — POS_CONFIG registry, token/creds handling. +- `lib/notify.sh` — artifact-path-only notify sends. + +--- + +## Handoff + +- **Status:** COMPLETE (investigation objective satisfied; no code changed). +- **Objective:** Security audit of command-execution surfaces, chat authorization, secret handling. +- **Evidence:** file:line citations throughout; classification by certainty (SAFE / DESIGNED-INTERACTIVE / VULNERABLE-INPUT / REVIEW) with provenance chains. +- **Affected areas:** Telegram & Matrix listeners, `pos-ai` eval, backup GPG, /dev/tcp probes, installer curl|sh. +- **Scope/decision boundary:** Read-only exploration only; **no changes proposed for implementation.** Minimal change sketch for the Telegram auth gate is provided at Section 2 (as exploration output, explicitly NOT implemented). +- **Verification performed:** Full source tracing of both listeners, pos-ai eval, backup, notify, and all grep'd primitives; git blame on the Telegram gate (unchanged since 2026-08-06, b9edd078 / 014d6be). +- **Remaining uncertainty:** live Telegram group behavior (V1), NET_PROBE env deploy state, MATRIX_ROOM_ID config state, real `ps`-argv threat model. +- **Recommended next agent:** **Architect** — the two listeners embody two different authorization philosophies (Telegram: chat-id-OR, no sender allowlist; Matrix: owner-sender with optional room). Aligning them into one "sender AND chat authorized" contract is a cross-component design decision (config schema + registry keys + both daemons), which is precisely an architectural boundary. Evidence above gives the exact guard lines and a minimal-change sketch to evaluate, not implement. + +### Scope-expansion note +Investigating Telegram/Matrix authorization surfaced that the auth contract is **not a single-file bug** but a **cross-component, decision-level** matter (two daemons, the `# POS_CONFIG:` registry, config templates, docs, and a new `TELEGRAM_OWNER_ID`-style key). That is an architectural decision, so the handoff above goes to **Architect** per the Explorer's scope rule. + +```text +Status: COMPLETE (with Architect handoff on scope expansion) +Reason: Fixing chat authorization correctly spans two listeners + config schema + registry + docs +Evidence: listener auth gates telegram:787 / matrix:550,539; POS_CONFIG headers in senders; no owner key anywhere +Affected areas: bin/pos-communication-telegram-listener, matrix-listener, telegram-sender (POS_CONFIG), config templates, DOC +Decision required: Architect +Out-of-scope changes: none +``` diff --git a/AgentsReport/explorer/2026-09-06_tooling-audit.md b/AgentsReport/explorer/2026-09-06_tooling-audit.md new file mode 100644 index 0000000..ac4e73f --- /dev/null +++ b/AgentsReport/explorer/2026-09-06_tooling-audit.md @@ -0,0 +1,242 @@ +# Explorer Report — 2026-09-06 — Tooling Audit (lint perf, install/uninstall symmetry, config-parsing duplication, shell correctness) + +## TL;DR + +- **Lint perf:** `scripts/lint-conventions.sh` (261 lines) has ~17 external-process hotspots; the dominant cost is **two full-file scans that spawn `printf | sed | tail` per line** (`uses_stdin` L67 and the top-level-`local` scan L143) — for a ~45-tool repo with several >1000-line tools this is roughly **60k+ subprocess forks per lint run** just from those two scans. Wall-time measurements are **UNVERIFIED** (sandbox denies `time`/`make`/`bash` execution); static hotspot inventory below is complete. +- **Install/uninstall symmetry:** a real uninstaller exists (`pos system uninstall`, 435 lines) but it is **PARTIAL** — 9 of 12 installed `lib/*.sh` have no removal path, the ScaleTail template clone (`/usr/local/share/linux_post_install/scale-tail`) and the feature-flag store (`/usr/local/share/linux_post_install/flags/`) are never removed, `~/.config/rclone/` and `/usr/local/bin/yt-dlp` survive, and **all runtime-created systemd *user* units** (`~/.config/systemd/user/`) are missed by every tier. +- **Config duplication:** **9 tools hand-roll byte-similar `load_config()`-style loaders** (env-wins export loop), plus at least 4 bespoke parsers; 2 shared key-value read/write libraries (`config-ui.sh` `cfg_value`/`cfg_write` and `entertainment-lib.sh` `config_value`/`write_config_key`) are duplicates of each other. CRLF-strip behavior splits 5-and-5; the `CONFIG_DIR`/XDG seam is honoured by self-contained tools but **bypassed by several common.sh-sourcing tools** that hardcode `$HOME/.config/linux_post_install/...`. Recommended owner: **`lib/config-ui.sh`**. +- **Shell correctness:** no high-confidence unquoted-`rm -rf`, unquoted-`[ $x ]`, or unguarded-`cd` bugs found in `bin/`; the flagged hotspots are `bin/pos:388/403` (`run eval "$cmd"` — AI-extracted command execution, deliberate but security-relevant), `bin/pos-system-uninstall:333` (`sed -i '/pos/d'` on user `.bash_completion`), and the per-line subprocess spawns in the lint script itself. + +--- + +## Task 1 — Lint performance (`scripts/lint-conventions.sh`, 261 lines) + +### 1.1 Measured timing + +| Command | Result | +|---|---| +| `time make lint` (run 1) | **UNVERIFIED** — sandbox denies `make`, `bash`, `time` execution | +| `time make lint` (run 2) | **UNVERIFIED** | +| `time bash scripts/lint-conventions.sh` | **UNVERIFIED** | + +Static hotspot analysis is complete and is the basis for the estimates (see 1.3). + +### 1.2 Per-line/per-file external-process hotspots (rule → implementation → bash-native alternative) + +F = per-file spawn, L = per-line spawn, 1× = one-off. + +| # | Lint rule | Location | Spawns per unit | Bash-native equivalent (no semantics change) | +|---|---|---|---|---| +| 1 | shebang check | L92 `head -1 "$f" \| grep -q` | F (2 procs/file) | `IFS= read -r first < "$f"` + `[[ $first == '#!/usr/bin/env bash' ]]` | +| 2 | strict-mode check | L95 `has_regex` → `grep -qE` | F (1 proc/file) | fold into the same first-line read as #1 | +| 3 | INTERACTIVE_CMDS extraction | L84 `sed -n … \| head -1` | 1× (2 procs) | single `read` with regex | +| 4 | `# POS:` header text | L110 `sed -n '/^# POS: /{…;q}'` | F (1 proc/tool) | read up to first `# POS:` line in bash loop | +| 5 | em-dash presence | L115 `grep -q ' — ' <<<"$headline"` | F (1 proc/tool, heredoc string) | `[[ $headline == *' — '* ]]` | +| 6 | `# POS:` line number | L118 `grep -nE … \| head -1 \| cut -d: -f1` | F (3 procs/tool) | captured in the same loop as #4 | +| 7 | deps-guard-before-help | L127 `first_guard_line` | F (0 extra — bash loop) | already bash-native | +| 8 | help-line number | L128 `first_line` → `grep -nE … \| while read` | F (1 proc/tool) | same first-match read loop as #4/#6 | +| 9 | top-level `local` scan | L137–157 with **L143 `printf '%s\n' "$line" \| sed -nE … \| tail -1`** | **L (3 procs per line of every tool)** | `[[ $line =~ <<-?[[:space:]]*([A-Za-z0-9_]+) ]]` in-bash | +| 10 | stdin-reader detection | L59–80 `uses_stdin` with **L67 `printf \| sed \| tail`** | **L (3 procs per line — scans every tool a 2nd time)** | same `[[ =~ ]]` regex; can also merge with #9 into ONE pass | +| 11 | POS.md reference | L169 `grep -q "$(basename "$f")" DOC/POS.md` | F (1 proc/tool) | read POS.md into a var once; `[[ $posmd == *$basename* ]]` | +| 12 | INTERACTIVE_CMDS entry check | L174–180 | — | fine | +| 13 | plugin common.sh / POS_PLUGIN / app uninstall fn+case / systemd / wrapper checks | L184,187,196,197,201,208,211,218,224 `has_regex` | F (1–2 procs/file each) | single-read first-match loop per file | +| 14 | wrapper line count | L221–222 `wc -l < "$f"` **twice** | F (2 procs/wrapper) | `mapfile -t lines < "$f"; ${#lines[@]}` | +| 15 | secret-literal scan | L229–243 `grep -nE` per file + **L235 `grep -qE … <<<"$body"` per matched line** | F + L (heredoc-string greps) | `[[ $body =~ (TOKEN|PASSWORD|…)= ]]` | +| 16 | system-path write scan | L245–258 `grep -nE` per file + **L251 two `grep -qE <<<"$body"` per matched line** | F + L | `[[ $body =~ (>|>>|tee ) ]] && [[ $body =~ (/etc/|\$HOME|/usr/local) ]]` | +| 17 | `last_line()` | L39–42 | **dead code — defined, never called** | delete | + +### 1.3 Estimated cost + +- The two per-line scans (#9, #10) each read every line of every `bin/pos-*` tool twice. The repo has 45 tools with several >1000-line files (pos-docker-vbox 1125, pos-media-ytsync 1213, pos-network-download 1108) — total tool lines ≈ 15–20k. At 3 forks/line × 2 scans ≈ **90k–120k `printf|sed|tail` subprocess forks per lint run** just from those two rules. +- Remaining rules add ≈ 10–15 forks per tool ≈ 500–700 more forks total. The secret/system-path scans add one grep per file plus per-matched-line heredoc greps. +- Expected effect: the lint time is dominated by process creation (fork/exec), not by grep itself. Replacing #9/#10 with `[[ =~ ]]` and merging into one pass should cut lint wall time by the largest factor; the heredoc-string greps (#5, #15, #16) are cheap per call but numerous. + +### 1.4 `scripts/check-sync.sh` (42 lines) — brief + +- `bash -n` per file, exec-bit loop, doc-sync via `gen-docs.sh --check`, 3 dispatch smokes. Per-file spawns are inherent to `bash -n` (must run bash anyway); no obvious perf bug. +- Correctness note: L14–16 glob list misses `features/*.sh`? — actually it includes `features/*.sh` (line 15 `features/*.sh`). It does NOT include `completions/*` other than `completions/pos.bash` (fine) and does not `bash -n` `install.sh`'s sourced libs beyond the list — libs are covered. No hotspot. + +### 1.5 `scripts/gen-docs.sh` (254 lines) — brief + +- `sed` per header per tool (L42–47: 6 sed calls/tool) — minor; only runs on `make gen`, not per commit. +- L207–210 check mode: `sed` block extract + `cat` + `diff` per block — fine for check. +- Correctness hotspot: docmap convergence loop (L244–252) re-runs `regen_block docmap` up to 5 times by design; each iteration re-does a full-file `sed` + `grep -n` + `wc -l` — acceptable (documented convergence), but on a 700-line file it is the single slowest part of gen; an in-memory line accounting would converge in one pass. Not a bug. + +### Task 1 — Ranked change list (no implementation) + +1. Merge the per-line `printf|sed|tail` delimiter extraction into a single bash-native pass using `[[ $line =~ <<-?[[:space:]]*([A-Za-z0-9_]+) ]]` — used by the top-level-`local` scan (L143) and `uses_stdin` (L67). Highest ROI; removes the ~90–120k fork estimate. +2. Replace per-line heredoc-string `grep -q <<<"$body"` with `[[ $body =~ … ]]` in the secret (L235) and system-path (L251) scans. +3. Replace per-file `grep`/`sed|head|cut`/`wc` with a single read of the first ~6 lines per tool (covers L92/95/110/115/118/128) plus `mapfile` line counts for wrappers (L221). +4. Replace `grep -q DOC/POS.md` (L169) with one preloaded POS.md content check. +5. Delete the dead `last_line()` (L39–42). + +--- + +## Task 2 — Install/uninstall symmetry + +### 2.1 What exists + +- **Installer:** `install.sh` phases 1–4 + optional apps. Uninstall path documented in `install.sh:68–71` (apps only) and provided as a **CLI tool** `bin/pos-system-uninstall` (not a `make uninstall`, not a scripts/ uninstaller — grep of `Makefile`, `scripts/`, `README.md` shows no `make uninstall`; `apps/install.sh --uninstall` handles optional desktop apps only). +- **Uninstaller:** `bin/pos-system-uninstall` — Tier 1 (always): binaries, plugins, known systemd services, shell integration; Tier 2 (`--config`): `~/.config/linux_post_install`; Tier 3 (`--data`): `~/.local/share/linux_post_install`. + +### 2.2 Install inventory vs uninstall coverage + +| Artifact | Installed by | Removal path | Verdict | +|---|---|---|---| +| `bin/*` → `/usr/local/bin/` (45 pos-*, pos, flag-*, wr-*, mp3/mp4/vbox/ssh-load-all) | Phase 2 (install.sh:136–140) | Tier 1: `/usr/local/bin/pos` + `compgen -G /usr/local/bin/pos-*` + legacy names (L53–83, 229–257) | **SYMMETRIC** | +| `lib/common.sh`, `lib/menu-lib.sh`, `lib/share-lib.sh` → `/usr/local/bin/` | Phase 2 (install.sh:143–144) | Tier 1 (pos-system-uninstall:62–64) | **SYMMETRIC** | +| `lib/{flags,notify,entertainment-lib,scheduler-lib,config-ui,user-timers-lib,entertainment-plugin-lib,usb-lib,registry}.sh` → `/usr/local/bin/` | Phase 2 (install.sh:143–144) | **none** | **INSTALL-ONLY** (9 of 12 libs) | +| `lib/ai-providers/*.sh` → `/usr/local/bin/ai-providers/` | Phase 2 (install.sh:154–161) | Tier 1 `rm -rf /usr/local/bin/ai-providers` (L244) | **SYMMETRIC** | +| `entertainment/*.sh` → `/usr/local/bin/` | Phase 2 (install.sh:167–173) | Tier 1 — **hardcoded list** `weather.sh gold.sh joke.sh` (L72, 248) | **SYMMETRIC today**; breaks automatically if a 4th plugin is added | +| `x64_bin|arm64_bin/*` → `/usr/local/bin/` | Phase 2 (install.sh:179–194) | Tier 1 hardcoded `wihotspot wihotspot-gui create_ap` (L86, 260) | **SYMMETRIC today**; same hardcode fragility | +| `features/*` → `/usr/local/bin/` (--feature) | Phase 2 (install.sh:197–222) + flag set | Tier 1 `autostart.sh usb-automount.sh` (L91, 265) | **SYMMETRIC today**; hardcoded | +| **Feature-flag store** `/usr/local/share/linux_post_install/flags/` | Phase 2 `flag_set` (install.sh:216) | **none** (no `flags`/`flag` match in pos-system-uninstall) | **INSTALL-ONLY** | +| **ScaleTail clone** `/usr/local/share/linux_post_install/scale-tail` | Phase 4 (install.sh:237–243) | **none** (only bash-completion under /usr/local/share is removed, L108) | **INSTALL-ONLY** | +| `completions/pos.bash` → `/usr/local/share/bash-completion/completions/pos.bash` | postinstall.sh:100–107 | Tier 1 (L108, 284) | **SYMMETRIC** | +| systemd `*.service`/`*.timer` → `/etc/systemd/system/` + enable | postinstall.sh:139–166 | Tier 1: 3 known + find `-name '*linux_post_install*' -o -name 'pos-*'` (L287–311) | **SYMMETRIC** (system units) | +| `config/authorized_keys` → `~/.ssh/authorized_keys` | postinstall.sh:110–137 | **none** (tier 2 only targets `~/.config/linux_post_install`) | **INSTALL-ONLY** (by design — user data) | +| `config/rclone.conf` → `~/.config/rclone/rclone.conf` | postinstall.sh:10–17 | **none** (tier 2 path is `linux_post_install` only) | **INSTALL-ONLY** | +| `config/{entertainment,system,notify,ai}.env` → `~/.config/linux_post_install/` | postinstall.sh:22–50 | Tier 2 (`--config`) find over the dir (L147–155) | **SYMMETRIC** (opt-in tier) | +| `config/schedule.d/*.env` → `~/.config/linux_post_install/schedule.d/` | postinstall.sh:57–77 | Tier 2 + rmdir schedule.d (L351–355) | **SYMMETRIC** (opt-in tier) | +| PATH line + completion line in `~/.bashrc` | postinstall.sh:80–98 | Tier 1 sed removals (L320–322) | **SYMMETRIC** | +| apt packages (25+) + yt-dlp → `/usr/local/bin/yt-dlp` + cpufreq | preinstall.sh:28–75 | **none** (uninstaller never touches apt or yt-dlp) | **INSTALL-ONLY** (likely deliberate — system packages) | + +### 2.3 Runtime-created state (created by tools at runtime, not install.sh) + +| Artifact | Created by | Uninstall path in pos-system-uninstall | Verdict | +|---|---|---|---| +| `~/.config/linux_post_install/.env` (ai, telegram, matrix, scrcpy, download, ytsync, grab, ai-aliases, compose) | tools' config writes | Tier 2 (`--config`) | **RUNTIME-STATE / SYMMETRIC** (removed with --config) | +| `~/.local/share/linux_post_install/{logs,ytsync,ai/models,entertainment/last,backups}` | bin/pos logging + tools | Tier 3 (`--data`) | **RUNTIME-STATE / SYMMETRIC** (removed with --data) | +| **systemd *user* units** `~/.config/systemd/user/`: `pos-aria2.service`+`pos-aria2-retry.{service,timer}` (pos-network-download:172–190,633–669), telegram-listener unit (pos-communication-telegram-listener:471–521), matrix-listener unit (pos-communication-matrix-listener:315–364), `pos-ai-server.service` (pos-ai-server:500–522), entertainment timers `pos-entertainment-*.timer` (user-timers-lib), scheduler per-job timers (scheduler-lib) | runtime tool subcommands | **none** — Tier 1 only scans `/etc/systemd/system` (L287–311); Tier 2 only `~/.config/linux_post_install`; `~/.config/systemd/user/` is outside both | **INSTALL-ONLY** (from the uninstaller's perspective; each tool's own `stop`/`disable` subcommand does remove its own unit, e.g. `pos network download stop` L209–211) | +| `~/.local/bin/pos-ai-hook.sh` + `ai-aliases.sh` wrappers | pos-ai-alias | Tier 1 pos-ai-hook + marker-managed alias scan (L96–105, 270–281) | **SYMMETRIC** | +| `~/.config/rclone/rclone.conf` (from postinstall) | postinstall.sh:10–17 | none | **INSTALL-ONLY** | + +### Task 2 — Ranked change list (no implementation) + +1. **Remove the 9 orphaned libs** (`flags.sh`, `notify.sh`, `entertainment-lib.sh`, `scheduler-lib.sh`, `config-ui.sh`, `user-timers-lib.sh`, `entertainment-plugin-lib.sh`, `usb-lib.sh`, `registry.sh`) in Tier 1 — the biggest INSTALL-ONLY gap (hardcoded `common.sh menu-lib.sh share-lib.sh` only, pos-system-uninstall:62–64). +2. **Remove ScaleTail templates** `/usr/local/share/linux_post_install/scale-tail` and the **feature-flag store** `/usr/local/share/linux_post_install/flags/` in Tier 1 (documented install outputs in AGENT_Context §3/§10, no removal). +3. **Add a user-unit sweep** to Tier 1: disable+remove matching units in `~/.config/systemd/user/` (prefixes `pos-*`, `pos-entertainment-*`, `pos-schedule-*` etc.), or document that per-tool `stop` is the supported path. +4. De-hardcode the entertainment-plugin / prebuilt-binary / feature names in the uninstaller to directory-driven discovery (mirror install.sh's loops) so future plugins/bins don't silently become INSTALL-ONLY. +5. Decide (and document) whether `~/.config/rclone`, `~/.ssh/authorized_keys` additions, apt packages, and `/usr/local/bin/yt-dlp` are intentionally outside uninstall — currently silent. + +--- + +## Task 3 — Config-parsing duplication + +### 3.1 Inventory + +**Shared loaders that exist:** + +| Loader | Location | Used by | +|---|---|---| +| `load_system_env()` (env-file → export, env-wins) | lib/common.sh:146–159 | pos-system-health, pos-system-backup, pos-media-sync (system.env) | +| `cfg_value()` / `cfg_write()` (key read/write, `KEY="value"`, chmod 600) | lib/config-ui.sh:311–344 | pos-config; pos-entertainment-config:L98 sources config-ui dynamically; config-scope registry consumers | +| `config_value()` / `write_config_key()` (key read/write, same semantics) | lib/entertainment-lib.sh:28–54 | pos-entertainment-{config,status,enable,disable,send} | +| inline `grep '^NOTIFY_PLATFORM=' \| tail -1 \| cut` | lib/notify.sh:41 | notify_send | + +**Hand-rolled near-identical `load_config()`-style loaders (9)** — each is the same ~14-line loop: `grep -E '^[A-Z_]+=' | while IFS='=' read k v` + quote-strip + `[ -z "${!k:-}" ] && export`: + +1. bin/pos-communication-telegram-sender:61–74 (`load_config`) +2. bin/pos-communication-matrix-listener:65–77 (`load_config`) +3. bin/pos-communication-telegram-listener:86–98 (`load_config`) +4. bin/pos-communication-matrix-sender:44–57 (`load_config`) +5. bin/pos-communication-scrcpy:14–28 (`load_config`) +6. bin/pos-ai:130–160 (`load_config`, plus legacy-file loop) +7. bin/pos-ai-server:21–35 (`load_config`) +8. bin/pos-ai-hf:30–44 (`load_hf_config`) +9. bin/pos-media-grab:10–23 (`load_grab_config`) + +**Bespoke parsers (4+):** + +- bin/pos-network-download:32–37 `load_secret` — single-key `grep'^RPC_SECRET=' | head -1 | cut -d= -f2-`; also duplicated inline at L153–154 +- bin/pos-docker-compose:9–10 + layered strategy (`template < global compose.env < per-service .env`, documented L31–48) — reads global config via `CONFIG_ENV` and per-service envs +- bin/pos-share-smb-server:97 `reload_config` — Samba-specific +- bin/pos-media-ytsync:28–30 `_YTSYNC_CFG` via `pos config ytsync` scope; plus the share-client (`pos-share-smb-client`) creds records parsing + +**Counts:** 45 `pos-*` tools; ~20 source `lib/common.sh`; **9 hand-roll their own file parser**; only `pos-config` and the entertainment tools use a shared key-value loader; 3 use `load_system_env`; the 5 self-contained communication tools duplicate the loader because they don't source common.sh (documented convention: guarded inline fallback copies in DEV.md). + +### 3.2 Consistency findings + +- **Precedence order** is `env > config-file > defaults` everywhere the hand-rolled loaders are used (`if [ -z "${!k:-}" ]` before export; defaults applied later via `${VAR:-default}`). CLI-vs-config precedence is declared `CLI > environment > config file` in the three tools that document it (telegram-sender:45, scrcpy:70, matrix-sender:29). pos-docker-compose is the outlier model (per-service file wins over global file; no env) — a different domain, but also the only tool where "config file" beats "global defaults" deliberately. +- **CRLF handling diverges:** 5 loaders strip `\r` (matrix-sender:53, scrcpy:23, ai:139, ai-server:30, ai-hf:39) but 5 do NOT (telegram-sender, matrix-listener, telegram-listener, media-grab, and `load_system_env` in common.sh:154). A Windows-edited `.env` parses differently depending on which tool reads it. +- **CONFIG_DIR / XDG seam divergence:** self-contained tools (+ config-ui.sh:32, notify.sh:27) carry the guarded `CONFIG_DIR="${CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/linux_post_install}"` copy; but several tools that **source** common.sh (which defines `CONFIG_DIR` at line 19) still hardcode `$HOME/.config/linux_post_install/...`: pos-ai:11, pos-ai-hf:26, pos-ai-server:15, pos-media-grab:11, common.sh load_system_env:147, entertainment-plugin-lib:15, pos-docker-compose:10. So `CONFIG_DIR`/`XDG_CONFIG_HOME` overrides work for some tools and are silently ignored by others. +- **Key-value writers duplicated** — `cfg_write` (config-ui.sh:323) and `write_config_key` (entertainment-lib.sh:36) are the same algorithm (grep -v + append, `-` deletes, chmod 600); only the value-quoting and the multi-line warning differ. + +### 3.3 Recommendation (no implementation) + +- **Owner: `lib/config-ui.sh`.** It already hosts the `POS_CONFIG` scope registry that `pos config` consumes, has secret masking/validation helpers, and is installed to `/usr/local/bin` alongside the tools. +- Add one generic loader there, e.g. `load_env_file ` (env-wins export loop with quote + CRLF strip parsed consistently) and have `load_system_env` delegate to it. +- Merge `entertainment-lib.sh` `config_value`/`write_config_key` into `cfg_value`/`cfg_write` (keep `config_value` as a thin alias for the entertainment tools, or migrate the 5 call sites). +- **Migration targets:** the 9 hand-rolled loaders → `load_env_file` (source `config-ui.sh` in the 5 self-contained communication tools, replacing their guarded inline copies and CONFIG_DIR blocks); `pos-network-download` → `cfg_value "$CONFIG_FILE" RPC_SECRET`; `pos-docker-compose` `config show` → `cfg_value`/`cfg_write` for the global config. +- Unify CRLF-strip and the `CONFIG_DIR` path source across every loader during the migration. + +### Task 3 — Ranked change list (no implementation) + +1. Add `load_env_file` to `lib/config-ui.sh`; make `common.sh load_system_env` delegate; fix the CRLF split in the process. +2. Migrate the 9 hand-rolled loaders (list in 3.1) to it; make the 5 self-contained communication tools source `config-ui.sh` instead of the inline CONFIG_DIR+load_config copies. +3. Fold `entertainment-lib.sh` read/write helpers into `cfg_value`/`cfg_write` (alias or migrate the 5 entertainment call sites). +4. Route `pos-network-download` `load_secret` and `pos-docker-compose` global-config reads through `cfg_value`. +5. Replace hardcoded `$HOME/.config/linux_post_install/...` in the common.sh-sourcing tools with the sourced `CONFIG_DIR` (pos-ai, pos-ai-hf, pos-ai-server, pos-media-grab, load_system_env, entertainment-plugin-lib, pos-docker-compose). + +--- + +## Task 4 — Shell-correctness hotspots (high-confidence only) + +Method: targeted scan of `bin/` and `lib/` for unquoted `$var` in args/array appends, `for x in $list`, `[ $x = … ]`, `rm -rf $VAR`, unguarded `cd`, missing `|| true` in pipelines under `set -euo pipefail`, `eval` of derived strings. Only high-confidence items below. + +### 4.1 High-confidence findings + +- **H-001 (WARN) — `bin/pos-ai:388,403` `run eval "$cmd"`.** `_prompt_run_command` executes a command string extracted from AI output. Interactive path prompts on `/dev/tty`; the `--trust` path (L385–388) auto-executes without confirmation. Deliberate feature, but any AI-output-derived command executed through `eval` is a shell-injection-relevant surface — recommend keeping, but it deserves explicit review of what `trusted=1` callers feed it. Classification: FACT (code), design concern. +- **H-002 (WARN) — `bin/pos-system-uninstall:333` `sed -i '/pos/d' "$HOME/.bash_completion"`.** Deletes **every** line containing the substring `pos` from a user-owned file, not just pos-managed lines (unlike the `pos-ai-hook` marker check at L277). A line like `complete -F _git checkout` is safe, but any unrelated completion containing "pos" (e.g. `repos`, `compose-help`, `dispose`) is silently removed — and this runs in default Tier 1. Classification: FACT. +- **H-003 (WARN) — `bin/pos-system-uninstall:320–322` `sed -i` on `~/.bashrc`.** Removal of PATH/completion/hook lines is line-based and unanchored at line start (`/source.*pos\.bash/d`, `/linux_post_install.*PATH/d`, `/source.*pos-ai-hook/d`); a user comment mentioning `pos.bash` is deleted. Lower risk than H-002 but same class. Classification: FACT. +- **H-004 (WARN) — `scripts/lint-conventions.sh:31,41,67,84,110,118,143,235,251`.** Under `set -euo pipefail`, the `grep | while read` and `... | tail -1 | cut` pipelines are only safe because of the `|| true` / `2>/dev/null` guards and the non-final elements' exit codes. The per-line `printf | sed | tail -1` inside the read loop (L67/L143) is the perf hotspot from Task 1 AND a correctness risk: if `sed` ever exits non-zero for a given line under `pipefail`, the surrounding `while read` loop aborts mid-scan. Classification: FACT (perf measured as static analysis); correctness risk is conditional, not observed. + +### 4.2 Checked and cleared (not bugs) + +- `rm -rf`/`rm -f` in `bin/` are consistently quoted (`pos-ai-hf:644,698,835,989`; `pos-system-uninstall` all lines; `pos-docker-vbox:1115`; app scripts). No unquoted/empty `rm -rf $VAR` found. +- Unquoted `[ $x … ]` comparisons: none found in `bin/` (only `"$var"` forms). +- `for x in $list` sites (`pos-docker-health:38`, `pos-docker-ps:36`, `pos-tree:71`, `pos-system-health:170`, `pos-network-checkport:408`, `pos-entertainment-status:39`, `pos:123`) intentionally word-split newline/comma-separated IDs or sorted output with no spaces in elements — not bugs at present, but a space in a future element (e.g. a plugin filename) would silently split. Low-priority hardening, not a defect. +- `cd` sites are guarded (`pos-docker-vbox:1029,1040` use `(cd "$d" 2>/dev/null && pwd) || …`; `pos-docker-compose:221–260` wrap in subshells with `set -e` context). +- `pos-share-smb-client:457` `sudo rm -f "$SMB_CREDS_DIR/$(basename "$where")"` — properly quoted. +- `find /etc/systemd/system/ -name '*linux_post_install*' -o -name 'pos-*'` (pos-system-uninstall:311) — `-o` binds both predicates to the stated path; matches both patterns as intended. Not a bug. +- `bin/pos` logging tee pipes: INTERACTIVE_CMDS handling verified by lint rule and existing registrations — no new finding. + +### Task 4 — Ranked change list (no implementation) + +1. Restrict `pos-system-uninstall` `.bash_completion`/`.bashrc` removal to anchored, marker-based patterns (e.g. only lines the installer itself added, or apply the `grep -q 'Managed by pos…'`-style marker check used for alias wrappers). +2. Review `bin/pos-ai` `_prompt_run_command` trust boundaries: confirm every `trusted=1` caller is user-flagged and document the eval surface (or re-run through `bash -c` with validation). +3. Convert lint L67/L143 per-line `printf|sed|tail` to `[[ =~ ]]` (also removes the pipefail-mid-loop abort risk). +4. Optional hardening: quote the `for x in $list` sites that consume plugin names/scheduled-job names where elements could contain spaces. + +--- + +## Uncertainties + +- **Lint wall time** could not be measured (sandbox denies `time`, `make`, `bash`). Estimates are derived from hotspot counts and file sizes (45 tools, 15–20k total lines); real numbers should be captured by a runner-capable agent (`make lint` ×2 + bare script) — see Handoff. +- The exact fork count per run is an INFERENCE (each `printf|sed|tail` is at least 3 forks; actual exec cost depends on PATH lookup and filesystem state). +- Whether apt packages / yt-dlp / `~/.ssh/authorized_keys` / `~/.config/rclone` are *supposed* to survive uninstall is a product decision, not verifiable from code. +- Whether the runtime-created user units are "expected to persist" is not documented anywhere in the repo; the uninstaller help text ("services") implies coverage, which is not delivered. + +## Important Files + +- `scripts/lint-conventions.sh` — all Task 1 hotspots (L31,41,67,84,92,95,110,115,118,137–157,162,169,221–222,235,251; dead `last_line` L39–42) +- `scripts/check-sync.sh`, `scripts/gen-docs.sh` — gates; convergence loop L244–252 +- `install.sh` — phases, `should_run` (L100–117), copy targets (L136–222), ScaleTail (L237–243) +- `preinstall.sh` — apt PACKAGES (L28–43), yt-dlp (L65–68) — no uninstall counterpart +- `postinstall.sh` — rclone/entertainment/system/notify/ai env templates, schedule.d, .bashrc, completion, systemd +- `bin/pos-system-uninstall` — tiers, lib list L62–64, user-unit gap, H-002/H-003, find L311 +- `bin/pos`, `bin/pos-ai`, `bin/pos-communication-{telegram,matrix}-{sender,listener}`, `bin/pos-communication-scrcpy`, `bin/pos-ai-server`, `bin/pos-ai-hf`, `bin/pos-media-grab`, `bin/pos-network-download`, `bin/pos-docker-compose`, `bin/pos-share-smb-server` — config-loading inventory (Task 3) +- `lib/common.sh` (load_system_env), `lib/config-ui.sh` (cfg_value/cfg_write), `lib/entertainment-lib.sh` (config_value/write_config_key), `lib/notify.sh` — loader candidates +- `DOC/DEV.md:182–213` — env-seam rules the loader centralization should preserve + +## Handoff + +- **Status:** OBJECTIVE_SATISFIED (plus measurement note) +- **Objective:** evidence audit of lint performance, install/uninstall symmetry, config-parsing duplication, shell-correctness hotspots — completed read-only. +- **Evidence / completed work:** this report; hotspot inventory with file:line; install/uninstall matrix; 9-loader duplication census with CRLF and CONFIG_DIR inconsistencies; 3 high-confidence shell hotspots. +- **Affected areas:** `scripts/lint-conventions.sh`, `bin/pos-system-uninstall`, `lib/config-ui.sh` + `lib/entertainment-lib.sh` + `lib/common.sh` (loader centralization), 9 tool files, `bin/pos-ai`. +- **Scope/decision boundary:** no code changed. Centralizing the loader (Task 3) is a deliberate cross-tool refactor with a doc convention ("guarded inline fallback copies" in DEV.md) — that is an Architect/Designer decision boundary, not a mechanical fix. +- **Verification performed:** full reads of lint/check-sync/gen-docs/install/preinstall/postinstall/uninstall/common/config-ui; greps across `bin/`+`lib/` for loaders, rm/cd/eval/for-splitting patterns; shared-memory check of maintainer/architect reports (no overlap: the 2026-09-06 convention-sweep was about POS header/doc drift, not these four areas). +- **Remaining uncertainty:** measured lint wall time (needs a runner-capable agent); intended persistence of apt packages/rclone/user units; actual fork count (inference). +- **Recommended next agent:** **Architect** (for the loader centralization decision: which library owns `load_env_file`, how self-contained tools source config-ui.sh without breaking the "no shared lib? inline fallbacks" convention) — and/or **Maintainer** for the uninstaller gaps + lint per-line hotspot rewrite if a decision is not needed. +- **Reason:** Task 3's fix crosses the documented DEV.md convention and 9 tool files (architectural boundary); Tasks 1/2/4 are mechanical cleanups that a Maintainer can implement once the loader decision is made. \ No newline at end of file diff --git a/AgentsReport/maintainer/2026-09-06_agendatodo-update.md b/AgentsReport/maintainer/2026-09-06_agendatodo-update.md new file mode 100644 index 0000000..4134120 --- /dev/null +++ b/AgentsReport/maintainer/2026-09-06_agendatodo-update.md @@ -0,0 +1,28 @@ +# Maintainer Report — 2026-09-06 AGENT_TODO maintenance + +## TL;DR + +- Fixed stale `DOC/APPS.md 15→16` count in the llamacpp Done entry (filetable is now 18 after stabilization pass). +- Added one new Done entry summarizing the full stabilization pass. +- Both edits are uncommitted; `AGENT_TODO.md` is not part of `make gen/check/lint`. + +## Step 1: Fix stale count in llamacpp Done entry + +**Finding:** `AGENT_TODO.md:45` says `DOC/APPS.md 15→16` — the security-track work later raised the filetable to 18. +**Status:** DONE +**Change:** Replaced `DOC/APPS.md 15→16` with `DOC/APPS.md app-table row count updated (15→16 at the time; 18 after the 2026-09-06 stabilization pass)`. +[DONE] + +## Step 2: Add stabilization-pass Done entry + +**Finding:** No Done entry summarizes the 17-point stabilization pass executed 2026-09-06. +**Status:** DONE +**Change:** Inserted a new `- **2026-09-06** — Stabilization pass …` entry at the top of the Done section (newest-last placement). +[DONE] + +## Verification + +- `git diff --stat AGENT_TODO.md` → `4 insertions` (new-newest Done entry + the count substitution). +- The diff shows exactly the two changed regions: `@@ -42,6 +42,10 @@ ## Done` — new entry inserted at top of Done, and the count substitution inside the llamacpp entry. +- No other entries moved/reordered; Later/Next untouched; no commit made. +[DONE] diff --git a/AgentsReport/reviewer/2026-09-06_llamacpp-app_review.md b/AgentsReport/reviewer/2026-09-06_llamacpp-app_review.md new file mode 100644 index 0000000..b93d784 --- /dev/null +++ b/AgentsReport/reviewer/2026-09-06_llamacpp-app_review.md @@ -0,0 +1,129 @@ +# Reviewer Report — 2026-09-06 — llamacpp optional-app installer + `ai` category + +## TL;DR + +- **Status:** REQUEST_CHANGES — 2 REQUIRED, 3 SUGGESTED, 3 NOTE. All functional/structure claims verified; two fixable defects block acceptance. +- **REQUIRED-1:** `apps/ai/llamacpp.sh` is **not executable** (mode 100644, untracked); every tracked `apps/*` file is 100755. Fix: `chmod +x` before `git add`. +- **REQUIRED-2:** `DOC/APPS.md:3` says "16 optional desktop application installers"; the catalog table (rows 73–90) and the filesystem both hold **18** app installers (17 pre-existing tracked + the new one). The 15→16 change propagated a stale count (correct: 18). +- **SUGGESTED:** (S1) failures inside the embedded `bash -c` install block are masked as success (no `set -e` in child — same class pattern as `scrcpy.sh`, so fix across the class); (S2) `DOC/DEV.md:292` categories list misses `ai`; (S3) unguarded `python3` dependency yields a misleading error when absent. +- Every other item in the brief verified [PASS]: shebang/strict-mode/source path, idempotence guard before network/sudo, `/releases?per_page=10` scanning, `.tar.gz` suffix matcher, `--strip-components=1`, symlink-loop + readlink-target uninstall (no blanket `rm llama*`), temp cleanup on success + matcher-failure path, `case` dispatch with `uninstall`, CAT_NAMES `[ai]`, template comment, exactly 3 text lines in `bin/pos-ai-server` (no logic change), 1-line `DOC/POS.md` hint (accurate — `install.sh --apps/--full` verified), AGENT_TODO Done entry, scope/hygiene clean. +- **Unverified (read-only boundary):** live GitHub API probe results, and `make gen/check/lint` runs (I did not re-run them); static evidence is consistent with the Builder's claims. + +## Step 1: `apps/ai/llamacpp.sh` — structure, idempotence, release discovery, cleanup, dispatch + +- [x] Shebang `#!/usr/bin/env bash` (line 1), `set -euo pipefail` (line 2), sources `lib/common.sh` via `$(dirname "$0")/../../lib/common.sh` (line 3) — correct depth for `apps/ai/`. +- [x] `install_llamacpp()` idempotence guard **first**: `command -v llama-server &>/dev/null && { log "llama.cpp already installed"; return 0; }` (line 16) — before any network (lines 25–27) and before any sudo (lines 49–55). +- [x] Arch mapping `x86_64→x64`, `aarch64→arm64`, unsupported arch errors (lines 19–23). +- [x] Discovery: fetches `/releases?per_page=10` (line 13); python3 matcher scans `r['assets']` for `a['name'].endswith('-bin-ubuntu-$arch.tar.gz')` (lines 30–40). `endswith` on the exact suffix ⇒ no false positives on `.sha256`/`.txt` assets. No-match path removes the temp JSON and `err`s with a clear message + GitHub URL (lines 41–44). +- [x] `--strip-components=1` extraction (line 51) — consistent with the claimed top-level-dir archive layout (Builder live probe; cannot re-probe read-only → see unverified). +- [x] Symlink loop links every `llama*` binary from the install dir to `/usr/local/bin`, guarded by `[ -f ] && [ -x ]` (lines 52–55). +- [x] Temp cleanup on success (line 56, both files) and on the matcher no-match path (line 42). No `trap`; remaining early-failure edge → NOTE-3. +- [x] `uninstall_llamacpp()`: idempotent guard (line 63), removes `/usr/local/lib/llama.cpp-*` (line 65), and removes **only** symlinks whose `readlink` target matches `/usr/local/lib/llama.cpp-*` (lines 69–77) — **no** blanket `rm -f /usr/local/bin/llama*`; unrelated `/usr/local/bin/llama*` files are left alone. Extra edge → NOTE-2. +- [x] `case "${1:-}"` dispatch with `uninstall` arm (lines 82–85). Matches lint rule `lint-conventions.sh:192-204` (function name + `uninstall)` dispatch present). +- [x] Dry-run: every mutation is inside `spawn` (common.sh `spawn` logs and returns 0 under `DRY_RUN=1`, lines 79–82) — no mutation under `DRY_RUN=1`. Nuance → NOTE-1. + +[PASS] + +## Step 2: `apps/install.sh` — CAT_NAMES + discovery + +- [x] `CAT_NAMES` gains `[ai]="AI / ML"` (line 45); alphabetical position consistent. +- [x] Discovery loop (lines 59–78) is directory-driven; no per-category registration — `apps/ai/` is picked up automatically; `bash apps/install.sh llamacpp` resolves via `find_app_category`; `--uninstall llamacpp` path works the same. + +[PASS] + +## Step 3: `templates/app.sh` — category comment + +- [x] Comment now reads "Categories: ai, browsers, development, media, networking, remote-access, system, utilities." (line 8) — `ai` added, nothing else drifted (single-hunk diff). + +[PASS] + +## Step 4: `bin/pos-ai-server` — exactly 3 text lines + +- [x] `git diff bin/pos-ai-server`: 3 hunks, +3/-3, string-only: + - help "Requires:" line (now line 256), + - `cmd_start` `err` (line 397), + - `cmd_status` `err` (line 563). +- [x] All three name the installer (`apps/ai/llamacpp.sh`), give the invocation (`bash apps/install.sh llamacpp`, `--apps`/`--full`), and keep the GitHub URL `https://github.com/ggerganov/llama.cpp`. +- [x] `--apps`/`--full` claim verified against root `install.sh:57-58,78-79` and `DOC/APPS.md:26` — accurate. +- [x] Phrasing mirrors the scrcpy model (`bin/pos-communication-scrcpy:11`: "… not found — install … with the app installer: 'apps/media/scrcpy.sh' … see 'pos help …'"). No logic changed — diff is text-only in message strings. + +[PASS] + +## Step 5: `DOC/APPS.md` — count, categories, row + +- [x] Categories line now includes `ai` (line 37); llama.cpp catalog row added (line 73) with method "GitHub release → `/usr/local/lib/llama.cpp-` + `/usr/local/bin` symlinks" and category `ai` — matches the installer (GitHub release `.tar.gz` → `/usr/local/lib`, cat `ai`). +- [FAIL] Count line (line 3): says "16 optional desktop application installers". Actual = **18**. Evidence: catalog rows 73–90 = 18 rows; `apps/**/*.sh` glob = 19 files incl. `apps/install.sh` (18 app installers); `git ls-files 'apps/*/*.sh'` = 17 tracked + 1 new. The pre-existing "15" was already stale (17 real installers at HEAD); the change kept the arithmetic wrong (correct value 18). REQUIRED-2. + +[FAIL] + +## Step 6: `DOC/POS.md` — single-line hint + +- [x] Diff shows exactly one line changed (line 125, `pos ai server` Flags line): appends "(install via `bash apps/install.sh llamacpp`)" to the existing sentence. Accurate. + +[PASS] + +## Step 7: `AGENT_TODO.md` — Done entry + +- [x] New entry dated **2026-09-06** inserted at the top of `## Done` (line 45), before the prior 2026-09-06 entries — accurate, no duplication, matches the implementation (including the "15→16" claim which carries the same count defect as the DOC — noted in R2). + +[PASS] + +## Step 8: Hygiene + executable mode + +- [x] No debugging artifacts, no hardcoded machine paths, no secrets, no committed temp files. `apps/ai/` contains only `llamacpp.sh`. +- [FAIL] Executable mode: **existing apps are tracked 100755** (`git ls-files -s apps` → every `apps/*` including `apps/install.sh` is `100755`; `templates/app.sh` also 100755). The new `apps/ai/llamacpp.sh` is **100644** (`git diff --no-index --summary /dev/null apps/ai/llamacpp.sh` → "create mode 100644"). The repo convention stores app exec bits; the new file breaks that uniformity and `git add` would persist 100644. Note: gates won't catch it (`check-sync.sh` exec loop covers only `bin/pos*`; `lint-conventions.sh` `executable_files()` covers `bin/pos-*` + `entertainment/*.sh` only). REQUIRED-1. + +[FAIL] + +## Step 9: Scope + +- [x] `git status --porcelain`: modified = AGENT_TODO.md, DOC/APPS.md, DOC/POS.md, apps/install.sh, bin/pos-ai-server, templates/app.sh; untracked = AgentsReport/builder/2026-09-06_llamacpp-app.md, apps/ai/. `git diff --stat` = 6 files, +11/-7 — consistent with the claimed change set. +- [x] No `lib/`, `config/`, `install.sh`/`preinstall.sh`/`postinstall.sh`, `completions/`, or GEN: block changes. Builder's "make gen changed nothing" is consistent with the absense of generated-block diffs. + +[PASS] + +## Findings + +| # | Severity | File:line | Finding | Evidence | +|---|----------|-----------|---------|----------| +| R1 | REQUIRED | `apps/ai/llamacpp.sh` (untracked, whole file) | New app installer is not executable (100644); all tracked app installers/template are 100755. `git add` will persist 100644 | `git diff --no-index --summary /dev/null apps/ai/llamacpp.sh` → "create mode 100644"; `git ls-files -s apps` → all 100755; `check-sync.sh:22-24` + `lint-conventions.sh:100-105` only enforce exec bits for `bin/pos*`/`entertainment/*.sh`, so gates are blind to this | +| R2 | REQUIRED | `DOC/APPS.md:3` | Count says 16; actual installer count is 18 (17 pre-existing tracked + llamacpp). Pre-existing "15" was already stale; change should have gone to 18 | Catalog table rows 73–90 (18 rows, read of DOC/APPS.md); glob `apps/**/*.sh` → 19 files incl. `apps/install.sh` (18 installers); `git ls-files 'apps/*/*.sh'` = 17 | +| S1 | SUGGESTED | `apps/ai/llamacpp.sh:46-57` | Embedded `bash -c` install block has no `set -e`; a failed `curl -fsSL`/`tar` continues to `rm` (rc 0) → `spawn` prints OK and line 59 logs "installed" on a failed install. Same class pattern as `apps/media/scrcpy.sh:26-34` — recommend adding `set -e` inside the child (or `&&`-chaining), ideally fixed across both installers | read of llamacpp.sh:46-57 vs scrcpy.sh:26-34; common.sh:90-115 (`spawn` runs `bash -c` child, no errexit inheritance) | +| S2 | SUGGESTED | `DOC/DEV.md:292` | "Adding an Optional App" categories list omits the new `ai` category (updated in `templates/app.sh:8` and `DOC/APPS.md:37` but not DEV.md) | read of DEV.md:292; diff of templates/app.sh + DOC/APPS.md | +| S3 | SUGGESTED | `apps/ai/llamacpp.sh:30-43` | `python3` is an undeclared dependency; if absent, `python3 -c` fails and the script prints the misleading "No llama.cpp Ubuntu x64 binary release found" error. A `command -v python3` guard (or using the `curl`+`jq` already required by `pos ai server`) would give a clearer error. Same pattern as scrcpy.sh:15 | llamacpp.sh:30-44; no deps guard for python3 anywhere in file | +| N1 | NOTE | `apps/ai/llamacpp.sh:30-44` + `lib/common.sh:79-82` | Under `DRY_RUN=1` no mutation occurs (spawn no-ops), but the python3 matcher runs outside spawn and hard-errors ("No … release found") because the probe file was never fetched — dry-run does not print a would-be trace. Mirrors scrcpy's dry-run behavior; no defect in mutation-safety | common.sh:79-82 (spawn DRY_RUN branch), llamacpp.sh:30-44 | +| N2 | NOTE | `apps/ai/llamacpp.sh:63` | Uninstall guard keys on `llama-server` on PATH; a partial install with no symlink leaves `/usr/local/lib/llama.cpp-*` behind on uninstall. Reinstall is safe (line 49 removes the dir first) — edge case only | llamacpp.sh:63, 49 | +| N3 | NOTE | `apps/ai/llamacpp.sh:25-27` + `lib/common.sh:114` | First-spawn (release JSON fetch) failure exits via `spawn`'s `exit "$rc"` before the JSON cleanup runs (line 42/56); a mid-transfer abort can leave a partial `/tmp/llamacpp-releases.json`. No `trap` for the process. Minor | common.sh:114, llamacpp.sh:25-27 | + +## Verification verified + +- All structure/idempotence/dispatch claims for `apps/ai/llamacpp.sh` (Step 1 list) — read-only inspection. +- `apps/install.sh` CAT_NAMES + auto-discovery; `templates/app.sh` one-comment-line diff; `bin/pos-ai-server` exactly 3 text lines (256/397/563), no logic change, GitHub URL kept, `--apps`/`--full` accurate (root `install.sh:57-58,78-79`); `DOC/POS.md` one-line hint; AGENT_TODO entry correct and non-duplicated; scope = exactly the claimed file set; hygiene clean; uninstall symlink-scoped removal (no blanket `rm llama*`); temp cleanup on success + matcher-failure paths; lint apps-class rules satisfied (`lint-conventions.sh:192-204`). + +## Verification unverified + +- Live GitHub API probe (nightly `bNNNNN` asset naming, `.tar.gz` not `.zip`, top-level dir layout, `/releases/latest` having no binaries) — external network probe; not re-run within read-only boundary. Static construction (per_page=10 scan, endswith matcher, strip-components=1) is consistent with the Builder's claimed evidence. +- `make gen` idempotence, `make check`, `make lint` runs — not re-run (do-not-run-make instruction); Builder-reported outputs are consistent with the script rules I inspected (gen scans only `bin/pos-*` `# POS:` headers; none changed). +- Runtime behavior of DRY_RUN and failure paths — analyzed statically only. + +## Scope compliance + +- In-scope confirmed: new `apps/ai/llamacpp.sh`; `apps/install.sh` line; `templates/app.sh` line; `DOC/APPS.md` 3-line edit; `bin/pos-ai-server` 3-line text edit; `DOC/POS.md` 1-line edit; `AGENT_TODO.md` 1 entry; builder report. +- Out-of-scope found: none. No lib/config/core-installer/generated-block changes. + +## Remaining uncertainty + +- Whether `chmod +x` is applied before commit (R1) and whether the count is corrected to 18 (R2). +- Whether the failure-masking (S1) should be addressed in this change or deferred as a class-level fix (scrcpy shares it). Not a blocker; owner decision. + +## Recommended next agent + +**Builder** + +## Reason + +Two REQUIRED findings within the approved scope of this change: (1) `chmod +x apps/ai/llamacpp.sh` before `git add` (100755 uniformity with all tracked apps); (2) correct `DOC/APPS.md:3` count from 16 → 18 (and, if touched, the AGENT_TODO wording). Both are small, understood fixes — no design/architecture input needed. After fixing, re-run `bash -n`, `make check`, `make lint`. + +## Changes made by Reviewer + +none \ No newline at end of file diff --git a/AgentsReport/reviewer/2026-09-06_stab_acceptance.md b/AgentsReport/reviewer/2026-09-06_stab_acceptance.md new file mode 100644 index 0000000..7312bfa --- /dev/null +++ b/AgentsReport/reviewer/2026-09-06_stab_acceptance.md @@ -0,0 +1,176 @@ +# Stability Pass Acceptance Review — 2026-09-06 + +## TL;DR + +- **Status: CHANGES_REQUIRED** (reject with block list). +- **Reviewed:** uncommitted stabilization pass (`git status` 33 modified + untracked `AgentsReport/`, `apps/ai/`, `config/{telegram,matrix}.env`, `tests/`) against architect `AgentsReport/architect/2026-09-06_stabilization-design.md` (D-A…D-F), explorer audits (V1-V7, D1-D4, M1-M4, H-002/H-003, dup-loaders), and llamacpp-app REQUIRED items. +- **Defects: 2 BLOCKING, 1 REQUIRED, 1 SUGGESTED, 3 NOTE** (below). +- **BLOCKING 1:** neither chat bridge passes `--no-command-execution` to `pos ai` (D-B criteria 5/6 fail; docs falsely claim the bridges rely on it). +- **BLOCKING 2:** D-A contract not implemented — owner/room unset must `err` + exit (acceptance criteria 1/5 fail); implemented as run-degraded fail-closed; needs per-contract fix **or** explicit Architect amendment. +- **Gates:** static verification only — **all empirical gate runs (make gen×2, make check, make lint, make test, bash -n, probes, `git diff --check`) are UNVERIFIED** from this sandbox (bash execution denied); Orchestrator must execute before merge. +- **Recommended next agent: Builder** for F1+F3 (and F2 if the per-contract `err` path is chosen); **Architect** if F2's soft-fail model is to be ratified instead. + +--- + +## Step 1: Scope and diff inventory + +Checked `git status --short`, `git diff --stat` (33 files, +685/−403), per-file diff mapping, and untracked files. + +- All 17 modified `bin/pos-*` trace to an approved decision area (D-A listeners/sender + templates; D-B pos-ai + `# POS_FLAGS:`/usage/completions + M4 adapter headers; D-D nine migrated tools + media-grab; D-E uninstall; V3 backup; V5 checkport/download/smb-client). +- `bin/pos` **not** modified (dispatcher/INTERACTIVE_CMDS untouched — `pos-ai` already listed at `bin/pos:269`). +- `bin/pos-docker-*` not modified → docker-compose loader **not** migrated (consistent with the accepted "no compose change" boundary). +- No accidental deletions, no stray files. Untracked files all expected (report dirs, `apps/ai/llamacpp.sh`, env templates, `tests/`). + +[PASS] + +## Step 2: D-A — owner/room AND-gates + +- `TELEGRAM_OWNER_ID` present in sender `# POS_CONFIG:` registry (`bin/pos-communication-telegram-sender:6`) and `config/telegram.env`; `pos config telegram` scope wired (sender line 46; template). +- Actual auth logic: telegram checks both chat and owner (`bin/pos-communication-telegram-listener:761,790`); matrix checks owner + room (`bin/pos-communication-matrix-listener:511`). +- **FAIL:** architect D-A required *refuse to start* on unset (`err` in `run_daemon`, decision lines 31/38; acceptance criteria 1/5 at lines 59/63). Implementation instead starts the daemon, warns, and ignores commands. Fail-closed security property holds (no unauthorized execution), but the approved contract is not met — **Finding 2 (BLOCKING)**. + +[FAIL] + +## Step 3: D-B — AI eval posture (`--no-command-execution`) + +- Flag implemented in `bin/pos-ai`: `# POS_FLAGS:` line 5, usage lines 43/78/83, parser 666-669 (last-wins vs `--trust`), guard in `_prompt_run_command` 370-404; completions/pos.bash:26 regenerated. +- **FAIL:** D-B acceptance criteria 5/6 (decision lines 137-138) — bridge invocations must include the flag. `grep -c "no-command-execution"` = **0** in both `bin/pos-communication-telegram-listener` and `bin/pos-communication-matrix-listener`; both call `pos ai gemini ask` without it (telegram:731, matrix:466). Security still holds today only via the non-tty guard (`pos-ai:376`), which is exactly the defense-in-depth the decision required to be made explicit — **Finding 1 (BLOCKING)**. +- **FAIL (minor):** criterion 4 ("no output of the command block") — implementation prints the block + "command execution disabled" before returning (`pos-ai:377-382`) — **Finding 3 (REQUIRED)**. +- Docs (DOC/POS.md `Command execution posture`; DOC/howto/ai.md same section) describe the print behavior and claim the bridges rely on the flag — the claim is false against the working tree (folded into F1). + +[FAIL] + +## Step 4: D-C — test suite + +Read `tests/run-tests.sh`, `tests/test-lib.sh`, `tests/README.md`, and 4 of the 12 t-*.sh files. + +- Harness: strict-mode runner in per-test subshell; `test_run`/`test_run_env` capture rc without triggering errexit; SKIP counted separately; **aborted test without FAIL is itself FAIL**; zero-assertion file → FAIL ("no assertions run — harness broken"). +- Tests are genuine behavioral tests (real production scripts, stub PATH/curl/llama-server/nvidia-smi/gpg, exact rc/output asserts) — not tautologies. +- `t-ai-server-flags.sh`: dedupe (`--ctx`+`--ctx-size` → one token), precedence, default+requested emission — meaningful. +- `t-matrix-auth.sh`: owner+room gating, exactly-one-reply (reply-loop detection), fail-closed no-exec — meaningful. +- `t-uninstall-manifest.sh`: install.sh ↔ POS_LIBS set-equality + every lib exists; XDG_CONFIG_HOME scan_tier1 extraction + behavioral run — meaningful. +- Makefile `test:` target added. + +[PASS] statically. Execution results (counts/timing/red-green) **UNVERIFIED** — Orchestrator must run `make test`. + +## Step 5: D-D — config loader migration + +- `lib/config-ui.sh:336-357` `load_env_file`: env-wins export, CRLF strip, quote-pair trim, quiet on missing file, `LOADED_ENV_KEYS` append, bare-basename resolution under CONFIG_DIR; precedence CLI > env > file > defaults documented and implemented (file only when env var unset/empty). +- Exactly 9 tools migrated (pos-ai, pos-ai-hf, pos-ai-server, telegram sender+listener, matrix sender+listener, scrcpy, media-grab); `load_secret` via loader in network-download; entertainment-lib thin wrappers `config_value`/`write_config_key` → `cfg_value`/`cfg_write` (`lib/entertainment-lib.sh:35-43`, CONFIG_FILE set at line 6). +- Legacy `load_system_env` remains in common.sh:146 with exactly the 3 unchanged callers (pos-system-backup, pos-system-health, pos-media-sync) — matches the accepted note. +- `cfg_write`/`cfg_value` semantics verified (env-file source of truth, grep-v+append value-safe writes, chmod 600). + +[PASS] + +## Step 6: D-E — uninstall manifest + +- `POS_LIBS` matches `install.sh` phase-2 lib list byte-for-byte (12 libs; `bin/pos-system-uninstall`; also asserted by `t-uninstall-manifest.sh`). +- ScaleTail path `/usr/local/share/linux_post_install/scale-tail` matches `install.sh:237`; flags dir matches `lib/flags.sh`; user-unit scan_tier1 honors XDG_CONFIG_HOME + `pos-*` prefix only; plugin removal stays POS_PLUGIN-marker driven; tier structure documented in DOC/POS.md. + +[PASS] + +## Step 7: D-F — llama-server flag validation + +Read `bin/pos-ai-server` end-to-end (743 lines): + +- Parse loop records `REQUESTED_FLAGS` (canonical tokens), `requested_from_env_config` captures config/env-sourced flags pre-parse (lines ~56-68), merged + deduped at 476-483, hard error for unsupported requested flags (validate_requested_flags) vs warn+omit for unsupported defaults (validate_default_flags; DEFAULT_*_OK gates in exec_cmd 527-574). +- Behavior matrix matches decision: readable `--help` → word-boundary version-aware matching; unreadable → warn + accept-all; unsupported default → one warning + omitted, never in the unit; unsupported requested → hard error naming flag+version. +- `LLAMACPP_HOST` honored (HOST default 127.0.0.1, used by health probe + exec line); dry-run prints exact ExecStart; systemd_quote for binary+model tokens. +- Docs (DOC/POS.md ai-server row, usage) updated to describe requested-vs-default semantics. + +[PASS] + +## Step 8: Security sweep (V1-V7, D1-D4, M1-M4, NET_PROBE, gpg, /dev/tcp) + +- NET_PROBE: `NET_PROBE="${NET_PROBE:-timeout 3 bash -c 'exec 3<>/dev/tcp/\$1/\$2' _ 8.8.8.8 53}"` — host/port escaped in default (`bin/pos-network-download:31`); all other `/dev/tcp` uses positional-arg composition (network-checkport 135/159/168/170 incl. udp fix, share-smb-client:94, share-lib.sh:61). +- gpg: only `--passphrase-fd 3` + `3<<<"$PASS"` in backup (197/207) with failure cleanup; **no** `--passphrase ` anywhere in bin/lib/scripts/apps/install.sh. +- D1 (`failed_files` init 624, single-file failure 706, meta suppressed 733, rc 1 738-742), M1 (`# POS_SUBCMDS:` incl. new verbs), M2/M4 (pos-ai subcmds + adapters), D2 (llamacpp provider honors LLAMACPP_HOST) — all present. +- Secrets: runtime token masking by `pos config`; templates carry no real secrets; `config/telegram.env`/`matrix.env` tracked by convention (like ai.env), no secret material. + +[PASS] + +## Step 9: llamacpp app REQUIRED items + +- `apps/ai/llamacpp.sh` mode 100755 (`git diff --no-index --summary` → create mode 100755); DOC/APPS.md row present + count 18; `apps/install.sh` CAT_NAMES has `[ai]`; `templates/app.sh` categories comment += ai; pos-ai-server install-hint strings present in help + both err lines (start/status). + +[PASS] + +## Step 10: Gates + +- `make gen` ×2 byte-identical, `make check`, `make lint` (0 FAIL/0 WARN on current repo), `make test` results, `bash -n` of changed scripts, planted-violation lint negative, test-mutation failure probes, `systemd-analyze` skip-path, `git diff --check`: **cannot be executed from this sandbox (bash denied). All UNVERIFIED — Orchestrator must run before merge.** +- Static lint comparison (HEAD vs working `scripts/lint-conventions.sh`): 23 fail/warn messages, 21 byte-identical; **two WARN messages lost the `:num` location** (secret-literal-assignment; system-path write) → "byte-identical output" claim holds only vacuously on a clean repo (0 WARN) — **Finding 4 (SUGGESTED)**. +- Deterministic generators: completion reorder observed (`llamacpp` moved in `_pos_subcmds[ai]`) consistent with LC_ALL=C sort. + +[BLOCKED: empirical gate execution requires Orchestrator] + +--- + +## Findings + +### Finding 1 — BLOCKING — chat bridges missing `--no-command-execution`; docs falsely claim they use it + +- Severity: BLOCKING · Certainty: FACT +- Evidence: `grep -c "no-command-execution"` = 0 in `bin/pos-communication-telegram-listener` / `bin/pos-communication-matrix-listener`; bridge invocations `timeout 120 pos ai gemini ask --session … --system … "$prompt"` at telegram:731 and matrix:466 carry no flag. Architect decision `AgentsReport/architect/2026-09-06_stabilization-design.md:112-113,126-127` and acceptance criteria 5/6 (lines 137-138) require the flag in the actual command string. DOC/POS.md and DOC/howto/ai.md state "this is the structural guard the chat bridges rely on" — false against the implementation. +- Why it matters: the approved D-B deliverable is not implemented; the accepted defense-in-depth (explicit flag so a future refactor cannot introduce bridge execution) does not exist; documentation misrepresents the implementation. +- Remediation: add `--no-command-execution` to both `pos ai gemini ask` invocations (telegram:731, matrix:466) and re-run `make check`. + +### Finding 2 — BLOCKING — D-A unset-owner/room contract not implemented (err+exit) — soft-fail substituted without amendment + +- Severity: BLOCKING · Certainty: FACT +- Evidence: architect decision lines 31/38 require `err` in `run_daemon` and acceptance criteria 1/5 (lines 59/63) require `--run` to exit with an error naming the key. Implementation: daemon starts, `warn` at telegram-listener:761/790 and matrix-listener:511, commands ignored. Docs (DOC/POS.md, howto/communication.md, config/telegram.env, config/matrix.env) and `t-matrix-auth.sh` consistently describe the soft-fail model — the whole round silently implements a different accepted decision. Security property (fail-closed, no unauthorized execution) is preserved by both designs. +- Why it matters: an approved, testable acceptance criterion is not met; the implementer changed design without a decision; unset-owner upgrades now keep the daemon alive (operator may not notice commands are dead). +- Remediation (either): + 1. Implement per contract: `[ -n "${TELEGRAM_OWNER_ID:-}" ] || err …` in run_daemon next to the token/chat-id guards; same for MATRIX_ROOM_ID; update docs/tests, or + 2. Architect formally amends D-A to the soft-fail model (then this finding downgrades and F2's code is accepted as-is). + +### Finding 3 — REQUIRED — `--no-command-execution` prints the command block (D-B criterion 4 not met) + +- Severity: REQUIRED · Certainty: FACT +- Evidence: `bin/pos-ai:377-382` prints "Command detected:" + cmd + "command execution disabled" before returning under NO_EXEC; decision line 136 ("no prompt, no output of the command block"). Docs document the print behavior (no doc bug); the deviation is the accepted criterion. +- Why it matters: contract mismatch on output semantics; per criterion the flag should return without printing. Harmless informationally, but violates the letter of the accepted decision. +- Remediation: in `_prompt_run_command`, check NO_EXEC before the "Command detected:" print and return 0; align DOC/POS.md/howto text. + +### Finding 4 — SUGGESTED — lint rewrite dropped `:num` locations from two WARN messages + +- Severity: SUGGESTED · Certainty: FACT +- Evidence: working `scripts/lint-conventions.sh` vs HEAD — `warn_ "$f: secret-like literal assignment (…)"` and `warn_ "$f: writes to a system path (…)"` lost `:num`; all other 21 messages byte-identical; no FAIL rule changed. Claim "byte-identical output" holds only when the repo has 0 WARNs. +- Why it matters: the two least-actionable warnings become file-only; brief's "output is actionable (file:line)" requirement degrades for those classes. +- Remediation: restore `:num` in those two messages; update the rewrite's verification claim to "identical on clean repo". + +### Notes + +- **N1 (NOTE):** `t-matrix-auth.sh` run 1 has no explicit `check_rc` after `timeout 5` — a daemon-hang regression (TERM trap broken, rc 124) would not fail the test. Add `check_rc` for run 1 and the matrix-room-unset run. +- **N2 (NOTE):** AGENT_TODO.md new Done entry says "DOC/APPS.md 15→16" — stale; the llamacpp bump was 17→18 (current count 18). Also the stabilization pass itself has no Done entry yet (acceptable pre-commit; add it in the commit per AGENTS.md). +- **N3 (NOTE):** `git diff --check` (whitespace) unobtainable here — folded into Step 10 UNVERIFIED. + +--- + +## Verification verified (static, fact-level) + +- D-B flag fully wired in pos-ai (header/usage/parser/last-wins guard) and completions; absent in bridges (F1). +- D-A registry + templates present; auth logic checks both gates; unset behavior deviates (F2). +- D-D loader/migration complete incl. legacy caller boundary; entertainment wrappers correct. +- D-E manifest/flags/ScaleTail/scan_tier1/plugin-marker; D-F full validation matrix; V1-V7 panels; D1/D2/M1/M2/M4; NET_PROBE + /dev/tcp positional hygiene; gpg fd-only passphrase; llamacpp app REQUIRED items; test-suite strictness (skip contract, aborted=Fail, zero-assertion=Fail). +- Scope: 33 modified + untracked files all trace to approved areas; no out-of-scope change found; bin/pos and docker-compose untouched. + +## Verification unverified (needs Orchestrator execution) + +- `make gen` ×2 byte-identical; `make check`; `make lint` (0 FAIL / 0 WARN); `make test` (12 files, counts, elapsed); `bash -n` on changed scripts; planted-violation lint negative; test-mutation failure probes; `systemd-analyze verify` skip-path; live bridge/`pos ai` behavioral probes; `git diff --check`. + +## Scope compliance + +- In-scope confirmed: all D-A..D-F areas, explorer audits, llamacpp REQUIRED items. +- Out-of-scope found: none (F1/F2 are *missing* accepted scope, not additions). +- Deviations from approved decisions: F1 (criterion 5/6 unmet), F2 (criterion 1/5 unmet, design substituted), F3 (criterion 4 unmet). + +## Remaining uncertainty + +- All empirical gate and behavioral results (Step 10). Whether F2 resolves to code (err+exit) or Architect amendment. Whether F3's print is acceptable after amendment. + +## Recommended next agent + +**Builder** (primary) — F1+F3 are well-understood scoped fixes (add flag to two invocations; move the NO_EXEC check before the print); F2 fix per contract also Builder. If the Orchestrator prefers to keep the soft-fail design, route F2 to **Architect** to amend D-A explicitly (then F2 downgrades and the round can be accepted after gates). + +## Changes made by Reviewer + +none \ No newline at end of file diff --git a/AgentsReport/reviewer/2026-09-06_stab_final_review.md b/AgentsReport/reviewer/2026-09-06_stab_final_review.md new file mode 100644 index 0000000..f6c89f3 --- /dev/null +++ b/AgentsReport/reviewer/2026-09-06_stab_final_review.md @@ -0,0 +1,139 @@ +# Stabilization Pass — Final Re-Verification (F1–F4 + test hardening) — 2026-09-06 + +## TL;DR + +- **Status: APPROVE_WITH_NOTES** (all four findings F1–F4 resolved; test hardening present; no BLOCKING/REQUIRED findings remain). +- **Reviewed (read-only):** re-verification of prior `CHANGES_REQUIRED` findings F1/F2/F3/F4 plus the t-matrix-auth test-hardening, against the working tree + architect amendment (`AgentsReport/architect/2026-09-06_da-failmode-decision.md`, `2026-09-06_stabilization-design.md` D-A "AMENDED" blocks). +- **Findings resolved:** F1 (BLOCKING→OK), F2 (BLOCKING→OK via Architect amendment), F3 (REQUIRED→OK), F4 (SUGGESTED→OK). Test hardening (item 5) present. +- **New defects: 0.** New NOTEs: 2 residual-verification notes (N1 rc=2 set-e-in-trap dependency; N2 lint perf-rewrite equivalence). +- **Gates:** static verification only. Empirical `make lint` result, `make test` (incl. rc=2), and behavioral probes of the lint rewrite are **UNVERIFIED-BY-REVIEWER** (bash execution denied in this sandbox) — Orchestrator must run before merge, as in the prior round. + +--- + +## Step 1: F1 — `--no-command-execution` in real bridge invocations + +**Evidence (grep of actual command strings, not help text):** +- `bin/pos-communication-telegram-listener:731`: + `timeout 120 pos ai gemini ask --no-command-execution --session "$session" --system "$AI_SYSTEM" "$prompt"` +- `bin/pos-communication-matrix-listener:466`: identical string with the flag. + +**Flag-ordering valid — walked the dispatch/parse chain:** +- `pos ai gemini ask ...` → dispatcher longest-prefix routes `pos ai gemini` → `bin/pos-ai-gemini` (thin forwarder, `bin/pos-ai-gemini:7`: `exec pos ai --provider gemini "$@"`), so `pos-ai` receives `--provider gemini ask --no-command-execution --session … --system … "$prompt"`. +- `bin/pos-ai` parse loop (`bin/pos-ai:645-677`) handles flags at ANY position (case arms; `--no-command-execution` → `NO_EXEC=1; TRUST_MODE=0` at 666-667; non-flag tokens become `cmd`/`args`). So a flag after the `ask` subcommand is accepted. `NO_EXEC` is set globally before `_prompt_run_command` runs. +- Last-wins vs `--trust` unchanged; bridges never pass `--trust` (nothing in the two bridge strings references it). + +**Docs now match implementation:** +- `DOC/POS.md:103` and `DOC/howto/ai.md:270` state the flag is "the structural guard the chat bridges rely on" — now true against the working tree (both bridges pass it). No false-claim remains. + +[PASS] — F1 resolved. + +## Step 2: F2 — D-A soft-fail ratified by Architect amendment + +**Inputs:** `AgentsReport/architect/2026-09-06_da-failmode-decision.md` (DECIDED) + `2026-09-06_stabilization-design.md` D-A amended blocks. + +**(a) Amendment coherent/self-consistent:** `2026-09-06_stabilization-design.md:33` (Telegram) and `:40` (Matrix) are marked **"AMENDED at review (2026-09-06): soft-fail ratified as shipped"**, present-tense soft-fail rules. TL;DR (`:11`), acceptance criteria (`:59-63`), and risk note (`:70`) all describe soft-fail. The only remaining strict-`err`/"requires" wording sits inside the amendment's past-tense descriptive paragraphs ("The original wording required…") — framed as what changed, not as the operative contract. No contradiction remains. + +**(b) No other design decision silently depends on old strict D-A:** the decision report (`da-failmode-decision.md`, Decision 2 + Files-changed) states no doc/template/test change was needed because the entire shipped surface already encodes soft-fail: `tests/t-telegram-auth.sh:5,76-78`, `tests/t-matrix-auth.sh:78-87`, `config/*.env`, `DOC/POS.md`, `DOC/howto/communication.md`. Spot-read confirms those describe soft-fail, not fail-stop. No dependent decision references the strict wording. + +**(c) Security property (fail-closed either way) — spot-check confirmed guards don't route around it:** +- Telegram: unset owner → `bin/pos-communication-telegram-listener:789-791` warns + `continue` **before** `handle_message` (line 800). AND-gate at `:796` requires `chat == TELEGRAM_CHAT_ID` AND `from_id == TELEGRAM_OWNER_ID`. +- Matrix: `bin/pos-communication-matrix-listener:510-512` warns on unset room; `:538` skips any room not matching `room_only` (empty when unset → every room skipped → fail-closed); sender gate `:549` (`sender == owner`). Dispatch only reached after both pass. +- Unauthorized/unset → command never executed; no hint reply. Fail-closed invariant holds in both models. + +[PASS] — F2 resolved by amendment (no code change required). + +## Step 3: F3 — NO_EXEC returns before any print + +**Evidence (`bin/pos-ai:370-402`):** +- `:376` tty guard (`[ -w /dev/tty ] || return 0`). +- `:377-379` `if [ "${NO_EXEC:-0}" -eq 1 ]; then return 0; fi` — placed immediately after the guard and **before** the `printf` at `:380-381`. +- So under `NO_EXEC=1` the function returns 0 with zero stdout/stderr (no "Command detected:", no cmd echo). +- Normal path (tty, NO_EXEC=0): `:380-381` prints the command; `y|Y` executes, `*` declines→history. Unchanged. +- Exit semantics: NO_EXEC path returns 0; matches prior behavior (rc 0). Builder probe (PROBE-A/B) claims 0 bytes both streams under NO_EXEC, prints on normal path. + +Docs (`DOC/POS.md:103`, `DOC/howto/ai.md:261-273`) updated to "neither printed nor run" — matches. No stale "still printed" text. + +[PASS] statically. (Empirical 0-byte probe is Orchestrator/UNVERIFIED.) + +## Step 4: F4 — lint WARN messages carry `:num` again + +**Evidence (`git diff HEAD -- scripts/lint-conventions.sh`):** +- Secret-literal: `warn_ "$f:$num: secret-like literal assignment (…)"` — working `:247`, and HEAD `:239` => **byte-identical format**. +- System-path: `warn_ "$f:$num: writes to a system path (…)"` — working `:297`, HEAD `:254` => **byte-identical**. +- `num` counter incremented first in-loop (before `continue` gates), so skipped lines still get correct numbers. + +**Rule list/count parity vs HEAD (no other lint behavior changed):** +- FAIL rules: 14/14 identical (shebang, set-euo, executable, POS-header, em-dash, -h|--help, deps-guard ordering, stdin-INTERACTIVE, INTERACTIVE_CMDS round-trip, plugin-no-common.sh, POS_PLUGIN, uninstall func, uninstall case, legacy-forward). +- WARN rules: 9/9 identical (POS-header line, local top-level, not-in-POS.md, TimeoutStopSec, WantedBy, legacy-lines, legacy-case, secret-literal, system-path). All message strings match HEAD byte-for-byte. + +[PASS] for rule/message parity and the two `:num` fixes. + +**NOTE (N2):** this file is a substantial *behavior-equivalence* rewrite (subprocess grep/sed → pure-bash single-pass: `_syspath_outer`, `_reads_stdin` + caller heredoc/depth state, POS.md preload). Rule parity is statically confirmed, but behavioral equivalence of the rewritten scan internals on edge cases needs empirical `make lint` (clean repo → 0 FAIL/0 WARN) plus the planted-violation negative — Orchestrator, UNVERIFIED-BY-REVIEWER. + +## Step 5: Test hardening (t-matrix-auth) + no-regression spot-read + +**Evidence (`tests/t-matrix-auth.sh`):** +- Run 1 (`:71-72`): `test_run_env … -- timeout --preserve-status -k 2 5 "$listener" --run` + `check_rc "daemon terminated via TERM trap, not killed (no hang)" 2 "$TR_RC"`. +- Run 2 / room-unset (`:82-83`): same `--preserve-status -k 2` pattern + `check_rc … 2`. +- `check_rc` helper exists unchanged (`tests/test-lib.sh:19`, delegating to `check_eq`; `test_run_env` captures `TR_RC` at `test-lib.sh:88-91`). +- rc=2 rationale: listeners run `set -euo pipefail` (`matrix-listener:2`, `telegram-listener:2`) with trap `kill $(jobs -p) 2>/dev/null; exit 0` (`matrix:525`, `telegram:767`). On empty `jobs`, bash `kill` hits a usage error (rc 2) and, under `set -e`, aborts before `exit 0` → rc 2. `--preserve-status` surfaces it; on a true daemon-hang the SIGKILL after `-k 2` yields 137 → the rc-2 assert genuinely fails → the mutation-probe hang claim is credible. + +**No-regression spot-read:** `tests/t-telegram-auth.sh` unchanged semantics, aligned with ratified soft-fail (`:5` "unset → fail-closed", `:76-78` assert warning + no marker + no sendMessage; no fail-stop assertion). No test asserts the old strict-D-A behavior. + +[PASS] statically. **N1 (NOTE):** the exact rc value 2 depends on bash `set -e` firing inside the TERM trap on the empty `kill`. If a given bash does not errexit-abort in a trap, the trap reaches `exit 0` → rc 0, which would make `check_rc … 2` **false-FAIL** a healthy run (the reverse failure direction). The 137-vs-2 hang discrimination is sound by design; the precise value must be confirmed by `make test` (Orchestrator). UNVERIFIED-BY-REVIEWER. + +--- + +## Findings + +### Resolved (prior round) + +**F1 (was BLOCKING) — RESOLVED.** `--no-command-execution` present in the actual `pos ai gemini ask` command strings (telegram:731, matrix:466); ordering valid through `pos-ai-gemini` → `pos-ai` parse loop (flag accepted post-subcommand); docs now truthful. Certainty: FACT. + +**F2 (was BLOCKING) — RESOLVED via Architect amendment.** D-A soft-fail ratified (decision report + amended design `:33/:40`); coherent and self-consistent; no dependent decision on old strict wording; fail-closed security verified (telegram AND-gate + unset-skip, matrix room-gate + sender gate; no route around). Certainty: FACT. + +**F3 (was REQUIRED) — RESOLVED.** `_prompt_run_command` returns 0 before any print under `NO_EXEC` (pos-ai:377-379); zero output; normal path unchanged; rc 0 both ways. Certainty: FACT (empirical 0-byte probe UNVERIFIED). + +**F4 (was SUGGESTED) — RESOLVED.** Both lint WARN messages carry `$f:$num:`, byte-identical to HEAD; 14 FAIL + 9 WARN rules and all message strings match HEAD. Certainty: FACT (rewrite behavior-equivalence UNVERIFIED). + +### New (this round) + +No new BLOCKING / REQUIRED / SUGGESTED findings. + +- **N1 (NOTE):** t-matrix-auth rc-2 assert depends on `set -e` aborting inside the TERM trap on empty `kill` (bash-version-sensitive). If errexit does not fire in-trap, healthy run yields rc 0 → check_rc(2) false-FAILs. Design intent (137-hang discrimination) is sound; the exact value is UNVERIFIED → Orchestrator `make test`. +- **N2 (NOTE):** `scripts/lint-conventions.sh` perf-rewrite (subprocess→pure-bash) is rule/message-identical per static diff, but behavior equivalence on edge cases is UNVERIFIED → Orchestrator `make lint` on clean repo + planted-violation negative. + +--- + +## Verification verified (static, fact-level) + +- F1 flag in both real bridge command strings (grep); dispatch+parse chain accepts it; docs accurate. +- F2 amendment coherent + no strict leftover + fail-closed guards verified (read of both listeners). +- F3 NO_EXEC early-return before print; rc semantics; docs aligned. +- F4 `$f:$num:` restored byte-identical to HEAD; rule/count parity (14 FAIL, 9 WARN) vs HEAD. +- t-matrix-auth both runs `timeout --preserve-status -k 2` + `check_rc 2`; check_rc unchanged (test-lib.sh:19); t-telegram-auth aligned with soft-fail. + +## Verification unverified (needs Orchestrator execution) + +- `make gen` ×2 byte-identical; `make check`; `make lint` (0 FAIL/0 WARN) incl. planted-violation negative; `make test` (all files, including the rc=2 asserts in t-matrix-auth and the no-hang 137 discrimination); `bash -n`; NO_EXEC 0-byte probe; `git diff --check`. +- Empirical rc=2 (set-e-in-trap) — N1. +- Lint rewrite behavior-equivalence on edge cases — N2. + +## Scope compliance + +- In-scope: F1, F2 (amendment), F3, F4, test hardening all trace to approved scope/decisions. +- Out-of-scope found: none. Builder F1/F3/F4 report confirms it touched only those files + literal-contradiction doc lines; D-A/decision-pending code regions untouched. + +## Remaining uncertainty + +- All empirical gate/test results (Step 5). N1 rc=2 value; N2 lint-equivalence. Whether `make test` passes with the new rc=2 asserts. + +## Recommended next agent + +**Orchestrator** — all findings statically resolved; this review is APPROVE_WITH_NOTES, and the standing empirical gates (`make test` incl. rc=2, `make lint` incl. negative, `make gen`/`check`) remain for the Orchestrator to execute before merge. If any gate regresses, hand that specific failure back to Builder (or Tester for a coverage fix). + +**Reason:** Reviewer is read-only and bash-denied; verdict is final and gated only on Orchestrator's empirical runs, none of which is a known defect. + +## Changes made by Reviewer + +none (read-only; report written only). diff --git a/AgentsReport/tester/2026-09-06_matrix-auth-rcfix.md b/AgentsReport/tester/2026-09-06_matrix-auth-rcfix.md new file mode 100644 index 0000000..94713ac --- /dev/null +++ b/AgentsReport/tester/2026-09-06_matrix-auth-rcfix.md @@ -0,0 +1,136 @@ +# Tester Report — Matrix-auth daemon-hang rc assertion (Reviewer N1) — 2026-09-06 + +## TL;DR + +- **Status: TESTS_READY** — regression-suite hardening for Reviewer `AgentsReport/reviewer/2026-09-06_stab_acceptance.md` Note N1: `t-matrix-auth.sh` run 1 lacked an exit-code assertion, so a daemon-hang regression could pass vacuously. +- **Defects added:** 2 new `check_rc` assertions (run 1 authorized + run 2 room-unset). File `t-matrix-auth.sh`: 8 → **10 checks**. +- **Suite:** `make test` → **12 files pass / 0 fail / 0 skip**, **179 checks pass**, runtime **45s**, exit 0. No flakes observed. +- **Production code: untouched.** Only `tests/t-matrix-auth.sh` modified; `tests/test-lib.sh` unchanged (the `check_rc` helper already exists). +- **Key correction to the brief:** asserting `TR_RC` under plain `timeout` is vacuous — GNU `timeout` reports **124 in BOTH** the healthy and broken-trap cases. The discriminating fix is `timeout --preserve-status -k 2 …` + assert the daemon's own TERM-trap exit status. + +--- + +## Step 1: Confirm the gap (Reviewer N1) + +Reviewed `tests/t-matrix-auth.sh` run 1: `test_run_env … -- timeout 5 "$listener" --run` with `check_contains`/`check_file_exists`/`check_eq` but **no rc assertion**. Confirmed the reviewer's concern: a broken TERM trap (daemon ignores TERM) would not fail any existing check. + +Also confirmed the deeper problem with the naive fix: plain `timeout 5` returns **124 whether or not the TERM trap works** (verified empirically — both the working daemon and a no-trap daemon yield 124). So `check_rc … 124` would be itself vacuous. + +Evidence (throwaway probes, `/tmp/opencode`, removed after): +``` +timeout 1 bash -c 'trap "exit 0" TERM; sleep 30' → rc=124 (trap WORKS) +timeout 1 bash -c 'sleep 30' → rc=124 (no trap) +timeout --preserve-status 1 … (trap WORKS) → rc=0 +timeout --preserve-status 1 … (no trap) → rc=143 (SIGTERM) +``` + +[PASS] + +## Step 2: Design the discriminating assertion + +Goal per the brief: *"so the test genuinely fails on a daemon-hang regression"*. The daemon is an infinite polling loop that only terminates via its `TERM` trap (`trap 'kill $(jobs -p) 2>/dev/null; exit 0' TERM INT`), so `timeout` must always be the one signalling it. To make the daemon's own exit status observable: + +- **`--preserve-status`** — `timeout` forwards the child's real exit status instead of forcing 124. +- **`-k 2` (`--kill-after`)** — bounds the wait: if a broken trap ignores TERM, `timeout` SIGKILLs at +2s so the test cannot hang the whole suite indefinitely. + +**Empirical healthy-path exit status (current production code):** the listener's TERM trap is `kill $(jobs -p) 2>/dev/null; exit 0`. Under `set -euo pipefail`, with no background jobs `kill` (no args) fails with **rc 2**, which triggers errexit **before** the `exit 0` and aborts the trap → the daemon actually exits with **rc 2** (verified on the real listener and a minimal `set -euo pipefail` repro: `rc=2`, the post-kill `exit 0` never runs). + +- A genuine **daemon-hang** (TERM ignored / trap non-exiting): `--kill-after` SIGKILLs → **rc 137**. +- A healthy daemon: TERM trap fires → **rc 2**. + +So `check_rc … 2` meaningfully discriminates: healthy = daemon's own trap exit (2, definitively not a forced kill), regression = 137/124. + +**Note (observation for Orchestrator, NOT fixed — out of scope):** the `kill $(jobs -p)` in the daemon's TERM trap fails under errexit, so the daemon exits 2 rather than the intended 0. Harmless to the daemon's operation (it still terminates, no hang) but the `exit 0` is effectively dead. Flagged for a possible future Builder fix; intentionally not addressed here (production code out of scope). The rc assert documents current healthy behavior and still catches a hang. + +[PASS] + +## Step 3: Change — `tests/t-matrix-auth.sh` + +- Run 1 (authorized): added `--preserve-status -k 2` and `check_rc "daemon terminated via TERM trap, not killed (no hang)" 2 "$TR_RC"`. +- Run 2 (room-unset): same `--preserve-status -k 2` + `check_rc … 2`. +- Comment explains why `--preserve-status -k 2` + rc 2 catches the regression and why plain `timeout` would be vacuous. + +Snippet (run 1): +```bash + # --preserve-status + --kill-after surface the daemon's own TERM-trap exit, + # so a broken trap (daemon-hang regression → SIGKILL 137 / timeout 124) + # genuinely fails the rc assert instead of passing vacuously. --kill-after + # also bounds the wait so a hung daemon can't stall the whole suite. + test_run_env "${common[@]}" -- timeout --preserve-status -k 2 5 "$listener" --run + check_rc "daemon terminated via TERM trap, not killed (no hang)" 2 "$TR_RC" +``` + +Run 2 mirrors it with the room-unset env (no `MATRIX_ROOM_ID`) and desc `"room-unset daemon terminated via TERM trap, not killed (no hang)"`. + +`tests/test-lib.sh` was not modified — `check_rc` already exists (`test-lib.sh:19`). + +[PASS] + +## Step 4: Mutation probe — the new assert genuinely fails on a hang + +Copied the listener into a scratch tree and replaced the TERM trap with a non-exiting handler (`trap 'hang…sleep 30' TERM INT`) to simulate the daemon-hang regression, then ran it under the same `timeout --preserve-status -k 2` invocation: + +``` +… timeout --preserve-status -k 2 3 "$listener" --run +ignoring TERM (regression), sleeping +rc=137 +``` + +Expected `2`, actual `137` → the `check_rc 2` assert **rejects** the regression and the test fails. Healthy path verified → `rc=2` → assert passes. Scratch cleanup performed; no production file touched. + +[PASS] + +## Step 5: Full suite — `make test` + +``` +Running 12 test file(s) — strict mode: no network, no sudo, no system changes. + PASS t-ai-hf-download.sh (10 checks) + PASS t-ai-llama-detect.sh ( 9 checks) + PASS t-ai-server-flags.sh (28 checks) + PASS t-config-precedence.sh (43 checks) + PASS t-gen-docs-drift.sh ( 4 checks) + PASS t-gpg-password.sh (14 checks) + PASS t-lint-gate.sh ( 5 checks) + PASS t-matrix-auth.sh (10 checks) + PASS t-systemd-unit.sh (11 checks) + PASS t-telegram-auth.sh ( 8 checks) + PASS t-uninstall-manifest.sh(18 checks) + PASS t-unsupported-flags.sh (19 checks) +────────────────────────────────────────────── +Summary: files 12 pass / 0 fail / 0 skip (of 12) +Checks : 179 pass / 0 fail / 0 skip +Runtime: 45s +``` + +Exit 0. `t-matrix-auth.sh` independently re-run at 10 checks (10s). No flake source observed; isolated result is representative (no mid-edit contention observed on the suite path — `bin/pos`/`bin/pos-*` are only read by the tests, never written here). + +[PASS] + +--- + +## Verification completed + +- Baseline `t-matrix-auth.sh` (8 checks, 10s) before edit — green. +- Post-edit standalone run (10 checks, 10s) — green. +- Mutation probe proves the new assert fails (rc 137) on a broken TERM trap / daemon hang. +- Full `make test` green: 179/179 checks, 45s. +- `test-lib.sh` untouched; no production code touched; no commits made. + +## Coverage note + +The new asserts close the reviewer's vacuous-pass gap: a daemon-hang (TERM-trap) regression now produces rc 137/124 and fails the test rather than passing silently. Both authz runs (authorized + fail-closed) carry the guard. + +## Remaining uncertainty / out of scope + +- The daemon's dormant `exit 0` in its TERM trap (exits 2 under errexit) — see Step 2 observation. Functional no-hang is preserved; left for Builder/Architect if they want the trap to truly exit 0. +- Not committed (per brief). + +## Recommended next agent + +**Reviewer** — the regression-suite gap is closed and green; suitable for independent verification of the rc assert. + +**Reason:** Tester completes measurement/verification; the change is in test-only scope, ready for reviewer sign-off. + +## Changes made by Tester + +- `tests/t-matrix-auth.sh` — added `--preserve-status -k 2` to both `timeout` invocations and two `check_rc … 2` (no-hang) assertions; explanatory comments. No other files touched, no commits. diff --git a/AgentsReport/tester/2026-09-06_stab-tests.md b/AgentsReport/tester/2026-09-06_stab-tests.md new file mode 100644 index 0000000..03972ad --- /dev/null +++ b/AgentsReport/tester/2026-09-06_stab-tests.md @@ -0,0 +1,93 @@ +# Tester Report — Regression Test Infrastructure + First Suite (2026-09-06) + +## TL;DR (updated continuously) + +- **Status:** TESTS_READY — 12/12 test files pass, 177 checks, runtime ~46s (`make test`). +- **Deliverables:** `tests/run-tests.sh` (zero-dep runner), `make test` target, 12 `tests/t-*.sh` files, `tests/README.md`. +- **Findings (production bugs discovered):** none — no production bug surfaced; all defects found during test iteration were in the test framework/stubs/test assertions themselves (see Step 4). +- **Suite timing / counts:** `make test` (2026-09-06): files **12 pass / 0 fail / 0 skip**, checks **177 pass / 0 fail / 0 skip**, runtime **46s** (44s on rerun); exit 0. +- **Gates:** `make lint` still `0 FAIL, 0 WARN`; `make check` now PASS (parallel-track gen drift resolved upstream during this session); tests/ has zero lint/check surface. + +--- + +## Step 1: Environment baseline (before adding tests) + +- `make check` at start: **FAILED** — `doc/code drift` (expected: parallel Builder tracks have uncommitted changes; gen output in the working tree not yet refreshed). Now resolves to PASS after upstream sync. +- `make lint` at start: **PASS** — `0 FAIL, 0 WARN` (3.5s). +- `make gen` idempotence on a pristine temp copy: **PASS** (2× ~1.4–1.8s; `git status --porcelain` empty after 2nd gen). +- `systemd-analyze verify` prototype: passes (rc 0) when ExecStart binary exists and model path is quoted. +- Config-loader migration (D-D): **landed in the working tree** — `load_env_file` present in `lib/config-ui.sh:336`; all 9 tools call it. Config-precedence tests target the final contract. + +## Step 2: Framework + suite files (status below) + +- [x] `tests/run-tests.sh` +- [x] `tests/test-lib.sh` +- [x] `tests/t-ai-server-flags.sh` +- [x] `tests/t-ai-hf-download.sh` +- [x] `tests/t-ai-llama-detect.sh` +- [x] `tests/t-unsupported-flags.sh` +- [x] `tests/t-systemd-unit.sh` +- [x] `tests/t-telegram-auth.sh` +- [x] `tests/t-matrix-auth.sh` +- [x] `tests/t-gpg-password.sh` +- [x] `tests/t-config-precedence.sh` +- [x] `tests/t-uninstall-manifest.sh` +- [x] `tests/t-gen-docs-drift.sh` +- [x] `tests/t-lint-gate.sh` +- [x] `Makefile` `test:` target +- [x] `tests/README.md` + +## Step 3: Full suite run (final) + +Command: `make test` (target: `./tests/run-tests.sh`) — 2026-09-06. + +``` +Running 12 test file(s) — strict mode: no network, no sudo, no system changes. + + PASS t-ai-hf-download.sh (10 checks) + PASS t-ai-llama-detect.sh (9 checks) + PASS t-ai-server-flags.sh (28 checks) + PASS t-config-precedence.sh (43 checks) + PASS t-gen-docs-drift.sh (4 checks) + PASS t-gpg-password.sh (14 checks) + PASS t-lint-gate.sh (5 checks) + PASS t-matrix-auth.sh (8 checks) + PASS t-systemd-unit.sh (11 checks) + PASS t-telegram-auth.sh (8 checks) + PASS t-uninstall-manifest.sh (18 checks) + PASS t-unsupported-flags.sh (19 checks) + +────────────────────────────────────────────── +Summary: files 12 pass / 0 fail / 0 skip (of 12) +Checks : 177 pass / 0 fail / 0 skip +Runtime: 46s +``` + +Exit code 0. Rerun via `make test`: files 12/12 pass, 44s. `make lint` unaffected (`0 FAIL, 0 WARN`), `make check` passes (parallel drift resolved upstream, not by this track). + +[PASS] + +## Step 4: Defects found and fixed during test iteration (all in test artifacts, none in production) + +1. `tests/test-lib.sh` `check_rc` — `$desc` read before `local desc="$1"` declaration → `set -u` crash on first use. Fixed. +2. `tests/run-tests.sh` — `set -e` in the runner killed the PARENT when a test subshell exited nonzero (e.g. test 4 aborted after 3 passing tests). Fixed: subshell wrapped in `if (…); then rc=0; else rc=$?; fi`; verified a failing test now records FAIL and continues. Also: bare-name args (`run-tests.sh t-gpg-password`) now resolve `$TEST_DIR/.sh`. +3. `tests/t-ai-hf-download.sh` stub — embedded JSON via `$(cat "$tree_resp")` broke stub quoting → replaced with `cat "$TREE_RESP"` env passthrough; `for (( ; i<=$#; i++ ))` expanded `$#` at stub-write time → escaped `\$#`; `base_env` typo → `env_base`; `return 1` at stub top level → `exit 1` (see #6). +4. `tests/t-ai-llama-detect.sh` — asserted literal `cpu`; tool emits `gpu: CPU` (case differs) → assertions corrected to actual token shape. +5. `tests/t-config-precedence.sh` — Part A env-wins probe `FOO=envval load_env_file …` evaluated in the PARENT shell (no persistence) → rewrote as explicit subprocess with `export` + captured output; Part B needed llama-server + nvidia-smi stubs for the deps guard; Part D legacy guard was a false positive — exactly 3 documented `load_system_env` callers (pos-media-sync, pos-system-backup, pos-system-health) → whitelist those and assert count == 3. +6. **Stub scripts: `return` at top level of a non-sourced script is an ERROR in bash and falls through** (`return: can only 'return' from a function or sourced script`), so every stub response silently gained a trailing `{"ok":true}` → corrupt JSON → listeners slept in a 5s retry loop and never processed (`jq -r '.ok'` returned `true\ntrue`). Fixed all stub heredocs to `exit 0` (telegram/matrix curl stubs; ai-hf already used `exit`). +7. `tests/t-telegram-auth.sh` / `tests/t-matrix-auth.sh` — two line-continuation bugs in `test_run_env` invocations: a missing trailing `\` meant the env-var list became a separate command and `test_run_env` ran bare `env` (prints the whole environment — the mysterious `SHELL=/bin/bash` output) with rc 0. Fixed by single-line invocation. Matrix reply count needle `m.room.message` also matched the URL-encoded sync filter on every `/sync` line → narrowed to `/send/m.room.message`. +8. `tests/t-gpg-password.sh` — artifact-leftover checks false-failed because run 1's `.gpg` remained on disk for runs 2/3 → now `rm -rf "$work"; mkdir` between runs; bare `--passphrase` guard now token-exact (`grep -c '^--passphrase$'`) since `--passphrase-fd` legitimately contains the substring. +9. `tests/t-lint-gate.sh` — negative case invoked the REAL lint (absolute path); `lint-conventions.sh` computes `ROOT="$(dirname "$0")/.."` and `cd`s THERE, so it linted the real repo (clean), not the planted copy. Fixed: run the copy's own `scripts/lint-conventions.sh` (relative path) from inside the copy. +10. `tests/t-uninstall-manifest.sh` — POS_LIBS extraction awk `<^POS_LIBS=( … {getline; while(1)…}` never matched a lone `^)` line because the block is `POS_LIBS=(… \⏎ …registry.sh)` (two lines, `)` on the second) → getline at EOF returns 0, loop spins forever at EOF → the whole test hung (this was the full-suite 300s hang). Replaced with a sed range `/^POS_LIBS=(/,/)$/p` + normalization; also the leftover-gap whitespace made the sorted diff fail (collapsed with `tr -s`), and plugin-removal marker check now greps `POS_PLUGIN` (the marker `installed_plugins()` scans for) instead of a literal `^# POS_PLUGIN:` in the uninstall script. +11. `tests/t-systemd-unit.sh` — systemd unit uses double quotes (not backslash escaping) for the model path → assertion corrected; `EnvironmentFile` check compared against the unit PATH instead of its content → `$(cat "$unit")`. +12. `tests/t-unsupported-flags.sh` — real error text is `installed llama.cpp does not expose — remove it or upgrade llama.cpp`, not "does not support" → assertions updated. + +None of the above touched production code. `make check` / `make lint` / `make gen` results are unchanged by this track (verify with `make check && make lint` — both currently green). + +## Step 5: Coverage notes & handoff + +- **Behavior covered per area:** ai-server flag seam (CLI/config/env/default precedence + unsupported-flag hard error + dedupe) 28; config file precedence + legacy loaders 43; gpg password hygiene (fd-only, no bare token, no secret in argv, artifact cleanup on enc/verify failure) 14; systemd unit generation (ExecStart quoting, environment/deps/secrets lines, `systemd-analyze verify`) 11; telegram/matrix authz fail-closed gates 8+8; ai hf download stub network behavior 10; llama detection stub 9; gen/lint gates (positive + planted-violation negative) 4+5; uninstall manifest symmetry + XDG scan tier + POS_PLUGIN marker 18; unsupported-flag matrix 19. +- **What is not covered (deliberately):** real network/sudo/docker paths (stubbed only); `pos entertainment send` live-plugin e2e (requires Telegram token); anything requiring root. These are outside the sandbox contract of this suite and remain manual checks. +- **Suite hygiene:** deterministic sorted order, per-test sandbox auto-clean, per-file logs, no network/sudo/system mutations, skip contract, total < 90s. + +[PASS] \ No newline at end of file diff --git a/DOC/AGENT_Context_Project.md b/DOC/AGENT_Context_Project.md index 3cf55e1..512454d 100644 --- a/DOC/AGENT_Context_Project.md +++ b/DOC/AGENT_Context_Project.md @@ -598,7 +598,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:` | `lib/common.sh` | 151 | Shared library (log/warn/err/run/spawn, dry-run aware, `load_system_env`, CONFIG_DIR) | | `lib/flags.sh` | 60 | Feature flag store (set/clear/is_set/value/list/status) | | `lib/notify.sh` | 87 | Multi-platform alerting (`notify_send`) — opt-in source, silent-fails | -| `lib/entertainment-lib.sh` | 311 | Entertainment module lib (ENABLED parsing, last-run state, scheduler sync via user-timers-lib) | +| `lib/entertainment-lib.sh` | 300 | Entertainment module lib (ENABLED parsing, last-run state, scheduler sync via user-timers-lib; config helpers wrap lib/config-ui.sh) | | `lib/entertainment-plugin-lib.sh` | 67 | Message-safe helpers for plugins (config load, require, fetch+retry) — plugins MAY source it | | `lib/scheduler-lib.sh` | 760 | Scheduler lib (job parsing, notify policies, per-job user timers via user-timers-lib, legacy migrate) | | `lib/user-timers-lib.sh` | 112 | Shared systemd **user** timer machinery (interval→OnCalendar, unit pair writer, linger) | @@ -615,15 +615,15 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:` | `bin/pos` | 302 | CLI dispatcher with smart arg matching + logging + category help | | `bin/pos-ai-alias` | 760 | manage AI agent aliases | | `bin/pos-ai-gemini` | 7 | Forward to pos ai --provider gemini (backward compat) | -| `bin/pos-ai-hf` | 1004 | Download AI models from Hugging Face (search, download, manage) | +| `bin/pos-ai-hf` | 993 | Download AI models from Hugging Face (search, download, manage) | | `bin/pos-ai-llamacpp` | 7 | Forward to pos ai --provider llamacpp (backward compat) | | `bin/pos-ai-openrouter` | 7 | Forward to pos ai --provider openrouter (backward compat) | -| `bin/pos-ai-server` | 659 | llama.cpp local inference server (start, stop, status, models, logs) | -| `bin/pos-communication-matrix-listener` | 568 | Matrix listener: map /command → bash, run them on room messages | -| `bin/pos-communication-matrix-sender` | 224 | Send messages to a Matrix room via the client-server API (send, test, login) | -| `bin/pos-communication-scrcpy` | 254 | Mirror/control an Android device via scrcpy+adb (mirror, devices, record, tcpip, connect, push, pull, screenshot, info) | -| `bin/pos-communication-telegram-listener` | 805 | Telegram bot listener: map /command → bash and → app, run them on chat messages | -| `bin/pos-communication-telegram-sender` | 221 | Send Telegram messages/files/links/stickers via Bot API (send, test) | +| `bin/pos-ai-server` | 743 | llama.cpp local inference server (start, stop, status, models, logs) | +| `bin/pos-communication-matrix-listener` | 567 | Matrix listener: map /command → bash, run them on room messages | +| `bin/pos-communication-matrix-sender` | 215 | Send messages to a Matrix room via the client-server API (send, test, login) | +| `bin/pos-communication-scrcpy` | 245 | Mirror/control an Android device via scrcpy+adb (mirror, devices, record, tcpip, connect, push, pull, screenshot, info) | +| `bin/pos-communication-telegram-listener` | 815 | Telegram bot listener: map /command → bash and → app, run them on chat messages | +| `bin/pos-communication-telegram-sender` | 212 | Send Telegram messages/files/links/stickers via Bot API (send, test) | | `bin/pos-docker-compose` | 487 | Docker Compose service manager (ls/up/down/restart/logs/update/config) | | `bin/pos-docker-health` | 107 | One-glance container health dashboard (exits 1 if unhealthy) | | `bin/pos-docker-ps` | 126 | Enhanced container overview (health, IPs, ports, uptime) | @@ -634,31 +634,31 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:` | `bin/pos-entertainment-enable` | 49 | Enable an auto-trigger for a plugin on a schedule | | `bin/pos-entertainment-send` | 95 | Run a public-API plugin and send its output via the configured notify platforms | | `bin/pos-entertainment-status` | 62 | Show enabled plugins and scheduler state | -| `bin/pos-media-grab` | 227 | Auto-download URL as audio or video (classify + route) | +| `bin/pos-media-grab` | 219 | Auto-download URL as audio or video (classify + route) | | `bin/pos-media-mp3` | 86 | Download audio as MP3 (yt-dlp) | | `bin/pos-media-mp4` | 132 | Download video as MP4 (smart/interactive format select) | | `bin/pos-media-sync` | 219 | Incremental Music → USB sync (mp3/mp4, add/update only) | | `bin/pos-media-ytsync` | 1213 | Incrementally sync YouTube channels/playlists into ~/Videos | -| `bin/pos-network-checkport` | 496 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view | -| `bin/pos-network-download` | 1108 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) | +| `bin/pos-network-checkport` | 498 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view | +| `bin/pos-network-download` | 1110 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) | | `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui | | `bin/pos-network-ip` | 69 | Show interfaces, routes, public IP + location | | `bin/pos-network-scan` | 272 | Parallel ping sweep of CIDR | | `bin/pos-share-nfs-client` | 504 | Mount NFS shares (ephemeral or persistent systemd mount units) | | `bin/pos-share-nfs-server` | 245 | Manage the NFS kernel server (status, share/unshare exports, enable/disable) | -| `bin/pos-share-smb-client` | 764 | Mount SMB/CIFS shares (ephemeral or persistent systemd mount units) | +| `bin/pos-share-smb-client` | 766 | Mount SMB/CIFS shares (ephemeral or persistent systemd mount units) | | `bin/pos-share-smb-server` | 441 | Manage the Samba server (status, share/unshare exports, users, enable/disable) | | `bin/pos-share-usb-server` | 362 | USB Redirector server control (--ls, --share; prompts when args omitted) | | `bin/pos-ssh-load-keys` | 31 | Load all SSH keys into the agent | -| `bin/pos-system-backup` | 293 | Encrypted (AES-256) folder snapshots (tar + gpg) | +| `bin/pos-system-backup` | 301 | Encrypted (AES-256) folder snapshots (tar + gpg) | | `bin/pos-system-firewall` | 325 | Interactive UFW management | | `bin/pos-system-health` | 209 | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL | | `bin/pos-system-schedule` | 151 | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently | -| `bin/pos-system-uninstall` | 435 | Remove pos toolkit binaries, services, shell integration, config, and data | -| `bin/pos-ai` | 706 | AI assistant: ask, chat, sessions, capture, models, providers | +| `bin/pos-system-uninstall` | 517 | Remove pos toolkit binaries, services, shell integration, config, and data | +| `bin/pos-ai` | 705 | AI assistant: ask, chat, sessions, capture, models, providers | | `bin/pos-config` | 80 | Interactive editor for the tools' runtime config (reads # POS_CONFIG: registry) | | `bin/pos-tree` | 118 | Show the pos CLI command tree: categories, commands, and subcommands | -| `completions/pos.bash` | 313 | Dynamic bash completion | +| `completions/pos.bash` | 314 | Dynamic bash completion | | `apps/install.sh` | 171 | App install/uninstall picker/orchestrator | diff --git a/DOC/APPS.md b/DOC/APPS.md index 5ab6928..4dc10dc 100644 --- a/DOC/APPS.md +++ b/DOC/APPS.md @@ -1,6 +1,6 @@ # Optional Apps Reference -`apps/` holds 15 optional desktop application installers, one script per app in `apps//.sh`. They are **not** installed by the core bootstrap — run the picker explicitly. +`apps/` holds 18 optional desktop application installers, one script per app in `apps//.sh`. They are **not** installed by the core bootstrap — run the picker explicitly. - [The picker — `apps/install.sh`](#the-picker--appsinstallsh) - [How an app installer works](#how-an-app-installer-works) @@ -34,7 +34,7 @@ bash apps/install.sh --uninstall brave # uninstall a specific app ### Configuration -- Categories: `browsers`, `development`, `media`, `networking`, `remote-access`, `system`, `utilities`. +- Categories: `ai`, `browsers`, `development`, `media`, `networking`, `remote-access`, `system`, `utilities`. - Adding an app = dropping `apps//.sh` into the folder. See [DEV.md](DEV.md) for the required installer conventions. --- @@ -70,6 +70,7 @@ Installation methods used across the catalog: | App | Category | What it is | Install method | |-----|----------|------------|----------------| +| llama.cpp | ai | Local LLM inference server (llama-server) | GitHub release → `/usr/local/lib/llama.cpp-` + `/usr/local/bin` symlinks | | Brave | browsers | Brave browser | apt repo + `apt install brave-browser` | | opencode | development | AI coding agent | official script → `~/.opencode/bin` | | VS Code | development | Code editor | Microsoft apt repo + `apt install code` | diff --git a/DOC/DEV.md b/DOC/DEV.md index b333181..012e458 100644 --- a/DOC/DEV.md +++ b/DOC/DEV.md @@ -156,6 +156,7 @@ Two kinds of config, don't mix them up: - **Machine defaults shipped by the installer:** place the file in `config/` and add copy logic to `postinstall.sh`. If it contains secrets, add to `.gitignore` and document in `DOC/`. - **Runtime tool config set by the user:** `~/.config/linux_post_install/.env` with `chmod 600`. Load it with env-var precedence (flags > environment > file). Patterns: `pos-docker-compose` (`compose.env`), `pos-communication-telegram-sender` (`telegram.env`, edited via `pos config telegram` — token masked), and the shared ones below. Never store tokens in the repo. + - **Canonical loader (new tools, D-D):** `load_env_file ` in `lib/config-ui.sh` — the one shared KEY=VALUE loader (reads `^[A-Z_]+=` lines, skips blank + `#` comments, strips trailing CR and one pair of quotes, exports a key only when the env var is not already set → env always wins over the file; loaded keys are appended to the global `LOADED_ENV_KEYS`). A bare basename resolves under the XDG-aware `CONFIG_DIR`; a full path is used as-is. Missing/unreadable file = quiet no-op; the loader never creates files and never chmods (chmod-600 stays with `cfg_write`/the tool's own writers). Source it with the standard fallback chain: `source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh"`. Legacy `load_system_env()` in `lib/common.sh` is functionally identical and stays for its existing three callers (`pos system health`, `pos system backup`, `pos media sync`) — new tools should use `load_env_file`. One deliberate exception: `pos-docker-compose`'s `load_global_config` still `source`s `compose.env`, because that file is shell-executed (variable expansion, file-beats-defaults layering, unquoted `config set` writes) — don't convert it to `load_env_file`. - `system.env` — shared "system" settings loaded via `load_system_env()` in `lib/common.sh` (currently `BACKUP_SERVICE_ROOTS`, `BACKUP_USB_ROOT`, `BACKUP_MOUNT_BASE`, `BACKUP_USB_BYID`, `HEALTH_BACKUP_MAX_AGE_DAYS`, plus `USB_MOUNT_BASE`/`USB_BYID`/`MEDIA_SYNC_SOURCE`/`MEDIA_SYNC_DEST` for `pos media sync`). Env already exported wins over the file. - `notify.env` — alerting platform selection (`NOTIFY_PLATFORM=telegram,matrix`), read by `lib/notify.sh`. diff --git a/DOC/POS.md b/DOC/POS.md index a31bfb1..dd64a52 100644 --- a/DOC/POS.md +++ b/DOC/POS.md @@ -100,6 +100,8 @@ Model precedence: `--model` flag > `AI_MODEL` env > provider-specific fallback ( **Messaging bridges:** the Telegram and Matrix listeners forward non-command messages starting with `ai ` (case-insensitive) to `pos ai ask` and reply with the model's answer — see [communication → listener](#communication). The Telegram bridge uses one session per chat (`telegram-`), the Matrix bridge one per room (`matrix-`). +**Command execution posture:** when an `ask`/`chat` answer contains a ``sh/shell`` fenced code block, `pos ai` offers to run it and the default is **deny**. On an interactive terminal it prompts `Run this command? [y/N]` — only an explicit `y`/`Y` runs it; anything else (including Enter) declines and adds the command to shell history. Without an interactive tty (pipes, scripts, cron, chat bridges) commands are **never** executed — the block is neither printed nor run. `--trust` auto-executes without the confirmation prompt, but only on an interactive terminal (no effect in a non-tty/bridge context). `--no-command-execution` disables execution entirely: the detected command is neither printed nor run and no prompt appears — this is the structural guard the chat bridges rely on so a future refactor cannot auto-execute. When both `--trust` and `--no-command-execution` are passed, the last one on the command line wins. + `pos ai hf` — Hugging Face model downloader: | Command | Behavior | @@ -122,7 +124,7 @@ Auth: `HF_TOKEN` in `~/.config/linux_post_install/ai.env` (same scope as `pos ai | `pos ai server models` | List `.gguf` files found in `HF_DOWNLOAD_DIR` with sizes | | `pos ai server logs [lines]` | Show recent server logs via `journalctl --user -u pos-ai-server` (default 50 lines) | -Flags: `--port ` (default 8088), `--host ` (default 127.0.0.1), `--model ` (overrides arg/config), `--ctx ` (context window, default 4096), `--gpu ` (-1=auto, 0=CPU, N=explicit, default -1), `--threads ` (default nproc), `--gpu-layers`/`--n-gpu-layers ` (GPU layers override), `--gpu-threads `, `--tensor-split `, `--batch-size `, `--ubatch-size `, `--temperature `, `--top-k `, `--top-p `, `--repetition-penalty `, `--mmap`, `--mlock`, `--kv-cache `, `--ctx-size `, `--metrics`, `--health`, `--slots `. Explicitly requested flags are validated against the installed llama.cpp's `--help` (version-aware: an unsupported flag errors with the detected version); if `--help` cannot be read the tool warns and proceeds. Config keys in `ai.env`: `LLAMACPP_PORT`, `LLAMACPP_HOST`, `LLAMACPP_MODEL`, `LLAMACPP_CTX_SIZE`, `LLAMACPP_GPU_LAYERS`, `LLAMACPP_THREADS`. Requires `curl` + `jq` and a `llama-server` binary on PATH. +Flags: `--port ` (default 8088), `--host ` (default 127.0.0.1), `--model ` (overrides arg/config), `--ctx ` (context window, default 4096), `--gpu ` (-1=auto, 0=CPU, N=explicit, default -1), `--threads ` (default nproc), `--gpu-layers`/`--n-gpu-layers ` (GPU layers override), `--gpu-threads `, `--tensor-split `, `--batch-size `, `--ubatch-size `, `--temperature `, `--top-k `, `--top-p `, `--repetition-penalty `, `--mmap`, `--mlock`, `--kv-cache `, `--ctx-size `, `--metrics`, `--health`, `--slots `. **Every flag that enters `ExecStart` is validated** against the installed llama.cpp's `--help` (word-boundary match, version-aware). Flag validation is split by intent: flags the user explicitly **requested** (via CLI, `LLAMACPP_*` config, or exported env) that are unsupported cause a **hard error** naming the flag + detected version; always-emitted **default** flags (`--port`, `--host`, `--n-gpu-layers`, `--ctx-size`, `--threads`) that the user did not request and that are unsupported are **omitted from the unit with a single warning** (never a hard error, never silently passing an unsupported flag). If `--help` cannot be read the tool warns and proceeds (all flags accepted). The generated unit and `--dry-run` contain only flags that passed validation. Config keys in `ai.env`: `LLAMACPP_PORT`, `LLAMACPP_HOST`, `LLAMACPP_MODEL`, `LLAMACPP_CTX_SIZE`, `LLAMACPP_GPU_LAYERS`, `LLAMACPP_THREADS`. Requires `curl` + `jq` and a `llama-server` binary on PATH (install via `bash apps/install.sh llamacpp`). ### network @@ -183,7 +185,7 @@ Runs a persistent `aria2c` JSON-RPC daemon (`localhost:6800`) as a **systemd use **`--tmux`:** after enqueueing, `add`/`torrent`/`metalink` open a detached tmux session `dl-` running `watch ` (name from `--out` or the URL basename, sanitized and truncated to 40 chars; `-2` suffix on collision). The session closes itself when the download finishes — attach with `tmux attach -t dl-`. -**Outage resilience:** `watch ` auto-restarts its download when the network comes back (it polls `NET_PROBE`, default `timeout 3 bash -c ' ()` line and `replace ` resumes it with a new URL. Both are dry-run aware. +**Outage resilience:** `watch ` auto-restarts its download when the network comes back (it polls `NET_PROBE`, default `timeout 3 bash -c 'exec 3<>/dev/tcp/$1/$2' _ 8.8.8.8 53` — host/port are positional args, never interpolated into the probe's shell source; the `NET_PROBE` env override is operator-controlled). For unattended machines the **retry healer** timer (`pos-aria2-retry.timer`, systemd **user** scope) runs `retry all --once --quiet` every 2 min; it arms automatically whenever a download starts (`add`/`torrent`/`metalink`/`restart`) and disables itself when no active, waiting, or errored downloads remain. When a source is genuinely gone (aria2 error 3, e.g. a 404), the download is marked permanent — `status` prints a `needs fresh link: ()` line and `replace ` resumes it with a new URL. Both are dry-run aware. ### docker @@ -281,11 +283,11 @@ reported as "N videos require sign-in — skipped" (escape hatch: | Command | File | Purpose | Configuration | |---------|------|---------|---------------| | `sudo pos system firewall` | `bin/pos-system-firewall` | Interactive UFW ("UFW POWER") menu: add/delete rules, status, enable/disable/reset, default policies | Must run as root. Every command is previewed and confirmed before execution; supports `--dry-run`; keeps a history of executed commands. Executed mutating changes are announced via `lib/notify.sh` | -| `pos system backup ` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES) only when encrypting. Artifact `_.tar.gz[.gpg]` in the current directory, `chmod 600`; `--no-encrypt` (or `BACKUP_ENCRYPT=0`) keeps a plain `.tar.gz` with no password prompt (headless/cron safe). Once the archive verifies, connected USB storage is offered (detected via `lsblk` TRAN with `lsusb`/by-id cross-check; unmounted sticks get a mount offer first — see `DOC/howto/system.md`; or pinned with `BACKUP_USB_ROOT`): the copy lands in `/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh`. Bare invocation on a terminal (or the `menu` subcommand) opens an interactive menu wrapping these modes (encrypted backup with typed folder, encrypted backup picked from the service roots, unencrypted variant) — each runs only after an explicit y/N confirm naming the folder; arguments stay scriptable | +| `pos system backup ` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored; the passphrase is fed to gpg on an internal fd — `--passphrase-fd`, never via argv, so it cannot leak through `ps`). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES) only when encrypting. Artifact `_.tar.gz[.gpg]` in the current directory, `chmod 600`; `--no-encrypt` (or `BACKUP_ENCRYPT=0`) keeps a plain `.tar.gz` with no password prompt (headless/cron safe). Once the archive verifies, connected USB storage is offered (detected via `lsblk` TRAN with `lsusb`/by-id cross-check; unmounted sticks get a mount offer first — see `DOC/howto/system.md`; or pinned with `BACKUP_USB_ROOT`): the copy lands in `/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh`. Bare invocation on a terminal (or the `menu` subcommand) opens an interactive menu wrapping these modes (encrypted backup with typed folder, encrypted backup picked from the service roots, unencrypted variant) — each runs only after an explicit y/N confirm naming the folder; arguments stay scriptable | | `pos system backup --service` | `bin/pos-system-backup` | Lists folders under `/srv` and `~/srv`, lets you pick one, then runs the same backup | Roots via `BACKUP_SERVICE_ROOTS` (space-separated, default `/srv $HOME/srv`) or `~/.config/linux_post_install/system.env` | | `pos system health` | `bin/pos-system-health` | Host health dashboard: disk per mount, RAM/swap, failed systemd units, backup age, fail2ban, docker containers. Exits 1 if any check FAILs | Console-only reporter — health itself never sends notifications; forward the output with a wrapper (e.g. the Telegram/Matrix listener map `/status=pos system health`) or schedule it via `pos system schedule` with a `NOTIFY` policy. `HEALTH_BACKUP_MAX_AGE_DAYS` (default 2) and `BACKUP_SERVICE_ROOTS` come from `~/.config/linux_post_install/system.env`; `--help` shows the effective values | | `pos system schedule ` | `bin/pos-system-schedule` | Scheduled jobs — run a command on a timer, notify (or stay silent): `run [name\|all]`, `list`, `config`, `enable [name\|all]`, `disable [name\|all]`, `status`, `migrate`. Each job is a file in `~/.config/linux_post_install/schedule.d/.env` with `INTERVAL` (`5m…59m`, `1h…23h`, `hourly`, `daily`, `weekly`, `OnCalendar=…`), `NOTIFY` policy, optional `MSG`, `RULE` (threshold only), and `COMMAND` = the literal rest of the line (pipes/quotes/`sudo` fine). Policies: `always` (full output every run), `onchange` (send when output differs from the last run; first run always sends), `onerror` (non-zero exit or empty output), `threshold` (first numeric output vs `RULE`, alert on false→true + one recovery — the old event-trigger behavior), `never` (side-effect jobs, no notify) | One systemd **user** timer pair per job (`pos-schedule-.timer` + oneshot `.service`, `Persistent=true`), reconciled on `enable`/`disable`; the legacy single `pos-event-trigger` timer is auto-removed. `migrate` converts a pre-existing `event.env` rule set into `schedule.d/rule-N.env` threshold jobs. `config` is an interactive editor (add/edit/remove/enable/disable, validates interval + threshold); alerts via `lib/notify.sh`; `--dry-run` previews runs/writes/sends; jobs are arbitrary shell commands (chmod 600, same trust model as the Telegram map); starter jobs in `config/schedule.d/` auto-installed no-clobber by postinstall. Bare invocation on a terminal (or the `menu` subcommand) opens an interactive hub over these verbs (list, timer status, run-now, enable, disable, config editor) — a menu run-now asks y/N first and goes through the same `run ` path the systemd timers use | -| `pos system uninstall` | `bin/pos-system-uninstall` | Safe, interactive uninstaller for the pos toolkit — scans and removes binaries, services, shell integration, config, and data in three tiers | Tier 1 (always): binaries in `/usr/local/bin/` (pos, pos-*, libs, ai-providers, entertainment plugins, prebuilt, features), systemd services (disable+remove), shell integration in `~/.bashrc` (PATH, completion, pos-ai-hook source), completion file. Tier 2 (`--config`): `~/.config/linux_post_install/` (.env files, schedule.d/, authorized_keys, rclone.conf). Tier 3 (`--data`): `~/.local/share/linux_post_install/` (ai sessions, logs, captured output). Flags: `--yes` (skip prompts, tier 1 only), `--config` (include tier 2), `--data` (include tier 3). Combine all three for nuclear removal. Git repo is never removed | +| `pos system uninstall` | `bin/pos-system-uninstall` | Safe, interactive uninstaller for the pos toolkit — scans and removes binaries, services, shell integration, config, and data in three tiers | Tier 1 (always): binaries in `/usr/local/bin/` (pos, pos-*, libs, ai-providers, entertainment plugins, prebuilt, features), systemd services (disable+remove) including runtime-created `~/.config/systemd/user/pos-*` user units, ScaleTail templates + feature-flag store under `/usr/local/share/linux_post_install/`, shell integration in `~/.bashrc` (PATH, completion, pos-ai-hook source), completion file. Tier 2 (`--config`): `~/.config/linux_post_install/` (.env files, schedule.d/, authorized_keys, rclone.conf). Tier 3 (`--data`): `~/.local/share/linux_post_install/` (ai sessions, logs, captured output). Flags: `--yes` (skip prompts, tier 1 only), `--config` (include tier 2), `--data` (include tier 3). Combine all three for nuclear removal. Git repo is never removed | A scheduled job is the recommended way to run the health dashboard on a timer: a `daily` job with `COMMAND=pos system health` and `NOTIFY=always` sends the dashboard output as the alert — no separate systemd unit needed (the old `pos-health.{service,timer}` units are gone; a legacy install may still have them failed/leftover — disable and remove them). @@ -332,9 +334,9 @@ Subcommands that need input prompt interactively when args are omitted. Bare inv | Command | File | Purpose | Configuration | |---------|------|---------|---------------| | `pos communication telegram sender send "text"` | `bin/pos-communication-telegram-sender` | Send a message, link, or media file (auto-detects the type) to a Telegram chat via the Bot API | Token + chat ID from `~/.config/linux_post_install/telegram.env` (`TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID`, chmod 600). Precedence: `--token`/`--chat-id` flags > env > config file | -| `pos communication telegram listener` | `bin/pos-communication-telegram-listener` | Telegram bot listener: map `/command` → bash commands and `` → apps, run them from chat; interactive editor for the map | Same `telegram.env` (the bot is the owner, `TELEGRAM_CHAT_ID`). Map lives in `~/.config/linux_post_install/telegram_commands.env` (`/cmd=bash command` lines); text-prefix app map in `telegram_prefixes.env` (`=command` lines) — both chmod 600 | +| `pos communication telegram listener` | `bin/pos-communication-telegram-listener` | Telegram bot listener: map `/command` → bash commands and `` → apps, run them from chat; interactive editor for the map | Same `telegram.env` (the bot is the owner, `TELEGRAM_CHAT_ID`; commands also require `TELEGRAM_OWNER_ID` — see the detail block). Map lives in `~/.config/linux_post_install/telegram_commands.env` (`/cmd=bash command` lines); text-prefix app map in `telegram_prefixes.env` (`=command` lines) — both chmod 600 | | `pos communication matrix sender send "text"` | `bin/pos-communication-matrix-sender` | Send a text message (plain or `--markdown`) to a Matrix room via the client-server API; also `login` (password → access token) and `test` | Homeserver + room from `~/.config/linux_post_install/matrix.env` (`MATRIX_HOMESERVER`, `MATRIX_ACCESS_TOKEN`, `MATRIX_USER_ID`, `MATRIX_ROOM_ID`, chmod 600, secrets masked by `pos config matrix`). Precedence: `--room` flag > env > config file | -| `pos communication matrix listener` | `bin/pos-communication-matrix-listener` | Matrix listener: map `/command` → bash commands and run them from room messages; interactive editor for the map | Same `matrix.env` (reacts to `MATRIX_USER_ID`'s own messages; watches `MATRIX_ROOM_ID` or all joined rooms). Map lives in `~/.config/linux_post_install/matrix_commands.env` (`/cmd=bash command` lines, chmod 600) | +| `pos communication matrix listener` | `bin/pos-communication-matrix-listener` | Matrix listener: map `/command` → bash commands and run them from room messages; interactive editor for the map | Same `matrix.env` (reacts to `MATRIX_USER_ID`'s own messages only, in the room set by `MATRIX_ROOM_ID` — unset = fail-closed, no commands run). Map lives in `~/.config/linux_post_install/matrix_commands.env` (`/cmd=bash command` lines, chmod 600) | | `pos communication scrcpy [cmd]` | `bin/pos-communication-scrcpy` | Mirror/control an Android device via scrcpy+adb: `devices`, `record`, `tcpip`, `connect`, `push`, `pull`, `screenshot`, `info` (bare = mirror) | `scrcpy.env` (`SCRCPY_SERIAL`, `SCRCPY_MAX_SIZE`, `SCRCPY_MAX_FPS`, `SCRCPY_BIT_RATE`, `SCRCPY_FULLSCREEN`, `SCRCPY_RECORD_DIR`, `SCRCPY_PUSH_TARGET`, `SCRCPY_EXTRA_FLAGS`) via `pos config scrcpy` | `pos communication telegram sender` in detail: @@ -367,7 +369,7 @@ The bot token is a secret — it is stored only in `~/.config/linux_post_install | `pos communication telegram listener prefix` | List the text-prefix map (`telegram_prefixes.env`: `=command` lines) plus the built-in Gemini bridge word | | `pos communication telegram listener prefix ` | Show one mapping, or map `` to a command with `prefix ` — non-command messages ` ` run the command with `` appended as ONE quoted argument (e.g. `prefix opencode opencode` → "opencode check cpu" runs `opencode "check cpu"`). `prefix -r ` removes. Mapped words shadow the Gemini bridge; the bridge word itself (`TELEGRAM_AI_PREFIX`, default `ai`) is set via `pos config telegram` | -The map file is re-read for every message — edits apply without a restart. The listener only reacts to the owner chat (`TELEGRAM_CHAT_ID`); anyone else's message is ignored. `/help` lists mapped commands; an unmapped command replies "Unknown command". +The map file is re-read for every message — edits apply without a restart. A command runs only when the message is sent to the owner chat (`TELEGRAM_CHAT_ID`) BY the owner account (`TELEGRAM_OWNER_ID`) — both must match, so a forwarded message or an impersonator can't trigger commands; with `TELEGRAM_OWNER_ID` unset the daemon starts but refuses every command (fail-closed). `/help` lists mapped commands; an unmapped command replies "Unknown command". **Text-prefix map** (`telegram_prefixes.env`, managed via the `prefix` verb): for apps, not bash snippets — a non-command message ` ` runs the mapped command with `` appended as ONE quoted argument, e.g. `opencode=opencode` turns "opencode check cpu" into `opencode "check cpu"`. First match wins (file order), matching is case-insensitive and the word must be space-delimited (bare `` with no trailing space replies Unknown command). Values are syntax-checked on save; `@quiet ` values suppress the reply; the 120s timeout + empty→`OK` + `exit ` reply mirror the command map. Routing order on every non-command message: text-prefix map → AI bridge → `/command` map → "Unknown command". @@ -397,7 +399,7 @@ The access token is a secret — it is stored only in `~/.config/linux_post_inst | `pos communication matrix listener --disable` | Stops, disables, and removes the service | | `pos communication matrix listener --run` | Run the polling loop in the foreground (what the service executes) | -The daemon long-polls `/sync` (30s timeout, per-sync `since` token, compact filter that drops presence/account_data/device noise and only requests `m.room.message` timeline events). It reacts only to messages **from `MATRIX_USER_ID`** (your own account — resolved via `/account/whoami` if unset); a `MATRIX_ROOM_ID` restricts it to one room, otherwise every joined room is watched. `/` and `!` prefixes both resolve (`!status` = `/status`). `/help` lists mapped commands; an unmapped command replies "Unknown command". Non-command text starting with `ai ` (case-insensitive, e.g. `ai what is Nvidia`) is forwarded to Gemini via `pos ai gemini ask` with a per-room session (`matrix-`; `ai /reset` clears it) and the answer is replied verbatim with markdown stripped. Replies are sent as `m.text` threaded with `m.in_reply_to` on your message. Commands run as your user via `timeout 60 bash -c "…"` (stdout + stderr are replied, truncated to ~3800 chars; empty output → `OK`; non-zero exit is prefixed with `exit `), so `sudo` inside them needs a NOPASSWD rule. A map value prefixed with `@quiet ` runs the command but does NOT reply — for commands that already send their own notification (e.g. `pos system backup` self-notifies, so `/backup=@quiet pos system backup $HOME/Documents` avoids a double message). Map lines may carry a `/cmd::description=…` description. `--enable` warns if linger is off — the service stops when you log out unless you run `sudo loginctl enable-linger $(whoami)`. +The daemon long-polls `/sync` (30s timeout, per-sync `since` token, compact filter that drops presence/account_data/device noise and only requests `m.room.message` timeline events). It reacts only to messages **from `MATRIX_USER_ID`** (your own account — resolved via `/account/whoami` if unset), and only in the room set by `MATRIX_ROOM_ID` — without it the daemon starts but runs no commands (fail-closed), so a bot account in many rooms can't be tricked. `/` and `!` prefixes both resolve (`!status` = `/status`). `/help` lists mapped commands; an unmapped command replies "Unknown command". Non-command text starting with `ai ` (case-insensitive, e.g. `ai what is Nvidia`) is forwarded to Gemini via `pos ai gemini ask` with a per-room session (`matrix-`; `ai /reset` clears it) and the answer is replied verbatim with markdown stripped. Replies are sent as `m.text` threaded with `m.in_reply_to` on your message. Commands run as your user via `timeout 60 bash -c "…"` (stdout + stderr are replied, truncated to ~3800 chars; empty output → `OK`; non-zero exit is prefixed with `exit `), so `sudo` inside them needs a NOPASSWD rule. A map value prefixed with `@quiet ` runs the command but does NOT reply — for commands that already send their own notification (e.g. `pos system backup` self-notifies, so `/backup=@quiet pos system backup $HOME/Documents` avoids a double message). Map lines may carry a `/cmd::description=…` description. `--enable` warns if linger is off — the service stops when you log out unless you run `sudo loginctl enable-linger $(whoami)`. `pos communication scrcpy` in detail: diff --git a/DOC/howto/ai.md b/DOC/howto/ai.md index c11106d..10d0743 100644 --- a/DOC/howto/ai.md +++ b/DOC/howto/ai.md @@ -25,6 +25,9 @@ attaches the latest pos command output or captured output (tail, max 4096 chars) to the question and notes on stderr which source was attached, its age, and a staleness warning once it is older than an hour (`ask` only; stdout stays pure answer). Use `capture` to save output from any command for `--last`. +`--trust` and `--no-command-execution` control how (if at all) agent-detected +command blocks run — see [Command execution posture](#command-execution-posture) +below. Backward compatibility: `pos ai gemini`, `pos ai openrouter`, and `pos ai llamacpp` still work as shorthand for `pos ai --provider gemini`, @@ -245,6 +248,31 @@ truncated). To disable: `unset __POS_CAPTURE_ACTIVE`. - On a non-2xx response the API's `error.message` is shown and the exit code is non-zero — so scripts can rely on `ask` failing loudly. +## Command execution posture + +When a model's answer contains a ```sh/shell fenced code block, `ask`/`chat` +offer to **run** it as a shell command on your machine. Because that code is +untrusted, AI-generated external authority, the default is to **deny**: + +- On an interactive terminal, `ask`/`chat` print the detected command and prompt + `Run this command? [y/N]` — you must type **`y`** (or `Y`) to execute it. + Enter or any other key **declines**: the command is added to your shell + history (press ↑ to recall and edit it) but never run. +- Without an interactive controlling tty (pipes, scripts, cron, the Telegram / + Matrix bridges) commands are **never** executed — the code block is neither + printed nor run. + +`--trust` auto-executes detected commands **without** the confirmation prompt — +but only on an interactive terminal. It is meant for trusted alias wrappers; +do not pass it unless you fully trust the agent's output. It has no effect in a +non-tty/chat-bridge context, which never executes commands anyway. + +`--no-command-execution` disables execution entirely: the detected command is +neither printed nor run and no confirmation prompt appears. It is the +structural guard used by the chat bridges so that a future refactor cannot +accidentally auto-execute model output. When both `--trust` and +`--no-command-execution` are given, the last one on the command line wins. + ## Troubleshooting - `ask` errors "No Gemini API key — run 'pos config ai'" → the key isn't set diff --git a/DOC/howto/communication.md b/DOC/howto/communication.md index 02e0345..637b3a9 100644 --- a/DOC/howto/communication.md +++ b/DOC/howto/communication.md @@ -24,14 +24,23 @@ health digests, backup alerts, firewall changes — and can be used directly. ```bash pos config telegram -# edit TELEGRAM_BOT_TOKEN and TELEGRAM_CHAT_ID (masked input), then test: +# edit TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID and TELEGRAM_OWNER_ID (masked input), then test: pos communication telegram sender test # config lives in ~/.config/linux_post_install/telegram.env (chmod 600) ``` -The bot token comes from @BotFather, the chat ID from @userinfobot (or by -starting a chat and reading it). `pos config telegram` shows the current values -(token masked). +The bot token comes from @BotFather; the chat ID **and your numeric user ID +(`TELEGRAM_OWNER_ID`)** from @userinfobot (or by starting a chat and reading +it). `pos config telegram` shows the current values (token masked). The +listener only executes commands when both the chat id and your owner id match +— with `TELEGRAM_OWNER_ID` unset it runs but ignores every command +(fail-closed, see below). + +> **Token in the process list:** the Telegram Bot API embeds the bot token in +> every API URL (`https://api.telegram.org/bot/…`), so while a send or +> poll runs the token is visible in `ps` output. This is inherent to the Bot +> API — keep the token scoped to a single-purpose bot and revoke it in +> BotFather if it ever leaks. ### Send @@ -92,8 +101,13 @@ pos communication telegram listener --disable # remove it falls back to the bash command. Telegram only registers lowercase `[a-z0-9_]` names (1–32 chars); `/Status` or `/my-cmd` are skipped from the menu but still work when typed. An empty map clears the menu. -- **Owner-only:** the bot only reacts to `TELEGRAM_CHAT_ID` (your own chat); - others are ignored. `/help` lists mapped commands; unknown → "Unknown command". +- **Owner-only (chat + account):** the bot executes only messages sent to + `TELEGRAM_CHAT_ID` (your chat) **BY** `TELEGRAM_OWNER_ID` (your account) — + both must match, so a forwarded message or an impersonator can't trigger + commands. With `TELEGRAM_OWNER_ID` unset the daemon runs but ignores every + command (fail-closed; `systemctl --user status pos-telegram-listener.service` + shows the message-time log lines). `/help` lists mapped commands; unknown → + "Unknown command". - **Runs as you:** mapped commands execute as your user with a 60s timeout, stdout + stderr are replied to the chat (truncated ~3800 chars; empty → `OK`). `sudo` inside a command needs a NOPASSWD rule. @@ -220,8 +234,10 @@ pos communication matrix listener --disable # remove it ``` - **Self-messaging:** the listener reacts to messages **from your own user id** (`MATRIX_USER_ID`) — in practice that means a second device (or another - account) sending the commands. If `MATRIX_ROOM_ID` is set it only watches - that room, otherwise every room you've joined. `/` and `!` both work + account) sending the commands. It only watches the room set by + `MATRIX_ROOM_ID`; with `MATRIX_ROOM_ID` unset the daemon runs but refuses to + execute any command (fail-closed), so a bot account that has joined many + rooms can't be tricked into running commands. `/` and `!` both work (`!status` = `/status`). `/help` lists mapped commands; unknown → "Unknown command". - **Runs as you:** mapped commands execute as your user with a 60s timeout, diff --git a/Makefile b/Makefile index 6f95c13..1b3bebb 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: check gen hook lint +.PHONY: check gen hook lint test ## check — verify repo self-consistency (syntax, exec bits, doc/code sync, smoke) check: @@ -12,6 +12,10 @@ gen: lint: ./scripts/lint-conventions.sh +## test — regression suite (zero-dependency runner; no network, no system changes) +test: + ./tests/run-tests.sh + ## hook — install the opt-in pre-commit hook (runs `make check`) hook: ./scripts/install-hooks.sh diff --git a/apps/ai/llamacpp.sh b/apps/ai/llamacpp.sh new file mode 100755 index 0000000..0b72626 --- /dev/null +++ b/apps/ai/llamacpp.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +set -euo pipefail +source "$(dirname "$0")/../../lib/common.sh" + +# llama.cpp — Local LLM inference server (llama-server) + CLI tools. +# +# Asset-naming note (probed live 2026-09-06): the vX.Y.Z milestone +# releases carry NO binary assets (only nightly-tag.txt); the prebuilt +# Ubuntu binaries ship on the nightly bNNNNN releases as +# llama--bin-ubuntu-x64.tar.gz / llama--bin-ubuntu-arm64.tar.gz +# so we scan the newest releases for the first one that ships our arch +# instead of hitting /releases/latest. +RELEASES_URL="https://api.github.com/repos/ggml-org/llama.cpp/releases?per_page=10" + +install_llamacpp() { + command -v llama-server &>/dev/null && { log "llama.cpp already installed"; return 0; } + + local arch + case "$(uname -m)" in + x86_64) arch="x64" ;; + aarch64) arch="arm64" ;; + *) err "Unsupported architecture: $(uname -m) (llama.cpp publishes x64/arm64 Ubuntu builds)" ;; + esac + + spawn "Fetching latest llama.cpp release info" bash -c " + curl -fsSL '$RELEASES_URL' -o /tmp/llamacpp-releases.json + " + + local tag asset_url + if ! { read -r tag && read -r asset_url; } < <(python3 -c " +import json, sys +rels = json.load(open('/tmp/llamacpp-releases.json')) +suffix = '-bin-ubuntu-$arch.tar.gz' +for r in rels: + for a in r['assets']: + if a['name'].endswith(suffix): + print(r['tag_name']) + print(a['browser_download_url']) + sys.exit(0) +sys.exit(1) +" 2>/dev/null); then + rm -f /tmp/llamacpp-releases.json + err "No llama.cpp Ubuntu $arch binary release found — see https://github.com/ggml-org/llama.cpp/releases" + fi + + spawn "Installing llama.cpp $tag ($arch)" bash -c " + install_dir=/usr/local/lib/llama.cpp-$tag + curl -fsSL '$asset_url' -o /tmp/llamacpp.tar.gz + sudo rm -rf \$install_dir + sudo mkdir -p \$install_dir + sudo tar xzf /tmp/llamacpp.tar.gz -C \$install_dir --strip-components=1 + for bin in \$install_dir/llama*; do + [ -f \"\$bin\" ] && [ -x \"\$bin\" ] || continue + sudo ln -sf \"\$bin\" /usr/local/bin/\$(basename \"\$bin\") + done + rm -f /tmp/llamacpp.tar.gz /tmp/llamacpp-releases.json + " + + log "llama.cpp $tag installed — run the server with 'pos ai server start '" +} + +uninstall_llamacpp() { + command -v llama-server &>/dev/null || { log "llama.cpp not installed"; return 0; } + + spawn "Removing llama.cpp files" sudo rm -rf /usr/local/lib/llama.cpp-* + + # Remove only the symlinks we created (targets inside the install dir); + # unrelated /usr/local/bin/llama* files are left alone. + spawn "Removing llama.cpp symlinks" bash -c " + for link in /usr/local/bin/llama*; do + [ -L \"\$link\" ] || continue + target=\$(readlink \"\$link\") + case \"\$target\" in + /usr/local/lib/llama.cpp-*) sudo rm -f \"\$link\" ;; + esac + done + " + + log "llama.cpp removed" +} + +case "${1:-}" in + uninstall) uninstall_llamacpp ;; + *) install_llamacpp ;; +esac \ No newline at end of file diff --git a/apps/install.sh b/apps/install.sh index c272f5d..1a170ab 100755 --- a/apps/install.sh +++ b/apps/install.sh @@ -42,6 +42,7 @@ APPS_DIR="$(dirname "$0")" # ── Category display names ───────────────────────────────────── declare -A CAT_NAMES=( + [ai]="AI / ML" [browsers]="Browsers" [development]="Development" [media]="Media" diff --git a/bin/pos-ai b/bin/pos-ai index a148bf5..bfba08f 100755 --- a/bin/pos-ai +++ b/bin/pos-ai @@ -1,14 +1,17 @@ #!/usr/bin/env bash set -euo pipefail # POS: ai ask — AI assistant: ask, chat, sessions, capture, models, providers -# POS_SUBCMDS: ask chat sessions capture models providers -# POS_FLAGS: --provider --model --session --system --full --last --trust +# POS_SUBCMDS: ask chat sessions capture models providers llamacpp +# POS_FLAGS: --provider --model --session --system --full --last --trust --no-command-execution # POS_CONFIG: ai | ai.env | AI_PROVIDER=:Provider (gemini, openrouter or llamacpp, default gemini) | @[AI_PROVIDER=gemini|] Gemini | *providers=gemini | @[AI_PROVIDER=openrouter] OpenRouter | *providers=openrouter | @[AI_PROVIDER=llamacpp] llamacpp | LLAMACPP_PORT=:Server port (default 8088) | LLAMACPP_HOST=:Bind address (default 127.0.0.1) | LLAMACPP_MODEL=:Default model path (GGUF) | LLAMACPP_CTX_SIZE=num:Context window size (default 4096) | LLAMACPP_GPU_LAYERS=num:GPU layers (-1=auto, 0=CPU, default -1) | LLAMACPP_THREADS=num:CPU threads (default: nproc) | *providers=llamacpp | @General | AI_SYSTEM_PROMPT=:Custom system prompt (overrides built-in, empty to reset) source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + # ── Paths & constants ────────────────────────────────────────── -CONFIG_FILE="$HOME/.config/linux_post_install/ai.env" +CONFIG_FILE="${CONFIG_FILE:-$CONFIG_DIR/ai.env}" SESSION_DIR="$HOME/.local/share/linux_post_install/ai" DISPATCH_LOG_DIR="$HOME/.local/share/linux_post_install/logs" # bin/pos per-run logs LAST_CMD_OUTPUT_FILE="$HOME/.local/share/linux_post_install/last_cmd_output" # --last fallback for any command @@ -32,12 +35,12 @@ DEFAULT_SYSTEM_PROMPT_HARD="You are a Linux CLI assistant. Rules: 5. Match the user's OS/package manager (apt/dnf/pacman)" # Legacy: kept for session migration and backward compat config -LEGACY_GEMINI_CONFIG="$HOME/.config/linux_post_install/ai.env" -LEGACY_OPENROUTER_CONFIG="$HOME/.config/linux_post_install/ai-openrouter.env" +LEGACY_GEMINI_CONFIG="${LEGACY_GEMINI_CONFIG:-$CONFIG_DIR/ai.env}" +LEGACY_OPENROUTER_CONFIG="${LEGACY_OPENROUTER_CONFIG:-$CONFIG_DIR/ai-openrouter.env}" usage() { cat <] [--model ] [--session ] [--system ] [--full] [--last] [--trust] +Usage: pos ai [subcommand] [--provider ] [--model ] [--session ] [--system ] [--full] [--last] [--trust] [--no-command-execution] AI assistant with pluggable providers (gemini, openrouter, llamacpp). @@ -73,8 +76,16 @@ Options: 'capture'. Notes on stderr which source was attached and its age; warns when stale (>60 min). --trust Auto-execute agent-detected commands without confirmation. - Used by trusted alias wrappers — do NOT pass manually - unless you fully trust the agent's output. + Interactive terminals only — it has no effect when stdout/ + stdin is not a tty (e.g. chat bridges, cron), which never + execute commands. Used by trusted alias wrappers — do NOT + pass manually unless you fully trust the agent's output. +--no-command-execution Never run agent-detected command blocks. The proposed + command is neither printed nor executed; no + confirmation prompt is shown (structural guard — used by + chat bridges so a future refactor cannot auto-execute). + Each of these two modes contradict the other: the last one + given on the command line wins. -h|--help This help. Config: $CONFIG_FILE (edit with 'pos config ai') @@ -126,36 +137,16 @@ load_provider() { source "$f" } -# ── ai.env loader (same pattern as telegram.env) ──────────────── +# ── ai.env loader (canonical loader + legacy files) ───────────── load_config() { - [ -f "$CONFIG_FILE" ] || return 0 - local k v - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - v="${v//$'\r'/}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) + load_env_file "$CONFIG_FILE" # Legacy provider-specific config files (fallback for old configs) # Both files are loaded — env-var precedence means unified AI_API_KEY wins. local legacy_files="$LEGACY_GEMINI_CONFIG $LEGACY_OPENROUTER_CONFIG" local legacy_env for legacy_env in $legacy_files; do [ -f "$legacy_env" ] && [ "$legacy_env" != "$CONFIG_FILE" ] || continue - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in \#*) continue ;; esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - v="${v//$'\r'/}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$legacy_env" || true) + load_env_file "$legacy_env" done } @@ -378,8 +369,14 @@ _extract_commands() { # ── Interactive prompt to run extracted commands ───────────────── _prompt_run_command() { local cmd="$1" trusted="${2:-0}" - # Only prompt on interactive terminals with a controlling tty + # Structural guards: commands never execute without a controlling tty, and + # never when --no-command-execution was passed. Both apply BEFORE any + # auto-execute path (including --trust), so a future refactor cannot + # introduce a non-interactive or bridge-forced execution. [ -w /dev/tty ] || return 0 + if [ "${NO_EXEC:-0}" -eq 1 ]; then + return 0 + fi printf '\n%s\n' "Command detected:" >&2 printf ' %s\n\n' "$cmd" >&2 if [ "$trusted" -eq 1 ]; then @@ -388,20 +385,19 @@ _prompt_run_command() { run eval "$cmd" return fi - printf 'Run this command? [Y/n] ' >&2 + printf 'Run this command? [y/N] ' >&2 local choice IFS= read -r choice /dev/null || true printf '%s\n' "Command added to history — press ↑ to recall, edit, and run." >&2 ;; - *) - # Y or Enter: execute - printf '%s\n' "$cmd" - run eval "$cmd" - ;; esac } @@ -642,6 +638,7 @@ MODEL_OVERRIDE="" FULL_MODE=0 LAST_MODE=0 TRUST_MODE=0 +NO_EXEC=0 PROVIDER="" cmd="" args=() @@ -665,7 +662,9 @@ while [ $# -gt 0 ]; do --last) LAST_MODE=1; shift ;; --trust) - TRUST_MODE=1; shift ;; + TRUST_MODE=1; NO_EXEC=0; shift ;; + --no-command-execution) + NO_EXEC=1; TRUST_MODE=0; shift ;; -*) err "Unknown option '$1' (see --help)" ;; *) if [ -z "$cmd" ]; then diff --git a/bin/pos-ai-gemini b/bin/pos-ai-gemini index 2a8b5a4..9ab9412 100755 --- a/bin/pos-ai-gemini +++ b/bin/pos-ai-gemini @@ -1,7 +1,7 @@ #!/usr/bin/env bash set -euo pipefail # POS: ai gemini — Forward to pos ai --provider gemini (backward compat) -# POS_SUBCMDS: ask chat models sessions capture +# POS_SUBCMDS: ask chat models sessions capture providers # Thin forwarder — all logic lives in bin/pos-ai + lib/ai-providers/gemini.sh case "${1:-}" in -h|--help) exec pos ai --provider gemini --help ;; esac exec pos ai --provider gemini "$@" diff --git a/bin/pos-ai-hf b/bin/pos-ai-hf index 5c2ebe2..1645d83 100755 --- a/bin/pos-ai-hf +++ b/bin/pos-ai-hf @@ -1,6 +1,7 @@ #!/usr/bin/env bash set -euo pipefail # POS: ai hf — Download AI models from Hugging Face (search, download, manage) +# POS_SUBCMDS: search download list remove info files cache # POS_FLAGS: --branch --gguf --list --output --quant --include --exclude --revision # POS_DEPS: curl jq # POS_CONFIG: ai | ai.env | HF_TOKEN=secret:Hugging Face API token (https://huggingface.co/settings/tokens) | HF_DOWNLOAD_DIR=:Model download directory (default ~/.local/share/linux_post_install/ai/models) @@ -18,29 +19,20 @@ set -euo pipefail source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + # ── Dependencies (before --help) ─────────────────────────────── command -v curl &>/dev/null || err "curl not found (install curl)" command -v jq &>/dev/null || err "jq not found (install jq)" # ── Config & defaults ────────────────────────────────────────── -CONFIG_FILE="${CONFIG_FILE:-$HOME/.config/linux_post_install/ai.env}" +CONFIG_FILE="${CONFIG_FILE:-$CONFIG_DIR/ai.env}" HF_TOKEN="${HF_TOKEN:-}" HF_DOWNLOAD_DIR="${HF_DOWNLOAD_DIR:-$HOME/.local/share/linux_post_install/ai/models}" load_hf_config() { - [ -f "$CONFIG_FILE" ] || return 0 - local k v - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - v="${v//$'\r'/}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) + load_env_file "$CONFIG_FILE" } load_hf_config @@ -698,7 +690,9 @@ cmd_download() { rm -rf "$temp_dir" trap - EXIT else - # Single file download - use original sequential approach + # Single-file download — the same honesty rules as the parallel path: + # a failure is recorded in failed_files so the model is never marked + # complete (.hf-meta suppressed) and the tool exits rc 1. while IFS= read -r file_json; do local fname fsize fname="$(printf '%s' "$file_json" | jq -r '.rfilename')" @@ -708,12 +702,8 @@ cmd_download() { local url="${HF_BASE}/${ns}/${repo}/resolve/${branch}/${fname}" local target="${target_dir}/${fname}" - if [ "$file_count" -gt 1 ]; then - downloaded=$((downloaded + 1)) - printf '[%d/%d] Downloading %s...\n' "$downloaded" "$file_count" "$fname" >&2 - fi - if ! hf_download_with_progress "$url" "$target"; then + failed_files+=("$fname") warn "Failed to download $fname" continue fi @@ -743,8 +733,15 @@ METAEOF warn "Not writing .hf-meta — ${repo_id} is incomplete (${#failed_files[@]} file(s) failed)" fi - # Summary - if [ "$file_count" -eq 1 ]; then + # Summary — honest in both single-file and parallel paths: any failure + # yields a success/failure count, never a false "downloaded" claim. + if [ "${#failed_files[@]}" -gt 0 ]; then + # Honest count: attempted = total files, success = total − failures + local success_count=$((file_count - ${#failed_files[@]})) + printf '📥 Downloaded: %s (%d of %d files, %d failed: %s)\n' \ + "$repo_id" "$success_count" "$file_count" "${#failed_files[@]}" "${failed_files[*]}" + printf '📁 %s/\n' "$target_dir" + elif [ "$file_count" -eq 1 ]; then local fname fname="$(printf '%s' "$filtered_files" | jq -r '.[0].rfilename')" local fsize @@ -754,22 +751,14 @@ METAEOF printf '📥 Downloaded: %s/%s (%s)\n' "$repo_id" "$fname" "$human_size" printf '📁 %s/%s\n' "$target_dir" "$fname" else - if [ "${#failed_files[@]}" -gt 0 ]; then - # Honest count: attempted = total files, success = total − failures - local success_count=$((file_count - ${#failed_files[@]})) - printf '📥 Downloaded: %s (%d of %d files, %d failed: %s)\n' \ - "$repo_id" "$success_count" "$file_count" "${#failed_files[@]}" "${failed_files[*]}" - else - local total_human - total_human="$(hf_human_size "$total_size")" - printf '📥 Downloaded: %s (%d files, %s)\n' "$repo_id" "$file_count" "$total_human" - fi + local total_human + total_human="$(hf_human_size "$total_size")" + printf '📥 Downloaded: %s (%d files, %s)\n' "$repo_id" "$file_count" "$total_human" printf '📁 %s/\n' "$target_dir" fi - # A partially-failed parallel batch must be detectable by scripts — - # exit non-zero. The sequential single-file path is unchanged: it never - # populates failed_files, so this clause only fires for the parallel path. + # Any failure — single-file or parallel batch — must be detectable by + # scripts: exit non-zero. if [ "${#failed_files[@]}" -gt 0 ]; then return 1 fi diff --git a/bin/pos-ai-llamacpp b/bin/pos-ai-llamacpp index ea130c2..03067b9 100755 --- a/bin/pos-ai-llamacpp +++ b/bin/pos-ai-llamacpp @@ -1,7 +1,7 @@ #!/usr/bin/env bash set -euo pipefail # POS: ai llamacpp — Forward to pos ai --provider llamacpp (backward compat) -# POS_SUBCMDS: ask chat models sessions capture +# POS_SUBCMDS: ask chat models sessions capture providers # Thin forwarder — all logic lives in bin/pos-ai + lib/ai-providers/llamacpp.sh case "${1:-}" in -h|--help) exec pos ai --provider llamacpp --help ;; esac exec pos ai --provider llamacpp "$@" diff --git a/bin/pos-ai-openrouter b/bin/pos-ai-openrouter index 6be9586..b1d5776 100755 --- a/bin/pos-ai-openrouter +++ b/bin/pos-ai-openrouter @@ -1,7 +1,7 @@ #!/usr/bin/env bash set -euo pipefail # POS: ai openrouter — Forward to pos ai --provider openrouter (backward compat) -# POS_SUBCMDS: ask chat sessions capture +# POS_SUBCMDS: ask chat sessions capture models providers # Thin forwarder — all logic lives in bin/pos-ai + lib/ai-providers/openrouter.sh case "${1:-}" in -h|--help) exec pos ai --provider openrouter --help ;; esac exec pos ai --provider openrouter "$@" diff --git a/bin/pos-ai-server b/bin/pos-ai-server index 0756b94..4a6d596 100755 --- a/bin/pos-ai-server +++ b/bin/pos-ai-server @@ -7,35 +7,50 @@ set -euo pipefail source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + # ── Dependencies (before --help) ─────────────────────────────── command -v curl &>/dev/null || err "curl not found (install curl)" command -v jq &>/dev/null || err "jq not found (install jq)" # ── Config / seams ───────────────────────────────────────────── -CONFIG_FILE="${CONFIG_FILE:-$HOME/.config/linux_post_install/ai.env}" +CONFIG_FILE="${CONFIG_FILE:-$CONFIG_DIR/ai.env}" USER_SYSTEMD_DIR="${USER_SYSTEMD_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user}" SERVICE="pos-ai-server.service" HF_DOWNLOAD_DIR="${HF_DOWNLOAD_DIR:-$HOME/.local/share/linux_post_install/ai/models}" -# ── Config loader (env-var precedence, same pattern as pos-ai-hf) ── +# ── Config loader (canonical env-var precedence, same pattern as pos-ai-hf) ── +# Loaded keys are also recorded in LOADED_ENV_KEYS (see below); the D-F +# requested-from-config tracking uses the exported values, so it is +# unaffected by the loader implementation. load_config() { - [ -f "$CONFIG_FILE" ] || return 0 - local k v - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - v="${v//$'\r'/}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) + load_env_file "$CONFIG_FILE" } load_config +# ── Config/env-requested flags (D-F) ─────────────────────────── +# Flags the user explicitly configured (via ai.env or exported env) are +# validated as "requested" (hard error if unsupported), distinct from the +# always-emitted tool defaults (warn + omit). This is captured here, BEFORE +# the CLI parse overwrites the LLAMACPP_* variables, so it reflects genuine +# user intent rather than the final defaulted values. +CONFIG_REQUESTED_FLAGS=() +requested_from_env_config() { + local var="$1" flag="$2" + [ -n "${!var:-}" ] || return 0 + case " ${CONFIG_REQUESTED_FLAGS[*]:-} " in + *" $flag "*) ;; # dedupe + *) CONFIG_REQUESTED_FLAGS+=("$flag") ;; + esac +} +requested_from_env_config LLAMACPP_PORT --port +requested_from_env_config LLAMACPP_HOST --host +requested_from_env_config LLAMACPP_CTX_SIZE --ctx-size +requested_from_env_config LLAMACPP_GPU_LAYERS --n-gpu-layers +requested_from_env_config LLAMACPP_THREADS --threads + # ── Binary detection ─────────────────────────────────────────── find_llamacpp() { local candidates=("llama-server" "llama.cpp/server" "server" "llama-server-cuda") @@ -80,12 +95,62 @@ validate_requested_flags() { *" $flag "*) continue ;; # dedupe alias-mapped flags (e.g. --gpu → --n-gpu-layers) esac seen+=("$flag") - if ! printf '%s' "$help_text" | grep -qF -- "$flag"; then + # Word-boundary match: the flag token must appear as a whole word in + # --help, not as a substring of a longer flag (D4 — e.g. --mmap must + # not match a --no-mmap entry). + if ! printf '%s' "$help_text" | grep -qE -- "(^|[[:space:]])${flag}([[:space:]]|=|$)"; then err "installed llama.cpp ${version} does not expose ${flag} — remove it or upgrade llama.cpp" fi done } +# ── Validate always-emitted default flags (D-F) ──────────────── +# validate_default_flags — for every +# flag the tool emits BY DEFAULT (with no explicit user intent), check it is +# supported in --help. Requested flags (CLI or config/env) are excluded — they +# were hard-validated upstream and must be kept. An unsupported DEFAULT is +# omitted from ExecStart with a single warn (never a hard error — the tool +# chose the default, not the user). Readable --help sets globals: +# DEFAULT_PORT_OK DEFAULT_HOST_OK DEFAULT_GPU_OK DEFAULT_CTX_OK DEFAULT_THREADS_OK +# (all 1 = keep; 0 = omit). Unreadable --help text sets all OK = 1 (warn). +validate_default_flags() { + local bin="$1" version="$2" + shift 2 + local requested=("$@") + local requested_str=" ${requested[*]:-} " + + DEFAULT_PORT_OK=1; DEFAULT_HOST_OK=1; DEFAULT_GPU_OK=1; DEFAULT_CTX_OK=1; DEFAULT_THREADS_OK=1 + + local help_text + help_text="$("$bin" --help 2>/dev/null)" || { + warn "Cannot obtain llama-server --help output — skipping default flag validation" + return 0 + } + + local spec flag ok_var + # flag | ok_var — the always-emitted defaults a user may not have requested. + local specs=( + "--port|DEFAULT_PORT_OK" + "--host|DEFAULT_HOST_OK" + "--n-gpu-layers|DEFAULT_GPU_OK" + "--ctx-size|DEFAULT_CTX_OK" + "--threads|DEFAULT_THREADS_OK" + ) + for spec in "${specs[@]}"; do + flag="${spec%%|*}" + ok_var="${spec#*|}" + case "$requested_str" in + *" $flag "*) continue ;; # requested → already hard-validated, keep + esac + if printf '%s' "$help_text" | grep -qE -- "(^|[[:space:]])${flag}([[:space:]]|=|$)"; then + eval "$ok_var=1" + else + warn "installed llama.cpp ${version} does not support default flag ${flag} — omitting it from the unit" + eval "$ok_var=0" + fi + done +} + # ── GPU detection ────────────────────────────────────────────── detect_gpu() { if command -v nvidia-smi &>/dev/null && nvidia-smi &>/dev/null 2>&1; then @@ -126,9 +191,11 @@ human_size() { # ── Health check ─────────────────────────────────────────────── check_health() { - local port="${LLAMACPP_PORT:-8088}" + # Probe the SAME host/port the unit binds (HOST/PORT, defaults + # 127.0.0.1/8088) — previously probes hardcoded 127.0.0.1 and missed a + # non-localhost LLAMACPP_HOST bind. local resp - resp="$(curl -sf "http://127.0.0.1:$port/health" 2>/dev/null)" || { echo "not running"; return 1; } + resp="$(curl -sf "http://$HOST:$PORT/health" 2>/dev/null)" || { echo "not running"; return 1; } local status status="$(printf '%s' "$resp" | jq -r '.status // "unknown"' 2>/dev/null)" echo "$status" @@ -253,7 +320,7 @@ Config (~/.config/linux_post_install/ai.env): LLAMACPP_GPU_LAYERS GPU layers: -1=auto, 0=CPU only (default -1) LLAMACPP_THREADS CPU threads (default: nproc) -Requires: llama-server binary (install llama.cpp: https://github.com/ggerganov/llama.cpp) +Requires: llama-server binary — install llama.cpp with the app installer: 'apps/ai/llamacpp.sh' (run 'bash apps/install.sh llamacpp', or pass '--apps'/'--full' to install.sh), see 'pos help ai server' (https://github.com/ggerganov/llama.cpp) EOF exit 0 } @@ -394,7 +461,7 @@ systemd_quote() { cmd_start() { # Resolve the llama-server binary local llamacpp_bin - llamacpp_bin="$(find_llamacpp)" || err "llama-server not found — install llama.cpp (https://github.com/ggerganov/llama.cpp)" + llamacpp_bin="$(find_llamacpp)" || err "llama-server not found — install llama.cpp with the app installer: 'apps/ai/llamacpp.sh' (run 'bash apps/install.sh llamacpp', or pass '--apps'/'--full' to install.sh), see 'pos help ai server' (https://github.com/ggerganov/llama.cpp)" local llamacpp_full llamacpp_full="$(command -v "$llamacpp_bin")" @@ -403,10 +470,21 @@ cmd_start() { local version version="$(detect_llama_version "$llamacpp_bin")" - # Validate explicitly requested flags against this binary's --help - if [ "${#REQUESTED_FLAGS[@]}" -gt 0 ]; then - validate_requested_flags "$llamacpp_bin" "$version" "${REQUESTED_FLAGS[@]}" + # Validate EVERY flag that will appear in ExecStart (D-F): + # - Requested flags (CLI OR config/env) → hard error if unsupported. + # - Always-emitted defaults → warn + omit if unsupported. + local all_requested=("${REQUESTED_FLAGS[@]}" "${CONFIG_REQUESTED_FLAGS[@]}") + local deduped=() flag + for flag in "${all_requested[@]}"; do + case " ${deduped[*]:-} " in + *" $flag "*) continue ;; + esac + deduped+=("$flag") + done + if [ "${#deduped[@]}" -gt 0 ]; then + validate_requested_flags "$llamacpp_bin" "$version" "${deduped[@]}" fi + validate_default_flags "$llamacpp_bin" "$version" "${deduped[@]}" # Resolve model local explicit_model="${SUBCMD_ARGS[0]:-}" @@ -441,11 +519,17 @@ cmd_start() { # unquoted whitespace, so the binary and the model path — the only tokens # that may contain spaces — are systemd_quote()d; plain flag/number # tokens stay unquoted. + # Build the command with ONLY the flags that passed validation. Requested + # flags (hard-validated) and supported defaults are always emitted; an + # unsupported DEFAULT is omitted here (validate_default_flags set the + # DEFAULT_*_OK globals) so the unit never carries an unsupported flag. local exec_cmd - exec_cmd="$(systemd_quote "$llamacpp_full") -m $(systemd_quote "$model") --port $PORT --host $HOST" - exec_cmd+=" --n-gpu-layers $gpu_layers" - exec_cmd+=" --ctx-size $CTX_SIZE" - exec_cmd+=" --threads $THREADS" + exec_cmd="$(systemd_quote "$llamacpp_full") -m $(systemd_quote "$model")" + if [ "$DEFAULT_PORT_OK" -eq 1 ]; then exec_cmd+=" --port $PORT"; fi + if [ "$DEFAULT_HOST_OK" -eq 1 ]; then exec_cmd+=" --host $HOST"; fi + if [ "$DEFAULT_GPU_OK" -eq 1 ]; then exec_cmd+=" --n-gpu-layers $gpu_layers"; fi + if [ "$DEFAULT_CTX_OK" -eq 1 ]; then exec_cmd+=" --ctx-size $CTX_SIZE"; fi + if [ "$DEFAULT_THREADS_OK" -eq 1 ]; then exec_cmd+=" --threads $THREADS"; fi if [ -n "$GPU_THREADS" ]; then exec_cmd+=" --gpu-threads $GPU_THREADS" fi @@ -560,7 +644,7 @@ cmd_status() { # llama-server must be present for the version probe below — same # actionable deps message as `start` if ! find_llamacpp >/dev/null 2>&1; then - err "llama-server not found — install llama.cpp (https://github.com/ggerganov/llama.cpp)" + err "llama-server not found — install llama.cpp with the app installer: 'apps/ai/llamacpp.sh' (run 'bash apps/install.sh llamacpp', or pass '--apps'/'--full' to install.sh), see 'pos help ai server' (https://github.com/ggerganov/llama.cpp)" fi # Service state @@ -573,7 +657,7 @@ cmd_status() { # Model (from health endpoint if running) if [ "$svc_state" = "running" ]; then local models_resp - models_resp="$(curl -sf "http://127.0.0.1:$PORT/v1/models" 2>/dev/null)" || true + models_resp="$(curl -sf "http://$HOST:$PORT/v1/models" 2>/dev/null)" || true local model_id model_id="$(printf '%s' "$models_resp" | jq -r '.data[0].id // "unknown"' 2>/dev/null)" || model_id="unknown" printf 'model: %s\n' "$model_id" diff --git a/bin/pos-communication-matrix-listener b/bin/pos-communication-matrix-listener index 70b5532..5f55952 100755 --- a/bin/pos-communication-matrix-listener +++ b/bin/pos-communication-matrix-listener @@ -9,6 +9,9 @@ MAP_FILE="$CONFIG_DIR/matrix_commands.env" SERVICE="pos-matrix-listener.service" USER_SYSTEMD_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + # System prompt for the "ai " bridge: replies are posted straight into the # room, so ask for concise, emoji-friendly Matrix-style answers. AI_SYSTEM="You are a friendly assistant chatting in a Matrix room. Keep replies concise, use emojis and light formatting to make them lively, and never claim to send messages yourself." @@ -31,7 +34,8 @@ Commands: --run Run the polling loop in the foreground (used by the service) Config: $CONFIG_FILE (MATRIX_HOMESERVER, MATRIX_ACCESS_TOKEN, - MATRIX_USER_ID — edit with 'pos config matrix') + MATRIX_USER_ID, MATRIX_ROOM_ID — edit with + 'pos config matrix') Map: $MAP_FILE — '/cmd=bash command' per line (optional '/cmd::short description=bash command') @@ -44,10 +48,11 @@ without '@quiet' and the listener forwards the output: '/status=pos system health' The listener only reacts to messages from your own Matrix user -(MATRIX_USER_ID). If MATRIX_ROOM_ID is set it only watches that room, -otherwise it watches every room you've joined. Commands run as your user, -so 'sudo' inside them needs a NOPASSWD rule. The interactive editor runs -'bash -n' to syntax-check commands before saving. +(MATRIX_USER_ID), and only in the room set by MATRIX_ROOM_ID — without it +the listener refuses to run any command (fail-closed), so a bot account +that has joined many rooms can't be tricked into executing commands. +Commands run as your user, so 'sudo' inside them needs a NOPASSWD rule. +The interactive editor runs 'bash -n' to syntax-check commands before saving. Commands are matched with a leading '/' or '!' — '/status' and '!status' both resolve. There is no equivalent of Telegram's bot "/" menu on Matrix, @@ -63,18 +68,7 @@ EOF # ── matrix.env (same pattern as pos-communication-matrix-sender) ── load_config() { - [ -f "$CONFIG_FILE" ] || return 0 - local k v - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) + load_env_file "$CONFIG_FILE" } api_base() { @@ -469,7 +463,7 @@ handle_message() { return fi log "ai: $prompt" - if answer="$(timeout 120 pos ai gemini ask --session "$session" --system "$AI_SYSTEM" "$prompt" 2>&1)"; then + if answer="$(timeout 120 pos ai gemini ask --no-command-execution --session "$session" --system "$AI_SYSTEM" "$prompt" 2>&1)"; then reply_room "$room" "$(strip_markdown "$answer")" "$event_id" else [ -n "$answer" ] || answer="timed out after 120s" @@ -513,6 +507,9 @@ run_daemon() { [ -n "${MATRIX_ACCESS_TOKEN:-}" ] || err "No access token — run 'pos communication matrix sender login'" local owner="${MATRIX_USER_ID:-}" local room_only="${MATRIX_ROOM_ID:-}" + if [ -z "$room_only" ]; then + warn "MATRIX_ROOM_ID unset — refusing to run commands (fail-closed); set it with 'pos config matrix'" + fi if [ -z "$owner" ]; then local who who="$(curl -fsS -m 30 -H "Authorization: Bearer ${MATRIX_ACCESS_TOKEN}" \ @@ -524,7 +521,7 @@ run_daemon() { local since="" filter_enc resp nb filter_enc="$(printf '%s' "$SYNC_FILTER" | jq -sRr @uri)" - log "listener running (owner ${owner}${room_only:+ — room ${room_only}}) — Ctrl+C to stop" + log "listener running (owner ${owner}, room ${room_only:-none — fail-closed}) — Ctrl+C to stop" trap 'kill $(jobs -p) 2>/dev/null; exit 0' TERM INT while true; do local args=("$(api_base)/_matrix/client/v3/sync?timeout=30000&filter=${filter_enc}") @@ -536,7 +533,9 @@ run_daemon() { local room ecount i roomid ev sender text eid for room in $(printf '%s' "$resp" | jq -r '.rooms.join // {} | to_entries[] | .key' 2>/dev/null); do - if [ -n "$room_only" ] && [ "$room" != "$room_only" ]; then + # Fail-closed: with MATRIX_ROOM_ID unset no room is watched, so no + # command can ever run (previously every joined room was watched). + if [ -z "$room_only" ] || [ "$room" != "$room_only" ]; then continue fi ecount="$(printf '%s' "$resp" | jq -r --arg r "$room" ".rooms.join[\"$room\"].timeline.events // [] | length")" diff --git a/bin/pos-communication-matrix-sender b/bin/pos-communication-matrix-sender index acdbe25..744e1e8 100755 --- a/bin/pos-communication-matrix-sender +++ b/bin/pos-communication-matrix-sender @@ -7,6 +7,9 @@ set -euo pipefail CONFIG_DIR="${CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/linux_post_install}" CONFIG_FILE="$CONFIG_DIR/matrix.env" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + usage() { cat <&2; exit 1; } log() { echo "[+] $*"; } load_config() { - [ -f "$CONFIG_FILE" ] || return 0 - local k v - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - v="${v//$'\r'/}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) + load_env_file "$CONFIG_FILE" } save_config() { diff --git a/bin/pos-communication-scrcpy b/bin/pos-communication-scrcpy index 9219a82..2ea59b6 100755 --- a/bin/pos-communication-scrcpy +++ b/bin/pos-communication-scrcpy @@ -6,25 +6,16 @@ set -euo pipefail source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh" -CONFIG_FILE="$HOME/.config/linux_post_install/scrcpy.env" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + +CONFIG_FILE="$CONFIG_DIR/scrcpy.env" command -v scrcpy &>/dev/null || err "scrcpy not found — install the latest release with the app installer: 'apps/media/scrcpy.sh' (or 'sudo apt install scrcpy' if your distro ships it; it bundles adb), see 'pos help communication scrcpy'" command -v adb &>/dev/null || err "adb not found — install it: 'sudo apt install adb'" load_config() { - [ -f "$CONFIG_FILE" ] || return 0 - local k v - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - v="${v//$'\r'/}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) + load_env_file "$CONFIG_FILE" } load_config diff --git a/bin/pos-communication-telegram-listener b/bin/pos-communication-telegram-listener index 0435931..895e0cf 100755 --- a/bin/pos-communication-telegram-listener +++ b/bin/pos-communication-telegram-listener @@ -12,6 +12,9 @@ API="https://api.telegram.org" SERVICE="pos-telegram-listener.service" USER_SYSTEMD_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + # System prompt for the " " AI bridge (default prefix: "ai"): replies # are posted straight into the chat, so ask for concise, emoji-friendly # Telegram-style answers. The trigger word is configurable via @@ -47,7 +50,8 @@ Commands: is set via 'pos config telegram' (TELEGRAM_AI_PREFIX). Config: $CONFIG_FILE (TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, - TELEGRAM_AI_PREFIX — edit with 'pos config telegram') + TELEGRAM_OWNER_ID, TELEGRAM_AI_PREFIX — edit with 'pos + config telegram') Map: $MAP_FILE — '/cmd=bash command' per line (optional '/cmd::short description=bash command' shown in the bot menu) Prefix map: $PREFIX_FILE — '=command' per line: a non-command @@ -65,7 +69,11 @@ To reply with a command's stdout (e.g. the health dashboard), just map it without '@quiet' and the listener forwards the output: '/status=pos system health' -The listener only reacts to the owner chat (TELEGRAM_CHAT_ID). Commands run +The listener only reacts to messages sent to the owner chat +(TELEGRAM_CHAT_ID) BY your account (TELEGRAM_OWNER_ID) — both must match, +so an impersonator or a forwarded message can't trigger commands. If +TELEGRAM_OWNER_ID is unset the daemon starts but refuses to run any chat +command (fail-closed); set it with 'pos config telegram'. Commands run as your user, so 'sudo' inside them needs a NOPASSWD rule. The interactive editor runs 'bash -n' to syntax-check commands before saving. @@ -84,18 +92,7 @@ EOF # ── telegram.env (same pattern as pos-communication-telegram) ──── load_config() { - [ -f "$CONFIG_FILE" ] || return 0 - local k v - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) + load_env_file "$CONFIG_FILE" } # ── command map (MAP_FILE) ────────────────────────────────────── @@ -731,7 +728,7 @@ handle_message() { if [ -n "$reply_text" ]; then prompt="[Reply context — the message you are replying to]\n${reply_text}\n\n${prompt}" fi - if answer="$(timeout 120 pos ai gemini ask --session "$session" --system "$AI_SYSTEM" "$prompt" 2>&1)"; then + if answer="$(timeout 120 pos ai gemini ask --no-command-execution --session "$session" --system "$AI_SYSTEM" "$prompt" 2>&1)"; then reply "$(strip_markdown "$answer")" "$msg_id" else [ -n "$answer" ] || answer="timed out after 120s" @@ -758,10 +755,15 @@ run_daemon() { load_config [ -n "${TELEGRAM_BOT_TOKEN:-}" ] || err "No bot token — run 'pos config telegram'" [ -n "${TELEGRAM_CHAT_ID:-}" ] || err "No chat id — run 'pos config telegram'" + if [ -n "${TELEGRAM_OWNER_ID:-}" ]; then + log "owner id ${TELEGRAM_OWNER_ID} — commands authorized" + else + warn "TELEGRAM_OWNER_ID unset — chat commands WILL BE IGNORED (fail-closed); set it with 'pos config telegram'" + fi sync_bot_commands || true local offset=0 - log "listener running (owner chat ${TELEGRAM_CHAT_ID}) — Ctrl+C to stop" + log "listener running (chat ${TELEGRAM_CHAT_ID}, owner ${TELEGRAM_OWNER_ID:-unset}) — Ctrl+C to stop" trap 'kill $(jobs -p) 2>/dev/null; exit 0' TERM INT while true; do local resp n i @@ -784,7 +786,15 @@ run_daemon() { reply_text="$(printf '%s' "$resp" | jq -r ".result[$i].message.reply_to_message.text // .result[$i].message.reply_to_message.caption // empty")" offset=$((u + 1)) [ -n "$text" ] || continue - if [ -n "$chat" ] && [ "$chat" != "$TELEGRAM_CHAT_ID" ] && [ "$from_id" != "$TELEGRAM_CHAT_ID" ]; then + if [ -z "${TELEGRAM_OWNER_ID:-}" ]; then + warn "TELEGRAM_OWNER_ID unset — ignoring command (set it with 'pos config telegram')" + continue + fi + # Fail-closed owner check: the message must be in the owner chat + # AND sent by the owner account. Previously a chat-id OR sender-id + # match was enough — anyone who knew the chat id could run commands. + if [ "$chat" != "$TELEGRAM_CHAT_ID" ] || [ "$from_id" != "$TELEGRAM_OWNER_ID" ]; then + warn "ignoring message in chat ${chat:-?} from ${from_id:-?} (not the owner chat/account)" continue fi handle_message "$text" "$msg_id" "$reply_text" diff --git a/bin/pos-communication-telegram-sender b/bin/pos-communication-telegram-sender index 2531faa..d9be649 100755 --- a/bin/pos-communication-telegram-sender +++ b/bin/pos-communication-telegram-sender @@ -3,12 +3,15 @@ set -euo pipefail # POS: communication telegram-sender — Send Telegram messages/files/links/stickers via Bot API (send, test) # POS_FLAGS: --type --caption --parse-mode --no-preview --token --chat-id --markdown # POS_SUBCMDS: send test -# POS_CONFIG: telegram | telegram.env | TELEGRAM_BOT_TOKEN=secret:Bot token from @BotFather | TELEGRAM_CHAT_ID=digits:Numeric chat id from @userinfobot | TELEGRAM_AI_PREFIX=:AI-bridge trigger word in the telegram listener (default ai)::ai +# POS_CONFIG: telegram | telegram.env | TELEGRAM_BOT_TOKEN=secret:Bot token from @BotFather | TELEGRAM_CHAT_ID=digits:Numeric chat id from @userinfobot | TELEGRAM_OWNER_ID=digits:Numeric Telegram user id (your account) allowed to run chat commands | TELEGRAM_AI_PREFIX=:AI-bridge trigger word in the telegram listener (default ai)::ai CONFIG_DIR="${CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/linux_post_install}" CONFIG_FILE="$CONFIG_DIR/telegram.env" API="https://api.telegram.org" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + usage() { cat < Override token for one send --chat-id Override chat id for one send -Config: $CONFIG_FILE (TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID) - edit it with 'pos config telegram' +Config: $CONFIG_FILE (TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, + TELEGRAM_OWNER_ID — edit it with 'pos config telegram') Precedence: CLI flags > environment > config file. @@ -59,19 +62,7 @@ EOF err() { echo "ERROR: $*" >&2; exit 1; } load_config() { - [ -f "$CONFIG_FILE" ] || return 0 - local k v - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - v="${v//$'\r'/}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) + load_env_file "$CONFIG_FILE" } send_request() { diff --git a/bin/pos-media-grab b/bin/pos-media-grab index 39a27e5..d0da341 100755 --- a/bin/pos-media-grab +++ b/bin/pos-media-grab @@ -6,20 +6,12 @@ set -euo pipefail source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + # Load grab.env config (env-seam: GRAB_DEFAULT) load_grab_config() { - local f="$HOME/.config/linux_post_install/grab.env" k v - [ -f "$f" ] || return 0 - while IFS='=' read -r k v; do - [ -n "$k" ] || continue - case "$k" in - \#*) continue ;; - esac - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - if [ -z "${!k:-}" ]; then - export "$k"="$v" - fi - done < <(grep -E '^[A-Z_]+=' "$f" || true) + load_env_file "$CONFIG_DIR/grab.env" } load_grab_config diff --git a/bin/pos-network-checkport b/bin/pos-network-checkport index d531ec3..9bba25b 100755 --- a/bin/pos-network-checkport +++ b/bin/pos-network-checkport @@ -130,7 +130,9 @@ port_meta() { # --- fallback engine (no nmap): bash /dev/tcp + nc --------------------------- check_tcp() { local ip="$1" port="$2" tmo="$3" rc="" - timeout "$tmo" bash -c "exec 3<>/dev/tcp/$ip/$port" 2>/dev/null + # host/port are positional args ($1/$2), never interpolated into the + # command source — a hostile host string stays a literal argument. + timeout "$tmo" bash -c 'exec 3<>/dev/tcp/$1/$2' _ "$ip" "$port" 2>/dev/null rc=$? if [ "$rc" -eq 0 ]; then printf 'open' elif [ "$rc" -eq 124 ]; then printf 'timeout' @@ -154,7 +156,7 @@ check_udp() { elif printf '%s' "$out" | grep -qiE 'refus|unreachable'; then printf 'closed' else printf 'no-reply'; fi else - timeout "$tmo" bash -c "exec 3<>/dev/udp/$ip/$port; printf 'x' >&3" 2>/dev/null + timeout "$tmo" bash -c 'exec 3<>/dev/udp/$1/$2; printf "x" >&3' _ "$ip" "$port" 2>/dev/null printf 'no-reply' fi } @@ -163,9 +165,9 @@ probe() { local ip="$1" port="$2" tmo="$3" case "$port" in 80|8000|8080|8081|8888|3000|5000|9090|3128|10000) - timeout "$tmo" bash -c "exec 3<>/dev/tcp/$ip/$port; printf 'HEAD / HTTP/1.0\r\n\r\n' >&3; head -c 300 <&3" 2>/dev/null ;; + timeout "$tmo" bash -c 'exec 3<>/dev/tcp/$1/$2; printf "HEAD / HTTP/1.0\r\n\r\n" >&3; head -c 300 <&3' _ "$ip" "$port" 2>/dev/null ;; *) - timeout "$tmo" bash -c "exec 3<>/dev/tcp/$ip/$port; head -c 200 <&3" 2>/dev/null ;; + timeout "$tmo" bash -c 'exec 3<>/dev/tcp/$1/$2; head -c 200 <&3' _ "$ip" "$port" 2>/dev/null ;; esac } diff --git a/bin/pos-network-download b/bin/pos-network-download index e397a73..54f2765 100755 --- a/bin/pos-network-download +++ b/bin/pos-network-download @@ -11,6 +11,9 @@ set -euo pipefail source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh" source "$(dirname "$0")/../lib/menu-lib.sh" 2>/dev/null || source "$(dirname "$0")/menu-lib.sh" +# Shared config loader (canonical KEY=VALUE parser, env-wins precedence) +source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh" + command -v aria2c &>/dev/null || err "aria2c not found (install aria2)" command -v jq &>/dev/null || err "jq not found (install jq)" command -v curl &>/dev/null || err "curl not found (install curl)" @@ -25,14 +28,13 @@ SERVICE="pos-aria2.service" HEALER_SERVICE="pos-aria2-retry.service" HEALER_TIMER="pos-aria2-retry.timer" RETRY_STATE="$CONFIG_DIR/download.retry" -NET_PROBE="${NET_PROBE:-timeout 3 bash -c ''}" +NET_PROBE="${NET_PROBE:-timeout 3 bash -c 'exec 3<>/dev/tcp/\$1/\$2' _ 8.8.8.8 53}" RETRY_INTERVAL="${RETRY_INTERVAL:-30}" RETRY_VERIFY_SLEEP="${RETRY_VERIFY_SLEEP:-3}" load_secret() { - if [ -z "${RPC_SECRET:-}" ] && [ -f "$CONFIG_FILE" ]; then - RPC_SECRET=$(grep -E '^RPC_SECRET=' "$CONFIG_FILE" 2>/dev/null | head -1 | cut -d= -f2-) - fi + # Canonical loader: env-wins, so an exported RPC_SECRET always beats the file. + load_env_file "$CONFIG_FILE" RPC_SECRET="${RPC_SECRET:-}" } load_secret @@ -150,8 +152,8 @@ rpc() { # rpc [json-args...] daemon_active() { systemctl --user is-active --quiet "$SERVICE" 2>/dev/null; } cmd_start() { - if [ -z "$RPC_SECRET" ] && [ -f "$CONFIG_FILE" ]; then - RPC_SECRET=$(grep -E '^RPC_SECRET=' "$CONFIG_FILE" 2>/dev/null | head -1 | cut -d= -f2-) + if [ -z "$RPC_SECRET" ]; then + load_secret fi if [ -z "$RPC_SECRET" ]; then RPC_SECRET=$(head -c 24 /dev/urandom | od -An -tx1 | tr -d ' \n') diff --git a/bin/pos-share-smb-client b/bin/pos-share-smb-client index ec8ea69..0180c6b 100755 --- a/bin/pos-share-smb-client +++ b/bin/pos-share-smb-client @@ -89,7 +89,9 @@ split_share() { # TCP connect probe against the server before touching passwords or mounts. probe_server() { local host="$1" - if timeout 3 bash -c "exec 3<>/dev/tcp/${host}/${SMB_PORT}" 2>/dev/null; then + # host/port are positional args, never interpolated into the command + # source — a hostile share/address string stays a literal argument. + if timeout 3 bash -c 'exec 3<>/dev/tcp/$1/$2' _ "$host" "$SMB_PORT" 2>/dev/null; then return 0 fi err "Server ${host} not reachable on port ${SMB_PORT} (SMB) — nothing was mounted. diff --git a/bin/pos-system-backup b/bin/pos-system-backup index 3354661..a50329d 100755 --- a/bin/pos-system-backup +++ b/bin/pos-system-backup @@ -192,14 +192,22 @@ run_backup() { unset CONFIRM log "Encrypting backup..." - gpg --batch --yes --passphrase "$PASS" --symmetric --cipher-algo AES256 "$ARCHIVE" + # The passphrase is fed on fd 3 (here-string), never via argv, so it + # cannot leak through 'ps' / /proc//cmdline while gpg runs. + if ! gpg --batch --yes --passphrase-fd 3 --symmetric --cipher-algo AES256 "$ARCHIVE" 3<<<"$PASS"; then + rm -f "$ARCHIVE" + err "encryption failed — plaintext archive removed, nothing left behind" + fi rm -f "$ARCHIVE" ARCHIVE="${ARCHIVE}.gpg" chmod 600 "$ARCHIVE" log "Verifying encrypted backup..." - gpg --batch --quiet --passphrase "$PASS" --decrypt "$ARCHIVE" | tar -tzf - > /dev/null + if ! gpg --batch --quiet --passphrase-fd 3 --decrypt "$ARCHIVE" 3<<<"$PASS" | tar -tzf - > /dev/null; then + rm -f "$ARCHIVE" + err "decryption verification failed — corrupt artifact removed" + fi unset PASS else diff --git a/bin/pos-system-uninstall b/bin/pos-system-uninstall index dd6dc32..dc46a9d 100755 --- a/bin/pos-system-uninstall +++ b/bin/pos-system-uninstall @@ -25,6 +25,14 @@ Examples: pos system uninstall # interactive, tier 1 only pos system uninstall --yes # non-interactive, tier 1 only pos system uninstall --yes --config --data # nuclear option + +Intentionally NOT removed (user-managed): + - apt packages (system packages installed by preinstall.sh) + - /usr/local/bin/yt-dlp (manually installed) + - ~/.config/rclone/ (rclone manages its own config) + - ~/.ssh/authorized_keys additions (user SSH access) + - pos-owned config files (removed by --config tier) + - pos-owned data files (removed by --data tier) EOF exit 0 } @@ -44,6 +52,27 @@ while [ $# -gt 0 ]; do esac done +# ── pos-owned installed files (single source of truth, mirrors install.sh) ── +# lib/*.sh list shipped by install.sh phase 2 to /usr/local/bin. +POS_LIBS=(common.sh flags.sh notify.sh entertainment-lib.sh scheduler-lib.sh config-ui.sh \ + user-timers-lib.sh entertainment-plugin-lib.sh usb-lib.sh share-lib.sh menu-lib.sh registry.sh) +# Exact lines postinstall.sh appends to ~/.bashrc. +PATH_LINE='export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$HOME/.local/bin:$PATH"' +COMPLETION_LINE='source /usr/local/share/bash-completion/completions/pos.bash 2>/dev/null || true' +# Pos-ai-hook source lines are not written by the installer (user/legacy-added), +# so they are matched by an anchored source-pattern, never a bare substring. +HOOK_PATTERN='^[[:space:]]*source[[:space:]].*pos-ai-hook\.sh' + +# Entertainment plugins installed by install.sh — discovered by POS_PLUGIN marker +# so future plugins are removed without touching this file. +installed_plugins() { + local ep + for ep in /usr/local/bin/*.sh; do + [ -f "$ep" ] || continue + grep -q '^# POS_PLUGIN:' "$ep" 2>/dev/null && echo "$ep" + done | sort +} + # ── Scan functions ────────────────────────────────────────────── scan_tier1() { local found=() @@ -58,20 +87,18 @@ scan_tier1() { found+=("$f") done < <(compgen -G /usr/local/bin/pos-* 2>/dev/null | sort || true) - # Lib files shipped by install.sh - for f in common.sh menu-lib.sh share-lib.sh; do + # Lib files shipped by install.sh (single source: POS_LIBS) + for f in "${POS_LIBS[@]}"; do [ -f "/usr/local/bin/$f" ] && found+=("/usr/local/bin/$f") done # AI providers subdirectory [ -d /usr/local/bin/ai-providers ] && found+=("/usr/local/bin/ai-providers/") - # Entertainment plugins installed by install.sh + # Entertainment plugins installed by install.sh (discovered by POS_PLUGIN marker) while IFS= read -r f; do - found+=("$f") - done < <(for ep in weather.sh gold.sh joke.sh; do - [ -f "/usr/local/bin/$ep" ] && echo "/usr/local/bin/$ep" - done | sort) + [ -n "$f" ] && found+=("$f") + done < <(installed_plugins) # Legacy forwarders for f in wr-* mp3 mp4 vbox ssh-load-all; do @@ -107,11 +134,11 @@ scan_tier1() { # Completion file [ -f /usr/local/share/bash-completion/completions/pos.bash ] && found+=("/usr/local/share/bash-completion/completions/pos.bash") - # ~/.bash_completion entries + # ~/.bash_completion entries (anchored source lines only, never /pos/ substring) if [ -f "$HOME/.bash_completion" ]; then while IFS= read -r line; do found+=("~/.bash_completion: $(echo "$line" | sed 's/^[[:space:]]*//' | cut -c1-70)") - done < <(grep -n 'pos' "$HOME/.bash_completion" 2>/dev/null || true) + done < <(grep -nE '^[[:space:]]*source[[:space:]].*pos\.bash' "$HOME/.bash_completion" 2>/dev/null || true) fi # ── Systemd services ── @@ -134,11 +161,24 @@ scan_tier1() { [ "$already" -eq 0 ] && found+=("service: $svc_name") done < <(systemctl list-unit-files --type=service 2>/dev/null | grep -i 'linux_post_install\|pos-' || true) - # ── Shell integration (~/.bashrc) ── + # ── ScaleTail templates + feature-flag store (install.sh phase 2/4) ── + [ -d /usr/local/share/linux_post_install/scale-tail ] && found+=("/usr/local/share/linux_post_install/scale-tail/") + [ -d /usr/local/share/linux_post_install/flags ] && found+=("/usr/local/share/linux_post_install/flags/") + + # ── Runtime-created user units (pos-* in ~/.config/systemd/user) ── + local user_unit_dir="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user" + if [ -d "$user_unit_dir" ]; then + local ufile + while IFS= read -r ufile; do + [ -n "$ufile" ] && found+=("user-unit: $(basename "$ufile")") + done < <(find "$user_unit_dir" -maxdepth 1 -name 'pos-*' -type f 2>/dev/null || true) + fi + + # ── Shell integration (~/.bashrc) — exact installer lines + anchored hook ── if [ -f "$HOME/.bashrc" ]; then while IFS= read -r line; do found+=("~/.bashrc: $(echo "$line" | sed 's/^[[:space:]]*//' | cut -c1-70)") - done < <(grep -n 'source.*pos-ai-hook\|linux_post_install.*PATH\|source.*pos\.bash\|pos completion' "$HOME/.bashrc" 2>/dev/null || true) + done < <({ grep -nF "$PATH_LINE" "$HOME/.bashrc"; grep -nF "$COMPLETION_LINE" "$HOME/.bashrc"; grep -nE "$HOOK_PATTERN" "$HOME/.bashrc"; } 2>/dev/null | sort -u || true) fi printf '%s\n' "${found[@]}" @@ -234,9 +274,9 @@ remove_tier1() { [ -f "$f" ] && { rm -f "$f" && count=$((count+1)); } done < <(compgen -G /usr/local/bin/pos-* 2>/dev/null | sort || true) - # Lib files - for f in /usr/local/bin/common.sh /usr/local/bin/menu-lib.sh /usr/local/bin/share-lib.sh; do - [ -f "$f" ] && { rm -f "$f" && count=$((count+1)); } + # Lib files (single source: POS_LIBS) + for f in "${POS_LIBS[@]}"; do + [ -f "/usr/local/bin/$f" ] && { rm -f "/usr/local/bin/$f" && count=$((count+1)); } done # AI providers directory @@ -244,10 +284,10 @@ remove_tier1() { rm -rf /usr/local/bin/ai-providers && count=$((count+1)) fi - # Entertainment plugins - for f in /usr/local/bin/weather.sh /usr/local/bin/gold.sh /usr/local/bin/joke.sh; do - [ -f "$f" ] && { rm -f "$f" && count=$((count+1)); } - done + # Entertainment plugins (discovered by POS_PLUGIN marker) + while IFS= read -r f; do + [ -n "$f" ] && [ -f "$f" ] && { rm -f "$f" && count=$((count+1)); } + done < <(installed_plugins) # Legacy forwarders for pat in 'wr-*' mp3 mp4 vbox ssh-load-all; do @@ -313,28 +353,70 @@ remove_tier1() { # Reload daemon after service changes systemctl daemon-reload 2>/dev/null || true - # ── Shell integration (~/.bashrc) ── - if [ -f "$HOME/.bashrc" ]; then - local before - before=$(wc -l < "$HOME/.bashrc") - sed -i '/source.*pos-ai-hook/d' "$HOME/.bashrc" - sed -i '/linux_post_install.*PATH/d' "$HOME/.bashrc" - sed -i '/source.*pos\.bash/d' "$HOME/.bashrc" - local after - after=$(wc -l < "$HOME/.bashrc") - local removed=$((before - after)) - count=$((count + removed)) + # ── Runtime-created USER systemd units (pos-* in ~/.config/systemd/user) ── + # Wrapped `|| true`: no user session (e.g. running as a scheduled task) is fine. + local user_unit_dir="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user" + if [ -d "$user_unit_dir" ]; then + local ufile uniname + while IFS= read -r ufile; do + [ -n "$ufile" ] || continue + uniname="$(basename "$ufile")" + systemctl --user disable --now "$uniname" 2>/dev/null || true + rm -f "$ufile" && count=$((count+1)) + done < <(find "$user_unit_dir" -maxdepth 1 -name 'pos-*' -type f 2>/dev/null || true) + systemctl --user daemon-reload 2>/dev/null || true fi - # ── Shell completion (~/.bash_completion) ── + # ── ScaleTail templates (submodule clone — guard existence) ── + if [ -d /usr/local/share/linux_post_install/scale-tail ]; then + rm -rf /usr/local/share/linux_post_install/scale-tail && count=$((count+1)) + fi + + # ── Feature-flag store ── + if [ -d /usr/local/share/linux_post_install/flags ]; then + rm -rf /usr/local/share/linux_post_install/flags && count=$((count+1)) + fi + + # Clean up parent dir if empty + rmdir /usr/local/share/linux_post_install 2>/dev/null || true + + # ── Shell integration (~/.bashrc) — exact installer lines + anchored hook ── + # Only lines postinstall.sh itself added are removed by exact literal match; + # pos-ai-hook lines are matched anchored (never a bare substring). + if [ -f "$HOME/.bashrc" ]; then + local before after removed tmp + before=$(wc -l < "$HOME/.bashrc") + tmp="$(mktemp)" + awk -v p="$PATH_LINE" -v c="$COMPLETION_LINE" -v h="$HOOK_PATTERN" ' + $0 == p || $0 == c || $0 ~ h { next } + { print } + ' "$HOME/.bashrc" > "$tmp" + after=$(wc -l < "$tmp") + removed=$((before - after)) + if [ "$removed" -gt 0 ]; then + chmod --reference="$HOME/.bashrc" "$tmp" + mv "$tmp" "$HOME/.bashrc" + count=$((count + removed)) + else + rm -f "$tmp" + fi + fi + + # ── Shell completion (~/.bash_completion) — anchored source lines only ── if [ -f "$HOME/.bash_completion" ]; then - local before + local before after removed tmp before=$(wc -l < "$HOME/.bash_completion") - sed -i '/pos/d' "$HOME/.bash_completion" - local after - after=$(wc -l < "$HOME/.bash_completion") - local removed=$((before - after)) - count=$((count + removed)) + tmp="$(mktemp)" + awk '/^[[:space:]]*source[[:space:]].*pos\.bash/ { next } { print }' "$HOME/.bash_completion" > "$tmp" + after=$(wc -l < "$tmp") + removed=$((before - after)) + if [ "$removed" -gt 0 ]; then + chmod --reference="$HOME/.bash_completion" "$tmp" + mv "$tmp" "$HOME/.bash_completion" + count=$((count + removed)) + else + rm -f "$tmp" + fi fi ok "Removed $count items (tier 1)" diff --git a/completions/pos.bash b/completions/pos.bash index 220b682..b0d3c33 100644 --- a/completions/pos.bash +++ b/completions/pos.bash @@ -23,15 +23,16 @@ _pos_flags[share-usb-server]="--ls --ls-shared --share --unshare --auto-share -- _pos_flags[system-backup]="--service --no-encrypt" _pos_flags[system-schedule]="--dry-run" _pos_flags[system-uninstall]="--yes --config --data" -_pos_flags[ai]="--provider --model --session --system --full --last --trust" +_pos_flags[ai]="--provider --model --session --system --full --last --trust --no-command-execution" _pos_flags[tree]="--depth" # GEN:END posflags # GEN:START possubcmds declare -A _pos_subcmds _pos_subcmds[ai-alias]="create edit remove list show" -_pos_subcmds[ai-gemini]="ask chat models sessions capture" -_pos_subcmds[ai-llamacpp]="ask chat models sessions capture" -_pos_subcmds[ai-openrouter]="ask chat sessions capture" +_pos_subcmds[ai-gemini]="ask chat models sessions capture providers" +_pos_subcmds[ai-hf]="search download list remove info files cache" +_pos_subcmds[ai-llamacpp]="ask chat models sessions capture providers" +_pos_subcmds[ai-openrouter]="ask chat sessions capture models providers" _pos_subcmds[ai-server]="start stop status models logs" _pos_subcmds[communication-matrix-sender]="send test login" _pos_subcmds[communication-scrcpy]="devices record tcpip connect push pull screenshot info" @@ -48,7 +49,7 @@ _pos_subcmds[share-smb-client]="mount unmount list persist unpersist menu" _pos_subcmds[share-smb-server]="status share unshare list adduser deluser reload enable disable menu" _pos_subcmds[system-backup]="menu" _pos_subcmds[system-schedule]="run list config enable disable status migrate menu" -_pos_subcmds[ai]="ask chat sessions capture models providers alias gemini hf llamacpp openrouter server" +_pos_subcmds[ai]="ask chat sessions capture models providers llamacpp alias gemini hf openrouter server" # GEN:END possubcmds # GEN:START posconfigscopes declare -a _pos_config_scopes=(ai compose entertainment grab matrix notify scrcpy system telegram ytsync) diff --git a/config/matrix.env b/config/matrix.env new file mode 100644 index 0000000..5c1ec97 --- /dev/null +++ b/config/matrix.env @@ -0,0 +1,13 @@ +# ~/.config/linux_post_install/matrix.env — Matrix config for 'pos communication matrix' +# Reference template (not auto-copied by install.sh). The runtime file is +# created and edited via: pos config matrix (chmod 600). +# +# MATRIX_HOMESERVER=https://matrix.example.org # Homeserver URL +# MATRIX_ACCESS_TOKEN= # Access token (secret; 'pos communication matrix sender login') +# MATRIX_USER_ID=@you:example.org # Your Matrix user id (owner) +# MATRIX_ROOM_ID=#pos:example.org # Room id or alias to watch. +# # REQUIRED for the listener (fail-closed): +# # without it the listener refuses to run +# # any command and watches nothing. +# +# Set values with: pos config matrix \ No newline at end of file diff --git a/config/telegram.env b/config/telegram.env new file mode 100644 index 0000000..cc8d474 --- /dev/null +++ b/config/telegram.env @@ -0,0 +1,13 @@ +# ~/.config/linux_post_install/telegram.env — Telegram bot config for 'pos communication telegram' +# Reference template (not auto-copied by install.sh). The runtime file is +# created and edited via: pos config telegram (chmod 600). +# +# TELEGRAM_BOT_TOKEN= # Bot token from @BotFather (secret) +# TELEGRAM_CHAT_ID= # Numeric chat id from @userinfobot +# TELEGRAM_OWNER_ID= # Your numeric Telegram user id (id from @userinfobot) +# # REQUIRED for the listener: only messages sent to +# # TELEGRAM_CHAT_ID BY your account are executed. +# # Leave unset and the listener ignores all commands. +# TELEGRAM_AI_PREFIX=ai # AI-bridge trigger word in the listener (default ai) +# +# Set values with: pos config telegram \ No newline at end of file diff --git a/lib/ai-providers/llamacpp.sh b/lib/ai-providers/llamacpp.sh index 57cf34e..a554087 100644 --- a/lib/ai-providers/llamacpp.sh +++ b/lib/ai-providers/llamacpp.sh @@ -9,15 +9,18 @@ provider_name() { printf 'Local llama.cpp'; } provider_default_model() { - local port="${LLAMACPP_PORT:-8088}" + # Honor LLAMACPP_HOST — must match the address the server binds (default + # 127.0.0.1); otherwise the adapter talks to a different host than the one + # the server actually listens on. + local host="${LLAMACPP_HOST:-127.0.0.1}" port="${LLAMACPP_PORT:-8088}" local model - model="$(curl -sf "http://127.0.0.1:$port/v1/models" 2>/dev/null | jq -r '.data[0].id // empty')" + model="$(curl -sf "http://$host:$port/v1/models" 2>/dev/null | jq -r '.data[0].id // empty')" [ -n "$model" ] && printf '%s' "$model" || printf '(no model loaded)' } # $1=model $2=messages JSON ({"messages":[{role,content}]}) $3=optional system prompt provider_generate() { - local model="$1" messages="$2" system="${3:-}" port="${LLAMACPP_PORT:-8088}" + local model="$1" messages="$2" system="${3:-}" host="${LLAMACPP_HOST:-127.0.0.1}" port="${LLAMACPP_PORT:-8088}" local body resp code body_out # Build messages array with optional system prompt if [ -n "$system" ]; then @@ -28,7 +31,7 @@ provider_generate() { fi body="$(printf '%s' "$body" | jq -nc --arg m "$model" --argjson msgs "$body" \ '{model:$m, messages:$msgs, stream:false}')" - resp="$(curl -sS -m 120 -X POST "http://127.0.0.1:$port/v1/chat/completions" \ + resp="$(curl -sS -m 120 -X POST "http://$host:$port/v1/chat/completions" \ -H "Content-Type: application/json" \ --write-out $'\n%{http_code}' \ --data "$body")" || { echo "request failed (curl exit $?)" >&2; return 1; } @@ -43,8 +46,8 @@ provider_generate() { # $1=current default model → stdout=formatted model list provider_models_list() { - local model="$1" port="${LLAMACPP_PORT:-8088}" resp code body - resp="$(curl -sf "http://127.0.0.1:$port/v1/models" \ + local model="$1" host="${LLAMACPP_HOST:-127.0.0.1}" port="${LLAMACPP_PORT:-8088}" resp code body + resp="$(curl -sf "http://$host:$port/v1/models" \ --write-out $'\n%{http_code}')" || { echo "server not running" >&2; return 1; } code="${resp##*$'\n'}" body="${resp%$'\n'*}" diff --git a/lib/config-ui.sh b/lib/config-ui.sh index a3188ea..1fb9a0b 100644 --- a/lib/config-ui.sh +++ b/lib/config-ui.sh @@ -307,6 +307,55 @@ cfg_scope_keys() { return 0 } +# ── Canonical env-file loader ──────────────────────────────────── +# load_env_file [scope] +# The one shared KEY=VALUE config loader for every pos tool (D-D). +# env file path; a bare basename (no '/') is resolved under +# ${CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/linux_post_install}/, +# so callers may pass "ai.env", "$CONFIG_DIR/ai.env", or any path. +# [scope] optional label (informational only; reserved, not used). +# Reads KEY=VALUE lines, skipping blank and '#' comment lines; strips a +# trailing CR from every value (CRLF files parse cleanly); trims one pair +# of surrounding quotes. Exports each key, but ONLY when the variable is +# not already set in the environment, so an exported env var always wins +# over the file. Full precedence contract, matching the historic behavior +# of every migrated tool: +# CLI flags > environment > config file > defaults +# (CLI flags are applied by each tool's own arg parser, defaults via +# ${VAR:-default} at declaration — the loader implements the middle step.) +# Loaded keys are APPENDED to the global LOADED_ENV_KEYS array so callers +# can tell which values came from the file (a caller that needs only one +# file's set resets LOADED_ENV_KEYS=() before the call). The loader never +# creates files and never chmods — chmod-600 semantics stay with cfg_write +# and the tools' own writers. A missing/unreadable file is a quiet no-op. +# +# NOTE: load_env_file supersedes lib/common.sh's load_system_env(), which is +# functionally identical (env-wins export loop). common.sh deliberately does +# NOT source this file — new tools should use load_env_file; the three legacy +# load_system_env callers keep working unchanged. +load_env_file() { + local f="$1" _scope="${2:-}" k v + if ! declare -p LOADED_ENV_KEYS &>/dev/null 2>&1; then + LOADED_ENV_KEYS=() + fi + case "$f" in + */*) : ;; # full path as given + *) f="${CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/linux_post_install}/$f" ;; + esac + [ -f "$f" ] || return 0 + while IFS='=' read -r k v; do + [ -n "$k" ] || continue + case "$k" in \#*) continue ;; esac + v="${v//$'\r'/}" + v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" + if [ -z "${!k:-}" ]; then + export "$k"="$v" + LOADED_ENV_KEYS+=("$k") + fi + done < <(grep -E '^[A-Z_]+=' "$f" || true) + return 0 +} + # Current value of a key in an env file (file is the source of truth, never sourced). cfg_value() { local file="$1" key="$2" v diff --git a/lib/entertainment-lib.sh b/lib/entertainment-lib.sh index cf898b3..6fce6e2 100644 --- a/lib/entertainment-lib.sh +++ b/lib/entertainment-lib.sh @@ -21,36 +21,25 @@ source "$(dirname "${BASH_SOURCE[0]}")/../lib/user-timers-lib.sh" 2>/dev/null \ || source "$(dirname "$0")/../lib/user-timers-lib.sh" 2>/dev/null \ || source "$(dirname "$0")/user-timers-lib.sh" +# Canonical config read/write + env loader (cfg_value/cfg_write/load_env_file). +source "$(dirname "${BASH_SOURCE[0]}")/../lib/config-ui.sh" 2>/dev/null \ + || source "$(dirname "${BASH_SOURCE[0]}")/config-ui.sh" 2>/dev/null \ + || source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null \ + || source "$(dirname "$0")/config-ui.sh" + # Per-plugin last-run state (rc + timestamp + first output line). LAST_RUN_DIR="${LAST_RUN_DIR:-$HOME/.local/share/linux_post_install/entertainment/last}" -# ── Config file helpers (file is the source of truth, never sourced) ── +# ── Config file helpers (thin wrappers over lib/config-ui.sh — the +# canonical read/write API; same semantics, CRLF-safe, chmod 600) ── config_value() { - local k="$1" v - [ -f "$CONFIG_FILE" ] || return 0 - v="$(sed -n "s|^${k}=||p" "$CONFIG_FILE" | tail -1)" - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - printf '%s' "$v" + local k="$1" + cfg_value "$CONFIG_FILE" "$k" } write_config_key() { - local key="$1" val="$2" tmp - val="${val//$'\r'/}" - val="${val%%$'\n'*}" - mkdir -p "$CONFIG_DIR" - if [ "$val" = "-" ]; then - [ -f "$CONFIG_FILE" ] || return 0 - tmp="$(mktemp)" - grep -v "^${key}=" "$CONFIG_FILE" >"$tmp" || true - mv "$tmp" "$CONFIG_FILE" - chmod 600 "$CONFIG_FILE" - return 0 - fi - tmp="$(mktemp)" - grep -v "^${key}=" "$CONFIG_FILE" 2>/dev/null >"$tmp" || true - printf '%s="%s"\n' "$key" "$val" >>"$tmp" - mv "$tmp" "$CONFIG_FILE" - chmod 600 "$CONFIG_FILE" + local key="$1" val="$2" + cfg_write "$CONFIG_FILE" "$key" "$val" } # ── Plugin lookup ────────────────────────────────────────────────── diff --git a/lib/share-lib.sh b/lib/share-lib.sh index 7e099c7..3f9d435 100644 --- a/lib/share-lib.sh +++ b/lib/share-lib.sh @@ -56,7 +56,9 @@ share_require_bin() { # rc 0 reachable within 3s · rc 1 unreachable/no-route. Message policy (targeted # hints, firewall wording) belongs to the caller. share_port_probe() { - timeout 3 bash -c "exec 3<>/dev/tcp/${1}/${2}" 2>/dev/null + # host/port are positional args ($1/$2), never interpolated into the + # command source — a hostile host string stays a literal argument. + timeout 3 bash -c 'exec 3<>/dev/tcp/$1/$2' _ "${1}" "${2}" 2>/dev/null } # ── systemd unit state probe ─────────────────────────────────── diff --git a/scripts/lint-conventions.sh b/scripts/lint-conventions.sh index 053f2bf..cec4f8e 100755 --- a/scripts/lint-conventions.sh +++ b/scripts/lint-conventions.sh @@ -26,21 +26,6 @@ has_regex() { return 1 } -first_line() { - local file="$1" re="$2" - grep -nE "$re" "$file" 2>/dev/null | while IFS=: read -r ln rest; do - [ -z "$ln" ] && continue - [[ "$rest" =~ ^[[:space:]]*# ]] && continue - printf '%s' "$ln" - break - done -} - -last_line() { - local file="$1" re="$2" - grep -nE "$re" "$file" 2>/dev/null | tail -1 | cut -d: -f1 || true -} - # First deps guard: a `command -v` line that hard-fails the tool when the binary # is absent — i.e. `command -v X … || err`, `if ! command -v X …`, or a # `command -v X … \` multi-line continuation. Runtime capability probes like @@ -56,27 +41,19 @@ first_guard_line() { printf '%s' "" } -uses_stdin() { - local file="$1" line heredoc="" - while IFS= read -r line; do - if [ -n "$heredoc" ]; then - [ "$line" = "$heredoc" ] && heredoc="" - continue - fi - local delim - delim="$(printf '%s\n' "$line" | sed -nE 's/.*<<-?[[:space:]]*([A-Za-z0-9_]+).*/\1/p' | tail -1)" - [ -n "$delim" ] && { heredoc="$delim"; continue; } - case "$line" in - *'read -'*|*'read '*|*'select '*|*'confirm '*|*'confirm('*) ;; - *) continue ;; - esac - [[ "$line" =~ ^[[:space:]]*# ]] && continue - [[ "$line" == *"/dev/tty"* ]] && continue - [[ "$line" =~ (while|until)[[:space:]].*read ]] && continue - [[ "$line" =~ [[:space:]]\< ]] && continue - return 0 - done < "$file" - return 1 +# Single-line stdin-reader test. Pure bash; no subprocess. Heredoc/while-loop +# exclusions are handled by the caller's per-file loop state. +_reads_stdin() { + local ln="$1" + case "$ln" in + *'read -'*|*'read '*|*'select '*|*'confirm '*|*'confirm('*) ;; + *) return 1 ;; + esac + [[ "$ln" =~ ^[[:space:]]*# ]] && return 1 + [[ "$ln" == *"/dev/tty"* ]] && return 1 + [[ "$ln" =~ (while|until)[[:space:]].*read ]] && return 1 + [[ "$ln" =~ [[:space:]]\< ]] && return 1 + return 0 } INTERACTIVE_CMDS="" @@ -89,10 +66,11 @@ for f in $(shell_files); do case "$f" in lib/*.sh) continue ;; # libraries are sourced, never executed esac - if ! head -1 "$f" | grep -q '^#!/usr/bin/env bash'; then + read -r first < "$f" || first="" + if [ "$first" != '#!/usr/bin/env bash' ]; then fail "$f: missing '#!/usr/bin/env bash' shebang" fi - if ! has_regex "$f" '^set -euo pipefail'; then + if ! grep -qE '^set -euo pipefail' "$f"; then fail "$f: missing 'set -euo pipefail'" fi done @@ -104,18 +82,69 @@ for f in $(executable_files); do fi done +# Preload DOC/POS.md once — the per-tool presence check below must not re-read +# the file (and re-spawn grep) for every tool. +posmd="" +[ -f DOC/POS.md ] && posmd="$(cat DOC/POS.md)" + for f in bin/pos-*; do [ -f "$f" ] || continue - headline="$(sed -n '/^# POS: /{s/^# POS: //;p;q}' "$f" 2>/dev/null)" + # ── single-pass metadata scan (shebang/pos-header/help/local/stdin) ── + headline="" + posline=0 + local_line="" + help_line="" + uses_stdin=0 + lineno=0 heredoc="" depth=0 + while IFS= read -r line || [ -n "$line" ]; do + lineno=$((lineno + 1)) + # # POS: header (whole-file scan, matches original `sed -n '/^# POS: /…'`) + if [ "$posline" -eq 0 ] && [[ "$line" == "# POS: "* ]]; then + posline=$lineno + headline="${line#*POS: }" + fi + # First non-comment -h|--help line (matches original first_line lookup) + if [ -z "$help_line" ] && ! [[ "$line" =~ ^[[:space:]]*# ]] && [[ "$line" == *-h* || "$line" == *--help* ]] && [[ "$line" =~ -h\|--help ]]; then + help_line=$lineno + fi + [ -n "$heredoc" ] && { [ "$line" = "$heredoc" ] && heredoc=""; continue; } + # Heredoc delimiter: greedy `.*` selects the LAST << / <<- on the line, + # matching the original sed `s/.*<<-?[[:space:]]*([A-Za-z0-9_]+).*/\1/p | tail -1`. + if [[ "$line" == *"<<"* ]] && [[ "$line" =~ .*\<\<-?[[:space:]]*([A-Za-z0-9_]+) ]]; then + heredoc="${BASH_REMATCH[1]}" + continue + fi + # Top-level `local` (WARN) — first occurrence at brace depth 0. Depth + # tracking is only needed until the first one is found, so the two + # full-line brace-count expansions are gated off after that. + if [ -z "$local_line" ]; then + if [[ "$line" == *local* ]] && [[ "$line" =~ ^[[:space:]]*local[[:space:]] ]]; then + if [ "$depth" -eq 0 ]; then + [ -n "$local_line" ] || local_line="$lineno" + fi + fi + opens="${line//[^{]/}" + closes="${line//[^\}]/}" + depth=$((depth + ${#opens} - ${#closes})) + [ "$depth" -lt 0 ] && depth=0 + fi + # stdin-reader detection (non-heredoc lines only; cheap string gate + # keeps the expensive regex work inside `_reads_stdin` for read-like lines) + if [ "$uses_stdin" -eq 0 ]; then + case "$line" in + *'read -'*|*'read '*|*'select '*|*'confirm '*|*'confirm('*) _reads_stdin "$line" && uses_stdin=1 ;; + esac + fi + done < "$f" + if [ -z "$headline" ]; then fail "$f: missing '# POS:' header" continue fi - if ! grep -q ' — ' <<<"$headline"; then + if [[ "$headline" != *' — '* ]]; then fail "$f: '# POS:' header missing em-dash ' — ' (format: '# POS: — ')" fi - posline="$(grep -nE '^# POS: ' "$f" | head -1 | cut -d: -f1 || true)" if [ "${posline:-99}" -gt 6 ]; then warn_ "$f: '# POS:' header on line $posline (convention: right after shebang/strict-mode)" fi @@ -125,48 +154,23 @@ for f in bin/pos-*; do fi guard="$(first_guard_line "$f")" - help_line="$(first_line "$f" '\-h\|\-\-help')" if [ -n "$guard" ] && [ -n "$help_line" ] && [ "$help_line" -lt "$guard" ]; then fail "$f: -h|--help (line $help_line) dispatched before deps guards (line $guard) — help must error on missing deps" fi - local_line="" - lineno=0 - depth=0 - heredoc="" - while IFS= read -r line || [ -n "$line" ]; do - lineno=$((lineno + 1)) - if [ -n "$heredoc" ]; then - [ "$line" = "$heredoc" ] && heredoc="" - continue - fi - delim="$(printf '%s\n' "$line" | sed -nE 's/.*<<-?[[:space:]]*([A-Za-z0-9_]+).*/\1/p' | tail -1)" - if [ -n "$delim" ]; then - heredoc="$delim" - continue - fi - if [[ "$line" =~ ^[[:space:]]*local[[:space:]] ]]; then - if [ "$depth" -eq 0 ]; then - [ -n "$local_line" ] || local_line="$lineno" - fi - fi - opens="${line//[^{]/}" - closes="${line//[^\}]/}" - depth=$((depth + ${#opens} - ${#closes})) - [ "$depth" -lt 0 ] && depth=0 - done < "$f" if [ -n "$local_line" ]; then warn_ "$f: '$local_line': 'local' at top-level brace depth (invalid in bash outside a function)" fi - if uses_stdin "$f"; then + if [ "$uses_stdin" -eq 1 ]; then name="${f#bin/pos-}" if ! [[ " $INTERACTIVE_CMDS " == *" $name "* ]]; then fail "$f: reads stdin but NOT in INTERACTIVE_CMDS in bin/pos (log tee will swallow/hang prompts)" fi fi - if ! grep -q "$(basename "$f")" DOC/POS.md 2>/dev/null; then + base="${f#bin/}" + if ! [[ "$posmd" == *"$base"* ]]; then warn_ "$f: file not referenced in DOC/POS.md" fi done @@ -226,35 +230,74 @@ for f in bin/wr-* bin/mp3 bin/mp4 bin/vbox bin/ssh-load-all; do fi done +# Secret-like literal assignment scan (WARN). Original used a grep pre-filter +# + a per-line heredoc grep; here both are bash `[[ =~ ]]` in one read per file. while IFS= read -r f; do [ -f "$f" ] || continue + num=0 while IFS= read -r line; do - [[ "$line" =~ ^[0-9]+: ]] || continue - num="${line%%:*}" - body="${line#*:}" - if grep -qE '(TOKEN|PASSWORD|PASSWD|SECRET|API[_-]?KEY|ACCESS[_-]?TOKEN|AUTH[_-]?KEY)=' <<<"$body"; then - val="${body#*=}" - case "$val" in - ""|*'$'*) ;; - *) warn_ "$f:$num: secret-like literal assignment (manual review for hardcoded credentials)" ;; - esac - fi - done < <(grep -nE '^[[:space:]]*[A-Za-z_][A-Za-z0-9_]*(TOKEN|PASSWORD|PASSWD|SECRET|API[_-]?KEY|ACCESS[_-]?TOKEN|AUTH[_-]?KEY)=' "$f" 2>/dev/null || true) + num=$((num + 1)) + # cheap substring gate before the expensive assignment regex + [[ "$line" == *TOKEN* || "$line" == *PASSWORD* || "$line" == *PASSWD* || "$line" == *SECRET* || "$line" == *API* || "$line" == *ACCESS* || "$line" == *KEY* || "$line" == *AUTH* ]] || continue + [[ "$line" =~ ^[[:space:]]*[A-Za-z_][A-Za-z0-9_]*(TOKEN|PASSWORD|PASSWD|SECRET|API[_-]?KEY|ACCESS[_-]?TOKEN|AUTH[_-]?KEY)= ]] || continue + [[ "$line" =~ (TOKEN|PASSWORD|PASSWD|SECRET|API[_-]?KEY|ACCESS[_-]?TOKEN|AUTH[_-]?KEY)= ]] || continue + val="${line#*=}" + case "$val" in + ""|*'$'*) ;; + *) warn_ "$f:$num: secret-like literal assignment (manual review for hardcoded credentials)" ;; + esac + done < "$f" done < <(printf '%s\n' bin/pos bin/pos-* lib/*.sh features/*.sh entertainment/*.sh install.sh preinstall.sh postinstall.sh) +# Early-occurrence scan for one of the three system paths in a line. +# strindex sets the global _SI to the index of $2 in $1 (or 2147483647 if absent). +_SI=0 +strindex() { local pre="${1%%"$2"*}"; if [ "$pre" = "$1" ]; then _SI=2147483647; else _SI=${#pre}; fi; } + +# Faithful re-implementation of the original outer filter +# grep -E '(\btee\b|>>?)[^#]*?(/etc/|\$HOME|/usr/local)' +# (operator, then any non-'#' chars, then a system path). Bash `=~` does not +# honour the lazy `[^#]*?` the same way, so we walk path occurrences manually, +# checking that the segment before each path (after the last '#') holds an +# operator (`>`/`>>` or a word-bounded `tee`). +_syspath_outer() { + local ln="$1" best=2147483647 bestcand="" idx c pre seg h + for c in '/etc/' '$HOME' '/usr/local'; do + strindex "$ln" "$c" + [ "$_SI" -lt "$best" ] && { best="$_SI"; bestcand="$c"; } + done + [ "$best" -ge 2147483647 ] && return 1 + pre="${ln:0:best}" + h="${pre%#*}" + if [ "$h" = "$pre" ]; then + seg="$pre" # no '#' before the path + else + seg="${pre:${#h}+1}" # after the last '#' before the path + fi + if [[ "$seg" == *">"* ]] || [[ "$seg" =~ \btee\b ]]; then + return 0 + fi + _syspath_outer "${ln:best+${#bestcand}}" +} + +# System-path write scan (WARN). Route lines through the same outer filter then +# the same per-line heuristic the original applied (check + exclusions). while IFS= read -r f; do [ -f "$f" ] || continue + num=0 while IFS= read -r line; do - [[ "$line" =~ ^[0-9]+: ]] || continue - num="${line%%:*}" - body="${line#*:}" - if grep -qE '(>|>>|tee )' <<<"$body" && grep -qE '(/etc/|\$HOME|/usr/local)' <<<"$body"; then - case "$body" in + num=$((num + 1)) + # cheap string gate: outer filter needs both an operator and a path + [[ "$line" == *">"* || "$line" == *tee* ]] || continue + [[ "$line" == */etc/* || "$line" == *'$HOME'* || "$line" == */usr/local* ]] || continue + _syspath_outer "$line" || continue + if [[ "$line" =~ (>|>>|tee[[:space:]]) ]] && [[ "$line" =~ (/etc/|\$HOME|/usr/local) ]]; then + case "$line" in *'command -v'*|*'|| echo'*) ;; *) warn_ "$f:$num: writes to a system path (verify a VAR=\"\${VAR:-path}\" test seam exists)" ;; esac fi - done < <(grep -nE '(\btee\b|>>?)[^#]*?(/etc/|\$HOME|/usr/local)' "$f" 2>/dev/null || true) + done < "$f" done < <(printf '%s\n' bin/pos-* lib/*.sh features/*.sh entertainment/*.sh) printf '\n%d FAIL, %d WARN (convention lint)\n' "$fails" "$warns" diff --git a/templates/app.sh b/templates/app.sh index bdc9784..67a4aed 100755 --- a/templates/app.sh +++ b/templates/app.sh @@ -5,7 +5,7 @@ set -euo pipefail # TEMPLATE — new optional app installer # # 1. Copy: cp templates/app.sh apps//.sh -# Categories: browsers, development, media, networking, +# Categories: ai, browsers, development, media, networking, # remote-access, system, utilities. # 2. Fill in install_myapp() / uninstall_myapp() (rename to your app). # 3. Docs: add a row to the catalog table in DOC/APPS.md. diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 0000000..4c61329 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,54 @@ +# tests/ — regression test suite + +Zero-dependency Bash regression tests for the Linux_post_install repository. + +## Run + +```bash +make test # discover tests/t-*.sh, run everything +./tests/run-tests.sh # same +./tests/run-tests.sh t-telegram-auth.sh # run one file +``` + +Exit code is non-zero if any check failed. The suite is designed to be +deterministic: tests stub every external dependency they touch (curl, gpg, +sudo, systemctl, llama-server, nvidia-smi…) and run against sandbox temp +dirs. No network, no sudo, no system changes. + +## Adding a test + +1. Create `tests/t-.sh` with a `run_test()` function. `set -euo pipefail` + is already active (the runner re-asserts it); `$ROOT` is the repo root, + `$TEST_TMP` a per-test temp dir that is cleaned automatically. +2. Use the helpers in `tests/test-lib.sh` for every assertion: + `check`, `check_eq`, `check_rc`, `check_contains`, `check_not_contains`, + `check_file_exists`, `check_file_absent`, `test_run` / `test_run_env`, + `mksandbox`, `tracked_tree_copy`, `count_token`. +3. If a case cannot run in the current environment (missing binary, missing + analyzer), call `skip_case "" ""` — never fake a pass and + never silently return. +4. Keep total suite runtime under 90 seconds. + +## Skip contract (hard) + +A test that cannot run must say `[SKIP] reason`. The runner counts skips in +the summary and a file whose `run_test()` produced zero checks and zero skips +is reported as FAIL ("no assertions") — a broken harness can never pass +silently. + +## Suite contents + +| File | What it verifies | +| --- | --- | +| `t-ai-server-flags.sh` | `pos-ai-server` ExecStart flag set: defaults, CLI, config, dedupe, one-token-per-flag | +| `t-ai-hf-download.sh` | `pos ai hf download` success + failure honesty (no `.hf-meta` on partial failure, rc != 0) | +| `t-ai-llama-detect.sh` | `pos ai-server status` version detection, "unknown", missing-binary failure | +| `t-unsupported-flags.sh` | unsupported-option handling: CLI/config/env hard errors, dropped defaults, word-boundary match | +| `t-systemd-unit.sh` | generated unit: one ExecStart, quoted paths, `systemd-analyze verify` | +| `t-telegram-auth.sh` | Telegram listener chat+owner gate and owner-unset fail-closed | +| `t-matrix-auth.sh` | Matrix listener room+owner gate and room-unset fail-closed | +| `t-gpg-password.sh` | backup passphrase on fd 3 (never argv), plaintext/corrupt cleanup | +| `t-config-precedence.sh` | `load_env_file` contract + CLI > env > file > defaults across tools | +| `t-uninstall-manifest.sh` | install.sh ↔ POS_LIBS symmetry, user-unit discovery, marker-driven plugin removal | +| `t-gen-docs-drift.sh` | `make gen` idempotence on a pristine tracked tree (CI drift gate) | +| `t-lint-gate.sh` | `make lint` green on the real tree; planted violations are caught and named | \ No newline at end of file diff --git a/tests/run-tests.sh b/tests/run-tests.sh new file mode 100755 index 0000000..cfe8bc5 --- /dev/null +++ b/tests/run-tests.sh @@ -0,0 +1,167 @@ +#!/usr/bin/env bash +set -euo pipefail +# tests/run-tests.sh — zero-dependency Bash regression runner. +# +# Discovers tests/t-*.sh, runs each in an isolated subshell (temp sandbox per +# test, cleaned up automatically), collects PASS/FAIL/SKIP, prints a summary +# table, and exits non-zero when any check failed. +# +# Usage: tests/run-tests.sh [test-file ...] +# No args → run every tests/t-*.sh (sorted) +# With args → run only the named test files (path or bare name) +# +# Skip contract (hard): a test that cannot run in this environment MUST call +# skip_case "" "" — it prints `[SKIP] reason`, is counted in the +# summary, and never counts as PASS. A test file whose run_test() produced +# zero checks and zero skips is reported as FAIL ("no assertions") so a broken +# harness can never silently pass. Tests must never lie. + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +TEST_DIR="$(cd "$(dirname "$0")" && pwd)" + +PASS=0; FAIL=0; SKIP=0 +files_total=0; files_pass=0; files_fail=0; files_skip=0 +declare -a FAILED_FILES=() +START_TS="$(date +%s)" + +# ── per-test state set by the runner before sourcing a test file ── +TEST_NAME="" +TEST_TMP="" +TEST_TRAP_SET=0 + +cleanup_test_tmp() { + if [ -n "${TEST_TMP:-}" ] && [ -d "$TEST_TMP" ]; then + rm -rf "$TEST_TMP" + fi + TEST_TMP="" +} + +# Run one test file in a subshell; stdout/stderr go to the per-test log. +# Returns 0 = no FAIL lines, 1 = FAIL, 2 = SKIP-only. +run_one() { + local test_file="$1" + local log="$TEST_TMP/$TEST_NAME.log" + local rc=0 + + # Run in a subshell; the exit status must be captured via `if` because a + # plain `( ... ); rc=$?` would let the subshell's failure trigger errexit + # in the parent (set -e applies to subshells as ordinary commands). + local rc=0 + if ( + # Isolated sandbox for this test. + TEST_NAME="$(basename "$test_file")" + TEST_TMP="$(mktemp -d "${TMPDIR:-/tmp}/pos-test-${TEST_NAME}.XXXXXX")" + export TEST_NAME TEST_TMP + trap cleanup_test_tmp EXIT + + # Shared helpers + strict mode (the runner already has -euo pipefail; + # re-assert it in the subshell for belt-and-braces). + set -euo pipefail + source "$TEST_DIR/test-lib.sh" + source "$test_file" + run_test + ) >"$log" 2>&1; then + rc=0 + else + rc=$? + fi + + local f_p=0 f_f=0 f_s=0 + f_p="$(grep -c '^ PASS ' "$log" 2>/dev/null || true)" + f_f="$(grep -c '^ FAIL ' "$log" 2>/dev/null || true)" + f_s="$(grep -c '^ SKIP ' "$log" 2>/dev/null || true)" + PASS=$((PASS + f_p)); FAIL=$((FAIL + f_f)); SKIP=$((SKIP + f_s)) + files_total=$((files_total + 1)) + + # A test that aborted (rc != 0) WITHOUT a checked FAIL is still a FAIL — + # set -e / errexit killed it mid-run. + if [ "$rc" -ne 0 ] && [ "$f_f" -eq 0 ]; then + f_f=1 + FAIL=$((FAIL + 1)) + printf '\n FAIL test aborted (exit %s) before a check failed — see log below\n' "$rc" >>"$log" + fi + + if [ "$f_f" -gt 0 ]; then + files_fail=$((files_fail + 1)) + FAILED_FILES+=("$test_file") + printf ' FAIL %s (%s checks, %s failed, %s skipped)\n' "$TEST_NAME" "$f_p" "$f_f" "$f_s" + elif [ "$f_p" -gt 0 ]; then + files_pass=$((files_pass + 1)) + if [ "$f_s" -gt 0 ]; then + printf ' PASS %s (%s checks, %s skipped)\n' "$TEST_NAME" "$f_p" "$f_s" + else + printf ' PASS %s (%s checks)\n' "$TEST_NAME" "$f_p" + fi + elif [ "$f_s" -gt 0 ]; then + files_skip=$((files_skip + 1)) + local reasons + reasons="$(grep '^ SKIP ' "$log" 2>/dev/null | sed 's/^ SKIP //' | tr '\n' '; ' )" + printf ' SKIP %s (%s reasons: %s)\n' "$TEST_NAME" "$f_s" "${reasons:-none}" + else + files_fail=$((files_fail + 1)) + FAILED_FILES+=("$test_file") + printf ' FAIL %s (no assertions run — harness broken)\n' "$TEST_NAME" + fi +} + +main() { + local tests=() f + if [ $# -gt 0 ]; then + for f in "$@"; do + case "$f" in + */*) tests+=("$f") ;; + *) + if [ -f "$TEST_DIR/$f.sh" ]; then + tests+=("$TEST_DIR/$f.sh") + elif [ -f "$TEST_DIR/$f" ]; then + tests+=("$TEST_DIR/$f") + else + echo "No such test: $f" >&2 + exit 2 + fi + ;; + esac + done + else + # Deterministic order: sorted by name. + while IFS= read -r f; do + tests+=("$f") + done < <(find "$TEST_DIR" -maxdepth 1 -name 't-*.sh' -type f | sort) + fi + + [ "${#tests[@]}" -gt 0 ] || { echo "No test files found in $TEST_DIR" >&2; exit 1; } + + echo "Running ${#tests[@]} test file(s) — strict mode: no network, no sudo, no system changes." + echo + + local one + for one in "${tests[@]}"; do + TEST_NAME="$(basename "$one")" + TEST_TMP="$(mktemp -d "${TMPDIR:-/tmp}/pos-run.XXXXXX")" + run_one "$one" + # keep the per-test sandbox only on FAILURE for debugging + if [ -f "$TEST_TMP/$TEST_NAME.log" ]; then + if [ "$(grep -c '^ FAIL ' "$TEST_TMP/$TEST_NAME.log" 2>/dev/null || true)" -gt 0 ]; then + echo "─── $TEST_NAME log ───" + sed 's/^/ /' "$TEST_TMP/$TEST_NAME.log" + echo "──────────────────────" + fi + rm -f "$TEST_TMP/$TEST_NAME.log" + fi + rm -rf "$TEST_TMP" + done + TEST_TMP="" + + local elapsed=$(( $(date +%s) - START_TS )) + echo + echo "──────────────────────────────────────────────" + echo "Summary: files ${files_pass} pass / ${files_fail} fail / ${files_skip} skip (of ${files_total})" + echo "Checks : ${PASS} pass / ${FAIL} fail / ${SKIP} skip" + echo "Runtime: ${elapsed}s" + if [ "${#FAILED_FILES[@]}" -gt 0 ]; then + printf 'Failed : %s\n' "${FAILED_FILES[*]}" + fi + [ "$FAIL" -eq 0 ] && [ "$files_fail" -eq 0 ] +} + +main "$@" \ No newline at end of file diff --git a/tests/t-ai-hf-download.sh b/tests/t-ai-hf-download.sh new file mode 100644 index 0000000..6cdc2ad --- /dev/null +++ b/tests/t-ai-hf-download.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-ai-hf-download.sh — `pos ai hf download` single-file path (D-C layout): +# - success: target file + .hf-meta written, rc 0, honest summary line; +# - failure: "Failed to download", NO .hf-meta (incomplete model is never +# advertised as complete), rc != 0, honest (0 of N files) summary. +# curl is stubbed (no network): /tree/ + /models/ APIs return canned JSON, +# /resolve/ writes content or fails via HF_FAIL_DOWNLOAD. + +run_test() { + require_cmd jq "ai hf download" || return 0 + require_cmd timeout "ai hf download" || return 0 + + local sandbox stubs dl tree_resp + sandbox="$(mksandbox ai-hf-download)" + stubs="$sandbox/stubs" + dl="$sandbox/models" + mkdir -p "$stubs" "$dl" + + local tree_resp="$sandbox/tree.json" + printf '%s' '[{"type":"file","path":"model.gguf","size":12345}]' > "$tree_resp" + + cat > "$stubs/curl" < "\$val"; fi + ;; + esac +done +url="\${@: -1}" +case "\$url" in + *"/tree/"*) + [ "\$has_w" -eq 1 ] && printf '200' + cat "\$TREE_RESP" > "\$out" + ;; + *"/models/"*) + [ "\$has_w" -eq 1 ] && printf '200' + printf '%s' '{"defaultBranch":null}' > "\$out" + ;; + *"/resolve/"*) + if [ "\${HF_FAIL_DOWNLOAD:-0}" = "1" ]; then + exit 1 + fi + printf 'stub model binary content\n' > "\$out" + ;; + *) + [ "\$has_w" -eq 1 ] && printf '200' + ;; +esac +STUB + chmod +x "$stubs/curl" + + local tool="$ROOT/bin/pos-ai-hf" + local env_base=(PATH="$stubs:/usr/bin:/bin" HF_DOWNLOAD_DIR="$dl" TREE_RESP="$tree_resp") + + # ── 1. success ── + test_run_env "${env_base[@]}" -- timeout 60 "$tool" download ns/test-model model.gguf + check_rc "download succeeds" 0 "$TR_RC" + check_contains "success summary names file" "Downloaded: ns/test-model/model.gguf" "$TR_OUT" + check_file_exists "target model downloaded" "$dl/ns-test-model/model.gguf" + check_file_exists "metadata written on full success" "$dl/ns-test-model/.hf-meta" + check_contains "metadata records repo" "ns/test-model" "$(cat "$dl/ns-test-model/.hf-meta")" + + # ── 2. download failure → no metadata, rc != 0 ── + rm -rf "$dl/ns-test-model" + test_run_env "${env_base[@]}" HF_FAIL_DOWNLOAD=1 -- timeout 60 "$tool" download ns/test-model model.gguf + check_contains "failure warns per file" "Failed to download model.gguf" "$TR_OUT" + check_contains "failure refuses to write metadata" "Not writing .hf-meta" "$TR_OUT" + check_contains "failure summary is honest (0 of 1)" "0 of 1 files, 1 failed" "$TR_OUT" + check_file_absent "no .hf-meta on partial failure" "$dl/ns-test-model/.hf-meta" + if [ "$TR_RC" -ne 0 ]; then + printf ' PASS download failure exits nonzero\n' + else + printf ' FAIL download failure exited 0\n' + fi +} \ No newline at end of file diff --git a/tests/t-ai-llama-detect.sh b/tests/t-ai-llama-detect.sh new file mode 100644 index 0000000..5e6e0d4 --- /dev/null +++ b/tests/t-ai-llama-detect.sh @@ -0,0 +1,74 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-ai-llama-detect.sh — `pos ai-server status` behavior: +# - prints the detected llama-server version from `--version`, +# - "unknown" when the stub omits a parseable version, +# - fails cleanly (rc != 0, actionable message) when the binary is missing. + +run_test() { + require_cmd timeout "ai-server status runs" || return 0 + + local sandbox stubs + sandbox="$(mksandbox ai-llama-detect)" + stubs="$sandbox/stubs" + mkdir -p "$stubs" + + # stub systemctl: everything inactive/disabled; nvidia-smi: no GPU → + # deterministic CPU path. + cat > "$stubs/systemctl" <<'STUB' +#!/usr/bin/env bash +# any subcommand: act like service is inactive/disabled +exit 1 +STUB + cat > "$stubs/nvidia-smi" <<'STUB' +#!/usr/bin/env bash +exit 1 +STUB + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "llama.cpp build 1.2.3 (abcdef)" ;; + --help) echo "usage: llama-server [options]" ;; +esac +STUB + chmod +x "$stubs/systemctl" "$stubs/nvidia-smi" "$stubs/llama-server" + + local server="$ROOT/bin/pos-ai-server" + local env_base=(PATH="$stubs:/usr/bin:/bin" USER_SYSTEMD_DIR="$sandbox/userunits") + + # 1. version detected from `--version` + test_run_env "${env_base[@]}" -- timeout 20 "$server" status + check_rc "status exits 0" 0 "$TR_RC" + check_contains "detects stub version 1.2.3" "1.2.3" "$TR_OUT" + check_contains "reports CPU backend (no GPU)" "gpu: CPU" "$TR_OUT" + check_not_contains "no GPU layers offload on CPU-only box" "gpu: GPU" "$TR_OUT" + + # 2. version not parseable → "unknown" (never a lie / never a crash) + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "custom llama build" ;; + --help) echo "usage: llama-server [options]" ;; +esac +STUB + chmod +x "$stubs/llama-server" + test_run_env "${env_base[@]}" -- timeout 20 "$server" status + check_rc "status exits 0 with unparseable version" 0 "$TR_RC" + check_contains "reports unknown version honestly" "unknown" "$TR_OUT" + + # 3. binary missing → clean, actionable failure (rc != 0, no hang) + local nopath="$sandbox/nopath" + mkdir -p "$nopath" + cp "$stubs/systemctl" "$nopath/systemctl" + cp "$stubs/nvidia-smi" "$nopath/nvidia-smi" + chmod +x "$nopath/systemctl" "$nopath/nvidia-smi" + test_run_env PATH="$nopath:/usr/bin:/bin" USER_SYSTEMD_DIR="$sandbox/userunits" -- timeout 20 "$server" status + check_contains "missing-binary status fails" "llama-server" "$TR_OUT" + check_contains "missing-binary error is actionable" "not found" "$TR_OUT" + # rc must be nonzero OR the message must clearly refuse (fail-closed) + if [ "$TR_RC" -eq 0 ]; then + printf ' FAIL status rc nonzero when llama-server missing (got rc 0)\n' + else + printf ' PASS status rc nonzero when llama-server missing\n' + fi +} \ No newline at end of file diff --git a/tests/t-ai-server-flags.sh b/tests/t-ai-server-flags.sh new file mode 100644 index 0000000..3f9b6e5 --- /dev/null +++ b/tests/t-ai-server-flags.sh @@ -0,0 +1,104 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-ai-server-flags.sh — pos-ai-server argument generation: flag set from +# CLI-explicit vs config vs defaults; dedupe; every emitted flag comes from a +# validated source. Behavior asserted through the tool's own dry-run ExecStart +# output (no unit write, no systemd). + +run_test() { + local sbin sandbox stubs cfg models + sandbox="$(mksandbox ai-server-flags)" + stubs="$sandbox/stubs" + cfg="$sandbox/cfg" + models="$sandbox/models" + mkdir -p "$stubs" "$cfg" "$models" + : > "$cfg/ai.env" + touch "$models/my-model.gguf" + + # ── stub llama-server: version + full help; nvidia-smi forced to fail so + # GPU detection deterministically resolves to "cpu" → --n-gpu-layers 0. + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "llama.cpp 1.2.3" ;; + --help) + cat <<'HELP' +usage: llama-server [options] +options: + --host bind address + --port server port + --n-gpu-layers layers to offload + --ctx-size context size + --threads cpu threads + --mmap memory mapping + --tensor-split tensor split +HELP + ;; +esac +STUB + cat > "$stubs/nvidia-smi" <<'STUB' +#!/usr/bin/env bash +exit 1 +STUB + chmod +x "$stubs/llama-server" "$stubs/nvidia-smi" + + local server="$ROOT/bin/pos-ai-server" + local base_env=(PATH="$stubs:/usr/bin:/bin" DRY_RUN=1 + CONFIG_FILE="$cfg/ai.env" USER_SYSTEMD_DIR="$sandbox/userunits") + + # 1. defaults: no CLI flags, no config → the always-emitted default set. + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" + check_rc "start with no flags exits 0" 0 "$TR_RC" + local exec_line + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_contains "dry-run ExecStart printed" "ExecStart:" "$TR_OUT" + check_contains "default --port 8088" "--port 8088" "$exec_line" + check_contains "default --host 127.0.0.1" "--host 127.0.0.1" "$exec_line" + check_contains "default --n-gpu-layers 0 (cpu)" "--n-gpu-layers 0" "$exec_line" + check_contains "default --ctx-size 4096" "--ctx-size 4096" "$exec_line" + check_contains "default --threads emitted" "--threads" "$exec_line" + check_contains "model path present via -m" "-m \"$models/my-model.gguf\"" "$exec_line" + + # 2. CLI-explicit flags override defaults and are emitted. + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" --port 9090 --ctx-size 2048 + check_rc "start with CLI flags exits 0" 0 "$TR_RC" + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_contains "CLI --port 9090 emitted" "--port 9090" "$exec_line" + check_contains "CLI --ctx-size 2048 emitted" "--ctx-size 2048" "$exec_line" + check_not_contains "CLI --port wins over default 8088" "--port 8088" "$exec_line" + + # 3. dedupe: --ctx and --ctx-size map to the same canonical flag → one token + # (validate_requested_flags dedupes alias-mapped flags). + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" --ctx 1024 --ctx-size 2048 + check_rc "start with alias pair exits 0" 0 "$TR_RC" + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_eq "--ctx-size emitted exactly once after dedupe" 1 "$(count_token --ctx-size "$exec_line")" + check_contains "dedupe keeps last value 2048" "--ctx-size 2048" "$exec_line" + + # 4. config-sourced values are emitted (no CLI). + printf 'LLAMACPP_CTX_SIZE=512\nLLAMACPP_PORT=9999\n' > "$cfg/ai.env" + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" + check_rc "start with config exits 0" 0 "$TR_RC" + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_contains "config LLAMACPP_CTX_SIZE=512 emitted" "--ctx-size 512" "$exec_line" + check_contains "config LLAMACPP_PORT=9999 emitted" "--port 9999" "$exec_line" + + # 5. CLI beats config (precedence contract: CLI > env > file > defaults). + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" --port 1234 + check_rc "start with CLI over config exits 0" 0 "$TR_RC" + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_contains "CLI --port 1234 beats config 9999" "--port 1234" "$exec_line" + check_not_contains "config port 9999 omitted when CLI given" "--port 9999" "$exec_line" + + # 6. only validated flags emitted: every default + requested flag appears + # exactly once in ExecStart. + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" --mmap + check_rc "start with --mmap exits 0" 0 "$TR_RC" + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_eq "--port exactly once" 1 "$(count_token --port "$exec_line")" + check_eq "--host exactly once" 1 "$(count_token --host "$exec_line")" + check_eq "--n-gpu-layers exactly once" 1 "$(count_token --n-gpu-layers "$exec_line")" + check_eq "--ctx-size exactly once" 1 "$(count_token --ctx-size "$exec_line")" + check_eq "--threads exactly once" 1 "$(count_token --threads "$exec_line")" + check_eq "--mmap exactly once" 1 "$(count_token --mmap "$exec_line")" +} \ No newline at end of file diff --git a/tests/t-config-precedence.sh b/tests/t-config-precedence.sh new file mode 100644 index 0000000..e603ee3 --- /dev/null +++ b/tests/t-config-precedence.sh @@ -0,0 +1,229 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-config-precedence.sh — canonical config loader + full precedence contract: +# CLI flags > environment > config file > defaults +# Part A: load_env_file (lib/config-ui.sh) contract — parsing, quotes, CRLF, +# comments, env-wins, missing-file no-op, basename under CONFIG_DIR. +# Part B: pos-ai-server behavioral dry-run (defaults → file → env → CLI). +# Part C: pos-communication-telegram-sender observable send URL (file/env/CLI). +# Part D: static guards — every migrated tool uses the shared loader; no +# hand-rolled load_system_env remains. + +run_test() { + local sandbox cfg stubs + sandbox="$(mksandbox config-precedence)" + cfg="$sandbox/cfg" + stubs="$sandbox/stubs" + mkdir -p "$cfg" "$stubs" + + # ═══ Part A: load_env_file contract ═══ + source "$ROOT/lib/config-ui.sh" + + local f="$sandbox/loader.env" + : > "$f" + + # A1: missing file = quiet no-op (rc 0, nothing exported) + unset MISSING_KEY || true + test_run load_env_file "$sandbox/does-not-exist.env" + check_rc "missing file is a quiet no-op" 0 "$TR_RC" + if [ -z "${MISSING_KEY:-}" ]; then + printf ' PASS missing file exports nothing\n' + else + printf ' FAIL missing file exported something\n' + fi + + # A2: KEY=VALUE parsed and exported, key registered in LOADED_ENV_KEYS + printf 'FOO=fileval\n' > "$f" + LOADED_ENV_KEYS=() + unset FOO || true + load_env_file "$f" + check_eq "file value exported" "fileval" "${FOO:-}" + check_contains "key registered in LOADED_ENV_KEYS" "FOO" "${LOADED_ENV_KEYS[*]}" + + # A3: environment wins over file (and file key NOT registered). The export + # must already exist in the environment when the loader runs — a temp + # `FOO=envval func` assignment would vanish with the call. + ( + unset FOO || true + export FOO=envval + LOADED_ENV_KEYS=() + load_env_file "$f" + printf '%s' "${FOO:-}" > "$sandbox/a3.val" + printf '%s' "${LOADED_ENV_KEYS[*]}" > "$sandbox/a3.keys" + ) + check_eq "environment beats file (env-wins)" "envval" "$(cat "$sandbox/a3.val")" + check_not_contains "env-won key not registered as file-loaded" "FOO" "$(cat "$sandbox/a3.keys")" + + # A4: CRLF line endings stripped + printf 'BAR=crlfvalue\r\n' > "$f" + LOADED_ENV_KEYS=() + unset BAR || true + load_env_file "$f" + check_eq "CRLF stripped" "crlfvalue" "${BAR:-}" + + # A5: surrounding quotes stripped (double and single) + printf 'BQ="doubleq"\nSQ='"'"'singleq'"'"'\n' > "$f" + LOADED_ENV_KEYS=() + unset BQ SQ || true + load_env_file "$f" + check_eq "double-quoted value unquoted" "doubleq" "${BQ:-}" + check_eq "single-quoted value unquoted" "singleq" "${SQ:-}" + + # A6: comments and blank lines skipped + printf '# COM=skipme\n\nREAL=yes\n' > "$f" + LOADED_ENV_KEYS=() + unset COM REAL || true + load_env_file "$f" + check_eq "non-comment key parsed" "yes" "${REAL:-}" + if [ -z "${COM:-}" ]; then + printf ' PASS comment key skipped\n' + else + printf ' FAIL comment key was loaded\n' + fi + + # A7: bare basename resolves under CONFIG_DIR + printf 'FROM_BASE=resolved\n' > "$cfg/ai.env" + LOADED_ENV_KEYS=() + unset FROM_BASE || true + CONFIG_DIR="$cfg" load_env_file "ai.env" + check_eq "basename resolved under CONFIG_DIR" "resolved" "${FROM_BASE:-}" + + # A8: only keys of the form KEY=VALUE (grep filter) are read + printf 'NOEQUALS\n=novalue\nOK=yes\n' > "$f" + LOADED_ENV_KEYS=() + unset NOEQUALS OK || true + load_env_file "$f" + check_eq "line without '=' ignored, valid line loaded" "yes" "${OK:-}" + if [ -z "${NOEQUALS:-}" ]; then + printf ' PASS bare line ignored\n' + else + printf ' FAIL bare line loaded\n' + fi + + # ═══ Part B: pos-ai-server behavioral precedence (dry-run ExecStart) ═══ + # llama-server stub (deps guard + flag validation) and nvidia-smi stub + # (forces CPU path → deterministic --n-gpu-layers 0). + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "llama.cpp 1.2.3" ;; + --help) + cat <<'HELP' +usage: llama-server [options] +options: + --host bind address + --port server port + --n-gpu-layers layers to offload + --ctx-size context size + --threads cpu threads + --mmap memory mapping +HELP + ;; +esac +STUB + printf '#!/usr/bin/env bash\nexit 1\n' > "$stubs/nvidia-smi" + chmod +x "$stubs/llama-server" "$stubs/nvidia-smi" + + local models="$sandbox/models" + mkdir -p "$models" + touch "$models/my-model.gguf" + local server="$ROOT/bin/pos-ai-server" + local base_env=(PATH="$stubs:/usr/bin:/bin" DRY_RUN=1 CONFIG_FILE="$cfg/ai.env") + local exec_line + : > "$cfg/ai.env" + + # B1: default when nothing set + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_contains "default port 8088 (no config/env/CLI)" "--port 8088" "$exec_line" + + # B2: config file beats default + printf 'LLAMACPP_PORT=9999\n' > "$cfg/ai.env" + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_contains "config file value used" "--port 9999" "$exec_line" + + # B3: env beats config file + test_run_env "${base_env[@]}" LLAMACPP_PORT=7777 -- "$server" start "$models/my-model.gguf" + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_contains "environment beats file" "--port 7777" "$exec_line" + check_not_contains "file value suppressed by env" "--port 9999" "$exec_line" + + # B4: CLI beats env (and therefore file + default) + test_run_env "${base_env[@]}" LLAMACPP_PORT=7777 -- "$server" start "$models/my-model.gguf" --port 5555 + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_contains "CLI beats env" "--port 5555" "$exec_line" + check_not_contains "env value suppressed by CLI" "--port 7777" "$exec_line" + + # ═══ Part C: telegram-sender observable send URL ═══ + local curl_log="$sandbox/curl.log" + : > "$curl_log" + cat > "$stubs/curl" <> "$curl_log" +printf '%s' '{"ok":true,"result":{"message_id":1}}' +STUB + chmod +x "$stubs/curl" + local sender="$ROOT/bin/pos-communication-telegram-sender" + local s_env=(PATH="$stubs:/usr/bin:/bin" CONFIG_DIR="$cfg") + + # C1: no token anywhere → fail-closed, no send + : > "$cfg/telegram.env" + test_run_env "${s_env[@]}" -- "$sender" send hello --chat-id 456 + check_contains "no token → hard error" "No bot token" "$TR_OUT" + [ "$TR_RC" -ne 0 ] && printf ' PASS no token exits nonzero\n' || printf ' FAIL no token exited 0\n' + + # C2: config file token used + printf 'TELEGRAM_BOT_TOKEN=FILE_TOKEN\n' > "$cfg/telegram.env" + test_run_env "${s_env[@]}" -- "$sender" send hello --chat-id 456 + check_rc "send succeeds from file config" 0 "$TR_RC" + check_contains "file token in send URL" "/botFILE_TOKEN/sendMessage" "$(cat "$curl_log")" + + # C3: env beats file + : > "$curl_log" + test_run_env "${s_env[@]}" TELEGRAM_BOT_TOKEN=ENV_TOKEN -- "$sender" send hello --chat-id 456 + check_contains "env token in send URL" "/botENV_TOKEN/sendMessage" "$(cat "$curl_log")" + check_not_contains "file token not used when env set" "/botFILE_TOKEN/sendMessage" "$(cat "$curl_log")" + + # C4: CLI flag beats env + : > "$curl_log" + test_run_env "${s_env[@]}" TELEGRAM_BOT_TOKEN=ENV_TOKEN -- "$sender" send hello --chat-id 456 --token CLI_TOKEN + check_contains "CLI token in send URL" "/botCLI_TOKEN/sendMessage" "$(cat "$curl_log")" + check_not_contains "env token suppressed by CLI" "/botENV_TOKEN/sendMessage" "$(cat "$curl_log")" + + # ═══ Part D: static migration guards ═══ + # every config-consuming tool must use the shared loader (D-D part 1) + local tool + for tool in pos-ai pos-ai-hf pos-ai-server pos-communication-matrix-listener \ + pos-communication-matrix-sender pos-communication-scrcpy \ + pos-communication-telegram-listener pos-communication-telegram-sender \ + pos-media-grab pos-network-download; do + if grep -q "load_env_file" "$ROOT/bin/$tool"; then + printf ' PASS %s uses shared load_env_file\n' "$tool" + else + printf ' FAIL %s does not use load_env_file\n' "$tool" + fi + done + # the three legacy load_system_env callers are DELIBERATE (documented in + # lib/config-ui.sh's loader NOTE) — assert no OTHER tool uses the legacy + # loader and the legacy function still exists for them. + local legacy + legacy="$(grep -l "load_system_env" "$ROOT"/bin/pos-* 2>/dev/null || true)" + local legacy_set + legacy_set="$(printf '%s\n' "$legacy" | grep -c . || true)" + check_eq "exactly the 3 documented legacy loaders remain" "3" "$legacy_set" + local lf + for lf in $legacy; do + case "$lf" in + */pos-media-sync|*/pos-system-backup|*/pos-system-health) + printf ' PASS legacy loader allowed: %s\n' "$(basename "$lf")" ;; + *) + printf ' FAIL unexpected legacy loader: %s\n' "$lf" ;; + esac + done + if grep -q "load_system_env()" "$ROOT/lib/common.sh"; then + printf ' PASS load_system_env still defined for the legacy callers\n' + else + printf ' FAIL load_system_env definition missing from lib/common.sh\n' + fi +} \ No newline at end of file diff --git a/tests/t-gen-docs-drift.sh b/tests/t-gen-docs-drift.sh new file mode 100644 index 0000000..603524e --- /dev/null +++ b/tests/t-gen-docs-drift.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-gen-docs-drift.sh — generated-docs drift gate: +# - `make gen` on a pristine tracked tree must be idempotent: running it +# twice produces zero diff (CI's `git diff --exit-code` style check), +# - the very first `make gen` on a baseline must succeed. +# Runs in a temp tracked-file copy (git ls-files) so the dirty working tree +# (parallel tracks) cannot influence the result. + +run_test() { + require_cmd git "gen-docs drift" || return 0 + require_cmd make "gen-docs drift" || return 0 + require_cmd timeout "gen-docs drift" || return 0 + + local sandbox copy + sandbox="$(mksandbox gen-docs-drift)" + copy="$sandbox/tree" + tracked_tree_copy "$copy" || { skip_case "gen-docs drift" "tracked tree copy failed"; return 0; } + + ( + cd "$copy" + git init -q + git config user.email test@example.invalid + git config user.name "test" + git add -A + git commit -qm base + + timeout 120 make gen >"$sandbox/gen1.out" 2>&1 + local rc1=$? + git add -A + git commit -qm "after gen 1" + + timeout 120 make gen >"$sandbox/gen2.out" 2>&1 + local rc2=$? + local dirty after2 + dirty="$(git status --porcelain)" + git diff --exit-code >/dev/null 2>&1 + local diffrc=$? + printf '%s %s\n' "$rc1" "$rc2" > "$sandbox/make-rcs" + printf '%s\n' "$dirty" > "$sandbox/dirty" + printf '%s\n' "$diffrc" > "$sandbox/diffrc" + ) + local ok=no + [ -f "$sandbox/make-rcs" ] && ok=yes + if [ "$ok" = "no" ]; then + printf ' SKIP gen-docs drift (git/make unavailable in sandbox)\n' + return 0 + fi + + local make_rcs dirty diffrc + make_rcs="$(cat "$sandbox/make-rcs")" + dirty="$(cat "$sandbox/dirty")" + diffrc="$(cat "$sandbox/diffrc")" + local rc1="${make_rcs%% *}" rc2="${make_rcs##* }" + check_rc "first make gen succeeds" 0 "$rc1" + check_rc "second make gen succeeds (idempotence run)" 0 "$rc2" + check_eq "no dirty files after second gen" "" "$dirty" + check_rc "git diff --exit-code clean after second gen" 0 "$diffrc" +} \ No newline at end of file diff --git a/tests/t-gpg-password.sh b/tests/t-gpg-password.sh new file mode 100644 index 0000000..d12caa6 --- /dev/null +++ b/tests/t-gpg-password.sh @@ -0,0 +1,108 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-gpg-password.sh — backup encryption passphrase handling (D3): +# - passphrase fed to gpg on fd 3 (here-string), NEVER as argv token, +# - plaintext archive removed after successful encryption; only .gpg remains, +# - encryption failure: plaintext removed, rc != 0, nothing left behind, +# - verification failure: corrupt .gpg removed, rc != 0, +# - end-to-end success path. gpg/sudo stubbed; tar real; stdin piped (no tty). + +run_test() { + require_cmd timeout "gpg password" || return 0 + command -v tar >/dev/null 2>&1 || { skip_case "gpg password" "tar not available"; return 0; } + + local sandbox stubs work data + sandbox="$(mksandbox gpg-password)" + stubs="$sandbox/stubs" + work="$sandbox/work" + data="$sandbox/data/My Data" + mkdir -p "$stubs" "$work" "$data" + printf 'important file content\n' > "$data/notes.txt" + + local gpg_log="$sandbox/gpg.log" + : > "$gpg_log" + + printf 'canned\n' > "$sandbox/canned.txt" + tar -czf "$sandbox/fake.tar.gz" -C "$sandbox" canned.txt || \ + { skip_case "gpg password" "cannot create canned tar fixture"; return 0; } + + # stub gpg: log every argv token (one per line), simulate failures via + # STUB_GPG_FAIL, decrypt phase emits canned tar.gz stream + cat > "$stubs/gpg" <> "$gpg_log" +input="" +for a in "\$@"; do + case "\$a" in + -*) ;; + *) input="\$a" ;; + esac +done +case "\$*" in + *--symmetric*) + if [ "\${STUB_GPG_FAIL:-}" = "encrypt" ]; then exit 1; fi + cp "\$input" "\$input.gpg" + ;; + *--decrypt*) + if [ "\${STUB_GPG_FAIL:-}" = "verify" ]; then exit 1; fi + cat "$sandbox/fake.tar.gz" + ;; + *) ;; +esac +STUB + printf '#!/usr/bin/env bash\nexec "$@"\n' > "$stubs/sudo" + chmod +x "$stubs/gpg" "$stubs/sudo" + + local backup="$ROOT/bin/pos-system-backup" + local common_env=(PATH="$stubs:/usr/bin:/bin" NOTIFY_PLATFORM="" + BACKUP_USB_ROOT="$sandbox/usbroot") + + # 1. success path + ( cd "$work" && printf 's3cr3t-pass\ns3cr3t-pass\n' \ + | timeout 30 env "${common_env[@]}" "$backup" "$data" >"$sandbox/run1.out" 2>&1 ) + local rc1=$? + check_rc "backup success exits 0" 0 "$rc1" + check_file_exists "encrypted artifact created (.tar.gz.gpg)" "$work"/*.tar.gz.gpg + check_eq "plaintext archive removed after encryption" 0 \ + "$(find "$work" -maxdepth 1 -name '*.tar.gz' | wc -l)" + check_contains "success logs completion" "Backup completed" "$(cat "$sandbox/run1.out")" + check_contains "gpg invoked with --passphrase-fd" "--passphrase-fd" "$(cat "$gpg_log")" + check_eq "gpg passphrase-fd used twice (encrypt+decrypt)" 2 \ + "$(count_token --passphrase-fd "$(cat "$gpg_log")")" + # the security property: no bare --passphrase argv token (check token-exact, + # because --passphrase-fd legitimately CONTAINS the substring) + local bare + bare="$(grep -c '^--passphrase$' "$gpg_log" || true)" + check_eq "never a bare --passphrase argv token" 0 "$bare" + check_not_contains "secret never appears in gpg argv" "s3cr3t-pass" "$(cat "$gpg_log")" + + # 2. encryption failure → plaintext removed, rc != 0, nothing left + rm -rf "$work"; mkdir -p "$work" + : > "$gpg_log" + ( cd "$work" && printf 's3cr3t-pass\ns3cr3t-pass\n' \ + | timeout 30 env "${common_env[@]}" STUB_GPG_FAIL=encrypt "$backup" "$data" >"$sandbox/run2.out" 2>&1 ) + local rc2=$? + check_contains "encrypt-failure reports cleanup" "plaintext archive removed" "$(cat "$sandbox/run2.out")" + if [ "$rc2" -ne 0 ]; then + printf ' PASS encryption failure exits nonzero\n' + else + printf ' FAIL encryption failure exited 0\n' + fi + check_eq "nothing left behind after encrypt failure" 0 \ + "$(find "$work" -maxdepth 1 \( -name '*.tar.gz' -o -name '*.tar.gz.gpg' \) | wc -l)" + + # 3. verification failure → corrupt .gpg removed, rc != 0, nothing left + rm -rf "$work"; mkdir -p "$work" + : > "$gpg_log" + ( cd "$work" && printf 's3cr3t-pass\ns3cr3t-pass\n' \ + | timeout 30 env "${common_env[@]}" STUB_GPG_FAIL=verify "$backup" "$data" >"$sandbox/run3.out" 2>&1 ) + local rc3=$? + check_contains "verify-failure reports cleanup" "corrupt artifact removed" "$(cat "$sandbox/run3.out")" + if [ "$rc3" -ne 0 ]; then + printf ' PASS verification failure exits nonzero\n' + else + printf ' FAIL verification failure exited 0\n' + fi + check_eq "nothing left behind after verify failure" 0 \ + "$(find "$work" -maxdepth 1 \( -name '*.tar.gz' -o -name '*.tar.gz.gpg' \) | wc -l)" +} \ No newline at end of file diff --git a/tests/t-lint-gate.sh b/tests/t-lint-gate.sh new file mode 100644 index 0000000..5d16efb --- /dev/null +++ b/tests/t-lint-gate.sh @@ -0,0 +1,58 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-lint-gate.sh — convention lint gate behavior: +# positive: `make lint` on the real tree passes with "0 FAIL, 0 WARN"; +# negative: a planted tool without a `# POS:` header and a planted app +# without an uninstall function are both reported by name (rc != 0). +# The negative case runs in a temp tracked-file copy so the dirty tree state +# cannot mask or fabricate violations. + +run_test() { + require_cmd make "lint gate" || return 0 + + local lint="$ROOT/scripts/lint-conventions.sh" + + # ── positive: real tree lints clean ── + test_run timeout 120 bash "$lint" + check_rc "lint on real tree exits 0" 0 "$TR_RC" + check_contains "lint reports zero failures" "0 FAIL, 0 WARN" "$TR_OUT" + + # ── negative: planted violations are caught and named ── + local sandbox copy + sandbox="$(mksandbox lint-gate)" + copy="$sandbox/tree" + tracked_tree_copy "$copy" || { skip_case "lint gate" "tracked tree copy failed"; return 0; } + + # plant 1: executable tool without a # POS: header + cat > "$copy/bin/pos-zz-test-broken" <<'EOF' +#!/usr/bin/env bash +set -euo pipefail +usage() { cat < "$copy/apps/zztest/zztest.sh" <<'EOF' +#!/usr/bin/env bash +set -euo pipefail +echo "zztest installed" +EOF + + # lint resolves its own ROOT from $0 and `cd`s there — so run the COPY's + # lint (relative path) to lint the copy, not the real repo + ( cd "$copy" && timeout 120 bash scripts/lint-conventions.sh ) >"$sandbox/lint.out" 2>&1 + local rc=$? + if [ "$rc" -eq 0 ]; then + printf ' FAIL lint did not reject planted violations (rc 0)\n' + else + printf ' PASS lint rejects planted violations (rc %s)\n' "$rc" + fi + check_contains "violating tool named in lint output" "pos-zz-test-broken" "$(cat "$sandbox/lint.out")" + check_contains "violating app named in lint output" "zztest" "$(cat "$sandbox/lint.out")" +} \ No newline at end of file diff --git a/tests/t-matrix-auth.sh b/tests/t-matrix-auth.sh new file mode 100644 index 0000000..4164cc8 --- /dev/null +++ b/tests/t-matrix-auth.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-matrix-auth.sh — Matrix listener authorization (D2): +# - with MATRIX_ROOM_ID[+MATRIX_USER_ID] set: ONLY the owner in the watched +# room runs commands; other senders / other rooms are ignored; +# - with MATRIX_ROOM_ID unset: fail-closed, no command ever runs; +# - exactly one reply (m.room.message send) per authorized event. +# Stub curl serves the sync batch once then empty batches; timeout kills the +# daemon (TERM trap → exit 0). + +run_test() { + require_cmd jq "matrix auth" || return 0 + require_cmd timeout "matrix auth" || return 0 + + local sandbox stubs cfg marker curl_log + sandbox="$(mksandbox matrix-auth)" + stubs="$sandbox/stubs" + cfg="$sandbox/cfg" + marker="$sandbox/executed.ping" + curl_log="$sandbox/curl.log" + mkdir -p "$stubs" "$cfg" + : > "$curl_log" + + local batch="$sandbox/batch.json" + cat > "$batch" <<'JSON' +{"next_batch":"b1","rooms":{"join":{ + "!room:example.org":{"timeline":{"events":[ + {"type":"m.room.message","event_id":"evt1","sender":"@owner:example.org","content":{"msgtype":"m.text","body":"/ping"}}, + {"type":"m.room.message","event_id":"evt2","sender":"@other:example.org","content":{"msgtype":"m.text","body":"/ping"}} + ]}}, + "!other:example.org":{"timeline":{"events":[ + {"type":"m.room.message","event_id":"evt3","sender":"@owner:example.org","content":{"msgtype":"m.text","body":"/ping"}}, + {"type":"m.room.message","event_id":"evt4","sender":"@other:example.org","content":{"msgtype":"m.text","body":"/ping"}} + ]}} +}}} +JSON + cat > "$stubs/curl" <> "$curl_log" +for a in "\$@"; do + case "\$a" in + *sync*) + if [ ! -e "$sandbox/served.once" ]; then + touch "$sandbox/served.once" + cat "$batch" + else + printf '%s' '{"next_batch":"b2"}' + fi + exit 0 + ;; + esac +done +printf '%s' '{"ok":true}' +STUB + chmod +x "$stubs/curl" + + : > "$cfg/matrix_commands.env" + printf '/ping=touch %s\n' "$marker" >> "$cfg/matrix_commands.env" + + local listener="$ROOT/bin/pos-communication-matrix-listener" + local common=(PATH="$stubs:/usr/bin:/bin" CONFIG_DIR="$cfg" + MATRIX_HOMESERVER=https://matrix.example.org MATRIX_ACCESS_TOKEN=secret + MATRIX_USER_ID=@owner:example.org MATRIX_ROOM_ID=!room:example.org) + + # ── run 1: room + owner set → exactly one command runs ── + rm -f "$sandbox/served.once" "$marker"; : > "$curl_log" + # --preserve-status + --kill-after surface the daemon's own TERM-trap exit, + # so a broken trap (daemon-hang regression → SIGKILL 137 / timeout 124) + # genuinely fails the rc assert instead of passing vacuously. --kill-after + # also bounds the wait so a hung daemon can't stall the whole suite. + test_run_env "${common[@]}" -- timeout --preserve-status -k 2 5 "$listener" --run + check_rc "daemon terminated via TERM trap, not killed (no hang)" 2 "$TR_RC" + check_contains "authorized /ping in watched room ran" "exec: /ping" "$TR_OUT" + check_file_exists "authorized command created marker" "$marker" + check_eq "exactly one reply sent (no reply loop)" 1 "$(count_occurrences '/send/m.room.message' "$(cat "$curl_log")")" + check_not_contains "no unknown-command reply for ignored events" "Unknown command" "$TR_OUT" + + # ── run 2: MATRIX_ROOM_ID unset → fail-closed, nothing runs ── + rm -f "$sandbox/served.once" "$marker"; : > "$curl_log" + test_run_env PATH="$stubs:/usr/bin:/bin" CONFIG_DIR="$cfg" \ + MATRIX_HOMESERVER=https://matrix.example.org MATRIX_ACCESS_TOKEN=secret \ + MATRIX_USER_ID=@owner:example.org -- timeout --preserve-status -k 2 5 "$listener" --run + check_rc "room-unset daemon terminated via TERM trap, not killed (no hang)" 2 "$TR_RC" + check_contains "room-unset fail-closed warning" "MATRIX_ROOM_ID unset — refusing to run commands" "$TR_OUT" + check_not_contains "no exec without MATRIX_ROOM_ID" "exec: /ping" "$TR_OUT" + check_file_absent "no marker without MATRIX_ROOM_ID" "$marker" + check_eq "no replies without MATRIX_ROOM_ID" 0 "$(count_occurrences '/send/m.room.message' "$(cat "$curl_log")")" +} \ No newline at end of file diff --git a/tests/t-systemd-unit.sh b/tests/t-systemd-unit.sh new file mode 100644 index 0000000..991b165 --- /dev/null +++ b/tests/t-systemd-unit.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-systemd-unit.sh — generated systemd unit correctness (D5): +# - exactly one ExecStart= line, binary + model paths quoted (spaces safe), +# - each runtime flag emitted exactly once, +# - EnvironmentFile= present with the ai.env path, +# - `systemd-analyze verify` passes on the generated unit (skip if the +# analyzer is unavailable). +# Runs the REAL unit-write path: stub systemctl/curl/llama-server so no +# system service or network is touched. + +run_test() { + local sandbox stubs unitdir models + sandbox="$(mksandbox systemd-unit)" + stubs="$sandbox/stubs" + unitdir="$sandbox/userunits" + models="$sandbox/models" + mkdir -p "$stubs" "$unitdir" "$models" + # model path deliberately contains spaces + touch "$models/my model file.gguf" + + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "llama.cpp 1.2.3" ;; + --help) + cat <<'HELP' +usage: llama-server [options] +options: + --host bind address + --port server port + --n-gpu-layers layers to offload + --ctx-size context size + --threads cpu threads + --mmap memory mapping +HELP + ;; +esac +STUB + # systemctl: pretend every operation succeeds silently. + printf '#!/usr/bin/env bash\nexit 0\n' > "$stubs/systemctl" + # curl (health check) returns an "ok" JSON body. + printf '#!/usr/bin/env bash\nprintf "%%s" '"'"'{"status":"ok"}'"'"'\n' > "$stubs/curl" + # nvidia-smi: no GPU → cpu path deterministic. + printf '#!/usr/bin/env bash\nexit 1\n' > "$stubs/nvidia-smi" + chmod +x "$stubs/llama-server" "$stubs/systemctl" "$stubs/curl" "$stubs/nvidia-smi" + + local server="$ROOT/bin/pos-ai-server" + local model="$models/my model file.gguf" + + # REAL run: writes the unit to USER_SYSTEMD_DIR, waits ~2s for health. + test_run_env PATH="$stubs:/usr/bin:/bin" USER_SYSTEMD_DIR="$unitdir" -- \ + timeout 60 "$server" start "$model" + check_rc "real start writes unit and exits 0" 0 "$TR_RC" + check_file_exists "unit file created" "$unitdir/pos-ai-server.service" + + local unit="$unitdir/pos-ai-server.service" + local exec_lines unit_exec + exec_lines="$(grep -c '^ExecStart=' "$unit" || true)" + check_eq "exactly one ExecStart= line" 1 "$exec_lines" + unit_exec="$(grep '^ExecStart=' "$unit")" + check_contains "binary path (with spaces) quoted" "\"$stubs/llama-server\"" "$unit_exec" + check_contains "model path (with spaces) double-quoted in unit" "\"$model\"" "$unit_exec" + check_contains "port flag in unit" "--port 8088" "$unit_exec" + check_contains "host flag in unit" "--host 127.0.0.1" "$unit_exec" + check_contains "gpu layers flag in unit" "--n-gpu-layers 0" "$unit_exec" + check_contains "ctx-size flag in unit" "--ctx-size 4096" "$unit_exec" + check_contains "EnvironmentFile ai.env referenced" "EnvironmentFile=-%h/.config/linux_post_install/ai.env" "$(cat "$unit")" + + if command -v systemd-analyze >/dev/null 2>&1; then + test_run systemd-analyze verify "$unit" + check_rc "systemd-analyze verify accepts generated unit" 0 "$TR_RC" + else + skip_case "systemd-analyze verify" "systemd-analyze not available" + fi +} \ No newline at end of file diff --git a/tests/t-telegram-auth.sh b/tests/t-telegram-auth.sh new file mode 100644 index 0000000..ce75709 --- /dev/null +++ b/tests/t-telegram-auth.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-telegram-auth.sh — Telegram listener authorization (D1): +# - chat ID AND sender ID must both match (previously chat-OR-sender); +# - unset TELEGRAM_OWNER_ID → fail-closed, no command ever runs; +# - authorized message runs the mapped command exactly once; +# - exactly one sendMessage for the authorized message (no reply loop). +# jq is a hard dep of the listener; stub curl serves a canned getUpdates batch +# once, then empty batches forever; timeout kills the daemon. + +run_test() { + require_cmd jq "telegram auth" || return 0 + require_cmd timeout "telegram auth" || return 0 + + local sandbox stubs cfg marker curl_log + sandbox="$(mksandbox telegram-auth)" + stubs="$sandbox/stubs" + cfg="$sandbox/cfg" + marker="$sandbox/executed.ping" + curl_log="$sandbox/curl.log" + mkdir -p "$stubs" "$cfg" + : > "$curl_log" + + # ── stub curl: log argv; serve one getUpdates batch then empty; answer + # sendMessage/setMyCommands with {ok:true}. + local batch="$sandbox/batch.json" + cat > "$batch" <<'JSON' +{"ok":true,"result":[ + {"update_id":1,"message":{"message_id":10,"from":{"id":123,"is_bot":false},"chat":{"id":456,"type":"private"},"date":0,"text":"/ping"}}, + {"update_id":2,"message":{"message_id":11,"from":{"id":999,"is_bot":false},"chat":{"id":456,"type":"private"},"date":0,"text":"/ping"}}, + {"update_id":3,"message":{"message_id":12,"from":{"id":123,"is_bot":false},"chat":{"id":789,"type":"private"},"date":0,"text":"/ping"}}, + {"update_id":4,"message":{"message_id":13,"from":{"id":999,"is_bot":false},"chat":{"id":789,"type":"private"},"date":0,"text":"/ping"}} +]} +JSON + cat > "$stubs/curl" <> "$curl_log" +for a in "\$@"; do + case "\$a" in + *getUpdates*) + if [ ! -e "$sandbox/served.once" ]; then + touch "$sandbox/served.once" + cat "$batch" + else + printf '%s' '{"ok":true,"result":[]}' + fi + exit 0 + ;; + esac +done +printf '%s' '{"ok":true}' +STUB + chmod +x "$stubs/curl" + + : > "$cfg/telegram_commands.env" + printf '/ping=touch %s\n' "$marker" >> "$cfg/telegram_commands.env" + + local listener="$ROOT/bin/pos-communication-telegram-listener" + + # ── run 1: all 4 authorization cases in one batch ── + local common=(PATH="$stubs:/usr/bin:/bin" CONFIG_DIR="$cfg" + TELEGRAM_BOT_TOKEN=testbot TELEGRAM_CHAT_ID=456 TELEGRAM_OWNER_ID=123) + rm -f "$sandbox/served.once" "$marker"; : > "$curl_log" + test_run_env "${common[@]}" -- timeout 5 "$listener" --run + check_contains "authorized /ping from owner chat ran" "exec: /ping" "$TR_OUT" + check_file_exists "authorized command created marker" "$marker" + check_eq "exactly one sendMessage (no reply loop)" 1 "$(count_occurrences sendMessage "$(cat "$curl_log")")" + check_eq "three unauthorized messages warned" 3 "$(count_occurrences 'ignoring message in chat' "$TR_OUT")" + check_not_contains "no unknown-command reply (wrong-chat/from msgs never handled)" "Unknown command" "$TR_OUT" + + # ── run 2: owner id unset → fail-closed, nothing runs, no sends ── + rm -f "$sandbox/served.once" "$marker"; : > "$curl_log" + test_run_env PATH="$stubs:/usr/bin:/bin" CONFIG_DIR="$cfg" \ + TELEGRAM_BOT_TOKEN=testbot TELEGRAM_CHAT_ID=456 -- \ + timeout 5 "$listener" --run + check_contains "owner-unset fail-closed warning" "TELEGRAM_OWNER_ID unset — ignoring command" "$TR_OUT" + check_file_absent "no marker without owner id" "$marker" + check_eq "no sendMessage without owner id" 0 "$(count_occurrences sendMessage "$(cat "$curl_log")")" +} \ No newline at end of file diff --git a/tests/t-uninstall-manifest.sh b/tests/t-uninstall-manifest.sh new file mode 100644 index 0000000..bb33e13 --- /dev/null +++ b/tests/t-uninstall-manifest.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-uninstall-manifest.sh — install/uninstall symmetry (D-E): +# - the installed-file manifest (POS_LIBS in pos-system-uninstall) matches +# install.sh's phase-2 lib list EXACTLY (set equality) and every named +# lib exists, +# - scan_tier1() user-unit discovery honors XDG_CONFIG_HOME and only picks +# pos-* units (never unrelated user units) — extracted function run in a +# sandboxed XDG_CONFIG_HOME, +# - plugin removal stays marker-driven (POS_PLUGIN), not hardcoded. + +run_test() { + local sandbox + sandbox="$(mksandbox uninstall-manifest)" + local uninstall="$ROOT/bin/pos-system-uninstall" + + # ── static manifest equality ── + local install_libs uninstall_libs + install_libs="$(sed -n 's/.*for lf in \(.*\); do.*/\1/p' "$ROOT/install.sh" | head -1)" + if [ -z "$install_libs" ]; then + printf ' FAIL could not extract install.sh lib list\n' + return 0 + fi + # POS_LIBS spans one or two lines: "(… \⏎ …)" or "(…)". Strip markers, + # backslash continuations and parens, join to one token string. + uninstall_libs="$(sed -n '/^POS_LIBS=(/,/)$/p' "$uninstall" \ + | sed -e '1s/^POS_LIBS=(//' -e 's/[()\\]//g' \ + | tr '\n' ' ')" + # normalize whitespace (line-continuation gap collapses to single spaces) + uninstall_libs="$(printf '%s' "$uninstall_libs" | tr -s ' \t\n' ' ' | sed -e 's/^ //' -e 's/ $//')" + + local il uil + il="$(printf '%s' "$install_libs" | tr ' ' '\n' | sort | tr '\n' ' ')" + uil="$(printf '%s' "$uninstall_libs" | tr ' ' '\n' | sort | tr '\n' ' ')" + check_eq "install.sh lib list == POS_LIBS (sorted)" "$il" "$uil" + + local lib + for lib in $install_libs; do + if [ -f "$ROOT/lib/$lib" ]; then + printf ' PASS lib/%s exists\n' "$lib" + else + printf ' FAIL lib/%s missing\n' "$lib" + fi + done + + # ── behavioral: user-unit discovery honors XDG_CONFIG_HOME, pos-* only ── + local fakeroot="$sandbox/fakeroot" + mkdir -p "$fakeroot/systemd/user" + touch "$fakeroot/systemd/user/pos-aria2.service" + touch "$fakeroot/systemd/user/pos-entertainment-weather.timer" + touch "$fakeroot/systemd/user/unrelated.service" + + # extract POS_LIBS + installed_plugins + scan_tier1 verbatim + local extract + extract="$(awk ' + /^POS_LIBS=\(/ { infn=1 } + /^scan_tier1\(\) \{/ { inscan=1 } + infn { print } + inscan && /^}/ { exit } + ' "$uninstall")" + if [ -z "$extract" ]; then + printf ' FAIL could not extract scan_tier1 function body\n' + return 0 + fi + local runner="$sandbox/run-scan.sh" + { + printf '#!/usr/bin/env bash\nset -euo pipefail\n' + printf '%s\n' "$extract" + printf '\nscan_tier1\n' + } > "$runner" + + test_run env XDG_CONFIG_HOME="$fakeroot" bash "$runner" + check_contains "scan_tier1 finds pos-* unit file" "user-unit: pos-aria2.service" "$TR_OUT" + check_contains "scan_tier1 finds pos-* timer file" "user-unit: pos-entertainment-weather.timer" "$TR_OUT" + check_not_contains "scan_tier1 ignores unrelated user unit" "user-unit: unrelated.service" "$TR_OUT" + + # ── plugin removal stays marker-driven ── + if grep -q "POS_PLUGIN" "$uninstall"; then + printf ' PASS plugin removal is POS_PLUGIN-marker driven\n' + else + printf ' FAIL plugin removal is hardcoded (no POS_PLUGIN marker)\n' + fi + if command -v compgen >/dev/null 2>&1; then + printf ' PASS compgen-based discovery available (bash builtin)\n' + else + printf ' FAIL compgen not available\n' + fi +} \ No newline at end of file diff --git a/tests/t-unsupported-flags.sh b/tests/t-unsupported-flags.sh new file mode 100644 index 0000000..8d83dfc --- /dev/null +++ b/tests/t-unsupported-flags.sh @@ -0,0 +1,153 @@ +#!/usr/bin/env bash +set -euo pipefail +# t-unsupported-flags.sh — unsupported-option handling (D4): +# - CLI-explicit feature the model does not expose → hard error naming the +# unsupported flag and the detected llama version, +# - config-requested / env-requested unsupported flags → same hard error, +# - unsupported *default* flags → silently dropped with a warning (start +# still proceeds), never written into ExecStart, +# - word-boundary matching: `--no-mmap` in help must NOT satisfy a request +# for `--mmap` (the old grep -qF substring bug). + +run_test() { + local sandbox stubs cfg models + sandbox="$(mksandbox unsupported-flags)" + stubs="$sandbox/stubs" + cfg="$sandbox/cfg" + models="$sandbox/models" + mkdir -p "$stubs" "$cfg" "$models" + : > "$cfg/ai.env" + touch "$models/my-model.gguf" + + local server="$ROOT/bin/pos-ai-server" + local base_env=(PATH="$stubs:/usr/bin:/bin" DRY_RUN=1 + CONFIG_FILE="$cfg/ai.env" USER_SYSTEMD_DIR="$sandbox/userunits") + + # stub that omits --tensor-split (every default flag supported) + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "llama.cpp 1.2.3" ;; + --help) + cat <<'HELP' +usage: llama-server [options] +options: + --host bind address + --port server port + --n-gpu-layers layers to offload + --ctx-size context size + --threads cpu threads + --mmap memory mapping +HELP + ;; +esac +STUB + chmod +x "$stubs/llama-server" + + # 1. CLI-explicit unsupported flag → rc 1, names flag + model version + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" --tensor-split 1:2:3 + check_contains "CLI unsupported --tensor-split: rc message" "does not expose" "$TR_OUT" + check_contains "names the unsupported flag" "--tensor-split" "$TR_OUT" + check_contains "names the detected model version" "1.2.3" "$TR_OUT" + [ "$TR_RC" -ne 0 ] && printf ' PASS CLI unsupported flag exits nonzero\n' || printf ' FAIL CLI unsupported flag exits 0\n' + + # 2. unsupported flag requested from CONFIG → same hard error (config + # requested flags go through the same validation) + printf 'LLAMACPP_PORT=9999\n' > "$cfg/ai.env" + # stub without --port in help: config-requested --port is unsupported + sed -i '/--port/d' "$stubs/llama-server" + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" + check_contains "config-requested unsupported flag: message" "does not expose" "$TR_OUT" + check_contains "config-requested flag named" "--port" "$TR_OUT" + [ "$TR_RC" -ne 0 ] && printf ' PASS config-requested unsupported flag exits nonzero\n' || printf ' FAIL config-requested unsupported flag exits 0\n' + + # 3. same via environment export (env feeds config-requested flags too) + local envcfg="$sandbox/envcfg" + mkdir -p "$envcfg" + : > "$envcfg/ai.env" + test_run_env PATH="$stubs:/usr/bin:/bin" DRY_RUN=1 \ + CONFIG_FILE="$envcfg/ai.env" USER_SYSTEMD_DIR="$sandbox/userunits" \ + LLAMACPP_PORT=9999 -- "$server" start "$models/my-model.gguf" + check_contains "env-requested unsupported flag: message" "does not expose" "$TR_OUT" + check_contains "env-requested flag named" "--port" "$TR_OUT" + [ "$TR_RC" -ne 0 ] && printf ' PASS env-requested unsupported flag exits nonzero\n' || printf ' FAIL env-requested unsupported flag exits 0\n' + + # 4. unsupported default flag: start SUCCEEDS, warns, omits the flag + # (--threads absent from help; every other default supported) + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "llama.cpp 1.2.3" ;; + --help) + cat <<'HELP' +usage: llama-server [options] +options: + --host bind address + --port server port + --n-gpu-layers layers to offload + --ctx-size context size + --mmap memory mapping +HELP + ;; +esac +STUB + chmod +x "$stubs/llama-server" + : > "$cfg/ai.env" + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" + check_rc "start succeeds with unsupported default" 0 "$TR_RC" + check_contains "warns about dropped default flag" "--threads" "$TR_OUT" + local exec_line + exec_line="$(printf '%s\n' "$TR_OUT" | grep 'dry-run) ExecStart:' | sed 's/.*ExecStart: //')" + check_not_contains "dropped default never reaches ExecStart" "--threads" "$exec_line" + check_contains "other defaults still emitted" "--n-gpu-layers 0" "$exec_line" + + # 5. word-boundary bug (D4): help advertises --no-mmap, NOT --mmap + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "llama.cpp 1.2.3" ;; + --help) + cat <<'HELP' +usage: llama-server [options] +options: + --no-mmap disable memory mapping + --host bind address + --port server port + --n-gpu-layers layers to offload + --ctx-size context size + --threads cpu threads +HELP + ;; +esac +STUB + chmod +x "$stubs/llama-server" + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" --mmap + check_contains "requested --mmap rejected (only --no-mmap in help)" "does not expose" "$TR_OUT" + check_contains "rejection names --mmap" "--mmap" "$TR_OUT" + [ "$TR_RC" -ne 0 ] && printf ' PASS word-boundary: --no-mmap does not satisfy --mmap (rc != 0)\n' || printf ' FAIL word-boundary: --no-mmap wrongly satisfied --mmap (rc 0)\n' + + # 6. control: full-support stub → all defaults + requested flags pass + cat > "$stubs/llama-server" <<'STUB' +#!/usr/bin/env bash +case "$1" in + --version) echo "llama.cpp 1.2.3" ;; + --help) + cat <<'HELP' +usage: llama-server [options] +options: + --host bind address + --port server port + --n-gpu-layers layers to offload + --ctx-size context size + --threads cpu threads + --mmap memory mapping + --tensor-split tensor split +HELP + ;; +esac +STUB + chmod +x "$stubs/llama-server" + test_run_env "${base_env[@]}" -- "$server" start "$models/my-model.gguf" --mmap --tensor-split 1:2:3 + check_rc "control: full-support stub starts cleanly" 0 "$TR_RC" + check_not_contains "control: no unsupported warning" "does not expose" "$TR_OUT" +} \ No newline at end of file diff --git a/tests/test-lib.sh b/tests/test-lib.sh new file mode 100644 index 0000000..f52a63c --- /dev/null +++ b/tests/test-lib.sh @@ -0,0 +1,159 @@ +#!/usr/bin/env bash +# tests/test-lib.sh — shared helpers for t-*.sh test files. +# Sourced by tests/run-tests.sh inside each test's isolated subshell. +# Keep this file dependency-free (bash builtins + coreutils only). + +# ── assertions ────────────────────────────────────────────────── +# Each check prints PASS/FAIL to stdout; counts are derived by the runner +# from the log (lines match '^ PASS ' / '^ FAIL ' / '^ SKIP '). + +check_eq() { # check_eq + local desc="$1" expected="$2" actual="$3" + if [ "$expected" = "$actual" ]; then + printf ' PASS %s\n' "$desc" + else + printf ' FAIL %s (expected=[%s] actual=[%s])\n' "$desc" "$expected" "$actual" + fi +} + +check_rc() { # check_rc + local desc="$1" + check_eq "$desc (rc)" "$2" "$3" +} + +check_contains() { # check_contains + local desc="$1" needle="$2" haystack="$3" + case "$haystack" in + *"$needle"*) printf ' PASS %s (contains: %s)\n' "$desc" "$needle" ;; + *) printf ' FAIL %s (missing: [%s] in [%s])\n' "$desc" "$needle" "$haystack" ;; + esac +} + +check_not_contains() { # check_not_contains + local desc="$1" needle="$2" haystack="$3" + case "$haystack" in + *"$needle"*) printf ' FAIL %s (unexpected: [%s] found in [%s])\n' "$desc" "$needle" "$haystack" ;; + *) printf ' PASS %s (absent: %s)\n' "$desc" "$needle" ;; + esac +} + +check_file_exists() { # check_file_exists + if [ -e "$2" ]; then + printf ' PASS %s (exists: %s)\n' "$1" "$2" + else + printf ' FAIL %s (missing file: %s)\n' "$1" "$2" + fi +} + +check_file_absent() { # check_file_absent + if [ -e "$2" ]; then + printf ' FAIL %s (unexpected file: %s)\n' "$1" "$2" + else + printf ' PASS %s (absent: %s)\n' "$1" "$2" + fi +} + +count_occurrences() { # count_occurrences → echo count + local needle="$1" haystack="$2" n=0 line + while IFS= read -r line; do + case "$line" in + *"$needle"*) n=$((n + 1)) ;; + esac + done <<< "$haystack" + printf '%s' "$n" +} + +# ── execution helpers ─────────────────────────────────────────── +# test_run — runs a command that may fail, captures combined output +# into $TR_OUT and exit code into $TR_RC. Never triggers set -e. +test_run() { + set +e + TR_OUT="$("$@" 2>&1)" + TR_RC=$? + set -e +} + +# test_run_env -- — env overrides then run. +test_run_env() { + local args=() + while [ $# -gt 0 ]; do + if [ "$1" = "--" ]; then + shift + break + fi + args+=("$1") + shift + done + local envs=("${args[@]}") + set +e + TR_OUT="$(env "${envs[@]}" "$@" 2>&1)" + TR_RC=$? + set -e +} + +# ── sandbox helpers ───────────────────────────────────────────── +# mksandbox — fresh temp dir under $TEST_TMP; auto-cleaned by runner. +mksandbox() { + local name="${1:-sandbox}" + local d="$TEST_TMP/$name" + mkdir -p "$d" + printf '%s' "$d" +} + +# write_env_file ... — deterministic env-file fixture. +write_env_file() { + local path="$1"; shift + local line + for line in "$@"; do + printf '%s\n' "$line" >>"$path" + done +} + +# ── skip contract ─────────────────────────────────────────────── +# skip_case — a case that cannot run in this environment. +skip_case() { + printf ' SKIP %s (%s)\n' "$1" "$2" +} + +# require_cmd — SKIP + abort the current function (usage: +# require_cmd jq "telegram auth" || return 0 +require_cmd() { + if command -v "$1" >/dev/null 2>&1; then + return 0 + fi + skip_case "$2" "$1 not available" + return 1 +} + +# ── repo helpers ──────────────────────────────────────────────── +# repo_root — absolute path of the checkout under test. +repo_root() { + printf '%s' "$ROOT" +} + +# tool_path — absolute path of a bin/pos-* tool. +tool_path() { + printf '%s' "$ROOT/bin/$1" +} + +# tracked_tree_copy — copy only git-tracked files (works on dirty +# trees; excludes .git and untracked files) using git ls-files + tar. +tracked_tree_copy() { + local dest="$1" + mkdir -p "$dest" + ( + cd "$ROOT" + git ls-files -z | tar --null -T - -cf - | (cd "$dest" && tar xf -) + ) +} + +# counts a token in a multiline string +count_token() { # count_token → echo N + local token="$1" haystack="$2" n=0 line + while IFS= read -r line; do + case "$line" in + *"$token"*) n=$((n + 1)) ;; + esac + done <<< "$haystack" + printf '%s' "$n" +} \ No newline at end of file