Compare commits

..

6 Commits

Author SHA1 Message Date
Your Name df1cca478d fix: Telegram listener — async command execution + singleton guard
gates / consistency-and-conventions (push) Successful in 23s
Root cause: run_and_reply() blocked the entire listener synchronously.
FFmpeg hung because child processes inherited stdin (waiting for 'q').
Long-running commands froze the listener for up to 120s.

Fix:
- Commands run in background with stdin=/dev/null, output to temp file
- reap_commands() collects output non-blocking after each getUpdates cycle
- SIGCHLD handler pre-caches exit codes via wait -n
- TERM/INT trap kills background processes and cleans temp files
- Singleton guard (flock) prevents duplicate listeners racing getUpdates

Tests:
- t-telegram-listener-exec.sh: 12 hermetic checks (echo, pipes, stderr,
  compound commands, long-running, quiet mode)
- t-telegram-listener-singleton.sh: 8 checks (lock acquire/release/status)

Architect verdict: accepted as-is, no re-architecture needed.
2026-09-09 17:17:56 -04:00
Your Name f14d24950a feat: pos media yt — unified YouTube tools + subtitles (POS--9)
gates / consistency-and-conventions (push) Successful in 20s
2026-09-09 07:04:22 -04:00
Your Name 9a164462e7 feat: install.sh version gate — --force + per-commit 0.0cN
gates / consistency-and-conventions (push) Successful in 33s
Architect-approved (C): install.sh now skips+aborts (exit 0) when the
installed version equals the current git-derived version, with a --force
flag to re-install. Version scheme 0.0cN (N = git commit count) bumps by
construction on every commit.

- install_version(): derives 0.0c$count via git rev-list; empty when .git
  absent (gate skipped); INSTALL_VERSION_OVERRIDE presence-check seam for
  tests (empty override simulates no-git deterministically)
- Gate after arg parse, before phases; numeric comparison (strip 0.0c, -eq);
  messages: 'Already installed (X). Use --force to re-install.' and
  '(dry-run) Would skip install: already at version X', both exit 0
- flag_set installed_version after 'Bootstrap complete' banner (not in
  dry-run, even under --force, never on phase failure)
- Docs: SCRIPTS.md flag table + gate description, AGENT_Context line count
  248->301 + flags + flow diagram, README --force row, tests/README row,
  AGENT_TODO Done entry

Verified: new tests/t-install-version.sh 21 checks (9 contracted cases,
real install.sh + hermetic env seams); suite 20 files / 461 checks /
0 fail / 0 skip; make gen byte-idempotent; make check OK; make lint
0 FAIL, 0 WARN; bash -n clean; git diff --check clean; Reviewer
APPROVE_WITH_NOTES with 3 mutation disproofs (8/21, 9/21, 4/21 fail)
2026-09-08 10:17:31 -04:00
Your Name 9ef42c5fd1 fix: pos ai — honor legacy AI_API_KEY as fallback (provider key stays primary)
gates / consistency-and-conventions (push) Successful in 27s
Architect decision C on the API-key contract mismatch: docs claimed
AI_API_KEY was the required primary key, but resolve_key() only read
provider-specific keys (7ae2e77 removed shared-key priority to fix
cross-provider leakage; docs never updated).

- bin/pos-ai resolve_key(): provider key wins, legacy AI_API_KEY honored
  read-only when the provider's own key is empty, llamacpp unchanged;
  cmd_providers() configured check mirrors the same set
- require_key() error messages byte-stable (test-locked)
- AI_API_KEY NOT re-added to the # POS_CONFIG:/# PROVIDER_CONFIG: registry
- Docs reworded: POS.md rows 91/96/98 + precedence sentence, howto/ai.md
  first-run hints, HOWTO.md row, AGENT_Context prose (2 spots), config/ai.env
  legacy comment
- New regression tests/t-ai-key-resolution.sh: 24 checks / 10 cases
  (provider-key-only, AI_API_KEY-only, both -> provider wins, env-wins,
  llamacpp no-key, missing-key message, providers configured status)

Verified: suite 19 files / 440 checks / 0 fail / 0 skip; make gen
byte-idempotent; make check OK; make lint 0 FAIL, 0 WARN; bash -n clean;
git diff --check clean; Reviewer APPROVE_WITH_NOTES (mutation disproof:
inverted precedence -> C3/C6 fail)
2026-09-07 13:32:42 -04:00
Your Name aedc9b5c8f feat: add opencode project skill for Linux_post_install operations
gates / consistency-and-conventions (push) Successful in 29s
- .opencode/skills/linux-post-install/SKILL.md: operational playbook for agents working on the repo
- Covers: repo shape (45 pos tools, lib/, templates, gen blocks), pos tool model
  (# POS: header system, exec-bit, deps-guard-before-help, INTERACTIVE_CMDS, determinism),
  doc authority order (MAINTENANCE Phase 0: templates -> DEV.md -> AGENTS.md -> code),
  Definition of Done (make gen x2 idempotent -> make check -> make lint 0/0 -> make test),
  test conventions (hard-skip contract, negative controls, tests/README table),
  repo commands (ci-status, gitea API, pos tree/config)
- Frontmatter validated: name matches folder, description with trigger keywords
- Auto-discovered at .opencode/skills/ — no opencode.json change needed; restart to load

Verified: git diff --check clean; skill frontmatter name/description valid
2026-09-07 09:06:16 -04:00
Your Name 5b9dea2118 test: share-client ask_mountpoint — regression for t=type manual mountpoint
gates / consistency-and-conventions (push) Successful in 22s
Permanent coverage for the 2026-09-07 share mountpoint UX change
(commit 35eb90a): the REAL ask_mountpoint body is brace-extracted from
each client file (smb + nfs), exercised through the real common.sh +
share-lib.sh helper chain via the deterministic non-TTY stdin path, with
a fake sudo shim (records to SUDO_LOG, honors SUDO_FAIL) so no real
filesystem is touched.

Covers: existing dir used as-is (rc 0, provable NO confirm / NO mkdir
via empty SUDO_LOG) added by 35eb90a; new-dir confirm=y creates via sudo
mkdir; confirm=n and confirm=EOF both fail closed rc 1; existing
non-directory rejected 'not a directory'; relative / trailing-slash /
system paths (/etc, /root, /home/*/.ssh*) and empty-EOF all rc 1 with no
side effects; mkdir failure rc 1. Stdout purity (path only). Static
guards fail if anyone regresses n=new / n|N / ask_new_mountpoint, and
verify the existing-dir [-d] branch precedes confirm/mkdir. Byte-symmetry
guard keeps smb/nfs bodies identical.

117 checks new (18 files / 416 total). Every case can genuinely fail
(negative-control verified by Tester). Hard-skip contract respected: 0
skips — all cases deterministic non-interactively.

Verified: make check OK, make lint 0 FAIL/0 WARN, make test 18 files /
416 checks / 0 fail / 0 skip (~46s), bash -n clean, git diff --check clean.
2026-09-07 08:30:38 -04:00
33 changed files with 2141 additions and 524 deletions
@@ -0,0 +1,63 @@
---
name: linux-post-install
description: Use when working on the Linux_post_install repo — creating, modifying, or fixing pos CLI tools (bin/pos-*, install.sh, features/, apps/, lib/, systemd/, completions/), regenerating docs via make gen, or running the make check / make lint / make test gates. Front-loads the # POS: header system, the pos tool model, the doc-authority order (templates > DEV.md > AGENTS.md > code), the definition of done, and the test conventions.
---
# Linux_post_install — Toolkit Operations
Operational playbook for changing the Linux_post_install repo (Debian/Ubuntu bootstrap + homelab toolkit, Bash). Read `DOC/AGENT_Context_Project.md` first for any non-trivial task — it opens with a **Document Map** (auto-generated line ranges) to jump straight to the relevant section. For day-to-day *usage* docs see `DOC/HOWTO.md`; for the `pos` CLI reference see `DOC/POS.md`.
## Repo shape
- `bin/pos-<cat>-<cmd>` — tools (45 of them); category-less `bin/pos-<cmd>` for dispatcher/dev-level (`pos-config`, `pos-tree`). Legacy `bin/wr-*`, `mp3`, `mp4`, `vbox`, `ssh-load-all` are thin forwarders — keep them that way.
- `lib/` — shared libraries: `common.sh` (run/spawn/err/warn, `ensure_user_bus`), `registry.sh` (POS_* header query API — consumers source it, never re-implement sed/grep), `menu-lib.sh` (`menu_ask_value`, `--allow-empty`), `jq-seam.sh` (stub-friendly jq), `config.sh` (env files, `pos config` scopes), `share-lib.sh`, `usb-lib.sh`, `ai-lib.sh` (use the `llamafile` seam in tests).
- `features/`, `apps/`, `templates/` (pos-tool.sh, app.sh, feature.sh — starting points for new files), `systemd/`, `completions/pos.bash` (gen output), `tests/` (suite, see below), `x64_bin/`+`arm64_bin/` (prebuilt hotspot binaries), `scripts/` (gen-docs, check-sync, lint-conventions, ci-status), `Makefile` (check/gen/lint/test/hook), `.gitea/workflows/lint.yml` (CI).
- Gen blocks `GEN:START`/`GEN:END` (AGENT_Context filetable/tree/dispatch/selfcontained/docmap + `completions/pos.bash`) are `make gen` output — never hand-edit.
## Tool model
- Format: shebang → `set -euo pipefail``# POS: <cat> <cmd> — <desc>` header (missing header hard-fails `make gen`) → `# POS_FLAGS:` / `# POS_SUBCMDS:` / `# POS_CONFIG:` / `# POS_DEPS:` / `# POS_EXAMPLES:` headers → `command -v` deps guards **before** `-h|--help` dispatch → case-based help.
- File must be executable (`100755`) and run standalone from `/usr/local/bin` after install (source `lib/common.sh` via `$(dirname "$0")/../lib/common.sh` fallback chain).
- New tools are auto-discovered; `make gen` only uses the text after the first `— ` in the header.
- Stdin readers must be added to `INTERACTIVE_CMDS` in `bin/pos` or the logging `tee` pipe hangs/swallows prompts.
- Deps: apt → `PACKAGES` array in `preinstall.sh`; non-apt/manual installers → `command -v <bin> || err` guard inside the tool. Secrets never committed; runtime config `~/.config/linux_post_install/<tool>.env` (chmod 600), env-var precedence.
- Entertainment plugins (`entertainment/`) must NOT source `lib/common.sh` (stdout is the Telegram message); markers `# POS_PLUGIN:` + `# POS_KEYS:`.
- ScaleTail compose is a git submodule — `git submodule update --init` before `pos docker compose *`.
- Generators must be byte-order deterministic (`LC_ALL=C`, sort) or CI `git diff --exit-code` trips.
## Doc authority (MAINTENANCE.md Phase 0)
1. `templates/*.sh` — codified current convention; required starting point for new files (`cp templates/pos-tool.sh bin/pos-<cat>-<cmd>`).
2. `DOC/DEV.md` — convention detail, checklists (wins on detail).
3. `AGENTS.md` — operational/process facts (wins on process).
4. Code + `# POS:` headers — ground truth for behavior and GEN blocks.
Drift in POS/HOWTO/README/SCRIPTS/SYSTEMD/APPS docs is a doc bug — fix the doc.
## Definition of done
After touching `bin/pos-*` (or anything structural):
```bash
make gen # regenerates gen blocks + completions
make gen # run twice — must be byte-idempotent
make check # self-consistency: bash -n + exec bits + doc-sync + dispatch smoke
make lint # convention gate — must end "0 FAIL, 0 WARN"
make test # full suite (18 files / 416 checks) unless tests/README.md says otherwise
```
Also: `bash -n <file>` for new/edited scripts, `git diff --check`, CI gate (`.gitea/workflows/lint.yml`, job `gates`) runs the same four commands on push to main and tags `ci-ok/<sha>`; query with `scripts/ci-status.sh [--wait] [<sha>]`.
## Test conventions (`tests/`)
- Suite lives in `tests/`; run standalone after suite load: `test_run <name> <label> <stdin> <expected_rc>` with `expect_eq`/`expect_ok`/`expect_fail` helpers; each file prints `^ <name>: <n> checks, <m> fail, <k> skip`.
- Read `tests/README.md` — it documents the harness and suite table; add a row when adding a file.
- Hard-skip contract: tests must never fake passes. Infeasible non-TTY cases are documented `skip`; use `# POS:` header checks, `extract_fn` brace extraction, stub-PATH seams, and non-TTY stdin pipes where possible. Zero-skip is preferred but never silence real failures. Negative controls (assert the new check actually catches the old bug) are required for regression tests.
## Commands relevant to this repo
- `make gen` + `make check` + `make lint` (+ `make test`) — the gates above.
- `make hook` — opt-in pre-commit gate.
- `scripts/ci-status.sh [--wait] [<sha>]` — CI tag lookup (0 green / 1 red / 2 pending); reliable path is the Gitea API `https://gitea.skink-platy.ts.net/api/v1/repos/admin/Linux_post_install/commits/<sha>/status`.
- `pos tree` — authoritative structure (needs a built/installed tree; falls back to reading `bin/pos-*` directly).
- `pos config <scope>` — runtime env-file config (`~/.config/linux_post_install/<tool>.env`); mask tokens in output.
+11
View File
@@ -42,6 +42,17 @@ summary (newest last).
## Done ## Done
- **2026-09-09** — Telegram listener single-instance guard (Toolsmith): `bin/pos-communication-telegram-listener --run` now takes a `flock(1)` on `${XDG_RUNTIME_DIR:-/tmp}/pos-telegram-listener.lock` inside `run_daemon()` (before config load/sync/poll loop) — a second `--run` on the same token fails fast (exit 1, `ERROR: listener already running (single instance) — check: systemctl --user status pos-telegram-listener`), never racing getUpdates (Telegram 409/command stealing). Kernel auto-release → no stale-lock bookkeeping, systemd `Restart=always` restarts clean. `--status` first line now reports `listener: running (single instance lock held)` / `listener: not running` via the same `lock_held()` probe. `flock` dep guard added (`util-linux`). New regression `tests/t-telegram-listener-singleton.sh` (7 checks: first acquires+loops, second exits 1 with exact message, lock releases → third starts clean, status reports both states; stubbed curl/systemctl, sandboxed XDG_RUNTIME_DIR — hermetic, no network). Verified: `bash -n`, `make gen` ×2 byte-idempotent, `make check` OK, `make lint` 0 FAIL / 0 WARN, `make test` green, `git diff --check` clean.
- **2026-09-09** — Unified YouTube tools into `pos media yt` + new `subtitles` (Architect POS--9). New `bin/pos-media-yt` dispatcher (mp3/mp4/grab/ytsync/subtitles) + `bin/pos-media-yt-{mp3,mp4,grab,subtitles,ytsync}`; the ytsync file is a forwarder to the existing `pos media ytsync`; legacy `bin/pos-media-{mp3,mp4,grab}` became thin forwarders to the `yt` forms. New `lib/yt-lib.sh` (deps/URL-validation/echo/classify helpers; `classify_url` migrated from grab, `yt_validate_url` is a return-1 checker — never exits, so callers can prefix errors). `bin/pos` INTERACTIVE_CMDS += `media-yt-mp4` (interactive format pick reads stdin). `pos-media-yt-subtitles` extracts captions via `--write-subs --write-auto-subs --sub-langs best`, `--lang en,ar` (one `--sub-langs` arg), `srt|vtt|txt` (txt = srt→txt conversion stripping timestamps/HTML), `--auto-only`, `--list-subs` probe, `--output`, no-ffmpeg dep (yt-dlp only; dry-run skips deps entirely). Docs: DOC/POS.md media section rewritten (yt group + forwarder rows), DOC/howto/media.md `yt` commands + subtitles section, AGENT_Context hand-maintained `lib/yt-lib.sh` row, tests/README row. New `tests/t-pos-media-yt.sh` (72 checks: dispatcher/forwarder resolution, full `pos media mp3` dispatch chain, yt-lib helpers, per-tool flags/dry-run/`YT_OUT_DIR` seam/`GRAB_DEFAULT` config, 3 mandated negative controls — unsafe-URL no-expansion, `--lang en,ar` single arg, txt timestamp-stripping, unavailable-subs detection). `tests/t-config-precedence.sh` Part D config-consumer list updated `pos-media-grab``pos-media-yt-grab`. Verified: `bash -n` all; `make gen` ×2 byte-idempotent; `make check` OK; `make lint` 0 FAIL / 0 WARN; `make test` 21 files / 533 checks / 0 fail / 0 skip; `git diff --check` clean; smokes — `pos media yt --help`, `yt mp3/mp4/subtitles --help`, `yt ytsync --help` (reaches `pos media ytsync`), `pos media mp3 --help` forwarder, `pos tree` shows the `yt` subtree (with repo-first PATH; system `/usr/local/bin` has a stale pre-POS--9 install that shadows it otherwise).
- **2026-09-08** — `install.sh` version gate (Architect→Builder): skip+abort when installed version == current version, `--force` to bypass, version scheme `0.0c<git commit count>` (auto-bumps per commit). `install_version()` derives `0.0c$(git rev-list --count HEAD)`; empty when `.git` absent → gate skipped (silently); `INSTALL_VERSION_OVERRIDE` env var (presence-check) = test seam. Gate after arg-parse, before phases, numeric comparison (strip `0.0c`, `-eq`); `log "Already installed ($CURRENT_VERSION). Use --force to re-install."` / `--dry-run``(dry-run) Would skip install: already at version $CURRENT_VERSION`, both exit 0. `FORCE=0` init, `--force` parse + usage. `flag_set installed_version "$CURRENT_VERSION"` after "Bootstrap complete" banner (only when DRY_RUN≠1 and version non-empty; even under --force). New `tests/t-install-version.sh` (21 checks / 9 cases). Docs: README/SCRIPTS/AGENT_Context (flags, flow, line count 248→301, tests/README row). Verified: `bash -n` clean; `make gen` idempotent; `make check` OK; `make lint` 0 FAIL / 0 WARN; `make test` suite green.
- **2026-09-07** — `pos ai` API-key contract mismatch fixed (Architect→Builder→Reviewer; docs/history evidence): docs claimed `AI_API_KEY` was the required primary key, but `resolve_key()` read only provider-specific keys (`7ae2e77` had removed shared-key priority to fix cross-provider leakage; docs never updated). Decision C: provider key stays primary (leakage guard intact), legacy `AI_API_KEY` honored as backward-compat fallback when the provider's own key is empty; `cmd_providers()` "configured" mirrors it; `require_key()` messages byte-stable; `AI_API_KEY` NOT re-added to `# POS_CONFIG:`/`# PROVIDER_CONFIG:` registry. Docs reworded (POS.md rows 91/96/98 + precedence, howto/ai.md first-run hints, HOWTO.md, AGENT_Context 2 prose spots, config/ai.env comment). New regression `tests/t-ai-key-resolution.sh` (24 checks / 10 cases: gemini+openrouter via provider key only, via AI_API_KEY only, both→provider wins, env-wins, llamacpp no-key, missing-key message, providers configured status). Verified: suite 19 files / 440 checks / 0 fail / 0 skip; `make gen` byte-idempotent; `make check` OK; `make lint` 0 FAIL, 0 WARN; Reviewer APPROVE_WITH_NOTES with mutation-based disproof (inverted precedence → C3/C6 fail).
- **2026-09-07** — opencode project skill: created `.opencode/skills/linux-post-install/SKILL.md` (repo had no `.opencode/`). Skill encodes the repo's operational playbook for agents: repo shape, `pos` tool model (`# POS:` header system, exec-bit, deps-guard-before-help, INTERACTIVE_CMDS, determinism), doc authority order (MAINTENANCE Phase 0: templates → DEV.md → AGENTS.md → code), Definition of Done gates (`make gen` ×2 idempotent → `make check``make lint` 0/0 → `make test` 18 files/416 checks), test conventions (hard-skip contract, negative controls), and repo commands (ci-status, gitea API, pos tree/config). Frontmatter validated (name matches folder, description with trigger keywords); auto-discovered at `.opencode/skills/` — no opencode.json change needed; restart opencode to load.
- **2026-09-06** — Repo cleanup: removed 114 temp/process files (AgentsReport/ 86 + reportAgents/ 28 agent reports), stale task/plan/audit docs (tmp_request.md, FINAL_SUMMARY.md, IMPLEMENTATION_PLAN.md, AUDIT.md, AUDIT_TABLE.md, registry-design doc), stray `To`/`.n`/`reports/` artifacts; gitignored AgentsReport/ so agent process reports stay local-only. DOC/, tools-docs/ytsync.md, AGENT_TODO.md, bin/lib/apps/tests/config/scripts etc. kept untouched. - **2026-09-06** — Repo cleanup: removed 114 temp/process files (AgentsReport/ 86 + reportAgents/ 28 agent reports), stale task/plan/audit docs (tmp_request.md, FINAL_SUMMARY.md, IMPLEMENTATION_PLAN.md, AUDIT.md, AUDIT_TABLE.md, registry-design doc), stray `To`/`.n`/`reports/` artifacts; gitignored AgentsReport/ so agent process reports stay local-only. DOC/, tools-docs/ytsync.md, AGENT_TODO.md, bin/lib/apps/tests/config/scripts etc. kept untouched.
- **2026-09-06** — `pos ai` OpenRouter 402 + unbounded session — Architect→Builder→Reviewer. User hit OpenRouter 402 on the `assist` alias: "You requested up to 131072 tokens, but can only afford 4511" — no provider sent `max_tokens`, so OpenRouter pre-bills the routed model's full worst-case output (131072 on `openrouter/auto`); user also asked to bound session history to last-5 requests. Architect decisions: `AI_MAX_TOKENS` (num, default **2048**, real cost cap) sent as `max_tokens` on OpenRouter and `generationConfig.maxOutputTokens` on Gemini (llamacpp skipped — local/free); `AI_SESSION_TURNS` (messages, 2 per exchange; default 40 kept back-compat; 10 = last 5 conversations) resolved lazily in `session_push` because config loads after the hardcoded line-25 default; both registered in the `bin/pos-ai` `# POS_CONFIG:` `@General` section → visible in `pos config ai`. Reviewer hardening (CHANGES_REQUIRED → fixed): unguarded env input could reach jq `tonumber` (0/-5/010/abc) — both providers + `session_push` now guard with `^[1-9][0-9]*$` fallback-to-default. Verified: fake-curl shim smoke (16 provider-body + 12 session-window checks incl. 010-regression proof), `make gen` idempotent, `make check` OK, `make lint` 0/0, `make test` 17 files/299 checks green; Reviewer ACCEPT (twice). Tester regression round not run this cycle (user's call); permanent coverage remains a follow-up. - **2026-09-06** — `pos ai` OpenRouter 402 + unbounded session — Architect→Builder→Reviewer. User hit OpenRouter 402 on the `assist` alias: "You requested up to 131072 tokens, but can only afford 4511" — no provider sent `max_tokens`, so OpenRouter pre-bills the routed model's full worst-case output (131072 on `openrouter/auto`); user also asked to bound session history to last-5 requests. Architect decisions: `AI_MAX_TOKENS` (num, default **2048**, real cost cap) sent as `max_tokens` on OpenRouter and `generationConfig.maxOutputTokens` on Gemini (llamacpp skipped — local/free); `AI_SESSION_TURNS` (messages, 2 per exchange; default 40 kept back-compat; 10 = last 5 conversations) resolved lazily in `session_push` because config loads after the hardcoded line-25 default; both registered in the `bin/pos-ai` `# POS_CONFIG:` `@General` section → visible in `pos config ai`. Reviewer hardening (CHANGES_REQUIRED → fixed): unguarded env input could reach jq `tonumber` (0/-5/010/abc) — both providers + `session_push` now guard with `^[1-9][0-9]*$` fallback-to-default. Verified: fake-curl shim smoke (16 provider-body + 12 session-window checks incl. 010-regression proof), `make gen` idempotent, `make check` OK, `make lint` 0/0, `make test` 17 files/299 checks green; Reviewer ACCEPT (twice). Tester regression round not run this cycle (user's call); permanent coverage remains a follow-up.
+48 -24
View File
@@ -10,19 +10,19 @@
<!-- GEN:START docmap --> <!-- GEN:START docmap -->
| ## 1. Project Overview | 2843 | | ## 1. Project Overview | 2843 |
| ## 2. Directory Structure | 44210 | | ## 2. Directory Structure | 44216 |
| ## 3. Installation Flow | 211264 | | ## 3. Installation Flow | 217275 |
| ## 4. The `pos` CLI System | 265346 | | ## 4. The `pos` CLI System | 276363 |
| ## 5. Shared Library — `lib/common.sh` | 347378 | | ## 5. Shared Library — `lib/common.sh` | 364395 |
| ## 6. Docker Compose / ScaleTail | 379421 | | ## 6. Docker Compose / ScaleTail | 396438 |
| ## 7. Optional Apps (`apps/`) | 422451 | | ## 7. Optional Apps (`apps/`) | 439468 |
| ## 8. Entertainment Module | 452465 | | ## 8. Entertainment Module | 469482 |
| ## 9. Systemd Services | 466477 | | ## 9. Systemd Services | 483494 |
| ## 10. Configuration Files | 478504 | | ## 10. Configuration Files | 495521 |
| ## 11. Coding Conventions | 505537 | | ## 11. Coding Conventions | 522554 |
| ## 12. Development Workflow | 538590 | | ## 12. Development Workflow | 555607 |
| ## 13. Key File Quick Reference | 591666 | | ## 13. Key File Quick Reference | 608690 |
| ## 14. Common Tasks for Agents | 667700 | | ## 14. Common Tasks for Agents | 691724 |
<!-- GEN:END docmap --> <!-- GEN:END docmap -->
## 1. Project Overview ## 1. Project Overview
@@ -90,7 +90,13 @@ Linux_post_install/
│ ├── pos-media-mp4 # Download video as MP4 (smart/interactive format select) │ ├── pos-media-mp4 # Download video as MP4 (smart/interactive format select)
│ ├── pos-media-sync # Incremental Music → USB sync (mp3/mp4, add/update only) │ ├── pos-media-sync # Incremental Music → USB sync (mp3/mp4, add/update only)
│ │ [deps: lsblk jq] │ │ [deps: lsblk jq]
│ ├── pos-media-yt-grab # Auto-download URL as audio or video (classify + route)
│ ├── pos-media-yt-mp3 # Download audio as MP3 (yt-dlp)
│ ├── pos-media-yt-mp4 # Download video as MP4 (smart/interactive format select)
│ ├── pos-media-yt-subtitles # Extract subtitles/captions from a URL (yt-dlp)
│ ├── pos-media-yt-ytsync # Incrementally sync YouTube channels/playlists into ~/Videos
│ ├── pos-media-ytsync # Incrementally sync YouTube channels/playlists into ~/Videos │ ├── pos-media-ytsync # Incrementally sync YouTube channels/playlists into ~/Videos
│ ├── pos-media-yt # YouTube download tools (mp3/mp4/grab/ytsync/subtitles)
│ ├── pos-network-checkport # Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view │ ├── pos-network-checkport # Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view
│ ├── pos-network-download # aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) │ ├── pos-network-download # aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits)
│ │ [deps: aria2c jq curl] │ │ [deps: aria2c jq curl]
@@ -211,7 +217,11 @@ Linux_post_install/
## 3. Installation Flow ## 3. Installation Flow
``` ```
User runs: ./install.sh [--apps|--full|--feature|--dry-run|--skip <phase>|--steps <spec>] User runs: ./install.sh [--apps|--full|--feature|--dry-run|--force|--skip <phase>|--steps <spec>]
├─ Version gate: compares installed version (flag) vs current (0.0c<commit count>)
│ └─ match + no --force → "Already installed. Use --force to re-install." / exit 0
│ (--dry-run variant prints "Would skip install"); no git or no flag → skip gate
├─ Phase 1: preinstall.sh (requires root) ├─ Phase 1: preinstall.sh (requires root)
│ └─ apt update + installs 25+ packages + yt-dlp + fail2ban │ └─ apt update + installs 25+ packages + yt-dlp + fail2ban
@@ -249,6 +259,7 @@ User runs: ./install.sh [--apps|--full|--feature|--dry-run|--skip <phase>|--step
| `--full` | Core install + all apps (non-interactive) | | `--full` | Core install + all apps (non-interactive) |
| `--feature` | Install `features/` scripts to `/usr/local/bin/` (asks before overwriting), set their flags | | `--feature` | Install `features/` scripts to `/usr/local/bin/` (asks before overwriting), set their flags |
| `--dry-run` | Preview without executing | | `--dry-run` | Preview without executing |
| `--force` | Re-install even if the version matches |
| `--skip <phase>` | Skip a phase (repeatable): `preinstall`, `scripts`, `postinstall`, `scalepoint`, `apps` | | `--skip <phase>` | Skip a phase (repeatable): `preinstall`, `scripts`, `postinstall`, `scalepoint`, `apps` |
| `--steps <spec>` | Run only specific phases. Format: `1,3,4` or `1-3` | | `--steps <spec>` | Run only specific phases. Format: `1,3,4` or `1-3` |
| `--no-color` | Disable colored output | | `--no-color` | Disable colored output |
@@ -306,7 +317,13 @@ All non-interactive `pos` commands log output to `~/.local/share/linux_post_inst
| media | mp3 | `pos-media-mp3` | Download audio as MP3 (yt-dlp) | | | | media | mp3 | `pos-media-mp3` | Download audio as MP3 (yt-dlp) | | |
| media | mp4 | `pos-media-mp4` | Download video as MP4 (smart/interactive format select) | | | | media | mp4 | `pos-media-mp4` | Download video as MP4 (smart/interactive format select) | | |
| media | sync | `pos-media-sync` | Incremental Music → USB sync (mp3/mp4, add/update only) | lsblk jq | pos media sync --mp3 → Sync only MP3 files to USB · pos media sync --mp4 --dry-run → Preview MP4 sync without copying | | media | sync | `pos-media-sync` | Incremental Music → USB sync (mp3/mp4, add/update only) | lsblk jq | pos media sync --mp3 → Sync only MP3 files to USB · pos media sync --mp4 --dry-run → Preview MP4 sync without copying |
| media | yt-grab | `pos-media-yt-grab` | Auto-download URL as audio or video (classify + route) | | |
| media | yt-mp3 | `pos-media-yt-mp3` | Download audio as MP3 (yt-dlp) | | |
| media | yt-mp4 | `pos-media-yt-mp4` | Download video as MP4 (smart/interactive format select) | | |
| media | yt-subtitles | `pos-media-yt-subtitles` | Extract subtitles/captions from a URL (yt-dlp) | | |
| media | yt-ytsync | `pos-media-yt-ytsync` | Incrementally sync YouTube channels/playlists into ~/Videos | | |
| media | ytsync | `pos-media-ytsync` | Incrementally sync YouTube channels/playlists into ~/Videos | | | | media | ytsync | `pos-media-ytsync` | Incrementally sync YouTube channels/playlists into ~/Videos | | |
| media | yt | `pos-media-yt` | YouTube download tools (mp3/mp4/grab/ytsync/subtitles) | | |
| network | checkport | `pos-network-checkport` | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view | | | | network | checkport | `pos-network-checkport` | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view | | |
| network | download | `pos-network-download` | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) | aria2c jq curl | pos network download add https://example.com/file.zip → Enqueue an HTTP download (auto-starts daemon) · pos network download status → Daemon health + global transfer stats · pos network download watch → Live progress view | | network | download | `pos-network-download` | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) | aria2c jq curl | pos network download add https://example.com/file.zip → Enqueue an HTTP download (auto-starts daemon) · pos network download status → Daemon health + global transfer stats · pos network download watch → Live progress view |
| network | hotspot | `pos-network-hotspot` | Wi-Fi hotspot via create_ap + wihotspot-gui | | | | network | hotspot | `pos-network-hotspot` | Wi-Fi hotspot via create_ap + wihotspot-gui | | |
@@ -371,7 +388,7 @@ source "$(dirname "$0")/../lib/common.sh"
**Scripts that do NOT source common.sh** (self-contained): **Scripts that do NOT source common.sh** (self-contained):
<!-- GEN:START selfcontained --> <!-- GEN:START selfcontained -->
`pos`, `pos-ai-gemini`, `pos-ai-llamacpp`, `pos-ai-openrouter`, `pos-communication-matrix-sender`, `pos-communication-telegram-listener`, `pos-communication-telegram-sender`, `pos-network-checkport`, `pos-network-hotspot`, `pos-network-ip`, `pos-network-scan`, `pos-ssh-load-keys`, `pos-system-firewall`. `pos`, `pos-ai-gemini`, `pos-ai-llamacpp`, `pos-ai-openrouter`, `pos-communication-matrix-sender`, `pos-communication-telegram-listener`, `pos-communication-telegram-sender`, `pos-media-grab`, `pos-media-mp3`, `pos-media-mp4`, `pos-media-yt-ytsync`, `pos-network-checkport`, `pos-network-hotspot`, `pos-network-ip`, `pos-network-scan`, `pos-ssh-load-keys`, `pos-system-firewall`.
<!-- GEN:END selfcontained --> <!-- GEN:END selfcontained -->
--- ---
@@ -488,7 +505,7 @@ All `.service` files in `systemd/` are automatically copied to `/etc/systemd/sys
- `~/.config/linux_post_install/entertainment.env` — entertainment plugin defaults: weather location + `ENABLED` auto-trigger list (`plugin, interval` pairs scheduled via `pos entertainment enable/disable`, systemd user timers); auto-installed from `config/entertainment.env` by `postinstall.sh` (no clobber, template printed) - `~/.config/linux_post_install/entertainment.env` — entertainment plugin defaults: weather location + `ENABLED` auto-trigger list (`plugin, interval` pairs scheduled via `pos entertainment enable/disable`, systemd user timers); auto-installed from `config/entertainment.env` by `postinstall.sh` (no clobber, template printed)
- `~/.config/linux_post_install/system.env` — shared "system" tool settings (loaded by `pos system health` / `pos system backup` via `load_system_env()` in `lib/common.sh`; env already exported wins over the file); template `config/system.env` - `~/.config/linux_post_install/system.env` — shared "system" tool settings (loaded by `pos system health` / `pos system backup` via `load_system_env()` in `lib/common.sh`; env already exported wins over the file); template `config/system.env`
- `~/.config/linux_post_install/notify.env` — alerting platform selection (`NOTIFY_PLATFORM=telegram,matrix`, comma-separated = fan out); read by `lib/notify.sh`; template `config/notify.env` - `~/.config/linux_post_install/notify.env` — alerting platform selection (`NOTIFY_PLATFORM=telegram,matrix`, comma-separated = fan out); read by `lib/notify.sh`; template `config/notify.env`
- `~/.config/linux_post_install/ai.env` — AI provider config (`AI_PROVIDER`, `AI_API_KEY` secret, `AI_MODEL`, `AI_SYSTEM_PROMPT`, `AI_MAX_TOKENS`, `AI_SESSION_TURNS`, plus legacy fallbacks `AI_GEMINI_API_KEY`, `AI_GEMINI_MODEL`, `OPENROUTER_API_KEY`, `OPENROUTER_MODEL`); read by `pos ai`; template `config/ai.env`, auto-installed by postinstall, edit with `pos config ai` - `~/.config/linux_post_install/ai.env` — AI provider config (`AI_PROVIDER`, `AI_MODEL`, `AI_SYSTEM_PROMPT`, `AI_MAX_TOKENS`, `AI_SESSION_TURNS`, provider keys `AI_GEMINI_API_KEY`/`OPENROUTER_API_KEY` (secrets), legacy shared fallback `AI_API_KEY` accepted when the provider's key is unset, plus model fallbacks `AI_GEMINI_MODEL`, `OPENROUTER_MODEL`); read by `pos ai`; template `config/ai.env`, auto-installed by postinstall, edit with `pos config ai`
- `~/.bashrc` — Modified by postinstall (PATH, bash completion) - `~/.bashrc` — Modified by postinstall (PATH, bash completion)
### Feature Flags ### Feature Flags
@@ -592,7 +609,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| File | Lines | Purpose | | File | Lines | Purpose |
|------|-------|---------| |------|-------|---------|
| `install.sh` | 248 | Main orchestrator — 4 phases with CLI flags, `--feature`, prebuilt arch bins | | `install.sh` | 301 | Main orchestrator — 4 phases with CLI flags, `--feature`, `--force`, version gate, prebuilt arch bins |
| `preinstall.sh` | 76 | System packages + hotspot deps + yt-dlp + fail2ban | | `preinstall.sh` | 76 | System packages + hotspot deps + yt-dlp + fail2ban |
| `postinstall.sh` | 168 | fail2ban config, PATH, bash completion, systemd (flag-gated) | | `postinstall.sh` | 168 | fail2ban config, PATH, bash completion, systemd (flag-gated) |
| `lib/common.sh` | 151 | Shared library (log/warn/err/run/spawn, dry-run aware, `load_system_env`, CONFIG_DIR) | | `lib/common.sh` | 151 | Shared library (log/warn/err/run/spawn, dry-run aware, `load_system_env`, CONFIG_DIR) |
@@ -606,6 +623,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `lib/share-lib.sh` | 318 | Domain layer for the share suite (usbsrv/smbclient record parsers, folder+mountpoint candidates, remote listings, service/firewall advisories; EOF-safe) + compat shims to `lib/menu-lib.sh` — used by all five `pos share *` tools | | `lib/share-lib.sh` | 318 | Domain layer for the share suite (usbsrv/smbclient record parsers, folder+mountpoint candidates, remote listings, service/firewall advisories; EOF-safe) + compat shims to `lib/menu-lib.sh` — used by all five `pos share *` tools |
| `lib/menu-lib.sh` | 362 | Category-neutral interactive menu primitives (`menu_guard` tty guard, `menu_run` looping boxed menu, `menu_pick` type-to-filter picker, `menu_ask_value` prompt-with-default via raw-mode bracketed-paste-safe `menu_read_value`; stderr render, fail-closed on non-tty/EOF) — sourced by `share-lib.sh`, open to any category | | `lib/menu-lib.sh` | 362 | Category-neutral interactive menu primitives (`menu_guard` tty guard, `menu_run` looping boxed menu, `menu_pick` type-to-filter picker, `menu_ask_value` prompt-with-default via raw-mode bracketed-paste-safe `menu_read_value`; stderr render, fail-closed on non-tty/EOF) — sourced by `share-lib.sh`, open to any category |
| `lib/registry.sh` | 199 | Shared query API for POS tool metadata headers (`# POS_*:`) — `reg_scan`/`reg_list`/`reg_lookup`/`reg_each`/config scope helpers; used by `pos-tree` and `gen-docs.sh` | | `lib/registry.sh` | 199 | Shared query API for POS tool metadata headers (`# POS_*:`) — `reg_scan`/`reg_list`/`reg_lookup`/`reg_each`/config scope helpers; used by `pos-tree` and `gen-docs.sh` |
| `lib/yt-lib.sh` | 50 | Shared YouTube helpers for `pos media yt *` (`yt_check_deps`, `yt_validate_url`, `yt_echo_cmd`, `classify_url`) — sourced by `yt-mp3`/`yt-mp4`/`yt-grab`/`yt-subtitles` |
| `bin/flag-reader` | 58 | Inspect flags (list/status/`--raw`) | | `bin/flag-reader` | 58 | Inspect flags (list/status/`--raw`) |
| `bin/flag-set` | 21 | Set a flag (optionally with a value) | | `bin/flag-set` | 21 | Set a flag (optionally with a value) |
| `bin/flag-clear` | 21 | Unset a flag | | `bin/flag-clear` | 21 | Unset a flag |
@@ -622,7 +640,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `bin/pos-communication-matrix-listener` | 582 | Matrix listener: map /command → bash, run them on room messages | | `bin/pos-communication-matrix-listener` | 582 | Matrix listener: map /command → bash, run them on room messages |
| `bin/pos-communication-matrix-sender` | 215 | Send messages to a Matrix room via the client-server API (send, test, login) | | `bin/pos-communication-matrix-sender` | 215 | Send messages to a Matrix room via the client-server API (send, test, login) |
| `bin/pos-communication-scrcpy` | 245 | Mirror/control an Android device via scrcpy+adb (mirror, devices, record, tcpip, connect, push, pull, screenshot, info) | | `bin/pos-communication-scrcpy` | 245 | Mirror/control an Android device via scrcpy+adb (mirror, devices, record, tcpip, connect, push, pull, screenshot, info) |
| `bin/pos-communication-telegram-listener` | 815 | Telegram bot listener: map /command → bash and <prefix> → app, run them on chat messages | | `bin/pos-communication-telegram-listener` | 889 | Telegram bot listener: map /command → bash and <prefix> → app, run them on chat messages |
| `bin/pos-communication-telegram-sender` | 212 | Send Telegram messages/files/links/stickers via Bot API (send, test) | | `bin/pos-communication-telegram-sender` | 212 | Send Telegram messages/files/links/stickers via Bot API (send, test) |
| `bin/pos-docker-compose` | 487 | Docker Compose service manager (ls/up/down/restart/logs/update/config) | | `bin/pos-docker-compose` | 487 | Docker Compose service manager (ls/up/down/restart/logs/update/config) |
| `bin/pos-docker-health` | 107 | One-glance container health dashboard (exits 1 if unhealthy) | | `bin/pos-docker-health` | 107 | One-glance container health dashboard (exits 1 if unhealthy) |
@@ -634,11 +652,17 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `bin/pos-entertainment-enable` | 49 | Enable an auto-trigger for a plugin on a schedule | | `bin/pos-entertainment-enable` | 49 | Enable an auto-trigger for a plugin on a schedule |
| `bin/pos-entertainment-send` | 95 | Run a public-API plugin and send its output via the configured notify platforms | | `bin/pos-entertainment-send` | 95 | Run a public-API plugin and send its output via the configured notify platforms |
| `bin/pos-entertainment-status` | 62 | Show enabled plugins and scheduler state | | `bin/pos-entertainment-status` | 62 | Show enabled plugins and scheduler state |
| `bin/pos-media-grab` | 219 | Auto-download URL as audio or video (classify + route) | | `bin/pos-media-grab` | 5 | Auto-download URL as audio or video (classify + route) |
| `bin/pos-media-mp3` | 86 | Download audio as MP3 (yt-dlp) | | `bin/pos-media-mp3` | 5 | Download audio as MP3 (yt-dlp) |
| `bin/pos-media-mp4` | 132 | Download video as MP4 (smart/interactive format select) | | `bin/pos-media-mp4` | 5 | Download video as MP4 (smart/interactive format select) |
| `bin/pos-media-sync` | 219 | Incremental Music → USB sync (mp3/mp4, add/update only) | | `bin/pos-media-sync` | 219 | Incremental Music → USB sync (mp3/mp4, add/update only) |
| `bin/pos-media-yt-grab` | 208 | Auto-download URL as audio or video (classify + route) |
| `bin/pos-media-yt-mp3` | 86 | Download audio as MP3 (yt-dlp) |
| `bin/pos-media-yt-mp4` | 132 | Download video as MP4 (smart/interactive format select) |
| `bin/pos-media-yt-subtitles` | 169 | Extract subtitles/captions from a URL (yt-dlp) |
| `bin/pos-media-yt-ytsync` | 5 | Incrementally sync YouTube channels/playlists into ~/Videos |
| `bin/pos-media-ytsync` | 1213 | Incrementally sync YouTube channels/playlists into ~/Videos | | `bin/pos-media-ytsync` | 1213 | Incrementally sync YouTube channels/playlists into ~/Videos |
| `bin/pos-media-yt` | 33 | YouTube download tools (mp3/mp4/grab/ytsync/subtitles) |
| `bin/pos-network-checkport` | 498 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view | | `bin/pos-network-checkport` | 498 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view |
| `bin/pos-network-download` | 1113 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) | | `bin/pos-network-download` | 1113 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) |
| `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui | | `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui |
@@ -655,10 +679,10 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `bin/pos-system-health` | 209 | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL | | `bin/pos-system-health` | 209 | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL |
| `bin/pos-system-schedule` | 151 | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently | | `bin/pos-system-schedule` | 151 | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently |
| `bin/pos-system-uninstall` | 517 | Remove pos toolkit binaries, services, shell integration, config, and data | | `bin/pos-system-uninstall` | 517 | Remove pos toolkit binaries, services, shell integration, config, and data |
| `bin/pos-ai` | 709 | AI assistant: ask, chat, sessions, capture, models, providers | | `bin/pos-ai` | 714 | AI assistant: ask, chat, sessions, capture, models, providers |
| `bin/pos-config` | 80 | Interactive editor for the tools' runtime config (reads # POS_CONFIG: registry) | | `bin/pos-config` | 80 | Interactive editor for the tools' runtime config (reads # POS_CONFIG: registry) |
| `bin/pos-tree` | 118 | Show the pos CLI command tree: categories, commands, and subcommands | | `bin/pos-tree` | 118 | Show the pos CLI command tree: categories, commands, and subcommands |
| `completions/pos.bash` | 314 | Dynamic bash completion | | `completions/pos.bash` | 316 | Dynamic bash completion |
<!-- GEN:END filetable --> <!-- GEN:END filetable -->
| `apps/install.sh` | 171 | App install/uninstall picker/orchestrator | | `apps/install.sh` | 171 | App install/uninstall picker/orchestrator |
@@ -692,7 +716,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| Modify Music→USB sync logic | Edit `bin/pos-media-sync` / shared USB layer `lib/usb-lib.sh` (seams `MEDIA_SYNC_SOURCE`/`MEDIA_SYNC_DEST`/`USB_MOUNT_BASE`/`USB_BYID` in `~/.config/linux_post_install/system.env`) | | Modify Music→USB sync logic | Edit `bin/pos-media-sync` / shared USB layer `lib/usb-lib.sh` (seams `MEDIA_SYNC_SOURCE`/`MEDIA_SYNC_DEST`/`USB_MOUNT_BASE`/`USB_BYID` in `~/.config/linux_post_install/system.env`) |
| Modify YouTube channel sync logic | Edit `bin/pos-media-ytsync` (state in `~/.local/share/linux_post_install/ytsync`; config scope `ytsync` via `pos config ytsync`; research notes `tools-docs/ytsync.md`) | | Modify YouTube channel sync logic | Edit `bin/pos-media-ytsync` (state in `~/.local/share/linux_post_install/ytsync`; config scope `ytsync` via `pos config ytsync`; research notes `tools-docs/ytsync.md`) |
| Modify the scheduler / scheduled jobs | Edit `bin/pos-system-schedule` / `lib/scheduler-lib.sh` (jobs in `~/.config/linux_post_install/schedule.d/`) | | Modify the scheduler / scheduled jobs | Edit `bin/pos-system-schedule` / `lib/scheduler-lib.sh` (jobs in `~/.config/linux_post_install/schedule.d/`) |
| Modify AI logic | Edit `bin/pos-ai` (main tool) + `lib/ai-providers/*.sh` (provider adapters); config scope `ai` via `pos config ai`; `AI_API_KEY`/`AI_MODEL`/`AI_PROVIDER` in `~/.config/linux_post_install/ai.env` | | Modify AI logic | Edit `bin/pos-ai` (main tool) + `lib/ai-providers/*.sh` (provider adapters); config scope `ai` via `pos config ai`; `AI_PROVIDER`, `AI_MODEL`, per-provider `AI_GEMINI_API_KEY`/`OPENROUTER_API_KEY` (legacy shared `AI_API_KEY` accepted as fallback) in `~/.config/linux_post_install/ai.env` |
| Modify UFW/firewall logic | Edit `bin/pos-system-firewall` | | Modify UFW/firewall logic | Edit `bin/pos-system-firewall` |
| Modify pos logging | Edit log setup in `bin/pos` | | Modify pos logging | Edit log setup in `bin/pos` |
| Modify install phases/flags | Edit arg parsing in `install.sh` | | Modify install phases/flags | Edit arg parsing in `install.sh` |
+1 -1
View File
@@ -42,7 +42,7 @@ templates (without overwriting an existing file):
| `system.env` | `pos system health`, `pos system backup` | `BACKUP_SERVICE_ROOTS`, `HEALTH_BACKUP_MAX_AGE_DAYS` | | `system.env` | `pos system health`, `pos system backup` | `BACKUP_SERVICE_ROOTS`, `HEALTH_BACKUP_MAX_AGE_DAYS` |
| `compose.env` | `pos docker compose` | `TS_AUTHKEY`, `TZ`, `DNS_SERVER`, `SERVICES_BASE` | | `compose.env` | `pos docker compose` | `TS_AUTHKEY`, `TZ`, `DNS_SERVER`, `SERVICES_BASE` |
| `entertainment.env` | `pos entertainment *` | plugin keys (`WEATHER_LAT`…), `ENABLED` | | `entertainment.env` | `pos entertainment *` | plugin keys (`WEATHER_LAT`…), `ENABLED` |
| `ai.env` | `pos ai` | `AI_PROVIDER`, `AI_API_KEY`, `AI_MODEL`, `AI_SYSTEM_PROMPT`, `AI_MAX_TOKENS`, `AI_SESSION_TURNS`, `AI_GEMINI_API_KEY`, `AI_GEMINI_MODEL`, `OPENROUTER_API_KEY`, `OPENROUTER_MODEL` | | `ai.env` | `pos ai` | `AI_PROVIDER`, `AI_MODEL`, `AI_SYSTEM_PROMPT`, `AI_MAX_TOKENS`, `AI_SESSION_TURNS`, provider keys `AI_GEMINI_API_KEY` / `OPENROUTER_API_KEY` (+ legacy `AI_API_KEY` fallback) |
| `schedule.d/` | `pos system schedule` | one `<name>.env` per job: `INTERVAL`, `NOTIFY`, `MSG`, `RULE`, `COMMAND` | | `schedule.d/` | `pos system schedule` | one `<name>.env` per job: `INTERVAL`, `NOTIFY`, `MSG`, `RULE`, `COMMAND` |
```bash ```bash
+13 -7
View File
@@ -88,17 +88,17 @@ Backward compatibility: `pos ai gemini`, `pos ai openrouter`, and `pos ai llamac
| Key | Required | Default | Purpose | | Key | Required | Default | Purpose |
|-----|----------|---------|---------| |-----|----------|---------|---------|
| `AI_PROVIDER` | no | `gemini` | Active provider (gemini\|openrouter\|llamacpp) | | `AI_PROVIDER` | no | `gemini` | Active provider (gemini\|openrouter\|llamacpp) |
| `AI_API_KEY` | yes | — | API key for the active provider (secret — masked in `pos config ai`) | | `AI_API_KEY` | no (legacy fallback) | — | Legacy shared API key, used when the active provider's key is empty; not part of the `pos config ai` prompt (set via env or hand-edit); secret |
| `AI_MODEL` | no | per provider | Model id used by `ask`/`chat`/`models` | | `AI_MODEL` | no | per provider | Model id used by `ask`/`chat`/`models` |
| `AI_SYSTEM_PROMPT` | no | built-in terse prompt | Custom system prompt (overrides built-in; empty to reset) | | `AI_SYSTEM_PROMPT` | no | built-in terse prompt | Custom system prompt (overrides built-in; empty to reset) |
| `AI_MAX_TOKENS` | no | `2048` | Max output tokens per request (OpenRouter/Gemini cost cap) | | `AI_MAX_TOKENS` | no | `2048` | Max output tokens per request (OpenRouter/Gemini cost cap) |
| `AI_SESSION_TURNS` | no | `40` | Session message cap — 2 per exchange; 10 = last 5 exchanges | | `AI_SESSION_TURNS` | no | `40` | Session message cap — 2 per exchange; 10 = last 5 exchanges |
| `AI_GEMINI_API_KEY` | fallback | — | Legacy: Gemini API key (used when `AI_API_KEY` is empty) | | `AI_GEMINI_API_KEY` | yes (gemini) | — | Gemini API key (the active key when provider is gemini; secret — masked in `pos config ai`) |
| `AI_GEMINI_MODEL` | fallback | `gemini-2.5-flash` | Legacy: Gemini model id (used when `AI_MODEL` is empty) | | `AI_GEMINI_MODEL` | fallback | `gemini-2.5-flash` | Legacy: Gemini model id (used when `AI_MODEL` is empty) |
| `OPENROUTER_API_KEY` | fallback | — | Legacy: OpenRouter API key (used when `AI_API_KEY` is empty) | | `OPENROUTER_API_KEY` | yes (openrouter) | — | OpenRouter API key (the active key when provider is openrouter; secret — masked in `pos config ai`) |
| `OPENROUTER_MODEL` | fallback | `openrouter/auto` | Legacy: OpenRouter model id (used when `AI_MODEL` is empty) | | `OPENROUTER_MODEL` | fallback | `openrouter/auto` | Legacy: OpenRouter model id (used when `AI_MODEL` is empty) |
Model precedence: `--model` flag > `AI_MODEL` env > provider-specific fallback (`AI_GEMINI_MODEL`/`OPENROUTER_MODEL`) > provider default. API key precedence: `AI_API_KEY` env > provider-specific fallback (`AI_GEMINI_API_KEY`/`OPENROUTER_API_KEY`) > error. `postinstall.sh` copies the repo's `config/ai.env` template to `~/.config/linux_post_install/ai.env` on install (no clobber). Dependencies: `curl` + `jq` (both in `preinstall.sh` PACKAGES). Sessions are stored in OpenAI `messages` format universally; old Gemini-format sessions (`contents[]`) are auto-migrated on load. Model precedence: `--model` flag > `AI_MODEL` env > provider-specific fallback (`AI_GEMINI_MODEL`/`OPENROUTER_MODEL`) > provider default. API key precedence: `<provider>_API_KEY` (`AI_GEMINI_API_KEY` for gemini / `OPENROUTER_API_KEY` for openrouter) > legacy `AI_API_KEY` fallback > error. `AI_API_KEY` is an internal adapter shim and a backward-compat input — it is not offered by `pos config ai`. `postinstall.sh` copies the repo's `config/ai.env` template to `~/.config/linux_post_install/ai.env` on install (no clobber). Dependencies: `curl` + `jq` (both in `preinstall.sh` PACKAGES). Sessions are stored in OpenAI `messages` format universally; old Gemini-format sessions (`contents[]`) are auto-migrated on load.
**Messaging bridges:** the Telegram and Matrix listeners forward non-command messages starting with `ai ` (case-insensitive) to `pos ai ask` and reply with the model's answer — see [communication → listener](#communication). The Telegram bridge uses one session per chat (`telegram-<chat id>`), the Matrix bridge one per room (`matrix-<room>`). **Messaging bridges:** the Telegram and Matrix listeners forward non-command messages starting with `ai ` (case-insensitive) to `pos ai ask` and reply with the model's answer — see [communication → listener](#communication). The Telegram bridge uses one session per chat (`telegram-<chat id>`), the Matrix bridge one per room (`matrix-<room>`).
@@ -263,9 +263,15 @@ The standalone `vbox` command still works and forwards to `pos docker vbox` (see
| Command | File | Purpose | Configuration | | Command | File | Purpose | Configuration |
|---------|------|---------|---------------| |---------|------|---------|---------------|
| `pos media mp3 <url>` | `bin/pos-media-mp3` | Download audio as MP3 via yt-dlp, with thumbnail + metadata | Output to `~/Music/%(title)s.%(ext)s`, `--audio-quality 0` | | `pos media yt` | `bin/pos-media-yt` | YouTube download tools dispatcher (mp3/mp4/grab/ytsync/subtitles); bare invocation prints help listing subcommands | Subcommands resolved via `bin/pos-media-yt-<sub>` files |
| `pos media mp4 <url>` | `bin/pos-media-mp4` | Download video via yt-dlp with **interactive format selection** | Lists formats (`yt-dlp -F`), asks for a format ID, saves to `~/Videos/` | | `pos media yt mp3 <url>` | `bin/pos-media-yt-mp3` | Download audio as MP3 via yt-dlp, with thumbnail + metadata | Output to `~/Music/%(title)s.%(ext)s`, `--audio-quality 0`; `--by-artist` organizes as `<dir>/<artist>/<title>.mp3`. Env seam: `YT_OUT_DIR` overrides the default output dir |
| `pos media grab <url>` | `bin/pos-media-grab` | Auto-download a URL as audio or video (classify + route to mp3/mp4) | Domain-based classification (YouTube Music/SoundCloud/Bandcamp → audio; YouTube/Vimeo/Twitch → video); `--audio`/`--video` force the mode; `--best` default for video (non-interactive); prints a clean summary (🎵/🎬 title, path, size). Config: `GRAB_DEFAULT` (`pos config grab`, default `video`) for unknown domains | | `pos media yt mp4 <url>` | `bin/pos-media-yt-mp4` | Download video via yt-dlp with **interactive format selection** | Lists formats (`yt-dlp -F`), asks for a format ID, saves to `~/Videos/`; `-f/--best/--worst` skip the prompt. Env seam: `YT_OUT_DIR` overrides the default output dir. Reads stdin (interactive format pick) → in `INTERACTIVE_CMDS` |
| `pos media yt grab <url>` | `bin/pos-media-yt-grab` | Auto-download a URL as audio or video (classify + route to yt-mp3/yt-mp4) | Domain-based classification (YouTube Music/SoundCloud/Bandcamp → audio; YouTube/Vimeo/Twitch → video); `--audio`/`--video` force the mode; `--best` default for video (non-interactive); prints a clean summary (🎵/🎬 title, path, size). Config: `GRAB_DEFAULT` (`pos config grab`, default `video`) for unknown domains |
| `pos media yt ytsync [add\|sync\|list\|remove]` | `bin/pos-media-yt-ytsync` | Thin forwarder → `pos media ytsync` (incremental YouTube channel/playlist sync into `~/Videos`) | See `bin/pos-media-ytsync` below |
| `pos media yt subtitles <url>` | `bin/pos-media-yt-subtitles` | Extract subtitles/captions from a URL (yt-dlp) | Default `--write-subs --write-auto-subs --sub-langs best`; `--lang en,ar` (comma = single `--sub-langs` arg); `--format srt` (default) / `vtt` / `txt` (srt→txt post-conversion); `--auto-only` drops manual subs; `--list-subs` probe; output to current dir (`-o ./%(title)s.%(sub_lang)s.%(ext)s`), `--output <dir>` overrides. Non-interactive; needs only yt-dlp (no ffmpeg) |
| `pos media mp3 <url>` | `bin/pos-media-mp3` | **Forwarder** → `pos media yt mp3` (backward-compat alias) | Legacy spelling still works |
| `pos media mp4 <url>` | `bin/pos-media-mp4` | **Forwarder** → `pos media yt mp4` (backward-compat alias) | Legacy spelling still works |
| `pos media grab <url>` | `bin/pos-media-grab` | **Forwarder** → `pos media yt grab` (backward-compat alias) | Legacy spelling still works |
| `pos media sync [--mp3\|--mp4]` | `bin/pos-media-sync` | Incremental Music → USB sync (add/update only — never deletes) | Copies mp3/mp4 from `$HOME/Music` (or `--source <dir>`) into `<usb>/Music/`, preserving the tree; missing or changed (size/mtime) files are copied, identical ones skipped. Same USB detection as `pos system backup` (lsblk TRAN + lsusb/by-id, mount offer for unmounted sticks, multi-stick picker). `--mp3`/`--mp4` filter by extension, neither = both; `--dry-run` previews. Config: `MEDIA_SYNC_SOURCE`, `MEDIA_SYNC_DEST`, shared `USB_MOUNT_BASE`/`USB_BYID` from `~/.config/linux_post_install/system.env`. Result notified via `lib/notify.sh`. Bare invocation on a terminal (or the `menu` subcommand) opens an interactive menu wrapping these actions (sync now mp3+mp4, dry-run preview, mp3-only, mp4-only, change source folder); flags stay scriptable | | `pos media sync [--mp3\|--mp4]` | `bin/pos-media-sync` | Incremental Music → USB sync (add/update only — never deletes) | Copies mp3/mp4 from `$HOME/Music` (or `--source <dir>`) into `<usb>/Music/`, preserving the tree; missing or changed (size/mtime) files are copied, identical ones skipped. Same USB detection as `pos system backup` (lsblk TRAN + lsusb/by-id, mount offer for unmounted sticks, multi-stick picker). `--mp3`/`--mp4` filter by extension, neither = both; `--dry-run` previews. Config: `MEDIA_SYNC_SOURCE`, `MEDIA_SYNC_DEST`, shared `USB_MOUNT_BASE`/`USB_BYID` from `~/.config/linux_post_install/system.env`. Result notified via `lib/notify.sh`. Bare invocation on a terminal (or the `menu` subcommand) opens an interactive menu wrapping these actions (sync now mp3+mp4, dry-run preview, mp3-only, mp4-only, change source folder); flags stay scriptable |
| `pos media ytsync [add\|sync\|list\|remove]` | `bin/pos-media-ytsync` | Incremental YouTube channel/playlist sync — first run asks for a URL (bare invocation = interactive menu; empty state goes straight to the prompt), repeat runs fetch only new videos | One yt-dlp call per new video (`bestvideo*+bestaudio/best` → MP4, metadata/chapters/thumbnail embedded, `--no-overwrites`, `--windows-filenames --trim-filenames 120`); per-source `--download-archive` (`~/.local/share/linux_post_install/ytsync/archive/<slug>.txt`) makes runs crash-safe and idempotent; registry tracks slug/type/url/subdir. Verbs never prompt (scheduler/timer safe); non-tty interactive entry prints a guard line and exits 0. `--dry-run` probes + plans with zero writes. Notify digest only when new>0 or failed>0 via `lib/notify.sh`. Config: `YTSYNC_VIDEOS_DIR`, `YTSYNC_EXTRA_ARGS` via `pos config ytsync`; automate with `pos system schedule` (`COMMAND=pos media ytsync sync`, `NOTIFY=never`) | | `pos media ytsync [add\|sync\|list\|remove]` | `bin/pos-media-ytsync` | Incremental YouTube channel/playlist sync — first run asks for a URL (bare invocation = interactive menu; empty state goes straight to the prompt), repeat runs fetch only new videos | One yt-dlp call per new video (`bestvideo*+bestaudio/best` → MP4, metadata/chapters/thumbnail embedded, `--no-overwrites`, `--windows-filenames --trim-filenames 120`); per-source `--download-archive` (`~/.local/share/linux_post_install/ytsync/archive/<slug>.txt`) makes runs crash-safe and idempotent; registry tracks slug/type/url/subdir. Verbs never prompt (scheduler/timer safe); non-tty interactive entry prints a guard line and exits 0. `--dry-run` probes + plans with zero writes. Notify digest only when new>0 or failed>0 via `lib/notify.sh`. Config: `YTSYNC_VIDEOS_DIR`, `YTSYNC_EXTRA_ARGS` via `pos config ytsync`; automate with `pos system schedule` (`COMMAND=pos media ytsync sync`, `NOTIFY=never`) |
+3 -1
View File
@@ -34,7 +34,8 @@ Everything that runs during the bootstrap install: `install.sh`, `preinstall.sh`
1. **Pre-parse `--no-color`** before anything else, so colors are disabled early (`TERM=dumb` is exported). 1. **Pre-parse `--no-color`** before anything else, so colors are disabled early (`TERM=dumb` is exported).
2. Source `lib/common.sh` (logging, `run`, `spawn`, …) and `lib/flags.sh` (feature flags). 2. Source `lib/common.sh` (logging, `run`, `spawn`, …) and `lib/flags.sh` (feature flags).
3. Parse CLI options. 3. Parse CLI options.
4. For each phase, `should_run <num> <name>` decides whether to run it: 4. **Version gate:** derive the current version (`0.0c<git commit count>` via `install_version()`; empty when `.git` is absent). If the installed version (stored as the `installed_version` flag) matches and `--force` is not given, skip the install — with `--dry-run` it prints `(dry-run) Would skip install: already at version <v>`, otherwise `Already installed (<v>). Use --force to re-install.` and exits 0. When no `installed_version` flag exists or the current version cannot be determined (no `.git`), the gate is skipped.
5. For each phase, `should_run <num> <name>` decides whether to run it:
- `--skip <phase>` removes a phase (takes precedence). - `--skip <phase>` removes a phase (takes precedence).
- `--steps <spec>` restricts the run to the listed phases only (`1,3,4` or `1-3`). - `--steps <spec>` restricts the run to the listed phases only (`1,3,4` or `1-3`).
- Phase map: `1=preinstall`, `2=scripts`, `3=postinstall`, `4=scalepoint` (+ `apps` handled separately). - Phase map: `1=preinstall`, `2=scripts`, `3=postinstall`, `4=scalepoint` (+ `apps` handled separately).
@@ -63,6 +64,7 @@ No config file — everything is command-line:
| `--full` | Core install + every app (non-interactive) | | `--full` | Core install + every app (non-interactive) |
| `--feature` | Install `features/` scripts to `/usr/local/bin/` (prompts on overwrite), sets their flags | | `--feature` | Install `features/` scripts to `/usr/local/bin/` (prompts on overwrite), sets their flags |
| `--dry-run` | Log every action instead of executing. **Note:** applies to `install.sh` itself; `postinstall.sh` runs as a subprocess and does not inherit `DRY_RUN` | | `--dry-run` | Log every action instead of executing. **Note:** applies to `install.sh` itself; `postinstall.sh` runs as a subprocess and does not inherit `DRY_RUN` |
| `--force` | Re-install even if the version matches |
| `--skip <phase>` | Skip a phase (repeatable): `preinstall`, `scripts`, `postinstall`, `scalepoint`, `apps` | | `--skip <phase>` | Skip a phase (repeatable): `preinstall`, `scripts`, `postinstall`, `scalepoint`, `apps` |
| `--steps <spec>` | Run only listed phases: `1,3,4` or `1-3` | | `--steps <spec>` | Run only listed phases: `1,3,4` or `1-3` |
| `--no-color` | Disable colored output | | `--no-color` | Disable colored output |
+6 -3
View File
@@ -69,7 +69,7 @@ exactly as before (no added blank lines), so scripting stays byte-stable.
2. Configure it (masked input): 2. Configure it (masked input):
```bash ```bash
pos config ai # enter AI_API_KEY (or AI_GEMINI_API_KEY) pos config ai # enter AI_GEMINI_API_KEY
``` ```
3. Test: 3. Test:
@@ -81,7 +81,10 @@ exactly as before (no added blank lines), so scripting stays byte-stable.
``` ```
`ai.env` lives at `~/.config/linux_post_install/ai.env` (chmod 600); `pos config ai` `ai.env` lives at `~/.config/linux_post_install/ai.env` (chmod 600); `pos config ai`
is the only place the key is written. The key is never printed by `pos`. is where the provider keys are written, and keys are never printed by `pos`. The
legacy shared `AI_API_KEY` is still honored as a fallback when the provider's own
key is unset — set it via the shell environment or by hand-editing `ai.env` (it is
not part of the `pos config ai` prompt).
## OpenRouter — many providers, one key ## OpenRouter — many providers, one key
@@ -98,7 +101,7 @@ pos ai openrouter ask "hi"
Configure the API key: Configure the API key:
```bash ```bash
pos config ai # enter AI_API_KEY (or OPENROUTER_API_KEY) pos config ai # enter OPENROUTER_API_KEY
``` ```
The default model is `openrouter/auto` (OpenRouter picks the best available The default model is `openrouter/auto` (OpenRouter picks the best available
+68 -27
View File
@@ -3,26 +3,34 @@
Download audio and video from the web via `yt-dlp`, auto-classify URLs, Download audio and video from the web via `yt-dlp`, auto-classify URLs,
sync your library to a USB stick, and keep YouTube channels incrementally sync your library to a USB stick, and keep YouTube channels incrementally
up to date. up to date.
Tools: `grab`, `mp3`, `mp4`, `sync`, `ytsync`. Tools: `yt mp3`, `yt mp4`, `yt grab`, `yt subtitles`, `sync`, `ytsync`.
The YouTube download tools live under `pos media yt` (`mp3`, `mp4`, `grab`,
`subtitles`, `ytsync`). The legacy standalone names `pos media mp3`, `pos media
mp4` and `pos media grab` still work — they are thin forwarders to the `yt`
forms.
| Tool | What it does | | Tool | What it does |
|------|--------------| |------|--------------|
| `pos media grab` | Auto-classify URL and download as audio or video | | `pos media yt mp3` | Download audio, convert to MP3 |
| `pos media mp3` | Download audio, convert to MP3 | | `pos media yt mp4` | Download video with smart/interactive format selection |
| `pos media mp4` | Download video with smart/interactive format selection | | `pos media yt grab` | Auto-classify URL and download as audio or video |
| `pos media yt subtitles` | Extract subtitles/captions from a URL |
| `pos media yt ytsync` | Forwarder → `pos media ytsync` |
| `pos media sync` | Incrementally copy `~/Music` onto a USB stick (mp3/mp4) | | `pos media sync` | Incrementally copy `~/Music` onto a USB stick (mp3/mp4) |
| `pos media ytsync` | Track YouTube channels/playlists and download only new videos into `~/Videos` | | `pos media ytsync` | Track YouTube channels/playlists and download only new videos into `~/Videos` |
Requires `yt-dlp` and `ffmpeg` (`sudo apt install yt-dlp ffmpeg`); the tools Requires `yt-dlp` and `ffmpeg` (`sudo apt install yt-dlp ffmpeg`) for `mp3`/
fail with a clean error message instead of a raw `command not found` if either `mp4`/`grab`; `subtitles` needs only `yt-dlp` (no ffmpeg). The tools fail with a
is missing. clean error message instead of a raw `command not found` if a dependency is
missing.
--- ---
## `pos media mp3` — audio as MP3 ## `pos media yt mp3` — audio as MP3
```bash ```bash
pos media mp3 <url> pos media yt mp3 <url> # or the legacy: pos media mp3 <url>
``` ```
Extracts and converts the audio track to MP3 in `~/Music/`. With `--by-artist` Extracts and converts the audio track to MP3 in `~/Music/`. With `--by-artist`
@@ -30,8 +38,8 @@ files land in `~/Music/<artist>/<title>.mp3` (falls back to the uploader name
when there's no artist tag), so a library stays organized. when there's no artist tag), so a library stays organized.
```bash ```bash
pos media mp3 https://youtube.com/watch?v=dQw4w9WgXcQ pos media yt mp3 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp3 --by-artist https://youtu.be/dQw4w9WgXcQ pos media yt mp3 --by-artist https://youtu.be/dQw4w9WgXcQ
``` ```
MP3s are encoded at best quality with title/artist/album/date/chapters embedded MP3s are encoded at best quality with title/artist/album/date/chapters embedded
@@ -48,15 +56,15 @@ cover art as a JPEG thumbnail. Existing files are never overwritten.
**Recipe:** batch — loop over a list of URLs: **Recipe:** batch — loop over a list of URLs:
```bash ```bash
while read -r url; do pos media mp3 --by-artist "$url"; done < urls.txt while read -r url; do pos media yt mp3 --by-artist "$url"; done < urls.txt
``` ```
--- ---
## `pos media mp4` — video with smart format choice ## `pos media yt mp4` — video with smart format choice
```bash ```bash
pos media mp4 <url> pos media yt mp4 <url> # or the legacy: pos media mp4 <url>
``` ```
Without a format flag, the available formats are shown as a **short curated Without a format flag, the available formats are shown as a **short curated
@@ -77,8 +85,8 @@ Non-interactive (scripting-friendly):
| `--dry-run` | Print the yt-dlp command without running it | | `--dry-run` | Print the yt-dlp command without running it |
```bash ```bash
pos media mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ pos media yt mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ pos media yt mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ
``` ```
Videos merge to MP4 with metadata, chapters, subtitles (all languages) and the Videos merge to MP4 with metadata, chapters, subtitles (all languages) and the
@@ -90,13 +98,13 @@ video+audio and merges them.
--- ---
## `pos media grab` — auto-classify URL and download ## `pos media yt grab` — auto-classify URL and download
```bash ```bash
pos media grab <url> pos media yt grab <url> # or the legacy: pos media grab <url>
``` ```
Smart URL classifier that routes to `pos media mp3` or `pos media mp4` Smart URL classifier that routes to `pos media yt mp3` or `pos media yt mp4`
automatically based on the domain. Send a URL from your phone via Telegram and automatically based on the domain. Send a URL from your phone via Telegram and
the bot downloads it to the right place without you thinking about it. the bot downloads it to the right place without you thinking about it.
@@ -116,16 +124,16 @@ unrecognized domains is `video` — change it with `pos config grab` or set
`GRAB_DEFAULT=audio` in `~/.config/linux_post_install/grab.env`. `GRAB_DEFAULT=audio` in `~/.config/linux_post_install/grab.env`.
```bash ```bash
pos media grab https://music.youtube.com/watch?v=abc # → ~/Music pos media yt grab https://music.youtube.com/watch?v=abc # → ~/Music
pos media grab https://youtube.com/watch?v=xyz # → ~/Videos pos media yt grab https://youtube.com/watch?v=xyz # → ~/Videos
pos media grab --audio https://vimeo.com/123 # force mp3 pos media yt grab --audio https://vimeo.com/123 # force mp3
pos media grab --worst https://youtu.be/abc # lowest quality pos media yt grab --worst https://youtu.be/abc # lowest quality
pos media grab --dry-run https://soundcloud.com/artist/track # preview only pos media yt grab --dry-run https://soundcloud.com/artist/track # preview only
``` ```
Non-interactive by design — `pos media mp4` receives `--best` by default so it Non-interactive by design — `pos media yt mp4` receives `--best` by default so
never prompts for a format (critical for Telegram bot context where there's no it never prompts for a format (critical for Telegram bot context where there's
TTY). Pass `--worst` if you want the smallest file. no TTY). Pass `--worst` if you want the smallest file.
| Flag | Meaning | | Flag | Meaning |
|------|---------| |------|---------|
@@ -140,6 +148,39 @@ TTY). Pass `--worst` if you want the smallest file.
--- ---
## `pos media yt subtitles` — extract subtitles/captions
```bash
pos media yt subtitles <url>
```
Downloads subtitles/captions from a URL via yt-dlp. Fetches manual captions and
auto-generated captions by default (`--write-subs --write-auto-subs
--sub-langs best` — "best" picks the manually-created track when available,
otherwise the auto one). Output files land in the current directory as
`<title>.<lang>.<ext>`.
```bash
pos media yt subtitles https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt subtitles --lang en https://youtu.be/dQw4w9WgXcQ
pos media yt subtitles --lang en,ar --format txt https://youtube.com/watch?v=dQw4w9WgXcQ
```
| Flag | Meaning |
|------|---------|
| `--lang <list>` | Subtitle languages, comma-separated (default `best`) — `en,ar` is passed as ONE `--sub-langs` arg |
| `--format <fmt>` | `srt` (default) / `vtt` / `txt``txt` converts srt→txt (timestamps, seq numbers and HTML tags stripped) |
| `--auto-only` | Only auto-generated captions (no manual subs) |
| `-o, --output <dir>` | Output directory (default: current directory) |
| `--list-subs` | List available subtitles for the URL and exit (probe only, no download) |
| `--no-playlist` | Download only the single video |
| `--dry-run` | Print the yt-dlp command without running it |
If a video has no available subtitles the tool reports
`unavailable subtitles for this video (try --list-subs to check)`.
---
## `pos media sync` — music onto a USB stick ## `pos media sync` — music onto a USB stick
```bash ```bash
+1
View File
@@ -35,6 +35,7 @@ cd Linux_post_install
| `--apps` | Interactive app picker after core install | | `--apps` | Interactive app picker after core install |
| `--full` | Core install + all apps (non-interactive) | | `--full` | Core install + all apps (non-interactive) |
| `--dry-run` | Preview without executing | | `--dry-run` | Preview without executing |
| `--force` | Re-install even if the version matches |
| `--skip <phase>` | Skip a phase: `preinstall`, `scripts`, `postinstall`, `scalepoint`, `apps` | | `--skip <phase>` | Skip a phase: `preinstall`, `scripts`, `postinstall`, `scalepoint`, `apps` |
| `--steps <spec>` | Run only specific phases, e.g. `--steps 1,3` | | `--steps <spec>` | Run only specific phases, e.g. `--steps 1,3` |
| `--no-color` | Disable colored output | | `--no-color` | Disable colored output |
+1 -1
View File
@@ -266,7 +266,7 @@ MAIN_LOG="$LOG_DIR/pos.log"
log_cmd() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $* → exit $2" >> "$MAIN_LOG"; } log_cmd() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $* → exit $2" >> "$MAIN_LOG"; }
# Commands that read from stdin interactively — only log invocation # Commands that read from stdin interactively — only log invocation
INTERACTIVE_CMDS="docker-compose docker-vbox network-hotspot system-firewall media-mp4 media-sync system-backup system-uninstall share-usb-server share-smb-server share-smb-client share-nfs-client share-nfs-server communication-telegram-listener communication-matrix-listener ai ai-gemini ai-openrouter ai-llamacpp ai-alias system-schedule entertainment-config config" INTERACTIVE_CMDS="docker-compose docker-vbox network-hotspot system-firewall media-mp4 media-yt-mp4 media-sync system-backup system-uninstall share-usb-server share-smb-server share-smb-client share-nfs-client share-nfs-server communication-telegram-listener communication-matrix-listener ai ai-gemini ai-openrouter ai-llamacpp ai-alias system-schedule entertainment-config config"
for ((i=n-1; i>=0; i--)); do for ((i=n-1; i>=0; i--)); do
cmd="pos" cmd="pos"
+10 -5
View File
@@ -156,10 +156,14 @@ load_config() {
resolve_key() { resolve_key() {
load_config load_config
local p="${PROVIDER:-gemini}" local p="${PROVIDER:-gemini}"
# Each provider has its own API key — set AI_API_KEY internally for adapters # Provider-specific key wins (leakage guard); legacy shared AI_API_KEY is a
# read-only fallback when the provider's own key is empty — load_config/env
# already exported it (env-wins), so never overwrite it here.
case "$p" in case "$p" in
gemini) [ -n "${AI_GEMINI_API_KEY:-}" ] && export AI_API_KEY="$AI_GEMINI_API_KEY" && return 0 ;; gemini) [ -n "${AI_GEMINI_API_KEY:-}" ] && export AI_API_KEY="$AI_GEMINI_API_KEY" && return 0
openrouter) [ -n "${OPENROUTER_API_KEY:-}" ] && export AI_API_KEY="$OPENROUTER_API_KEY" && return 0 ;; [ -n "${AI_API_KEY:-}" ] && return 0 ;;
openrouter) [ -n "${OPENROUTER_API_KEY:-}" ] && export AI_API_KEY="$OPENROUTER_API_KEY" && return 0
[ -n "${AI_API_KEY:-}" ] && return 0 ;;
llamacpp) return 0 ;; # No API key needed for local server llamacpp) return 0 ;; # No API key needed for local server
esac esac
return 1 return 1
@@ -626,9 +630,10 @@ cmd_providers() {
[ -n "$pmodel" ] || pmodel="unknown" [ -n "$pmodel" ] || pmodel="unknown"
# Check if API key exists for this provider # Check if API key exists for this provider
configured="not configured" configured="not configured"
# Same key set as resolve_key(): provider key, or legacy AI_API_KEY fallback
case "$name" in case "$name" in
gemini) [ -n "${AI_GEMINI_API_KEY:-}" ] && configured="configured" ;; gemini) { [ -n "${AI_GEMINI_API_KEY:-}" ] || [ -n "${AI_API_KEY:-}" ]; } && configured="configured" ;;
openrouter) [ -n "${OPENROUTER_API_KEY:-}" ] && configured="configured" ;; openrouter) { [ -n "${OPENROUTER_API_KEY:-}" ] || [ -n "${AI_API_KEY:-}" ]; } && configured="configured" ;;
llamacpp) configured="configured" ;; # Local server — always configured llamacpp) configured="configured" ;; # Local server — always configured
esac esac
current="" current=""
+88 -14
View File
@@ -36,10 +36,12 @@ Commands:
(none) Interactive editor for the /command → bash map (none) Interactive editor for the /command → bash map
--enable Install + start the systemd user service (autostarts on login) --enable Install + start the systemd user service (autostarts on login)
--disable Stop + disable + remove the service --disable Stop + disable + remove the service
--status Show service state and the command map --status Show service state (single-instance lock) and the command map
--sync-commands --sync-commands
Push the mapped /commands to the bot's "/" menu (setMyCommands) Push the mapped /commands to the bot's "/" menu (setMyCommands)
--run Run the polling loop in the foreground (used by the service) --run Run the polling loop in the foreground (used by the service).
Single instance: only one --run may poll the bot token at a
time — a second --run exits immediately with an error.
prefix [word [command...]] prefix [word [command...]]
Manage the text-prefix map (telegram_prefixes.env): any Manage the text-prefix map (telegram_prefixes.env): any
non-command message '<prefix> <text>' runs the mapped command non-command message '<prefix> <text>' runs the mapped command
@@ -337,24 +339,67 @@ prefix_map_show() {
return 1 return 1
} }
# Run a mapped command line and reply with its output: empty output → "OK", # ── async command execution ─────────────────────────────────────
# non-zero exit → "exit <rc>" + output; quiet=1 suppresses the reply (for # Commands run in the background so the listener never blocks. stdin is
# '@quiet ' entries that self-notify). Used by the /command map (60s cap) # /dev/null (prevents interactive hangs — FFmpeg reading 'q', scripts
# and the text-prefix bridge (120s cap for app calls). # waiting for prompts); stdout+stderr go to a temp file; output is collected
# and replied asynchronously from the main loop.
#
# PID → metadata arrays (populated by run_and_reply, drained by reap_commands)
declare -A _CMD_OUT _CMD_MSG _CMD_QUIET
# Exit codes stored by the SIGCHLD handler (wait -n) so reap_commands can
# retrieve them without calling blocking wait.
declare -A _EXIT_CODES
# run_and_reply <cmdline> <msg_id> [timeout] [quiet]
# Starts the command in the background and returns immediately. The main
# loop calls reap_commands after each getUpdates cycle to collect output
# and send replies.
run_and_reply() { run_and_reply() {
local cmdline="$1" msg_id="$2" tmo="${3:-120}" quiet="${4:-0}" output rc local cmdline="$1" msg_id="$2" tmo="${3:-120}" quiet="${4:-0}"
if output="$(timeout "$tmo" bash -c "$cmdline" 2>&1)"; then local out_file
rc=0 out_file="$(mktemp /tmp/pos-cmd.XXXXXX)"
# stdin=/dev/null: prevents interactive hangs (FFmpeg 'q', read prompts).
# The child inherits nothing from the listener's own stdin.
timeout "$tmo" bash -c "$cmdline" </dev/null >"$out_file" 2>&1 &
local pid=$!
_CMD_OUT[$pid]="$out_file"
_CMD_MSG[$pid]="$msg_id"
_CMD_QUIET[$pid]="$quiet"
}
# reap_commands — called from the main loop after each getUpdates cycle.
# Checks every tracked PID with kill -0 (non-blocking); when a process has
# exited, reads its output file and sends the reply. Never blocks the loop.
reap_commands() {
local pid
for pid in "${!_CMD_OUT[@]}"; do
# Non-blocking: has the process exited?
if ! kill -0 "$pid" 2>/dev/null; then
# Retrieve exit code (SIGCHLD handler stores it; fallback to wait).
local rc="${_EXIT_CODES[$pid]:-}"
if [ -n "$rc" ]; then
unset _EXIT_CODES[$pid]
else else
rc=$? wait "$pid" 2>/dev/null; rc=$?
fi fi
[ "$quiet" -eq 1 ] && return local out_file="${_CMD_OUT[$pid]}"
local msg_id="${_CMD_MSG[$pid]}"
local quiet="${_CMD_QUIET[$pid]}"
local output=""
[ -s "$out_file" ] && output="$(cat "$out_file" 2>/dev/null)"
rm -f "$out_file"
if [ "$quiet" -ne 1 ]; then
[ -n "$output" ] || output="OK" [ -n "$output" ] || output="OK"
if [ "$rc" -ne 0 ]; then if [ "$rc" -ne 0 ]; then
reply "$(printf 'exit %s\n%s' "$rc" "$output")" "$msg_id" "$rc" reply "$(printf 'exit %s\n%s' "$rc" "$output")" "$msg_id" "$rc"
else else
reply "$output" "$msg_id" reply "$output" "$msg_id"
fi fi
fi
unset _CMD_OUT[$pid] _CMD_MSG[$pid] _CMD_QUIET[$pid]
fi
done
} }
ui_run_command() { ui_run_command() {
@@ -521,8 +566,8 @@ disable_service() {
} }
status() { status() {
if systemctl --user is-active --quiet "$SERVICE" 2>/dev/null; then if lock_held; then
echo "listener: running" echo "listener: running (single instance lock held)"
else else
echo "listener: not running" echo "listener: not running"
fi fi
@@ -750,7 +795,31 @@ handle_message() {
run_and_reply "$value" "$msg_id" 60 "$quiet" run_and_reply "$value" "$msg_id" 60 "$quiet"
} }
# ── single-instance guard ──────────────────────────────────────
# flock(1) on a runtime lockfile — the kernel drops the lock when the process
# dies, so there is no stale-lock/pidfile bookkeeping and the systemd
# Restart=always unit restarts cleanly. Two getUpdates loops on one bot token
# cause Telegram 409 conflicts and command stealing, so a second --run fails
# closed instead of racing the active listener.
LOCK_FILE="${XDG_RUNTIME_DIR:-/tmp}/pos-telegram-listener.lock"
acquire_lock() {
exec 9>"$LOCK_FILE"
flock -n 9 || err "listener already running (single instance) — check: systemctl --user status pos-telegram-listener"
}
lock_held() {
# Non-blocking probe: acquiring then dropping the flock in a subshell
# succeeds only when nobody else holds it. Returns 0 when held.
if ( flock -n 9 ) 9>"$LOCK_FILE" 2>/dev/null; then
return 1
fi
return 0
}
run_daemon() { run_daemon() {
command -v flock &>/dev/null || err "flock not found (install util-linux)"
acquire_lock
command -v jq &>/dev/null || err "jq not found (install jq — in preinstall PACKAGES)" command -v jq &>/dev/null || err "jq not found (install jq — in preinstall PACKAGES)"
load_config load_config
[ -n "${TELEGRAM_BOT_TOKEN:-}" ] || err "No bot token — run 'pos config telegram'" [ -n "${TELEGRAM_BOT_TOKEN:-}" ] || err "No bot token — run 'pos config telegram'"
@@ -764,7 +833,10 @@ run_daemon() {
local offset=0 local offset=0
log "listener running (chat ${TELEGRAM_CHAT_ID}, owner ${TELEGRAM_OWNER_ID:-unset}) — Ctrl+C to stop" log "listener running (chat ${TELEGRAM_CHAT_ID}, owner ${TELEGRAM_OWNER_ID:-unset}) — Ctrl+C to stop"
trap 'kill $(jobs -p) 2>/dev/null; exit 0' TERM INT # SIGCHLD: reap finished children and store their exit codes so
# reap_commands can retrieve them without blocking.
trap 'local _p; while _p=$(wait -n 2>/dev/null); do _EXIT_CODES[$_p]=$?; done' CHLD
trap 'kill $(jobs -p) 2>/dev/null; rm -f /tmp/pos-cmd.* 2>/dev/null; wait 2>/dev/null; exit 0' TERM INT
while true; do while true; do
local resp n i local resp n i
resp="$(curl -fsS -m 45 "${API}/bot${TELEGRAM_BOT_TOKEN}/getUpdates" \ resp="$(curl -fsS -m 45 "${API}/bot${TELEGRAM_BOT_TOKEN}/getUpdates" \
@@ -799,6 +871,8 @@ run_daemon() {
fi fi
handle_message "$text" "$msg_id" "$reply_text" handle_message "$text" "$msg_id" "$reply_text"
done done
# Collect output from finished background commands and send replies.
reap_commands
done done
} }
+2 -216
View File
@@ -1,219 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
# POS: media grab — Auto-download URL as audio or video (classify + route) # POS: media grab — Auto-download URL as audio or video (classify + route)
# POS_FLAGS: --audio --video --best --worst --output --no-playlist --cookies --dry-run # -h|--help forwarded via exec below
# POS_CONFIG: grab | grab.env | GRAB_DEFAULT=:Default mode for unknown domains (video or audio, default video) exec pos media yt grab "$@"
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
# Shared config loader (canonical KEY=VALUE parser, env-wins precedence)
source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh"
# Load grab.env config (env-seam: GRAB_DEFAULT)
load_grab_config() {
load_env_file "$CONFIG_DIR/grab.env"
}
load_grab_config
# ── URL classification ─────────────────────────────────────────
classify_url() {
local url="$1" mode="${GRAB_DEFAULT:-video}"
case "$url" in
*music.youtube.com*) echo "audio" ;;
*soundcloud.com*) echo "audio" ;;
*bandcamp.com*) echo "audio" ;;
*youtube.com*|*youtu.be*) echo "video" ;;
*vimeo.com*) echo "video" ;;
*twitch.tv*) echo "video" ;;
*) echo "$mode" ;;
esac
}
usage() {
cat <<EOF
Usage: pos media grab [options] <url>
Auto-download a URL as audio or video. Classifies the domain and delegates
to 'pos media mp3' (audio) or 'pos media mp4' (video).
Options:
--audio Force audio (mp3) download
--video Force video (mp4) download
--best Best quality for video (default for non-interactive)
--worst Lowest quality for video
-o, --output <dir> Output directory (passed to mp3/mp4)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the command that would run, don't execute
-h, --help This help
Examples:
pos media grab https://music.youtube.com/watch?v=abc
pos media grab https://youtube.com/watch?v=xyz
pos media grab --audio https://vimeo.com/123
pos media grab --worst https://youtu.be/abc
pos media grab --dry-run https://soundcloud.com/artist/track
EOF
exit 0
}
# ── Arg parsing ────────────────────────────────────────────────
URL=""
FORCE_AUDIO=0
FORCE_VIDEO=0
BEST=0
WORST=0
DRY_RUN=0
EXTRA_ARGS=()
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
--audio) FORCE_AUDIO=1; shift ;;
--video) FORCE_VIDEO=1; shift ;;
--best) BEST=1; shift ;;
--worst) WORST=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "pos media grab: --output needs a value"
EXTRA_ARGS+=(--output "$2"); shift 2 ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "pos media grab: --cookies needs a value"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "pos media grab: Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "pos media grab: Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
# Validate URL scheme
case "$URL" in
http://*|https://*) ;;
*) err "pos media grab: not a valid URL: $URL (must start with http:// or https://)" ;;
esac
# Validate mutually exclusive overrides
[ "$FORCE_AUDIO" -eq 1 ] && [ "$FORCE_VIDEO" -eq 1 ] && \
err "pos media grab: --audio and --video are mutually exclusive"
[ "$BEST" -eq 1 ] && [ "$WORST" -eq 1 ] && \
err "pos media grab: --best and --worst are mutually exclusive"
# ── Classification ─────────────────────────────────────────────
mode=""
if [ "$FORCE_AUDIO" -eq 1 ]; then
mode="audio"
elif [ "$FORCE_VIDEO" -eq 1 ]; then
mode="video"
else
mode="$(classify_url "$URL")"
fi
# ── Build delegated command ────────────────────────────────────
DELEGATE_ARGS=()
if [ "$mode" = "audio" ]; then
DELEGATE_ARGS=(pos media mp3 "${EXTRA_ARGS[@]}")
else
# mp4 route: --best by default (non-interactive), --worst if user passes it
if [ "$WORST" -eq 1 ]; then
DELEGATE_ARGS=(pos media mp4 --worst "${EXTRA_ARGS[@]}")
else
DELEGATE_ARGS=(pos media mp4 --best "${EXTRA_ARGS[@]}")
fi
fi
# ── Dry run ────────────────────────────────────────────────────
if [ "$DRY_RUN" -eq 1 ]; then
echo "${DELEGATE_ARGS[*]} $URL"
exit 0
fi
# ── Execute ────────────────────────────────────────────────────
output=""
rc=0
if output=$("${DELEGATE_ARGS[@]}" "$URL" 2>&1); then
rc=0
else
rc=$?
fi
if [ "$rc" -ne 0 ]; then
# Summarize stderr for the user
summary="$(printf '%s' "$output" | grep -i 'error\|fail' | head -1 || true)"
[ -z "$summary" ] && summary="exit code $rc"
err "pos media grab: ❌ Download failed: $summary"
fi
# ── Metadata + summary ────────────────────────────────────────
title=""
duration=""
file_path=""
# Determine expected output directory
if [ "$mode" = "audio" ]; then
out_dir="$HOME/Music"
for (( i=0; i<${#EXTRA_ARGS[@]}; i++ )); do
if [ "${EXTRA_ARGS[$i]}" = "--output" ] && [ $(( i + 1 )) -lt ${#EXTRA_ARGS[@]} ]; then
out_dir="${EXTRA_ARGS[$(( i + 1 ))]}"
break
fi
done
file_ext="mp3"
else
out_dir="$HOME/Videos"
for (( i=0; i<${#EXTRA_ARGS[@]}; i++ )); do
if [ "${EXTRA_ARGS[$i]}" = "--output" ] && [ $(( i + 1 )) -lt ${#EXTRA_ARGS[@]} ]; then
out_dir="${EXTRA_ARGS[$(( i + 1 ))]}"
break
fi
done
file_ext="mp4"
fi
# Fetch metadata (fast, no download)
if command -v yt-dlp &>/dev/null; then
meta="$(yt-dlp --print title --print duration_string --no-warnings "$URL" 2>/dev/null || true)"
title="$(printf '%s' "$meta" | sed -n '1p')"
duration="$(printf '%s' "$meta" | sed -n '2p')"
fi
# Find the downloaded file (most recent matching extension in out_dir)
if [ -d "$out_dir" ]; then
file_path="$(find "$out_dir" -maxdepth 1 -name "*.$file_ext" -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2- || true)"
fi
# Build summary
[ -z "$title" ] && title="$(basename "$URL" | sed 's/[?#].*//')"
[ -z "$duration" ] && duration="?"
if [ "$mode" = "audio" ]; then
emoji="🎵"
else
emoji="🎬"
fi
echo "$emoji Downloaded: $title ($duration)"
if [ -n "$file_path" ] && [ -f "$file_path" ]; then
file_size="$(stat --printf='%s' "$file_path" 2>/dev/null || echo "0")"
# Format size in human-readable form
if [ "$file_size" -ge 1073741824 ]; then
size_human="$(awk "BEGIN { printf \"%.1f GB\", $file_size / 1073741824 }")"
elif [ "$file_size" -ge 1048576 ]; then
size_human="$(awk "BEGIN { printf \"%.1f MB\", $file_size / 1048576 }")"
elif [ "$file_size" -ge 1024 ]; then
size_human="$(awk "BEGIN { printf \"%.1f KB\", $file_size / 1024 }")"
else
size_human="${file_size} B"
fi
# Show path relative to HOME
rel_path="${file_path/#$HOME/\~}"
echo "📁 $rel_path ($size_human)"
else
echo "📁 $out_dir/ ($file_ext)"
fi
+2 -83
View File
@@ -1,86 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
# POS: media mp3 — Download audio as MP3 (yt-dlp) # POS: media mp3 — Download audio as MP3 (yt-dlp)
# POS_FLAGS: --output --no-playlist --cookies --by-artist --dry-run # -h|--help forwarded via exec below
exec pos media yt mp3 "$@"
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
# --dry-run is parsed before the deps guards (below): it previews the yt-dlp
# command without requiring yt-dlp/ffmpeg to be installed.
DRY_RUN=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
done
# Deps guards sit before -h|--help (help also errors on a box missing the deps).
if [ "$DRY_RUN" -eq 0 ]; then
command -v yt-dlp &>/dev/null || err "yt-dlp not found — install it with: sudo apt install yt-dlp"
command -v ffmpeg &>/dev/null || err "ffmpeg not found (needed for MP3 conversion) — install it with: sudo apt install ffmpeg"
fi
OUT_DIR="$HOME/Music"
BY_ARTIST=0
EXTRA_ARGS=()
usage() {
cat <<EOF
Usage: pos media mp3 [options] <url>
Download audio from a URL and convert to MP3 via yt-dlp.
Options:
-o, --output <dir> Output directory (default: $OUT_DIR)
--by-artist Organize as <dir>/<artist>/<title>.mp3
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media mp3 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp3 --by-artist --no-playlist https://youtu.be/dQw4w9WgXcQ
pos media mp3 --cookies ~/cookies.txt https://vimeo.com/123
EOF
exit 0
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--by-artist) BY_ARTIST=1; shift ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "--cookies needs a value"
[ -f "$2" ] || err "cookies file not found: $2"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
if [ "$BY_ARTIST" -eq 1 ]; then
OUT_TEMPLATE="$OUT_DIR/%(artist,uploader)s/%(title)s.%(ext)s"
else
OUT_TEMPLATE="$OUT_DIR/%(title)s.%(ext)s"
fi
args=(yt-dlp -x --audio-format mp3 --audio-quality 0
--embed-metadata --embed-chapters
--embed-thumbnail --convert-thumbnails jpg
--parse-metadata "%(artist,uploader)s:%(artist)s"
--no-overwrites
-o "$OUT_TEMPLATE"
"${EXTRA_ARGS[@]}" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
echo "yt-dlp ${args[*]:1}"
else
spawn "downloading audio → $OUT_DIR" "${args[@]}"
fi
+2 -129
View File
@@ -1,132 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
# POS: media mp4 — Download video as MP4 (smart/interactive format select) # POS: media mp4 — Download video as MP4 (smart/interactive format select)
# POS_FLAGS: --format --best --worst --output --no-playlist --cookies --dry-run # -h|--help forwarded via exec below
exec pos media yt mp4 "$@"
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
# --dry-run is parsed before the deps guards (below): it previews the yt-dlp
# command without requiring yt-dlp/ffmpeg to be installed.
DRY_RUN=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
done
# Deps guards sit before -h|--help (help also errors on a box missing the deps).
if [ "$DRY_RUN" -eq 0 ]; then
command -v yt-dlp &>/dev/null || err "yt-dlp not found — install it with: sudo apt install yt-dlp"
command -v ffmpeg &>/dev/null || err "ffmpeg not found (needed for MP4 merge) — install it with: sudo apt install ffmpeg"
fi
OUT_DIR="$HOME/Videos"
FORMAT=""
BEST=0
WORST=0
EXTRA_ARGS=()
usage() {
cat <<EOF
Usage: pos media mp4 [options] <url>
Download video from a URL via yt-dlp. Without -f/--best/--worst, formats are
listed and picked interactively.
Options:
-f, --format <id> Download that format id directly (no prompt)
--best Best video + audio (no prompt)
--worst Lowest quality (no prompt)
-o, --output <dir> Output directory (default: $OUT_DIR)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media mp4 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ
EOF
exit 0
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
-f|--format)
[ $# -ge 2 ] || err "--format needs a value"
FORMAT="$2"; shift 2 ;;
--best) BEST=1; shift ;;
--worst) WORST=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "--cookies needs a value"
[ -f "$2" ] || err "cookies file not found: $2"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
[ -n "$FORMAT" ] && { [ "$BEST" -eq 1 ] || [ "$WORST" -eq 1 ]; } \
&& err "--format and --best/--worst are mutually exclusive"
[ "$BEST" -eq 1 ] && [ "$WORST" -eq 1 ] && err "--best and --worst are mutually exclusive"
# Resolve the format selector: explicit flag → smart preset → interactive.
# The curated table goes to stderr; stdout carries ONLY the chosen format id
# (same ui_pick lesson — a $(...) capture must not swallow the menu).
fmt_spec=""
if [ -n "$FORMAT" ]; then
fmt_spec="$FORMAT"
elif [ "$BEST" -eq 1 ]; then
fmt_spec="bestvideo*+bestaudio/best"
elif [ "$WORST" -eq 1 ]; then
fmt_spec="worst"
else
table="$(yt-dlp -F "$URL" 2>&1)" || {
echo "ERROR: no formats listed (unsupported site or age-gated content)" >&2
exit 1
}
printf '%s' "$table" | awk -F'[[:space:]][[:space:]]+' '
NR <= 2 { print; next }
$0 ~ /^-+$/ { print; next }
$0 ~ /audio only/ && $0 !~ /storyboard/ && $1 !~ /^sb/ { print " [audio] " $0; next }
$0 ~ /video only/ && $1 !~ /^sb/ { print " [video] " $0; next }
$1 ~ /^[0-9]+$/ && $2 ~ /^(mp4|webm)$/ && $3 ~ /x/ { print " [combo] " $0 }
' >&2
echo >&2
read -rp "Format ID ('best'/'worst'/empty = best): " chosen
case "${chosen:-best}" in
best) fmt_spec="bestvideo*+bestaudio/best" ;;
worst) fmt_spec="worst" ;;
*)
if printf '%s' "$table" | grep -qE "^[[:space:]]*${chosen}[[:space:]]"; then
fmt_spec="$chosen"
else
echo "ERROR: unknown format id '$chosen'" >&2
exit 1
fi
;;
esac
fi
args=(yt-dlp -f "$fmt_spec"
--merge-output-format mp4
--embed-metadata --embed-chapters
--embed-subs --sub-langs all
--embed-thumbnail
--no-overwrites
-o "$OUT_DIR/%(title)s.%(ext)s"
"${EXTRA_ARGS[@]}" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
echo "yt-dlp ${args[*]:1}"
else
spawn "downloading video → $OUT_DIR" "${args[@]}"
fi
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt — YouTube download tools (mp3/mp4/grab/ytsync/subtitles)
# POS_SUBCMDS: mp3 mp4 grab ytsync subtitles
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
usage() {
cat <<'EOF'
Usage: pos media yt <command> [args]
YouTube download tools: audio, video, auto-grab, subtitles, channel sync.
Commands:
mp3 <url> Download audio as MP3
mp4 <url> Download video as MP4 (interactive format select)
grab <url> Auto-download as audio or video
ytsync [cmd] Incremental YouTube channel sync
subtitles <url> Extract subtitles/captions
Options:
-h, --help This help
Run 'pos media yt <command> --help' for details.
EOF
exit 0
}
case "${1:-}" in
-h|--help|"") usage ;;
esac
# Subcommands are resolved by the dispatcher via pos-media-yt-<sub> files.
# This file only handles bare 'pos media yt' (help) and unknown args.
err "unknown yt command: $1 (see 'pos media yt --help')"
+208
View File
@@ -0,0 +1,208 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-grab — Auto-download URL as audio or video (classify + route)
# POS_FLAGS: --audio --video --best --worst --output --no-playlist --cookies --dry-run
# POS_CONFIG: grab | grab.env | GRAB_DEFAULT=:Default mode for unknown domains (video or audio, default video)
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
# Shared config loader (canonical KEY=VALUE parser, env-wins precedence)
source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh"
# Shared yt helpers (classify_url, yt_validate_url, yt_check_deps, yt_echo_cmd)
source "$(dirname "$0")/../lib/yt-lib.sh" 2>/dev/null || source "$(dirname "$0")/yt-lib.sh"
# Load grab.env config (env-seam: GRAB_DEFAULT)
load_grab_config() {
load_env_file "$CONFIG_DIR/grab.env"
}
load_grab_config
usage() {
cat <<EOF
Usage: pos media yt grab [options] <url>
Auto-download a URL as audio or video. Classifies the domain and delegates
to 'pos media yt mp3' (audio) or 'pos media yt mp4' (video).
Options:
--audio Force audio (mp3) download
--video Force video (mp4) download
--best Best quality for video (default for non-interactive)
--worst Lowest quality for video
-o, --output <dir> Output directory (passed to mp3/mp4)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the command that would run, don't execute
-h, --help This help
Examples:
pos media yt grab https://music.youtube.com/watch?v=abc
pos media yt grab https://youtube.com/watch?v=xyz
pos media yt grab --audio https://vimeo.com/123
pos media yt grab --worst https://youtu.be/abc
pos media yt grab --dry-run https://soundcloud.com/artist/track
The legacy spelling 'pos media grab' still works (forwarder) and is equivalent.
EOF
exit 0
}
# ── Arg parsing ────────────────────────────────────────────────
URL=""
FORCE_AUDIO=0
FORCE_VIDEO=0
BEST=0
WORST=0
DRY_RUN=0
EXTRA_ARGS=()
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
--audio) FORCE_AUDIO=1; shift ;;
--video) FORCE_VIDEO=1; shift ;;
--best) BEST=1; shift ;;
--worst) WORST=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "yt-grab: --output needs a value"
EXTRA_ARGS+=(--output "$2"); shift 2 ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "yt-grab: --cookies needs a value"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "yt-grab: Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "yt-grab: Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
# Validate URL scheme
yt_validate_url "$URL" 2>/dev/null || \
err "yt-grab: not a valid URL: $URL (must start with http:// or https://)"
# Validate mutually exclusive overrides
[ "$FORCE_AUDIO" -eq 1 ] && [ "$FORCE_VIDEO" -eq 1 ] && \
err "yt-grab: --audio and --video are mutually exclusive"
[ "$BEST" -eq 1 ] && [ "$WORST" -eq 1 ] && \
err "yt-grab: --best and --worst are mutually exclusive"
# ── Classification ─────────────────────────────────────────────
mode=""
if [ "$FORCE_AUDIO" -eq 1 ]; then
mode="audio"
elif [ "$FORCE_VIDEO" -eq 1 ]; then
mode="video"
else
mode="$(classify_url "$URL")"
fi
# ── Build delegated command ────────────────────────────────────
DELEGATE_ARGS=()
if [ "$mode" = "audio" ]; then
DELEGATE_ARGS=(pos media yt mp3 "${EXTRA_ARGS[@]}")
else
# mp4 route: --best by default (non-interactive), --worst if user passes it
if [ "$WORST" -eq 1 ]; then
DELEGATE_ARGS=(pos media yt mp4 --worst "${EXTRA_ARGS[@]}")
else
DELEGATE_ARGS=(pos media yt mp4 --best "${EXTRA_ARGS[@]}")
fi
fi
# ── Dry run ────────────────────────────────────────────────────
if [ "$DRY_RUN" -eq 1 ]; then
echo "${DELEGATE_ARGS[*]} $URL"
exit 0
fi
# ── Execute ────────────────────────────────────────────────────
output=""
rc=0
if output=$("${DELEGATE_ARGS[@]}" "$URL" 2>&1); then
rc=0
else
rc=$?
fi
if [ "$rc" -ne 0 ]; then
# Summarize stderr for the user
summary="$(printf '%s' "$output" | grep -i 'error\|fail' | head -1 || true)"
[ -z "$summary" ] && summary="exit code $rc"
err "yt-grab: ❌ Download failed: $summary"
fi
# ── Metadata + summary ────────────────────────────────────────
title=""
duration=""
file_path=""
# Determine expected output directory
if [ "$mode" = "audio" ]; then
out_dir="$HOME/Music"
for (( i=0; i<${#EXTRA_ARGS[@]}; i++ )); do
if [ "${EXTRA_ARGS[$i]}" = "--output" ] && [ $(( i + 1 )) -lt ${#EXTRA_ARGS[@]} ]; then
out_dir="${EXTRA_ARGS[$(( i + 1 ))]}"
break
fi
done
file_ext="mp3"
else
out_dir="$HOME/Videos"
for (( i=0; i<${#EXTRA_ARGS[@]}; i++ )); do
if [ "${EXTRA_ARGS[$i]}" = "--output" ] && [ $(( i + 1 )) -lt ${#EXTRA_ARGS[@]} ]; then
out_dir="${EXTRA_ARGS[$(( i + 1 ))]}"
break
fi
done
file_ext="mp4"
fi
# Fetch metadata (fast, no download)
if command -v yt-dlp &>/dev/null; then
meta="$(yt-dlp --print title --print duration_string --no-warnings "$URL" 2>/dev/null || true)"
title="$(printf '%s' "$meta" | sed -n '1p')"
duration="$(printf '%s' "$meta" | sed -n '2p')"
fi
# Find the downloaded file (most recent matching extension in out_dir)
if [ -d "$out_dir" ]; then
file_path="$(find "$out_dir" -maxdepth 1 -name "*.$file_ext" -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2- || true)"
fi
# Build summary
[ -z "$title" ] && title="$(basename "$URL" | sed 's/[?#].*//')"
[ -z "$duration" ] && duration="?"
if [ "$mode" = "audio" ]; then
emoji="🎵"
else
emoji="🎬"
fi
echo "$emoji Downloaded: $title ($duration)"
if [ -n "$file_path" ] && [ -f "$file_path" ]; then
file_size="$(stat --printf='%s' "$file_path" 2>/dev/null || echo "0")"
# Format size in human-readable form
if [ "$file_size" -ge 1073741824 ]; then
size_human="$(awk "BEGIN { printf \"%.1f GB\", $file_size / 1073741824 }")"
elif [ "$file_size" -ge 1048576 ]; then
size_human="$(awk "BEGIN { printf \"%.1f MB\", $file_size / 1048576 }")"
elif [ "$file_size" -ge 1024 ]; then
size_human="$(awk "BEGIN { printf \"%.1f KB\", $file_size / 1024 }")"
else
size_human="${file_size} B"
fi
# Show path relative to HOME
rel_path="${file_path/#$HOME/\~}"
echo "📁 $rel_path ($size_human)"
else
echo "📁 $out_dir/ ($file_ext)"
fi
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-mp3 — Download audio as MP3 (yt-dlp)
# POS_FLAGS: --output --no-playlist --cookies --by-artist --dry-run
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
source "$(dirname "$0")/../lib/yt-lib.sh" 2>/dev/null || source "$(dirname "$0")/yt-lib.sh"
# --dry-run is parsed before the deps guards (below): it previews the yt-dlp
# command without requiring yt-dlp/ffmpeg to be installed.
DRY_RUN=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
done
# Deps guards sit before -h|--help (help also errors on a box missing the deps).
# yt_check_deps with dry-run=1 skips the dep checks (dry-run preview needs none).
yt_check_deps "$DRY_RUN" "MP3 conversion"
# Env seam: YT_OUT_DIR overrides the default (keeps HOME sandbox-compatible).
OUT_DIR="${YT_OUT_DIR:-$HOME/Music}"
BY_ARTIST=0
EXTRA_ARGS=()
usage() {
cat <<EOF
Usage: pos media yt mp3 [options] <url>
Download audio from a URL and convert to MP3 via yt-dlp.
Options:
-o, --output <dir> Output directory (default: $OUT_DIR)
--by-artist Organize as <dir>/<artist>/<title>.mp3
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media yt mp3 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp3 --by-artist --no-playlist https://youtu.be/dQw4w9WgXcQ
pos media yt mp3 --cookies ~/cookies.txt https://vimeo.com/123
EOF
exit 0
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--by-artist) BY_ARTIST=1; shift ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "--cookies needs a value"
[ -f "$2" ] || err "cookies file not found: $2"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
if [ "$BY_ARTIST" -eq 1 ]; then
OUT_TEMPLATE="$OUT_DIR/%(artist,uploader)s/%(title)s.%(ext)s"
else
OUT_TEMPLATE="$OUT_DIR/%(title)s.%(ext)s"
fi
args=(yt-dlp -x --audio-format mp3 --audio-quality 0
--embed-metadata --embed-chapters
--embed-thumbnail --convert-thumbnails jpg
--parse-metadata "%(artist,uploader)s:%(artist)s"
--no-overwrites
-o "$OUT_TEMPLATE"
"${EXTRA_ARGS[@]}" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
echo "yt-dlp ${args[*]:1}"
else
spawn "downloading audio → $OUT_DIR" "${args[@]}"
fi
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-mp4 — Download video as MP4 (smart/interactive format select)
# POS_FLAGS: --format --best --worst --output --no-playlist --cookies --dry-run
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
source "$(dirname "$0")/../lib/yt-lib.sh" 2>/dev/null || source "$(dirname "$0")/yt-lib.sh"
# --dry-run is parsed before the deps guards (below): it previews the yt-dlp
# command without requiring yt-dlp/ffmpeg to be installed.
DRY_RUN=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
done
# Deps guards sit before -h|--help (help also errors on a box missing the deps).
# yt_check_deps with dry-run=1 skips the dep checks (dry-run preview needs none).
yt_check_deps "$DRY_RUN" "MP4 merge"
# Env seam: YT_OUT_DIR overrides the default (keeps HOME sandbox-compatible).
OUT_DIR="${YT_OUT_DIR:-$HOME/Videos}"
FORMAT=""
BEST=0
WORST=0
EXTRA_ARGS=()
usage() {
cat <<EOF
Usage: pos media yt mp4 [options] <url>
Download video from a URL via yt-dlp. Without -f/--best/--worst, formats are
listed and picked interactively.
Options:
-f, --format <id> Download that format id directly (no prompt)
--best Best video + audio (no prompt)
--worst Lowest quality (no prompt)
-o, --output <dir> Output directory (default: $OUT_DIR)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media yt mp4 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ
EOF
exit 0
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
-f|--format)
[ $# -ge 2 ] || err "--format needs a value"
FORMAT="$2"; shift 2 ;;
--best) BEST=1; shift ;;
--worst) WORST=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "--cookies needs a value"
[ -f "$2" ] || err "cookies file not found: $2"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
[ -n "$FORMAT" ] && { [ "$BEST" -eq 1 ] || [ "$WORST" -eq 1 ]; } \
&& err "--format and --best/--worst are mutually exclusive"
[ "$BEST" -eq 1 ] && [ "$WORST" -eq 1 ] && err "--best and --worst are mutually exclusive"
# Resolve the format selector: explicit flag → smart preset → interactive.
# The curated table goes to stderr; stdout carries ONLY the chosen format id
# (same ui_pick lesson — a $(...) capture must not swallow the menu).
fmt_spec=""
if [ -n "$FORMAT" ]; then
fmt_spec="$FORMAT"
elif [ "$BEST" -eq 1 ]; then
fmt_spec="bestvideo*+bestaudio/best"
elif [ "$WORST" -eq 1 ]; then
fmt_spec="worst"
else
table="$(yt-dlp -F "$URL" 2>&1)" || {
echo "ERROR: no formats listed (unsupported site or age-gated content)" >&2
exit 1
}
printf '%s' "$table" | awk -F'[[:space:]][[:space:]]+' '
NR <= 2 { print; next }
$0 ~ /^-+$/ { print; next }
$0 ~ /audio only/ && $0 !~ /storyboard/ && $1 !~ /^sb/ { print " [audio] " $0; next }
$0 ~ /video only/ && $1 !~ /^sb/ { print " [video] " $0; next }
$1 ~ /^[0-9]+$/ && $2 ~ /^(mp4|webm)$/ && $3 ~ /x/ { print " [combo] " $0 }
' >&2
echo >&2
read -rp "Format ID ('best'/'worst'/empty = best): " chosen
case "${chosen:-best}" in
best) fmt_spec="bestvideo*+bestaudio/best" ;;
worst) fmt_spec="worst" ;;
*)
if printf '%s' "$table" | grep -qE "^[[:space:]]*${chosen}[[:space:]]"; then
fmt_spec="$chosen"
else
echo "ERROR: unknown format id '$chosen'" >&2
exit 1
fi
;;
esac
fi
args=(yt-dlp -f "$fmt_spec"
--merge-output-format mp4
--embed-metadata --embed-chapters
--embed-subs --sub-langs all
--embed-thumbnail
--no-overwrites
-o "$OUT_DIR/%(title)s.%(ext)s"
"${EXTRA_ARGS[@]}" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
echo "yt-dlp ${args[*]:1}"
else
spawn "downloading video → $OUT_DIR" "${args[@]}"
fi
+169
View File
@@ -0,0 +1,169 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-subtitles — Extract subtitles/captions from a URL (yt-dlp)
# POS_FLAGS: --lang --format --auto-only --output --list-subs --no-playlist --dry-run
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
source "$(dirname "$0")/../lib/yt-lib.sh" 2>/dev/null || source "$(dirname "$0")/yt-lib.sh"
# --dry-run / --list-subs are parsed before the deps guards (below): dry-run
# previews the yt-dlp command without requiring yt-dlp to be installed.
DRY_RUN=0
LIST_SUBS=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
[ "$arg" = "--list-subs" ] && LIST_SUBS=1
done
# Deps guards sit before -h|--help. Subtitles needs only yt-dlp (no ffmpeg).
# dry-run skips the deps check entirely (preview needs no binary).
if [ "$DRY_RUN" -eq 0 ]; then
command -v yt-dlp &>/dev/null || err "yt-dlp not found — install it with: sudo apt install yt-dlp"
fi
LANG_SPEC=""
FMT="srt"
AUTO_ONLY=0
OUT_DIR=""
NO_PLAYLIST=0
usage() {
cat <<EOF
Usage: pos media yt subtitles [options] <url>
Extract subtitles/captions from a URL via yt-dlp. Fetches manual captions and
auto-generated captions by default.
Options:
--lang <list> Subtitle languages (comma-separated), default: best
--format <fmt> Output format: srt (default), vtt, or txt
--auto-only Only auto-generated captions (no manual subs)
-o, --output <dir> Output directory (default: current directory)
--list-subs List available subtitles and exit (probe only)
--no-playlist Download only the single video
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media yt subtitles https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt subtitles --lang en,ar https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt subtitles --format txt https://youtu.be/dQw4w9WgXcQ
pos media yt subtitles --list-subs https://youtube.com/watch?v=dQw4w9WgXcQ
EOF
exit 0
}
# convert_srt_to_txt <srt-file> — strip SRT timestamps, seq numbers, HTML tags
convert_srt_to_txt() {
local srt_file="$1" txt_file="${1%.srt}.txt"
sed '/^[0-9][0-9]:[0-9][0-9]/d; /^$/d; s/<[^>]*>//g' "$srt_file" > "$txt_file"
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
--lang)
[ $# -ge 2 ] || err "--lang needs a value"
LANG_SPEC="$2"; shift 2 ;;
--format)
[ $# -ge 2 ] || err "--format needs a value"
case "$2" in
srt|vtt|txt) FMT="$2" ;;
*) err "unknown format '$2' (use srt, vtt, or txt)" ;;
esac
shift 2 ;;
--auto-only) AUTO_ONLY=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--list-subs) LIST_SUBS=1; shift ;;
--no-playlist) NO_PLAYLIST=1; shift ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || err "missing URL (see --help)"
yt_validate_url "$URL" 2>/dev/null || err "not a valid URL: $URL (must start with http:// or https://)"
# ── Build the yt-dlp command ─────────────────────────────────
if [ "$LIST_SUBS" -eq 1 ]; then
args=(yt-dlp --list-subs "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
yt_echo_cmd "${args[@]:1}"
exit 0
fi
spawn "listing subtitles" "${args[@]}"
exit 0
fi
# Output template
OUT_TEMPLATE=""
if [ -n "$OUT_DIR" ]; then
OUT_TEMPLATE="$OUT_DIR/%(title)s.%(sub_lang)s.%(ext)s"
else
OUT_TEMPLATE="./%(title)s.%(sub_lang)s.%(ext)s"
fi
args=(yt-dlp)
if [ "$AUTO_ONLY" -eq 1 ]; then
args+=(--write-auto-subs)
else
args+=(--write-subs --write-auto-subs)
fi
if [ -n "$LANG_SPEC" ]; then
args+=(--sub-langs "$LANG_SPEC")
else
args+=(--sub-langs best)
fi
case "$FMT" in
vtt) args+=(--sub-format vtt) ;;
txt) args+=(--sub-format srt) ;;
*) args+=(--sub-format srt) ;;
esac
[ "$NO_PLAYLIST" -eq 1 ] && args+=(--no-playlist)
args+=(-o "$OUT_TEMPLATE" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
yt_echo_cmd "${args[@]:1}"
exit 0
fi
# ── Execute ───────────────────────────────────────────────────
set +e
out="$(mktemp)"
err_tmp="$(mktemp)"
"${args[@]}" >"$out" 2>"$err_tmp"
rc=$?
set -e
if [ "$rc" -ne 0 ]; then
# Unavailable-subtitle detection: exit!=0 AND stderr matches no-subs patterns
if grep -qiE 'no subtitles|no captions|unable to extract|subtitles not available' "$err_tmp"; then
rm -f "$out" "$err_tmp"
err "unavailable subtitles for this video (try --list-subs to check)"
fi
# Re-surface the real yt-dlp error otherwise
sed 's/^/ /' "$err_tmp" >&2
rm -f "$out" "$err_tmp"
exit "$rc"
fi
rm -f "$out" "$err_tmp"
# ── Post-conversion for txt format ────────────────────────────
if [ "$FMT" = "txt" ]; then
search_dir="."
[ -n "$OUT_DIR" ] && search_dir="$OUT_DIR"
while IFS= read -r -d '' f; do
case "$f" in
*.srt) convert_srt_to_txt "$f" ;;
esac
done < <(find "$search_dir" -maxdepth 1 -name '*.srt' -print0 2>/dev/null || true)
fi
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-ytsync — Incrementally sync YouTube channels/playlists into ~/Videos
# -h|--help forwarded via exec below
exec pos media ytsync "$@"
+5 -3
View File
@@ -11,10 +11,11 @@ _pos_flags[communication-telegram-sender]="--type --caption --parse-mode --no-pr
_pos_flags[docker-stack]="-a --all" _pos_flags[docker-stack]="-a --all"
_pos_flags[docker-vbox]="--dir --gpu --device --port --cpus --memory --network" _pos_flags[docker-vbox]="--dir --gpu --device --port --cpus --memory --network"
_pos_flags[entertainment-send]="--print --markdown" _pos_flags[entertainment-send]="--print --markdown"
_pos_flags[media-grab]="--audio --video --best --worst --output --no-playlist --cookies --dry-run"
_pos_flags[media-mp3]="--output --no-playlist --cookies --by-artist --dry-run"
_pos_flags[media-mp4]="--format --best --worst --output --no-playlist --cookies --dry-run"
_pos_flags[media-sync]="--mp3 --mp4 --source --dry-run" _pos_flags[media-sync]="--mp3 --mp4 --source --dry-run"
_pos_flags[media-yt-grab]="--audio --video --best --worst --output --no-playlist --cookies --dry-run"
_pos_flags[media-yt-mp3]="--output --no-playlist --cookies --by-artist --dry-run"
_pos_flags[media-yt-mp4]="--format --best --worst --output --no-playlist --cookies --dry-run"
_pos_flags[media-yt-subtitles]="--lang --format --auto-only --output --list-subs --no-playlist --dry-run"
_pos_flags[media-ytsync]="--dry-run" _pos_flags[media-ytsync]="--dry-run"
_pos_flags[network-checkport]="--tcp --udp --ping --no-banner --versions --timeout" _pos_flags[network-checkport]="--tcp --udp --ping --no-banner --versions --timeout"
_pos_flags[network-download]="--dir --out --split --seed --force --upload --gid --tmux" _pos_flags[network-download]="--dir --out --split --seed --force --upload --gid --tmux"
@@ -42,6 +43,7 @@ _pos_subcmds[docker-compose]="ls installed up down restart logs update config me
_pos_subcmds[docker-vbox]="create enter stop start rm ls menu" _pos_subcmds[docker-vbox]="create enter stop start rm ls menu"
_pos_subcmds[media-sync]="menu" _pos_subcmds[media-sync]="menu"
_pos_subcmds[media-ytsync]="add sync list remove" _pos_subcmds[media-ytsync]="add sync list remove"
_pos_subcmds[media-yt]="mp3 mp4 grab ytsync subtitles"
_pos_subcmds[network-download]="start stop status add torrent metalink list info files peers pause resume remove purge move limit set watch restart retry replace menu" _pos_subcmds[network-download]="start stop status add torrent metalink list info files peers pause resume remove purge move limit set watch restart retry replace menu"
_pos_subcmds[share-nfs-client]="mount unmount list persist unpersist menu" _pos_subcmds[share-nfs-client]="mount unmount list persist unpersist menu"
_pos_subcmds[share-nfs-server]="status share unshare list reload enable disable menu" _pos_subcmds[share-nfs-server]="status share unshare list reload enable disable menu"
+1
View File
@@ -8,6 +8,7 @@
# API keys (each provider needs its own): # API keys (each provider needs its own):
# AI_GEMINI_API_KEY=<key> # Gemini API key from aistudio.google.com # AI_GEMINI_API_KEY=<key> # Gemini API key from aistudio.google.com
# OPENROUTER_API_KEY=<key> # OpenRouter API key from openrouter.ai # OPENROUTER_API_KEY=<key> # OpenRouter API key from openrouter.ai
# Legacy fallback: AI_API_KEY=<key> is honored when the provider's own key is unset
# #
# Model overrides (optional, defaults per provider): # Model overrides (optional, defaults per provider):
# AI_GEMINI_MODEL=<model> # Gemini model id (default: gemini-2.5-flash) # AI_GEMINI_MODEL=<model> # Gemini model id (default: gemini-2.5-flash)
+40
View File
@@ -14,11 +14,26 @@ fi
source "$(dirname "$0")/lib/common.sh" source "$(dirname "$0")/lib/common.sh"
source "$(dirname "$0")/lib/flags.sh" source "$(dirname "$0")/lib/flags.sh"
# Return the current install.sh version (e.g. "0.0c174").
# Derived from git commit count. Empty when .git is absent.
# Override with INSTALL_VERSION_OVERRIDE for testing.
install_version() {
if [ "${INSTALL_VERSION_OVERRIDE+x}" ]; then
printf '%s' "$INSTALL_VERSION_OVERRIDE"
return
fi
local repo_dir count
repo_dir="$(cd "$(dirname "$0")" && pwd)"
count="$(git -C "$repo_dir" rev-list --count HEAD 2>/dev/null)" || { printf ''; return; }
printf '0.0c%s' "$count"
}
# Exported so child phases (preinstall.sh, postinstall.sh) inherit it — # Exported so child phases (preinstall.sh, postinstall.sh) inherit it —
# otherwise '--dry-run' silently executes them for real. # otherwise '--dry-run' silently executes them for real.
export DRY_RUN=0 export DRY_RUN=0
RUN_APPS=0 RUN_APPS=0
RUN_FEATURES=0 RUN_FEATURES=0
FORCE=0
SKIP_PHASES="" SKIP_PHASES=""
STEPS_SPEC="" STEPS_SPEC=""
@@ -58,6 +73,7 @@ Options:
--full Core install + all optional apps (non-interactive) --full Core install + all optional apps (non-interactive)
--feature Install features/ scripts (prompts before overwriting) --feature Install features/ scripts (prompts before overwriting)
--dry-run Show what would be done without executing --dry-run Show what would be done without executing
--force Re-install even if the version matches
--skip <phase> Skip a phase (repeatable): --skip <phase> Skip a phase (repeatable):
preinstall, scripts, postinstall, scalepoint, apps preinstall, scripts, postinstall, scalepoint, apps
--steps <spec> Run only specific phases. Format: 1,3,4 or 1-3 --steps <spec> Run only specific phases. Format: 1,3,4 or 1-3
@@ -79,6 +95,7 @@ while [[ $# -gt 0 ]]; do
--full) RUN_APPS=2; shift ;; --full) RUN_APPS=2; shift ;;
--feature) RUN_FEATURES=1; shift ;; --feature) RUN_FEATURES=1; shift ;;
--dry-run) DRY_RUN=1; shift ;; --dry-run) DRY_RUN=1; shift ;;
--force) FORCE=1; shift ;;
--skip) --skip)
[ -z "${2:-}" ] && err "Missing value for --skip" [ -z "${2:-}" ] && err "Missing value for --skip"
SKIP_PHASES="${SKIP_PHASES:+$SKIP_PHASES,}$2" SKIP_PHASES="${SKIP_PHASES:+$SKIP_PHASES,}$2"
@@ -95,6 +112,24 @@ while [[ $# -gt 0 ]]; do
esac esac
done done
# ── Version gate ────────────────────────────────────────────────
CURRENT_VERSION="$(install_version)"
if [ "${FORCE:-0}" -ne 1 ] && [ -n "$CURRENT_VERSION" ]; then
INSTALLED_VERSION="$(flag_value installed_version 2>/dev/null)" || true
if [ -n "$INSTALLED_VERSION" ]; then
current_num="${CURRENT_VERSION#0.0c}"
installed_num="${INSTALLED_VERSION#0.0c}"
if [ "$current_num" -eq "$installed_num" ] 2>/dev/null; then
if [ "${DRY_RUN:-0}" -eq 1 ]; then
log "(dry-run) Would skip install: already at version $CURRENT_VERSION"
else
log "Already installed ($CURRENT_VERSION). Use --force to re-install."
fi
exit 0
fi
fi
fi
# ── Phase runner ──────────────────────────────────────────────── # ── Phase runner ────────────────────────────────────────────────
# Phase names → numbers: preinstall=1 scripts=2 postinstall=3 scalepoint=4 # Phase names → numbers: preinstall=1 scripts=2 postinstall=3 scalepoint=4
should_run() { should_run() {
@@ -248,6 +283,11 @@ echo "${GREEN}══════════════════════
echo "${GREEN} Bootstrap complete ($(timer_stop))${RESET}" echo "${GREEN} Bootstrap complete ($(timer_stop))${RESET}"
echo "${GREEN}════════════════════════════════════════════${RESET}" echo "${GREEN}════════════════════════════════════════════${RESET}"
# ── Record installed version ────────────────────────────────────
if [ "${DRY_RUN:-0}" -ne 1 ] && [ -n "${CURRENT_VERSION:-}" ]; then
flag_set installed_version "$CURRENT_VERSION"
fi
# ── Optional apps ────────────────────────────────────────────── # ── Optional apps ──────────────────────────────────────────────
if [ "$RUN_APPS" -eq 1 ]; then if [ "$RUN_APPS" -eq 1 ]; then
echo echo
+50
View File
@@ -0,0 +1,50 @@
# lib/yt-lib.sh — shared helpers for yt-dlp-based media tools.
# Sourced opt-in by yt-* subcommand files. Uses err() from lib/common.sh.
# yt_check_deps [dry_run] [ffmpeg_purpose]
# Check yt-dlp (+ ffmpeg unless dry_run=1 and caller wants dry-run to skip deps).
# ffmpeg_purpose is the parenthetical in the error message (default: "MP3 conversion").
# ytsync overrides this deliberately (needs yt-dlp+jq even in dry-run).
yt_check_deps() {
local dry="${1:-0}" purpose="${2:-MP3 conversion}"
command -v yt-dlp &>/dev/null || err "yt-dlp not found — install it with: sudo apt install yt-dlp"
if [ "$dry" -eq 0 ]; then
command -v ffmpeg &>/dev/null || err "ffmpeg not found (needed for $purpose) — install it with: sudo apt install ffmpeg"
fi
}
# yt_validate_url <url>
# Returns 1 (does NOT exit) when URL doesn't start with http:// or https://;
# prints the diagnostic to stderr. Callers surface the error themselves (e.g.
# `yt_validate_url "$url" 2>/dev/null || err "…"` with their own prefix) so a
# helper whose only job is to check can never kill the caller's error path.
yt_validate_url() {
local url="$1"
case "$url" in
http://*|https://*) return 0 ;;
*) echo "not a valid URL: $url (must start with http:// or https://)" >&2; return 1 ;;
esac
}
# yt_echo_cmd <args...>
# Prints the yt-dlp command for --dry-run mode
yt_echo_cmd() {
echo "yt-dlp $*"
}
# classify_url <url>
# Domain classification: music.youtube/soundcloud/bandcamp → audio;
# youtube/youtu.be/vimeo/twitch → video; unknown → ${GRAB_DEFAULT:-video}
# Migrated from bin/pos-media-grab
classify_url() {
local url="$1" mode="${GRAB_DEFAULT:-video}"
case "$url" in
*music.youtube.com*) echo "audio" ;;
*soundcloud.com*) echo "audio" ;;
*bandcamp.com*) echo "audio" ;;
*youtube.com*|*youtu.be*) echo "video" ;;
*vimeo.com*) echo "video" ;;
*twitch.tv*) echo "video" ;;
*) echo "$mode" ;;
esac
}
+5
View File
@@ -42,6 +42,7 @@ silently.
| --- | --- | | --- | --- |
| `t-ai-server-flags.sh` | `pos-ai-server` ExecStart flag set: defaults, CLI, config, dedupe, one-token-per-flag | | `t-ai-server-flags.sh` | `pos-ai-server` ExecStart flag set: defaults, CLI, config, dedupe, one-token-per-flag |
| `t-ai-hf-download.sh` | `pos ai hf download` success + failure honesty (no `.hf-meta` on partial failure, rc != 0) | | `t-ai-hf-download.sh` | `pos ai hf download` success + failure honesty (no `.hf-meta` on partial failure, rc != 0) |
| `t-ai-key-resolution.sh` | `pos ai` API-key contract: provider key > legacy `AI_API_KEY` fallback > error; leakage guard, env-wins, providers status sync |
| `t-ai-llama-detect.sh` | `pos ai-server status` version detection, "unknown", missing-binary failure | | `t-ai-llama-detect.sh` | `pos ai-server status` version detection, "unknown", missing-binary failure |
| `t-unsupported-flags.sh` | unsupported-option handling: CLI/config/env hard errors, dropped defaults, word-boundary match | | `t-unsupported-flags.sh` | unsupported-option handling: CLI/config/env hard errors, dropped defaults, word-boundary match |
| `t-systemd-unit.sh` | generated unit: one ExecStart, quoted paths, `systemd-analyze verify` | | `t-systemd-unit.sh` | generated unit: one ExecStart, quoted paths, `systemd-analyze verify` |
@@ -52,3 +53,7 @@ silently.
| `t-uninstall-manifest.sh` | install.sh ↔ POS_LIBS symmetry, user-unit discovery, marker-driven plugin removal | | `t-uninstall-manifest.sh` | install.sh ↔ POS_LIBS symmetry, user-unit discovery, marker-driven plugin removal |
| `t-gen-docs-drift.sh` | `make gen` idempotence on a pristine tracked tree (CI drift gate) | | `t-gen-docs-drift.sh` | `make gen` idempotence on a pristine tracked tree (CI drift gate) |
| `t-lint-gate.sh` | `make lint` green on the real tree; planted violations are caught and named | | `t-lint-gate.sh` | `make lint` green on the real tree; planted violations are caught and named |
| `t-install-version.sh` | install.sh version gate: match→skip, mismatch→proceed, --force bypass, dry-run variant, flag write, numeric comparison |
| `t-share-mountpoint.sh` | share-client `ask_mountpoint` UX: existing/new/declined/rejected paths, confirm gate, mkdir side effects, non-TTY stdin contract, static `n``t` guards |
| `t-pos-media-yt.sh` | unified `pos media yt` suite: dispatcher + forwarder resolution, shared yt-lib helpers, yt-mp3/mp4/grab/subtitles flags, dry-run deps, `YT_OUT_DIR` seam, `GRAB_DEFAULT` config, negative controls (unsafe-URL no-expansion, `--lang en,ar` single arg, txt timestamp-stripping) |
| `t-telegram-listener-singleton.sh` | Telegram listener single-instance guard: first `--run` acquires the flock, second `--run` fails fast with the exact message, lock auto-releases so the next start is clean, `--status` reports the lock state |
+128
View File
@@ -0,0 +1,128 @@
#!/usr/bin/env bash
set -euo pipefail
# t-ai-key-resolution.sh — `pos ai` API-key resolution contract:
# provider-specific key (AI_GEMINI_API_KEY / OPENROUTER_API_KEY)
# > legacy shared AI_API_KEY fallback > error; llamacpp needs no key.
#
# Regression for the 7ae2e77 "per-provider keys" migration: AI_API_KEY must be
# honored ONLY when the active provider's own key is empty (backward-compat for
# howto-taught single-key configs) and must NEVER override a provider key
# (cross-provider leakage guard). Also pins env-wins (exported env beats
# ai.env) and `pos ai providers` agreeing with resolve_key().
#
# Seam: `pos ai models` with a curl stub on PATH that records -H headers to a
# log and returns deterministic per-provider JSON (no sessions are written by
# `models`, unlike ask/chat).
run_test() {
require_cmd jq "pos ai key resolution" || return 0
local sandbox cfg home stubs curl_log
sandbox="$(mksandbox ai-key-resolution)"
cfg="$sandbox/cfg"
home="$sandbox/home"
stubs="$sandbox/stubs"
mkdir -p "$cfg" "$home" "$stubs"
curl_log="$sandbox/curl.log"
: > "$curl_log"
: > "$cfg/ai.env"
# curl stub: log every -H <header> pair; reply with provider-shaped JSON +
# the trailing \n%{http_code} line the adapters' --write-out expects.
cat > "$stubs/curl" <<STUB
#!/usr/bin/env bash
all="\$*"
while [ \$# -gt 0 ]; do
case "\$1" in
-H) printf 'HDR %s\n' "\$2" >> "$curl_log"; shift 2 ;;
*) shift ;;
esac
done
case "\$all" in
*generativelanguage.googleapis.com*)
printf '%s\n200' '{"models":[{"name":"models/gemini-2.5-flash","supportedGenerationMethods":["generateContent"]}]}' ;;
*openrouter.ai*)
printf '%s\n200' '{"data":[{"id":"openrouter/auto"}]}' ;;
*)
printf '%s\n200' '{"data":[{"id":"loaded-model"}]}' ;;
esac
STUB
chmod +x "$stubs/curl"
local posai="$ROOT/bin/pos-ai"
# Hermetic env: `env -i` drops every inherited variable (incl. any ambient
# AI_*), so each case exercises exactly the values declared below.
local path_env=(-i PATH="$stubs:/usr/bin:/bin" CONFIG_FILE="$cfg/ai.env" HOME="$home")
# ── Case 1: gemini via provider key only ──
printf 'AI_GEMINI_API_KEY=GEMFILEKEY\n' > "$cfg/ai.env"
test_run_env "${path_env[@]}" -- "$posai" models
check_rc "C1 gemini via AI_GEMINI_API_KEY works" 0 "$TR_RC"
check_contains "C1 gemini header carries provider key" "x-goog-api-key: GEMFILEKEY" "$(cat "$curl_log")"
# ── Case 2: gemini via legacy AI_API_KEY only (core regression) ──
: > "$curl_log"
printf 'AI_API_KEY=AIKEYONLY\n' > "$cfg/ai.env"
test_run_env "${path_env[@]}" -- "$posai" models
check_rc "C2 gemini via legacy AI_API_KEY works" 0 "$TR_RC"
check_contains "C2 gemini header carries legacy key" "x-goog-api-key: AIKEYONLY" "$(cat "$curl_log")"
# ── Case 3: gemini with both → provider key wins (leakage guard) ──
: > "$curl_log"
printf 'AI_GEMINI_API_KEY=GEMWIN\nAI_API_KEY=AIKEYLOSE\n' > "$cfg/ai.env"
test_run_env "${path_env[@]}" -- "$posai" models
check_rc "C3 both set still works" 0 "$TR_RC"
check_contains "C3 provider key wins" "x-goog-api-key: GEMWIN" "$(cat "$curl_log")"
check_not_contains "C3 shared key not leaked" "AIKEYLOSE" "$(cat "$curl_log")"
# ── Case 4: openrouter via provider key only ──
: > "$curl_log"
printf 'OPENROUTER_API_KEY=ORFILEKEY\n' > "$cfg/ai.env"
test_run_env "${path_env[@]}" -- "$posai" --provider openrouter models
check_rc "C4 openrouter via OPENROUTER_API_KEY works" 0 "$TR_RC"
check_contains "C4 openrouter auth carries provider key" "Authorization: Bearer ORFILEKEY" "$(cat "$curl_log")"
# ── Case 5: openrouter via legacy AI_API_KEY only ──
: > "$curl_log"
printf 'AI_API_KEY=AIKEYONLY\n' > "$cfg/ai.env"
test_run_env "${path_env[@]}" -- "$posai" --provider openrouter models
check_rc "C5 openrouter via legacy AI_API_KEY works" 0 "$TR_RC"
check_contains "C5 openrouter auth carries legacy key" "Authorization: Bearer AIKEYONLY" "$(cat "$curl_log")"
# ── Case 6: openrouter with both → provider key wins ──
: > "$curl_log"
printf 'OPENROUTER_API_KEY=ORWIN\nAI_API_KEY=AIKEYLOSE\n' > "$cfg/ai.env"
test_run_env "${path_env[@]}" -- "$posai" --provider openrouter models
check_rc "C6 both set still works" 0 "$TR_RC"
check_contains "C6 openrouter provider key wins" "Authorization: Bearer ORWIN" "$(cat "$curl_log")"
check_not_contains "C6 shared key not leaked" "AIKEYLOSE" "$(cat "$curl_log")"
# ── Case 7: env-wins — exported provider key beats ai.env file value ──
: > "$curl_log"
printf 'AI_GEMINI_API_KEY=FILEVALUE\n' > "$cfg/ai.env"
test_run_env "${path_env[@]}" AI_GEMINI_API_KEY=ENVVALUE -- "$posai" models
check_rc "C7 env-wins run works" 0 "$TR_RC"
check_contains "C7 exported env key used" "x-goog-api-key: ENVVALUE" "$(cat "$curl_log")"
check_not_contains "C7 file value suppressed by env" "FILEVALUE" "$(cat "$curl_log")"
# ── Case 8: llamacpp needs no key ──
: > "$curl_log"
: > "$cfg/ai.env"
test_run_env "${path_env[@]}" -- "$posai" --provider llamacpp models
check_rc "C8 llamacpp works with no key" 0 "$TR_RC"
check_contains "C8 llamacpp lists models" "Local llama.cpp models:" "$TR_OUT"
# ── Case 9: missing key → rc 1 + byte-stable message ──
test_run_env "${path_env[@]}" -- "$posai" models
check_rc "C9 missing key exits nonzero" 1 "$TR_RC"
check_contains "C9 stable error message" "No Gemini API key — run 'pos config ai' and set AI_GEMINI_API_KEY" "$TR_OUT"
# ── Case 10: providers status agrees with resolve_key (AI_API_KEY only) ──
local prov
printf 'AI_API_KEY=SHAREDKEY\n' > "$cfg/ai.env"
test_run_env "${path_env[@]}" -- "$posai" providers
prov="$(printf '%s\n' "$TR_OUT" | grep configured || true)"
check_contains "C10 gemini row shows configured" "gemini" "$prov"
check_contains "C10 openrouter row shows configured" "openrouter" "$prov"
check_not_contains "C10 no 'not configured' row" "not configured" "$TR_OUT"
}
+1 -1
View File
@@ -201,7 +201,7 @@ STUB
for tool in pos-ai pos-ai-hf pos-ai-server pos-communication-matrix-listener \ for tool in pos-ai pos-ai-hf pos-ai-server pos-communication-matrix-listener \
pos-communication-matrix-sender pos-communication-scrcpy \ pos-communication-matrix-sender pos-communication-scrcpy \
pos-communication-telegram-listener pos-communication-telegram-sender \ pos-communication-telegram-listener pos-communication-telegram-sender \
pos-media-grab pos-network-download; do pos-media-yt-grab pos-network-download; do
if grep -q "load_env_file" "$ROOT/bin/$tool"; then if grep -q "load_env_file" "$ROOT/bin/$tool"; then
printf ' PASS %s uses shared load_env_file\n' "$tool" printf ' PASS %s uses shared load_env_file\n' "$tool"
else else
+174
View File
@@ -0,0 +1,174 @@
#!/usr/bin/env bash
set -euo pipefail
# t-install-version.sh — install.sh version gate contract:
# match→skip with "Already installed", mismatch→proceed, --force bypass,
# no-git (empty) skip, dry-run variant, flag write after run,
# numeric comparison correctness (0.0c10 > 0.0c9).
#
# Seam: INSTALL_VERSION_OVERRIDE env var (presence check → empty = no-git path),
# FLAGS_DIR env var (existing flag store seam).
#
# Proceed-path cases (2/3/4/5/8) use --skip all phases instead of --dry-run
# because postinstall.sh is not fully dry-run safe (cat on non-existent file
# after dry-run copy). Skipping all phases is functionally equivalent — the
# gate fires before phases, and the banner + flag write still execute.
run_test() {
local sandbox stubs flags_dir install_sh
sandbox="$(mksandbox install-version)"
stubs="$sandbox/stubs"
flags_dir="$sandbox/flags"
mkdir -p "$stubs" "$flags_dir"
install_sh="$ROOT/install.sh"
# sudo stub: just exec its arguments (bypasses real sudo in sandbox)
cat > "$stubs/sudo" <<'STUB'
#!/usr/bin/env bash
exec "$@"
STUB
chmod +x "$stubs/sudo"
# install stub: delegate to the real install binary (needed so -d/-m flags
# work; we only shadow 'sudo' to avoid needing a tty/password)
cat > "$stubs/install" <<'STUB'
#!/usr/bin/env bash
exec /usr/bin/install "$@"
STUB
chmod +x "$stubs/install"
# mkdir stub: just exec its arguments
cat > "$stubs/mkdir" <<'STUB'
#!/usr/bin/env bash
exec "$@"
STUB
chmod +x "$stubs/mkdir"
local base_env=(PATH="$stubs:/usr/bin:/bin"
HOME="$sandbox/home"
FLAGS_DIR="$flags_dir")
# Helper: run install.sh with env overrides and capture output+rc.
run_install() {
local envs=() args=()
while [ $# -gt 0 ]; do
if [ "$1" = "--" ]; then shift; break; fi
envs+=("$1"); shift
done
args=("$@")
set +e
TR_OUT="$(env "${base_env[@]}" "${envs[@]}" bash "$install_sh" "${args[@]}" 2>&1)"
TR_RC=$?
set -e
}
# Helper: set a flag value directly (for pre-seeding installed_version)
set_flag() {
local name="$1" value="$2"
mkdir -p "$flags_dir"
printf '%s' "$value" > "$flags_dir/$name"
}
# Helper: read a flag value
get_flag() {
local name="$1"
local f="$flags_dir/$name"
[ -f "$f" ] && cat "$f" || printf ''
}
# ═══════════════════════════════════════════════════════════════
# Case 1: Match → skip ("Already installed")
# Gate exits BEFORE any phase — safe without --dry-run or --skip.
# ═══════════════════════════════════════════════════════════════
set_flag "installed_version" "0.0c10"
run_install INSTALL_VERSION_OVERRIDE=0.0c10
check_rc "C1 match→skip exit code" 0 "$TR_RC"
check_contains "C1 match→skip message" "Already installed (0.0c10)" "$TR_OUT"
# ═══════════════════════════════════════════════════════════════
# Case 2: Mismatch → proceed (gate doesn't match, phases skipped)
# ═══════════════════════════════════════════════════════════════
rm -f "$flags_dir/installed_version"
set_flag "installed_version" "0.0c10"
run_install INSTALL_VERSION_OVERRIDE=0.0c11 -- --skip preinstall,scripts,postinstall,scalepoint
check_rc "C2 mismatch→proceed exit code" 0 "$TR_RC"
check_not_contains "C2 mismatch does not skip" "Already installed" "$TR_OUT"
# ═══════════════════════════════════════════════════════════════
# Case 3: Force bypasses match
# ═══════════════════════════════════════════════════════════════
rm -f "$flags_dir/installed_version"
set_flag "installed_version" "0.0c10"
run_install INSTALL_VERSION_OVERRIDE=0.0c10 -- --skip preinstall,scripts,postinstall,scalepoint --force
check_rc "C3 force bypasses exit code" 0 "$TR_RC"
check_not_contains "C3 force bypasses skip" "Already installed" "$TR_OUT"
# ═══════════════════════════════════════════════════════════════
# Case 4: No git (empty version via presence check)
# INSTALL_VERSION_OVERRIDE="" with ${var+x} → empty → gate skipped
# ═══════════════════════════════════════════════════════════════
rm -f "$flags_dir/installed_version"
set_flag "installed_version" "0.0c10"
run_install INSTALL_VERSION_OVERRIDE="" -- --skip preinstall,scripts,postinstall,scalepoint
check_rc "C4 empty override→proceeds exit code" 0 "$TR_RC"
check_not_contains "C4 empty override does not skip" "Already installed" "$TR_OUT"
# ═══════════════════════════════════════════════════════════════
# Case 5: No installed flag → gate doesn't block
# ═══════════════════════════════════════════════════════════════
rm -f "$flags_dir/installed_version"
run_install INSTALL_VERSION_OVERRIDE=0.0c10 -- --skip preinstall,scripts,postinstall,scalepoint
check_rc "C5 no flag→proceeds exit code" 0 "$TR_RC"
check_not_contains "C5 no flag does not skip" "Already installed" "$TR_OUT"
# ═══════════════════════════════════════════════════════════════
# Case 6: Dry-run + match → "Would skip" (not "Already installed")
# ═══════════════════════════════════════════════════════════════
rm -f "$flags_dir/installed_version"
set_flag "installed_version" "0.0c10"
run_install INSTALL_VERSION_OVERRIDE=0.0c10 -- --dry-run
check_rc "C6 dry-run match exit code" 0 "$TR_RC"
check_contains "C6 dry-run shows Would skip" "Would skip install" "$TR_OUT"
check_not_contains "C6 dry-run not Already installed" "Already installed" "$TR_OUT"
# ═══════════════════════════════════════════════════════════════
# Case 7: Flag written after successful run
# All phases skipped; sudo/install/mkdir stubs on PATH so
# flag_set's `run sudo install` actually creates the flag file.
# ═══════════════════════════════════════════════════════════════
rm -f "$flags_dir/installed_version"
set_flag "installed_version" "0.0c10"
run_install INSTALL_VERSION_OVERRIDE=0.0c11 -- --skip preinstall,scripts,postinstall,scalepoint
check_rc "C7 all-skip exit code" 0 "$TR_RC"
local flag_val
flag_val="$(get_flag installed_version)"
check_eq "C7 flag written with current version" "0.0c11" "$flag_val"
# ═══════════════════════════════════════════════════════════════
# Case 8: Numeric comparison — 0.0c10 > 0.0c9 (not blocked)
# ═══════════════════════════════════════════════════════════════
rm -f "$flags_dir/installed_version"
set_flag "installed_version" "0.0c9"
run_install INSTALL_VERSION_OVERRIDE=0.0c10 -- --skip preinstall,scripts,postinstall,scalepoint
check_rc "C8 numeric 10>9 proceeds exit code" 0 "$TR_RC"
check_not_contains "C8 numeric 10>9 does not skip" "Already installed" "$TR_OUT"
# Also test: 0.0c9 with installed 0.0c10 → NOT a match
rm -f "$flags_dir/installed_version"
set_flag "installed_version" "0.0c10"
run_install INSTALL_VERSION_OVERRIDE=0.0c9 -- --skip preinstall,scripts,postinstall,scalepoint
check_rc "C8 numeric 9≠10 proceeds exit code" 0 "$TR_RC"
check_not_contains "C8 numeric 9≠10 does not skip" "Already installed" "$TR_OUT"
# ═══════════════════════════════════════════════════════════════
# Case 9: Force + match → still writes flag
# ═══════════════════════════════════════════════════════════════
rm -f "$flags_dir/installed_version"
set_flag "installed_version" "0.0c10"
run_install INSTALL_VERSION_OVERRIDE=0.0c10 -- --skip preinstall,scripts,postinstall,scalepoint --force
check_rc "C9 force+match exit code" 0 "$TR_RC"
local flag_val9
flag_val9="$(get_flag installed_version)"
check_eq "C9 force writes flag" "0.0c10" "$flag_val9"
}
+279
View File
@@ -0,0 +1,279 @@
#!/usr/bin/env bash
set -euo pipefail
# t-pos-media-yt.sh — POS--9 verification for the unified `pos media yt`
# suite and its forwarders.
#
# Part 1: dispatcher + forwarder resolution (bin/pos-media-yt, pos-media-mp3/
# mp4/grab forwarders, full chain through bin/pos)
# Part 2: lib/yt-lib.sh shared helpers (deps, url validation, classify)
# Part 3: yt-mp3 behaviors (flags, dry-run, YT_OUT_DIR seam, unsafe-URL guard)
# Part 4: yt-mp4 behaviors (format select flags, mutual exclusions, seam)
# Part 5: yt-grab classification + delegation (dry-run command strings) + config
# Part 6: yt-subtitles (defaults, --lang en,ar single-arg, formats),
# txt conversion (timestamps stripped), unavailable-subs detection
# Part 7: negative controls mandated by the Architect contract
run_test() {
local sandbox stubs cfg
sandbox="$(mksandbox media-yt)"
stubs="$sandbox/stubs"
cfg="$sandbox/cfg"
mkdir -p "$stubs" "$cfg"
local ytdlp_log="$sandbox/ytdlp.log"
: > "$ytdlp_log"
# fake yt-dlp logs its exact argv (one per line) — lets us assert arg
# counting / no-expansion WITHOUT any network or real binary.
cat > "$stubs/yt-dlp" <<STUB
#!/usr/bin/env bash
printf 'yt-dlp %s\\n' "\$*" >> "$ytdlp_log"
STUB
# ffmpeg stub: satisfies the non-dry-run deps guard; not actually used.
printf '#!/usr/bin/env bash\nexit 0\n' > "$stubs/ffmpeg"
chmod +x "$stubs/yt-dlp" "$stubs/ffmpeg"
local RB="$ROOT/bin"
local yt ytd="$RB/pos-media-yt" mp3="$RB/pos-media-yt-mp3" \
mp4="$RB/pos-media-yt-mp4" grab="$RB/pos-media-yt-grab" \
subs="$RB/pos-media-yt-subtitles"
local env_base=(PATH="$stubs:$RB:/usr/bin:/bin" CONFIG_DIR="$cfg" \
YT_OUT_DIR="$sandbox/out")
# ═══ Part 1: dispatcher + forwarders ═══
# 1.0 bare dispatcher → usage listing subcommands (exit 0)
test_run_env "${env_base[@]}" -- "$ytd"
check_rc "dispatcher bare invocation = usage exit 0" 0 "$TR_RC"
check_contains "dispatcher usage lists mp3" "mp3" "$TR_OUT"
check_contains "dispatcher usage lists subtitles" "subtitles" "$TR_OUT"
# 1.1 dispatcher unknown subcommand → hard error, nonzero
test_run_env "${env_base[@]}" -- "$ytd" bogus
[ "$TR_RC" -ne 0 ] && printf ' PASS dispatcher unknown subcommand exits nonzero\n' \
|| printf ' FAIL dispatcher unknown subcommand exited 0\n'
check_contains "dispatcher unknown subcommand message" "unknown yt command" "$TR_OUT"
# 1.2 forwarder helpers: --help passes through to the real yt usage
test_run_env "${env_base[@]}" -- "$RB/pos-media-mp3" --help
check_rc "pos media mp3 --help forwards cleanly" 0 "$TR_RC"
check_contains "mp3 forwarder shows yt-mp3 usage" "pos media yt mp3 [options]" "$TR_OUT"
test_run_env "${env_base[@]}" -- "$RB/pos-media-mp4" --help
check_rc "pos media mp4 --help forwards cleanly" 0 "$TR_RC"
check_contains "mp4 forwarder shows yt-mp4 usage" "pos media yt mp4 [options]" "$TR_OUT"
test_run_env "${env_base[@]}" -- "$RB/pos-media-grab" --help
check_rc "pos media grab --help forwards cleanly" 0 "$TR_RC"
check_contains "grab forwarder shows yt-grab usage" "pos media yt grab [options]" "$TR_OUT"
# 1.3 ytsync forwarder --help forwards to pos media ytsync
test_run_env "${env_base[@]}" -- "$RB/pos-media-yt-ytsync" --help
check_contains "ytsync forwarder references media ytsync" "media ytsync" "$TR_OUT"
# 1.4 full dispatch chain: `pos media mp3 <url>` → yt-mp3 (fake yt-dlp called).
# `media-yt-mp3` is non-interactive so bin/pos execs it without a tee.
: > "$ytdlp_log"
test_run_env "${env_base[@]}" -- "$RB/pos" media mp3 "http://example.com/x"
check_rc "chain pos media mp3 <url> resolves" 0 "$TR_RC"
check_contains "chain reaches yt-dlp with -x (audio path)" "-x" "$(cat "$ytdlp_log")"
# ═══ Part 2: lib/yt-lib.sh helpers ═══
source "$ROOT/lib/common.sh"
source "$ROOT/lib/yt-lib.sh"
# 2.0 yt_validate_url accepts http(s), rejects junk
yt_validate_url "https://youtube.com/watch?v=x" && printf ' PASS url valid http/https accepted\n' \
|| printf ' FAIL valid https url rejected\n'
test_run yt_validate_url "ftp://bad"
[ "$TR_RC" -ne 0 ] && printf ' PASS invalid url rejected\n' || printf ' FAIL invalid url accepted\n'
check_contains "invalid url error message" "not a valid URL" "$TR_OUT"
# 2.1 classify_url: audio / video / unknown-with-GRAB_DEFAULT
check_eq "music.youtube → audio" "audio" "$(classify_url "https://music.youtube.com/watch?v=x")"
check_eq "soundcloud → audio" "audio" "$(classify_url "https://soundcloud.com/a/b")"
check_eq "youtube.com → video" "video" "$(classify_url "https://youtube.com/watch?v=x")"
check_eq "vimeo → video" "video" "$(classify_url "https://vimeo.com/123")"
check_eq "unknown → default video" "video" "$(classify_url "https://example.com/x")"
check_eq "unknown → GRAB_DEFAULT=audio" "audio" "$(GRAB_DEFAULT=audio classify_url "https://example.com/x")"
# ═══ Part 3: yt-mp3 ═══
# 3.0 dry-run requires no deps and prints the yt-dlp command (audio flags)
test_run_env "${env_base[@]}" -- "$mp3" --dry-run "https://youtube.com/watch?v=dQw4w9WgXcQ"
check_rc "mp3 dry-run ok" 0 "$TR_RC"
check_contains "mp3 dry-run has -x --audio-format mp3" "--audio-format mp3" "$TR_OUT"
check_contains "mp3 dry-run has --embed-metadata" "--embed-metadata" "$TR_OUT"
# 3.1 --by-artist template
test_run_env "${env_base[@]}" -- "$mp3" --dry-run --by-artist "https://youtu.be/x"
check_contains "mp3 --by-artist organizes by artist/uploader" "%(artist,uploader)s/" "$TR_OUT"
# 3.2 --no-playlist propagated
test_run_env "${env_base[@]}" -- "$mp3" --dry-run --no-playlist "https://youtube.com/watch?v=x"
check_contains "mp3 --no-playlist passed through" "--no-playlist" "$TR_OUT"
# 3.3 --cookies must exist
test_run_env "${env_base[@]}" -- "$mp3" --dry-run --cookies "$sandbox/nope.txt" "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS mp3 missing cookies file errors\n' \
|| printf ' FAIL mp3 accepted missing cookies file\n'
# 3.4 YT_OUT_DIR seam overrides default output
test_run_env "${env_base[@]}" -- "$mp3" --dry-run "https://youtube.com/watch?v=x"
check_contains "mp3 uses YT_OUT_DIR seam" "$sandbox/out/%(title)s.%(ext)s" "$TR_OUT"
# 3.5 missing dep in non-dry-run → hard error (no yt-dlp on PATH at all)
test_run_env PATH="/usr/bin:/bin" -- "$mp3" "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS mp3 without yt-dlp in path errors\n' \
|| printf ' FAIL mp3 ran without yt-dlp present\n'
# 3.6 no URL → usage exit 0
test_run_env "${env_base[@]}" -- "$mp3" --dry-run
check_rc "mp3 no URL → usage exit 0" 0 "$TR_RC"
check_contains "mp3 no URL prints usage" "Usage: pos media yt mp3" "$TR_OUT"
# 3.7 NEGATIVE CONTROL: unsafe URL not expanded — passed as ONE literal arg.
# `$(echo pwned)` stays literal inside the URL token; no second pwned word
# appears in the argv log (which would mean command substitution ran).
: > "$ytdlp_log"
bad='$(echo pwned)'
test_run_env "${env_base[@]}" -- "$mp3" "http://example.com/$bad"
check_rc "unsafe URL run (fake yt-dlp) ok" 0 "$TR_RC"
local logged; logged="$(cat "$ytdlp_log")"
check_contains "url passed literally (single arg)" "\$(echo pwned)" "$logged"
check_not_contains "no separate pwned word from substitution" "example.com/ pwned" "$logged"
# ═══ Part 4: yt-mp4 ═══
# 4.0 --best
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --best "https://youtube.com/watch?v=x"
check_contains "mp4 --best selects bestvideo+bestaudio" "bestvideo*+bestaudio/best" "$TR_OUT"
# 4.1 --worst
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --worst "https://youtube.com/watch?v=x"
check_contains "mp4 --worst selects worst" "-f worst" "$TR_OUT"
# 4.2 -f id skips prompt
test_run_env "${env_base[@]}" -- "$mp4" --dry-run -f 22 "https://youtube.com/watch?v=x"
check_contains "mp4 -f 22 (no prompt)" "-f 22" "$TR_OUT"
# 4.3 mutual exclusions
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --best --worst "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS mp4 --best/--worst mutually exclusive\n' \
|| printf ' FAIL mp4 allowed --best and --worst together\n'
test_run_env "${env_base[@]}" -- "$mp4" --dry-run -f 22 --best "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS mp4 --format/--best mutually exclusive\n' \
|| printf ' FAIL mp4 allowed --format and --best together\n'
# 4.4 YT_OUT_DIR seam
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --best "https://youtube.com/watch?v=x"
check_contains "mp4 uses YT_OUT_DIR seam" "$sandbox/out/%(title)s.%(ext)s" "$TR_OUT"
# 4.5 no URL → usage
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --best
check_rc "mp4 no URL → usage exit 0" 0 "$TR_RC"
check_contains "mp4 no URL prints usage" "Usage: pos media yt mp4" "$TR_OUT"
# ═══ Part 5: yt-grab classification + delegation (dry-run) + config ═══
# 5.0 music.youtube → audio route (dry-run prints delegate command, no exec)
test_run_env "${env_base[@]}" -- "$grab" --dry-run "https://music.youtube.com/watch?v=abc"
check_contains "grab music.youtube routes to yt mp3" "pos media yt mp3" "$TR_OUT"
check_not_contains "grab music.youtube not routed to mp4" "pos media yt mp4" "$TR_OUT"
# 5.1 youtube.com → video route (default --best)
test_run_env "${env_base[@]}" -- "$grab" --dry-run "https://youtube.com/watch?v=xyz"
check_contains "grab youtube routes to yt mp4" "pos media yt mp4 --best" "$TR_OUT"
# 5.2 --audio forces audio despite video domain
test_run_env "${env_base[@]}" -- "$grab" --dry-run --audio "https://youtube.com/watch?v=abc"
check_contains "grab --audio forces yt mp3" "pos media yt mp3" "$TR_OUT"
# 5.3 --worst → video --worst
test_run_env "${env_base[@]}" -- "$grab" --dry-run --worst "https://youtube.com/watch?v=abc"
check_contains "grab --worst routes to mp4 --worst" "pos media yt mp4 --worst" "$TR_OUT"
# 5.4 --audio + --video mutually exclusive
test_run_env "${env_base[@]}" -- "$grab" --dry-run --audio --video "https://youtube.com/watch?v=abc"
[ "$TR_RC" -ne 0 ] && printf ' PASS grab --audio/--video mutually exclusive\n' \
|| printf ' FAIL grab allowed --audio and --video together\n'
# 5.5 invalid URL
test_run_env "${env_base[@]}" -- "$grab" "not-a-url"
[ "$TR_RC" -ne 0 ] && printf ' PASS grab rejects invalid url\n' \
|| printf ' FAIL grab accepted invalid url\n'
check_contains "grab error prefixed yt-grab" "yt-grab:" "$TR_OUT"
# 5.6 unknown domain honors GRAB_DEFAULT via config file (audio)
printf 'GRAB_DEFAULT=audio\n' > "$cfg/grab.env"
test_run_env "${env_base[@]}" -- "$grab" --dry-run "https://example.com/x"
check_contains "grab unknown domain honors GRAB_DEFAULT=audio" "pos media yt mp3" "$TR_OUT"
# ═══ Part 6: yt-subtitles ═══
# 6.0 defaults: --write-subs --write-auto-subs --sub-langs best
test_run_env "${env_base[@]}" -- "$subs" --dry-run "https://youtube.com/watch?v=x"
check_contains "subs default writes manual subs" "--write-subs" "$TR_OUT"
check_contains "subs default writes auto subs" "--write-auto-subs" "$TR_OUT"
check_contains "subs default sub-langs best" "--sub-langs best" "$TR_OUT"
# 6.1 NEGATIVE CONTROL: --lang en,ar becomes ONE --sub-langs arg
test_run_env "${env_base[@]}" -- "$subs" --dry-run --lang en,ar "https://youtube.com/watch?v=x"
check_contains "subs --lang en,ar single arg" "--sub-langs en,ar" "$TR_OUT"
check_not_contains "subs does not split --lang (no single 'en')" "--sub-langs en " "$TR_OUT"
# 6.2 --format vtt
test_run_env "${env_base[@]}" -- "$subs" --dry-run --format vtt "https://youtube.com/watch?v=x"
check_contains "subs --format vtt" "--sub-format vtt" "$TR_OUT"
# 6.3 --auto-only drops --write-subs
test_run_env "${env_base[@]}" -- "$subs" --dry-run --auto-only "https://youtube.com/watch?v=x"
check_not_contains "subs --auto-only drops manual subs" "--write-subs" "$TR_OUT"
check_contains "subs --auto-only keeps auto subs" "--write-auto-subs" "$TR_OUT"
# 6.4 --output dir template
test_run_env "${env_base[@]}" -- "$subs" --dry-run --output "$sandbox/subs" "https://youtube.com/watch?v=x"
check_contains "subs --output template" "$sandbox/subs/%(title)s.%(sub_lang)s.%(ext)s" "$TR_OUT"
# 6.5 invalid --format
test_run_env "${env_base[@]}" -- "$subs" --dry-run --format bogus "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS subs rejects unknown format\n' \
|| printf ' FAIL subs accepted unknown format\n'
# 6.6 unavailable subs: fake yt-dlp fails with no-subtitle stderr → friendly error
mkdir -p "$stubs/subsfail"
cat > "$stubs/subsfail/yt-dlp" <<STUB
#!/usr/bin/env bash
printf 'ERROR: no subtitles found' >&2
exit 1
STUB
chmod +x "$stubs/subsfail/yt-dlp"
test_run_env PATH="$stubs/subsfail:/usr/bin:/bin" -- "$subs" "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS subs no-subtitles exits nonzero\n' \
|| printf ' FAIL subs no-subtitles exited 0\n'
check_contains "subs unavailable-subs message" "unavailable subtitles" "$TR_OUT"
# 6.7 NEGATIVE CONTROL: txt conversion strips SRT timestamps — run the real
# tool non-dry with a fake yt-dlp that writes an .srt into the cwd; the txt
# converter must strip timestamps/html and keep the words.
local txtdir="$sandbox/txtdir"
mkdir -p "$txtdir" "$stubs/txtsubs"
cat > "$stubs/txtsubs/yt-dlp" <<STUB
#!/usr/bin/env bash
printf '1\\n00:00:01,000 --> 00:00:02,000\\nHello <i>world</i>\\n\\n2\\n00:00:03,000 --> 00:00:04,000\\nSecond line\\n' > foo.srt
exit 0
STUB
chmod +x "$stubs/txtsubs/yt-dlp"
(
cd "$txtdir"
env PATH="$stubs/txtsubs:/usr/bin:/bin" "$subs" --format txt "https://youtube.com/watch?v=x"
) >"$sandbox/txtrun.log" 2>&1
local txt; txt="$(cat "$txtdir/foo.txt" 2>/dev/null || true)"
check_not_contains "txt output strips timestamps" "-->" "$txt"
check_not_contains "txt output strips html tags" "<i>" "$txt"
check_contains "txt output keeps words" "Hello world" "$txt"
# 6.8 subs dry-run skips deps (yt-dlp not in path) — needs no binaries
test_run_env PATH="/usr/bin:/bin" -- "$subs" --dry-run "https://youtube.com/watch?v=x"
check_rc "subs dry-run works without yt-dlp (skips deps)" 0 "$TR_RC"
check_contains "subs dry-run prints yt-dlp cmd" "yt-dlp" "$TR_OUT"
# 6.9 subs no URL → error (not usage), exit 1
test_run_env "${env_base[@]}" -- "$subs" --dry-run
[ "$TR_RC" -ne 0 ] && printf ' PASS subs no URL exits nonzero\n' \
|| printf ' FAIL subs no URL exited 0\n'
check_contains "subs no URL error message" "missing URL (see --help)" "$TR_OUT"
}
+235
View File
@@ -0,0 +1,235 @@
#!/usr/bin/env bash
set -euo pipefail
# t-share-mountpoint.sh — permanent regression for the share-client
# ask_mountpoint() manual-entry flow (2026-09-07 `t=type` UX change in
# bin/pos-share-smb-client + bin/pos-share-nfs-client; byte-identical bodies).
#
# Behaviour contract under test (both clients):
# existing dir → printed as-is, rc 0, NO confirm, NO mkdir
# new dir + y → confirm gate passes, sudo mkdir -p, dir created, rc 0
# new dir + n → rc 1, nothing created (confirm EOF also denies)
# existing file → rc 1 + "not a directory" warning
# relative / trailing-slash / system paths / empty-EOF → rc 1, no side effects
# mkdir failure → rc 1 + "Could not create" warning
# stream → display on stderr, result on stdout
#
# Strategy (production logic is never re-typed): the REAL ask_mountpoint body
# is brace-extracted verbatim from each client file (extract_fn, same pattern
# as t-menu-allow-empty.sh / t-ai-server-validate.sh) and sourced; the REAL
# helper chain lib/common.sh (warn/confirm/run/log) + lib/share-lib.sh
# (share_ask_value → menu_ask_value → menu_read_value) is exercised through
# the deterministic NON-TTY stdin path (stty -g fails on a pipe → plain
# IFS= read, common.sh:181-186). Input is fed as `printf '%b' | fn`, exactly
# like t-menu-allow-empty.sh feeds menu_ask_value. A fake `sudo` on PATH
# records every invocation and honors SUDO_FAIL, so the sandbox never touches
# the real system. NOTE: bash suppresses read -rp prompt text on a pipe, so
# the confirm prompt string is never asserted — the confirm gate is proven
# behaviourally (feed y → created; feed n → rc 1; missing answer → EOF-deny
# rc 1) and by SUDO_LOG (mkdir attempted or not).
# extract_fn <source-file> <fnname> — print one brace-delimited function body.
extract_fn() {
local file="$1" fn="$2"
awk -v fn="$fn" '
BEGIN { found=0; depth=0 }
{
if (!found && $0 ~ ("^" fn "\\(\\)")) { found=1; depth=0 }
if (found) {
n_open = gsub(/\{/, "{")
n_close = gsub(/\}/, "}")
depth = depth + n_open - n_close
print
if (depth <= 0) exit
}
}
' "$file"
}
run_test() {
source "$ROOT/lib/common.sh" # warn / confirm / run / log
source "$ROOT/lib/share-lib.sh" # share_ask_value → REAL menu_ask_value
local sandbox stubs
sandbox="$(mksandbox share-mountpoint)"
stubs="$sandbox/stubs"
mkdir -p "$stubs"
# Fake sudo: records every invocation to $SUDO_LOG; exits 1 (no side
# effect) when SUDO_FAIL=1, otherwise `exec "$@"` (real mkdir) so the
# confirmed-create case genuinely creates the dir inside the sandbox.
cat > "$stubs/sudo" <<'STUB'
#!/usr/bin/env bash
printf 'sudo %s\n' "$*" >> "${SUDO_LOG:?fake sudo needs SUDO_LOG}"
if [ "${SUDO_FAIL:-0}" = "1" ]; then
printf 'sudo: permission denied (fake)\n' >&2
exit 1
fi
[ $# -gt 0 ] && exec "$@"
exit 0
STUB
chmod +x "$stubs/sudo"
export PATH="$stubs:$PATH"
export SUDO_LOG="$sandbox/sudo.log"
export SUDO_FAIL=0
: > "$SUDO_LOG"
# ═══ Part A0: non-TTY stdin contract of the real reader chain ═══
# The whole matrix below is only deterministic if menu_read_value falls
# back to plain `IFS= read` on a pipe and confirm() also reads plain
# stdin. Prove both against the REAL helpers.
local probe_out probe_rc
set +e
probe_out="$(printf 'x\n' | menu_read_value "probe" 2>/dev/null)"; probe_rc=$?
set -e
check_rc "non-TTY: menu_read_value pipe → rc 0 (plain-read fallback)" 0 "$probe_rc"
check_eq "non-TTY: menu_read_value pipe → value" "x" "$probe_out"
set +e
printf '' | stty -g >/dev/null 2>&1; probe_rc=$?
set -e
check_rc "non-TTY: stty -g fails on a pipe (plain-read path active)" 1 "$probe_rc"
set +e
printf 'y\n' | confirm probe n >/dev/null 2>&1; probe_rc=$?
set -e
check_rc "non-TTY: confirm y → rc 0" 0 "$probe_rc"
set +e
printf 'n\n' | confirm probe n >/dev/null 2>&1; probe_rc=$?
set -e
check_rc "non-TTY: confirm n → rc 1" 1 "$probe_rc"
set +e
printf '' | confirm probe n >/dev/null 2>&1; probe_rc=$?
set -e
check_rc "non-TTY: confirm EOF → rc 1 (fail-closed)" 1 "$probe_rc"
# ═══ Part A/B runner over the REAL extracted ask_mountpoint ═══
# mp_case <prefix> <desc> <feed> <expect_rc> <expect_out>
# [<stderr-needle> [<stderr-absent> [<sudo_fail>]]]
# feed = literal bytes: line 1 = mountpoint, line 2 (when present) = the
# confirm answer. SUDO_LOG resets per case; side effects asserted by the
# caller. SUDO_FAIL applies to the ask_mountpoint call only (assignment
# prefix on the pipeline element; exported attribute carries it to the
# fake sudo child process).
mp_case() {
local prefix="$1" desc="$2" feed="$3" expect_rc="$4" expect_out="$5"
local needle="${6:-}" absent="${7:-}" failflag="${8:-0}"
local out rc errf="$sandbox/mp.err"
: > "$SUDO_LOG"
set +e
out="$(printf '%b' "$feed" | SUDO_FAIL="$failflag" ask_mountpoint 2>"$errf")"
rc=$?
set -e
check_rc "$prefix: $desc (rc)" "$expect_rc" "$rc"
check_eq "$prefix: $desc (stdout)" "$expect_out" "$out"
if [ -n "$needle" ]; then
check_contains "$prefix: $desc (stderr)" "$needle" "$(cat "$errf")"
fi
if [ -n "$absent" ]; then
check_not_contains "$prefix: $desc (stderr)" "$absent" "$(cat "$errf")"
fi
}
local client file
for client in smb nfs; do
case "$client" in
smb) file="$ROOT/bin/pos-share-smb-client" ;;
nfs) file="$ROOT/bin/pos-share-nfs-client" ;;
esac
local fn_file="$sandbox/ask_mountpoint-$client.sh"
extract_fn "$file" ask_mountpoint > "$fn_file"
source "$fn_file" # REAL body — a syntax error here aborts (FAIL, never silent)
local cdir="$sandbox/$client"
mkdir -p "$cdir/existing"
touch "$cdir/file"
# 1. existing dir → as-is, rc 0. Feed has NO y/n line, so reaching the
# confirm gate would hit EOF → rc 1; rc 0 + empty SUDO_LOG proves
# the confirm/mkdir branch was never entered.
mp_case "$client" "existing dir → used as-is, no create" \
"$cdir/existing\n" 0 "$cdir/existing" "" "Created mount point"
check_eq "$client: existing dir → sudo NOT attempted" "" "$(cat "$SUDO_LOG")"
check_file_exists "$client: existing dir → fixture intact" "$cdir/existing"
# 2. new dir + confirm y → sudo mkdir -p, dir created, printed, logged
mp_case "$client" "new dir confirm=y → rc 0 + created" \
"$cdir/newdir\ny\n" 0 "$cdir/newdir" "Created mount point"
check_contains "$client: new dir → sudo mkdir -p attempted" \
"sudo mkdir -p $cdir/newdir" "$(cat "$SUDO_LOG")"
check_file_exists "$client: new dir → created" "$cdir/newdir"
# 3. new dir + confirm n → rc 1, nothing created
mp_case "$client" "new dir confirm=n → rc 1" \
"$cdir/declined\nn\n" 1 "" "" ""
check_eq "$client: declined → sudo NOT attempted" "" "$(cat "$SUDO_LOG")"
check_file_absent "$client: declined → no dir created" "$cdir/declined"
# 4. new dir + confirm EOF → rc 1 (gate fails closed)
mp_case "$client" "new dir confirm=EOF → rc 1 (fail-closed)" \
"$cdir/eof\n" 1 "" "" ""
check_eq "$client: confirm-EOF → sudo NOT attempted" "" "$(cat "$SUDO_LOG")"
check_file_absent "$client: confirm-EOF → no dir created" "$cdir/eof"
# 5. existing non-directory → rc 1 + not-a-directory warning
mp_case "$client" "existing file → rc 1 not-a-directory" \
"$cdir/file\n" 1 "" "not a directory" ""
check_eq "$client: file → sudo NOT attempted" "" "$(cat "$SUDO_LOG")"
# 6. relative path → rc 1, warned, no write
mp_case "$client" "relative path → rc 1" \
"relative/path\n" 1 "" "not an absolute path" ""
check_eq "$client: relative → sudo NOT attempted" "" "$(cat "$SUDO_LOG")"
# 7. trailing slash → rc 1, warned, no write
mp_case "$client" "trailing slash → rc 1" \
"$cdir/existing/\n" 1 "" "must not end with a slash" ""
check_eq "$client: trailing slash → sudo NOT attempted" "" "$(cat "$SUDO_LOG")"
# 8. system paths → rc 1 (shape glob fires before any FS access)
mp_case "$client" "system path /etc → rc 1" "/etc\n" 1 "" "Refusing system path" ""
mp_case "$client" "system path /root → rc 1" "/root\n" 1 "" "Refusing system path" ""
mp_case "$client" "system path /home/*/.ssh* → rc 1" \
"/home/ci-user/.ssh/authorized_keys\n" 1 "" "Refusing system path" ""
check_eq "$client: system paths → sudo NEVER attempted" "" "$(cat "$SUDO_LOG")"
# 9. empty/EOF on the path input → rc 1 (share_ask_value cancel)
mp_case "$client" "EOF on path → rc 1" "" 1 "" "" ""
check_eq "$client: EOF → sudo NOT attempted" "" "$(cat "$SUDO_LOG")"
# 10. mkdir failure (sudo shim fails) → rc 1 + could-not-create warning
mp_case "$client" "mkdir fails → rc 1" \
"$cdir/failmkdir\ny\n" 1 "" "Could not create" "" 1
check_file_absent "$client: mkdir-fail → no dir created" "$cdir/failmkdir"
# ═══ Part B: static guards — the old n=new flow must not regress ═══
local thint=0 tarm=0
grep -q 't=type' "$file" && thint=1
grep -q 't | T)' "$file" && tarm=1
check_eq "$client: t=type hint present" 1 "$thint"
check_eq "$client: t | T arm present" 1 "$tarm"
check_eq "$client: no n=new hint" 0 "$(grep -c 'n=new' "$file" || true)"
check_eq "$client: no n | N arm" 0 "$(grep -c 'n | N)' "$file" || true)"
check_eq "$client: no ask_new_mountpoint" 0 "$(grep -c 'ask_new_mountpoint' "$file" || true)"
# Behavioural contract: the existing-dir branch must precede the
# confirm/mkdir create flow inside the extracted body, so an existing
# dir can never reach confirm/mkdir.
check_eq "$client: ask_mountpoint has -d branch" 1 \
"$(grep -cF '[ -d "$dir" ]' "$fn_file" || true)"
check_eq "$client: ask_mountpoint has create flow" 1 \
"$(grep -cF 'confirm "Create mountpoint' "$fn_file" || true)"
local dline cline order=0
dline="$(grep -nF '[ -d "$dir" ]' "$fn_file" | head -1 | cut -d: -f1 || true)"
cline="$(grep -nF 'confirm "Create mountpoint' "$fn_file" | head -1 | cut -d: -f1 || true)"
[ -n "$dline" ] && [ -n "$cline" ] && [ "$dline" -lt "$cline" ] && order=1
check_eq "$client: existing-dir branch precedes confirm/mkdir" 1 "$order"
unset -f ask_mountpoint
done
# ═══ Part C: symmetry — the two shipped bodies must stay identical ═══
local same=0
cmp -s "$sandbox/ask_mountpoint-smb.sh" "$sandbox/ask_mountpoint-nfs.sh" && same=1
check_eq "ask_mountpoint bodies byte-identical (smb == nfs)" 1 "$same"
}
+133
View File
@@ -0,0 +1,133 @@
#!/usr/bin/env bash
set -euo pipefail
# t-telegram-listener-exec.sh — async command execution in the Telegram
# listener. Proves the listener can execute ANY valid Bash command without
# blocking: simple output, compound commands, pipes, stderr, long-running
# (timeout), and that the listener stays responsive while a command runs.
#
# Hermetic: stubbed curl (serves a canned getUpdates batch with /command
# messages, then empty batches), stubbed systemctl, real jq/timeout.
# No network, no real Telegram, no FFmpeg (unless /dev/video0 exists).
run_test() {
require_cmd jq "telegram exec" || return 0
require_cmd timeout "telegram exec" || return 0
local sandbox stubs cfg curl_log marker listener batch
sandbox="$(mksandbox telegram-exec)"
stubs="$sandbox/stubs"
cfg="$sandbox/cfg"
curl_log="$sandbox/curl.log"
marker="$sandbox/executed.log"
listener="$ROOT/bin/pos-communication-telegram-listener"
mkdir -p "$stubs" "$cfg"
: > "$curl_log"
: > "$marker"
# ── command map: one /command per line, each triggers a known behavior ──
cat > "$cfg/telegram_commands.env" <<'MAP'
/echo_hello=echo hello
/compound=sleep 0.2 && echo done
/stdout_test=printf 'line1\nline2\n'
/stderr_test=bash -c 'echo error_msg >&2; echo output_msg'
/pipe_test=echo "hello world" | tr ' ' '\n'
/long_run=sleep 30
/no_output=true
/quiet_test=@quiet echo hello_quiet
MAP
: > "$cfg/telegram_prefixes.env"
# ── stub curl ──
# Serve a batch with 8 commands (one per mapped /command), then empty.
local batch_file="$sandbox/batch.json"
cat > "$batch_file" <<'JSON'
{"ok":true,"result":[
{"update_id":1,"message":{"message_id":10,"from":{"id":123},"chat":{"id":456},"text":"/echo_hello"}},
{"update_id":2,"message":{"message_id":11,"from":{"id":123},"chat":{"id":456},"text":"/compound"}},
{"update_id":3,"message":{"message_id":12,"from":{"id":123},"chat":{"id":456},"text":"/stdout_test"}},
{"update_id":4,"message":{"message_id":13,"from":{"id":123},"chat":{"id":456},"text":"/stderr_test"}},
{"update_id":5,"message":{"message_id":14,"from":{"id":123},"chat":{"id":456},"text":"/pipe_test"}},
{"update_id":6,"message":{"message_id":15,"from":{"id":123},"chat":{"id":456},"text":"/long_run"}},
{"update_id":7,"message":{"message_id":16,"from":{"id":123},"chat":{"id":456},"text":"/no_output"}},
{"update_id":8,"message":{"message_id":17,"from":{"id":123},"chat":{"id":456},"text":"/quiet_test"}}
]}
JSON
cat > "$stubs/curl" <<STUB
#!/usr/bin/env bash
printf 'curl %s\n' "\$*" >> "$curl_log"
for a in "\$@"; do
case "\$a" in
*getUpdates*)
if [ ! -e "$sandbox/served.once" ]; then
touch "$sandbox/served.once"
cat "$batch_file"
else
sleep 1
printf '%s' '{"ok":true,"result":[]}'
fi
exit 0
;;
esac
done
printf '%s' '{"ok":true}'
STUB
chmod +x "$stubs/curl"
printf '#!/usr/bin/env bash\nexit 1\n' > "$stubs/systemctl"
chmod +x "$stubs/systemctl"
local common=(PATH="$stubs:/usr/bin:/bin" CONFIG_DIR="$cfg"
TELEGRAM_BOT_TOKEN=testbot TELEGRAM_CHAT_ID=456 TELEGRAM_OWNER_ID=123)
# ── run the listener ──
# /long_run (sleep 30) runs in background — the listener does NOT block.
# The 45s outer timeout proves the listener stayed responsive.
test_run_env "${common[@]}" -- timeout 45 "$listener" --run
local curl_content
curl_content="$(cat "$curl_log")"
# ── all commands were dispatched ──
check_contains "listener processed /echo_hello" "exec: /echo_hello" "$TR_OUT"
check_contains "listener processed /compound" "exec: /compound" "$TR_OUT"
check_contains "listener processed /long_run" "exec: /long_run" "$TR_OUT"
# ── /echo_hello → "hello" ──
check_contains "/echo_hello reply" "text=hello" "$curl_content"
# ── /compound (sleep 0.2 && echo done) → "done" ──
check_contains "/compound reply" "text=done" "$curl_content"
# ── /stdout_test → multi-line stdout captured ──
check_contains "/stdout_test reply" "text=line1" "$curl_content"
# ── /stderr_test → stderr+stdout both captured ──
# Output is "error_msg\noutput_msg" (newline-separated).
# The curl log may split this across lines, so check each token alone.
check_contains "/stderr_test stderr captured" "error_msg" "$curl_content"
check_contains "/stderr_test stdout captured" "output_msg" "$curl_content"
# ── /pipe_test → pipe works ──
check_contains "/pipe_test reply" "text=hello" "$curl_content"
# ── /no_output → "OK" (no output → default reply) ──
check_contains "/no_output reply" "text=OK" "$curl_content"
# ── /quiet_test → NO sendMessage with "hello_quiet" ──
# The setMyCommands call may contain "hello_quiet" in the description,
# so we check that no sendMessage line contains it.
local quiet_send_count
quiet_send_count="$(printf '%s' "$curl_content" | grep 'sendMessage' | grep -c 'hello_quiet' || true)"
check_eq "/quiet_test suppresses reply" 0 "$quiet_send_count"
# ── the daemon exited within the outer timeout (not hung) ──
# rc=124 means `timeout` killed it — listener was alive and processing.
# rc=0 means it exited cleanly. Both prove no hang.
if [ "${TR_RC:-0}" -eq 124 ] || [ "${TR_RC:-0}" -eq 0 ]; then
printf ' PASS daemon exited cleanly (rc=%s, not hung)\n' "${TR_RC}"
else
printf ' FAIL daemon exited with unexpected rc=%s\n' "${TR_RC:-?}"
fi
}
+129
View File
@@ -0,0 +1,129 @@
#!/usr/bin/env bash
set -euo pipefail
# t-telegram-listener-singleton.sh — single-instance guard for the Telegram
# listener daemon (flock on ${XDG_RUNTIME_DIR:-/tmp}/pos-telegram-listener.lock):
# (a) the first --run acquires the lock and reaches its poll loop;
# (b) a second --run on the same runtime dir fails fast (exit 1) with the
# exact single-instance message — no 409/getUpdates race;
# (c) the flock auto-releases when the first instance exits, so the next
# --run starts cleanly (systemd Restart=always path);
# (d) --status reports the lock through the same primitives.
# Hermetic: stubbed curl (no network) + systemctl (no user bus), real
# jq/flock/timeout, sandboxed XDG_RUNTIME_DIR + CONFIG_DIR.
run_test() {
require_cmd jq "telegram singleton guard" || return 0
require_cmd flock "telegram singleton guard" || return 0
require_cmd timeout "telegram singleton guard" || return 0
local sandbox stubs cfg runtime home listener curl_log marker first_log
sandbox="$(mksandbox telegram-singleton)"
stubs="$sandbox/stubs"
cfg="$sandbox/cfg"
runtime="$sandbox/runtime"
home="$sandbox/home"
listener="$ROOT/bin/pos-communication-telegram-listener"
curl_log="$sandbox/curl.log"
marker="$sandbox/loop.started"
first_log="$sandbox/first.log"
mkdir -p "$stubs" "$cfg" "$runtime" "$home"
: > "$curl_log"
# Stub curl: never touches the network. getUpdates serves an empty batch
# forever (first call touches $marker so the test knows the daemon reached
# its poll loop — which only happens AFTER the lock was acquired and the
# config checks passed); everything else returns {ok:true}. The small
# sleep keeps the empty-poll loop from spinning while the test runs.
cat > "$stubs/curl" <<STUB
#!/usr/bin/env bash
printf 'curl %s\n' "\$*" >> "$curl_log"
for a in "\$@"; do
case "\$a" in
*getUpdates*)
touch "$marker"
sleep 1
printf '%s' '{"ok":true,"result":[]}'
exit 0
;;
esac
done
printf '%s' '{"ok":true}'
STUB
chmod +x "$stubs/curl"
# Stub systemctl: deterministic exit 1 — --status must not reach the real
# user bus; the autostart line is not what this test asserts.
printf '#!/usr/bin/env bash\nexit 1\n' > "$stubs/systemctl"
chmod +x "$stubs/systemctl"
: > "$cfg/telegram_commands.env"
: > "$cfg/telegram_prefixes.env"
local common=(PATH="$stubs:/usr/bin:/bin" CONFIG_DIR="$cfg"
XDG_RUNTIME_DIR="$runtime" HOME="$home"
TELEGRAM_BOT_TOKEN=testbot TELEGRAM_CHAT_ID=456 TELEGRAM_OWNER_ID=123)
# ── (a) first instance acquires the lock and runs ──
rm -f "$marker"
env "${common[@]}" timeout 10 "$listener" --run >"$first_log" 2>&1 &
local first_pid=$!
local waited=0
until [ -e "$marker" ]; do
sleep 0.1
waited=$((waited + 1))
if [ "$waited" -ge 100 ]; then
printf ' FAIL first listener never reached the poll loop (log below)\n'
cat "$first_log"
kill "$first_pid" 2>/dev/null || true
wait "$first_pid" 2>/dev/null || true
return 0
fi
done
printf ' PASS first listener acquired lock and reached the poll loop\n'
test_run_env "${common[@]}" -- "$listener" --status
check_rc "status while daemon up exits 0" 0 "$TR_RC"
check_contains "status reports lock held while running" \
"listener: running (single instance lock held)" "$TR_OUT"
# ── (b) second instance fails fast with the exact message ──
test_run_env "${common[@]}" -- timeout 3 "$listener" --run
check_rc "second instance fails fast (exit 1)" 1 "$TR_RC"
check_contains "second instance prints exact single-instance message" \
"ERROR: listener already running (single instance) — check: systemctl --user status pos-telegram-listener" \
"$TR_OUT"
# ── (c) lock releases when the first instance ends ──
kill "$first_pid" 2>/dev/null || true
wait "$first_pid" 2>/dev/null || true
test_run_env "${common[@]}" -- "$listener" --status
check_contains "status reports not running after first exits" \
"listener: not running" "$TR_OUT"
rm -f "$marker"
env "${common[@]}" timeout 10 "$listener" --run >"$sandbox/third.log" 2>&1 &
local third_pid=$!
waited=0
until [ -e "$marker" ]; do
sleep 0.1
waited=$((waited + 1))
if [ "$waited" -ge 100 ]; then
printf ' FAIL third listener never reached the poll loop (log below)\n'
cat "$sandbox/third.log"
kill "$third_pid" 2>/dev/null || true
wait "$third_pid" 2>/dev/null || true
return 0
fi
done
printf ' PASS third listener starts cleanly after the lock was released\n'
kill "$third_pid" 2>/dev/null || true
wait "$third_pid" 2>/dev/null || true
test_run_env "${common[@]}" -- "$listener" --status
check_contains "status reports not running after third exits" \
"listener: not running" "$TR_OUT"
}