feat: pos media yt — unified YouTube tools + subtitles (POS--9)
gates / consistency-and-conventions (push) Successful in 20s

This commit is contained in:
Your Name
2026-09-09 07:04:22 -04:00
parent 9a164462e7
commit f14d24950a
19 changed files with 1093 additions and 482 deletions
+2
View File
@@ -42,6 +42,8 @@ summary (newest last).
## Done
- **2026-09-09** — Unified YouTube tools into `pos media yt` + new `subtitles` (Architect POS--9). New `bin/pos-media-yt` dispatcher (mp3/mp4/grab/ytsync/subtitles) + `bin/pos-media-yt-{mp3,mp4,grab,subtitles,ytsync}`; the ytsync file is a forwarder to the existing `pos media ytsync`; legacy `bin/pos-media-{mp3,mp4,grab}` became thin forwarders to the `yt` forms. New `lib/yt-lib.sh` (deps/URL-validation/echo/classify helpers; `classify_url` migrated from grab, `yt_validate_url` is a return-1 checker — never exits, so callers can prefix errors). `bin/pos` INTERACTIVE_CMDS += `media-yt-mp4` (interactive format pick reads stdin). `pos-media-yt-subtitles` extracts captions via `--write-subs --write-auto-subs --sub-langs best`, `--lang en,ar` (one `--sub-langs` arg), `srt|vtt|txt` (txt = srt→txt conversion stripping timestamps/HTML), `--auto-only`, `--list-subs` probe, `--output`, no-ffmpeg dep (yt-dlp only; dry-run skips deps entirely). Docs: DOC/POS.md media section rewritten (yt group + forwarder rows), DOC/howto/media.md `yt` commands + subtitles section, AGENT_Context hand-maintained `lib/yt-lib.sh` row, tests/README row. New `tests/t-pos-media-yt.sh` (72 checks: dispatcher/forwarder resolution, full `pos media mp3` dispatch chain, yt-lib helpers, per-tool flags/dry-run/`YT_OUT_DIR` seam/`GRAB_DEFAULT` config, 3 mandated negative controls — unsafe-URL no-expansion, `--lang en,ar` single arg, txt timestamp-stripping, unavailable-subs detection). `tests/t-config-precedence.sh` Part D config-consumer list updated `pos-media-grab``pos-media-yt-grab`. Verified: `bash -n` all; `make gen` ×2 byte-idempotent; `make check` OK; `make lint` 0 FAIL / 0 WARN; `make test` 21 files / 533 checks / 0 fail / 0 skip; `git diff --check` clean; smokes — `pos media yt --help`, `yt mp3/mp4/subtitles --help`, `yt ytsync --help` (reaches `pos media ytsync`), `pos media mp3 --help` forwarder, `pos tree` shows the `yt` subtree (with repo-first PATH; system `/usr/local/bin` has a stale pre-POS--9 install that shadows it otherwise).
- **2026-09-08** — `install.sh` version gate (Architect→Builder): skip+abort when installed version == current version, `--force` to bypass, version scheme `0.0c<git commit count>` (auto-bumps per commit). `install_version()` derives `0.0c$(git rev-list --count HEAD)`; empty when `.git` absent → gate skipped (silently); `INSTALL_VERSION_OVERRIDE` env var (presence-check) = test seam. Gate after arg-parse, before phases, numeric comparison (strip `0.0c`, `-eq`); `log "Already installed ($CURRENT_VERSION). Use --force to re-install."` / `--dry-run``(dry-run) Would skip install: already at version $CURRENT_VERSION`, both exit 0. `FORCE=0` init, `--force` parse + usage. `flag_set installed_version "$CURRENT_VERSION"` after "Bootstrap complete" banner (only when DRY_RUN≠1 and version non-empty; even under --force). New `tests/t-install-version.sh` (21 checks / 9 cases). Docs: README/SCRIPTS/AGENT_Context (flags, flow, line count 248→301, tests/README row). Verified: `bash -n` clean; `make gen` idempotent; `make check` OK; `make lint` 0 FAIL / 0 WARN; `make test` suite green.
+37 -18
View File
@@ -10,19 +10,19 @@
<!-- GEN:START docmap -->
| ## 1. Project Overview | 2843 |
| ## 2. Directory Structure | 44210 |
| ## 3. Installation Flow | 211269 |
| ## 4. The `pos` CLI System | 270351 |
| ## 5. Shared Library — `lib/common.sh` | 352383 |
| ## 6. Docker Compose / ScaleTail | 384426 |
| ## 7. Optional Apps (`apps/`) | 427456 |
| ## 8. Entertainment Module | 457470 |
| ## 9. Systemd Services | 471482 |
| ## 10. Configuration Files | 483509 |
| ## 11. Coding Conventions | 510542 |
| ## 12. Development Workflow | 543595 |
| ## 13. Key File Quick Reference | 596671 |
| ## 14. Common Tasks for Agents | 672705 |
| ## 2. Directory Structure | 44216 |
| ## 3. Installation Flow | 217275 |
| ## 4. The `pos` CLI System | 276363 |
| ## 5. Shared Library — `lib/common.sh` | 364395 |
| ## 6. Docker Compose / ScaleTail | 396438 |
| ## 7. Optional Apps (`apps/`) | 439468 |
| ## 8. Entertainment Module | 469482 |
| ## 9. Systemd Services | 483494 |
| ## 10. Configuration Files | 495521 |
| ## 11. Coding Conventions | 522554 |
| ## 12. Development Workflow | 555607 |
| ## 13. Key File Quick Reference | 608690 |
| ## 14. Common Tasks for Agents | 691724 |
<!-- GEN:END docmap -->
## 1. Project Overview
@@ -90,7 +90,13 @@ Linux_post_install/
│ ├── pos-media-mp4 # Download video as MP4 (smart/interactive format select)
│ ├── pos-media-sync # Incremental Music → USB sync (mp3/mp4, add/update only)
│ │ [deps: lsblk jq]
│ ├── pos-media-yt-grab # Auto-download URL as audio or video (classify + route)
│ ├── pos-media-yt-mp3 # Download audio as MP3 (yt-dlp)
│ ├── pos-media-yt-mp4 # Download video as MP4 (smart/interactive format select)
│ ├── pos-media-yt-subtitles # Extract subtitles/captions from a URL (yt-dlp)
│ ├── pos-media-yt-ytsync # Incrementally sync YouTube channels/playlists into ~/Videos
│ ├── pos-media-ytsync # Incrementally sync YouTube channels/playlists into ~/Videos
│ ├── pos-media-yt # YouTube download tools (mp3/mp4/grab/ytsync/subtitles)
│ ├── pos-network-checkport # Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view
│ ├── pos-network-download # aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits)
│ │ [deps: aria2c jq curl]
@@ -311,7 +317,13 @@ All non-interactive `pos` commands log output to `~/.local/share/linux_post_inst
| media | mp3 | `pos-media-mp3` | Download audio as MP3 (yt-dlp) | | |
| media | mp4 | `pos-media-mp4` | Download video as MP4 (smart/interactive format select) | | |
| media | sync | `pos-media-sync` | Incremental Music → USB sync (mp3/mp4, add/update only) | lsblk jq | pos media sync --mp3 → Sync only MP3 files to USB · pos media sync --mp4 --dry-run → Preview MP4 sync without copying |
| media | yt-grab | `pos-media-yt-grab` | Auto-download URL as audio or video (classify + route) | | |
| media | yt-mp3 | `pos-media-yt-mp3` | Download audio as MP3 (yt-dlp) | | |
| media | yt-mp4 | `pos-media-yt-mp4` | Download video as MP4 (smart/interactive format select) | | |
| media | yt-subtitles | `pos-media-yt-subtitles` | Extract subtitles/captions from a URL (yt-dlp) | | |
| media | yt-ytsync | `pos-media-yt-ytsync` | Incrementally sync YouTube channels/playlists into ~/Videos | | |
| media | ytsync | `pos-media-ytsync` | Incrementally sync YouTube channels/playlists into ~/Videos | | |
| media | yt | `pos-media-yt` | YouTube download tools (mp3/mp4/grab/ytsync/subtitles) | | |
| network | checkport | `pos-network-checkport` | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view | | |
| network | download | `pos-network-download` | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) | aria2c jq curl | pos network download add https://example.com/file.zip → Enqueue an HTTP download (auto-starts daemon) · pos network download status → Daemon health + global transfer stats · pos network download watch → Live progress view |
| network | hotspot | `pos-network-hotspot` | Wi-Fi hotspot via create_ap + wihotspot-gui | | |
@@ -376,7 +388,7 @@ source "$(dirname "$0")/../lib/common.sh"
**Scripts that do NOT source common.sh** (self-contained):
<!-- GEN:START selfcontained -->
`pos`, `pos-ai-gemini`, `pos-ai-llamacpp`, `pos-ai-openrouter`, `pos-communication-matrix-sender`, `pos-communication-telegram-listener`, `pos-communication-telegram-sender`, `pos-network-checkport`, `pos-network-hotspot`, `pos-network-ip`, `pos-network-scan`, `pos-ssh-load-keys`, `pos-system-firewall`.
`pos`, `pos-ai-gemini`, `pos-ai-llamacpp`, `pos-ai-openrouter`, `pos-communication-matrix-sender`, `pos-communication-telegram-listener`, `pos-communication-telegram-sender`, `pos-media-grab`, `pos-media-mp3`, `pos-media-mp4`, `pos-media-yt-ytsync`, `pos-network-checkport`, `pos-network-hotspot`, `pos-network-ip`, `pos-network-scan`, `pos-ssh-load-keys`, `pos-system-firewall`.
<!-- GEN:END selfcontained -->
---
@@ -611,6 +623,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `lib/share-lib.sh` | 318 | Domain layer for the share suite (usbsrv/smbclient record parsers, folder+mountpoint candidates, remote listings, service/firewall advisories; EOF-safe) + compat shims to `lib/menu-lib.sh` — used by all five `pos share *` tools |
| `lib/menu-lib.sh` | 362 | Category-neutral interactive menu primitives (`menu_guard` tty guard, `menu_run` looping boxed menu, `menu_pick` type-to-filter picker, `menu_ask_value` prompt-with-default via raw-mode bracketed-paste-safe `menu_read_value`; stderr render, fail-closed on non-tty/EOF) — sourced by `share-lib.sh`, open to any category |
| `lib/registry.sh` | 199 | Shared query API for POS tool metadata headers (`# POS_*:`) — `reg_scan`/`reg_list`/`reg_lookup`/`reg_each`/config scope helpers; used by `pos-tree` and `gen-docs.sh` |
| `lib/yt-lib.sh` | 50 | Shared YouTube helpers for `pos media yt *` (`yt_check_deps`, `yt_validate_url`, `yt_echo_cmd`, `classify_url`) — sourced by `yt-mp3`/`yt-mp4`/`yt-grab`/`yt-subtitles` |
| `bin/flag-reader` | 58 | Inspect flags (list/status/`--raw`) |
| `bin/flag-set` | 21 | Set a flag (optionally with a value) |
| `bin/flag-clear` | 21 | Unset a flag |
@@ -639,11 +652,17 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `bin/pos-entertainment-enable` | 49 | Enable an auto-trigger for a plugin on a schedule |
| `bin/pos-entertainment-send` | 95 | Run a public-API plugin and send its output via the configured notify platforms |
| `bin/pos-entertainment-status` | 62 | Show enabled plugins and scheduler state |
| `bin/pos-media-grab` | 219 | Auto-download URL as audio or video (classify + route) |
| `bin/pos-media-mp3` | 86 | Download audio as MP3 (yt-dlp) |
| `bin/pos-media-mp4` | 132 | Download video as MP4 (smart/interactive format select) |
| `bin/pos-media-grab` | 5 | Auto-download URL as audio or video (classify + route) |
| `bin/pos-media-mp3` | 5 | Download audio as MP3 (yt-dlp) |
| `bin/pos-media-mp4` | 5 | Download video as MP4 (smart/interactive format select) |
| `bin/pos-media-sync` | 219 | Incremental Music → USB sync (mp3/mp4, add/update only) |
| `bin/pos-media-yt-grab` | 208 | Auto-download URL as audio or video (classify + route) |
| `bin/pos-media-yt-mp3` | 86 | Download audio as MP3 (yt-dlp) |
| `bin/pos-media-yt-mp4` | 132 | Download video as MP4 (smart/interactive format select) |
| `bin/pos-media-yt-subtitles` | 169 | Extract subtitles/captions from a URL (yt-dlp) |
| `bin/pos-media-yt-ytsync` | 5 | Incrementally sync YouTube channels/playlists into ~/Videos |
| `bin/pos-media-ytsync` | 1213 | Incrementally sync YouTube channels/playlists into ~/Videos |
| `bin/pos-media-yt` | 33 | YouTube download tools (mp3/mp4/grab/ytsync/subtitles) |
| `bin/pos-network-checkport` | 498 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view |
| `bin/pos-network-download` | 1113 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) |
| `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui |
@@ -663,7 +682,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `bin/pos-ai` | 714 | AI assistant: ask, chat, sessions, capture, models, providers |
| `bin/pos-config` | 80 | Interactive editor for the tools' runtime config (reads # POS_CONFIG: registry) |
| `bin/pos-tree` | 118 | Show the pos CLI command tree: categories, commands, and subcommands |
| `completions/pos.bash` | 314 | Dynamic bash completion |
| `completions/pos.bash` | 316 | Dynamic bash completion |
<!-- GEN:END filetable -->
| `apps/install.sh` | 171 | App install/uninstall picker/orchestrator |
+9 -3
View File
@@ -263,9 +263,15 @@ The standalone `vbox` command still works and forwards to `pos docker vbox` (see
| Command | File | Purpose | Configuration |
|---------|------|---------|---------------|
| `pos media mp3 <url>` | `bin/pos-media-mp3` | Download audio as MP3 via yt-dlp, with thumbnail + metadata | Output to `~/Music/%(title)s.%(ext)s`, `--audio-quality 0` |
| `pos media mp4 <url>` | `bin/pos-media-mp4` | Download video via yt-dlp with **interactive format selection** | Lists formats (`yt-dlp -F`), asks for a format ID, saves to `~/Videos/` |
| `pos media grab <url>` | `bin/pos-media-grab` | Auto-download a URL as audio or video (classify + route to mp3/mp4) | Domain-based classification (YouTube Music/SoundCloud/Bandcamp → audio; YouTube/Vimeo/Twitch → video); `--audio`/`--video` force the mode; `--best` default for video (non-interactive); prints a clean summary (🎵/🎬 title, path, size). Config: `GRAB_DEFAULT` (`pos config grab`, default `video`) for unknown domains |
| `pos media yt` | `bin/pos-media-yt` | YouTube download tools dispatcher (mp3/mp4/grab/ytsync/subtitles); bare invocation prints help listing subcommands | Subcommands resolved via `bin/pos-media-yt-<sub>` files |
| `pos media yt mp3 <url>` | `bin/pos-media-yt-mp3` | Download audio as MP3 via yt-dlp, with thumbnail + metadata | Output to `~/Music/%(title)s.%(ext)s`, `--audio-quality 0`; `--by-artist` organizes as `<dir>/<artist>/<title>.mp3`. Env seam: `YT_OUT_DIR` overrides the default output dir |
| `pos media yt mp4 <url>` | `bin/pos-media-yt-mp4` | Download video via yt-dlp with **interactive format selection** | Lists formats (`yt-dlp -F`), asks for a format ID, saves to `~/Videos/`; `-f/--best/--worst` skip the prompt. Env seam: `YT_OUT_DIR` overrides the default output dir. Reads stdin (interactive format pick) → in `INTERACTIVE_CMDS` |
| `pos media yt grab <url>` | `bin/pos-media-yt-grab` | Auto-download a URL as audio or video (classify + route to yt-mp3/yt-mp4) | Domain-based classification (YouTube Music/SoundCloud/Bandcamp → audio; YouTube/Vimeo/Twitch → video); `--audio`/`--video` force the mode; `--best` default for video (non-interactive); prints a clean summary (🎵/🎬 title, path, size). Config: `GRAB_DEFAULT` (`pos config grab`, default `video`) for unknown domains |
| `pos media yt ytsync [add\|sync\|list\|remove]` | `bin/pos-media-yt-ytsync` | Thin forwarder → `pos media ytsync` (incremental YouTube channel/playlist sync into `~/Videos`) | See `bin/pos-media-ytsync` below |
| `pos media yt subtitles <url>` | `bin/pos-media-yt-subtitles` | Extract subtitles/captions from a URL (yt-dlp) | Default `--write-subs --write-auto-subs --sub-langs best`; `--lang en,ar` (comma = single `--sub-langs` arg); `--format srt` (default) / `vtt` / `txt` (srt→txt post-conversion); `--auto-only` drops manual subs; `--list-subs` probe; output to current dir (`-o ./%(title)s.%(sub_lang)s.%(ext)s`), `--output <dir>` overrides. Non-interactive; needs only yt-dlp (no ffmpeg) |
| `pos media mp3 <url>` | `bin/pos-media-mp3` | **Forwarder**`pos media yt mp3` (backward-compat alias) | Legacy spelling still works |
| `pos media mp4 <url>` | `bin/pos-media-mp4` | **Forwarder**`pos media yt mp4` (backward-compat alias) | Legacy spelling still works |
| `pos media grab <url>` | `bin/pos-media-grab` | **Forwarder**`pos media yt grab` (backward-compat alias) | Legacy spelling still works |
| `pos media sync [--mp3\|--mp4]` | `bin/pos-media-sync` | Incremental Music → USB sync (add/update only — never deletes) | Copies mp3/mp4 from `$HOME/Music` (or `--source <dir>`) into `<usb>/Music/`, preserving the tree; missing or changed (size/mtime) files are copied, identical ones skipped. Same USB detection as `pos system backup` (lsblk TRAN + lsusb/by-id, mount offer for unmounted sticks, multi-stick picker). `--mp3`/`--mp4` filter by extension, neither = both; `--dry-run` previews. Config: `MEDIA_SYNC_SOURCE`, `MEDIA_SYNC_DEST`, shared `USB_MOUNT_BASE`/`USB_BYID` from `~/.config/linux_post_install/system.env`. Result notified via `lib/notify.sh`. Bare invocation on a terminal (or the `menu` subcommand) opens an interactive menu wrapping these actions (sync now mp3+mp4, dry-run preview, mp3-only, mp4-only, change source folder); flags stay scriptable |
| `pos media ytsync [add\|sync\|list\|remove]` | `bin/pos-media-ytsync` | Incremental YouTube channel/playlist sync — first run asks for a URL (bare invocation = interactive menu; empty state goes straight to the prompt), repeat runs fetch only new videos | One yt-dlp call per new video (`bestvideo*+bestaudio/best` → MP4, metadata/chapters/thumbnail embedded, `--no-overwrites`, `--windows-filenames --trim-filenames 120`); per-source `--download-archive` (`~/.local/share/linux_post_install/ytsync/archive/<slug>.txt`) makes runs crash-safe and idempotent; registry tracks slug/type/url/subdir. Verbs never prompt (scheduler/timer safe); non-tty interactive entry prints a guard line and exits 0. `--dry-run` probes + plans with zero writes. Notify digest only when new>0 or failed>0 via `lib/notify.sh`. Config: `YTSYNC_VIDEOS_DIR`, `YTSYNC_EXTRA_ARGS` via `pos config ytsync`; automate with `pos system schedule` (`COMMAND=pos media ytsync sync`, `NOTIFY=never`) |
+68 -27
View File
@@ -3,26 +3,34 @@
Download audio and video from the web via `yt-dlp`, auto-classify URLs,
sync your library to a USB stick, and keep YouTube channels incrementally
up to date.
Tools: `grab`, `mp3`, `mp4`, `sync`, `ytsync`.
Tools: `yt mp3`, `yt mp4`, `yt grab`, `yt subtitles`, `sync`, `ytsync`.
The YouTube download tools live under `pos media yt` (`mp3`, `mp4`, `grab`,
`subtitles`, `ytsync`). The legacy standalone names `pos media mp3`, `pos media
mp4` and `pos media grab` still work — they are thin forwarders to the `yt`
forms.
| Tool | What it does |
|------|--------------|
| `pos media grab` | Auto-classify URL and download as audio or video |
| `pos media mp3` | Download audio, convert to MP3 |
| `pos media mp4` | Download video with smart/interactive format selection |
| `pos media yt mp3` | Download audio, convert to MP3 |
| `pos media yt mp4` | Download video with smart/interactive format selection |
| `pos media yt grab` | Auto-classify URL and download as audio or video |
| `pos media yt subtitles` | Extract subtitles/captions from a URL |
| `pos media yt ytsync` | Forwarder → `pos media ytsync` |
| `pos media sync` | Incrementally copy `~/Music` onto a USB stick (mp3/mp4) |
| `pos media ytsync` | Track YouTube channels/playlists and download only new videos into `~/Videos` |
Requires `yt-dlp` and `ffmpeg` (`sudo apt install yt-dlp ffmpeg`); the tools
fail with a clean error message instead of a raw `command not found` if either
is missing.
Requires `yt-dlp` and `ffmpeg` (`sudo apt install yt-dlp ffmpeg`) for `mp3`/
`mp4`/`grab`; `subtitles` needs only `yt-dlp` (no ffmpeg). The tools fail with a
clean error message instead of a raw `command not found` if a dependency is
missing.
---
## `pos media mp3` — audio as MP3
## `pos media yt mp3` — audio as MP3
```bash
pos media mp3 <url>
pos media yt mp3 <url> # or the legacy: pos media mp3 <url>
```
Extracts and converts the audio track to MP3 in `~/Music/`. With `--by-artist`
@@ -30,8 +38,8 @@ files land in `~/Music/<artist>/<title>.mp3` (falls back to the uploader name
when there's no artist tag), so a library stays organized.
```bash
pos media mp3 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp3 --by-artist https://youtu.be/dQw4w9WgXcQ
pos media yt mp3 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp3 --by-artist https://youtu.be/dQw4w9WgXcQ
```
MP3s are encoded at best quality with title/artist/album/date/chapters embedded
@@ -48,15 +56,15 @@ cover art as a JPEG thumbnail. Existing files are never overwritten.
**Recipe:** batch — loop over a list of URLs:
```bash
while read -r url; do pos media mp3 --by-artist "$url"; done < urls.txt
while read -r url; do pos media yt mp3 --by-artist "$url"; done < urls.txt
```
---
## `pos media mp4` — video with smart format choice
## `pos media yt mp4` — video with smart format choice
```bash
pos media mp4 <url>
pos media yt mp4 <url> # or the legacy: pos media mp4 <url>
```
Without a format flag, the available formats are shown as a **short curated
@@ -77,8 +85,8 @@ Non-interactive (scripting-friendly):
| `--dry-run` | Print the yt-dlp command without running it |
```bash
pos media mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ
```
Videos merge to MP4 with metadata, chapters, subtitles (all languages) and the
@@ -90,13 +98,13 @@ video+audio and merges them.
---
## `pos media grab` — auto-classify URL and download
## `pos media yt grab` — auto-classify URL and download
```bash
pos media grab <url>
pos media yt grab <url> # or the legacy: pos media grab <url>
```
Smart URL classifier that routes to `pos media mp3` or `pos media mp4`
Smart URL classifier that routes to `pos media yt mp3` or `pos media yt mp4`
automatically based on the domain. Send a URL from your phone via Telegram and
the bot downloads it to the right place without you thinking about it.
@@ -116,16 +124,16 @@ unrecognized domains is `video` — change it with `pos config grab` or set
`GRAB_DEFAULT=audio` in `~/.config/linux_post_install/grab.env`.
```bash
pos media grab https://music.youtube.com/watch?v=abc # → ~/Music
pos media grab https://youtube.com/watch?v=xyz # → ~/Videos
pos media grab --audio https://vimeo.com/123 # force mp3
pos media grab --worst https://youtu.be/abc # lowest quality
pos media grab --dry-run https://soundcloud.com/artist/track # preview only
pos media yt grab https://music.youtube.com/watch?v=abc # → ~/Music
pos media yt grab https://youtube.com/watch?v=xyz # → ~/Videos
pos media yt grab --audio https://vimeo.com/123 # force mp3
pos media yt grab --worst https://youtu.be/abc # lowest quality
pos media yt grab --dry-run https://soundcloud.com/artist/track # preview only
```
Non-interactive by design — `pos media mp4` receives `--best` by default so it
never prompts for a format (critical for Telegram bot context where there's no
TTY). Pass `--worst` if you want the smallest file.
Non-interactive by design — `pos media yt mp4` receives `--best` by default so
it never prompts for a format (critical for Telegram bot context where there's
no TTY). Pass `--worst` if you want the smallest file.
| Flag | Meaning |
|------|---------|
@@ -140,6 +148,39 @@ TTY). Pass `--worst` if you want the smallest file.
---
## `pos media yt subtitles` — extract subtitles/captions
```bash
pos media yt subtitles <url>
```
Downloads subtitles/captions from a URL via yt-dlp. Fetches manual captions and
auto-generated captions by default (`--write-subs --write-auto-subs
--sub-langs best` — "best" picks the manually-created track when available,
otherwise the auto one). Output files land in the current directory as
`<title>.<lang>.<ext>`.
```bash
pos media yt subtitles https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt subtitles --lang en https://youtu.be/dQw4w9WgXcQ
pos media yt subtitles --lang en,ar --format txt https://youtube.com/watch?v=dQw4w9WgXcQ
```
| Flag | Meaning |
|------|---------|
| `--lang <list>` | Subtitle languages, comma-separated (default `best`) — `en,ar` is passed as ONE `--sub-langs` arg |
| `--format <fmt>` | `srt` (default) / `vtt` / `txt``txt` converts srt→txt (timestamps, seq numbers and HTML tags stripped) |
| `--auto-only` | Only auto-generated captions (no manual subs) |
| `-o, --output <dir>` | Output directory (default: current directory) |
| `--list-subs` | List available subtitles for the URL and exit (probe only, no download) |
| `--no-playlist` | Download only the single video |
| `--dry-run` | Print the yt-dlp command without running it |
If a video has no available subtitles the tool reports
`unavailable subtitles for this video (try --list-subs to check)`.
---
## `pos media sync` — music onto a USB stick
```bash
+1 -1
View File
@@ -266,7 +266,7 @@ MAIN_LOG="$LOG_DIR/pos.log"
log_cmd() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $* → exit $2" >> "$MAIN_LOG"; }
# Commands that read from stdin interactively — only log invocation
INTERACTIVE_CMDS="docker-compose docker-vbox network-hotspot system-firewall media-mp4 media-sync system-backup system-uninstall share-usb-server share-smb-server share-smb-client share-nfs-client share-nfs-server communication-telegram-listener communication-matrix-listener ai ai-gemini ai-openrouter ai-llamacpp ai-alias system-schedule entertainment-config config"
INTERACTIVE_CMDS="docker-compose docker-vbox network-hotspot system-firewall media-mp4 media-yt-mp4 media-sync system-backup system-uninstall share-usb-server share-smb-server share-smb-client share-nfs-client share-nfs-server communication-telegram-listener communication-matrix-listener ai ai-gemini ai-openrouter ai-llamacpp ai-alias system-schedule entertainment-config config"
for ((i=n-1; i>=0; i--)); do
cmd="pos"
+2 -216
View File
@@ -1,219 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media grab — Auto-download URL as audio or video (classify + route)
# POS_FLAGS: --audio --video --best --worst --output --no-playlist --cookies --dry-run
# POS_CONFIG: grab | grab.env | GRAB_DEFAULT=:Default mode for unknown domains (video or audio, default video)
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
# Shared config loader (canonical KEY=VALUE parser, env-wins precedence)
source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh"
# Load grab.env config (env-seam: GRAB_DEFAULT)
load_grab_config() {
load_env_file "$CONFIG_DIR/grab.env"
}
load_grab_config
# ── URL classification ─────────────────────────────────────────
classify_url() {
local url="$1" mode="${GRAB_DEFAULT:-video}"
case "$url" in
*music.youtube.com*) echo "audio" ;;
*soundcloud.com*) echo "audio" ;;
*bandcamp.com*) echo "audio" ;;
*youtube.com*|*youtu.be*) echo "video" ;;
*vimeo.com*) echo "video" ;;
*twitch.tv*) echo "video" ;;
*) echo "$mode" ;;
esac
}
usage() {
cat <<EOF
Usage: pos media grab [options] <url>
Auto-download a URL as audio or video. Classifies the domain and delegates
to 'pos media mp3' (audio) or 'pos media mp4' (video).
Options:
--audio Force audio (mp3) download
--video Force video (mp4) download
--best Best quality for video (default for non-interactive)
--worst Lowest quality for video
-o, --output <dir> Output directory (passed to mp3/mp4)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the command that would run, don't execute
-h, --help This help
Examples:
pos media grab https://music.youtube.com/watch?v=abc
pos media grab https://youtube.com/watch?v=xyz
pos media grab --audio https://vimeo.com/123
pos media grab --worst https://youtu.be/abc
pos media grab --dry-run https://soundcloud.com/artist/track
EOF
exit 0
}
# ── Arg parsing ────────────────────────────────────────────────
URL=""
FORCE_AUDIO=0
FORCE_VIDEO=0
BEST=0
WORST=0
DRY_RUN=0
EXTRA_ARGS=()
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
--audio) FORCE_AUDIO=1; shift ;;
--video) FORCE_VIDEO=1; shift ;;
--best) BEST=1; shift ;;
--worst) WORST=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "pos media grab: --output needs a value"
EXTRA_ARGS+=(--output "$2"); shift 2 ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "pos media grab: --cookies needs a value"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "pos media grab: Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "pos media grab: Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
# Validate URL scheme
case "$URL" in
http://*|https://*) ;;
*) err "pos media grab: not a valid URL: $URL (must start with http:// or https://)" ;;
esac
# Validate mutually exclusive overrides
[ "$FORCE_AUDIO" -eq 1 ] && [ "$FORCE_VIDEO" -eq 1 ] && \
err "pos media grab: --audio and --video are mutually exclusive"
[ "$BEST" -eq 1 ] && [ "$WORST" -eq 1 ] && \
err "pos media grab: --best and --worst are mutually exclusive"
# ── Classification ─────────────────────────────────────────────
mode=""
if [ "$FORCE_AUDIO" -eq 1 ]; then
mode="audio"
elif [ "$FORCE_VIDEO" -eq 1 ]; then
mode="video"
else
mode="$(classify_url "$URL")"
fi
# ── Build delegated command ────────────────────────────────────
DELEGATE_ARGS=()
if [ "$mode" = "audio" ]; then
DELEGATE_ARGS=(pos media mp3 "${EXTRA_ARGS[@]}")
else
# mp4 route: --best by default (non-interactive), --worst if user passes it
if [ "$WORST" -eq 1 ]; then
DELEGATE_ARGS=(pos media mp4 --worst "${EXTRA_ARGS[@]}")
else
DELEGATE_ARGS=(pos media mp4 --best "${EXTRA_ARGS[@]}")
fi
fi
# ── Dry run ────────────────────────────────────────────────────
if [ "$DRY_RUN" -eq 1 ]; then
echo "${DELEGATE_ARGS[*]} $URL"
exit 0
fi
# ── Execute ────────────────────────────────────────────────────
output=""
rc=0
if output=$("${DELEGATE_ARGS[@]}" "$URL" 2>&1); then
rc=0
else
rc=$?
fi
if [ "$rc" -ne 0 ]; then
# Summarize stderr for the user
summary="$(printf '%s' "$output" | grep -i 'error\|fail' | head -1 || true)"
[ -z "$summary" ] && summary="exit code $rc"
err "pos media grab: ❌ Download failed: $summary"
fi
# ── Metadata + summary ────────────────────────────────────────
title=""
duration=""
file_path=""
# Determine expected output directory
if [ "$mode" = "audio" ]; then
out_dir="$HOME/Music"
for (( i=0; i<${#EXTRA_ARGS[@]}; i++ )); do
if [ "${EXTRA_ARGS[$i]}" = "--output" ] && [ $(( i + 1 )) -lt ${#EXTRA_ARGS[@]} ]; then
out_dir="${EXTRA_ARGS[$(( i + 1 ))]}"
break
fi
done
file_ext="mp3"
else
out_dir="$HOME/Videos"
for (( i=0; i<${#EXTRA_ARGS[@]}; i++ )); do
if [ "${EXTRA_ARGS[$i]}" = "--output" ] && [ $(( i + 1 )) -lt ${#EXTRA_ARGS[@]} ]; then
out_dir="${EXTRA_ARGS[$(( i + 1 ))]}"
break
fi
done
file_ext="mp4"
fi
# Fetch metadata (fast, no download)
if command -v yt-dlp &>/dev/null; then
meta="$(yt-dlp --print title --print duration_string --no-warnings "$URL" 2>/dev/null || true)"
title="$(printf '%s' "$meta" | sed -n '1p')"
duration="$(printf '%s' "$meta" | sed -n '2p')"
fi
# Find the downloaded file (most recent matching extension in out_dir)
if [ -d "$out_dir" ]; then
file_path="$(find "$out_dir" -maxdepth 1 -name "*.$file_ext" -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2- || true)"
fi
# Build summary
[ -z "$title" ] && title="$(basename "$URL" | sed 's/[?#].*//')"
[ -z "$duration" ] && duration="?"
if [ "$mode" = "audio" ]; then
emoji="🎵"
else
emoji="🎬"
fi
echo "$emoji Downloaded: $title ($duration)"
if [ -n "$file_path" ] && [ -f "$file_path" ]; then
file_size="$(stat --printf='%s' "$file_path" 2>/dev/null || echo "0")"
# Format size in human-readable form
if [ "$file_size" -ge 1073741824 ]; then
size_human="$(awk "BEGIN { printf \"%.1f GB\", $file_size / 1073741824 }")"
elif [ "$file_size" -ge 1048576 ]; then
size_human="$(awk "BEGIN { printf \"%.1f MB\", $file_size / 1048576 }")"
elif [ "$file_size" -ge 1024 ]; then
size_human="$(awk "BEGIN { printf \"%.1f KB\", $file_size / 1024 }")"
else
size_human="${file_size} B"
fi
# Show path relative to HOME
rel_path="${file_path/#$HOME/\~}"
echo "📁 $rel_path ($size_human)"
else
echo "📁 $out_dir/ ($file_ext)"
fi
# -h|--help forwarded via exec below
exec pos media yt grab "$@"
+2 -83
View File
@@ -1,86 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media mp3 — Download audio as MP3 (yt-dlp)
# POS_FLAGS: --output --no-playlist --cookies --by-artist --dry-run
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
# --dry-run is parsed before the deps guards (below): it previews the yt-dlp
# command without requiring yt-dlp/ffmpeg to be installed.
DRY_RUN=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
done
# Deps guards sit before -h|--help (help also errors on a box missing the deps).
if [ "$DRY_RUN" -eq 0 ]; then
command -v yt-dlp &>/dev/null || err "yt-dlp not found — install it with: sudo apt install yt-dlp"
command -v ffmpeg &>/dev/null || err "ffmpeg not found (needed for MP3 conversion) — install it with: sudo apt install ffmpeg"
fi
OUT_DIR="$HOME/Music"
BY_ARTIST=0
EXTRA_ARGS=()
usage() {
cat <<EOF
Usage: pos media mp3 [options] <url>
Download audio from a URL and convert to MP3 via yt-dlp.
Options:
-o, --output <dir> Output directory (default: $OUT_DIR)
--by-artist Organize as <dir>/<artist>/<title>.mp3
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media mp3 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp3 --by-artist --no-playlist https://youtu.be/dQw4w9WgXcQ
pos media mp3 --cookies ~/cookies.txt https://vimeo.com/123
EOF
exit 0
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--by-artist) BY_ARTIST=1; shift ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "--cookies needs a value"
[ -f "$2" ] || err "cookies file not found: $2"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
if [ "$BY_ARTIST" -eq 1 ]; then
OUT_TEMPLATE="$OUT_DIR/%(artist,uploader)s/%(title)s.%(ext)s"
else
OUT_TEMPLATE="$OUT_DIR/%(title)s.%(ext)s"
fi
args=(yt-dlp -x --audio-format mp3 --audio-quality 0
--embed-metadata --embed-chapters
--embed-thumbnail --convert-thumbnails jpg
--parse-metadata "%(artist,uploader)s:%(artist)s"
--no-overwrites
-o "$OUT_TEMPLATE"
"${EXTRA_ARGS[@]}" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
echo "yt-dlp ${args[*]:1}"
else
spawn "downloading audio → $OUT_DIR" "${args[@]}"
fi
# -h|--help forwarded via exec below
exec pos media yt mp3 "$@"
+2 -129
View File
@@ -1,132 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media mp4 — Download video as MP4 (smart/interactive format select)
# POS_FLAGS: --format --best --worst --output --no-playlist --cookies --dry-run
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
# --dry-run is parsed before the deps guards (below): it previews the yt-dlp
# command without requiring yt-dlp/ffmpeg to be installed.
DRY_RUN=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
done
# Deps guards sit before -h|--help (help also errors on a box missing the deps).
if [ "$DRY_RUN" -eq 0 ]; then
command -v yt-dlp &>/dev/null || err "yt-dlp not found — install it with: sudo apt install yt-dlp"
command -v ffmpeg &>/dev/null || err "ffmpeg not found (needed for MP4 merge) — install it with: sudo apt install ffmpeg"
fi
OUT_DIR="$HOME/Videos"
FORMAT=""
BEST=0
WORST=0
EXTRA_ARGS=()
usage() {
cat <<EOF
Usage: pos media mp4 [options] <url>
Download video from a URL via yt-dlp. Without -f/--best/--worst, formats are
listed and picked interactively.
Options:
-f, --format <id> Download that format id directly (no prompt)
--best Best video + audio (no prompt)
--worst Lowest quality (no prompt)
-o, --output <dir> Output directory (default: $OUT_DIR)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media mp4 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ
pos media mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ
EOF
exit 0
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
-f|--format)
[ $# -ge 2 ] || err "--format needs a value"
FORMAT="$2"; shift 2 ;;
--best) BEST=1; shift ;;
--worst) WORST=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "--cookies needs a value"
[ -f "$2" ] || err "cookies file not found: $2"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
[ -n "$FORMAT" ] && { [ "$BEST" -eq 1 ] || [ "$WORST" -eq 1 ]; } \
&& err "--format and --best/--worst are mutually exclusive"
[ "$BEST" -eq 1 ] && [ "$WORST" -eq 1 ] && err "--best and --worst are mutually exclusive"
# Resolve the format selector: explicit flag → smart preset → interactive.
# The curated table goes to stderr; stdout carries ONLY the chosen format id
# (same ui_pick lesson — a $(...) capture must not swallow the menu).
fmt_spec=""
if [ -n "$FORMAT" ]; then
fmt_spec="$FORMAT"
elif [ "$BEST" -eq 1 ]; then
fmt_spec="bestvideo*+bestaudio/best"
elif [ "$WORST" -eq 1 ]; then
fmt_spec="worst"
else
table="$(yt-dlp -F "$URL" 2>&1)" || {
echo "ERROR: no formats listed (unsupported site or age-gated content)" >&2
exit 1
}
printf '%s' "$table" | awk -F'[[:space:]][[:space:]]+' '
NR <= 2 { print; next }
$0 ~ /^-+$/ { print; next }
$0 ~ /audio only/ && $0 !~ /storyboard/ && $1 !~ /^sb/ { print " [audio] " $0; next }
$0 ~ /video only/ && $1 !~ /^sb/ { print " [video] " $0; next }
$1 ~ /^[0-9]+$/ && $2 ~ /^(mp4|webm)$/ && $3 ~ /x/ { print " [combo] " $0 }
' >&2
echo >&2
read -rp "Format ID ('best'/'worst'/empty = best): " chosen
case "${chosen:-best}" in
best) fmt_spec="bestvideo*+bestaudio/best" ;;
worst) fmt_spec="worst" ;;
*)
if printf '%s' "$table" | grep -qE "^[[:space:]]*${chosen}[[:space:]]"; then
fmt_spec="$chosen"
else
echo "ERROR: unknown format id '$chosen'" >&2
exit 1
fi
;;
esac
fi
args=(yt-dlp -f "$fmt_spec"
--merge-output-format mp4
--embed-metadata --embed-chapters
--embed-subs --sub-langs all
--embed-thumbnail
--no-overwrites
-o "$OUT_DIR/%(title)s.%(ext)s"
"${EXTRA_ARGS[@]}" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
echo "yt-dlp ${args[*]:1}"
else
spawn "downloading video → $OUT_DIR" "${args[@]}"
fi
# -h|--help forwarded via exec below
exec pos media yt mp4 "$@"
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt — YouTube download tools (mp3/mp4/grab/ytsync/subtitles)
# POS_SUBCMDS: mp3 mp4 grab ytsync subtitles
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
usage() {
cat <<'EOF'
Usage: pos media yt <command> [args]
YouTube download tools: audio, video, auto-grab, subtitles, channel sync.
Commands:
mp3 <url> Download audio as MP3
mp4 <url> Download video as MP4 (interactive format select)
grab <url> Auto-download as audio or video
ytsync [cmd] Incremental YouTube channel sync
subtitles <url> Extract subtitles/captions
Options:
-h, --help This help
Run 'pos media yt <command> --help' for details.
EOF
exit 0
}
case "${1:-}" in
-h|--help|"") usage ;;
esac
# Subcommands are resolved by the dispatcher via pos-media-yt-<sub> files.
# This file only handles bare 'pos media yt' (help) and unknown args.
err "unknown yt command: $1 (see 'pos media yt --help')"
+208
View File
@@ -0,0 +1,208 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-grab — Auto-download URL as audio or video (classify + route)
# POS_FLAGS: --audio --video --best --worst --output --no-playlist --cookies --dry-run
# POS_CONFIG: grab | grab.env | GRAB_DEFAULT=:Default mode for unknown domains (video or audio, default video)
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
# Shared config loader (canonical KEY=VALUE parser, env-wins precedence)
source "$(dirname "$0")/../lib/config-ui.sh" 2>/dev/null || source "$(dirname "$0")/config-ui.sh"
# Shared yt helpers (classify_url, yt_validate_url, yt_check_deps, yt_echo_cmd)
source "$(dirname "$0")/../lib/yt-lib.sh" 2>/dev/null || source "$(dirname "$0")/yt-lib.sh"
# Load grab.env config (env-seam: GRAB_DEFAULT)
load_grab_config() {
load_env_file "$CONFIG_DIR/grab.env"
}
load_grab_config
usage() {
cat <<EOF
Usage: pos media yt grab [options] <url>
Auto-download a URL as audio or video. Classifies the domain and delegates
to 'pos media yt mp3' (audio) or 'pos media yt mp4' (video).
Options:
--audio Force audio (mp3) download
--video Force video (mp4) download
--best Best quality for video (default for non-interactive)
--worst Lowest quality for video
-o, --output <dir> Output directory (passed to mp3/mp4)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the command that would run, don't execute
-h, --help This help
Examples:
pos media yt grab https://music.youtube.com/watch?v=abc
pos media yt grab https://youtube.com/watch?v=xyz
pos media yt grab --audio https://vimeo.com/123
pos media yt grab --worst https://youtu.be/abc
pos media yt grab --dry-run https://soundcloud.com/artist/track
The legacy spelling 'pos media grab' still works (forwarder) and is equivalent.
EOF
exit 0
}
# ── Arg parsing ────────────────────────────────────────────────
URL=""
FORCE_AUDIO=0
FORCE_VIDEO=0
BEST=0
WORST=0
DRY_RUN=0
EXTRA_ARGS=()
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
--audio) FORCE_AUDIO=1; shift ;;
--video) FORCE_VIDEO=1; shift ;;
--best) BEST=1; shift ;;
--worst) WORST=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "yt-grab: --output needs a value"
EXTRA_ARGS+=(--output "$2"); shift 2 ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "yt-grab: --cookies needs a value"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "yt-grab: Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "yt-grab: Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
# Validate URL scheme
yt_validate_url "$URL" 2>/dev/null || \
err "yt-grab: not a valid URL: $URL (must start with http:// or https://)"
# Validate mutually exclusive overrides
[ "$FORCE_AUDIO" -eq 1 ] && [ "$FORCE_VIDEO" -eq 1 ] && \
err "yt-grab: --audio and --video are mutually exclusive"
[ "$BEST" -eq 1 ] && [ "$WORST" -eq 1 ] && \
err "yt-grab: --best and --worst are mutually exclusive"
# ── Classification ─────────────────────────────────────────────
mode=""
if [ "$FORCE_AUDIO" -eq 1 ]; then
mode="audio"
elif [ "$FORCE_VIDEO" -eq 1 ]; then
mode="video"
else
mode="$(classify_url "$URL")"
fi
# ── Build delegated command ────────────────────────────────────
DELEGATE_ARGS=()
if [ "$mode" = "audio" ]; then
DELEGATE_ARGS=(pos media yt mp3 "${EXTRA_ARGS[@]}")
else
# mp4 route: --best by default (non-interactive), --worst if user passes it
if [ "$WORST" -eq 1 ]; then
DELEGATE_ARGS=(pos media yt mp4 --worst "${EXTRA_ARGS[@]}")
else
DELEGATE_ARGS=(pos media yt mp4 --best "${EXTRA_ARGS[@]}")
fi
fi
# ── Dry run ────────────────────────────────────────────────────
if [ "$DRY_RUN" -eq 1 ]; then
echo "${DELEGATE_ARGS[*]} $URL"
exit 0
fi
# ── Execute ────────────────────────────────────────────────────
output=""
rc=0
if output=$("${DELEGATE_ARGS[@]}" "$URL" 2>&1); then
rc=0
else
rc=$?
fi
if [ "$rc" -ne 0 ]; then
# Summarize stderr for the user
summary="$(printf '%s' "$output" | grep -i 'error\|fail' | head -1 || true)"
[ -z "$summary" ] && summary="exit code $rc"
err "yt-grab: ❌ Download failed: $summary"
fi
# ── Metadata + summary ────────────────────────────────────────
title=""
duration=""
file_path=""
# Determine expected output directory
if [ "$mode" = "audio" ]; then
out_dir="$HOME/Music"
for (( i=0; i<${#EXTRA_ARGS[@]}; i++ )); do
if [ "${EXTRA_ARGS[$i]}" = "--output" ] && [ $(( i + 1 )) -lt ${#EXTRA_ARGS[@]} ]; then
out_dir="${EXTRA_ARGS[$(( i + 1 ))]}"
break
fi
done
file_ext="mp3"
else
out_dir="$HOME/Videos"
for (( i=0; i<${#EXTRA_ARGS[@]}; i++ )); do
if [ "${EXTRA_ARGS[$i]}" = "--output" ] && [ $(( i + 1 )) -lt ${#EXTRA_ARGS[@]} ]; then
out_dir="${EXTRA_ARGS[$(( i + 1 ))]}"
break
fi
done
file_ext="mp4"
fi
# Fetch metadata (fast, no download)
if command -v yt-dlp &>/dev/null; then
meta="$(yt-dlp --print title --print duration_string --no-warnings "$URL" 2>/dev/null || true)"
title="$(printf '%s' "$meta" | sed -n '1p')"
duration="$(printf '%s' "$meta" | sed -n '2p')"
fi
# Find the downloaded file (most recent matching extension in out_dir)
if [ -d "$out_dir" ]; then
file_path="$(find "$out_dir" -maxdepth 1 -name "*.$file_ext" -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2- || true)"
fi
# Build summary
[ -z "$title" ] && title="$(basename "$URL" | sed 's/[?#].*//')"
[ -z "$duration" ] && duration="?"
if [ "$mode" = "audio" ]; then
emoji="🎵"
else
emoji="🎬"
fi
echo "$emoji Downloaded: $title ($duration)"
if [ -n "$file_path" ] && [ -f "$file_path" ]; then
file_size="$(stat --printf='%s' "$file_path" 2>/dev/null || echo "0")"
# Format size in human-readable form
if [ "$file_size" -ge 1073741824 ]; then
size_human="$(awk "BEGIN { printf \"%.1f GB\", $file_size / 1073741824 }")"
elif [ "$file_size" -ge 1048576 ]; then
size_human="$(awk "BEGIN { printf \"%.1f MB\", $file_size / 1048576 }")"
elif [ "$file_size" -ge 1024 ]; then
size_human="$(awk "BEGIN { printf \"%.1f KB\", $file_size / 1024 }")"
else
size_human="${file_size} B"
fi
# Show path relative to HOME
rel_path="${file_path/#$HOME/\~}"
echo "📁 $rel_path ($size_human)"
else
echo "📁 $out_dir/ ($file_ext)"
fi
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-mp3 — Download audio as MP3 (yt-dlp)
# POS_FLAGS: --output --no-playlist --cookies --by-artist --dry-run
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
source "$(dirname "$0")/../lib/yt-lib.sh" 2>/dev/null || source "$(dirname "$0")/yt-lib.sh"
# --dry-run is parsed before the deps guards (below): it previews the yt-dlp
# command without requiring yt-dlp/ffmpeg to be installed.
DRY_RUN=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
done
# Deps guards sit before -h|--help (help also errors on a box missing the deps).
# yt_check_deps with dry-run=1 skips the dep checks (dry-run preview needs none).
yt_check_deps "$DRY_RUN" "MP3 conversion"
# Env seam: YT_OUT_DIR overrides the default (keeps HOME sandbox-compatible).
OUT_DIR="${YT_OUT_DIR:-$HOME/Music}"
BY_ARTIST=0
EXTRA_ARGS=()
usage() {
cat <<EOF
Usage: pos media yt mp3 [options] <url>
Download audio from a URL and convert to MP3 via yt-dlp.
Options:
-o, --output <dir> Output directory (default: $OUT_DIR)
--by-artist Organize as <dir>/<artist>/<title>.mp3
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media yt mp3 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp3 --by-artist --no-playlist https://youtu.be/dQw4w9WgXcQ
pos media yt mp3 --cookies ~/cookies.txt https://vimeo.com/123
EOF
exit 0
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--by-artist) BY_ARTIST=1; shift ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "--cookies needs a value"
[ -f "$2" ] || err "cookies file not found: $2"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
if [ "$BY_ARTIST" -eq 1 ]; then
OUT_TEMPLATE="$OUT_DIR/%(artist,uploader)s/%(title)s.%(ext)s"
else
OUT_TEMPLATE="$OUT_DIR/%(title)s.%(ext)s"
fi
args=(yt-dlp -x --audio-format mp3 --audio-quality 0
--embed-metadata --embed-chapters
--embed-thumbnail --convert-thumbnails jpg
--parse-metadata "%(artist,uploader)s:%(artist)s"
--no-overwrites
-o "$OUT_TEMPLATE"
"${EXTRA_ARGS[@]}" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
echo "yt-dlp ${args[*]:1}"
else
spawn "downloading audio → $OUT_DIR" "${args[@]}"
fi
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-mp4 — Download video as MP4 (smart/interactive format select)
# POS_FLAGS: --format --best --worst --output --no-playlist --cookies --dry-run
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
source "$(dirname "$0")/../lib/yt-lib.sh" 2>/dev/null || source "$(dirname "$0")/yt-lib.sh"
# --dry-run is parsed before the deps guards (below): it previews the yt-dlp
# command without requiring yt-dlp/ffmpeg to be installed.
DRY_RUN=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
done
# Deps guards sit before -h|--help (help also errors on a box missing the deps).
# yt_check_deps with dry-run=1 skips the dep checks (dry-run preview needs none).
yt_check_deps "$DRY_RUN" "MP4 merge"
# Env seam: YT_OUT_DIR overrides the default (keeps HOME sandbox-compatible).
OUT_DIR="${YT_OUT_DIR:-$HOME/Videos}"
FORMAT=""
BEST=0
WORST=0
EXTRA_ARGS=()
usage() {
cat <<EOF
Usage: pos media yt mp4 [options] <url>
Download video from a URL via yt-dlp. Without -f/--best/--worst, formats are
listed and picked interactively.
Options:
-f, --format <id> Download that format id directly (no prompt)
--best Best video + audio (no prompt)
--worst Lowest quality (no prompt)
-o, --output <dir> Output directory (default: $OUT_DIR)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media yt mp4 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ
EOF
exit 0
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
-f|--format)
[ $# -ge 2 ] || err "--format needs a value"
FORMAT="$2"; shift 2 ;;
--best) BEST=1; shift ;;
--worst) WORST=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--no-playlist) EXTRA_ARGS+=(--no-playlist); shift ;;
--cookies)
[ $# -ge 2 ] || err "--cookies needs a value"
[ -f "$2" ] || err "cookies file not found: $2"
EXTRA_ARGS+=(--cookies "$2"); shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || usage
[ -n "$FORMAT" ] && { [ "$BEST" -eq 1 ] || [ "$WORST" -eq 1 ]; } \
&& err "--format and --best/--worst are mutually exclusive"
[ "$BEST" -eq 1 ] && [ "$WORST" -eq 1 ] && err "--best and --worst are mutually exclusive"
# Resolve the format selector: explicit flag → smart preset → interactive.
# The curated table goes to stderr; stdout carries ONLY the chosen format id
# (same ui_pick lesson — a $(...) capture must not swallow the menu).
fmt_spec=""
if [ -n "$FORMAT" ]; then
fmt_spec="$FORMAT"
elif [ "$BEST" -eq 1 ]; then
fmt_spec="bestvideo*+bestaudio/best"
elif [ "$WORST" -eq 1 ]; then
fmt_spec="worst"
else
table="$(yt-dlp -F "$URL" 2>&1)" || {
echo "ERROR: no formats listed (unsupported site or age-gated content)" >&2
exit 1
}
printf '%s' "$table" | awk -F'[[:space:]][[:space:]]+' '
NR <= 2 { print; next }
$0 ~ /^-+$/ { print; next }
$0 ~ /audio only/ && $0 !~ /storyboard/ && $1 !~ /^sb/ { print " [audio] " $0; next }
$0 ~ /video only/ && $1 !~ /^sb/ { print " [video] " $0; next }
$1 ~ /^[0-9]+$/ && $2 ~ /^(mp4|webm)$/ && $3 ~ /x/ { print " [combo] " $0 }
' >&2
echo >&2
read -rp "Format ID ('best'/'worst'/empty = best): " chosen
case "${chosen:-best}" in
best) fmt_spec="bestvideo*+bestaudio/best" ;;
worst) fmt_spec="worst" ;;
*)
if printf '%s' "$table" | grep -qE "^[[:space:]]*${chosen}[[:space:]]"; then
fmt_spec="$chosen"
else
echo "ERROR: unknown format id '$chosen'" >&2
exit 1
fi
;;
esac
fi
args=(yt-dlp -f "$fmt_spec"
--merge-output-format mp4
--embed-metadata --embed-chapters
--embed-subs --sub-langs all
--embed-thumbnail
--no-overwrites
-o "$OUT_DIR/%(title)s.%(ext)s"
"${EXTRA_ARGS[@]}" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
echo "yt-dlp ${args[*]:1}"
else
spawn "downloading video → $OUT_DIR" "${args[@]}"
fi
+169
View File
@@ -0,0 +1,169 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-subtitles — Extract subtitles/captions from a URL (yt-dlp)
# POS_FLAGS: --lang --format --auto-only --output --list-subs --no-playlist --dry-run
source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh"
source "$(dirname "$0")/../lib/yt-lib.sh" 2>/dev/null || source "$(dirname "$0")/yt-lib.sh"
# --dry-run / --list-subs are parsed before the deps guards (below): dry-run
# previews the yt-dlp command without requiring yt-dlp to be installed.
DRY_RUN=0
LIST_SUBS=0
for arg in "$@"; do
[ "$arg" = "--dry-run" ] && DRY_RUN=1
[ "$arg" = "--list-subs" ] && LIST_SUBS=1
done
# Deps guards sit before -h|--help. Subtitles needs only yt-dlp (no ffmpeg).
# dry-run skips the deps check entirely (preview needs no binary).
if [ "$DRY_RUN" -eq 0 ]; then
command -v yt-dlp &>/dev/null || err "yt-dlp not found — install it with: sudo apt install yt-dlp"
fi
LANG_SPEC=""
FMT="srt"
AUTO_ONLY=0
OUT_DIR=""
NO_PLAYLIST=0
usage() {
cat <<EOF
Usage: pos media yt subtitles [options] <url>
Extract subtitles/captions from a URL via yt-dlp. Fetches manual captions and
auto-generated captions by default.
Options:
--lang <list> Subtitle languages (comma-separated), default: best
--format <fmt> Output format: srt (default), vtt, or txt
--auto-only Only auto-generated captions (no manual subs)
-o, --output <dir> Output directory (default: current directory)
--list-subs List available subtitles and exit (probe only)
--no-playlist Download only the single video
--dry-run Print the yt-dlp command without running it
-h, --help This help
Examples:
pos media yt subtitles https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt subtitles --lang en,ar https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt subtitles --format txt https://youtu.be/dQw4w9WgXcQ
pos media yt subtitles --list-subs https://youtube.com/watch?v=dQw4w9WgXcQ
EOF
exit 0
}
# convert_srt_to_txt <srt-file> — strip SRT timestamps, seq numbers, HTML tags
convert_srt_to_txt() {
local srt_file="$1" txt_file="${1%.srt}.txt"
sed '/^[0-9][0-9]:[0-9][0-9]/d; /^$/d; s/<[^>]*>//g' "$srt_file" > "$txt_file"
}
URL=""
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage ;;
--lang)
[ $# -ge 2 ] || err "--lang needs a value"
LANG_SPEC="$2"; shift 2 ;;
--format)
[ $# -ge 2 ] || err "--format needs a value"
case "$2" in
srt|vtt|txt) FMT="$2" ;;
*) err "unknown format '$2' (use srt, vtt, or txt)" ;;
esac
shift 2 ;;
--auto-only) AUTO_ONLY=1; shift ;;
-o|--output)
[ $# -ge 2 ] || err "--output needs a value"
OUT_DIR="$2"; shift 2 ;;
--list-subs) LIST_SUBS=1; shift ;;
--no-playlist) NO_PLAYLIST=1; shift ;;
--dry-run) DRY_RUN=1; shift ;;
-*) err "Unknown option: $1 (see --help)" ;;
*)
[ -z "$URL" ] && URL="$1" && shift || err "Unexpected argument: $1" ;;
esac
done
[ -n "$URL" ] || err "missing URL (see --help)"
yt_validate_url "$URL" 2>/dev/null || err "not a valid URL: $URL (must start with http:// or https://)"
# ── Build the yt-dlp command ─────────────────────────────────
if [ "$LIST_SUBS" -eq 1 ]; then
args=(yt-dlp --list-subs "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
yt_echo_cmd "${args[@]:1}"
exit 0
fi
spawn "listing subtitles" "${args[@]}"
exit 0
fi
# Output template
OUT_TEMPLATE=""
if [ -n "$OUT_DIR" ]; then
OUT_TEMPLATE="$OUT_DIR/%(title)s.%(sub_lang)s.%(ext)s"
else
OUT_TEMPLATE="./%(title)s.%(sub_lang)s.%(ext)s"
fi
args=(yt-dlp)
if [ "$AUTO_ONLY" -eq 1 ]; then
args+=(--write-auto-subs)
else
args+=(--write-subs --write-auto-subs)
fi
if [ -n "$LANG_SPEC" ]; then
args+=(--sub-langs "$LANG_SPEC")
else
args+=(--sub-langs best)
fi
case "$FMT" in
vtt) args+=(--sub-format vtt) ;;
txt) args+=(--sub-format srt) ;;
*) args+=(--sub-format srt) ;;
esac
[ "$NO_PLAYLIST" -eq 1 ] && args+=(--no-playlist)
args+=(-o "$OUT_TEMPLATE" "$URL")
if [ "$DRY_RUN" -eq 1 ]; then
yt_echo_cmd "${args[@]:1}"
exit 0
fi
# ── Execute ───────────────────────────────────────────────────
set +e
out="$(mktemp)"
err_tmp="$(mktemp)"
"${args[@]}" >"$out" 2>"$err_tmp"
rc=$?
set -e
if [ "$rc" -ne 0 ]; then
# Unavailable-subtitle detection: exit!=0 AND stderr matches no-subs patterns
if grep -qiE 'no subtitles|no captions|unable to extract|subtitles not available' "$err_tmp"; then
rm -f "$out" "$err_tmp"
err "unavailable subtitles for this video (try --list-subs to check)"
fi
# Re-surface the real yt-dlp error otherwise
sed 's/^/ /' "$err_tmp" >&2
rm -f "$out" "$err_tmp"
exit "$rc"
fi
rm -f "$out" "$err_tmp"
# ── Post-conversion for txt format ────────────────────────────
if [ "$FMT" = "txt" ]; then
search_dir="."
[ -n "$OUT_DIR" ] && search_dir="$OUT_DIR"
while IFS= read -r -d '' f; do
case "$f" in
*.srt) convert_srt_to_txt "$f" ;;
esac
done < <(find "$search_dir" -maxdepth 1 -name '*.srt' -print0 2>/dev/null || true)
fi
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
# POS: media yt-ytsync — Incrementally sync YouTube channels/playlists into ~/Videos
# -h|--help forwarded via exec below
exec pos media ytsync "$@"
+5 -3
View File
@@ -11,10 +11,11 @@ _pos_flags[communication-telegram-sender]="--type --caption --parse-mode --no-pr
_pos_flags[docker-stack]="-a --all"
_pos_flags[docker-vbox]="--dir --gpu --device --port --cpus --memory --network"
_pos_flags[entertainment-send]="--print --markdown"
_pos_flags[media-grab]="--audio --video --best --worst --output --no-playlist --cookies --dry-run"
_pos_flags[media-mp3]="--output --no-playlist --cookies --by-artist --dry-run"
_pos_flags[media-mp4]="--format --best --worst --output --no-playlist --cookies --dry-run"
_pos_flags[media-sync]="--mp3 --mp4 --source --dry-run"
_pos_flags[media-yt-grab]="--audio --video --best --worst --output --no-playlist --cookies --dry-run"
_pos_flags[media-yt-mp3]="--output --no-playlist --cookies --by-artist --dry-run"
_pos_flags[media-yt-mp4]="--format --best --worst --output --no-playlist --cookies --dry-run"
_pos_flags[media-yt-subtitles]="--lang --format --auto-only --output --list-subs --no-playlist --dry-run"
_pos_flags[media-ytsync]="--dry-run"
_pos_flags[network-checkport]="--tcp --udp --ping --no-banner --versions --timeout"
_pos_flags[network-download]="--dir --out --split --seed --force --upload --gid --tmux"
@@ -42,6 +43,7 @@ _pos_subcmds[docker-compose]="ls installed up down restart logs update config me
_pos_subcmds[docker-vbox]="create enter stop start rm ls menu"
_pos_subcmds[media-sync]="menu"
_pos_subcmds[media-ytsync]="add sync list remove"
_pos_subcmds[media-yt]="mp3 mp4 grab ytsync subtitles"
_pos_subcmds[network-download]="start stop status add torrent metalink list info files peers pause resume remove purge move limit set watch restart retry replace menu"
_pos_subcmds[share-nfs-client]="mount unmount list persist unpersist menu"
_pos_subcmds[share-nfs-server]="status share unshare list reload enable disable menu"
+50
View File
@@ -0,0 +1,50 @@
# lib/yt-lib.sh — shared helpers for yt-dlp-based media tools.
# Sourced opt-in by yt-* subcommand files. Uses err() from lib/common.sh.
# yt_check_deps [dry_run] [ffmpeg_purpose]
# Check yt-dlp (+ ffmpeg unless dry_run=1 and caller wants dry-run to skip deps).
# ffmpeg_purpose is the parenthetical in the error message (default: "MP3 conversion").
# ytsync overrides this deliberately (needs yt-dlp+jq even in dry-run).
yt_check_deps() {
local dry="${1:-0}" purpose="${2:-MP3 conversion}"
command -v yt-dlp &>/dev/null || err "yt-dlp not found — install it with: sudo apt install yt-dlp"
if [ "$dry" -eq 0 ]; then
command -v ffmpeg &>/dev/null || err "ffmpeg not found (needed for $purpose) — install it with: sudo apt install ffmpeg"
fi
}
# yt_validate_url <url>
# Returns 1 (does NOT exit) when URL doesn't start with http:// or https://;
# prints the diagnostic to stderr. Callers surface the error themselves (e.g.
# `yt_validate_url "$url" 2>/dev/null || err "…"` with their own prefix) so a
# helper whose only job is to check can never kill the caller's error path.
yt_validate_url() {
local url="$1"
case "$url" in
http://*|https://*) return 0 ;;
*) echo "not a valid URL: $url (must start with http:// or https://)" >&2; return 1 ;;
esac
}
# yt_echo_cmd <args...>
# Prints the yt-dlp command for --dry-run mode
yt_echo_cmd() {
echo "yt-dlp $*"
}
# classify_url <url>
# Domain classification: music.youtube/soundcloud/bandcamp → audio;
# youtube/youtu.be/vimeo/twitch → video; unknown → ${GRAB_DEFAULT:-video}
# Migrated from bin/pos-media-grab
classify_url() {
local url="$1" mode="${GRAB_DEFAULT:-video}"
case "$url" in
*music.youtube.com*) echo "audio" ;;
*soundcloud.com*) echo "audio" ;;
*bandcamp.com*) echo "audio" ;;
*youtube.com*|*youtu.be*) echo "video" ;;
*vimeo.com*) echo "video" ;;
*twitch.tv*) echo "video" ;;
*) echo "$mode" ;;
esac
}
+2 -1
View File
@@ -54,4 +54,5 @@ silently.
| `t-gen-docs-drift.sh` | `make gen` idempotence on a pristine tracked tree (CI drift gate) |
| `t-lint-gate.sh` | `make lint` green on the real tree; planted violations are caught and named |
| `t-install-version.sh` | install.sh version gate: match→skip, mismatch→proceed, --force bypass, dry-run variant, flag write, numeric comparison |
| `t-share-mountpoint.sh` | share-client `ask_mountpoint` UX: existing/new/declined/rejected paths, confirm gate, mkdir side effects, non-TTY stdin contract, static `n``t` guards |
| `t-share-mountpoint.sh` | share-client `ask_mountpoint` UX: existing/new/declined/rejected paths, confirm gate, mkdir side effects, non-TTY stdin contract, static `n``t` guards |
| `t-pos-media-yt.sh` | unified `pos media yt` suite: dispatcher + forwarder resolution, shared yt-lib helpers, yt-mp3/mp4/grab/subtitles flags, dry-run deps, `YT_OUT_DIR` seam, `GRAB_DEFAULT` config, negative controls (unsafe-URL no-expansion, `--lang en,ar` single arg, txt timestamp-stripping) |
+1 -1
View File
@@ -201,7 +201,7 @@ STUB
for tool in pos-ai pos-ai-hf pos-ai-server pos-communication-matrix-listener \
pos-communication-matrix-sender pos-communication-scrcpy \
pos-communication-telegram-listener pos-communication-telegram-sender \
pos-media-grab pos-network-download; do
pos-media-yt-grab pos-network-download; do
if grep -q "load_env_file" "$ROOT/bin/$tool"; then
printf ' PASS %s uses shared load_env_file\n' "$tool"
else
+279
View File
@@ -0,0 +1,279 @@
#!/usr/bin/env bash
set -euo pipefail
# t-pos-media-yt.sh — POS--9 verification for the unified `pos media yt`
# suite and its forwarders.
#
# Part 1: dispatcher + forwarder resolution (bin/pos-media-yt, pos-media-mp3/
# mp4/grab forwarders, full chain through bin/pos)
# Part 2: lib/yt-lib.sh shared helpers (deps, url validation, classify)
# Part 3: yt-mp3 behaviors (flags, dry-run, YT_OUT_DIR seam, unsafe-URL guard)
# Part 4: yt-mp4 behaviors (format select flags, mutual exclusions, seam)
# Part 5: yt-grab classification + delegation (dry-run command strings) + config
# Part 6: yt-subtitles (defaults, --lang en,ar single-arg, formats),
# txt conversion (timestamps stripped), unavailable-subs detection
# Part 7: negative controls mandated by the Architect contract
run_test() {
local sandbox stubs cfg
sandbox="$(mksandbox media-yt)"
stubs="$sandbox/stubs"
cfg="$sandbox/cfg"
mkdir -p "$stubs" "$cfg"
local ytdlp_log="$sandbox/ytdlp.log"
: > "$ytdlp_log"
# fake yt-dlp logs its exact argv (one per line) — lets us assert arg
# counting / no-expansion WITHOUT any network or real binary.
cat > "$stubs/yt-dlp" <<STUB
#!/usr/bin/env bash
printf 'yt-dlp %s\\n' "\$*" >> "$ytdlp_log"
STUB
# ffmpeg stub: satisfies the non-dry-run deps guard; not actually used.
printf '#!/usr/bin/env bash\nexit 0\n' > "$stubs/ffmpeg"
chmod +x "$stubs/yt-dlp" "$stubs/ffmpeg"
local RB="$ROOT/bin"
local yt ytd="$RB/pos-media-yt" mp3="$RB/pos-media-yt-mp3" \
mp4="$RB/pos-media-yt-mp4" grab="$RB/pos-media-yt-grab" \
subs="$RB/pos-media-yt-subtitles"
local env_base=(PATH="$stubs:$RB:/usr/bin:/bin" CONFIG_DIR="$cfg" \
YT_OUT_DIR="$sandbox/out")
# ═══ Part 1: dispatcher + forwarders ═══
# 1.0 bare dispatcher → usage listing subcommands (exit 0)
test_run_env "${env_base[@]}" -- "$ytd"
check_rc "dispatcher bare invocation = usage exit 0" 0 "$TR_RC"
check_contains "dispatcher usage lists mp3" "mp3" "$TR_OUT"
check_contains "dispatcher usage lists subtitles" "subtitles" "$TR_OUT"
# 1.1 dispatcher unknown subcommand → hard error, nonzero
test_run_env "${env_base[@]}" -- "$ytd" bogus
[ "$TR_RC" -ne 0 ] && printf ' PASS dispatcher unknown subcommand exits nonzero\n' \
|| printf ' FAIL dispatcher unknown subcommand exited 0\n'
check_contains "dispatcher unknown subcommand message" "unknown yt command" "$TR_OUT"
# 1.2 forwarder helpers: --help passes through to the real yt usage
test_run_env "${env_base[@]}" -- "$RB/pos-media-mp3" --help
check_rc "pos media mp3 --help forwards cleanly" 0 "$TR_RC"
check_contains "mp3 forwarder shows yt-mp3 usage" "pos media yt mp3 [options]" "$TR_OUT"
test_run_env "${env_base[@]}" -- "$RB/pos-media-mp4" --help
check_rc "pos media mp4 --help forwards cleanly" 0 "$TR_RC"
check_contains "mp4 forwarder shows yt-mp4 usage" "pos media yt mp4 [options]" "$TR_OUT"
test_run_env "${env_base[@]}" -- "$RB/pos-media-grab" --help
check_rc "pos media grab --help forwards cleanly" 0 "$TR_RC"
check_contains "grab forwarder shows yt-grab usage" "pos media yt grab [options]" "$TR_OUT"
# 1.3 ytsync forwarder --help forwards to pos media ytsync
test_run_env "${env_base[@]}" -- "$RB/pos-media-yt-ytsync" --help
check_contains "ytsync forwarder references media ytsync" "media ytsync" "$TR_OUT"
# 1.4 full dispatch chain: `pos media mp3 <url>` → yt-mp3 (fake yt-dlp called).
# `media-yt-mp3` is non-interactive so bin/pos execs it without a tee.
: > "$ytdlp_log"
test_run_env "${env_base[@]}" -- "$RB/pos" media mp3 "http://example.com/x"
check_rc "chain pos media mp3 <url> resolves" 0 "$TR_RC"
check_contains "chain reaches yt-dlp with -x (audio path)" "-x" "$(cat "$ytdlp_log")"
# ═══ Part 2: lib/yt-lib.sh helpers ═══
source "$ROOT/lib/common.sh"
source "$ROOT/lib/yt-lib.sh"
# 2.0 yt_validate_url accepts http(s), rejects junk
yt_validate_url "https://youtube.com/watch?v=x" && printf ' PASS url valid http/https accepted\n' \
|| printf ' FAIL valid https url rejected\n'
test_run yt_validate_url "ftp://bad"
[ "$TR_RC" -ne 0 ] && printf ' PASS invalid url rejected\n' || printf ' FAIL invalid url accepted\n'
check_contains "invalid url error message" "not a valid URL" "$TR_OUT"
# 2.1 classify_url: audio / video / unknown-with-GRAB_DEFAULT
check_eq "music.youtube → audio" "audio" "$(classify_url "https://music.youtube.com/watch?v=x")"
check_eq "soundcloud → audio" "audio" "$(classify_url "https://soundcloud.com/a/b")"
check_eq "youtube.com → video" "video" "$(classify_url "https://youtube.com/watch?v=x")"
check_eq "vimeo → video" "video" "$(classify_url "https://vimeo.com/123")"
check_eq "unknown → default video" "video" "$(classify_url "https://example.com/x")"
check_eq "unknown → GRAB_DEFAULT=audio" "audio" "$(GRAB_DEFAULT=audio classify_url "https://example.com/x")"
# ═══ Part 3: yt-mp3 ═══
# 3.0 dry-run requires no deps and prints the yt-dlp command (audio flags)
test_run_env "${env_base[@]}" -- "$mp3" --dry-run "https://youtube.com/watch?v=dQw4w9WgXcQ"
check_rc "mp3 dry-run ok" 0 "$TR_RC"
check_contains "mp3 dry-run has -x --audio-format mp3" "--audio-format mp3" "$TR_OUT"
check_contains "mp3 dry-run has --embed-metadata" "--embed-metadata" "$TR_OUT"
# 3.1 --by-artist template
test_run_env "${env_base[@]}" -- "$mp3" --dry-run --by-artist "https://youtu.be/x"
check_contains "mp3 --by-artist organizes by artist/uploader" "%(artist,uploader)s/" "$TR_OUT"
# 3.2 --no-playlist propagated
test_run_env "${env_base[@]}" -- "$mp3" --dry-run --no-playlist "https://youtube.com/watch?v=x"
check_contains "mp3 --no-playlist passed through" "--no-playlist" "$TR_OUT"
# 3.3 --cookies must exist
test_run_env "${env_base[@]}" -- "$mp3" --dry-run --cookies "$sandbox/nope.txt" "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS mp3 missing cookies file errors\n' \
|| printf ' FAIL mp3 accepted missing cookies file\n'
# 3.4 YT_OUT_DIR seam overrides default output
test_run_env "${env_base[@]}" -- "$mp3" --dry-run "https://youtube.com/watch?v=x"
check_contains "mp3 uses YT_OUT_DIR seam" "$sandbox/out/%(title)s.%(ext)s" "$TR_OUT"
# 3.5 missing dep in non-dry-run → hard error (no yt-dlp on PATH at all)
test_run_env PATH="/usr/bin:/bin" -- "$mp3" "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS mp3 without yt-dlp in path errors\n' \
|| printf ' FAIL mp3 ran without yt-dlp present\n'
# 3.6 no URL → usage exit 0
test_run_env "${env_base[@]}" -- "$mp3" --dry-run
check_rc "mp3 no URL → usage exit 0" 0 "$TR_RC"
check_contains "mp3 no URL prints usage" "Usage: pos media yt mp3" "$TR_OUT"
# 3.7 NEGATIVE CONTROL: unsafe URL not expanded — passed as ONE literal arg.
# `$(echo pwned)` stays literal inside the URL token; no second pwned word
# appears in the argv log (which would mean command substitution ran).
: > "$ytdlp_log"
bad='$(echo pwned)'
test_run_env "${env_base[@]}" -- "$mp3" "http://example.com/$bad"
check_rc "unsafe URL run (fake yt-dlp) ok" 0 "$TR_RC"
local logged; logged="$(cat "$ytdlp_log")"
check_contains "url passed literally (single arg)" "\$(echo pwned)" "$logged"
check_not_contains "no separate pwned word from substitution" "example.com/ pwned" "$logged"
# ═══ Part 4: yt-mp4 ═══
# 4.0 --best
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --best "https://youtube.com/watch?v=x"
check_contains "mp4 --best selects bestvideo+bestaudio" "bestvideo*+bestaudio/best" "$TR_OUT"
# 4.1 --worst
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --worst "https://youtube.com/watch?v=x"
check_contains "mp4 --worst selects worst" "-f worst" "$TR_OUT"
# 4.2 -f id skips prompt
test_run_env "${env_base[@]}" -- "$mp4" --dry-run -f 22 "https://youtube.com/watch?v=x"
check_contains "mp4 -f 22 (no prompt)" "-f 22" "$TR_OUT"
# 4.3 mutual exclusions
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --best --worst "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS mp4 --best/--worst mutually exclusive\n' \
|| printf ' FAIL mp4 allowed --best and --worst together\n'
test_run_env "${env_base[@]}" -- "$mp4" --dry-run -f 22 --best "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS mp4 --format/--best mutually exclusive\n' \
|| printf ' FAIL mp4 allowed --format and --best together\n'
# 4.4 YT_OUT_DIR seam
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --best "https://youtube.com/watch?v=x"
check_contains "mp4 uses YT_OUT_DIR seam" "$sandbox/out/%(title)s.%(ext)s" "$TR_OUT"
# 4.5 no URL → usage
test_run_env "${env_base[@]}" -- "$mp4" --dry-run --best
check_rc "mp4 no URL → usage exit 0" 0 "$TR_RC"
check_contains "mp4 no URL prints usage" "Usage: pos media yt mp4" "$TR_OUT"
# ═══ Part 5: yt-grab classification + delegation (dry-run) + config ═══
# 5.0 music.youtube → audio route (dry-run prints delegate command, no exec)
test_run_env "${env_base[@]}" -- "$grab" --dry-run "https://music.youtube.com/watch?v=abc"
check_contains "grab music.youtube routes to yt mp3" "pos media yt mp3" "$TR_OUT"
check_not_contains "grab music.youtube not routed to mp4" "pos media yt mp4" "$TR_OUT"
# 5.1 youtube.com → video route (default --best)
test_run_env "${env_base[@]}" -- "$grab" --dry-run "https://youtube.com/watch?v=xyz"
check_contains "grab youtube routes to yt mp4" "pos media yt mp4 --best" "$TR_OUT"
# 5.2 --audio forces audio despite video domain
test_run_env "${env_base[@]}" -- "$grab" --dry-run --audio "https://youtube.com/watch?v=abc"
check_contains "grab --audio forces yt mp3" "pos media yt mp3" "$TR_OUT"
# 5.3 --worst → video --worst
test_run_env "${env_base[@]}" -- "$grab" --dry-run --worst "https://youtube.com/watch?v=abc"
check_contains "grab --worst routes to mp4 --worst" "pos media yt mp4 --worst" "$TR_OUT"
# 5.4 --audio + --video mutually exclusive
test_run_env "${env_base[@]}" -- "$grab" --dry-run --audio --video "https://youtube.com/watch?v=abc"
[ "$TR_RC" -ne 0 ] && printf ' PASS grab --audio/--video mutually exclusive\n' \
|| printf ' FAIL grab allowed --audio and --video together\n'
# 5.5 invalid URL
test_run_env "${env_base[@]}" -- "$grab" "not-a-url"
[ "$TR_RC" -ne 0 ] && printf ' PASS grab rejects invalid url\n' \
|| printf ' FAIL grab accepted invalid url\n'
check_contains "grab error prefixed yt-grab" "yt-grab:" "$TR_OUT"
# 5.6 unknown domain honors GRAB_DEFAULT via config file (audio)
printf 'GRAB_DEFAULT=audio\n' > "$cfg/grab.env"
test_run_env "${env_base[@]}" -- "$grab" --dry-run "https://example.com/x"
check_contains "grab unknown domain honors GRAB_DEFAULT=audio" "pos media yt mp3" "$TR_OUT"
# ═══ Part 6: yt-subtitles ═══
# 6.0 defaults: --write-subs --write-auto-subs --sub-langs best
test_run_env "${env_base[@]}" -- "$subs" --dry-run "https://youtube.com/watch?v=x"
check_contains "subs default writes manual subs" "--write-subs" "$TR_OUT"
check_contains "subs default writes auto subs" "--write-auto-subs" "$TR_OUT"
check_contains "subs default sub-langs best" "--sub-langs best" "$TR_OUT"
# 6.1 NEGATIVE CONTROL: --lang en,ar becomes ONE --sub-langs arg
test_run_env "${env_base[@]}" -- "$subs" --dry-run --lang en,ar "https://youtube.com/watch?v=x"
check_contains "subs --lang en,ar single arg" "--sub-langs en,ar" "$TR_OUT"
check_not_contains "subs does not split --lang (no single 'en')" "--sub-langs en " "$TR_OUT"
# 6.2 --format vtt
test_run_env "${env_base[@]}" -- "$subs" --dry-run --format vtt "https://youtube.com/watch?v=x"
check_contains "subs --format vtt" "--sub-format vtt" "$TR_OUT"
# 6.3 --auto-only drops --write-subs
test_run_env "${env_base[@]}" -- "$subs" --dry-run --auto-only "https://youtube.com/watch?v=x"
check_not_contains "subs --auto-only drops manual subs" "--write-subs" "$TR_OUT"
check_contains "subs --auto-only keeps auto subs" "--write-auto-subs" "$TR_OUT"
# 6.4 --output dir template
test_run_env "${env_base[@]}" -- "$subs" --dry-run --output "$sandbox/subs" "https://youtube.com/watch?v=x"
check_contains "subs --output template" "$sandbox/subs/%(title)s.%(sub_lang)s.%(ext)s" "$TR_OUT"
# 6.5 invalid --format
test_run_env "${env_base[@]}" -- "$subs" --dry-run --format bogus "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS subs rejects unknown format\n' \
|| printf ' FAIL subs accepted unknown format\n'
# 6.6 unavailable subs: fake yt-dlp fails with no-subtitle stderr → friendly error
mkdir -p "$stubs/subsfail"
cat > "$stubs/subsfail/yt-dlp" <<STUB
#!/usr/bin/env bash
printf 'ERROR: no subtitles found' >&2
exit 1
STUB
chmod +x "$stubs/subsfail/yt-dlp"
test_run_env PATH="$stubs/subsfail:/usr/bin:/bin" -- "$subs" "https://youtube.com/watch?v=x"
[ "$TR_RC" -ne 0 ] && printf ' PASS subs no-subtitles exits nonzero\n' \
|| printf ' FAIL subs no-subtitles exited 0\n'
check_contains "subs unavailable-subs message" "unavailable subtitles" "$TR_OUT"
# 6.7 NEGATIVE CONTROL: txt conversion strips SRT timestamps — run the real
# tool non-dry with a fake yt-dlp that writes an .srt into the cwd; the txt
# converter must strip timestamps/html and keep the words.
local txtdir="$sandbox/txtdir"
mkdir -p "$txtdir" "$stubs/txtsubs"
cat > "$stubs/txtsubs/yt-dlp" <<STUB
#!/usr/bin/env bash
printf '1\\n00:00:01,000 --> 00:00:02,000\\nHello <i>world</i>\\n\\n2\\n00:00:03,000 --> 00:00:04,000\\nSecond line\\n' > foo.srt
exit 0
STUB
chmod +x "$stubs/txtsubs/yt-dlp"
(
cd "$txtdir"
env PATH="$stubs/txtsubs:/usr/bin:/bin" "$subs" --format txt "https://youtube.com/watch?v=x"
) >"$sandbox/txtrun.log" 2>&1
local txt; txt="$(cat "$txtdir/foo.txt" 2>/dev/null || true)"
check_not_contains "txt output strips timestamps" "-->" "$txt"
check_not_contains "txt output strips html tags" "<i>" "$txt"
check_contains "txt output keeps words" "Hello world" "$txt"
# 6.8 subs dry-run skips deps (yt-dlp not in path) — needs no binaries
test_run_env PATH="/usr/bin:/bin" -- "$subs" --dry-run "https://youtube.com/watch?v=x"
check_rc "subs dry-run works without yt-dlp (skips deps)" 0 "$TR_RC"
check_contains "subs dry-run prints yt-dlp cmd" "yt-dlp" "$TR_OUT"
# 6.9 subs no URL → error (not usage), exit 1
test_run_env "${env_base[@]}" -- "$subs" --dry-run
[ "$TR_RC" -ne 0 ] && printf ' PASS subs no URL exits nonzero\n' \
|| printf ' FAIL subs no URL exited 0\n'
check_contains "subs no URL error message" "missing URL (see --help)" "$TR_OUT"
}