fix: ytsync — classify youtu.be/<id> short links as videos; sync tools-docs thumbnail flag
gates / consistency-and-conventions (push) Successful in 2m34s
gates / consistency-and-conventions (push) Successful in 2m34s
This commit is contained in:
@@ -129,3 +129,4 @@ summary (newest last).
|
|||||||
- **2026-08-14** — CI green-check via plain git (no SSH to the runner, no API tokens — user chose "CI tags + git ls-remote" + "scripts/ci-status.sh helper"): `.gitea/workflows/lint.yml` scoped to `on: push: branches: [main]` (tag pushes no longer re-trigger it) and the gate step now reports its own outcome as a lightweight tag — `ci-ok/$GITHUB_SHA` on success / `ci-fail/$GITHUB_SHA` on failure, pushed over HTTP with the jobs automatic `GITEA_TOKEN` to `http://oauth2:${GITEA_TOKEN}@gitea.skink-platy.ts.net:3000/admin/Linux_post_install.git` (runner container already host-maps that hostname to 100.111.241.54); `steps.gates.conclusion` decides ok/fail, `if: always()` (guarded to `push` events) covers failed gate runs, and an existing-tag guard makes re-runs idempotent. New executable `scripts/ci-status.sh [--wait] [<sha>]` reads the tags via `git ls-remote` (origin, `CI_STATUS_REMOTE` override): GREEN (0) / RED (1) / PENDING (2); `--wait` polls every 10s up to 10 min. DEV.md §CI gains a "Checking green without SSH" bullet. Verified: `bash -n`, yaml-parse OK, `make gen && make check`, `make lint` 0 FAIL / 0 WARN; first live-tag verification pending the push (fallback if Gitea clamps token-push: PAT as workflow secret).
|
- **2026-08-14** — CI green-check via plain git (no SSH to the runner, no API tokens — user chose "CI tags + git ls-remote" + "scripts/ci-status.sh helper"): `.gitea/workflows/lint.yml` scoped to `on: push: branches: [main]` (tag pushes no longer re-trigger it) and the gate step now reports its own outcome as a lightweight tag — `ci-ok/$GITHUB_SHA` on success / `ci-fail/$GITHUB_SHA` on failure, pushed over HTTP with the jobs automatic `GITEA_TOKEN` to `http://oauth2:${GITEA_TOKEN}@gitea.skink-platy.ts.net:3000/admin/Linux_post_install.git` (runner container already host-maps that hostname to 100.111.241.54); `steps.gates.conclusion` decides ok/fail, `if: always()` (guarded to `push` events) covers failed gate runs, and an existing-tag guard makes re-runs idempotent. New executable `scripts/ci-status.sh [--wait] [<sha>]` reads the tags via `git ls-remote` (origin, `CI_STATUS_REMOTE` override): GREEN (0) / RED (1) / PENDING (2); `--wait` polls every 10s up to 10 min. DEV.md §CI gains a "Checking green without SSH" bullet. Verified: `bash -n`, yaml-parse OK, `make gen && make check`, `make lint` 0 FAIL / 0 WARN; first live-tag verification pending the push (fallback if Gitea clamps token-push: PAT as workflow secret).
|
||||||
- **2026-08-14** — `pos media sync` (`bin/pos-media-sync`) — incremental Music → USB sync, plus the shared USB layer it builds on. **New lib `lib/usb-lib.sh`** (194 lines, installed by install.sh): `usb_detect` (lsblk JSON, TRAN + lsusb/by-id cross-check → `USB_MOUNTED`/`USB_UNMOUNTED`), `usb_related_present`, `usb_mount_offer` (`/media/<label>` mount-offer, `usb-automount` scheme), `usb_pick_root <prefix> <subfolder> <giveup-msg>` (detect → mount-offer → single/multi picker → `USB_ROOT`); seams `USB_MOUNT_BASE`/`USB_BYID` with `BACKUP_MOUNT_BASE`/`BACKUP_USB_BYID` aliases so existing `system.env` lines keep working; TRAN-fallback warning deduped to once per scan. `pos-system-backup` refactored onto it (216 lines, was 364) — re-ran the backup stub suite: 65/65 green. **Sync tool**: add/update only, never deletes (user choice); `--mp3`/`--mp4` filter (neither = both), `--source <dir>` (default `MEDIA_SYNC_SOURCE`/`$HOME/Music`), `--dry-run` preview with counts; copies missing/changed (size/mtime) files into `<usb>/Music/` (`MEDIA_SYNC_DEST`) preserving the tree via `cp --preserve=timestamps`; result notified via `lib/notify.sh`; `media-sync` added to `INTERACTIVE_CMDS`; deps guards (`lsblk`/`jq`) before `-h|--help`. Docs: POS.md media row, howto/media.md section, SCRIPTS.md lib section + Phase-2 lib list, system.env seams, DEV.md env-seam registry, AGENT_Context Common Tasks + hand-maintained lib row (+usb-lib 194) + gen'd tree/dispatch/filetable/flags. Verified: new stub suite `/tmp/opencode/msync-run.sh` 46/46 green (fresh/no-op/update/filter/dry-run/multi-stick/mount-offer/no-USB skip/never-delete/--source/TRAN-fallback/notify) — caught and fixed an inverted `needs_copy` return; `make gen && make check`, `make lint` 0 FAIL / 0 WARN; dispatch via `pos media sync --help` + `pos media` listing.
|
- **2026-08-14** — `pos media sync` (`bin/pos-media-sync`) — incremental Music → USB sync, plus the shared USB layer it builds on. **New lib `lib/usb-lib.sh`** (194 lines, installed by install.sh): `usb_detect` (lsblk JSON, TRAN + lsusb/by-id cross-check → `USB_MOUNTED`/`USB_UNMOUNTED`), `usb_related_present`, `usb_mount_offer` (`/media/<label>` mount-offer, `usb-automount` scheme), `usb_pick_root <prefix> <subfolder> <giveup-msg>` (detect → mount-offer → single/multi picker → `USB_ROOT`); seams `USB_MOUNT_BASE`/`USB_BYID` with `BACKUP_MOUNT_BASE`/`BACKUP_USB_BYID` aliases so existing `system.env` lines keep working; TRAN-fallback warning deduped to once per scan. `pos-system-backup` refactored onto it (216 lines, was 364) — re-ran the backup stub suite: 65/65 green. **Sync tool**: add/update only, never deletes (user choice); `--mp3`/`--mp4` filter (neither = both), `--source <dir>` (default `MEDIA_SYNC_SOURCE`/`$HOME/Music`), `--dry-run` preview with counts; copies missing/changed (size/mtime) files into `<usb>/Music/` (`MEDIA_SYNC_DEST`) preserving the tree via `cp --preserve=timestamps`; result notified via `lib/notify.sh`; `media-sync` added to `INTERACTIVE_CMDS`; deps guards (`lsblk`/`jq`) before `-h|--help`. Docs: POS.md media row, howto/media.md section, SCRIPTS.md lib section + Phase-2 lib list, system.env seams, DEV.md env-seam registry, AGENT_Context Common Tasks + hand-maintained lib row (+usb-lib 194) + gen'd tree/dispatch/filetable/flags. Verified: new stub suite `/tmp/opencode/msync-run.sh` 46/46 green (fresh/no-op/update/filter/dry-run/multi-stick/mount-offer/no-USB skip/never-delete/--source/TRAN-fallback/notify) — caught and fixed an inverted `needs_copy` return; `make gen && make check`, `make lint` 0 FAIL / 0 WARN; dispatch via `pos media sync --help` + `pos media` listing.
|
||||||
- **2026-08-22** — `pos media ytsync` (`bin/pos-media-ytsync`) — incremental YouTube channel/playlist sync into `~/Videos`, implemented per the Architect decisions D1–D9 + Designer UX contract (`reportAgents/2026-08-22-*.md`). Subcommands `add [url] / sync [name] / list / remove <name>` + `--dry-run`; bare invocation = interactive menu (`/dev/tty` reads, EOF-safe, NOT in INTERACTIVE_CMDS so dispatcher tee logging is kept; empty state goes straight to the URL prompt). One yt-dlp call per new video (`bestvideo*+bestaudio/best` → MP4, metadata/chapters/thumbnail, `--no-overwrites`, `--windows-filenames --trim-filenames 120`, retries 3), per-video `[n/N] title` heartbeat lines, LF-only logs (spinner TTY-gated); probe = `yt-dlp --flat-playlist -J` parsed with jq, new-list diffed against the per-source `--download-archive` BEFORE downloads (exact counts, exact dry-run plans, zero speculative downloads). State machine-owned outside ~/Videos: `$YTSYNC_STATE_DIR/{registry(\x1f-delimited slug⇥type⇥url⇥subdir⇥playlist_title⇥added_ts), archive/<slug>.txt, history.log}`, atomic temp+mv writes; `remove` keeps files AND archive (re-add resumes incrementally); `?v=`+`&list=` URLs download the single video only. Exit codes: 0 incl. no-op/cancel/non-tty-guard; 1 reserved for missing deps, invalid explicit URL, unknown/ambiguous name, wholesale source failure. Notify digest only when new>0 or failed>0 (+ ERR-trap alarm around download passes) via opt-in `lib/notify.sh`. Config scope `ytsync`: `YTSYNC_VIDEOS_DIR` / `YTSYNC_EXTRA_ARGS` (`pos config ytsync`); automation documented as a `pos system schedule` job (`COMMAND=pos media ytsync sync`, `NOTIFY=never`). Deps guards before `-h|--help` with yt-dlp+jq active under `--dry-run` (the preview IS the probe; ffmpeg skipped there). Docs: `tools-docs/ytsync.md` (new dir), POS.md media row + notes, HOWTO.md row, howto/media.md section + troubleshooting, AGENT_Context Common-Tasks row, `bin/pos` EXAMPLES line. Verified: `make gen && make check && make lint` 0 FAIL / 0 WARN; PATH-stub yt-dlp suite (add happy path, incremental 0-new idempotency, playlist NNN numbering, dry-run zero writes, non-tty guard rc0, remove-keeps-archive) with real $HOME byte-untouched via seams.
|
- **2026-08-22** — `pos media ytsync` (`bin/pos-media-ytsync`) — incremental YouTube channel/playlist sync into `~/Videos`, implemented per the Architect decisions D1–D9 + Designer UX contract (`reportAgents/2026-08-22-*.md`). Subcommands `add [url] / sync [name] / list / remove <name>` + `--dry-run`; bare invocation = interactive menu (`/dev/tty` reads, EOF-safe, NOT in INTERACTIVE_CMDS so dispatcher tee logging is kept; empty state goes straight to the URL prompt). One yt-dlp call per new video (`bestvideo*+bestaudio/best` → MP4, metadata/chapters/thumbnail, `--no-overwrites`, `--windows-filenames --trim-filenames 120`, retries 3), per-video `[n/N] title` heartbeat lines, LF-only logs (spinner TTY-gated); probe = `yt-dlp --flat-playlist -J` parsed with jq, new-list diffed against the per-source `--download-archive` BEFORE downloads (exact counts, exact dry-run plans, zero speculative downloads). State machine-owned outside ~/Videos: `$YTSYNC_STATE_DIR/{registry(\x1f-delimited slug⇥type⇥url⇥subdir⇥playlist_title⇥added_ts), archive/<slug>.txt, history.log}`, atomic temp+mv writes; `remove` keeps files AND archive (re-add resumes incrementally); `?v=`+`&list=` URLs download the single video only. Exit codes: 0 incl. no-op/cancel/non-tty-guard; 1 reserved for missing deps, invalid explicit URL, unknown/ambiguous name, wholesale source failure. Notify digest only when new>0 or failed>0 (+ ERR-trap alarm around download passes) via opt-in `lib/notify.sh`. Config scope `ytsync`: `YTSYNC_VIDEOS_DIR` / `YTSYNC_EXTRA_ARGS` (`pos config ytsync`); automation documented as a `pos system schedule` job (`COMMAND=pos media ytsync sync`, `NOTIFY=never`). Deps guards before `-h|--help` with yt-dlp+jq active under `--dry-run` (the preview IS the probe; ffmpeg skipped there). Docs: `tools-docs/ytsync.md` (new dir), POS.md media row + notes, HOWTO.md row, howto/media.md section + troubleshooting, AGENT_Context Common-Tasks row, `bin/pos` EXAMPLES line. Verified: `make gen && make check && make lint` 0 FAIL / 0 WARN; PATH-stub yt-dlp suite (add happy path, incremental 0-new idempotency, playlist NNN numbering, dry-run zero writes, non-tty guard rc0, remove-keeps-archive) with real $HOME byte-untouched via seams.
|
||||||
|
- **2026-08-23** — ytsync post-review fixes (from `reportAgents/2026-08-23-reviewer-ytsync.md`, ACCEPT_WITH_NITS): `classify_url` now treats `youtu.be/<id>` short links (with or without `&list=`, incl. scheme-less + `?si=` forms) as single videos — same path as `?v=` — so they get type `video` + `--no-playlist` instead of being misfiled as playlists; usage() watch-link note reworded; tools-docs classification table gains the short-link row and the invocation block gains the previously undocumented `--convert-thumbnails jpg`; POS.md/howto media wording extended. Verified: classify_url matrix (6 URL shapes) + stub-PATH end-to-end add (registry type=video, download call carries `--no-playlist` + canonical watch URL); `make gen && make check && make lint` 0 FAIL / 0 WARN.
|
||||||
|
|||||||
@@ -605,7 +605,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
|
|||||||
| `bin/pos-media-mp3` | 86 | Download audio as MP3 (yt-dlp) |
|
| `bin/pos-media-mp3` | 86 | Download audio as MP3 (yt-dlp) |
|
||||||
| `bin/pos-media-mp4` | 132 | Download video as MP4 (smart/interactive format select) |
|
| `bin/pos-media-mp4` | 132 | Download video as MP4 (smart/interactive format select) |
|
||||||
| `bin/pos-media-sync` | 164 | Incremental Music → USB sync (mp3/mp4, add/update only) |
|
| `bin/pos-media-sync` | 164 | Incremental Music → USB sync (mp3/mp4, add/update only) |
|
||||||
| `bin/pos-media-ytsync` | 1180 | Incrementally sync YouTube channels/playlists into ~/Videos |
|
| `bin/pos-media-ytsync` | 1190 | Incrementally sync YouTube channels/playlists into ~/Videos |
|
||||||
| `bin/pos-network-checkport` | 496 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view |
|
| `bin/pos-network-checkport` | 496 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view |
|
||||||
| `bin/pos-network-download` | 951 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) |
|
| `bin/pos-network-download` | 951 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) |
|
||||||
| `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui |
|
| `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui |
|
||||||
|
|||||||
+2
-1
@@ -214,7 +214,8 @@ The standalone `vbox` command still works and forwards to `pos docker vbox` (see
|
|||||||
|
|
||||||
A watch link carrying **both** `?v=` and `&list=` downloads only that single video
|
A watch link carrying **both** `?v=` and `&list=` downloads only that single video
|
||||||
(`--no-playlist`) — nobody accidentally backfills a 500-video playlist from a watch
|
(`--no-playlist`) — nobody accidentally backfills a 500-video playlist from a watch
|
||||||
link; a pure playlist link becomes a tracked playlist source with numbered
|
link; `youtu.be/<id>` short links count as watch links too. A pure playlist link
|
||||||
|
becomes a tracked playlist source with numbered
|
||||||
`<NNN> - <title>.mp4` files. `pos media ytsync remove <name>` stops tracking but
|
`<NNN> - <title>.mp4` files. `pos media ytsync remove <name>` stops tracking but
|
||||||
keeps the downloaded files AND the archive — re-adding the same source later
|
keeps the downloaded files AND the archive — re-adding the same source later
|
||||||
resumes incrementally instead of re-downloading. Members-only/age-gated videos are
|
resumes incrementally instead of re-downloading. Members-only/age-gated videos are
|
||||||
|
|||||||
+1
-1
@@ -183,7 +183,7 @@ Where files land:
|
|||||||
Notes:
|
Notes:
|
||||||
|
|
||||||
- A watch link with **both** `?v=` and `&list=` downloads only that single video,
|
- A watch link with **both** `?v=` and `&list=` downloads only that single video,
|
||||||
never the whole playlist.
|
never the whole playlist; `youtu.be/<id>` short links count as watch links too.
|
||||||
- Retitled/renamed videos keep their local filenames (the archive is keyed by video
|
- Retitled/renamed videos keep their local filenames (the archive is keyed by video
|
||||||
id); inserting a video mid-playlist shifts future numbering only.
|
id); inserting a video mid-playlist shifts future numbering only.
|
||||||
- Members-only / age-gated videos are skipped with a count ("N videos require
|
- Members-only / age-gated videos are skipped with a count ("N videos require
|
||||||
|
|||||||
+18
-8
@@ -80,10 +80,10 @@ Layout:
|
|||||||
Notes:
|
Notes:
|
||||||
Existing files are never overwritten; renamed/retitled videos keep their
|
Existing files are never overwritten; renamed/retitled videos keep their
|
||||||
local filename. Unattended/scheduled use: 'pos media ytsync sync'.
|
local filename. Unattended/scheduled use: 'pos media ytsync sync'.
|
||||||
A watch link carrying BOTH ?v= and &list= downloads only that single video
|
A watch link (?v=…, incl. youtu.be/<id> short links) downloads only that
|
||||||
(--no-playlist), never the whole playlist. 'remove' keeps the downloaded
|
single video (--no-playlist), even when it carries &list= — never the whole
|
||||||
files AND the archive — re-adding the same source later resumes
|
playlist. 'remove' keeps the downloaded files AND the archive — re-adding
|
||||||
incrementally instead of re-downloading.
|
the same source later resumes incrementally instead of re-downloading.
|
||||||
|
|
||||||
Examples:
|
Examples:
|
||||||
pos media ytsync # interactive
|
pos media ytsync # interactive
|
||||||
@@ -154,12 +154,22 @@ is_youtube_url() {
|
|||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
classify_url() { # list= without v= → playlist; v= → single video; else channel
|
classify_url() { # names-one-video (v= or youtu.be/<id>) → video; list= → playlist; else channel
|
||||||
local u="$1"
|
local u="$1" vid=""
|
||||||
if [[ "$u" == *list=* && "$u" != *v=* ]]; then
|
case "$u" in
|
||||||
printf 'playlist\n'
|
*youtu.be/*)
|
||||||
|
vid="${u#*youtu.be/}" # short link: video id is the first path segment
|
||||||
|
vid="${vid%%\?*}"
|
||||||
|
vid="${vid%%\#*}"
|
||||||
|
vid="${vid%%/*}"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
if [ -n "$vid" ]; then
|
||||||
|
printf 'video\n'
|
||||||
elif [[ "$u" == *v=* ]]; then
|
elif [[ "$u" == *v=* ]]; then
|
||||||
printf 'video\n'
|
printf 'video\n'
|
||||||
|
elif [[ "$u" == *list=* ]]; then
|
||||||
|
printf 'playlist\n'
|
||||||
else
|
else
|
||||||
printf 'channel\n'
|
printf 'channel\n'
|
||||||
fi
|
fi
|
||||||
|
|||||||
@@ -43,8 +43,9 @@ bytes ever reach the dispatcher's tee'd logs.
|
|||||||
|
|
||||||
| URL contains | Type | Behavior |
|
| URL contains | Type | Behavior |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `list=` without `v=` | `playlist` | tracked source; files land as `<videos>/<owner>/<playlist>/<NNN> - <title>.<ext>` |
|
| `youtu.be/<id>` path (with or without `list=`) | `video` | short links name one video — treated exactly like `v=` |
|
||||||
| `v=` present (with or without `list=`) | `video` | single video, downloaded with `--no-playlist`; nobody backfills a 500-video playlist by pasting a watch link |
|
| `v=` present (with or without `list=`) | `video` | single video, downloaded with `--no-playlist`; nobody backfills a 500-video playlist by pasting a watch link |
|
||||||
|
| `list=` without any video-naming part | `playlist` | tracked source; files land as `<videos>/<owner>/<playlist>/<NNN> - <title>.<ext>` |
|
||||||
| neither | `channel` | flat source; `<videos>/<channel>/<title>.<ext>` |
|
| neither | `channel` | flat source; `<videos>/<channel>/<title>.<ext>` |
|
||||||
|
|
||||||
## Download invocation (one yt-dlp call PER NEW VIDEO)
|
## Download invocation (one yt-dlp call PER NEW VIDEO)
|
||||||
@@ -53,6 +54,7 @@ bytes ever reach the dispatcher's tee'd logs.
|
|||||||
yt-dlp -f "bestvideo*+bestaudio/best"
|
yt-dlp -f "bestvideo*+bestaudio/best"
|
||||||
--merge-output-format mp4 # parity with pos media mp4
|
--merge-output-format mp4 # parity with pos media mp4
|
||||||
--embed-metadata --embed-chapters --embed-thumbnail
|
--embed-metadata --embed-chapters --embed-thumbnail
|
||||||
|
--convert-thumbnails jpg
|
||||||
--no-overwrites # parity; collisions become "exists, kept" warnings
|
--no-overwrites # parity; collisions become "exists, kept" warnings
|
||||||
--download-archive <state>/archive/<slug>.txt # crash-safe per-video recording
|
--download-archive <state>/archive/<slug>.txt # crash-safe per-video recording
|
||||||
--windows-filenames # USB/Samba/TV-safe names
|
--windows-filenames # USB/Samba/TV-safe names
|
||||||
|
|||||||
Reference in New Issue
Block a user