Files
Your Name f14d24950a
gates / consistency-and-conventions (push) Successful in 20s
feat: pos media yt — unified YouTube tools + subtitles (POS--9)
2026-09-09 07:04:22 -04:00

14 KiB
Raw Permalink Blame History

How-To: pos media

Download audio and video from the web via yt-dlp, auto-classify URLs, sync your library to a USB stick, and keep YouTube channels incrementally up to date. Tools: yt mp3, yt mp4, yt grab, yt subtitles, sync, ytsync.

The YouTube download tools live under pos media yt (mp3, mp4, grab, subtitles, ytsync). The legacy standalone names pos media mp3, pos media mp4 and pos media grab still work — they are thin forwarders to the yt forms.

Tool What it does
pos media yt mp3 Download audio, convert to MP3
pos media yt mp4 Download video with smart/interactive format selection
pos media yt grab Auto-classify URL and download as audio or video
pos media yt subtitles Extract subtitles/captions from a URL
pos media yt ytsync Forwarder → pos media ytsync
pos media sync Incrementally copy ~/Music onto a USB stick (mp3/mp4)
pos media ytsync Track YouTube channels/playlists and download only new videos into ~/Videos

Requires yt-dlp and ffmpeg (sudo apt install yt-dlp ffmpeg) for mp3/ mp4/grab; subtitles needs only yt-dlp (no ffmpeg). The tools fail with a clean error message instead of a raw command not found if a dependency is missing.


pos media yt mp3 — audio as MP3

pos media yt mp3 <url>        # or the legacy: pos media mp3 <url>

Extracts and converts the audio track to MP3 in ~/Music/. With --by-artist files land in ~/Music/<artist>/<title>.mp3 (falls back to the uploader name when there's no artist tag), so a library stays organized.

pos media yt mp3 https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp3 --by-artist https://youtu.be/dQw4w9WgXcQ

MP3s are encoded at best quality with title/artist/album/date/chapters embedded (--embed-metadata --embed-chapters, artist falls back to uploader) plus the cover art as a JPEG thumbnail. Existing files are never overwritten.

Flag Meaning
-o, --output <dir> Output directory (default ~/Music)
--by-artist Organize as <dir>/<artist>/<title>.mp3
--no-playlist Download only the single video, not the playlist
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it

Recipe: batch — loop over a list of URLs:

while read -r url; do pos media yt mp3 --by-artist "$url"; done < urls.txt

pos media yt mp4 — video with smart format choice

pos media yt mp4 <url>        # or the legacy: pos media mp4 <url>

Without a format flag, the available formats are shown as a short curated list (grouped [audio] / [video] / [combo], with the raw dump's clutter dropped) and you pick one — the id is validated before downloading. Entering nothing (or best) picks the best video+audio automatically.

Non-interactive (scripting-friendly):

Flag Meaning
-f, --format <id> Download that format id directly (no prompt)
--best Best video + audio, no prompt
--worst Lowest quality, no prompt
-o, --output <dir> Output directory (default ~/Videos)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the yt-dlp command without running it
pos media yt mp4 --best https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt mp4 -f 22 https://youtube.com/watch?v=dQw4w9WgXcQ

Videos merge to MP4 with metadata, chapters, subtitles (all languages) and the thumbnail embedded (--embed-metadata --embed-chapters --embed-subs --sub-langs all --embed-thumbnail); existing files are never overwritten.

Recipe: grab a 4K stream for later — --best already picks the best video+audio and merges them.


pos media yt grab — auto-classify URL and download

pos media yt grab <url>       # or the legacy: pos media grab <url>

Smart URL classifier that routes to pos media yt mp3 or pos media yt mp4 automatically based on the domain. Send a URL from your phone via Telegram and the bot downloads it to the right place without you thinking about it.

Classification rules:

Domain Routes to Why
music.youtube.com mp3 Audio streaming
soundcloud.com mp3 Audio-first platform
bandcamp.com mp3 Audio-first platform
youtube.com / youtu.be mp4 Video content
vimeo.com / twitch.tv mp4 Video platforms
Everything else mp4 (default) Safe fallback

Override the classification with --audio or --video. The default for unrecognized domains is video — change it with pos config grab or set GRAB_DEFAULT=audio in ~/.config/linux_post_install/grab.env.

pos media yt grab https://music.youtube.com/watch?v=abc          # → ~/Music
pos media yt grab https://youtube.com/watch?v=xyz                # → ~/Videos
pos media yt grab --audio https://vimeo.com/123                  # force mp3
pos media yt grab --worst https://youtu.be/abc                   # lowest quality
pos media yt grab --dry-run https://soundcloud.com/artist/track  # preview only

Non-interactive by design — pos media yt mp4 receives --best by default so it never prompts for a format (critical for Telegram bot context where there's no TTY). Pass --worst if you want the smallest file.

Flag Meaning
--audio Force audio (mp3) download
--video Force video (mp4) download
--best Best quality for video (default)
--worst Lowest quality for video
-o, --output <dir> Output directory (passed to mp3/mp4)
--no-playlist Download only the single video
--cookies <file> Netscape cookies.txt for age-gated content
--dry-run Print the command that would run, don't execute

pos media yt subtitles — extract subtitles/captions

pos media yt subtitles <url>

Downloads subtitles/captions from a URL via yt-dlp. Fetches manual captions and auto-generated captions by default (--write-subs --write-auto-subs --sub-langs best — "best" picks the manually-created track when available, otherwise the auto one). Output files land in the current directory as <title>.<lang>.<ext>.

pos media yt subtitles https://youtube.com/watch?v=dQw4w9WgXcQ
pos media yt subtitles --lang en https://youtu.be/dQw4w9WgXcQ
pos media yt subtitles --lang en,ar --format txt https://youtube.com/watch?v=dQw4w9WgXcQ
Flag Meaning
--lang <list> Subtitle languages, comma-separated (default best) — en,ar is passed as ONE --sub-langs arg
--format <fmt> srt (default) / vtt / txttxt converts srt→txt (timestamps, seq numbers and HTML tags stripped)
--auto-only Only auto-generated captions (no manual subs)
-o, --output <dir> Output directory (default: current directory)
--list-subs List available subtitles for the URL and exit (probe only, no download)
--no-playlist Download only the single video
--dry-run Print the yt-dlp command without running it

If a video has no available subtitles the tool reports unavailable subtitles for this video (try --list-subs to check).


pos media sync — music onto a USB stick

pos media sync          # copy everything (mp3 + mp4) from ~/Music to the stick
pos media sync --mp3    # only the .mp3 files
pos media sync --mp4    # only the .mp4 files

Detects connected USB storage exactly like pos system backup (same shared lib/usb-lib.sh): a plugged-in but unmounted stick is offered a mount first (/media/<label>, world-writable, mirrors usb-automount), multiple sticks are listed for you to pick, and if nothing is plugged in it re-scans after you press Enter. Files are mirrored into <usb>/Music/ (change with MEDIA_SYNC_DEST) preserving the artist/album tree.

Sync semantics — add/update only, never delete. Files missing on the stick are copied; files whose size or mtime changed are overwritten; everything identical is skipped. Files on the stick that are no longer in the source are left alone — a playback stick can never lose files to a mirror mistake. Copies keep the source timestamps (cp --preserve=timestamps), so a re-run is a no-op. Preview before copying with --dry-run:

pos media sync --mp4 --dry-run   # shows "would copy" list + counts, copies nothing

The source folder may be a symlink to a library elsewhere (~/Music -> /mnt/data/music) — it is followed, the artist/album tree is mirrored under the symlink's target.

Target picking. Mounted USB partitions are listed with size, label and filesystem (single candidate → confirm prompt; several → numbered picker, one row per partition). EFI system partitions (e.g. a Ventoy stick's VTOYEFI, 32 MB) are never offered as a target — they are boot machinery, not storage. Before any copy the tool verifies the payload fits (df vs the exact bytes to copy) and fails fast with Not enough free space on … instead of dying mid-copy with No space left on device.

Flag Meaning
--mp3 Sync only *.mp3 (neither flag = both)
--mp4 Sync only *.mp4 (neither flag = both)
--source <dir> Source folder (default ~/Music)
--dry-run Preview what would be copied, copy nothing

When it finishes it announces the result via lib/notify.sh (Music sync completed: N added, M updated → <usb>/Music).

Recipes:

pos media sync                    # keep the car stick up to date (both formats)
pos media sync --mp3 --dry-run    # check what a new batch will bring first
pos media sync --source /data/Music   # sync a library that lives elsewhere

Config (all in ~/.config/linux_post_install/system.env or exported): MEDIA_SYNC_SOURCE (default $HOME/Music), MEDIA_SYNC_DEST (default Music), plus the shared USB_MOUNT_BASE / USB_BYID seams.


pos media ytsync — incremental YouTube channel sync

pos media ytsync                                  # interactive
pos media ytsync add https://youtube.com/@SomeChannel
pos media ytsync sync                             # fetch new videos from every source
pos media ytsync list
pos media ytsync remove "Some Channel"

The first run asks for a channel or playlist URL, shows what was resolved (Resolved : Linus Tech Tips (channel · 2140 videos) + the target library path), and downloads everything. Every later run probes the same URL, diffs it against a per-source download archive, and fetches only new videos — one yt-dlp call per video (bestvideo*+bestaudio/best merged to MP4 with metadata, chapters and the thumbnail embedded; existing files are never overwritten).

Where files land:

~/Videos/<channel>/<title>.mp4                              # channels & single videos
~/Videos/<channel>/<playlist>/<NNN> - <title>.mp4           # playlist sources (playlist order)
Command Meaning
add [url] Register a source + first download (asks for the URL if omitted)
sync [name] Incremental pass — all tracked sources, or one by slug/display name
list Tracked sources table: type, archived count, last sync date, destination
remove <name> Stop tracking. Downloaded files AND the archive are kept

Notes:

  • A watch link with both ?v= and &list= downloads only that single video, never the whole playlist; youtu.be/<id> short links count as watch links too.
  • Retitled/renamed videos keep their local filenames (the archive is keyed by video id); inserting a video mid-playlist shifts future numbering only.
  • Members-only / age-gated videos are skipped with a count ("N videos require sign-in"). Escape hatch: put YTSYNC_EXTRA_ARGS=--cookies-from-browser firefox (or --cookies <file>) into the config file below.
  • State lives outside ~/Videos, in ~/.local/share/linux_post_install/ytsync/ (registry, per-source archives, run history) — the media tree stays pure media.

Recipe: daily automation — no built-in timers; use the shared scheduler:

pos system schedule config     # name: ytsync
#   INTERVAL=daily            (or hourly / weekly / OnCalendar=…)
#   NOTIFY=never              # ytsync sends its own digest; don't double-notify
#   MSG="ytSync"
#   COMMAND=pos media ytsync sync
pos system schedule enable ytsync
pos system schedule run ytsync # test once, right now

ytsync never prompts on sync, so the job is timer-safe by construction. It notifies via lib/notify.sh only when something happened (new videos or failures); a scheduled no-op run stays silent — hence NOTIFY=never on the job to avoid double alerts.

Previewing: --dry-run works on add and sync — real probe + plan block (New : 12 would be downloaded (800 already present) + example filenames), zero writes anywhere:

pos media ytsync sync --dry-run
pos media ytsync add https://youtube.com/@SomeChannel --dry-run

Config (~/.config/linux_post_install/ytsync.env, chmod 600 — materialize/edit via pos config ytsync; exported environment wins):

Key Default Meaning
YTSYNC_VIDEOS_DIR $HOME/Videos Videos root for synced sources
YTSYNC_EXTRA_ARGS (empty) Extra yt-dlp flags appended to every download call (cookies, format overrides, …)

Troubleshooting

  • Format list is empty / download fails → the site or age-gate requires cookies; pass --cookies ~/cookies.txt (export it from your browser), or update yt-dlp (sudo apt upgrade yt-dlp).
  • Error about a missing postprocessor → sudo apt install ffmpeg.
  • --by-artist leaves files loose → the source has no artist/uploader tag; it falls back to the uploader name in the artist slot.
  • Very large downloads: ensure free space; files land in ~/Music/~/Videos (or your -o directory).
  • ytsync: "could not reach YouTube" at add → connectivity/DNS; the interactive flow re-prompts up to 3×. "source not found or private" → wrong handle or a deleted/private source.
  • ytsync: many "exists, kept" warnings after a YouTube-side change → the channel's internal id changed, so re-adding computed a new archive. Rename the old archive file (~/.local/share/linux_post_install/ytsync/archive/<old>.txt) to the new slug before syncing (see tools-docs/ytsync.md).
  • ytsync: "disk full — stopping … mid-run" → free space on the videos volume and run pos media ytsync sync again; unfetched videos simply stay "new".