Linux_post_install/AgentsReport/detective/2026-09-04_ytsync-channel-handle.md

ytsync channel handle bug — root cause investigation

Date: 2026-09-04 Agent: Detective (read-only) Status: ROOT_CAUSE_ESTABLISHED

TL;DR

• Confirmed root cause: run_probe() probes the user's channel URL verbatim; yt-dlp --flat-playlist -J on a bare channel URL (@handle, /c/, /user/, /channel/<ID>, music.youtube.com/channel/<ID>) returns the channel's TAB structure (Videos/Live/Shorts: _type:"playlist", url:null, id==channel_id) instead of videos. collect_entries() records every entry unfiltered, so the three tabs become three "new videos" that download_video() tries to fetch as watch?v=<channel_id> → [youtube] <channel_id>: This video is unavailable.
• Fix (validated, not implemented): (a) probe-time canonicalization — new helper canonical_channel_url() appends /videos to bare channel URLs, called at the top of run_probe() (fixes both add and sync of already-stored bare-handle registry entries, no migration); (b) defense-in-depth filter in collect_entries() keeping only watchable entry URLs (watch?v=, youtu.be/, /shorts/), plus a _type=="video" guard on the single-object fallback (handles empty channels gracefully).
• Artifacts: probes under /tmp/opencode/*.json; validated stub harness at /tmp/opencode/ytsync-test/run-tests.sh (17 assertions; 5 fail on the unfixed script, all must pass after the fix).
• Classification: FACT (reproduced live + function-level).

Step 1: Read evidence + code

Read /tmp/opencode/ytsync-probe.json, /tmp/opencode/ytsync-vtab.json, and all of bin/pos-media-ytsync (1191 lines). Key code facts:

• run_probe() line 316: yt-dlp --flat-playlist -J --no-warnings -- "$url".
• collect_entries() lines 374-392: records .id+.title of every .entries[] element; no _type/url filter; fallback to single object when no entries.
• is_signin_skipped() line 401: only skips empty ids / [Private/[Deleted/[Unavailable titles — tab entries pass.
• classify_url() line 157: video / playlist / channel.
• Registry stores the original URL (finish_add line 773-774). sync re-probes the stored URL verbatim (run_sync_set line 923 → pass_prepare line 559 → run_probe "$S_URL").

[DONE]

Step 2: Probe channel URL forms (my own reproductions)

All probes: yt-dlp 2026.08.19 --flat-playlist -J --no-warnings. Summarized as shapes → outcome:

Probe URL	Probe shape	Verdict
https://www.youtube.com/@3blue1brown	_type:playlist, id:@3blue1brown, playlist_count:3; entries = 3 tabs (_type:"playlist", url:null, id==channel_id)	TABS — the bug
https://www.youtube.com/@3blue1brown/videos	playlist_count:151; entries = 151 real videos (_type:"url", url:watch?v=…)	canonical target OK
https://www.youtube.com/@3blue1brown/shorts	playlist_count:81; real entries, url:youtube.com/shorts/<id>	OK (needs /shorts/ in filter)
https://www.youtube.com/@3blue1brown/streams	playlist_count:10; real entries watch?v=	OK
https://www.youtube.com/@3blue1brown/live	rc=1, stdout null, stderr The channel is not currently live	probe-fail path (unchanged)
https://www.youtube.com/@3blue1brown/playlists	playlist_count:24; entries _type:"url" but url:playlist?list=…	false-positive if unfiltered
https://www.youtube.com/@3blue1brown/featured	title "… - Home", 7 entries, id:null, playlist/tab urls	false-positive if unfiltered
https://www.youtube.com/c/3Blue1Brown	tabs (playlist_count:3)	needs canonicalization
https://www.youtube.com/c/3Blue1Brown/videos	151 real videos	OK
https://www.youtube.com/user/3Blue1Brown (+/videos)	rc=1, HTTP 404	user/ dead for this channel
https://www.youtube.com/user/pewdiepie	tabs (playlist_count:2)	works → needs canonicalization
https://www.youtube.com/user/MarquesBrownlee	tabs (playlist_count:3)	works → needs canonicalization
https://www.youtube.com/channel/UCYO_…	tabs	needs canonicalization
https://www.youtube.com/channel/UCYO_…/videos	151 real videos	OK
https://www.youtube.com/playlist?list=…	_type:playlist; entries real watch?v=	OK — must stay untouched
https://www.youtube.com/watch?v=… / https://youtu.be/…	_type:"video" single object, no entries	OK — fallback path
@3blue1brown (no domain)	rc=1 [generic] not a valid URL	latent ytsync wart (is_youtube_url accepts @*)
youtube.com/@3blue1brown (no proto)	tabs	works; canonicalizable
youtube.com/@3blue1brown/videos	151 real videos	OK
…/@3blue1brown/videos/ (trailing slash)	151 real videos	OK — suffix check must strip slash
…/@3blue1brown/videos?view=0&sort=dd	151 real videos	OK — suffix check must strip query
…/@3blue1brown/VIDEOS (uppercase)	rc=1 channel does not have a VIDEOS tab	yt-dlp suffix is case-sensitive
https://music.youtube.com/channel/UCYO_…	tabs	needs canonicalization
https://music.youtube.com/channel/UCYO_…/videos	151 real videos	OK

Live functional reproduction (read-only, --dry-run writes nothing — registry write is behind DRY_RUN -eq 0, history only in non-dry runs):

$ bin/pos-media-ytsync sync --dry-run
Source   : https://www.youtube.com/@3blue1brown          ← stored URL re-probed verbatim
Resolved : 3Blue1Brown (channel · 3 videos)              ← the 3 TABS
New      : 3 would be downloaded (0 already present)
  3Blue1Brown - Videos.mp4 / - Live.mp4 / - Shorts.mp4   ← tab titles as "videos"

Matches the reported add output exactly (first divergence: parse_probe+collect_entries treating tab entries as videos).

[DONE]

Step 3: Trace the call surface (who probes what URL)

run_probe() callers:

1. cmd_add line 876 — run_probe "$url" (user-supplied add URL; explicit mode).
2. ask_url_interactive line 839 — run_probe "$u" (interactive add; finish_add stores $ASKED_URL).
3. pass_prepare line 559 — run_probe "$S_URL"; S_URL comes from the registry line (read at run_sync_set line 923).

Mandatory conclusion: the stored registry URL is re-probed verbatim on every sync (confirmed by code trace AND the live dry-run above). Therefore canonicalization inside run_probe() fixes both flows at once — the existing machine registry entry (3blue1brown … https://www.youtube.com/@3blue1brown) needs no migration; it simply re-canonicalizes each probe.

parse_probe/P_KEY impact: both the bare-handle and /videos probe shapes carry uploader_id:"@3blue1brown" → P_KEY="@3blue1brown" → slug 3blue1brown (matches existing registry slug). P_TITLE changes cosmetically (3Blue1Brown → 3Blue1Brown - Videos); it is stored as the registry S_TITLE field but never displayed by cmd_list/digests. No functional impact.

[DONE]

Step 4: Fix spec (validated, Builder-executable)

Recommended combination: canonicalization (primary) + entry filter + fallback guard (defense-in-depth). Canonicalization alone fixes the report end-to-end; the filter alone would degrade a bare-handle add to "0 videos" (graceful but useless). Both are needed; both validated by simulation below.

4.1 New helper — insert after classify_url() (after line 176), before sanitize_component()

# ── Channel URL canonicalization ─────────────────────────────────────
# yt-dlp --flat-playlist on a bare channel URL returns the channel's
# TAB list (Videos/Live/Shorts; _type "playlist", url null, id==channel_id),
# not videos. Appending /videos makes the probe return the real videos.
canonical_channel_url() {   # add /videos to bare channel URLs; echo canonical
    local u="$1"
    case "$u" in
        @*) u="https://www.youtube.com/$u" ;;   # bare 'handle' → full URL
    esac
    [ "$(classify_url "$u")" = "channel" ] || { printf '%s' "$u"; return 0; }
    local path="${u%%\?*}"
    path="${path%%\#*}"
    path="${path%/}"
    case "${path##*/}" in
        videos | shorts | streams | live | playlists | featured | releases | podcasts | search)
            printf '%s' "$u" ;;
        *)
            printf '%s/videos' "$u" ;;
    esac
}

Behavior (all 16 cases tested PASS in the harness): bare @handle full URL → …/videos; bare @3blue1brown (no domain, fixes the latent generic-error failure) → https://www.youtube.com/@3blue1brown/videos; youtube.com/@… no-protocol → + /videos; /c/NAME, /user/NAME, /channel/ID → + /videos; music.youtube.com/channel/ID → + /videos; already-suffixed /videos /shorts /streams /live /playlists /featured, with trailing slash or query → untouched; ?list= / watch?v= / youtu.be/<id> → untouched.

4.2 run_probe() — one line, after local url="$1" (line 317)

    url="$(canonical_channel_url "$url")"

That is the whole integration point: add (both modes) and sync (stored URLs) now probe the Videos tab. Nothing downstream changes (S_URL stays the original; display is cosmetic).

4.3 collect_entries() — two edits (lines 374-392)

(a) Line 381 — add a select so only watchable entries are collected (exact in-file quoting verified):

        mapfile -t pairs < <(jq -r '.entries[] | select((.url // "") | test("watch\\?v=|youtu\\.be/|/shorts/")) | ((.id // "") + "\u001f" + (.title // ""))' "$PROBE_JSON")

(b) Lines 388-391 — guard the single-object fallback so empty tab/playlist probes never become one bogus video:

    elif [ "$(jq -r '._type // ""' "$PROBE_JSON")" = "video" ]; then
        ENTRY_IDS+=("$(jq -r '.id // ""' "$PROBE_JSON")")
        ENTRY_TITLES+=("$(jq -r '.title // ""' "$PROBE_JSON")")
    fi

Filter counts (validated): tab probe → 0; videos → 151; shorts → 81; streams → 10; playlists-tab → 0; featured → 0; playlist ?list= → 16; video ?v= → 1 (fallback). Empty channel (entries:[], _type:"playlist") → 0 → pass_execute line 624 prints Sync complete: 0 new, 0 already present, 0 failed (no 3-failure spam).

4.4 Error behavior — empty channel / not-live channel

• Channel with zero videos: /videos probe → entries:[] → 0 valid → graceful "0 new" summary (verified with empty-videos.json fixture).
• Explicit /live when not live: probe rc=1 → existing notfound/unreachable handling (sync: skip + FAILED (probe) history; add: retry prompt). Unchanged.
• Explicit /playlists / /featured URLs: untouched by canonicalization; filter now drops all playlist-URL entries → 0 new / 0 failed (previously N failed downloads). Graceful improvement, no new error path.

4.5 Things deliberately NOT changed

• parse_probe, P_KEY, registry line format, download loop, classify_url — untouched.
• Registry migration — none required (probe-time canonicalization covers stored URLs).
• Storing the canonical URL in the registry for new adds — optional cosmetic enhancement, NOT needed for correctness; recommended to skip to keep the change minimal.

[DONE]

Step 5: Test plan

Harness (new, stub-based — no ytsync harness exists in repo): /tmp/opencode/ytsync-test/run-tests.sh

• Self-contained: generates 6 inline fixture JSONs (tab-probe, videos+shorts-tab, playlist, playlists-tab, single-video, empty-videos), stub common.sh/notify.sh, a recording fake yt-dlp; truncates bin/pos-media-ytsync at the # ── Argument dispatch ── marker and sources it (function-level tests, no network/state).
• 17 assertions: 7× classify_url regression; 16× canonical_channel_url matrix; 8× collect_entries fixture behavior; 1× run_probe URL recording.
• Baseline on the unfixed script: 12 PASS / 5 FAIL (exactly the fix-specific assertions fail — proves the harness discriminates the bug). After the fix: all 17 must pass.

Recommended Builder self-verification (in order):

1. bash -n bin/pos-media-ytsync
2. bash /tmp/opencode/ytsync-test/run-tests.sh → 0 failed
3. ./bin/pos-media-ytsync sync --dry-run → Resolved : 3Blue1Brown (channel · 151 videos), real titles, 151 would be downloaded
4. Real limited download into temp dirs (never the real ~/Videos): YTSYNC_STATE_DIR=/tmp/yts-state YTSYNC_VIDEOS_DIR=/tmp/yts-vids ./bin/pos-media-ytsync add https://www.youtube.com/@3blue1brown then interrupt after the first videos (safe: temp dir; yt-dlp renames atomically, archive records completed ones); verify archive/3blue1brown.txt grows and flat <title>.mp4 layout.
5. make check and make lint (must end 0 FAIL, 0 WARN); commit with conventional prefix + AGENT_TODO.md Done move.

Edge-case matrix (URL form → after-fix behavior):

• bare @handle / /c/ / /user/ (resolvable) / /channel/ID / music.…/channel/ID → canonicalized to /videos → real videos.
• @handle (no domain) → full canonical URL → real videos (fixes latent failure).
• /videos, /shorts, /streams, /live, /playlists, /featured explicit suffixes → untouched; shorts/streams pass filter; playlists/featured yield graceful 0; live fails gracefully if not live.
• ?list=, ?v=, youtu.be/<id> → untouched, behave as today.
• /user/ URLs that 404 → unchanged notfound failure path (pre-existing; /user/ is deprecated by YouTube).
• Uppercase suffix (@h/VIDEOS) → untouched → yt-dlp error (pre-existing; suffix check is intentionally case-sensitive).
• Existing stored bare-handle registry entries → work without migration; archive has no stale ids (the 3 broken downloads never reached the archive).

[DONE]

Handoff

Status: ROOT_CAUSE_ESTABLISHED

Symptom: pos media ytsync add https://www.youtube.com/@3blue1brown (and sync of the stored URL) resolves the channel's 3 tabs as videos and fails each download with [youtube] UCYO_jab_es: This video is unavailable.

Expected: resolve the channel's actual videos and download new ones incrementally.

Actual: probe returns tab entries (Videos/Live/Shorts); collect_entries records them unfiltered; 3 failed downloads; summary "0 new … 3 failed".

Root cause: run_probe probes the bare channel URL verbatim; yt-dlp flat-playlist on channel URLs returns channel tabs (_type:"playlist", url:null, id==channel_id), not videos; collect_entries has no non-video entry filter, so tabs are downloaded as watch?v=<channel_id> and fail.

Classification: FACT (live reproduction: sync dry-run shows the 3 tab titles; function-level reproduction: tab fixture yields 3 entries; probe evidence table).

Evidence: probe JSONs /tmp/opencode/ytsync-{probe,vtab}.json + my additional probes; filter counts above; harness baseline 12/17.

Tests performed: 24 URL-form probes (matrix above); jq filter validation on 6 shapes; fallback guard validation on 4 fixtures; live sync --dry-run reproduction; harness mechanics + baseline run.

Alternatives eliminated:

• "yt-dlp version regression" — no; current yt-dlp behavior is inherent for channel URLs (tabs), /videos suffix returns videos (probed).
• "URL should be classified as playlist" — no; channel classification is correct; the probe target is the problem.
• "Filter _type=="url" only" — rejected: /playlists-tab entries are _type:"url" with playlist IDs; the watch-URL test is the correct discriminator (validated).
• "Registry migration needed" — no; probe-time canonicalization covers stored URLs (traced + live dry-run).

Affected components: bin/pos-media-ytsync — run_probe (line 316), collect_entries (lines 374-392); new helper canonical_channel_url; no change to registry format.

Scope / decision boundary: none — pure bug fix within the tool; no architectural decisions required.

Recommended next agent: Builder

Reason: root cause and the exact code-level fix (validated by simulation and the stub harness) are established; implementation + make gen/check/lint + harness green are Builder work.

Changes made by Detective: none (repo untouched; scaffolding + harness in /tmp/opencode/ only).