diff --git a/AGENT_TODO.md b/AGENT_TODO.md index 712e926..c437d1b 100644 --- a/AGENT_TODO.md +++ b/AGENT_TODO.md @@ -42,6 +42,7 @@ summary (newest last). ## Done +- **2026-08-16** — `pos media sync` hardening (3 approved fixes): (1) **trailing-slash source** — with `--source /x/` (or `MEDIA_SYNC_SOURCE=…/`) GNU find normalizes the slash on the starting point, so the rel prefix `"${f#"$SRC/"}"` became `/x//` which never matched and every file silently nested under `/Music//data/Music/...` each run; `SRC` is now `%/`-normalized with a non-empty guard right after option parsing, before the `-d` check (covers flag + env forms). (2) **silent find failure → false "Sync complete"** — both `find -H` invocations ran through process substitution, which hides find's exit code and stderr from `set -e`/`pipefail`: an unreadable subdir made find exit 1 with "Permission denied" yet the tool announced a full success on a partial tree. find now runs ONCE into a temp list + temp stderr (`|| find_rc=$?`, sorted in place with `sort -o`, temp files removed via `trap EXIT`); rc != 0 or non-empty stderr prints a visible `warn` ("results may be incomplete") with the find stderr lines indented (spawn-style) instead of continuing silently; both the space scan and the copy loop read the same captured list, so the double find scan is gone and the file set is identical. (3) **inner symlinks silently skipped** — `find -H` follows only the command-line source symlink, so symlinks inside the tree never synced; an inner-symlink count (`find -H "$SRC" -mindepth 1 -type l`, root symlink excluded) now warns "N symlink(s) inside the source are not followed (find -H) — their targets will not be synced". `find -H` kept (no `-L`); `needs_copy` mtime logic untouched (FAT32 granularity deferred). Docs: howto/media.md symlink paragraph notes inner symlinks are skipped with a count warning; AGENT_Context filetable regenerated (pos-media-sync 164→202). Verified: stub harness `/tmp/opencode/media-sync-fix-test/` (lsblk JSON fixture with rm=true/type=part/mountpoint/TRAN=usb, `y` confirm, `MEDIA_SYNC_SOURCE`/`MEDIA_SYNC_DEST`/`USB_BYID` seams, HOME isolation, telegram-sender stub) 32/32 green — trailing-slash `--source` and env forms land under `/Music/` with no absolute-path nesting, unreadable subdir → visible "results may be incomplete" warn + indented find stderr + exit 0 + accessible files still copied, inner symlinks → count warning, symlink root still works with NO inner-symlink warning (23d69b7 regression incl. trailing-slash combo), re-run after success → `0 added, 0 updated, 2 unchanged`; the same suite against the pre-fix script fails exactly those assertions (24/32). `make gen && make check` green, `make lint` 0 FAIL / 0 WARN. - **2026-08-15** — `pos docker stack` (`bin/pos-docker-stack`) — containers grouped by their Docker Compose project. Each stack is a section (project name, sorted) with lines `container-name status ports`; containers with no compose project land in a `Standalone` section at the end; ends with `Stacks: N containers: N standalone: N`. Running only by default, `-a|--all` includes stopped/exited (like `docker ps -a`). Status colored on a terminal (`Up*` green, `Exited*`/`Dead*`/`Created*` red, `Paused*`/`Restarting*` yellow); exit 0 also when no containers. Data via `docker ps` with `--format '{{.Names}}{{"\u001f"}}{{.Label "com.docker.compose.project"}}{{"\u001f"}}{{.Status}}{{"\u001f"}}{{.Ports}}'` (compose v2 sets the project label; `{{"\u001f"}}` escapes in the Go template), parsed with `awk -F'\x1f'` + `IFS=$'\x1f' read` everywhere — tab/pipe delimiters are IFS whitespace or inside values, so `\x1f` (DEV.md:213 gotcha); dash padding via `sed` not `tr` (tr corrupts multi-byte `─`). Deps guard (`docker`) before `--help`; no stdin → not in `INTERACTIVE_CMDS`; `# POS_FLAGS: -a --all`. Docs: POS.md docker row + detail, howto/docker.md table + section, `bin/pos` usage EXAMPLES, AGENT_Context §14 row. Verified: stub-PATH suite `/tmp/opencode/docker-stack-test/run-tests.sh` 23/23 (grouping, sorted stacks, `-a` shows exited, standalone, empty daemon rc=0, colored status, missing docker rc=1, `--help` after deps guard); live runs against the real daemon (affine/audiobookshelf/convertx/gitea stacks, `affine_migration_job Exited (0)` + `lab1 Exited (137)` under `-a`); dispatch via `pos docker stack`; `make gen && make check`, `make lint` 0 FAIL / 0 WARN. - **2026-08-15** — Fix `pos media sync` offering a Ventoy stick's **EFI partition** as the sync target: with the data partition unmounted, the 32 MB `VTOYEFI` ESP was the only mounted USB partition, `usb_detect` offered it with no context, and `cp` died mid-copy with `No space left on device` (live-box report). `usb_detect` now fetches `FSTYPE`/`PARTTYPENAME` and excludes EFI system partitions (Ventoy `VTOYEFI`, `/boot/efi`) from **both** the mounted list and the mount-offer list; `USB_MOUNTED` entries carry `mp|label|size|model|fs` and `usb_pick_root` shows that in the single-stick confirm and the multi-stick/partition picker (`1) /media/Ventoy (1.1T, Ventoy, exfat)`), while `USB_ROOT` stays a bare mountpoint (`${root%|*}`) so `pos system backup` (`${root%/}/backups`) is unaffected. `pos-media-sync` gained a pre-flight space check (measures exactly what `needs_copy` would copy vs `df -Pk`, `err`/`warn` before any copy) — no more mid-copy ENOSPC. Docs: howto/media.md target-picking note, SCRIPTS.md usb-lib paragraph, AGENT_Context hand-maintained lib row (194→205). Verified: new stub harness `/tmp/opencode/vtoyefi-run.sh` (ESP filtered from mounted + mount-offer, multi-pick shows only the data partition, space fit/too-small/dry-run-warn) green; `/tmp/opencode/backup-test` still green; live check `printf 'n\ns\n' | bash bin/pos-media-sync --mp3` no longer offers VTOYEFI (offers unmounted `sda1` Ventoy instead); `make gen && make check`, `make lint` 0 FAIL / 0 WARN. - **2026-08-15** — Fix `pos media sync` reporting success with 0 files when the source is a symlink: it enumerated with plain `find "$SRC"`, and GNU find (default `-P`) does not descend a command-line symlink to a directory — `~/Music -> /mnt/hdd/…/music` therefore yielded zero matches, the loop never ran, and the tool printed `0 added, 0 updated, 0 unchanged` without creating the target dir (live-box report). Switched to `find -H "$SRC"` (follows only command-line symlinks; inner-symlink semantics unchanged). howto/media.md sync section notes symlinked sources are followed. Caught live, not by the 46-case stub suite (which used a real temp dir source — lesson: add a symlink-root fixture). Verified: `printf 'y\n' | bash bin/pos-media-sync --mp3 --dry-run` now lists all 31 mp3s as "would copy"; `make gen && make check` green. diff --git a/DOC/AGENT_Context_Project.md b/DOC/AGENT_Context_Project.md index 754c72b..3560c98 100644 --- a/DOC/AGENT_Context_Project.md +++ b/DOC/AGENT_Context_Project.md @@ -602,7 +602,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:` | `bin/pos-entertainment-status` | 62 | Show enabled plugins and scheduler state | | `bin/pos-media-mp3` | 86 | Download audio as MP3 (yt-dlp) | | `bin/pos-media-mp4` | 132 | Download video as MP4 (smart/interactive format select) | -| `bin/pos-media-sync` | 164 | Incremental Music → USB sync (mp3/mp4, add/update only) | +| `bin/pos-media-sync` | 202 | Incremental Music → USB sync (mp3/mp4, add/update only) | | `bin/pos-network-checkport` | 496 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view | | `bin/pos-network-download` | 951 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) | | `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui | diff --git a/DOC/howto/media.md b/DOC/howto/media.md index c4e9f7c..1b9cf1e 100644 --- a/DOC/howto/media.md +++ b/DOC/howto/media.md @@ -114,7 +114,8 @@ pos media sync --mp4 --dry-run # shows "would copy" list + counts, copies noth The source folder may be a symlink to a library elsewhere (`~/Music -> /mnt/data/music`) — it is followed, the artist/album tree is -mirrored under the symlink's target. +mirrored under the symlink's target. Symlinks *inside* the tree are not +followed (`find -H`), and a warning reports how many were skipped. **Target picking.** Mounted USB partitions are listed with size, label and filesystem (single candidate → confirm prompt; several → numbered picker, one diff --git a/bin/pos-media-sync b/bin/pos-media-sync index 0f08bb4..c5582c8 100755 --- a/bin/pos-media-sync +++ b/bin/pos-media-sync @@ -69,8 +69,26 @@ while [[ $# -gt 0 ]]; do done [ "$MP3" -eq 1 ] || [ "$MP4" -eq 1 ] || { MP3=1; MP4=1; } +# Normalize a trailing slash on the source: GNU find strips it on the starting +# point (find -H /x/ -type f emits /x/a.mp3, not /x//a.mp3), so with +# SRC=/x/ the rel prefix "${f#"$SRC/"}" becomes /x// which never matches and +# files would nest under /Music//x/... instead of mirroring the tree. +# Covers both --source /x/ and MEDIA_SYNC_SOURCE=.../. +SRC="${SRC%/}" +[ -n "$SRC" ] || err "Source not found: $SRC" + [ -d "$SRC" ] || err "Source not found: $SRC" +# find -H follows only the command-line source symlink; symlinks INSIDE the +# tree are never followed, so their targets silently never sync. Count them +# (mindepth 1: a symlink source itself IS followed and must not count) and +# surface the skip instead of leaving a partial mirror unexplained. +inner_links="$(find -H "$SRC" -mindepth 1 -type l 2>/dev/null | wc -l)" || true +inner_links="${inner_links// }" +if [ "$inner_links" -gt 0 ]; then + warn "${inner_links} symlink(s) inside the source are not followed (find -H) — their targets will not be synced" +fi + trap 'notify_send "Music sync FAILED"' ERR section "Music sync" @@ -112,6 +130,26 @@ if [ "$DRY_RUN" -eq 0 ]; then mkdir -p "$dest_root" space_path="$dest_root" fi +# One find pass into a temp list, sorted once, shared by the space scan and +# the copy loop so both see the identical file set (the old double find is +# gone). The old process substitution swallowed find's exit code and stderr +# (invisible to set -e / pipefail): an unreadable subdir made find exit 1 +# with "Permission denied" yet the tool still announced a false "Sync +# complete" on a partial tree. Capture both and surface them explicitly +# instead of continuing silently. +find_list="$(mktemp)" +find_err="$(mktemp)" +trap 'rm -f "$find_list" "$find_err"' EXIT +find_rc=0 +find -H "$SRC" "${find_expr[@]}" >"$find_list" 2>"$find_err" || find_rc=$? +sort -o "$find_list" "$find_list" +if [ "$find_rc" -ne 0 ] || [ -s "$find_err" ]; then + warn "find of the source reported problems — results may be incomplete:" + if [ -s "$find_err" ]; then + sed 's/^/ /' "$find_err" + fi +fi + need_kb=0 while IFS= read -r f; do rel="${f#"$SRC/"}" @@ -119,7 +157,7 @@ while IFS= read -r f; do sz="$(stat -c %s "$f" 2>/dev/null || printf 0)" need_kb=$((need_kb + (sz + 1023) / 1024)) fi -done < <(find -H "$SRC" "${find_expr[@]}") +done < "$find_list" have_kb="$(df -Pk "$space_path" 2>/dev/null | awk 'NR==2 {print $4}')" have_kb="${have_kb:-0}" if [ "$need_kb" -gt "$have_kb" ]; then @@ -154,7 +192,7 @@ while IFS= read -r f; do else unchanged=$((unchanged + 1)) fi -done < <(find -H "$SRC" "${find_expr[@]}" | sort) +done < "$find_list" if [ "$DRY_RUN" -eq 1 ]; then echo "DRY RUN — nothing copied. Would sync: ${added} new, ${updated} updated, ${unchanged} unchanged → $dest_root"