fix: pos media sync — normalize trailing-slash source + surface find errors
This commit is contained in:
@@ -42,6 +42,7 @@ summary (newest last).
|
||||
|
||||
## Done
|
||||
|
||||
- **2026-08-16** — `pos media sync` hardening (3 approved fixes): (1) **trailing-slash source** — with `--source /x/` (or `MEDIA_SYNC_SOURCE=…/`) GNU find normalizes the slash on the starting point, so the rel prefix `"${f#"$SRC/"}"` became `/x//` which never matched and every file silently nested under `<stick>/Music//data/Music/...` each run; `SRC` is now `%/`-normalized with a non-empty guard right after option parsing, before the `-d` check (covers flag + env forms). (2) **silent find failure → false "Sync complete"** — both `find -H` invocations ran through process substitution, which hides find's exit code and stderr from `set -e`/`pipefail`: an unreadable subdir made find exit 1 with "Permission denied" yet the tool announced a full success on a partial tree. find now runs ONCE into a temp list + temp stderr (`|| find_rc=$?`, sorted in place with `sort -o`, temp files removed via `trap EXIT`); rc != 0 or non-empty stderr prints a visible `warn` ("results may be incomplete") with the find stderr lines indented (spawn-style) instead of continuing silently; both the space scan and the copy loop read the same captured list, so the double find scan is gone and the file set is identical. (3) **inner symlinks silently skipped** — `find -H` follows only the command-line source symlink, so symlinks inside the tree never synced; an inner-symlink count (`find -H "$SRC" -mindepth 1 -type l`, root symlink excluded) now warns "N symlink(s) inside the source are not followed (find -H) — their targets will not be synced". `find -H` kept (no `-L`); `needs_copy` mtime logic untouched (FAT32 granularity deferred). Docs: howto/media.md symlink paragraph notes inner symlinks are skipped with a count warning; AGENT_Context filetable regenerated (pos-media-sync 164→202). Verified: stub harness `/tmp/opencode/media-sync-fix-test/` (lsblk JSON fixture with rm=true/type=part/mountpoint/TRAN=usb, `y` confirm, `MEDIA_SYNC_SOURCE`/`MEDIA_SYNC_DEST`/`USB_BYID` seams, HOME isolation, telegram-sender stub) 32/32 green — trailing-slash `--source` and env forms land under `<stick>/Music/<rel>` with no absolute-path nesting, unreadable subdir → visible "results may be incomplete" warn + indented find stderr + exit 0 + accessible files still copied, inner symlinks → count warning, symlink root still works with NO inner-symlink warning (23d69b7 regression incl. trailing-slash combo), re-run after success → `0 added, 0 updated, 2 unchanged`; the same suite against the pre-fix script fails exactly those assertions (24/32). `make gen && make check` green, `make lint` 0 FAIL / 0 WARN.
|
||||
- **2026-08-15** — `pos docker stack` (`bin/pos-docker-stack`) — containers grouped by their Docker Compose project. Each stack is a section (project name, sorted) with lines `container-name status ports`; containers with no compose project land in a `Standalone` section at the end; ends with `Stacks: N containers: N standalone: N`. Running only by default, `-a|--all` includes stopped/exited (like `docker ps -a`). Status colored on a terminal (`Up*` green, `Exited*`/`Dead*`/`Created*` red, `Paused*`/`Restarting*` yellow); exit 0 also when no containers. Data via `docker ps` with `--format '{{.Names}}{{"\u001f"}}{{.Label "com.docker.compose.project"}}{{"\u001f"}}{{.Status}}{{"\u001f"}}{{.Ports}}'` (compose v2 sets the project label; `{{"\u001f"}}` escapes in the Go template), parsed with `awk -F'\x1f'` + `IFS=$'\x1f' read` everywhere — tab/pipe delimiters are IFS whitespace or inside values, so `\x1f` (DEV.md:213 gotcha); dash padding via `sed` not `tr` (tr corrupts multi-byte `─`). Deps guard (`docker`) before `--help`; no stdin → not in `INTERACTIVE_CMDS`; `# POS_FLAGS: -a --all`. Docs: POS.md docker row + detail, howto/docker.md table + section, `bin/pos` usage EXAMPLES, AGENT_Context §14 row. Verified: stub-PATH suite `/tmp/opencode/docker-stack-test/run-tests.sh` 23/23 (grouping, sorted stacks, `-a` shows exited, standalone, empty daemon rc=0, colored status, missing docker rc=1, `--help` after deps guard); live runs against the real daemon (affine/audiobookshelf/convertx/gitea stacks, `affine_migration_job Exited (0)` + `lab1 Exited (137)` under `-a`); dispatch via `pos docker stack`; `make gen && make check`, `make lint` 0 FAIL / 0 WARN.
|
||||
- **2026-08-15** — Fix `pos media sync` offering a Ventoy stick's **EFI partition** as the sync target: with the data partition unmounted, the 32 MB `VTOYEFI` ESP was the only mounted USB partition, `usb_detect` offered it with no context, and `cp` died mid-copy with `No space left on device` (live-box report). `usb_detect` now fetches `FSTYPE`/`PARTTYPENAME` and excludes EFI system partitions (Ventoy `VTOYEFI`, `/boot/efi`) from **both** the mounted list and the mount-offer list; `USB_MOUNTED` entries carry `mp|label|size|model|fs` and `usb_pick_root` shows that in the single-stick confirm and the multi-stick/partition picker (`1) /media/Ventoy (1.1T, Ventoy, exfat)`), while `USB_ROOT` stays a bare mountpoint (`${root%|*}`) so `pos system backup` (`${root%/}/backups`) is unaffected. `pos-media-sync` gained a pre-flight space check (measures exactly what `needs_copy` would copy vs `df -Pk`, `err`/`warn` before any copy) — no more mid-copy ENOSPC. Docs: howto/media.md target-picking note, SCRIPTS.md usb-lib paragraph, AGENT_Context hand-maintained lib row (194→205). Verified: new stub harness `/tmp/opencode/vtoyefi-run.sh` (ESP filtered from mounted + mount-offer, multi-pick shows only the data partition, space fit/too-small/dry-run-warn) green; `/tmp/opencode/backup-test` still green; live check `printf 'n\ns\n' | bash bin/pos-media-sync --mp3` no longer offers VTOYEFI (offers unmounted `sda1` Ventoy instead); `make gen && make check`, `make lint` 0 FAIL / 0 WARN.
|
||||
- **2026-08-15** — Fix `pos media sync` reporting success with 0 files when the source is a symlink: it enumerated with plain `find "$SRC"`, and GNU find (default `-P`) does not descend a command-line symlink to a directory — `~/Music -> /mnt/hdd/…/music` therefore yielded zero matches, the loop never ran, and the tool printed `0 added, 0 updated, 0 unchanged` without creating the target dir (live-box report). Switched to `find -H "$SRC"` (follows only command-line symlinks; inner-symlink semantics unchanged). howto/media.md sync section notes symlinked sources are followed. Caught live, not by the 46-case stub suite (which used a real temp dir source — lesson: add a symlink-root fixture). Verified: `printf 'y\n' | bash bin/pos-media-sync --mp3 --dry-run` now lists all 31 mp3s as "would copy"; `make gen && make check` green.
|
||||
|
||||
@@ -602,7 +602,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
|
||||
| `bin/pos-entertainment-status` | 62 | Show enabled plugins and scheduler state |
|
||||
| `bin/pos-media-mp3` | 86 | Download audio as MP3 (yt-dlp) |
|
||||
| `bin/pos-media-mp4` | 132 | Download video as MP4 (smart/interactive format select) |
|
||||
| `bin/pos-media-sync` | 164 | Incremental Music → USB sync (mp3/mp4, add/update only) |
|
||||
| `bin/pos-media-sync` | 202 | Incremental Music → USB sync (mp3/mp4, add/update only) |
|
||||
| `bin/pos-network-checkport` | 496 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view |
|
||||
| `bin/pos-network-download` | 951 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) |
|
||||
| `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui |
|
||||
|
||||
+2
-1
@@ -114,7 +114,8 @@ pos media sync --mp4 --dry-run # shows "would copy" list + counts, copies noth
|
||||
|
||||
The source folder may be a symlink to a library elsewhere
|
||||
(`~/Music -> /mnt/data/music`) — it is followed, the artist/album tree is
|
||||
mirrored under the symlink's target.
|
||||
mirrored under the symlink's target. Symlinks *inside* the tree are not
|
||||
followed (`find -H`), and a warning reports how many were skipped.
|
||||
|
||||
**Target picking.** Mounted USB partitions are listed with size, label and
|
||||
filesystem (single candidate → confirm prompt; several → numbered picker, one
|
||||
|
||||
+40
-2
@@ -69,8 +69,26 @@ while [[ $# -gt 0 ]]; do
|
||||
done
|
||||
[ "$MP3" -eq 1 ] || [ "$MP4" -eq 1 ] || { MP3=1; MP4=1; }
|
||||
|
||||
# Normalize a trailing slash on the source: GNU find strips it on the starting
|
||||
# point (find -H /x/ -type f emits /x/a.mp3, not /x//a.mp3), so with
|
||||
# SRC=/x/ the rel prefix "${f#"$SRC/"}" becomes /x// which never matches and
|
||||
# files would nest under <stick>/Music//x/... instead of mirroring the tree.
|
||||
# Covers both --source /x/ and MEDIA_SYNC_SOURCE=.../.
|
||||
SRC="${SRC%/}"
|
||||
[ -n "$SRC" ] || err "Source not found: $SRC"
|
||||
|
||||
[ -d "$SRC" ] || err "Source not found: $SRC"
|
||||
|
||||
# find -H follows only the command-line source symlink; symlinks INSIDE the
|
||||
# tree are never followed, so their targets silently never sync. Count them
|
||||
# (mindepth 1: a symlink source itself IS followed and must not count) and
|
||||
# surface the skip instead of leaving a partial mirror unexplained.
|
||||
inner_links="$(find -H "$SRC" -mindepth 1 -type l 2>/dev/null | wc -l)" || true
|
||||
inner_links="${inner_links// }"
|
||||
if [ "$inner_links" -gt 0 ]; then
|
||||
warn "${inner_links} symlink(s) inside the source are not followed (find -H) — their targets will not be synced"
|
||||
fi
|
||||
|
||||
trap 'notify_send "Music sync FAILED"' ERR
|
||||
|
||||
section "Music sync"
|
||||
@@ -112,6 +130,26 @@ if [ "$DRY_RUN" -eq 0 ]; then
|
||||
mkdir -p "$dest_root"
|
||||
space_path="$dest_root"
|
||||
fi
|
||||
# One find pass into a temp list, sorted once, shared by the space scan and
|
||||
# the copy loop so both see the identical file set (the old double find is
|
||||
# gone). The old process substitution swallowed find's exit code and stderr
|
||||
# (invisible to set -e / pipefail): an unreadable subdir made find exit 1
|
||||
# with "Permission denied" yet the tool still announced a false "Sync
|
||||
# complete" on a partial tree. Capture both and surface them explicitly
|
||||
# instead of continuing silently.
|
||||
find_list="$(mktemp)"
|
||||
find_err="$(mktemp)"
|
||||
trap 'rm -f "$find_list" "$find_err"' EXIT
|
||||
find_rc=0
|
||||
find -H "$SRC" "${find_expr[@]}" >"$find_list" 2>"$find_err" || find_rc=$?
|
||||
sort -o "$find_list" "$find_list"
|
||||
if [ "$find_rc" -ne 0 ] || [ -s "$find_err" ]; then
|
||||
warn "find of the source reported problems — results may be incomplete:"
|
||||
if [ -s "$find_err" ]; then
|
||||
sed 's/^/ /' "$find_err"
|
||||
fi
|
||||
fi
|
||||
|
||||
need_kb=0
|
||||
while IFS= read -r f; do
|
||||
rel="${f#"$SRC/"}"
|
||||
@@ -119,7 +157,7 @@ while IFS= read -r f; do
|
||||
sz="$(stat -c %s "$f" 2>/dev/null || printf 0)"
|
||||
need_kb=$((need_kb + (sz + 1023) / 1024))
|
||||
fi
|
||||
done < <(find -H "$SRC" "${find_expr[@]}")
|
||||
done < "$find_list"
|
||||
have_kb="$(df -Pk "$space_path" 2>/dev/null | awk 'NR==2 {print $4}')"
|
||||
have_kb="${have_kb:-0}"
|
||||
if [ "$need_kb" -gt "$have_kb" ]; then
|
||||
@@ -154,7 +192,7 @@ while IFS= read -r f; do
|
||||
else
|
||||
unchanged=$((unchanged + 1))
|
||||
fi
|
||||
done < <(find -H "$SRC" "${find_expr[@]}" | sort)
|
||||
done < "$find_list"
|
||||
|
||||
if [ "$DRY_RUN" -eq 1 ]; then
|
||||
echo "DRY RUN — nothing copied. Would sync: ${added} new, ${updated} updated, ${unchanged} unchanged → $dest_root"
|
||||
|
||||
Reference in New Issue
Block a user