feat: pos system backup — smart USB detection with mount offer for unmounted sticks
gates / consistency-and-conventions (push) Successful in 45s

This commit is contained in:
he
2026-08-14 16:58:47 -04:00
parent 16c7cc32ae
commit 28fae8a684
7 changed files with 175 additions and 10 deletions
+1
View File
@@ -42,6 +42,7 @@ summary (newest last).
## Done
- **2026-08-14** — `pos system backup` — smart USB detection: lsblk TRAN (lsusb/by-id cross-check), mount offer for plugged-in-but-unmounted sticks, sha256-verified copy (stub-suite 54/54).
- **2026-08-05** — `pos communication telegram``--parse-mode` (plain/markdown/html).
- **2026-08-05** — doc/code sync gate — `make gen` + `make check` + pre-commit hook.
- **2026-08-05** — `pos usb server` — USB Redirector control tool (494eae2).
+1 -1
View File
@@ -607,7 +607,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `bin/pos-share-smb-server` | 253 | Manage the Samba server (status, share/unshare exports, users, enable/disable) |
| `bin/pos-share-usb-server` | 218 | USB Redirector server control (--ls, --share; prompts when args omitted) |
| `bin/pos-ssh-load-keys` | 31 | Load all SSH keys into the agent |
| `bin/pos-system-backup` | 220 | Encrypted (AES-256) folder snapshots (tar + gpg) |
| `bin/pos-system-backup` | 347 | Encrypted (AES-256) folder snapshots (tar + gpg) |
| `bin/pos-system-firewall` | 308 | Interactive UFW management |
| `bin/pos-system-health` | 209 | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL |
| `bin/pos-system-schedule` | 81 | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently |
+1 -1
View File
@@ -154,7 +154,7 @@ Two kinds of config, don't mix them up:
- **Machine defaults shipped by the installer:** place the file in `config/` and add copy logic to `postinstall.sh`. If it contains secrets, add to `.gitignore` and document in `DOC/`.
- **Runtime tool config set by the user:** `~/.config/linux_post_install/<tool>.env` with `chmod 600`. Load it with env-var precedence (flags > environment > file). Patterns: `pos-docker-compose` (`compose.env`), `pos-communication-telegram-sender` (`telegram.env`, edited via `pos config telegram` — token masked), and the shared ones below. Never store tokens in the repo.
- `system.env` — shared "system" settings loaded by `pos-system-*` tools via `load_system_env()` in `lib/common.sh` (currently `BACKUP_SERVICE_ROOTS`, `BACKUP_USB_ROOT`, `HEALTH_BACKUP_MAX_AGE_DAYS`). Env already exported wins over the file.
- `system.env` — shared "system" settings loaded by `pos-system-*` tools via `load_system_env()` in `lib/common.sh` (currently `BACKUP_SERVICE_ROOTS`, `BACKUP_USB_ROOT`, `BACKUP_MOUNT_BASE`, `BACKUP_USB_BYID`, `HEALTH_BACKUP_MAX_AGE_DAYS`). Env already exported wins over the file.
- `notify.env` — alerting platform selection (`NOTIFY_PLATFORM=telegram,matrix`), read by `lib/notify.sh`.
### 5. Add SSH keys (if needed)
+1 -1
View File
@@ -212,7 +212,7 @@ The standalone `vbox` command still works and forwards to `pos docker vbox` (see
| Command | File | Purpose | Configuration |
|---------|------|---------|---------------|
| `sudo pos system firewall` | `bin/pos-system-firewall` | Interactive UFW ("UFW POWER") menu: add/delete rules, status, enable/disable/reset, default policies | Must run as root. Every command is previewed and confirmed before execution; supports `--dry-run`; keeps a history of executed commands. Executed mutating changes are announced via `lib/notify.sh` |
| `pos system backup <folder-path>` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES). Artifact `<name>_<date>.tar.gz.gpg` in the current directory, `chmod 600`. Once the archive verifies, connected USB storage is offered (auto-detected removable mounts via `lsblk`, or pinned with `BACKUP_USB_ROOT`): the copy lands in `<usb>/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh` |
| `pos system backup <folder-path>` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES). Artifact `<name>_<date>.tar.gz.gpg` in the current directory, `chmod 600`. Once the archive verifies, connected USB storage is offered (detected via `lsblk` TRAN with `lsusb`/by-id cross-check; unmounted sticks get a mount offer first — see `DOC/howto/system.md`; or pinned with `BACKUP_USB_ROOT`): the copy lands in `<usb>/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh` |
| `pos system backup --service` | `bin/pos-system-backup` | Lists folders under `/srv` and `~/srv`, lets you pick one, then runs the same backup | Roots via `BACKUP_SERVICE_ROOTS` (space-separated, default `/srv $HOME/srv`) or `~/.config/linux_post_install/system.env` |
| `pos system health` | `bin/pos-system-health` | Host health dashboard: disk per mount, RAM/swap, failed systemd units, backup age, fail2ban, docker containers. Exits 1 if any check FAILs | Console-only reporter — health itself never sends notifications; forward the output with a wrapper (e.g. the Telegram/Matrix listener map `/status=pos system health`) or schedule it via `pos system schedule` with a `NOTIFY` policy. `HEALTH_BACKUP_MAX_AGE_DAYS` (default 2) and `BACKUP_SERVICE_ROOTS` come from `~/.config/linux_post_install/system.env`; `--help` shows the effective values |
| `pos system schedule <cmd>` | `bin/pos-system-schedule` | Scheduled jobs — run a command on a timer, notify (or stay silent): `run [name\|all]`, `list`, `config`, `enable [name\|all]`, `disable [name\|all]`, `status`, `migrate`. Each job is a file in `~/.config/linux_post_install/schedule.d/<name>.env` with `INTERVAL` (`5m…59m`, `1h…23h`, `hourly`, `daily`, `weekly`, `OnCalendar=…`), `NOTIFY` policy, optional `MSG`, `RULE` (threshold only), and `COMMAND` = the literal rest of the line (pipes/quotes/`sudo` fine). Policies: `always` (full output every run), `onchange` (send when output differs from the last run; first run always sends), `onerror` (non-zero exit or empty output), `threshold` (first numeric output vs `RULE`, alert on false→true + one recovery — the old event-trigger behavior), `never` (side-effect jobs, no notify) | One systemd **user** timer pair per job (`pos-schedule-<name>.timer` + oneshot `.service`, `Persistent=true`), reconciled on `enable`/`disable`; the legacy single `pos-event-trigger` timer is auto-removed. `migrate` converts a pre-existing `event.env` rule set into `schedule.d/rule-N.env` threshold jobs. `config` is an interactive editor (add/edit/remove/enable/disable, validates interval + threshold); alerts via `lib/notify.sh`; `--dry-run` previews runs/writes/sends; jobs are arbitrary shell commands (chmod 600, same trust model as the Telegram map); starter jobs in `config/schedule.d/` auto-installed no-clobber by postinstall |
+26
View File
@@ -106,11 +106,37 @@ pos system backup ~/Documents
# OK Transfer verified 100% (sha256 match): .../backups/docs_2026-08-13.tar.gz.gpg
```
Detection reads `lsblk` and treats a device as USB when `TRAN == usb` (the
per-device deciding signal). Removable-but-not-USB slots (e.g. a SATA card
reader) are skipped. If a device shows no `TRAN` at all, `lsblk`'s answer is
cross-checked against `/dev/disk/by-id/usb-*` symlinks and `lsusb` before it is
offered.
**Unmounted stick?** If the USB stick is plugged in but only shows as `sdax`
with no mountpoint (common on CLI boxes with no automounter), you're offered a
**mount first**, then it copies there:
```bash
# [!] Found USB storage not mounted: /dev/sda1 (7.5G, DataTraveler)
# Mount it at /media/usb-sda1 (world-writable) so the backup can go there? [y/N]
# (y) OK Mounted /dev/sda1 at /media/usb-sda1
# [+] Copying to /media/usb-sda1/backups/docs_2026-08-13.tar.gz.gpg ...
# OK Transfer verified 100% (sha256 match): .../backups/docs_2026-08-13.tar.gz.gpg
```
The mount mirrors `usb-automount` (`/media/<label>`, fallback
`/media/usb-<devname>`, `-o umask=000` world-writable so the copy works without
root). Decline it and you get the exact `sudo mkdir -p` / `sudo mount` commands
to run yourself, then `Enter` re-checks. `s` or EOF (cron) skips silently and
the backup stays local — it never blocks.
- Multiple sticks mounted → pick by number; `0` skips; `n`/EOF skips silently
(cron runs never block).
- Pin a fixed stick (no detection, no prompt on cron) with
`BACKUP_USB_ROOT=/mnt/usb` in `system.env` — the copy still lands in
`<root>/backups/` and is still sha256-verified.
- Mount base and by-id dir are configurable: `BACKUP_MOUNT_BASE` (default
`/media`) and `BACKUP_USB_BYID` (default `/dev/disk/by-id`).
### Recipes
+134 -7
View File
@@ -26,13 +26,17 @@ Modes:
The final artifact <name>_<date>.tar.gz.gpg is written to the current directory.
After it verifies, connected USB storage is offered: the copy lands in
<usb>/backups/ and is sha256-verified 100% before it is announced.
<usb>/backups/ and is sha256-verified 100% before it is announced. A stick
that is plugged in but not mounted is offered a mount first (sudo, mirrors
the usb-automount scheme) before the copy.
Environment:
BACKUP_SERVICE_ROOTS Space-separated roots for --service
(effective: ${EFF_ROOTS})
BACKUP_USB_ROOT USB root to copy finished backups to
(default: auto-detect mounted removable storage)
(default: auto-detect mounted USB storage)
BACKUP_MOUNT_BASE Where to mount an unmounted USB stick offered
during the copy (default: /media)
(loaded from ~/.config/linux_post_install/system.env unless exported)
EOF
exit 0
@@ -46,10 +50,18 @@ command -v gpg &>/dev/null || err "gpg not found (install gnupg)"
# the archive was being made is found. The copy lands in <usb>/backups/
# and the transfer is proven 100% (sha256 source vs copy) before any
# success is announced. BACKUP_USB_ROOT pins the root and skips
# detection; otherwise mounted removable storage is auto-detected.
# detection; otherwise USB storage is auto-detected.
#
# "Is it really USB?" — lsblk's TRAN column decides per device (rm==1
# card readers / hot-swap bays report sata and are skipped). When TRAN
# is unavailable, /dev/disk/by-id/usb-* symlinks and lsusb text
# corroborate. A stick that is plugged in but not mounted (the CLI-box
# case) is offered a mount (sudo, usb-automount scheme: /media/<label>,
# fallback /media/usb-<devname>) before the copy; cron/EOF skips.
usb_copy_offer() {
local archive="$1" pass=0 resp="" i=0 root="" dest_dir="" dest="" src_sum="" dst_sum=""
local -a roots=()
local offered=0
command -v lsblk &>/dev/null || { warn "lsblk not found — USB copy skipped"; return 0; }
command -v jq &>/dev/null || { warn "jq not found — USB copy skipped"; return 0; }
@@ -63,10 +75,21 @@ usb_copy_offer() {
if [ -n "${BACKUP_USB_ROOT:-}" ]; then
roots+=("$BACKUP_USB_ROOT")
else
while IFS= read -r mp; do
[ -n "$mp" ] && roots+=("$mp")
done < <(lsblk -J -o MOUNTPOINT,RM,TYPE 2>/dev/null \
| jq -r '.. | objects | select(.rm == true and .mountpoint != null and (.type == "part" or .type == "disk")) | .mountpoint')
detect_usb
roots=("${USB_MOUNTED[@]}")
if [ ${#roots[@]} -eq 0 ] && [ ${#USB_UNMOUNTED[@]} -gt 0 ]; then
if [ "$offered" -ge 1 ]; then
warn "USB stick still not mounted — backup stays local: $archive"
return 0
fi
offered=1
mount_offer "${USB_UNMOUNTED[@]}" || {
log "Skipped — backup stays local: $archive"
return 0
}
continue
fi
fi
[ ${#roots[@]} -gt 0 ] && break
@@ -125,6 +148,110 @@ usb_copy_offer() {
notify_send "Backup copied to USB: $dest (sha256 verified)"
}
# ── USB detection ────────────────────────────────────────────────
# Fills USB_MOUNTED (mountpoints, one per removable USB storage) and
# USB_UNMOUNTED ("path|label|size|model" entries) from lsblk JSON.
# TRAN=="usb" is the per-device deciding signal; when TRAN is empty,
# usb_related_present() corroborates. Non-USB removables are skipped.
detect_usb() {
local out path="" mp="" label="" size="" model="" tran="" type="" children=""
USB_MOUNTED=()
USB_UNMOUNTED=()
out="$(lsblk -J -o NAME,PATH,LABEL,MOUNTPOINT,RM,TYPE,TRAN,SIZE,MODEL 2>/dev/null)" || return 0
while IFS=$'\x1f' read -r path mp label size model tran type children; do
[ -n "$path" ] || continue
if [ "$tran" = "usb" ]; then
:
elif [ -z "$tran" ] && usb_related_present; then
warn "TRAN unavailable — assuming USB (lsusb/by-id corroboration)"
else
continue
fi
if [ -n "$mp" ]; then
USB_MOUNTED+=("$mp")
elif [ "$children" = "0" ]; then
USB_UNMOUNTED+=("$path|$label|$size|$model")
fi
done < <(printf '%s' "$out" | jq -r '
.. | objects
| select(.rm == true and (.type == "part" or .type == "disk"))
| [.path, (.mountpoint // ""), (.label // ""), (.size // ""),
(.model // ""), (.tran // ""), (.type // ""),
((.children // []) | length)]
| join("\u001f")')
}
# Whole-system "is any USB storage attached?" — udev by-id usb-* symlinks
# are definitive; lsusb text is a secondary hint for storage-ish devices.
# BACKUP_USB_BYID points at the by-id dir (test seam; default real one).
usb_related_present() {
[ -n "$(ls "${BACKUP_USB_BYID:-/dev/disk/by-id}"/usb-* 2>/dev/null)" ] && return 0
command -v lsusb &>/dev/null \
&& lsusb 2>/dev/null | grep -qiE 'mass storage|card reader|flash disk|usb.*(disk|drive|storage)|reader|external'
}
# ── Mount an unmounted USB stick (CLI-box case) ──────────────────
# Asks to mount a detected-but-unmounted device (sudo, mirroring
# usb-automount: /media/<label>, fallback /media/usb-<devname>, -o
# umask=000 world-writable). Returns 0 → caller re-scans (mounted now
# or the user will mount manually); 1 → skipped/EOF (backup stays local).
mount_offer() {
local -a devs=("$@")
local i=0 dev="" label="" size="" model="" mp="" resp="" n=""
local -a chosen=()
if [ ${#devs[@]} -gt 1 ]; then
echo "Multiple unmounted USB storages found:"
for i in "${!devs[@]}"; do
IFS='|' read -r dev label size model <<< "${devs[$i]}"
printf "%2d) %s (%s, %s)\n" "$((i + 1))" "$dev" "$size" "${model:-no label}"
done
read -rp "Copy backup to which one? [1-${#devs[@]}] (0 = skip): " resp || return 1
if ! [[ "$resp" =~ ^[0-9]+$ ]] || (( resp < 1 || resp > ${#devs[@]} )); then
return 1
fi
chosen=("${devs[$((resp - 1))]}")
else
chosen=("${devs[0]}")
fi
IFS='|' read -r dev label size model <<< "${chosen[0]}"
label="${label//\//_}"
mp="${BACKUP_MOUNT_BASE:-/media}/${label:-usb-$(basename "$dev")}"
if [ -d "$mp" ] && mountpoint -q "$mp" 2>/dev/null; then
n=2
while [ -d "${mp}-${n}" ] && mountpoint -q "${mp}-${n}" 2>/dev/null; do
n=$((n + 1))
done
mp="${mp}-${n}"
fi
echo "Found USB storage not mounted: $dev ($size, ${model:-no label})"
if confirm "Mount it at $mp (world-writable) so the backup can go there?" n; then
run sudo mkdir -p "$mp"
if ! run sudo mount -o umask=000 "$dev" "$mp"; then
run sudo mount "$dev" "$mp" || true
fi
if [ "${DRY_RUN:-0}" -eq 1 ] || mountpoint -q "$mp" 2>/dev/null; then
ok "Mounted $dev at $mp"
return 0
fi
warn "Mount failed — do it manually, then we'll copy:"
log " sudo mkdir -p $mp && sudo mount $dev $mp"
else
warn "Mount it manually, then we'll copy:"
log " sudo mkdir -p $mp"
log " sudo mount $dev $mp"
fi
read -rp "Press Enter once mounted (or 's' to skip): " resp || return 1
case "$resp" in
s|S) return 1 ;;
esac
return 0
}
SERVICE=0
case "${1:-}" in
-h|--help) usage ;;
+11
View File
@@ -10,3 +10,14 @@
# Max backup age in days before `pos system health` raises a WARN. Default: 2
#HEALTH_BACKUP_MAX_AGE_DAYS=3
# Pin a fixed USB stick for `pos system backup` (no detection, no prompt on
# cron). The copy lands in <root>/backups/ and is still sha256-verified.
#BACKUP_USB_ROOT=/mnt/usb
# Where `pos system backup` mounts an unmounted stick it found. Default: /media
#BACKUP_MOUNT_BASE=/media
# Dir holding by-id symlinks used to cross-check devices that show no TRAN in
# lsblk. Default: /dev/disk/by-id
#BACKUP_USB_BYID=/dev/disk/by-id