feat: pos system backup — smart USB detection with mount offer for unmounted sticks
gates / consistency-and-conventions (push) Successful in 45s
gates / consistency-and-conventions (push) Successful in 45s
This commit is contained in:
@@ -42,6 +42,7 @@ summary (newest last).
|
||||
|
||||
## Done
|
||||
|
||||
- **2026-08-14** — `pos system backup` — smart USB detection: lsblk TRAN (lsusb/by-id cross-check), mount offer for plugged-in-but-unmounted sticks, sha256-verified copy (stub-suite 54/54).
|
||||
- **2026-08-05** — `pos communication telegram` — `--parse-mode` (plain/markdown/html).
|
||||
- **2026-08-05** — doc/code sync gate — `make gen` + `make check` + pre-commit hook.
|
||||
- **2026-08-05** — `pos usb server` — USB Redirector control tool (494eae2).
|
||||
|
||||
@@ -607,7 +607,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
|
||||
| `bin/pos-share-smb-server` | 253 | Manage the Samba server (status, share/unshare exports, users, enable/disable) |
|
||||
| `bin/pos-share-usb-server` | 218 | USB Redirector server control (--ls, --share; prompts when args omitted) |
|
||||
| `bin/pos-ssh-load-keys` | 31 | Load all SSH keys into the agent |
|
||||
| `bin/pos-system-backup` | 220 | Encrypted (AES-256) folder snapshots (tar + gpg) |
|
||||
| `bin/pos-system-backup` | 347 | Encrypted (AES-256) folder snapshots (tar + gpg) |
|
||||
| `bin/pos-system-firewall` | 308 | Interactive UFW management |
|
||||
| `bin/pos-system-health` | 209 | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL |
|
||||
| `bin/pos-system-schedule` | 81 | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently |
|
||||
|
||||
+1
-1
@@ -154,7 +154,7 @@ Two kinds of config, don't mix them up:
|
||||
|
||||
- **Machine defaults shipped by the installer:** place the file in `config/` and add copy logic to `postinstall.sh`. If it contains secrets, add to `.gitignore` and document in `DOC/`.
|
||||
- **Runtime tool config set by the user:** `~/.config/linux_post_install/<tool>.env` with `chmod 600`. Load it with env-var precedence (flags > environment > file). Patterns: `pos-docker-compose` (`compose.env`), `pos-communication-telegram-sender` (`telegram.env`, edited via `pos config telegram` — token masked), and the shared ones below. Never store tokens in the repo.
|
||||
- `system.env` — shared "system" settings loaded by `pos-system-*` tools via `load_system_env()` in `lib/common.sh` (currently `BACKUP_SERVICE_ROOTS`, `BACKUP_USB_ROOT`, `HEALTH_BACKUP_MAX_AGE_DAYS`). Env already exported wins over the file.
|
||||
- `system.env` — shared "system" settings loaded by `pos-system-*` tools via `load_system_env()` in `lib/common.sh` (currently `BACKUP_SERVICE_ROOTS`, `BACKUP_USB_ROOT`, `BACKUP_MOUNT_BASE`, `BACKUP_USB_BYID`, `HEALTH_BACKUP_MAX_AGE_DAYS`). Env already exported wins over the file.
|
||||
- `notify.env` — alerting platform selection (`NOTIFY_PLATFORM=telegram,matrix`), read by `lib/notify.sh`.
|
||||
|
||||
### 5. Add SSH keys (if needed)
|
||||
|
||||
+1
-1
@@ -212,7 +212,7 @@ The standalone `vbox` command still works and forwards to `pos docker vbox` (see
|
||||
| Command | File | Purpose | Configuration |
|
||||
|---------|------|---------|---------------|
|
||||
| `sudo pos system firewall` | `bin/pos-system-firewall` | Interactive UFW ("UFW POWER") menu: add/delete rules, status, enable/disable/reset, default policies | Must run as root. Every command is previewed and confirmed before execution; supports `--dry-run`; keeps a history of executed commands. Executed mutating changes are announced via `lib/notify.sh` |
|
||||
| `pos system backup <folder-path>` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES). Artifact `<name>_<date>.tar.gz.gpg` in the current directory, `chmod 600`. Once the archive verifies, connected USB storage is offered (auto-detected removable mounts via `lsblk`, or pinned with `BACKUP_USB_ROOT`): the copy lands in `<usb>/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh` |
|
||||
| `pos system backup <folder-path>` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES). Artifact `<name>_<date>.tar.gz.gpg` in the current directory, `chmod 600`. Once the archive verifies, connected USB storage is offered (detected via `lsblk` TRAN with `lsusb`/by-id cross-check; unmounted sticks get a mount offer first — see `DOC/howto/system.md`; or pinned with `BACKUP_USB_ROOT`): the copy lands in `<usb>/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh` |
|
||||
| `pos system backup --service` | `bin/pos-system-backup` | Lists folders under `/srv` and `~/srv`, lets you pick one, then runs the same backup | Roots via `BACKUP_SERVICE_ROOTS` (space-separated, default `/srv $HOME/srv`) or `~/.config/linux_post_install/system.env` |
|
||||
| `pos system health` | `bin/pos-system-health` | Host health dashboard: disk per mount, RAM/swap, failed systemd units, backup age, fail2ban, docker containers. Exits 1 if any check FAILs | Console-only reporter — health itself never sends notifications; forward the output with a wrapper (e.g. the Telegram/Matrix listener map `/status=pos system health`) or schedule it via `pos system schedule` with a `NOTIFY` policy. `HEALTH_BACKUP_MAX_AGE_DAYS` (default 2) and `BACKUP_SERVICE_ROOTS` come from `~/.config/linux_post_install/system.env`; `--help` shows the effective values |
|
||||
| `pos system schedule <cmd>` | `bin/pos-system-schedule` | Scheduled jobs — run a command on a timer, notify (or stay silent): `run [name\|all]`, `list`, `config`, `enable [name\|all]`, `disable [name\|all]`, `status`, `migrate`. Each job is a file in `~/.config/linux_post_install/schedule.d/<name>.env` with `INTERVAL` (`5m…59m`, `1h…23h`, `hourly`, `daily`, `weekly`, `OnCalendar=…`), `NOTIFY` policy, optional `MSG`, `RULE` (threshold only), and `COMMAND` = the literal rest of the line (pipes/quotes/`sudo` fine). Policies: `always` (full output every run), `onchange` (send when output differs from the last run; first run always sends), `onerror` (non-zero exit or empty output), `threshold` (first numeric output vs `RULE`, alert on false→true + one recovery — the old event-trigger behavior), `never` (side-effect jobs, no notify) | One systemd **user** timer pair per job (`pos-schedule-<name>.timer` + oneshot `.service`, `Persistent=true`), reconciled on `enable`/`disable`; the legacy single `pos-event-trigger` timer is auto-removed. `migrate` converts a pre-existing `event.env` rule set into `schedule.d/rule-N.env` threshold jobs. `config` is an interactive editor (add/edit/remove/enable/disable, validates interval + threshold); alerts via `lib/notify.sh`; `--dry-run` previews runs/writes/sends; jobs are arbitrary shell commands (chmod 600, same trust model as the Telegram map); starter jobs in `config/schedule.d/` auto-installed no-clobber by postinstall |
|
||||
|
||||
@@ -106,11 +106,37 @@ pos system backup ~/Documents
|
||||
# OK Transfer verified 100% (sha256 match): .../backups/docs_2026-08-13.tar.gz.gpg
|
||||
```
|
||||
|
||||
Detection reads `lsblk` and treats a device as USB when `TRAN == usb` (the
|
||||
per-device deciding signal). Removable-but-not-USB slots (e.g. a SATA card
|
||||
reader) are skipped. If a device shows no `TRAN` at all, `lsblk`'s answer is
|
||||
cross-checked against `/dev/disk/by-id/usb-*` symlinks and `lsusb` before it is
|
||||
offered.
|
||||
|
||||
**Unmounted stick?** If the USB stick is plugged in but only shows as `sdax`
|
||||
with no mountpoint (common on CLI boxes with no automounter), you're offered a
|
||||
**mount first**, then it copies there:
|
||||
|
||||
```bash
|
||||
# [!] Found USB storage not mounted: /dev/sda1 (7.5G, DataTraveler)
|
||||
# Mount it at /media/usb-sda1 (world-writable) so the backup can go there? [y/N]
|
||||
# (y) OK Mounted /dev/sda1 at /media/usb-sda1
|
||||
# [+] Copying to /media/usb-sda1/backups/docs_2026-08-13.tar.gz.gpg ...
|
||||
# OK Transfer verified 100% (sha256 match): .../backups/docs_2026-08-13.tar.gz.gpg
|
||||
```
|
||||
|
||||
The mount mirrors `usb-automount` (`/media/<label>`, fallback
|
||||
`/media/usb-<devname>`, `-o umask=000` world-writable so the copy works without
|
||||
root). Decline it and you get the exact `sudo mkdir -p` / `sudo mount` commands
|
||||
to run yourself, then `Enter` re-checks. `s` or EOF (cron) skips silently and
|
||||
the backup stays local — it never blocks.
|
||||
|
||||
- Multiple sticks mounted → pick by number; `0` skips; `n`/EOF skips silently
|
||||
(cron runs never block).
|
||||
- Pin a fixed stick (no detection, no prompt on cron) with
|
||||
`BACKUP_USB_ROOT=/mnt/usb` in `system.env` — the copy still lands in
|
||||
`<root>/backups/` and is still sha256-verified.
|
||||
- Mount base and by-id dir are configurable: `BACKUP_MOUNT_BASE` (default
|
||||
`/media`) and `BACKUP_USB_BYID` (default `/dev/disk/by-id`).
|
||||
|
||||
### Recipes
|
||||
|
||||
|
||||
+134
-7
@@ -26,13 +26,17 @@ Modes:
|
||||
|
||||
The final artifact <name>_<date>.tar.gz.gpg is written to the current directory.
|
||||
After it verifies, connected USB storage is offered: the copy lands in
|
||||
<usb>/backups/ and is sha256-verified 100% before it is announced.
|
||||
<usb>/backups/ and is sha256-verified 100% before it is announced. A stick
|
||||
that is plugged in but not mounted is offered a mount first (sudo, mirrors
|
||||
the usb-automount scheme) before the copy.
|
||||
|
||||
Environment:
|
||||
BACKUP_SERVICE_ROOTS Space-separated roots for --service
|
||||
(effective: ${EFF_ROOTS})
|
||||
BACKUP_USB_ROOT USB root to copy finished backups to
|
||||
(default: auto-detect mounted removable storage)
|
||||
(default: auto-detect mounted USB storage)
|
||||
BACKUP_MOUNT_BASE Where to mount an unmounted USB stick offered
|
||||
during the copy (default: /media)
|
||||
(loaded from ~/.config/linux_post_install/system.env unless exported)
|
||||
EOF
|
||||
exit 0
|
||||
@@ -46,10 +50,18 @@ command -v gpg &>/dev/null || err "gpg not found (install gnupg)"
|
||||
# the archive was being made is found. The copy lands in <usb>/backups/
|
||||
# and the transfer is proven 100% (sha256 source vs copy) before any
|
||||
# success is announced. BACKUP_USB_ROOT pins the root and skips
|
||||
# detection; otherwise mounted removable storage is auto-detected.
|
||||
# detection; otherwise USB storage is auto-detected.
|
||||
#
|
||||
# "Is it really USB?" — lsblk's TRAN column decides per device (rm==1
|
||||
# card readers / hot-swap bays report sata and are skipped). When TRAN
|
||||
# is unavailable, /dev/disk/by-id/usb-* symlinks and lsusb text
|
||||
# corroborate. A stick that is plugged in but not mounted (the CLI-box
|
||||
# case) is offered a mount (sudo, usb-automount scheme: /media/<label>,
|
||||
# fallback /media/usb-<devname>) before the copy; cron/EOF skips.
|
||||
usb_copy_offer() {
|
||||
local archive="$1" pass=0 resp="" i=0 root="" dest_dir="" dest="" src_sum="" dst_sum=""
|
||||
local -a roots=()
|
||||
local offered=0
|
||||
|
||||
command -v lsblk &>/dev/null || { warn "lsblk not found — USB copy skipped"; return 0; }
|
||||
command -v jq &>/dev/null || { warn "jq not found — USB copy skipped"; return 0; }
|
||||
@@ -63,10 +75,21 @@ usb_copy_offer() {
|
||||
if [ -n "${BACKUP_USB_ROOT:-}" ]; then
|
||||
roots+=("$BACKUP_USB_ROOT")
|
||||
else
|
||||
while IFS= read -r mp; do
|
||||
[ -n "$mp" ] && roots+=("$mp")
|
||||
done < <(lsblk -J -o MOUNTPOINT,RM,TYPE 2>/dev/null \
|
||||
| jq -r '.. | objects | select(.rm == true and .mountpoint != null and (.type == "part" or .type == "disk")) | .mountpoint')
|
||||
detect_usb
|
||||
roots=("${USB_MOUNTED[@]}")
|
||||
|
||||
if [ ${#roots[@]} -eq 0 ] && [ ${#USB_UNMOUNTED[@]} -gt 0 ]; then
|
||||
if [ "$offered" -ge 1 ]; then
|
||||
warn "USB stick still not mounted — backup stays local: $archive"
|
||||
return 0
|
||||
fi
|
||||
offered=1
|
||||
mount_offer "${USB_UNMOUNTED[@]}" || {
|
||||
log "Skipped — backup stays local: $archive"
|
||||
return 0
|
||||
}
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
[ ${#roots[@]} -gt 0 ] && break
|
||||
@@ -125,6 +148,110 @@ usb_copy_offer() {
|
||||
notify_send "Backup copied to USB: $dest (sha256 verified)"
|
||||
}
|
||||
|
||||
# ── USB detection ────────────────────────────────────────────────
|
||||
# Fills USB_MOUNTED (mountpoints, one per removable USB storage) and
|
||||
# USB_UNMOUNTED ("path|label|size|model" entries) from lsblk JSON.
|
||||
# TRAN=="usb" is the per-device deciding signal; when TRAN is empty,
|
||||
# usb_related_present() corroborates. Non-USB removables are skipped.
|
||||
detect_usb() {
|
||||
local out path="" mp="" label="" size="" model="" tran="" type="" children=""
|
||||
USB_MOUNTED=()
|
||||
USB_UNMOUNTED=()
|
||||
out="$(lsblk -J -o NAME,PATH,LABEL,MOUNTPOINT,RM,TYPE,TRAN,SIZE,MODEL 2>/dev/null)" || return 0
|
||||
|
||||
while IFS=$'\x1f' read -r path mp label size model tran type children; do
|
||||
[ -n "$path" ] || continue
|
||||
if [ "$tran" = "usb" ]; then
|
||||
:
|
||||
elif [ -z "$tran" ] && usb_related_present; then
|
||||
warn "TRAN unavailable — assuming USB (lsusb/by-id corroboration)"
|
||||
else
|
||||
continue
|
||||
fi
|
||||
if [ -n "$mp" ]; then
|
||||
USB_MOUNTED+=("$mp")
|
||||
elif [ "$children" = "0" ]; then
|
||||
USB_UNMOUNTED+=("$path|$label|$size|$model")
|
||||
fi
|
||||
done < <(printf '%s' "$out" | jq -r '
|
||||
.. | objects
|
||||
| select(.rm == true and (.type == "part" or .type == "disk"))
|
||||
| [.path, (.mountpoint // ""), (.label // ""), (.size // ""),
|
||||
(.model // ""), (.tran // ""), (.type // ""),
|
||||
((.children // []) | length)]
|
||||
| join("\u001f")')
|
||||
}
|
||||
|
||||
# Whole-system "is any USB storage attached?" — udev by-id usb-* symlinks
|
||||
# are definitive; lsusb text is a secondary hint for storage-ish devices.
|
||||
# BACKUP_USB_BYID points at the by-id dir (test seam; default real one).
|
||||
usb_related_present() {
|
||||
[ -n "$(ls "${BACKUP_USB_BYID:-/dev/disk/by-id}"/usb-* 2>/dev/null)" ] && return 0
|
||||
command -v lsusb &>/dev/null \
|
||||
&& lsusb 2>/dev/null | grep -qiE 'mass storage|card reader|flash disk|usb.*(disk|drive|storage)|reader|external'
|
||||
}
|
||||
|
||||
# ── Mount an unmounted USB stick (CLI-box case) ──────────────────
|
||||
# Asks to mount a detected-but-unmounted device (sudo, mirroring
|
||||
# usb-automount: /media/<label>, fallback /media/usb-<devname>, -o
|
||||
# umask=000 world-writable). Returns 0 → caller re-scans (mounted now
|
||||
# or the user will mount manually); 1 → skipped/EOF (backup stays local).
|
||||
mount_offer() {
|
||||
local -a devs=("$@")
|
||||
local i=0 dev="" label="" size="" model="" mp="" resp="" n=""
|
||||
local -a chosen=()
|
||||
|
||||
if [ ${#devs[@]} -gt 1 ]; then
|
||||
echo "Multiple unmounted USB storages found:"
|
||||
for i in "${!devs[@]}"; do
|
||||
IFS='|' read -r dev label size model <<< "${devs[$i]}"
|
||||
printf "%2d) %s (%s, %s)\n" "$((i + 1))" "$dev" "$size" "${model:-no label}"
|
||||
done
|
||||
read -rp "Copy backup to which one? [1-${#devs[@]}] (0 = skip): " resp || return 1
|
||||
if ! [[ "$resp" =~ ^[0-9]+$ ]] || (( resp < 1 || resp > ${#devs[@]} )); then
|
||||
return 1
|
||||
fi
|
||||
chosen=("${devs[$((resp - 1))]}")
|
||||
else
|
||||
chosen=("${devs[0]}")
|
||||
fi
|
||||
|
||||
IFS='|' read -r dev label size model <<< "${chosen[0]}"
|
||||
label="${label//\//_}"
|
||||
mp="${BACKUP_MOUNT_BASE:-/media}/${label:-usb-$(basename "$dev")}"
|
||||
if [ -d "$mp" ] && mountpoint -q "$mp" 2>/dev/null; then
|
||||
n=2
|
||||
while [ -d "${mp}-${n}" ] && mountpoint -q "${mp}-${n}" 2>/dev/null; do
|
||||
n=$((n + 1))
|
||||
done
|
||||
mp="${mp}-${n}"
|
||||
fi
|
||||
|
||||
echo "Found USB storage not mounted: $dev ($size, ${model:-no label})"
|
||||
if confirm "Mount it at $mp (world-writable) so the backup can go there?" n; then
|
||||
run sudo mkdir -p "$mp"
|
||||
if ! run sudo mount -o umask=000 "$dev" "$mp"; then
|
||||
run sudo mount "$dev" "$mp" || true
|
||||
fi
|
||||
if [ "${DRY_RUN:-0}" -eq 1 ] || mountpoint -q "$mp" 2>/dev/null; then
|
||||
ok "Mounted $dev at $mp"
|
||||
return 0
|
||||
fi
|
||||
warn "Mount failed — do it manually, then we'll copy:"
|
||||
log " sudo mkdir -p $mp && sudo mount $dev $mp"
|
||||
else
|
||||
warn "Mount it manually, then we'll copy:"
|
||||
log " sudo mkdir -p $mp"
|
||||
log " sudo mount $dev $mp"
|
||||
fi
|
||||
|
||||
read -rp "Press Enter once mounted (or 's' to skip): " resp || return 1
|
||||
case "$resp" in
|
||||
s|S) return 1 ;;
|
||||
esac
|
||||
return 0
|
||||
}
|
||||
|
||||
SERVICE=0
|
||||
case "${1:-}" in
|
||||
-h|--help) usage ;;
|
||||
|
||||
@@ -10,3 +10,14 @@
|
||||
|
||||
# Max backup age in days before `pos system health` raises a WARN. Default: 2
|
||||
#HEALTH_BACKUP_MAX_AGE_DAYS=3
|
||||
|
||||
# Pin a fixed USB stick for `pos system backup` (no detection, no prompt on
|
||||
# cron). The copy lands in <root>/backups/ and is still sha256-verified.
|
||||
#BACKUP_USB_ROOT=/mnt/usb
|
||||
|
||||
# Where `pos system backup` mounts an unmounted stick it found. Default: /media
|
||||
#BACKUP_MOUNT_BASE=/media
|
||||
|
||||
# Dir holding by-id symlinks used to cross-check devices that show no TRAN in
|
||||
# lsblk. Default: /dev/disk/by-id
|
||||
#BACKUP_USB_BYID=/dev/disk/by-id
|
||||
|
||||
Reference in New Issue
Block a user