feat: pos system backup copies to USB after verification — sha256-proven 100%
Once the archive verifies, USB detection runs (so a stick plugged in while
the backup ran is found): mounted removable storage is auto-detected via
lsblk -J + a recursive jq filter (rm, mounted, type part|disk — JSON makes
spacey mountpoints safe), or BACKUP_USB_ROOT pins a fixed stick and skips
detection. None mounted → one re-scan prompt ('s' skips, EOF from cron
skips silently, rc stays 0); one stick → y/N confirm; several → numbered
pick (0 = skip). The copy lands in <usb>/backups/ (mkdir -p, chmod 600
best-effort — a vfat chmod failure warns, never fails the copy) and the
transfer is proven 100% by sha256 source-vs-copy before any success is
announced; a mismatch warns with both hashes, notifies 'USB copy FAILED',
and exits 1. The ERR trap is re-armed before the USB phase so a copy
failure no longer notifies 'Backup FAILED'. Docs: usage() Environment,
POS.md backup row, howto/system.md (USB section + env table + mismatch
troubleshooting), DEV.md system.env list. Stub suite
(/tmp/opencode/backup-test, HOME-isolated, sudo/gpg/lsblk/sender stubs,
corrupting-cp + vfat-chmod overrides, per-test lsblk JSON fixtures):
40/40 green.
This commit is contained in:
Your Name
@@ -16,6 +16,7 @@ summary (newest last).
|
||||
|
||||
## Done
|
||||
|
||||
- **2026-08-13** — `pos system backup` copies the finished backup to a USB stick. Detection runs **after** the archive verifies (so a stick plugged in while the backup ran is found; if none is mounted, one re-scan prompt before giving up — `s` skips, EOF from cron skips silently, rc stays 0). Single stick → y/N confirm; several → numbered pick (0 = skip). Copy lands in `<usb>/backups/` (mkdir -p; `chmod 600` best-effort — vfat chmod failures warn, never fail), and the transfer is proven 100% by **sha256 source-vs-copy** before any success is announced: mismatch → warn with both hashes + `notify_send "USB copy FAILED…"` + rc=1 (the ERR trap is re-armed mid-script so a USB-phase failure no longer notifies "Backup FAILED"). Detection: `lsblk -J` → recursive jq filter (rm==true && mounted && type part|disk, space-safe via JSON) or pinned `BACKUP_USB_ROOT` seam (= `<root>/backups/`, skips detection — also the test seam). Docs: usage() Environment, POS.md backup row, howto/system.md (USB section + env table + mismatch troubleshooting), DEV.md system.env list. Verified with a stub suite (`/tmp/opencode/backup-test` — sudo/gpg/lsblk/sender stubs, HOME isolation, per-test lsblk JSON fixtures, corrupting-cp + vfat-chmod override stubs): 40/40 green (skip s/EOF, seam y/n, detect single, multi pick 2/0, re-scan after replug, corrupt copy rc=1 + honest notify, vfat tolerance). `make gen && make check` green.
|
||||
- **2026-08-13** — `pos share smb-server share` now guards the two common `NT_STATUS_ACCESS_DENIED` causes at share time (warnings only): `--users` entries missing from the Samba passdb (`pdbedit -L`, cut to user column, `grep -qxF` per user — pointer to `pos share smb-server adduser <user>`), and ancestors of the share path lacking `other:+x` traversal (sticky dirs like `/tmp` count as traversable via the `t` slot; fix hint `chmod o+x <dir>`). Both wired into the `share` case after `require_root_dir`; howto/share.md SMB section + troubleshooting updated. Rooted in `reports/bug-report-smb-server-access-denied.md` (committed as the spec). Verified with a stub-PATH suite (`/tmp/opencode/smb-test` — pdbedit/systemctl/smbcontrol/testparm/smbpasswd stubs, `SMB_CONF` seam): 16/16 green.
|
||||
- **2026-08-13** — Docs hardening from the schedule-session review (sole-developer call: terse, session-learned). DEV.md §7 env-seam registry now lists `USER_SYSTEMD_DIR` (`bin/pos-network-download`, `bin/pos-communication-{telegram,matrix}-listener`, `lib/scheduler-lib.sh`) + the scheduler's `SCHEDULE_*` seams, and documents the missing-`:-`-guard gotcha (a `VAR="${XDG…:-…}"` without leading `VAR:-` overrides the seam — stub runs then silently write to the real `$HOME`; fix: `USER_SYSTEMD_DIR="${USER_SYSTEMD_DIR:-…}"`). New-tool test checklist gains an env-seam review step (grep for unguarded config writes + prove with `VAR=/tmp/x`). §7 notes stub harnesses are throwaway by design — build in `/tmp/opencode/<tool>-test/`, leave there, keep only the pattern. howto/schedule.md documents that `migrate` copies the rule LHS **verbatim** as `COMMAND` (old tool never had `disk root`/`loadavg` shorthands — rewrite those jobs with real commands). `make check` green.
|
||||
- **2026-08-12** — `pos system event-trigger` (eventer) generalized into `pos system schedule` — the scheduler replaces the single-timer threshold monitor with **per-job systemd user timers** (`pos-schedule-<name>.{timer,service}`, `Persistent`, ExecStart `run <name>`, reconciled on `enable`/`disable` — orphan units + the legacy `pos-event-trigger` timer auto-removed). Each job is a chmod-600 file `~/.config/linux_post_install/schedule.d/<name>.env`: `INTERVAL` (`5m..59m`/`1h..23h`/`hourly`/`daily`/`weekly`/`OnCalendar=…`), `NOTIFY` policy, optional `MSG`, `RULE` (threshold only), and `COMMAND` = **literal remainder of the line** (pipes/quotes/`sudo` need no escaping). Policies: `always` (full output every run), `onchange` (diff vs last run, first run always sends), `onerror` (non-zero exit or empty output), `threshold` (old event-trigger behavior: first numeric vs `RULE`, alert on false→true + recovery, per-job firing state), `never` (silent side-effect jobs — no notify; run log + last-run record still kept). Per-run logs/state in `~/.local/share/linux_post_install/schedule/{logs,state}/`. Subcommands: `run [name|all]`, `list`, `config` (interactive add/edit/remove/enable/disable with validation), `enable [name|all]`, `disable [name|all]`, `status`, `migrate` (converts legacy `event.env` rules → `schedule.d/rule-N.env` threshold jobs, adopts the legacy timer's OnCalendar or 5m, removes the old timer). Files: `bin/pos-system-event-trigger` → `bin/pos-system-schedule`, `lib/eventer-lib.sh` → `lib/scheduler-lib.sh` (git mv; installed by install.sh), `config/event.env` + `config/event-rules.template` → `config/schedule.d/` starter jobs (nvme-health via `sudo -n smartctl` with the user's exact grep — sudoers NOPASSWD documented; cpu-temp + disk-root thresholds; silent log-cleanup), postinstall installs them no-clobber into an empty `schedule.d/` (legacy `event.env` users get a migrate hint instead). `bin/pos` EXAMPLES + INTERACTIVE_CMDS (`system-schedule config`) updated. Supersedes the "Tier 2: watch plugins" backlog idea. Docs: POS.md system row rewritten, howto/event-trigger.md → howto/schedule.md (job syntax, policies, NVMe recipe, migration), HOWTO.md index row + config table + scheduling bullet, AGENT_Context lib row + Common Tasks row. `make gen && make check` green; stub-harness suite (fake `systemctl`/`sudo`/`smartctl`/`sensors`/`df` + fake telegram sender logging, env seams `SCHEDULE_DIR`/`SCHEDULE_STATE_DIR`/`SCHEDULE_LOG_DIR`/`USER_SYSTEMD_DIR`/`SCHED_LEGACY_ENV`) covers all 5 policies (threshold cross/recover/no-repeat, onchange first/diff/same, onerror, always, never-silent), COMMAND literal-pipe parsing, enable/disable/status + orphan/legacy cleanup, migrate (incl. skip-existing + dry-run), and dispatch.
|
||||
|
||||
@@ -601,7 +601,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
|
||||
| `bin/pos-share-smb-server` | 253 | Manage the Samba server (status, share/unshare exports, users, enable/disable) |
|
||||
| `bin/pos-share-usb-server` | 218 | USB Redirector server control (--ls, --share; prompts when args omitted) |
|
||||
| `bin/pos-ssh-load-keys` | 31 | Load all SSH keys into the agent |
|
||||
| `bin/pos-system-backup` | 126 | Encrypted (AES-256) folder snapshots (tar + gpg) |
|
||||
| `bin/pos-system-backup` | 220 | Encrypted (AES-256) folder snapshots (tar + gpg) |
|
||||
| `bin/pos-system-firewall` | 291 | Interactive UFW management |
|
||||
| `bin/pos-system-health` | 209 | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL |
|
||||
| `bin/pos-system-schedule` | 81 | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently |
|
||||
|
||||
+1
-1
@@ -153,7 +153,7 @@ Two kinds of config, don't mix them up:
|
||||
|
||||
- **Machine defaults shipped by the installer:** place the file in `config/` and add copy logic to `postinstall.sh`. If it contains secrets, add to `.gitignore` and document in `DOC/`.
|
||||
- **Runtime tool config set by the user:** `~/.config/linux_post_install/<tool>.env` with `chmod 600`. Load it with env-var precedence (flags > environment > file). Patterns: `pos-docker-compose` (`compose.env`), `pos-communication-telegram-sender` (`telegram.env`, edited via `pos config telegram` — token masked), and the shared ones below. Never store tokens in the repo.
|
||||
- `system.env` — shared "system" settings loaded by `pos-system-*` tools via `load_system_env()` in `lib/common.sh` (currently `BACKUP_SERVICE_ROOTS`, `HEALTH_BACKUP_MAX_AGE_DAYS`). Env already exported wins over the file.
|
||||
- `system.env` — shared "system" settings loaded by `pos-system-*` tools via `load_system_env()` in `lib/common.sh` (currently `BACKUP_SERVICE_ROOTS`, `BACKUP_USB_ROOT`, `HEALTH_BACKUP_MAX_AGE_DAYS`). Env already exported wins over the file.
|
||||
- `notify.env` — alerting platform selection (`NOTIFY_PLATFORM=telegram,matrix`), read by `lib/notify.sh`.
|
||||
|
||||
### 5. Add SSH keys (if needed)
|
||||
|
||||
+1
-1
@@ -212,7 +212,7 @@ The standalone `vbox` command still works and forwards to `pos docker vbox` (see
|
||||
| Command | File | Purpose | Configuration |
|
||||
|---------|------|---------|---------------|
|
||||
| `sudo pos system firewall` | `bin/pos-system-firewall` | Interactive UFW ("UFW POWER") menu: add/delete rules, status, enable/disable/reset, default policies | Must run as root. Every command is previewed and confirmed before execution; supports `--dry-run`; keeps a history of executed commands. Executed mutating changes are announced via `lib/notify.sh` |
|
||||
| `pos system backup <folder-path>` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES). Artifact `<name>_<date>.tar.gz.gpg` in the current directory, `chmod 600`. Success/failure are announced via `lib/notify.sh` |
|
||||
| `pos system backup <folder-path>` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES). Artifact `<name>_<date>.tar.gz.gpg` in the current directory, `chmod 600`. Once the archive verifies, connected USB storage is offered (auto-detected removable mounts via `lsblk`, or pinned with `BACKUP_USB_ROOT`): the copy lands in `<usb>/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh` |
|
||||
| `pos system backup --service` | `bin/pos-system-backup` | Lists folders under `/srv` and `~/srv`, lets you pick one, then runs the same backup | Roots via `BACKUP_SERVICE_ROOTS` (space-separated, default `/srv $HOME/srv`) or `~/.config/linux_post_install/system.env` |
|
||||
| `pos system health [--send] [--markdown]` | `bin/pos-system-health` | Host health dashboard: disk per mount, RAM/swap, failed systemd units, backup age, fail2ban, docker containers. Exits 1 if any check FAILs | `--send`/`--markdown` are notification-only: they send the summary via `lib/notify.sh` to every platform in `NOTIFY_PLATFORM` and do NOT print the dashboard (so wrappers like the Telegram listener don't echo it back — pair with the listener's `@quiet` marker). `HEALTH_BACKUP_MAX_AGE_DAYS` (default 2) and `BACKUP_SERVICE_ROOTS` come from `~/.config/linux_post_install/system.env`; `--help` shows the effective values. Platform list from `~/.config/linux_post_install/notify.env` |
|
||||
| `pos system schedule <cmd>` | `bin/pos-system-schedule` | Scheduled jobs — run a command on a timer, notify (or stay silent): `run [name\|all]`, `list`, `config`, `enable [name\|all]`, `disable [name\|all]`, `status`, `migrate`. Each job is a file in `~/.config/linux_post_install/schedule.d/<name>.env` with `INTERVAL` (`5m…59m`, `1h…23h`, `hourly`, `daily`, `weekly`, `OnCalendar=…`), `NOTIFY` policy, optional `MSG`, `RULE` (threshold only), and `COMMAND` = the literal rest of the line (pipes/quotes/`sudo` fine). Policies: `always` (full output every run), `onchange` (send when output differs from the last run; first run always sends), `onerror` (non-zero exit or empty output), `threshold` (first numeric output vs `RULE`, alert on false→true + one recovery — the old event-trigger behavior), `never` (side-effect jobs, no notify) | One systemd **user** timer pair per job (`pos-schedule-<name>.timer` + oneshot `.service`, `Persistent=true`), reconciled on `enable`/`disable`; the legacy single `pos-event-trigger` timer is auto-removed. `migrate` converts a pre-existing `event.env` rule set into `schedule.d/rule-N.env` threshold jobs. `config` is an interactive editor (add/edit/remove/enable/disable, validates interval + threshold); alerts via `lib/notify.sh`; `--dry-run` previews runs/writes/sends; jobs are arbitrary shell commands (chmod 600, same trust model as the Telegram map); starter jobs in `config/schedule.d/` auto-installed no-clobber by postinstall |
|
||||
|
||||
@@ -39,6 +39,7 @@ Environment (effective values):
|
||||
```bash
|
||||
# ~/.config/linux_post_install/system.env (comment-only defaults — uncomment to override)
|
||||
BACKUP_SERVICE_ROOTS=/srv $HOME/srv # roots for backup-age check + backup --service
|
||||
BACKUP_USB_ROOT=/mnt/usb # optional: copy finished backups to <root>/backups/ (auto-detects a mounted USB when unset)
|
||||
HEALTH_BACKUP_MAX_AGE_DAYS=3 # WARN if newest backup older (default 2)
|
||||
# ~/.config/linux_post_install/notify.env
|
||||
NOTIFY_PLATFORM=telegram
|
||||
@@ -86,6 +87,29 @@ trap) a `notify_send` alert is sent.
|
||||
`--service` lists folders under the roots in `BACKUP_SERVICE_ROOTS`
|
||||
(default `/srv $HOME/srv`; override via `system.env` or env) and lets you pick.
|
||||
|
||||
### Copy to a USB stick
|
||||
|
||||
After the archive verifies, connected USB storage is **detected** (so a stick
|
||||
plugged in while the backup was running is found — if none is mounted you get
|
||||
one chance to plug one in and re-check) and you're asked whether to copy the
|
||||
backup there. The copy lands in `<usb>/backups/` and is **verified 100%**
|
||||
(sha256 source vs copy) before any success is announced:
|
||||
|
||||
```bash
|
||||
pos system backup ~/Documents
|
||||
# ... after the archive verifies:
|
||||
# [!] No USB storage detected
|
||||
# Plug a USB drive in now and press Enter to re-check (or 's' to skip):
|
||||
# [+] Copying to /media/you/USB-DISK/backups/docs_2026-08-13.tar.gz.gpg ...
|
||||
# OK Transfer verified 100% (sha256 match): .../backups/docs_2026-08-13.tar.gz.gpg
|
||||
```
|
||||
|
||||
- Multiple sticks mounted → pick by number; `0` skips; `n`/EOF skips silently
|
||||
(cron runs never block).
|
||||
- Pin a fixed stick (no detection, no prompt on cron) with
|
||||
`BACKUP_USB_ROOT=/mnt/usb` in `system.env` — the copy still lands in
|
||||
`<root>/backups/` and is still sha256-verified.
|
||||
|
||||
### Recipes
|
||||
|
||||
- **Nightly service backup + health check:**
|
||||
@@ -105,6 +129,9 @@ trap) a `notify_send` alert is sent.
|
||||
- Forgot the password → backups are unrecoverable; keep the passphrase in a
|
||||
password manager. Nothing is stored anywhere else.
|
||||
- `sudo tar` prompt: ensure the user has sudo rights for the source dir.
|
||||
- "USB copy FAILED verification — checksum mismatch" → the copy is corrupt
|
||||
(bad stick or transfer); the local archive is untouched — copy it again
|
||||
manually and replace the file on the stick.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -25,10 +25,14 @@ Modes:
|
||||
--service List folders under /srv and ~/srv, pick one, back it up.
|
||||
|
||||
The final artifact <name>_<date>.tar.gz.gpg is written to the current directory.
|
||||
After it verifies, connected USB storage is offered: the copy lands in
|
||||
<usb>/backups/ and is sha256-verified 100% before it is announced.
|
||||
|
||||
Environment:
|
||||
BACKUP_SERVICE_ROOTS Space-separated roots for --service
|
||||
(effective: ${EFF_ROOTS})
|
||||
BACKUP_USB_ROOT USB root to copy finished backups to
|
||||
(default: auto-detect mounted removable storage)
|
||||
(loaded from ~/.config/linux_post_install/system.env unless exported)
|
||||
EOF
|
||||
exit 0
|
||||
@@ -37,6 +41,90 @@ EOF
|
||||
command -v tar &>/dev/null || err "tar not found"
|
||||
command -v gpg &>/dev/null || err "gpg not found (install gnupg)"
|
||||
|
||||
# ── USB copy (optional post-backup step) ─────────────────────────
|
||||
# Detection runs AFTER the backup finished, so a stick plugged in while
|
||||
# the archive was being made is found. The copy lands in <usb>/backups/
|
||||
# and the transfer is proven 100% (sha256 source vs copy) before any
|
||||
# success is announced. BACKUP_USB_ROOT pins the root and skips
|
||||
# detection; otherwise mounted removable storage is auto-detected.
|
||||
usb_copy_offer() {
|
||||
local archive="$1" pass=0 resp="" i=0 root="" dest_dir="" dest="" src_sum="" dst_sum=""
|
||||
local -a roots=()
|
||||
|
||||
command -v lsblk &>/dev/null || { warn "lsblk not found — USB copy skipped"; return 0; }
|
||||
command -v jq &>/dev/null || { warn "jq not found — USB copy skipped"; return 0; }
|
||||
|
||||
section "USB copy"
|
||||
|
||||
while :; do
|
||||
pass=$((pass + 1))
|
||||
roots=()
|
||||
|
||||
if [ -n "${BACKUP_USB_ROOT:-}" ]; then
|
||||
roots+=("$BACKUP_USB_ROOT")
|
||||
else
|
||||
while IFS= read -r mp; do
|
||||
[ -n "$mp" ] && roots+=("$mp")
|
||||
done < <(lsblk -J -o MOUNTPOINT,RM,TYPE 2>/dev/null \
|
||||
| jq -r '.. | objects | select(.rm == true and .mountpoint != null and (.type == "part" or .type == "disk")) | .mountpoint')
|
||||
fi
|
||||
|
||||
[ ${#roots[@]} -gt 0 ] && break
|
||||
|
||||
# None found — offer one re-scan before giving up.
|
||||
if [ "$pass" -ge 2 ]; then
|
||||
warn "Still no USB storage detected — backup stays local: $archive"
|
||||
return 0
|
||||
fi
|
||||
warn "No USB storage detected"
|
||||
read -rp "Plug a USB drive in now and press Enter to re-check (or 's' to skip): " resp || return 0
|
||||
case "$resp" in
|
||||
s|S) log "Skipped — backup stays local: $archive"; return 0 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ ${#roots[@]} -eq 1 ]; then
|
||||
root="${roots[0]}"
|
||||
if ! confirm "Copy backup to ${root%/}/backups/?" n; then
|
||||
log "Skipped — backup stays local: $archive"
|
||||
return 0
|
||||
fi
|
||||
else
|
||||
echo "Multiple USB storages found:"
|
||||
for i in "${!roots[@]}"; do
|
||||
printf "%2d) %s\n" "$((i + 1))" "${roots[$i]}"
|
||||
done
|
||||
read -rp "Copy backup to which one? [1-${#roots[@]}] (0 = skip): " resp || return 0
|
||||
if ! [[ "$resp" =~ ^[0-9]+$ ]] || (( resp < 1 || resp > ${#roots[@]} )); then
|
||||
log "Skipped — backup stays local: $archive"
|
||||
return 0
|
||||
fi
|
||||
root="${roots[$((resp - 1))]}"
|
||||
fi
|
||||
|
||||
dest_dir="${root%/}/backups"
|
||||
dest="$dest_dir/$(basename "$archive")"
|
||||
|
||||
mkdir -p "$dest_dir"
|
||||
log "Copying to $dest ..."
|
||||
cp "$archive" "$dest_dir/"
|
||||
chmod 600 "$dest" 2>/dev/null \
|
||||
|| warn "Could not chmod 600 the USB copy (vfat filesystem?)"
|
||||
|
||||
log "Verifying transfer (sha256)..."
|
||||
src_sum="$(sha256sum "$archive" | cut -d' ' -f1)"
|
||||
dst_sum="$(sha256sum "$dest" | cut -d' ' -f1)"
|
||||
if [ "$src_sum" != "$dst_sum" ]; then
|
||||
warn "USB copy FAILED verification — checksum mismatch:"
|
||||
warn " source: $src_sum $archive"
|
||||
warn " copy : $dst_sum $dest"
|
||||
notify_send "USB copy FAILED for $archive — checksum mismatch on $dest"
|
||||
exit 1
|
||||
fi
|
||||
ok "Transfer verified 100% (sha256 match): $dest"
|
||||
notify_send "Backup copied to USB: $dest (sha256 verified)"
|
||||
}
|
||||
|
||||
SERVICE=0
|
||||
case "${1:-}" in
|
||||
-h|--help) usage ;;
|
||||
@@ -124,3 +212,9 @@ unset PASS
|
||||
echo
|
||||
log "Backup completed: $ARCHIVE"
|
||||
notify_send "Backup completed: $ARCHIVE"
|
||||
|
||||
# Optional: detect a USB stick connected after the backup finished, offer to
|
||||
# copy the archive to <usb>/backups/, and prove the transfer 100%. From here
|
||||
# on a failure is a USB-copy problem, not a backup problem.
|
||||
trap 'notify_send "USB copy FAILED: ${ARCHIVE:-unknown}"' ERR
|
||||
usb_copy_offer "$ARCHIVE"
|
||||
|
||||
Reference in New Issue
Block a user