From 03dd92370ce2e8b3b78473e1ff17ea6d2358ea95 Mon Sep 17 00:00:00 2001 From: he Date: Fri, 14 Aug 2026 17:19:17 -0400 Subject: [PATCH] feat: pos system backup -- optional --no-encrypt (BACKUP_ENCRYPT=0) --- AGENT_TODO.md | 1 + DOC/AGENT_Context_Project.md | 2 +- DOC/POS.md | 2 +- DOC/howto/system.md | 12 +++++- bin/pos-system-backup | 75 ++++++++++++++++++++++-------------- completions/pos.bash | 2 +- config/system.env | 4 ++ 7 files changed, 64 insertions(+), 34 deletions(-) diff --git a/AGENT_TODO.md b/AGENT_TODO.md index bdf16ac..fe8a883 100644 --- a/AGENT_TODO.md +++ b/AGENT_TODO.md @@ -121,3 +121,4 @@ summary (newest last). - **2026-08-14** β€” Gitea Actions gate is now **live and green end-to-end**: act_runner (v0.6.1, labels `ubuntu-latest`) registered on `100.100.1.2` (`~/srv/gitea/runner/`, standalone compose next to the ScaleTail gitea; `CONFIG_FILE=/config.yaml` env required or `run.sh` never reads the config; `--add-host gitea.skink-platy.ts.net:100.111.241.54` so the job container reaches gitea). First real runs **caught a deterministic gen-drift**: plain `sort` in `scripts/gen-docs.sh` is locale-dependent (category-less tool keys start with `|`, which collates after letters under the CI container's locale β†’ `pos-config`/`pos-tree` reordered), so the `git diff --exit-code` step failed. Fixed with `export LC_ALL=C` in gen-docs.sh (byte-order sort) + regenerated `DOC/AGENT_Context_Project.md` (config/tree now sort after the letter categories); `make check` OK, `make lint` 0 FAIL / 0 WARN. Live CI verdicts: the run for `e0b5b11` (workflow commit) and the empty trigger `98a767c` both **FAILED** on the drift; the run for `9d058b7` (the fix) **SUCCEEDED** (`🏁 Job succeeded`). - **2026-08-14** β€” Gitea Actions gate added: `.gitea/workflows/lint.yml` runs `make gen` + `git diff --exit-code` (gen-drift) + `make check` + `make lint` on every push/PR. Verified locally the exact four steps pass (gen idempotent, check OK, lint 0 FAIL / 0 WARN). "no CI" lines updated in AGENTS.md (Quick facts β†’ CI bullet, notes a registered act_runner is required) and DEV.md (stub harnesses note: CI runs static gates only, not behaviour suites). Gitea 1.26.4 confirmed reachable; runner registration completed the same day (see the entry above). - **2026-08-14** β€” Convention-drift maintenance fix session (completed the audit backlog `MAINTENANCE.md`, M-001..M-023, all VERIFIED; gate `scripts/lint-conventions.sh` + `make lint` now 0 FAIL / 0 WARN; `make gen && make check` green). P0 bugs: M-002/003/004 added `docker-compose docker-vbox network-hotspot` to `INTERACTIVE_CMDS` (stdin/log-pipe prompt swallow); M-005 `install.sh --steps` now expands documented `N-M` ranges via `normalize_steps_spec()` (dry-run verified); M-006 feature-vs-docs decision: `--send`/`--markdown` **not restored** (health is a console-only reporter by design since fe7708f; scheduler `NOTIFY=always` covers delivery) β€” 5 docs corrected instead; M-007 `lib/notify.sh:57` fallback routed to stderr (stdout-leak on standalone source). P1: M-008..M-014 deps guards moved **before** `-h|--help` in docker-health/docker-ps (converted to `command -v X || err`), network-scan, share-usb-server, media-mp3/mp4 (guards before help with a `--dry-run` pre-scan preserving the documented no-deps preview); system-health documented as the sanctioned graceful-degradation no-guard pattern in DEV.md β€” lint refined accordingly (`first_guard_line` only matches real guards; `first_line` skips comments; precision fixes, not weakenings); M-015 system-firewall gained `usage()`+`-h|--help` (root-gated first; verified via sudo); M-016 `ffmpeg` added to preinstall PACKAGES. P2: M-017/M-018 autostart + usb-automount gained the feature-template preamble (flags.sh load, usage); M-019 `chmod +x apps/media/scrcpy.sh`; M-020 `SCALE_DIR`/`CONFIG_ENV` `:-` seams in pos-docker-compose (verified via overrides; follow-on fix: `DIM` color var missing from common.sh crashed `pos docker compose config` β€” added it); M-021 `CONFIG_DIR` centralized as the canonical XDG-aware seam in common.sh, per-file duplicates dropped (standalone-sourced notify.sh/config-ui.sh/matrix+telegram tools keep an identical guarded copy β€” "no shared lib? inline fallbacks"); M-022 `plugin_*` prefix collision resolved by renaming the internal registry helpers to `ent_plugin_*` (the documented plugin-authoring API `plugin_have`/`plugin_require`/`plugin_load_config`/`plugin_http_json` kept for user plugins); M-023 six tools (pos-config, pos-tree, pos-entertainment-{config,enable,disable,status}) now filename-referenced in DOC/POS.md. Hand-maintained AGENT_Context line-count rows bumped (install.sh 223β†’248, preinstall 75β†’76, common.sh 144β†’151, notify.sh 76β†’87 stale-corrected, autostart 14β†’50, usb-automount 134β†’138); `make lint` target wired in the Makefile. `MAINTENANCE.md` kept as the working record (uncommitted by design). +- **2026-08-14** β€” `pos system backup` optional encryption (`--no-encrypt` flag + `BACKUP_ENCRYPT=0` env, flag-or-env β€” user chose "Flag + env only"): plain path keeps a verified `.tar.gz` with no password prompt (headless/cron safe); encrypt path unchanged (prompt β†’ gpg AES-256 β†’ decrypt-verify; the gpg dep-guard moved into the encrypt branch so plain backups no longer require `gnupg`). Arg parsing rewritten as a loop over `"$@"` so `pos system backup --no-encrypt` works with the flag after the folder; usage() documents all three forms + the plain artifact name; `# POS_FLAGS: --service --no-encrypt`; `config/system.env` template gains `#BACKUP_ENCRYPT=0`; POS.md row + howto/system.md section updated. Verified: stub suite +2 cases (T18 flag / T19 env: plain .tar.gz artifact, gpg never called via `$GPG_CALLED`, USB copy + sha256 of the plain archive, notify wording) β€” 65/65 green; `bash -n`, `make gen && make check`, `make lint` 0 FAIL / 0 WARN. diff --git a/DOC/AGENT_Context_Project.md b/DOC/AGENT_Context_Project.md index 362e118..767d8b5 100644 --- a/DOC/AGENT_Context_Project.md +++ b/DOC/AGENT_Context_Project.md @@ -607,7 +607,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:` | `bin/pos-share-smb-server` | 253 | Manage the Samba server (status, share/unshare exports, users, enable/disable) | | `bin/pos-share-usb-server` | 218 | USB Redirector server control (--ls, --share; prompts when args omitted) | | `bin/pos-ssh-load-keys` | 31 | Load all SSH keys into the agent | -| `bin/pos-system-backup` | 347 | Encrypted (AES-256) folder snapshots (tar + gpg) | +| `bin/pos-system-backup` | 364 | Encrypted (AES-256) folder snapshots (tar + gpg) | | `bin/pos-system-firewall` | 308 | Interactive UFW management | | `bin/pos-system-health` | 209 | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL | | `bin/pos-system-schedule` | 81 | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently | diff --git a/DOC/POS.md b/DOC/POS.md index 210c12f..4780b9a 100644 --- a/DOC/POS.md +++ b/DOC/POS.md @@ -212,7 +212,7 @@ The standalone `vbox` command still works and forwards to `pos docker vbox` (see | Command | File | Purpose | Configuration | |---------|------|---------|---------------| | `sudo pos system firewall` | `bin/pos-system-firewall` | Interactive UFW ("UFW POWER") menu: add/delete rules, status, enable/disable/reset, default policies | Must run as root. Every command is previewed and confirmed before execution; supports `--dry-run`; keeps a history of executed commands. Executed mutating changes are announced via `lib/notify.sh` | -| `pos system backup ` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES). Artifact `_.tar.gz.gpg` in the current directory, `chmod 600`. Once the archive verifies, connected USB storage is offered (detected via `lsblk` TRAN with `lsusb`/by-id cross-check; unmounted sticks get a mount offer first β€” see `DOC/howto/system.md`; or pinned with `BACKUP_USB_ROOT`): the copy lands in `/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh` | +| `pos system backup ` | `bin/pos-system-backup` | Create a gpg-encrypted (AES-256) `tar.gz` snapshot of a folder and verify it | Prompts twice for a password (never stored). Uses `sudo tar`; needs `gnupg` (in `preinstall.sh` PACKAGES) only when encrypting. Artifact `_.tar.gz[.gpg]` in the current directory, `chmod 600`; `--no-encrypt` (or `BACKUP_ENCRYPT=0`) keeps a plain `.tar.gz` with no password prompt (headless/cron safe). Once the archive verifies, connected USB storage is offered (detected via `lsblk` TRAN with `lsusb`/by-id cross-check; unmounted sticks get a mount offer first β€” see `DOC/howto/system.md`; or pinned with `BACKUP_USB_ROOT`): the copy lands in `/backups/` and is proven 100% by sha256 before it is announced. Success/failure are announced via `lib/notify.sh` | | `pos system backup --service` | `bin/pos-system-backup` | Lists folders under `/srv` and `~/srv`, lets you pick one, then runs the same backup | Roots via `BACKUP_SERVICE_ROOTS` (space-separated, default `/srv $HOME/srv`) or `~/.config/linux_post_install/system.env` | | `pos system health` | `bin/pos-system-health` | Host health dashboard: disk per mount, RAM/swap, failed systemd units, backup age, fail2ban, docker containers. Exits 1 if any check FAILs | Console-only reporter β€” health itself never sends notifications; forward the output with a wrapper (e.g. the Telegram/Matrix listener map `/status=pos system health`) or schedule it via `pos system schedule` with a `NOTIFY` policy. `HEALTH_BACKUP_MAX_AGE_DAYS` (default 2) and `BACKUP_SERVICE_ROOTS` come from `~/.config/linux_post_install/system.env`; `--help` shows the effective values | | `pos system schedule ` | `bin/pos-system-schedule` | Scheduled jobs β€” run a command on a timer, notify (or stay silent): `run [name\|all]`, `list`, `config`, `enable [name\|all]`, `disable [name\|all]`, `status`, `migrate`. Each job is a file in `~/.config/linux_post_install/schedule.d/.env` with `INTERVAL` (`5m…59m`, `1h…23h`, `hourly`, `daily`, `weekly`, `OnCalendar=…`), `NOTIFY` policy, optional `MSG`, `RULE` (threshold only), and `COMMAND` = the literal rest of the line (pipes/quotes/`sudo` fine). Policies: `always` (full output every run), `onchange` (send when output differs from the last run; first run always sends), `onerror` (non-zero exit or empty output), `threshold` (first numeric output vs `RULE`, alert on falseβ†’true + one recovery β€” the old event-trigger behavior), `never` (side-effect jobs, no notify) | One systemd **user** timer pair per job (`pos-schedule-.timer` + oneshot `.service`, `Persistent=true`), reconciled on `enable`/`disable`; the legacy single `pos-event-trigger` timer is auto-removed. `migrate` converts a pre-existing `event.env` rule set into `schedule.d/rule-N.env` threshold jobs. `config` is an interactive editor (add/edit/remove/enable/disable, validates interval + threshold); alerts via `lib/notify.sh`; `--dry-run` previews runs/writes/sends; jobs are arbitrary shell commands (chmod 600, same trust model as the Telegram map); starter jobs in `config/schedule.d/` auto-installed no-clobber by postinstall | diff --git a/DOC/howto/system.md b/DOC/howto/system.md index c0a0671..198bf5a 100644 --- a/DOC/howto/system.md +++ b/DOC/howto/system.md @@ -78,14 +78,22 @@ sudo rm -f /etc/systemd/system/pos-health.{service,timer} && sudo systemctl daem ## `pos system backup` β€” encrypted folder snapshots ```bash -pos system backup # encrypt to ./_.tar.gz.gpg -pos system backup --service # pick a folder from /srv + ~/srv +pos system backup # encrypt to ./_.tar.gz.gpg +pos system backup --no-encrypt # plain ./_.tar.gz, no password +pos system backup --service # pick a folder from /srv + ~/srv ``` Uses `sudo tar` + gpg AES-256. The password is prompted **twice and never stored**; the artifact is `chmod 600`. On success (and on failure, via ERR trap) a `notify_send` alert is sent. +**Skip encryption** with `--no-encrypt` (or `BACKUP_ENCRYPT=0` in +`system.env`): the archive stays a plain `.tar.gz`, no password is prompted, +and the file is still `chmod 600` + USB-copy verified. This is the +**headless/cron-safe** mode β€” the encrypted path prompts for a password, so +under cron it needs `--no-encrypt` with a fixed folder +(`pos system backup ~/Documents --no-encrypt`). + `--service` lists folders under the roots in `BACKUP_SERVICE_ROOTS` (default `/srv $HOME/srv`; override via `system.env` or env) and lets you pick. diff --git a/bin/pos-system-backup b/bin/pos-system-backup index 375bb4d..b77ad70 100755 --- a/bin/pos-system-backup +++ b/bin/pos-system-backup @@ -1,7 +1,7 @@ #!/usr/bin/env bash set -euo pipefail # POS: system backup β€” Encrypted (AES-256) folder snapshots (tar + gpg) -# POS_FLAGS: --service +# POS_FLAGS: --service --no-encrypt # POS_CONFIG: notify | notify.env | NOTIFY_PLATFORM=:Comma-separated notify platforms (default telegram) β€” shared by backup, firewall, share nfs client/server source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh" @@ -15,22 +15,28 @@ trap 'notify_send "Backup FAILED: ${FOLDER:-unknown}"' ERR usage() { cat < + pos system backup --no-encrypt pos system backup --service Create a gpg-encrypted (AES-256) tar.gz snapshot of a folder and verify it. -The archive password is prompted twice and never stored. +The archive password is prompted twice and never stored. With --no-encrypt +(or BACKUP_ENCRYPT=0) the backup is kept as a plain .tar.gz β€” no password, +headless/cron safe. Modes: Back up that folder directly. + --no-encrypt Skip encryption (no password prompt, artifact stays .tar.gz). --service List folders under /srv and ~/srv, pick one, back it up. -The final artifact _.tar.gz.gpg is written to the current directory. +The final artifact _.tar.gz[.gpg] is written to the current directory. After it verifies, connected USB storage is offered: the copy lands in /backups/ and is sha256-verified 100% before it is announced. A stick that is plugged in but not mounted is offered a mount first (sudo, mirrors the usb-automount scheme) before the copy. Environment: + BACKUP_ENCRYPT Set to 0 to skip encryption (same as --no-encrypt) + (default: 1) BACKUP_SERVICE_ROOTS Space-separated roots for --service (effective: ${EFF_ROOTS}) BACKUP_USB_ROOT USB root to copy finished backups to @@ -43,7 +49,6 @@ EOF } command -v tar &>/dev/null || err "tar not found" -command -v gpg &>/dev/null || err "gpg not found (install gnupg)" # ── USB copy (optional post-backup step) ───────────────────────── # Detection runs AFTER the backup finished, so a stick plugged in while @@ -253,12 +258,17 @@ mount_offer() { } SERVICE=0 -case "${1:-}" in - -h|--help) usage ;; - --service) SERVICE=1 ;; - "") err "Missing folder path (or use --service)" ;; - *) FOLDER="$1" ;; -esac +ENCRYPT=1 +[ "${BACKUP_ENCRYPT:-1}" = "0" ] && ENCRYPT=0 +for arg in "$@"; do + case "$arg" in + -h|--help) usage ;; + --service) SERVICE=1 ;; + --no-encrypt) ENCRYPT=0 ;; + *) FOLDER="$arg" ;; + esac +done +{ [ "$SERVICE" -eq 1 ] || [ -n "${FOLDER:-}" ]; } || err "Missing folder path (or use --service)" if [ "$SERVICE" -eq 1 ]; then if [ -n "${BACKUP_SERVICE_ROOTS:-}" ]; then @@ -313,29 +323,36 @@ log "Verifying archive..." tar -tzf "$ARCHIVE" > /dev/null log "Archive verified" -while true; do - read -s -rp "Enter backup password: " PASS - echo - read -s -rp "Confirm backup password: " CONFIRM - echo - if [ -n "$PASS" ] && [ "$PASS" = "$CONFIRM" ]; then - break - fi - warn "Passwords are empty or do not match β€” try again" -done -unset CONFIRM +if [ "$ENCRYPT" -eq 1 ]; then + command -v gpg &>/dev/null || err "gpg not found (install gnupg)" -log "Encrypting backup..." -gpg --batch --yes --passphrase "$PASS" --symmetric --cipher-algo AES256 "$ARCHIVE" + while true; do + read -s -rp "Enter backup password: " PASS + echo + read -s -rp "Confirm backup password: " CONFIRM + echo + if [ -n "$PASS" ] && [ "$PASS" = "$CONFIRM" ]; then + break + fi + warn "Passwords are empty or do not match β€” try again" + done + unset CONFIRM -rm -f "$ARCHIVE" -ARCHIVE="${ARCHIVE}.gpg" -chmod 600 "$ARCHIVE" + log "Encrypting backup..." + gpg --batch --yes --passphrase "$PASS" --symmetric --cipher-algo AES256 "$ARCHIVE" -log "Verifying encrypted backup..." -gpg --batch --quiet --passphrase "$PASS" --decrypt "$ARCHIVE" | tar -tzf - > /dev/null + rm -f "$ARCHIVE" + ARCHIVE="${ARCHIVE}.gpg" + chmod 600 "$ARCHIVE" -unset PASS + log "Verifying encrypted backup..." + gpg --batch --quiet --passphrase "$PASS" --decrypt "$ARCHIVE" | tar -tzf - > /dev/null + + unset PASS +else + chmod 600 "$ARCHIVE" + log "No encryption requested β€” keeping $ARCHIVE" +fi echo log "Backup completed: $ARCHIVE" notify_send "Backup completed: $ARCHIVE" diff --git a/completions/pos.bash b/completions/pos.bash index 4a7a00f..af7d79e 100644 --- a/completions/pos.bash +++ b/completions/pos.bash @@ -14,7 +14,7 @@ _pos_flags[network-checkport]="--tcp --udp --ping --no-banner --versions --timeo _pos_flags[network-download]="--dir --out --split --seed --force --upload --gid --tmux" _pos_flags[network-hotspot]="--foreground" _pos_flags[share-usb-server]="--ls --ls-shared --share --unshare --auto-share --callback --close-callback --auto-connect --disconnect --nickname --timeout --port --info --version" -_pos_flags[system-backup]="--service" +_pos_flags[system-backup]="--service --no-encrypt" _pos_flags[system-schedule]="--dry-run" _pos_flags[tree]="--depth" # GEN:END posflags diff --git a/config/system.env b/config/system.env index fdee820..7b5b434 100644 --- a/config/system.env +++ b/config/system.env @@ -8,6 +8,10 @@ # in `pos system health`. Default: /srv $HOME/srv #BACKUP_SERVICE_ROOTS=/srv /home/you/srv +# Skip encryption in `pos system backup` (plain .tar.gz, no password prompt β€” +# headless/cron safe). Equivalent to the --no-encrypt flag. Default: 1 +#BACKUP_ENCRYPT=0 + # Max backup age in days before `pos system health` raises a WARN. Default: 2 #HEALTH_BACKUP_MAX_AGE_DAYS=3