Linux_post_install/DOC/howto/system.md

How-To: pos system

Host care: encrypted backups, firewall, health dashboard, and uninstall. Tools: backup, firewall, health, uninstall.

Tool	What it does
pos system health	Host health dashboard (disk, RAM, services, backup age, fail2ban, docker)
pos system backup	gpg-encrypted (AES-256) folder snapshots
pos system firewall	Interactive UFW ("UFW POWER") management
pos system uninstall	Safe, interactive uninstaller for the pos toolkit

---

pos system health — host health dashboard

pos system health                 # console report; exits 1 if any check FAILs

Checks: disk per mount (>90% = FAIL), RAM/swap, failed systemd units, backup age, fail2ban, docker containers. Header shows hostname, uptime, load, public IP.

Health is a console-only reporter — it never sends notifications; deliver its output with a wrapper or a scheduled job (below). --help prints the effective config values (env > system.env > default), e.g.:

Environment (effective values):
  HEALTH_BACKUP_MAX_AGE_DAYS  2
  BACKUP_SERVICE_ROOTS        /srv $HOME/srv

($HOME is resolved at runtime — on this host that is /srv /home/unknown/srv.)

Configuration

# ~/.config/linux_post_install/system.env   (comment-only defaults — uncomment to override)
BACKUP_SERVICE_ROOTS=/srv $HOME/srv    # roots for backup-age check + backup --service
BACKUP_USB_ROOT=/mnt/usb               # optional: copy finished backups to <root>/backups/ (auto-detects a mounted USB when unset)
HEALTH_BACKUP_MAX_AGE_DAYS=3           # WARN if newest backup older (default 2)

Daily digest (automated)

Run the health report on a timer with a scheduled job (no systemd unit needed):

pos system schedule config        # add a job: INTERVAL=daily, NOTIFY=always,
                                  #   COMMAND=pos system health
systemctl --user list-timers | grep pos-schedule
pos system schedule run <name>    # run once now

The NOTIFY=always policy sends the job's full output — i.e. the dashboard — as the alert. The old pos-health.{service,timer} systemd units are gone — a legacy install may still have them failed/leftover; disable and remove them:

sudo systemctl disable --now pos-health.timer pos-health.service 2>/dev/null
sudo rm -f /etc/systemd/system/pos-health.{service,timer} && sudo systemctl daemon-reload

Recipes:

• Watch the backup age without email: add the daily digest job; if the backup check turns WARN you'll see it in the morning report.
• Exit code in a cron/scheduled check: pos system health >/dev/null 2>&1 || notify_send "health FAIL".

Troubleshooting:

• [WARN] fail2ban installed but not running → expected unless you have it active; start it (sudo systemctl enable --now fail2ban) or ignore.
• [FAIL] services: nbd-server.service … → a failed unit; inspect with systemctl status <unit>.

---

pos system backup — encrypted folder snapshots

pos system backup <folder-path>                 # encrypt to ./<name>_<date>.tar.gz.gpg
pos system backup <folder-path> --no-encrypt    # plain ./<name>_<date>.tar.gz, no password
pos system backup --service                     # pick a folder from /srv + ~/srv

Uses sudo tar + gpg AES-256. The password is prompted twice and never stored; the artifact is chmod 600. On success (and on failure, via ERR trap) a notify_send alert is sent.

Skip encryption with --no-encrypt (or BACKUP_ENCRYPT=0 in system.env): the archive stays a plain .tar.gz, no password is prompted, and the file is still chmod 600 + USB-copy verified. This is the headless/cron-safe mode — the encrypted path prompts for a password, so under cron it needs --no-encrypt with a fixed folder (pos system backup ~/Documents --no-encrypt).

--service lists folders under the roots in BACKUP_SERVICE_ROOTS (default /srv $HOME/srv; override via system.env or env) and lets you pick.

Copy to a USB stick

After the archive verifies, connected USB storage is detected (so a stick plugged in while the backup was running is found — if none is mounted you get one chance to plug one in and re-check) and you're asked whether to copy the backup there. The copy lands in <usb>/backups/ and is verified 100% (sha256 source vs copy) before any success is announced:

pos system backup ~/Documents
# ... after the archive verifies:
#   [!] No USB storage detected
#   Plug a USB drive in now and press Enter to re-check (or 's' to skip):
#   [+] Copying to /media/you/USB-DISK/backups/docs_2026-08-13.tar.gz.gpg ...
#   OK Transfer verified 100% (sha256 match): .../backups/docs_2026-08-13.tar.gz.gpg

Detection reads lsblk and treats a device as USB when TRAN == usb (the per-device deciding signal). Removable-but-not-USB slots (e.g. a SATA card reader) are skipped. If a device shows no TRAN at all, lsblk's answer is cross-checked against /dev/disk/by-id/usb-* symlinks and lsusb before it is offered.

Unmounted stick? If the USB stick is plugged in but only shows as sdax with no mountpoint (common on CLI boxes with no automounter), you're offered a mount first, then it copies there:

#   [!] Found USB storage not mounted: /dev/sda1 (7.5G, DataTraveler)
#   Mount it at /media/usb-sda1 (world-writable) so the backup can go there? [y/N]
#   (y)  OK Mounted /dev/sda1 at /media/usb-sda1
#   [+] Copying to /media/usb-sda1/backups/docs_2026-08-13.tar.gz.gpg ...
#   OK Transfer verified 100% (sha256 match): .../backups/docs_2026-08-13.tar.gz.gpg

The mount mirrors usb-automount (/media/<label>, fallback /media/usb-<devname>, -o umask=000 world-writable so the copy works without root). Decline it and you get the exact sudo mkdir -p / sudo mount commands to run yourself, then Enter re-checks. s or EOF (cron) skips silently and the backup stays local — it never blocks.

• Multiple sticks mounted → pick by number; 0 skips; n/EOF skips silently (cron runs never block).
• Pin a fixed stick (no detection, no prompt on cron) with BACKUP_USB_ROOT=/mnt/usb in system.env — the copy still lands in <root>/backups/ and is still sha256-verified.
• Mount base and by-id dir are configurable: BACKUP_MOUNT_BASE (default /media) and BACKUP_USB_BYID (default /dev/disk/by-id).

Recipes

• Nightly service backup + health check:

  cd ~/backups && pos system backup --service
  pos system health          # "backup: 0d old" turns OK
  

• Cron it and get alerted:

  30 3 * * * cd ~/backups && pos system backup --service >> ~/backups/backup.log 2>&1
  

  Success/failure are sent to Telegram automatically.

Troubleshooting:

• "Root not found: …" → the default roots don't exist; set BACKUP_SERVICE_ROOTS in system.env.
• Forgot the password → backups are unrecoverable; keep the passphrase in a password manager. Nothing is stored anywhere else.
• sudo tar prompt: ensure the user has sudo rights for the source dir.
• "USB copy FAILED verification — checksum mismatch" → the copy is corrupt (bad stick or transfer); the local archive is untouched — copy it again manually and replace the file on the stick.

---

pos system firewall — interactive UFW ("UFW POWER")

Must run as root:

sudo pos system firewall

Interactive menu: add rule (port/service/IP/directional), delete by number or text, status (simple/verbose/numbered), enable/disable/reset, default policies, and an executed-command history. Supports --dry-run:

sudo pos system firewall --dry-run

Every command is previewed and confirmed before execution. Executed mutating changes are announced via notify_send (read-only ufw status is not).

Recipes

• Open SSH + a service: add rule → port/service → 22/tcp, then 8080/tcp; finish with enable.
• See current rules for deletion: menu 3) Show status → numbered.
• --dry-run to rehearse a rule batch safely.

Troubleshooting:

• "Please run as root" → you need sudo pos system firewall (the notify config still uses your user's $HOME, so alerts keep working).
• Accidentally locked yourself out of SSH → console into the host, sudo ufw allow 22/tcp, then sudo ufw reload.
• ufw reset requires typing RESET — deliberate.

pos system uninstall — remove the pos toolkit

pos system uninstall                       # interactive scan + confirm tier 1
pos system uninstall --yes                 # non-interactive, tier 1 only
pos system uninstall --yes --config --data # remove everything (nuclear option)

Scans the system for installed pos components and removes them in three tiers:

Tier	What it removes	How to include
Tier 1	Binaries (/usr/local/bin/pos*, libs, entertainment plugins, prebuilt, features), systemd services (disable+remove), shell integration (~/.bashrc PATH/completion/pos-ai-hook entries), completion file	Always (default)
Tier 2	Config files (~/.config/linux_post_install/ — .env files, schedule.d/, authorized_keys, rclone.conf)	--config flag
Tier 3	Session/log data (~/.local/share/linux_post_install/ — AI sessions, logs, captured output)	--data flag

The default mode is interactive: it shows what will be removed and asks for confirmation. The git repo is never removed — delete it manually if desired.

Recipes:

• Quick cleanup: pos system uninstall --yes
• Full wipe: pos system uninstall --yes --config --data
• Safe preview: run pos system uninstall without --yes to see the plan first

Troubleshooting:

• "Nothing to remove" → pos toolkit is not installed (or already removed)
• After uninstall, run source ~/.bashrc or restart your shell

---

Related

• Reference: DOC/POS.md → system
• Notify platform config: communication.md
• Backup roots shared with health: system.env (DOC/POS.md)