refactor: pos system event-trigger → pos system schedule — per-job timers, notify policies, legacy migrate

The single-timer threshold monitor generalizes into a scheduler: each job is a
chmod-600 file in schedule.d/<name>.env (INTERVAL 5m..59m/1h..23h/hourly/daily/
weekly/OnCalendar=..., NOTIFY policy, optional MSG, RULE for threshold,
COMMAND = literal rest of line) with its own systemd user timer pair
(pos-schedule-<name>.timer + oneshot .service, Persistent, reconciled on
enable/disable — orphan units + the legacy pos-event-trigger timer
auto-removed). Policies: always (full output every run), onchange (diff vs
last run, first run sends), onerror (non-zero exit or empty output),
threshold (old event-trigger behavior: first numeric vs RULE, alert on
false→true + recovery, per-job firing state), never (silent side-effect jobs).
run [name|all], list, config (interactive add/edit/remove/enable/disable with
validation), enable/disable [name|all], status, migrate (converts legacy
event.env rules → rule-N.env threshold jobs, verbatim LHS as COMMAND, adopts
the legacy timer's OnCalendar or 5m, removes the old timer). Per-run logs +
state in ~/.local/share/linux_post_install/schedule/{logs,state}/.
config/event.env + event-rules.template → config/schedule.d/ starter jobs
(nvme-health, cpu-temp, disk-root, silent log-cleanup); postinstall installs
them no-clobber into an empty schedule.d/ (legacy event.env users get a
migrate hint instead). bin/pos EXAMPLES + INTERACTIVE_CMDS
(system-schedule config) updated; install.sh ships the renamed tool+lib.
This commit is contained in:
Your Name
2026-08-13 02:14:25 -04:00
parent 8b3435a8d5
commit 414a990801
18 changed files with 1134 additions and 727 deletions
+9 -8
View File
@@ -22,7 +22,7 @@
| ## 11. Coding Conventions | 472504 |
| ## 12. Development Workflow | 505557 |
| ## 13. Key File Quick Reference | 558614 |
| ## 14. Common Tasks for Agents | 615643 |
| ## 14. Common Tasks for Agents | 615644 |
<!-- GEN:END docmap -->
## 1. Project Overview
@@ -87,9 +87,9 @@ Linux_post_install/
│ ├── pos-share-usb-server # USB Redirector server control (--ls, --share; prompts when args omitted)
│ ├── pos-ssh-load-keys # Load all SSH keys into the agent
│ ├── pos-system-backup # Encrypted (AES-256) folder snapshots (tar + gpg)
│ ├── pos-system-event-trigger # State-based rule monitors; alerts via notify when a check crosses a threshold
│ ├── pos-system-firewall # Interactive UFW management
│ ├── pos-system-health # Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL
│ ├── pos-system-schedule # Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently
│ ├── pos-tree # Show the pos CLI command tree: categories, commands, and subcommands
<!-- GEN:END tree -->
│ ├── flag-reader # Inspect feature flags (list/status/--raw)
@@ -288,9 +288,9 @@ All non-interactive `pos` commands log output to `~/.local/share/linux_post_inst
| share | usb-server | `pos-share-usb-server` | USB Redirector server control (--ls, --share; prompts when args omitted) |
| ssh | load-keys | `pos-ssh-load-keys` | Load all SSH keys into the agent |
| system | backup | `pos-system-backup` | Encrypted (AES-256) folder snapshots (tar + gpg) |
| system | event-trigger | `pos-system-event-trigger` | State-based rule monitors; alerts via notify when a check crosses a threshold |
| system | firewall | `pos-system-firewall` | Interactive UFW management |
| system | health | `pos-system-health` | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL |
| system | schedule | `pos-system-schedule` | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently |
| | tree | `pos-tree` | Show the pos CLI command tree: categories, commands, and subcommands |
<!-- GEN:END dispatch -->
@@ -566,13 +566,13 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `lib/flags.sh` | 60 | Feature flag store (set/clear/is_set/value/list/status) |
| `lib/notify.sh` | 76 | Multi-platform alerting (`notify_send`) — opt-in source, silent-fails |
| `lib/entertainment-lib.sh` | 350 | Entertainment module lib (ENABLED parsing, scheduler sync) |
| `lib/eventer-lib.sh` | 312 | Eventer lib (rule parsing, float compare, per-rule state, user-timer sync) |
| `lib/scheduler-lib.sh` | 822 | Scheduler lib (job parsing, notify policies, per-job user timers, legacy migrate) |
| `bin/flag-reader` | 58 | Inspect flags (list/status/`--raw`) |
| `bin/flag-set` | 21 | Set a flag (optionally with a value) |
| `bin/flag-clear` | 21 | Unset a flag |
| `features/autostart.sh` | 14 | Boot-time feature (moved from `bin/`, flag-gated service) |
<!-- GEN:START filetable -->
| `bin/pos` | 291 | CLI dispatcher with smart arg matching + logging + category help |
| `bin/pos` | 292 | CLI dispatcher with smart arg matching + logging + category help |
| `bin/pos-ai-gemini` | 311 | Chat with Google Gemini (ask, chat, models, sessions) |
| `bin/pos-communication-matrix-listener` | 565 | Matrix listener: map /command → bash, run them on room messages |
| `bin/pos-communication-matrix-sender` | 224 | Send messages to a Matrix room via the client-server API (send, test, login) |
@@ -591,20 +591,20 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| `bin/pos-media-mp3` | 80 | Download audio as MP3 (yt-dlp) |
| `bin/pos-media-mp4` | 126 | Download video as MP4 (smart/interactive format select) |
| `bin/pos-network-checkport` | 496 | Check TCP/UDP port reachability (nmap, or bash/nc fallback) + local interface view |
| `bin/pos-network-download` | 874 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) |
| `bin/pos-network-download` | 949 | aria2 RPC daemon + queue control (add/torrent/metalink, watch, limits) |
| `bin/pos-network-hotspot` | 93 | Wi-Fi hotspot via create_ap + wihotspot-gui |
| `bin/pos-network-ip` | 69 | Show interfaces, routes, public IP + location |
| `bin/pos-network-scan` | 271 | Parallel ping sweep of CIDR |
| `bin/pos-share-nfs-client` | 138 | Mount NFS shares (ephemeral or persistent systemd mount units) |
| `bin/pos-share-nfs-server` | 134 | Manage the NFS kernel server (status, share/unshare exports, enable/disable) |
| `bin/pos-share-smb-client` | 183 | Mount SMB/CIFS shares (ephemeral or persistent systemd mount units) |
| `bin/pos-share-smb-server` | 226 | Manage the Samba server (status, share/unshare exports, users, enable/disable) |
| `bin/pos-share-smb-server` | 253 | Manage the Samba server (status, share/unshare exports, users, enable/disable) |
| `bin/pos-share-usb-server` | 218 | USB Redirector server control (--ls, --share; prompts when args omitted) |
| `bin/pos-ssh-load-keys` | 31 | Load all SSH keys into the agent |
| `bin/pos-system-backup` | 126 | Encrypted (AES-256) folder snapshots (tar + gpg) |
| `bin/pos-system-event-trigger` | 219 | State-based rule monitors; alerts via notify when a check crosses a threshold |
| `bin/pos-system-firewall` | 291 | Interactive UFW management |
| `bin/pos-system-health` | 209 | Host health dashboard (disk, RAM, services, backup age, fail2ban, docker); exit 1 if any FAIL |
| `bin/pos-system-schedule` | 81 | Scheduled jobs: run a command on a timer; notify on threshold/change/error/always or silently |
| `bin/pos-tree` | 112 | Show the pos CLI command tree: categories, commands, and subcommands |
| `completions/pos.bash` | 291 | Dynamic bash completion |
<!-- GEN:END filetable -->
@@ -635,6 +635,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`
| Modify aria2 download daemon / queue logic | Edit `bin/pos-network-download` |
| Modify NFS share logic | Edit `bin/pos-share-nfs-server` / `bin/pos-share-nfs-client` |
| Modify SMB share logic | Edit `bin/pos-share-smb-server` / `bin/pos-share-smb-client` |
| Modify the scheduler / scheduled jobs | Edit `bin/pos-system-schedule` / `lib/scheduler-lib.sh` (jobs in `~/.config/linux_post_install/schedule.d/`) |
| Modify AI/Gemini logic | Edit `bin/pos-ai-gemini` (config scope `ai` via `pos config ai`; `AI_GEMINI_API_KEY`/`AI_GEMINI_MODEL` in `~/.config/linux_post_install/ai.env`) |
| Modify UFW/firewall logic | Edit `bin/pos-system-firewall` |
| Modify pos logging | Edit log setup in `bin/pos` |
+4 -1
View File
@@ -14,7 +14,7 @@ authoritative one-line reference (every command + flag), see
| `pos docker` | Compose services, container dashboards, disposable VMs | [docker](howto/docker.md) |
| `pos media` | Download audio/video via yt-dlp | [media](howto/media.md) |
| `pos system` | Backups, firewall, health dashboard | [system](howto/system.md) |
| `pos system event-trigger` | Threshold-rule monitors that alert on crossing | [event-trigger](howto/event-trigger.md) |
| `pos system schedule` | Scheduled jobs: run a command on a timer, notify on threshold/change/error or silently | [schedule](howto/schedule.md) |
| `pos ssh` | Load keys into the agent | [ssh](howto/ssh.md) |
| `pos share` | Share USB devices & filesystems over the network (USB, NFS, SMB) | [share](howto/share.md) |
| `pos communication` | Send Telegram/Matrix messages & alerts, /command listeners | [communication](howto/communication.md) |
@@ -42,6 +42,7 @@ templates (without overwriting an existing file):
| `compose.env` | `pos docker compose` | `TS_AUTHKEY`, `TZ`, `DNS_SERVER`, `SERVICES_BASE` |
| `entertainment.env` | `pos entertainment *` | plugin keys (`WEATHER_LAT`…), `ENABLED` |
| `ai.env` | `pos ai gemini` | `AI_GEMINI_API_KEY`, `AI_GEMINI_MODEL` |
| `schedule.d/` | `pos system schedule` | one `<name>.env` per job: `INTERVAL`, `NOTIFY`, `MSG`, `RULE`, `COMMAND` |
```bash
pos config telegram # set TELEGRAM_BOT_TOKEN / TELEGRAM_CHAT_ID
@@ -73,6 +74,8 @@ Adding another platform = create `bin/pos-communication-<p>` implementing
enabled by postinstall once `telegram.env` exists. See [system](howto/system.md).
- **Entertainment auto-triggers** — per-plugin `pos entertainment enable <plugin> <interval>`,
uses systemd user timers (or cron fallback). See [entertainment](howto/entertainment.md).
- **`pos system schedule` jobs** — run any command on a per-job timer and notify
on threshold/change/error/always or silently. See [schedule](howto/schedule.md).
### Gotcha: run from anywhere
-88
View File
@@ -1,88 +0,0 @@
# How-To: `pos system event-trigger`
State-based threshold monitors: each line of `event.env` is an independent rule;
when a rule's check crosses its threshold you get one alert (plus one recovery
message when it clears). Alerts go through `lib/notify.sh` — Telegram by
default, `NOTIFY_PLATFORM` for more.
```bash
pos system event-trigger config # interactive rule editor
pos system event-trigger list # rules + live check values
pos system event-trigger enable 5m # evaluate every 5 minutes via systemd
pos system event-trigger status # timer + rule count
pos system event-trigger disable # stop monitoring
pos system event-trigger run --dry-run # preview what would fire
```
---
## Rule format
One rule per line in `~/.config/linux_post_install/event.env` (chmod 600):
```
["<message>" if ] <check-command> <op> <threshold>
```
| Part | Meaning |
|------|---------|
| `"<message>" if` | Optional custom alert text (quote-stripped); without it the message is auto-composed |
| `<check-command>` | Any shell command; its **first numeric output** is the value (pipes/args fine) |
| `<op>` | `>` `<` `>=` `<=` `==` `!=` |
| `<threshold>` | Number with optional unit suffix — `60c`, `80%`, `10g` all work |
The operator is detected as the rightmost `op threshold` pair in the line, so
check commands containing their own `>`/`<` (redirection, awk) don't confuse it.
Examples:
```
# event.env
"CPU too hot" if sensors -u coretemp-isa-0000 | awk '/Package id 0:/{f=1} f && /temp1_input:/{print $2; exit}' > 60c
"Disk nearly full" if df -P / | awk 'NR==2{print $5+0}' > 80%
"Load high" if uptime | sed 's/.*load average: //; s/,.*//' >= 4
```
A ready-made starter set (CPU, memory, disk space/health, NVMe, network,
processes) lives in `config/event-rules.template` — copy it to
`~/.config/linux_post_install/event.env` or add the lines you want via
`pos system event-trigger config`.
Behavior:
- The check runs on every pass. Non-numeric/empty output, or an unparseable
line → the rule is skipped with a warning (other rules still run). The
`run` summary reports how many rules evaluated and how many were skipped.
- `sensors` rules need `lm-sensors` installed (`preinstall.sh` installs it;
verify with `sensors -u` before adding a rule — sensor names differ by
chip, check `sensors -u` output for the real `*_input` key). **Gotcha:**
a bare `grep -m1 temp1_input` can match a *different* chip's `temp1_input`
first (e.g. `acpitz`'s case temp) instead of the CPU — pin the chip with
`sensors -u <chip>` (find it in `sensors -u`, e.g. `coretemp-isa-0000`)
as in the example above.
- Alerts fire **once** when the condition turns true, and once more when it
recovers — a hot CPU for two hours is one message, not twenty.
- State is tracked per rule in `~/.local/share/linux_post_install/eventer/state/`
(keyed by a hash of the rule line — editing a rule resets its state).
## Scheduling
`enable [interval]` installs a systemd **user timer** (`pos-event-trigger.timer`
+ oneshot `.service` that runs `pos system event-trigger run`). Intervals:
`5m 10m 15m 30m 45m hourly 2h 6h 12h daily weekly`, or a raw `OnCalendar=…`.
Requires a reachable user systemd manager; run
`sudo loginctl enable-linger $USER` once so timers fire without login (the tool
tries this and warns if it can't).
## Alerting
`run` sends via `lib/notify.sh notify_send`, which delivers to every platform in
`NOTIFY_PLATFORM` (default `telegram`; comma-separated = fan out). Adding a
Matrix/Synapse sender later needs no changes here — see DOC/DEV.md → Alerting.
---
## Related
- Reference: [DOC/POS.md → system](../POS.md#system)
- Notifications: [communication](howto/communication.md) / `lib/notify.sh`
+151
View File
@@ -0,0 +1,151 @@
# How-To: `pos system schedule`
Scheduled jobs: run any shell command on a timer and have the result sent via
`lib/notify.sh` (Telegram by default, `NOTIFY_PLATFORM` for more) — or run it
silently. Unlike the old `event-trigger` threshold monitors, each job has its
own interval and its own notify policy.
```bash
pos system schedule list # jobs + notify policy + last run
pos system schedule enable # start all job timers (per-job intervals)
pos system schedule status # per-job timer state + next run
pos system schedule run nvme-health # run one job now
pos system schedule run all --dry-run # preview everything
pos system schedule disable cpu-temp # stop a job, keep the file
pos system schedule migrate # convert old event.env rules
pos system schedule config # interactive job editor
```
---
## Job format
One file per job in `~/.config/linux_post_install/schedule.d/<name>.env`
(chmod 600):
```
INTERVAL=hourly # 5m..59m | 1h..23h | hourly daily weekly | OnCalendar=…
NOTIFY=onchange # always | onchange | onerror | threshold | never
MSG="NVMe health" # optional label; threshold alert text when NOTIFY=threshold
RULE="> 60c" # threshold only: op + threshold (unit suffix fine: 60c, 80%)
COMMAND=… # literal rest of the line — pipes/quotes/sudo fine
ENABLED=false # written by `disable`; default enabled
```
The `COMMAND=` value is everything after the prefix — no quoting/escaping of
pipes, quotes, or `sudo`. The job runs it with `bash -c` and observes the
exit code. Only `INTERVAL` is required (default `5m`); `NOTIFY` defaults to
`threshold` when `RULE` is present, otherwise `onchange`.
### Notify policies
| Policy | Behavior |
|--------|----------|
| `threshold` | Runs the command, compares its **first numeric output** against `RULE` (float-safe). Alerts once on false→true plus one recovery message on true→false — a hot CPU for two hours is one message, not twenty. `MSG` is the alert text (auto-composed when absent) |
| `onchange` | Sends the output only when it differs from the last run. First run always sends. Best for "tell me when this changes" reports |
| `always` | Sends the full output on every run |
| `onerror` | Sends only when the command fails (non-zero exit or empty output), with the `rc` |
| `never` | Runs and logs, never notifies — side-effect jobs (cleanups, backups) |
Every run also writes a per-job log + last-run record
(`~/.local/share/linux_post_install/schedule/{logs,state}/`) so `list`/`status`
show the result even for `never` jobs.
## Example — NVMe health every hour
Your exact use case:
```
# ~/.config/linux_post_install/schedule.d/nvme-health.env
INTERVAL=hourly
NOTIFY=onchange
MSG=NVMe health
COMMAND=sudo -n smartctl -a /dev/nvme0n1 | grep -Ei 'critical_warning|temperature|available_spare|percentage_used|media_errors|error_information'
```
Two setup notes:
1. **Passwordless sudo for smartctl.** User timers have no tty, so `sudo` needs
a NOPASSWD rule (smartmontools is in `preinstall.sh` PACKAGES):
```bash
echo '%sudo ALL=(ALL) NOPASSWD: /usr/sbin/smartctl' | sudo tee /etc/sudoers.d/smartctl
```
(Use `/usr/sbin/smartctl` — check `command -v smartctl`.) The `-n` flag makes
`sudo` fail instead of hanging if the rule is missing, which `onerror` will
report.
2. **`temperature` drifts**, so an `onchange` job on this grep will fire most
hours. If you only want fault-field alerts, drop the `temperature`
alternative: `grep -Ei 'critical_warning|available_spare|percentage_used|media_errors|error_information'`.
## Example — threshold rule (old event-trigger style)
```
# cpu-temp.env
INTERVAL=5m
NOTIFY=threshold
MSG=CPU too hot
RULE="> 60c"
COMMAND=sensors -u coretemp-isa-0000 | awk '/Package id 0:/{f=1} f && /temp1_input:/{print $2; exit}'
```
Gotchas carried over from event-trigger:
- Non-numeric/empty output → the job is skipped with a warning (others still run).
- **Pin the chip in `sensors` rules** — a bare `grep -m1 temp1_input` can match a
different chip's `temp1_input` first (e.g. `acpitz`'s case temp). Find the
real name in `sensors -u` (e.g. `coretemp-isa-0000`) and pin it as above.
- Unit suffixes on the threshold are fine: `60c`, `80%`, `10g`.
## Example — silent side-effect job
```
# log-cleanup.env — never notifies
INTERVAL=weekly
NOTIFY=never
COMMAND=find ~/.local/share/linux_post_install/logs -type f -name '*_pos_*.log' -mtime +30 -delete
```
## Scheduling
Each enabled job gets its own systemd **user timer pair**
(`pos-schedule-<name>.timer` + oneshot `.service` running
`pos system schedule run <name>`, `Persistent=true` — missed runs fire on next
boot/login). `enable`/`disable` reconcile the timers with `schedule.d/` and
remove orphaned units; the legacy single `pos-event-trigger` timer is cleaned
up automatically. Requires a reachable user systemd manager; run
`sudo loginctl enable-linger $USER` once so timers fire without login (the tool
tries this and warns if it can't).
## Migrating from event-trigger
If you have rules in `~/.config/linux_post_install/event.env`, convert them in
place:
```bash
pos system schedule migrate
```
Each rule becomes `schedule.d/rule-N.env` with `NOTIFY=threshold`, the legacy
timer's interval (or `5m`), and the rule split into `MSG`/`RULE`/`COMMAND`.
The old timer is disabled and removed; run `pos system schedule enable` to
start the migrated jobs.
`migrate` copies the rule's left side **verbatim** as `COMMAND` — the old tool
ran it literally and never had `disk root`/`loadavg`-style shorthands. A rule
like `disk root > 80%` migrates, but its job will log "produced no number" on
every run. After migrating, rewrite such jobs with a real command, e.g.
`COMMAND=df -P / | awk 'NR==2{print $5+0}'` (see the `config/schedule.d/`
starter jobs for patterns).
## Alerting
`run` sends via `lib/notify.sh notify_send`, which delivers to every platform
in `NOTIFY_PLATFORM` (default `telegram`; comma-separated = fan out). Adding a
Matrix/Synapse sender needs no changes here — see DOC/DEV.md → Alerting.
---
## Related
- Reference: [DOC/POS.md → system](../POS.md#system)
- Notifications: [communication](howto/communication.md) / `lib/notify.sh`