# AGENT Context — Linux_post_install Project

> **Purpose:** Single-source context document so any AI agent can understand the project, navigate the codebase, and make correct contributions.

---

## 1. Project Overview

**Linux_post_install** is a personal bootstrap and homelab toolkit for Debian/Ubuntu. One command turns a bare install into a fully productive machine:

- Automated system package installation (25+ packages)
- A unified CLI (`pos`) for network, Docker, media, system, and SSH tasks
- Optional desktop application installers (15 apps)
- Docker Compose service management via ScaleTail templates (119+ self-hosted services with Tailscale sidecar)
- Systemd service management for boot-time automation

**Repository:** `https://gitea.skink-platy.ts.net/admin/Linux_post_install`
**Target OS:** Debian / Ubuntu (uses `apt`)
**Shell:** Bash (`#!/usr/bin/env bash`)

---

## 2. Directory Structure

```
Linux_post_install/
├── install.sh              # Main orchestrator — entry point
├── preinstall.sh           # Phase 1: system packages via apt + yt-dlp
├── postinstall.sh          # Phase 3: PATH, bash completion, systemd services
│
├── lib/
│   ├── common.sh           # Shared library (colors, logging, spinner, timer, run)
│   └── flags.sh            # Feature flag store (flag_set/clear/is_set/value/list/status)
│
├── bin/                    # CLI tools — installed to /usr/local/bin/
│   ├── pos                 # Main dispatcher — smart arg matching to pos-* scripts
│   ├── pos-network-ip      # Show interfaces, routes, public IP
│   ├── pos-network-checkport  # TCP port checker
│   ├── pos-network-scan    # Parallel ping sweep of CIDR subnet
│   ├── pos-docker-ps       # Enhanced docker ps (health, IPs, ports, uptime)
│   ├── pos-docker-health   # Quick one-glance health dashboard
│   ├── pos-docker-compose  # Docker Compose service manager (largest script, 317 lines)
│   ├── pos-media-mp3       # Audio downloader (yt-dlp → MP3)
│   ├── pos-media-mp4       # Video downloader (yt-dlp → MP4, interactive format select)
│   ├── pos-system-firewall # Interactive UFW manager (menu-driven, 284 lines)
│   ├── pos-ssh-load-keys   # Load SSH keys into ssh-agent
│   ├── pos-vbox            # Disposable Docker-based "VMs"
│   ├── pos-network-hotspot # Wi-Fi hotspot (create_ap + wihotspot-gui)
│   ├── flag-reader         # Inspect feature flags (list/status/--raw)
│   ├── flag-set            # Set a feature flag (optionally with a value)
│   ├── flag-clear          # Unset a feature flag
│   ├── wr-*                # Legacy wrappers → pos (backward compat)
│   ├── mp3, mp4, vbox      # Legacy convenience wrappers → pos
│   └── ssh-load-all        # Legacy wrapper → pos ssh load-keys
│
├── features/               # User-customizable scripts (installed via --feature)
│   └── autostart.sh        # Boot-time script (via systemd, flag-gated)
│
├── x64_bin/                # Precompiled binaries, copied to /usr/local/bin on x86_64
│   ├── create_ap           # Wi-Fi AP CLI (bash script)
│   ├── wihotspot           # Wrapper → wihotspot-gui
│   └── wihotspot-gui       # GTK3 hotspot GUI (x86-64 ELF)
│                           # future: arm64_bin/ picked up automatically on aarch64
│
├── apps/                   # Optional desktop app installers (by category)
│   ├── install.sh          # Interactive picker / orchestrator
│   ├── browsers/
│   │   └── brave.sh       # Brave Browser (APT repo)
│   ├── development/
│   │   ├── opencode.sh    # opencode AI agent (official script)
│   │   └── vscode.sh      # VS Code (Microsoft APT repo)
│   ├── media/
│   │   ├── obs.sh         # OBS Studio (apt)
│   │   ├── scrcpy.sh      # scrcpy Android mirror (GitHub release)
│   │   └── vlc.sh         # VLC media player (apt)
│   ├── networking/
│   │   ├── netbird.sh     # NetBird VPN (official script)
│   │   ├── tailscale.sh   # Tailscale VPN (official script)
│   │   └── zerotier.sh    # ZeroTier VPN (official script)
│   ├── remote-access/
│   │   ├── termius.sh     # Termius SSH client (.deb)
│   │   └── vnc-viewer.sh  # TigerVNC Viewer (apt)
│   ├── system/
│   │   ├── docker.sh      # Docker Engine (get.docker.com)
│   │   └── qemu.sh        # QEMU + libvirt + KVM (apt)
│   └── utilities/
│       ├── affine.sh      # AFFiNE knowledge base (AppImage)
│       ├── btop.sh        # btop resource monitor (apt)
│       └── localsend.sh   # LocalSend (flatpak)
│
├── completions/
│   └── pos.bash            # Bash tab-completion for the pos CLI
│
├── config/
│   └── authorized_keys     # SSH public keys (gitignored)
│
├── compose/
│   └── scale-tail/         # Git submodule → ScaleTail templates (119+ services)
│
├── systemd/
│   ├── autostart.service   # Runs autostart.sh on boot
│   └── ssh-agent.service   # System-wide SSH agent socket
│
├── README.md               # User-facing intro + quick start (links into DOC/)
│
├── DOC/                    # All documentation
│   ├── README.md           # Docs index
│   ├── SCRIPTS.md          # Installer scripts, libs, features — reference
│   ├── POS.md              # pos CLI reference
│   ├── APPS.md             # Optional apps reference
│   ├── SYSTEMD.md          # Systemd units + completion
│   ├── DEV.md              # Developer guide
│   ├── AGENT_Context_Project.md  # This file — AI agent context
│   └── algorithm.md        # Algorithm diagrams
│
├── .gitignore              # Excludes secrets, Python artifacts, OS files
└── .gitmodules             # Submodule: compose/scale-tail → ScaleTail
```

---

## 3. Installation Flow

```
User runs: ./install.sh [--apps|--full|--feature|--dry-run|--skip <phase>|--steps <spec>]
│
├─ Phase 1: preinstall.sh        (requires root)
│   └─ apt update + installs 25+ packages + yt-dlp + fail2ban
│
├─ Phase 2: install.sh           (requires root)
│   └─ Copies bin/* → /usr/local/bin/ (chmod 755)
│   └─ Copies lib/common.sh + lib/flags.sh → /usr/local/bin/ (chmod 644)
│   └─ Copies x64_bin/* → /usr/local/bin/ on x86_64 (arm64_bin/ on aarch64)
│   └─ [if --feature] Copies features/* → /usr/local/bin/ (asks before overwriting),
│                      then sets the matching feature flag
│
├─ Phase 3: postinstall.sh       (runs as user)
│   └─ Configures fail2ban (SSH jail: 5 retries, 1h ban)
│   └─ PATH export in ~/.bashrc
│   └─ Bash completion for pos CLI
│   └─ Copies systemd/*.service → /etc/systemd/system/, enables them
│      (autostart.service only when the `autostart` flag is set)
│
├─ Phase 4: ScaleTail clone
│   └─ Shallow-clones ScaleTail templates to /usr/local/share/linux_post_install/scale-tail
│
└─ [if --apps or --full]: apps/install.sh
    └─ Interactive picker (or --all for non-interactive)
```

**After install, the repo can be deleted** — all tools live in `/usr/local/bin/` and templates in `/usr/local/share/linux_post_install/`.

### install.sh Flags

| Flag | Purpose |
|------|---------|
| `--apps` | Run interactive app picker after core install |
| `--full` | Core install + all apps (non-interactive) |
| `--feature` | Install `features/` scripts to `/usr/local/bin/` (asks before overwriting), set their flags |
| `--dry-run` | Preview without executing |
| `--skip <phase>` | Skip a phase (repeatable): `preinstall`, `scripts`, `postinstall`, `scalepoint`, `apps` |
| `--steps <spec>` | Run only specific phases. Format: `1,3,4` or `1-3` |
| `--no-color` | Disable colored output |

### pos Output Logging

All non-interactive `pos` commands log output to `~/.local/share/linux_post_install/logs/`:
- Per-command files: `YYYYMMDD_HHMMSS_pos_<cmd>.log` (full stdout+stderr)
- Main log: `pos.log` (command + timestamp + exit code for every invocation)
- Interactive commands (`system-firewall`, `media-mp4`) only log invocation, not output

---

## 4. The `pos` CLI System

### How It Works

`bin/pos` is the main dispatcher. It:
1. Scans its own directory for all executable `pos-*` files
2. Extracts category-subcommand names from filenames
3. Uses variable-length argument matching to find the right script

**Example:** `pos docker compose up jellyfin`
- Tries `pos-docker-compose-up-jellyfin` (not found)
- Tries `pos-docker-compose-up` (not found)
- Finds `pos-docker-compose` (runs with args `up jellyfin`)

### Available Commands

| Category | Command | Script | Description |
|----------|---------|--------|-------------|
| network | ip | `pos-network-ip` | Show interfaces, routes, public IP |
| network | checkport | `pos-network-checkport` | Check TCP port connectivity |
| network | scan | `pos-network-scan` | Parallel ping sweep of CIDR |
| docker | ps | `pos-docker-ps` | Enhanced container overview |
| docker | health | `pos-docker-health` | Quick health dashboard (exits 1 if unhealthy) |
| docker | compose | `pos-docker-compose` | Service manager (ls/up/down/restart/logs/update/config) |
| media | mp3 | `pos-media-mp3` | Download audio as MP3 |
| media | mp4 | `pos-media-mp4` | Download video with format select |
| system | firewall | `pos-system-firewall` | Interactive UFW management |
| ssh | load-keys | `pos-ssh-load-keys` | Load SSH keys into agent |
| vbox | create | `pos-vbox create` | Create disposable VM (asks "Enter now?") |
| vbox | enter | `pos-vbox enter` | Start and exec into container |
| vbox | ls | `pos-vbox ls` | List vbox-managed containers only (label-filtered) |
| vbox | start/stop/rm | `pos-vbox start/stop/rm` | Lifecycle management |

### Legacy Wrappers

These forward to `pos` transparently: `wr-ip`, `wr-checkport`, `wr-scan-ping`, `wr-docker`, `wr-compose`, `wr-ufw`, `mp3`, `mp4`, `vbox`, `ssh-load-all`.

### pos vbox Details

`pos-vbox` manages disposable Docker containers as lightweight VMs:

- **Container labeling:** All created containers get `linux_post_install.vbox=true` label
- **`ls` filtering:** `docker ps --filter label=linux_post_install.vbox=true` — only shows vbox-managed containers
- **Post-create prompt:** After `create`, asks "Enter now? [Y/n]" using `confirm` helper
- **Working dir detection:** `enter` auto-detects bind mount path from container labels
- **Custom dirs:** `--dir <path>` or `--dir .` for current directory

---

## 5. Shared Library — `lib/common.sh`

Sourced by most scripts. Provides:

| Function | Purpose |
|----------|---------|
| `log "msg"` | Green `[+]` status message |
| `warn "msg"` | Yellow `[!]` warning |
| `err "msg"` | Red `ERROR:` + exit 1 |
| `ok "msg"` | Green `OK` prefix |
| `section "title"` | Cyan-bordered section header |
| `step N T "msg"` | Numbered step header (e.g., `[1/4] Installing`) |
| `run cmd` | Executes command, respects `$DRY_RUN` |
| `spawn "msg" cmd` | Runs with animated braille spinner, elapsed time, OK/FAIL status |
| `timer_start` / `timer_stop` | Elapsed time tracking |
| `confirm "prompt" [default]` | y/N or Y/n prompt |

**Auto-detects TTY** — disables colors when piped.

**Source pattern:**
```bash
source "$(dirname "$0")/../lib/common.sh"
```

**Scripts that do NOT source common.sh** (self-contained): `bin/pos`, `pos-network-ip`, `pos-network-checkport`, `pos-network-scan`, `pos-media-mp3`, `pos-media-mp4`, `pos-ssh-load-keys`, `pos-system-firewall`.

---

## 6. Docker Compose / ScaleTail

### Architecture

ScaleTail provides 119+ Docker Compose templates with a Tailscale sidecar pattern (`network_mode: service:tailscale`). Each service gets a `tail-xxxxx.ts.net` URL with optional automatic HTTPS.

```
/usr/local/share/linux_post_install/scale-tail/    # Templates (git repo)
└── services/<name>/
    ├── compose.yaml
    └── .env

~/.config/linux_post_install/compose.env           # Global defaults (TS_AUTHKEY, TZ, DNS_SERVER, SERVICES_BASE)

/srv/<service>/                         # Active deployments (default base)
    ├── compose.yaml                    # From template (refreshed on update)
    ├── .env                            # User config (preserved across updates)
    ├── config/
    └── data/
```

### Key Commands

| Command | Description |
|---------|-------------|
| `pos docker compose ls` | List all available ScaleTail services |
| `pos docker compose up <svc>` | Deploy service to SERVICES_BASE |
| `pos docker compose down <svc>` | Stop a deployed service |
| `pos docker compose restart <svc>` | Restart a service |
| `pos docker compose logs <svc> [-f]` | View/follow logs |
| `pos docker compose update` | Pull latest templates, refresh compose.yaml (preserves .env) |
| `pos docker compose config set K=V` | Set global config value |
| `pos docker compose config show` | Display current config |

### Global Config Keys

- `TS_AUTHKEY` — Tailscale auth key (required)
- `TZ` — Timezone
- `DNS_SERVER` — Custom DNS
- `SERVICES_BASE` — Deployment root (default: `/srv`)

---

## 7. Optional Apps (`apps/`)

### How They Work

- `apps/install.sh` auto-discovers all `apps/<category>/*.sh` files (excluding itself)
- Three modes: interactive (default), `--all`, or specific app names as arguments
- `--uninstall` switches to uninstall mode (same selection, invokes app scripts with `uninstall` argument)
- Interactive TUI groups apps by category with section headers
- Each app script is standalone, idempotent, sources `lib/common.sh`
- Every app script defines `install_<name>()` **and** `uninstall_<name>()`, dispatched via `case "${1:-}" in uninstall) ...`

### Installation Methods

| Method | Apps |
|--------|------|
| `apt install` | btop, obs, vlc, vnc-viewer, qemu |
| APT repo (GPG + repo) | brave, vscode |
| Official `curl \| sh` | docker, tailscale, netbird, zerotier, opencode |
| AppImage | affine |
| GitHub release binary | scrcpy |
| Flatpak | localsend |
| .deb package | termius |

### Adding a New App

1. Create `apps/<name>.sh` following the template in DOC/DEV.md
2. It auto-appears in the interactive picker — no registration needed

---

## 8. Systemd Services

| Service | File | Purpose |
|---------|------|---------|
| `ssh-agent.service` | `systemd/ssh-agent.service` | System-wide SSH agent, socket at `/run/ssh-agent/socket` |
| `autostart.service` | `systemd/autostart.service` | Runs `autostart.sh` on boot |

All `.service` files in `systemd/` are automatically copied to `/etc/systemd/system/` and enabled by `postinstall.sh`.

---

## 9. Configuration Files

### Gitignored Secrets

- `config/rclone.conf` — rclone remote config (OAuth tokens)
- `config/authorized_keys` — SSH public keys

### Runtime Config

- `~/.config/linux_post_install/compose.env` — Docker Compose global defaults
- `~/.bashrc` — Modified by postinstall (PATH, bash completion)

### Feature Flags

System-wide flag store at `/usr/local/share/linux_post_install/flags/`:
- One file per flag; **presence = set**, **file content = optional value** (dir 755, files 644).
- Library: `lib/flags.sh` (installed as `/usr/local/bin/flags.sh`) — `flag_set <name> [value]`, `flag_clear <name>`, `flag_is_set <name>`, `flag_value <name>`, `flag_list`, `flag_status <name>`.
- CLI: `flag-reader` (list / status / `--raw`), `flag-set`, `flag-clear`.
- Set by `./install.sh --feature`; read by `postinstall.sh` to gate systemd enablement (e.g. `autostart.service` requires the `autostart` flag).
- Writes use `run` + `sudo`, so they respect `--dry-run`. `FLAGS_DIR` is env-overridable for tests.

---

## 10. Coding Conventions

### Script Standards

- **Shebang:** `#!/usr/bin/env bash`
- **Strict mode:** `set -euo pipefail`
- **Help:** Every script accepts `-h`/`--help` via `case` pattern
- **Idempotency:** Check existence before creating/modifying
- **Exit codes:** 0 = success, 1 = error

### Naming Conventions

- `pos-<category>-<command>` — canonical tool names
- `wr-*` — legacy wrappers
- `apps/<category>/<name>.sh` — optional app installers
- Hyphens for word separation, lowercase always

### Error Handling

- `command -v <tool> &>/dev/null` to check tool availability
- `set -euo pipefail` for fail-fast
- `err()` for fatal errors, `warn()` for non-fatal
- Confirmation prompts for destructive actions

### Security

- Never hardcode secrets in scripts
- Use `chmod 600` for sensitive files
- Validate user input before shell commands
- Use `sudo` only where necessary

---

## 11. Development Workflow

### Adding a New App

1. Create `apps/<category>/<name>.sh` following the template in DOC/DEV.md
2. It auto-appears in the interactive picker — no registration needed

### Adding a New Tool

1. Create `bin/pos-<category>-<command>` following conventions
2. Add system deps to `PACKAGES` array in `preinstall.sh` (if needed)
3. Add config logic to `postinstall.sh` (if needed, with `.gitignore` for secrets)
4. Update `DOC/POS.md` (and root `README.md` only if the category list changes)
5. Test: `bash -n bin/your-tool && shellcheck bin/your-tool`

### Testing

```bash
# Syntax check all scripts
for f in bin/* apps/*/*.sh lib/common.sh install.sh preinstall.sh postinstall.sh; do
    bash -n "$f" || echo "FAIL: $f"
done

# ShellCheck linting
shellcheck bin/my-script

# Test in Docker
docker run --rm -it -v $PWD:/repo ubuntu:22.04 bash
# inside: cd /repo && ./install.sh

# Test apps interactively
./apps/install.sh --all
./apps/install.sh docker vscode
```

### Commit Conventions

Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`

---

## 12. Key File Quick Reference

| File | Lines | Purpose |
|------|-------|---------|
| `install.sh` | 192 | Main orchestrator — 4 phases with CLI flags, `--feature`, prebuilt arch bins |
| `preinstall.sh` | 54 | System packages + hotspot deps + yt-dlp + fail2ban |
| `postinstall.sh` | 97 | fail2ban config, PATH, bash completion, systemd (flag-gated) |
| `lib/common.sh` | 121 | Shared library |
| `lib/flags.sh` | 60 | Feature flag store (set/clear/is_set/value/list/status) |
| `bin/flag-reader` | 58 | Inspect flags (list/status/`--raw`) |
| `bin/flag-set` | 21 | Set a flag (optionally with a value) |
| `bin/flag-clear` | 21 | Unset a flag |
| `features/autostart.sh` | 14 | Boot-time feature (moved from `bin/`, flag-gated service) |
| `bin/pos` | 146 | CLI dispatcher with smart arg matching + logging |
| `bin/pos-docker-compose` | 363 | Largest script — full compose management |
| `bin/pos-system-firewall` | 284 | Interactive UFW manager |
| `bin/pos-docker-ps` | 127 | Enhanced container overview |
| `bin/pos-docker-health` | 109 | Quick health dashboard |
| `bin/pos-vbox` | 156 | Docker-based disposable VMs (label-filtered, auto-enter prompt) |
| `bin/pos-network-hotspot` | 91 | Wi-Fi hotspot: `create_ap` (start with background prompt/`--foreground`, stop, status) + `wihotspot-gui` |
| `completions/pos.bash` | 118 | Dynamic bash completion |
| `apps/install.sh` | 171 | App install/uninstall picker/orchestrator |

---

## 13. Common Tasks for Agents

| Task | Where to Edit |
|------|---------------|
| Add a new CLI tool | Create `bin/pos-<cat>-<cmd>`, add deps in `preinstall.sh` |
| Add a new app installer | Create `apps/<name>.sh` (auto-discovered) |
| Add a feature | Create `features/<name>.sh` (installed on demand via `./install.sh --feature`) |
| Add a systemd service | Create `systemd/<name>.service` (auto-installed by postinstall; gate on a flag if it backs a feature) |
| Add a precompiled binary | Drop it in `x64_bin/` (or `arm64_bin/` later) — auto-copied by Phase 2 |
| Inspect/set feature flags | `flag-reader`, `flag-set`, `flag-clear` (or source `lib/flags.sh`) |
| Modify package list | Edit `PACKAGES` array in `preinstall.sh` |
| Change PATH or bash config | Edit `postinstall.sh` |
| Modify fail2ban config | Edit jail.local section in `postinstall.sh` |
| Add bash completion | Edit `completions/pos.bash` |
| Modify Docker Compose logic | Edit `bin/pos-docker-compose` |
| Modify Docker health check | Edit `bin/pos-docker-health` |
| Modify UFW/firewall logic | Edit `bin/pos-system-firewall` |
| Modify pos logging | Edit log setup in `bin/pos` |
| Modify install phases/flags | Edit arg parsing in `install.sh` |
| Update documentation | Edit the relevant doc under `DOC/` (index: `DOC/README.md`) |
| Add a secret config file | Add to `config/`, update `.gitignore`, add copy logic in `postinstall.sh` |