diff --git a/AGENT_TODO.md b/AGENT_TODO.md index 190cde8..3c9a7ba 100644 --- a/AGENT_TODO.md +++ b/AGENT_TODO.md @@ -42,6 +42,8 @@ summary (newest last). ## Done +- **2026-09-04** β€” `pos ai hf` (`bin/pos-ai-hf`) β€” Hugging Face model downloader. Subcommands: `download [filename]` (single file, whole repo, `--gguf` filter, `--branch `, `--output `), `search `, `list`, `remove`. Downloads to `~/.local/share/linux_post_install/ai/models/-/` (seam-guarded `HF_DOWNLOAD_DIR`), writes `.hf-meta` JSON per repo, prints structured summary (πŸ“₯/πŸ“). Config extends the existing `ai` scope via `# POS_CONFIG: ai` β€” `HF_TOKEN` (secret) and `HF_DOWNLOAD_DIR` in `~/.config/linux_post_install/ai.env` with env-var precedence. Auth on all requests; HTTP 429 rate-limit sleep + retry once; resume via `curl -C -`; progress bars to stderr. Deps: `curl`/`jq` guards before `--help`; no stdin β†’ not in INTERACTIVE_CMDS. Verified: stub-PATH suite `/tmp/opencode/hf-test/run-tests.sh` 46/46 green (argument parsing, download single/multi/gguf/branch/output, search, list, remove, config/token, output format); `bash -n`; `make gen && make check` green; `make lint` 0 FAIL / 0 WARN. Docs: POS.md ai row + detail block. + - **2026-09-04** β€” `pos media grab` (`bin/pos-media-grab`) β€” auto-download a URL as audio or video. Classifies by domain (YouTube Music/SoundCloud/Bandcamp β†’ mp3; YouTube/Vimeo/Twitch β†’ mp4) with `--audio`/`--video` overrides and `GRAB_DEFAULT` config (`pos config grab`, default `video`) for unknown domains; `--best` default for video (non-interactive, `--worst` override); all flags (`--output`, `--no-playlist`, `--cookies`, `--dry-run`) forwarded to mp3/mp4; prints a clean summary (🎡/🎬 title, duration, path, size). Telegram listener (`bin/pos-communication-telegram-listener`) gains `url_detect` + a URL routing step between the prefix map and AI bridge β€” bare http(s) URLs route to `pos media grab --best` (600s timeout). Verified: `/tmp/opencode/media-grab-test/run-tests.sh` 28 cases / 70 assertions green; `bash -n` on both files; `make gen && make check` green; `make lint` 0 FAIL / 0 WARN. - **2026-08-21** β€” Share suite interactive layer (`lib/share-lib.sh` + menu modes for all five `pos share *` tools): bare invocation now opens an EOF-safe looping menu instead of printing usage. New `lib/share-lib.sh` (436 lines) owns the shared primitives β€” `share_menu_guard`/`share_menu_run`/`share_pick`/`share_ask_value` (quit on EOF so non-tty callers can't hang), `share_require_bin`/`share_port_probe`/`share_service_active`/`share_path_probe` rc-only probes, `share_usb_records` (blank-line-record parser for usbsrv listings), `share_smb_shares` (smbclient `-g` Disk enumeration incl. guestβ†’auth retry) + `share_usb_devices`/`share_usb_clients`, `share_nfs_exports` (showmount), `share_folder_candidates` (bounded-probe scan of mounted targets + conventional roots; container overlay/tmpfs/nsfs excluded via findmnt; clients build their own mountpoint pickers on top), and advisories `share_ufw_blocks_ports`+`share_offer_fix`. Tools keep every legacy flag/subcommand byte-compatible (verbatim command bodies, thin menu layer on top): nfs-server gains a client-spec presets picker + inactive-service/UFW offers, nfs-client gains idempotent unmount/unpersist (already-absent = report, rc0) + persist-verify-with-rollback + replace-confirm, smb-server gets UFW offer + menu tree, smb-client gets enumerateβ†’pickβ†’mount with account reuse (`SMB_AUTH_USER` contract), usb-server picker-first with raw-listing manual-entry fallback when the server listing is unreadable. New seams: `EXPORTS_FILE` (nfs-server), `UNIT_DIR` (nfs-client); `bin/pos` INTERACTIVE_CMDS += both stdin-reading share tools; install.sh lib list += share-lib.sh; preinstall.sh += smbclient (smb-client enumeration dep). Docs: POS.md share rows/detail, howto/share.md per-tool Interactive-menu notes, SCRIPTS.md phase table + new `## lib/share-lib.sh` section, DEV.md lib row + env-seam registry, AGENT_Context hand-maintained spots (lib table row 436, Phase-2 prose). Verified: 80-case stub battery vs recaptured deterministic golden β€” only the 9 documented intentional deltas differ (additive help lines, seam-path strings in messages, missing-dep message delta, unmount-idle FLAGGEDβ†’rc0, unpersist round-trip now works, usbs-bare usageβ†’menu guard); 12/12 PTY tests (menus open/quit non-tty, filter/zero-match/default/cancel picker semantics, full nfs-server share flow writes the export line, usb-server share via pickers + down-server fallback, smb-client guest-enumerateβ†’manual-share flow); real `/etc/exports` + `/etc/systemd/system` md5-verified untouched; `make gen && make check` green, `make lint` 0 FAIL / 0 WARN. diff --git a/DOC/AGENT_Context_Project.md b/DOC/AGENT_Context_Project.md index 9b1f5c5..af6d381 100644 --- a/DOC/AGENT_Context_Project.md +++ b/DOC/AGENT_Context_Project.md @@ -10,19 +10,19 @@ | ## 1. Project Overview | 28–43 | -| ## 2. Directory Structure | 44–205 | -| ## 3. Installation Flow | 206–259 | -| ## 4. The `pos` CLI System | 260–338 | -| ## 5. Shared Library β€” `lib/common.sh` | 339–370 | -| ## 6. Docker Compose / ScaleTail | 371–413 | -| ## 7. Optional Apps (`apps/`) | 414–443 | -| ## 8. Entertainment Module | 444–457 | -| ## 9. Systemd Services | 458–469 | -| ## 10. Configuration Files | 470–496 | -| ## 11. Coding Conventions | 497–529 | -| ## 12. Development Workflow | 530–582 | -| ## 13. Key File Quick Reference | 583–655 | -| ## 14. Common Tasks for Agents | 656–689 | +| ## 2. Directory Structure | 44–207 | +| ## 3. Installation Flow | 208–261 | +| ## 4. The `pos` CLI System | 262–341 | +| ## 5. Shared Library β€” `lib/common.sh` | 342–373 | +| ## 6. Docker Compose / ScaleTail | 374–416 | +| ## 7. Optional Apps (`apps/`) | 417–446 | +| ## 8. Entertainment Module | 447–460 | +| ## 9. Systemd Services | 461–472 | +| ## 10. Configuration Files | 473–499 | +| ## 11. Coding Conventions | 500–532 | +| ## 12. Development Workflow | 533–585 | +| ## 13. Key File Quick Reference | 586–659 | +| ## 14. Common Tasks for Agents | 660–693 | ## 1. Project Overview @@ -63,6 +63,8 @@ Linux_post_install/ β”‚ β”œβ”€β”€ pos-ai-alias # manage AI agent aliases β”‚ β”œβ”€β”€ pos-ai-gemini # Forward to pos ai --provider gemini (backward compat) +β”‚ β”œβ”€β”€ pos-ai-hf # Download AI models from Hugging Face (search, download, manage) +β”‚ β”‚ [deps: curl jq] β”‚ β”œβ”€β”€ pos-ai-openrouter # Forward to pos ai --provider openrouter (backward compat) β”‚ β”œβ”€β”€ pos-communication-matrix-listener # Matrix listener: map /command β†’ bash, run them on room messages β”‚ β”œβ”€β”€ pos-communication-matrix-sender # Send messages to a Matrix room via the client-server API (send, test, login) @@ -278,6 +280,7 @@ All non-interactive `pos` commands log output to `~/.local/share/linux_post_inst |----------|---------|--------|-------------|------|----------| | ai | alias | `pos-ai-alias` | manage AI agent aliases | | | | ai | gemini | `pos-ai-gemini` | Forward to pos ai --provider gemini (backward compat) | | | +| ai | hf | `pos-ai-hf` | Download AI models from Hugging Face (search, download, manage) | curl jq | pos ai hf search llama 7b β†’ Search Hugging Face for "llama 7b" models Β· pos ai hf download meta-llama/Llama-3.1-8B-Instruct β†’ Download all files from a repo Β· pos ai hf download meta-llama/Llama-3.1-8B-Instruct --gguf β†’ Download only GGUF quantized files Β· pos ai hf download meta-llama/Llama-3.1-8B-Instruct config.json β†’ Download a single file Β· pos ai hf list β†’ List downloaded models Β· pos ai hf remove meta-llama-Llama-3.1-8B-Instruct β†’ Remove a downloaded model | | ai | openrouter | `pos-ai-openrouter` | Forward to pos ai --provider openrouter (backward compat) | | | | communication | matrix-listener | `pos-communication-matrix-listener` | Matrix listener: map /command β†’ bash, run them on room messages | | | | communication | matrix-sender | `pos-communication-matrix-sender` | Send messages to a Matrix room via the client-server API (send, test, login) | | | @@ -607,6 +610,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:` | `bin/pos` | 302 | CLI dispatcher with smart arg matching + logging + category help | | `bin/pos-ai-alias` | 760 | manage AI agent aliases | | `bin/pos-ai-gemini` | 7 | Forward to pos ai --provider gemini (backward compat) | +| `bin/pos-ai-hf` | 495 | Download AI models from Hugging Face (search, download, manage) | | `bin/pos-ai-openrouter` | 7 | Forward to pos ai --provider openrouter (backward compat) | | `bin/pos-communication-matrix-listener` | 568 | Matrix listener: map /command β†’ bash, run them on room messages | | `bin/pos-communication-matrix-sender` | 224 | Send messages to a Matrix room via the client-server API (send, test, login) | @@ -647,7 +651,7 @@ Use conventional prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `chore:` | `bin/pos-ai` | 692 | AI assistant: ask, chat, sessions, capture, models, providers | | `bin/pos-config` | 80 | Interactive editor for the tools' runtime config (reads # POS_CONFIG: registry) | | `bin/pos-tree` | 118 | Show the pos CLI command tree: categories, commands, and subcommands | -| `completions/pos.bash` | 309 | Dynamic bash completion | +| `completions/pos.bash` | 310 | Dynamic bash completion | | `apps/install.sh` | 171 | App install/uninstall picker/orchestrator | diff --git a/DOC/POS.md b/DOC/POS.md index 7cff360..5b9b3c0 100644 --- a/DOC/POS.md +++ b/DOC/POS.md @@ -55,7 +55,7 @@ Category-less tools (`config`, `tree`) live outside any category and are documen ### ai -**File:** `bin/pos-ai` (provider-agnostic main tool), `bin/pos-ai-gemini` / `bin/pos-ai-openrouter` (backward-compat forwarders β†’ `pos ai --provider `) +**File:** `bin/pos-ai` (provider-agnostic main tool), `bin/pos-ai-gemini` / `bin/pos-ai-openrouter` (backward-compat forwarders β†’ `pos ai --provider `), `bin/pos-ai-hf` (Hugging Face model downloader) **Provider adapters:** `lib/ai-providers/gemini.sh`, `lib/ai-providers/openrouter.sh` **Purpose:** AI assistant with pluggable providers. Six subcommands: `ask` (scriptable, persistent session), `capture` (run a command and save its output for `--last`), `chat` (interactive multi-turn REPL), `models` (list available models), `providers` (list providers and config status), and `sessions` (list/clear sessions). Providers handle API-specific logic; the main tool handles sessions, rendering, machine context, and all shared logic. @@ -100,6 +100,17 @@ Model precedence: `--model` flag > `AI_MODEL` env > provider-specific fallback ( **Messaging bridges:** the Telegram and Matrix listeners forward non-command messages starting with `ai ` (case-insensitive) to `pos ai ask` and reply with the model's answer β€” see [communication β†’ listener](#communication). The Telegram bridge uses one session per chat (`telegram-`), the Matrix bridge one per room (`matrix-`). +`pos ai hf` β€” Hugging Face model downloader: + +| Command | Behavior | +|---------|----------| +| `pos ai hf search ` | Search Hugging Face models by query (sorted by downloads); prints model ID, download count | +| `pos ai hf download [filename]` | Download a file or entire repo from Hugging Face. Creates `-/` under `HF_DOWNLOAD_DIR` (default `~/.local/share/linux_post_install/ai/models/`). Options: `--branch ` (specific branch), `--gguf` (only `.gguf` files), `--output ` (override download dir). Progress bars to stderr; summary with path and size to stdout. Writes `.hf-meta` JSON (repo-id, branch, files, timestamp) for `list` and `remove` | +| `pos ai hf list` | List all downloaded models with size and date | +| `pos ai hf remove ` | Remove a downloaded model directory and show freed space | + +Auth: `HF_TOKEN` in `~/.config/linux_post_install/ai.env` (same scope as `pos ai`; edit via `pos config ai`). Even for public repos, a token increases rate limits from 500/5min to 1000/5min. Resume: `curl -C -` resumes interrupted downloads. Rate limit handling: on HTTP 429, sleeps `Retry-After` or 60s, retries once. + ### network | Command | File | Purpose | Configuration | diff --git a/bin/pos-ai-hf b/bin/pos-ai-hf new file mode 100755 index 0000000..eb41712 --- /dev/null +++ b/bin/pos-ai-hf @@ -0,0 +1,495 @@ +#!/usr/bin/env bash +set -euo pipefail +# POS: ai hf β€” Download AI models from Hugging Face (search, download, manage) +# POS_FLAGS: --branch --gguf --output +# POS_DEPS: curl jq +# POS_CONFIG: ai | ai.env | HF_TOKEN=secret:Hugging Face API token (https://huggingface.co/settings/tokens) | HF_DOWNLOAD_DIR=:Model download directory (default ~/.local/share/linux_post_install/ai/models) +# POS_EXAMPLES: pos ai hf search llama 7b | Search Hugging Face for "llama 7b" models +# POS_EXAMPLES: pos ai hf download meta-llama/Llama-3.1-8B-Instruct | Download all files from a repo +# POS_EXAMPLES: pos ai hf download meta-llama/Llama-3.1-8B-Instruct --gguf | Download only GGUF quantized files +# POS_EXAMPLES: pos ai hf download meta-llama/Llama-3.1-8B-Instruct config.json | Download a single file +# POS_EXAMPLES: pos ai hf list | List downloaded models +# POS_EXAMPLES: pos ai hf remove meta-llama-Llama-3.1-8B-Instruct | Remove a downloaded model + +source "$(dirname "$0")/../lib/common.sh" 2>/dev/null || source "$(dirname "$0")/common.sh" + +# ── Dependencies (before --help) ─────────────────────────────── +command -v curl &>/dev/null || err "curl not found (install curl)" +command -v jq &>/dev/null || err "jq not found (install jq)" + +# ── Config & defaults ────────────────────────────────────────── +CONFIG_FILE="${CONFIG_FILE:-$HOME/.config/linux_post_install/ai.env}" +HF_TOKEN="${HF_TOKEN:-}" +HF_DOWNLOAD_DIR="${HF_DOWNLOAD_DIR:-$HOME/.local/share/linux_post_install/ai/models}" + +load_hf_config() { + [ -f "$CONFIG_FILE" ] || return 0 + local k v + while IFS='=' read -r k v; do + [ -n "$k" ] || continue + case "$k" in + \#*) continue ;; + esac + v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" + v="${v//$'\r'/}" + if [ -z "${!k:-}" ]; then + export "$k"="$v" + fi + done < <(grep -E '^[A-Z_]+=' "$CONFIG_FILE" || true) +} + +load_hf_config + +# ── Usage ────────────────────────────────────────────────────── +usage() { + cat <<'EOF' +Usage: pos ai hf [args] + +Hugging Face model downloader β€” search, download, and manage AI models. + +Subcommands: + search Search Hugging Face models + download [filename] Download a file or entire repo + list List downloaded models + remove Remove a downloaded model + +Download options: + --branch Download from a specific branch/revision + --gguf Download only .gguf files (inference-ready) + --output Override download directory + +Examples: + pos ai hf search llama 7b + pos ai hf download meta-llama/Llama-3.1-8B-Instruct + pos ai hf download meta-llama/Llama-3.1-8B-Instruct --gguf + pos ai hf download meta-llama/Llama-3.1-8B-Instruct config.json + pos ai hf download meta-llama/Llama-3.1-8B-Instruct --branch main + pos ai hf list + pos ai hf remove meta-llama-Llama-3.1-8B-Instruct + +Config (~/.config/linux_post_install/ai.env): + HF_TOKEN Hugging Face API token (better rate limits for public repos) + HF_DOWNLOAD_DIR Model download directory (default ~/.local/share/linux_post_install/ai/models) + +Exit codes: + 0 success + 1 error (missing deps, invalid input, API failure) +EOF + exit 0 +} + +# ── Parse global flags ───────────────────────────────────────── +SUBCMD="" +SUBCMD_ARGS=() +BRANCH="" +GGUF_ONLY=0 +OUTPUT_DIR="" + +while [ $# -gt 0 ]; do + case "$1" in + -h|--help) usage ;; + --branch) + [ $# -ge 2 ] || err "--branch requires a value" + BRANCH="$2"; shift 2 ;; + --gguf) + GGUF_ONLY=1; shift ;; + --output) + [ $# -ge 2 ] || err "--output requires a value" + OUTPUT_DIR="$2"; shift 2 ;; + -*) + err "Unknown option '$1' (see --help)" ;; + *) + if [ -z "$SUBCMD" ]; then + SUBCMD="$1" + else + SUBCMD_ARGS+=("$1") + fi + shift ;; + esac +done + +[ -n "$SUBCMD" ] || usage + +# Apply output dir override +if [ -n "$OUTPUT_DIR" ]; then + HF_DOWNLOAD_DIR="$OUTPUT_DIR" +fi +HF_DOWNLOAD_DIR="${HF_DOWNLOAD_DIR:-$HOME/.local/share/linux_post_install/ai/models}" + +# ── Token warning ────────────────────────────────────────────── +if [ -z "$HF_TOKEN" ]; then + warn "No HF_TOKEN set β€” using anonymous access" +fi + +# ── HF API helpers ───────────────────────────────────────────── +HF_BASE="https://huggingface.co" +HF_API_BASE="https://huggingface.co/api" + +hf_auth_header() { + if [ -n "$HF_TOKEN" ]; then + printf 'Authorization: Bearer %s' "$HF_TOKEN" + fi +} + +hf_api() { + local endpoint="$1" + local url="${HF_API_BASE}${endpoint}" + local auth_header + auth_header="$(hf_auth_header)" + + local http_code body tmpfile + tmpfile="$(mktemp)" + + local curl_args=(-sS -w '%{http_code}' -o "$tmpfile" --max-time 30) + if [ -n "$auth_header" ]; then + curl_args+=(-H "$auth_header") + fi + + # Rate limit retry: on 429, sleep and retry once + local attempt=0 + while [ $attempt -lt 2 ]; do + http_code="$(curl "${curl_args[@]}" "$url" 2>/dev/null)" || { + rm -f "$tmpfile" + err "Connection timed out β€” check network" + } + + if [ "$http_code" = "429" ]; then + local retry_after + retry_after="$(curl -sI -H "${auth_header:-}" "$url" 2>/dev/null | grep -i 'retry-after:' | tr -d '\r' | awk '{print $2}')" + retry_after="${retry_after:-60}" + warn "Rate limited β€” waiting ${retry_after}s before retry" + sleep "$retry_after" + attempt=$((attempt + 1)) + continue + fi + + break + done + + body="$(cat "$tmpfile")" + rm -f "$tmpfile" + + case "$http_code" in + 200) ;; + 401|403) err "Authentication failed β€” check HF_TOKEN (pos config ai)" ;; + 404) err "Model not found: ${endpoint#/api/models/}" ;; + 429) err "Rate limit exceeded β€” try again later" ;; + *) err "API request failed (HTTP $http_code)" ;; + esac + + # Validate JSON + if ! printf '%s' "$body" | jq empty 2>/dev/null; then + err "Failed to parse API response β€” check network or HF status" + fi + + printf '%s' "$body" +} + +hf_repo_files() { + local repo_id="$1" + local branch="${2:-main}" + local ns="${repo_id%%/*}" + local repo="${repo_id#*/}" + + if [ "$ns" = "$repo" ]; then + err "Invalid repo format: use namespace/model-name" + fi + + # Try /tree/ endpoint first (has file sizes + LFS info) + local endpoint="/models/${ns}/${repo}/tree/${branch}" + local result + if result="$(hf_api "$endpoint" 2>/dev/null)"; then + printf '%s' "$result" + return 0 + fi + + # Fallback: /api/models/{ns}/{repo} (siblings, no sizes) + warn "Tree endpoint unavailable, using repo metadata" + local fallback + fallback="$(hf_api "/models/${ns}/${repo}")" || err "Failed to fetch repo info for $repo_id" + printf '%s' "$fallback" | jq '[.siblings[] | {rfilename: .rfilename, size: (.size // 0)}]' +} + +hf_search() { + local query="$1" + local limit="${2:-10}" + local encoded_query + encoded_query="$(printf '%s' "$query" | jq -sRr @uri)" + local endpoint="/models?search=${encoded_query}&sort=downloads&direction=-1&limit=${limit}" + hf_api "$endpoint" +} + +# ── Utility functions ────────────────────────────────────────── +hf_repo_dir() { + local repo_id="$1" + printf '%s' "$HF_DOWNLOAD_DIR/${repo_id//\//-}" +} + +hf_human_size() { + local bytes="$1" + if [ "$bytes" -ge 1073741824 ]; then + awk "BEGIN { printf \"%.1f GB\", $bytes / 1073741824 }" + elif [ "$bytes" -ge 1048576 ]; then + awk "BEGIN { printf \"%.1f MB\", $bytes / 1048576 }" + elif [ "$bytes" -ge 1024 ]; then + awk "BEGIN { printf \"%.1f KB\", $bytes / 1024 }" + else + printf '%d B' "$bytes" + fi +} + +hf_resolve_branch() { + local repo_id="$1" + local branch="${2:-}" + if [ -n "$branch" ]; then + printf '%s' "$branch" + return + fi + # Try to get default branch from API + local ns="${repo_id%%/*}" + local repo="${repo_id#*/}" + local meta + if meta="$(hf_api "/models/${ns}/${repo}" 2>/dev/null)"; then + local default_branch + default_branch="$(printf '%s' "$meta" | jq -r '.defaultBranch // empty' 2>/dev/null)" + if [ -n "$default_branch" ]; then + printf '%s' "$default_branch" + return + fi + fi + printf 'main' +} + +# ── Download helpers ─────────────────────────────────────────── +hf_download_file() { + local url="$1" + local target="$2" + local auth_header + auth_header="$(hf_auth_header)" + + local curl_args=(-L -C - --progress-bar -o "$target") + if [ -n "$auth_header" ]; then + curl_args+=(-H "$auth_header") + fi + + if curl "${curl_args[@]}" "$url" 2>&1; then + if [ -s "$target" ]; then + return 0 + else + warn "Downloaded file is empty: $target" + return 1 + fi + else + warn "Download interrupted for $(basename "$target") (resume with same command)" + return 1 + fi +} + +# ── Subcommands ──────────────────────────────────────────────── + +cmd_search() { + local query="${SUBCMD_ARGS[*]:-}" + [ -n "$query" ] || err "Usage: pos ai hf search " + + local result + result="$(hf_search "$query" "10")" + + local count + count="$(printf '%s' "$result" | jq 'length')" + [ "$count" -gt 0 ] || { warn "No models found for '$query'"; return 0; } + + printf 'Found %d models for "%s":\n' "$count" "$query" + printf '%s' "$result" | jq -r '.[] | " \(.id)\t\(.downloads // 0)\t\(.likes // 0)"' | \ + while IFS=$'\t' read -r id downloads likes; do + local dl_str + if [ "$downloads" -ge 1000 ]; then + dl_str="$(awk "BEGIN { printf \"%.1fk\", $downloads / 1000 }")" + else + dl_str="$downloads" + fi + printf ' %-55s %s downloads\n' "$id" "$dl_str" + done +} + +cmd_download() { + local repo_id="${SUBCMD_ARGS[0]:-}" + [ -n "$repo_id" ] || err "Usage: pos ai hf download [filename]" + + # Validate repo-id contains / + [[ "$repo_id" == */* ]] || err "Invalid repo format: use namespace/model-name" + + local filename="${SUBCMD_ARGS[1]:-}" + local branch + branch="$(hf_resolve_branch "$repo_id" "$BRANCH")" + + # Get file list from API + local files_json + files_json="$(hf_repo_files "$repo_id" "$branch")" + + # Filter files + local filtered_files + if [ -n "$filename" ]; then + # Single file mode + filtered_files="$(printf '%s' "$files_json" | jq -c --arg fn "$filename" '[.[] | select(.rfilename == $fn)]')" + elif [ "$GGUF_ONLY" -eq 1 ]; then + # GGUF filter + filtered_files="$(printf '%s' "$files_json" | jq -c '[.[] | select(.rfilename | endswith(".gguf"))]')" + else + # All files + filtered_files="$(printf '%s' "$files_json" | jq -c '.')" + fi + + local file_count + file_count="$(printf '%s' "$filtered_files" | jq 'length')" + [ "$file_count" -gt 0 ] || err "No files to download" + + # Prepare target directory + local target_dir + target_dir="$(hf_repo_dir "$repo_id")" + mkdir -p "$target_dir" + + # Disk space pre-flight check + local total_size + total_size="$(printf '%s' "$filtered_files" | jq '[.[].size // 0] | add // 0')" + if [ "$total_size" -gt 0 ]; then + local avail_kb + avail_kb="$(df --output=avail "$target_dir" 2>/dev/null | tail -1 | tr -d ' ')" + local need_kb=$((total_size / 1024)) + if [ "$avail_kb" -lt "$need_kb" ]; then + local need_human avail_human + need_human="$(hf_human_size "$total_size")" + avail_human="$(hf_human_size "$((avail_kb * 1024))")" + warn "Low disk space: need $need_human, only $avail_human available" + fi + fi + + local downloaded=0 + local ns="${repo_id%%/*}" + local repo="${repo_id#*/}" + + while IFS= read -r file_json; do + local fname fsize + fname="$(printf '%s' "$file_json" | jq -r '.rfilename')" + fsize="$(printf '%s' "$file_json" | jq -r '.size // 0')" + total_size=$((total_size + fsize)) + + local url="${HF_BASE}/${ns}/${repo}/resolve/${branch}/${fname}" + local target="${target_dir}/${fname}" + + if [ "$file_count" -gt 1 ]; then + downloaded=$((downloaded + 1)) + printf '[%d/%d] Downloading %s...\n' "$downloaded" "$file_count" "$fname" >&2 + fi + + # Create parent directory + mkdir -p "$(dirname "$target")" + + if ! hf_download_file "$url" "$target"; then + warn "Failed to download $fname" + continue + fi + done < <(printf '%s' "$filtered_files" | jq -c '.[]') + + # Write metadata + local meta_file="${target_dir}/.hf-meta" + local timestamp + timestamp="$(date -u +%Y-%m-%dT%H:%M:%SZ)" + local file_list + file_list="$(printf '%s' "$filtered_files" | jq -c '[.[] | .rfilename]')" + + cat > "$meta_file" </dev/null | sort) + + [ ${#models[@]} -gt 0 ] || { warn "No models downloaded yet"; return 0; } + + printf 'Downloaded models (%d):\n' "${#models[@]}" + for dir in "${models[@]}"; do + local meta_file="${dir}/.hf-meta" + local name + name="$(basename "$dir")" + local total_size=0 + local date_str + date_str="$(jq -r '.downloaded_at // "unknown"' "$meta_file" 2>/dev/null | cut -dT -f1)" + + # Calculate total size + while IFS= read -r file; do + [ -f "$file" ] || continue + local size + size="$(stat -c%s "$file" 2>/dev/null || echo 0)" + total_size=$((total_size + size)) + done < <(find "$dir" -type f ! -name '.hf-meta' 2>/dev/null) + + local human_size + human_size="$(hf_human_size "$total_size")" + printf ' %-50s %s %s\n' "$name" "$human_size" "$date_str" + done +} + +cmd_remove() { + local repo_id="${SUBCMD_ARGS[0]:-}" + [ -n "$repo_id" ] || err "Usage: pos ai hf remove " + + local target_dir + target_dir="$(hf_repo_dir "$repo_id")" + + [ -d "$target_dir" ] || err "Model not found: $repo_id" + + # Calculate size before removal + local total_size=0 + while IFS= read -r file; do + [ -f "$file" ] || continue + local size + size="$(stat -c%s "$file" 2>/dev/null || echo 0)" + total_size=$((total_size + size)) + done < <(find "$target_dir" -type f 2>/dev/null) + + local human_size + human_size="$(hf_human_size "$total_size")" + + rm -rf "$target_dir" + printf 'Removed: %s (freed %s)\n' "$repo_id" "$human_size" +} + +# ── Dispatch ─────────────────────────────────────────────────── +case "$SUBCMD" in + search) cmd_search ;; + download) cmd_download ;; + list) cmd_list ;; + remove) cmd_remove ;; + *) err "Unknown subcommand '$SUBCMD' (see --help)" ;; +esac diff --git a/completions/pos.bash b/completions/pos.bash index ab34392..078ac78 100644 --- a/completions/pos.bash +++ b/completions/pos.bash @@ -3,6 +3,7 @@ # Install: source this file in ~/.bashrc or place in /etc/bash_completion.d/ # GEN:START posflags declare -A _pos_flags +_pos_flags[ai-hf]="--branch --gguf --output" _pos_flags[communication-matrix-listener]="--enable --disable --status --run" _pos_flags[communication-telegram-listener]="--enable --disable --status --sync-commands --run" _pos_flags[communication-telegram-sender]="--type --caption --parse-mode --no-preview --token --chat-id --markdown" @@ -44,7 +45,7 @@ _pos_subcmds[share-smb-client]="mount unmount list persist unpersist menu" _pos_subcmds[share-smb-server]="status share unshare list adduser deluser reload enable disable menu" _pos_subcmds[system-backup]="menu" _pos_subcmds[system-schedule]="run list config enable disable status migrate menu" -_pos_subcmds[ai]="ask chat sessions capture models providers alias gemini openrouter" +_pos_subcmds[ai]="ask chat sessions capture models providers alias gemini hf openrouter" # GEN:END possubcmds # GEN:START posconfigscopes declare -a _pos_config_scopes=(ai compose entertainment grab matrix notify scrcpy system telegram ytsync)