Files

T

gates / consistency-and-conventions (push) Successful in 2m16s

Details

fix: review-driven hardening of pos ai hf/server + llamacpp provider

Adversarial review of the AI tools (commits 387f23f/0856b25) found 2
BLOCKING + 5 REQUIRED defects; all fixed:

- pos-ai-hf --include/--exclude: bash-case glob filtering (array-safe,
  no jq regex interpolation, composes gguf->filename->include->exclude)
- pos-ai-server: ExecStart rebuilt as single-line properly-quoted command
  (systemd_quote for executable + model path; systemd-analyze verify rc=0)
- --branch/--revision aliased (last wins), dead BRANCH variable removed
- parallel download drains all jobs: per-pid wait, honest
  'X of Y files, N failed' summary, rc=1 on partial failure, no .hf-meta
  for half-downloaded models, EXIT-trap temp cleanup
- detect_llama_version guarded; validate_requested_flags errors on
  unsupported explicit flags with version-aware message
- pos ai hf cache [status|clear]: real implementation, fail-closed confirm
- new bin/pos-ai-llamacpp thin forwarder + llamacpp shorthand in bin/pos-ai
  (pos ai llamacpp <subcmd> = pos ai --provider llamacpp <subcmd>)
- docs synced: bin/pos-ai usage(), DOC/POS.md AI_PROVIDER row, howto/ai.md
  (adapter list, --provider backends, shorthand, providers table); gen
  regenerated (tree/dispatch/completions)

Verified: bash -n all bin/pos*; make gen idempotent; make check green;
make lint 0 FAIL, 0 WARN. Reviewer acceptance: APPROVE_WITH_NOTES
(0 REQUIRED). Audit deliverables + agent reports included for context.

2026-09-06 03:45:53 -04:00

4.8 KiB

Raw Blame History

Audit of POS AI Tools Implementation

Current Status Analysis

Based on my examination of the codebase, I can see that the pos ai hf and pos ai server tools are implemented but appear to be incomplete compared to the underlying applications they're supposed to wrap.

Audit Findings

1. `pos ai hf` - Current State vs. Hugging Face CLI

Current Implementation:

Supports search, download, list, remove commands
Basic GGUF filtering capabilities
Branch/revision support
Authentication via HF_TOKEN
Basic file listing and downloading

Missing Hugging Face CLI Features:

Repository Discovery: The tool lacks advanced repository discovery features
Model Information: No info or show commands to get repository details
Advanced Filtering: Missing --include and --exclude patterns
Revision Support: Limited branch support, no tag/commit support
Cache Management: No cache inspection or management
Authentication: Only basic token support, no credential helper integration
Multi-Shard Models: Limited support for sharded GGUF models
Progress Indicators: Basic progress, no detailed download metrics
Environment Variables: Limited environment variable handling

2. `pos ai server` - Current State vs. llama.cpp

Current Implementation:

Supports start, stop, status, models, logs commands
Basic GPU detection and auto-config
Port and host configuration
Model selection
Basic health checking

Missing llama.cpp Features:

GPU Configuration: No support for detailed GPU offloading (-ngl, multi-GPU)
Memory Management: No support for KV cache configuration
Performance Options: Missing batch sizes, ubatch sizes, continuous batching
Sampling Parameters: No temperature, top-k, top-p, repetition penalty controls
Advanced Features: Missing JSON/schema, tool calling, reasoning options
Server Configuration: No support for all server options like metrics, health endpoints
Version Detection: No capability to detect and validate llama-server version
Graceful Shutdown: Limited shutdown handling

Technical Limitations

Hugging Face CLI Analysis

Based on the Hugging Face documentation and typical CLI patterns, the actual hf command provides:

hf list - List repositories
hf info <repo> - Show repository information
hf files <repo> - List repository files
hf download <repo> - Download repository with various options
--include/--exclude - File pattern filtering
--revision - Specific revision support
--cache - Cache management
Authentication with tokens or credential helpers
Detailed progress reporting

llama.cpp Analysis

Based on llama.cpp documentation, the actual llama-server provides:

--model <path> - Model file specification
--port <port> - Port binding
--host <host> - Host binding
--ctx-size <n> - Context size
--n-gpu-layers <n> - GPU layer count
--tensor-split - Tensor split configuration
--split-mode - Split mode (none, layer, row)
--flash-attn - Flash attention support
--threads <n> - Thread count
--mmap - Memory mapping
--mlock - Memory locking
--batch-size - Batch size
--ubatch-size - UBatch size
--log-disable - Logging control
--health - Health endpoint
--slots - Concurrent request slots
--metrics - Metrics endpoint

Required Improvements

For `pos ai hf`:

Add Info Command: pos ai hf info <repo>
Add Files Command: pos ai hf files <repo>
Add Cache Management: pos ai hf cache
Enhance Download: Support include/exclude, revision, and better progress
Model Information: Show model details, size, and metadata
Repository Files: List files with size and metadata
Authentication: Better credential handling
Version Support: Detect and support version-specific features

For `pos ai server`:

GPU Configuration: Support detailed GPU offloading parameters
Memory Management: Context size, KV cache, memory allocation
Performance Tuning: Batch size, ubatch size, continuous batching
Sampling Controls: Temperature, top-k, top-p, repetition penalty
Advanced Features: JSON/schema, tool calling, reasoning
Server Options: Health, metrics, concurrency control
Version Awareness: Detect and validate supported options
Process Management: Better monitoring and graceful shutdown

Implementation Approach

Given that we don't have the actual underlying CLI tools installed in this environment, I'll need to:

Create a comprehensive audit document
Design the proper interface based on documented capabilities
Implement stubs and placeholders for actual functionality
Ensure all the missing features are properly accounted for in the plan

4.8 KiB Raw Blame History