fix: review-driven hardening of pos ai hf/server + llamacpp provider
gates / consistency-and-conventions (push) Successful in 2m16s
gates / consistency-and-conventions (push) Successful in 2m16s
Adversarial review of the AI tools (commits 387f23f/0856b25) found 2 BLOCKING + 5 REQUIRED defects; all fixed: - pos-ai-hf --include/--exclude: bash-case glob filtering (array-safe, no jq regex interpolation, composes gguf->filename->include->exclude) - pos-ai-server: ExecStart rebuilt as single-line properly-quoted command (systemd_quote for executable + model path; systemd-analyze verify rc=0) - --branch/--revision aliased (last wins), dead BRANCH variable removed - parallel download drains all jobs: per-pid wait, honest 'X of Y files, N failed' summary, rc=1 on partial failure, no .hf-meta for half-downloaded models, EXIT-trap temp cleanup - detect_llama_version guarded; validate_requested_flags errors on unsupported explicit flags with version-aware message - pos ai hf cache [status|clear]: real implementation, fail-closed confirm - new bin/pos-ai-llamacpp thin forwarder + llamacpp shorthand in bin/pos-ai (pos ai llamacpp <subcmd> = pos ai --provider llamacpp <subcmd>) - docs synced: bin/pos-ai usage(), DOC/POS.md AI_PROVIDER row, howto/ai.md (adapter list, --provider backends, shorthand, providers table); gen regenerated (tree/dispatch/completions) Verified: bash -n all bin/pos*; make gen idempotent; make check green; make lint 0 FAIL, 0 WARN. Reviewer acceptance: APPROVE_WITH_NOTES (0 REQUIRED). Audit deliverables + agent reports included for context.
This commit is contained in:
Your Name
@@ -0,0 +1,331 @@
|
||||
# Implementation Plan for POS AI Tools
|
||||
|
||||
## Overview
|
||||
|
||||
This document outlines the comprehensive implementation plan for enhancing the `pos ai hf` and `pos ai server` tools to make them robust, useful wrappers around the actual Hugging Face CLI and llama.cpp server.
|
||||
|
||||
## 1. `pos ai hf` Enhancement Plan
|
||||
|
||||
### Current Limitations Identified
|
||||
|
||||
Based on audit, the current implementation is missing:
|
||||
- Repository information (`info` command)
|
||||
- File listing (`files` command)
|
||||
- Cache management
|
||||
- Advanced filtering (`--include`, `--exclude`)
|
||||
- Revision support
|
||||
- Better progress reporting
|
||||
- Enhanced authentication handling
|
||||
|
||||
### Required Enhancements
|
||||
|
||||
#### A. Add New Commands
|
||||
|
||||
**1. Info Command**
|
||||
```bash
|
||||
pos ai hf info <repo>
|
||||
```
|
||||
- Show repository metadata (size, downloads, likes, tags, etc.)
|
||||
- Display model architecture information
|
||||
- Show commit history and version information
|
||||
|
||||
**2. Files Command**
|
||||
```bash
|
||||
pos ai hf files <repo>
|
||||
```
|
||||
- List all files in repository with sizes
|
||||
- Show file types and metadata
|
||||
- Support pattern matching
|
||||
|
||||
**3. Cache Command**
|
||||
```bash
|
||||
pos ai hf cache
|
||||
```
|
||||
- Show cache status
|
||||
- Clear cache
|
||||
- Manage local cache
|
||||
|
||||
#### B. Enhanced Download Capabilities
|
||||
|
||||
**1. Advanced Filtering Support**
|
||||
- `--include`/`--exclude` patterns
|
||||
- File globbing support
|
||||
- Wildcard matching
|
||||
|
||||
**2. Revision Support**
|
||||
- `--revision` for commits/tags/branches
|
||||
- Specific version targeting
|
||||
|
||||
**3. Progress Reporting**
|
||||
- Detailed download progress
|
||||
- Transfer speed indicators
|
||||
- Estimated time remaining
|
||||
|
||||
#### C. Authentication Improvements
|
||||
|
||||
**1. Enhanced Token Handling**
|
||||
- Support for credential helpers
|
||||
- Better error messages for authentication failures
|
||||
- Token validation
|
||||
|
||||
#### D. Integration Improvements
|
||||
|
||||
**1. Better Error Handling**
|
||||
- More descriptive error messages
|
||||
- Context-specific help
|
||||
- Graceful degradation
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
#### Phase 1: Core Infrastructure (Week 1)
|
||||
1. Add new command structure to POS registry
|
||||
2. Implement basic command routing
|
||||
3. Add enhanced error handling
|
||||
4. Update documentation
|
||||
|
||||
#### Phase 2: New Commands (Week 2)
|
||||
1. Implement `info` command
|
||||
2. Implement `files` command
|
||||
3. Implement `cache` command
|
||||
4. Add command-specific help text
|
||||
|
||||
#### Phase 3: Advanced Features (Week 3)
|
||||
1. Add `--include`/`--exclude` support
|
||||
2. Add revision support
|
||||
3. Enhance progress reporting
|
||||
4. Improve authentication handling
|
||||
|
||||
## 2. `pos ai server` Enhancement Plan
|
||||
|
||||
### Current Limitations Identified
|
||||
|
||||
Based on audit, the current implementation is missing:
|
||||
- Detailed GPU configuration (`-ngl`, multi-GPU)
|
||||
- Memory management parameters
|
||||
- Performance tuning options
|
||||
- Advanced sampling controls
|
||||
- Server configuration options
|
||||
- Version awareness
|
||||
- Process monitoring
|
||||
|
||||
### Required Enhancements
|
||||
|
||||
#### A. GPU Configuration
|
||||
|
||||
**1. Detailed GPU Support**
|
||||
```bash
|
||||
pos ai server start --gpu-layers <n> --gpu-threads <n> --tensor-split <n>
|
||||
```
|
||||
- Support for `--n-gpu-layers`
|
||||
- Support for tensor splitting
|
||||
- Multi-GPU configuration
|
||||
|
||||
**2. Device Selection**
|
||||
- GPU device selection
|
||||
- CPU fallback handling
|
||||
|
||||
#### B. Memory and Context Management
|
||||
|
||||
**1. Context Size Control**
|
||||
```bash
|
||||
pos ai server start --ctx-size <n> --kv-cache <size>
|
||||
```
|
||||
|
||||
**2. Memory Allocation**
|
||||
- Support for `--mmap`, `--mlock`
|
||||
- KV cache configuration
|
||||
|
||||
#### C. Performance Tuning
|
||||
|
||||
**1. Batch Size Configuration**
|
||||
```bash
|
||||
pos ai server start --batch-size <n> --ubatch-size <n>
|
||||
```
|
||||
|
||||
**2. Continuous Batching**
|
||||
- Support for continuous batching options
|
||||
- Parallel request handling
|
||||
|
||||
#### D. Sampling Controls
|
||||
|
||||
**1. Advanced Sampling**
|
||||
```bash
|
||||
pos ai server start --temperature <n> --top-k <n> --top-p <n> --repetition-penalty <n>
|
||||
```
|
||||
|
||||
**2. Advanced Features**
|
||||
- JSON/schema support
|
||||
- Tool calling capabilities
|
||||
- Reasoning options
|
||||
|
||||
#### E. Server Configuration
|
||||
|
||||
**1. Endpoint Configuration**
|
||||
- Health endpoints
|
||||
- Metrics endpoints
|
||||
- Authentication handling
|
||||
|
||||
**2. Process Management**
|
||||
- Graceful shutdown
|
||||
- Process monitoring
|
||||
- Log management
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
#### Phase 1: Core Infrastructure (Week 1)
|
||||
1. Extend command structure for server options
|
||||
2. Add version detection capability
|
||||
3. Implement enhanced GPU detection
|
||||
4. Add memory management support
|
||||
|
||||
#### Phase 2: Configuration Options (Week 2)
|
||||
1. Add GPU parameter support
|
||||
2. Implement memory context controls
|
||||
3. Add performance tuning options
|
||||
4. Add sampling controls
|
||||
|
||||
#### Phase 3: Advanced Features (Week 3)
|
||||
1. Add server configuration options
|
||||
2. Implement version-aware command generation
|
||||
3. Add process monitoring
|
||||
4. Enhance error handling and validation
|
||||
|
||||
## 3. Version Awareness Implementation
|
||||
|
||||
### Approach
|
||||
1. **Version Detection**: Implement `llama-server --version` detection
|
||||
2. **Feature Support Matrix**: Create support matrix for different versions
|
||||
3. **Validation**: Validate configuration against supported features
|
||||
4. **Error Handling**: Provide clear error messages for unsupported features
|
||||
|
||||
### Example Implementation
|
||||
```bash
|
||||
detect_llama_version() {
|
||||
local version
|
||||
version="$(llama-server --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)"
|
||||
echo "$version"
|
||||
}
|
||||
|
||||
validate_options() {
|
||||
local version="$1"
|
||||
local options="$2"
|
||||
# Check if options are supported in this version
|
||||
# Return error if unsupported
|
||||
}
|
||||
```
|
||||
|
||||
## 4. Testing Strategy
|
||||
|
||||
### `pos ai hf` Tests
|
||||
1. **Model Download Tests**
|
||||
- Basic download functionality
|
||||
- Specific file download
|
||||
- Include/exclude patterns
|
||||
- Revision handling
|
||||
|
||||
2. **Repository Tests**
|
||||
- Info command
|
||||
- Files command
|
||||
- Cache management
|
||||
|
||||
3. **Error Handling Tests**
|
||||
- Authentication failures
|
||||
- Nonexistent repositories
|
||||
- Network failures
|
||||
|
||||
### `pos ai server` Tests
|
||||
1. **Command Generation Tests**
|
||||
- Basic server start
|
||||
- GPU configuration
|
||||
- Memory settings
|
||||
|
||||
2. **Configuration Tests**
|
||||
- Version detection
|
||||
- Feature validation
|
||||
- Unsupported option handling
|
||||
|
||||
3. **Integration Tests**
|
||||
- Process start/stop
|
||||
- Health checking
|
||||
- Graceful shutdown
|
||||
|
||||
## 5. Documentation Updates
|
||||
|
||||
### Help Text Updates
|
||||
1. Update `pos ai hf --help`
|
||||
2. Update `pos ai server --help`
|
||||
3. Add examples for new features
|
||||
4. Include GPU/memory configuration examples
|
||||
|
||||
### Usage Examples
|
||||
1. **Basic Model Download**
|
||||
```bash
|
||||
pos ai hf download meta-llama/Llama-3.1-8B-Instruct
|
||||
```
|
||||
|
||||
2. **Specific GGUF File**
|
||||
```bash
|
||||
pos ai hf download meta-llama/Llama-3.1-8B-Instruct model-00001-of-00006.gguf
|
||||
```
|
||||
|
||||
3. **Server Configuration**
|
||||
```bash
|
||||
pos ai server start --model model.gguf --gpu-layers 35 --ctx-size 4096
|
||||
```
|
||||
|
||||
## 6. Backward Compatibility
|
||||
|
||||
### Maintained Features
|
||||
1. All existing commands must continue to work
|
||||
2. All existing flags must continue to work
|
||||
3. Default behavior unchanged
|
||||
4. Configuration files remain compatible
|
||||
|
||||
### New Features
|
||||
1. Additions are optional
|
||||
2. Existing workflows unchanged
|
||||
3. No breaking changes introduced
|
||||
|
||||
## 7. Risk Mitigation
|
||||
|
||||
### Technical Risks
|
||||
1. **Version Compatibility**: Different llama.cpp versions may have different options
|
||||
2. **Dependency Issues**: May require additional system packages
|
||||
3. **Integration Complexity**: Complex server process management
|
||||
|
||||
### Mitigation Strategies
|
||||
1. **Version Detection**: Detect and validate supported options
|
||||
2. **Graceful Degradation**: Fallback to basic functionality when features unavailable
|
||||
3. **Comprehensive Testing**: Test across different scenarios and configurations
|
||||
|
||||
## 8. Timeline
|
||||
|
||||
### Week 1: Core Implementation
|
||||
- Command structure enhancements
|
||||
- Basic GPU/memory support
|
||||
- Version detection
|
||||
|
||||
### Week 2: Feature Implementation
|
||||
- Advanced download capabilities
|
||||
- Server configuration options
|
||||
- Error handling improvements
|
||||
|
||||
### Week 3: Testing and Documentation
|
||||
- Comprehensive testing
|
||||
- Documentation updates
|
||||
- Final validation
|
||||
|
||||
## 9. Expected Benefits
|
||||
|
||||
1. **Enhanced Functionality**: Complete feature set matching underlying tools
|
||||
2. **Better User Experience**: More intuitive workflows and better error messages
|
||||
3. **Improved Reliability**: Better error handling and validation
|
||||
4. **Version Safety**: Proper version detection and compatibility
|
||||
5. **Performance**: Optimized server configuration options
|
||||
|
||||
## 10. Future Considerations
|
||||
|
||||
1. **Integration with POS Ecosystem**: Seamless integration with other pos tools
|
||||
2. **Extensibility**: Easy to add new features
|
||||
3. **Scalability**: Support for larger deployments
|
||||
4. **Cross-platform**: Consistent behavior across different systems
|
||||
Reference in New Issue
Block a user