feat: pos ai server — llama.cpp local inference server

Service manager (start/stop/status/models/logs) with systemd user service generation, GPU auto-detection, model selection from pos ai hf downloads. Provider adapter integrates with pos ai ask as --provider llamacpp. Config extends existing ai scope with LLAMACPP_* keys. 87 test cases / 0 failed. make gen/check/lint 0 FAIL / 0 WARN.
2026-09-04 12:40:50 -04:00
parent 99c033c6c6
commit 5e312b3207
7 changed files with 554 additions and 19 deletions
@@ -16,3 +16,11 @@
 #
 # System prompt:
 #   AI_SYSTEM_PROMPT=<prompt>     # Custom system prompt (overrides built-in; empty to reset)
+#
+# llama.cpp local inference server (pos ai server):
+#   LLAMACPP_PORT=8088            # Server port (default 8088)
+#   LLAMACPP_HOST=127.0.0.1       # Bind address (default 127.0.0.1)
+#   LLAMACPP_MODEL=<path>         # Default model path (GGUF file)
+#   LLAMACPP_CTX_SIZE=4096        # Context window size (default 4096)
+#   LLAMACPP_GPU_LAYERS=-1        # GPU layers: -1=auto, 0=CPU only (default -1)
+#   LLAMACPP_THREADS=<n>          # CPU threads (default: nproc)