Local AI Cat personal markLocal AI Cat business markLocal AI Catkeep your paws off my datafor very serious cat business
← All modules

Local AI Cat module

GGUF backend (llama.cpp sidecar)

Current shipping, testing, and product context for this part of Local AI Cat.

Status

Where it stands

Lane: StableStatus: BetaTesting: prov. +behavioral
Platforms
macOS
Release channels
Alpha · Beta · Main
Distribution
Outdoor
  • AlphamacOS
  • BetamacOS
  • Outdoor · Direct downloadmacOS

Description

What it does

macOS / Outdoor + Beta only. Sidecar-first (see docs/planning/done/gguf-backend-spike-notes.md): FeatureGGUF's LlamaServerSidecar actor locates a llama-server binary, spawns it on a free loopback port for the selected .gguf model, health-checks /health, detects crashes and restarts (bounded), and terminates gracefully. GGUFChatBackend proxies /v1/chat/completions SSE into LocalAITextGenerationEvents behind the SAME LocalAITextGenerating seam RemoteCatBackend uses, registered through ChatBackendRegistry with a `gguf:`-prefixed model contributor. GGUF-only LM Studio dirs are no longer rejected by ExternalModelSource — they carry format=.gguf and are served here instead of the MLX loader. Physically absent from App Store + iOS (package unlinked, FEATURE_GGUF undefined). Deferred: bundling llama-server (size + signing/notarization) and GGUF model-download UX.

Headless Local API

Drive it over the loopback API

API parity: PartialLogging: L0 · prov.

Logging / observability: provisional grade L0 — structured=0 print=0 sentry=0.

discover
not applicable
execute
present
observe
not applicable
configure
not applicable
cancel/stop
not applicable
reset
not applicable
localApi
  • /v1/local/app/action
  • /v1/local/app/artifacts/{id}
  • /v1/local/app/navigate
  • /v1/local/app/screenshot
  • /v1/local/app/state
  • /v1/local/app/surfaces
  • /v1/local/app/transactions/{id}
  • /v1/local/artifacts
  • /v1/local/artifacts/{artifact_id}
  • /v1/local/artifacts/{artifact_id}/content

Media

See it in action

Walkthrough coming soonUI-test video · GIF fallback
A short walkthrough will appear here when the module's UI test media is ready.

Backlog / improvements

What comes next

No improvements are listed for this module yet.

History

How it got here

No change history has been published for this module yet.