av/harbor

Harbor is a CLI tool that orchestrates a complete local LLM stack with a single command.

The tool solves the problem of complex manual configuration when setting up local AI infrastructure. Rather than individually installing and wiring together multiple services, Harbor uses Docker Compose to manage the entire stack. Users specify which services they want—LLM backends like Ollama, llama.cpp, or vLLM; frontends like Open WebUI; and supporting services such as SearXNG for web search, Speaches for voice chat, and ComfyUI for image generation—and Harbor handles deployment, configuration, and cross-service connectivity automatically. The companion app provides a graphical interface alongside the CLI for managing the running stack.

Harbor suits developers and researchers who want to experiment with local LLM deployments without infrastructure expertise. It eliminates the friction of manual Docker Compose configuration and service integration, making it practical to spin up complex multi-service stacks for local development, homelab setups, or self-hosted AI exploration. The tool is particularly valuable for those wanting to keep their AI work local and private while avoiding the tedium of hand-configuring dozens of environment variables and service dependencies.

The project shows active maintenance with frequent updates addressing both new backend support and stability improvements. Recent work includes adding new backends like PrismML for ternary quantized models and Halogen Flash Server for AMD hardware, alongside fixes for specific failure modes such as Hugging Face download permissions and long-running generation timeouts. The changelog indicates attention to user-facing reliability, with updates covering cache recovery, atomic configuration repair, and per-service environment defaults. Development includes expanding the service catalog and improving the developer experience through features like scoped service control and code-driven model interaction through the Boost codemode module.