!Harbor project logo (https://github.com/av/harbor/raw/main/docs/harbor-2.png)

GitHub Tag (https://img.shields.io/github/v/tag/av/harbor) NPM Version (https://img.shields.io/npm/v/%40avcodes%2Fharbor?labelColor=red&color=white) PyPI - Version (https://img.shields.io/pypi/v/llm-harbor?labelColor=blue) !GitHub repo size (https://img.shields.io/github/repo-size/av/harbor) !GitHub repo file or directory count (https://img.shields.io/github/directory-file-count/av/harbor?type=file&extension=yml&label=compose%20files&color=orange) Visitors (https://api.visitorbadge.io/api/visitors?path=av%2Fharbor&countColor=%23263759&style=flat) !GitHub language count (https://img.shields.io/github/languages/count/av/harbor) Discord (https://img.shields.io/badge/Discord-Harbor-blue?logo=discord&logoColor=white) !Harbor Ko-fi (https://img.shields.io/badge/Ko--fi-white?style=social&logo=kofi)

Setup your local LLM stack effortlessly.

# Starts fully configured Open WebUI and Ollama
harbor up

# Now, Open WebUI can do Web RAG and TTS/STT
harbor up searxng speaches

Harbor is a containerized LLM toolkit that allows you to run LLM backends, frontends and related useful services. It consists of a CLI and a companion App.

!Screenshot of Harbor CLI and App together (https://github.com/av/harbor/wiki/harbor-app-3.png)

Documentation

• Installing Harbor (https://github.com/av/harbor/wiki/1.0.-Installing-Harbor) Guides to install Harbor CLI and App
• Harbor User Guide (https://github.com/av/harbor/wiki/1.-Harbor-User-Guide) High-level overview of working with Harbor
• Harbor App (https://github.com/av/harbor/wiki/1.1-Harbor-App) Overview and manual for the Harbor companion application
• Harbor Services (https://github.com/av/harbor/wiki/2.-Services) Catalog of services available in Harbor
• Harbor CLI Reference (https://github.com/av/harbor/wiki/3.-Harbor-CLI-Reference) Read more about Harbor CLI commands and options. Read about supported services and the ways to configure them.
• Join our Discord (https://discord.gg/8nDRphrhSF) Get help, share your experience, and contribute to the project.

What can Harbor do?

!Diagram outlining Harbor's service structure (https://raw.githubusercontent.com/wiki/av/harbor/harbor-arch-diag.png)

✦ Local LLMs

Run LLMs and related services locally, with no or minimal configuration, typically in a single command or click.

# All backends are pre-connected to Open WebUI
harbor up ollama
harbor up llamacpp
harbor up vllm

# Set and remember args for llama.cpp
harbor llamacpp args -ngl 32

Cutting Edge Inference

Harbor supports most of the major inference engines as well as a few of the lesser-known ones.

# We sincerely hope you'll never try to run all of them at once
harbor up vllm llamacpp tgi litellm tabbyapi aphrodite sglang ktransformers mistralrs airllm

Tool Use

Enjoy the benefits of MCP ecosystem, extend it to your use-cases.

# Manage MCPs with a convenient Web UI
harbor up metamcp

# Connect MCPs to Open WebUI
harbor up metamcp mcpo

Generate Images

Harbor includes ComfyUI + Flux + Open WebUI integration.

# Use FLUX in Open WebUI in one command
harbor up comfyui

Local Web RAG / Deep Research

Harbor includes SearXNG that is pre-connected to a lot of services out of the box: Perplexica, ChatUI, Morphic, Local Deep Research and more.

# SearXNG is pre-connected to Open WebUI
harbor up searxng

# And to many other services
harbor up searxng chatui
harbor up searxng morphic
harbor up searxng perplexica
harbor up searxng ldr