A list of 139+ LLM services, tools, and infrastructure for running AI locally. Criteria for inclusion: - Open Source - Self-hostable - Friendly to containerization (Docker, Podman, etc.) - Relates to homelab or personal AI use cases - Well-documented with setup instructions
Relevance score (
0–100%): a composite metric of Popularity (logarithm of stars) and Recency (exponential decay with a 90-day half-life). This highlights projects that are both widely recognized and actively maintained.
Contents
Frontends - Chat interfaces and web UIs (19)
Backends - Inference engines and model servers (24)
Satellites - Companion services and integrations (91)
Workflow & Automation - Visual programming platforms (13)
API & Proxies - LLM gateways and aggregators (25)
Audio & Speech - TTS and STT services (7)
CLI Tools - Terminal-based LLM tools (23)
Evaluation - Benchmarking and testing (2)
MCP Tools - Model Context Protocol (20)
Frontends
Chat interfaces and web applications for interacting with language models.
AnythingLLM
84% 
64.5k 
issues 3.7k (296 open, 3.4k closed) 
2026-08-06
The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
AstrBot
82% 38.8k issues 6.4k (1.0k open, 5.4k closed) 2026-08-08
All-in-one agentic chatbot platform for multi-LLM conversations across messaging platforms with plugin system.
BionicGPT
58% 
2.3k issues 406 (0 open, 406 closed) 2026-08-03
on-premise LLM web UI with support for OpenAI-compatible backends
Chat Nio
22% 9.3k issues 307 (24 open, 283 closed) 2026-03-12
Comprehensive LLM web interface with built-in marketplace
ComfyUI
91% 125.0k issues 9.0k (4.0k open, 5.0k closed) 2026-08-09
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
Hollama
5% 1.2k issues 206 (67 open, 139 closed) 2025-10-12
A minimal web-UI for talking to Ollama servers.
HuggingFace ChatUI
71% 10.9k issues 740 (182 open, 558 closed) 2026-08-06
A chat interface using open source models, eg OpenAssistant or Llama. It is a SvelteKit app and it powers the HuggingChat app on hf.co/chat.
KoboldCpp
69% 11.4k issues 1.5k (493 open, 1.0k closed) 2026-08-02
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.
LibreChat
81% 41.8k issues 4.8k (350 open, 4.5k closed) 2026-08-06
Open-source ChatGPT UI alternative supporting multiple AI providers (Anthropic, AWS, OpenAI, Azure, Groq, Mistral, Google) with features like model switching, message search, and multi-user support. Includes integration with DALL-E-3 and various APIs.
Lobe Chat
88% 81.4k issues 6.3k (254 open, 6.1k closed) 2026-08-09
An open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek / MiniMax), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system.
Mikupad
9% 
751 issues 67 (18 open, 49 closed) 2025-12-27
LLM Frontend in a single HMTL file
ol1
59% 3.2k issues 187 (57 open, 130 closed) 2026-08-02
A simple Gradio app implementing an o1-like chain of reasoning with Ollama.
Omnichain
1% 356 issues 11 (1 open, 10 closed) 2025-05-13
Visual programming for AI language models
Onyx
79% 31.5k issues 1.3k (99 open, 1.2k closed) 2026-08-07
Open Source AI Platform with Chat UI, RAG, MCP support, and 40+ document connectors.
Open WebUI
84% 148.3k issues 9.3k (304 open, 9.0k closed) 2026-07-27
widely adopted and feature rich web interface for interacting with LLMs. Supports OpenAI-compatible and Ollama backends, multi-users, multi-model chats, custom prompts, TTS, Web RAG, RAG, and much much more.
oterm
59% 2.4k issues 156 (4 open, 152 closed) 2026-08-06
The text-based terminal client for Ollama.
Parllama
38% 484 issues 40 (3 open, 37 closed) 2026-07-11
TUI for Ollama
SillyTavern
63% 31.8k issues 2.6k (417 open, 2.2k closed) 2026-07-07
Feature-rich LLM chat frontend for power users. Supports multiple AI backends, personas, advanced prompting, and extensions.
Voicebox
76% 49.8k issues 668 (478 open, 190 closed) 2026-07-27
Local-first AI voice studio for voice cloning, preset voices, speech generation, dictation, transcription, and agent speech.
Backends
Inference engines and model serving platforms. These power the actual LLM responses.
AirLLM
80% 30.2k issues 243 (98 open, 145 closed) 2026-08-08
70B inference with single 4GB GPU (very slow, though)
Aphrodite
56% 1.8k issues 250 (79 open, 171 closed) 2026-08-04
Large-scale LLM inference engine
Docker Model Runner
Docker-managed local model runner exposed to Harbor as an OpenAI-compatible backend.
faster-whisper-server
27% 3.6k issues 331 (107 open, 224 closed) 2026-04-18
Legacy version of Speaches, use that instead.
ik_llama.cpp
62% 3.0k issues 589 (54 open, 535 closed) 2026-08-08
llama.cpp fork with additional quantization formats and CPU/CUDA performance improvements.
KoboldCpp
69% 11.4k issues 1.5k (493 open, 1.0k closed) 2026-08-02
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.
KTransformers
76% 19.2k issues 1.3k (463 open, 833 closed) 2026-08-08
A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
Lemonade
66% 5.3k issues 1.3k (356 open, 902 closed) 2026-08-08
Local AI server with LLM chat, speech-to-text, text-to-speech, and image generation. Optimized for AMD hardware.
llama.cpp
90% 123.2k issues 9.3k (687 open, 8.7k closed) 2026-08-08
LLM inference in C/C++
mistral.rs
64% 7.6k issues 709 (234 open, 475 closed) 2026-07-29
Blazingly fast LLM inference.
MLX
66% 6.5k issues 546 (227 open, 319 closed) 2026-08-05
Host-native Apple Silicon MLX inference managed by Harbor through an OpenAI-compatible proxy.
Modular MAX
79% 26.7k issues 3.6k (799 open, 2.8k closed) 2026-08-08
MAX is a platform from Modular (creators of Mojo) for running LLMs.
Needle
55% 3.4k issues 16 (13 open, 3 closed) 2026-07-22
Small function-calling model served through an OpenAI-compatible tool-calling API.
Nexa SDK
69% 8.3k issues 339 (22 open, 317 closed) 2026-08-07
Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models.
Ollama
94% 178.1k issues 10.9k (2.4k open, 8.5k closed) 2026-08-09
Get up and running with Llama 3.2, Mistral, Gemma 3, and other large language models.
oMLX
76% 18.5k issues 1.5k (667 open, 876 closed) 2026-08-08
Host-native Apple Silicon oMLX inference with continuous batching, SSD KV cache, and OpenAI-compatible Harbor integrations.
openedai-speech
1% 859 issues 85 (7 open, 78 closed) 2025-02-02
An OpenAI API compatible text to speech server using Coqui AI's xtts_v2 and/or piper tts as the backend.
Parler
1% 5.6k issues 157 (118 open, 39 closed) 2024-12-10
Inference and training library for high-quality TTS models.
SGLang
81% 31.6k issues 6.9k (768 open, 6.1k closed) 2026-08-09
SGLang is a fast serving framework for large language models and vision language models.
Speaches
27% 3.6k issues 331 (107 open, 224 closed) 2026-04-18
an OpenAI API-compatible speech server (formerly faster-whisper-server), both TTS and STT
Speech To Speech
74% 12.1k issues 163 (93 open, 70 closed) 2026-08-10
Modular realtime voice pipeline (VAD, STT, LLM, TTS) exposing an OpenAI Realtime-compatible WebSocket API.
TabbyAPI
55% 1.3k issues 237 (12 open, 225 closed) 2026-08-08
An OAI compatible exllamav2 API that's both lightweight and fast
Text Generation Inference
24% 10.9k issues 1.6k (285 open, 1.3k closed) 2026-03-21
Inference engine from HuggingFace.
vLLM
89% 88.6k issues 17.1k (2.1k open, 15.0k closed) 2026-08-09
A high-throughput and memory-efficient inference and serving engine for LLMs
Satellites
Companion services, research tools, and integrations that enhance LLM workflows.
Activepieces
78% 23.6k issues 3.0k (370 open, 2.6k closed) 2026-08-08
Open-source workflow automation platform with AI capabilities and 200+ app connectors.
Agent Zero
72% 18.8k issues 821 (94 open, 727 closed) 2026-08-01
General-purpose personal assistant with Web RAG, persistent memory, tools, browser use and more.
aichat
20% 10.3k issues 660 (47 open, 613 closed) 2026-02-23
All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents.
Aider
46% 48.1k issues 4.3k (1.3k open, 3.0k closed) 2026-05-22
Aider is AI pair programming in your terminal.
Airweave
41% 6.5k issues 114 (41 open, 73 closed) 2026-06-05
Airweave lets agents search any app by transforming its contents into agent-ready knowledge.
AstrBot
82% 38.8k issues 6.4k (1.0k open, 5.4k closed) 2026-08-08
All-in-one agentic chatbot platform for multi-LLM conversations across messaging platforms with plugin system.
autogpt
91% 186.4k issues 3.9k (300 open, 3.6k closed) 2026-08-05
Create, deploy, and manage continuous AI agents that automate complex workflows.
Beszel
77% 24.1k issues 1.3k (272 open, 1.0k closed) 2026-08-06
Lightweight server monitoring with historical data, docker stats, and alerts.
Bifrost
68% 7.2k issues 1.1k (352 open, 732 closed) 2026-08-08
Fast OpenAI-compatible AI gateway with provider routing, failover, observability, and Harbor backend bootstrapping.
Bolt.new
19% 19.7k issues 961 (78 open, 883 closed) 2026-02-07
Prompt, run, edit, and deploy full-stack web applications.
Browser Use
39% 16.3k issues 448 (272 open, 176 closed) 2026-05-15
AI-powered browser automation with web UI
cloudflared
74% 15.1k issues 1.3k (488 open, 850 closed) 2026-08-07
A helper service allowing to expose Harbor services over the internet.
cmdh
11% 133 issues 16 (9 open, 7 closed) 2026-02-25
Create Linux commands from natural language, in the shell.
Cognee
80% 29.9k issues 891 (148 open, 743 closed) 2026-08-08
Knowledge engine for AI agent memory. Transforms documents into searchable graphs and vectors with built-in pipelines.
Daytona
61% 72.0k issues 1.2k (312 open, 932 closed) 2026-06-25
Self-hosted sandbox platform for AI agents with Docker-in-Docker, computer use, and GPU support.
DBHub
63% 3.3k issues 165 (1 open, 164 closed) 2026-08-08
Zero-dependency, token-efficient database MCP server for Postgres, MySQL, SQL Server, MariaDB, and SQLite.
DeerFlow
87% 79.6k issues 1.8k (596 open, 1.2k closed) 2026-08-08
Community-driven deep research framework combining LLMs with web search, crawling, and multi-agent workflows for comprehensive research reports.
Dify
93% 151.8k issues 18.6k (270 open, 18.3k closed) 2026-08-09
An open-source LLM app development platform.
Docling
85% 64.4k issues 1.9k (889 open, 1.1k closed) 2026-08-08
Transform documents into format ready for LLMs.
Drawio
81% 34.4k issues 326 (137 open, 189 closed) 2026-08-08
AI-powered diagram creation tool - generate draw.io diagrams from natural language.
Fabric
80% 43.4k issues 858 (29 open, 829 closed) 2026-08-04
LLM-driven processing of the text data in the terminal.
facts
25% 198 issues 0 (0 open, 0 closed) 2026-06-03
Fact-driven development CLI for project specs that agents can verify and implement.
Flowise
84% 55.3k issues 2.7k (700 open, 2.0k closed) 2026-08-07
Drag & drop UI to build your customized LLM flow.
gptme
65% 4.4k issues 599 (4 open, 595 closed) 2026-08-08
A simple CLI tool to interact with LLMs.
Harbor Bench
59% 3.2k issues 187 (57 open, 130 closed) 2026-08-02
Harbor's own tool to evaluate LLMs and inference backends against custom tasks.
Harbor Boost
59% 3.2k issues 187 (57 open, 130 closed) 2026-08-02
Connects to downstream LLM API and serves a wrapper with custom workflow. For example, it can be used to add a CoT (Chain of Thought) to an existing LLM API, and much more. Scriptable with Python.
Hermes Agent
96% 227.6k issues 20.3k (9.9k open, 10.4k closed) 2026-08-09
Autonomous AI agent with persistent memory, skills, and multi-platform gateway.
Home Assistant
88% 89.8k issues 72.8k (2.7k open, 70.2k closed) 2026-08-08
Open source home automation platform for managing and controlling smart home devices.
JupyterLab
74% 15.2k issues 9.3k (2.4k open, 6.9k closed) 2026-08-08
Helper service to author/run Jupyter notebooks in Python with access to Harbor services.
K6
77% 31.2k issues 2.7k (708 open, 1.9k closed) 2026-08-03
A modern load testing tool, using Go and JavaScript - https://k6.io
Karakeep
75% 28.2k issues 1.7k (597 open, 1.1k closed) 2026-08-02
Self-hosted bookmark manager with AI-powered automatic tagging via OpenAI or Ollama.
Khoj
77% 36.4k issues 603 (96 open, 507 closed) 2026-08-02
AI second brain for chat, search, and agents with your docs. Supports local and cloud LLMs.
KoboldCpp
69% 11.4k issues 1.5k (493 open, 1.0k closed) 2026-08-02
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.
Kotaemon
46% 25.7k issues 472 (214 open, 258 closed) 2026-05-30
An open-source RAG-based tool for chatting with your documents.
LangFlow
92% 153.0k issues 3.4k (216 open, 3.2k closed) 2026-08-08
A low-code app builder for RAG and multi-agent AI applications.
LangFuse
80% 32.8k issues 2.8k (288 open, 2.5k closed) 2026-08-07
Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets.
Latent Scope
42% 786 issues 79 (17 open, 62 closed) 2026-07-12
A new kind of workflow + tool for visualizing and exploring datasets through the lens of latent spaces.
LibreTranslate
75% 15.9k issues 596 (124 open, 472 closed) 2026-08-08
A free and open-source machine translation.
LiteLLM
85% 55.9k issues 12.0k (1.6k open, 10.3k closed) 2026-08-09
LLM proxy that can aggregate multiple inference APIs together into a single endpoint.
LitLytics
0% 103 issues 16 (3 open, 13 closed) 2024-11-25
Simple analytics platform that leverages LLMs to automate data analysis.
llama-swap
66% 5.3k issues 451 (46 open, 405 closed) 2026-08-08
Runs multiple llama.cpp servers on demand for seamless switching between them.
lm-evaluation-harness
60% 13.6k issues 1.7k (583 open, 1.1k closed) 2026-07-13
A de-facto standard framework for the few-shot evaluation of language models.
Local Deep Research
70% 8.9k issues 623 (210 open, 413 closed) 2026-08-08
Transforms complex questions into comprehensive, cited reports.
LocalAI
83% 48.3k issues 1.6k (93 open, 1.5k closed) 2026-08-08
Complete AI stack for running AI models locally. Allows downloading variety of LLMs, TTS/STT/Image models and running them locally via Web UI.
MCP Forge
64% 4.3k issues 3.0k (960 open, 2.0k closed) 2026-08-07
Gateway and admin UI for managing Model Context Protocol (MCP) servers, tools, and resources.
mcpo
19% 4.3k issues 136 (38 open, 98 closed) 2026-02-27
Turn MCP servers into OpenAPI REST APIs - use them anywhere.
MetaMCP
42% 2.6k issues 198 (91 open, 107 closed) 2026-06-22
Allows to manage MCPs via a WebUI, exposes multiple MCPs as a single server.
mi
27% 101 issues 0 (0 open, 0 closed) 2026-07-02
Minimal agentic coding CLI that runs against OpenAI-compatible local or cloud model APIs.
MindsDB
65% 39.5k issues 4.4k (2 open, 4.4k closed) 2026-07-10
AI platform for integrating ML models with data sources via HTTP and MySQL APIs.
ML Intern
48% 10.7k issues 56 (32 open, 24 closed) 2026-06-18
Hugging Face ML engineering agent that researches papers, trains models, and ships ML code with local or cloud LLM backends.
Morphic
70% 9.0k issues 278 (55 open, 223 closed) 2026-08-08
An AI-powered search engine with a generative UI, similar to Perplexity and Perplexica.
n8n
95% 199.9k issues 9.9k (335 open, 9.6k closed) 2026-08-09
Fair-code workflow automation platform with native AI capabilities.
nanobot
83% 46.8k issues 1.4k (195 open, 1.2k closed) 2026-08-08
Ultra-lightweight personal AI assistant with multi-channel messaging and scheduled tasks.
Netdata
88% 80.1k issues 8.3k (185 open, 8.1k closed) 2026-08-09
Real-time infrastructure monitoring with per-second metrics for systems, containers, and applications.
npcsh
46% 460 issues 38 (8 open, 30 closed) 2026-08-03
Agentic shell and NPC team server with OpenAI-compatible endpoints, jinxes, and local backend integrations.
OmniParser
68% 25.2k issues 243 (173 open, 70 closed) 2026-07-20
A simple screen parsing tool towards pure vision based GUI agent.
Open Design
87% 84.6k issues 1.7k (453 open, 1.3k closed) 2026-08-07
Local-first design artifact workspace with BYOK model providers, design systems, decks, prototypes, and Harbor Ollama integration.
Open Interpreter
86% 67.9k issues 1.1k (1 open, 1.1k closed) 2026-08-08
A natural language interface for computers.
Open Notebook
77% 36.5k issues 569 (101 open, 468 closed) 2026-08-01
AI-powered research and note-taking platform with multi-provider LLM support, podcast generation, and content analysis.
Open Terminal
62% 3.0k issues 90 (23 open, 67 closed) 2026-08-09
Remote terminal and file-management API for AI agents, pre-wired for Open WebUI and Harbor backends.
Open WebUI Pipelines
4% 2.4k issues 280 (155 open, 125 closed) 2025-08-18
UI-Agnostic OpenAI API Plugin Framework.
OpenClaw
100% 385.6k issues 46.4k (3.4k open, 43.0k closed) 2026-08-09
Personal AI assistant with a self-hosted gateway, Control UI, and multi-channel messaging support.
OpenCode
94% 195.2k issues 23.1k (3.8k open, 19.3k closed) 2026-08-08
AI coding assistant with server API, TUI, and IDE extensions. Supports multiple LLM providers.
OpenFang
38% 18.1k issues 733 (73 open, 660 closed) 2026-05-12
Rust-based Agent Operating System for running autonomous AI agents with scheduling, 40 channel adapters, and 27 LLM providers.
OpenHands
88% 83.5k issues 4.4k (274 open, 4.2k closed) 2026-08-09
A platform for software development agents powered by AI.
OptiLLM
55% 4.2k issues 99 (17 open, 82 closed) 2026-07-18
Optimising LLM proxy that implements many advanced workflows to boost the performance of the LLMs.
Perplexica
32% 36.1k issues 672 (228 open, 444 closed) 2026-04-11
An AI-powered search engine (project evolved into Vane). Open source alternative to Perplexity AI.
PhotoPrism
82% 40.0k issues 2.6k (428 open, 2.2k closed) 2026-08-08
AI-powered photo management app with face recognition, image classification, and automatic organization.
Plandex
7% 15.6k issues 206 (39 open, 167 closed) 2025-10-03
AI driven development in your terminal.
Postiz
81% 34.4k issues 781 (158 open, 623 closed) 2026-08-08
AI-powered social media scheduling tool for managing posts, analytics, and team collaboration.
Presenton
70% 9.4k issues 287 (28 open, 259 closed) 2026-08-07
Open-source AI presentation generator with custom layouts, multi-model support, and PDF/PPTX export.
Promptfoo
77% 24.1k issues 1.3k (109 open, 1.1k closed) 2026-08-07
Test your prompts, agents, and RAGs. A developer-friendly local tool for testing LLM applications.
Qdrant
78% 33.9k issues 1.9k (484 open, 1.5k closed) 2026-08-04
Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine.
RAGLite
43% 1.2k issues 41 (12 open, 29 closed) 2026-07-09
Python toolkit for Retrieval-Augmented Generation (RAG)
Repopack
79% 27.7k issues 259 (102 open, 157 closed) 2026-08-08
A powerful tool that packs your entire repository into a single, AI-friendly file.
Resume Matcher
74% 28.1k issues 251 (25 open, 226 closed) 2026-07-30
AI-powered tool for comparing resumes against job descriptions using local LLMs via Ollama.
ROS MCP Server
38% 1.4k issues 99 (22 open, 77 closed) 2026-06-18
Model Context Protocol (MCP) server for Robot Operating System (ROS). Connects LLMs to robots.
SearXNG
80% 35.2k issues 1.8k (177 open, 1.7k closed) 2026-08-06
A privacy-respecting, hackable metasearch engine. Highly configurable and can be used for Web RAG use-cases.
Sim Studio
79% 29.4k issues 465 (116 open, 349 closed) 2026-08-07
Open-source platform to build and deploy AI agent workflows with visual canvas editor.
Solo CLI
18% 298 issues 13 (7 open, 6 closed) 2026-04-11
Fastest way to deploy Physical AI on your edge devices. Manages model servers and supports MCP tools.
SQL Chat
29% 5.8k issues 83 (22 open, 61 closed) 2026-04-21
Chat-based SQL client, which uses natural language to communicate with the database.
SuperGateway
6% 2.8k issues 85 (41 open, 44 closed) 2025-10-09
A simple and powerful API gateway for LLMs.
SurfSense
74% 15.8k issues 461 (117 open, 344 closed) 2026-08-07
Private NotebookLM-style research workspace with RAG, citations, browser capture, reports, and podcast generation.
TextGrad
3% 3.7k issues 123 (45 open, 78 closed) 2025-07-25
Automatic "Differentiation" via Text - using large language models to backpropagate textual gradients.
tokscale
66% 4.9k issues 306 (57 open, 249 closed) 2026-08-09
Monitor AI token usage and costs across coding assistants.
Traefik
83% 64.4k issues 6.8k (689 open, 6.1k closed) 2026-08-04
A modern HTTP reverse proxy and load balancer that makes deploying microservices easy.
txtai RAG
29% 459 issues 32 (2 open, 30 closed) 2026-06-05
RAG WebUI built with txtai.
Unsloth
87% 69.7k issues 3.9k (742 open, 3.2k closed) 2026-08-09
Jupyter Lab environment with Unsloth for fast LLM fine-tuning - 2x faster training with 70% less memory.
Unsloth Studio
87% 69.7k issues 3.9k (742 open, 3.2k closed) 2026-08-09
No-code web UI for fine-tuning LLMs with Unsloth. 2x faster training and 70% less memory, powered by llama.cpp + Hugging Face.
Webtop
63% 4.3k issues 162 (7 open, 155 closed) 2026-08-04
Linux in a web browser supporting popular desktop environments.
Windmill
75% 17.5k issues 1.6k (552 open, 1.0k closed) 2026-08-07
Open-source developer platform for internal tools, workflows, and UIs with multi-language script support.
Workflow & Automation
Visual programming, workflow automation, and orchestration platforms for building LLM applications.
Activepieces
78% 23.6k issues 3.0k (370 open, 2.6k closed) 2026-08-08
Open-source workflow automation platform with AI capabilities and 200+ app connectors.
ComfyUI
91% 125.0k issues 9.0k (4.0k open, 5.0k closed) 2026-08-09
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
DeerFlow
87% 79.6k issues 1.8k (596 open, 1.2k closed) 2026-08-08
Community-driven deep research framework combining LLMs with web search, crawling, and multi-agent workflows for comprehensive research reports.
Dify
93% 151.8k issues 18.6k (270 open, 18.3k closed) 2026-08-09
An open-source LLM app development platform.
Flowise
84% 55.3k issues 2.7k (700 open, 2.0k closed) 2026-08-07
Drag & drop UI to build your customized LLM flow.
LangFlow
92% 153.0k issues 3.4k (216 open, 3.2k closed) 2026-08-08
A low-code app builder for RAG and multi-agent AI applications.
LitLytics
0% 103 issues 16 (3 open, 13 closed) 2024-11-25
Simple analytics platform that leverages LLMs to automate data analysis.
n8n
95% 199.9k issues 9.9k (335 open, 9.6k closed) 2026-08-09
Fair-code workflow automation platform with native AI capabilities.
Onyx
79% 31.5k issues 1.3k (99 open, 1.2k closed) 2026-08-07
Open Source AI Platform with Chat UI, RAG, MCP support, and 40+ document connectors.
Open WebUI Pipelines
4% 2.4k issues 280 (155 open, 125 closed) 2025-08-18
UI-Agnostic OpenAI API Plugin Framework.
Sim Studio
79% 29.4k issues 465 (116 open, 349 closed) 2026-08-07
Open-source platform to build and deploy AI agent workflows with visual canvas editor.
SurfSense
74% 15.8k issues 461 (117 open, 344 closed) 2026-08-07
Private NotebookLM-style research workspace with RAG, citations, browser capture, reports, and podcast generation.
Windmill
75% 17.5k issues 1.6k (552 open, 1.0k closed) 2026-08-07
Open-source developer platform for internal tools, workflows, and UIs with multi-language script support.
API & Proxies
API gateways, proxies, and aggregation services for managing multiple LLM endpoints.
Bifrost
68% 7.2k issues 1.1k (352 open, 732 closed) 2026-08-08
Fast OpenAI-compatible AI gateway with provider routing, failover, observability, and Harbor backend bootstrapping.
cloudflared
74% 15.1k issues 1.3k (488 open, 850 closed) 2026-08-07
A helper service allowing to expose Harbor services over the internet.
Cognee
80% 29.9k issues 891 (148 open, 743 closed) 2026-08-08
Knowledge engine for AI agent memory. Transforms documents into searchable graphs and vectors with built-in pipelines.
Daytona
61% 72.0k issues 1.2k (312 open, 932 closed) 2026-06-25
Self-hosted sandbox platform for AI agents with Docker-in-Docker, computer use, and GPU support.
DBHub
63% 3.3k issues 165 (1 open, 164 closed) 2026-08-08
Zero-dependency, token-efficient database MCP server for Postgres, MySQL, SQL Server, MariaDB, and SQLite.
Docker Model Runner
Docker-managed local model runner exposed to Harbor as an OpenAI-compatible backend.
Harbor Boost
59% 3.2k issues 187 (57 open, 130 closed) 2026-08-02
Connects to downstream LLM API and serves a wrapper with custom workflow. For example, it can be used to add a CoT (Chain of Thought) to an existing LLM API, and much more. Scriptable with Python.
LangFuse
80% 32.8k issues 2.8k (288 open, 2.5k closed) 2026-08-07
Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets.
Lemonade
66% 5.3k issues 1.3k (356 open, 902 closed) 2026-08-08
Local AI server with LLM chat, speech-to-text, text-to-speech, and image generation. Optimized for AMD hardware.
LiteLLM
85% 55.9k issues 12.0k (1.6k open, 10.3k closed) 2026-08-09
LLM proxy that can aggregate multiple inference APIs together into a single endpoint.
llama-swap
66% 5.3k issues 451 (46 open, 405 closed) 2026-08-08
Runs multiple llama.cpp servers on demand for seamless switching between them.
MindsDB
65% 39.5k issues 4.4k (2 open, 4.4k closed) 2026-07-10
AI platform for integrating ML models with data sources via HTTP and MySQL APIs.
ML Intern
48% 10.7k issues 56 (32 open, 24 closed) 2026-06-18
Hugging Face ML engineering agent that researches papers, trains models, and ships ML code with local or cloud LLM backends.
MLX
66% 6.5k issues 546 (227 open, 319 closed) 2026-08-05
Host-native Apple Silicon MLX inference managed by Harbor through an OpenAI-compatible proxy.
Needle
55% 3.4k issues 16 (13 open, 3 closed) 2026-07-22
Small function-calling model served through an OpenAI-compatible tool-calling API.
npcsh
46% 460 issues 38 (8 open, 30 closed) 2026-08-03
Agentic shell and NPC team server with OpenAI-compatible endpoints, jinxes, and local backend integrations.
oMLX
76% 18.5k issues 1.5k (667 open, 876 closed) 2026-08-08
Host-native Apple Silicon oMLX inference with continuous batching, SSD KV cache, and OpenAI-compatible Harbor integrations.
Open Terminal
62% 3.0k issues 90 (23 open, 67 closed) 2026-08-09
Remote terminal and file-management API for AI agents, pre-wired for Open WebUI and Harbor backends.
Open WebUI Pipelines
4% 2.4k issues 280 (155 open, 125 closed) 2025-08-18
UI-Agnostic OpenAI API Plugin Framework.
OpenFang
38% 18.1k issues 733 (73 open, 660 closed) 2026-05-12
Rust-based Agent Operating System for running autonomous AI agents with scheduling, 40 channel adapters, and 27 LLM providers.
OptiLLM
55% 4.2k issues 99 (17 open, 82 closed) 2026-07-18
Optimising LLM proxy that implements many advanced workflows to boost the performance of the LLMs.
ROS MCP Server
38% 1.4k issues 99 (22 open, 77 closed) 2026-06-18
Model Context Protocol (MCP) server for Robot Operating System (ROS). Connects LLMs to robots.
Speech To Speech
74% 12.1k issues 163 (93 open, 70 closed) 2026-08-10
Modular realtime voice pipeline (VAD, STT, LLM, TTS) exposing an OpenAI Realtime-compatible WebSocket API.
Traefik
83% 64.4k issues 6.8k (689 open, 6.1k closed) 2026-08-04
A modern HTTP reverse proxy and load balancer that makes deploying microservices easy.
Voicebox
76% 49.8k issues 668 (478 open, 190 closed) 2026-07-27
Local-first AI voice studio for voice cloning, preset voices, speech generation, dictation, transcription, and agent speech.
Audio & Speech
Text-to-speech (TTS), speech-to-text (STT), and audio processing services.
faster-whisper-server
27% 3.6k issues 331 (107 open, 224 closed) 2026-04-18
Legacy version of Speaches, use that instead.
Lemonade
66% 5.3k issues 1.3k (356 open, 902 closed) 2026-08-08
Local AI server with LLM chat, speech-to-text, text-to-speech, and image generation. Optimized for AMD hardware.
openedai-speech
1% 859 issues 85 (7 open, 78 closed) 2025-02-02
An OpenAI API compatible text to speech server using Coqui AI's xtts_v2 and/or piper tts as the backend.
Parler
1% 5.6k issues 157 (118 open, 39 closed) 2024-12-10
Inference and training library for high-quality TTS models.
Speaches
27% 3.6k issues 331 (107 open, 224 closed) 2026-04-18
an OpenAI API-compatible speech server (formerly faster-whisper-server), both TTS and STT
Speech To Speech
74% 12.1k issues 163 (93 open, 70 closed) 2026-08-10
Modular realtime voice pipeline (VAD, STT, LLM, TTS) exposing an OpenAI Realtime-compatible WebSocket API.
Voicebox
76% 49.8k issues 668 (478 open, 190 closed) 2026-07-27
Local-first AI voice studio for voice cloning, preset voices, speech generation, dictation, transcription, and agent speech.
CLI Tools
Command-line interfaces and terminal-based tools for LLM interaction.
aichat
20% 10.3k issues 660 (47 open, 613 closed) 2026-02-23
All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents.
Aider
46% 48.1k issues 4.3k (1.3k open, 3.0k closed) 2026-05-22
Aider is AI pair programming in your terminal.
cloudflared
74% 15.1k issues 1.3k (488 open, 850 closed) 2026-08-07
A helper service allowing to expose Harbor services over the internet.
cmdh
11% 133 issues 16 (9 open, 7 closed) 2026-02-25
Create Linux commands from natural language, in the shell.
Fabric
80% 43.4k issues 858 (29 open, 829 closed) 2026-08-04
LLM-driven processing of the text data in the terminal.
facts
25% 198 issues 0 (0 open, 0 closed) 2026-06-03
Fact-driven development CLI for project specs that agents can verify and implement.
gptme
65% 4.4k issues 599 (4 open, 595 closed) 2026-08-08
A simple CLI tool to interact with LLMs.
Harbor Bench
59% 3.2k issues 187 (57 open, 130 closed) 2026-08-02
Harbor's own tool to evaluate LLMs and inference backends against custom tasks.
Hermes Agent
96% 227.6k issues 20.3k (9.9k open, 10.4k closed) 2026-08-09
Autonomous AI agent with persistent memory, skills, and multi-platform gateway.
K6
77% 31.2k issues 2.7k (708 open, 1.9k closed) 2026-08-03
A modern load testing tool, using Go and JavaScript - https://k6.io
lm-evaluation-harness
60% 13.6k issues 1.7k (583 open, 1.1k closed) 2026-07-13
A de-facto standard framework for the few-shot evaluation of language models.
mi
27% 101 issues 0 (0 open, 0 closed) 2026-07-02
Minimal agentic coding CLI that runs against OpenAI-compatible local or cloud model APIs.
nanobot
83% 46.8k issues 1.4k (195 open, 1.2k closed) 2026-08-08
Ultra-lightweight personal AI assistant with multi-channel messaging and scheduled tasks.
npcsh
46% 460 issues 38 (8 open, 30 closed) 2026-08-03
Agentic shell and NPC team server with OpenAI-compatible endpoints, jinxes, and local backend integrations.
Open Interpreter
86% 67.9k issues 1.1k (1 open, 1.1k closed) 2026-08-08
A natural language interface for computers.
OpenCode
94% 195.2k issues 23.1k (3.8k open, 19.3k closed) 2026-08-08
AI coding assistant with server API, TUI, and IDE extensions. Supports multiple LLM providers.
oterm
59% 2.4k issues 156 (4 open, 152 closed) 2026-08-06
The text-based terminal client for Ollama.
Plandex
7% 15.6k issues 206 (39 open, 167 closed) 2025-10-03
AI driven development in your terminal.
Promptfoo
77% 24.1k issues 1.3k (109 open, 1.1k closed) 2026-08-07
Test your prompts, agents, and RAGs. A developer-friendly local tool for testing LLM applications.
Repopack
79% 27.7k issues 259 (102 open, 157 closed) 2026-08-08
A powerful tool that packs your entire repository into a single, AI-friendly file.
Solo CLI
18% 298 issues 13 (7 open, 6 closed) 2026-04-11
Fastest way to deploy Physical AI on your edge devices. Manages model servers and supports MCP tools.
SuperGateway
6% 2.8k issues 85 (41 open, 44 closed) 2025-10-09
A simple and powerful API gateway for LLMs.
tokscale
66% 4.9k issues 306 (57 open, 249 closed) 2026-08-09
Monitor AI token usage and costs across coding assistants.
Evaluation
Benchmarking, evaluation, and testing tools for measuring LLM performance.
Harbor Bench
59% 3.2k issues 187 (57 open, 130 closed) 2026-08-02
Harbor's own tool to evaluate LLMs and inference backends against custom tasks.
lm-evaluation-harness
60% 13.6k issues 1.7k (583 open, 1.1k closed) 2026-07-13
A de-facto standard framework for the few-shot evaluation of language models.
MCP Tools
Model Context Protocol servers and tool integration services.
AstrBot
82% 38.8k issues 6.4k (1.0k open, 5.4k closed) 2026-08-08
All-in-one agentic chatbot platform for multi-LLM conversations across messaging platforms with plugin system.
DBHub
63% 3.3k issues 165 (1 open, 164 closed) 2026-08-08
Zero-dependency, token-efficient database MCP server for Postgres, MySQL, SQL Server, MariaDB, and SQLite.
facts
25% 198 issues 0 (0 open, 0 closed) 2026-06-03
Fact-driven development CLI for project specs that agents can verify and implement.
Hermes Agent
96% 227.6k issues 20.3k (9.9k open, 10.4k closed) 2026-08-09
Autonomous AI agent with persistent memory, skills, and multi-platform gateway.
MCP Forge
64% 4.3k issues 3.0k (960 open, 2.0k closed) 2026-08-07
Gateway and admin UI for managing Model Context Protocol (MCP) servers, tools, and resources.
mcpo
19% 4.3k issues 136 (38 open, 98 closed) 2026-02-27
Turn MCP servers into OpenAPI REST APIs - use them anywhere.
MetaMCP
42% 2.6k issues 198 (91 open, 107 closed) 2026-06-22
Allows to manage MCPs via a WebUI, exposes multiple MCPs as a single server.
mi
27% 101 issues 0 (0 open, 0 closed) 2026-07-02
Minimal agentic coding CLI that runs against OpenAI-compatible local or cloud model APIs.
ML Intern
48% 10.7k issues 56 (32 open, 24 closed) 2026-06-18
Hugging Face ML engineering agent that researches papers, trains models, and ships ML code with local or cloud LLM backends.
nanobot
83% 46.8k issues 1.4k (195 open, 1.2k closed) 2026-08-08
Ultra-lightweight personal AI assistant with multi-channel messaging and scheduled tasks.
Needle
55% 3.4k issues 16 (13 open, 3 closed) 2026-07-22
Small function-calling model served through an OpenAI-compatible tool-calling API.
npcsh
46% 460 issues 38 (8 open, 30 closed) 2026-08-03
Agentic shell and NPC team server with OpenAI-compatible endpoints, jinxes, and local backend integrations.
Open Design
87% 84.6k issues 1.7k (453 open, 1.3k closed) 2026-08-07
Local-first design artifact workspace with BYOK model providers, design systems, decks, prototypes, and Harbor Ollama integration.
Open Terminal
62% 3.0k issues 90 (23 open, 67 closed) 2026-08-09
Remote terminal and file-management API for AI agents, pre-wired for Open WebUI and Harbor backends.
OpenClaw
100% 385.6k issues 46.4k (3.4k open, 43.0k closed) 2026-08-09
Personal AI assistant with a self-hosted gateway, Control UI, and multi-channel messaging support.
OpenFang
38% 18.1k issues 733 (73 open, 660 closed) 2026-05-12
Rust-based Agent Operating System for running autonomous AI agents with scheduling, 40 channel adapters, and 27 LLM providers.
Postiz
81% 34.4k issues 781 (158 open, 623 closed) 2026-08-08
AI-powered social media scheduling tool for managing posts, analytics, and team collaboration.
ROS MCP Server
38% 1.4k issues 99 (22 open, 77 closed) 2026-06-18
Model Context Protocol (MCP) server for Robot Operating System (ROS). Connects LLMs to robots.
Solo CLI
18% 298 issues 13 (7 open, 6 closed) 2026-04-11
Fastest way to deploy Physical AI on your edge devices. Manages model servers and supports MCP tools.
SuperGateway
6% 2.8k issues 85 (41 open, 44 closed) 2025-10-09
A simple and powerful API gateway for LLMs.
Contributing
This list is auto-generated from Harbor's service metadata.
Made with 
by the Harbor community