butter
Melt away latency with ultra-smooth local inference
原文来源AI 开源 Releases
查看官方原文
Melt away latency with ultra-smooth local inference
RELATED SIGNALS
同厂商与同频道动态
cachellm
Cut token costs by caching LLM responses locally for OpenAI-compatible providers, eliminating duplicate API calls.
llm-eval-harness
A tiny, dependency-free Python harness for scoring LLM answers against a gold set (exact / contains / numeric matching, CI-friendly).
ha-ai-pool
Home Assistant integration: pool AI providers for ai_task, conversation, TTS and STT with quota rotation and error-aware failover
ai-agents.nvim
One picker for every AI coding agent in Neovim - Claude Code, Codex, Copilot, opencode, Gemini and API models - with a single accept/reject flow for the changes they propose
pi-omniroute
OmniRoute provider extension for pi — self-hosted gateway routing 350+ AI providers (Claude, GPT, Gemini, DeepSeek, …) with quota-aware auto-fallback and token compression, exposed as an OpenAI-compatible provider.
tokcodec
Token codec for AI coding tools: same signal, 30–95% fewer tokens. Works with Claude Code via npx tokcodec install.