Jesse Gross
556245843a
A draft cache pairs each slot with the token that follows it, so the deepest stored pair always names one token past what a prefix match can verify - at generation end, the sampled-but-never-committed final token. Restoring at the match point reuses that pair blind: a stop token stripped from the next prompt, or any divergence at the boundary, leaves it stale, and pairing never rewrites below the resume position, quietly lowering draft acceptance. Key the trie by token pairs instead: the key for offset i packs (token i, token i+1), so matching k keys verifies k+1 tokens and every match is a valid restore point. A pair is reused only if the token it names matched, and prefill re-evaluates the boundary token, rebuilding its pair with the token that actually follows. A token gets a key only once its successor is recorded, so endings record the final sampled token - never forwarded - and the trie stays level with the caches. Without a look-ahead the keys are the tokens and behavior is unchanged. The recorded tokens' slice bounds used to reject state past them for free; close now checks the invariant against the stored keys directly. The test harness rests requests the way the pipeline does - the deepest recorded token never enters the caches.
Ollama
Start building with open models.
Download
macOS
curl -fsSL https://ollama.com/install.sh | sh
Windows
irm https://ollama.com/install.ps1 | iex
Linux
curl -fsSL https://ollama.com/install.sh | sh
Docker
The official Ollama Docker image ollama/ollama is available on Docker Hub.
Libraries
Community
Get started
ollama
You'll be prompted to run a model or connect Ollama to your existing agents or applications such as Claude Code, OpenClaw, OpenCode , Codex, Copilot, and more.
Coding
To launch a specific integration:
ollama launch claude
Supported integrations include Claude Code, Codex, Copilot CLI, Droid, and OpenCode.
AI assistant
Use OpenClaw to turn Ollama into a personal AI assistant across WhatsApp, Telegram, Slack, Discord, and more:
ollama launch openclaw
Chat with a model
Run and chat with Gemma 4:
ollama run gemma4
See ollama.com/library for the full list.
See the quickstart guide for more details.
REST API
Ollama has a REST API for running and managing models.
curl http://localhost:11434/api/chat -d '{
"model": "gemma4",
"messages": [{
"role": "user",
"content": "Why is the sky blue?"
}],
"stream": false
}'
See the API documentation for all endpoints.
Python
pip install ollama
from ollama import chat
response = chat(model='gemma4', messages=[
{
'role': 'user',
'content': 'Why is the sky blue?',
},
])
print(response.message.content)
JavaScript
npm i ollama
import ollama from "ollama";
const response = await ollama.chat({
model: "gemma4",
messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);
Supported backends
- llama.cpp project founded by Georgi Gerganov.
Documentation
Community Integrations
Want to add your project? Open a pull request.
Chat Interfaces
Web
- Open WebUI - Extensible, self-hosted AI interface
- Onyx - Connected AI workspace
- LibreChat - Enhanced ChatGPT clone with multi-provider support
- Lobe Chat - Modern chat framework with plugin ecosystem (docs)
- NextChat - Cross-platform ChatGPT UI (docs)
- Perplexica - AI-powered search engine, open-source Perplexity alternative
- big-AGI - AI suite for professionals
- Lollms WebUI - Multi-model web interface
- ChatOllama - Chatbot with knowledge bases
- Bionic GPT - On-premise AI platform
- Chatbot UI - ChatGPT-style web interface
- Hollama - Minimal web interface
- Chatbox - Desktop and web AI client
- chat - Chat web app for teams
- Ollama RAG Chatbot - Chat with multiple PDFs using RAG
- Tkinter-based client - Python desktop client
Desktop
- Dify.AI - LLM app development platform
- AnythingLLM - All-in-one AI app for Mac, Windows, and Linux
- Maid - Cross-platform mobile and desktop client
- Witsy - AI desktop app for Mac, Windows, and Linux
- Cherry Studio - Multi-provider desktop client
- Ollama App - Multi-platform client for desktop and mobile
- PyGPT - AI desktop assistant for Linux, Windows, and Mac
- Alpaca - GTK4 client for Linux and macOS
- SwiftChat - Cross-platform including iOS, Android, and Apple Vision Pro
- Enchanted - Native macOS and iOS client
- RWKV-Runner - Multi-model desktop runner
- Ollama Grid Search - Evaluate and compare models
- macai - macOS client for Ollama and ChatGPT
- AI Studio - Multi-provider desktop IDE
- Reins - Parameter tuning and reasoning model support
- ConfiChat - Privacy-focused with optional encryption
- LLocal.in - Electron desktop client
- MindMac - AI chat client for Mac
- Msty - Multi-model desktop client
- BoltAI for Mac - AI chat client for Mac
- IntelliBar - AI-powered assistant for macOS
- Kerlig AI - AI writing assistant for macOS
- Hillnote - Markdown-first AI workspace
- Perfect Memory AI - Productivity AI personalized by screen and meeting history
Mobile
- Ollama Android Chat - One-click Ollama on Android
SwiftChat, Enchanted, Maid, Ollama App, Reins, and ConfiChat listed above also support mobile platforms.
Code Editors & Development
- Cline - VS Code extension for multi-file/whole-repo coding
- Continue - Open-source AI code assistant for any IDE
- Void - Open source AI code editor, Cursor alternative
- Copilot for Obsidian - AI assistant for Obsidian
- twinny - Copilot and Copilot chat alternative
- gptel Emacs client - LLM client for Emacs
- Ollama Copilot - Use Ollama as GitHub Copilot
- Obsidian Local GPT - Local AI for Obsidian
- Ellama Emacs client - LLM tool for Emacs
- orbiton - Config-free text editor with Ollama tab completion
- AI ST Completion - Sublime Text 4 AI assistant
- VT Code - Rust-based terminal coding agent with Tree-sitter
- QodeAssist - AI coding assistant for Qt Creator
- AI Toolkit for VS Code - Microsoft-official VS Code extension
- Open Interpreter - Natural language interface for computers
Libraries & SDKs
- LiteLLM - Unified API for 100+ LLM providers
- Semantic Kernel - Microsoft AI orchestration SDK
- LangChain4j - Java LangChain (example)
- LangChainGo - Go LangChain (example)
- Spring AI - Spring framework AI support (docs)
- LangChain and LangChain.js with example
- Ollama for Ruby - Ruby LLM library
- any-llm - Unified LLM interface by Mozilla
- OllamaSharp for .NET - .NET SDK
- LangChainRust - Rust LangChain (example)
- Agents-Flex for Java - Java agent framework (example)
- Elixir LangChain - Elixir LangChain
- Ollama-rs for Rust - Rust SDK
- LangChain for .NET - .NET LangChain (example)
- chromem-go - Go vector database with Ollama embeddings (example)
- LangChainDart - Dart LangChain
- LlmTornado - Unified C# interface for multiple inference APIs
- Ollama4j for Java - Java SDK
- Ollama for Laravel - Laravel integration
- Ollama for Swift - Swift SDK
- LlamaIndex and LlamaIndexTS - Data framework for LLM apps
- Haystack - AI pipeline framework
- Firebase Genkit - Google AI framework
- Ollama-hpp for C++ - C++ SDK
- PromptingTools.jl - Julia LLM toolkit (example)
- Ollama for R - rollama - R SDK
- Portkey - AI gateway
- Testcontainers - Container-based testing
- LLPhant - PHP AI framework
Frameworks & Agents
- AutoGPT - Autonomous AI agent platform
- crewAI - Multi-agent orchestration framework
- Strands Agents - Model-driven agent building by AWS
- Cheshire Cat - AI assistant framework
- any-agent - Unified agent framework interface by Mozilla
- Stakpak - Open source DevOps agent
- Hexabot - Conversational AI builder
- Neuro SAN - Multi-agent orchestration (docs)
RAG & Knowledge Bases
- RAGFlow - RAG engine based on deep document understanding
- R2R - Open-source RAG engine
- MaxKB - Ready-to-use RAG chatbot
- Minima - On-premises or fully local RAG
- Chipper - AI interface with Haystack RAG
- ARGO - RAG and deep research on Mac/Windows/Linux
- Archyve - RAG-enabling document library
- Casibase - AI knowledge base with RAG and SSO
- BrainSoup - Native client with RAG and multi-agent automation
Bots & Messaging
- LangBot - Multi-platform messaging bots with agents and RAG
- AstrBot - Multi-platform chatbot with RAG and plugins
- Discord-Ollama Chat Bot - TypeScript Discord bot
- Ollama Telegram Bot - Telegram bot
- LLM Telegram Bot - Telegram bot for roleplay
Terminal & CLI
- aichat - All-in-one LLM CLI with Shell Assistant, RAG, and AI tools
- oterm - Terminal client for Ollama
- gollama - Go-based model manager for Ollama
- tlm - Local shell copilot
- tenere - TUI for LLMs
- ParLlama - TUI for Ollama
- llm-ollama - Plugin for Datasette's LLM CLI
- ShellOracle - Shell command suggestions
- LLM-X - Progressive web app for LLMs
- cmdh - Natural language to shell commands
- VT - Minimal multimodal AI chat app
Productivity & Apps
- AppFlowy - AI collaborative workspace, self-hostable Notion alternative
- Screenpipe - 24/7 screen and mic recording with AI-powered search
- Vibe - Transcribe and analyze meetings
- Page Assist - Chrome extension for AI-powered browsing
- NativeMind - Private, on-device browser AI assistant
- Ollama Fortress - Security proxy for Ollama
- 1Panel - Web-based Linux server management
- Writeopia - Text editor with Ollama integration
- QA-Pilot - GitHub code repository understanding
- Raycast extension - Ollama in Raycast
- Painting Droid - Painting app with AI integrations
- Serene Pub - AI roleplaying app
- Mayan EDMS - Document management with Ollama workflows
- TagSpaces - File management with AI tagging
Observability & Monitoring
- Opik - Debug, evaluate, and monitor LLM applications
- OpenLIT - OpenTelemetry-native monitoring for Ollama and GPUs
- Lunary - LLM observability with analytics and PII masking
- Langfuse - Open source LLM observability
- HoneyHive - AI observability and evaluation for agents
- MLflow Tracing - Open source LLM observability
Database & Embeddings
- pgai - PostgreSQL as a vector database (guide)
- MindsDB - Connect Ollama with 200+ data platforms
- chromem-go - Embeddable vector database for Go (example)
- Kangaroo - AI-powered SQL client
Infrastructure & Deployment
Cloud
- Google Cloud
- Fly.io
- Koyeb
- Harbor - Containerized LLM toolkit with Ollama as default backend