ORBIT โ Open Retrieval-Based Inference Toolkit
git clone https://github.com/schmitech/orbit.git && cd orbit/docker
docker compose up -dThen test it:
curl -X POST http://localhost:3000/v1/chat \
-H 'Content-Type: application/json' \
-H 'X-API-Key: default-key' \
-H 'X-Session-ID: local-test' \
-d '{
"messages": [{"role": "user", "content": "Summarize ORBIT in one sentence."}],
"stream": false
}'That's it. ORBIT is listening on port 3000 with an admin panel at localhost:3000/admin (default login: admin / admin123).
For GPU acceleration: docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d
Adapter wiring and sample domains live in config/adapters/ and examples/intent-templates/.
| Layer | Coverage |
|---|---|
| LLM & inference | 29 providers โ OpenAI, Anthropic, Gemini, Cohere, Groq, DeepSeek, Mistral, xAI, AWS Bedrock, Azure, Vertex, Together, Fireworks, Perplexity, Replicate, OpenRouter, Watson, NVIDIA, Hugging Face, Ollama (local/cloud/remote), vLLM, TensorRT-LLM, llama.cpp, Shimmy, BitNet (1.58-bit), Transformers, Z.ai |
| Data sources | 17 โ Postgres, MySQL, MariaDB, SQL Server, Oracle, SQLite, MongoDB, Redis, Cassandra, DuckDB, Athena, Elasticsearch, Supabase + HTTP/REST, GraphQL, Firecrawl |
| Vector stores | Chroma, Qdrant, Pinecone, Milvus, Weaviate, Elasticsearch |
| Embeddings | 10 providers โ OpenAI, Cohere, Jina, Voyage, Mistral, Gemini, OpenRouter, Ollama, llama.cpp, Sentence-Transformers |
| Rerankers | 6 providers โ Cohere, Jina, Voyage, OpenAI, Anthropic, Ollama |
| Moderation / guardrails | OpenAI, Anthropic, Llama Guard (local), pluggable chain |
| Voice | Full-duplex speech-to-speech via PersonaPlex; STT (Whisper, Google, Gemini), TTS (OpenAI, ElevenLabs, Coqui) |
| Protocols | OpenAI-compatible chat API + MCP โ drop-in tool server for OpenClaw, Claude Desktop, Cursor, or any MCP client |
Most AI gateways stop at provider routing. ORBIT is built for the messy parts of production RAG.
- Intent-based retrieval, not just vector search โ ship real queries, not vector guesses. Users ask in natural language; ORBIT picks the right template and runs the query against your data. Learn more.
- Cross-adapter RAG across mixed databases + APIs โ one question, many sources. Fan a query out to SQL, MongoDB, HTTP, and more in parallel and let the LLM merge the answers. Learn more.
- Template diagnostics โ iterate on intent templates without burning LLM tokens. Learn more.
- Conversation threading with cached datasets โ branch off any turn; follow-ups reuse the retrieved data instead of re-querying the DB. Learn more.
- Circuit breakers + parallel fan-out โ resilient adapter orchestration that survives provider hiccups. Learn more.
- Autocomplete that knows your data โ fuzzy-matched suggestions sourced from your intent templates. Learn more.
- Two-layer rate limiting โ IP limits plus per-API-key quotas with progressive throttling. Learn more.
- Multilingual by default โ 100+ languages with conversation stickiness so the model doesn't flap between turns. Learn more.
- OpenClaw / MCP integration โ drop ORBIT into any OpenClaw agent as a tool server with a single config entry. Learn more.
- Ask your database questions in any language โ connect Postgres, MySQL, MongoDB, DuckDB, Elasticsearch, or any of the other 12 sources and query them with natural language.
- Query across Postgres + MongoDB + a REST API in one prompt โ true multi-source RAG, no pipeline glue.
- Switch LLM providers without changing code โ swap between 29 providers with a single config line.
- Build full-duplex voice agents โ speech-to-speech with interruption handling via PersonaPlex.
- Plug ORBIT into OpenClaw in minutes โ one config entry turns ORBIT into a tool server for any OpenClaw agent. Walkthrough.
- Power agentic workflows โ MCP-compatible with Claude Desktop, Cursor, and custom agents.
- Upload files and get answers โ RAG over PDFs, images, and documents out of the box.
- Add guardrails and moderation โ chain OpenAI, Anthropic, or local Llama Guard moderators.
- Keep everything private โ self-host on your own infrastructure with RBAC, audit logs, and two-layer rate limiting.
ORBIT is used in production at PoliceStats.ca, a public-facing AI search and analytics site for Canadian municipal police open data.
PoliceStats uses ORBIT to:
- Route users across many dataset-specific adapters for cities like Toronto, Ottawa, Montreal, Edmonton, Hamilton, Winnipeg, Saskatoon, Vancouver, and Canada-wide statistics
- Query structured public-safety datasets using natural language
- Return grounded answers with source citations back to the relevant open data portal
- Support both broad city assistants and narrow subdomain assistants
- Power a production web chat experience with typed and voice interaction
PoliceStats is a useful reference if you want to see ORBIT applied to a real vertical product instead of only toy examples: one OpenAI-compatible API, many adapters, structured retrieval over public data, and answers designed for end users rather than internal analysts.
ORBIT is Apache 2.0 โ you can build and sell commercial products on top of it without royalties or per-seat licensing back to the project. Below are product patterns that map cleanly to ORBIT's primitives (intent adapters, composite routing, voice, MCP, guardrails). Each row links to a cookbook recipe with a working configuration you can fork as a starting point.
| Product pattern | What you're actually selling | Starting point |
|---|---|---|
| Vertical database copilot | A natural-language query layer over one industry's schemas โ healthcare claims, legal discovery, logistics, retail ops, public safety โ sold per-seat to analysts who don't write SQL | Database copilot ยท NL-BI |
| Private AI gateway for regulated data | On-prem or VPC-only deployment for banks, hospitals, insurers, and government โ RBAC, audit logs, guardrails, and data residency baked in | Private gateway |
| Full-duplex voice assistant | Voice-first copilots for call centers, field service, drive-throughs, and lobby kiosks โ priced per concurrent channel or per minute | Field service voice ยท PersonaPlex overview |
| Enterprise system chat connectors | Chat-native access to ServiceNow, Salesforce, Jira, SAP, or internal APIs โ sold per connector or as an integration bundle | ServiceNow connector ยท REST / GraphQL |
| Document Q&A for teams | Knowledge-base SaaS over PDFs, contracts, policies, and manuals โ semantic search with citations, team subscriptions | File-upload RAG ยท Vector stores |
| Managed ORBIT hosting | Run ORBIT-as-a-Service for customers who don't want to operate it โ SLA-backed, failover, metered usage, multi-tenant API keys | Production deployment ยท Resilient gateway ยท Rate limits & quotas |
| MCP tools for agent platforms | Package domain-specific ORBIT adapters as MCP tools and sell them to Claude, Cursor, OpenClaw, and custom-agent users | MCP / OpenClaw |
| Market-intel chat over the public web | Subscription products that answer questions about competitor pricing, regulatory filings, and news by chat-indexing live web content | Firecrawl web queries |
Already shipped in production: see PoliceStats.ca above for a vertical public-data product built on ORBIT. If you build something on top, open an issue and we'll feature it here.
| Without ORBIT | With ORBIT |
|---|---|
| One SDK per provider, rewrites when you switch | One OpenAI-compatible API across 29 providers |
| Separate pipelines for retrieval and inference | Unified model + retrieval + tooling gateway |
| Fragile glue scripts between data sources and LLMs | 9 intent-adapter archetypes with template diagnostics |
| Separate tools for each database โ no way to combine them | Composite adapters fan one prompt across SQL + NoSQL + HTTP, merged by the LLM |
| Cascading failures when a provider hiccups | Circuit breakers, parallel fan-out, progressive throttling |
| No visibility into what models are doing | Built-in RBAC, quota-aware rate limiting, and audit logging |
| Client | Description |
|---|---|
| Web Chat | React UI |
| CLI | pip install schmitech-orbit-client |
| Mobile | iOS & Android (Expo) |
| Node SDK | Or use any OpenAI-compatible SDK |
Docker Compose (fastest path)
git clone https://github.com/schmitech/orbit.git && cd orbit/docker
docker compose up -dStarts ORBIT + Ollama with SmolLM2, auto-pulls models, and exposes the API on port 3000. The web admin UI is at /admin on the same host. Connect orbitchat from your host:
ORBIT_ADAPTER_KEYS='{"simple-chat":"default-key"}' npx orbitchatSee the full Docker Guide for GPU mode, volumes, and configuration.
Pre-built image (server only)
docker pull schmitech/orbit:basic
docker run -d --name orbit-basic -p 3000:3000 schmitech/orbit:basicIf Ollama runs on your host, add -e OLLAMA_HOST=host.docker.internal:11434 so the container can reach it. Includes simple-chat only.
From release tarball (production)
curl -L https://github.com/schmitech/orbit/releases/download/v2.6.6/orbit-2.6.6.tar.gz -o orbit-2.6.6.tar.gz
tar -xzf orbit-2.6.6.tar.gz && cd orbit-2.6.6
cp env.example .env && ./install/setup.sh
source venv/bin/activate
./bin/orbit.sh start && cat ./logs/orbit.log- Step-by-Step Tutorial โ Chat with your own data in minutes
- Cookbook โ 20+ recipes: database copilots, voice assistants, fault tolerance, MCP agents, private gateways
- Documentation โ Full architecture and setup guides
- GitHub Issues โ Bug reports and feature requests
Contributions are welcome! Check the issues for good first tasks, or open a new one to discuss your idea.
If you find ORBIT useful, a star helps others discover the project.
Apache 2.0 โ see LICENSE.