# llm-gateway

> Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any            compatible backend. Identity-based access, virtual A

- **URL**: https://www.freshcrate.ai/projects/llm-gateway
- **Author**: openziti
- **Category**: Infrastructure
- **Latest version**: `v0.1.4` (2026-04-08)
- **License**: Apache-2.0
- **Source**: https://github.com/openziti/llm-gateway
- **Language**: Go
- **GitHub**: 55 stars, 2 forks
- **Registry**: github (`openziti/llm-gateway`)
- **Tags**: `agentic-ai`, `ai-gateway`, `ai-infrastructure`, `ai-security`, `anthropic`, `e2e-encryption`, `go`, `llm-gateway`, `llm-proxy`

## Description

Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any            compatible backend. Identity-based access, virtual API keys, and end-to-end encryption via OpenZiti

## Recent releases

| Version | Date | Urgency | Changes |
| --- | --- | --- | --- |
| `v0.1.4` | 2026-04-08 | Medium | ## CHANGELOG  CHANGE: Removed specificity in docs and implementation tying local instances to Ollama; we really support any local OpenAI-compatible inference. Works with Ollama, llama-server, vLLM, SGLang, or anything that exposes `/v1/chat/completions`.  FIX: Improved ziti context handling to eliminate an issue with leaks on long-running instances. (https://github.com/openziti/llm-gateway/issues/7) |
| `v0.1.3` | 2026-03-19 | Low | ## CHANGELOG  Initial public release. |

## Citation

- HTML: https://www.freshcrate.ai/projects/llm-gateway
- Markdown: https://www.freshcrate.ai/projects/llm-gateway.md
- Dependencies JSON: https://www.freshcrate.ai/api/projects/llm-gateway/deps

_Generated by freshcrate.ai. Indexes github releases for AI-agent ecosystem packages._