# deltallm

> Route, manage, and analyze your LLM requests across multiple providers with a unified API interface

- **URL**: https://www.freshcrate.ai/projects/deltallm
- **Author**: deltawi
- **Category**: MCP Servers
- **Latest version**: `v0.1.24-rc2` (2026-05-06)
- **License**: MIT
- **Source**: https://github.com/deltawi/deltallm
- **Homepage**: https://deltallm.io
- **Language**: Python
- **GitHub**: 4 stars, 1 forks
- **Registry**: github
- **Tags**: `ai-gateway`, `ai-infrastructure`, `api-gateway`, `kubernetes`, `llm-gateway`, `llm-proxy`, `llm-routing`, `mcp`, `model-context-protocol`, `python`

## Description

Route, manage, and analyze your LLM requests across multiple providers with a unified API interface

## Recent releases

| Version | Date | Urgency | Changes |
| --- | --- | --- | --- |
| `v0.1.24-rc2` | 2026-05-06 | High | Release candidate for v0.1.24 including OpenAI Batch API compatibility fixes for LiteLLM. |
| `v0.1.24` | 2026-05-06 | High | # v0.1.24  This release focuses on batch reliability, chat batch support, model access governance, upstream/provider correctness, and admin UI polish.  Compared against `v0.1.23`.  ## Highlights  ### Batch API and worker reliability  - Added chat batch support with concurrent execution for OpenAI-compatible providers, `sync_microbatch` configuration, safe per-item fallback, cancellation/finalization hardening, and model UI support for chat batching parameters. (#150) - Added structured batch ret |
| `v0.1.23` | 2026-04-25 | High | ## What's Changed * Include boto3 in runtime dependencies by @deltawi in https://github.com/deltawi/deltallm/pull/118   **Full Changelog**: https://github.com/deltawi/deltallm/compare/v0.1.22...v0.1.23 |
| `v0.1.20-rc3` | 2026-04-24 | High | ## What's Changed * hotfix: make audit detail lookup text-safe by @deltawi in https://github.com/deltawi/deltallm/pull/114   **Full Changelog**: https://github.com/deltawi/deltallm/compare/v0.1.21-rc2...v0.1.20-rc3 |
| `v0.1.21-rc1` | 2026-04-18 | High | ## What's Changed * Fix deployment health classification for upstream request failures by @deltawi in https://github.com/deltawi/deltallm/pull/101   **Full Changelog**: https://github.com/deltawi/deltallm/compare/v0.1.20-rc2...v0.1.21-rc1 |
| `v0.1.20-rc2` | 2026-04-18 | High | ## What's Changed * Fix batch embedding output compatibility and refactor worker internals by @deltawi in https://github.com/deltawi/deltallm/pull/99   **Full Changelog**: https://github.com/deltawi/deltallm/compare/v0.1.20-rc1...v0.1.20-rc2 |
| `v0.1.20-rc2` | 2026-04-18 | High | ## What's Changed * Fix batch embedding output compatibility and refactor worker internals by @deltawi in https://github.com/deltawi/deltallm/pull/99   **Full Changelog**: https://github.com/deltawi/deltallm/compare/v0.1.20-rc1...v0.1.20-rc2 |
| `v0.1.20-rc2` | 2026-04-18 | High | ## What's Changed * Fix batch embedding output compatibility and refactor worker internals by @deltawi in https://github.com/deltawi/deltallm/pull/99   **Full Changelog**: https://github.com/deltawi/deltallm/compare/v0.1.20-rc1...v0.1.20-rc2 |
| `v0.1.20-rc1` | 2026-04-15 | High | ## v0.1.20-rc1    v0.1.20-rc1 is a release candidate focused on a major new asynchronous embeddings batch pipeline, plus deployment   hardening, audit-log improvements, and a few UI/documentation updates.    ### Highlights    #### New: Embeddings Batch API    DeltaLLM now supports asynchronous embeddings batches end to end.    What’s included:    - Upload JSONL batch input files with purpose=batch   - Create batches through /v1/batches   - Queue, execute, finalize, and retain ba |
| `v0.1.19` | 2026-04-11 | High | ## Highlights  - **Named credentials** for provider connection settings — share one credential across many model deployments and rotate it once. (#65, fixes #58) - **Embedding batch overhaul** — three phases of reliability, streaming/throughput, and operator hardening, plus upstream **microbatching** to coalesce eligible embedding requests into fewer provider round-trips. (#66, #69, #70, #76, #77, #78) - **Custom upstream auth headers** for OpenAI-compatible providers that don't use `Authorizati |

## Citation

- HTML: https://www.freshcrate.ai/projects/deltallm
- Markdown: https://www.freshcrate.ai/projects/deltallm.md
- Dependencies JSON: https://www.freshcrate.ai/api/projects/deltallm/deps

_Generated by freshcrate.ai. Indexes github releases for AI-agent ecosystem packages._