# xgrammar > Efficient, Flexible and Portable Structured Generation - **URL**: https://www.freshcrate.ai/projects/xgrammar - **Author**: MLC Team - **Category**: RAG & Memory - **Latest version**: `v0.2.1` (2026-05-17) - **License**: Apache 2.0 - **Source**: https://github.com/mlc-ai/xgrammar - **Homepage**: https://pypi.org/project/xgrammar/ - **Language**: C++ - **GitHub**: 1,637 stars, 141 forks - **Registry**: pypi (`xgrammar`) - **Tags**: `inference`, `learning`, `machine`, `pypi` ## Description
logo [![Documentation](https://img.shields.io/badge/docs-latest-green)](https://xgrammar.mlc.ai/docs/) [![License](https://img.shields.io/badge/license-apache_2-blue)](https://github.com/mlc-ai/xgrammar/blob/main/LICENSE) [![PyPI](https://img.shields.io/pypi/v/xgrammar)](https://pypi.org/project/xgrammar) [![PyPI Downloads](https://static.pepy.tech/badge/xgrammar)](https://pepy.tech/projects/xgrammar) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/mlc-ai/xgrammar) **Efficient, Flexible and Portable Structured Generation** [Get Started](#get-started) | [Documentation](https://xgrammar.mlc.ai/docs/) | [Blogpost](https://blog.mlc.ai/2024/11/22/achieving-efficient-flexible-portable-structured-generation-with-xgrammar) | [Technical Report](https://arxiv.org/abs/2411.15100)
## News - [2025/12] XGrammar has been officially integrated into [Mirai](https://github.com/trymirai/uzu) - [2025/09] XGrammar has been officially integrated into [OpenVINO GenAI](https://github.com/openvinotoolkit/openvino.genai) - [2025/02] XGrammar has been officially integrated into [Modular's MAX](https://docs.modular.com/max/serve/structured-output) - [2025/01] XGrammar has been officially integrated into [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM). - [2024/12] XGrammar has been officially integrated into [vLLM](https://github.com/vllm-project/vllm). - [2024/12] We presented research talks on XGrammar at CMU, UC Berkeley, MIT, THU, SJTU, Ant Group, LMSys, Qingke AI, Camel AI. The slides can be found [here](https://docs.google.com/presentation/d/1iS7tu2EV4IKRWDaR0F3YD7ubrNqtGYUStSskceneelc/edit?usp=sharing). - [2024/11] XGrammar has been officially integrated into [SGLang](https://github.com/sgl-project/sglang). - [2024/11] XGrammar has been officially integrated into [MLC-LLM](https://github.com/mlc-ai/mlc-llm). - [2024/11] We officially released XGrammar v0.1.0! ## Overview XGrammar is an open-source library for efficient, flexible, and portable structured generation. It leverages constrained decoding to ensure **100% structural correctness** of the output. It supports general context-free grammar to enable a broad range of structures, including **JSON**, **regex**, **custom context-free grammar**, etc. XGrammar uses careful optimizations to achieve extremely low overhead in structured generation. It has achieved **near-zero overhead** in JSON generation, making it one of the fastest structured generation engines available. XGrammar features **universal deployment**. It supports: * **Platforms**: Linux, macOS, Windows * **Hardware**: CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU, etc. * **Languages**: Python, C++, and JavaScript APIs * **Models**: Qwen, Llama, DeepSeek, Phi, Gemma, etc. XGrammar is very easy to integrate with LLM inference engines. It is the default structured generation backend for most LLM inference engines, including [**vLLM**](https://github.com/vllm-project/vllm), [**SGLang**](https://github.com/sgl-project/sglang), [**TensorRT-LLM**](https://github.com/NVIDIA/TensorRT-LLM), and [**MLC-LLM**](https://github.com/mlc-ai/mlc-llm), as well as many other companies. You can also try out their structured generation modes! ## Get Started Install XGrammar: ```bash pip install xgrammar ``` For use with MPS on Apple Silicon, install with: ```bash pip install "xgrammar[metal]" ``` Import XGrammar: ```python import xgrammar as xgr ``` Please visit our [documentation](https://xgrammar.mlc.ai/docs/) to get started with XGrammar. - [Installation](https://xgrammar.mlc.ai/docs/start/installation) - [Quick start](https://xgrammar.mlc.ai/docs/start/quick_start) ## Third-Party Bindings - **Rust**: [xgrammar-rs](https://github.com/trymirai/xgrammar-rs) — Community Rust bindings for XGrammar. ## Collaborators XGrammar has been widely adopted in industry, open-source projects, and academia. Our collaborators include:
[](https://github.com/NVIDIA/TensorRT-LLM)   [](https://www.databricks.com/)   [](https://about.meta.com/)   [](https://about.google/)   [](https://github.com/NVIDIA/TensorRT-LLM)   [](http ## Recent releases | Version | Date | Urgency | Changes | | --- | --- | --- | --- | | `v0.2.1` | 2026-05-17 | High | ## What's Changed * format: fix the format of the test files. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/624 * fix: wrap Kimi auto tool calls in section markers by @JustinTong0323 in https://github.com/mlc-ai/xgrammar/pull/623 * feat: expose normalize_tool_choice and unify Qwen XML structural tag builders by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/625 * fix: stop emitting \/ for forward slashes in JSON output by @Ubospica in https://github.com/mlc-ai/xgrammar/p | | `v0.2.0` | 2026-05-01 | High | ## What's Changed * refactor: unify reasoning parameter and rename model keys by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/609 * fix: fix gpt-oss's tool-calling format. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/607 * docs: add Perplexity collaborator logo by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/612 * Expose draft tree traversal on GrammarMatcher by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/613 * perf: reduce structural-tag compile tim | | `v0.1.34` | 2026-04-29 | High | ## What's Changed * Reapply "refactor: migrate the binding logic into `tvm_ffi`. (#550)"" by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/576 * fix: accept {n, -1} as unbounded repeat in EBNF parser by @ushiromiya-lion in https://github.com/mlc-ai/xgrammar/pull/579 * fix: AnyTokensFormat with exclude_tokens should be treated as self-terminating by @ushiromiya-lion in https://github.com/mlc-ai/xgrammar/pull/578 * fix: fix the building of the website. by @Seven-Streams in https:// | | `0.1.33` | 2026-04-21 | Low | Imported from PyPI (0.1.33) | | `v0.1.33` | 2026-03-27 | Medium | ## What's Changed * refactor: simplify TagDispatch by removing stop_eos and stop_str by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/554 * feat: provide `PlusFormat`, `OptionalFormat`, `StarFormat` to enhance `StructuralTag`. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/557 * feat: support `structural_tag`-level cache. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/553 * feat: token-level grammar support with Token/ExcludeToken/TokenTagDispatch edges by | | `v0.1.33` | 2026-03-27 | Medium | ## What's Changed * refactor: simplify TagDispatch by removing stop_eos and stop_str by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/554 * feat: provide `PlusFormat`, `OptionalFormat`, `StarFormat` to enhance `StructuralTag`. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/557 * feat: support `structural_tag`-level cache. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/553 * feat: token-level grammar support with Token/ExcludeToken/TokenTagDispatch edges by | | `v0.1.33` | 2026-03-27 | Medium | ## What's Changed * refactor: simplify TagDispatch by removing stop_eos and stop_str by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/554 * feat: provide `PlusFormat`, `OptionalFormat`, `StarFormat` to enhance `StructuralTag`. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/557 * feat: support `structural_tag`-level cache. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/553 * feat: token-level grammar support with Token/ExcludeToken/TokenTagDispatch edges by | | `v0.1.33` | 2026-03-27 | Medium | ## What's Changed * refactor: simplify TagDispatch by removing stop_eos and stop_str by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/554 * feat: provide `PlusFormat`, `OptionalFormat`, `StarFormat` to enhance `StructuralTag`. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/557 * feat: support `structural_tag`-level cache. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/553 * feat: token-level grammar support with Token/ExcludeToken/TokenTagDispatch edges by | | `v0.1.33` | 2026-03-27 | Low | ## What's Changed * refactor: simplify TagDispatch by removing stop_eos and stop_str by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/554 * feat: provide `PlusFormat`, `OptionalFormat`, `StarFormat` to enhance `StructuralTag`. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/557 * feat: support `structural_tag`-level cache. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/553 * feat: token-level grammar support with Token/ExcludeToken/TokenTagDispatch edges by | | `v0.1.33` | 2026-03-27 | Low | ## What's Changed * refactor: simplify TagDispatch by removing stop_eos and stop_str by @Ubospica in https://github.com/mlc-ai/xgrammar/pull/554 * feat: provide `PlusFormat`, `OptionalFormat`, `StarFormat` to enhance `StructuralTag`. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/557 * feat: support `structural_tag`-level cache. by @Seven-Streams in https://github.com/mlc-ai/xgrammar/pull/553 * feat: token-level grammar support with Token/ExcludeToken/TokenTagDispatch edges by | ## Citation - HTML: https://www.freshcrate.ai/projects/xgrammar - Markdown: https://www.freshcrate.ai/projects/xgrammar.md - Dependencies JSON: https://www.freshcrate.ai/api/projects/xgrammar/deps _Generated by freshcrate.ai. Indexes pypi releases for AI-agent ecosystem packages._