# tokenizer

> High-Performance Tokenizer implementation in PHP.

- **URL**: https://www.freshcrate.ai/projects/tokenizer
- **Author**: neuron-core
- **Category**: Frameworks
- **Latest version**: `1.0.0` (2025-09-16)
- **License**: MIT
- **Source**: https://github.com/neuron-core/tokenizer
- **Homepage**: https://neuron-ai.dev
- **Language**: PHP
- **GitHub**: 6 stars, 1 forks
- **Registry**: github (`neuron-core/tokenizer`)
- **Tags**: `agentic-framework`, `agentic-workflow`, `ai`, `ai-agents`, `ai-framework`, `bpe-tokenizer`, `llm`, `php`

## Description

High-Performance Tokenizer implementation in PHP.

## Recent releases

| Version | Date | Urgency | Changes |
| --- | --- | --- | --- |
| `1.0.0` | 2025-09-16 | Low | Latest release: 1.0.0 |

## Citation

- HTML: https://www.freshcrate.ai/projects/tokenizer
- Markdown: https://www.freshcrate.ai/projects/tokenizer.md
- Dependencies JSON: https://www.freshcrate.ai/api/projects/tokenizer/deps

_Generated by freshcrate.ai. Indexes github releases for AI-agent ecosystem packages._