Kaizen

Continuous prompt optimization platform for AI applications.
Instrument LLM calls, collect user feedback, and let Kaizen refine your prompts automatically.

Kaizen is a self-hosted platform that ingests runtime traces and feedback from your AI application, runs DSPy-based prompt optimization (GEPA by default, MIPROv2 optional) in a background worker, versions the result, and — when configured — opens a pull request against your repository so the change can be reviewed like any other code.

The stack ships as a FastAPI service, a Celery worker, a Postgres + Redis backend, a Next.js dashboard, and a Python SDK that lives in src/sdk/.

• Language: Python 3.12 (>=3.12,<3.13)
• API: FastAPI 0.135, Uvicorn
• Worker: Celery 5.6 (Redis broker, Postgres result backend)
• Optimization: DSPy 3.1 + LiteLLM 1.80
• ORM / Migrations: SQLAlchemy 2.0 (async, psycopg driver) + Alembic 1.18
• Database: PostgreSQL 16
• Cache / Queue: Redis 7.4
• Dashboard: Next.js 14, TypeScript, Tailwind, TanStack Query, Radix UI, Recharts
• Docs site: Next.js 14 + Nextra 4.6
• Package manager: uv (Python), npm (JS)

• Python 3.12 (3.13 is not supported by the pinned runtime)
• Node.js 20+ (dashboard and docs site)
• Docker / Podman with Compose plugin
• uv for local Python development
• An OpenAI-compatible LLM endpoint and API key (used by LiteLLM for teacher/judge models)
• A Git provider access token if you want auto-PR (GitHub, Bitbucket Server, or GitLab)

git clone <your-fork-url> kaizen
cd kaizen
cp .env.example .env

Then edit .env and set at minimum:

docker compose up -d      # or: podman-compose up -d

This boots:

On first run, bootstrap a key through the dashboard at http://localhost:3000, or from the command line:

docker compose exec api python -m src create-key --label "my-team"

Use the returned kaizen_... key as the X-API-Key header on all API requests.

The SDK is not published to PyPI yet. Install it directly from this repo:

pip install ./src/sdk                # from a checkout of this repo
# or:
pip install "git+https://github.com/<your-fork>/kaizen.git#subdirectory=src/sdk"

Minimal usage:

import kaizen_sdk as kaizen

kaizen.init(
    api_key="kaizen_...",
    base_url="http://localhost:8000",
    feedback_threshold=5,
    teacher_model="openai/gpt-4o",
    judge_model="openai/gpt-4o-mini",
    # Optional — enables auto-PR:
    git_provider="github",            # github | bitbucket_server | gitlab
    git_repo="your-org/your-repo",
    git_base_branch="main",
    git_token="ghp_...",
)

# Wrap an LLM call — the SDK records inputs/outputs into a per-request buffer
result = await kaizen.trace("summarize_ticket", chain.ainvoke, {"text": "..."})

# When the user rates the response, flush buffered traces with a score in [0, 1]
await kaizen.flush(score=0.85)

# Later, fetch the currently active optimized prompt for a task
prompt = await kaizen.get_prompt("summarize_ticket")

The public SDK surface is: init, trace, trace_sync, flush, flush_sync, get_prompt, get_buffered_traces, reset_buffer, close.

kaizen/
├── src/
│   ├── api/          # FastAPI app, routes, auth, schemas
│   ├── models/       # SQLAlchemy ORM models
│   ├── services/     # Git providers (GitHub/Bitbucket/GitLab), auto-PR, prompt file IO
│   ├── worker/       # Celery app, DSPy pipeline, evaluators, cost estimator
│   ├── sdk/          # kaizen_sdk Python package (installable)
│   ├── utils/        # Crypto helpers, PR templates
│   ├── config.py     # pydantic-settings configuration
│   └── __main__.py   # `python -m src create-key` CLI
├── dashboard/        # Next.js 14 web UI
├── docs/             # Nextra documentation site
├── alembic/          # Database migrations
├── tests/            # Pytest suite (api, config, feedback loop)
├── docker-compose.yml
├── Dockerfile
└── pyproject.toml

All endpoints live under /api/v1/ and require an X-API-Key header except where noted. OpenAPI/Swagger is served at http://localhost:8000/docs while the API is running.

uv sync                         # installs project + dev deps from uv.lock

# Bring up only the infrastructure containers
docker compose up -d postgres redis

# Apply migrations
uv run alembic upgrade head

# Run the API with auto-reload
uv run uvicorn src.api.main:app --reload --port 8000

# In another shell: run the Celery worker
uv run celery -A src.worker.celery_app worker -l info

cd dashboard && npm install && npm run dev     # http://localhost:3000
cd docs      && npm install && npm run dev     # http://localhost:4000

uv run pytest

tests/ contains API, config, and end-to-end feedback-loop tests. pytest-asyncio is configured via pyproject.toml.

uv run alembic revision --autogenerate -m "describe the change"
uv run alembic upgrade head
uv run alembic downgrade -1

The API container runs alembic upgrade head on startup, so no manual step is required in the Docker workflow.

The provided Dockerfile builds a single Python image (used by api, worker, and beat) via uv sync --locked --no-dev. The dashboard/ and docs/ directories have their own Dockerfiles.

docker compose build
docker compose up -d

Production considerations:

• Provide a long-lived GIT_TOKEN_ENCRYPTION_KEY. Rotating it invalidates previously encrypted Git tokens.
• Override TEACHER_MODEL, JUDGE_MODEL, MAX_TRIALS_DEFAULT, COST_BUDGET_DEFAULT, and OPTIMIZATION_WALL_TIMEOUT in .env to tune optimization behavior and spend.
• Set SSL_VERIFY=false only if you are pointing OPENAI_API_BASE at an internal, self-signed endpoint.
• DEFAULT_OPTIMIZER defaults to gepa; set it to miprov2 to switch back to the MIPROv2 optimizer.

See CONTRIBUTING.md for development setup, commit conventions, and PR guidelines.

MIT