⚔ token-savior

• server.py: 2,439 -> 1,002 lines (-59%), split into focused modules
• Cache size: -57% via LazyLines (file content lazy-loaded from disk)
• 865 tests (up from 856)
• ProjectQueryEngine class replaces 705-line closure
• Automated benchmarks (benchmarks/run_benchmarks.py)
• Path traversal fix in create_checkpoint
• Output cap on get_change_impact / get_dependents (default 50,000 chars)
• 2 deprecated tool aliases with migration path (removal in v1.1.0)

An MCP server that indexes your codebase structurally and exposes surgical query tools — so your AI agent reads 200 characters instead of 200 files.

find_symbol("send_message")           →  67 chars    (was: 41M chars of source)
get_change_impact("LLMClient")        →  16K chars   (154 direct + 492 transitive deps)
get_function_source("compile")        →  4.5K chars  (exact source, no grep, no cat)
analyze_config()                      →  finds duplicates, secrets, orphan keys

Measured across 5,074 queries in 137 real sessions: 97% token reduction.

Every AI coding session starts the same way: the agent grabs cat or grep, reads a dozen files to find one function, then bloats its context trying to understand what else might break. By the end, half your token budget is gone before the first edit.

token-savior replaces that pattern entirely. It builds a structural index once, keeps it in sync with git automatically, and answers "where is X", "what calls X", and "what breaks if I change X" in sub-millisecond time — with responses sized to the answer, not the codebase.

"Tokens (naive)" = estimated tokens the agent would have consumed with cat/grep. These savings are model-agnostic -- the index reduces context window pressure regardless of provider. Updated at each release via automated benchmarks.

Since v1.0.0, file content is not stored in the cache (lazy-loaded from disk via LazyLines), reducing cache size by ~57%. With the persistent cache, subsequent restarts skip the full build. CPython goes from 56s -> under 1s on cache hit.

Runs three checks (individually toggleable via the checks parameter):

• Duplicates — Same key defined twice in the same file, plus Levenshtein-based typo detection (e.g. db_hsot vs db_host)
• Secrets — Regex patterns for known secret formats (API keys, tokens, private keys) plus Shannon entropy analysis for high-entropy strings
• Orphans — Cross-references config keys against actual code usage. Detects keys your code never reads and env vars your code expects but aren't set. Understands os.environ, process.env, os.Getenv, std::env::var, and more.

Supported formats: .yaml, .yml, .toml, .ini, .cfg, .properties, .env, .xml, .plist, .hcl, .tf, .conf, .json

LSP answers "where is this defined?" — token-savior answers "what breaks if I change it?"

LSP is point queries: one symbol, one file, one position. It can find where LLMClient is defined and who references it directly. Ask "what breaks transitively if I refactor LLMClient?" and LSP has nothing — the AI would need to chain dozens of find-reference calls recursively, reading files at every step.

get_change_impact("TestCase") on CPython finds 154 direct dependents and 492 transitive dependents in 0.45ms, returning 16K chars instead of reading 41M. And unlike LSP, it requires zero language servers — one binary covers Python + TS/JS + Go + Rust + C# + C/GLSL + config files + Dockerfiles out of the box.

uvx token-savior

No venv, no clone. Runs directly from PyPI.

git clone https://github.com/Mibayy/token-savior
cd token-savior
python3 -m venv ~/.local/token-savior-venv
~/.local/token-savior-venv/bin/pip install -e ".[mcp]"

Add to .mcp.json in your project root:

{
  "mcpServers": {
    "token-savior": {
      "command": "uvx",
      "args": ["token-savior"],
      "env": {
        "WORKSPACE_ROOTS": "/path/to/project1,/path/to/project2",
        "TOKEN_SAVIOR_CLIENT": "claude-code"
      }
    }
  }
}

Add to ~/.hermes/config.yaml:

mcp_servers:
  token-savior:
    command: ~/.local/token-savior-venv/bin/token-savior
    env:
      WORKSPACE_ROOTS: /path/to/project1,/path/to/project2
      TOKEN_SAVIOR_CLIENT: hermes
    timeout: 120
    connect_timeout: 30

TOKEN_SAVIOR_CLIENT is optional but lets the live dashboard attribute savings by client.

AI assistants default to grep and cat even when better tools are available. Soft instructions get rationalized away. Add this to your CLAUDE.md or equivalent:

## Codebase Navigation — MANDATORY

You MUST use token-savior MCP tools FIRST.

- ALWAYS start with: find_symbol, get_function_source, get_class_source,
  search_codebase, get_dependencies, get_dependents, get_change_impact
- Only fall back to Read/Grep when token-savior tools genuinely don't cover it
- If you catch yourself reaching for grep to find code, STOP

One server instance covers every project on the machine:

WORKSPACE_ROOTS=/root/myapp,/root/mybot,/root/docs token-savior

Each root gets its own isolated index, loaded lazily on first use. list_projects shows all registered roots. switch_project sets the active one.

The server detects changes via two mechanisms:

1. mtime scanning -- os.scandir() checks directory modification times before every query (~1-2ms). Changed files are re-parsed incrementally.
2. git tracking -- git diff and git status for symbol-level change detection.

No manual reindex after edits, branch switches, or pulls.

The index is saved to .token-savior-cache.json after every build -- human-readable JSON, inspectable when things go wrong, safe across Python versions. File content is not stored in the cache (lazy-loaded from disk on demand), reducing cache size by ~57%.

from token_savior.project_indexer import ProjectIndexer
from token_savior.query_api import ProjectQueryEngine

indexer = ProjectIndexer("/path/to/project")
index = indexer.index()
engine: ProjectQueryEngine = ProjectQueryEngine(index)

print(engine.get_project_summary())
print(engine.find_symbol("MyClass"))
print(engine.get_change_impact("send_message"))

# Dict-based access (backward compatible)
query = engine.as_dict()
print(query["find_symbol"]("MyClass"))

src/token_savior/
  server.py            MCP transport, tool routing, usage stats (~1,000 lines)
  tool_schemas.py       53 tool definitions, DEPRECATED_TOOLS set (extracted v1.0.0)
  slot_manager.py       Multi-project lifecycle, incremental mtime updates
  cache_ops.py          JSON persistence, legacy cache migration
  query_api.py          ProjectQueryEngine -- 22 query methods + as_dict()
  models.py             ProjectIndex, LazyLines, AnnotatorProtocol
  project_indexer.py    File discovery, structural indexing, dependency graphs
  brace_matcher.py      Shared brace matching for C/C#/Rust/Go (factored v1.0.0)
  annotator.py          Language dispatch via _ANNOTATOR_MAP
  *_annotator.py        Per-language annotators (Python, TS, Go, Rust, C#, C/GLSL)

pip install -e ".[dev,mcp]"
pytest tests/ -v          # 865 tests
ruff check src/ tests/

• Live-editing window: The index is git-aware and updates on query, not on save. If you edit a file and immediately call get_function_source, you may get the pre-edit version. The next git-tracked change triggers a re-index.
• Cross-language tracing: get_change_impact stops at language boundaries. Python calling a shell script calling a JSON config — the chain breaks after Python.
• JSON value semantics: The JSON annotator indexes key structure, not value meaning. Tracing what a config value propagates to across files is still manual.
• Windows paths: Not tested. Contributions welcome.
• Max files: Default 10,000 per project (configurable via TOKEN_SAVIOR_MAX_FILES env var).
• Max file size: Default 1 MB (configurable via TOKEN_SAVIOR_MAX_FILE_SIZE_MB env var).