Jarvis

Chiming into conversations (the magic moment):

👤 Alice: I wonder what the weather will be like tomorrow
👤 Bob: Yeah, we should check before planning the picnic
👤 Alice: Jarvis, what do you think?
  📝 Heard: "What do you think Jarvis?"
  🧠 Intent (wake word): directed → "what do you think about the weather for the picnic"

✨ Working on it: what do you think about the weather for the picnic
  🧰 Tool: getWeather…
  💬 Generating response...

🤖 Jarvis
Tomorrow looks great for a picnic! Sunny with highs around 22°C...

Jarvis understood the entire conversation and gave a contextual answer — no need to repeat the question.

Browser automation via MCP:

  📝 Heard: "Open YouTube Jarvis."
  🧠 Intent (wake word): directed → "open YouTube"

✨ Working on it: open YouTube
  💬 Generating response...

🤖 Jarvis
I have opened YouTube for you.

Health-aware advice:

  📝 Heard: "Should I order pizza tonight considering my health goals Jarvis?"
  🧠 Intent (wake word): directed → "should I order pizza tonight considering my health goals"

✨ Working on it: should I order pizza tonight considering my health goals
  🧰 Tool: fetchMeals…
  💬 Generating response...

🤖 Jarvis
Looking at your week — you mentioned wanting to lose 5kg by June...
Today you've had about 860 calories so far. Given your 1,800 calorie target,
you've got room for pizza! But maybe consider thin crust to save calories.

Personalised news search:

  📝 Heard: "What are some news from today that might interest me Jarvis?"
  🧠 Intent (wake word): directed → "what are some news from today that might interest me"

✨ Working on it: what are some news from today that might interest me
  🧰 Tool: searchMemory…
  🧰 Tool: webSearch…
  💬 Generating response...

🤖 Jarvis
Here's a quick snapshot of today's headlines...

Natural wake word placement (say "Jarvis" anywhere):

  📝 Heard: "Give me a random topic to discuss Jarvis."
  🧠 Intent (wake word): directed → "give me a random topic to discuss"

✨ Working on it: give me a random topic to discuss
  💬 Generating response...

🤖 Jarvis
How about the Fermi Paradox? Given the vast number of stars...

Echo detection (Jarvis ignores its own speech):

🤖 Jarvis
I have opened YouTube for you.

👂 Listening for follow-up (3s)...
  📝 Heard: "I have opened YouTube for you."
  🔇 Heard (echo): "i have opened youtube for you."
💤 Returning to wake word mode

• Multilingual (default, 99 languages): "whisper_model": "medium"
• English Only (slightly better English accuracy): "whisper_model": "medium.en"

Speed is relative to the original large model. Source

If you have an NVIDIA GPU, Jarvis can use CUDA for much faster speech recognition. The Windows installer offers an optional CUDA download during setup. For development:

pip install nvidia-cublas-cu12 nvidia-cudnn-cu12

CUDA is detected automatically — no configuration needed.

LLM Intent Judge - Jarvis uses gemma4:e2b for intelligent voice intent classification (echo detection, query extraction, stop commands). This model is automatically installed alongside your chosen chat model during setup. The intent judge cannot be disabled but gracefully falls back to simpler text matching if Ollama is unavailable.

Tool Router - When "tool_selection_strategy": "llm" (the default), Jarvis asks a small LLM to pick which tools are relevant for each query, shrinking the tool catalogue the chat model sees. By default this routing call reuses the intent-judge model — it's already warm and small enough not to stall the turn. Override with "tool_router_model": "<name>" to dedicate a different model to routing. Other strategies: "keyword" (fast, no LLM), "embedding" (nomic-embed-text), "all" (no filtering).

Small chat models (~2B, e.g. gemma4:e2b) degrade sharply as their prompt grows. Jarvis runs two cheap distil passes to keep the prompt tight:

• Memory digest — boils diary + graph recall into a short relevance-filtered note before injecting it as background context.
• Tool-result digest — boils a raw tool payload (especially webSearch UNTRUSTED WEB EXTRACT blocks) into a short attributed fact note before it reaches the main reply model.

Both auto-enable for small models and stay off for large models that ground on raw payloads reliably. Override in ~/.config/jarvis/config.json:

{
  "memory_digest_enabled": null,          // null = auto-on for SMALL, false to force off, true to force on
  "tool_result_digest_enabled": null,     // same semantics
  "llm_digest_timeout_sec": 8.0           // tight ceiling shared by both passes
}

Field logs show 🧩 Memory digest: … and 🧩 Tool digest: … lines when a pass ran, so you can see when the substrate was replaced.

Piper TTS (default) - Neural TTS that auto-downloads on first use (~60MB):

• Works out of the box - no setup required
• High-quality British English male voice (en_GB-alan-medium)
• Fast local synthesis with exact duration tracking

To use different Piper voices, download from HuggingFace and set:

{
  "tts_piper_model_path": "~/.local/share/jarvis/models/piper/en_GB-alan-medium.onnx"
}

Chatterbox - AI voice with emotion control (requires running from source):

{ "tts_engine": "chatterbox" }

Voice cloning with Chatterbox - add a 3-10 second .wav sample:

{
  "tts_engine": "chatterbox",
  "tts_chatterbox_audio_prompt": "/path/to/voice.wav"
}

Jarvis can provide location-aware responses (weather, local time, etc.) using a local GeoLite2 database — no cloud geolocation services are used.

IP detection chain (in order of preference):

1. Manual IP — configure location_ip_address in settings
2. UPnP — queries your local router (no traffic leaves LAN)
3. Socket heuristic — determines which interface routes externally (no data sent)
4. OpenDNS DNS query — single myip.opendns.com lookup to 208.67.222.222 (only external query)

If your ISP uses carrier-grade NAT (CGNAT), Jarvis automatically resolves your true public IP via the same OpenDNS DNS query. This can be disabled:

{
  "location_cgnat_resolve_public_ip": false
}

Setup: Register for a free MaxMind GeoLite2 account, download the City database (MMDB format), and save it to ~/.local/share/jarvis/geoip/GeoLite2-City.mmdb. The setup wizard will guide you through this.

Connect Jarvis to external tools via MCP servers:

{
  "mcps": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": { "GITHUB_TOKEN": "your-token" }
    }
  }
}

Popular integrations:

• Home Assistant - Voice control for smart home
• Google Workspace - Gmail, Calendar, Drive, Docs
• GitHub - Issues, PRs, workflows
• Notion - Knowledge management
• Slack/Discord - Team communication
• Databases - MySQL, PostgreSQL, MongoDB
• Composio - 500+ apps in one integration

See full MCP setup guide below.

1. Add MCP Server integration in Home Assistant (Settings → Devices & services)
2. Expose entities you want to control (Settings → Voice assistants → Exposed entities)
3. Create Long-lived Access Token (Profile → Security → Create token)
4. Install proxy: uv tool install git+https://github.com/sparfenyuk/mcp-proxy
5. Add to config:

{
  "mcps": {
    "home_assistant": {
      "command": "mcp-proxy",
      "args": ["http://localhost:8123/mcp_server/sse"],
      "env": { "API_ACCESS_TOKEN": "YOUR_TOKEN" }
    }
  }
}

"Jarvis, turn on the living room lights" / "set bedroom to 72°" / "run good night scene"

{
  "mcps": {
    "google_workspace": {
      "command": "npx",
      "args": ["-y", "google-workspace-mcp"],
      "env": {
        "GOOGLE_CLIENT_ID": "your-client-id",
        "GOOGLE_CLIENT_SECRET": "your-client-secret"
      }
    }
  }
}

Setup: taylorwilsdon/google_workspace_mcp

Notion:

{ "mcps": { "notion": { "command": "npx", "args": ["-y", "@makenotion/mcp-server-notion"], "env": { "NOTION_API_KEY": "your-token" } } } }

Slack:

{ "mcps": { "slack": { "command": "npx", "args": ["-y", "slack-mcp-server"], "env": { "SLACK_BOT_TOKEN": "xoxb-...", "SLACK_USER_TOKEN": "xoxp-..." } } } }

Discord:

{ "mcps": { "discord": { "command": "npx", "args": ["-y", "discord-mcp-server"], "env": { "DISCORD_BOT_TOKEN": "your-token" } } } }

Databases: bytebase/dbhub (SQL), mongodb-mcp-server (MongoDB)

{
  "mcps": {
    "composio": {
      "command": "npx",
      "args": ["-y", "@composiohq/rube"],
      "env": { "COMPOSIO_API_KEY": "your-key" }
    }
  }
}

Get API key at composio.dev

First startup takes a bit - Jarvis pre-warms the Whisper, chat, and intent-judge models before announcing "Listening!" so the first engagement feels instant. This adds a few seconds on cold start and is bounded at 60 s — if Ollama is slow, Jarvis will start listening anyway and load the models on demand.

Jarvis doesn't hear me - Check microphone permissions, speak clearly after "Jarvis"

Responses are slow - Ensure you have enough VRAM (8GB+ for default model; see System Requirements for other models)

Windows: App won't start - Extract full zip first, check Windows Defender

macOS: "App can't be opened" - Right-click → Open, or System Settings → Privacy & Security → Allow

Linux: No tray icon - sudo apt install libayatana-appindicator3-1

git clone https://github.com/isair/jarvis.git
cd jarvis

# macOS
bash scripts/run_macos.sh

# Windows (with Micromamba)
pwsh -ExecutionPolicy Bypass -File scripts\run_windows.ps1

# Linux
bash scripts/run_linux.sh

Running from source enables Chatterbox TTS (AI voice with emotion/cloning). Piper TTS works in both bundled and source modes.

{
  "web_search_enabled": false,
  "wikipedia_fallback_enabled": false,
  "brave_search_api_key": "",
  "mcps": {},
  "location_auto_detect": false,
  "location_cgnat_resolve_public_ip": false,
  "location_enabled": false
}

Verify: sudo lsof -i -n -P | grep jarvis (should only show 127.0.0.1 to Ollama)

When DuckDuckGo is rate-limited or returns nothing fetchable, Jarvis walks a small fallback chain before giving up rather than confabulating:

1. Brave Search — opt-in, requires brave_search_api_key. Free tier: 2,000 queries/month. Get a key at api.search.brave.com.
2. Wikipedia — zero-config, on by default, uses the Wikipedia host matching the language Whisper auto-detected on the utterance (so a Turkish question gets a Turkish answer). Disable with wikipedia_fallback_enabled: false.
3. Honest failure — if every provider fails, the reply tells you the search was blocked rather than making something up.

The whole chain is bounded by a ~20s wall-clock deadline so a stalled provider can't run out the voice-assistant latency budget.