freshcrate — #evaluation

Home > #evaluation

Tag: #evaluation

14 packages • ⭐ 130,490 total stars

mlflowv3.11.1🌱 Seedling⭐25,285

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controllin

agentops agents ai ai-governance apache-spark evaluation langchain llm-evaluation pythonby mlflow

langfusev3.169.0🌿 Growing⭐24,578

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

analytics autogen evaluation langchain large-language-models llama-index llm llm-evaluation prompt-engineering typescriptby langfuse

promptfoocode-scan-action-0.1.5🌿 Growing⭐19,943

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and

ci ci-cd cicd evaluation evaluation-framework llm llm-eval llm-evaluation typescriptby promptfoo

opik2.0.6🌳 Mature⭐18,767

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

evaluation hacktoberfest hacktoberfest2025 langchain llama-index llm llm-evaluation llm-observability pythonby comet-ml

WeKnorav0.4.0🌳 Mature⭐13,819

LLM-powered framework for deep document understanding, semantic retrieval, and context-aware answers using RAG paradigm.

agent agentic ai chatbot chatbots embeddings evaluation generative-ai goby Tencent

ragasv0.4.3🌱 Seedling⭐13,329

Supercharge Your LLM Application Evaluations 🚀

evaluation llm llmops pythonby explodinggradients

AutoRAGv0.3.22🌱 Seedling⭐4,693

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

analysis automl benchmarking document-parser embeddings evaluation llm llm-evaluation pythonby Marker-Inc-Korea

agentav0.96.7🌳 Mature⭐4,011

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

agents evaluation llm-as-a-judge llm-evaluation llm-framework llm-monitoring llm-observability llm-platform prompt-engineering typescriptby Agenta-AI

langwatchskills@v0.3.0🌿 Growing⭐3,193

The platform for LLM evaluations and AI agent testing

ai analytics datasets dspy evaluation gpt llm llm-ops typescriptby langwatch

ouroborosv0.28.8🌳 Mature⭐2,107

Stop prompting. Start specifying.

ai-agent claude-code codex-cli devtools evaluation llm mcp multi-agent prompt-engineering pythonby Q00

OpenClawProBenchmain@2026-04-15🌿 Growing⭐340

OpenClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability.

agent benchmark evaluation harness leaderboard llm openclaw pythonby suyoumo

aragv0.1.0🌿 Growing⭐247

A-RAG: Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces. State-of-the-art RAG framework with keyword, semantic, and chunk read tools for multi-hop QA.

agent agentic-ai agenticrag deepresearch evaluation graphrag llm llmagents pythonby Ayanami0730

evalsv0.1.15🌿 Growing⭐103

A comprehensive evaluation framework for AI agents and LLM applications.

agentic agentic-ai ai evaluation machine-learning python strands-agentsby strands-agents

arthur-engine2.1.529🌿 Growing⭐75

Make AI work for Everyone - Monitoring and governing for your AI/ML

agentic benchmarking evaluation genai guardrails llm ml monitoring pythonby arthur-ai