# RAGElo

> RAGElo is a set of tools that helps you selecting the best RAG-based LLM agents by using an Elo ranker

- **URL**: https://www.freshcrate.ai/projects/RAGElo
- **Author**: zetaalphavector
- **Category**: RAG & Memory
- **Latest version**: `0.5.0` (2026-05-04)
- **License**: Apache-2.0
- **Source**: https://github.com/zetaalphavector/RAGElo
- **Language**: Python
- **GitHub**: 128 stars, 6 forks
- **Registry**: github
- **Tags**: `python`

## Description

RAGElo is a set of tools that helps you selecting the best RAG-based LLM agents by using an Elo ranker

## Recent releases

| Version | Date | Urgency | Changes |
| --- | --- | --- | --- |
| `0.5.0` | 2026-05-04 | High | ## RAGElo v0.5.0 - The Rubrics release. The main change in this release is the addition of RubricEvaluators. It allows the evaluator to ground its selection of best Agent in a multiple-criteria rubric that can be either directly given to the Evaluator or automatically created based on the retrieved documents.   Other new features include: - Added support for multi-turn conversations in all answer evaluators. - Stronger typing for evaluators, rankers, results, and evaluables. Less mypy error |
| `0.4.0` | 2026-03-02 | Low | ## What's Changed  ### New Features    - External LLM adapter support (#76): BaseLLMProvider can now be used without LLMProviderConfig, making it easy to integrate third-party or custom LLM clients into RAGElo without subclassing the full provider stack. This paves to way to better integration with Zeta Alpha's [agents-sdk](https://github.com/zetaalphavector/platform).  ### Improvements    - Improved experiment loading (#74): Major overhaul of Experiment state management and loading logi |
| `0.3.1` | 2025-09-06 | Low | - Dropped required Tenacity version to <9 - Fixed issue with GPT-5 and Openai O-models and temperature - Fixed bug introduced in 0.3 with document filtering  ## What's Changed * Tenacity 8 by @ArthurCamara in https://github.com/zetaalphavector/RAGElo/pull/69   **Full Changelog**: https://github.com/zetaalphavector/RAGElo/compare/0.3...0.3.1 |
| `0.3` | 2025-09-05 | Low | This new release adds a bunch of quality-of-life improvements. Mainly, all evaluators now use Jinja Templates instead of plain strings, which were a pain to format.  We also changed how most of the evaluators return their answers, relying more on Pydantic BaseModels, and less on finicky strings and JSONs.  Finally, we moved all the codebase to Pydantic-V2, and updated the OpenAI provider to use the responses API instead of the old and deprecated chat_responses.   ## What's Changed * Merge c |
| `0.2.1` | 2025-06-24 | Low | ## What's Changed * Small fixes and notebook update by @ArthurCamara in https://github.com/zetaalphavector/RAGElo/pull/61   **Full Changelog**: https://github.com/zetaalphavector/RAGElo/compare/0.2.0...0.2.1 |
| `0.2.0` | 2025-04-09 | Low | ## What's Changed * Example fixes by @ArthurCamara in https://github.com/zetaalphavector/RAGElo/pull/54   **Full Changelog**: https://github.com/zetaalphavector/RAGElo/compare/0.1.9...0.2.0 |
| `0.1.9` | 2025-04-08 | Low | ## What's Changed * add chat pairwise evaluation by @pavlosZakkas in https://github.com/zetaalphavector/RAGElo/pull/42 * Ollama and OpenAI updates by @ArthurCamara in https://github.com/zetaalphavector/RAGElo/pull/43 * Big overhaul: Added Experiments, JSON and Structuted LLM outputs, more evaluatoers and more by @ArthurCamara in https://github.com/zetaalphavector/RAGElo/pull/45 * Add StrEnum to RAGElo types by @din0s in https://github.com/zetaalphavector/RAGElo/pull/47 * Add seed to base ll |
| `0.1.8` | 2024-08-29 | Low | ## New features: The `Query` object now supports two new methods for easier evaluation of your retrieval pipeline: - `query.get_runs()` returns a dictionary of TREC-style runs for all the agents that retrieved documents for that query. (the mapping is agent_id -> query_id->document_id->retrieval_score). - `query.get_qrels()` returns a TREC-style qrels dictionary with the judgement scores assigned by an Evaluator. The mapping is query_id->document_id->relevance).  You can explore how these t |
| `0.1.7` | 2024-08-23 | Low | ## What's Changed * Python3.8 fixes by @ArthurCamara in https://github.com/zetaalphavector/RAGElo/pull/37   **Full Changelog**: https://github.com/zetaalphavector/RAGElo/compare/0.1.6...0.1.7 |
| `0.1.6` | 2024-07-02 | Low | ## What's Changed * Fix issue with RDNAM parsing of answer by @matprst in https://github.com/zetaalphavector/RAGElo/pull/32 * docs: update README.md by @eltociear in https://github.com/zetaalphavector/RAGElo/pull/33 * Elo Ranker returns dictionary with agents scores by @ArthurCamara in https://github.com/zetaalphavector/RAGElo/pull/34  ## New Contributors * @matprst made their first contribution in https://github.com/zetaalphavector/RAGElo/pull/32 * @eltociear made their first contributio |

## Dependency audit

- **Score**: 76/100
- **Total deps**: 10
- **Resolved**: 4
- **Unresolved**: 6
- **License conflicts**: 0
- **Warnings**: 6
- **Scanned**: 2026-05-04

## Citation

- HTML: https://www.freshcrate.ai/projects/RAGElo
- Markdown: https://www.freshcrate.ai/projects/RAGElo.md
- Dependencies JSON: https://www.freshcrate.ai/api/projects/RAGElo/deps

_Generated by freshcrate.ai. Indexes github releases for AI-agent ecosystem packages._