Methodology at a glance
- Dataset
- EnterpriseRAG-Bench
- Primary metric
- Retrieval recall
- Sample
- 29-question sample including 10 basic questions
- Evaluation date
- 2026-08-19
- Baselines
- No model baseline claimed; results are reported against the benchmark's required-document labels
- Model and build versions
- Ask Poth current build evaluated with the official EnterpriseRAG-Bench framework
- Limitations
- Sample result, not the full 500-question set; retrieval recall does not measure final answer quality
- Artifacts
- Internal evaluation output (not public) summarized in this article; https://github.com/onyx-dot-app/EnterpriseRAG-Bench
Published August 19, 2026. Last updated August 20, 2026.
A grounded answer starts with retrieval. If the right documents never surface, no amount of reasoning saves the answer. This benchmark measures whether Ask Poth finds them.
What we measured
EnterpriseRAG-Bench is a benchmark from Onyx built to look like a real company's internal knowledge: roughly 500,000 synthetic documents across nine source types (Slack, Gmail, Linear, Google Drive, HubSpot, Fireflies, GitHub, Jira, and Confluence). The corpus is deliberately messy. It contains misfiled documents, near-duplicates, and conflicting information, because real company data does too.
Its central retrieval metric is recall: for each question, did the system surface the documents a correct answer needs?
The result
On a 29-question sample scored with the benchmark's own evaluation framework:
- 72.2% retrieval recall across the full sample.
- 90% retrieval recall on the basic question set (9 of 10 questions).
In plain terms: for close to three out of four questions over half a million noisy documents, Ask Poth pulled up everything a correct answer required, and on straightforward lookups it almost never missed.
Methodology and caveats
- Scored with EnterpriseRAG-Bench's official evaluation framework on a 29-question sample; the basic set within it is 10 questions.
- A 29-question sample is enough to show where the system is strong, not to settle decimal places. A run over the full 500-question set is next.
- Retrieval recall measures finding the evidence, not the finished answer. We publish our answer-quality numbers on this benchmark once we consider them worth your time; answer synthesis is where our current work is focused.
- The benchmark, corpus, and evaluation harness are public: EnterpriseRAG-Bench on GitHub.