V8 evaluation suite
Measure retrieval and routing instead of trusting demos.
The benchmark suite contains judged queries with expected intents topics and relevant evidence. V8 reports classifier accuracy MRR and recall@5 so regressions are visible.
6cases
50%intent accuracy
0.00MRR
0%recall@5
| query | intent | topic | first relevant rank |
|---|---|---|---|
| compare q learning and contextual bandits | compare | reinforcement-learning | miss |
| explain bm25 for rag | definition | nlp-retrieval | miss |
| how do i debug a prisma database connection | debug | data-science | miss |
| build a reinforcement learning study plan | howto | reinforcement-learning | miss |
| what is a transformer architecture | project | deep-learning | miss |
| how does tf idf represent text | howto | nlp-retrieval | miss |