Benchmarks / Kai / Choosing among many options
Kai · CapabilityChoosing among many options
Accuracy at each number of listed options, every option scored, the same cases put to both models.
Latest
History
3 results, newest first. Superseded runs stay, so a revision's progress can be read.
| Measured | Subject | Metric | Value | Baseline | Result | Status |
|---|---|---|---|---|---|---|
| Oct 16, 2026 | kai-1.4 · c1856a77 · 4 options | Accuracy | 94.3% | Jev 90.5% | Win | Live · Hanzo-measured |
| Oct 16, 2026 | kai-1.4 · c1856a77 · 16 options | Accuracy | 92.0% | Jev 85.3% | Win | Live · Hanzo-measured |
| Oct 16, 2026 | kai-1.4 · c1856a77 · 77 options | Accuracy | 88.0% | Jev 84.3% | Win | Live · Hanzo-measured |
Conditions
choice sizes: 4, 16, 77every option scoredJev: typesafe/jev-1.13, measured Sep 27, 2026Hardware
CPU F32-Q8_0 on evo
Source
research run cap.cardinality-cases-kai-1.4.c1856a77-kai
Reproduce
bench cap --harness . --who kai,jev --kai <checkpoint> # in hanzoai/benchmarks/decision
More Kai benchmarks
AG News · DAIR Emotion · Banking77 · Support triage · Email spam · Phishing · Toxicity · RAG relevance · Model routing · Typed decisions · MASSIVE · Latency of one decision
Every result for Choosing among many options on the shelf · How these are measured