Benchmarks / Kai / Typed decisions
Kai · Decision harnessTyped decisions
Kai and Jev on the same 2,000 questions of Typed decisions, one run per Kai revision.
Latest
History
5 results, newest first. Superseded runs stay, so a revision's progress can be read.
| Measured | Subject | Metric | Value | Baseline | Result | Status |
|---|---|---|---|---|---|---|
| Oct 16, 2026 | kai-1.4 · c1856a77 | Accuracy | 74.6% | Jev 73.6% | Win | Live · Hanzo-measured |
| Oct 3, 2026 | kai-1 · 0834a74f | Accuracy | 75.9% | Jev 73.6% | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a8 · 0cfeef05 | Accuracy | 76.1% | Jev 73.6% | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a7 · 0834a74f | Accuracy | 75.9% | Jev 73.6% | Win | Superseded · Hanzo-measured |
| Sep 28, 2026 | Kai a6 · a211cc70 | Accuracy | 75.0% | Jev 73.6% | Win | Superseded · Hanzo-measured |
Conditions
split: harness2,000 questions, 2,000 answereddataset dde140a63fc56 Kai revisions measuredJev: typesafe/jev-1.13-20260917, measured Sep 25, 2026Hardware
CPU AS SERVED
Source
research run typed_decisions-harness-kai-1.4.c1856a77-kai
Reproduce
bench score --harness . --kai results/kai/preds.json.gz --out results/scores.json # in hanzoai/benchmarks/decision @ 5f616dd
More Kai benchmarks
AG News · DAIR Emotion · Banking77 · Support triage · Email spam · Phishing · Toxicity · RAG relevance · Model routing · MASSIVE · Choosing among many options · Latency of one decision
Every result for Typed decisions on the shelf · How these are measured