CalibratedDecisions.

Repo · Benchmarks & research

open-jev-typed-decision-engine

Open reproduction of TypeSafe Jev: a 150M typed decision engine (noul/choice/score in one non-autoregressive pass, calibrated confidence).

Open github.com ↗

From the repository

Open reproduction of TypeSafe Jev: a 150M typed decision engine (noul/choice/score in one non-autoregressive pass, calibrated confidence). 0.697 vs Jev's 0.727, 2.5x better calibrated, 4x faster, free. Trains on a Colab T4 in 30 min.

agent-observabilitycalibrationcolabdecision-modelexpected-calibration-errorhuggingfacejevllm-alternativellm-routingmodernbertnon-autoregressiveonnxpytorchstructured-outputstext-classificationtyped-decisionstypesafe-aizero-cost-inference

How builders describe it

Open reproduction of TypeSafe Jev: a 150M typed decision engine (noul/choice/score in one non-autoregressive pass, calibrated confidence). 0.697 vs Jev's 0.727, 2.5x better calibrated, 4x faster, free. Trains on a Colab T4 in 30 min.
Open 150M encoder that reproduces the typed noul, choice, and score interface in one forward pass, trainable on a free GPU and runnable locally for nothing.

The decision Jev makes

Benchmark questions with known answers, to check accuracy and confidence.

Where it fits

Head-to-head tests, calibration studies and independent research into how well Jev decides, how fast, and at what cost. All 682 benchmarks & research projects →

Related projects