CalibratedDecisions.

Repo · Benchmarks & research

open-alternative-jev

Typed, calibrated decisions from any open-weights model, on your own GPU.

Open github.com ↗

From the repository

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with honest benchmarks

calibrationclassificationjevjev-alternativellmlogprobsopen-jevopen-source-jevopen-weightsqwenstructured-decisionsstructured-outputsystem-onesystem-one-modeltransformerstyped-decisionstypesafetypesafe-jevvllm

How builders describe it

Open alternative to Jev: typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with benchmarks.
A Python research library that extracts typed choices and probability distributions from open-weight language models, comparing packed questions with separate inference and optional calibration.
An Apache-licensed Python library that reads option-token probabilities from open-weight models through Transformers or vLLM, giving Jev-style typed decisions in one forward pass without an HTTP server.

The decision Jev makes

Benchmark questions with known answers, to check accuracy and confidence.

Where it fits

Head-to-head tests, calibration studies and independent research into how well Jev decides, how fast, and at what cost. All 682 benchmarks & research projects →

Related projects