Repo · Benchmarks & research
jev-align (Sutro)
Turns human labels into calibrated Jev classifiers with GEPA.
Open github.com ↗From the repository
Build calibrated AI Functions from human feedback using Jev and GEPA.
active-learningclassificationcligepahuman-in-the-loopjevprompt-optimizationtypesafe-ai
How builders describe it
Active-learning CLI that uses human labels and GEPA to improve Jev decision definitions.
Experimental CLI (jeva / jev-align) that builds calibrated AI Functions with TypeSafe Jev evaluations plus GEPA optimization from human labels—binary, multiclass, multilabel, or score tasks.
An experimental Sutro CLI that evaluates CSV, Parquet, and JSONL data with Jev, asks you to label ambiguous and random audit rows, and uses GEPA to propose improved function definitions.
Jev is cool, but like any foundation model it needs to be calibrated to your decision criteria. We launched jev-align: an open-source CLI to quickly teach Jev what good and bad looks like using GEPA. Try it out! Via DAIR.AI Jev Field Notes (research paraphrase, snapshot 2026-09-20).
The decision Jev makes
Benchmark questions with known answers, to check accuracy and confidence.
Where it fits
Head-to-head tests, calibration studies and independent research into how well Jev decides, how fast, and at what cost. All 682 benchmarks & research projects →
Related projects
Benchmarks & researchMicaA 4B decision model: calibrated probabilities for yes/no, choice and score questions over long inputs.GitHub · ★ 37 · akivet
Benchmarks & researchawesome-jev-papers📄 Research papers on Jev, TypeSafe's System One model, and the open models built in its shape.GitHub · ★ 5 · OmniJev
Benchmarks & researchjev-typesafe-real-financial-use-casesFifty real-world financial use cases for TypeSafe's Jev model: typed, structured LLM answers over ledgers, fraud, portfolios, trades and filings,…GitHub · ★ 3 · IslamBaraka90
Benchmarks & researchjev-healthcare-labOpen experiment comparing TypeSafe Jev vs DeepSeek on 96 healthcare tasks across 12 scenarios (quality, latency, cost) with per-task data and…GitHub · JuneYaooo
Benchmarks & researchtypesafe-jev-ecosystemExplore the TypeSafe Jev ecosystem: open-source integrations, compatible decision models, SDKs, agent tools, and research.GitHub · ★ 4 · VoltAgent
Benchmarks & researchBeelineBenchHow well do decision models plan across a multi-step problem?GitHub · ★ 3 · kyle-pena-nlp