Repo · Routing & model choice
jev-certify
Finite-sample routing certificates and prediction-powered audits for TypeSafe Jev probabilities: conformal risk control for thresholds, PPI for cheap…
Open github.com ↗From the repository
Finite-sample guarantees for Jev (TypeSafe's System One). Conformal risk control turns calibrated probabilities into certified routing thresholds; prediction-powered inference audits them. 2,412 decisions on CLINC150 for $0.23 — including the shift and prevalence cases where the guarantee breaks.
clinc150conformal-predictiondecision-modelsintent-classificationjevllm-routingopenrouterprediction-powered-inferencerisk-controlsystem-onetypesafeuncertainty-quantification
How builders describe it
Finite-sample routing certificates and prediction-powered audits for TypeSafe Jev probabilities: conformal risk control for thresholds, PPI for cheap live audits, plus measured failure modes.
jev-certify turns Jev's probabilities into a bounded claim: conformal risk control picks a routing threshold with a finite-sample guarantee on silently misrouted queries, and prediction-powered inference audits it with about 40 hand labels. Measured on 2,412 Jev 1.13 decisions over CLINC150.
The decision Jev makes
Pick an agent, a model, a skill from a catalog, or a function and its arguments.
Where it fits
Deciding which agent, model, skill or function should handle a request. Jev is a natural router: the options are fixed and the confidence tells you when to fall back. All 348 routing & model choice projects →
Related projects
Routing & model choiceAgentNetwork-jevEvent-driven multi-agent network where TypeSafe Jev observes live network state and assigns pending work to idle general-purpose agents each time one…GitHub · ★ 3 · Kelly-Leeee
Routing & model choicemodelrudderFree local community preview that adds model routing to the native Codex terminal: your own TypeSafe Jev key classifies each request so the strongest…GitHub · ★ 2 · cybrking
Routing & model choicejev-orcMy JEV Orchestrator to supply my needs.GitHub · ★ 2 · hiyoku
Routing & model choicejev-gateway (dragpk247)Sub-300ms intelligent LLM router with Zero-Leak privacy sandbox, Obsidian RAG, and multi-tier local GPU & cloud offloading.GitHub · ★ 1 · dragpk247
Routing & model choicepbi-orchestration-jevClaude Code skill that turns multi-file implementation into Product Backlog Items for sub-agents (Codex, Antigravity, Claude Code Agent), with…GitHub · ★ 1 · reeschan
Routing & model choiceAI Model PickerNative macOS assistant: describe a task, review rule-ranked app/model/effort choices, and open or prepare the task in the client you already use;…GitHub · Gavinluo19