Post · Benchmarks & research

Internal classifier field note

Matched-precision comparison against a private fine-tuned classifier.

Open x.com ↗

was pumped to get early access to @typesafe_ai's Jev, pointed it at an internal benchmark

Jev zero-shot was within ~5 pts of recall of our fine-tune at matched precision. cost-wise its roughly ~$70/mo for our full volume and sub-second latency;

so, fine-tuned qwen still beats

Sep 17, 2026Show the live post from X

How builders describe it

An internal benchmark report: Jev zero-shot came within about 5 points of recall of a fine-tuned model at matched precision, at roughly $70 a month.

i benchmarked it on an internal classifier, its getting eerily close (and cheaper!!) to beating fine-tuned qwen 9b

The decision Jev makes

Benchmark questions with known answers, to check accuracy and confidence.

Where it fits

Head-to-head tests, calibration studies and independent research into how well Jev decides, how fast, and at what cost. All 682 benchmarks & research projects →

Related projects