CalibratedDecisions.

Post · Documents & research

A thousand papers sorted, then graded

Jev does the 24-topic sort, Opus 5 grades the labels.

Open x.com ↗
Stanislav Sorokin @stas_sorokin_

1,000 AI papers sorted into 24 topics for $0.0585. Then Opus 5 graded the labels.

@nutlope's Jev paper map went viral, but the pipeline never shipped and the eval was "still running". So I rebuilt both and opened them.

The first judge run came back empty: Opus spent its whole

Image from the post

The decision Jev makes

Assign a topic, choose a document category, or answer whether a passage supports a citation.

Where it fits

Sorting, labelling and checking documents is classic decision work: topic labels for papers, categories for files, and whether a source actually supports a claim. All 45 documents & research projects →

Related projects