CalibratedDecisions.

Post · Agents & browsers

Computer use without screenshots

On-device segmentation and OCR turn the screen into text for Jev.

Open x.com ↗
Milind S @milindlabs

Okay so Jev can actually do computer use really well

Without any screenshots, or LLMs and no Pixels leave my mac

I dont even read the Dom elements

A local CoreML model segments every button and UI element on screen.

On-device OCR reads the labels. That text is all Jev gets.

Video from the post

How builders describe it

Computer use with Jev and local vision model! Its so goddamn fast

The decision Jev makes

Pick the next click, field or navigation from the legal actions on screen.

Where it fits

Browser agents need to choose an action on every step. With Jev each step is a choice from the actions actually available on the page, which keeps agents fast and on the rails. All 299 agents & browsers projects →

Related projects