Jev vs. Kev: open-source Jev alternative tested side by side

Opper compared TypeSafe’s new decision model Jev with Kev 4B, an open reproduction by Jared Palmer, by hosting Kev in the EU and sending identical requests to the same Opper endpoint with only the model name changed. Tests used 362 items published after both releases (arXiv abstracts, recent Stack Exchange questions, and new GitHub issues), each sent as a single-question request. Accuracy was essentially tied across tasks (examples: arXiv 96.9% vs 95.0%, Stack Exchange 97.5% vs 98.3%, GitHub 95.1% vs 93.9%), latency was similar (Kev ~220 ms median, Jev ~275 ms), but Jev produced noticeably better calibration (its stated probabilities matched actual accuracy more closely).

The clearest operational difference was token accounting: Jev adds a fixed ~257-token overhead to every request (likely an internal prompt), so short one-line inputs cost far more on Jev than Kev (one-line request: 280 vs 23 tokens, ~12× cost). The overhead is per request, so batching questions helps Jev’s cost ratio. Kev 4B is an Apache‑2.0 fine-tune of Alibaba’s Qwen3.5-4B-Base and, despite no access to Jev’s internals, matched performance on straightforward classification but lagged on calibration, subtle meaning and long documents. Caveats include modest sample size, relatively easy fresh tasks, Kev’s prior exposure to many public datasets, and Jev’s unpublished training data. Benchmark code and test sets are available for reproduction.