A single function Jev-like wrapper for LLMs, including vision models

This describes a lightweight Jev-style wrapper that reads LLM token probabilities (logprobs) to turn models into quick, structured scorers. The core trick is prompting the model to emit a single-token answer (e.g., a letter) and requesting logprobs/top_logprobs so the wrapper can interpret alternative-token scores as option probabilities. That approach supports choice questions, boolean (named "noul") questions, and ordinal scores, and can be KV-cached to avoid recomputing shared state. The same mechanism is extended to multimodal models by adding an attachments field for images: webcam frames are encoded as base64 JPEGs, sent along with the JSON state, and the model returns per-question probabilities. On an RTX 3090 with Gemma 4 12B via llama.cpp the example gets roughly 1 FPS for three questions per frame; using OpenAI gpt-6-luna measured about 0.2 FPS.

The included standalone Python example shows how images are loaded or encoded, how questions are declared (person, plant, setting, light), and how the wrapper maps tokens to options, normalizes weights, checks for omitted options, and returns a winning choice plus probability distributions or expected ordinal scores. It handles API differences between llama.cpp (Chat Completions) and OpenAI (Responses), and the post supplies commands to download Gemma and run a local llama server. The author highlights that specialist vision models are faster but this method trades efficiency for flexible, text-described conditions.