Show HN: Ru-Jev – an open model that scores caller-supplied options on CPU (MIT)

Ru-Jev v1.3 is an open, Russian-first model for choosing among caller-supplied answers rather than generating text. It handles yes/no decisions, selections from a list, and ordinal ratings, returning probabilities as well as a choice. Built by fine-tuning a 307-million-parameter mmBERT encoder with a separate scoring head, it is intended to run on CPU and within private infrastructure. The model is released under MIT and trained on open, human-labelled data; its creators say it was developed independently and not trained on outputs from a commercial decision API. A key integration warning is that standard Transformers zero-shot pipelines silently load a randomly initialized classification head, producing unstable, misleading scores. Users must load the model in the specified way.

Version 1.3 addresses a weakness in v1.1: performance collapsed when familiar category labels were renamed, while confidence remained too high. Training examples now vary option wording, encouraging the model to interpret labels rather than memorize exact strings. Tests on 1,000 routing questions compare five- and eighteen-option choices. On held-out wordings, gains include 0.235 and 0.142 accuracy for jargon on a strict label set, and 0.166 and 0.050 for office slang, respectively. The improvement is strongest for meaningful words, including borrowed terms; it does not reliably extend to abbreviations or opaque codes, which remain difficult. The evaluation distinguishes familiar wording controls from genuinely unseen forms and flags overlap between some test labels and training options, so the reported slices should not be compared as if they used identical labels.