Contrastive Language Models: A Fast, Generalizable System One Model

Contrastive Language Models (CLM) present a "system one" approach that trains separate state and action encoders with a contrastive (InfoNCE) objective so states are pulled toward their ground-truth actions and pushed away from others. The released CLM-8B model uses frozen LLM backbones plus 20M-parameter projection heads, with embeddings cached and reused to make inference cheap: one embedding per novel text and a dot product per candidate action. The training pipeline is staged - pre-trained on 60M Nemotron Q&A pairs, mid-trained on 30M synthetic hard negatives, and post-trained on 1M agentic trajectories - with a bidirectional InfoNCE loss (extended to include hard negatives) and reported power-law scaling of test loss with compute, model and data size.

Empirically CLM-8B matches a strong baseline (Jev) across computer-use, gaming and tool-calling tasks while running up to 9× faster, with the biggest gains when many candidates exist or actions are reused. As a verifier, lightweight fine-tuning yields state-of-the-art results on agentic coding benchmarks (Terminal-Bench 2.1: 87.6%, DeepSWE: 81.6%) while running 4.1-5.7× faster than Jev. The repository provides an easy install, a clm-serve API and playground, an in-process Engine for ranking and typed-question primitives (Noul/Choice/Score), and instructions to fine-tune or reproduce evaluations.