Show HN: Strot – Compile an LLM's intent once, run decisions locally forever

STROT is an open-source "decision compiler" that uses an LLM once at compile time to synthesize a deterministic, self-routing local runtime that classifies, scores, and routes records as type-safe JSON. The compile step ingests an intent, schema, and samples, generates handlers and teacher labels, calibrates confidence, and emits a frozen Python artifact. At runtime the artifact projects each record, runs a compact decision model (default: 144M Julia‑1 on CPU, ~60 ms), applies a confidence gate that can abstain, and returns typed choices, scores, or booleans with probabilities and calibrated confidence. Two APIs exist: run() for routing+handler execution and decide() for per-record typed questions.

Benchmarks report substantial gains from compilation versus naive local inference (e.g., grounded routing 92.5% vs 74.8%; semantic text 86.7% vs 72.7%) with zero marginal LLM calls and ~60 ms latency per record. Key guarantees include content-addressed artifacts, golden-record promotion (must match known answers before serving), temperature scaling plus split conformal prediction to bound error rates, measured label scoring and distilled heads, and elimination of structural hallucination by restricting runtime outputs to selection and computation over a registry. Installation is pip-based with variants for local or hosted decision models; the project is MIT licensed and implements ideas from arXiv:2505.01636.