To Make, or to Buy: The Economics of Decision Models

Explains a concrete framework for using decision-focused models (exemplified by Jev) that emit structured outputs - choice (probability distribution over options), score (probability-weighted rating), and "noul" (probability a statement is true). Emphasizes practical advantages: predefined answer spaces, per-answer odds that enable arithmetic reasoning, negligible marginal cost for output tokens, and low latency. Demonstrates a JSON request/response to show how such models report top probabilities, confidence, and legends, and argues these properties make decision models a cheap, fast layer for triaging routine cases before invoking full LLMs or humans.

Proposes a decision hierarchy analogous to computer memory layers: fast rule caches at the top, decision models in the middle, LLMs below, and humans at the bottom; each level handles inputs it is confident about and forwards the rest. Applies Chow’s rule to decide whether to answer or abstain: answer only if expected cost of answering (1 − p_max) is less than the cost of delegating (NX_err + NX_price/NX_errcost). Uses a numerical example (p_max=0.6, NX_err=0.05, NX_price=$0.01, NX_errcost=$2) to show delegation is cheaper. Draws two implications: when errors are costly, accuracy dominates price; when errors are cheap, price matters. Notes real-world caveats - confident-but-wrong decisions and unequal mistake costs - and suggests mitigations like random audits and weighting error costs by output category.