Show HN: Flide: Make Jev-like decision models up to 9.5x faster

Flide is an inference engine for Jev-style decision models, which answer structured questions about a shared piece of context, such as choosing an action or judging whether something is urgent. Its central technique, exact state forking, processes the shared context once and branches the model’s state at the final token for each question. This avoids repeated computation and the extra delimiters used by multi-item scoring, which can change a model’s answers. Each question therefore gets the answer it would receive if asked on its own, with probabilities for the available options.

On one H200 GPU, Flide reports up to 9.4 times the speed of vLLM and SGLang for Qwen3.5-0.8B, with gains varying by model and workload; it also reports 2.4 times their throughput with 64 clients. Its accuracy is within 1.1 points of a float32 reference, while multi-item scoring shifts results by as much as 14.8 points. An optional batch-invariant mode makes outputs identical regardless of which other requests share a batch, at some speed cost. Flide supports a broad collection of decision-model families, as well as Qwen and Gemma models, with text, image, and in some cases video inputs. It exposes a Jev-compatible HTTP API, and includes session caching for agent workflows that reuse earlier context.