What if Jev spoke Arrow?

Jev is TypeSafe AI’s model that converts natural language plus application state into typed decisions, returning selected options, per-option probabilities, and confidence measures. The write-up explains how Jev enables practical probabilistic patterns - speculative fan-out, confidence-gated routing, composite scoring, and intent routing - that let code treat AI outputs as first-class, structured signals. To integrate those signals into high-performance data stacks without the JSON conversion cost, an Apache Arrow schema was designed mapping Jev’s three answer shapes (Choice, Noul, Score) onto Arrow types and metadata. Labels and legends live in schema metadata, predictions and probability vectors live in fixed-size buffers, and Arrow extension types attach semantic meaning so consumers can reconstruct typed answers without copying.

Because the TypeSafe API lacks a native bulk endpoint, Columnar built Jevaro, a Python proxy plus JavaScript client that accepts many states with a shared questions map, issues concurrent per-state Jev calls, and streams an Arrow IPC result preserving input order. Throughput tuning used HTTP/2 connection reuse, a sliding window of asynchronous requests, and SDK retries; a 10,000-message benchmark returned all rows in 21.5 seconds (~464 states/sec) at an estimated $0.20 for 30,000 answers. Jevaro demonstrates practical integration and cost-effectiveness but still pays per-state HTTP/JSON overhead; a native bulk Arrow output from TypeSafe would remove that bottleneck and fold Jev directly into Arrow-native analytics pipelines.