Show HN: InstinctFlash – Run 5B world-action models in real time on Jetson Thor

InstinctFlash is a full-source, high-performance serving framework that accelerates robotics world-action models (including 5B-class families) for real-time control on edge hardware such as Jetson Thor and desktop GPUs (RTX 4090/5090). The release bundles eight model families, vendor-specific acceleration kernels, and a unified Runtime that can run models with different precision tiers (BITEXACT, NUMERIC, FP8) and step schedules. Measured Jetson Thor results claim up to a 33.78× end-to-end latency improvement for LingBot-VA when combining FP8 and aggressive sampling changes; a published table gives per-model p50 prediction times and speedups across LingBot-VA, VLA variants, pi0.5, GR00T, Cosmos3 Edge/Nano and DreamZero. Benchmarks, reproduction recipes, and a native Thor CUDA backend are included.

The system uses pinned per-family Python environments and a bootstrap installer; a checkpoint can be served directly (instinctflash serve) and loaded in Python via Runtime.from_pretrained. Runtime exposes an episode/predict API for stepwise action inference and supports a msgpack-over-websocket protocol compatible with openpi clients. Tooling covers preflight (dry_run), smoke tests, visualization, checkpoint validation/certification, and paired evaluation (latency, action agreement, task success) for reproducible comparisons. Advanced options include dynamic step caching (DreamZero requires behavioral tier), shared BF16 fusion (NUMERIC), and explicit policies for FP8 and numerical compilation.