Bypassing the AVX2 Tax,Part 4:ONNX CPU EP on Ryzen 7430U(Vs OpenVINO,DirectML)

The tests compare ONNX Runtime’s CPU execution provider (EP), OpenVINO, and DirectML on a Firebat R3 mini PC with a Ryzen 5 7430U, 12 threads, and integrated Radeon graphics. The Fruit Inspector pipeline captures NV12 video on the CPU, runs YOLOX-nano at 640×640 to detect and track fruit, processes a 256×256 region of interest, then classifies it with a LightGBM ONNX model using 44 features. The CPU EP runs inference in about 6.8-7.5 ms, or roughly 77-88 frames per second. OpenVINO’s AUTO mode reaches about 78-80 fps, but its bundled GPU support targets Intel rather than Radeon. DirectML takes about 11 ms per inference: for this small model, D3D12 and GPU-dispatch overhead outweigh the benefit of the iGPU. The CPU EP is therefore the fastest choice on this AMD system.

The Windows release prioritizes a simple deployment for factory-floor PCs: one folder containing the executable, DLLs, and models, with no Python or CUDA installation. The public demo includes only the CPU library; OpenVINO and DirectML remain in the development tree, though users can switch providers through settings without rebuilding. OpenVINO is excluded to reduce archive size and operator complexity, while DirectML should be adopted only after benchmarking. The broader comparison - about 124-132 fps on an i5-11400, 77-88 on the Ryzen, and 11-15 on an older i3-2120 without AVX2 - underscores that newer SIMD support or an iGPU does not automatically make inference faster.