Llama.cpp: 6.5x faster prompt processing on Arm with a NEON kernel

A patch adds an Arm NEON kernel using the SDOT instruction to llama.cpp’s tiled low-bit matrix multiplication. Without it, several quantized formats - especially IQ formats - use an Arm fallback that repeatedly unpacks weights during prompt processing. The kernel subtracts each format’s bias as weights are loaded, allowing exact integer dot products, and interleaves prompt rows so each instruction accumulates results for multiple rows without a later reduction. The change is 164 lines across two files and is enabled on AArch64 CPUs with the dot-product extension.

On three AWS Graviton generations, tests with Qwen2.5 1.5B, four threads, and a 512-token prompt show prompt-processing gains from roughly 1.6× to 6.5× for affected formats; the largest is IQ3_M on Neoverse N1. Q3_K_M and Q2_K improve by about 1.7-2.1×, while formats that already had Arm kernels show essentially no change. Generation speed is unchanged. Correctness tests and randomized tests passed on all three chips, but the study covers only one small model and prompt setup. It does not measure phones, Apple Silicon, higher thread counts, or real-prompt model output, and the patch has not been submitted upstream. Results also remain below the x86 reference, and the cause of that gap is not analyzed.