As of version 0.1.4, the gearhash Rust crate adds a NEON backend that automatically accelerates ARM64 (aarch64) builds and delivers roughly 2× throughput at typical chunk sizes used for content-defined chunking. Gearhash implements the GEAR rolling hash used by FastCDC; demand from real-world use (notably the Xet client) plus the rise of performant ARM64 hardware motivated optimizing beyond the original scalar path. The NEON path is backwards compatible and selected automatically, so updating the crate is sufficient to benefit from the speedups.
The speedups come from turning a latency-bound serial hash into a throughput-oriented SIMD kernel: unrolling multiple intermediate states, collapsing four byte loads into one unaligned 32-bit load with shifts, and combining four boundary tests inside the NEON vector unit to avoid multiple branches. Incremental changes moved performance from a direct port (0.92×) to 1.81× in 0.1.4. Benchmarks show mask density matters: for an 11-bit mask (≈2 KiB chunks) NEON is 1.80× (1981 → 3565 MB/s), for 16-bit (64 KiB) 2.14× (1999 → 4277 MB/s), and it plateaus near 2.17× at larger chunks; NEON becomes slower than scalar for very small average chunks (<≈350 bytes), so falling back to scalar on very sparse masks is a simple mitigation.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.