hn.today

Speeding up gearhash on ARM64 (2× faster)

sam.dev3 points0 comments
Screenshot of Speeding up gearhash on ARM64 (2× faster)

As of version 0.1.4, the gearhash Rust crate adds a NEON backend that automatically accelerates ARM64 (aarch64) builds and delivers roughly 2× throughput at typical chunk sizes used for content-defined chunking. Gearhash implements the GEAR rolling hash used by FastCDC; demand from real-world use (notably the Xet client) plus the rise of performant ARM64 hardware motivated optimizing beyond the original scalar path. The NEON path is backwards compatible and selected automatically, so updating the crate is sufficient to benefit from the speedups.

The speedups come from turning a latency-bound serial hash into a throughput-oriented SIMD kernel: unrolling multiple intermediate states, collapsing four byte loads into one unaligned 32-bit load with shifts, and combining four boundary tests inside the NEON vector unit to avoid multiple branches. Incremental changes moved performance from a direct port (0.92×) to 1.81× in 0.1.4. Benchmarks show mask density matters: for an 11-bit mask (≈2 KiB chunks) NEON is 1.80× (1981 → 3565 MB/s), for 16-bit (64 KiB) 2.14× (1999 → 4277 MB/s), and it plateaus near 2.17× at larger chunks; NEON becomes slower than scalar for very small average chunks (<≈350 bytes), so falling back to scalar on very sparse masks is a simple mitigation.

Read on sam.dev0 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Programming

Code Scans

Code Scans

Devin introduces Code Scans, a tool that helps teams identify and implement codebase improvements based on engineering goals. It uses parallel investigation and automates pull requests, saving time and enhancing software quality. (devin.ai)

Bend

Bend

Bend is a high-speed programming language that emphasizes correctness through proofs and laws, ensuring AI-generated code is error-free. It compiles to native code, runs parallel across multiple cores or GPUs, and verifies AI adherence to rules with rapid type checking and proof systems. (bend-lang.com)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.