4-Bit Rotational Quantization: -45% RAM, <1% recall drop vs. TurboQuant

Weaviate 1.39 extends Rotational Quantization (RQ) to 4-bit codes and delivers multiple system-level improvements - faster FWHT random rotations via SIMD, optimized encode kernels, SIMD nibble distance functions, and architecture-specific byte-dot instructions - yielding large speedups (FWHT 3.1-6.1×; RQ8 encoding up to 3.8×; RQ4 encoding in ~5.6-8.1 μs across CPUs). Distance kernels now run in the low tens of nanoseconds and efficient prefetching adds 7-11% query throughput and ~12% faster imports on large HNSW indices. Centering (subtracting a fitted mean) and storing the two largest rotated coordinates exactly further recover recall at 4 bits.

Results show that RQ4 (and a centered variant RQ4c) achieves high brute-force recall@10 across diverse datasets (e.g., dbpedia, DPR, SIFT, GloVe) with RQ4c often within a few percentage points of RQ8, while reducing heap use from 4.2 GiB (RQ8 in 1.38) to 2.3 GiB (RQ4/4c in 1.39) - about a 45% memory reduction. Imports are faster (RQ8 16% faster than before; RQ4 ~32-37% faster than the older baseline). Recall remains stable from 1M to 250M vectors, and the centered mean can be fit with ~10k samples with negligible loss. A head-to-head comparison reports under a 1% recall drop versus TurboQuant.