Nvidia Vera's System Architecture: Pushing Bandwidth

Nvidia Vera is a high-performance Arm server chip built around beefy Olympus cores and a second-generation Scalable Coherency Fabric (SCF) mesh that stitches cores, caches, LPDDR5X memory controllers, PCIe and NVLink together. SCF uses coherency switch nodes that each host a pair of cores, memory switch nodes that feed a 1024‑bit LPDDR5X‑9600 interface (about 1.228 TB/s), and bridge nodes for 88 PCIe lanes (≈665.5 GB/s). Memory is field-serviceable via SOCAMM2 modules. Measured local DRAM latency is comparable to DDR5 and remains under ~200 ns on pointer-chase tests even under full bandwidth load, avoiding the sharp latency rise seen on many platforms. Cross-socket NVLink C2C provides huge throughput (reads ~587 GB/s, bi‑directional ~780 GB/s) but exhibits higher latency and variable behavior depending on offered bandwidth.

Caching and core behavior trade bandwidth for core count: a 164 MB non‑inclusive L3 plus 2 MB private L2s yields up to ~340 MB effective capacity and ~3.35 TB/s L3 read bandwidth (≈33.1 GB/s/core under full contention; single-core peaks ~88.5 GB/s). Intra-socket core‑to‑core latency averages ~148.5 ns (SMT <10 ns), while cross‑socket coherency and core‑to‑core latencies are high (~309-660 ns). SPEC CPU2026 shows very strong per‑core throughput and decisive wins in DRAM‑bound FP workloads versus higher‑core competitors like Graviton 5, but lower overall socket throughput where core count matters. Vera therefore prioritizes per‑thread performance and massive memory bandwidth, making it compelling for latency‑sensitive and memory‑heavy server workloads.