NVIDIA’s RTX PRO 4500 Blackwell Server Edition is a single-slot, passively cooled server GPU rated at 165W with 32GB GDDR7, full Blackwell features (fifth‑generation Tensor Cores with FP4, fourth‑gen RT cores), three NVENC/NVDEC engines, MIG support for two isolated 16GB instances, and PCIe 5.0 x16. It’s positioned as a right‑sized upgrade for brownfield data centers and edge deployments that need Blackwell performance without new chassis or heavy cooling, fitting common 2U servers but not 1U boxes that accepted the half‑height L4. The board uses a 16‑pin power connector, passive cooling that depends on chassis airflow, and trades display outputs for data‑center features like vGPU 20 and confidential computing.
Benchmarking with vLLM across multiple models and contexts shows the 4500 delivers L40S‑class inference at much lower power: at 512‑token input/output it matched or beat the L40S on FP8 and BF16 (e.g., 1,107 vs. 995 tokens/s on M3 14B FP8; 2,006 vs. 1,988 on M3 3B) while drawing ~47% of the L40S’s power. It produces roughly 2.8-3.6x the L4’s throughput at short context for ~2.3x the L4’s board power. Limits appear at large contexts: 32GB KV cache fills and throughput flattens at 8K input, trailing the L40S by 17-33%. NVFP4 performance is currently a software story - hardware FP4 peaks far higher in microbenchmarks but NVFP4 paths in vLLM didn’t scale with memory bandwidth. The RTX PRO 6000’s main advantage is memory capacity and bandwidth, roughly doubling 4500 throughput at short context and increasing gains at long context.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Open weight models now process over 71.5% of tokens and account for nearly 23% of AI Gateway spend, quadrupling their share since June. Spending on open weight models has increased more than four times in three months, reflecting rapid adoption and shifting AI usage patterns. (vercel.com)
An AI system has successfully designed 16 new viruses, including bacteriophages. This breakthrough raises concerns about potential biosecurity risks. (cnn.com)
Flux 3 Image Klein is a 5.6 billion parameter open weights model for text-to-image generation, editing, and multi-reference creation. It offers high-quality, ground-connected images with efficient performance on consumer GPUs, enabling local and on-device use. (bfl.ai)
Yum! Brands has implemented AI in its restaurants, such as automating drive-thru orders, which has improved employee retention by making jobs easier. Despite public criticism of AI, company leaders see its benefits in increasing workplace satisfaction and efficiency. (fortune.com)
Transformers function as sophisticated memory systems that recall information via self-attention, minimizing interference and boosting capacity. Different neural layers, from linear maps to self-attention, are explained as various forms of memory storage and retrieval. (alexwang.ai)
Granite Switch 4.1 integrates chat, decision-making, and speech-to-text functions into a single endpoint, enhancing efficiency and speed. It features models optimized for safety, decision tasks, and transcription, all with reduced GPU requirements. (huggingface.co)
Today's best Hacker News stories, summarized and screenshotted, one email a day.