How many GPUs is 1M/B/T tokens?

A tokens-to-GPUs calculator turns a token-volume target into a hardware plan by combining a chosen model, GPU type, serving time window, token mix (input vs output), average utilization and per-GPU throughput estimates. It uses presets (measured when available, estimated otherwise) for tokens-per-second per GPU, allows swapping GPU reference types, and exposes advanced assumptions - input-token cost, multi‑GPU efficiency and memory fit - so planners can replace defaults with their own measurements. The tool shows a headline GPU count, a plausible low/high range, per‑GPU detail, a sensitivity chart that ranks which assumptions move the result most, and clear limits (rough 2× planning error, latency-dependence, no prompt caching or speculative decoding, and theoretical values for some older GPUs).

Using the calculator: converting 1 trillion tokens in 30 days for Llama 3.3 70B with a 47.5% output token share and 50% utilization yields 367 H100 GPUs (range 211-853). Equivalent base estimates for other hardware are ~815 A100, ~2,547 V100 (theoretical), and ~92 B200 single GPUs; in 8‑GPU nodes those are ~46 8×H100 nodes or ~12 8×B200 nodes. The sensitivity analysis shows per‑GPU speed, average utilization and input-token cost drive the result most, so measuring model throughput and choosing utilization targets are the priority actions for accurate capacity planning.