A Million Agents Is a Distributed System Problem

The piece argues that large-scale collections of AI agents are literally distributed systems and must be treated like processes rather than immortal minds. Practical limits - context windows, compute, memory, money - break long-lived-agent thinking. Systems research already converges on OS-like solutions: AIOS turns agent actions into system calls with scheduling and context snapshots and reports up to 2.1× speedups; LLM-as-Scheduler cuts token use by 43% and latency by >36% for modest accuracy loss. Controlled experiments show coordination is the core cost: a Google Research study across 180 configurations found multi-agent setups worsened strictly sequential planning by 39-70% while centralized coordination boosted parallel tasks up to 80.9%. Parallel, uncoordinated workers amplified errors 17.2×; a centralized orchestrator reduced that to 4.4×. Silo-Bench (ACL 2026) ran 1,620 experiments with 2-100 agents and found success rates falling with scale - hard global-shuffle tasks hit 0% success at 50 agents and average performance dropped from 61% (2 agents) to 18% (100 agents).

The operational conclusion is concrete: durable state, scheduling, failure detection, leader election, ownership, checkpoints, backpressure, and observability matter more than keeping any single agent alive. Centralized orchestration often helps but creates bottlenecks and single points of failure; decentralized routing and emergent coordination are proposed alternatives, yet both require classic distributed primitives. The right infrastructure schedules and recovers ephemeral intelligent workers, persisting goals, plans, and artifacts so new agents can pick up work - scaling millions of agents is a distributed-systems engineering problem, not an AI algorithmic one.