AWS Emulator and the engineer's job when agents write the code

AWS Emulator provides coding agents with an offline, local environment that mimics more than 200 AWS services, so they can test applications without repeatedly using the live cloud. Its design draws on earlier work testing Alf, a Paxos-based distributed lock service that underpins systems including S3 and DynamoDB. Because failures in distributed systems depend on precise event sequences, conventional large-scale and chaos testing could not reliably reproduce every problem. The Alf team instead ran the service in a controlled simulation, where they could inject packet loss, trigger specific event orderings, and combine testing with formal methods such as TLA+. This exposed many concurrency bugs before deployment, though real-world testing still found issues simulations could not capture.

Building such test environments by hand is impractical across AWS’s more than 200 services, so agents were tasked with creating an emulator from public API specifications and tests checked against real AWS. Those fidelity tests capture actual service behavior; agents use them both to build the emulator and to reveal where it differs from the cloud. The project now covers over 400 API models, runs free and offline, and its code is released into the public domain. The central argument is that engineers’ work shifts from writing most code to setting goals and establishing the specifications, tests, security reviews, and merge gates that determine whether agents’ work is sound. A replication-engine rebuild took about two hours once an agent had a specification and test environment, compared with four months of original development.