Delta: Highly available, strongly consistent storage using chain replication (2022)

Delta is a purpose-built object storage service designed to provide a highly available, strongly consistent foundation for critical bootstrap and recovery workflows, trading latency and storage efficiency for simplicity and resilience. It exposes only four operations (put, get, delete, list), intentionally avoids filesystem semantics and aggressive optimization, and is positioned at the bottom of the infrastructure stack to minimize dependencies. Reliability and recoverability drive design decisions: Delta prioritizes predictable behavior under failure and straightforward operational recovery over throughput or storage savings.

The core mechanism is productionized chain replication: objects are partitioned into chains (typically four or more servers) chosen by a consistent hash; writes go to the head and are pipelined down to the tail, which acknowledges only after durable persistence, while reads are served from the tail to guarantee strong consistency. Compared with quorum-based replication, chain replication is less storage-efficient and yields higher write latency but can match or exceed fault tolerance (a chain of n nodes can tolerate up to n-2 failures) and avoids complex leader election. Buckets maintain authoritative chain-host mappings, chains are spread across failure domains and can be rebalanced horizontally, and failure handling relies on fail-stop detection via heartbeats and a two-vote suspicion rule with automated repair and resynchronization of returning hosts.