Zerodha built Blobasaur to replace SSDB, a single-node blob store that had become an unmaintained operational risk and a scaling constraint. Its trading platform stores users’ customizable layouts, chart settings, and other preferences as opaque JSON blobs, from a few kilobytes to several megabytes, with thousands of read and write requests per second. Existing alternatives either performed poorly on large blobs, carried substantial operating complexity or cost, or lacked needs such as exact per-key expiry. Blobasaur is an open-source, Rust-based server that uses SQLite, speaks the Redis protocol, and supports key and hash operations with TTLs. That compatibility lets existing services switch without changing their Redis clients; the team accepts relaxed durability because these preferences are not financially critical.
Each node partitions keys across multiple SQLite files using consistent hashing. A dedicated writer per shard batches queued writes into transactions, avoiding SQLite’s single-writer bottleneck, while pooled read-only connections serve concurrent reads. Blobasaur compresses payloads before storage, primarily with zstd; in one hour of production hash traffic, the average 63.6 KiB payload became 5.5 KiB, an approximately elevenfold reduction. In a benchmark with 64 concurrent connections and 64 KiB payloads, increasing from one shard to 16 raised write-only throughput from 7,740 to 25,324 operations per second and mixed read/write throughput from 14,227 to 29,538. Read-only throughput changed little, consistent with SQLite WAL’s support for concurrent readers. The results support SQLite as a compact, economical storage engine when combined with compression and sharding.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.