hn.today

We solved SQLite's single-writer limitation

marcobambini.substack.com5 points1 comments
Screenshot of We solved SQLite's single-writer limitation

This announces sqlite-multiwriter, a compatibility-first solution that removes SQLite’s single-writer bottleneck without changing SQLite itself or its on-disk format. It implements a custom SQLite Virtual File System (VFS) that gives each writer a private write-ahead log and an independent snapshot, then validates and publishes compatible commits with durable storage. An optional rebase mechanism can resolve some page-level conflicts when different rows on the same page are modified. The design meets four constraints: keep existing databases compatible, avoid modifying SQLite source, support multithreading and multiprocessing, and remain transparent to existing SQLite APIs and SQL.

Benchmarks show large gains for low-conflict workloads: with 16 concurrent threads, transactions/sec jumped from ~8.6k to ~49.3k (5.7x) and p99.9 latency fell from 157 ms to 2.08 ms; with 16 processes throughput rose to ~26k (3.0x) and p99.9 latency to 1.54 ms. Genuine conflicts still require retries and the project documents limitations; different workloads will produce different results. Released under Apache 2.0, the code, tests, and reproducible benchmarks are open-source and the authors invite testing and contributions, especially from teams building AI agents, local-first apps, and other high-concurrency systems.

Read on marcobambini.substack.com1 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Other

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.