Shlemiel the Painter Is Back: How Zig's Libc Rewrite Made SQLite 12,000× Slower

A Rust-based chess analysis service began taking minutes to insert roughly 40,000 JSON records into SQLite, despite storage normally being expected to take milliseconds. The CPU-bound slowdown appeared only in Linux builds: SQLite was compiled using Zig’s cross-platform C compiler, while the developer’s Mac build did not use Zig. The cause was a rewrite of several C string functions in Zig’s standard-library replacement. In particular, the rewritten strspn measured the full remaining string length each time it was called. SQLite’s tokenizer repeatedly calls such functions as it advances through text, so each step rescanned a shrinking suffix. Tokenizing a 1.2 MB input was about 12,000 times slower, and the cost grew quadratically with input size.

The problem illustrates the “Shlemiel the Painter” trap: a locally simple operation becomes ruinously expensive when repeated, because it redoes work instead of carrying forward useful state. Similar issues affect rewrites of strcspn, strpbrk, and strstr. The developer first reduced the cost by stripping whitespace from JSON before passing it to SQLite, then moved Linux builds to Linux containers to avoid the affected toolchain. The account emphasizes that performance failures can hide in basic library routines and emerge only under realistic workloads; profiling exposed the unexpected bottleneck. It also recalls an earlier Firefox string-concatenation problem, where repeated allocation and copying hurt benchmark performance until buffer over-allocation reduced the repeated work. The broader lesson is that fundamental algorithmic complexity still matters, even in modern tools and seemingly trivial code.