hn.today

Tin: full-text search for Postgres

planetscale.com143 points62 comments
Screenshot of Tin: full-text search for Postgres

TIN (Text INdex) is a Postgres extension that provides a full-featured, high-performance full-text search index designed to support real application needs: Boolean, phrase and span queries; fuzzy, wildcard and regular-expression term matching; case and accent folding; COUNT() queries and BM25 top-k ranking; and correct behavior with joins, complex WHERE clauses, continuous updates, replication, backups and transaction visibility. It aims to replace existing Postgres text indexes that lacked one or more of these capabilities while remaining tightly integrated with Postgres semantics.

Benchmarks on an 85 GB Stack Exchange export (150M documents) run inside an 8‑vCPU/32 GB Postgres container show TIN outperforming ParadeDB, pg_textsearch and Postgres GIN across build and query workloads. TIN built a 50.7 GB index in 8m10s using 32 GB RAM; competitors needed more RAM and longer builds. For mixed top‑10 BM25 queries TIN served ~25× the QPS of ParadeDB with ~26× lower p99 latency; for conjunction/phrase top‑10 queries it delivered ~10× ParadeDB and ~541× GIN QPS with much lower latencies; under disjunction queries with 1,000 concurrent updates/sec TIN handled ~36-57× the read QPS of pg_textsearch/ParadeDB and completed ≈270,279 updates in ten minutes (vs ≈185,584 and 735). TIN also reads far fewer MB/query, reducing I/O and cache impact; GIN frequently ran out of memory on disjunction workloads.

Read on planetscale.com62 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Web

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.