hn.today

Bringing PostgreSQL Closer to the Edge at Cloudflare

infoq.com5 points0 comments
Screenshot of Bringing PostgreSQL Closer to the Edge at Cloudflare

Cloudflare describes how it moved relational data closer to users by operating distributed PostgreSQL clusters across its global edge. The control plane stores customer configuration and transactional state - DNS changes, firewall rules, billing entitlements - while handling extreme throughput (hundreds of millions of HTTP requests and tens of millions of row operations per second across ~250 PoPs and >50 TB of data). The stack runs on bare metal for fine-grained tuning, with PgBouncer connection pools fronted by an anycast BGP layer to route reads locally and forward writes to a primary region. High availability relies on HAProxy, stolon and etcd for leadership and replication to many read replicas; PostgreSQL is used for stored procedures and as an outbox queue, with a daemon pushing events into Kafka so latency-critical edge services can bypass direct DB access.

The core argument is that colocating storage and compute at the edge delivers large latency benefits but demands hard tradeoffs around consistency, replication lag, and degraded-mode behavior. Replication lag in a distributed replicated system is a dominant operational challenge - handling degraded states is harder than handling outright failures - so Cloudflare uses cross-region distribution for resilience and fast failovers and designs around eventual consistency where necessary. Running on-premise hardware gives control but prevents instant autoscaling, and the company contends that embedded, colocated relational databases at the edge are the practical path forward when paired with careful architecture and operational practices.

Read on infoq.com0 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Web

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.