RC RANDOM CHAOS

pgrust rewrites Postgres in Rust, claims 300x analytics speedup over the original

· via Hacker News

Original source

Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD

Hacker News →

pgrust, a Rust reimplementation of Postgres, has shipped version 0.2 with a rebuilt query engine that its authors say makes analytical queries up to 300x faster than stock Postgres — fast enough, they claim, to edge out ClickHouse on the ClickBench benchmark while still running 30% faster than Postgres on OLTP workloads. Roughly 10x of that 300x gain comes from the query engine alone. The core argument is architectural: Postgres was designed in the 1980s when disk I/O dominated performance, but with datasets now fitting in RAM, NVMe storage that’s orders of magnitude faster than spinning disks, and analytics workloads that scan data in bulk, the real bottleneck has shifted to CPU and memory bandwidth. The query engine is the database’s main CPU consumer, so that’s where the rewrite focuses.

The post walks through the optimizations by building a miniature Postgres executor in Rust and improving it step by step. Stock Postgres uses the classic Volcano model, where every plan node exposes a next() method that returns one row at a time — simple to implement but expensive, since per-row function calls defeat CPU pipelining and other hardware optimizations. Batching is the first fix: processing rows in fixed-size chunks (1024 at a time) using a stack-allocated buffer to avoid runtime memory allocation, which drops a sample sum-of-500-million query from 1.3s toward the 358ms of a raw loop. The next bottleneck becomes the buffer copy itself, addressed through operator fusion — collapsing a sequential scan and an aggregation into a single combined node so the intermediate copy disappears entirely.

The broader lesson is that most of Postgres’s overhead on modern hardware is incidental rather than fundamental: locking, storage-format parsing, and the row-at-a-time execution model all add cost that a from-scratch, hardware-conscious engine can strip away. Batching, operator fusion, and SIMD are presented as the layered techniques that together close most of the gap between a real database and hand-written Rust — a useful case study for anyone weighing how much analytical performance is being left on the table by a general-purpose OLTP engine.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.