Databricks' LTAP: Postgres and analytics on one copy of data, no CDC
Traditional databases keep their write-ahead log and data files on a single machine’s disk, and Databricks argues that this monolithic design is the source of most operational pain: commits can silently vanish on misconfigured flushes or disk failure, read replicas and high-availability standbys each require a full physical clone of the dataset, and heavy analytical queries starve latency-sensitive transactions because both fight over the same row-oriented storage. Lakebase, the company’s serverless Postgres, breaks this apart by making compute stateless — the WAL moves to a distributed service called SafeKeeper (durability via replicated quorum rather than local disk flush) and the data files move to PageServer. Postgres engines can then be started, stopped, and branched freely because they no longer own the data, yielding elastic compute, cheap durable storage, simpler HA, and near-instant provisioning.
LTAP is the next step: operational data is materialized once in open columnar formats on object storage that both Postgres and lakehouse query engines can read. Analytics then run against the same fresh data a transaction just committed, with no change-data-capture pipeline, no mirrored second copy, and no drag on the transactional workload. The pitch contrasts this with HTAP, which forces both transactional and analytical workloads through one engine and inevitably compromises each. LTAP instead unifies at the storage layer and lets a purpose-built engine handle each job.
The significance is architectural rather than incremental: by decoupling durability and storage from compute and standardizing on open columnar formats, Databricks is positioning Lakebase as a way to collapse the long-standing operational-versus-analytical divide without the ETL plumbing and duplicated infrastructure that divide normally demands — a natural extension of its lakehouse strategy into the OLTP layer.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.