Microsoft open-sources pg_durable, a Postgres extension for durable workflows
Microsoft has released pg_durable, a PostgreSQL extension that brings durable execution—the pattern popularized by Temporal, Airflow, and Step Functions—directly into the database. Workflows are defined in SQL as graphs of steps using composable operators like ~> and |=>, started via df.start(), and checkpointed by the runtime so that crashes, restarts, or step failures resume from the last successful point rather than forcing a full replay. State, retry counters, and progress live in Postgres tables such as df.instances, using the same auth, backup, and visibility model as the rest of the database.
The pitch is eliminating the usual scaffolding around long-running jobs: pg_cron plus status tables, external orchestrators calling back into Postgres, or queues coordinated with bespoke worker code. Target use cases include vector embedding pipelines feeding pgvector, batch ingest and dedup, scheduled maintenance with approval gates, fan-out aggregation, and external API enrichment from SQL. The extension intentionally does not try to replace general-purpose orchestrators when logic spans many heterogeneous systems or needs rich in-memory control flow.
Distribution is via Debian packages for PostgreSQL 17 and 18 on amd64, with source builds requiring Rust nightly and cargo-pgrx 0.16.1. Security is opt-in: CREATE EXTENSION grants nothing to PUBLIC, row-level security scopes each user to their own instances, and admins explicitly call df.grant_usage() per role. The background worker role must be a superuser to bypass RLS across tenants, and grants must be re-issued after extension upgrades to cover newly added functions.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.