RC RANDOM CHAOS

Put your workflow engine inside Postgres and partial failures disappear

· via Hacker News

Original source

Postgres transactions are a distributed systems superpower

Hacker News →

DBOS argues that durable workflow systems shouldn’t just persist their state in Postgres — they should live in the same Postgres database as the application itself. The usual instinct is to separate workflow metadata from business data, but co-location turns out to be an advantage: because both can be written in a single database transaction, the partial-failure window that plagues distributed systems is eliminated.

The payoff shows up in two classic hard problems. For idempotency, ordinary durable workflows checkpoint each step after it runs, leaving a gap where a step (say, crediting a bank account) commits but crashes before its checkpoint is recorded — on recovery it runs again and double-credits. A co-located engine instead writes the step’s data change and its checkpoint in one transaction, so the step either commits exactly once or rolls back entirely. That makes application-level bookkeeping tables and manual idempotency guards unnecessary for transactional steps. For atomicity across systems, the standard fix is a transactional outbox plus a separate poller, which adds infrastructure and can drift out of sync. DBOS replaces it with a Postgres user-defined function that enqueues a workflow row in the same transaction as the data update; a worker later dequeues and runs it asynchronously.

The significance is less about a specific product than about the design principle: when workflow state and application data share a transaction boundary, exactly-once and atomic-across-systems semantics come almost for free, cutting out reconciliation jobs, outbox pollers, and guard tables. The tradeoff — coupling your workflow engine to your primary database — is the point rather than a compromise, and it fits the broader “just use Postgres” trend of collapsing separate pieces of infrastructure back into the database.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.