A 4B model outplans Postgres

A 4B-parameter model can produce Postgres query plans that execute up to 81% faster than the plans Postgres chooses for itself - but only on a narrow class of queries, and only because the model never touches the data path directly. It does not rewrite your SQL. It does not replace the executor. It looks at a query, proposes a join order and plan shape, and every candidate it suggests gets validated by the real system before anything runs in production. The 81% figure comes from complex multi-join analytic queries, the exact place where Postgres’s own cost estimates tend to drift furthest from reality. On a two-table lookup, the model gives you nothing, because Postgres already gets those right.

The reason a model this small is enough has nothing to do with intelligence and everything to do with the shape of the problem. Join ordering is a bounded search over a structured space with a measurable reward: actual execution time. You are not asking the model to reason freely or generate language. You are asking it to rank candidate plans for a query whose structure it can see fully - the tables, the predicates, the available indexes, the cardinality hints. That is a narrow, well-defined task, and 4B parameters is more than sufficient to learn it. Scaling to a larger model buys you latency and cost, not accuracy, because the ceiling here is set by the quality of your training signal, not by parameter count.

What you are actually buying is bounded. The model needs training data drawn from your own workload, because a plan ranker learned on one schema does not transfer cleanly to another. It carries inference latency, so it belongs at planning time or in an offline optimization pass, not bolted onto the per-query hot path unless you can amortize the cost across repeated queries. And it degrades on query shapes it has never seen. Treated as a planning-time advisor with a validation gate behind it, it earns its place. Treated as a drop-in replacement for the Postgres optimizer, it will hurt you the first time it confidently proposes a plan that regresses.

To see why the model has room to win at all, you have to look at how Postgres plans a query in the first place. The planner is a cost-based optimizer. It enumerates candidate execution strategies - which join order, which join method, nested loop versus hash versus merge, which index versus sequential scan - and it assigns each a cost estimate derived from table statistics collected by ANALYZE. It then picks the cheapest plan by that estimate. For a small number of joins it searches this space with dynamic programming; once the join count climbs past a threshold, it hands off to GEQO, a genetic algorithm that samples the space instead of exhausting it. The output is a plan that is optimal according to the planner’s model of cost, which is not the same thing as optimal according to the clock.

The gap between those two lives almost entirely in cardinality estimation. Postgres estimates how many rows each operation will emit, and those estimates feed every downstream decision. The trouble is that estimation error compounds across joins. A predicate on a skewed column, a correlation between two columns the statistics assume are independent, a join that fans out harder than the histograms predict - any of these throws the row estimate off, and the error multiplies as it propagates up the join tree. By the time the planner is choosing between a nested loop and a hash join near the top of a five-way join, it may be working from a row estimate that is wrong by two orders of magnitude. It then picks a plan that is genuinely cheapest for the wrong cardinalities. This is where the 81% comes from: not from Postgres being poorly engineered, but from a single systematic blind spot that the model learns to correct.

The model attacks exactly that blind spot. You train it on triples of query structure, candidate plan, and measured runtime, so it learns to predict which join order and plan shape actually run fastest - correcting for the estimator errors that Postgres cannot see. The input is not raw SQL text but the structured features that matter: the join graph, the predicates, the relevant statistics, the index availability. The output is a plan directive, typically expressed through a hinting layer like pg_hint_plan, that forces Postgres down the join order the model prefers. Around all of this sits deterministic control. You never trust the model’s predicted runtime as truth. You take its proposed plan, force it, and compare it against the planner’s default on the real executor. Whichever wins, wins. The model narrows the search and proposes; the executor remains the only source of ground truth. That separation is the whole design - a probabilistic ranker wrapped in a deterministic validation loop, so a bad suggestion costs you a comparison, never a production regression.

The first misread is that the model is writing better SQL. It is not. Your queries are untouched, character for character. What changes is the physical plan Postgres uses to answer them - the join order and the operators - and that plan is a separate layer from the SQL you wrote. People hear 4B model and query optimization in the same sentence and picture an LLM rewriting statements, which is the wrong mental model and leads to the wrong architecture. Nothing here depends on natural language generation. The model consumes a structured representation of a query and emits a structured plan directive.

The second is the assumption that this replaces the Postgres planner. It sits beside it. Postgres still parses, still plans, still executes; the model injects a preference into the join ordering and the planner does the rest under a hint. Remove the model and the database keeps working exactly as before. That is not a weakness to apologize for - it is the property that makes the approach safe to deploy, because your fallback is the unmodified planner you already trust. Anyone who frames this as ripping out the optimizer has skipped the part that makes it operationally viable. The related mistake is believing a larger model would do better. The bottleneck is the reward signal and the coverage of your training workload, not parameter count. A 4B model is the right size precisely because it keeps inference cheap enough to run at planning time; a larger one would raise cost and latency without moving accuracy, since the task is already saturated at this scale.

The third and most damaging error is trusting the 81% as if it were a global speedup and shipping model-picked plans without a gate. That number is a benchmark result on a specific class of complex queries. Simple queries see no change, and some queries will regress, because the model is a probabilistic ranker and it will occasionally be confidently wrong. A production setup that forces the model’s plan blindly is one bad prediction away from a slow query storming through your latency budget. The discipline that makes this work is unglamorous: validate every proposed plan against the baseline before adopting it, keep the planner’s default as a fallback, and monitor for regressions after deployment so a plan that ages badly gets caught and reverted. The capability is real, but the value is in the loop around the model, not the model itself - the validation gate, the fallback path, and the comparison against ground truth are what turn an interesting benchmark into something you can run under load.

What works is treating the model as one component inside a loop, not as the thing you call and trust. The working system has six moving parts: a classifier that decides which queries are even worth routing, a deterministic feature extractor, the 4B plan ranker, a validation gate, a plan cache, and a regression monitor. The model is the smallest and least important of these. Strip it out, drop in a random plan generator in its place, and the loop still protects you - it just stops winning. That is the tell that the leverage lives in the orchestration, not the weights.

Start by scoping. Fingerprint every query by normalising its literals, then classify by join count and shape. Anything with two or three tables goes straight to the Postgres planner untouched, because there is no gap to close there and every millisecond of model latency would be pure waste. Only the complex multi-join analytic queries - the six, seven, and eight-way joins where cardinality error compounds - get routed into the optimisation path. This single decision does most of the work, because it aims the expensive machinery at the narrow class of queries where it actually pays and keeps it away from the queries Postgres already handles well.

The feature extraction is deterministic and has nothing to do with the model. You pull the join graph, the predicates, the relevant column statistics from pg_statistic, the index availability, and the planner’s own row estimates out of EXPLAIN. That structured representation is the model’s input - never raw SQL text. Training data comes from the same pipeline run offline on a replica: for each query you force a set of candidate plans through pg_hint_plan, measure real execution time with EXPLAIN ANALYZE, and log the triple of features, plan, and runtime. The model learns the mapping from query structure to the plan that actually runs fastest, and nothing beyond that.

Then the gate, which is the part you never skip. At optimisation time the model proposes its top few plans; you force each one and the planner’s default, run them on the real executor against a representative data sample, and keep whichever is measurably fastest. The winning plan directive gets cached keyed by the query fingerprint. Serving is then a cache lookup and a hint injection - no model call on the hot path, no inference latency in front of the user. A cache miss falls back to the unmodified planner and queues the query for the next offline pass. And because table statistics drift, you monitor runtime per fingerprint over time; when a cached plan starts aging badly you revert it to the default and re-optimise. The fallback is always the database you already trust.

Picture a reporting service sitting on a Postgres warehouse: a star schema, a couple of billion-row fact tables, a dozen dimensions, and roughly two hundred parameterised analytic queries behind a set of dashboards. Most are heavy - six to eight joins, aggregations, filters on columns that are skewed and correlated in ways ANALYZE does not capture. Region correlates with currency, one status value covers ninety-five percent of rows, and the planner treats every column as independent and uniform. The result is familiar: on the worst dashboards Postgres picks a nested loop where a hash join would win, and a query that should return in under a second takes four.

You run the offline pass over the query catalogue on a replica. For each of the two hundred fingerprints the model proposes a handful of join orders, the gate forces them alongside the default, and the fastest measured plan wins. Most queries improve; the heaviest analytic ones drop from around four seconds to under one, which is where the headline speedup actually comes from. Some do not move at all, and the gate keeps the default for those. At least one query the model was confident about ran slower when forced - the gate measured it, saw the regression, and discarded the model’s plan. That rejection is the system working exactly as designed, not a failure of it. The surviving directives get cached, and the dashboards now serve pre-validated plans with zero model latency at request time.

A month later a data load shifts the distribution of one fact table, and a cached plan that used to win starts losing. The monitor catches the runtime creeping up on that fingerprint, reverts it to the planner default automatically, and flags it for the next offline pass, where the model re-proposes against the new statistics. No one woke up at 3am, no dashboard fell over, and the worst case at every step was a query running at plain-Postgres speed. That is the entire point of the design: the downside is bounded to the baseline you started with, and the upside is real on exactly the queries that were hurting.

The 81% is real, but it is not the achievement. The achievement is a probabilistic ranker wrapped so tightly in deterministic control that a wrong answer costs you a comparison and never a production incident. The model narrows the search; the executor decides. Take that framing seriously and the small model size stops being surprising - you are not buying reasoning, you are buying a learned correction to one specific blind spot in Postgres cost estimation, and 4B parameters saturate that task.

The pattern generalises well beyond query planning. Any time you have a bounded search space, a measurable reward, and an existing deterministic system that is good but not optimal, the same shape applies: a model proposes, a validator built on ground truth accepts or rejects, and the trusted system stays the fallback. Plan selection, index recommendation, cache warming, resource scheduling - they all fit that mould. What does not fit is the version where you delete the validator to save latency, because then you have swapped a reliable system for a probabilistic one and called it progress.

So the honest scope: this belongs in an offline or planning-time optimisation pass over a stable set of recurring analytic queries, trained on your own workload, with a gate in front and a fallback behind. It is not a drop-in Postgres replacement, it does no rewriting of your SQL, and it does nothing for simple queries. Build the loop first and the model becomes a component you can improve, replace, or remove without risk. Build the model first and skip the loop, and the day it is confidently wrong is the day it takes production down with it. The capability was never the hard part. The control around it is.