Postgres Replication Slots Hold WAL Until Disk Fills

Oct 20, 2026 By Yusuke Tanaka

A replication slot is a promise: Postgres will keep the WAL a consumer still needs. When the consumer disappears, the promise outlives its purpose, and the WAL directory grows until the disk fills. This piece covers how slots pin segments, how the failure unfolds, and the settings and habits that keep retention bounded.

The Slot That Ate The Disk

Replication slots were added to Postgres to solve a real problem: a standby or logical consumer could fall behind and the primary would recycle WAL it had not yet received. The slot tracks the consumer's position and prevents removal of segments beyond it. That guarantee is the point, and it is also the hazard.

The slot does not know whether the consumer is alive, paused, or gone. It knows only the last confirmed position. An inactive slot holds that position indefinitely by default. There is no built-in timeout, and no warning that escalates with age. A slot created for a migration that finished months ago will still pin every segment written since.

The failure is quiet until it is not. Disk usage climbs, checkpoints begin to fail, and the write path hits a PANIC. A single forgotten slot can halt a primary that has run cleanly for years. The fix is usually one pg_drop_replication_slot call, but reaching it requires a working connection and enough free space to operate.

The pattern repeats often enough that it has a recognizable shape. A team stands up a logical replica for an analytics pipeline, points it at the primary, and the pipeline gets deprecated six months later. The replica host is decommissioned. The slot remains, because the slot lives on the primary and nobody dropped it. WAL keeps accumulating at whatever rate the workload produces, and the volume fills weeks or months later, long after the person who created the slot has moved to another project. The delay between cause and symptom is what makes this failure so easy to miss.

How Slots Pin WAL Segments

Each slot records two positions: restart_lsn, the oldest WAL still needed, and confirmed_flush_lsn, the last position the consumer acknowledged. For logical slots, confirmed_flush_lsn advances only when the consumer confirms receipt. Physical slots advance through feedback from the standby.

The WAL recycler computes a retention bound from the minimum restart_lsn across all slots. Segments newer than that bound stay in pg_wal, even after a checkpoint would otherwise recycle them. The directory grows in 16 MB segments by default, so a slot lagging by a few gigabytes is visible as a few hundred files.

Postgres does expose a warning. When retention exceeds max_slot_wal_keep_size, the slot's wal_status moves to reserved, then eventually lost, and the slot is invalidated. The default for that setting is -1, meaning unlimited. That default is the reason the failure mode exists at scale.

It helps to separate the two slot kinds here, because they fail differently. A physical slot tracks a standby's replay position, and if the standby is reachable, feedback keeps the position moving. A logical slot tracks a decoding consumer, and its confirmed_flush_lsn only moves when that consumer explicitly confirms. A logical consumer that connects, reads a little, and then stalls without confirming leaves the slot pinned at the last confirmation, not at the last byte it read. The distinction matters when you are reading the view and trying to decide whether a slot is genuinely stuck or merely a few seconds behind.

There is also a subtler retention source: wal_keep_size and the archive settings can keep segments around independently of slots. When disk fills, confirm which mechanism is actually holding the segments before you start dropping things. The pg_replication_slots view tells you the slot side of the story; the archive configuration tells you the rest.

When The Failure Arrives

The sequence is predictable. WAL accumulates until the filesystem reports no free space. The next checkpoint cannot write, and the checkpointer process exits. Writes then fail with a PANIC, and the postmaster begins a crash recovery cycle that cannot complete because it needs to write WAL.

Reads may survive briefly on an already-open connection, but new connections typically fail during recovery. The server is effectively down, and the usual administrative tools need the same disk the server cannot write to. Teams often free space by removing old base backups or logs before they can drop the slot.

Recovery requires dropping the offending slot or moving pg_wal to a larger volume. Dropping the slot immediately recycles WAL up to the next consumer's position, which can free space in seconds. If the slot was a logical consumer that still needs data, that data is gone, and the consumer must be rebuilt from a fresh copy.

The rebuild is the part teams underestimate. A logical consumer that has to resync from scratch reads a full copy of the tables it tracks, which on a large database can take hours and load the primary with the read traffic. If the consumer was feeding a downstream system, that system is stale for the duration. Budget for the resync window when you decide whether to cap retention or let it run unbounded.

Monitoring The Right Signals

pg_replication_slots is the primary view. The active column shows whether a consumer is currently connected. A slot with active = false for more than a few hours is a candidate for investigation. The restart_lsn and confirmed_flush_lsn columns show the lag in bytes when compared to the current WAL position.

Directory size matters more than any single metric. Track the total size of pg_wal against free space on the volume, and alert when the directory exceeds a fixed threshold or when free space drops below a margin. The pg_stat_replication view covers connected standbys and their replay lag, but it says nothing about disconnected slots.

The gap between pg_wal growth and disk capacity is the real signal. A slot lagging by tens of gigabytes on a volume with hundreds free is survivable; the same lag on a smaller volume is an outage waiting. Set thresholds from the volume size, not from a generic number copied between clusters.

Two derived numbers are worth computing on a schedule. The first is the retention distance: the difference between the current WAL insert position and the oldest restart_lsn, converted to bytes. The second is the time to fill: that distance divided by the recent WAL generation rate, which you can estimate from the growth of pg_wal over an hour or a day. Time to fill is the number that tells you whether you have a week or an afternoon. A slot that has been inactive for a month on a busy primary may have a time-to-fill measured in hours, and that is the case worth paging on.

Watch the wal_status column too. A slot that has moved to reserved is already inside the retention bound and heading for invalidation; that transition is a leading indicator, not a routine state.

Coping Patterns That Work

max_slot_wal_keep_size is the direct control. Setting it to a disk-aware bound, such as a fraction of the volume, lets Postgres invalidate a slot that exceeds retention rather than fill the disk. The trade-off is real: the slot becomes unusable and the consumer must resync. For a logical replica that is already far behind, that may be the cheaper outcome.

Temporary slots suit one-off logical copies. Create the slot, run the copy, drop the slot in the same script. A slot that exists only for the duration of a job cannot be forgotten. For long-lived consumers, pair every slot with an owner and an expiry date recorded somewhere outside the database.

pg_waldump and the pg_replication_slots view together identify what a slot is for. If the slot name does not map to a known consumer, treat it as abandoned. Before decommissioning a replica, drop its slot first; the reverse order leaves an orphan. The same discipline applies to client libraries that manage connections, as this site has noted in Postgres wire protocol changes.

The choice between capping retention and leaving it unlimited is a choice about which failure you prefer. With a cap, you trade a slot invalidation and a resync for a bounded disk footprint; you accept that a long-stalled consumer loses its position and must be rebuilt. Without a cap, you keep every consumer's position intact and accept that a forgotten slot can take down the primary. Neither is free. The cap converts an unbounded outage into a bounded, recoverable event, and for most production clusters that is the right trade.

Operational Habits To Adopt

  1. Audit pg_replication_slots weekly and flag any slot with active = false for more than a day.
  2. Set max_slot_wal_keep_size to a value derived from the volume size, not copied from another cluster.
  3. Alert when pg_wal exceeds a fixed size threshold or when free space on the volume drops below a defined margin.
  4. Record an owner and an expiry date for every slot at creation, and review the list monthly.
  5. Test dropping a slot and recovering from the resulting WAL recycle in a staging cluster before you need to do it under pressure.

These habits are unglamorous, and they cost a few minutes a week. The alternative is a primary that stops accepting writes at an inconvenient hour because a slot nobody remembers is still holding a promise. A related piece on certificate pinning makes a similar point about silent failures that surface only when the margin is gone.

Recommend Posts
Tech

Anycast Routing and DNS Steering Diverge Under Load

By Sara Park/Oct 20, 2026

Anycast and DNS steering both promise proximity, but they diverge under load. A craft breakdown of where each fails and how to instrument both planes.
Tech

Maintainer Burnout Rises While Tooling Metrics Stay Green

By Sara Park/Oct 20, 2026

CI dashboards keep reporting green while the people behind the tooling run out of capacity. A look at why build metrics miss maintainer burnout and what to track instead.
Tech

Retry Budgets Decide Which Microservice Failures Become Outages

By Yusuke Tanaka/Oct 20, 2026

Retry budgets cap the ratio of retries to original requests, stopping retry storms from turning a slow dependency into a full outage. How Envoy, Istio, gRPC, and Finagle implement them, what the trade-off costs in user-visible errors, and how to pick a budget without guesswork.
Tech

Open Source Maintainers Trade Salary for Ownership in Hiring Deals

By Sara Park/Oct 20, 2026

Open-source maintainers increasingly negotiate governance authority and equity instead of top salary. What that trade costs, and how to structure the deal before signing.
Tech

Open Source Licenses and Cloud Vendor Contracts Diverge on Support Revenue

By Deepa Iyer/Oct 20, 2026

Open-source licenses grant rights, not revenue. The money lives in cloud support contracts, indemnity terms, and renewal pricing. Here is where the margin actually sits.
Tech

Testing Push Notification Delivery on Android Before Trusting Analytics

By Sara Park/Oct 20, 2026

Analytics dashboards count sends, not deliveries. Here is how Android push actually moves through FCM, Doze, and OEM layers, and how to instrument the real path.
Tech

Mutual TLS Deployment Stalls Where Certificate Rotation Lacks Automation

By Yusuke Tanaka/Oct 20, 2026

Mutual TLS demos cleanly but stalls at renewal. A practical look at what mTLS changes on the wire, where manual rotation breaks, and automation patterns that survive.
Tech

Developer Tooling Careers Trade Depth for Breadth in Hiring

By Lucas Mendes/Oct 20, 2026

Developer tooling careers reward breadth in hiring loops, but depth in package managers, build systems, and CI pays off long-term. How specialists can bridge the gap.
Tech

Why Token Rotation Logs Miss the Sessions Attackers Keep

By Lucas Mendes/Oct 20, 2026

Token rotation logs report success while stolen sessions stay live. A payments team post-mortem shows why session state escapes audit and how to close the gap.
Tech

Maintainer Contracts Rarely Cover the On-Call Hours Behind a Security Fix

By Lucas Mendes/Oct 20, 2026

Security fixes ship on time, but the unpaid on-call hours behind them rarely appear in maintainer contracts. A breakdown of who bears the cost and how to fix it.
Tech

Editors and Package Managers Split Over Who Pays for CI Minutes

By Lucas Mendes/Oct 20, 2026

CI minutes have become a real budget line. Editors, package managers and platforms disagree about who absorbs the cost, and the contracts decide the winner.
Tech

Nvidia's Driver Stack Carries the Correctness Burden Its Hardware Cannot

By Deepa Iyer/Oct 20, 2026

Nvidia's GPUs do not guarantee correct math on their own. The driver and CUDA runtime carry that promise, and the burden is shifting toward users.
Tech

Postgres Wire Protocol Changes and the Client Libraries That Must Track Them

By Lucas Mendes/Oct 20, 2026

How Postgres client libraries track wire protocol changes, where message parsing breaks, and the testing practices that keep drivers compatible across server versions.
Tech

Postgres Replication Slots Hold WAL Until Disk Fills

By Yusuke Tanaka/Oct 20, 2026

Replication slots pin WAL segments indefinitely, filling disk and halting writes. Learn how slots retain WAL, the signals to watch, and the settings that cap the damage.
Tech

Chrome and Firefox Repaint Differently When CSS Grid Changes

By Yusuke Tanaka/Oct 20, 2026

Chrome and Firefox invalidate and repaint CSS Grid changes differently. A look at Blink's invalidation sets, Gecko's display lists, and how to profile both.
Tech

Machine-Learning Teams Rewrite Their Deployment Playbooks as Inference Standards Shift

By Yusuke Tanaka/Oct 20, 2026

Serving engines like vLLM and TensorRT-LLM now define wire behavior. What changes for ML teams at the API level, in on-call rotations, and in hiring loops.
Tech

Open Source Foundations Fund Compliance Work From Trademark Royalties

By Sara Park/Oct 20, 2026

Open source foundations increasingly fund license compliance work through trademark royalties from certification marks, creating a business model with real governance tensions.
Tech

Android Fragmentation Forces Cross-Platform Stacks Into Per-Device Testing

By Sara Park/Oct 20, 2026

Android fragmentation turns one cross-platform codebase into hundreds of test targets. A practical breakdown of device tiering, snapshot matrices, and hardening the mobile pipeline.
Tech

Check Which Maintainers Hold Signing Keys Before Your Next Update

By Yusuke Tanaka/Oct 20, 2026

Code signing only proves what a key says. Here is how to check which maintainers actually hold signing keys, where those keys live, and what to fix this quarter.
Tech

Certificate Pinning Breaks Silently When Rotation Runs Behind Revocation

By Deepa Iyer/Oct 20, 2026

Certificate pinning fails quietly when revocation outruns rotation. A practical breakdown of pin sets, overlap windows, revocation drills and handshake telemetry.