Skip to Content
Why a Mirror?

Why a Mirror?

KoldStore already uses PostgreSQL WAL as its change source (logical decoding into a database worker). It still maintains a per-table latest-state mirror (koldstore.<table>__cl) because WAL and the mirror solve different problems.

WAL is an excellent change transport. The mirror is an operational data structure for flush, change feeds, and primary-key lookups.

Compare

NeedPostgreSQL WALLatest-state mirror (__cl)
Crash recovery / PITR / physical replicaDesigned for thisNot its job
Decode every committed changeNativeApplied once by the worker
Latest state of one primary keyReplay + resolveIndexed row (WHERE id = …)
Rows pending flushReconstruct from historyThe table is the queue
Resume change feed from a cursorSlot LSN + filter/replayWHERE seq > $last
Per-table isolationShared stream; filter laterOne __cl per managed table
Inspect / debug with SQLDecoding toolingNormal SELECT
Append-only history of every UPDATEYesNo — one row per PK

Latest state, not another WAL

Four WAL records for the same key become one mirror row:

TEXT
WAL:    INSERT id=10 → UPDATE → UPDATE → DELETEMirror: id=10, op=DELETE, seq=8471294123

Flush and catch-up need “what is pending for this key now,” not a full rewrite of intermediate versions.

What the mirror makes cheap

SQL
-- Pending flush batchSELECT * FROM koldstore.messages__clORDER BY seq LIMIT 1000; -- Change-feed resumeSELECT * FROM koldstore.messages__clWHERE seq > 91234123ORDER BY seq; -- Still hot before archive?SELECT 1 FROM koldstore.messages__cl WHERE id = 10;

Those are ordinary indexed queries. Doing the same against raw WAL means decoding, replaying, visibility, and reconstructing latest state on every use.

Complementary, not competing

TEXT
PostgreSQL WALLogical decodingMaterialize latest stateMirror (__cl)Flush + changes_since + hot/cold lifecycle

Today that path is already live: heap commit → WAL → applier → __cl. A future capture change would still target the same mirror and the same external APIs.

Summary

  • WAL moves durable changes through PostgreSQL.
  • Mirror materializes latest-per-PK state for flush queues, resume cursors, and PK lookups.
  • They are layers in one design, not alternatives.

See also: Under the Hood, Change Feed.

Last updated on