Change Feed

koldstore.changes_since is the packaged SQL change API for managed tables. It merges the hot change-log mirror and cold Parquet into one latest-state stream ordered by seq.

This is catch-up for clients (and aligns with KalamDB subscribe semantics for from / newest-N). It is not a full temporal audit log of every historical event.

Signature

SQL

1koldstore.changes_since(2  table_name regclass,3  since_seq  bigint  DEFAULT 0,      -- exclusive resume (seq > since_seq)4  limit_rows integer DEFAULT 1000,   -- forward page size5  last_rows  integer DEFAULT NULL    -- newest-N rewind when since_seq = 06)7RETURNS TABLE (8  seq        bigint,9  op         smallint,   -- 1=INSERT, 2=UPDATE, 3=DELETE10  pk         jsonb,11  deleted    boolean,12  row_image  jsonb,13  source     text        -- 'hot' | 'cold'14)

Modes

Mode	Arguments	Behavior
Resume	`since_seq > 0`	Rows with `seq > since_seq`, ascending, capped by `limit_rows`. `last_rows` is ignored.
Rewind	`since_seq = 0` and `last_rows` set	Newest N retained changes, delivered oldest → newest
From start	`since_seq = 0`, no `last_rows`	Start of retained history

Precedence rule: a positive since_seq always wins over last_rows.

Examples

Fence first when you need a strong boundary against the heap:

SQL

1SELECT koldstore.wait_for_async_mirror();

Page from the beginning

SQL

1SELECT seq, op, pk, deleted, source2FROM koldstore.changes_since(3  table_name => 'app.messages'::regclass,4  since_seq  => 0,5  limit_rows => 1006);

Resume after the last consumed seq

SQL

1SELECT seq, op, pk, deleted, source2FROM koldstore.changes_since(3  table_name => 'app.messages'::regclass,4  since_seq  => 184467440737095516,  -- last seq you already applied5  limit_rows => 5006);

Advance your cursor from the highest seq you successfully processed.

SQL

1SELECT seq, op, pk, deleted, source2FROM koldstore.changes_since(3  table_name => 'app.messages'::regclass,4  since_seq  => 0,5  limit_rows => 1000,6  last_rows  => 507);

last_rows must be <= limit_rows.

After flush

The same cursor continues to return flushed rows from cold (source = 'cold'). Do not treat “mirror empty” as “caught up” if cold still holds retained history.

Semantics and caveats

Exclusive cursor: seq > since_seq, never inclusive.
Authoritative seq values come only from the serialized WAL applier.
Latest-state per PK — updates replace prior images; deletes appear as tombstones until pruned by flush retention rules.
since_seq = 0 means “start of retained history,” not “I fell behind retention.”
A positive since_seq older than the retained cold floor raises a retention-gap error — do not treat an empty page as success in that case.
Hot + cold merge; source tells you which tier contributed the row image.

Direct mirror SQL (advanced)

You can inspect the hot mirror for debugging:

SQL

1SELECT id, seq, op2FROM koldstore.messages__cl3WHERE seq > $14ORDER BY seq5LIMIT 100;

Prefer changes_since in application code so cold history stays visible after flush and retention-gap errors stay consistent.