Schema Evolution and Migrations

KalamDB projects store schema history under kalam/migrations/ (configured in [migrations].dir). The CLI diffs schema.sql against a local baseline, maintains a working _draft.sql, and tracks applied migrations on the server per namespace.

Complete schema evolution lifecycle

Follow this sequence for each change: edit the schema → review a migration → apply locally → migrate existing data → regenerate and test clients → stage → deploy → verify. Keep the server, CLI, and SDK versions aligned. Start a project with init and dev; environment URLs and codegen targets live in kalam.toml.

1. Add a compatible column

Suppose app.todos already contains rows and older applications still write only title. Add this nullable field to its existing declaration in schema.sql:

1completed BOOLEAN

For this explicit workflow, pause the kalam dev watcher before editing so it does not apply the same change through a draft. Keep the target database running independently. Create and inspect the numbered migration:

1kalam db migration create add_completed2kalam db migration status

The reviewed -- UP operation should be:

1ALTER TABLE app.todos ADD COLUMN completed BOOLEAN;

Review both -- UP and -- DOWN; keep unrelated operations out. A generated diff is a proposal, not proof that a conversion is safe. Do not edit a migration after it has been applied to a shared environment: checksums protect history. Correct it with a new migration.

Alternatively, keep kalam dev running and use its draft review/apply flow below. Choose one workflow for a change; do not generate a second numbered migration for a draft already applied.

2. Apply and migrate existing data

1kalam db migrate2kalam db migration status

Check the actual catalog and rows, not only the local schema file:

1SELECT id, title, completed FROM app.todos;2UPDATE app.todos SET completed = FALSE WHERE completed IS NULL;3SELECT COUNT(*) AS remaining FROM app.todos WHERE completed IS NULL;

Treat the update as a reviewed data migration or tracked backfill job. For a large table, process bounded batches with progress and retries; this small example uses one statement. The predicate makes repeating the update safe for rows already filled.

For a USER table, these queries affect the effective authenticated user’s partition. Running them as root does not backfill every user’s data. Run authorized backfills for each required identity, or let the application fill rows within each user’s scope. For SHARED tables, verify the operator’s policies and the intended tenant scope.

Older clients can still insert NULL while they omit the new column. Keep a read fallback (completed ?? false in TypeScript/Dart, or unwrap_or(false) for a Rust Option<bool>) until those writers have been upgraded. Backfill completion is not permission to immediately require the field.

3. Regenerate and test each application

1kalam schema gen2kalam schema gen --check

These commands use the languages enabled in [schema].languages; current targets include TypeScript, Dart, and Rust. A specific --languages filter must name an enabled target. Generation reads the project schema; passing --check does not prove the database was migrated.

Rebuild the actual applications and run their tests. For React/Drizzle, regenerate its ORM schema when that is your application’s schema source; it is a separate generator. Review type mapping limitations rather than assuming every generated type is exact.

Verify an existing row, a new row from the updated client, and a write from the previous client version. Check the nullable field and fallback in each SDK. Regenerate routine contracts and build their implementations too when function signatures change.

4. Verify live queries and local caches

Include the new field in explicit SELECT projections and update row mappers. Recreate affected subscriptions against the committed schema; renames and type changes can invalidate existing subscriptions. Test insert, update, delete, reconnect, and a fresh snapshot with the SDK examples.

Version persisted caches and their projections with the schema. A cursor belongs to its cached data, query, and identity; do not reuse it with an incompatible decoder or an empty cache. For Flutter offline apps, migrate local state deliberately and preserve pending writes before rebuilding a cache. See Kalam Sync and schema-change reconnection.

5. Stage, deploy, and verify

Commit the schema source, reviewed numbered migrations, function implementations, and generated artifacts according to your repository policy. Use the same reviewed revision in staging and production. Configure and verify each environment’s URL, schema, and credentials first.

For migration-only rollout:

1kalam db migration status --env staging2kalam db migrate --env staging3kalam db migration status --env staging

For a project with server functions:

1kalam deploy --env staging --dry-run2kalam deploy --env staging3kalam functions status --env staging

deploy --dry-run generates/builds locally but does not apply migrations or activate functions. A real deploy generates artifacts, builds functions, applies migrations, activates the module, and checks target health. It does not publish or start your frontend, mobile app, or service processes. Release those through your application’s deployment process after verifying schema compatibility.

Repeat the verified commands with --env prod after staging passes and a suitable backup/recovery plan is ready. Check migration status, authenticated reads/writes, a function call if applicable, and a live subscription. Database health alone does not prove application health.

Breaking schema changes and older clients

Use an expand–migrate–contract rollout:

A direct rename or cast can be appropriate during a controlled outage, but must be reviewed against the supported table DDL and named-type evolution. Mobile releases and cached browser sessions can remain on old contracts after the new deployment. Do not drop old columns merely because the newest application builds.

Failure recovery and rollback

kalam db migration repair --mark-applied only changes migration bookkeeping. Use it after independently proving that the intended effects exist. It does not execute missing SQL or restore lost data. --force is not a universal repair command.

The current CLI has no automatic migration-down command. A -- DOWN section is reviewable SQL, not a promise that rollback is safe or automatic. Prefer a corrective forward migration; destructive data loss may require a verified restore. Function rollback restores a module revision only—it does not undo schema or backfill changes. Ensure the old function/app still supports the current schema before reverting it.

Application schema evolution is separate from a server storage-format upgrade. Follow Upgrade to 0.8 for that lifecycle.

Migration files

Draft files are not applied to production deployments until sealed into a numbered migration.

Typical development flow

1# 1. Edit schema.sql2vim schema.sql3 4# 2. Let kalam dev detect changes (or run pipeline manually)5kalam dev6 7# 3. When prompted, apply the draft — or seal and migrate explicitly:8kalam db migration seal9kalam db migrate10 11# 4. Check state12kalam db migration status

During kalam dev, edits to schema.sql update _draft.sql automatically. Numbered migrations apply on startup and after each watched change; the draft waits for your confirmation unless you use kalam dev --force (which skips the prompt but still leaves sealing/applying the draft to the Apply path or explicit kalam db migration seal).

Commands

kalam db migration create <name>

Create a numbered migration immediately from the current diff (without going through the draft file):

1kalam db migration create add_messages_table2# creates kalam/migrations/20260611120000_add_messages_table.sql

kalam db migration status

Report migration state for the resolved environment:

1kalam db migration status2kalam db migration status --env staging

Groups: Applied, Pending, Failed, Applying.

kalam db migration seal

Promote _draft.sql into the next numbered migration file (rename only — does not apply to the database):

1kalam db migration seal

Run kalam db migrate afterward to apply the sealed migration.

kalam db migrate

Apply all pending numbered migrations to the linked database:

1kalam db migrate2kalam db migrate --env prod

• Does not include _draft.sql unless you sealed it first
• Ensures the target namespace exists before applying
• Validates checksums for migrations already marked applied

kalam db reset

Clear local dev project state and, when appropriate, drop the linked namespace on the server so the next kalam dev can re-apply migrations cleanly.

1kalam db reset2kalam db reset --yes   # confirm namespace drop without prompting3kalam dev

Options

• --project-dir <PATH>
• --env <ENV>
• --yes — drop the namespace on a remote or non-project server without prompting (required in non-interactive shells when the server is not this project’s kalam/server)

Removes locally (when present):

• kalam/server/ — entire local server directory (data, logs, and server.toml)
• kalam/.schema-baseline.sql — schema diff baseline

Keeps:

• kalam/migrations/ — migration SQL files on disk

Server namespace drop (when the linked KalamDB server is reachable):

Dropping the namespace removes tables and server-side migration records for that namespace (including failed or stuck migrations in system.migrations).

If you decline the prompt, or run in a non-interactive shell without --yes, local files are still cleared but the server namespace is left unchanged. You may see migration failed previously on the next kalam dev until you run kalam db reset --yes or drop the namespace manually.

Stop kalam dev before reset when wiping kalam/server/ so RocksDB files are not locked.

After reset, run kalam dev again. Pending numbered migrations re-apply on startup.

The CLI prints each removed path, reports when nothing was present, and shows whether the namespace was dropped or skipped.

kalam db migration retry <id>

Re-queue a failed migration for another apply attempt:

1kalam db migration retry 20260610120000_add_users

kalam db migration repair <id> --mark-applied

Manually mark a migration as applied on the server (use when you’ve fixed state out of band):

1kalam db migration repair 20260610120000_add_users --mark-applied

How drafting works

1. Baseline — kalam/.schema-baseline.sql mirrors the last successfully applied schema
2. Diff — the CLI compares schema.sql to the baseline and generates UP/DOWN SQL
3. Draft — changes land in kalam/migrations/_draft.sql
4. Seal — kalam db migration seal renames the draft to timestamp_name.sql
5. Apply — kalam db migrate or the dev pipeline executes UP statements and records checksums on the server
6. Baseline update — after apply, baseline is refreshed from schema.sql

If the diff is empty, a stale _draft.sql is removed automatically.

Dev vs production behavior

Deployment: Use kalam db migrate --env prod for migration-only rollout, or kalam deploy for the schema/functions pipeline. Application deployment remains separate.

Stuck or failed migrations

During apply, the CLI may prompt when a migration was interrupted or failed previously:

• Retry — attempt apply again
• Skip / mark applied — for failed migrations when you’ve verified state manually
• Abort — stop the apply run

Pass --force on kalam dev to retry automatically in some stuck states.

For manual recovery:

1kalam db migration status2kalam db migration retry <migration_id>3# or4kalam db migration repair <migration_id> --mark-applied

Global flags

All migration commands accept:

Migration file format

1-- Migration: add_messages_table2-- Created: 2026-06-11T12:00:00Z3 4-- UP5CREATE TABLE app.messages (6  id BIGINT PRIMARY KEY,7  body TEXT8);9 10-- DOWN11DROP TABLE app.messages;

Related

• Local Development — watch loop and draft prompts
• Project Init — kalam.toml and migrations directory
• SQL reference

PostgreSQL schema dependencies

The 0.8 CLI diffs a project snapshot of tables, named types and routines, then orders dependent changes before applying them. Review destructive or conversion operations in the generated SQL. Migration files use lexer-aware -- UP and -- DOWN markers, so markers inside quoted routine bodies do not split a migration. Fenced migration receipts record application state and help prevent competing runs from treating the same migration as independently applied. Verify status after a failure before retrying or repairing.

An application schema migration is different from upgrading an older server data directory. Read Upgrade to 0.8 for the fresh-database requirement.