Storage Commands
KalamDB supports multiple storage backends for cold-tier data. These commands manage storage configuration, health checks, and maintenance.
For practical table-level storage selection (STORAGE_ID, USE_USER_STORAGE, and verification queries), see Storage ID Usage.
For S3-compatible local deployment setup, see MinIO (S3-Compatible).
CREATE STORAGE
CREATE STORAGE <storage_id> TYPE '<filesystem|s3|gcs|azure>' [NAME '<storage_name>'] [DESCRIPTION '<description>'] [PATH '<path>'] [BUCKET '<bucket_or_s3_url>'] [REGION '<region>'] [BASE_DIRECTORY '<path_or_url>'] [SHARED_TABLES_TEMPLATE '<template>'] [USER_TABLES_TEMPLATE '<template>'] [CREDENTIALS '<json_credentials>'] [CONFIG '<json_config>'];Examples
Local filesystem:
CREATE STORAGE local TYPE 'filesystem' PATH './data';Amazon S3:
CREATE STORAGE s3_prod TYPE 's3' BUCKET 'my-bucket' REGION 'us-west-2' CREDENTIALS '{"access_key_id":"...","secret_access_key":"..."}';Google Cloud Storage:
CREATE STORAGE gcs_prod TYPE 'gcs' BUCKET 'my-gcs-bucket' CREDENTIALS '{"service_account_key":"..."}';Azure Blob Storage:
CREATE STORAGE azure_prod TYPE 'azure' BUCKET 'my-container' CREDENTIALS '{"account_name":"...","account_key":"..."}';Storage Type Aliases
| SQL Type | Alias | Base Directory Prefix |
|---|---|---|
filesystem | — | Local path (e.g., ./data/storage) |
s3 | — | s3://bucket/prefix |
gcs | gs | gs://bucket/prefix |
azure | az | az://container/prefix |
CONFIG JSON Fields by Backend
S3 CONFIG
{ "type": "s3", "region": "us-east-1", "endpoint": "http://127.0.0.1:9120", "allow_http": true, "access_key_id": "...", "secret_access_key": "...", "session_token": "..."}| Field | Type | Default | Notes |
|---|---|---|---|
region | string | "us-east-1" | AWS region or custom region for S3-compatible |
endpoint | string | null | Custom endpoint for MinIO / S3-compatible |
allow_http | boolean | false | Must be true for non-TLS endpoints |
access_key_id | string | null | Static credentials (omit for IAM roles) |
secret_access_key | string | null | Static credentials |
session_token | string | null | Temporary session credentials |
For S3-compatible local setup, see MinIO (S3-Compatible).
GCS CONFIG
{ "type": "gcs", "service_account_json": "{...}"}| Field | Type | Default | Notes |
|---|---|---|---|
service_account_json | string | null | Service account JSON key. Omit to use Application Default Credentials (ADC). |
Azure CONFIG
{ "type": "azure", "account_name": "mystorageaccount", "access_key": "...", "sas_token": "..."}| Field | Type | Default | Notes |
|---|---|---|---|
account_name | string | null | Azure storage account name |
access_key | string | null | Account access key |
sas_token | string | null | Shared Access Signature token (alternative to access_key) |
Local/Filesystem CONFIG
{ "type": "local", "root": "./data/storage"}| Field | Type | Default | Notes |
|---|---|---|---|
root | string | null | Informational; actual path comes from BASE_DIRECTORY or PATH. |
ALTER STORAGE
ALTER STORAGE <storage_id> [SET NAME '<new_name>'] [SET DESCRIPTION '<new_description>'] [SET SHARED_TABLES_TEMPLATE '<new_template>'] [SET USER_TABLES_TEMPLATE '<new_template>'] [SET CONFIG '<json_config>'];DROP STORAGE
DROP STORAGE <storage_id>;DROP STORAGE IF EXISTS <storage_id>;SHOW STORAGES
SHOW STORAGES;STORAGE CHECK
Verify storage backend health:
STORAGE CHECK <storage_id>;STORAGE CHECK <storage_id> EXTENDED;STORAGE FLUSH
Force flushing data from hot tier (RocksDB) to cold tier (Parquet):
-- Flush a specific tableSTORAGE FLUSH TABLE <namespace>.<table_name>; -- Flush all tables in a namespaceSTORAGE FLUSH ALL IN <namespace>;STORAGE FLUSH ALL IN NAMESPACE <namespace>; -- Flush everythingSTORAGE FLUSH ALL;STORAGE COMPACT
Merge and optimize cold-tier Parquet segments:
-- Compact a specific tableSTORAGE COMPACT TABLE <namespace>.<table_name>; -- Compact all tables in a namespaceSTORAGE COMPACT ALL IN <namespace>;STORAGE COMPACT ALL IN NAMESPACE <namespace>; -- Compact everythingSTORAGE COMPACT ALL;SHOW MANIFEST
Display the storage manifest (schema + segment index):
SHOW MANIFEST;Last updated on