MinIO (S3-Compatible)

KalamDB uses the S3 storage backend for MinIO. You configure MinIO through CREATE STORAGE ... TYPE 's3' and the storage CONFIG JSON.

Quick Start with Docker Compose

The KalamDB repository includes a MinIO service in docker/utils/docker-compose.yml:

minio:
  image: minio/minio:latest
  environment:
    MINIO_ROOT_USER: minioadmin
    MINIO_ROOT_PASSWORD: minioadmin
    MINIO_CI: "on"
  ports:
    - "9120:9000"   # S3 API
    - "9121:9001"   # Console UI
  volumes:
    - ./minio-data:/data

Start it:

docker compose up minio -d

MinIO Console: http://127.0.0.1:9121 (login: minioadmin / minioadmin)
S3 API Endpoint: http://127.0.0.1:9120

The compose entrypoint auto-creates a kalamdb-test bucket and sets public anonymous access for development.

S3 CONFIG Fields

For S3-compatible backends (including MinIO), KalamDB supports these CONFIG JSON fields:

Example: Register MinIO Storage

CREATE STORAGE s3_minio
  TYPE 's3'
  NAME 'MinIO Local Storage'
  DESCRIPTION 'MinIO bucket for local development'
  BASE_DIRECTORY 's3://kalamdb-test/'
  CONFIG '{
    "type": "s3",
    "region": "us-east-1",
    "endpoint": "http://127.0.0.1:9120",
    "allow_http": true,
    "access_key_id": "minioadmin",
    "secret_access_key": "minioadmin"
  }'
  SHARED_TABLES_TEMPLATE '{namespace}/{tableName}'
  USER_TABLES_TEMPLATE '{namespace}/{tableName}/{userId}';

S3 Factory Behavior

When a custom endpoint is set (as with MinIO), the factory applies special handling:

• Path-style requests: virtual_hosted_style_request is set to false automatically. This is required for MinIO and most S3-compatible services.
• Timeout skip: ClientOptions (request/connect timeouts from [storage.remote_timeouts]) are not applied to S3-compatible endpoints. This avoids timeout conflicts with non-AWS backends. Timeouts only apply to standard AWS S3.
• Region fallback: If region is omitted, it defaults to "us-east-1".

Notes on CONFIG

• Use an s3://... BASE_DIRECTORY for S3-compatible storage.
• allow_http = true is required for any non-TLS endpoint (local MinIO, test environments).
• The type field in the CONFIG JSON must match the TYPE in the SQL command.

Remote Timeout Tuning (server.toml)

Remote object store timeouts are configured globally in server.toml:

[storage.remote_timeouts]
request_timeout_secs = 60
connect_timeout_secs = 10

Important: These timeouts apply to standard AWS S3 only. They are skipped for S3-compatible endpoints (MinIO, etc.) to avoid conflicts.

Credential Placement

Recommended options:

• Development: Put credentials in CONFIG JSON directly.
• Production (AWS S3): Use IAM instance roles or environment-based AWS credential chains — omit access_key_id/secret_access_key from CONFIG.
• Production (MinIO): Use scoped credentials; rotate regularly.
• Avoid committing production secrets to source control.

Verify Storage Health

After creating the storage, verify connectivity:

STORAGE CHECK s3_minio;
STORAGE CHECK s3_minio EXTENDED;

Other Cloud Storage Backends

KalamDB also supports GCS and Azure Blob Storage. See Storage Commands for all backend types and CONFIG formats.

Related Resources

• Storage Commands
• Storage ID Usage
• Storage Tiers