Broker

The broker subcommand is used to manage topics in a Nisshi cluster. If no subcommand is present, broker is the default.

nisshi broker --help

Usage: nisshi broker [OPTIONS] [COMMAND]

Commands:
  iceberg  Schema topics are written as Apache Iceberg tables
  delta    Schema topics are written as Delta Lake tables
  parquet  Schema topics are written in Parquet format
  help     Print this message or the help of the given subcommand(s)

Options:
      --cluster-id <CLUSTER_ID>
          All members of the same cluster should use the same id [env: CLUSTER_ID=nisshi] [default: nisshi_cluster] [aliases: --kafka-cluster-id]
      --listener-url <LISTENER_URL>
          The broker will listen on this address [env: LISTENER_URL=] [default: tcp://0.0.0.0:9092] [aliases: --kafka-listener-url]
      --advertised-listener-url <ADVERTISED_LISTENER_URL>
          This location is advertised to clients in metadata [env: ADVERTISED_LISTENER_URL=] [default: tcp://localhost:9092] [aliases: --kafka-advertised-listener-url]
      --storage-engine <STORAGE_ENGINE>
          Storage engine examples are: postgres://postgres:postgres@localhost, memory://nisshi/ or s3://nisshi/ [env: STORAGE_ENGINE=] [default: memory://nisshi/]
      --schema-registry <SCHEMA_REGISTRY>
          Schema registry examples are: file://./etc/schema or s3://nisshi/, containing: topic.json, topic.proto or topic.avsc [env: SCHEMA_REGISTRY=]
      --schema-registry-cache-expiry <SCHEMA_REGISTRY_CACHE_EXPIRY>
          Schema registry cache expiry duration
      --otlp-endpoint-url <OTLP_ENDPOINT_URL>
          OTEL Exporter OTLP endpoint [env: OTEL_EXPORTER_OTLP_ENDPOINT=]
  -h, --help
          Print help

The broker subcommand will automatically load environment variables from a file named .env in the current directory or any of its parents, and will be shown as [env: VARIABLE=value].

Example

nisshi broker --storage-engine=sqlite://nisshi -schema-registry=file://./etc/schema

Iceberg

Schema topics are written as Apache Iceberg tables

Usage: nisshi broker iceberg [OPTIONS] --location <LOCATION> --catalog <CATALOG>

Options:
      --location <LOCATION>    Apache Parquet files are written to this location, examples are: file://./lake or s3://lake/ [env: DATA_LAKE=s3://lake]
      --catalog <CATALOG>      Apache Iceberg Catalog, examples are: http://localhost:8181/ [env: ICEBERG_CATALOG=http://localhost:8181/catalog]
      --namespace <NAMESPACE>  Iceberg namespace [env: ICEBERG_NAMESPACE=] [default: nisshi]
      --warehouse <WAREHOUSE>  Iceberg warehouse [env: ICEBERG_WAREHOUSE=nisshi]
  -h, --help                   Print help

Delta

Schema topics are written as Delta Lake tables

Usage: nisshi broker delta [OPTIONS] --location <LOCATION>

Options:
      --location <LOCATION>
          Apache Parquet files are written to this location, examples are: file://./lake or s3://lake/ [env: DATA_LAKE=s3://lake]
      --database <DATABASE>
          Delta database [env: DELTA_DATABASE=] [default: nisshi]
      --records-per-second <RECORDS_PER_SECOND>
          Throttle the maximum number of records per second
  -h, --help
          Print help

Parquet

Schema topics are written in Parquet format

Usage: nisshi broker parquet --location <LOCATION>

Options:
      --location <LOCATION>  Apache Parquet files are written to this location, examples are: file://./lake or s3://lake/ [env: DATA_LAKE=s3://lake]
  -h, --help                 Print help