Skip to content

Topics and consumer groups

List, create and change Kafka topics, and view, reset and delete consumer groups.

Updated View as Markdown

A Kafka cluster’s Topics and Consumer groups tabs list everything in the broker: what you created through EZGH (Created in EZGH, origin: api) and what apps created themselves (Created by app, origin: broker). You can change or delete either kind. EZGH never deletes or rewrites a topic you didn’t ask it to: a topic keeps the partitions, replication factor and settings it was created with.

Each object, and each list, has observedAt: the time the cluster reported it.

Topics

ezgh hosted-queues topics list hqc_4k2m9x0a7qpe
ezgh hosted-queues topics create hqc_4k2m9x0a7qpe orders --partitions 6 \
  --config retention.ms=259200000 --wait

A topic has partitions, replicationFactor, config and messagesPerSecond.

  • Partitions can be added, never removed: ezgh hosted-queues topics update … --partitions 12.
  • Replication factor is set when a topic is created. On a replicated cluster, a topic created through EZGH, or by an app without its own replication factor, has 3 copies; a topic an app created with its own keeps it. It can be raised later, never lowered: ezgh hosted-queues topics update … --replication-factor 3, or every topic at once with Raise to 3 copies.
  • config takes retention.ms, retention.bytes, cleanup.policy, min.insync.replicas, max.message.bytes, compression.type, segment.bytes, segment.ms, delete.retention.ms, min.compaction.lag.ms, max.compaction.lag.ms, min.cleanable.dirty.ratio and message.timestamp.type, and can change later.

Deleting a topic deletes its messages:

ezgh hosted-queues topics delete hqc_4k2m9x0a7qpe orders --wait

Kafka’s internal topics, such as __consumer_offsets, are listed and can’t be changed.

Cluster settings

Setting
autoCreateTopics Whether a topic is created when an app first uses it. On by default.
defaultRetentionMs How long topics keep messages unless they set retention.ms. 7 days by default.
defaultPartitions Partitions for automatically created topics.
ezgh hosted-queues clusters update hqc_4k2m9x0a7qpe --auto-create-topics=false --wait

Consumer groups

ezgh hosted-queues consumer-groups list hqc_4k2m9x0a7qpe
ezgh hosted-queues consumer-groups get hqc_4k2m9x0a7qpe billing

A group shows its state, its members, and its lag per topic and in total. Groups are created by the consumers that join them.

A group with no members can have its offsets reset, to earliest, latest or a time, or be deleted:

ezgh hosted-queues consumer-groups reset-offsets hqc_4k2m9x0a7qpe billing --topic orders --to earliest --wait
ezgh hosted-queues consumer-groups delete hqc_4k2m9x0a7qpe billing --wait

Both are refused while the group has members. Lag is also a metric, kafka_consumergroup_lag, in the project’s Observability.

Names in the API

Topic and group names go in the URL path, percent-encoded: orders.v1 as is, a/b as a%2Fb.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close