Skip to content

Create a cluster

Choose an engine, version, mode, size and storage, decide who can connect, and set the maintenance window.

Updated View as Markdown

Before you start

A cluster needs a subnet of a VPC. It’s reached only from inside the VPC: there’s no public address.

Its first user’s password goes to a new Secret in the project, written as you, so creating a cluster also needs secrets.secrets.create and secrets.versions.create in the project.

Create

  1. Open Hosted Queues and choose Create cluster.
  2. Pick Kafka or RabbitMQ, a version, and Single node or Replicated.
  3. Pick a size, storage and a subnet, and whether to serve the plaintext port.
  4. Review, then choose Create cluster.
ezgh hosted-queues clusters create events --engine kafka --engine-version 4.4 --mode replicated \
  --instance-type gp-1.medium --storage-size-gib 100 --subnet sbn_2m9x0a7qpe4k --wait
ezgh hosted-queues clusters create jobs --engine rabbitmq --engine-version 4.3 --mode single \
  --instance-type gs-1.small --storage-size-gib 20 --subnet sbn_2m9x0a7qpe4k --wait

A create takes minutes, and --wait waits until the cluster is available. The cluster exists from the start, creating, with its ID (hqc_…). Its endpoint is null until its names resolve.

The cluster comes with one user, app, with full access to its topics or vhosts, whose password is in a Secret. See Users and permissions.

Settings

Setting
engine kafka or rabbitmq. Fixed.
engineVersion ezgh hosted-queues clusters engine-versions lists them. Upgrades move to a newer one.
mode single or replicated. A single-node cluster can grow to replicated; a replicated one stays replicated.
instanceType The size of each node, a gp-1 or gs-1 type; every node has the same one. hp-1 types take single-node clusters only. Can change later.
storage Each node’s storage: sizeGiB, and optionally iops and throughputMBps. Can grow later.
subnet The subnet it’s in. Fixed.
allowedSourceCidrs Who can connect.
plaintextPort Whether port 9092 (Kafka) or 5672 (RabbitMQ) also serves without TLS. See The plaintext port.
settings Kafka: autoCreateTopics, defaultRetentionMs, defaultPartitions. See Topics.
maintenance.windowUtc The maintenance window.
deletionProtection Refuses deletes while on.

Storage

Each node keeps everything it’s sent on its storage. A cluster is storage_low when more than 85% of its storage is used, and the project’s notifications say so. Grow it while it serves:

ezgh hosted-queues clusters update hqc_4k2m9x0a7qpe --storage-size-gib 200 --wait

Storage only grows. When it’s full, RabbitMQ blocks publishers until there’s room, and Kafka stops taking writes. Kafka topics keep messages for 7 days by default (defaultRetentionMs, or a topic’s own retention.ms).

Who can connect

By default, any address in the cluster’s VPC can connect to its client ports. allowedSourceCidrs narrows that to the ranges you list:

ezgh hosted-queues clusters update hqc_4k2m9x0a7qpe --allowed-source-cidr 10.0.1.0/24 \
  --allowed-source-cidr 10.0.2.0/24 --wait

An instance’s own firewall also has to allow its outbound connection. A firewall rule can name the cluster as its peer (hqc_4k2m9x0a7qpe) in place of its addresses.

Maintenance

Patch releases and other maintenance happen in the cluster’s maintenance window (maintenance.windowUtc, such as sun:06:00-sun:07:00). A replicated cluster is maintained one node at a time while the others serve. See Timeouts and reconnects.

Change the size

ezgh hosted-queues clusters update hqc_4k2m9x0a7qpe --instance-type gp-1.large --wait

A replicated cluster moves one node at a time while the others serve.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close