EZGH Hosted Queues runs RabbitMQ and Kafka clusters for you, in a subnet of your VPC, for apps that need a broker: Sentry and PostHog on Kafka, Celery and MassTransit apps on RabbitMQ. You point an app at the cluster’s name with a user and password that are already in Secrets, and the app’s stock settings work.
Create a cluster
Engines, versions, modes, sizes, storage, who can connect and maintenance.
Connect your apps
Names, ports, TLS and the CA bundle, the plaintext port and client versions.
Timeouts and reconnects
What clients see during maintenance and upgrades, and how to set them up.
Connect Sentry
Self-hosted Sentry on a Kafka cluster.
Connect PostHog
Self-hosted PostHog on a Kafka cluster.
Connect Celery
Celery 5.5 and later on RabbitMQ, with quorum queues.
Connect MassTransit
.NET apps using MassTransit on RabbitMQ.
Users and permissions
Users, their permissions on each engine, and their passwords in Secrets.
Topics and consumer groups
Kafka topics, their settings, and consumer groups and their offsets.
Vhosts and queues
RabbitMQ virtual hosts, queues, exchanges, bindings and policies.
Single node and replicated
What each mode does, and growing a cluster to replicated.
Upgrades
Moving a cluster to a newer engine version.
The Hosted Queues API is at https://hostedqueues-api.ezghcloud.com. In the console, open
Hosted Queues. Hosted Queues is a separate product from Queues.
Engines
| Kafka | RabbitMQ | |
|---|---|---|
| Versions | 4.3, 4.4 | 4.3, 4.4 |
| Modes | single or replicated |
single or replicated |
| Ports | 9093 (SASL_SSL); 9092 (SASL_PLAINTEXT, optional) |
5671 (AMQPS); 5672 (AMQP, optional) |
| Authentication | SASL/SCRAM-SHA-512 or SCRAM-SHA-256 | Username and password |
| Contents | Topics and consumer groups | Vhosts, queues, exchanges, bindings and policies |
single: one node.replicated: three nodes, each with its own copy of the data. A replicated cluster keeps serving, and loses no acknowledged message, when any one node stops. See Single node and replicated.- RabbitMQ clusters include the consistent-hash exchange (
x-consistent-hash). There’s no management UI, delayed-message exchange, MQTT, STOMP, shovel or federation.
Status
| Status | Means |
|---|---|
creating |
It’s being created. |
available |
It’s serving. |
modifying |
A change is being applied. It serves meanwhile. |
upgrading |
It’s moving to a new engine version. See Upgrades. |
degraded |
(Replicated) It’s serving with a node out of service, and returns to available by itself. |
recovering |
(Single node) It’s coming back after its node stopped. Clients reconnect when it’s available. |
storage_low |
More than 85% of its storage is used. Grow the storage. |
deleting |
It’s being deleted. |
deleted |
It’s gone. |
failed |
It couldn’t be created. |
New statuses may appear.
Monitoring
Every cluster sends its metrics to its project in Observability, free and
automatically, every 30 seconds, with the cluster’s ID as the cluster label and the node’s index
(0, 1 or 2) as node: on Kafka, bytes and messages per topic, partitions, consumer group lag
(kafka_consumergroup_lag, by group and topic), request latency and storage; on RabbitMQ, ready and
unacknowledged messages per queue, publish, deliver and ack rates, consumers, connections, channels,
alarms and quorum queue health. The cluster’s Monitoring tab shows a built-in dashboard for its
engine. Its Events tab lists maintenance, upgrades and growth, newest first:
ezgh hosted-queues clusters events hqc_4k2m9x0a7qpeDelete a cluster
ezgh hosted-queues clusters delete hqc_4k2m9x0a7qpe --waitDeleting a cluster deletes its data and its users’ Secrets.
It deletes the Secrets as you, so it also needs secrets.secrets.delete. It’s refused with
409 deletion_protected while deletion protection is on (deletionProtection).
Pricing
Each node of a cluster is billed by the hour at the hourly price of an instance of its instance type: one node for a single-node cluster, three for a replicated one. Storage isn’t billed yet. See Pricing for current prices.
Limits
| Limit | Value |
|---|---|
| Clusters per project | 5 by default, up to 40 |
| Users per cluster | 50 by default, up to 200 |
| Kafka clients | Version 2.1 or later (see Client versions) |
IAM
| Policy | Allows |
|---|---|
HostedqueuesFullAccess |
Everything here. |
HostedqueuesReadOnlyAccess |
Listing and viewing clusters, users and broker objects. ReadOnlyAccess includes the same. |
Actions are hostedqueues.<collection>.<verb>, such as hostedqueues.clusters.create,
hostedqueues.topics.create and hostedqueues.queues.purge. Creating a cluster, creating a user and
rotating a password write a Secret as you, so they also need secrets.secrets.create and
secrets.versions.create in the project; deleting a user or a cluster deletes Secrets, so it needs
secrets.secrets.delete.