EZGH Queues stores messages between the parts of your application that produce work and the parts that do it. Producers send messages to a queue. Consumers either receive them with a long poll and acknowledge each one when they’ve handled it, or have Queues push them in batches to an HTTPS endpoint. A message that keeps failing moves to a dead-letter queue, where you can inspect it and send it back.
- Standard queues deliver every message at least once, in roughly the order sent.
- FIFO queues deliver the messages of each message group in order, one batch at a time, and don’t store a send whose deduplication ID they saw in the last 5 minutes.
Quickstart
Create a queue, send a message, receive it and acknowledge it, with curl or the ezgh CLI.
Concepts
Queues, messages, visibility timeouts, receipt handles and retention.
Pull consumers
Long-poll receive, acknowledging, and extending or releasing messages.
Push subscriptions
Batches pushed to your HTTPS endpoint, signature checks and partial failures.
FIFO queues
Message groups, ordering and deduplication.
Dead-letter queues
Redrive policies, inspecting failed messages and moving them back.
IAM recipes
Producers, consumers, operators, and policies for one queue.
Queues and Workflows
Start workflow runs from a queue, resume waiting runs, send from a step, and get run results.
Limits
Sizes, rate limits, quotas and usage.
Hosts
| Host | What it serves |
|---|---|
https://queues-api.ezghcloud.com |
The Queues API: queues, subscriptions, purges, redrives and limits; sending and peeking at messages. Called with an API key or your console session |
https://queues-live.ezghcloud.com |
Receiving messages, with an API key or your console session; acknowledging them and changing their visibility, with their receipt handles and no credential |
The console’s Queues pages (in Application integration) use both hosts.
Permissions
| Action | Allows | Checked against |
|---|---|---|
queues.queues.list |
List queues | Project |
queues.queues.get |
View a queue, its settings and its message counts | Queue |
queues.queues.create |
Create queues | Project |
queues.queues.update |
Change a queue’s settings and tags | Queue |
queues.queues.delete |
Delete a queue, its subscription and its messages | Queue |
queues.queues.purge |
Delete every message in a queue | Queue |
queues.messages.send |
Send messages | Queue |
queues.messages.receive |
Receive messages, which hides them from other consumers | Queue |
queues.messages.peek |
View a queue’s messages without receiving them | Queue |
queues.redrives.start |
Move a dead-letter queue’s messages back | The dead-letter queue, and the destination queue when you name one |
queues.redrives.list |
List a dead-letter queue’s redrives | Queue |
queues.redrives.get |
View a redrive’s progress | Queue |
queues.subscriptions.list |
List subscriptions | Project |
queues.subscriptions.get |
View a subscription and its delivery status | Subscription |
queues.subscriptions.create |
Create a queue’s subscription | Queue |
queues.subscriptions.update |
Change a subscription, or rotate its signing secret | Subscription |
queues.subscriptions.delete |
Delete a subscription | Subscription |
queues.subscriptions.pause |
Pause and resume a subscription | Subscription |
queues.limits.get |
View a project’s rate limits and caps | Project |
Acknowledging a message and changing its visibility need no permission: the message’s receipt handle, returned by a receive, is what allows them. See Pull consumers.
| Managed policy | What it allows |
|---|---|
QueuesFullAccess |
Every queues.* action. |
QueuesReadOnlyAccess |
Every list and view action, including peeking at messages: queues and their counts, messages, redrives, subscriptions and limits. Not receiving, which consumes messages. |
ReadOnlyAccess allows what QueuesReadOnlyAccess does. AdministratorAccess and the root user
can do all of these. See IAM recipes.
Resources
| Resource | Resource name |
|---|---|
| Queue | ezgh:us-west-1:<org>:<project>:que_<slug> |
| Subscription | ezgh:us-west-1:<org>:<project>:qsb_<slug> |
Paths name queues and subscriptions by their id, a UUID. The slug (que_ or qsb_ and 12
characters) is in the resource name.
Data
- Message bodies and attribute values are never logged or recorded in Trails. A message is kept until it’s acknowledged, its queue’s retention ends, or it’s purged or deleted with its queue.
- A subscription’s signing secret is stored encrypted and shown once, when it’s created or rotated.
Trails
Queues’ events have the event source queues.ezghcloud.com.
- Recorded: every call to the Queues API, as the rules for reads
say, with event names such as
CreateQueue,PurgeQueue,StartRedrive,CreateSubscriptionandRotateSubscriptionSecret. PeekMessages is recorded as a data event, without the messages. - Recorded only when refused: SendMessages and
ReceiveMessages, for example a
403. - Never recorded: acknowledgements and visibility changes, message bodies and attributes, and signing secrets.