A delivery sends your organization’s Trails events, from the moment you create it, into one project’s Observability as log records. Trails keeps events for 30 days; the delivered copy stays as long as the destination dataset’s log retention, and you search it, chart it and alert on it like any other logs.
- Organization-wide: a delivery carries every event of the organization, from every project and the organization itself, into one project.
- One environment: the events go to the dataset of the environment you choose (
productionby default), which is created on first use like any other. - What Trails keeps, or reads too: a delivery carries the events Trails keeps (What Trails records). With Include read events, it also carries every read-only API call, most of which Trails itself doesn’t keep.
- From now on: events from before the delivery was created aren’t delivered.
An organization can have up to 5 deliveries.
Create a delivery
You need audit.deliveries.create, and obs.logs.ingest on the destination project (see
Permissions).
- In the console, open Trails and select the Deliveries tab.
- Select Create delivery.
- Enter a Name, choose the Project, and set the Environment (
productionunless you change it). - Turn on Include read events to deliver read-only API calls too.
- Select Create delivery. The delivery’s page opens.
The delivery’s page shows its status, when it last delivered and how many events were dropped. View logs opens the environment’s logs in Observability, filtered to Trails’ records. Edit changes the name and Include read events; Pause and Resume stop and restart delivery; Delete removes it.
ezgh trails deliveries create delivers to the project given by --project (or EZGH_PROJECT,
or the profile’s project):
ezgh trails deliveries create "Audit archive" --project security --environment audit
ezgh trails deliveries create "Everything" --project security --include-reads| Command | Does |
|---|---|
ezgh trails deliveries list |
Lists deliveries: name, destination, reads and status. |
ezgh trails deliveries get <delivery> |
Shows a delivery by ID or slug (tdl_…), with its status, last delivery, last error and dropped events. |
ezgh trails deliveries create <name> |
Creates a delivery. --environment (default production), --include-reads, and --enabled=false to create it paused. |
ezgh trails deliveries update <delivery> |
Changes only the flags given: --name, --include-reads=true|false, and --enabled=false to pause or --enabled=true to resume. |
ezgh trails deliveries delete <delivery> |
Deletes a delivery. It asks first; --yes confirms. |
$ ezgh trails deliveries list
NAME DESTINATION READS STATUS ID
Everything security · production yes active 0199a3f0-0000-7000-8000-0000000d0002
Audit archive security · audit no active 0199a3f0-0000-7000-8000-0000000d0001| Operation | Request | Needs |
|---|---|---|
| CreateTrailDelivery | POST /v1/organizations/<organization ID>/deliveries |
audit.deliveries.create, and obs.logs.ingest on the project |
| ListTrailDeliveries | GET …/deliveries |
audit.deliveries.list |
| GetTrailDelivery | GET …/deliveries/<delivery ID or slug> |
audit.deliveries.get |
| UpdateTrailDelivery | PATCH …/deliveries/<delivery ID or slug> |
audit.deliveries.update, and obs.logs.ingest on the project to turn enabled or includeReads on |
| DeleteTrailDelivery | DELETE …/deliveries/<delivery ID or slug> |
audit.deliveries.delete |
curl -H "Authorization: Bearer $EZGH_API_KEY" -H "Content-Type: application/json" \
-H "Idempotency-Key: $(uuidgen)" \
-d '{"name":"Audit archive","projectId":"'"$PROJECT_ID"'","environment":"audit","includeReads":false}' \
"https://trails.ezghcloud.com/v1/organizations/$ORG_ID/deliveries"{
"id": "0199a3f0-0000-7000-8000-0000000d0001",
"slug": "tdl_4k2m9x0a7qpe",
"resourceName": "ezgh:us-west-1:org_k3f9a0x2m7qp:tdl_4k2m9x0a7qpe",
"organizationId": "0199a3c2-5b1e-7d40-9f3a-2c8e6b1d4f70",
"region": "us-west-1",
"name": "Audit archive",
"projectId": "0199a3c2-6c2f-7e51-8a4b-3d9f7c2e5a81",
"environment": "audit",
"includeReads": false,
"enabled": true,
"status": "active",
"lastDeliveredAt": null,
"lastError": null,
"lastErrorAt": null,
"droppedEvents": 0,
"createdAt": "2026-10-14T09:00:00.000Z",
"updatedAt": "2026-10-14T09:00:00.000Z"
}CreateTrailDeliveryanswers201with aLocationheader. It takes anIdempotency-Key: a retry with the same key within 24 hours answers the first201again, withIdempotent-Replayed: true.nameis 1 to 100 characters.environmentis trimmed and lowercased, then must match^[a-z0-9][a-z0-9._-]{0,62}$.includeReadsdefaults tofalseandenabledtotrue.projectIdandenvironmentcan’t change: aPATCHnaming them is400 invalid_request. Create another delivery instead.- A project that isn’t a live project of the organization is
400 invalid_request. - Without
obs.logs.ingeston the project, the answer is403 access_denied, naming it. - The sixth delivery is
409 quota_exceeded, with the quotaaudit.deliveries.perOrganization(an allocation of 5) and how many are in use. - The list is newest first, 20 a page by default and at most 100 (
limit), withnextCursorfor the next page (cursor). DeleteTrailDeliveryanswers204.
Read events and billing
Delivered records are ordinary Observability logs in the destination project: their bytes and storage count in that project’s Observability meters and go through its rate limits and transform rules. Trails itself adds no charge.
Include read events delivers every read-only API call as well, such as reading a project or listing bots. Reads are most API calls; they’re billed as Observability log ingest in the destination project like the rest.
The record
Each event becomes one log record. Its body is the event’s JSON exactly, as the Trails API returns it, so anything that reads Trails events can read the delivered copy.
Resource attributes
| Key | Value |
|---|---|
service.name |
ezgh.trails |
ezgh.metadata.application |
trails |
ezgh.metadata.environment |
The delivery’s environment |
ezgh.metadata.project |
The event’s project ID; absent for organization-level events |
Record
| Field | Value |
|---|---|
| Time | The event’s eventTime |
| Observed time | When Trails received the event |
| Severity | WARN when the event has an errorCode, else INFO |
| Body | The event’s JSON, as a string |
| Scope | trails.ezghcloud.com/delivery |
Record attributes copy the event’s main fields, for search. A field the event doesn’t have is left out, never sent empty.
| Key | From |
|---|---|
trails.eventId, trails.eventName, trails.eventSource, trails.eventType |
The same fields |
trails.readOnly |
"true" or "false" |
trails.errorCode |
errorCode |
trails.userIdentity.type, trails.userIdentity.principalId, trails.userIdentity.userName, trails.userIdentity.accessKeyId, trails.userIdentity.invokedBy |
The same fields of userIdentity |
trails.sourceIpAddress, trails.requestId |
The same fields |
trails.organizationId |
organizationId |
trails.resources |
The resources’ names, comma-separated |
trails.deliveryId |
The delivery |
An example record, as OTLP JSON (the body shortened):
{
"timeUnixNano": "1791964800123000000",
"observedTimeUnixNano": "1791964801250000000",
"severityNumber": 9,
"severityText": "INFO",
"body": { "stringValue": "{\"additionalEventData\":null,\"errorCode\":null,…,\"eventName\":\"CreateDataset\",…}" },
"attributes": [
{ "key": "trails.eventId", "value": { "stringValue": "0199a3f2-7c1d-7a4b-9e2f-1b3c4d5e6f70" } },
{ "key": "trails.eventName", "value": { "stringValue": "CreateDataset" } },
{ "key": "trails.eventSource", "value": { "stringValue": "obs.ezghcloud.com" } },
{ "key": "trails.eventType", "value": { "stringValue": "ApiCall" } },
{ "key": "trails.readOnly", "value": { "stringValue": "false" } },
{ "key": "trails.userIdentity.type", "value": { "stringValue": "User" } },
{ "key": "trails.userIdentity.principalId", "value": { "stringValue": "user_7f3k2m" } },
{ "key": "trails.userIdentity.userName", "value": { "stringValue": "ada@example.com" } },
{ "key": "trails.sourceIpAddress", "value": { "stringValue": "203.0.113.7" } },
{ "key": "trails.requestId", "value": { "stringValue": "req_1" } },
{ "key": "trails.organizationId", "value": { "stringValue": "01990000-0000-7000-8000-000000000001" } },
{ "key": "trails.resources", "value": { "stringValue": "ezgh:us-west-1:org_k3f9a0x2m7qp:prj_x:ds_web" } },
{ "key": "trails.deliveryId", "value": { "stringValue": "01990000-0000-7000-8000-0000000000d1" } }
]
}Your project’s transform rules apply to delivered records like any others. Trails remains the record of what happened for its 30 days.
Search the logs
In the console, open Observability, then Logs, choose the delivery’s environment, and
filter on the service ezgh.trails; a delivery’s View logs opens exactly that. Add
attribute filters such as trails.eventName, trails.userIdentity.userName or
trails.userIdentity.accessKeyId.
With the CLI:
# Every bot deleted in the last day.
ezgh obs search --project security --environment audit --from 1d \
--service ezgh.trails --attr trails.eventName=DeleteBot
# Refused calls.
ezgh obs search --project security --environment audit --from 7d \
--service ezgh.trails --attr trails.errorCode=access_deniedStatus and dropped events
| Status | Means |
|---|---|
active |
Delivering. |
paused |
Turned off (enabled: false). Nothing is delivered until it’s resumed; events from while it was paused aren’t delivered. |
failing |
The last attempt to deliver failed after the last success. lastError says why. |
A delivery that can’t deliver (Observability unavailable, or the project’s limits refusing it)
keeps its events and retries, with increasing waits, for up to 3 days. Events still
undelivered after 3 days are dropped. droppedEvents counts every event that was never
delivered: those dropped after 3 days, and records Observability rejected, such as one whose
eventTime falls outside the dataset’s accepted window.
Delivery is at least once: after a retry, a record can arrive twice. Use trails.eventId to
tell repeats apart.
Deleting the destination project, or the organization, deletes its deliveries. Deleting a delivery drops the events it hasn’t delivered; what it delivered stays in Observability.
Permissions
| Action | Allows | In AdministratorAccess and TrailsFullAccess |
In ReadOnlyAccess |
|---|---|---|---|
audit.deliveries.list |
List the organization’s deliveries | Yes | Yes |
audit.deliveries.get |
View a delivery and its status | Yes | Yes |
audit.deliveries.create |
Create a delivery | Yes | No |
audit.deliveries.update |
Rename, pause or resume a delivery, or turn read events on or off | Yes | No |
audit.deliveries.delete |
Delete a delivery | Yes | No |
Creating a delivery, or turning enabled or includeReads on, sends events into the
destination project’s Observability, so it also needs obs.logs.ingest on that project, as
creating an ingest key does. Reading the delivered logs needs the
usual Observability actions, such as obs.logs.search.
Creating, changing and deleting a delivery are recorded in Trails as CreateTrailDelivery,
UpdateTrailDelivery and DeleteTrailDelivery, with the delivery (TrailDelivery) in
resources. A delivery deleted with its project or organization is a service event by
organizations.ezghcloud.com.