---
title: "Data reference"
description: "The gen_ai.* attributes AI Center reads, the content limit and media placeholders, the generated ezgh_genai_* metrics, retention, billing and permissions."
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.ezghcloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data reference

## Attributes read

A span is a GenAI span when it has `gen_ai.operation.name`. AI Center reads these attributes from
it:

| Attribute | Used as |
| --- | --- |
| `gen_ai.operation.name` | The operation: `chat`, `text_completion`, `generate_content` and `embeddings` are model calls, `execute_tool` tool calls and `invoke_agent` agent invocations |
| `gen_ai.provider.name` | The provider |
| `gen_ai.response.model`, else `gen_ai.request.model` | The model |
| `gen_ai.agent.name` | The agent |
| `gen_ai.tool.name` | The tool |
| `gen_ai.tool.call.id` | Matches a tool call span to the tool call in the messages |
| `gen_ai.conversation.id` | The conversation |
| `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | Input and output tokens |
| `gen_ai.usage.cache_read.input_tokens`, `gen_ai.usage.cache_creation.input_tokens` | Cache read and cache write tokens |
| `gen_ai.response.finish_reasons` | The finish reasons |
| `error.type`, and the span's status | An error: status `ERROR` or an `error.type` |

The service and environment come from the span's resource, as for every span: `service.name` and
the dataset's environment.

## Content

| Attribute | Content |
| --- | --- |
| `gen_ai.system_instructions` | System instructions |
| `gen_ai.input.messages` | Input messages |
| `gen_ai.output.messages` | Output messages |
| `gen_ai.tool.definitions` | Tool definitions |
| `gen_ai.tool.call.arguments` | A tool call's arguments |
| `gen_ai.tool.call.result` | A tool call's result |

Content is read from the span's attributes, and from the attributes of a
`gen_ai.client.inference.operation.details` span event; when both have the same attribute, the
span's wins. Send each content attribute as a string: the JSON text the conventions describe, or any other
text. A string is kept as sent.

Content is stored apart from its span. In [Traces](/observability/traces), a GenAI span has no
content attributes, and its operation details events keep their other attributes. The span has
`ezgh.genai.content_bytes` instead: the size of its content as sent, in bytes. Observability sets
it; a value you send is replaced.

### Text only

AI Center keeps text. In system instructions and input and output messages, a part whose `type`
is `text`, `tool_call`, `tool_call_response` or `reasoning` is kept as sent. Any other part, such
as an image, audio or a file (`blob`, `file`, `uri`), is replaced by a placeholder with its type,
MIME type (`null` when the part has none) and the size of its data as sent, in bytes (for a
`blob`, the length of its base64 text):

```json
{ "type": "blob", "mime_type": "image/png", "bytes": 48213 }
```

### Limits

| Limit | Value |
| --- | --- |
| Content per span | 1 MiB, after media parts are replaced |
| Span size | 64 KiB, with its resource and scope, not counting its content |

A GenAI span's content doesn't count toward the [span size limit](/observability/traces#limits);
every other trace limit applies as usual, request sizes included.

Past 1 MiB, content is cut: fields are kept in the order of the [table above](#content), the field
that reaches the limit is cut at a character boundary (its JSON is then incomplete), and later
fields are left out. The span is stored, and its step in the run has `truncated` set, with the
size as sent in `contentBytes`.

### Sampled datasets

In a dataset with [trace sampling](/observability/sampling), content and runs are stored for kept
traces only: a dropped trace's run isn't in AI Center. The [generated metrics](#generated-metrics)
count every GenAI span received, kept or dropped, as [span metrics](/observability/span-metrics)
do.

## Generated metrics

Observability generates these series from GenAI spans, in each environment's dataset, and reads
them with PromQL like [span metrics](/observability/span-metrics). They're free: not counted in
ingested or stored bytes, a dataset's active series, or ingest rate limits. They power the
overview, and work in dashboards and [alert rules](/observability/alerting/rules#metrics-promql).

| Series | Type | Counts | Labels |
| --- | --- | --- | --- |
| `ezgh_genai_tokens_total` | Counter | Tokens in model calls | `type` (`input`, `output`, `cache_read`, `cache_write`), `operation`, `provider`, `model`, `agent` |
| `ezgh_genai_calls_total` | Counter | Model calls | `status`, `operation`, `provider`, `model`, `agent` |
| `ezgh_genai_call_duration_seconds_bucket`, `_sum`, `_count` | Histogram, in seconds | Model call durations | `operation`, `provider`, `model`, `agent` |
| `ezgh_genai_tool_calls_total` | Counter | Tool calls | `status`, `tool`, `agent` |
| `ezgh_genai_tool_duration_seconds_bucket`, `_sum`, `_count` | Histogram, in seconds | Tool call durations | `tool`, `agent` |
| `ezgh_genai_agent_invocations_total` | Counter | Agent invocations | `status`, `agent` |
| `ezgh_genai_agent_duration_seconds_bucket`, `_sum`, `_count` | Histogram, in seconds | Agent invocation durations | `agent` |

- `status` is `ok` or `error`.
- `agent` is the span's own `gen_ai.agent.name`, absent when the span has none. Instrumentations
  that set it only on `invoke_agent` spans have tool and model call series without `agent`.
- Every series also has `job` (the service key), `environment` and `application`, as span metrics
  do. Histograms have `le`.
- A `provider`, `model`, `agent` or `tool` value over 128 bytes, or with control characters, is
  `__other__`.

| Histogram | Bucket bounds, in seconds |
| --- | --- |
| Model calls | `0.25`, `0.5`, `1`, `2`, `4`, `8`, `15`, `30`, `60`, `120`, `+Inf` |
| Tool calls | `0.01`, `0.025`, `0.05`, `0.1`, `0.25`, `0.5`, `1`, `2.5`, `5`, `10`, `30`, `+Inf` |
| Agent invocations | `1`, `2`, `5`, `10`, `20`, `30`, `60`, `120`, `300`, `600`, `+Inf` |

| Cap | Value | Past it |
| --- | --- | --- |
| Models, agents, tools | 200 distinct values each per environment, each hour | New values in that hour are counted as `__other__` |
| Generated series | About 20,000 per environment, apart from span metrics' | New series aren't written |

The series are written and counted as span metrics are; see
[Accuracy](/observability/span-metrics#accuracy). Your own `gen_ai.client.*` metrics, if you send
them, are stored as sent.

```promql
# Tokens per hour by agent and type
sum by (agent, type) (increase(ezgh_genai_tokens_total{environment="production"}[1h]))

# Cache hit rate: cache read tokens over input tokens
sum(increase(ezgh_genai_tokens_total{environment="production", type="cache_read"}[1h]))
  / sum(increase(ezgh_genai_tokens_total{environment="production", type="input"}[1h]))

# p95 model call latency by model
histogram_quantile(0.95, sum by (model, le) (rate(ezgh_genai_call_duration_seconds_bucket{environment="production"}[5m])))

# Tool error ratio by tool
sum by (tool) (rate(ezgh_genai_tool_calls_total{environment="production", status="error"}[5m]))
  / sum by (tool) (rate(ezgh_genai_tool_calls_total{environment="production"}[5m]))
```

## Retention

Runs and their content follow the dataset's traces: they're kept for its
[retention class](/observability#retention-classes), in hot and cold storage, and leave with
their spans. There's no separate retention setting. The generated metrics follow the dataset's
metrics retention.

## Billing

GenAI spans are traces. A GenAI span is metered as traces ingested bytes at its size as sent,
content included, and its content's storage is counted in traces stored bytes; see
[Pricing](/observability/traces#pricing). AI Center has no meter of its own, and the generated
metrics are free.

## Permissions

| To | Needs |
| --- | --- |
| See the overview, and pick filter values on Runs and Conversations | `obs.metrics.query` |
| List and search runs and conversations | `obs.traces.search` |
| Read a run's steps, and its waterfall | `obs.traces.getSpans` |
| Read a run's content: prompts, completions, tool arguments and results | `obs.genai.readContent` |
| Search content (`text`, `--text`, the Runs search box) | `obs.traces.search` and `obs.genai.readContent` |

All are checked against the dataset. `ReadOnlyAccess`, `ObsReadOnlyAccess` and `ObsFullAccess`
include the first three. `obs.genai.readContent` has access level Write and is only in
`ObsFullAccess`; give it to others with a custom policy. Without it, the console shows a run's
steps without their text, disables the Runs search box and names the permission. See
[Policies](/iam/policies).

Reads aren't recorded in [Trails](/trails), as for other trace reads.

Source: https://docs.ezghcloud.com/observability/ai-center/data-reference/index.mdx
