Project limits
| Limit | Default | Over it |
|---|---|---|
runStartsPerSecond |
100 run starts a second, in bursts of up to 200 (runStartsBurst) |
Starting a run or triggering a schedule answers 429 too_many_requests with Retry-After |
openRuns |
100,000 runs that haven’t ended | Starting a run or triggering a schedule answers 429 too_many_requests with Retry-After |
concurrentInvocations |
1,000 invocations running at once | Further invocations wait until one ends. Nothing fails |
- Run starts are counted on StartRun and
TriggerSchedule. Runs started by a schedule’s timetable
and child workflows don’t count against
runStartsPerSecond. Retry-Afteris how long until a start would be allowed. The SDK clients andezghwait for it and retry.- Concurrent invocations count push deliveries and pull workers’ invocations together. Over the limit, push deliveries queue, and pull workers’ polls wait for a slot or answer with no tasks.
To see the limits in force for a project, open Service quotas in the console, run
ezgh workflows limits, or read them from the API
(workflows.limits.get). Each limit has
a name, value, unit, and a scope: default, or organization or project when it was
changed for you. New limits may be added to the list. To raise a limit, contact support.
$ ezgh workflows limits
NAME VALUE UNIT SCOPE
runStartsPerSecond 100 runsPerSecond default
runStartsBurst 200 runs default
concurrentInvocations 1,000 invocations default
openRuns 100,000 runs defaultService quotas
| Quota | Default | Over it |
|---|---|---|
workflows.apps.perProject |
100 apps per project | 409 quota_exceeded |
workflows.schedules.perProject |
1,000 schedules per project | 409 quota_exceeded |
See Service quotas.
Fixed limits
| What | Limit |
|---|---|
| A run’s input, a step’s result, a workflow’s output, a callback’s body, a schedule’s input | 1 MiB of JSON |
| Steps per run, child workflows included | 1,000 |
| Steps started together | 1,000 |
| Step invocations running at once, per run | 32; more wait their turn |
| Child workflow nesting | 10 levels |
| A run’s duration | 1 year |
| A run’s data after it ends | Kept 30 days, then deleted |
| A push invocation | The app’s maxInvocationSeconds, at most 15 minutes |
| A pull invocation | The longest timeout of the app’s announced workflows, from 15 minutes to 24 hours |
| A pull invocation without a heartbeat | 60 seconds, then it’s given to another worker |
| A pull app’s partitions | 1 to 32, 4 by default; they can grow, never shrink |
| Tasks per poll | 1 to 10 |
| A poll’s hold | 55 seconds |
| Workflows a pull worker serves | 1,000 |
| Callback completions by URL | 20 a second per client IP address, in bursts of up to 60 |
| A schedule’s cron | At most once a minute |
| A schedule’s jitter | 0 to 3,600 seconds |
| A schedule’s catch-up window | 0 to 86,400 seconds, 3,600 by default |
Usage
Workflows measures usage per app:
| Meter | Measures |
|---|---|
workflows.steps.executions |
Step attempts, retries included, and each invocation of a workflow’s own code |
workflows.runs.storedBytes |
The bytes of retained run data (steps, inputs, outputs and callback bodies), sampled hourly |
Both are priced at $0 at launch and nothing is charged for them. They don’t appear in Billing yet.
Deleting a project or an organization
While an organization’s deletion is pending, its Workflows stop starting new work:
- Its schedules are paused by EZGH (
pausedBy: "system"), and resumed if the deletion is cancelled. - Creating apps and schedules and starting runs answer
409 organization_deletion_pending. - A workflow’s child workflow fails to start with
organization_deletion_pending.
When a project or organization is deleted, within an hour EZGH:
- Cancels its runs that haven’t ended.
- Deletes its schedules, then its apps. These are recorded in Trails as events EZGH
made (
ServiceEvent), invoked byorganizations.ezghcloud.com. - Reports its final usage.
- Deletes its run data: steps, history, inputs, outputs and callback bodies, and its runs from the run list.