A command run sends one command, or one version of a saved script, to a set of nodes. Each node’s part is an invocation, which reports a status and an exit code. Output isn’t kept: the agent reads and discards what the command prints.
Run a command
- Open Fleets > Command runs and choose Run command, or choose Run command on a node’s page.
- Choose the targets (nodes, or tags), the account, and a command or a script with its parameters. Set the limits if the defaults don’t suit.
- The run’s page shows each node’s invocation as it runs.
ezgh fleets run --node web-1 --node web-2 --as root --command 'systemctl restart nginx' --wait
ezgh fleets run --tag env=prod --tag role=web --as root --script restart-service \
--param service=nginx --max-concurrency 25% --max-errors 1
ezgh fleets runs get cmd_7k2m9x0a4qpeWith --wait, ezgh waits for the run to end, prints each node’s result and exits 1 unless
every invocation succeeded.
SendCommand with targets, osUser and exactly one of command
or script. ListCommandInvocations lists each node’s
result.
- Targets are nodes by ID, slug or name, or every node in the project with all the given tags. Tags are matched once, when the run is created. A run has 1 to 1,000 nodes.
- The account is required, and defaults to
ezgh-userin the console andezgh. - A command is run with
/bin/sh -c, at most 64 KiB. - Access is checked once, for every target, when the run is created. If you may not run it on
any one target, the whole run is refused (
403, naming the node), so a run never partly starts.
Limits
| Setting | Default | Range |
|---|---|---|
| Max concurrency: invocations running at once | 50 | 1 to 1,000, or a percentage of the targets |
| Max errors: invocations that may fail; once more fail, no more start | 0 | 0 to 1,000, or a percentage of the targets |
| Timeout: how long each invocation may run | 1 hour | 1 second to 48 hours |
| Delivery timeout: how long an offline node has to come online | 1 hour | 30 seconds to 48 hours |
Once more invocations fail than max errors allows, no more start, and those already running finish. A
node that stays offline past the delivery timeout ends undelivered.
Statuses
A run is pending, then running, and ends:
| Status | Meaning |
|---|---|
succeeded |
The failed invocations didn’t exceed max errors |
failed |
More invocations failed than max errors allows |
cancelled |
Someone cancelled the run |
An invocation is pending, dispatched (sent to the node), running, then succeeded (exit
code 0), failed, timed_out, cancelled or undelivered (the node didn’t come online before
the delivery deadline; it has no reason). Others that didn’t succeed have a reason:
| Reason | Meaning |
|---|---|
osUser_not_found |
The account doesn’t exist on the node |
script_hash_mismatch |
The node received a script body other than the version’s, and refused it |
start_failed |
The node couldn’t start the command |
agent_restarted |
The agent restarted while it ran. It isn’t run again |
no_result |
The node never reported a result |
node_offline |
It was running and timed out (timed_out) while its node was offline |
node_deregistered |
The node was deregistered |
project_deleted |
The project was deleted |
Each invocation runs at most once.
Cancel a run
Choose Cancel run on its page, or ezgh fleets runs cancel cmd_…. No more invocations start,
and running ones are stopped (SIGTERM, then SIGKILL after 10 seconds).
Scripts
A script is a saved, versioned body for sh or bash with typed parameters. Each version is
fixed once created: changing a script’s body or parameters adds a version, and a run names the
version it uses (the latest by default).
ezgh fleets scripts create restart-service --file restart.sh --shell bash \
--param service:string:required --param graceful:boolean=true
ezgh fleets scripts new-version restart-service --file restart.sh --param service:string:required-
A body is at most 64 KiB. A script has at most 50 parameters.
-
A parameter has a name, a type (
string,integerorboolean), and optionallyrequired, a default and a pattern its value must match. -
Parameters reach the script as environment variables,
EZGH_PARAM_<NAME>with the name upper-cased, never pasted into the body, so a value can’t inject shell syntax:#!/bin/bash systemctl restart "$EZGH_PARAM_SERVICE"
Running a script needs fleets.scripts.run on the script and fleets.scripts.runOn on each
node’s account, so a policy can allow a saved script as root without allowing any command as
root. See IAM.
A command run, with its command text and parameter values, is kept for 30 days after it ends,
and anyone with fleets.commands.get can read it. Don’t put secrets in commands or parameters:
read them on the machine instead, for example from Secrets.