Skip to content

Run commands and scripts

Run a command or a saved script on nodes by name or by tag, with concurrency, error and time limits, and save versioned scripts with typed parameters.

Updated View as Markdown

A command run sends one command, or one version of a saved script, to a set of nodes. Each node’s part is an invocation, which reports a status and an exit code. Output isn’t kept: the agent reads and discards what the command prints.

Run a command

  1. Open Fleets > Command runs and choose Run command, or choose Run command on a node’s page.
  2. Choose the targets (nodes, or tags), the account, and a command or a script with its parameters. Set the limits if the defaults don’t suit.
  3. The run’s page shows each node’s invocation as it runs.
ezgh fleets run --node web-1 --node web-2 --as root --command 'systemctl restart nginx' --wait
ezgh fleets run --tag env=prod --tag role=web --as root --script restart-service \
  --param service=nginx --max-concurrency 25% --max-errors 1
ezgh fleets runs get cmd_7k2m9x0a4qpe

With --wait, ezgh waits for the run to end, prints each node’s result and exits 1 unless every invocation succeeded.

SendCommand with targets, osUser and exactly one of command or script. ListCommandInvocations lists each node’s result.

  • Targets are nodes by ID, slug or name, or every node in the project with all the given tags. Tags are matched once, when the run is created. A run has 1 to 1,000 nodes.
  • The account is required, and defaults to ezgh-user in the console and ezgh.
  • A command is run with /bin/sh -c, at most 64 KiB.
  • Access is checked once, for every target, when the run is created. If you may not run it on any one target, the whole run is refused (403, naming the node), so a run never partly starts.

Limits

Setting Default Range
Max concurrency: invocations running at once 50 1 to 1,000, or a percentage of the targets
Max errors: invocations that may fail; once more fail, no more start 0 0 to 1,000, or a percentage of the targets
Timeout: how long each invocation may run 1 hour 1 second to 48 hours
Delivery timeout: how long an offline node has to come online 1 hour 30 seconds to 48 hours

Once more invocations fail than max errors allows, no more start, and those already running finish. A node that stays offline past the delivery timeout ends undelivered.

Statuses

A run is pending, then running, and ends:

Status Meaning
succeeded The failed invocations didn’t exceed max errors
failed More invocations failed than max errors allows
cancelled Someone cancelled the run

An invocation is pending, dispatched (sent to the node), running, then succeeded (exit code 0), failed, timed_out, cancelled or undelivered (the node didn’t come online before the delivery deadline; it has no reason). Others that didn’t succeed have a reason:

Reason Meaning
osUser_not_found The account doesn’t exist on the node
script_hash_mismatch The node received a script body other than the version’s, and refused it
start_failed The node couldn’t start the command
agent_restarted The agent restarted while it ran. It isn’t run again
no_result The node never reported a result
node_offline It was running and timed out (timed_out) while its node was offline
node_deregistered The node was deregistered
project_deleted The project was deleted

Each invocation runs at most once.

Cancel a run

Choose Cancel run on its page, or ezgh fleets runs cancel cmd_…. No more invocations start, and running ones are stopped (SIGTERM, then SIGKILL after 10 seconds).

Scripts

A script is a saved, versioned body for sh or bash with typed parameters. Each version is fixed once created: changing a script’s body or parameters adds a version, and a run names the version it uses (the latest by default).

ezgh fleets scripts create restart-service --file restart.sh --shell bash \
  --param service:string:required --param graceful:boolean=true
ezgh fleets scripts new-version restart-service --file restart.sh --param service:string:required
  • A body is at most 64 KiB. A script has at most 50 parameters.

  • A parameter has a name, a type (string, integer or boolean), and optionally required, a default and a pattern its value must match.

  • Parameters reach the script as environment variables, EZGH_PARAM_<NAME> with the name upper-cased, never pasted into the body, so a value can’t inject shell syntax:

    #!/bin/bash
    systemctl restart "$EZGH_PARAM_SERVICE"

Running a script needs fleets.scripts.run on the script and fleets.scripts.runOn on each node’s account, so a policy can allow a saved script as root without allowing any command as root. See IAM.

A command run, with its command text and parameter values, is kept for 30 days after it ends, and anyone with fleets.commands.get can read it. Don’t put secrets in commands or parameters: read them on the machine instead, for example from Secrets.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close