Skip to content

Node groups

Run a cluster's nodes as instances in your project, scaled by Cluster Autoscaler and upgraded with drains.

Updated View as Markdown

A node group is a set of nodes of one instance type in one private subnet. Its nodes are ordinary instances in your project, launched by an autoscaling group the node group owns, and billed as instances.

Create a node group

ezgh kubernetes node-groups create general --cluster kcl_3k9d0a1m2x7q \
  --instance-type gp-1.large --subnet sbn_2m9x0a7qpe4k --bot bot_5t7y9u1i3oqa \
  --min 2 --max 6 --label team=shop --taint dedicated=shop:NoSchedule --wait
Field
instanceType The nodes’ instance type.
subnet A private subnet of the cluster’s VPC. Pods take their addresses from it too.
minSize, maxSize 0 to 100 nodes. maxSize is at least 1.
desiredSize How many nodes to start with, from minSize to maxSize. After that, Cluster Autoscaler sets it.
version The nodes’ Kubernetes version: the cluster’s by default, never newer.
rootVolumeSizeGiB Each node’s root volume, 8 to 16,384 GiB. The default is 20.
labels Kubernetes labels on each node, at most 50. Keys under kubernetes.io, k8s.io and kubernetes.ezghcloud.com are reserved.
taints Taints on each node, at most 50. The effect is NoSchedule, PreferNoSchedule or NoExecute.
sshKeyIds SSH keys on each node. Optional.
bot The node bot. See below.
updateConfig.maxUnavailablePercent The share of nodes replaced at once in an upgrade, 1 to 100. The default is 25.

A node holds up to 110 pods, and as many PersistentVolumes as its instance type’s limits.volumes less one for the root volume.

The node bot

Each node acts as the node group’s bot, its Instance Bot, to take pod addresses, pull images from Container Registry and send telemetry to Obs. Attach the managed policy KubernetesNodeAccess to it. With it, pods run images from your organization’s private repositories with no image pull secret: use the image’s full reference, such as <id>.registry.ezghcloud.com/team-a/api:v1 (ezgh registry get shows the host).

Creating a node group needs bots.pass on the node bot, and so does the cluster’s bot, which launches the nodes: give the cluster bot a policy allowing bots.pass on the node bot. In the console, Create a bot for nodes does both.

Pods never act as the node bot: they can’t reach the instance metadata service. A pod that calls EZGH APIs uses pod identity.

The autoscaling group

The node group’s autoscaling group is in your project, named after the cluster and the node group and ending in the node group’s ID (prod-general-kng_7h3n2c5d8wqa). Its nodes are tagged ezgh:kubernetes-cluster and ezgh:kubernetes-node-group. The node group manages it: a change made to the autoscaling group directly is replaced by the node group’s settings. Change the node group instead.

Node groups can’t change their instance type, subnet, root volume size, bot or SSH keys. Create a new node group, then delete the old one.

Scaling

Cluster Autoscaler runs with the cluster’s control plane. It adds nodes when pods can’t be scheduled, and removes nodes it no longer needs, draining each first. It stays between the node group’s minSize and maxSize:

ezgh kubernetes node-groups update kng_7h3n2c5d8wqa --cluster kcl_3k9d0a1m2x7q --min 3 --max 12 --wait

Upgrades and draining

Upgrading a node group, or changing its labels or taints, replaces its nodes a few at a time (maxUnavailablePercent of them). Each node is cordoned and drained before it’s terminated, honouring PodDisruptionBudgets, and a new node counts only once it’s Ready. A drain that hasn’t finished after 15 minutes ends, and the node is terminated.

ezgh kubernetes node-groups update kng_7h3n2c5d8wqa --cluster kcl_3k9d0a1m2x7q --version 1.37 --wait

Upgrade the cluster first: a node group can’t be newer than its cluster. See Versions and upgrades.

Deleting a node group drains and terminates its nodes, then deletes its autoscaling group.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close