A node group is a set of nodes of one instance type in one private subnet. Its nodes are ordinary instances in your project, launched by an autoscaling group the node group owns, and billed as instances.
Create a node group
ezgh kubernetes node-groups create general --cluster kcl_3k9d0a1m2x7q \
--instance-type gp-1.large --subnet sbn_2m9x0a7qpe4k --bot bot_5t7y9u1i3oqa \
--min 2 --max 6 --label team=shop --taint dedicated=shop:NoSchedule --wait| Field | |
|---|---|
instanceType |
The nodes’ instance type. |
subnet |
A private subnet of the cluster’s VPC. Pods take their addresses from it too. |
minSize, maxSize |
0 to 100 nodes. maxSize is at least 1. |
desiredSize |
How many nodes to start with, from minSize to maxSize. After that, Cluster Autoscaler sets it. |
version |
The nodes’ Kubernetes version: the cluster’s by default, never newer. |
rootVolumeSizeGiB |
Each node’s root volume, 8 to 16,384 GiB. The default is 20. |
labels |
Kubernetes labels on each node, at most 50. Keys under kubernetes.io, k8s.io and kubernetes.ezghcloud.com are reserved. |
taints |
Taints on each node, at most 50. The effect is NoSchedule, PreferNoSchedule or NoExecute. |
sshKeyIds |
SSH keys on each node. Optional. |
bot |
The node bot. See below. |
updateConfig.maxUnavailablePercent |
The share of nodes replaced at once in an upgrade, 1 to 100. The default is 25. |
A node holds up to 110 pods, and as many PersistentVolumes as its instance type’s
limits.volumes less one for the root volume.
The node bot
Each node acts as the node group’s bot, its Instance Bot, to take pod
addresses, pull images from Container Registry and send telemetry to Obs. Attach the managed
policy KubernetesNodeAccess to it. With it, pods run images from your organization’s private
repositories with no image pull secret: use the image’s full reference, such as
<id>.registry.ezghcloud.com/team-a/api:v1 (ezgh registry get shows the host).
Creating a node group needs bots.pass on the node bot, and so does the cluster’s bot, which launches
the nodes: give the cluster bot a policy allowing bots.pass on the node bot. In the console, Create a
bot for nodes does both.
Pods never act as the node bot: they can’t reach the instance metadata service. A pod that calls EZGH APIs uses pod identity.
The autoscaling group
The node group’s autoscaling group is in your project, named after the cluster and the node group and
ending in the node group’s ID (prod-general-kng_7h3n2c5d8wqa). Its nodes are tagged
ezgh:kubernetes-cluster and ezgh:kubernetes-node-group. The node group manages it: a change made to the autoscaling group directly
is replaced by the node group’s settings. Change the node group instead.
Node groups can’t change their instance type, subnet, root volume size, bot or SSH keys. Create a new node group, then delete the old one.
Scaling
Cluster Autoscaler runs with the cluster’s control plane. It adds nodes when pods can’t be scheduled,
and removes nodes it no longer needs, draining each first. It stays between the node group’s
minSize and maxSize:
ezgh kubernetes node-groups update kng_7h3n2c5d8wqa --cluster kcl_3k9d0a1m2x7q --min 3 --max 12 --waitUpgrades and draining
Upgrading a node group, or changing its labels or taints, replaces its nodes a few at a time
(maxUnavailablePercent of them). Each node is cordoned and drained before it’s terminated, honouring
PodDisruptionBudgets, and a new node counts only once it’s Ready. A drain that hasn’t finished after
15 minutes ends, and the node is terminated.
ezgh kubernetes node-groups update kng_7h3n2c5d8wqa --cluster kcl_3k9d0a1m2x7q --version 1.37 --waitUpgrade the cluster first: a node group can’t be newer than its cluster. See Versions and upgrades.
Deleting a node group drains and terminates its nodes, then deletes its autoscaling group.