EZGH Kubernetes runs a Kubernetes cluster’s control plane for you. Your nodes are ordinary
instances in your project, launched by node groups. Pods get addresses in your
VPC, LoadBalancer Services and Ingresses become EZGH load balancers,
PersistentVolumeClaims become volumes or Files access points, pods act
as bots with no stored key, and kubectl signs in with EZGH IAM. You install nothing in
the cluster for any of this.
kubectl access
Sign in with EZGH IAM, access entries and the public endpoint.
Node groups
Instance types, sizes, node bots, scaling and draining.
Load balancers
LoadBalancer Services, Ingress and the Gateway API.
Storage
Block Storage volumes and Files volumes for PersistentVolumeClaims.
Pod identity
Pods that call EZGH APIs as a bot, with no stored key.
Secrets
Mount Secrets fields into pods.
Observability
Control-plane logs and metrics, and node and workload telemetry in Obs.
Versions and upgrades
Supported versions, patches, minor upgrades and end of support.
The Kubernetes API is at https://kubernetes-api.ezghcloud.com. In the console, open Kubernetes.
Before you start
A cluster needs:
- A VPC with an attached Network Gateway.
- A private subnet of it whose route table
sends
0.0.0.0/0to that gateway. The cluster’s endpoint takes 3 addresses in this subnet. - A cluster bot: the bot the cluster acts as when it creates load balancers, volumes
and nodes in your project. Attach the managed policy
KubernetesClusterAccessto it. Creating the cluster needsbots.passon that bot.
What the cluster bot does shows in Trails as that bot.
Create a cluster
-
Create the cluster. It takes about 15 minutes, and
--waitwaits until it’sactive:ezgh kubernetes clusters create prod --subnet sbn_2m9x0a7qpe4k --bot bot_8k2l4m6n8p0q --wait -
Add a node group. Its bot needs the managed policy
KubernetesNodeAccess, and you needbots.passon it (Node groups):ezgh kubernetes node-groups create general --cluster kcl_3k9d0a1m2x7q \ --instance-type gp-1.large --subnet sbn_2m9x0a7qpe4k --bot bot_5t7y9u1i3oqa \ --min 2 --max 6 --wait -
Point
kubectlat the cluster, then list its nodes:ezgh kubernetes update-kubeconfig kcl_3k9d0a1m2x7q kubectl get nodes
- Open Kubernetes, then Create cluster. Pick the version, the VPC and subnet, and the cluster
bot. Create a bot makes one with
KubernetesClusterAccessattached. - On the cluster’s Node groups tab, choose Add node group. Create a bot for nodes makes the
node bot with
KubernetesNodeAccessattached and lets the cluster’s bot use it. - Copy the
ezgh kubernetes update-kubeconfigcommand from the cluster’s Overview, run it, then runkubectl get nodes.
In the API, CreateCluster answers 202 with an operation. The
cluster exists from then on, creating, with its ID. Its certificateAuthority is null until it’s
active. Clusters, node groups and add-ons are created, changed and deleted through operations; access
entries and pod identity associations change at once.
Whoever creates the cluster gets cluster admin access through an access
entry, unless they turn that off
(bootstrapCreatorAccess: false, --no-creator-access).
Add-ons
Each cluster runs managed add-ons. EZGH installs, configures and upgrades them, and they upgrade with the cluster unless you pin a version.
| Add-on | What it is | Default |
|---|---|---|
vpc-cni |
Pod networking with VPC addresses, and NetworkPolicy | Installed; can’t be removed |
kube-proxy |
Services | Installed; can’t be removed |
coredns |
Cluster DNS | Installed; can’t be removed |
pod-identity-agent |
Pod identity | Installed; removable |
blockstore-csi |
Block Storage volumes | Installed; removable |
files-csi |
Files volumes | Optional |
secrets-store |
Secrets mounted into pods | Optional |
obs-agent |
Node and workload telemetry | Optional |
metrics-server |
Resource metrics for kubectl top and autoscalers |
Optional |
GET /public/v1/addon-versions?clusterVersion=1.37 lists every add-on’s versions for a cluster
version, which add-ons are installed by default and which can be removed (ezgh kubernetes addon-versions list --cluster-version 1.37).
An add-on’s configuration is a JSON object, checked against the schema of the add-on version it’s
for; a configuration that doesn’t fit is refused. EZGH manages the objects an add-on installs: a field
EZGH sets is overwritten if it’s changed in the cluster.
ezgh kubernetes addons create metrics-server --cluster kcl_3k9d0a1m2x7q --wait
ezgh kubernetes addons update coredns --cluster kcl_3k9d0a1m2x7q --configuration-file coredns.json --wait--version pins an add-on to a version; --follow-cluster unpins it, so it’s upgraded with the cluster.
Quotas and limits
| Quota | Default |
|---|---|
| Clusters per organization | 10 |
| Node groups per cluster | 10 |
| Nodes per cluster (the sum of its node groups’ maximum sizes) | 300 |
These are Service quotas. Each node group’s autoscaling group also counts against Compute’s autoscaling groups quota, and its nodes against Compute’s instance and vCPU quotas.
| Limit | |
|---|---|
| Nodes per node group | 100 |
| Pods per node | 110 |
| PersistentVolumes per node | The instance type’s limits.volumes, less one for the root volume |
Delete a cluster
Delete a cluster’s node groups first: a cluster with node groups can’t be deleted
(409 cluster_has_node_groups). Deleting the cluster then:
- deletes the load balancers it created for Services, Ingresses and Gateways, and its node firewall;
- keeps the volumes it created for PersistentVolumeClaims, tagged with the cluster. Delete the PersistentVolumeClaims first to delete their volumes with them.
ezgh kubernetes node-groups delete kng_7h3n2c5d8wqa --cluster kcl_3k9d0a1m2x7q --yes --wait
ezgh kubernetes clusters delete kcl_3k9d0a1m2x7q --yes --wait