Skip to content

Kubernetes

Managed Kubernetes clusters in your VPC, with node groups of your own instances and every EZGH integration built in.

Updated View as Markdown

EZGH Kubernetes runs a Kubernetes cluster’s control plane for you. Your nodes are ordinary instances in your project, launched by node groups. Pods get addresses in your VPC, LoadBalancer Services and Ingresses become EZGH load balancers, PersistentVolumeClaims become volumes or Files access points, pods act as bots with no stored key, and kubectl signs in with EZGH IAM. You install nothing in the cluster for any of this.

The Kubernetes API is at https://kubernetes-api.ezghcloud.com. In the console, open Kubernetes.

Before you start

A cluster needs:

  • A VPC with an attached Network Gateway.
  • A private subnet of it whose route table sends 0.0.0.0/0 to that gateway. The cluster’s endpoint takes 3 addresses in this subnet.
  • A cluster bot: the bot the cluster acts as when it creates load balancers, volumes and nodes in your project. Attach the managed policy KubernetesClusterAccess to it. Creating the cluster needs bots.pass on that bot.

What the cluster bot does shows in Trails as that bot.

Create a cluster

  1. Create the cluster. It takes about 15 minutes, and --wait waits until it’s active:

    ezgh kubernetes clusters create prod --subnet sbn_2m9x0a7qpe4k --bot bot_8k2l4m6n8p0q --wait
  2. Add a node group. Its bot needs the managed policy KubernetesNodeAccess, and you need bots.pass on it (Node groups):

    ezgh kubernetes node-groups create general --cluster kcl_3k9d0a1m2x7q \
      --instance-type gp-1.large --subnet sbn_2m9x0a7qpe4k --bot bot_5t7y9u1i3oqa \
      --min 2 --max 6 --wait
  3. Point kubectl at the cluster, then list its nodes:

    ezgh kubernetes update-kubeconfig kcl_3k9d0a1m2x7q
    kubectl get nodes
  1. Open Kubernetes, then Create cluster. Pick the version, the VPC and subnet, and the cluster bot. Create a bot makes one with KubernetesClusterAccess attached.
  2. On the cluster’s Node groups tab, choose Add node group. Create a bot for nodes makes the node bot with KubernetesNodeAccess attached and lets the cluster’s bot use it.
  3. Copy the ezgh kubernetes update-kubeconfig command from the cluster’s Overview, run it, then run kubectl get nodes.

In the API, CreateCluster answers 202 with an operation. The cluster exists from then on, creating, with its ID. Its certificateAuthority is null until it’s active. Clusters, node groups and add-ons are created, changed and deleted through operations; access entries and pod identity associations change at once.

Whoever creates the cluster gets cluster admin access through an access entry, unless they turn that off (bootstrapCreatorAccess: false, --no-creator-access).

Add-ons

Each cluster runs managed add-ons. EZGH installs, configures and upgrades them, and they upgrade with the cluster unless you pin a version.

Add-on What it is Default
vpc-cni Pod networking with VPC addresses, and NetworkPolicy Installed; can’t be removed
kube-proxy Services Installed; can’t be removed
coredns Cluster DNS Installed; can’t be removed
pod-identity-agent Pod identity Installed; removable
blockstore-csi Block Storage volumes Installed; removable
files-csi Files volumes Optional
secrets-store Secrets mounted into pods Optional
obs-agent Node and workload telemetry Optional
metrics-server Resource metrics for kubectl top and autoscalers Optional

GET /public/v1/addon-versions?clusterVersion=1.37 lists every add-on’s versions for a cluster version, which add-ons are installed by default and which can be removed (ezgh kubernetes addon-versions list --cluster-version 1.37).

An add-on’s configuration is a JSON object, checked against the schema of the add-on version it’s for; a configuration that doesn’t fit is refused. EZGH manages the objects an add-on installs: a field EZGH sets is overwritten if it’s changed in the cluster.

ezgh kubernetes addons create metrics-server --cluster kcl_3k9d0a1m2x7q --wait
ezgh kubernetes addons update coredns --cluster kcl_3k9d0a1m2x7q --configuration-file coredns.json --wait

--version pins an add-on to a version; --follow-cluster unpins it, so it’s upgraded with the cluster.

Quotas and limits

Quota Default
Clusters per organization 10
Node groups per cluster 10
Nodes per cluster (the sum of its node groups’ maximum sizes) 300

These are Service quotas. Each node group’s autoscaling group also counts against Compute’s autoscaling groups quota, and its nodes against Compute’s instance and vCPU quotas.

Limit
Nodes per node group 100
Pods per node 110
PersistentVolumes per node The instance type’s limits.volumes, less one for the root volume

Delete a cluster

Delete a cluster’s node groups first: a cluster with node groups can’t be deleted (409 cluster_has_node_groups). Deleting the cluster then:

  • deletes the load balancers it created for Services, Ingresses and Gateways, and its node firewall;
  • keeps the volumes it created for PersistentVolumeClaims, tagged with the cluster. Delete the PersistentVolumeClaims first to delete their volumes with them.
ezgh kubernetes node-groups delete kng_7h3n2c5d8wqa --cluster kcl_3k9d0a1m2x7q --yes --wait
ezgh kubernetes clusters delete kcl_3k9d0a1m2x7q --yes --wait
Navigation

Type to search…

↑↓ navigate↵ selectEsc close