Skip to content

Pull-through cache

Pull Docker Hub images through your project's registry host with your own Docker Hub credentials, and keep a copy in the project.

Updated View as Markdown

A pull-through cache rule maps a prefix on your project’s registry host to Docker Hub. The first pull of <host>/<prefix>/<namespace>/<repository> fetches the image from Docker Hub with the rule’s credentials and stores it in the project. Later pulls are served from that copy.

docker pull <id>.registry.ezghcloud.com/docker-hub/library/nginx:1.27
#                                       └prefix──┘ └repository─┘

Set it up

  1. Create a Docker Hub access token. In Docker Hub, create a personal access token (or an organization access token) with read-only access to public repositories, and to any private repositories you want to pull.

  2. Put it in a secret. In Secrets, create a secret in the project with two fields, username (your Docker Hub username, or the organization’s name for an organization token) and accessToken:

    printf %s "$DOCKERHUB_TOKEN" | ezgh secrets create dockerhub/pull --set username=acme-ci --set accessToken=-
  3. Create the rule. In the console, open Container Registry, then Pull-through cache, and Create rule: a prefix, the upstream (Docker Hub) and the secret. Or:

    ezgh registry cache-rules create --prefix docker-hub --upstream docker-hub --secret dockerhub/pull

    Creating the rule also grants it read access to the secret in Secrets, as you (grants). That needs secrets.grants.create and secrets.versions.access on the secret. If the grant fails, the rule is still created: retry it from the console, or with ezgh registry cache-rules update docker-hub --secret dockerhub/pull.

  • The prefix is one lowercase name part (letters, digits, ., _, __ or -), at most 30 characters. It can’t change, and it must not be the scope of an existing repository. While the rule exists, nobody can create or push a repository under it.
  • Credentials are required. Anonymous pulls from Docker Hub aren’t offered.
  • A project can have up to 25 rules.

Pull through it

Docker Hub repository names have two parts, so cache repositories have three: <prefix>/<namespace>/<repository>. Official images are under library/:

Docker Hub Through the rule
nginx:1.27 <host>/docker-hub/library/nginx:1.27
bitnami/redis:7.4 <host>/docker-hub/bitnami/redis:7.4
node@sha256:… <host>/docker-hub/library/node@sha256:…

Sign in to your registry host as for any pull. The first pull of a repository creates it in the project, as a private repository with a Cache label. Pull it like any other repository. You can’t push to it, or make it public.

Kubernetes

Give the cluster an API key for a bot that can pull (see IAM) in an image pull secret:

kubectl create secret docker-registry ezgh-registry \
  --docker-server=<id>.registry.ezghcloud.com \
  --docker-username=ezgh \
  --docker-password="$EZGH_API_KEY"
spec:
  imagePullSecrets:
    - name: ezgh-registry
  containers:
    - name: web
      image: <id>.registry.ezghcloud.com/docker-hub/library/nginx:1.27

CI

Sign in to the registry host, then use cache paths in place of Docker Hub’s, including in FROM lines:

# GitHub Actions
- run: echo "${{ secrets.EZGH_API_KEY }}" | docker login <id>.registry.ezghcloud.com -u ezgh --password-stdin
- run: docker build -t <id>.registry.ezghcloud.com/team-a/api:${{ github.sha }} .
FROM <id>.registry.ezghcloud.com/docker-hub/library/node:22-alpine

When Docker Hub is asked

  • Pulls by digest are served from the stored copy once it’s there. Docker Hub isn’t asked again.
  • Pulls by tag are served from the stored copy while it’s less than 24 hours since the tag was last checked. After that, the next pull checks the tag with Docker Hub (a HEAD request, which Docker Hub doesn’t count against pull limits). If it moved, the new image is fetched and stored, and the old one stays pullable by digest.
  • An image’s layers are fetched the first time someone pulls them, and stored before they’re served. For a large layer, the first pull waits for the whole download (several minutes for a multi-GB layer). Kubernetes may give up and retry the pull meanwhile; the download carries on, and a later retry gets the stored layer.
  • A multi-platform image stores the platforms that are actually pulled.
  • Layers already in the project (pushed, or cached through another rule) aren’t fetched again.

When Docker Hub is down

Images already stored keep pulling. A tag that’s due for its check is served from the stored copy when Docker Hub is unreachable, rate-limits the rule, refuses its credentials, or no longer has the tag; the next pull checks again. An image that was never pulled can’t be pulled until Docker Hub answers: the client gets 503 (429 when Docker Hub rate-limits the rule, with its Retry-After), or 404 when Docker Hub has no such repository or tag.

Read failures

Every failed fetch or tag check is recorded on the rule, without Docker Hub’s responses or any credentials. In the console, the rule’s Recent failures tab lists them, and the rules list shows Failing when one happened in the last 24 hours. Or:

ezgh registry cache-rules failures docker-hub
ezgh registry cache-rules failures docker-hub --upstream-repository library/nginx

Each row is an upstream repository, a reference (tag or digest) and a reason, with how many times it happened, when it last did, and whether the stored copy was served anyway. A row goes away once a pull of that reference succeeds, and after 30 days.

Reason Meaning
upstream_not_found Docker Hub has no such repository, tag or digest.
upstream_auth_refused Docker Hub refused the rule’s credentials.
credential_unavailable The secret, its grant to the rule, or its username and accessToken fields are missing.
upstream_rate_limited Docker Hub rate-limited the rule’s credentials.
upstream_unavailable Docker Hub was unreachable or failed.
secrets_unavailable Secrets was unavailable.
digest_mismatch Docker Hub’s content didn’t match its digest or size. Nothing was stored.
unsupported_manifest The image’s manifest type isn’t supported (Docker schema 1, or not an image).
too_large The manifest is over 4 MiB, or a layer over 10 GiB.
storage_limit The organization’s storage limit is reached.
repository_limit The project’s repository limit is reached.

Change or delete a rule

  • Change credentials in the rule’s Settings, or with ezgh registry cache-rules update <rule> --secret <secret>. The new secret is granted to the rule first. It’s used from the next fetch or tag check. Revoking the grant, or deleting the secret, stops new fetches and tag checks (credential_unavailable); what’s already stored keeps pulling.
  • Delete a cached image or repository as any other. A deleted image is fetched again on its next pull.
  • Deleting a rule deletes every repository it cached. Pulls under its prefix stop at once, and the repositories are deleted in the background, including ones the person deleting the rule couldn’t delete themselves. Then the prefix is free for a new rule. The console asks for the prefix to be typed and shows how many repositories go; the CLI asks unless --yes.

IAM

Action Allows Checked against
registry.cacheRules.list List a project’s rules The project
registry.cacheRules.get See a rule and its recent failures The rule
registry.cacheRules.create Create rules The new rule
registry.cacheRules.update Change a rule’s credentials and description The rule
registry.cacheRules.delete Delete rules, and every repository they cached The rule

RegistryFullAccess has all of them. RegistryReadOnlyAccess, RegistryPullAccess, RegistryPushAccess and ReadOnlyAccess have list and get.

The first pull of a Docker Hub repository creates it, so it needs registry.images.pull and registry.repositories.create on the new repository. Later pulls of that repository need only registry.images.pull. Cache repositories’ resource names end in their three-part name, ezgh:<region>:<org>:<project>:rep_<id>/docker-hub/library/nginx, so:

  • rep_*/docker-hub/* matches the whole cache;
  • rep_*/docker-hub/library/* matches official images only;
  • rep_*/docker-hub/bitnami/redis matches one image.

To let a cluster’s bot fill the cache without creating other repositories, give it RegistryPullAccess plus:

{
  "statements": [
    {
      "effect": "allow",
      "actions": ["registry.repositories.create"],
      "resources": ["ezgh:*:org_k3f9a0x2m7qp:prj_g7h8i9j0k1l2:rep_*/docker-hub/*"]
    }
  ]
}
Navigation

Type to search…

↑↓ navigate↵ selectEsc close