Skip to content

Concurrency and scaling

How many requests an instance serves at once, how instances start and stop, and the limits on a request.

Updated View as Markdown

Concurrency

An instance serves up to maxConcurrency requests at once (1 to 1000, default 80), like an ordinary web server. Run sends requests to instances with room, and starts more as the requests in flight grow. It never sends an instance more than maxConcurrency at once. Set it to what your server handles well: 1 for a server that can take one request at a time.

Scaling

  • Run keeps the number of instances near the requests in flight divided by 70% of maxConcurrency, between minInstances (0 to 100, default 0) and maxInstances (1 to 100, default 10). It follows the average over the last 60 seconds, and grows faster when requests double within seconds.
  • With minInstances 0, a service with no requests has no running instance. The next request waits while an instance starts or resumes; it isn’t refused.
  • When fewer instances are needed, Run stops sending one requests, lets its requests finish, and suspends it. A revision keeps up to 2 suspended instances, which resume faster than a new one starts. One unused for 24 hours is deleted.
  • Your instances count against Sandboxes’ quotas. When a quota, or anything else, stops Run from starting an instance, the revision shows why in its scalingError, and Run tries again.

Starting an instance

An instance runs the image’s entrypoint and command. It’s ready once something accepts a TCP connection on 127.0.0.1 at the revision’s port (default 8080; PORT holds it). It has startupTimeoutSeconds (1 to 3600, default 240) to get there. A revision whose first instance doesn’t is failed.

Listen on 0.0.0.0 or 127.0.0.1. What your container writes to stdout and stderr goes to the project’s logs in Observability.

Requests

Limit
Request timeout requestTimeoutSeconds, 1 to 3600, default 300: then 504
Request body 32 MiB: more is 413
A request body your server doesn’t read Ended after 30 seconds
Waiting for an instance with room Up to 30 seconds; past that, or with too many requests waiting, 429 with Retry-After
Protocol to your container HTTP/1.1. WebSockets, HTTP/2 and gRPC to the container aren’t supported

Other answers that come from Run rather than your container:

Status When
404 No service has the URL’s name, including the former URL of a service set to load balancers only (Serve a Run service through a load balancer)
503 The service has no ready revision yet, or is briefly busy; with Retry-After
502 Your container’s answer failed after it started (a closed connection)

A request that reached your container is never sent again. Requests are only retried on another instance when nothing read them.

Headers

Your container gets the client’s request headers, except hop-by-hop headers and headers starting with Ezgh-, X-Ezgh- or X-Envoy-, and these:

Header Value
X-Forwarded-For The client’s address, and any proxies before it
X-Forwarded-Proto https
X-Request-Id The request’s ID, also on the response
X-Ezgh-Principal With authenticated access, the ID of the user or bot that called
Navigation

Type to search…

↑↓ navigate↵ selectEsc close