ALL NOTES

build infrastructure · SEV-3

builds start failing with 429 Too Many Requests from a public package registry, with no change to the code

scroll to render

How to confirm it

  • Rule out port exhaustion before blaming the network

    gcloud logging read 'jsonPayload.allocation_status="DROPPED"' --limit=10

    Zero NAT drops means this is an application-layer 429, not a connection-tracking limit. Two very different fixes.

  • Count your egress addresses

    gcloud compute routers nats describe NAT --router ROUTER --region REGION --format='value(natIps)'

    One manually assigned address for an entire CI fleet is the finding. Compare against the workload gateway, which usually has several.

  • Prove the cache never persists

    kubectl -n NS get scaledjob NAME -o jsonpath='{.spec.jobTargetRef.template.spec.volumes}'

    An ephemeral volume for the workspace plus an unset cache-home variable means the cache dies with the pod. The log tell is a download of the full dependency set on every single run.

  • Check whether a proxy ever existed

    gcloud artifacts repositories list --format='table(name,format,mode)'

    Remote-mode repositories are the mirrors. If the only entries are standard mode, builds have been resolving public dependencies directly all along — the regression is volume, not a deleted proxy.

Read the source

Cause

Every CI agent egressed through a single NAT address, and no dependency cache survived a job — the workspace was ephemeral and the tool's cache directory was never pointed anywhere persistent. So every build cold-downloaded the full dependency graph from one IP. A single address plus a cold cache is a permanent ceiling; a busier day simply crossed it.

Fix

Put a pull-through mirror in the private registry so agents only ever talk to it, and let it fetch upstream server-side — then per-IP throttling is structurally impossible. Add addresses to the NAT gateway for immediate relief and persist the dependency cache so builds stop being cold. Retrying inside the window just re-throttles the same address.