Deployments fail to load metadata from docker.io

Incident Report for Fly.io

Resolved

This incident has been resolved.

Posted Sep 20, 2023 - 08:48 UTC

Monitoring

We restarted our registry mirror that proxies requests to docker.io, and that cleared up the issue. Our metrics show deployments are working again. We'll continue to monitor for a bit to make sure this doesn't recur.

This issue impacted deployments that built images and required one or more images from docker.io. This impacted deployment between 2023-09-20 4:05 UTC and 7:40 UTC.

Posted Sep 20, 2023 - 07:45 UTC

Investigating

Deployments that build new images are failing with errors like:

ERROR [internal] load metadata for docker.io...

We're investigating the issue now.

The failed deployments do not impact existing machines and nomad allocs. They only prevent new changes from getting deployed.

Posted Sep 20, 2023 - 04:05 UTC

This incident affected: Deployments.