Apps with volumes deployed to one of our FRA hosts are temporarily unavailable

Incident Report for Fly.io

Resolved

This incident has been resolved.

Posted Aug 26, 2021 - 18:59 UTC

Monitoring

All apps were restored. A redeploy may be required.

Please redeploy if your app was using a custom Dockerfile or a buildpack. If you were using a Docker image from Docker Hub, no need to redeploy.

Posted Aug 26, 2021 - 17:15 UTC

Update

Our registry and remote builds should now be back up. Marked as degraded performance for now as we monitor the situation.

Posted Aug 26, 2021 - 16:38 UTC

Update

We have started recovering apps and volumes.

Posted Aug 26, 2021 - 16:30 UTC

Update

We are continuing to work on a fix for this issue.

Posted Aug 26, 2021 - 15:47 UTC

Identified

By mistake, as part of a clean up attempt, all apps that had volumes on one of our FRA (Frankfurt) hosts were marked as deleted.

This situation is entirely recoverable and should be resolved rapidly.

Do not try recreating them.

Posted Aug 26, 2021 - 15:44 UTC

This incident affected: Platform and Tools (Remote Builds) and Regional Availability (FRA - Frankfurt, Germany).