Remote builder connectivity issues

Incident Report for Fly.io

Resolved

This incident has been resolved.

Posted May 16, 2022 - 17:47 UTC

Monitoring

Connectivity between our API servers and machine mesh has been restored. Remote builders should behave normally now.

Posted May 16, 2022 - 17:00 UTC

Identified

Our API server is experiencing connectivity issues to the NATS mesh we use to send start/stop signals to remote builders. This is preventing some remote machine builders from starting and leaving them in an unexpected state.

Posted May 16, 2022 - 15:35 UTC

Investigating

We are investigating reports of connectivity failures to remote builders. You can workaround this issue with a local Docker daemon + `flyctl deploy --local-only`.

Posted May 16, 2022 - 14:40 UTC

This incident affected: Platform and Tools (Remote Builds).