Setting secrets and creating apps is degraded

Incident Report for Fly.io

Resolved

This incident has been resolved.
Posted Mar 11, 2026 - 11:37 UTC

Update

While the secret storage service was in a read-only state, app creation requests queued up, due to the retry logic and insufficient request concurrency limits in our GraphQL API. This prevented our GraphQL API from serving any other requests. We have scaled up the GraphQL API and are continuing to monitor the situation.
Posted Mar 11, 2026 - 11:03 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Mar 11, 2026 - 10:14 UTC

Identified

An ongoing data migration in our secret storage service is causing degraded Machines API functionality.
Posted Mar 11, 2026 - 09:19 UTC
This incident affected: Machines API and Deployments.