This incident has been resolved.

We're continuing to monitor after having added more capacity to our DFW and IAD regions.

Deploys or machine starts using existing volumes in these regions may still hit a capacity issue. Users should use fly volume fork --vm-memory <size> to fork the volume to a host with more capacity, then retry the deploy or start command using the new volume.

We have added additional capacity in DFW and IAD regions and are monitoring the impact.

New machine creates and deploys without volumes are seeing improved success rates. Deploys using depot builders in those regions are also improving, with much quicker builder start times.

Deploys or machine starts using existing volumes in these regions may still hit a capacity issue. Users should use fly volume fork --vm-memory <size> to fork the volume to a host with more capacity, then retry the deploy or start command using the new volume.

We've identified some newly created Managed Postgres clusters are failing to come up healthy in these regions.

New machine creates in these regions might fail temporarily, and Depot builders may be unavailable. If you are having issues with Depot builders, consider moving them to a different region, or try --depot=false.