Host Instability In Chicago
Incident Report for Fly.io
Resolved
This incident has been resolved.

Several Chicago hosts experienced simultaneous disk failures causing significantly degraded states impacting application performance. These failures additionally prevented some existing apps on the affected hosts from stopping properly, causing deployments for these apps to get stuck in a 'pending' state while their hosts were unresponsive.

The issue was eventually resolved by identifying and repairing the bad disks on all affected hosts, and we will be making improvements to our deployment system to prevent similar stuck-deployment issues from occurring in the future.
Posted May 09, 2022 - 13:05 CDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted May 09, 2022 - 03:00 CDT
Identified
It definitely looks like a hardware issue! We're pulling the affected host out of the rotation and are continuing to monitor.
Posted May 08, 2022 - 22:10 CDT
Investigating
This is an issue with a specific host; it may be hardware-related. We're looking in to it.
Posted May 08, 2022 - 21:54 CDT
This incident affected: Application Hosting (ORD - Chicago, Illinois (US)).