High error rates across multiple regions
Incident Report for Fly.io
Resolved
This incident has been resolved.
Posted May 27, 2020 - 15:19 CDT
Update
All services restored. We are monitoring to make sure customer apps recovered gracefully.
Posted May 27, 2020 - 15:18 CDT
Monitoring
We've restored backhaul between all regions, error rates are decreasing. We are restarting numerous services to flush out old backhaul configurations.
Posted May 27, 2020 - 15:16 CDT
Update
We are continuing to investigate this issue.
Posted May 27, 2020 - 15:13 CDT
Update
Backhaul between regions is unstable and causing multiple connection failures.
Posted May 27, 2020 - 15:11 CDT
Investigating
Our monitoring is reporting errors across multiple Fly regions, we're investigating.
Posted May 27, 2020 - 15:05 CDT
This incident affected: Application Hosting (EWR1 Application Hosting (New Jersey), AMS1 Application Hosting (Amsterdam), YYZ1 Application Hosting (Toronto), SJC1 Application Hosting (San Jose, CA), NRT1 Application Hosting (Tokyo, Japan), SEA1 Application Hosting (Seattle, Washington), LAX1 Application Hosting (Los Angeles, CA), IAD1 Application Hosting (Ashburn, Virginia), ORD1 Application Hosting (Chicago, Illinois), ATL1 Application Hosting (Atlanta, Georgia), DFW2 Application Hosting (Dallas, Texas), FRA1 Application Hosting (Frankfurt, Germany), HKG1 Application Hosting (Hong Kong), SYD1 Application Hosting (Sydney, Australia), MRS1 Application Hosting (Marseille, France), SIN1 Application Hosting (Singapore)).