Update - We are continuing to investigate, we have not found the root cause yet.
Aug 10, 18:31 CDT
Investigating - We are investigating an issue causing return packets from UDP services over anycast IPs to be dropped.
Aug 10, 16:30 CDT
Monitoring - Rollback has restored access to DNS services.
Aug 10, 16:15 CDT
Identified - We are rolling back a networking change that might have interfered with DNS packets destined for external addresses.
Aug 10, 16:06 CDT
Update - We are continuing to investigate this issue.
Aug 10, 15:44 CDT
Update - We are investigating broken DNS resolution, as well as failures in UDP Anycast services on port 53. Applications may not currently be able to resolve public DNS records or accept UDP traffic on Anycast port 53.
Aug 10, 15:44 CDT
Investigating - We are investigating reports of an increase in DNS resolution errors for apps running on Fly.io.
Aug 10, 15:32 CDT
Application Hosting Operational
90 days ago
99.99 % uptime
Today
AMS - Amsterdam, Netherlands Operational
90 days ago
99.99 % uptime
Today
CDG - Paris, France Operational
90 days ago
100.0 % uptime
Today
DFW - Dallas, Texas (US) Operational
90 days ago
100.0 % uptime
Today
EWR - Secaucus, New Jersey (US) Operational
90 days ago
100.0 % uptime
Today
FRA - Frankfurt, Germany Operational
90 days ago
99.98 % uptime
Today
GRU - São Paulo, Brazil Operational
90 days ago
100.0 % uptime
Today
HKG - Hong Kong Operational
90 days ago
100.0 % uptime
Today
IAD - Ashburn, Virginia (US) Operational
90 days ago
100.0 % uptime
Today
LAX - Los Angeles, California (US) Operational
90 days ago
100.0 % uptime
Today
LHR - London, United Kingdom Operational
90 days ago
99.99 % uptime
Today
MAA - Chennai (Madras), India Operational
90 days ago
100.0 % uptime
Today
MAD - Madrid, Spain Operational
90 days ago
100.0 % uptime
Today
MIA - Miami, Florida (US) Operational
90 days ago
100.0 % uptime
Today
NRT - Tokyo, Japan Operational
90 days ago
100.0 % uptime
Today
ORD - Chicago, Illinois (US) Operational
90 days ago
100.0 % uptime
Today
SCL - Santiago, Chile Operational
90 days ago
100.0 % uptime
Today
SEA - Seattle, Washington (US) Operational
90 days ago
100.0 % uptime
Today
SIN - Singapore Operational
90 days ago
100.0 % uptime
Today
SJC - Sunnyvale, California (US) Operational
90 days ago
100.0 % uptime
Today
SYD - Sydney, Australia Operational
90 days ago
100.0 % uptime
Today
YYZ - Toronto, Canada Operational
90 days ago
100.0 % uptime
Today
ATL - Atlanta, Georgia (US) Operational
90 days ago
100.0 % uptime
Today
MRS - Marseille, France Operational
90 days ago
100.0 % uptime
Today
Persistent Storage (Volumes) ? Operational
90 days ago
100.0 % uptime
Today
Private DNS ? Operational
90 days ago
99.98 % uptime
Today
UDP Anycast ? Operational
90 days ago
100.0 % uptime
Today
Edge Proxy Partial Outage
90 days ago
99.99 % uptime
Today
EWR1 Edge (New Jersey) Operational
90 days ago
100.0 % uptime
Today
Routing Partial Outage
90 days ago
99.98 % uptime
Today
Platform and Tools Operational
90 days ago
99.87 % uptime
Today
API Operational
90 days ago
99.73 % uptime
Today
Deployments Operational
90 days ago
99.67 % uptime
Today
Dashboard Operational
90 days ago
99.92 % uptime
Today
Remote Builds Operational
90 days ago
99.93 % uptime
Today
Logs Operational
90 days ago
99.55 % uptime
Today
SSL/TLS Certificate Provisioning Operational
90 days ago
99.96 % uptime
Today
PostgreSQL Clusters Operational
90 days ago
100.0 % uptime
Today
Metrics ? Operational
90 days ago
99.96 % uptime
Today
GitHub Actions Integration Operational
90 days ago
100.0 % uptime
Today
Docker Registry Operational
90 days ago
99.99 % uptime
Today
JavaScript Edge Apps Operational
90 days ago
100.0 % uptime
Today
JavaScript Edge Hosting Operational
90 days ago
100.0 % uptime
Today
Stripe API Operational
DNSimple Name Servers Operational
AWS route53 Operational
AWS s3-sa-east-1 Operational
AWS s3-us-west-1 Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Registry API
Fetching
Past Incidents
Aug 10, 2022

Unresolved incidents: Anycast UDP outage, Increased DNS resolution errors.

Aug 9, 2022

No incidents reported.

Aug 8, 2022
Resolved - This issue has been resolved.
Aug 8, 10:22 CDT
Investigating - We're seeing elevated error rates connecting to the Fly API and dashboard. We're investigating.
Aug 8, 09:29 CDT
Aug 7, 2022

No incidents reported.

Aug 6, 2022

No incidents reported.

Aug 5, 2022

No incidents reported.

Aug 4, 2022

No incidents reported.

Aug 3, 2022

No incidents reported.

Aug 2, 2022

No incidents reported.

Aug 1, 2022

No incidents reported.

Jul 31, 2022

No incidents reported.

Jul 30, 2022

No incidents reported.

Jul 29, 2022

No incidents reported.

Jul 28, 2022
Resolved - At approximately 2:30UTC, we added newly acquired IP blocks to our anycast infrastructure. The new blocks surpassed a BGP announcement limit in place with an upstream provider. This triggered withdrawal of routes for some existing IP blocks.

We restored the missing routes at 2:35UTC. The effect on end users varied because BGP changes take time to propagate. In the worst cases, end users weren't able to connect to application IPs for 5 minutes.

Jul 28, 09:30 CDT
Jul 27, 2022
Resolved - This incident has been resolved.
Jul 27, 17:12 CDT
Update - We're still root-causing this. We've remediated some of the problems and are watching closely. The problems appear to stem from instability in our IAD region.
Jul 27, 16:44 CDT
Investigating - We're currently seeing evidence of instability in our API, which will impact deployment of new instances and Fly machines. We'll keep you posted.
Jul 27, 15:08 CDT