Resolved -
This incident has been resolved and all hosts in BOM are accurately reporting metrics.
Jan 12, 04:29 UTC
Update -
One host in BOM remains reporting delayed metrics as it continues to catch up. All other hosts in BOM are reporting metrics correctly.
If needed, users with impacted machines can use `fly machine clone` to create new machines in the region, which should land on a different host.
Jan 11, 18:43 UTC
Update -
Metrics have returned to normal for most hosts in BOM. Two hosts are still reporting delayed metrics, but are continuing to catch up.
Users with impacted machines can use `fly machine clone` to add new machines, which should land on a different host.
Jan 11, 05:36 UTC
Update -
Metrics have completed backfilling and are up to date on most hosts in the BOM region.
Two hosts are still working through the backlog; machines on those two hosts are still reporting delayed metrics at this time.
Jan 10, 01:28 UTC
Update -
We are continuing to process the metrics backlog in BOM. Progress is being made, but due to the volume of metrics this may still take some time to fully complete.
At this time users with machines in BOM should see metrics from the past 12h beginning to backfill into fly-metrics.net. However most will not be fully caught up yet.
Jan 9, 20:36 UTC
Update -
We are continuing to work through the backlog of metrics in BOM. Metrics will remain unavailable for BOM machines until this is complete.
Jan 9, 17:59 UTC
Update -
Our metrics cluster is continuing to working through the backlog of metrics in BOM. Metrics for machines running in the BOM region will continue to be unavailable until this is complete.
Jan 9, 15:20 UTC
Identified -
The cause of the issue has been identified and a fix is being implemented.
Metrics for machines in the BOM region remain unavailable in fly-metrics.net, however the machines themselves continue to run, start, and stop normally.
Jan 9, 14:07 UTC
Investigating -
We are currently investigating issues collecting machine metrics for machines running in the BOM (Mumbai, India) region.
Machines in the region continue to run, start, and stop normally, however metrics for these machines are not displaying in fly-metrics.net.
Jan 9, 12:53 UTC