On October 19th, 20th, and 22nd, Bigleaf Networks and a subset of Bigleaf customer sites/circuits in our NYC PoP were impacted by connectivity issues with an upstream network and backbone provider due a series of power failures at an upstream facility in NYC.
Times of impact:
10/19 - 8:32 PM EDT to 1:14 AM EDT 10/20 - 4:56 PM EDT to 1:31 AM EDT 10/22 - 6:43 AM EDT to 8:01 AM EDT
Per the upstream providers that were impacted, their datacenter/network facility in NYC lost their powergrid on 10/19 and have had several other power/backup generator issues following the initial failure.
Although the instances of downtime with this provider are significant, Bigleaf Networks was able to rely on alternative providers and paths to route around these issues once they were detected.
A subset of Bigleaf customers were impacted by these events during those routing changes and avoiding this affected network path - which resulted in some sites or circuits losing connectivity for 2-5 minutes during those affected times of impact above.
At this time, our team is working closely with the upstream provider to ensure all related issues are resolved and we continue to monitor closely.