Packet loss on upstream provider nationwide

Incident Report for Bigleaf Networks

Resolved

Shortly after 3pm PDT this afternoon we saw significant packet loss on all connections to one of our upstream providers. We quickly turned down BGP peering with the provider in our Chicago and New York City Gateway Clusters, where the effects appeared to be most severe, to redirect flows to redundant connections and limit the impact to customer traffic traversing those clusters.

At this time the situation looks to have stabilized at all our Gateway Clusters and we have reenabled all BGP peering with the provider. We are continuing to monitor those connections closely and are standing by to mitigate any further issues if they occur. At this time we consider the issues resolved, but we will provide updates if there are further events.

Posted Aug 09, 2017 - 15:31 PDT