Update - The DevOps team's monitoring has concluded that packet loss levels to subscribers on various ISPs is once again at normal levels. Our network provider for East US will continue to work with Level3 to return then transit to normal levels while also continue to build out the new direct peering relationship with Comcast.
Apr 18, 15:45 UTC
Update - While over the past 48 hours most connections through Level3 have only had very intermittent packet loss or had already been routed through different egress providers. Our network provider has disabled direct peering with Level3 until they are able to ensure a higher quality connection within the Level3 network.
If you or your end users are still experiencing packet loss or other network issues in transit please report the issue to the support team. The end user public IP and a MTR report is always appreciated.
Apr 18, 13:18 UTC
Update - The DevOps team and our network providers continue to monitor and reroute connections that experience higher than normal levels of packet loss. Please report any specific issues to the support team for optimization.
Our network provider has also begun the process to begin peering directly with Comcast to further reduce our transit over Level3 to end users on Comcast. This is not currently in production but will be in the near future.
Apr 15, 14:03 UTC
Update - The DevOps team and our network providers are continuing to evaluate transit through Level3, some paths have been restored for egress through our East Coast POP. We will continue to monitor and optimize the paths through out the day.
Apr 12, 14:17 UTC
Monitoring - The DevOps team continues to monitor transit through Level3, packet loss continues to show clean connections through the alternative paths. Our DevOps and network providers will continue monitor our primary peering on the East Coast will move traffic back once it is determined to be in good quality again.
Apr 11, 16:30 UTC
Identified - In efforts to reduce packet loss, Level 3 egress has been rerouted through our West Coast POP. While latency is slightly higher, packet loss has been reduced greatly.
Apr 11, 16:18 UTC
Update - The DevOps Team and our network provider continue to investigate peering with Level3, so far packet loss is occurring once within Level3 transit. The teams continue to investigate alternative paths to improve performance.
Apr 11, 16:12 UTC
Investigating - The DevOps team is currently working with our network providers investigating observed packet loss to some locations within the United States. As with the April 9th issue we have specifically identified customers who traverse through Level 3 are experiencing packet loss.
Apr 11, 15:33 UTC