Quality issues in Central Europe
Incident Report for Pexip Service
Resolved
After heavily monitoring throughout the European business day. No further issues have been observed or reported and all systems in EMEA are operational at this time.
Posted Feb 06, 2018 - 19:49 UTC
Monitoring
Our network provider for the region has completed their emergency maintenance and has begun moving traffic off of backup links.

We will continue to heavily monitor connections through Central Europe during peak hours on Tuesday, 6 Feb.
Posted Feb 06, 2018 - 00:13 UTC
Identified
Videxio has been in close communication with our network provider for the area.

At this time calls within or through Central Europe are considered stable, customers may continue to see very low levels of packet loss.

Our network provider for this region has temporarily moved traffic to backup links until off hours for the region tonight when they will perform emergency maintenance replacing some hardware components.

Once the emergency maintenance is complete and the replacement components have been qualified; traffic will be moved back to primary links. At this time Videxio will move this incident into a monitoring state while we monitor the serviceability to ensure packet loss and latency return to normal operating levels.
Posted Feb 05, 2018 - 14:52 UTC
Investigating
A new incident of Service Network instability is now being observed in the EMEA Theatre which has a noticeable impact on the subscribers' traffic.

Symptoms of the new incident: elevated values of packet loss, failure to complete some calls, device registration instability, and degraded quality of the calls in progress.

The Videxio Operations team has contacted the ISP, who immediately started working on this new incident.
Posted Feb 05, 2018 - 09:45 UTC
Update
Another incident of Service Network instability was observed in the EMEA Theatre which may have had a noticeable impact to subscriber traffic. Symptoms may have included dropped calls, failure to complete calls, device registration instability, and degraded quality of completed calls.

Videxio Operations contacted its Network Service Provider to report the instability, and the Service Provider stabilized traffic by re-routing traffic to bypass an identified infrastructure device suspected of causing the instability.

Videxio Operations is continuing to monitor this situation and is working closely with the Network Service Provider in this region.

Time of incident Start: 10:30 UTC
Time of Incident End: 11:05 UTC
Posted Feb 02, 2018 - 16:55 UTC
Monitoring
The Videxio ISP has followed up with their DC vendor on the issue. As of 09:56 UTC the connectivity has been observed as stable and as of 10:18 UTC confirmed stability over the backup route also by the ISP.

The Operations team continues to monitor the network closely.
Posted Feb 01, 2018 - 11:31 UTC
Investigating
The Operations team has been notified by the ISP about issues with their primary path in one of our PoPs in Central Europe.

The perception of the issues is with a degraded quality since 08:30 UTC and affects calls made from or to Central Europe.
Posted Feb 01, 2018 - 08:46 UTC