During the above window we will be carrying out service affecting maintenance on our PWAN edge device located in our Wolverhampton Data Center.
We expect all services terminated on this device to have a maximum downtime of 3 hours. Services include PWAN firewalls, PWAN co-location and psuedowires terminated into customer PWAN solutions.
We have experienced a brief loss of connectivity to our equipment at our Cambridge PoP. Engineers are currently investigating, updates to follow as soon as available.
Engineers are investigating an issue which is currently rendering the Mirada portal unavailable. Further updates will be provided as and when they become available.
We are currently experiencing intermittent loss of connectivity to the above device, some PWAN customers with equipment located within our suite in VeloxServ may have experienced a short outage. Remote hands at the DC are currently investigating and we will aim to provide an update shortly.
From the first drop at around 10:40 we have seen 2 subsequent drops. Upon initial investigation it does not appear to be an issue with the physical cabling causing a link loss. We are therefore going to perform an emergency reload of the switch stack to try and rectify the issue.
The switch stack has now reloaded and connectivity should start to be restored. Engineers are monitoring closely and continuing to investigate a possible cause for the initial losses of connectivity.
Connectivity has been lost again after the emergency reload. Work is continuing to try and bring back stability to the switch stack. We will provide further updates as they become available.
Work is continuing, however the longest period of stability since the initial outage this morning has just been observed. Our engineers may have found the possible cause however we will continue to provide updates until we are certain the issue has been fully resolved.
We now believe this incident to be resolved. Investigations are still ongoing as to the root cause, which may result in additional works being required. These works (if required) will be scheduled and conducted as per normal change process.
Jamie Smith says: Between the hours of 17:00 – 18:00 we will be carrying out emergency works to restore resiliency to this device by re-introducing the uplinks back into an ether-channel. This work is non service affecting however all PWAN services and PWAN colocation in this location should be considered at risk for the duration of this window.
Jamie Smith says: This work is now commencing
Jamie Smith says: This work has now been completed.
We are aware of an incident affecting some PWAN ADSL/VDSL services. Engineers are already engaged and are investigating.
Further updates will be provided as and when they become available.
Jordan Coleman says: We now believe this issue to be resolved. Any remaining customers that are affected are advised to reboot their router before calling into the Service Desk.
Saturday 23rd November 2019 22:00 – Sunday 24th November 2019 06:00
During the above window CityFibre will be carrying out a planned maintenance on our network in Leeds.
We expect all services to have a maximum downtime of 5 hours. Services include Leased lines, Metro Services and Pseudowires terminating at this location. Whilst actual downtime is expected to be no more than 5 hours, the full 8 hours may be required to account for any unforeseen issues.
All other traffic that would normally route through this location will take alternative paths.
Friday 15th November 2019 01:00 – 03:00
During the above window Vodafone will be completing a planned maintenance to upgrade firmware on a single handover point. It is expected for all affected connections to experience 1 hour downtime, however the full duration of the window may be required to account for any unforeseen issues.
Investigations highlight this only impacted leased line services in Edinburgh. We are starting to see services restore as we continue investigations to find the root cause.
After a state of monitoring all services restored and remained stable by 10:15. We apologise for any inconvenience this caused.