Change Summary: Wholesale Ethernet Planned Works Full Outage multiple Sky Circuits.
Type of Change: Planned
Impact: Full outage to service
Start Date: 30/10/2020 00:00:00 GMT
End Date: 31/10/2020 06:00:00 GMT
Start Date: 31/10/2020 00:00:00 GMT
End Date: 01/11/2020 06:00:00 GMT
Start Date: 01/11/2020 00:00:00 GMT
End Date: 02/11/2020 06:00:00 GMT
Reason for work: Maintenance-Third party work
Apologies for any disruption this may cause.
Good Morning.
We have had a large number of SKY circuits go offline for a period of 30 minutes early this morning. This has been due to unplanned works performed by SKY, but as of 00:55AM. All services should be restored.
We apologise in advance for the inconvenience caused. Given this happened yesterday and again this morning, we are taking further measures to ensure with the supplier there are no further unexpected works.
If you are experiencing any further issues since this morning’s works, please re-boot your service and proceed to call into the support desk if the issue remains.
Good Morning.
We are currently seeing a large number of SKY leased line circuits offline. At this moment in time, this is under investigation and we are currently in correspondence with suppliers for diagnostics.
Further updates will be provided when available.
jbiddulph says: Good Morning.
The majority of the circuits appear to now be restored.
Investigations are currently still underway to identify the cause of this outage.
Further updates will follow when available.
jbiddulph says: Good Morning.
We have had confirmation from the supplier that this was part of a maintenance change on the SKY side of an ENNI. All services on this ENNI were unexpectedly taken offline as a result.
We are now pushing for the specific details and RFO for the unplanned outage.
Further updates will follow when available.
jbiddulph says: Good Morning.
The supplier has confirmed the outage that took place was due to the movement of circuits on their side to a Device Card. This was to enable further services for their hardware, but unfortunately due to the lack of awareness made to ourselves, this was an unexpected outage.
We have full confirmation that all services involved are now back online and restored.
If you have any services that you believe to be related to this, please fully re-boot the service and re-test the service. If you are having continued issues after this, please contact the Technical Support Department.
Thursday 12th November 2020, 21:00 – Friday 13th November 2020 05:00
Due to on going improvements to the power infrastructure within the BT POP’s, Bristol TE requires both the ATS and UPS to be refreshed.
Therefore, BT will be undertaking maintenance work within the above window which will require the power feeds to be moved, whilst an ATS & UPS are swapped within the rack. Although the core network router is not expected to experience any loss of service, customers connecting into the adjacent equipment may experience a small period of downtime.
Thursday 05th November 2020, 21:00 – Friday 06th November 2020 05:00
Due to on going improvements to the power infrastructure within the BT POP’s, Stepney TE requires both the ATS and UPS to be refreshed.
Therefore, BT will be undertaking maintenance work within the above window which will require the power feeds to be moved, whilst an ATS & UPS are swapped within the rack. Although the core network router is not expected to experience any loss of service, customers connecting into the adjacent equipment will experience a small period of downtime.
Naz says: work is now commencing
Naz says: Work has been completed
We are currently seeing a number of Virgin Media leased line circuits as down. These circuits appear to be in the Birmingham area at present. We are currently in correspondence with suppliers for diagnostics.
Further updates will be provided when available.
Jonathan Clarke says: Suppliers has advised that this is already a known issue. Engineers are currently on route to site and their current ETA is 18:15, for further investigation.
Virgin media are proactively updating us again at 18:45.
Further updates will be provided when available.
Jonathan Clarke says: Suppliers has advised that the engineer is now on site.
ETA is 19:30
Further updates will be provided when available.
Jonathan Clarke says: Suppliers are now seeing services restored. Their engineer is still on site working on this issue.
When we have confirmation from them that that the issue is fully resolved we will update you again.
Services should still be considered at risk until we have had confirmation.
Matthew Davies says: Our suppliers have now advised that work has now been completed, and these circuits should no longer be at risk.
They advised the below work has been carried out:
A high level transmission card was reseated which restored services.
If you have a service that is still down, please reboot the equipment on site and contact the support department after this has been done.
We are currently aware of an issue affecting CityFibre provided pseudowire circuits through Telehouse North. Engineers are investigating and further updates will be provided as soon as possible.
david labouchardiere says: Engineers have investigated and believe the issue to now be resolved. Please ask any customers still having issues to reboot their router before contacting the service desk. We apologise for any inconvenience caused.
Thursday 29th October 2020, 21:00 – Friday 30th October 2020 05:00
Due to on going improvements to the power infrastructure within the BT POP’s, Slough TE requires both the ATS and UPS to be refreshed. Although only Slough is directly effected due to network design customers with services on Kingston & Colindale core’s should also be considered at risk.
Therefore, BT will be undertaking maintenance work within the above window which will require the power feeds to be moved, whilst an ATS & UPS are swapped within the rack. Although the core network router is not expected to experience any loss of service, customers connecting into the adjacent equipment will experience a small period of downtime.
Thursday 22nd October 2020, 21:00 – Friday 23rd October 2020 05:00
Due to on going improvements to the power infrastructure within the BT POP’s, Preston TE requires both the ATS and UPS to be refreshed.
Therefore, BT will be undertaking maintenance work within the above window which will require the power feeds to be moved, whilst an ATS & UPS are swapped within the rack. Although the core network router is not expected to experience any loss of service, customers connecting into the adjacent equipment will experience a small period of downtime.
Naz says: Work is now commencing
Naz says: Work has been completed successfully
The above incident is part of a 5 day planned works, starting from 28/10/2020. The details of this disruption are below:-
Change Summary: Wholesale Ethernet Planned Works Full Outage multiple circuits
Type of Change: Planned
Impact: Full outage to service
Start Date: 26/10/2020 00:00:00 GMT
End Date: 30/10/2020 06:00:00 GMT
Reason for work: Maintenance-Third party work
Apologies for any disruption this may case in the meantime.