Archive for the ‘Incidents’ Category

Incident: Leased lines in Sheffield, Peterborough and Nottingham

Posted: Tuesday, September 15th, 2020 at 12:03 by david labouchardiere

We are aware of an issue affecting leased lines in and around Sheffied, Peterborough and Nottingham. Our networking team are currently investigating the issue and further updates will be provided as soon as possible.

  1. mpurcell says:

    Sheffield and Nottingham core routers are back online but we still have an issue in Peterborough. Engineers are being dispatched.

  2. mpurcell says:

    Engineers are expected on site at 1500

  3. mpurcell says:

    The Engineer is due to arrive on site imminently.

  4. mpurcell says:

    Engineer is now on site and investigations have begun

  5. mpurcell says:

    NOC have now gained access to the core router and have begun troubleshooting the issue.

    Further updates to be published at 1730

  6. mpurcell says:

    Services have been restored. A reboot was required which has resulted in the data necessary to carry out further root cause analysis to be lost. A reboot in this instance was the last available option in order for us to restore service.

    Due to this we will continue to monitor for the next 48hours.

Incident: BT Edinburgh

Posted: Wednesday, August 12th, 2020 at 08:57 by David Brewis

A major incident has been declared by BT for an exchange in Edinburgh impacting broadband and ethernet services.

Loss of service is a result of water ingress into the location. The emergency command centre is onsite with majority of water removed, with engineers ensuring safe to enter to continue investigations and restore services as soon as possible.

Further updates will be provided.

  1. David Brewis says:

    Engineers remain on site with a plan in place, working to attempt to restore service. Equipment continues to remove remaining water with spare hardware and cabling on site and on route.

    Next update to follow by 13:30

  2. David Brewis says:

    Water has been removed from the building and clean up is in progress. Cables have been reconnected providing restoration steps with testing being carried out.

    Works will continue on site ensuring the location is secure and no further risk.

    Next update to follow by 15:30

  3. David Brewis says:

    Sessions appear to have returned from around 13:30 and we will continue to monitor our network equipment and the progress of clean up activities.

    No further updates to follow unless further disruption is caused by these activities as the incident is now resolved. If you experience a loss of service please call into the helpdesk following a reboot of your equipment.

Incident: Goswell Core

Posted: Friday, June 12th, 2020 at 04:58 by Matthew Davies
We have experienced a loss of connectivity to our Goswell core. This has been raised with our colocation providers, and their initial diagnostics suggest a wider power issue to be the cause.  Investigations are ongoing, and further updates will be provided when available.
  1. Matthew Davies says:

    Our colocation providers have confirmed a power outage at our Goswell POP. Engineers are working to restore power and a further update will be provided when available.

  2. Matthew Davies says:

    Our colocation providers have restored power at our Goswell POP. We apologise for any inconvenience caused.

Incident: L2TP broadband

Posted: Tuesday, March 31st, 2020 at 11:46 by david labouchardiere

We are currently aware of an issue affecting authentication on L2TP broadband circuits. Engineers are already engaged and are investigating.

Further updates will be provided as and when they become available.

 

  1. david labouchardiere says:

    Engineers confirm service has now been restored. Please ask any customers still having issues to reboot their router before contacting the service desk. We apologise for any inconvenience caused.

Incident: slo24-rtr-1

Posted: Wednesday, March 18th, 2020 at 14:40 by Jordan Coleman
We are currently aware of an issue affecting the slo24-rtr-1 PE. Initial diagnostics indicate a line card failure. Engineers are investigating and engaging with our supplier. Further updates will be provided when available.
  1. Aiden Spencer says:

    Engineers have completed troubleshooting and have identified a hardware failure. New hardware is en-route to site and will be installed upon arrival, expected within the next 2 hours.

  2. Aiden Spencer says:

    The issue has been resolved and the services have now been restored.