Archive for January 22nd, 2010

Outage: Telehouse-north0.core

Posted: Friday, January 22nd, 2010 at 13:08 by Adam Heath

We are seeing issues accross the network that inital investigation shows could be related to a possible fault in Telehouse-north0.core. Our engineers are investigating these issues as a matter of urgency. Further information will be provided when available.

  1. Martyn Haynes says:

    We have stabilised the issue however it is not resolved. An Engineer is tasked to site to investigate further. Further updates will be posted when available.

  2. Steve Lalonde says:

    We need to reload the router to eliminate the IOS version as the cause. the reload will be at 15:55

    All traffic that flows via the router has been rerouted over alternative paths where possible, all directly connected customers will experience an outage of approx 10 minutes.

  3. Steve Lalonde says:

    The router is now operational again we will keep the traffic routed over alternative paths while we monitor the system.

  4. Steve Lalonde says:

    We are now routing approximately 1/3 of the normal traffic back through the router, the current priority is to get all the resiliency that relies on this hardware operating as normal. This should be achieved in the next 30 minutes.

    We will continue to monitor the situation and update.

  5. Steve Lalonde says:

    All resilient links are now operational, we are continuing to route traffic over alternate paths.

  6. Steve Lalonde says:

    After monitoring for more than 24 hours, the router is stable and functioning normally, We have now routed traffic back on to the optimal paths.

  7. Richard Partridge says:

    There is now an outage report available for this issue. It can be found here.