We are currently seeing errors on our link between London and Manchester. To maintain network stability, we are taking this link out of service while investigations take place. All traffic will route via alternate paths, some customers may notice a small increase in latency.
Archive for the ‘Outage’ Category
Emergency Maintenance: telford-dc3.core.enta.net
Posted: Monday, February 1st, 2010 at 12:34 by Steve Lalonde
Tuesday 2nd Feb 2010, 03:00hrs – 04:00hrs
To resolve the intermittent packet loss issue on the Telford to Wolverhampton link it has become necessary to reload telford-dc3.core.enta.net. This emergency work should take approx 10 minutes to complete, during which all directly connected customers will lose connectivity.
-
Steve Lalonde says:
This work was completed successfully. Traffic is now on the preferred paths and full resilience is restored.
Outage: Telehouse-north0.core
Posted: Friday, January 22nd, 2010 at 13:08 by Martyn Haynes
We are seeing issues accross the network that inital investigation shows could be related to a possible fault in Telehouse-north0.core. Our engineers are investigating these issues as a matter of urgency. Further information will be provided when available.
-
Martyn Haynes says:
We have stabilised the issue however it is not resolved. An Engineer is tasked to site to investigate further. Further updates will be posted when available.
-
We need to reload the router to eliminate the IOS version as the cause. the reload will be at 15:55
All traffic that flows via the router has been rerouted over alternative paths where possible, all directly connected customers will experience an outage of approx 10 minutes.
-
The router is now operational again we will keep the traffic routed over alternative paths while we monitor the system.
-
We are now routing approximately 1/3 of the normal traffic back through the router, the current priority is to get all the resiliency that relies on this hardware operating as normal. This should be achieved in the next 30 minutes.
We will continue to monitor the situation and update.
-
All resilient links are now operational, we are continuing to route traffic over alternate paths.
-
After monitoring for more than 24 hours, the router is stable and functioning normally, We have now routed traffic back on to the optimal paths.
-
Richard Partridge says:
There is now an outage report available for this issue. It can be found here.
Emergency work: man-te.core.enta.net
Posted: Wednesday, January 20th, 2010 at 12:33 by Steve Lalonde
Due to router instability we have rebooted this router.
Outage: Birmingham.core
Posted: Tuesday, January 19th, 2010 at 18:16 by Martyn Haynes
Due to advance preparation for tonight’s maintenance work, Birmingham.core has unexpectedly reloaded. Customers connected via this node will have suffered a short (<4 minutes) outage. Please accept our apologies for any inconvenience this may have caused.
As a result of this unexpected reload, tonight’s maintenance window for Birmingham.core is no longer needed.
Outage: Services into Slough
Posted: Saturday, January 16th, 2010 at 17:47 by Martyn Haynes
We are aware of a faul in our Slough POP which is affecting leased lines and some EWCS customers, initial investigation suggests a hardware failiure, we are attempting to resolve now. All ADSL services via Slough should be routed via alternative paths.
More information will be posted as we have it.
-
Engineers tasked to site with replacement parts. Estimated time of repair 22:00
-
Ming-Yu Hsieh says:
Estimated time of repair 23:00.
Outage: Leeds to Sheffield
Posted: Thursday, January 7th, 2010 at 08:58 by Phil Davies
The link between Leeds and Sheffield is currently down. A fault has been raised with our provider and they have confirmed they are dealing with an incident 2km outside of Leeds and updates will be made as and when they are available. Traffic usually on this route is using an alternative path, there may be a small increase in latency due to the change in routing. Due to adverse weather conditions the fix time may be extended.
-
Phil Davies says:
Our provider has confirmed they have a fiber break and engineers are on site preparing a length of fiber to replace the damaged one and splicing will begin at 12pm.
-
The service was restored at 12:27pm.
Outage: Newcastle to Leeds
Posted: Wednesday, December 30th, 2009 at 17:55 by Phil Davies
The link between Newcastle and Leeds is currently down. A fault has been raised with our provider and they have confirmed they are dealing with an incident between York and Middlesbrough and updates will be made as and when they are available. Traffic usually on this route is using an alternative path, there may be a small increase in latency due to the change in routing.
-
Our provider has confirmed that a fiber break has been identified in York and engineers are on route for further investigation.
-
Service was restored at 4.15am and the provider has confirmed this fault has now been closed off.
Outage: London to Manchester
Posted: Thursday, December 17th, 2009 at 10:41 by Phil Davies
The link between London and Manchester is currently down. A fault has been raised with our provider and updates will be made as and well they are available.
Traffic usually on this route is using an alternative path, there may be a small increase in latency due to the change in routing.
-
Service was restored at 05:15, engineers closed the fault at 07:30
Outage: Stoke to Manchester
Posted: Wednesday, December 9th, 2009 at 17:28 by Martyn Haynes
We appear to have lost a link between Stoke and Manchester. We are logging this with our provider and updates will be made as and well they are available.
Traffic usually on this route is using an alternative path, there may be a small increase in latency due to the change in routing.
-
Initial indications suggest that this is related to a fibre cut in the Warrington area – we will update as soon as it is confirmed to us.
-
Our suppliers have advised that engineering work is currently underway to repair the fibre damage and restore service. No ETR has been specified as of yet. Further information will be made available as we receive it.
-
Marc Woolfenden says:
Our suppliers have advised that re-splicing work has been completed, however upon retest a secondary break in the fibre has been identified in the Harecastle area. As the secondary break is inside a tunnel on the rail network our suppliers are currently liaising with Network Rail in order to avoid high speed rail services entering the tunnel in order to preform the necessary work, however we have been advised that work may not be able to continue until 23:00. Our suppliers are attempting to re-route traffic around the affected bearer, however no ETA is availiable.
Further information will be made availiable as we receive it.
-
Our suppliers have advised that repair work has been partially completed, and our network link has been restored. This incident is still ongoing however as the repair work has had to be suspended as high speed rail services have resumed.
This link should be considered at risk until the repair work is completed, and further information will be made available as soon as we receive it.
Our circuit provider has identified a possible fault with a card in Manchester. Engineers are working to confirm this, there is no ETR at this time. Further updates will follow as we receive them.
Our provider has cleared the fault, we will test the circuit for a errors over the next hour.
We have seen no more errors on the circuit and it is therefore now back in service. We will continue to monitor as the load increases.