Planned Work: Capacity upgrades

Posted: December 10th, 2014 at 10:20 by Steve Lalonde

Tuesday 16th December 2014 00:01 – 06:00

During the above window we will be inserting multiplexing hardware in to the fibre routes between

  • Telehouse North and Telehouse East
  • Telehouse North and Interxion

This work is to improve capacity of our network to meet increasing demand.

Traffic on the affected links will be routed over alternate paths during the work.

No downtime is expected.

  1. Ming-Yu Hsieh says:

    This work is now complete.

Planned Work: Capacity upgrades

Posted: December 5th, 2014 at 14:56 by Steve Lalonde

Tuesday 9th December 2014 00:01 – 06:00

During the above window we will be inserting multiplexing hardware in to the fibre routes between

  • Telehouse East and Interxion
  • Telehouse Metro and Interxion
  • Telehouse Metro and Telecity Hex

This work is to improve capacity of our network to meet increasing demand.

Traffic on the affected links will be routed over alternate paths during the work.

No downtime is expected.

  1. Steve Lalonde says:

    This work is now complete.

Incident: telecity-hex89.core.enta.net

Posted: December 2nd, 2014 at 19:11 by Steve Lalonde

this router has crashed during a line card insertion, we are rebooting the device.

  1. Steve Lalonde says:

    the router is back online.

Incident: lts04.the.dsl.enta.net

Posted: December 2nd, 2014 at 14:44 by Steve Lalonde

The above router has just crashed with a critical alarm, engineers are investigating. all sessions on this router will have disconnected and will reconnect via alternate routers.

  1. Steve Lalonde says:

    The router has been power cycled and is now back in service, we will open a case with the manufacturer to diagnose the cause.

  2. Steve Lalonde says:

    The router has crashed again, we have removed it from the pool and added all the about to be retired routers back in to handle the load.

  3. Steve Lalonde says:

    A replacement router is being shipped to us overnight.

Wolverhampton to Stoke

Posted: November 19th, 2014 at 22:54 by Steve Lalonde

This circuit is currently down due to a suspected fibre break, traffic usually on the route has been automatically re-routed via alternative paths.

  1. Ryan Berrisford says:

    This is confirmed as a fibre break between Birmingham and Longton. Supplier engineers are currently onsite working to restore the damaged fibres and an estimated restore time has now been given as 09:30.

    Further updates will follow as they become available.

  2. Ryan Berrisford says:

    Our suppliers have confirmed that fibre splicing is continuing. Due to the amount of work required the estimated time of repair has now been moved to 11:30.

    We will keep in close contact with our suppliers and post further updates as they become available.

  3. Ryan Berrisford says:

    Repair work on the fibre is continuing and our suppliers are still working towards the estimated resolution time of 11:30.

    We are also still communicating closely with our suppliers and any change to the estimated resolution time will be communicated via this post.

  4. Ryan Berrisford says:

    Further information has now become available from our supplier.

    The onsite fibre crew has advised that the repair of the fiber work has been extended due to a broken fiber tube within a joint. The onsite crew are now in the process of a cut and rebuild of the affected section.

    Our suppliers are working as quickly as possible to get the issue resolved, however due to the unforeseen complications an estimated time of repair cannot be given at this point. We will keep working with the supplier and feed back information as we get it.

  5. Ryan Berrisford says:

    In further communication with our supplier they have advised that the repair work is still being carried out.

    We will post out further updates on the progress as they are fed back from the supplier engineers on site.

  6. Ryan Berrisford says:

    Our suppliers have confirmed that the fibre tube has been repaired and splicing has once again continued. Our core link is already back up but this has to be considered at risk until the full completion of the splicing work.

    We will post further updates as they become available.

  7. Ryan Berrisford says:

    Our suppliers have now confirmed that the splicing work has been completed. The service is therefore no longer considered at risk.

Incident: LONAP Peering

Posted: November 19th, 2014 at 15:11 by Richard Partridge

We’re aware of an issue within LONAP. We’ve temporarily disabled our peering to LONAP for the time being.

Incident: interxion2.core.enta.net

Posted: November 13th, 2014 at 15:18 by Wayne Hawtin

We are aware of an issue with interxion2.core.enta.net. Service appears to be returning to normal, however we are continuing to investigate.

  1. Wayne Hawtin says:

    After a period of monitoring, service has remained stable. An incident report will be made available as soon as possible.

  2. Neil Watson says:

    This was related to yesterday’s Interxion incident and as such the report can be found here

Incident: Interxion Blade Failure

Posted: November 12th, 2014 at 08:41 by Richard Partridge

We are currently investigating what appears to be a blade failure within our interxion.core.enta.net router.

Further updates will be provided as and when they become available.

  1. Richard Partridge says:

    Entanet engineers have been dispatched to Interxion with replacement hardware. We will update again once our engineers have arrived..

  2. Richard Partridge says:

    DSL customers affected by this incident are being re-routed however we’re receiving reports of packet loss. This is currently under investigation.

  3. Richard Partridge says:

    Further investigation has shown that DSL traffic is re-routing around the blade failure in Interxion. However, this is causing increased strain on equipment forming parts of the alternative paths currently being used to bypass the failed blade.

    Engineers are still en-route to Interxion with the aim of replacing the failed hardware, however we’re continuing to determine if there is anything that can be done in the meantime to alleviate the packet loss.

  4. Richard Partridge says:

    Engineers are still en-route and are expected to arrive on site shortly.

  5. Richard Partridge says:

    Our engineers have now arrived on site.

  6. Richard Partridge says:

    Work to replace the blade is currently ongoing.

  7. Richard Partridge says:

    The blade replacement hasn’t stabilised the issue, so we will be rebooting the Interxion core immediately.

  8. Richard Partridge says:

    Service appears to have been restored. We will be monitoring the situation closely.

  9. Richard Partridge says:

    An incident report on this event is currently being produced and will be made available as soon as possible.

  10. Neil Watson says:

    An incident report is now available and can be found here

Planned Work: Telehouse North

Posted: November 11th, 2014 at 18:33 by Steve Lalonde

Wednesday 19th November 22:00 – Thursday 20th November 04:00

During the above window we will be inserting multiplexing hardware in to the fibre routes between

  • Telehouse North and Telehouse East
  • Telehouse North and Interxion

This work is to improve capacity of our network to meet increasing demand.

Traffic on the affected links will be routed over alternate paths during the work.

No downtime is expected.

  1. Ming-Yu Hsieh says:

    Wrong date was entered which have been corrected now.

  2. Steve Lalonde says:

    Due to the fibre break between Wolverhampton and Stoke we are postponing the work tonight. The new hardware has been installed in our racks but will not be connected to the network.

Planned Work: Telehouse East

Posted: November 11th, 2014 at 18:20 by Steve Lalonde

Tuesday 18th November 22:00 – Wednesday 19th November 04:00

Entanet Wholesale Broadband Customers.

During the above maintenance window we will be replacing the following routers with new much more powerful devices

  • lts01.the.dsl.enta.net
  • lts02.the.dsl.enta.net
  • lts04.the.dsl.enta.net

All 3 routers have not been accepting new connections for several weeks and it is expected that there will be less than 100 users still connected across these routers at the start of the work. Those users will reconnect to alternative routers during the work.

No disruption to any other broadband users is expected.

Following the LTS upgrades we will also be moving several servers to make room for additional hardware no impact is expected as these are all redundant systems.

 

 

  1. Ming-Yu Hsieh says:

    Wrong date was entered which have been corrected now.

  2. Steve Lalonde says:

    This work is now complete.