EWCS (L2TP) LTS and Radius details – New IP details

Posted: May 14th, 2013 at 16:36 by Neil Watson

We wish to advise our EWCS partners that they may soon start to see sessions from additional LTS IP addresses. All addresses used will be within the previously announced ranges of:

Radius: 62.249.192.160/27, 78.33.48.32/28
LTS: 62.249.255.128/27, 78.33.252.0/23

The actual addresses that will be in use are

62.249.255.128/30 = WBC
62.249.255.136/30 = WBC
62.249.255.144/30 = WBC
62.249.255.148/30 = WBC
78.33.253.72/30 = WBC
78.33.253.76/30 = WBC
78.33.253.80/30 = WBC
78.33.253.84/30 = WBC

62.249.255.132/30 = IPSC
62.249.255.152/30 = IPSC
62.249.255.140/30 = IPSC
62.249.255.156/30 = IPSC

As these ranges have previously been announced, this notification is for information only as no config changes should be required by our partners. We would ask that you verify your configurations to make sure that traffic is poining to the correct LTS for the technology in use.

Planned Maintenance: DSL Network

Posted: May 9th, 2013 at 16:34 by Steve Lalonde

Friday 7th June 2013, 00:00 – 02:00

During the above window we will be modifying the routing configuration of the DSL network in Telehouse and Interxion to improve traffic flow and reduce the time taken to route around faults.

The work will cause some routing instability, we do not expect there to be any downtime.

This is rescheduled work from 11th May

 

Planned Maintenance: Telford based Virgin Media connectivity

Posted: April 26th, 2013 at 15:42 by Richard Partridge

Sunday 12th May 2013, 00:00 – 12:00

During the above window, necessary maintenance work will be carried out on the Virgin Media connectivity into our Telford headquarters.

This will only affect customers who take Virgin Media based services which terminate directly into our Telford Data Centre. These customers will be notified directly by a representative of the Support team.

  1. Ming-Yu Hsieh says:

    This maintenance work is now complete.

Planned Maintenance: DSL Network

Posted: April 25th, 2013 at 10:00 by Steve Lalonde

 Saturday 11th May 2013, 00:00 – 01:00

During the above window we will be modifying the routing configuration of the DSL network in Telehouse and Interxion to improve traffic flow and reduce the time taken to route around faults.

The work will cause some routing instability, we do not expect there to be any downtime.

This is the rescheduled maintenance originally planned for 27th April

  1. Steve Lalonde says:

    This work has been cancelled and will be rescheduled for a later date.

Planned Maintenance: DSL Network

Posted: April 22nd, 2013 at 17:20 by Steve Lalonde

 Saturday 27th April 2013, 00:00 – 01:00

During the above window we will be modifying the routing configuration of the DSL network in Telehouse and Interxion to improve traffic flow and reduce the time taken to route around faults.

The work will cause some routing instability, we do not expect there to be any downtime.

  1. Steve Lalonde says:

    This work has been cancelled and will be rescheduled for a later date.

PWAN Incident

Posted: April 22nd, 2013 at 13:52 by Richard Partridge

We are aware of an issue affecting PWAN connectivity into our Interxion Point of Presence. Our engineers are already investigating the cause of the problem.

 

Further updates will be provided once further information becomes available.

  1. Richard Partridge says:

    Our engineers are still investigating the cause of this issue. Further updates will follow.

  2. Richard Partridge says:

    Service has now been restored. Our engineers are continuing their investigation into the root cause of this incident.

  3. Richard Partridge says:

    An incident report has been made available for this issue. It can be found here.

Interxion DSL DDOS

Posted: March 28th, 2013 at 15:40 by Richard Partridge

We are currently dealing with an issue which is causing disconnections for a small number of DSL connections. This appears to be a DDOS directed at parts of our DSL network based in Interxion.

Our engineers are working on resolving the issue and further updates will follow as soon as they become available.

  1. Richard Partridge says:

    The inbound flow of traffic has now ceased so service has returned to normal. We are continuing to monitor the network and investigate the source of this attack.

Virgin Media Leased Lines

Posted: March 22nd, 2013 at 15:28 by Richard Partridge

We have just experienced a brief outage on several Virgin Media provided leased line circuits provided into Telehouse East.

This appears to have been the result of a Virgin Media engineer accidentally disconnecting part of the Virgin Media equipment. Connectivity has already been restored.

Connectivity Incident

Posted: March 22nd, 2013 at 11:07 by Richard Partridge

We are aware of an network incident affecting WBC connectivity this morning at approximately 10:00. Traffic was rerouted across alternative paths and service was reinstated minutes after.

Our initial investigations have suggested this may be a third party related issue however this is still being validated. Further updates will be provided as and when they become available.

  1. Richard Partridge says:

    We are aware of a repeat of the same issue experienced earlier this morning. We have identified a line card issue which we are expecting to rectify shortly. We will update again once we have further information.

  2. Richard Partridge says:

    Services has now been stabilised following the rerouting of connectivity away from the affected line card. We are continuing to work on rectifying the underlying issue. We are aiming to update this every 30 minutes until the problem is fully resolved.

    Following further investigation, the subject of this post has been modified to more accurately reflect the nature of this incident.

  3. Richard Partridge says:

    We believe the line card issue has been rectified, and it is now ready for normal operation once again. However, we are only passing minimal traffic through the card to give us an opportunity to monitor it before returning it to full service.

    As per the previous update, customer traffic continues to be routed around this card for the time being.

    We will update again in approximately 30 minutes.

  4. Richard Partridge says:

    The line card has remained stable since our last update. We are continuing to monitor the line card at the moment and customer traffic is still being routed around it.

    We will update again in approximately 30 minutes.

  5. Richard Partridge says:

    Our engineers are still monitoring the line card at the moment, and it has continued to remain stable since the previous update.

    We will update again in approximately one hour.

  6. Richard Partridge says:

    We are gradually increasing the amount of traffic routing through the original path and the service has remained stable as expected.

    We will continue to monitor the traffic and we will aim to provide another update at approximately 17:00 today.

  7. Richard Partridge says:

    The line card has remained stable, however the underlying cause which triggered this incident in the first place warrants further investigation with Cisco. As a result of this, the line card will remain running on a reduced workload and we will continue monitoring it’s status for the time being.

    Once the case is fully investigated and concluded an incident report will be published. We expect to be able to provide the next update on Monday 25th March.

  8. Wayne Hawtin says:

    The service has been monitored closely and remained stable as expected over the weekend. We will continue to reroute traffic until we are satisfied the risk is fully removed.

  9. Richard Partridge says:

    An incident report has been made available for this issue. It can be found here.

Network Incident

Posted: March 21st, 2013 at 19:42 by Richard Partridge

We’re currently aware of a network issue affecting portions of the customer base. Engineers are working towards a resolution at present.

 

More information will be provided as and when it becomes available.

  1. Richard Partridge says:

    Services are being restored gradually and we are seeing an increasing number of customers’ services reinstated. We are continuing to investigate and will provide further updates as soon as possible.

  2. Richard Partridge says:

    Between approximately 18.30hrs we began work to resolve the issue and by 20.45hrs all but two sites were reinstated successfully. The remaining sites were restored by approximately 21.30hrs and shortly after midnight. A detailed report is being prepared and will be published by 5:00pm today. We apologise for any inconvenience this issue may have caused.

  3. Neil Watson says:

    The Incident Report is now available for your information: here. We apologise for any inconvenience that this incident may have caused.