VoIP Enrich Issue

Posted: July 21st, 2016 at 10:00 by Adam Heath

We are currently aware of an issue affecting a portion of our Enrich users.  Initial reports highlight a problem with both inbound and outbound calls.  We suspect this is being caused by a 3rd party outage elsewhere, however are currently investigating further.  We will provide further information as it becomes available.

  1. Adam Heath says:

    The issue appears to be resolved following engineers restarting a process on one of our media servers. Initial suspicions of an outage elsewhere being the cause were incorrect, but considered due to timing of the events correlating.

Broadband Problems

Posted: July 20th, 2016 at 09:04 by Adam Heath

We are currently aware of problems affecting the broadband platform with reports of numerous services being offline. Sources suggest this may be an issue on the BT network rather than our own, but we are continuing to investigate and gather information. Unfortunately, bt.com and btwholesale.com are also currently offline, meaning we have limited resources available to investigate at present.

We will update when further information becomes available.

  1. Adam Heath says:

    The problem has been confirmed on BT’s side and they are currently investigating. We haven’t been provided anything relating to the cause as of yet, but we will update upon receipt of this information.

    Unfortunately the BT systems remain down preventing us from looking into individual line issues.

  2. Adam Heath says:

    BT released information a short while ago advising that the problems are ‘due to power issues at one of our internet peering partners’ sites in London’. Work continues to be undertaken on BT’s side to restore service and we will update in due course.

  3. Adam Heath says:

    BT have advised that they are restoring services now. We also currently have access to some of the BT systems which were earlier unavailable. We are treating these as still at risk however until confirmation of restoration comes from BT.

  4. Adam Heath says:

    We have received confirmation from BT regarding the restoration of their systems which we access. Reports of users having no connectivity have also ceased coming into our desk, in relation to the earlier outage.

    We will be following up with BT to obtain any further formal information they release.

Possible fibre cut

Posted: July 12th, 2016 at 22:09 by Steve Lalonde

We are investigating a fault between Telehouse and Interxion that appears to be a fibre cut affecting multiple circuits.

We are experiencing some congestion as traffic is flowing over alternative paths.

Engineers are working to rebalance traffic.

  1. Steve Lalonde says:

    Traffic balancing work has reduced the congestion

    We are expecting the broken circuits to be restored at approx 02:00

  2. Steve Lalonde says:

    Latest update from supplier is links should be restored shortly.

  3. Steve Lalonde says:

    Service has been restored

    We will continue to monitor

Planned Maintenance: interxion3.core.enta.net

Posted: July 6th, 2016 at 15:32 by Ming-Yu Hsieh

Friday 15th July 2016 0:01 ~ 06:00

During the above window we will be doing software upgrade on interxion3.core.enta.net.

We expect downtime of approximately 1 hour during the window.

Affected services are:

  • All customer interconnects ( Transit / EWCS / Wholesale handoff / Pseudowire ) terminating at interxion3.core.enta.net

Traffic that would normally be routed through this equipment will be routed via alternate paths.

  1. Ming-Yu Hsieh says:

    This work is now complete.

Planned Maintenance: telehouse-east4.core.enta.net

Posted: July 6th, 2016 at 15:29 by Ming-Yu Hsieh

Thursday 14th July 2016 0:01 ~ 06:00

During the above window we will be doing software upgrade on telehouse-east4.core.enta.net.

We expect downtime of approximately 1 hour during the window.

Affected services are:

  • All customer interconnects ( Transit / EWCS / Wholesale handoff / Pseudowire ) terminating at telehouse-east4.core.enta.net

Traffic that would normally be routed through this equipment will be routed via alternate paths.

  1. Ming-Yu Hsieh says:

    This work is now complete.

EWCS (L2TP) Service Improvements

Posted: July 6th, 2016 at 12:05 by Steve Lalonde

We are currently improving capacity and resiliency of our Broadband network. As a result of this we are making some changes to the LTS pool serving our wholesale customers.

The LTS subnet 78.33.252.0/23 will be split into two parts

  • 78.33.252.0/24 will serve IPSC connections
  • 78.33.253.0/24 will serve WBC connections

Connections will be from multiple addresses in the subnets and additional LTS can be added at any time as the number of connections increases, please make sure that your configuration allows connections from all of the IP addresses on the subnets.

Please make any changes required to your configuration ASAP You will start to see connections from the new LTS in the next 2 weeks.

We will also be retiring all of the LTS in 62.249.255.128/27 over the next few months. We will update you further when you can remove this range from your configuration.

Planned Maintenance: kingston.core.enta.net

Posted: June 29th, 2016 at 16:31 by Jake Turner

Thursday 7th July 2016 0:01 ~ 06:00

During the above window we will be replacing kingston.core.enta.net with a higher capacity router.

All services will be migrated to the new router, we expect downtime of approximately 1 hour during the window.

Affected services are:

  • All customer interconnects ( Transit / EWCS / Wholesale handoff / Pseudowire ) terminating at kingston.core.enta.net
  • IPSC and WBC interconnect

Traffic that would normally be routed through this equipment will be routed via alternate paths.

  1. Mark Yardley says:

    This work is now complete.

Incident: Viper Hosting Platform

Posted: June 26th, 2016 at 14:26 by Jake Turner

We are currently investigating an issue causing a loss of service affecting our Viper hosting platform.

Engineers are currently investigating and further updates will be provided when available.

  1. Jake Turner says:

    All services are now restored for Viper customers.

    We apologise for the inconvenience caused.

Telephone System

Posted: June 21st, 2016 at 14:57 by Ryan Berrisford

We are aware of an issue which is affecting our in-office telephone system. Inbound calls into the office are still working but some functionality of the telephone system has been lost. We are already engaged with our suppliers and further updates will follow.

  1. jonathan clarke says:

    Suppliers have advised that this issue is now a major service outage and are still working to resolve the fault as soon as possible. We will advise an update as soon as a further update is available.

  2. jonathan clarke says:

    Suppliers have advised that the issue remains at present and further monitoring and testing is taking place. Suppliers advise they will be in touch again when a further update is available.

  3. Ryan Berrisford says:

    Our suppliers have confirmed that after changes made by their Operations Team, normal service has been resumed. Furthermore our suppliers have confirmed they are carrying out investigations into the root cause of the incident.

    We apologise for any inconvenience caused to our customer base during the incident, and will post more information as it becomes available.

Telephone System

Posted: June 8th, 2016 at 15:32 by Ryan Berrisford

We are aware of an issue with incoming calls into our in-office telephone system. We are currently investigating the issue and further updates will follow.

  1. Ryan Berrisford says:

    Service is now restored and we apologise for any inconvenience caused by this incident.