VoIP Enrich Incident

Posted: April 18th, 2016 at 13:39 by Adam Heath

We are aware of a potential problem on the VoIP Enrich platform.  Engineers are currently investigating and we will post further information as it becomes available.

  1. Adam Heath says:

    Our NOC engineers have confirmed that one of the media servers had crashed. They restarted the process and service seems to have been restored.

    Engineers will be investigating the crash log in an attempt to ascertain the cause.

Emergency Maintenance: DSL network

Posted: April 11th, 2016 at 16:12 by Steve Lalonde

Tuesday 12 April 2016 0:00 ~ 8:00

During the above window we will be making some changes to the DSL network to reduce the packet loss some users have been reporting.

The work may cause some users to disconnect/reconnect

No other impact is expected.

  1. Steve Lalonde says:

    This work was completed at approx 8:30

Planned Maintenance: Interxion PWAN Infrastructure Final Phase

Posted: April 11th, 2016 at 10:40 by Ming-Yu Hsieh

Tuesday 19th April 2016 00:01 – 06:00

During the above window we will be migrating customers firewalls on the Interxion PWAN switch to our VPLS platform. Due to the amount of customers on this site, we will be migrating customers in phases, each customer will have downtime of up to 30 minutes during the window.

All affected customers will be contacted by our support team in advance.

  1. Phil Davies says:

    This work is now complete.

Planned maintenance: VoIP media servers

Posted: March 31st, 2016 at 15:31 by Ben Bliss

Thursday 7 April 2016 10:00-15:00

During the above window we will be making additional changes to the media servers on the VoIP system. This is a continuation of the work carried out on the 31st March 2016.

We will work on each server in turn, removing it from the pool and waiting for its calls to complete before performing the work. It will then be tested before returning to the pool and moving on to the next.

Please note that since the system is designed to cope with server failure we do not expect any customer impact from this. In the event of problems we will roll back to the previous state.

  1. Ben Bliss says:

    The work is now complete. Additional maintenance will be needed at a later date, which will be announced shortly.

Planned Maintenance: Interxion PWAN Infrastructure Phase 2

Posted: March 29th, 2016 at 14:55 by Ming-Yu Hsieh

Tuesday 5th April 2016 00:01 – 06:00

During the above window we will be migrating customers firewalls on the Interxion PWAN switch to our VPLS platform. Due to the amount of customers on this site, we will be migrating customers in phases, each customer will have downtime of up to 30 minutes during the window.

All affected customers will be contacted by our support team in advance.

  1. Ming-Yu Hsieh says:

    This work is now complete.

Incident: Telehouse North

Posted: March 15th, 2016 at 09:16 by Neil Watson

We are currently aware of issues affecting connectivity to our Telehouse North core router. Customers directly connected to this router will be affected.

Engineers are currently investigating and further updates will follow.

  1. Steve Lalonde says:

    We are starting to roll back the maintenance from last night as the new router has failed.

  2. Ryan Berrisford says:

    All affected customers will now have service restored. We apologise for the inconvenience caused and a full report will follow.

  3. Ryan Berrisford says:

    An incident report for this issue is now available here.

Planned maintenance: VOIP media servers

Posted: March 11th, 2016 at 15:08 by David Derrick

Thursday 31 March 2016 10:00-15:00

During the above window we will be making changes to the media servers on the VOIP system. We will work on each server in turn, removing it from the pool and waiting for its calls to complete before performing the work. It will then be tested before returning to the pool and moving on to the next.

Please note that since the system is designed to cope with server failure we do not expect any customer impact from this. In the event of problems we will roll back to the previous state.

  1. Ben Bliss says:

    The maintenance was a success, but due to time constraints, further work is needed to roll the changes out across the rest of the media server cluster.

    This will be scheduled for a later date.

Planned Maintenance: Telehouse-north.core.enta.net

Posted: March 4th, 2016 at 12:06 by Ming-Yu Hsieh

Tuesday 15th March 2016 00:01 – 06:00

During the above window we will be replacing telehouse-north.core.enta.net with a higher capacity router.

All services will be migrated to the new router, we expect downtime of approximately 1 hour during the window.

Affected services are:

  • All customer interconnects ( Transit / EWCS / Wholesale handoff / Pseudowire ) terminating at telehouse-north.core.enta.net

Traffic that would normally be routed through this equipment will be routed via alternate paths

  1. Ming-Yu Hsieh says:

    This work is now complete.

Planned Maintenance: Telehouse East Phase 2

Posted: February 24th, 2016 at 08:44 by Ming-Yu Hsieh

Thursday 3rd Mar 2016 0:01 – 06:00

During the above window we will be migrating customers on telehouse-east.core.enta.net and telehouse-east2.core.enta.net to new routers in Telehouse East. Due to the amount of customers on this site, we will be migrating customers in phases, each connection will have downtime of up to 30 minutes during the window.

Affected services are:

  • All customer interconnects ( Transit / EWCS / Wholesale handoff / Pseudowire ) terminating on telehouse-east.core.enta.net and telehouse-east2.core.enta.net
  1. Ming-Yu Hsieh says:

    This work is now complete.

Incident: Virgin Media Based Circuits

Posted: February 16th, 2016 at 12:57 by Scott Morgan

We are currently aware of an issue affecting a number of Virgin Media based circuits. We are working with our supplier who are now commencing investigation.

We will continue to work with our suppliers and publish updates as they become available.

  1. Scott Morgan says:

    We have identified this as affecting a single Virgin Media NTU impacting approximately 200 circuits.

    We are continuing to work closely with our suppliers in order to get the issue resolved and further updates will follow, as they become available.

  2. Ryan Berrisford says:

    Our suppliers are engaging and dispatching engineering resource to the point of presence in InterXion.

    An estimated time of arrival for the engineering resource will follow shortly, which will be updated via this post.

  3. Ryan Berrisford says:

    Further updates from our supplier have advised that the issue is located between InterXion and Brentford. Engineering resource is continuing to work on the issue and updates will follow as they become available.

  4. Conor Murphy says:

    Engineers are continuing to investigate the reported fault to isolate the issue further. We are keeping in constant contact with our suppliers and further updates will be posted as and when they become available.

  5. Ryan Berrisford says:

    Affected leased line have now been restored to normal service.

    We have been in contact with our supplier who have advised that although services have been restored, they will be considered at risk until the engineer has closed off the fault with his full findings.

    We are expecting full confirmation shortly and will update partners via this post.

  6. Conor Murphy says:

    Suppliers have confirmed service has been restored via a temporary work around to bypass a faulty fibre patch at the Poplar Metnet. Virgin Media will schedule future planned maintenance to fix the original fault.

    Services should no longer be considered at risk however we will update further once the planned maintenance to complete the repair has been scheduled.

  7. Ryan Berrisford says:

    Further to the incident yesterday, we have been in constant contact with our supplier to confirm the strategy for the planned maintenance.

    As the workaround in place no way diminishes the service we provide, it will remain in place for the foreseeable future whilst our supplier confirms the scheduled dates for the maintenance. The maintenance will then be notified to us via the normal change management channels so we can inform our customers.

    A full incident report has been requested from our suppliers which will be published as and when we receive it.

    We apologise the inconvenience caused during the course of yesterday afternoon.

  8. Scott Morgan says:

    After extensive correspondence with our supplier, developments within the Virgin Media network have deemed further fibre work is required between 2 different Virgin Media Transmode Muxes in the Poplar exchange. As the further work is critical to the service, a director level decision has been made to carry out emergency maintenance and this has been scheduled for tonight, out of business hours.

    The maintenance window is 18/02/2016 00:01 – 06:00. Virgin Media have advised the maximum amount of downtime is classed as 4 hours, although downtime for Entanet is expected to be much less.

    All customers with an ADSL backup service will failover and continue to work during this period.

    We apologise for the short notice and we will work with our suppliers to keep downtime to a minimum.

  9. Ryan Berrisford says:

    An incident report for this issue is now available here.