Archive for the ‘Incidents’ Category

Incident: Vodafone Hub Outage

Posted: Monday, May 20th, 2019 at 03:33 by William George

Following on from a Vodafone maintenance our monitoring has alerted us to a number of Vodafone circuits that are currently out of service. Our initial diagnostics indicate that the circuits are related to a single handoff point between ourselves and Vodafone. Our engineers are investigating and are engaged with the supplier and further updates will be provided via this site.

We apologise to any customers that are impacted and want to reassure you that we are working to return full service as soon as possible.

  1. William George says:

    Vodafone have confirmed they see the issue and are in the process of arranging a resource to go to site to address it. Further updates will be provided as they are available.

  2. David Brewis says:

    Engineers from the supplier are already on site investigating cause for disruption to service and will update with their findings. As this progresses updates will be posted for your notice. Please expect the next update by 10:20.

  3. David Brewis says:

    Investigations from field engineers have identified an equipment fault, work has begun with hardware vendors to replace and resolve service as soon as possible.

    We apologise for the ongoing disruption whilst this incident progresses.

    Please expect the next update by 11:20

  4. David Brewis says:

    Following contact by Vodafone to their hardware vendor, a replacement component has been shipped to site with the expected arrival time of 11:00. We are awaiting confirmation of this delivery to begin works for replacement to restore service.

    Please expect the next update by 12:20.

  5. David Brewis says:

    The replacement unit has been installed and initial troubleshooting by Vodafone demonstrates the incident has been resolved.

    We are completing our own triage before confirming the engineer can leave site.

    Please expect a further update by 12:30.

  6. David Brewis says:

    We can confirm the handoff to Vodafone has returned to working service and the engineer stood down.

    If your service remains offline please contact the helpdesk following a reboot of your devices.

    We apologise for any disruption caused by this incident and will continue discussions with Vodafone for root cause analysis.

Incident: Virgin outage in London

Posted: Saturday, May 4th, 2019 at 19:14 by David Wheeler

We are aware of an issue affecting connectivity on Virgin Circuits in the London area. This has been raised to Virgin and their engineers are currently investigating and on site. Updates to follow shortly when we have the information from the site engineers is received.

  1. David Wheeler says:

    All circuits have now re-established, but the circuits may be subject to further disruption whilst full fix is put in place. We will be getting a further update around 21:00 this evening.
    Please consider the circuits at risk until we have had confirmation of permanent resolution from Virgin.

  2. David Wheeler says:

    All services are now restored and there is no further outage anticipated.

Incident: Voip Platform

Posted: Monday, April 29th, 2019 at 09:43 by Matthew Davies

We are aware of an issue that was affecting some users on our VOIP platform. We believe the issue has now been resolved and request any users affected to now retest the service.

Incident: Colt circuits

Posted: Sunday, April 21st, 2019 at 05:42 by William George

We are aware of an issue affecting connectivity on some Colt Circuits. This has been raised to Colt and their engineers are currently investigating, updates to follow when available.

  1. John Benge says:

    Colt engineers have isolated the issue to circuits supplied through Telecity/Equinix and are currently continuing their investigations.

    Further updates will be provided as soon they are available.

  2. John Benge says:

    Colt engineers have located and rectified a cabling issue affecting circuits supplied through Telecity/Equinix. Services have been restored and will continue to be monitored for the next 24hrs.

Incident: Talk Talk Circuits

Posted: Tuesday, April 16th, 2019 at 10:16 by David Brewis

Our monitoring has alerted us to a number of Talk Talk circuits that are currently out of service. Our initial diagnostics indicate that the circuits are related to a single handoff point between Talk Talk and us. Our engineers are investigating and are engaged with the supplier. Further updates will be provided via this site. We apologise to any customers that are impacted and want to reassure you that we are working to return full service as soon as possible.

  1. david labouchardiere says:

    Our supplier advises they are not seeing any light levels from our hardware at the affected POP. Our NOC team have dispatched an engineer who is en route to the POP for further testing.

    Further updates will be provided when available.

  2. Jamie Smith says:

    An Engineer is currently on site Investigating the root cause of this outage, further updates will be provided as and when they are available

  3. david labouchardiere says:

    Service has been restored following repair by the attending enigneer. Customers are advised to reboot their equipment if they are still experiencing issues prior to contacting us via normal channels.
    We apologise for the inconvenience caused by this interruption.

Incident: Sheffield area

Posted: Wednesday, April 10th, 2019 at 23:24 by David Wheeler

We are currently aware of an issue within the Sheffield area affecting connectivity. Engineers are currently investigating, updates to follow as soon as available.

  1. David Wheeler says:

    All services have been restored.

    If any further issues are experienced please contact our Support Desk.

    We apologise for the disruption caused by this incident.

Incident: VoIP Enrich Platform

Posted: Tuesday, April 2nd, 2019 at 12:55 by Adam Heath

We are aware of an issue affecting inbound and outbound calls, in addition to registration on the VoIP Enrich platform.  We are currently investigating and will update shortly.

  1. David Brewis says:

    Following investigations service is now restored, please contact the desk if you continue to experience issues. We apologise for any inconvenience during this incident.

Incident: Coventry & Gloucester PoPs

Posted: Sunday, March 31st, 2019 at 13:09 by Gurinder Bansal

We are currently aware of an issue affecting our Coventry & Gloucester PoPs. Engineers are currently investigating, updates to follow as soon as available.

  1. Gurinder Bansal says:

    This incident was resolved on 31/3/19 at 4.30pm. We have continued to monitor with no further issues. We are working with suppliers to prevent further occurrences.

Incident: Wolverhampton PoPs

Posted: Friday, March 29th, 2019 at 10:19 by Jordan Coleman

We are currently aware of an issue affecting our Wolverhampton PoPs. Engineers are currently investigating, updates to follow as soon as available.

  1. Jordan Coleman says:

    Investigations have highlighted that there is a confirmed issue on the local power grid. This is affecting several post codes, which also cover the locations of the affected PoPs.

    Further updates will be provided when available.

  2. Jamie Smith says:

    Western power have confirmed the believe the Issue on the power grid to now be resolved.

    We apologise for any inconvenience this may have caused

Incident: Legacy VoIP

Posted: Sunday, March 10th, 2019 at 19:45 by Adam Heath
Our supplier identified an attempt at toll fraud and temporarily disabled outbound calls on Enrich/Express/Easy Access as a mitigation this evening. End users could of course still call 999 and inbound and on-net calls remained available.
We are working with our supplier to restore outbound calling.
Our Hosted and SIP Trunk products were not impacted.
  1. Adam Heath says:

    Service was restored at 22:46 on 10/03/2019.