Archive for the ‘VoIP’ Category

Incident: VoIP Disruption

Posted: Thursday, July 13th, 2017 at 10:46 by Richard Partridge

Engineers are currently investigating an incident that is causing a loss of service on our VoIP platform. Further information will be provided as and when it becomes available.

  1. Richard Partridge says:

    Service has now been restored. We apologise for any inconvenience.

VoIP disruption

Posted: Wednesday, April 12th, 2017 at 15:39 by Adam Heath

We are aware that a number of users are having issues placing and receiving calls on the VoIP Enrich platform.  Engineers are currently working to restore service and we will provide further information when it becomes available.

  1. Adam Heath says:

    Service has been restored following our engineers restarting a process on one of the VoIP servers.

Incident: VoIP Disruption

Posted: Wednesday, February 1st, 2017 at 09:37 by Mark Yardley

We are currently aware of an issue affecting some VoIP Enrich customers. Engineers are currently investigating and further updates will be provided as and when they become available.

  1. Richard Partridge says:

    The issue has been rectified and normal service has now resumed. Engineers will continue to monitor the platform closely.

Incident: VoIP Disruption

Posted: Friday, January 6th, 2017 at 10:18 by Richard Partridge

We are currently aware of an issue affecting outbound calls. Engineers are currently investigating and further updates will be provided as and when they become available.

  1. Richard Partridge says:

    The issue has now been rectified and normal service has now resumed. Engineers will continue to monitor the platform closely.

Incident: VoIP Disruption

Posted: Wednesday, November 2nd, 2016 at 15:45 by Richard Partridge

One of our upstream carriers is experiencing difficulties which is having an adverse affect on outbound call setups. Engineers are working to bypass this problem and direct outbound calls towards other carriers.

Further updates will be provided as and when they become available.

  1. Richard Partridge says:

    The affected carrier has now been bypassed.

  2. Richard Partridge says:

    The upstream carrier has acknowledged the problem and is working on a resolution. We will continue to re-route calls around this problem until a solution has been put in place.

  3. David Derrick says:

    The affected carrier resolved their problem and has now been brought back into service.

VoIP Enrich Incident

Posted: Monday, April 18th, 2016 at 13:39 by Adam Heath

We are aware of a potential problem on the VoIP Enrich platform.  Engineers are currently investigating and we will post further information as it becomes available.

  1. Adam Heath says:

    Our NOC engineers have confirmed that one of the media servers had crashed. They restarted the process and service seems to have been restored.

    Engineers will be investigating the crash log in an attempt to ascertain the cause.

Planned maintenance: VoIP media servers

Posted: Thursday, March 31st, 2016 at 15:31 by Ben Bliss

Thursday 7 April 2016 10:00-15:00

During the above window we will be making additional changes to the media servers on the VoIP system. This is a continuation of the work carried out on the 31st March 2016.

We will work on each server in turn, removing it from the pool and waiting for its calls to complete before performing the work. It will then be tested before returning to the pool and moving on to the next.

Please note that since the system is designed to cope with server failure we do not expect any customer impact from this. In the event of problems we will roll back to the previous state.

  1. Ben Bliss says:

    The work is now complete. Additional maintenance will be needed at a later date, which will be announced shortly.

Planned maintenance: VOIP media servers

Posted: Friday, March 11th, 2016 at 15:08 by David Derrick

Thursday 31 March 2016 10:00-15:00

During the above window we will be making changes to the media servers on the VOIP system. We will work on each server in turn, removing it from the pool and waiting for its calls to complete before performing the work. It will then be tested before returning to the pool and moving on to the next.

Please note that since the system is designed to cope with server failure we do not expect any customer impact from this. In the event of problems we will roll back to the previous state.

  1. Ben Bliss says:

    The maintenance was a success, but due to time constraints, further work is needed to roll the changes out across the rest of the media server cluster.

    This will be scheduled for a later date.

VoIP Disruption

Posted: Thursday, January 14th, 2016 at 09:39 by Ryan Berrisford

We are currently aware of an issue affecting in and outbound calls on our VoIP Enrich platform.

Engineers are investigating the issue and updates will follow as they become available.

  1. Ryan Berrisford says:

    Following engineering work, service has been restored to customers. Engineers however are continuing to investigate the root cause of the issues experienced.

    We apologise for the inconvenience caused.

  2. Adam Heath says:

    We have been aware that there were still some problems since the previous update, with making inbound and outbound calls on the Enrich platform. Our engineers believe they have now isolated the problem following further investigation and further actions have been taken to prevent this from re-occurring.

    We are continuing to monitor the situation, although currently services appear to be stable and functional.

  3. Adam Heath says:

    Our monitoring has alerted us to further reports of the outbound call issue still being experienced by some users. This is still being investigated following our previous actions not fully rectifying the scenario and further updates are to follow.

  4. Ryan Berrisford says:

    We are aware that a small portion of the customer base is still affected by the issue and engineers are continuing to work to rectify the problem. Further updates will follow shortly.

  5. Adam Heath says:

    Our engineers have made some further configuration changes to the primary VoIP platform, however to apply the same amendments to the Enrich platform, a brief reboot will occur at 23:00 this evening.

    We anticipate that this will rectify the problems experienced by some users of the services today and we are continuing to monitor the situation. Further updates will follow tomorrow morning.

  6. David Derrick says:

    The Enrich platform has been updated with the same fix as applied earlier to the primary platform.

  7. Adam Heath says:

    There does appear to be a continuation of problems on the VoIP platform this morning and our engineers are investigating currently. Our engineers advise that the configuration changes made yesterday evening do appear to have had a positive impact in terms of the number of successful calls now being made, however they are continuing to look into the cause of the ones that are failing.

    We would also like to remind any resellers or end users who may have had problems reaching the Support desk, to please ensure they are using our updated number. This is 0330 100 3551.

  8. Adam Heath says:

    Issues experienced earlier this morning came about due to a script on a VoIP server being modified incorrectly during a debugging process yesterday afternoon. Further investigation into the error allowed us to correct the script and service has been stable for the last couple of hours in which we have been monitoring.

    Further on from this an RFO will be released next week in order to clarify the sequence of events and provide a more detailed technical breakdown. We apologise for the inconvenience this has caused.

  9. Ryan Berrisford says:

    Due to complexity of this incident the RFO report is taking a little longer than expected to complete. We apologise for the delay and we anticipate this will be available by close of business tomorrow.

  10. Ryan Berrisford says:

    An incident report for this issue is now available here.

Emergency Work: VoIP Platform

Posted: Thursday, December 10th, 2015 at 17:09 by Steve Lalonde

Friday 11 December 2015 00:00-06:00 GMT

Following on from this weeks VoIP platform issues we will be replacing the primary Database server.

We expect the work to take approximately 2 hours during which VoIP platform will be unavailable.

The back out plan will be to revert to the existing server.

  1. Ming-Yu Hsieh says:

    we are still working on the database server.

  2. Ming-Yu Hsieh says:

    This work is now complete.