Archive for January 14th, 2016

VoIP Disruption

Posted: Thursday, January 14th, 2016 at 09:39 by Ryan Berrisford

We are currently aware of an issue affecting in and outbound calls on our VoIP Enrich platform.

Engineers are investigating the issue and updates will follow as they become available.

  1. Ryan Berrisford says:

    Following engineering work, service has been restored to customers. Engineers however are continuing to investigate the root cause of the issues experienced.

    We apologise for the inconvenience caused.

  2. Adam Heath says:

    We have been aware that there were still some problems since the previous update, with making inbound and outbound calls on the Enrich platform. Our engineers believe they have now isolated the problem following further investigation and further actions have been taken to prevent this from re-occurring.

    We are continuing to monitor the situation, although currently services appear to be stable and functional.

  3. Adam Heath says:

    Our monitoring has alerted us to further reports of the outbound call issue still being experienced by some users. This is still being investigated following our previous actions not fully rectifying the scenario and further updates are to follow.

  4. Ryan Berrisford says:

    We are aware that a small portion of the customer base is still affected by the issue and engineers are continuing to work to rectify the problem. Further updates will follow shortly.

  5. Adam Heath says:

    Our engineers have made some further configuration changes to the primary VoIP platform, however to apply the same amendments to the Enrich platform, a brief reboot will occur at 23:00 this evening.

    We anticipate that this will rectify the problems experienced by some users of the services today and we are continuing to monitor the situation. Further updates will follow tomorrow morning.

  6. David Derrick says:

    The Enrich platform has been updated with the same fix as applied earlier to the primary platform.

  7. Adam Heath says:

    There does appear to be a continuation of problems on the VoIP platform this morning and our engineers are investigating currently. Our engineers advise that the configuration changes made yesterday evening do appear to have had a positive impact in terms of the number of successful calls now being made, however they are continuing to look into the cause of the ones that are failing.

    We would also like to remind any resellers or end users who may have had problems reaching the Support desk, to please ensure they are using our updated number. This is 0330 100 3551.

  8. Adam Heath says:

    Issues experienced earlier this morning came about due to a script on a VoIP server being modified incorrectly during a debugging process yesterday afternoon. Further investigation into the error allowed us to correct the script and service has been stable for the last couple of hours in which we have been monitoring.

    Further on from this an RFO will be released next week in order to clarify the sequence of events and provide a more detailed technical breakdown. We apologise for the inconvenience this has caused.

  9. Ryan Berrisford says:

    Due to complexity of this incident the RFO report is taking a little longer than expected to complete. We apologise for the delay and we anticipate this will be available by close of business tomorrow.

  10. Ryan Berrisford says:

    An incident report for this issue is now available here.