Archive for the ‘VoIP’ Category

Planned maintenance: VOIP media servers

Posted: Friday, March 11th, 2016 at 15:08 by David Derrick

Thursday 31 March 2016 10:00-15:00

During the above window we will be making changes to the media servers on the VOIP system. We will work on each server in turn, removing it from the pool and waiting for its calls to complete before performing the work. It will then be tested before returning to the pool and moving on to the next.

Please note that since the system is designed to cope with server failure we do not expect any customer impact from this. In the event of problems we will roll back to the previous state.

  1. Ben Bliss says:

    The maintenance was a success, but due to time constraints, further work is needed to roll the changes out across the rest of the media server cluster.

    This will be scheduled for a later date.

VoIP Disruption

Posted: Thursday, January 14th, 2016 at 09:39 by Ryan Berrisford

We are currently aware of an issue affecting in and outbound calls on our VoIP Enrich platform.

Engineers are investigating the issue and updates will follow as they become available.

  1. Ryan Berrisford says:

    Following engineering work, service has been restored to customers. Engineers however are continuing to investigate the root cause of the issues experienced.

    We apologise for the inconvenience caused.

  2. Adam Heath says:

    We have been aware that there were still some problems since the previous update, with making inbound and outbound calls on the Enrich platform. Our engineers believe they have now isolated the problem following further investigation and further actions have been taken to prevent this from re-occurring.

    We are continuing to monitor the situation, although currently services appear to be stable and functional.

  3. Adam Heath says:

    Our monitoring has alerted us to further reports of the outbound call issue still being experienced by some users. This is still being investigated following our previous actions not fully rectifying the scenario and further updates are to follow.

  4. Ryan Berrisford says:

    We are aware that a small portion of the customer base is still affected by the issue and engineers are continuing to work to rectify the problem. Further updates will follow shortly.

  5. Adam Heath says:

    Our engineers have made some further configuration changes to the primary VoIP platform, however to apply the same amendments to the Enrich platform, a brief reboot will occur at 23:00 this evening.

    We anticipate that this will rectify the problems experienced by some users of the services today and we are continuing to monitor the situation. Further updates will follow tomorrow morning.

  6. David Derrick says:

    The Enrich platform has been updated with the same fix as applied earlier to the primary platform.

  7. Adam Heath says:

    There does appear to be a continuation of problems on the VoIP platform this morning and our engineers are investigating currently. Our engineers advise that the configuration changes made yesterday evening do appear to have had a positive impact in terms of the number of successful calls now being made, however they are continuing to look into the cause of the ones that are failing.

    We would also like to remind any resellers or end users who may have had problems reaching the Support desk, to please ensure they are using our updated number. This is 0330 100 3551.

  8. Adam Heath says:

    Issues experienced earlier this morning came about due to a script on a VoIP server being modified incorrectly during a debugging process yesterday afternoon. Further investigation into the error allowed us to correct the script and service has been stable for the last couple of hours in which we have been monitoring.

    Further on from this an RFO will be released next week in order to clarify the sequence of events and provide a more detailed technical breakdown. We apologise for the inconvenience this has caused.

  9. Ryan Berrisford says:

    Due to complexity of this incident the RFO report is taking a little longer than expected to complete. We apologise for the delay and we anticipate this will be available by close of business tomorrow.

  10. Ryan Berrisford says:

    An incident report for this issue is now available here.

Emergency Work: VoIP Platform

Posted: Thursday, December 10th, 2015 at 17:09 by Steve Lalonde

Friday 11 December 2015 00:00-06:00 GMT

Following on from this weeks VoIP platform issues we will be replacing the primary Database server.

We expect the work to take approximately 2 hours during which VoIP platform will be unavailable.

The back out plan will be to revert to the existing server.

  1. Ming-Yu Hsieh says:

    we are still working on the database server.

  2. Ming-Yu Hsieh says:

    This work is now complete.

Incident: VoIP Disruption

Posted: Tuesday, December 8th, 2015 at 14:10 by Ryan Berrisford

We are currently aware of a problem affecting calls on the VoIP platform.

Engineers are investigating the issue and further updates will be posted as they come available.

  1. Ryan Berrisford says:

    Service was restored to the affected customers shortly after the disruption.

    Post investigations have not identified any issues on the Entanet platform and therefore upstream providers have been engaged to help isolate the issue further.

    Updates will follow, as we receive them.

  2. Ryan Berrisford says:

    Our upstream providers have updated us to advise they did have problems with their database servers at the reported time. They have also advised they are carrying out a review of the issue for future improvements.

    We apologise for the inconvenience caused at the time.

Emergency maintenance: VOIP systems

Posted: Tuesday, December 8th, 2015 at 12:39 by David Derrick

Following on from Saturday’s disruption we have discovered a problem with the VOIP databases which are affecting calls for some customers. Attempts to gracefully recover have failed so we will have to briefly stop and restart the database. This may cause calls to drop. Downtime will be less than 60 seconds and is expected to happen at 12:45.

  1. David Derrick says:

    The database has restarted successfully.

  2. Adam Heath says:

    There seems to have been a continuation of the database issue we previously identified which may be causing issues with inbound calls and amendments to call forwards. We are investigating on our side and will provide further information when it becomes available.

  3. Adam Heath says:

    We have resolved the immediate database issue and services now appear to be working again. Further remedial work is required and will be scheduled in due course.

VoIP Disruption

Posted: Saturday, December 5th, 2015 at 16:08 by Mark Yardley

We are currently aware of an issue affecting some users of our VoIP platform. Engineers are investigating and a further update will be provided when available.

  1. Mark Yardley says:

    Engineers have now rectified the issue and can see calls are establishing normally. We will continue to monitor to ensure there are no further problems.

Planned maintenance:

Posted: Friday, December 4th, 2015 at 16:51 by David Derrick

Sunday 13 December 2015 01:00-02:00 GMT

During the above window we will be replacing the hardware for as part of a server refresh cycle. Downtime is expected to be less than five minutes but may be longer if we encounter problems. The back out plan will be to revert to the existing server.

  1. David Derrick says:

    This work has been postponed. We will post a new announcement when it has been rescheduled.

Incident: VoIP Disruption

Posted: Tuesday, November 17th, 2015 at 15:01 by Ben Bliss

We are currently aware of an issue which is affecting outbound calls across our VoIP platform.
Initial investigation shows this to be down to one of our upstream providers. We are now routing calls to alternate providers while this is being investigated.
Further information will be provided as and when it becomes available.

  1. Ben Bliss says:

    We are currently awaiting further information from our affected upstream provider.

    While currently not proven, indications are that the problems being experienced are as a direct result of the power issues currently being experienced within Telecity.

    We are continuing to monitor the situation, and will provide a further update as and when additional information is available.

  2. Ben Bliss says:

    Further to the last post, connectivity to the affected upstream provider was re-established shortly before 5pm.

    However, testing has indicated there are still issues with their systems, so we have not re-established call routing to them at this time.

    We will continue to monitor the situation, and as soon as we receive information to advise the issues within their network are resolved, we will provide a further update.

  3. Ben Bliss says:

    Following further investigation and the resolution of various issues within the upstream suppliers network caused by the earlier power failure in Telecity, we are satisfied that we can now start sending calls via the affected carrier.

    As such, we have recommenced sending calls to the affected provider, and all appears to be working correctly.

    We will continue to monitor the situation and provide further updates if necessary.

Incident: Incoming VOIP calls

Posted: Monday, March 23rd, 2015 at 10:15 by David Derrick

One of our suppliers has advised that they are currently experiencing an outage on their incoming call handling systems. Some customers may experience problems receiving calls from external phone networks.

  1. David Derrick says:

    Our supplier reports their incident is resolved and initial tests indicate that inbound calls are working correctly again.

Interxion Power – VoIP Impact

Posted: Monday, August 18th, 2014 at 09:51 by Jon Farmer

Further to the post dated 14/8/2014 this is to provide further information on how our VoIP platform will be impacted. The work on the VoIP Platform is scheduled to take place between 00:00 and 05:00 on 2/9/2014.

VoIP Express and VoIP EasyAccess should experience no downtime.

VoIP Enrich customers will receive a loss of both signalling and media capability for a short period of time. It is estimated that the loss of service will not exceed 15 minutes during the above window. If at the close of the window or a comment to this post that the work is complete you find that some VoIP Enrich phones are not working then as a first step reboot the phone before contacting our support team.

  1. Jon Farmer says:

    This work is now completed.