Archive for September, 2006

Network Issues

Posted: Monday, September 18th, 2006 at 17:57 by Neil Watson

We are currently experiencing problems on our network. The systems team is investigating and will update as soon as possible. We apologise for any inconvenience.

  1. Neil Watson says:

    This issue has now cleared and our network is now fully available. We will continue to investigate the cause of the issue and will closely monitor the situation.

Move of some Telford Services

Posted: Monday, September 18th, 2006 at 12:55 by Martyn Haynes

Date: 22nd September 2006, 06:30 – 08:00hrs

We will be moving a number of services from the old Telford.core router to a new Telford4 router which will enable us to decommission an obsolete switch.

Services affected will be Windows hosted sites, some ecommerce and co-location customers connected through colofirewalls 1-6 (except 5), viperfirewall and VDS, some local gateways and VPN Boxes.

Downtime should be less than 10 minutes for each service as routing converges.

Please note: telephone support may be unavailable for a 10 minute period while this work is completed.

  1. David Derrick says:

    This work has been completed, all services appear to be working correctly. Please report ant problems to support via the normal channels.

ADSL maintenance update

Posted: Friday, September 15th, 2006 at 09:39 by Neil Watson

We are currently experiencing an increased call volume regarding ADSL issues, which may be linked to last night’s maintenance. The majority of issues are due to authentication problems. We would advise all our customers to follow our previous advice: if you are unable to connect, please power down your equipment for a minimum of 15 minutes and retry. We currently have engineers on site in Telehouse who are looking to rectify any problems. Updates will be made as soon as more information becomes available.

  1. Neil Watson says:

    Our engineers have now confirmed that all problems have now been resolved. Call volumes in support are now at a normal level. We apologise to anyone who experienced issues this morning.

UPS replacement

Posted: Wednesday, September 13th, 2006 at 12:40 by Neil Watson

Following on from the previous power issues, we have identified some UPS equipment that was failing to perform as expected. As such we will be replacing the UPS equipment between 17.30hrs and 18.30hrs on Thursday 14th September 2006. This change will affect some co-located hosting customers who will be advised individually. We will also be exchanging the UPS that covers the following servers: pop, dns-cache1, and dial-gateway; a small period of outage will be required for the swap over. Please accept out apologies for any down time caused, but wish to assure you that the work taking place is essential.

  1. David Derrick says:

    Work is now completed. The colocated customers were returned to service on schedule but the Entanet servers were down for longer than planned due to difficulty removing the faulty UPS. Services appear to be back up and running again now.

Wolverhampton 1 Router Failed

Posted: Saturday, September 9th, 2006 at 20:09 by Ming-Yu Hsieh
wolverhampton 1 router has failed, engineers are currently looking into the problem.
  1. Steve Lalonde says:

    The router crashed and was stuck at a rommon prompt.

    Our out of band modem failed also so we had to visit the site to fix the problem.
    After reloading the router on site it crashed again inside 30 seconds with an out of memory error.

    When the router rebooted again it seemed to be stable but it was sending bad information into our IGP causing traffic to be sent to the wrong Destination.
    We have now down graded the IOS on the router to a known stable version and all is now operating as expected.
    We have also fixed the OOB modem so it should be posible to fix remotely if there is a further problem.

    Apologies for any inconvenience caused.

  2. Neil Watson says:

    Our engineers have now provided an Incident Report for this outage.

Planned ADSL maintenance

Posted: Friday, September 8th, 2006 at 16:30 by Neil Watson

Friday 15th September 2006, 01.00hrs – -5.00hrs

As part of our maintenance programme, we will be undertaking some work on the routers that terminate our ADSL service. This work involves the removal of and, with the current services from this equipment being transferred to a higher specification device, this will allow for future growth potential. At the same time it will also be necessary to reboot each LNS which will result in an outage for all DSL customers. Please be advised that during the maintenance window you may lose connection at several points. If, once maintenance is complete, you are unable to connect, please power down your equipment for a minimum of 15 minutes and retry.

Whilst completing this work we will also be renaming the LNS equipment. Please do not be alarmed by this name change as it is purely cosmetic and will have no effect on the end user experience.

  1. Steve Lalonde says:

    This maintenance slot is now complete. please refer any issues to the usual support channels.

Emergency Network Maintenance

Posted: Friday, September 8th, 2006 at 15:18 by Neil Watson

Thursday 14th September 2006 01.00hrs – 05.00hrs

Our network management team has become aware of a potential issue regarding core routers in our London ring. Whilst the issue poses no immediate threat to our network, it is essential that we apply an emergency software release to the routers to allow us to prevent any possible outage. This work will be carried out during the period specified above and will require a reboot of each core router in the network thereby causing outages at key points in our network. We will have engineers onsite to ensure that the service is fully restored as soon as possible.

During this work we will also be taking the opportunity to make improvements to our network configuration. This work will involve moving from a switched network model to a routed model, as a consequence of which, customers may perceive additional routing equipment showing in any trace routes performed. Please do no be alarmed by the visibility of additional equipment, as all traffic previously passed via this kit. The change will provide better diagnostic tools for any network issues and will ensure that our network capabilities are being utilised in the most efficient manner.

Please accept our apologies for any inconvenience that this may cause, we will as always endeavour to keep any disruption to a minimum

  1. Steve Lalonde says:

    This work is now completed

    any problems should be directed via the usual support channels

Email Delays

Posted: Tuesday, September 5th, 2006 at 14:30 by Neil Watson

Our mail systems are currently experiencing a heavy load and are subject to extended mail queues. Please be advised that mail sent through our systems may be delayed. Please accept our apologies and be assured that we are currently working to correct the situation.

  1. Neil Watson says:

    Our engineers have amended the configuration and mail will now flow normally, however delays of upto 30 minutes are expected whilst the backlog clears.

BT Planned Engineering Works

Posted: Tuesday, September 5th, 2006 at 13:24 by Neil Watson

BT has announced a series of engineering work that will affect around 30 DSLAM’s per night until October 2006. The schedule is split into regions of the UK as shown below.

  • Central Midlands 4th September and 5th September
  • South Midlands 6th September
  • West Midlands 7th September
  • East Midlands 8th to 13th September
  • North Scotland 14th September
  • West Scotland 15th September to 18th September
  • East Scotland 19th September
  • Wales (North) 20th September
  • Northern Ireland 21st September
  • East Anglia 22nd September to 25th September
  • South Downs 26th September
  • Southampton 27th September
  • Severn Side 28th September
  • South Wales 29th September
  • North London 2nd October
  • South London 3rd October
  • Central London 4th October
  • North Downs 5th October
  • Thamesway 6th October
  • Mid Yorkshire 9th October
  • South Lancashire 10th October
  • Lancashire 11th October
  • Liverpool 12th October
  • Manchester 12th October
  • North East 13th October
  • Cornwall 13th October

The work is part of network upgrades and will occur between midnight and 6am, with people possibly experiencing a downtime of 30 minutes on the night of the work. Therefore if your area is scheduled for work one evening and your connection drops do not panic. BT recommends that if after the outage that you cannot reconnect, that you switch your ADSL hardware off for a minute and then switch it back on. If you continue to have problems please feel free to call our support desk.

Telephone system problems

Posted: Tuesday, September 5th, 2006 at 09:27 by Neil Watson

Unfortunately we are currently experiencing some problems with our call system and are, at present, unable to take support calls. We are working on rectifying the situation as a matter of urgency and will ensure that normal service is resumed as soon as possible. As an alternative method of contact please mail, or alternatively dial 0870 770 9666 and select options as required.

  1. Neil Watson says:

    All phone system problems have now been resolved and support can be reached onthe standard number. Many thanks for you patience during this time.