Archive for November, 2006

Reboot of Remus Windows Server

Posted: Thursday, November 30th, 2006 at 15:40 by Adam Heath

Thursday 30th November 2006: 17.00hrs – 17.05hrs

In order to update our Windows server Remus, it is necessary for us to reboot the box. These updates should provide essential security fixes and resolutions for some minor issues with database issues. We have scheduled the reboot for 5pm today (30th November 2006). All sites hosted on this server will be affected for a maximum of 5 minutes.

Central Pipe Status

Posted: Wednesday, November 29th, 2006 at 01:23 by Steve Lalonde

A few users have asked what the colours mean on the status widget, so here is the simple explanation.

Green = less than 600Mb

Amber = less than 622Mb (or almost full)

Red = 622Mb or slightly above (we can get approx 630Mb before there is any real packet loss)

 Black = Packet loss expected due to BT’s router policing the central. :(

Nice and simple, the data comes straight out of our nms system.

I also added a Not live status so you can see the new centrals coming soon.

  1. Steve Lalonde says:

    Just a side note on the status, if you see the pipe you are on go black it would be usefull if you could reconnect ( don’t bother if both are black )

LNS & Central Pipe Configuration

Posted: Tuesday, November 28th, 2006 at 13:45 by Neil Watson

We have made some non intrusive changes to how our Central Pipes and LNS servers relate to each other, with them now having a direct correlation between the central you are on and the LNS server. This work will allow us to better try to balance our central pipes, whilst we wait for the third pipe to be installed. In order to see the changes it will be necessary to reconnect to the service, at which point it will be apparent which pipe you are connected to. If some users are experiencing high latency or packet loss, please disconnect and reconnect to the service until connected to the alternative pipe. We hope that this change will have a positive effect on the service we provide to you.

  1. Steve Lalonde says:

    OK here is the list of lns on each central.

    birch, chestnut, dogwood are on global switch central no1.

    ironwood, jujuba, katsura are on telehouse central no1.

    elm, fir, larch will be on telehouse central no2.

    all the other lns,lts boxes are taking connections from both centrals.

    also please note that if you have not reconnected in the 12 hours before this post you could be on any central.

3rd 622 Central Update

Posted: Tuesday, November 28th, 2006 at 09:38 by Steve Lalonde

Progress at last, BT have just arrived in Telehouse to install the fiber for the new central.

I am going to keep this item updated with comments so make sure you are subscribed to the comments RSS feed to stay informed.

  1. Steve Lalonde says:

    OK the fiber is in the rack and they are now splicing the tails on, after this they will need to schedule an end to end test.

  2. Steve Lalonde says:

    BT have finished the fiber install and will be back later this week to do the end to end test.

  3. I spoke to the Job Control this morning and they have confirmed that the fibre has been delivered and tested end to end. BOU alarm test will be next week then all we need is the routers to be devlivered and tested.

  4. Steve Lalonde says:

    BT will be in telehouse installing the NTEs for the new 622 between monday 11th and wednesday 13th.

  5. Steve Lalonde says:

    Good news we have BT engineers on site in telehouse right now. The 2 * 1 gig lines to the exchange are now up, all tests passed. Next on the list is the NTEs due on friday. :)

  6. Steve Lalonde says:

    BT and Entanet engineers will be in telehouse on Monday 18th for instalation and commisioning of the new 622.

  7. Steve Lalonde says:

    BT have now finished the hardware install, we will now begin testing the new pipe (well after we have had lunch :) )

  8. Steve Lalonde says:

    OK looking good so far we have moved 25 users on to the new pipe and all is working as expected. We will continue to monitor the pipe and will probably move some more users across later today, if all is still correct.

    This cant be done out of hours as BT BOU only do this kind of work 9-5 :(

  9. Steve Lalonde says:

    We are adding another 1k users in to the new pipe, this will be the last change today.

    BT had a small config error on the LTS boxes their end (missing route to some of the newer BRAS boxes) but thats now resolved.

    NOTE: Please do not keep trying to reconnect to get on the new pipe, We will be moving all the rest of the users tomorrow.

  10. Steve Lalonde says:

    The new pipe is now part of the main pool, all reconnects have a chance of getting on to the new pipe.

    Dont waste your time reconnecting to get on the new pipe let it happen naturally or you will all be overloading the new pipe while the old ones have spare capacity. :P

xDSL Stability

Posted: Sunday, November 26th, 2006 at 11:27 by Steve Lalonde

We are currently seeing some problems with our LNS pool, this is causing users to be randomly disconnected. We are working on the problem and will update asap. This problem is not affecting L2TP customers.

  1. Steve Lalonde says:

    Interestingly this has had the effect of balancing the traffic on both 622 centrals. (bet that does not last :( )

BT peering work

Posted: Wednesday, November 22nd, 2006 at 13:16 by Neil Watson

BT have advised that they will today be bringing the second Gb peering link with Entanet online. This will significantly increase the amount of peering that we are able to perform with them and will have positive effects for all customer who pass traffic through to the BT network. It is anticipated that this work will be completed by 16.00 today. No downtime or negative impact is anticipated. We will confirm once complete.

  1. Ming-Yu Hsieh says:

    This job has be completed and stable for last 2 hours.

High Latency on xDSL Services

Posted: Tuesday, November 21st, 2006 at 14:57 by Adam Heath

We are currently experiencing issues with high levels of latency for some xDSL users. We are currently investigating the cause of this, please accept our apologies for any inconvenience caused. Further information will be posted as and when it becomes available.

Reboot of Remus web server

Posted: Friday, November 17th, 2006 at 13:56 by Neil Watson

We are currently experiencing some issues with elements of our Windows based hosting which requires the server Remus to be restarted. This work will be carried out at approximately 14.00 today, which will cause a small period of downtime for some Windows based hosting users. This is expected to last approximately 5 minutes. We apologise for the inconvenience caused and wish to reassure you that we are taking all necessary steps to ensure continuity of service.

Forced disconnection of DSL users

Posted: Thursday, November 16th, 2006 at 11:28 by Neil Watson

In order to try to alleviate some of the issues currently being seen with latency on our DSL platform, it is necessary to disconnect all DSL users. This will be carried out as 10.30 today. We apologise for the short notice and any inconvenience this will cause.

Users should automatically reconnect, however, if problems are experienced we recommend  powering off the routing equipment for a period of approximately 15 minutes before retrying.

Emergency BT Maintenance

Posted: Wednesday, November 15th, 2006 at 17:31 by Neil Watson

Thursday 16th November 2006: 01.00 – 06.00hrs

BT have advised us of the following emergency work:


BT Wholesale would like to advise customers that Planned Engineering Work will be conducted on BT’s Broadband Network on 16th November 2006 from 01:00hrs, through to 06:00hrs.

During this period there should be no loss of service.

The reason for this work is a Software Upgrade

BT Wholesale will endeavour to complete the work promptly.