Archive for the ‘Emergency Work’ Category

« Older Entries

Planned Work: Telehouse DSL Platform

Posted: Tuesday, May 28th, 2024 at 15:05 by Jamie Smith

Thursday 13/06/2024 00:00 – 06:00

During the above window we will be performing a service impacting maintenance on the Telehouse DSL core.

At the start of the maintenance window, sessions connected to the Telehouse side of the platform will be forcibly disconnected and will allowed to reconnect to the Interxion infrastructure for the duration of the work. Once the rack migration has been completed, sessions will be rebalanced across the platform by performing additional disconnections from the Interxion side.

For the avoidance of doubt, all DSL services connected to the Telehouse side of the platform at the start of the window will experience a brief outage while sessions are torn down and reconnect. All services connected to the Interxion side of the platform may experience a brief outage towards the end of the window when rebalancing occurs.

We apologise for any inconvenience this may cause.

Posted in 21cn Updates, Emergency Work, EWCS | No Comments »

Emergency Work: lon4-rtr-1

Posted: Monday, January 29th, 2024 at 13:27 by Jamie Smith

Wednesday 31/01/2024 22:00 to Thursday 01/02/2024 06:00

During the above window, we will carry out emergency work to apply a critical software patch to this core device at Telehouse East. All directly connected services will experience an outage of up to 30 mins in total, however the full duration of the window should be accounted for. Any traffic that would usually be routed through this node will take an alternative path for the duration of the work.

We apologise in advance for any inconvenience caused.


Posted in Emergency Work | No Comments »

Emergency Power Work: Interxion Datacenter

Posted: Thursday, November 9th, 2023 at 16:19 by Jamie Smith

Wednesday 8th November 23:59 – Thursday 9th November 05:00

Following on from the two Major Incidents that occured over the weekend, we will be carrying out emergency work to our power Infrastructure within the affected rack.

There will be an impact to services within this location while this work occurs. Efforts have been carried out to minimze disruption to services and devices within the affected rack ensuring that work is only carried out on one PDU at a time.

We apologise for the inconnvience this may cause.

Posted in Emergency Work | No Comments »

Emergency Maintenance: man191-rtr-1

Posted: Friday, July 14th, 2023 at 18:49 by Jamie Smith

Friday 14th July 2023 19:00 – 20:00

During the above window, we will be carrying out a reboot of our core router in Manchester. This is required to address a memory issue being observed on the device.

All services currently terminated in this location will be subject to downtime. Services Include Leased lines, Wholesale handoff, Transit, EWCS & Psuedowires.

All other traffic that would normally route through this location will take alternative paths.

Posted in Emergency Work | 1 Comment »

  1. Jamie Smith says:
    July 14, 2023 at 20:42

    This is now starting

Emergency Joints Works Leeds Area

Posted: Sunday, July 2nd, 2023 at 16:55 by Dan Gluch

Saturday 1st July 00:01am – Tuesday 4th July 23:59:59pm

CityFibre are aware that some customers may be experiencing an unexpected service outage in the Leeds area. Please be aware that there are currently significant planned works ongoing in the area (Reference CHG0078467) between 00:01 on Saturday the 1st of July and 23:59 on the Tuesday the 4th of July. These works are required to address at risk infrastructure to reduce the likelihood of future unexpected service outages and to provide additional capacity.

We have endeavoured to issue advance notice to all customers. In the event you experience a service outage in the area between these times, please check change notifications prior to contacting our Technical Service Centre. Should you not have been notified and are experiencing an unexpected service outage, we apologise for this and ask that you notify our TSC of your service details in order to determine whether related to the works or a coincidental fault.

Current status as of 16:00 02/07/2023: 

Our field Teams have completed splicing activity in two of the three locations. Works are ongoing in the last location and additional teams are attending field splitters for testing with remote support from our TSC. It is not possible to provide specific timings or estimated times of customer restoration at this time however we anticipate that the majority of services will be restored by lunchtime Monday 03/07/2023. We ask that any customers who remain down at that time complete reboots of their equipment before contacting the TSC. Any services still down beyond that time will be investigated by our field team with the support of our ERS partner and TSC.

Posted in Emergency Work, Uncategorized | No Comments »

Emergency Maintenance: Core Router Software Upgrade – pre1-rtr-1

Posted: Thursday, June 22nd, 2023 at 23:22 by Jamie Smith

Friday 23rd June 23:00 – Saturday 24th June 03:00

During the above window, we will carry out an emergency software upgrades on our core router at Preston Telephone Exchange off the back of a Major Incident affecting the device on 21/6/23. All directly connected services will experience a number of outages of up to 60 minutes in total however the full duration of the window should be accounted for. Any traffic that would usually be routed through this node will take an alternative path for the duration of the work.

Posted in Emergency Work | No Comments »

Emergency Maintenance: lon55-rtr-1 & lon53-rtr-1

Posted: Friday, June 2nd, 2023 at 20:36 by Jamie Smith

Friday 2nd June 2023 23:00 – Saturday 3rd June 2023 00:00

During the above window, we will be carrying out a restart to the BGP process due to a software bug encountered on these devices. This will result in a brief outage for IP customers who terminate on this device. BGP Sessions are expected to automatically re-establish after the process restart. Any users that remain down following the work may need to reboot their Customer Premises Equipment to restore connectivity.

Posted in Emergency Work | No Comments »

Emergency Maintenance: PWAN Infrastructure Software Upgrades

Posted: Monday, April 3rd, 2023 at 09:13 by Jamie Smith

Wednesday 5th April 2023 00:00 – 05:00

During the above window we will be carrying out service affecting upgrades to two of our PWAN Infrastructure devices located within Interxion Datacenter (pwan-nx.inx.edge & pwan-nx2.inx.edge) this is due to a critical firmware bug on the SSD’s within these devices.

During this window an outage will occur to any PWAN customers who have managed or unmanged firewalls (Standalone & HA), psudowires or cross connects terminating on these devices.

We expect no longer than a 60 minute outage per PWAN however the full duration of the maintenance window should be accounted for.

We apologise for any inconvenience this may cause

Tags: ,
Posted in Emergency Work | 2 Comments »

  1. Jamie Smith says:
    April 5, 2023 at 00:21

    This work will soon be starting

  2. Jamie Smith says:
    April 5, 2023 at 01:50

    This work has now been completed

Supplier Emergency Power Work: Manchester Telephone Exchange

Posted: Friday, February 24th, 2023 at 12:48 by Jamie Smith

Friday 24th Feb

BT Locate are currently on-site carrying out an Emergency Power Change over. We are currently observing a loss of service to a small number of EAD circuits. The site should be considered at risk for the duration of this works.

BT have advised this work is due to complete at approx 16:00

Posted in Emergency Work | No Comments »

Third Party Emergency Work: Upgrade National Virgin Media Infinera Core

Posted: Wednesday, March 2nd, 2022 at 10:10 by Jamie Smith

Wednesday 9th March 2022 00:01 – Wednesday 9th March 2022 06:00

During the above window the third-party supplier Virgin Media will be carrying out an upgrade on the National Virgin Media Infinera Core. You may see a disruption to service which has been notified of up to 5 minutes as a precaution. This will occur at some point within the outage window and we are not able to specify a time.

We apologise for the short notice and any inconvenience this may cause

Posted in At-Risk, Emergency Work | No Comments »

« Older Entries