NATS failure down to bug from the 90s and redundant code

A bug present in the National Air Traffic Services (Nats) IT system since the 1990s has been identified as the root cause of the five-hour outage of UK air traffic control on 12 December 2014, according to an independent inquiry.

The Civil Aviation Authority's Nats System Failure 12 December 2014 – Final Report Independent Enquiry highlighted a bug in a module of the System Flight Server (SFS), which augments flight plan data with dynamic information, including clearance and co-ordination data from the controller.

“The fault had been present since the time the SFS system became operational in 2002. It was triggered because a set of circumstances arose that had not occurred previously in the system operation,” the Civil Aviation Authority's (CAA) report stated.

A redundant code called “watching mode”, designed to prevent flight controllers from seeing the wrong data, caused the SFS and the backup SFS to receive the same programming exception, which caused them both to shutdown.

“As the fault was in the SFS software, replaying the commands led to the second SFS system failing in exactly the same way as the first, resulting in the complete loss of SFS,” the CAA report stated. 

The problem was a software fault, which affected the code designed to manage hardware failure. 

“To test every combination of workstation modes at one second per test would take of the order of 100 years, without considering all the other parameters,” the CAA said. 

The authority praised the engineering team for identifying the problem, but urged Nats to apply “modern design principles” to the human machine interface of the forthcoming pan-European air traffic control systems, known as Sesar.

The report recommended Nats retain its log management and software release programmes to ensure Sesar is successful.