MoD did not tell the whole truth on Chinook software

Ever since the crash of a Chinook helicopter on the Mull of Kintyre in 1994, the Ministry of Defence has defended the quality of the Chinook helicopter's safety-critical engine control software by adopting a twin-track approach.

First, it has staunchly defended its decision to put the Chinook, with a new Full Authority Digital Engine Control (Fadec) system into service.

Secondly, it has attacked a technique used by its IT and airworthiness assessors at Boscombe Down for validating safety-critical software, known as static code analysis, which found a high incidence of flaws in the Fadec software.

Over the past three years, the MoD has told two Parliamentary committees, ministers and the public spending watchdogs - the National Audit Office - that static code analysis was inappropriate for testing the Chinook Mk2's Fadec. The technique, said the MoD, was used by the nuclear power industry rather than the defence community.

Now IT specialists have written to Computer Weekly revealing that the MoD was wrong in its repeated public statements concerning static code analysis. A branch of the MoD in fact developed the disputed methodology. And it was developed specifically to provide an independent method of validating safety-critical software, the quality of which may otherwise have been impossible to ascertain.

This means that one of the main justifications for the decision to blame the pilots for the crash on the Mull of Kintyre - that there was no evidence of any software problems - has now been undermined.

Specialists say the principal reason that static code analysis is a recommended technique of the ministry is that it detects anomalies that may be missed by other validation methods - anomalies that may cause the software to behave unpredictably.

In various public statements, the ministry has argued that Boscombe Down was unable to validate the Fadec software because of its use of static code analysis. If this argument is accepted then Boscombe Down, as well as the independent contractor EDS-Scicon, were using the wrong technique to assess the software.

Were this true it would disaffirm the findings of Boscombe Down that the software was unacceptable. It would also belittle the evaluation by EDS-Scicon that found a high incidence of anomalies in the software.

it is now clear that static code analysis is not only a defence standard but is MoD best practice for validating safety-critical software in aviation systems. This establishes that the techniques used by Boscombe Down and EDS were the correct ones after all. Therefore, Boscombe Down's findings have a credibility that has been wrongly undermined by the MoD.

The ministry's incorrect statements on static code analysis may also leave it open to the accusation that it has misled the National Audit Office, and Parliament.

On the basis of its MoD briefings, the National Audit Office (NAO) reported on the anomalies found by EDS-Scicon, but added that the contractor had used static code analysis, which it said was an "internal Boscombe Down policy, not supported by defence standards".

At a hearing of the Public Accounts Committee earlier this month, the MoD's top civil servant, Permanent Under-Secretary Kevin Tebbit, said he agreed with the NAO's statement on static code analysis.

He said static code analysis was Boscombe Down's "preferred" method of analysing the software. He also claimed that it was "inappropriate" for verifying or validating Fadec.

When asked by the committee why Boscombe Down was using an inappropriate method of testing the software, Tebbit said, "I do know that it was used in the nuclear power industry and was applied in this context".

If the ministry has inadvertently rather than deliberately denigrated Boscombe Down, it has done so consistently.

In 1998, MPs on the Commons' Defence Committee were told by the Ministry of Defence that, "static code analysis is... a requirement placed by British Nuclear Fuels on the safety of a nuclear system".

The Defence Committee concluded that Boscombe Down's "failure" to give final approval to the Fadec software was a "management failure".

In July last year, a senior civil servant at the Secretariat (Air Staff) of the Ministry of Defence wrote in a letter that, "Static code analysis does not validate the performance of the software and the department therefore had no requirement for it".

In August last year, the Ministry wrote to Defence Committee MP Michael Hancock saying that "Boscombe Down's preferred method of examination is static code analysis, a system of verification not widely in use but employed in the nuclear industry."

Also last year, the House of Lords was told by the ministry that, "Boscombe Down indicated a wish to assess the design of the Fadec software using static code analysis - a methodology used by the nuclear industry."

None of this gave a true impression of the importance to the MoD of static code analysis. Computer Weekly has a 1991 version of defence standard 00-55 which specifies static analysis for the validation of safety-critical software used in aircraft and other defence equipment.

And a letter to Computer Weekly from Martyn Thomas, one of the UK's most respected independent specialists in the field of safety-critical software, says that static code analysis was developed by the Royal Signals and Radar Estab-lishment, now the Defence Evaluation Research Agency, an agency of the Ministry of Defence.

"RSRE developed the secret technology so that they could verify security-critical software," he says in a letter.

"Work on static analysis was declassified as a matter of public policy, precisely so that it could be used on safety-critical software - such as the Chinook Fadec," he adds.

Another detailed letter to Computer Weekly from a software specialist says that the Lockheed C130J Hercules aircraft is undergoing static code analysis for the purposes of UK flight certification.

And separately, Bath-based Praxis Critical Systems, which develops software for the defence, banking and other industries, told Computer Weekly that static code analysis has been used to validate safety-critical software in aircraft such as the Tornado F3 and the Eurofighter.

It is also used in safety-critical aircraft support functions, such as the Sholis system that helps helicopters to land safely on ships. In addition, it has been used by the Government's communications centre GCHQ to spot viruses in software.

Praxis said the importance of static code analysis, which involves testing code without executing it, lies in its ability to highlight anomalies and faults that could remain undetected by dynamic testing.

It said dynamic testing, which involves executing the code, can highlight only a small number of potential problems. This is because it does not check paths through the software that can be taken by executable code.

But it was not only in the area of static code analysis that the Public Accounts Committee was given the wrong impression by the ministry. Tebbit mistakenly told the committee that the Fadec software was not designated as safety-critical, although Boeing had said it was.

The ministry also told the committee that Fadec had undergone 70,000 hours of testing. But it did not say that this was the cumulative total for a number a different versions of Fadec. And the 70,000 hours related mainly to a version of Fadec used by the US Army. This was not an identical system to Fadec as used in RAF Chinook helicopters.

But even if the ministry may have omitted to tell MPs the whole story of Fadec, and what it has told them may have sometimes been misleading, does this matter?

Some people believe it does. If Parliament accepts incorrect statements from the MoD, it suggests that nobody can ever be held accountable for the decision to put 25 of the UK's top intelligence officers into a helicopter which had known flaws in its safety-critical software.

The technique, said the MoD, was used by the nuclear power industry rather than the defence community

A branch of the MoD in fact developed the disputed methodology

This establishes that the techniques used by Boscombe Down and EDS were the correct ones after all

More Chinook news