Computer Weekly's evidence to the Public Accounts Committee

Mr K Brown

Public Accounts Committee
House of Commons
Westminster
London

Dear Mr Brown

Computer Weekly has made a particular study of the Chinook's FADEC and aviation accidents in which computers or the man-machine interface have been a suspected factor.

In the case of the Chinook's FADEC there has never, to Computer Weekly's knowledge, been a procurement or an implementation of safety-critical software that has had such a dense history of significant problems.

Therefore the issue of whether the Chinook's FADEC was poorly procured, whether it was ready for operational use when it went into service and whether flaws in the system could have caused the RAF's worst peacetime accident, are issues that give the Public Accounts Committee a unique opportunity to examine some of the wider matters that are of great significance.

The key question, in our view, is whether the pilots of Chinook ZD576 that crashed on the Mull were at fault, or whether, given the FADEC's history of causing engine surges, spurious cockpit warnings and engine run-downs, sometimes without leaving physical evidence of any software problems, the FADEC's software could have caused the accident on the Mull of Kintyre by again malfunctioning without leaving any physical evidence.

The far wider issues that have been raised by the National Audit Office's research into the Chinook's FADEC, and have not so far been considered by Parliament include:

- the feasibility of independent testing of safety critical and mission critical software

- what checks exist if any to stop departments sidelining any independent advice that goes against the grain

- the conflicts of interest that arise when only manufacturers understand their software sufficiently to identify any of its flaws after a major incident

- the accountability of civil servants to Parliament after a major incident

- the difficulties of establishing physical evidence of software problems after a major incident, and

- whether, in the light of a major incident, a department will seek to protect a supplier from criticism rather than allow some of the opprobrium to spill onto the department's lap.

These are not problems that are confined to aviation. Software now controls everything from City transportation systems to missiles, command and control systems, critical telecommunications equipment and the systems on which the UK's reputation as a major financial centre depend.

This last point is topical at the present point in time. It is currently more than a week after a computer failure brought down the London Stock Exchange for nearly an entire day, and the bug that brought down the system has not been identified because it has not proved possible as yet to replicate the exact problem. The inability to identify the bug is despite the forensic skills some of the most expert software specialists in the USA and the UK.

Having studied all the available evidence and more - including hundreds of pages of documents that have never been published by the Ministry of Defence - we believe that we can show that there is insufficient evidence to blame the pilots, and good grounds for believing that the software may have played a critical role.

The evidence we have studied supports the following conclusions (if the committee so requests, we will supply documentary evidence for any of the following conclusions):

- the FADEC was procured (without open competition) without sufficient controls by the prime contractor (Textron Lycoming) on the subcontractors;

- the RAF was kept at a distance from the development process rather than working in tandem, which is usually necessary in major software-intensive projects. Studies into the lessons from IT problem projects, such as the Stock Exchange's Taurus project and the "Croeso" system undertaken by South Western Electricity and its neighbouring utility South Wales Electricity underline the desirability of joint working to ensure that the final product fulfils the customer's objectives and also because the developers can be held accountable for the quality of their work by an independent organisation (the customer) on an ongoing basis.

- also for reasons of accountability, and in the light of the failure of the post Office/Benefit Agency "Pathway" project, HM Treasury have stressed to the DTI select committee the need for major software developments to be financed by an organisation that is entirely independent of the developers. In the case of the Chinook's FADEC, the project was financed by initially by the subcontractors who were also the developers.

- the original proposal for a Chinook FADEC contained an undertaking that then project would be low-risk. This low-risk concept was not carried through to implementation, however. A change was made. Instead of a mechanical backup the FADEC's main (primary) and backup (reversionary) lanes were controlled by software. So pilots were to have no direct mechanical control of engine acceleration or deceleration. They had to delegate "full authority" to the software. Two years ago, however, Textron Lycoming announced a joint project to develop a successor to FADEC that "unlike FADEC … will feature in independent mechanical backup subsystems for all critical control functions".

- the UK version of the FADEC software design was inadequately tested and flawed, as evidenced by the software's ongoing (flight critical) problems and the number of changes the software underwent after it was accepted by the MoD and the RAF.

- the system was brought into service to meet operational needs (the number of Chinooks available for operations in 1994 being at a record low), despite the fact that the MoD was at the time taking legal action against the FADEC suppliers over claims that the FADEC software had not been designed to international military or civil avionics standards;

- the RAF and the MoD did not take the advice of its airworthiness assessors at Boscombe Down that the software should be re-written. This was because of the disruption this would have caused to the timetable for the Mid-Life Update, and also because there was a resistance to giving in to Boscombe Down whose concerns over the FADEC were perceived at the highest level in the RAF as being exaggerated (memo reference ADD/308/04 dated 6 June 1994).

- the concerns about FADEC that were expressed in 1993 by an independent contractor EDS-Scicon, which was commissioned to analyse the software, were not addressed by the time the FADEC came into operational service. This was partly because the MoD was assured in a Textron Lycoming "White Paper" that the concerns of EDS-Scicon and Boscombe Down were misplaced. However the trust that the MoD placed on Textron's assurances about FADEC were in contradiction to the MoD's distrust of Textron's assurances that the Ministry expressed in its arbitration proceedings against Textron, proceedings which were at that time secret.

The above points suggest strongly that the FADEC was implemented against the conventions of best practice. The Committee may therefore wish to consider whether the procurement of the FADEC and the poor relationship in 1993 and 1994 between the MoD and its appointed independent arbiter of aircraft software quality at Boscombe Down is a matter of some concern.

It may be asked how it is possible for a government department to procure, accept and implement a safety-critical software product that has not been benchmarked and approved by qualified arbiters, to the satisfaction of those expert assessors, using methodologies and tools laid down in the MoD's main standard 00-55 which sets out procedures for designing, verifying and validating software in safety-related applications?

Another question the Committee could ask is: how can the Mod persist with its argument that the software was ready to be put into operational use when the software's performance and reliability was questioned by its own assessors and by independent private contractors?

So far, the MoD response has been to say that Boscombe Down was using an inappropriate methodology to validate the software. The Ministry's Permanent Under Secretary of State, Mr Kevin Tebbit, at the Public Accounts Committee's hearing on 8 March 2000 suggested that Boscombe Down's chosen methodology, static code analysis, was more appropriate for the nuclear industry.

"I would not like to comment on Static Code Analysis procedure on the general side. I do know that it was used in the nuclear power industry and was applied in this context."

However Martyn Thomas, of the UK's most respected independent safety-critical software specialists, has written to Computer Weekly pointing out that a branch of the MoD helped to develop static code analysis. Indeed it has emerged that static code analysis was, in 1994, and is today, a recommended methodology in 00-55, the MoD's benchmark standard for the design, verification and validation of safety-related software.

Thomas says in his letter says that it was the Royal Signals and Radar Establishment, now part of the Defence Evaluation Research Agency, an agency of the Ministry of Defence, that helped to develop static code analysis.

"The RSRE developed the secret technology so that they could verify security-critical software. Work on static analysis was declassified as a matter of public policy, precisely so that it could be used on safety-critical software, such as the Chinook FADEC".

Other letters to Computer Weekly say that the Lockheed C130J Hercules aircraft is undergoing static code analysis for the purposes of UK flight certification; and separately, Bath-based Praxis Critical Systems, which develops software for the defence, banking and other industries, says that static code analysis has been used to validate safety-critical software in aircraft such as the Tornado F3 and the Eurofighter.

It is also used in safety-critical aircraft support functions, such as the Sholis system that helps helicopters to land safely on ships. In addition, it has been used by the Government's communications centre GCHQ to spot viruses in software.

Praxis said the importance of static code analysis, which involves testing code without executing it, lies in its ability to highlight anomalies and faults that could remain undetected by dynamic testing.

It said dynamic testing, which involves executing the code, can highlight only a small number of potential problems. This is because it does not check paths through the software that can be taken by executable code.

What all this shows is that Boscombe Down's preferred methodology, that has been much-denigrated by the MoD, was and is the MoD's own preferred methodology.

The ministry's incorrect statements on static code analysis may leave it open to the accusation that it has misled the National Audit Office, and Parliament.

On the basis of its MoD briefings, the National Audit Office (NAO) reported on the anomalies found by EDS-Scicon, but added that the contractor had used static code analysis, which it said was an "internal Boscombe Down policy, not supported by defence standards".

In 1998, MPs on the Commons' Defence Committee were told by the Ministry of Defence that, "static code analysis is... a requirement placed by British Nuclear Fuels on the safety of a nuclear system".

In July last year, a senior civil servant at the Secretariat (Air Staff) of the Ministry of Defence wrote in a letter that, "Static code analysis does not validate the performance of the software and the department therefore had no requirement for it".

In August last year, the Ministry wrote to Defence Committee MP Michael Hancock saying that "Boscombe Down's preferred method of examination is static code analysis, a system of verification not widely in use but employed in the nuclear industry."

Also last year, the House of Lords was told by the Ministry that, "Boscombe Down indicated a wish to assess the design of the FADEC software using static code analysis - a methodology used by the nuclear industry."

None of this gives a true impression of the importance to the MoD of static code analysis. It should be pointed out that static code analysis is not the only method that should be used to validate software. Specialists who have written to us say that a combination of static and dynamic will help to spot flaws and potential causes of failure.

So, if Boscombe Down was dissatisfied with the software having used the Ministry's preferred methodology to test the code, why is the Mod persisting in its claims that the FADEC was not flawed?

Nobody is certain why, but it is evident that the issues surrounding the crash on the Mull and the FADEC have become mired in half-truths, doublespeak, and falsehoods. There are a number of examples of these, which the Public Accounts Committee has encountered first hand (see later comments relating to the MoD evidence to the committee).

If the MoD cannot argue rationally and logically on why their actions were justifiable in putting the FADEC into service despite expert reservations, can it argue with credibility that the pilots of Chinook ZD576 were to blame for the accident on the Mull of Kintyre?

The FADEC is unusually complex piece of equipment, with nearly two million lines of software code. Indeed we note that when the Public Accounts Committee asked Mr Tebbit, Sir Robert Walmsley, Chief of Defence Procurement, or Vice Admiral Sir Jeremy Blackham, Deputy Chief of Defence Staff (Equipment Capability), about how FADEC can cause engines surges, none knew sufficient about the way the FADEC impacts on the helicopter's flying capabilities to give the Committee an answer.

It took Computer Weekly researchers many months of studying the design documents, and dozens of conversations with pilots and technicians, to understand how FADEC interacts with the controls on a Chinook, what can happen when the system malfunctions, how pilots should react, and what faults would not leave any trace.

We were able to conclude that no evidence of technical malfunction does not mean no technical malfunction.

If, however, the MoD's contention that no evidence of technical malfunction is analagous to no technical malfunction, this sets a dangerous precedent.

The "Rand" report for the National Transportation Safety Board which investigates aviation accidents in the United States points to the fact that accidents caused by software may not be traceable to software because it tends to leave no physical trace of its behaviour in the wreckage.

Indeed it was a conclusion of the RAF Board of Inquiry into the crash on the Mull of Kintyre that: "an unforeseen technical malfunction of the type being experienced on the Chinook HC2, which would not necessarily have left an physical evidence, remained a possibility and could not be discounted".

This issue, of manufacturers being held accountable for the software they produce, is of concern to Computer Weekly readers. If the MoD's view of the crash on the Mull of Kintyre is accepted - that no evidence of technical malfunction means that there was no technical malfunction - this in our view provides a cushion of comfort for manufacturers who supply poor quality software.

It means that if software causes a fatal accident or fails in a mission-critical system, and the fault cannot be traced afterwards because software has left no physical evidence of its behaviour, then the manufacturers cannot be blamed. This would leave computer users, Parliament, and safety regulators unable to hold software manufacturers to account if their products caused critical systems to fail.

In the case of the crash on the Mull of Kintyre the pilots were found posthumously to have been grossly negligent in the absence of any concrete evidence of technical malfunction. If this verdict is accepted, irrespective of whether this represents an injustice or not to the families of the pilots, we believe this send the wrong signals to manufacturers.

Acceptance of the verdict would also, in effect, condone what we believe is MoD doublespeak. The MoD says it will examine with compassion any new evidence but it knows no evidence of software problems can be produced. Therefore the Ministry is relying on its detractors to produce evidence that it knows cannot physically be produced. Is this not evidence of the MoD's doublespeak?

This brings us to another of the wider issues. In its anxiety to defend the FADEC, and therefore the reputation and integrity of the MoD and RAF, statements have been made to Parliament that have been incorrect and/or misleading, sometimes patently so. This may raise questions that go beyond the Chinook and FADEC, and touch on matters related to accountability of the department to Parliament. To avoid making this letter too long we have not always given examples but can do so if requested. The Ministry has:

- withheld relevant information and, when this information has leaked out, has made incorrect statements about that information. For example, as attention focused on the FADEC in the light of the crash on the Mull of Kintyre, the MoD and the RAF made no mention of its litigation against Textron Lycoming. Therefore the families of the dead pilots or passengers were unaware that FADEC was capable of causing a Chinook to crash, and indeed had caused a serious accident in 1989, after which the Ministry had issued a writ against the FADEC supplier. When evidence of the arbitration proceedings leaked out in 1997, the MoD issued incorrect and contradictory statements about the matter. It made statements, for example, saying that the RAF Board of Inquiry and the Fatal Accident Inquiry was aware of the litigation. Then it conceded that neither inquiry was made aware was the litigation. Then it issued statements that the arbitration proceedings arose from faulty testing and had "nothing to do with the software". In fact the opening page of the government's writ against Textron was that the 1989 accident was "caused by respondent Textron's faulty design of a computerised engine fuel control device, FADEC." Indeed the legal papers denied that the accident was due to faulty testing. The MoD's case was that "Boeing's test procedures were reasonable and adequate".

- continued to issue incorrect statements even after these have been shown to be incorrect. For example the Ministry last year apologised for stating incorrectly to MPs on the Defence Committee that the FADEC was not safety-critical. However the Ministry last month repeated the original incorrect statement to the Public Accounts Committee.

- used selective quotations from the report of accident investigators. The effect of this has been to give an impression of certainty when investigators, in sentences immediately before or after the one selected by the MoD, have expressed uncertainty or a caveat. For example the MoD, in several letters to MPs and in one to the British Airline Pilots Association sought to show that the FADEC's main computer component (called the DECU - Digital Electronic Control Unit) was not at fault in the accident on the Mull. All of these letters quoted one particular part of the accident report which said: "Strip examination of the engines … revealed no signs of pre-impact failure or malfunction that could have affected the operation of either engine". However none of the letters quoted the very next sentence in the accident report which said: "Fire damage prevented assessment of the functionality of the No 1 (engine's) DECU and had destroyed its memories of the operating program and exceedance fault history".

- Omitted facts or parts of official reports that have not been consistent with the MoD's stated position that FADEC has never caused a Chinook accident, could not cause a Chinook accident, and has never, in its production versions, had any serious flaws. For example the MoD did not mention, in its dozens of letters to MPs and in Parliamentary Answers that one conclusion of the RAF Board of Inquiry was that: "… an unforeseen technical malfunction of the type being experienced on the Chinook HC2, which would not necessarily have left an physical evidence, remained a possibility and could not be discounted".

Mixed undisputed facts with disputed MoD speculation, without making the distinction clear, in such a way as to convey certainty when none exists. It has also made statements without context that have had the effect of giving an incorrect impression. For example, after the fatal crash of a US Army Chinook equipped with FADEC in 1996, the deceased pilots were blamed because no fault was found of a technical malfunction. Later it was found that the original verdict was wrong and that, contrary to the initial investigation report which found no evidence of any electrical problems, there had in fact been an electrical failure. When the MoD was asked about possible electrical problems on the aircraft that crashed on the Mull, the MoD replied that the findings on this were "unequivocal" and it quoted part of a sentence in the investigator's report which said that a "major pre-impact loss of electrical supplies had not occurred". The MoD omitted to mention the first part of the sentence which said "While none of the direct indications of electrical system behaviour was conclusive …" To cite another example, the US Army issued a warning to its Chinook community in 1999 that hydraulic contamination was a suspected cause of sudden, unexpected and potentially fatal manœuvres of the aircraft. When it was put to the MoD that investigators of the crash on the Mull had found a "considerable quantity" of particles in hydraulic fluid and had concluded that there was "pre-impact hydraulic system contamination," the MoD quoted from other parts of the accident report in which hydraulic components had been found without abnormalities. The Ministry also, in a letter to the MP Michael Hancock, dated August 1999, said that hydraulic contamination had been "ruled out" as a result of an investigation by the Air Accidents Investigation Branch. The letter added that the investigators found hydraulic contamination that was "consistent only with what would have been expected as a result of normal wear and tear". The MoD omitted to quote the reference in the investigation report to "pre-impact hydraulic system contamination and "high" wear rates. Also in its letter, the MoD omitted to mention that the accident investigators, among their final conclusion in the accident report had, remarked that there were "possible utility hydraulic system abnormalities".

- criticised all specialists who have not agreed with the MoD over the FADEC. The criticism has extended to Malcolm Perks, the MoD's own expert witness in its arbitration proceedings against Textron Lycoming, Malcolm Perks, who has expressed the view that FADEC could have caused the crash on the Mull. The MoD has also questioned the professionalism of Squadron Leader Robert Burke, a pilot with more test hours on Chinooks than any other at the time who expressed the view that that at an engine surge could have contributed to the crash on the Mull. The Ministry has also criticised the A&AEE at Boscombe Down for using inappropriate methods to test the FADEC, when in fact those methods were those recommended by the MoD. In addition the Ministry belittled the evidence of its independent contractor EDS-Scicon by saying that it raised issues on FADEC that were related mainly to documentation. In fact EDS-Scicon had raised concerns about the safety of the software. The Ministry has also criticised the media, sometimes by denigrating statements that the media did not make.

- issued a letter warning a respected aviation magazine that if it went ahead and published an article that was critical of the decision to blame the pilots for the crash on the Mull of Kintyre, it may face an action for defamation.

- defended FADEC by quoting the assurances of the Design Authority (Textron Lycoming) without drawing attention to the fact that the Ministry had undermined those same assurances during the arbitration proceedings, which the Ministry won.

But why, since the crash on the Mull, has the MoD has made so many misleading statements (many more than have been mentioned above) with the effect, apparently, of turning attention away from the FADEC and onto the pilots?

One possible explanation is that the Ministry does not want to give any ground to those who are critical of its decision to put the Chinook into operational service before the problems with the FADEC's software had been eliminated.

It could also be said that the MoD does not wish to lend weight to the concerns shared by the families of the dead pilots and the families of the dead VIP passengers on that last flight of Chinook ZD576, that FADEC may have been involved in the accident.

We are not certain, however, that these are the reasons for the MoD's position.

What we have seen is that, the more information that comes to light, from the US Army and elsewhere, which throws doubt on the original decision to blame the pilots, the more entrenched has become the MoD's defence of FADEC's implementation on the Chinook.

We cannot help but conclude that the reason for the MoD's position is that it has always in public defended the decision to blame the pilots and that it must, for the sake of consistency, continue to defend this decision, whatever new evidence or information arises.

It appears to us that that the matter of whether the FADEC was fundamentally flawed, or whether it caused the crash on the Mull, is in some ways subordinate to the need to sustain departmental pride and not admit that a mistake may have been made.

Indeed there appears to be, within the MoD, an institutional machismo that will admit no imposition on its decision making by those whom it regards as outsiders: particularly politicians and the media.

If this attitude continues to prevail - and we suspect it will - then no matter what information comes to light, or whatever political pressure is applied, the Ministry and particularly air marshals in the RAF will not allow the verdict against the pilots to be stood down.

Therefore we believe that the matter needs to be investigated by a body that is independent of the MoD. For if no action is taken to reinvestigate, or to question the MoD statements in relation to the FADEC, this will leave unanswered the question of whether the Ministry has inadvertently misled Parliament or has done so systematically rather than countenance the possibility that it made mistakes, firstly by rushing the FADEC into service and then defending the decision to blame two dead Special Forces pilots for a crash that has no identifiable cause but which could have involved FADEC.

CONCLUSION:

a) It is not possible to look at the procurement of the FADEC and say whether it was safe, procured properly and in a timely fashion without looking at its performance after it came into operational service. The greatest possible example of its malfunctioning could have been the crash on the Mull of Kintyre. The question of whether the FADEC worked properly could be related directly to that accident.

b) The MoD is failing its own legal defence team in the arbitration hearing. The Ministry used hundreds of pages of evidence to prosecute its case against the FADEC supplier Textron Lycoming but failed to make any of this material available to any of the inquiries into the performance of the FADEC including the Public Accounts Committee. Once details of the MoD's successful legal case against Textron were put into the public domain by the media, the MoD attacked its own evidence as irrelevant and peripheral to the performance of the FADEC.

c) Ultimately there are two issues: Did the FADEC pass all the tests set for it by the procurement team which rightly included Boscombe Down and EDS-Scicon, or for a host of operational reasons, was FADEC rushed into service, amid a hasty dismissal of expert concerns, with possible tragic consequences?

If the Committee wishes to see any or all of the documents on which this letter is based, we will be happy to oblige.

Yours sincerely

Karl Schneider
Editor

Executive Editor
Tony Collins

Hooman Bassirian
News Editor