What was the 'dangerous' flaw in safety-critical Chinook system?
Defence contractor EDS - now part of HP - warned the Ministry of Defence that flaws in a Fadec safety-critical system fitted to the Chinook Mk2 helicopter could cause it to malfunction, months before a fatal crash on the Mull of Kintyre.
Defence contractor EDS - now part of HP - warned the Ministry of Defence that flaws in a Fadec safety-critical system fitted to the Chinook Mk2 helicopter could cause it to malfunction, months before a fatal crash on the Mull of Kintyre.
The warning in a confidential report was ignored and the MoD allowed Chinook Mk2 helicopters to be fitted with a poorly-designed fuel control system. A helicopter of the type fitted with the system crashed on in June 1994, killing all 29 on board including 25 senior police and intelligence officers.
For the first time, Computer Weekly is publishing a detailed account of the software faults found by EDS when it examined the Chinook Mk2's fuel-control system, known as the Full Authority Digital Engine Control (Fadec) system, in July 1993.
The crash of Chinook ZD576 was one of the worst RAF accidents in peacetime.
An RAF Board of Inquiry did not rule out a Fadec-related problem as a factor in the accident. But two air marshals found that the pilots of ZD576, Flight Lieutenants Rick Cook and Jonathan Tapper, had been negligent to a gross degree.
The EDS report is one of the pieces of technical evidence that was withheld from the Air Accidents Investigation Branch which investigated the crash of Chinook ZD576 on the Mull.
The EDS report went unseen, too, by an RAF Board of Inquiry into the Mull accident.
The failure of the MoD and RAF to show to investigators the EDS report and other documents that highlighted flaws in the design of the crashed helicopter's safety critical systems will reinforce concerns among some defence IT experts that there were gaps in the investigation into the Chinook crash on the Mull of Kintyre.
In its report on the Chinook Mk2's Fadec software, EDS said it abandoned its evaluation after analysing 2,897 of the total 16,254 lines of code. because of the large number of anomalies found.
The 56 most severe anomalies, which EDS termed "category one", included blocks of superfluous code that should not have been present in safety-critical software.
According to safety-critical software expert Martyn Thomas, who was asked by Computer Weekly to comment on the EDS report, safety-critical software should not normally contain any unnecessary code in case it ended up being activated. In some circumstances, the activated code could cause the system to behave unpredictably, he said.
EDS's report said that the most serious anomaly it found was in the Fadec's back-up software, known as the "reversionary lane".
The Fadec's software was split into two parts: a primary software channel and the backup "reversionary" lane. If the primary software lane failed, the system would "revert" to the backup reversionary lane.
But EDS found that the correct operation of the reversionary lane depended on an "undocumented feature of the Intel ASM96 microcomputer". This meant that a minor change in the processor's manufacturing process could cause it to malfunction.
Computer Weekly has learned that the software's authors Hawker Siddeley - which later became part of BAe - assured EDS that the code behaved correctly. No code needed modifying, it said.
But EDS said in its report to the MoD that it was concerned that a "change in the mask or process of the ASM96 chip at some point in future "may cause incorrect operation of the Fadec".
EDS's concern might have been relevant if the ASM96 processors on Chinook Mk2 Fadec systems had ever been replaced with a slightly different version, for maintenance purposes.
Indeed Computer Weekly has evidence that a Fadec processor was changed on ZD576 about three weeks before it crashed on the Mull of Kintyre.
RAF engineers replaced one of ZD576's Fadec processors because of a suspected fault on 18 May 1994, according to an internal Mod "maintenance work order", form 707B.
Thomas, who is a visiting professor of software engineering at Oxford University Computing Laboratory, told Computer Weekly: "It's dangerous and highly unprofessional to use undocumented instructions or addressing modes."
The EDS report highlighted faults or examples of bad practice that should be corrected before the software went into service, he said.
"Overall they show a very poor level of quality management which should be enough, on its own, to deny the aircraft a release to service".
But the Chinook Mk2 went into operational service in late 1993, less than five months after EDS gave its report on the Fadec software to the MoD. without a correction of the software. Only after the crash on the Mull was the software subject to a major modification.
The MoD and its ministers have told MPs repeatedly that the Fadec caused only "nuisance" faults on the Chinook which were not serious. They have also claimed that the Fadec, even if it malfunctioned, could not endanger the helicopter. In fact, the MoD did not categorize the Fadec system as safety critical.
But last month, Computer Weekly and BBC Radio 4's Today programme published details of an internal MoD memo which described the reliance of the Fadec software on an undocumented feature in the Intel processor as "positively dangerous".
The memo, which was written on 30 September 1993 by the Superintendent of Engineering Systems at the MoD, Boscombe Down, said: ". . . the reliance on an undocumented and unproved feature of the [Fadec] processor is considered to be positively dangerous" in our opinion the software quality falls significantly short of the standard required and expected for a safety-critical system."
The memo added that pilot control of the engines through the Fadec could not be assured. "No assurance can be given concerning the fidelity of the software and hence the pilot's control of the engine (s) through Fadec cannot be assured."
Three fellows of the Royal Aeronautical Society who have studied the crash say that the Chinook Mk2 was not airworthy when it went into service. They point out that the MoD and RAF hierarchy may have a conflict of interest in continuing to blame pilots for the crash of an aircraft type that might not have been airworthy.
The defence secretary Bob Ainsworth and his minister Bill Rammell say that nothing new has come to light which casts doubt on the finding of gross negligence against the pilots of Chinook ZD576.
But the Conservatives have promised that, if they win power, they will appoint a senior judge to review the decision to blame the pilots.
EDS report into safety of Chinook software >>
Chinook Mk2's software was "positively dangerous" say newly-disclosed documents >>
Computer Weekly's 140-page "RAF Justice" report on the Mull crash and the Chinook Mk2 software >>
Campaign for Justice over the Chinook crash >>
