Why agile development failed for Universal Credit

Adapting to agile

DWP executives told Computer Weekly in 2011 they had already agreed contracts with major suppliers at the start of the programme. These were conventional contracts. They contracted Susan Atkinson, a lawyer with Gallen Alliance who had developed a model agile contract, to adapt the contract by adding extensions that made them amenable to agile development.

Atkinson now refuses to say whether she succeeded. Andrew Dyson – a DLA Piper lawyer contracted to work on major DWP contracts, and who had worked with Atkinson on the contract extensions – did not return calls.

The DWP had already completed contract negotiations with BT, HP and Accenture when it sought to turn Universal Credit into an agile project in June 2011, said programme director Malcolm Whitehouse at the time. The major suppliers were selected under the Application Development and Enhancement Programme, a £400m procurement project that had been on ice since 2009, before the coalition government came to power.

Between three and five months later, the DWP gave its applications contracts to Accenture, HP, IBM, and Capgemini. These companies would act as primary contractors to Universal Credit under the £1.12bn Application Deployment Services contract.

Yet the DWP insisted it would require these large system integrators to employ small, specialist suppliers to help them develop Universal Credit using agile methods.

Whitehouse and Steve Dover, then co-director of Universal Credit, courted such suppliers at an agile industry conference shortly before DWP signed its major systems integrator contracts in 2011.

Dover said the DWP would instruct the systems integrators to employ small suppliers on the project. Whitehouse said the DWP would contract small agile software houses through the Agile Delivery Network, a commercial alliance of such companies.

"We are going to be measured on the number of SMEs we use," said Dover at the time.

Computer Weekly has learned, however, that of 24 contractors employed on Universal Credit since 2010, only three have been SMEs.

Most of Universal Credit's suppliers - 20 of them, according to a freedom of information request - were companies that individually report annual sales of billions or hundreds of millions of dollars.

Of the rest, one was Thoughtworks, a global supplier of agile software project tools. Only three could claim to be small suppliers: a company that provides localisation services; Lucidus Consulting, a risk consultant; and Emergn, an agile consulting firm the DWP employed to train its systems integrators for Universal Credit.

Universal Credit was wrong from the beginning. Government needs to rethink the way it procures. Until they change procurement, the rules of the game are stacked against anyone doing agile properly. We were working against the tide

Former principal agile consultant on Universal Credit

Failure to deliver

Paul Wilson, a director of the Agile Delivery Network, said the DWP never delivered on its offer to contract small agile software developers. The Agile Delivery Network (ADN) was never contracted to work on the programme.

Andy Devale, another ADN director, said: "Malcolm Whitehouse said he was going to work with the Agile Delivery Network. It never happened. ADN had no involvement. It's been working with the Government Digital Service.

"There wasn't really a follow-up after the meeting. We tried to get in touch but it wasn't exactly pushing on an open door. Subsequent to that, Steve Dover went to forums where he gave robust speeches in agile," said Devale.

Another small supplier said at the time:  "It's all quietly tied up. No small company has a chance. There's no place in the contract for small players like us."

Alex Adamopoulos, CEO of Emergn, spoke out against the systems integrators he was contracted to work with on Universal Credit in 2011. Now, like Atkinson, he won't talk about it at all.

"I'm not the right person to ask," he said. "I've no time to speak. You might be better off speaking to the DWP directly."

He refused to say whether Emergn had been taken off the programme.

A consultant who worked on Universal Credit with Emergn, however, said agile contractors hit a wall when they arrived at the DWP. Its culture was more in tune with old software practices than the agile methods it purported to now follow.

"Universal Credit is very well known in the agile community - and not necessarily for the right reasons," he said.

"There was an extremely strong command and control culture at the DWP, which goes against agile. We were trying to alleviate that - but it wasn't working," said the source.

"My view is they weren't really embracing agile," said another leading member of the UK agile community.

"This is what I heard. The big systems integrators told the DWP they could do agile. The DWP trusted them. But if you look at the SI credentials, they were learning agile on the job. I've never seen them do agile before. They didn't have a track record.

"They were trying to disguise their more traditional processes as agile. Because they can make more money out of change requests," he said.

This was a characteristic of the waterfall regime rejected by the coalition government, and one specifically criticised by cabinet office minister Francis Maude, whose department had pushed agile onto Universal Credit and itself employed those ADN companies the DWP had shunned.

The senior project source said Universal Credit was consequentially agile in part. Agile development was expected to be incremental and iterative. Universal Credit was incremental, in that the project had been broken up into small deliverables. But it had not been iterative - it was delivering the increments one at a time, instead of gradual iterations of a continuously evolving software project.

Too big to be agile

Paul Wilson, UK director of agile developer Neo and a director of the Agile Delivery Network, said Universal Credit could never have been an agile project because it was simply too big in the first place.

"It's far too ambitious," he said. "These big plans don't work. All they are going to do is waste money."

If Universal Credit was agile, said Wilson, it would have started with a modest £250,000 budget and evolved from there.

But this approach may not have satisfied the political necessity of work and pensions secretary Iain Duncan Smith, who conceived Universal Credit, to get the project delivered just as specified by his own political requirement for a radical programme to reform the way government calculated, administered and delivered £74bn of benefits to 8 million people.

"Small changes do not make a glamorous political platform," said Wilson. "You can't take that to the ballot box."

The converging influences that undermined agile at the DWP were the conventional contracts it was already negotiating with the major suppliers and its own political requirement for big-bang reform.

Computer Weekly asked IBM if agile had not been up to the job, or whether it had simply not been commercially convenient for a large supplier. A spokesman said it was not prepared to talk about its work. Accenture said the same.

Just six months prior to that, Ian Duncan Smith publicly reasserted that Universal Credit was an agile project. But in weeks both his project directors departed. Insiders report that the late Philip Langsdale, then the DWP CIO, dismissed them as a part of his overhaul of the troubled Universal Credit IT project.

After Langsdale’s untimely death in January 2013, DWP stuck to his plan, which no longer involved agile.

Universal Credit came under the remit of Howard Shiplee, previously director of construction company Laing O’Rourke, where the waterfall project style would have been familiar and, arguably, more appropriate.

Universal Credit's costs have since been revealed by Computer Weekly to be £12.8bn, six times the figure ministers originally said they would be.