Fixing government digital transformation – lessons from the early days of GDS

Central controls

The GDS central controls are often now referred to as “spend controls”. While the financial aspects are important and helped provide GDS with HM Treasury-backed authority, this is only one reason why they were introduced.

When GDS started, there was little insight at the centre into what was happening across Whitehall. Without baseline evidence, it was hard to know what was going well, what was going badly and where central assistance or co-ordinated, collaborative working between departments might help.

To inform our planning, we needed a single view across government of everything that was happening or planned. It was an essential part of situational awareness and mapping the landscape – our own systematic discovery, if you like.

Our purpose in establishing the central controls was to provide this missing insight: the contracts in place (with whom, what for, their duration, cost, and so on), the work being planned, the open standards being worked to, the technologies, the costs and budgets and where they went, the legacy estate, and the staff and contractors in place. So, everything that could provide an informed, collective view across Whitehall and enable us to build out a shared and co-ordinated pipeline of current and planned policies, programmes and systems.

“Organisations are encouraged to work with controls teams to submit forward looks (including for their arms-length bodies) to the Cabinet Office on a regular basis through pipeline reports. Forward looks will also be complemented with meetings between organisations and controls officials.”

Source: Extract from ‘Organisation’s responsibilities, Cabinet Office controls 2014’

It proved remarkably difficult and slow to get the baseline data required. There was (and is) no common chart of accounts across Whitehall to identify what is being spent where, with whom, why and on what. Even within a single department, a comprehensive central view rarely exists. Some of the data we had to acquire directly from departments’ suppliers and industry analysts. Yes, it was that bad.

On the back of developing this single authoritative and public view, we also wanted to improve in-house capabilities. By gathering the data, we’d know what skills and talent were needed to tackle common needs and problems.

“The bench” became our internal nickname for the development of a virtual pool of talent to be provided both in-house, from small suppliers and others.

This technical resource could be drawn on by whoever needed it and would provide expert knowledge spanning everything from legacy systems running Cobol, to .Net to Ruby on Rails, and from Wardley mapping to using agile at scale – whatever skills and expertise were needed to enable us to work collectively and effectively together across Whitehall.

Agile

The use of agile in government long predates GDS, as this Computer Weekly article and the National Audit Office describe. However, it was not often well-applied either in-house or among the supplier base.

Where agile was used, it often tended to reinforce longstanding departmental policy stovepipes, technical design and isolated programme management and delivery, just as waterfall had done.

This problem was also highlighted as one of the significant “learnings” from the 25 GDS “digital exemplars”. While the exemplars aimed to work to shared principles and design standards, they also proceeded in silos, working alongside rather than with each other.

“Share resources: services, information, data and software components must be shared in order to encourage reuse, avoid duplication and prevent redundant investments. Reuse includes the use of existing services and capabilities that already exist outside of government where they provide best value for money – e.g. identity verification, fraud and debt management, cloud-based commodity services.”

Source: Extract from ‘The technology code of practice, GDS chief technology officer 2013’

Without a meaningful central controls process and a comprehensive view across Whitehall, even where agile was well used, it became hostage to existing policy, programme and organisational silos rather than contributing to an integrated cross-government approach. As has happened so often from 1994 onwards, the primary focus retreated into the improvement of government websites delivering silo experiences rather than on developing more modern, flexible and relevant public services.

Today, much of what is called agile just simply isn’t. Instead of being an iterative, continuing process of public service improvement, it’s a silo project. It’s waterfall by another name — a series of sequential steps labelled Discovery, Alpha, Beta, Live.

Discovery often lacks a meaningful evidential base and objective rigour. It seldom engages at the early critical stages of social or economic need, policy research, analysis, inception and design. It’s typically reduced instead to user research and website design to implement a predetermined, top-down policy decision. As such, it’s become worryingly equivalent to writing an old-school technology business case.

Neither should discovery be solely an upfront early stage. It needs to be a continuing process that informs improvements throughout alpha and beta stages and long after going live. Instead, the use of agile in Whitehall has largely defaulted to automating silo policies, processes and operations.

Afterwards

The abolition of the chief technology officer (CTO) role and associated Office of the CTO within GDS saw the push towards open standards, cross-government platforms, better use of data and application programming interfaces (APIs), updating of central platforms, “the bench” and so on, all begin to decline in importance relative to the focus on websites and website service design.

After a promising start and considerable work on developing a live, dynamic pipeline and collaborative series of pan-Whitehall common interest groups – open source, agile, legacy, open standards, data – a prevailing view developed among some senior officials that none of this was necessary.

“Old stuff” – meaning contracts, technologies, legacy IT and more – would simply be magically replaced by “new stuff” over the coming years. It was all somehow going to be easy, facilitated by sheep-dipping civil servants via “101 instant expert” courses at new digital academies – courses that largely reinforce the dysfunctional split between policy and technology by focusing on downstream implementation aspects such as agile and “digital” project management.

The single, authoritative central controls atrophied into the much narrower process of spend controls. As a result, there remains little overall insight at the centre. The openly published and continuously updated pan-Whitehall pipeline has never happened.

Agile has often become a camouflaging badge of convenience pinned on top of the status quo. As the National Audit Office has observed: “Effective governance and accountability structures are vital if teams are to use an agile approach to deliver successful projects.”

In the absence of the insight and collaboration provided by central controls, the ability to implement meaningful change at scale and to explore and pool policies, capabilities, standards, products, services and processes was also lost.

Lessons learned – breaking the cycle

This wasn’t the first attempt at radical change driven by a central team. The wellsprings of digital transformation reach back at least to the mid-1990s. The work of GDS mirrors many of the experiences of 1997 onwards, when open standards, shared platforms, improved in-house capabilities, agile processes etc. also became the norm under an earlier Cabinet Office team.

So what should we, and the new CDDO, take from all this, apart from the lack of Whitehall memory potentially condemning us to live forever in Groundhog Day?

Improving government, Whitehall and our public services are all political problems, not ones of technology alone. Public policy and technology need to be developed hand in hand, not by one trying to unilaterally lead and impose on the other.

Every political decision has technical implications and every technical decision has political implications, yet the two are rarely discussed, researched, developed and designed together. The potential dangers of this schism can be witnessed in technically driven efforts to “break down” Whitehall and implement single central technical platforms:

“Our unwritten constitution is a fragile thing. It relies on preventing a centralisation or abuse of powers though the intentional balance achieved by splitting and distributing government into different legal entities, and ensuring that there are checks and balances between those different branches or departments. Cross-government platforms, unless well-designed both individually and collectively, could fundamentally tilt this balance – and provide a single point of oversight, power and control.”

Source: Shared cross-government platforms

Until policymaking and politics catch up with the possibilities of technology, it’s hard to see how major improvements can be made to the way our public sector works to the benefit of employees, citizens, businesses and politicians alike.

Even when a strong political minister such as Ian McCartney or Francis Maude is involved, helping drive change and establish momentum, they remain one solitary voice rather than being part of a wider cross-party commitment to radical improvement. When these individuals move on, meaningful political sponsorship disappears with them too.

If we want to succeed where previous efforts have failed, our focus as technologists might be better rewarded by engaging more successfully and openly not just with politicians and policymakers, but with everyone who uses and cares about public services. If the phrase “meeting user needs” means anything, then perhaps it’s time we started there, and not with top-down models of technical change.

Jerry Fishenden was a senior technical advisor to GDS’s chief technology officer and later interim deputy CTO at GDS; James Duncan used data from spend controls to formulate the Crown Hosting strategy; James Findlay was technology leader for the Department for Transport.