Searching for the signal of open standards amid the growing noise of agile

For all its undeniable strengths, the use of agile in government is in danger of becoming a fetish

It’s now almost six years since I wrote a paper entitled Open Source and Open Standards: Reforming IT Procurement in Government for George Osborne, suggesting that, if elected, the Tories should place the rigorous pursuit of open standards at the heart of their approach to IT. 

The thinking was simple: the link between architecture, technology, and government contracts was not working for government, for one reason. Government was buying bespoke equipment in huge, integrated chunks. An approach with a number of flaws.

First, Whitehall departments were buying the same equipment over and over again without realising it. They were often buying off-the-peg technology at bespoke prices.

Because everything was integrated with everything else, a change to one small part - to accommodate new legislation, for example - often meant everything else needed re-engineering.

Then there was the fact only a handful of suppliers were capable of doing business with the government in huge, integrated chunks, and they knew it.

And it was only worth everyone’s while investing in huge, integrated systems once every few years, because buying equipment designed in this way was so expensive. This meant contracts were locked down and difficult to escape from.

It’s always easy to generalise in hindsight, but the picture in 2008 resulting from the above was that government bought huge, bespoke, expensive, integrated IT from a small handful of large suppliers, typically on long-term contracts that guaranteed those suppliers minimum revenues over many years. 

Open standards

For some of us at the time, the way out was open standards. Why?

Open standards was more than just a rallying cry for sharing equipment openly. It meant breaking down huge IT into smaller chunks of standardised, interoperable components, or transactions, which could be reused across government.

Unfortunately, for all its undeniable strengths, agile is in danger of becoming a fetish - something imbued with additional symbolic charge over and above its actual meaning

As the number of re-useable components for use across government grew, IT projects would become smaller, iterative, lower-budget, and more user-centric – standard building blocks that could be reassembled in locally appropriate ways.

So here’s the controversial bit: open standards were the foundational principle underlying an architecture of consumption. 

Open standards don’t care where the technology comes from, in the same way as you probably don’t care which version of Google you’re running. Providing it meets standardised service outcomes and overarching technical/security criteria, you’re happy to consume it. 

And government IT could be the same. If we could only have the self-discipline to standardise our demand across government for common building blocks, we would create a ready market, in which a host of large and small suppliers would willingly innovate and compete.

A properly architected government would create huge demand for standard functions, and would be able to sit back and consume, as the market responded.

No more upgrades, maintenance, or supplier lock-in. As a general rule government would not build and maintain equipment itself, but would opt instead to consume the best from a rapidly evolving, globalised marketplace. It would get its act together, stop bespoking equipment that should be commoditised, and join the mainstream – benefiting from innovation built on commodity components in much the same way that house builders have benefited from standard bricks.

Open source

In contrast to the consumption architecture underlying open standards, open source is different, because it does care where the technology comes from. And because of this, a government architecture founded on reuse of open source alone can never be an architecture of utility consumption.

Instead, the logic becomes one of in-house development and maintenance. The equipment doesn’t exist “out there”, so you need to build it. The tail wags the dog.

Of course, open source is a powerful tool that can be used to create and encourage the development of commodity components and a vibrant market. Many of the core components of our modern society, such as the internet, were forged by it.

Read more on government digital strategy

In practice, any open standard should ideally be provided as running “open source” code, eventually. However, the goal of government IT should be to build on commodity components, such as utility services, and to create an architecture of consumption. While open source helps in this path, only open standards define that end state.

This is where agile comes in, and the controversial bit. 

Agile

In the context of an architecture of consumption, agile is a powerful approach, since it allows the recombination and re-use of standard building-blocks, closely customised to users’ requirements. However, the local approach is always tempered by the iron discipline of consumption and re-use wherever possible – the principle of open standards.

In contrast, in the context of an approach involving in-house development and maintenance, agile becomes potentially dangerous, because, untempered by the discipline of consumption and reuse, the local approach can run amok. Apps are spun up with little regard for the overarching ecosystem in which they will need to exist. 

Agile becomes the end rather than the means, and government starts to build its own proprietary equipment all over again, but this time badged as “agile”, “using open source” or even “open”.

Something bespoked in Ruby – an open-source tool - is little better than something bespoked in proprietary Visual Basic if it commits government to a lifetime of maintenance upgrades and catch-up, while outside, the world moves on at a faster pace.

My understanding of the government’s IT strategy based on open standards is emphatically the former, rather than the latter scenario.  

Departmental, in-house system integrators are, from the perspective of open standards, not much of an advance on the previous arrangement using external system integrators - both detach government from the mainstream and from the lifeblood of innovation, consumption, and commoditisation that this offers. Both are “cathedral” visions rather than “bazaar” ecosystems.

Unfortunately, for all its undeniable strengths, agile is in danger of becoming a fetish, something imbued with additional symbolic charge over and above its actual meaning.

Agile will not work for government unless it is used in conjunction with the architectural and commercial discipline of open standards. Government departments need to build capability in open standards - “the why” - every bit as much as to embed agile in their technology practices - “the how”. 

Amid the growing noise across government of crowds enthusiastically chanting “agile!”, I worry that the signal of open standards is growing fainter – and with it, the possibility of a genuine IT and service revolution.

Am I the only one?

Mark Thompson (pictured above) is a senior lecturer in information systems at Cambridge Judge Business School, a key architect of the government’s open IT strategy, and strategy director at consultancy Methods Digital

Read more on Web development