Government opens £2bn chest for big data analytics

The Crown Commercial Service (CCS) has earmarked up to £2bn in potential spending for big data storage and analytics projects, by government departments, over a four-year period, from 2022 to 2026.

The early notification notice published by the CCS said the contract for a procurement framework will run from 6 April 2022, and will provide “big data and analytics” for central government departments and the rest of the public sector, including local authorities, health, police, fire and rescue, education and the devolved administrations.

The notice mentions the National Data Strategy’s “requirement to access and interrogate government data more effectively to improve public services” as a backdrop to the framework.

The stated intention of the framework, which has been developed by the CCS and HMRC, is to provide a “range of buying options and pricing mechanisms [enabling] competitive and agile procurements across a specialist pool of suppliers.

It is divided into two “lots”, one for services and one for software. Lot 1 includes “cognitive solutions” under the heading of services, and Lot 2 includes machine learning as well as reporting and analytics, and so goes deeper into artificial intelligence (AI) software than “analytics” would suggest.

The National Data Strategy itself has been long in gestation. Its core strategy document has had a long life on the Gov.uk website, reaching back into 2019. It can fairly be described as a “work in progress”.

Its latest iteration, published on 18 May 2021, reiterated the strategy having four pillars, described as: data foundations, data skills, data availability, and responsible data use.

Above the pillars are five missions: to unlock the value of data held across the economy, secure a pro-growth and trusted data regime, transform government’s use of data to drive efficiency and improve public services, ensure the security and resilience of the infrastructure on which data use relies, and champion the international flow of data. 

In June 2021, the Crown Commercial Service and HMRC ran a webinar for prospective suppliers to the big data and analytics framework, hosted by TechUK. 

On that webinar, Iain Smith, data and analytics lead at HMRC, said they are seeing a need for around £58m per annum in professional services in data analytics. They also spend around £14m each year on software. He said they want the framework “to be flexible enough to go to the market” and to find specialist provision innovation in data analytics that will be directional over a five-year period.

Calvin Jepson, head of data platform services at CCS, said: “This is not about one-off delivery, but about services and products that have a wider re-use and are built to evolve and to ensure we bring a regular pace of innovation.”

On the “build and run” service provision side (Lot 1) he mentioned, as examples, moving data from a third party into a data lake or data warehouse in a safe and consistent manner, building data quality rules, and working to deliver data virtualisation.

On the commercial off-the-shelf software side, he mentioned security and privacy software used for data obfuscation and “machine learning services from particular cloud providers”, as examples.

Alistair Flockhart, commercial lead at HMRC, speaking in respect of knowledge transfer, said to potential suppliers to the framework: “We need you to help us learn, develop and change the ways we operate. We will be looking for suppliers who can collaborate with our DevSecOps and DataOps teams.”

A form enabling suppliers to bid for the work will be available from 8 October.

A CCS spokesperson has contacted Computer Weekly and said: “no spending is ever guaranteed through procurement frameworks. The £2bn figure represents the maximum that could be spent through this agreement."