ICO launches guidance on AI and data protection

AI for the ICO

What is AI, in the eyes of the ICO? “We use the umbrella term ‘AI’ because it has become a standard industry term for a range of technologies. One prominent area of AI is machine learning, which is the use of computational techniques to create (often complex) statistical models using (typically) large quantities of data. Those models can be used to make classifications or predictions about new data points. While not all AI involves ML, most of the recent interest in AI is driven by ML in some way, whether in image recognition, speech-to-text, or classifying credit risk.

“This guidance therefore focuses on the data protection challenges that ML-based AI may present, while acknowledging that other kinds of AI may give rise to other data protection challenges.”

Of particular interest to the ICO is the concept of “explainability” in AI. The guidance goes on: “in collaboration with the Alan Turing Institute we have produced guidance on how organisations can best explain their use of AI to individuals. This resulted in the Explaining decisions made with AI guidance, which was published in May 2020”.

The guidance contains commentary about the distinction between a “controller” and a “processor”. It says “organisations that determine the purposes and means of processing will be controllers regardless of how they are described in any contract about processing services”.

This could be potentially relevant to the controversy surrounding the involvement of US data analytics company Palantir’s in the NHS Data Store project, where it has been repeatedly stressed, by Palantir, that the provider is merely a processor and not a controller – which is the NHS in that contractual relationship.

Biased data

The guidance also discusses such matters as bias in data sets leading to AIs making biased decisions, and offers this advice, among other pointers: “In cases of imbalanced training data, it may be possible to balance it out by adding or removing data about under/overrepresented subsets of the population (eg adding more data points on loan applications from women).

“In cases where the training data reflects past discrimination, you could either modify the data, change the learning process, or modify the model after training”.

Simon McDougall, deputy commissioner of regulatory innovation and technology at the ICO, said of the guidance: “Understanding how to assess compliance with data protection principles can be challenging in the context of AI. From the exacerbated, and sometimes novel, security risks that come from the use of AI systems, to the potential for discrimination and bias in the data. It is hard for technology specialists and compliance experts to navigate their way to compliant and workable AI systems.  

“The guidance contains recommendations on best practice and technical measures that organisations can use to mitigate those risks caused or exacerbated by the use of this technology. It is reflective of current AI practices and is practically applicable.”