Test police algorithms in court rather than legislate, Lords told

A conflict of interest between police and suppliers

According to the Court of Appeals judgment in the SWP case – which was handed down in August 2020 – the decision was made on the grounds that the force’s use of the technology was “not in accordance” with the claimants Article 8 privacy rights; that it did not conduct an appropriate Data Protection Impact Assessment (DPIA); and that it did not comply with its Public Sector Equality Duty (PSED) to consider how its policies and practices could be discriminatory.

In terms of the discriminatory impact of the technology, the ruling stated the “SWP have never sought to satisfy themselves, either directly or by way of independent verification, that the software program in this case does not have an unacceptable bias on grounds of race or sex”.

It added: “For reasons of commercial confidentiality, the manufacturer is not prepared to divulge the details so that it could be tested. That may be understandable but, in our view, it does not enable a public authority to discharge its own, non-delegable, duty under section 149.”

Addressing the HAJC in November 2021, the national policing chief scientific adviser at the National Police Chief’s Council (NPCC), Paul Taylor, highlighted the conflicting interests between police forces and their technology suppliers.

“To be able to sufficiently scrutinise these new technologies and all the permutations, to be able to 100% tell you what they are and aren’t doing, requires a level of resource that we simply do not have – the amount of testing and testing and testing one would need is just not there,” he said.

“If I turn around to industry and say, ‘I’m going to hold you responsible’, they will then want to do a level of testing that probably makes investment into that technology not viable anymore; not an interesting product for them to engage into.

“There’s a tension there [because] we don’t want to stifle the market by saying that you have to do all of this [testing], but equally, of course, we need them to do it, and so that’s a real challenge for us,” said Taylor.

He added that part of the problem is that police are often taking a fairly mature technology and trying to implement it in a policing context it was not explicitly designed for.

Malthouse’s view that it would be preferable to test new technologies through the courts also stands in contrast to the view of former biometrics commissioner, Paul Wiles, who told the House of Commons Science and Technology Committee in July 2021 that while there was currently a “general legal framework” governing the use of biometric technologies, their pervasive nature and rapid proliferation meant a more explicit legal framework was needed.

Wiles said the current framework governing their use had “not kept up with the development of new biometrics” and nor had “the government responded to judgments by both domestic courts and the European Court of Human Rights about the inadequacy of that current framework”.

During that same session, Malthouse commented on whether the government should legislate specifically on biometrics: “Obviously there is a framework at the moment, and that’s been adduced through the courts, but as technology advances we would like to get to a position where both the police and the public can be confident about the legislative architecture that enables the adoption of future technology.

“Whether that is required through legislation or not is a matter of a discussion, but we’ve got a manifesto commitment, so no doubt we’ll be bringing forward plans before the next election.”

A questionable legal basis

Speaking to the HAJC in October 2021, Karen Yeung – an Interdisciplinary Professorial Fellow in Law, Ethics and Informatics at Birmingham Law School – said a key issue with police deployments of new technologies is that authorities have started using them “just because we can … without clear evidence” about their efficacy or impacts.

On the trials of LFR tech conducted by the MPS specifically, Yeung said the force’s scientific methodology was “very unrigorous”, noting that because procedures were tweaked every time a trial was conducted, “we do not have a stable and rigorous set of data on the basis of these experiments”.

She added: “In those 11 trials, 500,000 faces were scanned to produce nine to 10 arrests, and many of those were individuals who were wanted for very trivial offences. All of this means the real-time location tracking of many, many hundreds of thousands of British people going about their lawful business, not bothering anyone.”

As with facial-recognition tech, Yeung said the development of crime prediction tools has been equally unrigorous, with historic arrest data being used a proxy for who is likely to commit a crime.

In response to the Science and Technology Committee’s July 2019 report, which called for a moratorium on police use of LFR until a proper legal framework was in place, the government claimed in March 2021 – after a delay of nearly two years – that there was “already a comprehensive legal framework for the management of biometrics, including facial recognition”.

The government said this framework included police common law powers to prevent and detect crime, the Data Protection Act 2018 (DPA), the Human Rights Act 1998, the Equality Act 2010, the Police and Criminal Evidence Act 1984 (PACE), the Protection of Freedoms Act 2012 (POFA), and police forces’ own published policies.