AI bias and privacy issues require more than clever tech

Bias is systemic

During a virtual panel discussion at the launch of the AI Barometer, Areeq Chowdhury, founder of WebRoots Democracy, discussed how technology inadvertently amplifies systemic discrimination. For instance, while there is a huge public debate about the accuracy rate of facial recognition systems to identify people from black and Asian minorities, the ongoing racial tension in the US has shown that the problem is wider than the actual technology.

According to Chowdhury, such systemic discrimination builds up from a collection of policies over a period of time.

The experts who took part in the CDEI analysis raised concerns about the lack of clarity over where oversight responsibility lies. “Despite AI and data being commonly used within and across sectors, it is often unclear who has formal ‘ownership’ of regulating its effects,” said the CDEI in the report.

AI needs cross-industry data regulations

Cathryn Ross, head of the Regulatory Horizon Council, who also took part in the panel discussion, said: “A biting constraint on the take-up of technology is public trust and legitimacy. Regulations can help to build public trust to enable tech innovation.”

Mirroring her remarks, fellow panellist Annemarie Naylor, director of policy and strategy at Future Care Capital, said: “Transparency has never been so important.”

The AI Barometer also reported that the experts the CDEI spoke to were concerned about low data quality, availability and infrastructure: It said: “The use of poor quality or unrepresentative data in the training of algorithms can lead to faulty or biased systems (eg diagnostic algorithms that are ineffective in identifying diseases among minority groups).

“Equally, the concentration of market power over data, the unwillingness or inability to share data (eg due to non-interoperable systems), and the difficulty of transitioning data from legacy and non-digital systems to modern applications can all stymie innovation.”

The CDEI noted that there is often disagreement among the public about how and where AI and data-driven technology should be deployed. Innovations can pose trade-offs such as between security and privacy, and between safety and free speech, which take time to work through.

However, the lockdown has shown that people are prepared to make radical changes very quickly if there are societal benefits. This has implications for data privacy policies. 

The challenge for regulators is that existing data regulations are often sector-specific. In Ross’s experience, technological innovation with AI cuts across different industry sectors. She said a fundamentally different approach that coordinated regulations was needed.

Discussing what the coronavirus has taught policy-makers and regulators about people’s attitudes to data, Ross said: “Society is prepared to take more risk for a bigger benefit, such as saving lives or reducing lockdown measures.”