IBM pushes boundaries of AI, but insists companies take an ethical approach

Challenging biases

“One of the concerns [with AI] is around bias – the fact that, especially in AI systems that are data-driven based on ML approaches, you can inject some bias,” says Francesca Rossi, global leader of AI ethics at IBM Research.

“If the training data is not inclusive enough, not diverse enough, then the AI system will not be fair in its decisions or recommendations for humans. You want to avoid that, so there is an issue around detecting and mitigating bias in training data, but also in models,” she says.

There are a variety of different kinds of biases. One is interaction bias, exemplified by Microsoft’s infamous chatbot Tay, which learned by observing how people interacted on Twitter. However, Twitter is not necessarily the best representation of real human interaction, meaning the bot had to be shut down within 24 hours due to the fact it began making racist statements.

Another, much more subtle kind of bias comes in the form of product recommendation, like the kind you see on Netflix or YouTube that recommends content you may be interested in based on what you have previously viewed. In these contexts the bias can be fairly innocuous, but when it comes to news or social media these AI filter bubbles become much more damaging through their creation of echo chambers.

Natural language bias, however, is the hardest type of bias to eliminate in AI.

“Certain types of bias are baked into language,” says Barry O’Sullivan, president of the European Artificial Intelligence Association. “If you look at the proximity of certain noun phrases to other noun phrases, in many languages you’ll find nouns that refer to authority positions, like president or leader or director, are very close in text with gender nouns, and that’s not the case for females who tend to be associated maybe with caring roles or supporting roles.”

According to Rossi, the intersectionality of bias means it may not be possible to completely eliminate bias. “You don’t want to be biased on various protected variables, like race and gender and age, but then by removing maybe one thing you might introduce a little more bias on another because they intersect with one another,” she says.

O’Sullivan adds that bias is ultimately a question of consensus. “We have to think very carefully about how we want to deal with bias because there are questions that we often can’t agree on, such as whether something has a bias or not. In general, the question of ethics and ethical codes ultimately comes down to whether or not there’s a consensus as to what is acceptable, and what is not,” he says.

But where does this consensus come from?

The trolley problem

The trolley problem is a theoretical question applied to self-driving cars. What it essentially boils down to is, if a crash is unavoidable and someone is going to die as a result, how does the car make that decision?

“Does it kill the young woman, or does it kill the two professional men, or does it sacrifice the two children in favour of the person who’s got world-leading expertise in neurosurgery?” says O’Sullivan.

“There is no consensus around what ethical principles should be because they’re very much dependent on societal norms. That’s really challenging for AI researchers”

A study published by Nature, called the Moral Machine experiment, asked 2.3 million people across the globe what they would like the autonomous vehicle to do in this life-or-death situation. It found that people’s ethical values and preferences differed wildly by culture and geographic location.

“There is actually no consensus around what ethical principles should be because they’re very much dependent on societal norms,” says O’Sullivan. “That’s really challenging for AI researchers.”

Ultimately, a self-driving car will never be designed specifically to kill someone, so although the life-or-death choice is an edge case, statistical versions of the trolley problem are likely to occur in the real world.

IBM’s Rossi describes a scenario where you are in a self-driving car travelling in the same lane as a truck travelling in the opposite direction towards your car, while there is also a bicycle travelling parallel to you. The choices are to move closer to the bike to create some space between you and the truck, but in doing so risk hitting the cyclist, or move to the other side which would mean less space between you and the truck. “It’s a matter of increasing one risk over another,” she says.

One solution to overcoming disagreements in ethical opinion is in the transparency of the technology – if we cannot agree on what is ethical, we at least need to know how the system is reaching its conclusions.

Building trust through transparency

To embed this transparency, IBM is proposing the use of factsheets for AI services, which it describes as being similar to food labels in the sense that, when you look at it, you can see the data it was trained on, who trained it, when it was trained, and so on.

The factsheet would be comprised of four main pillars: fairness – using data and models free of bias; robustness – making systems secure; explainability – people need to know what is going on inside the black box; and lineage – providing details of development, deployment and maintenance so the system can be audited throughout its lifecycle.

“We think every AI system, when it’s delivered, should be accompanied by something that describes all the design choices and also how we took care of bias, explainability, and so on,” says Rossi. “We think trust in the tech is achieved by being very transparent, not just about the data policy, but also very transparent on the design choices.”

IBM’s Project Debater, for example, is the first AI system to engage in a live public debate with humans, which it is able to do through unscripted reasoning.

“In this setup, we believe humans still prevail,” says Noam Slonim, the principle investigator for Project Debater. “Humans rely more on rhetoric, and when we measure it they usually deliver speech better than the system. On the other hand, the system advantage is usually reflected by the fact it can pinpoint high-quality evidence to support its case.”

“Trust in the tech is not enough. You also want the final users to trust whoever produces that technology. You need to build a system of corporate responsibility”

Due to AI’s ability to search huge amounts of data very quickly, there is potential for Project Debater to be used in assisting human decisions, as it can quickly identify facts and evidence, both in support and opposition to the arguments presented.

The more transparent and explainable the system is, the more those using it for these purposes will be able to trust the information it provides.

Building trust can also be helped if companies act responsibly with the powerful technologies in their hands. This is especially true in the context of rapid technological advancement, according to O’Sullivan. “Just because we can do something with AI, doesn’t mean we should go and do it,” he says.

“Humans ultimately need to be responsible for AI systems. An AI system should never be used as a way of a human being relinquishing or displacing his or her responsibility for taking a decision, so if an AI system is under your control, you are responsible,” adds O’Sullivan.

“Trust in the tech is not enough. You also want the final users to trust whoever produces that technology,” says Rossi. “You need to build a system of corporate responsibility.”

One way IBM is trying to do this is through collaborative initiatives with international bodies, national governments and other organisations developing AI technologies.

“You want to hear the voice of everybody, not just those who produce the AI system, but those communities that are affected by the AI,” concludes Rossi.