Why responsible AI is a business imperative

Start with principles

Responsible AI is both a business imperative and just good business, suggests Diya Wynn, responsible AI lead at AWS. She prefers the term to AI ethics because it broadens the conversation from moral connotations to the security, privacy and compliance perspectives organisations will need to address risks and unintended impacts.

The good news is that most companies with compliance teams in place for GDPR already have many of the structures necessary for AI governance, although you may need to add ethical expertise to the technical capabilities of data science teams.

Responsible AI is about quality, safety, fairness and reliability. To deliver that, Purcell urges organisations to start with a set of ethical AI principles covering accountability, competence, dependability, empathy, factual consistency, integrity and transparency that articulate corporate culture and values.

That will help when AI optimisations expose tensions between different business teams, such as AI lending tools improving sales by offering loans to riskier applicants, and give you a strong basis for adopting effective controls using tools increasingly designed for business leaders as well as data scientists. “AI ethics is a human discipline, not a technology category,” warns Buytendijk.

The metrics businesses care about when deploying AI systems are rarely the purely technical measures of machine learning accuracy in many early responsible AI tools. Instead, they are more technosocial measurements such as getting the right balance of productivity improvements, customer satisfaction and return on investment.

Generative AI chatbots that close calls faster and avoid escalations may not reduce the overall time human agents spend on the phone if it’s simple cases they handle quickly, leaving agents to deal with complex queries that take more time and expertise to address – but also matter most for high-value customer loyalty. Equally, time saved is irrelevant if customers give up and go to another vendor because the chatbot just gets in the way.

Many organisations want custom measures of how their AI system treats users, says Mehrnoosh Sameki, responsible AI tools lead for Azure: “Maybe they want friendliness to be a metric, or the apology score. We hear a lot that folks want to see to what extent their application is polite and apologetic, and a lot of customers want to understand the ‘emotional intelligence’ of their models.”

Emerging AI governance tools from vendors such as Microsoft, Google, Salesforce and AWS (which worked with Accenture on its Responsible Artificial Intelligence Platform) cover multiple stages of a responsible AI process: from picking models using model cards and transparency notes that cover the capabilities and risks, to creating input and output guardrails, grounding, managing user experience and monitoring production systems.

Gateways and guardrails for genAI

There are unique risks with generative AI models, over and above issues of fairness, transparency and bias that can occur with more traditional machine learning, requiring layered mitigations.

Input guardrails help keep an AI tool on topic: for example, a service agent that can process refunds and answer questions about the status of orders but passes any other queries to a human. That improves the accuracy of responses, which is good for both customer service and business reputation.

It also keeps costs down by avoiding expensive multi-turn conversations that could be either an attempted jailbreak or a frustrated customer trying to fix a problem the tool can’t help with. That becomes even more important as organisations start to deploy agents that can take action rather than just give answers.

Guardrails also address compliance concerns like making sure personally identifiable information (PII) and confidential information are never sent to the model, but it’s the output guardrails where ethical decisions may be most important, avoiding toxic or inaccurate responses as well as copyrighted material.

Azure AI Content Safety handles both, with new options added recently, Sameki explains. “It can filter for a lot of risky content, like sexual, hateful, harmful, violent content. It can detect that someone is attempting a jailbreak, either directly or indirectly, and filter that. It could filter IP protected materials as a part of the response. It could even see that your response is containing ungrounded, hallucinated content, and rewrite it for you.”

Hallucinations are probably the best known issue with generative AI: where the model accidentally misuses data or generates content that is not grounded in the context that was available to the model. But equally important, she suggests, are omissions, when the model is deliberately leaving out information in the responses.

Rather than just filtering out hallucinations, it’s better to ground the model with relevant data.

Handling hallucinations

Training data is a thorny question for generative AI, with questions over both the legality and ethics of mass scraping to create training sets for large language models. That’s a concern for employees as well as CIOs; in a recent Salesforce study, 54% said they don’t trust the data used to train their AI systems, and the majority of those who don’t trust that training data think AI doesn’t have the information to be useful and are hesitant to use it.

While we wait for legislation, licence agreements and other progress on LLM training data, organisations can do a lot to improve results by using their own data to better ground generative AI by teaching it what data to use to provide information in its answers. The most common technique for this is Retrieval Augmented Generation (RAG) although the tools often use catchier names: this is a form of metaprompting – improving the user’s original prompt, in this case by adding information from your own data sources.

While fully fine-tuning LLMs is too expensive for most organisations, you can look for models that have been tuned for specific, specialist domains using a technique called Low-Rank Adaptation (LoRA), which fine tunes a smaller number of parameters.

Collecting and using feedback is key to improving systems (on top of the usual considerations like tracking usage and costs). Content safety services from Azure and Salesforce, for example, include audit trails: if the AI system has to remove off-colour language from a prompt, you want to know that you caught it but you also want to know if it was a user expressing frustration because they’re not getting useful results, suggesting you need to give the model extra information. 

Similarly, monitoring hallucinations tells you not just the quality of AI outputs, but whether the model has retrieved the relevant documents to answer questions. Gather detailed feedback from users and you can effectively allow expert users to train the system with feedback.

Designing the user experience to make feedback natural also gives you a chance to consider how to effectively present AI information: not just making it clear what’s AI generated but also explaining how it was created or how reliable it is to avoid people accepting it without evaluating and verifying.

“This handoff between AI and people is really important, and people need to understand where they should trust the response coming from AI and where they may need to lean in and use judgement,” Goldman says.

Human-at-the-helm patterns can include what she calls “mindful friction”, such as the way demographics are never selected by default for marketing segmentation in Salesforce.

“Sometimes, when you’re creating a segment for a marketing campaign using demographic attribute,s a customer base may be perfectly appropriate. Other times, it may be an unintentional bias that limits your customer base,” Goldman adds.

Turn training into a trump card

How AI systems affect your employees is a key part of ethical AI usage. Customer support is one of the successful areas for generative AI adoption and the biggest benefits come not from replacing call centre employees, but helping them resolve difficult calls faster by using systems with RAG and fine tuning it to know the product, the documentation and the customer better, says Peter Guagenti, president of Tabnine.

“The most successful applications of this are highly trained on a company’s unique offerings, the unique relationship they have with their customers and the unique expectations of customers,” he says.

And training isn’t just important for AI systems. The European Unions (EU) AI Act will require businesses to foster “AI literacy” inside the organisation. As with other technologies, familiarity allows power users who have had specific, relevant training to get the most out of AI tools.

“The people who use the tool the most are the ones who make the best use of the tool and see the greatest benefit,” says Guagenti, and building on the automation efficiencies rather than attempting to replace users with AI will benefit the business.

“If you can coach and teach your people how to do these things, you’re going to benefit from it, and if you build an actual curriculum around it with your HR function, then you’ll become the employer of choice, because people know that these are critical skills.”

As well as making your investment in AI worthwhile by improving adoption, involving employees about “pain points they’re trying to solve” is also the best way to create effective AI tools, Goldman says.

Getting all of that right will require cooperation between business and development teams. Responsible AI operationalisation needs a governance layer and tools – such as the Azure impact assessment template or an open source generative AI governance assessment – can help to create a process where technical teams start by giving not just the business use case, but the risks that will need mitigating, so the CIO knows what to expect.

Bring in a red team to stress the models and test mitigations: the impact assessment means the CIO can clearly see if the system is ready to be approved and put into production – and monitored to see how it behaves.

A more overall assessment of what your risk exposure is from AI can pay unexpected dividends by finding places where you need to improve the way PII is treated and who has access to data, Purcell says. “One of the real benefits of this moment is using it as a catalyst to solve a lot of your existing governance problems, whether they’re specific to AI or not.”