US competition watchdog issues generative AI warning

The US Federal Trade Commission (FTC) has issued a warning to companies over the use of artificial intelligence (AI) to manipulate people’s behaviour for commercial gain.

In a blog post published 1 May 2023, FTC advertising practices attorney Michael Atleson said firms are already using the new wave of generative AI tools in ways that can influence people’s beliefs, emotions and behaviour.

This includes the use of chatbots designed to provide information, advice, support or companionship, many of which Atleson says “are effectively built to persuade” by answering queries in confident language, even when those answers are complete fiction.

He added that there is also a tendency for people to trust the output of such generative AI tools, due a mixture of automation bias (whereby people are unduly trusting of a machine due to the appearance of neutrality or impartiality) and anthropomorphism (whereby the chatbot is designed to appear more human by how it uses language or emojis, for example).

“People could easily be led to think they’re conversing with something that understands them and is on their side,” said Atleson.

“Many commercial actors are interested in these generative AI tools and their built-in advantage of tapping into unearned human trust.

“Concern about their malicious use goes well beyond FTC jurisdiction,” he said. “But a key FTC concern is firms using them in ways that, deliberately or not, steer people unfairly or deceptively into harmful decisions in areas such as finances, health, education, housing and employment.”

Atleson added that any companies considering the use of generative AI in their advertising practices should be aware that design elements intended to “trick” people are a common feature of FTC cases.

He further warned that companies could start placing ads within a generative AI feature, which could also lead to deceptive or unfair practices.

“Among other things, it should always be clear that an ad is an ad, and search results or any generative AI output should distinguish clearly between what is organic and what is paid,” said Atleson.

“People should know if an AI product’s response is steering them to a particular website, service provider, or product because of a commercial relationship,” he said. “And, certainly, people should know if they’re communicating with a real person or a machine.”

Referring to news from March 2023 that Microsoft had laid off an entire team dedicated to AI ethics and safety – despite being one of the leading firms pushing generative AI through its partnership with OpenAI – Atleson noted that “it’s perhaps not the best time for firms” to be reducing the personnel working on these issues.

“If the FTC comes calling and you want to convince us that you adequately assessed risks and mitigated harms, these reductions might not be a good look,” he said. “Among other things, your risk assessment and mitigations should factor in foreseeable downstream uses and the need to train staff and contractors, as well as monitoring and addressing the actual use and impact of any tools eventually deployed.”