Why diversity in AI remains a challenge and how to fix it

Each year that passes sees artificial intelligence (AI) integrating even further into major institutions. It is set to contribute $15.7tn to the global economy by 2030, which exceeds the current output of China and India combined. This game-changing technology is becoming increasingly embedded in our daily lives, from the devices on our bodies that measure our heart rates to those in our home that play our music.

With AI poised to play an increasingly important role in society, further regulations are likely to be introduced worldwide. Over the past five years, more than 60 countries have developed over 700 AI policy initiatives. Last year, the European Commission introduced its Proposal for a Regulation on a European Approach for Artificial Intelligence (also known as the Artificial Intelligence Act), designed to ban harmful uses of AI and foster more consumer transparency in AI-driven systems and applications.

A particularly challenging issue in AI is the diversity of those programming the technology itself. The World Economic Forum has shown that about 78% of global professionals with AI skills are male, while a study by the AI Now Institute of New York University showed that over 80% of AI professors are men.

Women make up only 15% of AI researchers working at Facebook and 10% of AI researchers working at Google. Also, only 2.5% of Google’s entire workforce and 4% of Facebook’s and Microsoft’s are black. These figures need to change – and quickly.

Beyond the moral imperative of preventing systemic racial and gender discrimination in designing new AI tools, there is also an economic one. Research has demonstrated that companies in the top quartile for gender diversity have been 21% more likely to experience above-average profitability, while ethnic and cultural diversity correlates with a 33% increase in performance.

Different viewpoints optimise AI development when it comes to explainability. This is the ability to explain the predictions made by a machine learning model: being able to understand why a model adopted a particular choice and representing it in a manner that humans can follow. Data cannot simply be fed into a learning system. Instead, the AI system needs to be designed not to omit data simply because it seems atypical based on a few data points.

A diverse AI workforce made up of different races, genders, ethnicities and ages will decrease the likelihood of racial, gender, ethnic and age discrimination by artificially intelligent systems. So how can we address the issue?

Addressing gender diversity starts with attracting more women into the industry as early as possible and helping more female teenagers to consider a career in AI while at secondary school. In 2020, a mere 0.4% of girls in the UK took computer science at A-level and were already under-represented when they reached university. Only 9% of female graduates in 2018 studied a core STEM (science, technology, engineering and maths) subject, arguably due to a lack of female role models in tech and misunderstandings about what a career in tech entails.

Beyond education, the next step would be to ensure that job descriptions are as inclusive as possible. Research has shown that male-gendered words can affect the willingness of women to apply for a role. Masculine wording could perpetuate a perceived bias towards men, with 44% of women reportedly being discouraged from applying if a role or workplace was described as “aggressive” in a job description.

Once more women are in AI education and employment, it is crucial to foster a culture in which everyone feels included, respected and valued. Culture is an enormous part of why women leave tech, and changing that culture can start with prioritising and developing robust mentoring programmes.

As Jim Boerkoel, co-chair of an undergraduate consortium designed to cultivate future AI researchers has pointed out, two of the most critical levers to fostering diversity in AI are mentoring and cohort-building. These give peers meaningful connections to help their careers evolve and can link them to mentors who make relevant connections and introductions throughout a career.

Mentorship has been demonstrated to increase minority representation at management level from 9% to 24%. The benefits of mentorships also include raising the promotion and retention rates for minorities and women from 15% to 38% compared with employees who were not mentored. Industry leaders must create partnerships with academic institutions to build these mentorship networks early in the career process.

As industry leaders, we must deal with the diversity issues in AI not by zeroing in on a single controversy or debate, but by taking a holistic view of the broader picture, welcoming creative solutions and genuine accountability for the generations to come.

Shawn Tan is CEO at Skymind, a company that focuses on AI ecosystems