Extended reality can be empathic reality

Extended reality (XR) – augmented and virtual reality (AR and VR) – entered the hype stage some time in 2021. Game developers have had a field day, and industrial companies foresee a time of increased productivity, while concerned journalists fear the day that fake news might become indistinguishable from authentic footage.

But XR also opens a door to conveying hopes, fears and emotions in visceral and accessible ways. The ability to elicit empathy will not only find use in training professionals who require such skills, but also for providing activists for civil causes with new tools to create understanding and support.

Using new types of media to create awareness about social issues is not a new development. What is new is the use of AR and VR to develop narratives, whereas in the past, video games provided the closest approach to immersive story-telling. In October 2016, when the IndieCade Foundation’s IndieCade Festival in Los Angeles showcased independent video games, developers presented a range of games that highlighted social issues.

Navid Khonsari presented 1979 Revolution: Black Friday, in which players assume the role of a photojournalist during the Iranian revolution. Khonsari lived in Iran when he was a child and he even included some of his family’s home movies from that period into the video game. Cynthia Miller featured We Are Chicago, which allows players to experience challenges and social issues faced by residents of Chicago’s South Side. And Simone Castagna showcased Blue Cat, in which the player represents a cat that experiences the owner’s struggle with depression. The game attempts to make the impact of depression comprehensible for players.

Also in 2016, the organisation Animal Equality presented at the Sundance Film Festival the 360-degree experience Factory Farm, which elicits viewers’ empathy towards animals in industrial farms and slaughterhouses. And just recently, Danny Pimentel, assistant professor of immersive psychology at the University of Oregon, looked at the “effects of embodying wildlife in virtual reality on conservation behaviours” in a June 2022 article in Scientific Reports. In particular, he wanted to understand how VR could affect “compassion fade” positively for loggerhead sea turtles.

Compassion fade is the decrease in empathy that people experience as the number of affected people and animals increases. “Compassion fade can be explained, in part, by differential processing of large- versus small-scale threats,” says Pimentel. “It is difficult to form empathic connections with unfamiliar masses versus singular victims.”

The researchers provided study participants with VR headsets to experience a turtle’s journey from hatchling to adult. Participants “became” turtles, with flippers instead of arms. They encountered ships and fishing gear and had to avoid being entangled or killed. Pimentel adds: “Embodiment of non-human bodies is a powerful tool that environmental storytellers can use.” The experience increased participants’ compassion for the animals.

It is only natural for these new capabilities of XR to find use to familiarise users with cultural and societal contexts. In 2015, Robert Hernandez, a professor at USC Annenberg, created Jovrnalism to give students the opportunity to tell stories with the new technology in immersive ways. For instance, stories about South Korea allow viewers to become part of the culture.

Understanding others’ fears, needs and hopes

The group also uses AR and VR to enable users to understand others’ fears, needs and hopes in visceral ways. For instance, a recent project collected experiences from witnesses of the 1992 Los Angeles riots and then used XR technologies to create stories in remembrance of the riots’ 30th anniversary.

In another project from 2018, a group of students talked to homeless people and shot AR footage to convey their living conditions. For example, Jennifer Flores, who lives in Los Angeles’s Echo Park, guides viewers around her tent. While doing so, viewers start to understand the challenges she is facing on a daily basis, such as keeping food cold and cell phones charged without access to electricity.

Governmental authorities also see advantages of XR in training personnel with a particular focus on emotional aspects. For example, the US Army is using photorealistic 3D avatars to convey the impact of hazing and sexual assault in the army. Voice recognition allows session participants to interact with captured conversations of real victims that avatars visualise. The Digital Survivor of Sexual Assault project uses the latest technology to educate and bring awareness of the horrific realities of sexual assault to US Army personnel.

The University of Southern California’s USC Shoah Foundation has a similar project to archive interviews with Holocaust survivors to carry on their stories. The foundation, which movie director Steven Spielberg founded in 1994, is nowadays capturing the narratives in 3D representations to elicit closeness between interviewees and the audience that aims to understand the horrors survivors encountered and the feelings they had and still have.

US police departments have also started using VR as a teaching tool, enabling officers to train their behaviour in dangerous situations safely. Cost advantages are another benefit of the technology, but VR also can help officers to experience interactions with citizens who are confused or frustrated or with victims who are in pain or feel loss.

A recent webinar, Beyond shoot-don’t shoot: the role of VR in police training and community policing, put participants into a variety of situations to make it easier for them to understand people’s behaviour. One of the presenters, Anna Queiroz, associate professor at the Universidade de São Paulo in Brazil, described the benefits of VR in helping officers to understand the situations homeless people face.

Police chief David Lash shared his first VR experience of walking across a plank that was 50 storeys high. He noted that his fear of heights transferred into the virtual environment, allowing him to understand how he would react in such a situation. VR enables users to repeat the same situation or adjust situations to learn how to correct their mistakes over time, but it also gives them a better idea what others might feel or go through in certain circumstances or situations.

For a long time, law enforcement has faced the issue of reacting appropriately to varying situations. Misconduct and violent behaviour of officers is a recurring theme in the media and in courts. But prospective business managers also can learn to understand better the potential impact they could have on segments of the population. Vijay Govindarajan, professor of international business at Dartmouth College Tuck School of Business, says: “The best and brightest leaders of today are those who are going to build an inclusive, responsible and compassionate capitalistic society.”

Govindarajan used VR to help his students understand the need to embed social thinking into their approach to business. He adds: “Humanising business means understanding that the seven billion people on planet Earth have the same needs and wants. Yet some people’s needs and wants are met, while others’ are not.”

He employed the technology to look at health and wellness challenges faced by Indian families who live below the poverty line. The goal was to enable his students to develop solutions to help this consumer group. More than 30 movies in VR360, as well as regular formats, enabled students to become part of Indian families’ environment.

Empathy training also plays a role in other professions, such as for medical nurses. Although nurses constantly encounter emotionally challenging situations, practitioners find that there is a difference between sympathy and empathy. Sympathy is the act of feeling sorry for a medical situation or outcome; empathy allows nurses to embrace what patients are feeling.

Sympathy and empathy

Whereas sympathy represents acknowledgment of a medical condition and is part of nurses’ professional conduct, empathy often requires training, “and that is what we are doing with virtual reality”, says Lynsey Steinberg, board-certified medical illustrator at Augusta University.  Elena Prendergast, assistant professor of nursing at Augusta University College of Nursing, adds: “What we felt was the most important thing was providing a safe environment where the students could be immersed in an end-of-life experience, but in a safe place.”

The training focused not only on the patients themselves, but also on family members who are trying to cope with their impending loss.

XR can find use in eliciting, even training, empathy. Many applications will benefit from features that can create deep immersive experiences. But the field is at an early stage and many questions exist. Practitioners distinguish between emotional, cognitive and compassionate empathy. There is even a distinction between taking the perspective of “being someone” and “being in someone else’s shoes.”  

Some researchers argue that VR can only find use in improving some types of empathy, but not others. Therefore, it is crucial to understand what kind of empathy is the focus of an application and how to measure improvements. Then, developers of VR applications will have to take into account the type of VR equipment and the degrees of immersion.

For example, creating virtual situations that are too realistic can actually induce stress in users of the application. So questions remain – and some concerns exist. But virtual environments are a promising approach to elicit and improve understanding of other people’s challenges and obstacles.

Martin Schwirn is the author of Small Data, Big Disruptions: How to Spot Signals of Change and Manage Uncertainty (ISBN 9781632651921). He is also senior adviser, strategic foresight at Business Finland, helping startups and incumbents to find their position in tomorrow’s marketplace.