Information in the metaverse is A Tale of Two Cities

Filter bubbles

There is the real danger that the metaverse will intensify already existing segmentation of public opinions. The separation of societies into so-called “filter bubbles” is seen as an increasing problem. Filter bubbles emerge when algorithms cater to individuals’ interests so effectively that topics or opinions outside of their field of immediate interest are no longer presented to these individuals. In the metaverse, entire environments will cater to one-sided and extreme views.

Such filter bubbles are information pockets that separate, and separated, segments of the population occupy – segments that will not take into account conflicting information or conflicting points of view. In effect, various groups of people live in differently perceived realities.

The issue is far beyond anecdotal. In December 2021, Public Agenda USA Today published a report headed “America’s hidden common ground: putting partisan animosity in perspective”. The report identified reasons for hope that Americans will be able to bridge some of their conflicts, but also showed that “only 9% of Americans think that political hostility and divisiveness between ordinary Americans will decrease in the next 10 years”. Instead, a 42% plurality think it will increase.

Finding common ground will become an increasingly challenging task. Unfortunately, the current Covid-19 vaccination debate provides us with a glimpse into how civil discourse can veer off into creating insurmountable walls of disagreement. The QAnon movement highlights the dangers that can emerge with more extreme points of view. Worse, what happens with information when such movements interact? A 2021 Rolling Stone article said: “What happens when a global pandemic, a vaccine-resistance movement and the age of conspiracy collide? A black hole of misinformation that poses a grave threat to public health.”

The disinformation dozen

It is worthwhile revisiting terms to ensure that different aspects of information threats are appropriately taken into consideration. Misinformation is false and incorrect information that a user might believe to be correct and is spread without the intent to deceive. Disinformation is information that is intended to mislead receivers and is spread to achieve desired outcomes. Intent matters, but the results can be indistinguishable.

Speaking to the power of social networks – and concerns that policymakers and civil rights groups have – only a small number of individuals can have tremendous impact on social networks and skew agendas and perceived majority opinions. In May 2021, researchers at the Centre for Countering Digital Hate published The disinformation dozen, a study that showed that just a dozen individuals accounted for the majority of misleading posts and disinformation about Covid vaccinations across the major social networks.

The infamous dozen individuals had a very large number of followers and created a substantial amount of anti-vaccination disinformation. “Analysis of a sample of anti-vaccine content that was shared or posted on Facebook and Twitter a total of 812,000 times between 1 February and 16 March 2021 shows that 65% of anti-vaccine content is attributable to the disinformation dozen,” said the study.

Unrelated, but acknowledging the issue and even referring to the disinformation dozen, in a July 2021 briefing by US press secretary Jen Psaki and US surgeon general Vivek H Murthy, government officials highlighted that social media providers should be held responsible for the misinformation on their networks  and suggested steps to address the spreading of false information.

To combat misinformation, the US government released Confronting health misinformation, which said: “Health misinformation is a serious threat to public health. It can cause confusion, sow mistrust, harm people’s health and undermine public health efforts. Limiting the spread of health misinformation is a moral and civic imperative that will require a whole-of-society effort.”

The document’s focus is health-related, but the issues and concerns reach every topic of political and civil importance.

State actor fakes

Perhaps naturally, professional efforts also try to sway attitudes and to establish specific views to affect public opinion. In September 2020, an exit memo by former Facebook data scientist Sophie Zhang brought to light the existence of fake accounts that were used globally to influence elections. In many countries – such as Afghanistan, India, Mexico and South Korea – such practices helped politicians to affect public opinion.

In July 2021, a Reuters investigation looked at Force 47, Vietnam’s military information warfare unit. The unit has thousands of soldiers tasked with developing, moderating and posting on Facebook groups that are government friendly. The investigation found that “in Vietnam, social media ‘influencers’ are more likely to be soldiers than celebrities”.

And in August 2021, the Centre for Information Resilience (CIR) released Analysis of the pro-China propaganda network targeting international narratives. The CIR found that “a coordinated influence operation on Twitter, Facebook and YouTube is using a mix of fake and repurposed accounts to push pro-China narratives and distort perceptions on important issues”. The accounts also feature problematic commentary on politics and events in the US.

Then there is the issue of deep fake and synthetic media, artificial intelligence (AI)-generated visuals that appear to be authentic photographic or film representations – the concept also relates to soundscapes or voice manipulation – but in fact are modified and changed media to potentially influence public opinion and voters. For instance, the CIR found in its investigation that many profile pictures on Twitter accounts were machine learning-generated user images.

The topic rose to the surface when a deep fake apparently showed US Speaker of the House Nancy Pelosi giving a speech while inebriated, leading Facebook to release a policy banning deep fake videos from its platform. The implications are concerning because, as the Brookings Institutions found in a January 2020 article: “If AI is reaching the point where it will be virtually impossible to detect audio and video representations of people saying things they never said, seeing will no longer be believing.”

The situation will get worse for two reasons. First, according to Sensity’s report The state of deepfakes 2020s, “non-consensual and harmful deep fake videos crafted by expert creators are now doubling roughly every six months – the number of deep fake videos detected up to December 2020 amounts to 85,047”. Second, there is a “growing gap between the capabilities to make deep fakes, the opportunities to claim a real video is a deep fake, and our ability to challenge that”, as Sam Gregory, programme director of Witness, explains.

Although spoofing is a serious consideration in the current communication landscape, the metaverse creates an entirely new range of criminal opportunities

There is no doubt that the virtual and augmented realities (VR/AR) in the metaverse cater to the use of deep fake and synthetic media, in part because that is what virtual elements in fact are. Also, the promise of immersive and experiential environments could create convincing realities that many users will perceive as authentic, because seeing and hearing is believing – still.

Also, although spoofing is a serious consideration in the current communication landscape, the metaverse creates an entirely new range of criminal opportunities. In spoofing attacks, perpetrators trick victims by pretending to be somebody else or by falsifying data. In the metaverse, entire environments could become what might be called “reality spoofs” – representations of legitimate companies, shops or marketplaces to extract money or gain access to valuable data and information.

The use of avatars that pretend to represent somebody trusted or trustworthy can enhance the perception of engaging with legitimate company representatives or service providers.

But it is not all doom and gloom. Clearly, the metaverse should be monitored closely to identify criminal activities early on and to rein in users’ worst instincts as effectively as possible. But there is another, very different, side to the metaverse – a side that can find use in doing good and supporting worthwhile social causes. This is where the metaverse can have a genuinely beneficial impact.

Virtual or augmented realities – depending on the applications we are considering – provide users with an immersive environment that can make experiential exposure to history, information and circumstances so much more visceral than any other existing means of transferring information. Education, communication and comprehension of existing issues or new developments can take on tangible qualities.

Users then can connect very intuitively and emotionally. Join Martin Luther King Jr in the Selma to Montgomery marches. Or descend with Neil Armstrong and Buzz Aldrin to the lunar surface during the Apollo 11 mission. The step from learning to understanding will be much more natural than with other media – at least in theory.

Connecting the disenfranchised

The internet has allowed disenfranchised individuals to connect with others who have experienced the same fate, background or experiences. Such groups then can become powerful support lines for people who might not be able to share their opinions and lives with others in their immediate surroundings. The metaverse will enable new pathways to connect with like-minded people and access news that might otherwise be censored, for instance.

For example, the organisation Reporters Without Borders has created The Uncensored library in the sandbox game Minecraft, which also features VR-focused environments. The library is meant as a “safe space for journalists” and is housed within the virtual representation of a library building, featuring articles and books for readers in oppressive countries that censor certain types of information. The metaverse can open doors to empathy, understanding and knowledge for people who might not be able to find such doors in the real world.

Since 2003, non-profit organisation CyArk has been trying to “digitally record, archive and share the world’s most significant cultural heritage and ensure that these places continue to inspire wonder and curiosity for decades to come”. The organisation is scanning, capturing and archiving cultural heritage sites all over the world, including Mount Rushmore in the US, Nineveh in Iraq, and the statues on Easter Island. The goal is to make sure that posterity will have access to these sites, many of which are endangered by natural disasters, terrorism, wars, and so on. The digital information then is used to create virtual representations for educational purposes. In this way, the metaverse could become a library of real-world sites, buildings and artwork.

Coincidentally, the Fraunhofer Society in Germany has a group, CultLab3D, which is working on similar efforts to “protect and preserve cultural heritage”. The group has created an automated 3D scanning system to capture artwork in museums and buildings for virtual applications. The group also employs its technology for industrial applications. The technology makes art tangible in schools and homes. The metaverse could provide the exhibition space to catalogue and display the art, the buildings, and the natural wonders of the world.

Raising virtual awareness

But static art and sites are only the beginning in an interactive, immersive and multimodal virtual landscape. As VR technologies become increasingly available, new messaging efforts have the potential to become increasingly captivating through the use of 3D movies and sound. New Reality Co developed the VR film Tree, which transforms the viewer into a rainforest tree: “With your arms as branches and your body as the trunk, you’ll experience the tree’s growth from a seedling into its fullest form and witness its fate first-hand.” Tree is drawing attention to the deforestation of tropical regions.

VR can also help to create empathy for others and their plights. Another VR experience by New Reality makes viscerally accessible the struggles of two parents to distract their young daughter in a war zone as bomb blasts come closer. Giant, which was inspired by real events, “transports the viewer into the family’s makeshift basement shelter”.

Organisations have gone to extraordinary lengths to make others’ struggles palpable in order to elicit understanding and compassion – and thereby, hopefully, support – for these groups. For instance, faced with the challenges of helping refugees attempting to cross the Mediterranean Sea, sea-rescue non-profit Sea-Watch implemented a novel messaging approach.

Every year, thousands of refugees die attempting the crossing. To raise awareness, Sea-Watch staged a mock sea passage with 40 volunteers in a lifeboat to simulate the experience that refugees face. The simulation, which took place in a training facility, used a cramped inflatable boat, hours of rocking motions, and lights and sounds to emulate a night-time crossing of the Mediterranean Sea.

Sea-Watch shot footage of the event and released a nine-minute documentary. The emotional and physical torment the volunteers experienced during the simulation is a potentially powerful tool for creating an immersive experience, and the video can serve as a story that people can forward to friends and colleagues. Lifeboat – the experiment won a 2020 Cresta International Advertising Award.

Imagine the effect when such purposeful messaging will be moving to virtual environments that provide a 360-degree, immersive experience with 3D sound-effects

No doubt, for participants of the simulation, the experience was physically immersive. For viewers, the footage is powerful but passive and two-dimensional. Imagine the effect when such purposeful messaging will be moving to virtual environments that provide a 360-degree, immersive experience with 3D sound-effects, potentially enhanced through novel interfaces that are becoming increasingly common, such as haptic vests. Messaging of social causes can envelop the target audience – literally.

Other public efforts could also benefit by the capabilities that advanced virtual worlds will be able to provide. Some law-enforcement agencies, for example, are looking at new ways to encourage the public’s participation in solving crimes. The hope is that intuitive information could create a close bond between victims and the public and that this bond could lead to the general public’s participation.

In 2019, detectives from the Newport Beach Police Department employed a novel outreach method to revive interest in a murder cold case. The police used Twitter’s social media platform to live-tweet a detailed first-person account (from the victim’s perspective) of the hours leading up to the murder in the hope of raising awareness about the cold case and uncovering new clues. Ultimately, the police identified and arrested a suspect because of a break in the case unrelated to the Twitter effort.

Although the police’s creative use of Twitter and storytelling was not responsible for solving this case, it represents a novel way to immerse audiences in a victim’s narrative and experiences and could even revive in some audiences useful memories from periods when certain crimes occurred.

Now, imagine how such a story would unfold in the metaverse, with accompanying soundscapes and streets with features and cars of the period – audiences might even be able to add information that could help others to get an even more detailed account of the last days of a victim in the hope of surfacing memories that might help law enforcement in their investigations.

Next step in information technologies

Many examples also exist of how governments are trying to sway and nudge citizens toward desired behaviours – such as hundreds of public service announcements for health-related, accident-avoidance or general safety causes. But targets of such campaigns often have no personal or direct experience with the issues the messages are about. In such situations, creating a meaningful, perhaps even visceral, connection between message and receiver can be challenging.

New approaches and experiments aim to create immersive messages that can help people understand social causes or improve the communication of relevant key elements of a campaign or initiative. And extended-reality applications and metaverse environments will see use in establishing an interactive backdrop for the delivery of such messages.

Advances in AR and VR – if not the development of a comprehensive metaverse – will enable new forms of communication and collaboration. Messages will become more accessible, distribution of news and opinions will become increasingly easy, and opportunities to misuse new-found capabilities will proliferate. But before condemning the metaverse’s problematic aspects, consider one thing: the invention of book printing enabled fast and inexpensive dissemination of false information and propaganda too, but most people would agree that this invention advanced societies and opened knowledge to the masses.

The metaverse constitutes the next step in information technologies. Policymakers and platform owners will have to find ways to rein in the worst abuses and users need to become more proficient in understanding how to judge and assess information and news. But the potential to create immersive and convincing stories and narratives opens completely new pathways to elicit understanding, address injustices, and open productive collaboration and opinion discourse.

Martin Schwirn is the author of Small data, big disruptions: How to spot signals of change and manage uncertainty. He is also senior advisor, strategic foresight at Business Finland, helping startups and incumbents to find their position in tomorrow’s marketplace