Should bad science be taken off social media?

How do you solve a problem like bad information?

When it comes to understanding science and making health decisions, it can have life-or-death consequences.

People dissuaded from taking vaccines as a result of reading misleading information online have ended up in hospital or even died.

And inaccurate or completely made-up claims about 5G and the origins of Covid-19 have been linked to violence and vandalism.

But completely removing information can look a lot like censorship, especially for scientists whose careers are based on the understanding that facts can and should be disputed, and that evidence changes.

The Royal Society is the world’s oldest continuously operating scientific institution, and it is attempting to grapple with the challenges posed by our newest ways of communicating information.

In a new report, it advises against social media companies removing content that is “legal but harmful”. Instead, the report authors believe, social media sites should adjust their algorithms to prevent it going viral – and stop people making money off false claims.

“Removing content may exacerbate feelings of distrust and be exploited by others to promote misinformation content.”

But not everyone agrees with that view – especially researchers who are experts in tracking the way misinformation spreads online, and how it harms people.

The Center for Countering Digital Hate (CCDH) maintains there are cases when the best thing to do is to remove content when it is very harmful, clearly wrong and spreading very widely.

The team points to Plandemic – a video that went viral at the start of the pandemic, making dangerous and false claims designed to scare people away from effective ways of reducing harm from the virus, like vaccines and masks, and was eventually taken down.

Social media companies were better primed for the video’s sequel Plandemic 2, which fell flat after being restricted on major platforms, having nothing like the same reach as the first video.

“It’s a political question…what balance we see between individual liberties and some form of restrictions on what people can and cannot say,” says Prof Rasmus Kleis Nielsen, director of the Reuters Institute for the Study of Journalism at the University of Oxford.

Prof Nielsen acknowledges that, although it’s a relatively small part of people’s media diets, science misinformation can lead to disproportionate harm.

But, he adds, given a lack of trust in institutions is a big driver of misinformation: “I imagine that there are quite a lot of citizens who would have their worst suspicions confirmed about how society works, if established institutions took a much more hands-on role in limiting people’s access to information.”

Echoing this concern, the Royal Society says: “Removing content may exacerbate feelings of distrust and be exploited by others to promote misinformation content.” This “may cause more harm than good by driving misinformation content…towards harder-to-address corners of the internet.”

The fact that those corners are “harder to reach”, though, is arguably part of the point. It reduces the risk that someone who is not already committed to potentially harmful beliefs, and isn’t seeking them out, will be exposed to them by chance.

Some of the violent protests that were driven at least in part by conspiracies had their origin not in obscure corners of the internet, but on Facebook. And there is little clear evidence that removing content drives people further into harmful beliefs.

Scientific misinformation is nothing new.

The incorrect belief in a link between the MMR vaccine and autism came from a published (and later retracted) academic paper, while widespread unevidenced beliefs in the harm of water fluoridation were driven by the print media, campaign groups and word of mouth.

What’s changed is the speed at which false facts travel, and the huge numbers of people who can end up reading them.

Rather than removing content, one way suggested by the report’s authors of tackling misinformation is making it harder to find and share, and less likely to appear automatically on someone’s feed.

This, Prof Gina Neff, a social scientist at the Oxford Internet Institute explained, was to “ensure that people still can speak their mind” – they just aren’t guaranteed an audience of millions.

“They can still post this information, but the platforms don’t have to make it go viral.”

The Institute for Strategic Dialogue (ISD), a think tank which monitors extremism, points out a substantial proportion of misinformation relies on the appropriation and misuse of genuine data and research.

“This is sometimes more dangerous than outright false information, because it can take substantially longer to debunk by explaining how and why this is a misreading or misuse of the data,” its spokesperson says.

That’s where fact-checking comes in – another tool which the Royal Society supports.

One of the most common pieces of vaccine misinformation over the past year – which the BBC has repeatedly fact-checked – was the notion that people are being harmed in high numbers by the jab. This claim is based on a misinterpretation of real figures.

The ISD says research has shown that a small group of accounts spreading misinformation had a “disproportionate influence on the public debate across social media”.

“Many of these accounts have been labelled by fact-checkers as sharing false or misleading content on multiple occasions, yet remain live.”

The Royal Society did not investigate removing the accounts of “influencers” who are especially prolific spreaders of harmful misinformation.

But this is seen as an important tool by many disinformation experts, and research into ISIS and the far-right suggests it can be successful.

When David Icke, a prolific spreader of Covid misinformation as well as anti-Semitic conspiracy theories, was removed from YouTube, research from the CCDH found his ability to reach people was considerably reduced.

While his videos remained on alternative video-hosting platform BitChute, their views fell from 150,000 on average before the YouTube ban to 6,711 afterwards. On YouTube, 64 of his videos had been viewed 9.6 million times.

Research from Cardiff University, found that the de-platforming of Kate Shemirani, a former nurse and prolific spreader of Covid misinformation, decreased her reach in the short term.

“Part of the issue is that current models of de-platforming need to be developed. It’s not enough to just take down a piece of content, or a small number of accounts,” one of the paper’s authors – Prof Martin Innes – explains.

Research from organised crime and counter-terrorism shows the need to disrupt the whole network, he says.

But he believes “this level of sophistication isn’t embedded yet” in the way we tackle disinformation that could put people in danger.