As tech giants figure out how to keep users from engaging with fake and misleading news online, a new Gallup poll suggests one potentially effective approach. In the survey, which was commissioned by journalism startup NewsGuard and its investor, the Knight Foundation, more than 60 percent of respondents said they were less likely to share stories from sites that were clearly labeled as unreliable. They were also more likely to trust stories from websites marked as credible.
NewsGuard was founded by media executives Steve Brill and Gordon Crovitz to do just that. The company, which launched last summer, has developed credibility ratings for more than 2,000 of the web’s most popular sites, using a team of journalists to vet each site based on a set list of criteria. Its browser extension affixes a red icon to websites that fail its test and a green icon to ones that pass. Users can also see a full “nutrition label” explaining each rating.
The extension has been live for months, but this survey is the first evidence of how people might respond to its ratings in the wild. The question was: Would users find NewsGuard itself to be trustworthy? Or would it face the same accusations of partisan bias that have dogged social media companies like Facebook and Twitter? While it’s not a peer-reviewed study, and the topic certainly warrants further inspection, it’s an encouraging sign not only for the startup but for anyone seeking remedies for the scourge of misinformation online.
Gallup sent the survey to a representative sample of 25,000 people who installed NewsGuard’s browser extension in November and used it for nearly two weeks. Of the 706 people who answered, 79 percent said they would give NewsGuard an overall rating of either excellent or good. More than 50 percent said red ratings made them less likely to read content from a given website, and 63 percent said it made them less likely to share content from those sites. Overall, more than 90 percent of respondents said the nutrition labels were either somewhat or very helpful. Even among people who said they disagreed with at least one rating, that figure still hovered over 80 percent.
There was, of course, a split between Democrats and Republicans. While 87 percent of Democrats rated NewsGuard as good or excellent, just 70 percent of Republicans said the same. (About 42 percent of respondents were Democrats, about 24 percent were Republicans, and about 34 percent were independents, roughly mirroring the electorate as a whole.) That Democrats would view NewsGuard more favorably than Republicans is to be expected given how divided the two parties are on the subject of media trustworthiness in general. An October 2018 Gallup poll found that while 76 percent of Democrats have at least a fair amount of trust in the mass media, just 21 percent of Republicans say the same. That makes NewsGuard’s 17-point split look tiny by comparison.
More than 50 percent of respondents said NewsGuard’s red ratings made them less likely to read content from a given website.
One important caveat to Gallup’s findings: While the initial sample of 25,000 participants was demographically representative of the US, the 706 survey respondents were not. This means, Gallup writes, that the survey results “may not be reflective of attitudes of the broader US adult population,” but merely represent a sample of likely NewsGuard users. It’s impossible to know whether the people who were least likely to agree with NewsGuard’s ratings were also the least likely to complete the survey.
Still, the company’s cofounders are encouraged by the numbers. “We knew in our bones we were being fair,” says Brill, who also founded The American Lawyer magazine and Court TV. “But we wanted to see if people who never met us agreed.”
When they launched Newsguard, Brill and Crovitz bet that trained journalists could do a better job judging a news outlet’s credibility than all of Silicon Valley’s algorithms combined. They hired a team of 20 reporters to analyze websites based on criteria like whether the page regularly publishes false content or whether it clearly discloses advertising. If adopted broadly by platforms like Facebook and Google, Brill and Crovitz believe the ratings could help curb the endless spread of fake news.
And yet, NewsGuard has seen plenty of pushback from people who say the ratings are all wrong. When NewsGuard launched, some media critics, like Joshua Benton at NiemanLab, questioned the startup’s decision to give FoxNews.com a green rating. And in November, Xeni Jardin skewered NewsGuard on Twitter for giving a red rating to Boing Boing, a 30-year-old website she coedits. (Jardin is a former WIRED contributor.) “Hilarious. @NewsGuardRating approached me and @boingboing in a manner that raised red flags, so we declined to participate in their scheme. They gave a 30-years-and-counting independent internet institution this failing grade. Here’s my response?? who made you the truth squad?” Jardin wrote.
Boing Boing publisher Jason Weisberger says he declined to answer NewsGuard’s questions about Boing Boing when a reporter there reached out, in part, because he was unsure who was behind it or whether it was reputable. According to NewsGuard, Boing Boing failed to meet several of its criteria, including distinguishing between news and opinion stories, clearly labeling advertising, and disclosing its sources of financing. Weisberger, however, says these characterizations are inaccurate. “All of our advertising is clearly called out. We disclose absolutely everything we work on and have for years. Any claims otherwise are bullshit,” he says. Weisberger also takes issue with a line in NewsGuard’s report noting that John Battelle, who helped launch WIRED, is a minority investor in Boing Boing. Weisberger confirmed that Battelle is a financial partner but that he hasn’t been actively involved in Boing Boing for years. (Battelle says he was made a partner “back in the day” but is no longer involved in Boing Boing “in any way.”) Weisberger declined to comment on other specific claims made in NewsGuard’s rating.
“I’m not looking to pick a fight with them. I’m not looking to discredit them. I don’t care,” he says. “We’re very happy they thought we were worthy to write about in their list, because we’ve been working on the site for a very long time, and we’re very proud of it.”
Boing Boing fans and Jardin’s followers rallied immediately to her side on Twitter and dismissed NewsGuard’s ratings broadly.
Brill and Crovitz, for their part, stand by the rating and say that Boing Boing’s staff is welcome to point out any specific inaccuracies they see. They also don’t mind fielding criticism for giving far right sites like the Daily Caller green ratings. Unlike tech companies whose black-box algorithms often make these decisions, they say, at least NewsGuard is being transparent about the criteria each site did or didn’t meet. “We’re happy to get that criticism, because think of the alternative,” Brill says. “Right now you have no idea whether Facebook rates Boing Boing higher or lower than the Daily Caller.”
Getting Facebook, Google, and other tech companies to adopt these ratings is, after all, the ultimate goal for NewsGuard. Right now, NewsGuard’s browser extension has just 30,000 active installations, in addition to installations at libraries across the country that have signed up for the service. But the only way to truly scale the idea would be for platforms with a critical mass of people to adopt it too. In the Gallup survey, nearly 70 percent of respondents said that these ratings would make them more trusting of social media platforms and search engines. So far, only Microsoft has started rolling out NewsGuard ratings on its Edge mobile browser. Brill says he expects additional announcements in the months to come, but he declined to comment on specifics.
Still, the dustup over Boing Boing illustrates why tech companies tasked with vetting an infinite number of websites are reluctant to take such a public stance, even if they’re already making these decisions behind closed doors. If the results of the Gallup survey hold true, those NewsGuard ratings hold real power. That power needs to be applied wisely.