Google

Google gives up on data voids

The company once warned users when they might be seeing low-quality results — but weeks before the 2024 election, the feature was quietly turned off

Casey Newton

Feb 24, 2025 — 9 min read

A low-quality search warning from 2022. The company quietly stopped showing them last year (Google)

Google quietly stopped showing warning banners that alerted users to potentially unreliable search results in the weeks leading up to the 2024 US presidential election, despite no obvious improvement in the quality of those results, according to a new study from researchers at Stanford and Carnegie Mellon University.

The study, led by Ronald E. Robertson of Stanford and Evan Williams of Carnegie Mellon, analyzed 1.4 million search queries shared on social media from October 2023 to September 2024. The study focused on “search directives” — social media posts encouraging people to search for specific terms, a practice often used by people who are spreading conspiracy theories. By encouraging people to “do their own research,” and then directing them to obscure results for which there are few high-quality links, they can trade on the trust that people have in Google to spread misinformation.

Beginning in 2022, Google began to show warning banners for some searches like this, which information researchers call “data voids.” Data voids are often exploited by actors seeking to spread misinformation on topics where little authoritative information is available.

In the early 2020s, as part of an industrywide effort to add context to potentially wrong or harmful information, Google added banners about three kinds of data voids: one where search results were of low relevance; one in cases where results are rapidly changing, such as after a natural disaster; and one — which researchers found that Google no longer displays — where the results are of low quality.

“It looks like there aren’t many great results for this search,” the banner for low quality results read. “The results below match your search terms, but some of them may not have reliable information on this topic.” The banner invited users to “check the source,” asking whether it is trustworthy.

The banners were shown relatively rarely: in about 1.5 percent of queries in the researchers’ March 2024 sample, the last before Google stopped showing the warnings. But when they were displayed, the banners often pointed out that results pointed to low-quality domains and conspiracy theories.

The study also found that conspiracy peddlers could ensure that no banners were displayed by including a site operator in the URL in the links they share. Site operators (such as “site:platformer.news”) direct Google to only search one specific domain. Google likely did not trigger warnings on such searches because it assumes anyone sophisticated enough to use a site operator knows what they’re doing. But the researchers’ work highlights how this eventually became an abuse vector for sharing conspiracies on social media, allowing conspiracy peddlers to launder fictional content related to QAnon, vaccines, and mythical lizard parasites through Google’s reputation.

The researchers collected their first data in October 2023 and March 2024. Initially, they were struck by how inconsistently Google showed its warnings. A deep learning model they built to analyze the queries suggested that the warnings should have appeared between 29 and 58 times as often as they did in practice. For motivated conspiracy peddlers, they were easy to evade: just adding quotation marks or a single letter to a query was typically enough to make the banner disappear.

The researchers were beginning to collect additional data last September when they discovered something unexpected: Google had stopped showing low-quality banners entirely, weeks before early voting began, without disclosing it.

Moreover, they wrote, the quality of results for the searches in their sample had not meaningfully improved.

“We found little evidence of substantive changes to the domain quality of the [search engine result pages] produced for queries that had previously received a low-quality banner,” they write. “Instead, we found that many of those queries continued to return low-quality domains in September 2024, they just no longer received a low-quality warning banner.”

The move was surprising, they wrote, given that elections so often generate conspiracy theories for which few high-quality results are available, or in which motivated actors quickly post pages filled with misinformation to support their political goals.

“Discontinuing these warning banners, without replacement or substantial improvements in domain quality, would be a surprising change to make in the months preceding the 2024 US elections,” the researchers write. “While we do not measure the impact of that change in this study, events with breaking news updates provide fertile grounds for data voids..”

The researchers called for "greater transparency around Google's warning banners, their prevalence, and the effects that their presence or absence has on real users."

Google confirmed to Platformer that it had discontinued the banners after finding that unspecified improvements to its core search engine had caused the warnings to trigger less often.

“As the result of a ranking quality improvement last year, the specific notice mentioned in this paper was not meeting our thresholds for helpfulness — it was surfacing extremely infrequently and was triggering false positives at a high rate, so we turned it down,” a spokesman said in an email.

I asked Google if it could share what “improvements” had resulted in the warnings appearing less often. (It’s not clear to me that this is a “ranking” problem, exactly — by definition, data voids have very few results.) The company declined to do so.

The company also said that the queries studied by researchers represent a tiny fraction of searches and are not representative of how most people use Google.

As an alternative to the banners, the company suggested that people use the “about this result” feature on the search engine results page, which uses information from Wikipedia to highlight websites known for spreading conspiracy theories. (Click or tap the three vertical dots next to the URL to find it.)

I asked danah boyd, a co-author of the original 2019 report about data voids, what she made of Google’s retreat.

“The whole point of “data voids” is that these are parts of the search query spaces where there is simply too little meaningful content to return without scraping the bottom of the barrel,” she told me.

boyd noted that not all data voids are dangerous. But it’s important that platforms continue to take it seriously, she said.

“As with all security issues, there is no magical ‘fix’ — there is only a constantly evolving battle between a system’s owners and its adversaries,” boyd said. “So if Google is getting rid of some of its tools to combat this security issue, is the company effectively saying ‘game on’ to any and all manipulators? That seems like a bad strategy.”