Not fake news: Major study finds no 'liberal bias' in media

Complaints about press bias are as old as the press itself, but in recent decades, conservatives have pushed one complaint above all other: The media is biased against them because it is overwhelmingly staffed by liberal journalists. A new study, forthcoming in Science Advances, provides the strongest evidence ever that they’re half-right — but only the least important half: Yes, reporters overall are significantly more liberal than the general population. In fact, almost one in six are more liberal than Rep. Alexandra Ocasio-Cortez, based on who they follow on Twitter. But no, that doesn’t matter — even for the most liberal cohort of them. The title of the study says it all: “There is No Liberal Media Bias in the News Political Journalists Choose to Cover.”

This article first appeared in Salon.

Even though “journalists are dominantly liberal and often fall far to the left of Americans,” the paper itself was emphatically clear in its conclusion:

In short, despite being dominantly liberals/Democrats, journalists do not seem to be exhibiting liberal media bias (or conservative media bias) in what they choose to cover. This null is vitally important — showing that overall, journalists do not display political gatekeeping bias in the stories they choose to cover.

In a way, that’s not that surprising: Journalists place a high value on objectivity and balance. Avoiding ideological bias “rates very high” among journalists, lead author Hans Hassell told Salon — 8.5 on scale of 10 in the survey these researchers conducted. But as Hassell acknowledged, “A response you give to a survey may be very different from the actual behaviors that you express in the things that you do.”

So the authors turned to a correspondence experiment, which Hassell said was “essentially an experiment to test for biases that individuals may not be willing to explicitly state, because it may be socially unacceptable to be biased in a certain way, or they may even be unintentionally biased.”

Historically, these kinds of experiments are "commonly used to detect racial bias,” Hassell said. (It’s primarily been used in employment studies, where it can detect gender bias as well.) Using this technique to detect ideological bias among journalists broke new ground.

“We created a fake state legislative campaign," Hassell explained. "We created an email account that purported to be from an individual in the community who was going to announce his candidacy for state legislature. Then we posed as a staffer for that campaign and sent an email to every single individual asked to participate in the survey,” Hassell explained.

The email provided basic candidate information and asked whether reporters they "would be interested in sitting down with the candidates and talking about their positions with regard to state government.” Crucially, researchers randomized whether employees of a given publication“received an email that indicated whether the candidate was a strong conservative, a moderate conservative, a moderate progressive or a strong progressive.”

The texts of the emails were the same in all cases, with the only differences being the candidate biographies included at the bottom of the emails. “We might expect progressives to be more interested in covering progressives and conservatives more interested in covering conservatives,” Hassell said, with a liberal bias predominating, simply because there are more liberals in the population at large. Instead, they found no significant evidence of bias, either for the respondents as a whole, or broken down by ideology or by the partisan leaning of the counties they served. In short, the premise that the profession still has governing standards was fully borne out by the experiment.

The overall results are shown in the following figure:

The paper explains:

As can be seen, there is no statistical or substantive difference in the probability of a journalist responding to the email based solely on the treatment conditions. Comparing the two poles, strong conservative candidates are, on average, a mere 0.4 percentage points less likely to get a response than strong progressive candidates. This effect is minuscule (being equivalent to 0.47% of a standard deviation) and is far from significantly different from 0 (p=0.87).

One possible explanation for the lack of bias might simply be journalists responding to the marketplace they serve — giving more coverage to conservatives in counties that voted for Trump, for example, and more coverage to progressives in counties that voted for Clinton. But in fact that wasn’t the case:

[W]e find that a journalist working for a newspaper in a county that voted for Trump is just as likely to respond to a request for an interview with a progressive candidate as they are to a request from a conservative candidate. This shows that even in spite of powerful economic incentives from the readership of one’s newspapers, journalists still show no signs of ideological gatekeeping bias in what they choose to cover.

The authors also divided journalists into three ideological categories — exploring the possibility that “we might not see evidence of bias overall, but, instead, we would see polarized coverage” — but the differences observed were barely perceptible, and even those minimal variations didn’t fall into a clear pattern: “[W]e find that journalists — regardless of their own ideology — treat candidates from different ideological backgrounds the same.”

These findings are all the more striking, given how liberal journalists as a whole were found to be. First, the authors conducted an extensive survey, emailing a list of just over 13,500 journalists with working email addresses. “We got about a 13% response rate, which is about in line with most surveys of political elites, and about double that of surveys of journalists that have been published in recent years,” Hassell said. There are two significant problems: First, a large number of journalists report being independents and/or report no ideology, and second, a large majority don’t respond to surveys. “So the question is, ‘Are the journalists who are not part of the online response going to be different in some ways?’”

To address these two problems, a team of eight undergraduates gathered all the Twitter handles they could find — about half the 13,500 journalists originally identified. Then the accounts were analyzed using a technique developed and validated by Pablo Barberá in a 2015 paper, “Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data.” As the paper explains:

The logic of this methodological technique is that individuals display their preferences (in this case, for ideological homogeneity) through their actions (in this case, who they follow on Twitter), just as they do with many revealed preferences.

Barberá showed that this method produces ideology measures that are strongly correlated with other measures — both self-reported ideology and party registration records — among both the public and elites in the U.S. and five European countries. Hassell’s team found they correlated strongly with self-reports of those who responded to their survey, as well. Thus, there’s high confidence in the validity of this measure, the results of are summarized visually here:

So if journalists actually reflecting their own views in covering the news, conservatives would have a strong case for bias. But as we’ve just seen, that’s not what Hassell’s team found: There was no significant bias at all.

Because these results are both surprising (even unbelievable) to some, and significant, it’s worth explaining a bit more about what the experiment reveals, and the thinking that went into it. Identifying gatekeeping bias is difficult, the paper notes, “because identifying the full population of news from which journalists could select stories is difficult.” As Hassell put it, “It could be that there aren’t an even number of liberal and conservative stories available” for journalists to choose from. The correspondence experiment responds to that difficulty by ensuring balance, and presenting a simplified representation, where the potential for bias is well-defined and measurable.

It’s also both sensitive and realistic. They didn’t want to do anything that could violate ethical standards — "presenting a scandal seemed unethical,” Hassell noted. “So we settled on this because we felt it was something that was logical in terms of something that would commonly happen,” and where judgment and discretion were involved. Some stories are “so compelling that everybody has to cover them," he observed, "and then there are other things that are not news at all, so this is kind of in the middle in the realm of journalistic discretion.” It was also something he knew from the other side, he later told me, which is part of why it was chosen. “As a student, I worked on a couple of election cycles working for a campaign, and I remember the candidates complaining, ‘Why can't I get coverage? Why won't the paper cover me? What's going on?’”

Another factor to consider is that state legislative races are themselves gateways to partisan politics: It’s plausible that someone you’ve never heard of could not just enter, but win such a race, and doing so could have long-term political consequences, depending in their subsequent career. Many state legislators go on to Congress, and a few to the Senate and national prominence. So, this experiment taps into something central to the way American politics works.

Still, experimental evidence is always stronger when support comes from lines of inquiry. In this case, Hassell’s team did some of that themselves. In the original survey, they “presented journalist respondents with two pairs of hypothetical candidates who were announcing their candidacy for governor in the state,” but with a set of limitations, so that only one could be covered. Who would that be? This randomized test also found minimal evidence of bias. “If anything, they are more predisposed to cover Republican candidates,” the paper reported, although the effect was deemed "not statistically significant."

But a stronger form of supporting evidence would come from asking a different sort of question entirely. I asked about that. “I think absolutely there should be other ways to do this,” Hassell said. Then he described some of the difficulties in doing so, starting with ethics. You don't want to damage a journalist's reputation with a false story lead, and you don't want to waste his or her time and resources "following up a story that really doesn't exist.” One idea Hassell suggested might work was to pose as a researcher announcing a new study, varying the topic. “Some topics are more conservative or more preferential to liberals,” he said. “That way you're not forcing the journalist to go out and do any research on it. You can be the researcher.”

From a broader perspective, one can certainly question how adequate this kind of evidence is, however.  As I’ve noted before, we can understand this in terms of  Daniel Hallin’s three-sphere model of how the media functions, from his 1986 book "The Uncensored War":

At the center is the sphere of consensus, mom-and-apple-pie country. Surrounding that, like a donut, is the sphere of legitimate debate, where journalists’ attention is usually focused, where there are two sides to every story and a need for objectivity and balance to be maintained.

Beyond that, though, is the sphere of deviance, the outer darkness in which dwell “political actors and views which journalists and the political mainstream of society reject as unworthy of being heard.”

In that piece, I went on to discuss the “shoddy fact-checking" directed at Alexandria Ocasio-Cortez by Glenn Kessler of the Washington Post, which reflected "a boundary-policing instinct, and an outdated one, considering that the entire political landscape has been irrevocably changed.”

All three of Hallin’s spheres can be problematic for journalists. Those rejected in the sphere of deviance might actually have the most accurate, and most important things to say: I cite a number of examples of how this applies to Ocasio-Cortez, but, for historical perspective one could also consider the early feminists who convened at Seneca Falls, New York, in 1848.

The sphere of legitimate debate can be problematic as well: Objectivity and balance can be sharply at odds with one another, as another example I cited pointed out:

As far back as 2004, a study found that “US prestige-press coverage of global warming from 1988 to 2002 has contributed to a significant divergence of popular discourse from scientific discourse,” and that “the prestige press's adherence to balance actually leads to biased coverage of both anthropogenic contributions to global warming and resultant action.”

Another problematic aspect of this sphere was pointed out to me by Jim Naureckas, the editor of FAIR.org, the website of Fairness & Accuracy In Reporting:

The traditional model of "objectivity" followed by corporate media uses the two major parties as the two poles that journalists are expected to drive their stories straight between. This "both sides" approach often leaves out several other sides — either to the left or right of the two big parties, or looking at politics from a different angle entirely.

One of the biggest biases we find in political coverage is toward seeing politics as a spectator sport. Sometimes when reporters are covering protests and other grassroots efforts to influence policy, I'm reminded of sports reporters writing about fans running onto the field.

This way of dismissing the very citizens who should be central to a democracy is hardly conducive to a healthy democracy over time. It stands in stark contrast to the kind of active engagement that health require.

Finally, the sphere of consensus at any point in history may be wildly at odds with what it was a generation or two earlier or later. There’s a reason we find past eras unfathomable at times: their spheres of consensus baffle us.

None of this is to question the value of Hassell and his colleagues' work. But the good news that it reveals — the extent to which journalists are guided by professional standards—strongly suggests the potential power of revising those standards to address the above-noted shortcomings, and other failings, to achieve the kind of objective, reliable reporting to which journalists aspire.

In addition, the innovative combination of approaches used in the study calls out for further breakthroughs that may help us better understand these broader sorts of challenges. I was reminded of a number studies I’ve written about here, such as UNC sociologist Christopher Bail’s book, "Terrified: How Anti-Muslim Fringe Organizations Became Mainstream" (author interview here), which used big data techniques to understand how factually deficient political actors came to have such disproportionate media and political influence, actually enabled by common media practices. There was also the Columbia Journalism Review study, "Don’t blame the election on fake news. Blame it on the media" (Salon story here) with this astonishing factoid: “The New York Times ran as many cover stories [10] about Hillary Clinton’s emails as they did about all policy issues combined in the 69 days leading up to the election.”

As these two examples (along with coverage of the climate crisis) suggest, supposedly “neutral” media practices can be heavily tilted toward conservative points of view, regardless of individual journalists’ attitudes. The fact that conservatives have a highly salient, although groundless, narrative of bias surely complicates matters for journalists trying to report the truth. They should take this study to heart as a validation that they are doing exactly what they claim, under extreme duress, and that charges of bias are unfounded at best, and quite likely deliberate attempts to manipulate reporters and shape coverage.

Such problems are especially vivid in matters of religion, where conservatives make a deeper claim: Liberals misunderstand them and thus systematically misrepresent them. In fact, the big problem is arguably the reverse. The media tendency to take the religious right at its word is arguably the greatest barrier to accurate reporting.   For some insight on this I contacted author Frederick Clarkson, senior research analyst at the progressive think tank Political Research Associates.

“Reporters and editors are often in a tricky spot.” Clarkson said. “Sometimes they lack the knowledge and vocabulary in this dynamic field. This can be particularly challenging when political figures or their prominent supporters are profoundly motivated by their religious views.”

He cited the example of the 2012 Republican primary, the early stages of which featured two candidates, Rep. Michele Bachmann of Minnesota and Gov. Rick Perry of Texas, “who were unambiguously influenced by overtly theocratic ideas and figures from well-established theocratic camps called Christian Reconstructionism and the Pentecostal, 'dominionist' movement called the New Apostolic Reformation.”

Leading figure in NAR, for example, staged a prayer rally of 30,000 people in Houston to launch Perry's campaign. “Journalists who had written accurately, about all this became the targets of a smear campaign," Clarkson said. "They were (falsely) said to be tarring evangelicals in the broader sense,” and “media coverage of dominionism collapsed.”

Clarkson continued: “This turning a blind eye to the theocratic politics animating much of public life remains more the rule more than the exception. Even though evangelical historian John Fea described Ted Cruz as a Dominionist in an essay for the Washington Post, reporters did not much follow his lead.”

Such willful blindness is difficult to square with any coherent notion of objectivity, and has contributed significantly to the ongoing state of bewilderment that Trump retains such high levels of support from white evangelicals.  The coverage of all American politics directly suffers from this blindness.

This study may help empower the media profession to be less easily intimidated going forward. There is a great deal that needs repair and renewal in American democracy, and journalism has a vital role to play in that process. By recognizing its own strengths, as revealed in this study, and building on them to address its weaknesses, the press can once again perform its democratic duty, and help the American public to navigate dark times.