Media Bias

A recent comment here by Brian expressed amusement at a citation of CBS news. Indeed, if you talk to a lay person about bias, one of the first examples that will come up is bias in the news media. A recent Zogby poll confirms widespread belief in the existence of such bias:

The vast majority of American voters believe media bias is alive and well – 83% of likely voters said the media is biased in one direction or another, while just 11% believe the media doesn’t take political sides…

I had a preconception that most people perceive bias as a function of difference from their own beliefs. Conservatives complain about liberal bias, while liberals complain about conservative bias. However the Zogby poll revealed that things are not as symmetrical as I had assumed:

While 97% of Republicans surveyed said the media are liberal, two-thirds of political independents feel the same, but fewer than one in four independents (23%) said they saw a conservative bias. Democrats, while much more likely to perceive a conservative bias than other groups, were not nearly as sure the media was against them as were the Republicans. While Republicans were unified in their perception of a left-wing media, just two-thirds of Democrats were certain the media skewed right – and 17% said the bias favored the left.

Overall, 64% perceived a liberal bias compared to 28% who see conservative bias.

In thinking about these results, it occurs to me that when liberals complain about conservative bias, they often focus their attention on just a few media outlets. The worst offender is Fox News, a network which advertises their "Fair and Balanced" coverage but which is widely perceived among liberals as being strongly biased to the right. It appears from these poll results that liberals are not as unhappy about opposing bias among the broad range of media.

An academic analysis of media bias from UCLA a couple of years ago attempts to offer a more objective appraisal (PDF preprint available here). This is of course a highly controversial topic and at first it might seem impossible to fairly quantify media bias. The methodology they developed is to study media sources, including newspapers, TV shows and web sites, and to look at how often they cite various think tanks and other policy groups. The idea would be that if they consistently cite liberal groups they are showing a liberal bias, and similarly for conservative groups.

But how to measure the groups, objectively?  Here is the real novelty. The study looks at how often members of Congress of various political views cite the same groups, and correlate those rates with the Congressperson’s scores from the ADA, a liberal group. ADA like other such groups gives Senators and Representatives numerical scores showing where they are on the liberal-conservative spectrum, and I understand that these ratings are generally considered to be reasonably objective and not too controversial.

By comparing the citation rates from the media with those from members of Congress with various ADA scores, the researchers come up with an equivalent ADA score for each media source. This then measures where they are on the political spectrum, compared to the average scores for Congressional representatives (the average ADA for Congress at the time was 50.1 on a scale from 0 to 100).

Although this methodology sounds reasonable, the results were surprising to me. The bottom line is that, of the 20 media outlets studied, almost all were had ADA scores above 50, putting them on the liberal side of the spectrum. Here are the results from Table IV of the paper, ranking media outlets in order of distance from the center, i.e. from least to most biased by the methodology of the study. For reference, the average ADA score for Congressional Democrats was 84.3:

  1. 55.8    Newshour with Jim Lehrer
  2. 56.0    CNN NewsNight with Aaron Brown
  3. 56.1    ABC Good Morning America
  4. 60.4    Drudge Report
  5. 39.7    Fox News’ Special Report with Brit Hume
  6. 61.0    ABC World News Tonight
  7. 61.6    NBC Nightly News
  8. 63.4    USA Today
  9. 64.0    NBC Today Show
  10. 35.4    Washington Times
  11. 65.4    Time Magazine
  12. 65.8    U.S. News and World Report
  13. 66.3    NPR Morning Edition
  14. 66.3    Newsweek
  15. 66.6    CBS Early Show
  16. 66.6    Washington Post
  17. 70.0    LA Times
  18. 73.7    CBS Evening News
  19. 73.7    New York Times
  20. 85.1    Wall Street Journal

If we accept these results at face value, Brian is right to be skeptical about the use of CBS news as a factual source, especially on a matter which reflects badly on a conservative administration. CBS Evening News was tied for the 2nd most biased source in the study. We also note that the one Fox News program does perhaps surprisingly well given the criticism the network often receives, coming in as the fifth least biased.

Granted, the study’s methodology is somewhat unorthodox and indirect. It occurs to me that the same results could occur if liberals and the media both cite mostly factual, unbiased sources (in keeping with the liberal claim to be the "reality based community") while conservatives and conservative media cite sources that have strong biases. On the other hand, the consistency between the results of this study and public polling results suggests that the reality is what we see here, a widespread if modest liberal bias in the media which is recognized accurately by the public.

GD Star Rating
Tagged as:
Trackback URL:
  • Trey

    I assume the numerical rating of 85.1 for the WSJ is an error, given that it is generally considered one of the most right-leaning news outlets. Perhaps the correct value is 35.1?

  • So The Wall Street Journal is more liberally biased than the average Congressional Democrat? Something smells fishy here.

  • The idea would be that if they consistently cite liberal groups they are showing a liberal bias, and similarly for conservative groups.

    Are the groups equal besides their tilt? Surely they focus on different issues, for example.

  • Carl Shulman

    Delving into the paper, it only addressed news coverage, so editorial content was excluded. The editorial and news sections of the WSJ are quite separate, and the consciously libertarian ethos of the former is not necessarily mimicked by the latter.

  • Obvious problem 1 with their study: by their methodology, the ACLU is a slightly conservative organization. This is because when a group from one side takes a nonintuitive stand, the other side cites them heavily. They tried to control for that, but it obviously didn’t work in a couple of high profile cases.

    Obvious problem 2: the truth has a well-known liberal bias. 🙂

  • fishbane

    Drudge is a liberal? Who knew.

  • Carl, you’re right, the WSJ result applies to their news pages, which by this study do come out to be highly liberal. I don’t know if that really makes sense based on what reading of the newspaper I have done.

    Dave, if more conservatives than liberals cite the ACLU (perhaps because of its reputation as being too extreme) then we might expect a conservative outlet like Fox to also cite the ACLU more than a relatively liberal one like CBS, for the same reasons. So the methodology can work even for these inverted citation patterns.

  • I thought the Wall Street Journal called consevatives “conservatives” in order to give them credit.

  • michael vassar

    Hmm, the only newspaper even remotely similar in score to Fox News, and the only one with a lower score is the Washington Times, which is owned and run at a loss of nearly $100,000,000/year by Reverend Sun Myung Moon, self-appointed Messiah (several prominent Republicans attended his anointment) and head of the Unification Church (the Moonies).

    Tied for second most ‘biased’ are CBS evening news and the “paper of record”, the New York Times.

    Most ‘biased’ by far and most ‘liberally biased’, by the metric used in this study is the Wall Street Journal, a paper which is definitely not generally perceived of as being left of the New York Times, but which is frequently considered, by those near the center of the political spectrum, to be of higher quality than the New York Times.

    To me it looks very much more likely that this study was attempting to measure bias and ended up measuring quality of sources instead. Since this was not the purpose of the study, it would seem to me to constitute be a VERY strong piece of evidence for the hypothesis that liberals constitute the “reality based community”.

    It also seems highly plausible to me a-priori that the perception of liberal bias is the result of the phrase ‘liberal bias’ being used non-technically by the public to mean ‘tendency to agree with liberal positions’. I think that such a usage of the word ‘bias’ is probably widespread and possibly dominant in public discourse. If people don’t generally have a formal definition for or meaning of normative belief, as is almost certainly the case, this is the meaning that would be expected. This seems consistent with the common complaint of scientists that ‘unbiased’ is often treated by even the educated public as equivalent to ‘presenting both sides of the issue’, or even in terms of giving ‘both sides’ equal time, for instance, teaching evolution and creation in schools

    A technically incorrect but suggestive model of these results would be to imagine the existence of a ‘reality loading factor’ in the news, and to imagine that these scores are approximately the ‘reality loading scores’ of different sources. It seems much more likely that the Wall Street Journal is more ‘reality loaded’ than a typical Democratic Congressperson than that it shows stronger “liberal bias” than a typical Democrat, half again more “liberal bias” than the New York Times, and 250% more total bias than Fox News.

  • TGGP

    I’d expect any attempt to have a “reality loading” measure or define what the normative center “ought” to be would get laughed at. And if no one else does the laughing, I will.

  • Carl Shulman


    The sources used for the study are primarily think tanks and advocacy groups, listed on page 55 of the paper. Many of them are cited simply to embody a position, e.g.: the NRA and Handgun Control Inc., NARAL and the national Right-to-Life coalition, etc. The NAACP, AARP, NOW, Christian Coalition, Sierra Club, etc influence scores extensively but are typically relied on for value statements rather than rigorous research. Neither Citizens for Tax Justice or Americans for Tax Reform will be very informative.

    The WSJ’s relatively intelligent and educated readership will be less socially conservative than the median voter, as will its reporters, which will influence the types of value statements cited. The positions expressed may be the outcome of a reality-based view, even though the sources in question may be low-quality shills.

    A handful of the 50 most cited sources, like the Federation of American Scientists or Council on Foreign Relations, support your hypothesis, but they make up only a small portion of the data.

  • Brian

    Sorry for starting this mess.

    I believe media bias is separate from the cognitive biases all humans are burdened with, the kind of things you guys discuss here. My theory would be that political biases are the cumulative effect of the many cognitive biases outlined in academic literature.

    In the interest of full disclosure, I’m a libertarian/conservative. I think it’s pretty obvious to most that the major US media organizations are consitently biased in terms of how they characterize events, sources, and people, as well as what issues they select to report. End of disclosure.

    I think it’s telling how liberals scream and howl whenever this issue is raised. The discussion quickly becomes ad hominem attacks on Rupert Murdoch or Matt Drudge (or others). They don’t even pretend to consider if it’s true.

    The libertarian side of me says that CBS News and the New York Times have every right in the world to slant the news as far left as they please. Wherever there is an open market for ideas and low barriers to entry, such as in cable, internet, or radio, the market will respond. I just wish every media outlet, right and left, would fess up to their political leanings. It seems that only relatively recently in the history of the media, post WWII perhaps, that they all pretend to be neutral. In the past, newspapers or journals were clearly pro-union or pro-prohibition, or whatever. I belive we’d be better served if outlets did what they used to, wear their political leanings on their sleeves.

    But maybe the bigger problem is political bias in the academy, but that’s a different can of worms!

  • Elliot Reed

    One thing that concerns me about the study methodology is that it’s far from clear that the median member of the House is a good approximation of the median voter. How much of an effect is the political party that controls the House having – what happens to the results when the median member is a Democrat?

  • Stuart Armstrong

    Sorry for starting this mess.

    I second that apology, for citing CBS in the first place – the first news link I found on the subject. I didn’t know much about it, or which way its bias goes ^_^

    I feel strongly about these results, so won’t comment on the reliability or not of the methodology – the possibility for confirmation/disconfiration bias is too high.

    What would be interesting would be to break down the general term “liberal”, which covers ideas like internationalism, paternalism, concern for the poor and obsession with fairness, interest in the arts, etc… and see what idea is the most significant in giving this liberal bias to the media. And is it the same liberal bias in the WSJ as the NTY?

    The WSJ is interesting here. If the editorial pages do in fact bear some relation to the news content, my guess is that what makes it liberal is reporting on those issues where liberal overlap with libertarian – or maybe where liberals overlap with the rich. Can someone who knows the WSJ well give me an opinion on this?

    the truth has a well-known liberal bias. 🙂
    I think it does – but only because of the social conservatives. They are fond of making unsupported pronouncements on various issues (“gay marriage causes the breakdown of the family”; “crime is caused by a lack of a strong father figure”) and are relatively mainstream in conservatism. Their equivalent on the hard left are less influential, and tend to make pronouncements that are harder to deny (“free trade depresses the wages of American workers!” – a statement that is very misleading, but technically correct in the short term). Maybe the liberal deference to experts means that most quoted experts will indeed follow liberal lines?

    Remove the social conservatives, and reality may well have a conservative bias.

  • Stuart Armstrong

    From the continent, the media in the UK is nearly seen as having a strong conservative bias. Are there any studies or results similar to this one for Britain?

  • Tex

    “Carl, you’re right, the WSJ result applies to their news pages, which by this study do come out to be highly liberal. I don’t know if that really makes sense based on what reading of the newspaper I have done.”

    The news desk is not the editorial desk. The study ranks the news, not the editorials.

    Read the news pages from both Investors Business Daily and The Wall Street Journal side by side for a week. The two papers clearly come at the same stories from different angles.

    There is a reason why Noam Chomsky quotes approvingly from the pages of The Wall Street Journal, not Investors Business Daily.

  • David Rothman

    Your estimator for bias is not an effective metric because it does not capture how the quoted sources are used. An article in one newspaper that quotes the Brady campaign may be biased in favor of gun control, while the same quote could be used in another publication to argue strongly against gun control.

    More fundamentally, I believe that you are conflating two very different kinds of biases. Your site seems primarily devoted to correcting unconscious biases due to misapprehension of reality (or if your prefer Baysean probability theory). These types of biases are usually unintentional, and there’s little resistance to correcting them once the error is demonstrated. Your mission seems to be to help us find and use more accurate scales.

    The bias in politics, journalism, advertising and education is of a very different type. Practitioners of this these trades have a vested interest in intentionally distorting their presentation of information to achieve results which they believe are desirable. They are not mistaken, they are simply lying. These professionals are like the butcher resting his thumb on the scale. Accuracy and precision are the very last thing they want.

    I think we all know that politicians and advertisers lie. But, why should journalists and educators lie? After all, aren’t they devoted to the pursuit of truth? In fact, many journalists and educators enter these fields with the specific intention of attempting to achieve political or social change that they think is desirable.

    In a recent speech broadcast on National Public Radio, Bill Moyer praised Antonio Gramsci and Michael Moore for showing the way for more “activitist” journalism. Michael Moore’s Farenheit 911 needs no introduction for the use of misinformation and bias in the pursuit of propaganda. Gramsci was an Italian communist who developed the concept of using the intelligensia to condition the proletariat for communism through the intentional manipulation of news, media and education.

    Media and academic bias are real and obvious to any observer. Fortunately, they are less effective than intended thanks to the highly attuned bullshit detectors which most people deveop. If you can not yet spot phonies, here are three simple heuristic tests that will effectively spot a large fraction of intentional lies:

    1) Is the action proposed in the interest of the person or organization making the appeal? In law this is called “cui bono?” (who gains?).

    2) Does the person or organization actually abide by the actions they demand of others?

    3) Does the person or organization making the appeal reject solutions to the problem they present if the solutions don’t agree with their overarching agenda?

    Can these tests lead you astray? Of course they can, but so can the most exquisite analysis of covariance testing. As George Box noted many years ago, “All models are wrong, some models are useful.”

  • This study seems to have no sensible underlying notion of bias, apart from “presenting both sides of an issue” Michael V wrote about.

    A natural way to judge the quality of a measure is looking at the effect of changes in the underlying data. Suppose that there was a strong shift to the right in the Democratic Party. This would probably lead the media to have a stronger “liberal bias” using the methodology of the article. The right becoming more extreme would have the same effect. But there is no corresponding change in the behavior of the media. I t^hink it is very hard to make sense of the ranking.

    Dave’s view that the truth has a liberal bias is certainly not inconsistent with the data…

  • Brian, no need to apologize, I had been meaning to write this ever since I read about the Gallup poll a couple of weeks ago and your comment goaded me into action.

    David, it may well be the case that some media bias is intentional. I see the value of the information I presented as helping us to become more aware of bias in the media information we consume. Although the Gallup poll results are broadly consistent with the UCLA study, it is clear that there is wide variation in the degree and direction of bias that individuals perceive. This information can help us to calibrate our judgments of bias in our information sources.

    Michael, ultimately the media rankings are based on the ADA rankings of Congress, correlated via citation patterns. If Democrats moved to the right and changed their citation patterns while the media did not; and if the ADA ranked Democrats more conservatively as a result, then the methodology of the study would not change media scores.

  • You are correct.

  • Avi Gur

    Recentely I’ve got my Ph.D. the subject of the study is:”Media Bias: A Framework for Analysis and case study, The Israeli Media and the Oslo Peace Process 1994 – 2004″.
    In this study I developed a new formula that ranked the main ISRAELI media (print and broadcast)according to the centric line and the results are in percent. I would like to have apportunity to introduce to you the abstract of the study.
    Avi Gur

  • Tom Elkin

    I understand Washington Times and Fox but listing WSJ as liberal is absurd and having Drudge as leaning liberal is almost as absurd. And you left out IBD and the New York Post which would skew this study even further. This study stinks.