To my surprise he came back more worried than ever. He had watched the program on the Bad Science page, but not looked very much at the damning criticism surrounding it. After all, a warning is much more salient than a critique. My friend is highly intelligent and careful about his biases, yet fell for this one.

There exists a feedback loop in cases like this. The public is concerned about a possible health threat (electromagnetic emissions, aspartame, GMOs) and demand that the potential threat is evaluated. Funding appears and researchers evaluate the threat. Their findings are reported back through media to the public, who update their risk estimates.

In an ideal world the end result is that everybody get better estimates. But this process very easily introduces bias: the initial concern will determine where the money goes, so issues the public is concerned about will get more funding regardless of where the real risks are. The media reporting will also introduce bias since the media favour reporting newsworthy news, and risk tends to cause greater interest than reports of no risk (or the arrival of reviews of the state of the knowledge). Hence studies warning of a risk will be overreported compared to risks downplaying it, and this will lead to a biased impression of the total risk. Finally, the public will have an availability bias that makes them take note of reported risks more than reported non-risks. And this leads to further concerns and demands for investigation.

Note that I leave out publication bias and funding bias here.There may also be a feedback from the public to media making media report things they estimate the public would want to hear about. These factors of course muddy things further in real life but mostly seem to support the feedback, not counter it.

A little model to estimate how serious the problem is: Imagine that there are N studies published, and that they have probability p of being right. On average we should expect Np correct conclusions and N(1-p) erroneous. Media will report a study with probability P1 if it finds a risk, and with probability P0 if there is no risk (P0 < P1). Finally, the public will notice risk reports with probability Q1 and non-risk reports with probability Q0 (Q0<Q1). If there is actually a risk the public will notice about  NpP1Q1 studies that warn against it, and N(1-p)P0Q0 studies that say there is no danger. As long as p/(1-p)>P0Q0/P1P1 people will get the correct impression that evidence tells them to increase their risk estimates. This is always the case given the above assumptions if p is close to 1. In the case there is no risk the public gets NpP0Q0 studies that say there is no problem, and N(1-p)P1Q1 studies that (erroneously) warn. If p/(1-p) < P1Q1/P0P0 the public will now be convinced that there is a risk.

If we assume that p is around 0.95, that only requires P1Q1/P0P0 to be larger than 19 to produce bias feedback. So if there is about a factor 4-5 overrepresentation of risk and availability bias each, we get a situation where the scientists are actually producing correct conclusions that there is no risk but they get consistently misrepresented to the extent that the public believes the risks are becoming more certain. Over time, the risk may become part of common knowledge (just like factoids like getting cramps from eating too close to swimming), promoting other biases like bandwagon effects and leading to irrational policy.

If p is lower, which is likely in many uncertain fields, the tendency to overestimate risk gets even worse for p=0.9 the bias factor needs to be larger than 9 to cause feedback, for p=0.8 just 4 and for p=0.7 just 2.3. In the last case we only need Q1/Q0 and P1/P0 to be 1.5 to get bias feedback. 50% overreporting and overattending to warnings doesn’t sound unlikely at all.

Even when this does not happen it seems likely that subgroups of the public will still be convinced that there is a risk and demand funding, and through another layer of availability bias (and decision makers tendency to act on demands rather than non-demands) this produces more funding for research that will keep the worried worried. This is apparently what is happening in cellphone-radiation research, where funding priorities are set externally by public concern rather than based on the best risk estimates.

How to overcome? Even if the media in this model were perfectly fair (P1=P0) the Q1/Q0 factor still introduces bias. It seems that the real threat here is the multiplication of these bias factors (we could introduce a publication bias factor increasing the likelihood of researchers publishing pro-risk papers, and a factor for filtering through experts, and so on). It doesn’t seem that unlikely to get a factor of at least 2, and then we just need two or three layers of bias before we risk serious feedback. Since reducing Q1/Q0 is likely hard, it is better to reduce the number of layers – read scientific papers directly, base funding decisions (somehow) not on public concern but on statistical estimates of risk, ignore or punish media for relying on experts (or worse, other media) rather than researchers themselves etc.

The problem here isn’t media per se, but that biases are compounding and possibly leading back to a distortion of the fact-finding process. Media priorities make things worse, but it is just an extra layer of compounding.

Some other things that came up while reading up on this:

Derek J. Paulsen, Wrong side of the tracks: exploring the role of newspaper coverage of homicide in socially constructing dangerous places,  Journal of Criminal Justice and Popular Culture, 9(3) (2002) 113-127

Even when the risk concern is right there might be biases due to focus on salient kinds of risk. The above paper shows that crime reporting accurately reports some crime hotspots (the central ones) but misses others. In the case of electromagnetic fields people are concerned with wifi and cellphones (availability again!) but not general fields, so funding becomes targeted at small subsets while the rational approach would be to try to figure out the whole picture.

Combs, B. & Slovic, P. (1979). Newspaper coverage of causes of death. Journalism Quarterly 56, 837-843. & Connie M. Kristiansen, Newspaper coverage of diseases and actual mortality statistics, European Journal of Social Psychology, Volume 13, Issue 2, Pages 193-194, 1983

Frequency of reporting in newspapers about causes of death is essentially uncorrelated with the frequencies of real causes of death (0.13 in the Coombs study). But the correlation between estimates made by people of the frequencies and the newspapers is high (0.7).

Kumanan Wilson, Catherine Code, Christopher Dornan, Nadya Ahmad, Paul Hébert and Ian Graham, The reporting of theoretical health risks by the media: Canadian newspaper reporting of potential blood transmission of Creutzfeldt-Jakob disease, BMC Public Health. 2004; 4: 1.

Newspapers primarily relied upon expert opinion as opposed to published medical evidence, and some of the activity led to policy effects (such as the Red Cross withdrawing blood out of fear of contamination, creating shortages of blood products and around $11 million in costs – but possibly a saved reputation).

Anders af Wåhlberg and Lennart Sjöberg, Risk perception and the media, Journal of Risk Research 3(1), 31-50 (2000)

A dissenting paper, arguing that media might be biased but tends to be biased in a diversity of ways, and in particular might affect people’s risk perception less than commonly believed. However, their argument seems to be that media does not affect personal risk perceptions as much; my argument in this essay is that impersonal risk perceptions might be significant in aggregate – even if someone doesn’t believe they are likely to be hurt by a risk, they might support further research or policy against it.

Basically, terrorists run into trouble because people use correspondent inference theory to infer the intentions of others: the results of their actions are assumed to be concordant with their intentions. If a person sweeps the floor we assume he wants it clean (but he could just be working off excess energy). If somebody hits somebody else, we assume the intention was to harm (but it could just be a game). Similarly, people infer that the horrific deaths of innocents is the primary motivation of a terrorist – which likely leads to a misunderstanding of the real goals of the terrorist.

This is bad news for terrorism as an effective coercive means to political or social ends. Although the terrorist can state his demands and goals, people will tend to assume that he is just a sadist rationalising. Possibly a dangerous sadist one has to occasionally acquiesce to, but the goals are not seen as essential to him. His "real" goals are assumed to be the destruction of society, and this makes accepting demands less favorable. Abrams finds empirical support for this in that terrorists are much more likely to succeed with their demands if they focus their attacks on military goals rather than civilian ones, and if they have minimalist goals (evicting a foreign power, winning control of a piece of territory). Attacking civilians or wanting to change the world makes people assume the intention is something else.

This analysis assumes bias among the non-terrorists making them unwilling to play along, but clearly there are plenty of biases among the terrorists too. The correspondence makes them impute evil intentions to governments that behave clumsily or violently. The emotional salience of terror probably introduces a lot of availability bias, impact bias makes terrorists overestimate the emotional effect of their actions, groupthink is likely pretty big within terrorist grooming communities and so on.

It seems that one could probably analyse terrorism in terms of cognitive biases quite fruitfully. Whether that will lead to ways of reducing terrorism is another matter. Maybe unbiased terrorists will simply see that the Bayesian thing to do is simply to go home since terror doesn’t work efficiently – or they would start making non-hyperbolic long-term plans for surgical strikes that simply cannot be misunderstood. Conversely, maybe terrorists could be incited to bias themselves into inefficiency, but highly biased people can occasionally be dangerous. Maybe the real aim should be an unbiased anti-terror strategy – but as long as politicians and public are biased they will likely see the unbiased strategy as wrong.

Geier, Ronzin & Doros demonstrated that people tends to regard a unit of some entity is the appropriate and optimal amount by measuring how much people consumed free Tootsie Rolls or pretzels when provided in different sizes, or M&M’s provided with differently sized spoons. This likely explains why people tend to eat more when served larger portions. The authors suggest that the unit bias in food might be social: people don’t want to seem to be gluttons. Another possibility they suggest is that there is a culture-norm interaction: we package things in appropriate sizes, we learn the appropriate amount by being exposed to standard packages.

A third possibility is of course an aversion to wasting, whether instilled by mother or evolution. I have a fourth neurocognitive possibility: we run on hierarchical motor programs and tend to switch behavior when one of them has concluded. So consuming a unit would presumably be a single iteration of one such program. We can certainly learn more elaborate programs like "take unit; consume until full; leave the rest", but that requires ongoing monitoring that may be cumbersome or easily distracted. I would expect unit bias to generalise outside food too. The researchers point out that double features are rare but long movies are not, and that people take one ride on an amusement park ride regardless of whether it is 1 or 5 minutes long. I would also expect unit bias to tend to round our thinking towards the nearest integer number of convenient units.

Some months ago when I moved to the UK I made the deliberate decision to only buy Coca Cola in sixpacks rather than 1.5 l bottles. The result is that I consume much less, since I now only take a can instead of more or less continually refilling my glass. So clearly unit bias can be used to downregulate food intake too. It is just that there is no incentive for the food sellers to do it. Maybe one solution to obesity would be easier ways of dividing bought food into convenient smaller units?

The authors claim that this result does not fit with "rational weigher" models where people try to maximize their utility, nor with "irrational weigher" models where cognitive biases and bounded rationality dominates. Rational  individualists and communitarians ought not differ on their risk evaluations, and the authors claim it is unlikely that different cultural backgrounds would cause differing biases. They suggest a "cultural weigher" model where individuals don’t simply weigh risks, but rather evaluate what one position or another on those risks will signify about how society should be organized. When people learn about nanotechnology or something similar, they do not update instrumental risk probabilities but develop a position with respect to the technology that will best express their cultural identities.

This does not bode well for public deliberations on new technologies (or political decisions on them), since it seems to suggest that the only thing that will be achieved in the deliberations is a fuller understanding of how to express already decided cultural/ideological identities in regards to the technology. It does suggest that storytelling around technologies, in particular stories about how they will fit various social projects, will have much more impact than commonly believed. Not very good for a rational discussion or decision-making, unless we can find ways of removing the cultural/ideological assumptions of participants, which is probably pretty hard work in deliberations and impossible in public decisionmaking. 

In Adam D. Galinsky, Joe C. Magee, M. Ena Inesi, and Deborah H Gruenfeld, Power and Perspectives Not Taken, Psychological Science, 17:12, 1068-1074 2006 researchers primed a group of test subjects by asking them to write down a memory where they held power over other people, while another group were asked to write about a time when others had power over them. Then the subjects were asked to quickly write the letter ‘E’ on their forehead.

High-power subjects were about three times as likely as low-power subjects to draw the letter oriented so it would be readable by themselves rather than readable by others.

In follow-up experiments it was found that high-power subjects also tended to assume other people had the same information that they had (the "telepathic boss" problem – the boss assumes that everybody knows what he knows and want). They were also less accurate than low-power subjects at judging emotional expressions. There were also anticorrelations between reports of general feelings of being in power in one’s life and tendency to take other’s perspective. Overall high-power people seem to anchor too heavily on their own vantage point and this impairs their ability to consider what others see, think and feel.

People with less power likely have to consider other people’s intentions and views more strongly, so perhaps the power bias is actually the real baseline and powerless people concentrate more on mind reading. But given the increase in errors in emotion reading the power mode people had compared to people primed neither with being powerful or powerless, this seems unlikely.

What are the implications of a power bias? In general power bias would make the empowered people tend to think they have more support from others in their views than they have. Altruists in power would be even less concerned with individual variations in goals and values – i.e. they would tend to become more egalitarian and paternalist. Egoists in power would become more concerned about the ambitions of others, i.e. paranoid.

Is this bias rational? When leading other people the cognitive load of taking their perspective might be cumbersome, and the increase in stereotyping that seem to occur in people in the "power mode" might also be a form of attention management. Imposing one’s own goals onto others might also make them obey more effectively. For leading people to perform particular goals this mode might work better. The downside is that if the task is heavily reliant on individual achievements meshing together or more based on voluntary action a lack of perspective risks missing early signs of trouble and will produce rebellion. The researchers suggest that power and perspective taking might not have to exclude each other and that responsible leadership might be possible by learning to take both into account. But they do not cite any actual experiments showing that it works.

Maybe we should just promote people with Asperger syndrome to management in favour of people with intact theory of mind. That way we will not reduce the total human ability to see things from other perspectives.

Another troubling source of bias is false confessions triggered by this confirmation bias and then strongly supporting the erroneous conclusion. The Psychology of Confessions by Kassin and Gudjonsson reviews this. During the preinterrogation review police, believing themselves to be better at detecting deception than they are, tend to confidently make false positive detections of deception in innocent people. Once they have convinced themselves they have caught a suspect a police interrogation then becomes guilt-presumptive and rather effective in generating false confessions, in particular in cognitively challenged people. And finally, juries and judges are easily convinced by the confessions.

Nothing of this may be total news to anybody on this blog, but it is still rather worrying how strong biases are accepted in police investigations and the legal system. Maybe the counter-hypothesis trick at least could be made part of police procedure: at certain points during investigations the investigators have to state possible disconfirming hypotheses for the record.

There have been some surveys of funding bias. Bekelman, Li & Gross find that 25% of investigators in their material had industry funding sources. Doing a meta-analysis of 8 articles themselves evaluating 1140 original studies they got a 3.6 odds ratio of industry favourable outcomes when there was industry sponsorship compared to no sponsorship. There are also problems with data sharing and publication bias. An AMA 2004 Council Report also points out that sponsered findings are less likely to be published and more likely to be delayed.

A case study of co-authoring a study with the tobacco industry by E. Yano describes both how the industry tried to fudge the results (probably more overtly than in most cases of funding bias) and how the equally fierce anti-tobacco campaigners then misrepresented the results; the poor researcher was in a no-win scenario.

Looking at these studies it seems that there is a general tendency even for unsponsored researchers to get industry-positive findings. My guess is that this is a form of publication bias: positive results are easier to publish, and in many fields there might be a correlation between positive results and them being positive for the industry (e.g. testing whether drugs work on various conditions).

Maybe there is also an anti-industry bias here? Leaving aside obvious cases of non-industry bias such as from government sponsored agencies seldom finding faults with government policies there could be signalling happening here. Do non-corporate funding agencies favour researchers not accepting corporate funding? Such researchers would be less likely to leave for corporate funding, making it more likely a long-term relationship could be built up. They would also be more interested in researching areas the agency finds relevant and in general be more “loyal”. This might be important if there is competition between funding agencies for “good” research projects (that will bring publicity, status, relevance and possibly advance political goals). To a researcher it might hence be rational to distance oneself from corporate funding in order to ensure more non-corporate funding.

I wonder whether the funding outcome bias is worse than the funding publication bias, and whether the 25% of research funding might not make up for some of the bias?

Warning, back-of-envelope calculation done before going to bed follows (this really should be modelled more properly, for example by simulating independent trials and summing them with Fisher’s method).

Unbiased studies get positive findings with probability f, and then get published with probability P. Negative findings get published with probability p (< P). Biased studies have a slightly higher positive finding probability kf (1/f > k>1), a higher publishing probability lP (l>1) and a lower negative finding publishing probability mp (m<1). The total amount of reported positive findings will be [0.75+0.25(kl)]Pf, and negative findings [0.75(1-f)+.25(1-kf)m]p.

The odds ratio between sponsored and unsponsored studies would be

OR = (kflP/(1-kf)mp) / (fP/(1-f)p) = (kl/m)(1-f)/(1-kf).

If f=0.5, a finding bias of k=1.56 would explain the JAMA findings (assuming no other biases). If f=0.1 (an area where positive results are very hard to get), it would have to be 2.8. Pumping it up equally with l and m is harder (a very low m is probably the best bet and hardest for outsiders to notice). Differentiating, I get the following sensitivities to changes in k,l and m: OR/k+kOR/(1-kf), OR/l and -OR/m. So for f=0.5, OR=3.6 and k,l,m around unity the sensitivity of k becomes 10.5, and the others have just 3.5. A small change of finding bias can produce a sizeable change in the OR, so that may be the best bet of where the main bias source would lie. 

This also implies that funding bias might be hard to get rid of without cutting funding totally. Experimenter bias is hard to get rid of, and just the emotion of thankfulness might be enough to induce a slight k. Encouraging the publication of negative results (increasing p) or forcing the publication of trials (increasing m) has relatively little effect.

The variance of the Bernouilli-distributed trials f(1-f), so estimating f from the studies would have variance f(1-f)/N where N is the number of trials. If N is reduced by 25% by removing all sponsored trials the variance increases by 33%. It seems that this would actually outweigh the benefits of removing the funding bias if the number of studies in a field are modest or the bias is not too large. So maybe just having researchers to declare their competing interests and then taking them into account when evaluating the research field might be the best way of getting a truth estimate?