Silencing Outsider Status

Me last week:

Paul Davies, chair of the group that decides what SETI scientists will do if evidence of aliens is ever found, thinks … until scientists can say something to the public with great (~99%) confidence, they should say nothing. … Most early low-probability signs … being false alarms is “damaging to the credibility of science.”  So until scientists can confidently say that an asteroid will hit us or that we see aliens, they should just whisper to each other. … One might justify this confidence-or-silence policy by arguing … reporters are biased to present low probability news as if it were high probability.

Today’s Post:

NASA … reopened a 14-year-old controversy, … reaffirming and offering support for its widely challenged assertion that a 4-billion-year-old meteorite that landed thousands of years ago on Antarctica shows evidence of microscopic life on Mars. … Fourteen years of relentless criticism have turned many scientists against the McKay results, and the Mars meteorite “discovery” has remained an unresolved and somewhat awkward issue.  This has continued even though the team’s central finding — that Mars once had living creatures — has gained broad acceptance. …

Critics had said that the magnetites could have just as easily existed without bacteria or biology — that they sometimes form as a result of the shock and searing heat that could come, for instance, from an asteroid strike. But … [a] recent paper … reported that the purity of the magnetites made that explanation impossible. … “All the criticisms of our original paper got widely distributed, but when we did the work to prove the critics were wrong, it hardly made a ripple. … We’re now in a position to say we’ve knocked down all the criticisms — and our biological explanation is the one left standing.” …

At the conference, a leading cautionary voice in astrobiology proposed that a special protocol be established to oversee release of any journal articles making dramatic extraterrestrial claims. Andrew Steele … compared the absence of astrobiology review with the formal procedures set up by scientists involved with the search for extraterrestrial life, or SETI.  He said that SETI leaders understood the societal sensitivity of their work and that it was time for researchers in astrobiology “grow up and do the same.” (more)

Yet another voice for muzzling!  It seems clear to me that scientists do not usually insist on such high standards of confidence for publication.  Most Research Findings Are False seems pretty clear evidence, as does the high rate of celebrated new medical treatments that are later repudiated, and the very low marginal health-effectiveness of medicine.  I suspect I see similarly low standards for publications that are pro-global warming, or that warn of low science funding or manpower.  If the standard of evidence for publication varies with the topic, we can’t explain it via a generic tendency for reporters to exaggerate findings.  So what explains this variation?

Here I’ll channel Tyler Cowen, and suggest this is mostly about how real events echoing stories we tell change which intellectuals get more status.  Think of all the movies you’ve ever seen of an outsider intellectual unfairly rejected by establishment scientists.  Evidence of aliens, or a Really Big Disaster are prototypical.  Well establishment scientists see those movies too, and they don’t want real stories like them to appear in the media. They correctly perceive, for example, that a story confirming aliens would raise the status of UFO nuts, relative to establishment academics.  Similarly, news about a really big disaster would raise the status of “the sky is falling” outsiders.

On the other hand, establishment academics correctly perceive their status would be raised, relative to outsiders, by more stories of promising new medical treatments, of the seriousness of global warming, of the need for more science funding, or that a new result “might lead to a new theory of everything.” Even if such stories turn out later to be wrong.  Why?  Because we hear many similar stories about heroic scientists discovering treatments, or warning of enviro disaster, and few stories about such scientists being later wrong.

I see two effects:

  1. There are some long standing disagreements between insider and outsider intellectuals in our society, and any news that confirms outsider claims raises outsider status.
  2. News about a real event about you that matches a commonly-told story in which you’d be a hero, raises your status.   If that news is later reversed, that won’t reverse your status, if there aren’t commonly told stories about you being a villain in a news reversal story.
GD Star Rating
loading...
Tagged as: , , , ,
Trackback URL:
  • http://t-a-w.blogspot.com/ Tomasz Wegrzanowski

    I wonder why you’re not opening intrade markets for things like existence of life on Mars – yes, it’s play money (real money markets are far more effort to setup), but it seems to work about as well. I’d gladly win some play money from you, as case of this meteorite seems quite solidly closed to me.

  • Khoth

    I’m unconvinced. Sure, discovering alien signals could improve the status of UFO nuts relative to scientists in general. But the silencing is being suggested by SETI scientists, and their status would be massively increased if they found something.

    It seems more likely that they’re worried about false alarms making them seem like the UFO nuts.

  • Bill

    Can there be an academic equivalent of yelling “Fire” in a crowded theatre?

    In other words, shouldn’t the level of certainty be related to the adverse consequences.

  • http://ynglingasaga.wordpress.com Rolf Andreassen

    On the subject of the certainty required for publication: In particle physics – which is quite high-status within science – there are rules for using the terms ‘evidence’ and ‘observation’. If you see a signal that is three standard deviations above your background, you can say this is ‘evidence for’ whatever you’re looking for. At five standard observations you can claim ‘observation of’. If it’s less than three sigma, you can only publish it as a ‘search for’. The origin of this rule is in the sixties when people were publishing low-quality results right and left and claiming new particles, basically because they were looking for “Anything interesting in such-and-such a mass range” and when you split a given range into a hundred bins, it’s fairly likely you’ll see something at two or three sigma. These days it is required that you state beforehand what you are looking for and where you expect to find it.

  • http://hanson.gmu.edu Robin Hanson

    Tomasz, intrade is real money. They don’t pay interest on deposits, which penalizes long term questions.
    Khoth, SETI scientists feel pressure from other scientists.
    Bill, what adverse consequences?
    Rolf, it is still the sixties in many other fields.

    • Bill

      Here are several examples of adverse consequences of some academic yelling “Fire” in a crowded room.

      Most stem from the problem of allocating resources.

      Someone yells: I have this improbable cure for cancer. Everyone else drops what they’re doing and pursues this avenue of research, depleting the funding of programs that have higher probabilities of success, including the one that is the cure for cancer.

      The person yelling “Fire” usually has a conflict as well. They are, afterall, the “Authority”, and so they will be hired to conduct the study.

      Now, with the proper funding from GMU, and specifically your research budget, I would be happy to discuss this further.

    • Bill

      One could even take the Mars biology example as well.

      Say the claim results in a big hunka funded project based on that rock study and the new study takes away funding for another project, making a cancer drug in a weightless atmosphere.

      I think you do have to consider tradeoff, and you also have to consider how the public will amplify sloppy scienctific claims, like it or not, making it difficult to do that which has a higher probability of success.

      Shouting Fire when you see what you think is a glimmer of light can cause a stampede.

    • http://hanson.gmu.edu Robin Hanson

      Bill the issue is why the consequences in this field are different, to justify different standards.

      • Bill

        OK, now I understand that this is limited to the field of space, particularly aliens and catastrophic asteroid crashes.

        And, the question is: why treat this differently than other events of a similar nature.

        The first question is: Is the premise true–Ask the question: Are there in fact things other than aliens and asteroid crashes where people seek more confirmatory evidence before they speak and excite the public. Yes. Potential nuclear reactor defects, potential underwater oil well leaks [oops, forget about that], tolerance levels for pesticides and combinations thereof. There are probably more, so fill in the blank here…..

        Second, what are the consequences of the Fire Drill–will resources be displaced that have greater risk elsewhere, will greater risks be assumed than the expected value of potential risks avoided [what is the probability the risk can be avoided may also be a question]. In other words, is the magnitude of the harm we create in avoiding a low probablility event even worse.

        Third, does the Fire Drill involve things that people have a difficult time conceptualizing or analyzing rationally, or, to put it another way, is it the kind of thing I will hear incessantly on Fox News and will devolve into something I will later see on South Park.

      • Bill

        And, finally, I am always skeptical of something that has a very high magnitude of harm and an infinitely small probability of occuring.

        Some people try to justify this as well, saying, look at the expected value of loss.

        Sort of like Pascals wager: if God exists even with a small probability, then I’d better be good; to which I reply: if there is a big hairy monster out there, and if you eat Tulips on Tuesday, then no harm will befall you from the hairy monster: that is the same argument too, and I do not eat Tulips on Tuesday.

        My experience does not include big hairy monsters, but they may exist, and in fact, I have a program you should fund to find out if they do or do not exist.

    • http://t-a-w.blogspot.com/ Tomasz Wegrzanowski

      Intrade has both real money (intrade.com) and play money (intrade.net) versions.

      As all real-money prediction markets suffer from serious problems (lack of interest, very high fees, dubious legality), I only use their play-money versions as an intellectual exercise.

  • Khoth

    I still don’t see it. If alien life is confirmed, the report will come out sooner or later and have all the status effects that a confirmed report of the existence of aliens would have. The case where the silencing makes a difference is when the evidence isn’t good and it turns out to be false alarm, and the status effect of that isn’t to elevate crackpots, it’s to drag scientists down to their level.

    • Khoth

      Oh, a relevant difference between this and the medicine ones:

      Scientists claiming that chemical X cures disease Y, or that it doesn’t, is commonplace, and even if that particular claim is wrong, it won’t have much of a status effect on anyone except for the few who published the claim (except in the case when it becomes widely accepted before being discredited).

      On the other hand, claiming to find alien life is very much in the domain of crackpots, so false claims of alien life will make scientists look like a bunch of nutters far more than false claims about drug efficiacy.

  • http://hanson.gmu.edu Robin Hanson

    Khoth, your argument would seem to apply equally well to tentative results in any field, yet the standards are set higher in the alien life field. The point is to explain variations in where the standard is set.

  • Jess Riedel

    This is tangential, but….

    > Most Research Findings Are False seems pretty clear evidence, as does the high rate of celebrated new medical treatments that are later repudiated, and the very low marginal health-effectiveness of medicine.

    Isn’t the very low marginal health-effectiveness of medicine explained simply by our willingness to generally spend almost any amount of money on emergent life-extending measures? (This willingness–which is hard to philosophically reconcile with the fact that we put an effective price on life in many other non-emergent situations–might then be explained by some sort of signaling if you wish.) This has always been true; it’s just become a problem recently because we’ve only just acquired the technology which enable us to spend huge sums of money for tiny life extensions. (In the past, there wasn’t anything you could do for someone with brain cancer.)

    So can we really take the very low marginal health-effectiveness of medicine as evidence “that scientists do not usually insist on such high standards of confidence for publication”?

  • http://ynglingasaga.wordpress.com Rolf Andreassen

    >Rolf, it is still the sixties in many other fields.

    Yes; I was holding up my own field as the standard to which others should aspire, not as an exemplar of how all science is done. :)

  • http://don-thelibertariandemocrat.blogspot.com/ Don the libertarian Democrat

    Given the current views on Illegal Immigration in the US, I seriously doubt that aliens and time travelers will be identifying themselves here.