Confidence or Silence

When prestigious academics evaluate the vita (i.e., publication list) of another academic, they want to see only top journals listed there.  A vita with five top journal articles and ten medium journal articles looks worse to them than a vita with just five top articles; if you can’t publish in the very top journals, they’d rather you didn’t publish at all.

Paul Davies is chair of the SETI Post-Detection Taskgroup.

Paul Davies, chair of the group that decides what SETI scientists will do if evidence of aliens is ever found, thinks similarly about science news: until scientists can say something to the public with great (~99%) confidence, they should say nothing.  (Quotes below.)  You see, frequent public updates on science issues of great popular interest, like evidence of aliens or asteroids headed toward Earth, would result in reporters bothering scientists at work with “mayhem”, disrupting their “lines of communication,” and disturbing their “dispassionate analysis.”  The fact that most early low-probability signs would end up being false alarms is “damaging to the credibility of science.”  So until scientists can confidently say that an asteroid will hit us or that we see aliens, they should just whisper to each other.

In the extreme case of receiving an actual alien message directed at us, Davies prefers scientists to kept quiet for the many years it would likely take to decode it fully.  And he prefers aliens to not send us any useful tech info, as then we would fight over who could decode it first.  How disruptive!

One might justify this confidence-or-silence policy by arguing either that non-scientists are biased to overreact to low confidence news, or that reporters are biased to present low probability news as if it were high probability, and non-scientists gullibly believe them.  I have not seen any systematic evidence presented in support of these claims, however.

Within academia, the bias against non-top articles seems like signaling.  Since folks confident they are great would not admit they’d ever done work that could not meet the highest standards, medium journal publications reveal a lack of confidence.  Similarly, I suspect signaling is behind the confidence-or-silence policy.  Since it is harder to credibly say something with great confidence than with low confidence, saying something with low confidence sends a bad signal about your abilities.  Keeping info secret is also a status move; info gives control and control marks status.

Quotes from Eerie Silence:

The fact that it may take days to be sure that a [SETI] signal is not manmade raises a very serious problem for managing the post-detection agenda. … Any hint of a positive result from a SETI project could immediately trigger media frenzy, and events might soon spiral out of control. … In the case of the SOHO satellite detection, the press got hold of the story even before the identification was made.  Fortunately the reporter concerned acted responsibly and waited for more data before rushing into print.  But not all members of the media can be relied upon to be so restrained. …

Because SETI astronomers are professional scientists, rigorous checking is an essential part of their training, and they want to be sure of their ground before making a definitive statement. History has shown that when scientists run to the press with sensational claims that haven’t been properly checked, the outcome is very damaging to the credibility of science itself, not to mention the reputations of the scientists involved.  A salutary lesson in how not to handle the media comes from the now largely discredited claim for cold nuclear fusion … They held a hasty press conference, and the media understandably had a field day. … It took many months for laboratories around the world to test the claim, and find it wanting. …

The lesson from that debacle is that it is wise to exercise restraint when dealing with the media about discoveries that carry sweeping implications for society. In the case of SETI, … once word got out, mayhem could ensue.  The astronomers might show up for work only to find their observatory besieged by journalists, … hardly an environment conducive to dispassionate analysis.  Even normal modes of communication are likely to be disrupted as lines become jammed by callers eager to check the rumors. …

It is in the nature of this type of investigation that false alarms greatly outnumber the real thing, so the above scenario might be played out many times, with the hullabaloo eventually subsiding as the story evaporates. A close analogy is the all too frequent announcement that civilization is menaced by an oncoming asteroid or comet. … When a new asteroid or comet seems to be moving on an Earth-crossing trajectory, … carefully checking takes time.  In the early days following the discovery, the projected orbits are uncertain because of normal measurement errors.  After the object has been followed for several days or weeks, the errors shrink enough that the astronomers can then work out whether it will or won’t hit Earth.  The most sensible strategy is to wait until the orbit has been properly determined, and only then, if there is still a clear and present danger, ‘wake the President’.  But … more often than not, the press get wind that a new object has been found that might strike our planet on the next orbital pass. It makes a wonderful scare story: ‘Killer asteroid may wipe out life as we know it!’  Headlines like that attract a lot of readers. … The known uncertainty in the measurements lets astronomers work out the probability of a collision – typically it is about one in 10,00 when the object is first identified. …

I personally believe the public does have a right to know, even if the news is bad – as soon as the situation is properly understood. I have yet to meet with a SETI scientist who doesn’t agree with this basic principle. There is no ‘code of secrecy’ in SETI. … Suppose the … discovery holds up at, say 99 per cent confidence level. …  If an astronomer were to spot something weird, which on closer inspection bore all the hallmarks of artificiality, then I believe it should be announced just like any other major astronomical discovery. … If an alien civilization were to send us a customized message, … I personally feel that the implications of simply  receiving such a message would be so startling and so disruptive that, all though eventual disclosure is essential, every effort should be made to delay a public announcement until a thorough evaluation of the content had been conducted, and the full consequences of  releasing the news carefully assessed in light of the Taskgroup’s recommendations. …

Information about the astronomical coordinates of the transmitter should be restricted to the astronomers involved … Even governments … no doubt would also want to take charge.  In my view, however, the less government involvement at the evaluation stage, the better. … Decoding it could take a very long time, perhaps involving years of meticulous work before we had any idea of what we were dealing with. … Most worring of all would be [a message] that merely handed us on a plate a revolutionary item of technology, e.g. a new source of energy, or a technique for engineering designier life forms reliably.  The problem here is that the gorup that possessed the knowledge first would be in a position of incomparable power. … Outright warfare might follow the scamble to grab the information. One can only hope that the aliens would recongnize the dangers and refrain from handing out scientific secretes like sweets.  [pp172-184]

GD Star Rating
loading...
Tagged as: , ,
Trackback URL:
  • Jess Riedel

    non-scientists are biased to overreact to low confidence news, or that reporters are biased to present low probability news as if it were high probability, and non-scientists gullibly believe them. I have not seen any systematic evidence presented in support of these claims, however.

    I think both of these are definitely true, but I am especially surprised that you aren’t confident about the first. People modify their diets all the time based off of summaries of sketchy studies. (I think eggs are a classical example of a food that has gone in and out of fashion amongst non-scientists a number of times based off flimsy evidence). People believe the new, extremely speculative theories in physics as if they were gospel. People believe historical reconstruction on the history channel. Etc., etc.

    Basically, most people will take anything reported as science to be true so long as they aren’t emotionally or materially invested in it.

    • http://robertwiblin.wordpress.com Robert Wiblin

      I agree with Jess, those things seem very probably true.

      There may be a status thing here – ‘we scientists know better than you all leave it to us’. In the situation Davies is describing that’s probably a reasonable opinion! Popular science reporting is constantly spreading results that are marginally significant and won’t be repeated. Knowledge of science would bet better if we weren’t constantly bombarded by such misinformation. At a glance, a cost-benefit analysis of ‘reveal SETI information’ immediately versus wait suggests waiting is the better idea.

  • http://hanson.gmu.edu Robin Hanson

    Jess and Robert, are you sure you guys are correcting sufficiently for observation selection? The cases where the public overreacts are far more visible than the cases where they under-react. There is a vast steady rain of small science news out there to which the public seems to hardly react at all.

    • http://robertwiblin.wordpress.com Robert Wiblin

      We only need to know whether the public would overreact wastefully to alien news, not all kinds of science news. Alien signal news is sexy, so I think we would over-react to events which have only a small chance of being real.

      Perhaps the reaction will be harmless – lots of news stories people enjoy reading. Perhaps it will be harmful as Davies says and make it hard for the scientists to get their work done.

      What do you think are the downsides of waiting, especially given the longest you could in practice go without leaks would be a few days?

    • Sarah

      Well, reaction is a different thing. Lots of people don’t react to science news; lots of people don’t even react to news about climate change, which is well popularized and generally goes in the same direction. People don’t necessarily act on what scientists tell them, but they do tend to understand science news as a representation of what scientists believe with certainty. (And then reject the scientists if it turns out not to be true.)

      The tendency to only publicize fairly certain findings may magnify this tendency in the public, though. I’ll hear statements like “Those scientists are always changing their minds!” (as a statement of scientists’ unreliability) instead of “Those scientists are never certain.”

  • Robert Koslover

    Thank goodness that the learned astronomers (peace be upon them!) are making sure that we uneducated, ignorant, and irrational peasants won’t have our delicate, unstable, and inferior brains damaged by exposure to potentially confusing or disturbing information sent to us by space aliens! Surely, these deeply noble folk have only our interests at heart! See: http://www.youtube.com/watch?v=kKQSghkrAds .

  • http://jensfi.blogspot.com/ dWj

    I think most scientists believe “reporters are biased to present low probability news as if it were high probability”. Most science reporting in major newspapers or even in some science magazines (like Discover) seems to conflate speculative theories with more well-established science; it tends to read to me like the writers simply don’t understand probability enough to understand the difference.

    • michael vassar

      I simply don’t see any plausible hypotheses as to how this could *not* be true. Selection that we have little reason to doubt exists guarantees that reporters will be like this. How often do you see science stories which emphasize the speculativeness of a story or which try to communicate precisely how strong the evidence is? It simply wouldn’t fit into the “science story” format.

    • http://hanson.gmu.edu Robin Hanson

      michael, it could be that readers expect such exaggeration, so no net harm done.

      • michael vassar

        That would still do net harm, as it would reduce the ability of scientists and journalists to communicate actually important and dramatic results. Dilution of any reputational currency, be it fiat money, facebook friends, grades, or reports of breakthroughs is inflationary and potentially destructive of information.

        BTW, I’d like some data to confirm the claim that 5 articles in top journals is better to publish than 5 in top journals and 10 in other journals. That doesn’t ring true to me.

      • http://meteuphoric.wordpress.com/ Katja Grace

        Readers expecting such exaggeration would be exactly the fear as far as scientists are concerned, though not a utilitarian justification of keeping quiet. Readers don’t expect just learn to expect exaggeration peacefully – they put blame for it on someone. Some of that goes to scientists.

  • noematic

    Scientists appear to view explanations of their findings, whether high or low confidence, to the public at large to be a fairly low status activity. However, wouldn’t it also be something of a capital preservation exercise for scientists to themselves reveal preliminary findings publicly, with the necessary caveats and in a manner over which they have control, pending formal findings, than have reporters selectively do so?

    This would, to some degree, minimise sexed up reporting, ensure that the scientists’ findings are not revealed, or selectively revealed, by some other source before they are formally presented and would leave the scientists relatively free to conduct further evaluations with minimal disruption. There is some risk in presenting preliminary findings if later formal findings turn out to be less significant than anticipated. However, this could be minimised by limiting definitive statements and conclusions drawn at the preliminary stage.

  • http://www.angryblog.org Brian Moore

    Robin: ” There is a vast steady rain of small science news out there to which the public seems to hardly react at all.”

    Such as this one, where, Livermore scientists confidently declare they’ll have fusion tech by this summer:

    http://edition.cnn.com/2010/TECH/science/04/28/laser.fusion.nif/index.html?hpt=C1

    Now, I’m certainly apprehensive, but then what do I know? All I know is that if you are really going to have fusion by this summer, that is huge news — and yet as far as I know no one really is talking about it.