Overcoming Bias

Share this post

Reporting Chains Swallow Extraordinary Evidence

www.overcomingbias.com

Discover more from Overcoming Bias

This is a blog on why we believe and do what we do, why we pretend otherwise, how we might do better, and what our descendants might do, if they don't all die.
Over 11,000 subscribers
Continue reading
Sign in

Reporting Chains Swallow Extraordinary Evidence

Robin Hanson
Mar 27, 2007
Share this post

Reporting Chains Swallow Extraordinary Evidence

www.overcomingbias.com
14
Share

In January I claimed:

An extraordinary claim is usually itself extraordinary evidence … I would be very unlikely to make such claims in situations where I did not have good reasons to think them true.  The times to be more skeptical of unlikely claims are when there is a larger than usual chance that someone would make such a claim even if it were not true. 

Eliezer responded, and then I outlined a formal model.  I now have a working paper.  In it, I consider the effect of people being organized into a reporting chain, such as up the levels of an organization, or from researcher to referee to editor to reporter to editor and so on.  The new interesting result:

When people are organized into a reporting chain, noise levels grow exponentially with chain length; long chains seem incapable of communicating extraordinary evidence.   

For example, imagine disaster frequency was inverse in deaths, that we faced one big enough to kill us all, and that nature gave someone a signal about the casualties, a signal with a (lognormal) standard deviation of a factor of ten.  But imagine the news of this signal passed through a chain of seven noisy people, each of whom adds 10% to the signal standard deviation.  While nature’s signal itself is clear evidence of an unprecedented disaster, the signal that appears at the chain’s end gives a median estimate of about one death; our warning is lost.

This suggests we reduce the length of our reporting chains, such as in efforts to make organizations more innovative by reducing layers of management:

Perhaps we need only the first few levels of reporting the discretion to adapt a claim to context; reports at further levels could be something like “Fred said that our best estimate of disaster deaths is one million."  Another alternative might be to use prediction markets to shorten reporting chains; the person who saw nature’s signal trades in the market, anyone who wants to correct that market price for context does so via trades, and everyone else is referred to the market price. 

Share this post

Reporting Chains Swallow Extraordinary Evidence

www.overcomingbias.com
14
Share
14 Comments
Share this discussion

Reporting Chains Swallow Extraordinary Evidence

www.overcomingbias.com
Overcoming Bias Commenter
May 15

This discussion has many resonances with Eliezer's discussion in the next post of the "decay" of knowledge with transmission from generation to generation. In that case we are more or less forced into a linear chain with no way (in the case of e.g. God speaking of Moses) of going back and rechecking the phenomenon. So the decay is structurally inevitable.

Science of course promotes short chains by encouraging replication and black boxing. Nobody has to "trust" the PCR works -- they get PCR materials and equipment and depend on it every day in their lab. All of molecular biology is a distributed daily replication of PCR. And this pattern holds for essentially all major scientific results, although less dramatically.

At the risk of getting boring, this is another piece of (what should be) a general theory of judgment aggregation. The community around this blog seems to be stepping smartly in that direction.

Expand full comment
Reply
Share
Robin Hanson
May 15

Dagon, no doubt it is easier for networks to communicate info that can be cheaply and repeatedly generated at will by many dispersed people. Unfortunately the most interesting extraordinary claims are rarely of this sort.

Expand full comment
Reply
Share
12 more comments...
Top
New
Community

No posts

Ready for more?

© 2023 Robin Hanson
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing