To Win Press, Feign Surprise

Fossil hunters have found a winning formula for getting media attention: pretend to believe behavior X appeared around the time of the earliest known fossil evidence for X, and then feign surprise when an earlier fossil overturns such estimates.  Consider these five media stories:

Mammals Linked to Earlier Flight Mammals may have taken to the skies much earlier than previously believed, … a fossil of a … creature that lived in Mongolia about 125 million years ago. It bears evidence [of] a skin membrane … providing enough lift for it to glide through the air. … lived tens of millions of years before the earliest confirmed record of bats taking wing about 51 million years ago. 


Scientists Mystified by 2,100-Year-Old Device The wheel was part of a device so sophisticated that its complexity would not be matched for a thousand years — it was also the world’s first known analog computer. … the device overturned conventional ideas that the ancient Greeks were primarily ivory tower thinkers who did not deign to muddy their hands with technical stuff.

Discovery of the World’s Earliest Sculpture … found a set of ivory carvings dating back to 30,000 years ago in … southwest Germany.  …  This discovery completely overthrows the traditional idea that art developed gradually from that of the primitive and rough style to the fine, contemporary craftsmanship today. "It seems that the first modern humans in Europe were astonishingly precocious in their skills," says Sinclair.

Tools unlock secrets of early man  New research shows early humans were living in Britain around 700,000 years ago, substantially earlier than had previously been thought.  … flint tools unearthed in Pakefield, Suffolk, were 200,000 years older than the previous oldest finds.  … One of the team, … said the discovery …  was startling.  "Until recently I certainly would not have believed that there would have been humans this far back," he said.

Earliest fire sheds light on hominids   There is already good evidence for hearths that are 250,000 years old, and it was widely believed that the first controlled handling of fire occurred 400,000 to 500,000 years ago.  But an analysis of burned remains … now proves that fire was tamed at least 300,000 years earlier than that. …a breakthrough in terms of understanding the evolution of hominids: the fact that they were using fire so early tells scientists a great deal about their abilities and behaviour at the time.

The public loves to hear a story of academics shocked, just shocked, by new findings.   But there is an obvious bias: we hear lots of stories about data forcing estimates to be earlier, but hardly ever stories about data forcing estimates to be later. 

Given continued new earliest fossil finds, it is quite unreasonable to estimate that the earliest behavior started about the time of the earliest known fossil.  Either an academic ban on "speculation" creates a bias in academic estimates, or fossil hunters allow misleading media impressions, thereby gaining media attention.  In either case this would be a great application for betting markets on science, to give the public a more accurate consensus.

Addendum:  I should note that our best estimate for anything should always follow a random walk; any systematic deviation from such a random walk suggests bias. 

GD Star Rating
loading...
Tagged as: , ,
Trackback URL:
  • http://profile.typekey.com/ocmpoma/ ocmpoma

    Not to disagree with your post, but I can see one factor at least contributing to the “new find pushes back date for x” tendency: If evidence for x is found in the fossil record, it will be assigned a date, as earlier fossils are found, the date can be pushed back. But it seems more unlikely to me that something could be found in the fossil record that contradicts evidence found earlier in the record. Thus, as earlier fossils are shown demonstrating x, academics can be duly shocked for the press. But it must be hard to find a fossil that both provides evidence for x and refutes other, earlier, fossils that were thought to demonstrate x.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Ocmpoma, agreed, new fossils are unlikely to contradict earlier fossils.

  • http://pdf23ds.net pdf23ds

    Being ignorant of archeology, I would ask whether any of their date estimates are based on largish samples of the thing under study. If you have found 20 or 200 instances of pre-historic sophisticated sculpture, and they all fall after a certain time period, then you probably have a good reason to believe that that kind of sculpture wasn’t common or even existent before that period. On the other hand, if you have only two or three relevant finds (which might be more common with fossils), then you really have no idea.

    Then there’s the possibility that, even in prehistory, you could have the rise and fall of cultural centers of some sort that make these things much more non-linear. Maybe there was a 30,000 year slump in the sculpture-making practice. If that sort of variability was common in late hominid evolution, then there might not be any reliable way to accurately assess the beginnings of such developments.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    pdf, yes of course these are not situations where you have enough data to estimate “accurately.” But estimate they do, which is not in itself unreasonable. The unreasonable thing is to have bias in the estimates.

  • http://pdf23ds.net pdf23ds

    Well, what I’m suspecting is that the bias is in the reporting instead of the science. For instance, say that before a find of artifact X, scientists thought that X’s began to be produced between 80,000 and 20,000 years ago (as a 90% confidence interval). X is discovered to be 30,000 years old, and scientists revise their estimates to 70,000 – 35,000. (70,000 because with each additional find close to the recent end of the range, it becomes less likely that anything older would be found.) I would not be at all surprised if, in general news reports on the matter, the far end of the range were elided, giving the false impression of bias you write about.

    In other words, never trust science reporting.

  • http://pancrit.org Chris Hibbert

    Another possibility is that the reporters are misinterpreting “earliest date we have evidence for” as “earliest date we believe”. And some scientists (predominately the field workers, right) would stick scrupulously to “earliest date we have evidence for.” Those doing the speculating would be less likely to be present at the dig site, or the ensuing press conference.

    In order to track the bias, we need to dig up what they said last week and last year, not what they’re currently saying they thought last week or last year.

  • http://econlog.econlib.org/archives/2006/12/robin_hansons_i.html EconLog

    Robin Hanson’s Intelligence Continues to Amaze Me

    Well, it shouldn’t, explains Robin. Consider the foolish way that the world reacts to new fossils: Fossil hunters have found…

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Chris and pdf, this is such a regular pattern that I don’t think it makes much sense to call it a misunderstanding. A fossil hunter who has seen a sequence of articles like this and is being interviewed is quite capable of making this point clear if he wants to. So either they choose not to make this point clear, or reporters pointedly ignore them when they do. I find the first theory more plausible than the second. (I’ve now had 160 media articles mention or quote me, so I know a bit about the reporter-reportee relation.)

  • Douglas Knight

    Current theories should not follow a symmetric random walk–there is skew. It’s easier for a single find to push the date back a lot in one step. Moreover, such a dramatic find is more likely to get media attention. Pushing the date forward would be more likely the result of a failure of evidence to show up, incremental and not newsworthy.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Douglas, I’ll admit that is theoretically possible, but I question whether that is what is actually going on.

  • David J. Balan

    The same phenomenon appears to work in reverse in Biblical Studies, where the way to make a splash is to find that some text was really written way later than anyone had previously thought.

  • http://videogameworkout.com Glen Raphael

    It works in climatology too, with the tendency to feign surprise at findings that it’s now “the hottest decade” since various earlier times. Whatever date you come up with, play it up in the press release as if you find it surprising and you’ll get headlines.

    An amusing recent example was when the National Academy of Sciences examined the “hockey stick” controversy and decided to /reject/ earlier controversial claims in the published literature that it’s recently “been the hottest in 1000 years” in favor of finding that one could only claim with confidence that it’s now the hottest it’s been in 400 years, which is to say that we’ve been on a warming trend since the Little Ice Age. Following the press release, headlines read “HOTTEST IN 400 YEARS”; you had to read the articles carefully to realize this was a backpedalling from earlier, much larger, estimates.

    (for instance: http://www.msnbc.msn.com/id/13474997/ )

  • http://drzeuss.blogspot.com Dr. Zeuss

    An alternative is that information that forces estimates to be earlier (like finding a new fossil) comes more rarely but changes our estimates drastically, whereas information that force estimates to be later (look, another year without a new fossil) comes more frequently but has smaller effects. If you assume that the media only reports changes in our estimate larger than a given absolute value, then we get the effect you’re talking about.