Average Your Guesses

What percentage of the world’s airports are in the USA? (Answer below the fold.)

Take a guess. Now, take another guess, different from the first one, and average them. According to research reported on in The Economist, averaging the 2nd guess improves accuracy by 6.5%. Better still, wait 3 weeks before taking a second guess. Averaging now improves accuracy by 16%. (Story found via Slashdot.)

Here is the full report from Pschological Science.  Some excerpts:

It is important that neither group knew they would be required to furnish a second guess, as this precluded subjects from misinterpreting their task as being to specify the two endpoints of a range.

That could make it a little tricky to do this on your own; you have to try to make your first guess as good as you can, and then start fresh for the second guess.

This benefit of averaging cannot be attributed to subjects’ finding more information between guesses, because second guesses were less accurate than first guesses

Hmmm, how can second guesses be less accurate than first guesses, yet averaging them is more accurate than either? I suppose it means that the two guesses tend (on average) to bracket the correct answer, with the second guess farther away than the first one. That means that your first instinct to improve your guess is more likely than not to be in the correct direction, but go too far. Knowing this might allow you to improve your guesses even more.

Oh, and as for the airports? According to the CIA World Factbook, there are 14,947 airports in the U.S., and 49,024 in the whole world, so 30% of the world’s airports are in the U.S. For the record, I guessed 25%, and then 15%, so averaging didn’t help me. But in general this might be a useful trick to easily improve guesses.

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • Unknown

    My first guess was 30%, my second 10%. So this supports the idea that the first guess is more accurate, but not that averaging helps.

  • Apep

    My first guess was 25% and my second was 35%. I thought both were too high.

  • http://yudkowsky.net/ Eliezer Yudkowsky

    My first guess was 25% (based on oil usage) and my second guess was 50% (thinking more about private airports).

  • Andy Wood

    My first guess was 20% and my second was 30%, so the trick brought me closer. Before looking at the answer, I wasn’t confident I was even in the ballpark.

  • Nominull

    I guessed 32% (off the top of my head) and then 12% (based off the fraction of the world’s economy that’s in the U.S.). So yeah, second-guessing myself didn’t help.

  • http://www.mccaughan.org.uk/g/ g

    Apep, what you’re saying is that you made a guess, you thought it was too high, so then when you had to guess again you guessed higher? Cor.

    Hal, your original post is missing a “y” in “Psychological Science”.

  • http://cabalamat.wordpress.com/ Philip Hunt

    I guessed 20% (first number that came into my head) and 25% (USA’s share of world economy).

    Incidently the average of everyone’s guesses so far is 25.3%.

  • Divide

    Hey, this doesn’t seem to work! I guessed 30% the first time. (With 10% being the second guess.)

  • Tiiba

    I remembered the US having about 25% of the world’s GDP (vague memory), and considered that airports are a luxury, so rich countries probably have more, relative to GDP. So I thought 1/3, which seems close enough. By the time I realized I need a second guess, it was too late. Can we play again?

  • http://uncrediblehallq.net/blog/ The Uncredible Hallq

    I guessed 50% and 1/6th, based on half-remembered ideas about natural resource usage, economy size, etc. Final result: 33%, almost exactly on the mark.

  • Anonymous Coward

    Hah. I guess I’m one of the ones for whom the averaging helped – I guessed 40 and 20, to end up with 30 on average. I didn’t have too much reasoning behind the guesses though. I thought the US would have a lot more than their share of airports, so I guessed 40, which seemed like a lot; I thought about it a bit more and figured I should adjust that guess downward, it seemed high, so I guessed 20.

    I could definitely have been biased by the fact that I would have to make two guesses, since I saw that sentence as I was thinking about what guess to make. This entry really isn’t that great of a test because of that. But hey, it was never really meant to be.

  • http://www.xuenay.net/ Kaj Sotala

    At first I guessed 20, then 30. Average is 25, so it helped.

  • http://hanson.gmu.edu Robin Hanson

    This fits with the idea of temporary mental noise – even if there is more mental noise in the second estimate it is better to average the two estimates to reduce the noise.

  • James Andrix

    I also used the 25% of the wealth number for my first guess.
    Then I visualized Major cities around the world (‘earth at night’ type images, mainly) but completely failed to think about minor airfeilds. So I guessed 50. Average 37.5

    Second guess was sloppy, so I lost 2.5%.

    On the other hand, My first guess might have only been so close because it I had a roughly related statistic to go on. if I had only remembered my other vague number about the US being 6% of the population, I might have started with that and increased it a bit to account for wealth. If I doubled it, and still had the same second guess I’d be cheering this method.

    So maybe a topic that is too well-known yeilds a first guess that is too good for this method to work reliably.

  • Si

    I’ll have to think twice before buying another lottery ticket.

  • Grant

    I wonder, does this second later guess say anything about the common folk-wisdom to “sleep on” important decisions?

  • http://imaginarypolitics.wordpress.com Unit

    First guess 60%
    Second guess 40%

  • Si

    @”I wonder, does this second later guess say anything about the common folk-wisdom to “sleep on” important decisions?”

    It suggests to me that you can’t ever “start fresh for the second guess” as Hal suggested you should.

  • Daniel Wolfe

    This reminds me of a novel by John Brunner, called The Shockwave Rider. In the novel, there exist these groups called Delphi Pools, which consist of a group of normal people who all make guesses as to what the turnout of something is going to be. Surprise surprise, their averaged guesstimate is actually pretty close to the mark.

    So, I know it’s been used in fiction before.

  • michael vassar

    First guess 25%, based on fraction of world economy 10-20 years ago, second guess, based on the observation that poor countries have limited highway infrastructure and need more very small airports, was 10%. I suspect that the second guess is more accurate for sufficiently small airports despite what the article says. I’d give even odds on the USSR having more airports than the US for light craft and the remainder of the world more than either alone, making the world factbook number impossible. I wonder what definitions they use.
    Sorry Robin and Hal but experience teaches me to be something that you probably consider overconfident.

  • Nick Tarleton

    “Airports: This entry gives the total number of airports or airfields recognizable from the air. The runway(s) may be paved (concrete or asphalt surfaces) or unpaved (grass, earth, sand, or gravel surfaces) but may include closed or abandoned installations. Airports or airfields that are no longer recognizable (overgrown, no facilities, etc.) are not included. Note that not all airports have accommodations for refueling, maintenance, or air traffic control.” – Notes and Definitions

  • Gwern

    Daniel Wolfe: That’s an interesting fictional analogue. How closely related to the Delphi method ( https://secure.wikimedia.org/wikipedia/en/wiki/Delphi_method ) are Brunner’s Delphi pools?

  • anonymous

    I guessed 10% and 50% – the average is 30%!

  • http://users.ox.ac.uk/~ball2568/ Pablo Stafforini

    Just like Hal Finney, I guessed 25% the first time and 15% the second time. The phenomenology of second-guessing is quite interesting. Updating your original guess in one direction feels right at one moment, and the next moment what feels right is an update in the opposite direction. I moved back and forth between 15% and 35%, and only settled for the former figure because I was impatient to see the right response.

  • Apep

    Yes, I thought my first guess was too high and I guessed higher the second time because that was what came to mind first. The mental sequence went more or less like this: “25%. Hmmm. That seems a little high. 35%. So maybe 15% is a better guess.”

  • http://shagbark.livejournal.com Phil Goetz

    I guessed 25%, but I had an unfair advantage – I designed NASA’s airspace simulator, and had a list of all the international airports in the world. I believe I had just under 20,000 US airports in my simulation. That would include all airports for which ETMS data is filed, but would exclude a large number of private & small-plane airports.

    And when I said I guessed 25%, I was really guessing 1/4.

  • http://cs.helsinki.fi/u/hirvinen Patrik Hirvinen

    Daniel Wolfe, Gwern: The Delphi method is different from but related to Delphi pools, which (IIRC) were initially just averages of guesses by a set of people but very soon incorporated betting on outcomes, essentially becoming prediction markets ( http://en.wikipedia.org/wiki/Prediction_market ), a topic Robin Hanson has written a lot about. See http://hanson.gmu.edu/ and search OB for those.

  • http://www.evangelicaloutpost.com/archives/2008/06/thirty-three-th-62.html the evangelical outpost

    Thirty Three Things (v. 68)

    1. The Market and Human Nature — from a 2005 QA with the conservative scholar Roger Scruton (via Rod Dreher): MG: What deleterious consequences result from the “free market ideology” you mention? Are there particular economic arrangements that co…

  • http://sarbayes.org/ctwardy/blog Charles Twardy

    Just a note: the paper does not claim that the average is always better than either guess! Only that you should expect it to be so. Of note, second guesses were worse than first guesses (overall, not always), but it was still better to average (overall, not always).

  • Phillip Huggan

    This effect is because you tend to guess in regular interval jumps. Like fives or tens. So your first guess is closest, and the next is 10% away in the right direction. Later guesses might tap unconscious brain work. 20% and 25% for me.

  • Pingback: links for 2008-07-31 [delicious.com] | The 'K' is not silent