Take Our Survey

Grad student Jason Briggeman and I would love for you to take our overcoming-bias related survey, and perhaps win a $100 Visa gift card.  I wish I could say more about it here, but we don’t want to bias your answers.  After we get enough answers and analyze them, we’ll tell you what we found.  Please don’t read the comments until you’ve taken the survey. 

GD Star Rating
Tagged as:
Trackback URL:
  • That was an interesting survey. I’m looking forward to seeing the results and analysis.

  • wcw

    I didn’t love it, but it could have been worse.

    Will you post the anonymized data set, or just email your tables?

  • Most. Unrepresentative. Sample. Ever.

  • Eliezer, we will advertize the survey in many places, and control for place in our analysis. If anyone wants to post an ad for the survey somewhere else, email me for a different link to use.

  • TGGP

    I would have figured in most moral/ethical quizzes my responses would be of the less respectable sort, but the nature of the questions allowed me to appear more benevolent. I agree on the unrepresentative nature of the quiz. I only got close to the correct answer on the foreign aid because I had already been reading Bryan Caplan on it. Since I’m opposed to any and all foreign aid, I would normally be the type of person who would greatly overstate it, but as it is I slightly underestimated it.

  • Hopefully Anonymous

    TGGP, that revelation is a shocker. I’d think like me you’d be more likely to tie your support of a foreign aid instance to your self-interest, rather than go with a bright line oppositional rule on something as complex and varied as every possible instance of foreign aid.

  • wcw

    RH, Shall I take your ignoring my question as a no? I am disappointed, in the event. Not surprised, but disappointed.

    EY, matching is a black art, but you can do a decent amount with the demos this survey and most others collect.

  • wcw, Jason and I never talked about that, so we don’t have a policy yet.

  • Mark Seecof

    About the foreign-aid question… some of us are well informed about nominal government spending on “foreign aid,” but think we should count (a) US private charity to foreigners (an order of magnitude larger from the USA than other developed countries), and (b) domestic spending on illegal aliens as part of the US’ “total foreign aid” spending.

  • Mark Seecof

    Hmmm. Next time, add another bonus question to your survey: “What percentage of the US Federal budget would you prefer to spend on foreign aid (apart from bribes for military cooperation, dumping of subsidized crops, and other spending for clearly non-altruistic reasons)?”

    I’d bet many people would pick a large fraction in a vacuum, a smaller number if first “primed” with information about current tax incidence and spending priorities (social programs, military operations, interest payments, etc.), and zero if they realized that the Federal government has no Constitutional authority to tax Americans to subsidize foreigners for supposedly-altruistic reasons.

  • I’m so good! I only overestimated last question by 15%. And I didn’t even know the number offhand. I did know that it’s one of those numbers that people tend to overestimate. But that knowledge didn’t bias my answer or make me reduce my guess. Well, I don’t think so anyway.

  • hesitant

    Do you track response times? (You should.)

    The one question I hesitated was increasing someone’s IQ by 60 points. Unless the random individual was exceptionally dumb to begin with, this would create a genius (4 SD), quite likely the sort who would have difficulties communicating with the the rather dull rest of the world.

    That might not make that person any happier. I also briefly considered the “evil genius”, but then decided that no person of that intelligence could CARE enough to do purposeful harm.

    In all other questions I did not hesitate to push the “do good” button.

  • Hopefully Anonymous

    I doubt increased intelligence results in increased communication difficulties. For example if “g” is applicable across multiple cognitive task domains, such as the task of effective communication.

  • Hesitant, I pondered that as well with the larger IQ changes, but I decided someone with an IQ ~300 could probably figure out a way around the problem. (Am I mis-recalling a +200 IQ question? Did we all get the same question set?)

    I was surprised to see that I guessed the foreign aid question exactly. I must have read that somewhere, because I was inexplicably drawn to the number despite thinking it was too low of a percentage to be right. I’ll blame Bryan Caplan, too.

  • Stuart Armstrong

    Must admit, I didn’t find the questionnaire very good.

    People’s answers are very much affected by the order of the questions in this questionnaire (is this something that you are trying to analyse?). I personally said yes to all IQ increases; but had I been more uncertain, the various number quoted would have forced me to an arbitrary decision based mainly on what the first ones to turn up were (the problem is not as bad for the hidden treasure, as the cash amounts are clearly differentiated and everyone has a good grasp of the amounts involved). Life extension falls somewhat between those two extremes (as everyone has a clear grasp on what extra years of life means, but the difference between 6 extra years and 4 is not so easy to grasp).

    After the first few questions, I was hoping that the test would cause me difficulty, or would expose contradictions in my reasoning, but that never happened (I might be atypical in this respect). What would have tripped me up is if the benefits had extended to more than one person (one person with +200 IQ is fine; one million people with +200 IQ is a different issue).

    Finally, two small points: the option of disapproving of giving extra treasure “because money is a positional good” was not included. And the question “strongly Democrat?” is unclear (damn! my politics exposed! 🙂 Would a Marxist count as strongly Democrat, or not Democrat at all? More to the point, would two Marxists interpret the question in the same way? What of those voting Democrat out of self-interest, rather than conviction? I follow US politics quite a lot, so I could answer these political questions adequately, but for most of the people over here, the equations are simply Democrat = Clinton = Good, and Republican = Bush = Bad, irrespective of their own political opinions.

    I know it’s far too late to change the questions, and I do appreciate your efforts in doing such a survey (it’s far more than what I’ve done today), but I thought you might want the feedback, even if negative.

  • So many of the reasons were things I hadn’t even thought of. I guess that’s one-reason decision-making for you.

    I did consider the “evil genius” problem on the +60 question, but decided that, conditioning on the rather unlikely event of the +4 SD hitting someone already on the far right, humanity was in enough trouble at this point that the risk of a bad outcome was worth the chance of a positive miracle. I don’t remember seeing a +200 question – that would have given me serious pause, since the result would probably depend very strongly on the form of the intelligence enhancement.

  • LP

    I did get the +200 IQ question, and elected not to push the button, because I have no idea whether an intelligence boost of that magnitude (if possible) would be a benefit or cost to the affected person.

  • TGGP

    Hopefully Anonymous, I suppose it is possible that the foreign aid could go to a person who invests the money in a high-return manner and, for no discernible reason, then gives the money to me. The way I figure it though, foreign aid taxes U.S citizens (and I am one) and gives the money to people who are non-U.S citizens (which, logically, I am not). I don’t really see a reasonable circumstance in which it is in my self-interest.

  • The “reasons not to give a random person $10M” section left out the most obvious one (for me): making a random person suddenly wealthy decreases my own relative purchasing power. I still would do it given the parameters of the problem (chances are the person will be a poor Indian slumdweller, and it won’t affect me), but consider the same problem over a small closed economy, say a town of 10000 people. The sudden appearance of a millionaire in a small economy would likely have dire effects on everybody else, if the millionaire chose to outbid everybody else for scarce resources.

    Whether or not you agree, I’m rather suprised that such an obvious argument would be left out — aren’t you folks economists?

  • savagehenry

    The only thing I didn’t press the button for was giving someone $1,000,000,000,000. Giving one person that much money who isn’t me seems like it might be a bad idea. I know I’d just spend it on old punk records and space ships; I don’t know what some random person would do with the ability to buy several nations. I wasn’t concerned with a decrease in relative purchasing power so much as that much money gives someone an awful lot of opportunity to do great harm. Even a couple billion dollars will only buy you so much harm (or so much good). Ten million is barely enough to buy a big house in some places, but a trillion dollars is too much to risk putting it in the wrong hands.

  • zoop

    mtraven: I might be missing something, but it seemed as though the question indicated that a person would discover wealth in the form of a treasure worth $10 million, rather than actual money in any form. The finder will presumably sell the treasure to an interested buyer for the quantity stated in those questions. Since it is wealth rather than money that is added to the economy, no one’s purchasing power will be diminished as a consequence of the discovery.

  • Richard Pointer

    I think it is funny that someone would worry about one human being having that much wealth. We give bureaucracies this kind of money all the time, and the last time I checked they can cause just as much havoc. See Iraq.

  • I probably would have been more generous had I not read Taleb’s books. His discussion of how one gets lucky, especially the comparison to Russian Roulette, would make me very reluctant to push that button unless the dollar amount wouldn’t have the person set for life. $100K would probably be my threshold, depending of course on how it’s closed in. Way back in the day (1992ish), I wrote some software for behavioral scientists to probe these thresholds experimentally. The results, as I understood them, were that the path to the indifference point was very important to the indifference point reached and could be parameterized on an individual basis.

    Taleb’s metaphor explained a lot of things I’ve observed in life. People who got lucky once beyond any reasonable explanation generally being aholes… People who made their luck repeatedly being worth trying to emulate. My own good and bad luck — things beyond my real control that made things great or made things stink. It sure will be interesting to see how a wide sample of people who are probably unfamiliar with Taleb’s framing of this issue answer the questions and the reasons they indicate.

  • Johan Edström

    I got increasingly frustrated at the questions as I felt the answers I gave would be misinterpreted. A part of it may have been personal realizations of my inconsistencies in value judgements.

    I was very stingy with my manipulations, mainly because I was unsure of wether the total of the effects would be positive or negative.

    In real life I can strategically *choose* options, places to interfere. This was more like poking around randomly in a very complex system, where if you don’t know what you are doing it’s unlikely you’ll get it right.

  • LemmusLemmus

    Uh-oh. I don’t know the survey’s objectives, but I feel that this questionnaire is going to reveal nothing but the limitations of Likert-type scales. Won’t almost everybody always say she’s “very likely” to do the thing that benefits the other person? No variation, no explanation.

    As for the foreign aid question, I knew it was somewhere around 1 percent, and guessed it would be a little less, so I typed in the number “2”.


  • I way undershot the foreign aid too. I knew it was higher, I think Africa gets close to 10, but I didn’t think I had undershot it by 15 billion. I thought it was closer to 15 or 20, but wanted to undershoot.

  • 1% is 24.7, not two.

  • I thought too hard on foreign aid. I think I knew it was in the mid-20s, but I also knew it was about 0.2% or 0.3% of GDP — and then I got confused between the budget (given) and GDP (not given)…

  • LemmusLemmus


    that was the point of the second part of my post.

  • tacking 1 day or 1 month to a person’s life is not necessarily a good thing. They’d probably be just more death-bed days which can be excrutiating.

    The foreign aid answer is questionable. Deductible charitable gifts are considered by most policy makers, and in Supreme Court dicta, to be “expenditures”. Americans give a lot to foreign causes. More important are military expenditures to be the world’s police (Iran & Iraq are expected to cost $161,000,000 in FY 2008) and the subsidies and other items mentioned previously.

  • Sorry, I just got too bored trying to plough through the (seemingly) interminable tables of how much attention I was supposed to have paid to multiple factors on some previous question, when I hadn’t analysed my thought processes in such detail in the first place (or attempted to remember them)…

  • agent00yak

    I said yes to the trillion dollar question only because I thought I could trade on the economic consequences of the treasure. I would short precious metals, go long auction houses and position my portfolio to benefit from a higher than expected liquidity environment. Otherwise I probably would have decided against pushing the button.

  • Michael Handy

    The 1 day and $1 trillion questions gave me some pause. but in the end i only lowered the 1 day and 1 trillion to somewhat likely. I figured that all things being equal, 1 day may well be highly beneficial in someones actions.

    While the possibility of $1 trillion destabilising the world economy was sobering,in the end I figured things would stabalise eventually, and it would be fun to see what happened and compare to economists models. The data gained from such an occurance may be worth the risk

  • Doug S.

    I didn’t get a $1 trillion question, just a $1 billion question. I hesitated there and in the end picked “prefer not to answer.” A billionaire is an extremely powerful person and assigning that much wealth to a random person may have negative consequences. (On the other hand, randomly making someone into a superhuman genius actually seems less likely to screw up the world; presumably they’d be smart enough to figure out that the world isn’t a zero-sum game and helping yourself means helping others and vice versa.)

    By the way, it would be awesome if someone found $1 trillion worth of platinum, although I’d be worried if they got to keep all the profits from selling it. Platinum is rare and is incredibly useful as a chemical catalyst; if there were more platinum, there really would be more wealth in the world. Gold is not that useful, as far as I can tell – it’s a great conductor of electricity (much like copper) but there don’t seem to be that many uses for large quantities of gold that can’t be accomplished with more readily available substitutes.

    The situation is different if the “buried treasure” was something of no intrinsic value. I wouldn’t want a random person to discover a large amount of buried cash; that would simply transfer resources from other people to the lucky finder through the mechanism of inflation. Finding rare collectibles – baseball cards, comic books, etc. would be neutral; they don’t increase the capacity of the world to produce goods and services, but they don’t reduce the wealth of people who don’t buy them.

    That economics textbook I had for my one mandatory economics course in college was the most effective indoctrination I’ve ever read. 😉 It’s certainly more persuasive and compelling than the various religious tracts I’ve come across. As a result, I’ve started to think more like an economist. For example, I now believe in comparative advantage and free trade. If I have to move to India to get a technical job, so be it.

  • Just thought of something.

    The average american IQ is ~105. What’s the distribution?

    I occasionally say to friends that “the average IQ is only 105, but probably none of you know anyone with that dumb, or even deal with one on a normal day.”

    What’s the mode? I bet the frequency of injury, illness, and syndromes and the seeming upper bound/low instances of savants to cause the curve to be skewed negative (or have multiple distributions).

  • This also reminds me of something, occasionally you’ll see that the average IQ is about 104 and the military is slightly higher at 105. But the military doesn’t admit the lower end, so it won’t be as skewed. It probably has a lower mode (but probably still has two curves, there’s a small subpopulation of very intelligent people.).

  • _Felix

    I don’t believe IQ tests measure anything meaningful, I don’t associate ethics with spirituality, I think obeying the law is important for selfish reasons but not the measure of a good citizen, and I never went to school or sat an exam but people tend to assume I have a degree. Therefore I had to rely on “prefer not to answer the question” a hell of a lot.

  • I picked “choose not to answer” when asked how much “ethical / spiritual considerations” (or something along those lines) influenced my answer. Reason being, I wasn’t sure of what was meant by that – in my view nearly all of the other reasons given would fall under the “ethical considerations” bit.

  • TGGP

    I don’t believe IQ tests measure anything meaningful
    What you believe is dead wrong, if you consider predictive power in hordes of life outcomes to be “meaningful”. You might want to read g: A precis

  • John Maxwell

    Gee, this survey sounds really interesting. Can someone send me a link to the survey and possibly the results?