What Do We Know That We Can’t Say?

I’ve been vacationing with family this week, and (re-) noticed a few things. When we played miniature golf, the winners were treated as if shown to be better skilled than previously thought, even though score differences were statistically insignificant. Same for arcade shooter contests. We also picked which Mexican place to eat at based on one person saying they had eaten there once and it was ok, even though given how random is who likes what when, that was unlikely to be a statistically significant difference for estimating what the rest of us would like.

The general point is that we quite often collect and act on rather weak info clues. This could make good info sense. We might be slowly collecting small clues that eventually add up to big clues. Or if we know well which parameters matter the most, it can make sense to act on weak clues; over a lifetime this can add up to net good decisions. When this is what is going on, then people will tend to know many things they cannot explicitly justify. They might have seen a long history of related situations, and have slowly accumulated enough relevant clues to form useful judgments, but not be able to explicitly point to most of those weak clues which were the basis of that judgement.

Another thing I noticed on vacation is that a large fraction of my relatives age 50 or older think that they know that their lives were personally saved by medicine. They can tell of one or more specific episodes where a good doctor did the right thing, and they’d otherwise be dead. But people just can’t on average have this much evidence, since we usually find it hard to see effects of medicine on health even when we have datasets with thousands of people. (I didn’t point this out to them – these beliefs seemed among the ones they held most deeply and passionately.) So clearly this intuitive collection of weak clues stuff goes very wrong sometimes, even on topics where people suffer large personal consequences. It is not just that random errors can show up; there are topics on which our minds are systematically structured, probably on purpose, to greatly mislead us in big ways.

One of the biggest questions we face is thus: when are judgements trustworthy? When can we guess that the intuitive slow accumulation of weak clues by others or ourselves embodies sufficient evidence to be a useful guide to action? At one extreme, one could try to never act on anything less than explicitly well-founded reasoning, but this would usually go very badly; we mostly have no choice but to rely heavily on such intuition. At the other extreme, many people go their whole lives relying almost entirely on their intuitions, and they seem to mostly do okay.

In between, people often act like they rely on intuition except when good solid evidence is presented to the contrary, but they usually rely on their intuition to judge when to look for explicit evidence, and when that is solid enough. So when those intuitions fail the whole process fails.

Prediction markets seem a robust way to cut through this fog; just believe the market prices when available. But people are usually resistant to creating such market prices, probably exactly because doing so might force them to drop treasured intuitive judgements.

On this blog I often present weak clues, relevant to important topics, but by themselves not sufficient to draw strong conclusions. Usually commenters are eager to indignantly point out this fact. Each and every time. But on many topics we have little other choice; until many weak clues are systematically collected into strong clues, weak clues are what we have. And the topics of where our intuitive conclusions are most likely to be systematically biased tend to be those sort of topics. So I’ll continue to struggle to collect whatever clues I can find there.

GD Star Rating
loading...
Tagged as: , ,
Trackback URL:
  • Stephen Diamond

    But on many topics we have little other choice; until many weak clues are systematically collected into strong clues, weak clues are what we have.

    1. You (or anybody else I know of) has never shown that knowledge develops by combining weak clues into strong clues. The modern Kuhnian account of science’s development isn’t consistent with your (Baconian?) perspective.

    2. When near-mode clues are excessively weak, one is often better off ignoring them and relying on far-mode intuition, despite its biases (if you can’t just say “I don’t know”). You haven’t shown that your weak clues are better than (biased) intuition.

    3. You’ve never presented arguments or evidence showing that prediction markets are anything more than a weak clues. You really have no idea of how much it would cost to properly motivate the “investors.” You don’t know that the cost would be lower or the results better than hiring a stable of experts for any given problem. At best, you know what kinds of issues prediction markets work best with; not how well they work. (And actual markets don’t work that great — http://tinyurl.com/ke2oj98 and http://tinyurl.com/cgnt4lq )

    • Michael Vassar

      This is scientism taken to the point of insanity. As far as I can tell, you seem to be saying that tacit knowledge hasn’t been proven to exist, which is ludicrously wrong by any scientific standard. Also, you are confusing Kuhn with Popper. Kuhn says that within a paradigm, weak anomalies are largely disregarded until enough of them accumulate to suggest a new paradigm and then the fight begins.

      • Stephen Diamond

        You need to look at Kuhn on preparadigm “science.” When a paradigm does develop, it ignores the “weak clues” previously accummulated.
        This has nothing to do with denying tacit knowledge, which rests on the intuition that “small clues” try to displace.

        Here’s an example. Current “futurologists” try to derive the probability of other intelligent life in our galaxy from a few facts, including the brute fact of our existence. (Katja used to be a specialist in this kind of “small clue” mongering.) What’s the likelihood that such “weak clues” will play a role in a paradigm science? They represent tangential trivia made salient by our almost total ignorance.

  • Russ

    The basic issue you are describing is something I have been observing in myself … The mental experience of being uncertain is uncomfortable, its not a pleasant experience to recognize that one don’t know much, its fundamentally unsettling, so I find myself taking information (or clues as you say), and extrapolating more extreme judgments than they justify, merely to make myself feel better and relax.

    It strikes me that we humans don’t like accepting the differing shades of grey, it needs to be black or white for us otherwise we experience the angst of dissonance.

    So its not surprising that people have more extreme views than the evidence justifies, this happens because it makes us feel better … I can’t see this changing anytime soon.

    • B_For_Bandana

      Recently I decided that I don’t know enough to have political positions on most issues, that I don’t have much power anyway, and so I wouldn’t waste time thinking about them. Amusingly, I soon became very confident of this, and proud that I was a rare person who understood this amazing, incontrovertibly true insight.

    • Stephen Diamond

      The basic issue you are describing is something I have been observing in myself.

      Not the same. You are talking about a bias to overemphasize weak clues. Robin is using what you call a bias to justify a “scientific” practice oriented toward “weak clues.”

  • Erich Schwarz

    “… a large fraction of my relatives age 50 or older … can tell of one
    or more specific episodes where a good doctor did the right thing, and they’d otherwise be dead. But people just can’t on average have this much evidence, since we usually find it hard to see effects of medicine on health even when we have datasets with thousands of people.”

    Or, maybe, the relatives are telling you the truth, and the studies were done incompetently. Has this possibility really never crossed your mind?

    • lemmycaution

      Doctors can help with infections especially the gastro-intestinal infections that kill boatloads of younger hunter gatherers. Antibiotics work. A lot of the stories are about heart disease and cancers for which the benefit of medical interventions is a lot shakier.

      http://courses.washington.edu/evpsych/Hiwi-hunter-gatherers-JHE2007.pdf

      Hiwi mortality data present a very different picture of human adaptation than that derived from mortality statistics in modern society. Not only are mortality rates much higher in the Hiwi, they are disproportionately higher among infants, children, and young adults (cf.Gurven and Kaplan, in press). The relative importance of major causes of death in the Hiwi vs. modern America is also striking. In the U.S., the major killers are heart disease and cancers, both almost nonexistent among hunter-gatherers. In modern America, respiratory diseases are among the top 10 causes of death, but gastrointestinal infections are virtually absent as a cause of death. Suicide is more important than homicide in modern America, and homicide accounts for only 1% of all deaths. Finally, because of automobiles, accidents make up almost an equal portion of deaths in both modern societies and in hunter-gatherers (all U.S. data are from Zopf, 1992).

      • Douglas Knight

        Antibiotics save lots of lives and the people whose lives they saved don’t even know it.

    • IMASBA

      “Or, maybe, the relatives are telling you the truth”

      Yeah, take a look at a random American street and you’ll see lots of people who would’ve been dead without medical care. The severely obese alone, there are millions of them… and then there are all the people who’ve had appendicitis. This is really a bad example, even more so because prediction markets can’t work for individuals (were YOU saved by a doctor?)

      • Erich Schwarz

        Why, actually, yes, I was saved by a doctor. I would have died of intestinal volvulus at age 36, without emergency surgery. Thanks for asking.

  • BenGolden1

    Doesn’t Bayesianism suggest that small clues, such as statistically insignificant score differences, actually should affect our beliefs, albeit marginally? Further, doesn’t it provide the math for how to aggregate small clues to create large ones?

    I suspect that people’s views on mini-golf talent aren’t actually that different after one game, especially if it’s close. With a larger sample size, our views change a lot more.

    • dmytryl

      > Further, doesn’t it provide the math for how to aggregate small clues to create large ones?

      The small clues are often not statistically independent so naive “Bayesian” aggregation, much touted by self described “bayesianists”, doesn’t work.

      Aggregate of weak clues in most case remains a weak clue.

      • BenGolden1

        In a simple case like mini-golf scores, the clues are largely independent, and the selection process is typically pretty unbiased, no? If we go a dozen rounds of putt-putt and compare our scores, each individual round is a small clue, but the aggregate is a potentially large clue. Is this a very special case?

      • dmytryl

        If you count them properly, yes, if you rely on memory of wins and losses, and when the skill disparity is not big, just consider how people end up really believing in role of luck and skill and such in the games of pure chance, in supernatural-ish foresight powers, and so on.

        Even more so for low grade evidence in form of what you think is near and what you think is far and so on.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        instead informing us of Hanson’s political views and such.

        Less politely, EMs are for Robin Hanson a sophisticated signaling device concerning his political alignment. (Signals gain effectiveness when they look unwitting or unintentional.)

        It would perhaps be more interesting to analyze what Robin is signaling than to discuss EMs (although Robin’s technical audience might disagree). He’s not signaling pure libertarianism; some erroneously think this means he isn’t engaging in political-team signaling at all. What he is signaling is quasi-libertarian economics combined with social and political authoritarianism. (Recall that Friedman supported Pinochet.) Robin has stated that he’s more worried about society erring toward permissiveness than toward excessive authority.

        This isn’t unusual: “libertarianism” usually means liberty for the upper-middle class and above; oppression for everyone else–and not only by “market forces.”

      • Marc Geddes

        Michael Anissimov (keen Yudkowsky fan) now calls himself a ‘neo-reactionary’ and on the ‘More Right’ blog he enthusiastically explores ideas such as restoration of a monarchy and benign dictatorship. I’m pretty sure that was the plan all along, only now he’s finally honest about it. The AIs will rule, with a monarchy of ‘rationalists’, and Yudkowsky as emperor.

        http://www.moreright.net/

      • http://entitledtoanopinion.wordpress.com TGGP

        Friedman didn’t “support” Pinochet. He visited Chile and gave the same talk that he gave in communist country. He explicitly opposed the regime. It was Hayek who supported Pinochet. And I don’t think Posner was ever considered a libertarian. He seems more like an irreligious conservative (though many conservatives may dislike his Keynesian macroeconomics). However, his focus on “efficient law” does sound similar to Hanson’s prioritization of efficiency over liberty.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        1. Sorry for the imprecision about Friedman: perhaps in memory I blended him with Hayek, who in any event is unquestionably libertarian, making the same point. Whether Friedman “supported” Pinochet is a matter of interpretation. What he did was meet personally with the blood-drenched dictator and wrote him a personal letter which formed the basis for Pinochet’s economic policies. After reviewing the facts, I would say his stance toward Pinochet was that we on the left would call “critical support” (kind of analogous to Trotsky’s stance on Stalin’s regime, although Trotsky was far more open about attacking Stalin’s murderous policies than Friedman was about Pinochet’s).

        2. Whether Posner was ever considered a libertarian depends on who is doing the considering. Perhaps “libertarian icon” was too strong, since some libertarians disown him, but much of the disowning is recent, following the changes in his economic analysis after capitalism’s recent economic debacle. But to say flatly that Posner was “never considered” a libertarian is also too strong, inasmuch as many people consider him libertarian and Posner has described himself in those terms. ( http://mises.org/daily/2470/ ) Posner is certainly more libertarian than an irreligious conservative would be expected to be–sometimes in a good way, such as opposing drug laws. I would say Posner is indeed much like Hanson in basic philosophy: a libertarianoid pragmatist.

      • http://entitledtoanopinion.wordpress.com TGGP

        Fair enough for the most part. I definitely think of Posner as more associated with “pragmatism”. His fondness for Oliver Wendell Holmes is held against him by many libertarians.

    • IMASBA

      “I suspect that people’s views on mini-golf talent aren’t actually that different after one game”

      Actually they are, this is one of those things our minds are minds are wired to mislead us on and it has devastating consequences in politics and the economy where tiny differences in skill, or even random fluctuations can catapult people to stardom because people tend to not believe in coincidence (“if you win you must be the best” is hardwired in our brains explains why people with an IQ below 200 can actually become billionaires).

  • snarkyxanf

    I’m curious what sort of examples you had in mind with “personally saved by medicine”. I’m not sure medical studies are good guidance as to the strength of medical evidence, since more or less by definition, a study is investigating questions which are at the limit of evidence.

    Conversely, some things in the standard of care have really large effect sizes. Blood transfusions for severe trauma or hemophilia, insulin for type I diabetes, dialysis for renal failure, etc, any responses to trauma (bandaging, compression, etc).

    It’s certainly possible that a large fraction of people have received basic care in emergencies that they could point to as otherwise lethal.

  • Sebastian_H

    “They can tell of one or more specific episodes where a good doctor did the right thing, and they’d otherwise be dead.”

    They are almost certainly right about the thought, but wrong about the instance. At least a few of them have certainly been saved by antibiotic prescriptions employed at the right times.

  • AspiringRationalist

    Aside from the medicine example. the examples you gave were of unimportant things where being right is not a major goal. With the Mexican restaurant example, the value of reaching a decision is considerably larger than the added value of an optimal vs. average decision, and a single commonly known piece of weak evidence provides a convenient Schelling point. With regards to mini-golf and arcade games, people probably discuss relative skill levels to avoid awkward silence, not to gain insight.

    • IMASBA

      You are technically right, but I think it’s safe to assume these examples are metaphors.

    • moridinamael

      Additionally, someone providing the evidence that they liked a certain restaurant indicates that the Mexican restaurant is probably not a total shithole, whereas any other restaurant which none of the group members might choose through abstract maximization might end up being terrible for unforeseeable reasons.

      In social situations it’s usually more valuable to *not* be the guy who suggested the horrible restaurant than it is to *be* the guy who suggested the great restaurant.

  • arch1

    I think that this posting touches on several things:

    1) What the economist Herbert Simon called “satisficing” (a hybrid of “satisfy” and “suffice”) which has also been referred to as the “theory of bounded rationality.”

    2) The tendency of people to err on the side of finding patterns where none exists (as their ancestors who erred on the other side were eaten by lurking leopards)

    3) The fact that probabilistic estimates have a second dimension, less widely appreciated and less well quantified than the primary dimension of estimated likelihood. The quality/completeness/sufficiency of information on which the likelihood estimate is based. Near one extreme of this sufficiency spectrum is e.g. an estimate from ignorance based on nothing but the symmetries of the situation.

  • IMASBA

    “When we played miniature golf, the winners were treated as if shown to be better skilled than previously thought, even though score differences were statistically insignificant. Same for arcade shooter contests. We also picked which Mexican place to eat at based on one person saying they had eaten there once and it was ok, even though given how random is who likes what when, that was unlikely to be a statistically significant difference for estimating what the rest of us would like.”

    “So clearly this intuitive collection of weak clues stuff goes very wrong sometimes, even on topics where people suffer large personal consequences. It is not just that random errors can show up; there are topics on which our minds are systematically structured, probably on purpose, to greatly mislead us in big ways.”

    Yeah, attributing success and skill to statistically insignificant scores, when someone has luck and wins, that might as well describe how an unfettered capitalist economy creates filthy rich executives, derivative traders and popstars. It’s amazing how an intelligent, educated, mathematically literate man like Hanson can make all these observations and still promote laissez-faire/unlimited reward/winner-takes-all capitalism.

  • mattmanhere

    Can we say that God may not exist, or is that too much?

    GreenWorld Farmland Investments

  • Paul Christiano

    Like others, I am a bit confused about your skepticism re: medical care (both here and in general). What studies do you think would have been expected to pick up effects of medical care on life expectancy? I assume you think the RAND trial is the best bet.

    The RAND trial covered only a few years of health care and saw a 2% reduction in mortality for the treatment group. The fact that there are big error bars seems like it is obviously not evidence against a 2% reduction, and indeed even if we’d observed an increase in mortality it would be extremely weak evidence against a reduction of a few percentage points. Basically the study is too weak to say anything about mortality reductions of a few percent.

    Moreover, the RAND trial involved only 3-5 years of health coverage, and it was the difference between a co-pay and free insurance, which has a relatively small impact on how much of the most important medical care that people receive. Scaled up from 3-5 years of coverage to 30-50 years of coverage you might expect a factor of 10 increase in the effect size. Scaled up from “free vs. co-pay” to “health care vs. no health care” you might expect another factor of 2-3 increase (since the difference between utilization of high-impact services between the two groups was only about 1/3 of the total utilization). Scaling up 2% by a factor of 20 would give you a 40% effect. So it seems like the results of this experiment are consistent with a significant fraction, and perhaps even a majority, of people being able to attribute their life to good medicine. Of course they are also perfectly consistent with no effect, but if anything they are weak evidence in favor of big effects.

    (The experiment also only covered mortality over the next decade or so, so if healthcare had any long-term effects we might expect to see another small multiplier; of course this wouldn’t appear in people’s reports about their lives being directly saved, so I’m happy to drop it.)

    Indeed, it would be extraordinarily surprising to me if the mortality results form the RAND experiment were large enough to be statistically significant. That would indicate that the impact of medicine is huge.

    On top of all that, when people say medicine saved their life they might often mean in practice that they averted a 10-20% risk of death from good medicine, which could make the number of people with such evidence much larger. Certainly if I avoided a 50% risk of death I would describe that as “my life was saved by medicine,” so I think its fair to give a factor of at least 2.

    Given that life expectancies from 40 have been steadily and significantly increasing in the US, and given that the experimental evidence seems too weak to say anything meaningful about effects of health care on all-cause mortality, I don’t see why you believe so strongly that the effect is small. It seems like the anecdotal evidence here is much stronger than the studies. (Though still very weak.)

    • Robin Hanson

      The RAND study is the best, but it is only one of dozens of studies I’ve seen over the years. I linked to many of those in previous post, and there’s another recent study I hope to post on soon. The RAND study did NOT see a 2% mortality reduction. Its best measure of health saw a *decrease* in health for those with more med. My relatives seem to believe in the life saving value of additional med they get that others don’t, not just in the common value of med that all get. And as far as I know, no one in health econ believes that the mortality ratio of med vs. less med groups should compound linearly in the number of years of treatment difference.

      • Paul Christiano

        Good point. The 2% reduction was for the estimated risk of dying, and is mostly driven by the well-reported drop in blood pressure. With respect to actual deaths they didn’t report their results because they lacked enough power to make it interesting. In any case, it seems like it is obvious that you wouldn’t be able to notice even a huge effect on mortality. Fair?

        The physiological measures in the RAND HIE had significant responses, though they aren’t really things we care directly about. The other measures, in particular “physical health and role functioning,” seem really dubious for this purpose, and mostly orthogonal to most claims that people want to make about health care (especially claims about not dying). It seems unsurprising that things that stop people from walking are not the kinds of things that they will stop treating when healthcare gets more expensive. As the authors of the RAND report say, “the majority of persons in a general non-aged population do not have measurable functional limitations; hence, the precision of these measures for testing hypotheses about effects of insurance plan on health status is somewhat restricted.” Casually that seems like an understatement, though I would have to do more looking into the measures to really know.

        If you ran 100 copies of the RAND experiment and got the same results each time, it would not make me any more confident that health care doesn’t help avert death. If you ran the RAND experiment and 99 much weaker studies, it would help even less. I agree that if you had more relevant outcome measures and generally saw no effect it would be suggestive, but I still don’t think it would bear on the claims people make about helpful treatment for acute problems.

        It seems like the strongest argument by far is that not enough people actually die for so many people to nearly die. But it seems like this is probably explained by (1) people overestimate how good their treatment is relative to others, in keeping with their behavior in many domains, and (2) people using `saved’ to mean `averted a 10% chance of death,’ or even more liberally, also in keeping with their behavior in most cases.

      • Robin Hanson

        There are individual studies with 1000x as many patients as the RAND study. E.g., http://www.nber.org/papers/w6513

        The RAND study was too small for actual mortality differences to be significant. They constructed their “risk of dying” index after they saw their data, and that was only a 1% estimated mortality effect for the whole population. The “General Health Index” was the measure they decided on before they saw their data. Role Functioning is an very relevant measure – it is after all one of the main reasons people want to be healthy, to function in their usual roles. Most people aren’t sick at any one time, yet the outcome we care about is sickness. That is a problem common to all these studies.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        One problem with a near-mode or inside view of the effects of medical care–as when you ask your relatives–is in their having few clues about whether they’ve been harmed by medical care. If I can generalize from my own experience as a provider of psychological services, medical care in the U.S. is immensely iatrogenic (because doctors, frankly, are money grubbers: the ignored case for socialized medicine). In psychological services, patients are, for example, often provided “supportive therapy,” which is an exercise in creating psychological dependency and undermining patient autonomy. It’s why I left the field after 12 years: pressures to deliver these kind of “services” became increasingly difficult to avoid.

        Patients committed to a form of therapy usually think they’ve improved and ignore or don’t notice the harm, and this is particularly true of the most iatrogenic supportive forms of “therapy,” because the therapist is, in conventional terms, being “nice” to the patient.

  • http://juridicalcoherence.blogspot.com/ Stephen Diamond

    People think they’ve benefited from medicine because there’s more information available for the benefits of medical interventions than for their disadvantages. Medicine isn’t special: it’s just more measurable, so the (weak) relationship between consumption and the desired consequences can be proven objectively. How do you show that status, generally, is overconsumed, although it surely is, because of the biases of a profit-driven market economy?

    The only place I’ve seen this bias made explicit is in Katja latest posting on Meteuphoric, where she explains that in the marketplace of ideas arguments against moral crusades are undersupplied due to a tragedy of the commons. As far as I know, this important insight is novel, but it can be immediately seen to apply to markets generally.

    If markets bias us to overproduce goods and services, what’s the alternative? Not to markets; that’s another discussion. ( http://tinyurl.com/ke2oj98 ) The alternative to oversupplying goods and services? It’s obviously leisure. Yet the trend is to increase the time worked.

    The bias for overproduction is exacerbated as markets reign more completely.

  • Philon

    “. . . because doing so might force them to drop treasured
    intuitive judgements.” Why do many people *treasure* their intuitive judgments, as opposed to merely thinking them plausible enough to be worth acting on?

    • http://juridicalcoherence.blogspot.com/ Stephen Diamond

      Hanson’s theory is that our intuitive judgments reflect self-serving (far-mode) illusions, which put our character in a positive light. Harboring these illusions help us signal our goodness, righteousness, and suitability as ally.

      Hanson’s wrong. Signaling is a near-mode operation. (For an application of my view of status signaling, see “Verbosity affronts the court”http://tinyurl.com/agft7ga )

  • Stuart Armstrong

    >On this blog I often present weak clues, relevant to important topics, but by themselves not sufficient to draw strong conclusions.

    What would be useful there is if you could describe an impartial process for seeking out this evidence. If the clues are weak, there must be clues in the other direction as well, of course – do you regularly present evidence for or against your positions, and make it easy to collate all the clues together on a particular topic?

  • Stuart Armstrong

    >But people just can’t on average have this much evidence, since we usually find it hard to see effects of medicine on health even when we have datasets with thousands of people.

    Is it true that medicine in total has little impact on health, or is it merely that medicine has no marginal impact on health?

    • http://overcomingbias.com RobinHanson

      Almost all the evidence we have is about marginal impacts.

      • Stuart Armstrong

        Then that’s perfectly compatible with many people having been saved by doctors.