At Less Wrong, Vladimir Nesov approvingly cites Tim Minchin's poem Storm, vid here, text here. It is an entertaining, passionate, articulate rant against fuzzy-headed thinking. It is also seriously misguided:
Alex, vaccination counts as med, but not food or washing, neither of which is plausibly "science."
[jaw drops] Food and washing aren't medicine, but not science? That's just crazy talk. Germ theory of disease. Vitamins. Iodized salt reduces endemic goiter. Come on!
I'm curious how the development of nuclear power would have gone in a world where the state didn't develop and fund it. Would it be as far developed as it is now, or would it only be developed when Gas/Coal was very rare and expensive? Also I'm not sure the internet would have looked the same as it does now if it hadn't spun off from arpa net. It would probably be far more heavily controlled, like compuserve etc.
So I'm not really sure how the economic historians can construct their counter factual hypotheses with any degree of accuracy.
@frelkins:I've read the majority of Robin's posts on health care, and have found very few of them convincing. Basically, I think the ad homs are an appropriate response to the quality of Robin's arguments. I've asked Robin to cite someone else who agrees with his views -- but he can't seem to find anyone. Their is obviously a career advantage to taking extreme views -- I'm not sure what form of "bias" that would be called.
Robin rarely has read the medical literature on an intervention he has posted. The most recent example comes to mind, when he cited some alternative-diet quack who was railing against statins. Robin found his "argument very convincing". And he is right -- the argument is convincing if you don't bother to do a pubmed search to find contradictory evidence. Alan Garber, who is director of health care policy at NBER recently wrotestatins are so effective, that they are killing research for other cardiovascular drugs
Lastly, Robin's favorite study has big holes in it, that he dismisses as "every study has flaws". Now maybe thats the standard of evidence for Economics -- but in medicine, a study with as many flaws as RAND would not be accepted until it had good replication.
So to sumarize -- Robin has a career advantage to take an extreme view, doesn't bother to look at the relevant medical literature (EBM or clinical trials), and ignores the gigantic holes in the studies supporting his views. My sense is that this justifies an ad hom attack.
BTW, the problem with the decision market on Robin being delusional, is coming up with objective decidable criterion that Robin would agree to as well.
Jason, every study has flaws. If you decide you can dismiss all aggregate med studies as meaningless, well then sure the default would be to accept the usual local med studies and claims at face value. I did post saying we should repeat the RAND experiment.
Walter, once I knew a treatment was "backed by unbiased study" that would be enough, regardless of if that treatment were "alternative" or not. Yes, it is the biases that make most med untrustworthy.
" If we used your favorite tool -- decision markets -- restricted to the overcoming bias audience, it seems like the majority would bet on you being delusional."
Of course you could toss a market up in a relatively short time, I know. I think you should; it would be a great idea. But my strong impression is that you are Quite Wrong. The majority of the steady OB readership in fact probably doesn't disagree with Hanson here, to the best of my current knowledge.
I'll give that 80%. Delusional seems ad hom. at best; do you have a real argument, one that's concise, with strong evidence? I don't mean to seem partisan, but you're not making it here that I can see.
Regular readers really don't seem to be participating in this discussion - you non-regulars who don't seem as familiar with Hanson's thinking or past posts and the links he has provided in them appear to be re-hashing your something that is looking more and more like a personal drama endlessly.
Perhaps the discussion would be more fruitful if you would do the work of reading what's available on the subject and then make a true case? I would be interested in reading that, thanks!
Posted by Robin: A colleague assures me most economic historians estimate we would be pretty much just as rich and healthy today had the only "scientists" been researchers funded directly by firms, with no government, charity, or student funding.
Mike, economic historians are well aware of your arguments, and still disagree with you.
This claim is very interesting. What are the arguments, and where is the evidence?
Robin, I take it that you believe yourself to be an anti-mystic. As such, I assume your argument is not that medicine based on good science has the same effectiveness as medicine that is not. If that is your argument, I would love a link to a piece that makes it in more detail. If it is as I assume, them maybe it is the wording of the post that causes such dissent.
A hypothetical: You are afflicted with a disease. You have a simple choice (I recognize this is unlikely). You could see an MD who you know to be honest, intelligent, empathetic, and competent. You know this person tries hard only to prescribe medication and treatment that is backed by unbiased study. You could also see a practitioner of "alternative medicine" who has the same personal qualities as the MD and is considered very highly in his/her profession by his/her peers. Should you see one or the other? Should you just stay home?
I suppose what I'm wondering is: do you think that medicine is failing to be empirical because of various reasons (biased studies, market factors, etc), and as a result it fails to be significantly more effective than alternatives? Or, that even striving for science-based medicine is folly? Or, something else completely?
You won't get as much argument from the first premise as from the second, is my thought. Forgive me if this has already been made clear elsewhere.
Robin, I guess I should put my question differently: why would we think that variation in the number of hospital beds across regions is driven by differing medical cultures rather than differing demand for hospital beds? If we were to regress the number of hospital beds on some measure of "local medical cultures" what would be the R^2? Even if just a third of the variation were driven by demand side factors, this would lead to seriously biased estimates because those factors may directly impact mortality. Fisher et. al. provide no evidence on this point which is central to their identification strategy. If they were to observe some direct measure of "local medical cultures" and use that as an instrument for resource expenditures, their finding would be much more believable.
Let me play devil's advocate here and state the case against your view as strongly as I can (and more strongly than is warranted given my own familiarity with the literature):
1) The RAND experiment is 30 years old and so may severely underestimate the benefits of health care today
2) These aggregate regressions don't meet the standards of evidence required for publication in top economics journals (can you link to any from the AER, JPE, QJE or ECTA?). They are systematically biased towards finding no effect of medical treatment or a negative effect because they fail to fully control for the fact that medical spending varies in response to the demand for medicine. It is not sufficient to point to the quantity of these studies because we know that that whole literatures can give systematically wrong answers if they fail to adequately grapple with identification (e.g. the class size literature prior to the Tennessee Star Experiment).
3) Even if we take them at face value, these aggregate regressions might not tell us the impact that increasing the price consumers face for medical treatment would have on which treatments they select. If your contention above is correct, the variation in current medical expenditures across regions is driven by supply-side factors ("local medical cultures"). If we increase the price of medical spending for consumers, that variation would still exist, and it is a separate empirical question what treatments consumers would forgo. As Cutler and Garber point out, there is some evidence that consumers are not good at forgoing only the least effective treatments. In fact, the marginal treatment induced by variation in "local medical cultures" is a treatment regarded as marginal by experts (some doctors choose to practice it and others don't), so we might be especially worried that this treatment would be less valuable than the treatment consumers would forgo if facing higher prices.
4) We just don't have good evidence on the question of whether large cuts in medical spending would lead to welfare gains
5) Repeating the RAND experiment could have potentially HUGE returns
6) We should attempt to reduce the number of dollars spent on many procedures which are known to be inefficient
To the extent that we disagree, presumably it is on points 2)-4).
Robin: I should make a post on it someday, but before I do I should understand your position in much greater detail. Can you recommend a book for someone who's not a trained economist, available at most good libraries, that states your views in a way you'd agree with? (also okay: something free on the Internet. I've read your OB posts but don't find those sufficient.)
Alex, vaccination counts as med, but not food or washing, neither of which is plausibly "science." Distain for study does not count as negative evidence, nor does being aware of a lack of evidence count as positive evidence.
Jason, there is a huge and well-documented "practice variation" due to differing local medical cultures. This easily and plausibly explains med scale variance.
Robin, as has been mentioned a hundred times, and like always you ignore religiously because it would actually require an ounce of knowledge of medicine. The MAJOR flaw with RAND is that preventative medicine -- which is basically what it was looking at -- was non-existent at the time the study was conducted. I've TOLD you OVER, and OVER, and OVER again, that the drugs used for prevention (or the trials documenting how to use them) were not available during the study period. This is called "generalizability" in the epidemiology literature -- which again you seem to be completely unaware of. I've even listed the interventions commonly used in preventive medicine for you so you can check the dates yourself. I've even told you how you could easily run a simulation using randomized trial data to SEE that I'm basically right and you are WRONG. OF course any verification of your religious beliefs apparently seems too much for you to bear.
I've also repeatedly asked for you to cite other people who make NEARLY as outrageous claims as you do. WHERE are these other "rationalists". Apparently other people on this blog, think you are delusional and this is a case study in bias. If we used your favorite tool -- decision markets -- restricted to the overcoming bias audience, it seems like the majority would bet on you being delusional.
Since this discussion is winding down (or moving to another venue), I thought I would link to Hanson's early post on his views on medical care for posterity:
Robin - thanks for the link to the Fisher et. al. study. After partialling out the controls for disease burden, where does the residual variation in the number of hospital beds come from? Why do some Hospital Referral Regions have more hospital beds than others? It seems there are many possible answers, but in most of these, hospital beds would be endogenous. For example, there may be more per capita hospital beds in regions with higher demand due to any disease characteristic not controlled for! This seems to be a natural explanation for why some regions would have more hospital beds than others after controlling for demographic characteristics, and if it's correct, then the regression results would be seriously biased. Is there some factor I'm missing which would could plausibly drive most of the variation in the number of hospital beds which is otherwise unrelated to mortality?
The RAND study is interesting, and I'm somewhat distressed not to have come accross it before.. Another bit of evidence for RH's argument is the repeated observation (in Canada, Scandenavia, and Israel) that when physicians go on protracted strikes, the mortality rate drops (PDQ Epidemiology, 1998, pg. 10; unfortunately the text doesn't cite the primary references).
While I think RH might overestimate the uselessness of medicine, the observation that more access to healthcare doesn't necessarily result in a healthier population is a usefull one.
... "the vast majority of medical treatments have no better supporting "scientific" evidence than the alternative medicine they deride"
Errr, sure, many conventional medical interventions have weak evidence bases (here's a great example ), but as Yvain points out, EBM is huge and we're generally aware of the problem. Contrast that with alternative medicine circles, where there is often outright hostility to criticism and disdain for the very pursuit of evidence. Equating the scope of the problem in mainstream vs. alternative medicine is inaccurate.
... "nor that modern medicine can only claim credit for a small fraction of our lifespan gains."
That, surprisingly, is true. I take it you're not considering hygene/vaccination/nutrition to be medicine (which I get) but you could reasonably chalk them up to non-mystical science, which I think is Minchin's point.
Alex, vaccination counts as med, but not food or washing, neither of which is plausibly "science."
[jaw drops] Food and washing aren't medicine, but not science? That's just crazy talk. Germ theory of disease. Vitamins. Iodized salt reduces endemic goiter. Come on!
I'm curious how the development of nuclear power would have gone in a world where the state didn't develop and fund it. Would it be as far developed as it is now, or would it only be developed when Gas/Coal was very rare and expensive? Also I'm not sure the internet would have looked the same as it does now if it hadn't spun off from arpa net. It would probably be far more heavily controlled, like compuserve etc.
So I'm not really sure how the economic historians can construct their counter factual hypotheses with any degree of accuracy.
@frelkins:I've read the majority of Robin's posts on health care, and have found very few of them convincing. Basically, I think the ad homs are an appropriate response to the quality of Robin's arguments. I've asked Robin to cite someone else who agrees with his views -- but he can't seem to find anyone. Their is obviously a career advantage to taking extreme views -- I'm not sure what form of "bias" that would be called.
Robin rarely has read the medical literature on an intervention he has posted. The most recent example comes to mind, when he cited some alternative-diet quack who was railing against statins. Robin found his "argument very convincing". And he is right -- the argument is convincing if you don't bother to do a pubmed search to find contradictory evidence. Alan Garber, who is director of health care policy at NBER recently wrotestatins are so effective, that they are killing research for other cardiovascular drugs
Lastly, Robin's favorite study has big holes in it, that he dismisses as "every study has flaws". Now maybe thats the standard of evidence for Economics -- but in medicine, a study with as many flaws as RAND would not be accepted until it had good replication.
So to sumarize -- Robin has a career advantage to take an extreme view, doesn't bother to look at the relevant medical literature (EBM or clinical trials), and ignores the gigantic holes in the studies supporting his views. My sense is that this justifies an ad hom attack.
BTW, the problem with the decision market on Robin being delusional, is coming up with objective decidable criterion that Robin would agree to as well.
Jason, every study has flaws. If you decide you can dismiss all aggregate med studies as meaningless, well then sure the default would be to accept the usual local med studies and claims at face value. I did post saying we should repeat the RAND experiment.
Walter, once I knew a treatment was "backed by unbiased study" that would be enough, regardless of if that treatment were "alternative" or not. Yes, it is the biases that make most med untrustworthy.
@diogenes
" If we used your favorite tool -- decision markets -- restricted to the overcoming bias audience, it seems like the majority would bet on you being delusional."
Of course you could toss a market up in a relatively short time, I know. I think you should; it would be a great idea. But my strong impression is that you are Quite Wrong. The majority of the steady OB readership in fact probably doesn't disagree with Hanson here, to the best of my current knowledge.
I'll give that 80%. Delusional seems ad hom. at best; do you have a real argument, one that's concise, with strong evidence? I don't mean to seem partisan, but you're not making it here that I can see.
Regular readers really don't seem to be participating in this discussion - you non-regulars who don't seem as familiar with Hanson's thinking or past posts and the links he has provided in them appear to be re-hashing your something that is looking more and more like a personal drama endlessly.
Perhaps the discussion would be more fruitful if you would do the work of reading what's available on the subject and then make a true case? I would be interested in reading that, thanks!
Posted by Robin: A colleague assures me most economic historians estimate we would be pretty much just as rich and healthy today had the only "scientists" been researchers funded directly by firms, with no government, charity, or student funding.
Mike, economic historians are well aware of your arguments, and still disagree with you.
This claim is very interesting. What are the arguments, and where is the evidence?
"...but not food or washing, neither of which is plausibly "science." "
Decidedly non-magical benefits of our better understanding of the world though, no? I don't think we really disagree here.
"Distain for study does not count as negative evidence, nor does being aware of a lack of evidence count as positive evidence."
That's not my claim. I merely think that the evidence problem in medicine vs. CAM is different in character, thus equating the two is misleading.
Robin, I take it that you believe yourself to be an anti-mystic. As such, I assume your argument is not that medicine based on good science has the same effectiveness as medicine that is not. If that is your argument, I would love a link to a piece that makes it in more detail. If it is as I assume, them maybe it is the wording of the post that causes such dissent.
A hypothetical: You are afflicted with a disease. You have a simple choice (I recognize this is unlikely). You could see an MD who you know to be honest, intelligent, empathetic, and competent. You know this person tries hard only to prescribe medication and treatment that is backed by unbiased study. You could also see a practitioner of "alternative medicine" who has the same personal qualities as the MD and is considered very highly in his/her profession by his/her peers. Should you see one or the other? Should you just stay home?
I suppose what I'm wondering is: do you think that medicine is failing to be empirical because of various reasons (biased studies, market factors, etc), and as a result it fails to be significantly more effective than alternatives? Or, that even striving for science-based medicine is folly? Or, something else completely?
You won't get as much argument from the first premise as from the second, is my thought. Forgive me if this has already been made clear elsewhere.
Robin, I guess I should put my question differently: why would we think that variation in the number of hospital beds across regions is driven by differing medical cultures rather than differing demand for hospital beds? If we were to regress the number of hospital beds on some measure of "local medical cultures" what would be the R^2? Even if just a third of the variation were driven by demand side factors, this would lead to seriously biased estimates because those factors may directly impact mortality. Fisher et. al. provide no evidence on this point which is central to their identification strategy. If they were to observe some direct measure of "local medical cultures" and use that as an instrument for resource expenditures, their finding would be much more believable.
Let me play devil's advocate here and state the case against your view as strongly as I can (and more strongly than is warranted given my own familiarity with the literature):
1) The RAND experiment is 30 years old and so may severely underestimate the benefits of health care today
2) These aggregate regressions don't meet the standards of evidence required for publication in top economics journals (can you link to any from the AER, JPE, QJE or ECTA?). They are systematically biased towards finding no effect of medical treatment or a negative effect because they fail to fully control for the fact that medical spending varies in response to the demand for medicine. It is not sufficient to point to the quantity of these studies because we know that that whole literatures can give systematically wrong answers if they fail to adequately grapple with identification (e.g. the class size literature prior to the Tennessee Star Experiment).
3) Even if we take them at face value, these aggregate regressions might not tell us the impact that increasing the price consumers face for medical treatment would have on which treatments they select. If your contention above is correct, the variation in current medical expenditures across regions is driven by supply-side factors ("local medical cultures"). If we increase the price of medical spending for consumers, that variation would still exist, and it is a separate empirical question what treatments consumers would forgo. As Cutler and Garber point out, there is some evidence that consumers are not good at forgoing only the least effective treatments. In fact, the marginal treatment induced by variation in "local medical cultures" is a treatment regarded as marginal by experts (some doctors choose to practice it and others don't), so we might be especially worried that this treatment would be less valuable than the treatment consumers would forgo if facing higher prices.
4) We just don't have good evidence on the question of whether large cuts in medical spending would lead to welfare gains
5) Repeating the RAND experiment could have potentially HUGE returns
6) We should attempt to reduce the number of dollars spent on many procedures which are known to be inefficient
To the extent that we disagree, presumably it is on points 2)-4).
Robin: I should make a post on it someday, but before I do I should understand your position in much greater detail. Can you recommend a book for someone who's not a trained economist, available at most good libraries, that states your views in a way you'd agree with? (also okay: something free on the Internet. I've read your OB posts but don't find those sufficient.)
Alex, vaccination counts as med, but not food or washing, neither of which is plausibly "science." Distain for study does not count as negative evidence, nor does being aware of a lack of evidence count as positive evidence.
Jason, there is a huge and well-documented "practice variation" due to differing local medical cultures. This easily and plausibly explains med scale variance.
Robin, as has been mentioned a hundred times, and like always you ignore religiously because it would actually require an ounce of knowledge of medicine. The MAJOR flaw with RAND is that preventative medicine -- which is basically what it was looking at -- was non-existent at the time the study was conducted. I've TOLD you OVER, and OVER, and OVER again, that the drugs used for prevention (or the trials documenting how to use them) were not available during the study period. This is called "generalizability" in the epidemiology literature -- which again you seem to be completely unaware of. I've even listed the interventions commonly used in preventive medicine for you so you can check the dates yourself. I've even told you how you could easily run a simulation using randomized trial data to SEE that I'm basically right and you are WRONG. OF course any verification of your religious beliefs apparently seems too much for you to bear.
I've also repeatedly asked for you to cite other people who make NEARLY as outrageous claims as you do. WHERE are these other "rationalists". Apparently other people on this blog, think you are delusional and this is a case study in bias. If we used your favorite tool -- decision markets -- restricted to the overcoming bias audience, it seems like the majority would bet on you being delusional.
Since this discussion is winding down (or moving to another venue), I thought I would link to Hanson's early post on his views on medical care for posterity:
http://www.overcomingbias.c...
Robin - thanks for the link to the Fisher et. al. study. After partialling out the controls for disease burden, where does the residual variation in the number of hospital beds come from? Why do some Hospital Referral Regions have more hospital beds than others? It seems there are many possible answers, but in most of these, hospital beds would be endogenous. For example, there may be more per capita hospital beds in regions with higher demand due to any disease characteristic not controlled for! This seems to be a natural explanation for why some regions would have more hospital beds than others after controlling for demographic characteristics, and if it's correct, then the regression results would be seriously biased. Is there some factor I'm missing which would could plausibly drive most of the variation in the number of hospital beds which is otherwise unrelated to mortality?
The RAND study is interesting, and I'm somewhat distressed not to have come accross it before.. Another bit of evidence for RH's argument is the repeated observation (in Canada, Scandenavia, and Israel) that when physicians go on protracted strikes, the mortality rate drops (PDQ Epidemiology, 1998, pg. 10; unfortunately the text doesn't cite the primary references).
While I think RH might overestimate the uselessness of medicine, the observation that more access to healthcare doesn't necessarily result in a healthier population is a usefull one.
... "the vast majority of medical treatments have no better supporting "scientific" evidence than the alternative medicine they deride"
Errr, sure, many conventional medical interventions have weak evidence bases (here's a great example ), but as Yvain points out, EBM is huge and we're generally aware of the problem. Contrast that with alternative medicine circles, where there is often outright hostility to criticism and disdain for the very pursuit of evidence. Equating the scope of the problem in mainstream vs. alternative medicine is inaccurate.
... "nor that modern medicine can only claim credit for a small fraction of our lifespan gains."
That, surprisingly, is true. I take it you're not considering hygene/vaccination/nutrition to be medicine (which I get) but you could reasonably chalk them up to non-mystical science, which I think is Minchin's point.
Yvian, why don't you compose a post where you lay out your critique/argument, so I can respond to that?