Imagine someone argued: Stock car races are a huge waste of resources. To find out which car models are faster, we can just have experts keep track of the speed of the cars they drive past, and write up their observations. Then we’ll have them debate each other. Sure some biases might slip in, but we shouldn’t pretend we can escape bias; stock car races can have biases too. For example, there might be a pebble on one side of the track that isn’t on the other side, or the sun might get in one driver’s eye for a moment but not in another’s.
The counter anecdote is a great example of precisely what we're trained NOT to do. Investigations are unnecessary, wasteful, harmful and to be avoided like the plague if they won't change our management/outcome.
Certainly we would get more bang for our buck from engineering and public health than from medicine on the margin, but the effects of cure are near and the effects of prevention are far, so who's going to get elected on a platform of diverting those funds?
Isn't that all statistics really is is a lot of boiled down anecdotes? Sort of like the reverse of Stalin's aphorism. All statistics are is a million murders (or anecdotes).
Couldn't 1,000 observational studies give us good ideas as to which treatments are harmful or wasteful at the margin? And which are beneficial (I'm sure some are)? We'd still have a picture of how useful medicine is at the margin, but we'd also get an idea of which treatments were helpful and which ones weren't.
If we get enough anecdotes in the comments, it will equal data, right?
Anyway, when my mom had a glioblastoma, she had two surgeries, radiation treatment and chemotherapy. She died within a year of the initial diagnosis. Interestingly, her GP told her, after the first surgery (which was indicated to confirm the type of brain tumor), not to undergo further treatment, as it would be painful and offer no meaningful benefit. However, the oncologists were much more bullish on the effects of treatment. Had we known then what we know now, we might have foregone treatment, but we deferred to the experts.
Clearly, in this case, considerable extra expense had no marginal benefit.
Would a RAND rerun really cost $500M and take 10 years?
As a practical matter, who would volunteer to enroll in the study? Today, I figure lots of people would sign up for a 50% chance at free medicine. But consider those who land in the arm where they have to pay almost everything out of pocket. In 2014, they can drop out and buy subsidized insurance instead. Rationally, anyone who expected to save money that way would do so. Those who remain would be healthier than average, skewing the results.
That reminded me of a counter-anecdote. One commenter's father had had a routine colonoscopy, but the man was already old and frail. If the colonoscopy had found colon cancer there would not have been anything useful to do about the cancer. Unfortunately there was a rare accident. The endoscope ruptured the man's colon and being old and frail he died of it.
That raised a strange question: does Spriteless' anecdote argue against Robin's claim, or in favour of it?
The problem is that the naive reading of "medicine is on average is near useless or harmful on the margin;" is that the margin is the homeopathy zone. The treatments do nothing and only money is at stake. That is not actually what Robin is saying. Useless on average covers both useless in every case and a balance of kill and cure.
Spriteless' anecdote offers us an image of the typical medical procedure as a reasonably priced surgery that is directly life saving. If that were the end of the story it would be easy to show that medical expenditures produced a large impact on mortality. It's not. So what is the rest of the story?
Accepting Robin's judgement that it is hard to demonstrate the marginal benefit of medicine in the aggregate figures, the implications of Spriteless' anecdote are perhaps inverted. If I'm persuaded that reasonably priced surgeries that are directly life saving are common place, then my heart sinks, for the implication is that the average is coming from a balancing factor of common place medical disaster. And Robin's point becomes about lives as well as money; there are many lives to be saved by cutting back on killer surgery.
I think Robin's point is about the marginal value of medecine. Getting rid of all of medecine will not make us healthier, obviously. But the question is, does a little more spending on medecine that what we have now increase our health at all? And does a little less decrease it?
It's pretty certain that diverticuli operations would not be the first things to be cut when the money goes down.
I'm just a layman, so I'd like to hear why in principle 1,000 observational studies should be better than a single randomized trial with equivalent funding (let's assume these are being done by a single person for the benefit of his own pursuit of truth). Is it because of diminishing returns in study size? You should be able to add more variables to observe or treatments to be given, even if this means not having as many experimental subjects, while still pursuing the one-big-study.
Spriteless: just so. On the other hand, it's rather hard to convince me that when my father and my mother-in-law both contracted opportunistic infections after semi-successful carpal tunnel and hernia surgeries (respectively), that medicine was especially helpful to either of them.
Medicine's successes are memorable, its worst failures are not. Indeed, medicine buries its gravest mistakes.
The worst of it is that we're not really sure if a lot of high-end, high-intervention surgeries actually improve medical outcomes more than doing nothing.
We've just come through a long period of prostate testing and surgery that, according to the best opinions today, mostly had no effect on the lifespan of the affected men (but it did frequently render them impotent).
It is hard for you to convince me that when my dad's diverticuli broke, scooping out all the rot and sewing up another means of defecation while his body healed had only a marginal effect on his health. He was kind of, you know, dying. Medicine is at its most memorable at moments like that, not in the decades of taking medicine that reduces the chance of a heart attack by 10% of .5%, rather than by 10% of 100%.
The counter anecdote is a great example of precisely what we're trained NOT to do. Investigations are unnecessary, wasteful, harmful and to be avoided like the plague if they won't change our management/outcome.
Certainly we would get more bang for our buck from engineering and public health than from medicine on the margin, but the effects of cure are near and the effects of prevention are far, so who's going to get elected on a platform of diverting those funds?
Its interesting that a hyper-rationalist like Robin comes to similar conclusions as hyper-skeptic Nassim Taleb regarding the value of Medicine.
Isn't that all statistics really is is a lot of boiled down anecdotes? Sort of like the reverse of Stalin's aphorism. All statistics are is a million murders (or anecdotes).
Well, what wouldn't you get if you had to pay out-of-pocket for it?
Robin,
Couldn't 1,000 observational studies give us good ideas as to which treatments are harmful or wasteful at the margin? And which are beneficial (I'm sure some are)? We'd still have a picture of how useful medicine is at the margin, but we'd also get an idea of which treatments were helpful and which ones weren't.
If we get enough anecdotes in the comments, it will equal data, right?
Anyway, when my mom had a glioblastoma, she had two surgeries, radiation treatment and chemotherapy. She died within a year of the initial diagnosis. Interestingly, her GP told her, after the first surgery (which was indicated to confirm the type of brain tumor), not to undergo further treatment, as it would be painful and offer no meaningful benefit. However, the oncologists were much more bullish on the effects of treatment. Had we known then what we know now, we might have foregone treatment, but we deferred to the experts.
Clearly, in this case, considerable extra expense had no marginal benefit.
Would a RAND rerun really cost $500M and take 10 years?
As a practical matter, who would volunteer to enroll in the study? Today, I figure lots of people would sign up for a 50% chance at free medicine. But consider those who land in the arm where they have to pay almost everything out of pocket. In 2014, they can drop out and buy subsidized insurance instead. Rationally, anyone who expected to save money that way would do so. Those who remain would be healthier than average, skewing the results.
That reminded me of a counter-anecdote. One commenter's father had had a routine colonoscopy, but the man was already old and frail. If the colonoscopy had found colon cancer there would not have been anything useful to do about the cancer. Unfortunately there was a rare accident. The endoscope ruptured the man's colon and being old and frail he died of it.
That raised a strange question: does Spriteless' anecdote argue against Robin's claim, or in favour of it?
The problem is that the naive reading of "medicine is on average is near useless or harmful on the margin;" is that the margin is the homeopathy zone. The treatments do nothing and only money is at stake. That is not actually what Robin is saying. Useless on average covers both useless in every case and a balance of kill and cure.
Spriteless' anecdote offers us an image of the typical medical procedure as a reasonably priced surgery that is directly life saving. If that were the end of the story it would be easy to show that medical expenditures produced a large impact on mortality. It's not. So what is the rest of the story?
Accepting Robin's judgement that it is hard to demonstrate the marginal benefit of medicine in the aggregate figures, the implications of Spriteless' anecdote are perhaps inverted. If I'm persuaded that reasonably priced surgeries that are directly life saving are common place, then my heart sinks, for the implication is that the average is coming from a balancing factor of common place medical disaster. And Robin's point becomes about lives as well as money; there are many lives to be saved by cutting back on killer surgery.
What would be cut, for example?
I think Robin's point is about the marginal value of medecine. Getting rid of all of medecine will not make us healthier, obviously. But the question is, does a little more spending on medecine that what we have now increase our health at all? And does a little less decrease it?
It's pretty certain that diverticuli operations would not be the first things to be cut when the money goes down.
I'm just a layman, so I'd like to hear why in principle 1,000 observational studies should be better than a single randomized trial with equivalent funding (let's assume these are being done by a single person for the benefit of his own pursuit of truth). Is it because of diminishing returns in study size? You should be able to add more variables to observe or treatments to be given, even if this means not having as many experimental subjects, while still pursuing the one-big-study.
Spriteless: just so. On the other hand, it's rather hard to convince me that when my father and my mother-in-law both contracted opportunistic infections after semi-successful carpal tunnel and hernia surgeries (respectively), that medicine was especially helpful to either of them.
Medicine's successes are memorable, its worst failures are not. Indeed, medicine buries its gravest mistakes.
The worst of it is that we're not really sure if a lot of high-end, high-intervention surgeries actually improve medical outcomes more than doing nothing.
We've just come through a long period of prostate testing and surgery that, according to the best opinions today, mostly had no effect on the lifespan of the affected men (but it did frequently render them impotent).
It is hard for you to convince me that when my dad's diverticuli broke, scooping out all the rot and sewing up another means of defecation while his body healed had only a marginal effect on his health. He was kind of, you know, dying. Medicine is at its most memorable at moments like that, not in the decades of taking medicine that reduces the chance of a heart attack by 10% of .5%, rather than by 10% of 100%.