Question Medical Findings

A recurring theme here is the difficulty in knowing whether some (much?) of modern healthcare is actually beneficial or not.  A couple of recent links that add support to that theme:

1. From JAMA, a new study analyzes more than two decades of heart care guidelines (that is, the guidelines that your doctor might follow in deciding how to treat you) from the American College of Cardiology and the American Heart Association. The study found that the overwhelming majority of recommendations are not supported by good evidence:

Level of evidence provides the link between recommendations and evidence base. Although there is significant variation among individual guidelines in available evidence supporting recommendations, the median of level of evidence A recommendations [i.e., those supported by more than one randomized trial] is only 11% across current guidelines, whereas the most common grade assigned is level of evidence C, indicating little to no objective empirical evidence for the recommended action. . . . Interestingly, our findings are reflective of a specialty — cardiology — that has a large pool of research to draw on for its care recommendations. Guidelines in other medical areas in which large clinical trials are performed less frequently may have an even weaker evidence-based foundation.

2. In this post, Dr. Eades criticizes (convincingly, I think) a recent study purporting to show that statins reduce mortality.

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • valter

    I cannot access the JAMA and check for myself, so I’ll ask:

    why would these results imply that cardiologists do not rely (enough) on evidence?

    Could it be that in those cases where they gave recommendations of levels B or C there were no possible recommendations of level A? In that case, even “little” empirical evidence in support of a given recommendation could be enough to justify it (depending of how large the possible negative side effects could be compared with the next best recommendation (e.g., do-nothing) and possibly on the cost of the treatments.)

    In other words, sometimes beating the null hypothesis at 95% confidence may be a worse rule than just checking whether the point estimate is positive.

    Of course, if there were many cases in which doctors recommended treatments for which the balance of evidence is strictly negative, well … then it would be another story.

  • frelkins

    @Stuart

    Ok I confess when I clicked on the Eades link I was very skeptical because of the “Protein Power” connection, which made that blog post look like a diet doctor with something to peddle. After reading the post, however, he may have a point. Of course the incentive to study statins, such as Lipitor, is great.

    Currently Lipitor brings Pfizer about US$12 billion a year and it has spent over $250 million advertising the drug over the last two years. So everyone has an interest in “proving” statins work – doctors, who collect cash from visits to prescribe and monitor those who take it; scientists, who get Pfizer cash or other indirect support to “study” it; the media, who benefits from the ad revenue.

    The corruption of medicine in this way is in fact one of the main reasons I became interested in Robin’s health care ideas. Cut medicine in half; pay all doctors & hospitals the same and only for procedures with real evidence, all that. Further, I wonder if we could best accomplish this by just putting everyone on Medicare and letting the HMOs provide care.

    Of course, as Robin has also shown, this reform is impossible as people prefer status affiliations to actual health outcomes. Unless we experience a European-style fiat forced-march to universal care – highly unlikely in the USA even in the present situation – sensible health care reform may in fact be impossible, and we should probably look at Cass Sunstein-type ideas for incremental improvement.

  • Douglas Knight

    Something that I think is related to both items is that once a treatment is established it becomes difficult to study, especially by controlled studies.

    #2 is way too long and deserves a summary. I almost didn’t finish reading it because I didn’t trust the author to consistently apply standards of evidence. The point of the post is that compliance can be correlated with outcome, even compliance with a placebo. This is probably a general phenomenon, thus way more important than the application to statins. While he doesn’t have a statin example to compare to the statin study, he has does have a cholesterol lowering drug comparison, so a similar population. A similar example, that I read in a Lewis Thomas book, is that eating breakfast is the best predictor of short-term mortality.

    It’s a pity that people who point out problems like this rarely try to reach their own conclusions. Eames just rails against observational studies and doesn’t ask, what happens if we account for the known predictions of compliance? I think the answer is that we estimate that statins do nothing. It’s not a very precise estimate because, eg, the populations are very different. But if people always did this kind of analysis on their controlled studies, we’d have estimates for lots of populations.
    I believe that most studies collect compliance data. Do they just not analyze it? Is it possible to get it from them and analyze it yourself?

  • http://diogens420.blogspot.com Diogenes

    I didn’t bother reading all of #2 — but your summary statement is extremely misleading. Statins have been show to work in multiple randomized controlled trials. The medscape article may be misleading and the study that purports to show improvement with adherence may not be exactly right — BUT the fact that statins reduce mortality is not really arguable.

  • http://profile.typekey.com/StuartBuck/ Stuart Buck

    My summary statement is merely that Dr. Eades criticizes a particular study — and it seems that you agree with me on that point (i.e., you seem to admit that it’s not very convincing for that particular study to have found mortality benefits to taking prescriptions conscientiously, even though conscientious people seem to be healthier anyway).

    As for randomized trials, I’m having trouble figuring out what the truth is. I’ve seen meta-analyses claiming that statins reduce mortality, as you say. But then the very authors of this particular study say otherwise:

    The value of primary prevention with statin therapy in the reduction of overall mortality has recently been questioned.

    A pooled analysis of 8 randomized trials in primary prevention populations showed that statins did not reduce overall mortality, indicating that lipid-lowering therapy with statins should not be prescribed for true primary prevention in women of any age or in men older than 69 years.

    So I have to admit a bit of bafflement here. In any event, I cited Dr. Eades for his criticism of the new cohort study, and I’m not seeing any actual disagreement with anything that Dr. Eades says on that point (whatever people might think of his other opinions).

  • Grant

    In the absence of empirical evidence one way or the other, I’ll trust a doctor’s recommendations. I’m skeptical that studies can easily encompass all the variables that a doctor’s recommendations might, and my health means a lot to me.

    Of course if the recommendations are costly, painful or dangerous, then my decision might change.

    frelkins, where has it been shown that people prefer status affiliations to health outcomes? Robin has shown that many people prefer to affiliate with high-status doctors, sure. But I don’t know that he’s ever shown that they’d willingly trade their health to do so. In addition, I don’t know of any studies showing that patients still prefer status affiliations when they are educated on actual health outcomes of different doctors.

    I really can’t see anyone who is really sick or in any kind pain caring about the status of their doctor over their health.

  • http://retiredurologist.com retired urologist

    For those who missed it (especially @ diogenes), David J. Balan’s post (and the distinguished review it cites) sums up the situation here.

  • http://diogenes42.blogspot.com diogenes

    @Stuart, yea sorry, I was being curt on the general point that statins are generally extremely effective drugs. In the guidelines the JAMA article studies — they are probably one of the most studied. The confusion on mortality data regards in the patients they are used in — if you have documented heart disease or diabetes or other disease of your blood vessels and you have high cholesterol statins CLEARLY improve mortality (secondary prevention — preventing someone from having another heart attack).

    If you’re a healthy 40 year old, with just high cholesterol, but nothing else against you — statins seem to help and there is a lotta data supporting there use, but its not as large a benefit (this is called primary prevention — e.g. preventing a heart attack in somoene who has never had one before). Here the guidelines on which patients should get a statin are still evolving. This should all be taken with the big grain of salt that I’m not a cardiologist.

    The point regarding my curtness is that statins are probably one of the top most well studied and efficacious treatments out there. On top of that, they are cheap (off patent now) and have minimal side effects. So they aren’t an ideal example to study bias in medicine as opposed to other incountable other drugs and procedures.

    @retired urologist: Pharma definitely biases studies, however there is already one statin off patent, and the others will follow shortly. There is much less incentive to game the system currently on this particular class of meds — and there effect on cholesterol is clear to anyone who has ever used them. So you have RCT data on mortality (maybe biased?, but many studies), biological mechanism, and easily quantifiable, objective effect in individual patients. I would say they are one of the most well studied and effective medical and surgical intervention.

  • http://hanson.gmu.edu Robin Hanson

    The key claim is:

    The only improvement in all-cause mortality has been in men under 65 who have been diagnosed with heart disease, and even that benefit is so small that many people question if the extra cost and side effects of the statins are worth it.

    Who has evidence to show this claim wrong? I’ve always heard that statins are one of our most clearly beneficial drugs; it is quite discouraging to see that the clearest is far from clear.

  • http://hanson.gmu.edu Robin Hanson

    OK, I’ve read Dr. Eades’s post and two others he links, and I’m convinced; he’s right on the money. Key quote:

    In contrast, the only large clinical trial funded by the government, rather than companies, found no statistically significant benefit at all…”

  • http://profile.typepad.com/halfinney Hal Finney

    Those two links make good and valid points, but one thing to keep in mind is that even though science makes mistakes, so does everyone else. The question is which is more accurate, a report in a scientific journal, or a random person’s opinion? Which is a better guide to the truth, the consensus of the scientific community, or a maverick collection of bloggers? Well, of course I am wording the comparisons to hint at my opinion! But at least science as an institution has mechanisms to self correct, and has a track record over hundreds of years of having come far closer to the truth during that time.

    The second link is to a diet doctor, Michael Eades, author of a book called Protein Power. He is apparently on record as being negative towards statins. What are the chances that you’re going to get an unbiased opinion from him, on statins or protein consumption? Reversing his views might undercut his credibility and interfere with his livelihood.

    Or maybe you will say that the right solution is to just look at the evidence and decide for yourself. But why believe that you, a layman, will come to a more accurate and informed opinion than your doctor, on things like statins? You would both be exposed to the same evidence, but he would have experience and training to put it into context, which you would not. Wouldn’t this advice (to judge the evidence for yourself) be a classic example of the overconfidence bias, thinking that you probably can figure things out better than the experts, on almost every topic?

  • http://hanson.gmu.edu Robin Hanson

    Hal, if we reject everyone who has ever stated a prior opinion on a related subject we will have few left to rely on. What if the consensus among meta scientists, who study medical science at a meta level, is that medical scientists are way too eager to draw conclusions that favor their drug company funders? What if I am one of those meta-scientists?

  • http://stuartbuck.blogspot.com Stuart Buck

    Which is a better guide to the truth, the consensus of the scientific community, or a maverick collection of bloggers? . . . But at least science as an institution has mechanisms to self correct, and has a track record over hundreds of years of having come far closer to the truth during that time

    I think what I’ve linked to in the post precisely consists of efforts of the scientific community (or portions thereof) to “self correct,” that is, to point out that some treatments or practices aren’t supported by good evidence.

  • http://diogenes42.blogspot.com Diogenes

    @Robin
    The govt. trial (ALLHAT), looked more at primary prevention and had a couple methodological problems (see their own website for more detail). See my earlier postings for the diff between primary and secondary prevention. Dr. Eade appears to make the same point, but it seems like you are missing it, because he’s mainly ranting. Anyone who says “on these drugs that are not particularly benign. ” — has an axe to grind. Of all the meds available, this class is probably one with the least amount of side effects.

    The QALY for a statin is ~ 15K-25K year for secondary prevention.

    All this info is easily accessible on up-to-date, which is available at any academic medical center.

    Statins are just not a good example regarding bias in medicine — they mainly display the bias of the person doing the ranting. Just like creationists clearly have their own agenda.

  • frelkins

    @Diogenes

    Haven’t you just re-inforced Robin’s point? The vast majority of the studies finding “benefit” seem to be sponsored in some way by big pharma, the people with the most to gain. Not a neutral, trustworthy source of science.

    The only independent, government study seems to show no significant benefit. You question the use of even the government study – therefore it seems there’s no quality, trustworthy evidence for statins either way at all! We know nothing scientific about drugs we spend many billions of dollars a year on. Why are we as a society paying for them?

    Can you please, Diogenes, offer OB several high-quality peer-reviewed ungated “clean” – independent, no pharma! – studies that show significant benefit for this class of drugs or any brand-name statin? They would be useful to see.

  • http://hanson.gmu.edu Robin Hanson

    Diogenes, every study has “methodological problems.” Give a link; your blog has only four posts, none of which mention statins or primary preventions.

  • http://diogenes42.blogspot.com Diogenes

    @robin,

    For people who already have coronary artery disease or something similiar, The UpToDate review is here. The first two trials are the 4S and the LIPID trial. Both I believe were funded by Pharma, but show highly significant results (by p-value) and reasonable effect sizes. These are highly significant results in the context of a highly plausible biological mechanism.

    In the case of primary prevention — i.e. preventing disease in patients who have never had a cardiac event — this is where pharma and a govt trial ALLHAT differ. Apparently, the researchers for ALLHAT, also think they had extensive methodological problems at their own site and an accompanying editorial to the published study, linked to from their site.

    @frelkins, @robin:

    This is not my area, but it seems that results from 4S came out in the middle of ALLHAT, and the “placebo/usual care” group in ALLHAT wound up receiving statins (~26%). This accounted for a smaller difference in LDL between the two groups (compared to pharma Trials), as compared to prior trials. This led to a non-significant difference in cardiovascular mortality.

    When evaluating a medical intervention — its important to consider, side effects, cost, and evidence. Side effects for statins are minimal (maybe a little more than tylenol). Cost is going to be nothing, as one is coming off patent. Evidence: multiple RCTS, and very convincing biological models.

  • http://www.hopeanon.typepad.com Hopefully Anonymous

    Robin, I like the term “meta-scientist”. Anyone in here abreast on the literature (I hope one exists) sussing out the degree to which pharma warps study results. Is it a black box? How exactly does it work? To what degree has this been quantitatively analyzed? For example, some useful numbers would be the percentage of pharma funded studies that show no or insufficient benefit of a drug, compared to the percentage of non-pharma funded studies, and the % of pharma funded studies showing drug benefit that later get disproven, as opposed to studies that don’t show benefit or that aren’t pharma funded.

    And beyond that I’m intersted in the microsocial epistemological mechanisms that result in the production scientific studies saying what benefits the profit motive of a pharmaceutical company rather than what benefits the production of social knowledge. What’s the history of this? What are the current instances? And what are effective restraints?

  • http://www.callcarenet.com shared decision making

    There is a lot we aren’t told as far as the medical industries go.
    But , just watching the news and staying up on current times you will see that doctors are constantly changing there opinions on caffeine, coffee, marijuana, etc,etc

    Snap out of it people
    Eyes wide shut, the news and government use propaganda , so don’t believe everything you hrear

  • http://www.healthtec-software.com/emr.htm emr

    It is truly amazing how one thing is good for you one day and it will kill you the next.

  • http://www.callcarenet.com/products-nursecare.asp health advocate

    It is very important to stay on top of your health and have a check up at least once a year to ensure that you remain healthy. Many health problems are curable but you need to catch them in time. Do not wait to go see a doctor because it may be too late.