In what may be among the longest-running and widest-ranging cases of academic fraud, one of the most prolific researchers in anesthesiology has admitted that he fabricated much of the data underlying his research, said a spokeswoman for the hospital where he works.

 The researcher, Dr. Scott S. Reuben, an anesthesiologist in Springfield, Mass., who practiced at Baystate Medical Center, never conducted the clinical trials that he wrote about in 21 journal articles dating from at least 1996, said Jane Albert, a spokeswoman for Baystate Health. . . .

The drug giant Pfizer underwrote much of Dr. Reuben’s research from 2002 to 2007. Many of his trials found that Celebrex and Lyrica, Pfizer drugs, were effective against postoperative pain. . . .

“When researchers are beholden to companies for much of their income, there is an incredible tendency to get results that are favorable to the company,” said Dr. Jerome Kassirer, a former editor of The New England Journal of Medicine and the author of a book about conflicts of interest.

Is it naive to be astonished that it took this long for someone to notice that 21 scholarly articles had been published about clinical trials that had never occurred?

1. From JAMA, a new study analyzes more than two decades of heart care guidelines (that is, the guidelines that your doctor might follow in deciding how to treat you) from the American College of Cardiology and the American Heart Association. The study found that the overwhelming majority of recommendations are not supported by good evidence:

Level of evidence provides the link between recommendations and evidence base. Although there is significant variation among individual guidelines in available evidence supporting recommendations, the median of level of evidence A recommendations [i.e., those supported by more than one randomized trial] is only 11% across current guidelines, whereas the most common grade assigned is level of evidence C, indicating little to no objective empirical evidence for the recommended action. . . . Interestingly, our findings are reflective of a specialty — cardiology — that has a large pool of research to draw on for its care recommendations. Guidelines in other medical areas in which large clinical trials are performed less frequently may have an even weaker evidence-based foundation.

2. In this post, Dr. Eades criticizes (convincingly, I think) a recent study purporting to show that statins reduce mortality.

I think what religion and politics have in common is that they become part of people's identity, and people can never have a fruitful argument about something that's part of their identity. By definition they're partisan.

Which topics engage people's identity depends on the people, not the topic. For example, a discussion about a battle that included citizens of one or more of the countries involved would probably degenerate into a political argument. But a discussion today about a battle that took place in the Bronze Age probably wouldn't. No one would know what side to be on. So it's not politics that's the source of the trouble, but identity. When people say a discussion has degenerated into a religious war, what they really mean is that it has started to be driven mostly by people's identities. . . .

The most intriguing thing about this theory, if it's right, is that it explains not merely which kinds of discussions to avoid, but how to have better ideas. If people can't think clearly about anything that has become part of their identity, then all other things being equal, the best plan is to let as few things into your identity as possible.

Most people reading this will already be fairly tolerant. But there is a step beyond thinking of yourself as x but tolerating y: not even to consider yourself an x. The more labels you have for yourself, the dumber they make you.

Via Andy McKenzie.

In a presidential campaign that promises straight talk and no gimmicks, why do both candidates champion one of medical care’s most pervasive myths? The myth is that like magic, preventive medicine will simultaneously reduce costs and improve health. . . .

The term "preventive medicine" no longer means what it used to: keeping people well by promoting healthy habits, like exercising, eating a balanced diet and not smoking. To their credit, both candidates ardently support that approach. But the medical model for prevention has become less about health promotion and more about early diagnosis. Both candidates appear to have bought into it: Mr. Obama encourages annual checkups and screening, Mr. McCain early testing and screening.

It boils down to encouraging the well to have themselves tested to make sure they are not sick. And that approach doesn’t save money; it costs money.

Increasing the amount of testing for an ever-expanding list of problems always identifies many more people as having disease and still more as being "at risk." Screening for heart disease, problems in major blood vessels and a variety of cancers has led to millions of diagnoses of these diseases in people who would never have become sick. . . .

These interventions do prevent advanced illness in some patients, but relatively few. Any savings from preventing those cases is dwarfed by the cost of intervening early in millions of additional patients. No wonder pharmaceutical companies and medical centers see preventive medicine as a great way to turn people into patients — and paying customers.  . . .

Early screening is like the "check engine" light in your car. It can alert you to problems that need to be fixed, but too often it picks up trivial abnormalities that don’t affect performance, like one sensor’s recognizing that another sensor isn’t sensing. . . .

It’s hard to ignore a "check-engine" light. Some mechanics reset them and see if they come on again, but often they lead you to a repair. And you may have had the unfortunate experience that a repair makes matters worse.

If so, you have some feel for the problem of overdiagnosis. Almost everybody with a diagnosis undergoes treatment. And all of our treatments have some harms. From 1 to 5 percent of patients die after major surgery, and as we are all increasingly aware, prescription medicines carry real risks. Recent experiences with hormone replacement (breast cancer) and Vioxx (heart attacks) are potent reminders that our "best" new treatments may harbor unpleasant surprises.

Taubes has several overarching themes; he contends, for example, that eating refined carbohydrates is what makes you obese, and that refined carbohydrates contribute to many of what used to be called "diseases of civilization" (such as heart disease, which seems to have been less common in traditional cultures that ate less processed food, including Northern cultures that ate almost exclusively meat).  (These arguments are still controversial, although new evidence continues to support them.)

The most important theme, however, suffuses the entire book: bias in scientific inquiry. 

Most of the chapters are headed by bias-related quotations, such as this from a 1921 book on philosophy of science: "In reality, those who repudiate a theory that they had once proposed, or a theory that they had accepted enthusiastically and with which they had identified themselves, are very rare.  The great majority of them shut their ears so as not to hear the crying facts, and shut their eyes so as not to see the glaring facts, in order to remain faithful to their theories in spite of all and everything." Or this quotation from a 1950 Fields Medal winner: "The thing is, it’s very dangerous to have a fixed idea. A person with a fixed idea will always find some way of convincing himself in the end that he is right."

Why is Taubes so interested in bias?  For several decades, it has been the conventional wisdom that dietary fat (and especially saturated fat) contributes to obesity, heart disease, and cancer.  Judging from Taubes’ exhaustive research — indeed, I’d be surprised if any other book examined bias within a particular scientific field in such detail — the conventional wisdom was based on unreliable and slender evidence that, once established and institutionalized in government funding, set a pattern of confirmation bias by which further research was judged (or ignored).  To take several examples (the book is full of many more):

• Researcher Ancel Keys is perhaps the most important figure behind the origins of the dietary fat hypothesis.  Two of his most famous studies found a strong correlation between diet and heart disease in a handful of countries, but he cherry-picked the countries to analyze, omitting countries that would have undermined or even eliminated the correlation entirely. (pp. 18, 31-33).
• Dietary researchers tended to ignore — or refused to allow publication of — studies showing that diet, cholesterol, and heart disease were not even correlated (pp. 27, 35), or even that low cholesterol raises other risks of death (in several studies, people with low cholesterol and/or people who ate low-fat diets were more likely to die of cancer, see pp. 37, 54, 71, 81).  As for cholesterol-lowering diets (which may include lots of polyunsaturated fat, and hence are different from low-fat diets per se), dietary researchers tended to rely on one positive result from a Helsinki study, while ignoring a politically incorrect result from a clinical trial in Minnesota (in which 269 mental patients assigned to a cholesterol-lowering diet died, compared to 206 in the control group). The Minnesota result wasn’t even published for 16 years; Taubes asked the researcher why, and got the response: "We were just disappointed in the way it came out."  (pp. 37-38).

• Similarly, in 1961, a conference of the Association of American Physicians included a presentation showing that in comparing heart disease patients in New Haven to a healthy population, the diseased patients were much more likely to have high triglycerides than high cholesterol, thus implicating high carbohydrate diets (which elevate triglycerides).  One of the researchers told Taubes, "It just about brought the house down.  People were so angry; they said they didn’t believe it."  Despite this result, later studies funded by the National Institutes of Health would completely ignore triglycerides, focusing only on cholesterol levels. (pp. 157-60). 

Taubes closes the book with these scathing words:

The institutionalized vigilance, “this unending exchange of critical judgment,” is nowhere to be found in the study of nutrition, chronic disease, and obesity, and it hasn’t been for decades.  For this reason, it is difficult to use the term “scientists” to describe those individuals who work in these disciplines, and, indeed, I have actively avoided doing so in this book.  It’s simply debatable, at best, whether what these individuals have practiced for the past fifty years, and whether the culture they have created, as a result, can reasonably be described as science.

Here are some informative interviews with Taubes:

PBS. 

MIT.

The Borzoi Reader.

The Telegraph. Notably, Taubes admits in this interview that he himself might be biased: “What are the chances of writing an article that says the entire medical establishment is wrong, and them going, ‘Good point, thank you, Gary. Can we give you an award?’ When people challenge the establishment, 99.9 per cent of the time they are wrong. If I was writing about me, I’d begin from the assumption that I am both wrong and a quack.”

One logical conjecture would be to expect this form of political behavior to be much like any other.  In other words, it would be disproportionately the province of well-educated, high-income populations.  Indeed, the frequency of general political discussion tracks closely with these characteristics of high socioeconomic status.  But the correlates of cross-cutting conversation are strikingly different.  As shown in Figure 2.3, there are clear patterns of difference with respect to race, income, and education, but they are not in the usual directions.  Nonwhites are significantly more likely to engage in cross-cutting political conversation than whites.  And as income increases, the frequency of disagreeable conversations declines.   Exposure to disagreement is highest among those who have completed less than a high school degree and lowest among those who have attended graduate school.

As sociologist William Weston notes in discussing Mutz’s findings:

I can testify to how easy it is for conversation among academics, the most educated group of people, to turn into a one-position echo chamber. Liberalism is taken to be an IQ test, and the rare conservative is encouraged to be quiet or go elsewhere. For political disagreement I go to the coffee house, which in our town draws a broader range of people than the faculty club contains.

Of course, one explanation would be that what looks like herd behavior and social conformity is really just what happens when a bunch of superior intellects independently settle on the objectively correct viewpoint.  But that’s rather a self-serving explanation, isn’t it?

From the abstract:

Brain images are believed to have a particularly persuasive influence on the public perception of research on cognition. Three experiments are reported showing that presenting brain images with articles summarizing cognitive neuroscience research resulted in higher ratings of scientific reasoning for arguments made in those articles, as compared to articles accompanied by bar graphs, a topographical map of brain activation, or no image. These data lend support to the notion that part of the fascination, and the credibility, of brain imaging research lies in the persuasive power of the actual brain images themselves. We argue that brain images are influential because they provide a physical basis for abstract cognitive processes, appealing to people’s affinity for reductionistic explanations of cognitive phenomena.

As the BPS blog elaborates:

David McCabe and Alan Castel presented university students with 300-word news stories about fictional cognitive research findings that were based on flawed scientific reasoning. For example, one story claimed that watching TV was linked to maths ability, based on the fact that both TV viewing and maths activate the temporal lobe. Crucially, students rated these stories to be more scientifically sound when they were accompanied by a brain image, compared with when the equivalent data were presented in a bar chart, or when there was no graphical illustration at all.

This fits in with the theme of how people tend to overvalue something that is dressed up in the attire of science. 

Via this blog, here’s comedian Lenny Bruce making a similar point:

I would be with a bunch of Kennedy fans watching the debate and their comment would be, “He’s really slaughtering Nixon.” Then we would all go to another apartment, and the Nixon fans would say, “How do you like the shellacking he gave Kennedy?” And then I realized that each group loved their candidate so that a guy would have to be this blatant — he would have to look into the camera and say: “I am a thief, a crook, do you hear me, I am the worst choice you could ever make for the Presidency!” And even then his following would say, “Now there’s an honest man for you. It takes a big guy to admit that. There’s the kind of guy we need for President.”

2.  Tyler Cowen has an article in The New Republic that is rather cynical about the value of most published research:

The sad truth is that "non-fiction" has been unreliable from the beginning, no matter how finely grained a section of human knowledge we wish to consider. For instance, in my own field, critics have tried to replicate the findings in academic journal articles by economists using the initial data sets. Usually, it is impossible to replicate the results of the article even half of the time. Note that the journals publishing these articles often use two or three referees–experts in the area–and typically they might accept only 10 percent of submitted papers. By the way, economics is often considered the most rigorous and the most demanding of the social sciences.

3.  Seth Roberts points out that the value of data is not binary, i.e., either convincing or worthless:

A vast number of scientists have managed to convince themselves that skepticism means, or at least includes, the opposite of value data. They tell themselves that they are being “skeptical” — properly, of course — when they ignore data. They ignore it in all sorts of familiar ways. They claim “correlation does not equal causation” — and act as if the correlation is meaningless. They claim that "the plural of anecdote is not data" — apparently believing that observations not collected as part of a study are worthless. Those are the low-rent expressions of this attitude. The high-rent version is when a high-level commission delegated to decide some question ignores data that does not come from a placebo-controlled double-blind study, or something similar.

So considering links 2 and 3, should we really downgrade the evidentiary value of published studies and upgrade the evidentiary value of anecdotes?  (That wouldn’t mean treating them both as equal, of course.) 

SCIENTISTS AT CALTECH and Stanford recently published the results of a peculiar wine tasting. They provided people with cabernet sauvignons at various price points, with bottles ranging from $5 to $90. Although the tasters were told that all the wines were different, the scientists were in fact presenting the same wines at different prices. The subjects consistently reported that the more expensive wines tasted better, even when they were actually identical to cheaper wines.

The experiment was even more unusual because it was conducted inside a scanner – the drinks were sipped via a network of plastic tubes – that allowed the scientists to see how the subjects’ brains responded to each wine. When subjects were told they were getting a more expensive wine, they observed more activity in a part of the brain known to be involved in our experience of pleasure.

What they saw was the power of expectations. People expect expensive wines to taste better, and then their brains literally make it so. Wine lovers shouldn’t feel singled out: Antonio Rangel, the Caltech neuroeconomist who led the study, insists that he could have used a variety of items to get similar results, from bottled water to modern art. …

Shiv cites research showing that cars made in the same factory, with the same parts, but sold under different brand names (such as Toyota and Geo) receive markedly different reliability ratings from consumers. When we drive a car with a less exalted brand name, we are more likely to notice minor mechanical problems.

A second item:

A 10-cent pill doesn’t kill pain as well as a $2.50 pill, even when they are identical placebos, according to a provocative study by Dan Ariely, a behavioral economist at Duke University. …
Ariely and a team of collaborators at the Massachusetts Institute of Technology used a standard protocol for administering light electric shock to participants’ wrists to measure their subjective rating of pain. The 82 study subjects were tested before getting the placebo and after. Half the participants were given a brochure describing the pill as a newly-approved pain-killer which cost $2.50 per dose and half were given a brochure describing it as marked down to 10 cents, without saying why.

In the full-price group, 85 percent of subjects experienced a reduction in pain after taking the placebo. In the low-price group, 61 percent said the pain was less.

A third item:

A GERMAN art expert was fooled into believing a painting done by a chimpanzee was the work of a master. The director of the State Art Museum of Moritzburg in Saxony-Anhalt, Katja Schneider, suggested the painting was by the Guggenheim Prize-winning artist Ernst Wilhelm Nay. "It looks like an Ernst Wilhelm Nay. He was famous for using such blotches of colour," Dr Schneider confidently asserted.

The canvas was actually the work of Banghi, a 31-year-old female chimp at the local zoo.

This same sort of misjudgment surely applies to the world of ideas. I wonder how often academics mistakenly but sincerely believe that Professor So-and-so’s idea is brilliant, even though they would have found the exact same idea deficient or perhaps just average if it were put forth by anyone else. 

To avoid this bias, you should try to be skeptical towards anything that you know you’re supposed to admire — whether it be a TV show or movie that everyone praises, the professors whose research is “hot” in any given field, etc. 

On the other hand, that does not mean that you should be contrarian for its own sake; the thing that everyone values may really be better. Put it this way: On a scale of 1 to 100, with 1 representing inventors of perpetual motion and 100 representing Frank Ramsey or John von Neumann, let’s say that the average “idea” in an academic field is at a value of 30, and that the ideas in the top quintile of highly-regardedness (to coin a word) average a value of 60.*  So you shouldn’t actually reject an idea just for being highly-regarded.  But suppose that, because of the herd effect described above, the most highly-regarded ideas are treated as if they have a value of 85 or 90 — far above their real value. What I’m saying is to be skeptical and ask whether an idea is really as insightful or worthy as is reputed, or whether it deserves to be taken down a notch to its real value (which may still be above average).

* These figures have a margin of error of plus-or-minus 2 points, by the way.