Darwinian medicine pioneer Randolph Nesse answers this year’s Edge question: I used to believe that you could find out what is true by finding the smartest people and finding out what they think. However, the most brilliant people keep turning out to be wrong. … A lot of the belief that smart people are right is an illusion caused by smart people being very convincing … even when they are wrong. I also used to believe that you could find out what is true by relying on smart experts who devote themselves to a topic. But most of us remember being told to eat margarine because it is safer than butter then it turned out that trans-fats are worse. … I shudder to think about all the false beliefs I have unknowingly but confidently passed on to my patients, thanks to my trust in experts. …
Hi Randoph, thanks for commenting. You'd be welcome to join this blog as a contributor, able to make your own posts here. For example, we did cover the antidepressant publication selection bias news, but I'm sure you would have done a more expert job of it. You may recall that we both presented at Susanne Lohmann's May 2001 conference "Why We Get Sick."
Thanks for those comments. They make me feel somewhat understood, and a bit less uncomfortable about posting a somewhat intemperate piece on EDGE. After all, I AM, or claim to be, an expert at a university, so…
Here is the back-story that may be of interest. I am spending this year writing a book about why depression exits, really about the evolutionary origins of low mood and the relatively fragile mechanisms that regulate mood. In the process of my research, I went looking for information on the state of evidence on stress hormones causing depression (a previous research topic of mine). As I read review after review, it quickly became clear that the reviewers were expert stress researchers and nearly every one emphasized the role of the HPA axis in causing depression. But when I went to look at the primary literature, I found reports showing no difference whatsoever in HPA activity between depressed and nondepressed people. I have more work to do on this, but it was quite frustrating to find I could not rely on the experts. Some are biased on purpose or for profit, but I think most, having devoted their lives to finding out how stress could cause depression, found all the positive evidence most salient, and the negative evidence uninteresting.
Then there was Turner’s Jan 20, 2008 publication in the NEJM showing that 37/38 studies showing antidepressants effective were published, while only 3/36 negative studies were published as a negative results. Those of us in psychiatry who try hard to use the evidence turn out to have had only a very biased selection of evidence available. I am afraid my suspicions about the opinions of experts turn out to not go nearly far enough. (Other fields may, of course, be profoundly different)
The other thing on my mind was the difficulty universities have in finding appropriate positions for researchers who are asking evolutionary questions about why bodies are vulnerable to disease. Even a university that wants to establish a program in Darwinian medicine must contend with established disciplines that, understandably, want to hire more of their own kind. It is hard for work in an entirely new area to get started. And as I thought about it, it also seemed clear that Departments tend to replicate and extend biases already present. So, for instance, Departments dominated by post-modernists will tend to exclude those who rely on numerical data. In biology, Departments with a tradition of studying adaptation tend to hire more people with those interests, while Departments dominated by anti-adaptationism would not consider such candidates. Such agreement make for congeniality, but not for engaging real issues. Again, there is no one to blame, this is just an emergent result of human tendencies. Nonetheless, it seems to me to result in insularity of the sort that makes it very hard to see, much less fight, bias.
Ideas futures is an interesting thought. I am spending this year in Berlin with a group working on Darwinian medicine and one of our main challenges is to find out some way assess which of a thousand evolutionary medicine hypotheses is true. I have a cookbook piece on my web to guide hypothesis testing, but it still takes days to really assess any one proposal. Just this week I decided to see if hunter gatherers really don’t get heart disease, as I and others have been saying for years—so far the evidence is looking thin indeed.
You site is interesting and the topic could not be more important. I have written a bit on how being objective is not good for one’s reproductive success, but that is whole different topic. I will read more on your site.
Constant, are you sure you are responding to this post?
When you include in your statement an endorsement of the attitude "that academia is the best we have", where "academia" can be more precisely specified as "smart university experts" (Mr. Nesse's words) or for short, "experts", you are endorsing the attitude that "experts are the best we have", and in doing so, implying that you need to endorse this attitude, thus implying that this attitude is less widespread than you would like it to be. So you are implying that too many people do not believe that experts are the best we have. And this, in turn, seems to be an ongoing issue for you.
Assuming: the expert is speaking in good faith, is correctly logical, and is working from the complete current best evidence.
Then: nobody could reach a different rational conclusion except by adding fresh evidence. This still applies if research is being prevented. Evidence that hasn't been discovered yet is not a base for conclusions. Frustration isn't evidence.
Corollary: people who ignore expert opinion are irrational, unless they're accusing the experts of malpractice. Being irrational, they aren't automatically wrong - but they can't tell right from wrong, and they can't update in a way that reliably improves.
Julian, I'm not sure I follow. Compared to what standard would you never be avoidably wrong (the suggestion implicit in Nesse's piece is that institutions are biased against pushing the state of the art). Also without insider or expert knowledge how does one determine who is a conscientious scientist or not?
The emerging picture is of 'he who pays the piper...'. When the media are paying the piper, the experts will pronounce to make a good story (Latest ! The Eat Butter And Drink Whisky Diet !!!). When the Universities are paying the piper, the experts will (according to Nesse, I don't agree) be collectively conservative and hidebound. When the pharm. labs are paying the piper, you get the publications bias illustrated in another thread. When Robin's Ideas Markets are paying the piper.... fill in the blanks yourselves. I don't profess to know, I'm just not a True Believer. Wall Street has shown it has to be heavily regulated, and Atlantic City isn't too healthy, by all accounts.
The thing about expert opinion, assuming the experts are conscientious scientists, is that it improves. At any one time the experts may give wrong advice, but if you track the state of the art you will never be avoidably wrong. The trouble with people who listen to experts, is that they tend to think the question is then closed - they can set it in stone and forget their curiosity. The lesson of the margarine mistake is: keep asking.
Paul, just because people must go to college in our society to be accepted into good jobs doesn't mean colleges can claim credit for everything college graduates do afterward.
Constant, are you sure you are responding to this post?
And yet... and yet... how easy it is to forget the stunning scientific advances that the universities have produced either directly (in-house) or indirectly (by training those produce them).
I think Neese is suffering from his own bias here. The failings are much more salient than the (vastly more numerous) successes.
It is not clear to me whether you think that people actually count expertise against the expert, or whether you think that people, while counting expertise in favor of the expert, do not count it highly enough. Your comments earlier suggest the former, but my own observation suggests that, if anything, the latter is the most that can be argued for.
If someone constructs a poor argument and someone else constructs a better argument, then I will tend to side with the latter. If the former turns out to be an expert and the latter a non-expert, then this will likely boost the credibility of the former. But it might not boost it enough to outweigh the superior arguments of the latter. As Eliezer has pointed out, argument screens off authority.
So I wonder whether you are observing the phenomenon of argument screening off authority, and misinterpreting it as a bias against expertise in favor of non-expertise.
Hi Randoph, thanks for commenting. You'd be welcome to join this blog as a contributor, able to make your own posts here. For example, we did cover the antidepressant publication selection bias news, but I'm sure you would have done a more expert job of it. You may recall that we both presented at Susanne Lohmann's May 2001 conference "Why We Get Sick."
Hi Everyone,
Thanks for those comments. They make me feel somewhat understood, and a bit less uncomfortable about posting a somewhat intemperate piece on EDGE. After all, I AM, or claim to be, an expert at a university, so…
Here is the back-story that may be of interest. I am spending this year writing a book about why depression exits, really about the evolutionary origins of low mood and the relatively fragile mechanisms that regulate mood. In the process of my research, I went looking for information on the state of evidence on stress hormones causing depression (a previous research topic of mine). As I read review after review, it quickly became clear that the reviewers were expert stress researchers and nearly every one emphasized the role of the HPA axis in causing depression. But when I went to look at the primary literature, I found reports showing no difference whatsoever in HPA activity between depressed and nondepressed people. I have more work to do on this, but it was quite frustrating to find I could not rely on the experts. Some are biased on purpose or for profit, but I think most, having devoted their lives to finding out how stress could cause depression, found all the positive evidence most salient, and the negative evidence uninteresting.
Then there was Turner’s Jan 20, 2008 publication in the NEJM showing that 37/38 studies showing antidepressants effective were published, while only 3/36 negative studies were published as a negative results. Those of us in psychiatry who try hard to use the evidence turn out to have had only a very biased selection of evidence available. I am afraid my suspicions about the opinions of experts turn out to not go nearly far enough. (Other fields may, of course, be profoundly different)
The other thing on my mind was the difficulty universities have in finding appropriate positions for researchers who are asking evolutionary questions about why bodies are vulnerable to disease. Even a university that wants to establish a program in Darwinian medicine must contend with established disciplines that, understandably, want to hire more of their own kind. It is hard for work in an entirely new area to get started. And as I thought about it, it also seemed clear that Departments tend to replicate and extend biases already present. So, for instance, Departments dominated by post-modernists will tend to exclude those who rely on numerical data. In biology, Departments with a tradition of studying adaptation tend to hire more people with those interests, while Departments dominated by anti-adaptationism would not consider such candidates. Such agreement make for congeniality, but not for engaging real issues. Again, there is no one to blame, this is just an emergent result of human tendencies. Nonetheless, it seems to me to result in insularity of the sort that makes it very hard to see, much less fight, bias.
Ideas futures is an interesting thought. I am spending this year in Berlin with a group working on Darwinian medicine and one of our main challenges is to find out some way assess which of a thousand evolutionary medicine hypotheses is true. I have a cookbook piece on my web to guide hypothesis testing, but it still takes days to really assess any one proposal. Just this week I decided to see if hunter gatherers really don’t get heart disease, as I and others have been saying for years—so far the evidence is looking thin indeed.
You site is interesting and the topic could not be more important. I have written a bit on how being objective is not good for one’s reproductive success, but that is whole different topic. I will read more on your site.
Thanks for having me over.
Randolph Nessehttp://nesse.us
Constant, are you sure you are responding to this post?
When you include in your statement an endorsement of the attitude "that academia is the best we have", where "academia" can be more precisely specified as "smart university experts" (Mr. Nesse's words) or for short, "experts", you are endorsing the attitude that "experts are the best we have", and in doing so, implying that you need to endorse this attitude, thus implying that this attitude is less widespread than you would like it to be. So you are implying that too many people do not believe that experts are the best we have. And this, in turn, seems to be an ongoing issue for you.
Corollary: people who ignore expert opinion are irrational
Agreed. But people who bow to expert opinion are gullible.
@alexa-blue
Assuming: the expert is speaking in good faith, is correctly logical, and is working from the complete current best evidence.
Then: nobody could reach a different rational conclusion except by adding fresh evidence. This still applies if research is being prevented. Evidence that hasn't been discovered yet is not a base for conclusions. Frustration isn't evidence.
Corollary: people who ignore expert opinion are irrational, unless they're accusing the experts of malpractice. Being irrational, they aren't automatically wrong - but they can't tell right from wrong, and they can't update in a way that reliably improves.
Julian, I'm not sure I follow. Compared to what standard would you never be avoidably wrong (the suggestion implicit in Nesse's piece is that institutions are biased against pushing the state of the art). Also without insider or expert knowledge how does one determine who is a conscientious scientist or not?
The emerging picture is of 'he who pays the piper...'. When the media are paying the piper, the experts will pronounce to make a good story (Latest ! The Eat Butter And Drink Whisky Diet !!!). When the Universities are paying the piper, the experts will (according to Nesse, I don't agree) be collectively conservative and hidebound. When the pharm. labs are paying the piper, you get the publications bias illustrated in another thread. When Robin's Ideas Markets are paying the piper.... fill in the blanks yourselves. I don't profess to know, I'm just not a True Believer. Wall Street has shown it has to be heavily regulated, and Atlantic City isn't too healthy, by all accounts.
The thing about expert opinion, assuming the experts are conscientious scientists, is that it improves. At any one time the experts may give wrong advice, but if you track the state of the art you will never be avoidably wrong. The trouble with people who listen to experts, is that they tend to think the question is then closed - they can set it in stone and forget their curiosity. The lesson of the margarine mistake is: keep asking.
Paul, just because people must go to college in our society to be accepted into good jobs doesn't mean colleges can claim credit for everything college graduates do afterward.
Constant, are you sure you are responding to this post?
And yet... and yet... how easy it is to forget the stunning scientific advances that the universities have produced either directly (in-house) or indirectly (by training those produce them).
I think Neese is suffering from his own bias here. The failings are much more salient than the (vastly more numerous) successes.
It is not clear to me whether you think that people actually count expertise against the expert, or whether you think that people, while counting expertise in favor of the expert, do not count it highly enough. Your comments earlier suggest the former, but my own observation suggests that, if anything, the latter is the most that can be argued for.
If someone constructs a poor argument and someone else constructs a better argument, then I will tend to side with the latter. If the former turns out to be an expert and the latter a non-expert, then this will likely boost the credibility of the former. But it might not boost it enough to outweigh the superior arguments of the latter. As Eliezer has pointed out, argument screens off authority.
So I wonder whether you are observing the phenomenon of argument screening off authority, and misinterpreting it as a bias against expertise in favor of non-expertise.