<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Disagreement Case Studies</title>
	<atom:link href="http://www.overcomingbias.com/2007/02/seeking_disagre.html/feed" rel="self" type="application/rss+xml" />
	<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html</link>
	<description>Overcoming Bias is economist Robin Hanson’s blog, on honesty, signaling, disagreement, forecasting, and the far future.</description>
	<lastBuildDate>Sat, 13 Mar 2010 23:30:14 -0500</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.5</generator>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
		<item>
		<title>By: Robin Hanson</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421685</link>
		<dc:creator>Robin Hanson</dc:creator>
		<pubDate>Sat, 12 May 2007 13:51:29 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421685</guid>
		<description>Kirk, this is helpful yes, thank you.

</description>
		<content:encoded><![CDATA[<p>Kirk, this is helpful yes, thank you.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Kirk Shanahan</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421684</link>
		<dc:creator>Kirk Shanahan</dc:creator>
		<pubDate>Fri, 11 May 2007 23:52:23 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421684</guid>
		<description>The first step is always to define what you&#039;re talking about.  In the case I am commenting on, I disagree that there exists a body of scientific information that establishes with a high probability that a nuclear process known as &#039;cold fusion&#039; exists which produces &#039;excess heat&#039;.  To that end, I formulated and published what I consider to be a non-nuclear explanation for the extant body of data regarding apparent excess heat.  That publication has been challeged directly once, and I have defended it.  And it has been indirectly challenged another time, and I defended that (both times in print).  So:

# Do you conclude just from the fact that they disagree that they must have more defects?
No.

# Do you think they realize that they can have defects, such as thinking errors or knowing less?
Yes, in principle, but in practice they believe they have eliminated all significant probability that they have errors.  (I believe they also believe a non-expert in the field could not identify significant defects.)

# Should the fact that you disagree be a clue to them about their defects?   Is it a clue about yours?
Normally, the fact that a disagreement has been published is both an indication that I disagee, which should be a clue to them, and it also indicates at least one other agrees with me (via the peer review system), which should also be a clue.  The fact that the one I am disagreeing with also has published an intended rebuttal indicates the same for that side, which should be a clue to me I suppose.  Both sides supposedly try to publish the clearest presentation of their position they can.

# Do they adjust their estimates enough for the possibility of their defects?  If not, why not?
This is a little hard to answer, but I would say no, because as I have
noted before, my counterproposal is ignored, rather than acted on.  In
science when a valid counterproposal is made, meaning one cannot find
a valid reason to reject it, the standard response is to go back to the
lab and conduct new and different experiments aimed at resolving the
issues.  Why they don&#039;t do this would be pure speculation on my part.
I have indicated that the one author I have dealt with in detail
seems to just simply reject my proposal for nonscientific reasons.

# What clues suggest to you that they have more defects, or under-adjust for them?
That their published proposal does not consider at all my
counter-proposal.  That when they are in receipt of my
counter-proposal they reject it for spurious reasons.

# What clues suggest to them that you have more defects, or under-adjust?
Good question.  The short answer is that I don&#039;t know.  They have
suggested that I am incapable of cogent commenting unless I have
personally conducted equivalent experiments, which I have not.
However, my proposal rests on data analysis methods, not data
collection methods, so their point is moot.  But this may not
be the primary reason they do not accept my proposal.

# Do you both have access to these clues, and if so do you interpret them differently?
Absolutely yes to both questions.  That is the crux of the issue.
We propose different mechanisms that produce the &#039;clues&#039; (data).

# Do you each realize some clues might be hidden?
Of course, finding the hidden clues (getting the data) is what science is all about.  A scientific publication, while supposedly presenting a very good case, is actually a request to the general audience for any relevant comments.  Theoretically, the &#039;deal is not ever done&#039;, and the best theories are subject to rejection.  In  practice, a lot of disagreements arise from people refusing to act like this is true.


# Does your inability to answer any of these questions suggest you have defects?
I wasn&#039;t unable to answer for myself.  I also don&#039;t have ESP, so
I can&#039;t say for the others.  Is that a defect?

# Consider all these questions again for your meta-disagreement about who has more defects.
OK, I take this question to mean I should consider why my &#039;opponents&#039;
won&#039;t listen to me, and vice versa.  The problem is that I have listened
to them, in great detail.  If my counter-proposal is not correct, their
position is the default fall-back position, which I have never denied
(in other words, cold fusion is a possibility if no other explanation
works).  On the other side, they don&#039;t accept my position, but are unable
to technically defend their lack of acceptance, which is the requirement
for doing &#039;good&#039; science, i.e. if they want to reject my proposal without
supplying new relevant data, they have to define a logical error.
which they have not done.



So does this help for your case study?
</description>
		<content:encoded><![CDATA[<p>The first step is always to define what you&#8217;re talking about.  In the case I am commenting on, I disagree that there exists a body of scientific information that establishes with a high probability that a nuclear process known as &#8216;cold fusion&#8217; exists which produces &#8216;excess heat&#8217;.  To that end, I formulated and published what I consider to be a non-nuclear explanation for the extant body of data regarding apparent excess heat.  That publication has been challeged directly once, and I have defended it.  And it has been indirectly challenged another time, and I defended that (both times in print).  So:</p>
<p># Do you conclude just from the fact that they disagree that they must have more defects?<br />
No.</p>
<p># Do you think they realize that they can have defects, such as thinking errors or knowing less?<br />
Yes, in principle, but in practice they believe they have eliminated all significant probability that they have errors.  (I believe they also believe a non-expert in the field could not identify significant defects.)</p>
<p># Should the fact that you disagree be a clue to them about their defects?   Is it a clue about yours?<br />
Normally, the fact that a disagreement has been published is both an indication that I disagee, which should be a clue to them, and it also indicates at least one other agrees with me (via the peer review system), which should also be a clue.  The fact that the one I am disagreeing with also has published an intended rebuttal indicates the same for that side, which should be a clue to me I suppose.  Both sides supposedly try to publish the clearest presentation of their position they can.</p>
<p># Do they adjust their estimates enough for the possibility of their defects?  If not, why not?<br />
This is a little hard to answer, but I would say no, because as I have<br />
noted before, my counterproposal is ignored, rather than acted on.  In<br />
science when a valid counterproposal is made, meaning one cannot find<br />
a valid reason to reject it, the standard response is to go back to the<br />
lab and conduct new and different experiments aimed at resolving the<br />
issues.  Why they don&#8217;t do this would be pure speculation on my part.<br />
I have indicated that the one author I have dealt with in detail<br />
seems to just simply reject my proposal for nonscientific reasons.</p>
<p># What clues suggest to you that they have more defects, or under-adjust for them?<br />
That their published proposal does not consider at all my<br />
counter-proposal.  That when they are in receipt of my<br />
counter-proposal they reject it for spurious reasons.</p>
<p># What clues suggest to them that you have more defects, or under-adjust?<br />
Good question.  The short answer is that I don&#8217;t know.  They have<br />
suggested that I am incapable of cogent commenting unless I have<br />
personally conducted equivalent experiments, which I have not.<br />
However, my proposal rests on data analysis methods, not data<br />
collection methods, so their point is moot.  But this may not<br />
be the primary reason they do not accept my proposal.</p>
<p># Do you both have access to these clues, and if so do you interpret them differently?<br />
Absolutely yes to both questions.  That is the crux of the issue.<br />
We propose different mechanisms that produce the &#8216;clues&#8217; (data).</p>
<p># Do you each realize some clues might be hidden?<br />
Of course, finding the hidden clues (getting the data) is what science is all about.  A scientific publication, while supposedly presenting a very good case, is actually a request to the general audience for any relevant comments.  Theoretically, the &#8216;deal is not ever done&#8217;, and the best theories are subject to rejection.  In  practice, a lot of disagreements arise from people refusing to act like this is true.</p>
<p># Does your inability to answer any of these questions suggest you have defects?<br />
I wasn&#8217;t unable to answer for myself.  I also don&#8217;t have ESP, so<br />
I can&#8217;t say for the others.  Is that a defect?</p>
<p># Consider all these questions again for your meta-disagreement about who has more defects.<br />
OK, I take this question to mean I should consider why my &#8216;opponents&#8217;<br />
won&#8217;t listen to me, and vice versa.  The problem is that I have listened<br />
to them, in great detail.  If my counter-proposal is not correct, their<br />
position is the default fall-back position, which I have never denied<br />
(in other words, cold fusion is a possibility if no other explanation<br />
works).  On the other side, they don&#8217;t accept my position, but are unable<br />
to technically defend their lack of acceptance, which is the requirement<br />
for doing &#8216;good&#8217; science, i.e. if they want to reject my proposal without<br />
supplying new relevant data, they have to define a logical error.<br />
which they have not done.</p>
<p>So does this help for your case study?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: rcriii</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421683</link>
		<dc:creator>rcriii</dc:creator>
		<pubDate>Thu, 01 Mar 2007 16:14:56 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421683</guid>
		<description>Robin,  That is a great paper, lots to absorb.

Yesterday I put on my &#039;outsider&#039; hat and looked at the effect of the configurations we were arguing about on past performance, and found little data to support any particular answer.  So should I throw up my hands and let the PM dictate configuration, or should I dictate my preferred answer (I still have all that theory, see)?  Or is the rational thing to stop wasting time on analysis and flip a coin?
</description>
		<content:encoded><![CDATA[<p>Robin,  That is a great paper, lots to absorb.</p>
<p>Yesterday I put on my &#8216;outsider&#8217; hat and looked at the effect of the configurations we were arguing about on past performance, and found little data to support any particular answer.  So should I throw up my hands and let the PM dictate configuration, or should I dictate my preferred answer (I still have all that theory, see)?  Or is the rational thing to stop wasting time on analysis and flip a coin?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: James Annan</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421682</link>
		<dc:creator>James Annan</dc:creator>
		<pubDate>Wed, 28 Feb 2007 09:56:56 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421682</guid>
		<description>I don&#039;t think a truth-seeker can ever simply trust an expert qua expert to give them &quot;the answer&quot;. At least, not a truth-seeker who is close enough to the coal-face (and sufficiently motivated) to actually look into the question themselves (it&#039;s probably a reasonable heuristic in practical cases). But each new question is never quite the same as any preceding one, and expertise is specialised and limited and based on what went before...
</description>
		<content:encoded><![CDATA[<p>I don&#8217;t think a truth-seeker can ever simply trust an expert qua expert to give them &#8220;the answer&#8221;. At least, not a truth-seeker who is close enough to the coal-face (and sufficiently motivated) to actually look into the question themselves (it&#8217;s probably a reasonable heuristic in practical cases). But each new question is never quite the same as any preceding one, and expertise is specialised and limited and based on what went before&#8230;</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Robin Hanson</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421681</link>
		<dc:creator>Robin Hanson</dc:creator>
		<pubDate>Wed, 28 Feb 2007 02:05:04 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421681</guid>
		<description>James, we should talk more about how to determine the &quot;real&quot; experts on topics.
</description>
		<content:encoded><![CDATA[<p>James, we should talk more about how to determine the &#8220;real&#8221; experts on topics.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: James Annan</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421680</link>
		<dc:creator>James Annan</dc:creator>
		<pubDate>Wed, 28 Feb 2007 00:47:46 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421680</guid>
		<description>Robin,

Assuming that truncated comment was simply &quot;are they not in fact more expert&quot; then I am sure that most people would consider them to be. However, it&#039;s worth noting that this is a rather new area of climate science and although they moved sideways into it before I did, they didn&#039;t have any real background in it before that time. Neither did anyone else in climate science, which is why they were able to establish themselves as experts without much meaningful review.

One clue that suggests their defects to me is that they have not been able to coherently and clearly state their case.

Another is that there are plenty of &quot;meta-experts&quot; who are far more expert in the field of probabilistic estimation in general - but not climate science in particular - who have been completely scathing about what has been going on in the field. In no small part they are frozen out by the established climate science clique.
</description>
		<content:encoded><![CDATA[<p>Robin,</p>
<p>Assuming that truncated comment was simply &#8220;are they not in fact more expert&#8221; then I am sure that most people would consider them to be. However, it&#8217;s worth noting that this is a rather new area of climate science and although they moved sideways into it before I did, they didn&#8217;t have any real background in it before that time. Neither did anyone else in climate science, which is why they were able to establish themselves as experts without much meaningful review.</p>
<p>One clue that suggests their defects to me is that they have not been able to coherently and clearly state their case.</p>
<p>Another is that there are plenty of &#8220;meta-experts&#8221; who are far more expert in the field of probabilistic estimation in general &#8211; but not climate science in particular &#8211; who have been completely scathing about what has been going on in the field. In no small part they are frozen out by the established climate science clique.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Robin Hanson</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421679</link>
		<dc:creator>Robin Hanson</dc:creator>
		<pubDate>Tue, 27 Feb 2007 20:12:29 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421679</guid>
		<description>James, you say most people defer to a few people who oppose you, thinking those few are more expert.  You focus your comments on these many followers, but don&#039;t say much about these few they follow.  What clues suggest their defects?  Are they not in fact more expert?
</description>
		<content:encoded><![CDATA[<p>James, you say most people defer to a few people who oppose you, thinking those few are more expert.  You focus your comments on these many followers, but don&#8217;t say much about these few they follow.  What clues suggest their defects?  Are they not in fact more expert?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Robin Hanson</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421678</link>
		<dc:creator>Robin Hanson</dc:creator>
		<pubDate>Tue, 27 Feb 2007 19:23:20 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421678</guid>
		<description>Rciii, thanks for trying this!  Your disagreement seems to a version of the classic &quot;inside view&quot; (detailed analysis) vs. &quot;outside view&quot; (experience with similar cases).  See Kahneman, D. and Lovallo, D. (1993) Timid choices and bold forecasts. A cognitive perspective on risk taking. Management Science, 39, 17-31.

Felix, thanks also!  I&#039;ll leave my comments on your blog.
</description>
		<content:encoded><![CDATA[<p>Rciii, thanks for trying this!  Your disagreement seems to a version of the classic &#8220;inside view&#8221; (detailed analysis) vs. &#8220;outside view&#8221; (experience with similar cases).  See Kahneman, D. and Lovallo, D. (1993) Timid choices and bold forecasts. A cognitive perspective on risk taking. Management Science, 39, 17-31.</p>
<p>Felix, thanks also!  I&#8217;ll leave my comments on your blog.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: rcriii</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421677</link>
		<dc:creator>rcriii</dc:creator>
		<pubDate>Tue, 27 Feb 2007 18:18:03 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421677</guid>
		<description>I wrote up a dispute and my answers to the questions &lt;a href=http://rcriii.backpackit.com/pub/973585 rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.
</description>
		<content:encoded><![CDATA[<p>I wrote up a dispute and my answers to the questions <a href=http://rcriii.backpackit.com/pub/973585 rel="nofollow">here</a>.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Hal Finney</title>
		<link>http://www.overcomingbias.com/2007/02/seeking_disagre.html#comment-421676</link>
		<dc:creator>Hal Finney</dc:creator>
		<pubDate>Tue, 27 Feb 2007 07:26:05 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2007/02/disagreement-case-studies.html#comment-421676</guid>
		<description>I see some interesting contrasts among the disagreements described by Felix, James and me. Felix and I find ourselves in a similar position: our respective disputants are relatively expert compared to us, and have strong and loyal followings. At the same time I think we are both advocating a generally mainstream view, while our disputants are mavericks in their fields, whose positions are in the minority. One difference is that Felix really thinks his arguments are superior, while I am more inclined to view quality of argument as not a very useful signifier of truth.

James is in something of the opposite position, acting as a lonely voice trying to persuade the larger mass of opinion to his view. Although it seems to have been an uphill battle for him, he persists, confident in the superior quality of his arguments.

From the &quot;meta&quot; perspective, the odds are against James. Most people who find themselves in positions like his turn out to be mistaken. Nevertheless, as a scientist, he perhaps has a professional obligation to advocate and defend his views so that they get a fair hearing. Just as we don&#039;t want a legal system where lawyers make their own judgments about guilt and then will only argue the side they think is right, society would probably not be benefited by a scientific community whose members were too quick to adopt the majority view, even though that would increase the average accuracy of individual scientists.
</description>
		<content:encoded><![CDATA[<p>I see some interesting contrasts among the disagreements described by Felix, James and me. Felix and I find ourselves in a similar position: our respective disputants are relatively expert compared to us, and have strong and loyal followings. At the same time I think we are both advocating a generally mainstream view, while our disputants are mavericks in their fields, whose positions are in the minority. One difference is that Felix really thinks his arguments are superior, while I am more inclined to view quality of argument as not a very useful signifier of truth.</p>
<p>James is in something of the opposite position, acting as a lonely voice trying to persuade the larger mass of opinion to his view. Although it seems to have been an uphill battle for him, he persists, confident in the superior quality of his arguments.</p>
<p>From the &#8220;meta&#8221; perspective, the odds are against James. Most people who find themselves in positions like his turn out to be mistaken. Nevertheless, as a scientist, he perhaps has a professional obligation to advocate and defend his views so that they get a fair hearing. Just as we don&#8217;t want a legal system where lawyers make their own judgments about guilt and then will only argue the side they think is right, society would probably not be benefited by a scientific community whose members were too quick to adopt the majority view, even though that would increase the average accuracy of individual scientists.</p>
]]></content:encoded>
	</item>
</channel>
</rss>
