<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Reveal Rejects?</title>
	<atom:link href="http://www.overcomingbias.com/2009/03/reveal-rejects.html/feed" rel="self" type="application/rss+xml" />
	<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html</link>
	<description>Overcoming Bias is economist Robin Hanson’s blog, on honesty, signaling, disagreement, forecasting, and the far future.</description>
	<lastBuildDate>Sat, 11 Feb 2012 23:03:19 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.1.3</generator>
	<item>
		<title>By: Barkley Rosser</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385654</link>
		<dc:creator>Barkley Rosser</dc:creator>
		<pubDate>Mon, 30 Mar 2009 16:01:19 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385654</guid>
		<description>But in a sports contest, especially ones involving teams, many of those doing the affiliating are affiliating with the losing team, and continue to do so, usually, even if the team loses.  They are paying to attend to see their team compete with the hope that they are going to win, or at least play well.

Those reading journal articles, especially those purchasing subscriptions to the journal, are doing so to read the articles published in the journal, or perhaps to affiliate with the journal.  But do these people think about &quot;affiliating&quot; with the authors or titles of papers rejected for publication?  Certainly not in the same way that fans affiliate with a losing team, or in the case of more individualized sports such as tennis or golf, with a losing player.
</description>
		<content:encoded><![CDATA[<p>But in a sports contest, especially ones involving teams, many of those doing the affiliating are affiliating with the losing team, and continue to do so, usually, even if the team loses.  They are paying to attend to see their team compete with the hope that they are going to win, or at least play well.</p>
<p>Those reading journal articles, especially those purchasing subscriptions to the journal, are doing so to read the articles published in the journal, or perhaps to affiliate with the journal.  But do these people think about &#8220;affiliating&#8221; with the authors or titles of papers rejected for publication?  Certainly not in the same way that fans affiliate with a losing team, or in the case of more individualized sports such as tennis or golf, with a losing player.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Robin Hanson</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385653</link>
		<dc:creator>Robin Hanson</dc:creator>
		<pubDate>Mon, 30 Mar 2009 13:01:24 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385653</guid>
		<description>Barkley, I only suggested publishing titles/authors/dates.  You say for sports &quot;the contest ... is the activity&quot; but &quot;for academic .. the bottom line is ... the contents of the ... paper&quot; but to me that just seems another way to rephrase the question.  I see both industries as ultimately allowing customers to affiliate with certified impressive folks, so the issue is about different ways to sort/certify impressiveness.
</description>
		<content:encoded><![CDATA[<p>Barkley, I only suggested publishing titles/authors/dates.  You say for sports &#8220;the contest &#8230; is the activity&#8221; but &#8220;for academic .. the bottom line is &#8230; the contents of the &#8230; paper&#8221; but to me that just seems another way to rephrase the question.  I see both industries as ultimately allowing customers to affiliate with certified impressive folks, so the issue is about different ways to sort/certify impressiveness.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Barkley Rosser</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385652</link>
		<dc:creator>Barkley Rosser</dc:creator>
		<pubDate>Mon, 30 Mar 2009 03:10:45 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385652</guid>
		<description>Robin,

Your main examples are sporting events and contests versus publishing academic research.  I do not think the matter really has to do with individual incentives as much as it doew with the structure and nature of the respective activities.  Thus, for sports, the contest between the rival participants is the activity, both when it is amateur and when it is professional, whereas for academic published research the bottom line is publicizing appropriately the contents of the &quot;winner&#039;s&quot; paper, not the contest between it and other papers not published.

This is clearest for the case of professional sports, where the outcomes involve peoples&#039; livelihoods, just as in academia, who gets to publish and where affects the livelihoods of those involved, especially for junior faculty atempting to get tenure in an academic department.  The money in professional sports comes from audiences paying to watch the contest between the contestants.  Certainly there come to be greater financial rewards from those who win more often than those who do not (and are perceived as playing well).  But even those who lose in the games, the players are providing a service for the spectators and thus deserve to be paid, at least somewhat.  If there were no losers, there would be no games.

In academic research, if there were enough pages in journals available, all research could be published, assuming that somebody is keeping an eye out to make sure that totally false garbage does not get through.  Much of what goes on in most fields is that there are page limits for each journal on how much can be published.  Thus, the high rejection rates one sees for more highly ranked economics journals do not mean that those who &quot;lose&quot; are doing anything bad or awful.  Most papers rejected from the journal I edit are not all that bad, indeed I would say only about 5% of submissions are just awful and totally unpublishable anywhere.  Most papers submitted have some sort of original idea and are not completely incompetent.  So, it is a matter of selecting the best papers to publish out of a set of possible ones.  The point for the &quot;spectators,&quot; the readers of a journal, is seeing the papers that make it through the selection process to get published.

There is also another problem not mentioned so far: copyrights.  It might be one thing to print titles of papers and names of authors (and perhaps of editors and referees).  But, the minute one starts talking about actually &quot;publishing&quot; the papers themselves (as some thinking about this think is what should be done) one runs into ruining the chance of a paper ever being properly published because of copyright.  Also, if a paper is rejected, does one &quot;publish&quot; the first draft, the last revision?  Does one publish the desk rejects?  Does one reveal to the public all these details, that this paepr was desk rejected while that one went through four revisions before finally getting the axe?
</description>
		<content:encoded><![CDATA[<p>Robin,</p>
<p>Your main examples are sporting events and contests versus publishing academic research.  I do not think the matter really has to do with individual incentives as much as it doew with the structure and nature of the respective activities.  Thus, for sports, the contest between the rival participants is the activity, both when it is amateur and when it is professional, whereas for academic published research the bottom line is publicizing appropriately the contents of the &#8220;winner&#8217;s&#8221; paper, not the contest between it and other papers not published.</p>
<p>This is clearest for the case of professional sports, where the outcomes involve peoples&#8217; livelihoods, just as in academia, who gets to publish and where affects the livelihoods of those involved, especially for junior faculty atempting to get tenure in an academic department.  The money in professional sports comes from audiences paying to watch the contest between the contestants.  Certainly there come to be greater financial rewards from those who win more often than those who do not (and are perceived as playing well).  But even those who lose in the games, the players are providing a service for the spectators and thus deserve to be paid, at least somewhat.  If there were no losers, there would be no games.</p>
<p>In academic research, if there were enough pages in journals available, all research could be published, assuming that somebody is keeping an eye out to make sure that totally false garbage does not get through.  Much of what goes on in most fields is that there are page limits for each journal on how much can be published.  Thus, the high rejection rates one sees for more highly ranked economics journals do not mean that those who &#8220;lose&#8221; are doing anything bad or awful.  Most papers rejected from the journal I edit are not all that bad, indeed I would say only about 5% of submissions are just awful and totally unpublishable anywhere.  Most papers submitted have some sort of original idea and are not completely incompetent.  So, it is a matter of selecting the best papers to publish out of a set of possible ones.  The point for the &#8220;spectators,&#8221; the readers of a journal, is seeing the papers that make it through the selection process to get published.</p>
<p>There is also another problem not mentioned so far: copyrights.  It might be one thing to print titles of papers and names of authors (and perhaps of editors and referees).  But, the minute one starts talking about actually &#8220;publishing&#8221; the papers themselves (as some thinking about this think is what should be done) one runs into ruining the chance of a paper ever being properly published because of copyright.  Also, if a paper is rejected, does one &#8220;publish&#8221; the first draft, the last revision?  Does one publish the desk rejects?  Does one reveal to the public all these details, that this paepr was desk rejected while that one went through four revisions before finally getting the axe?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Robin Hanson</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385651</link>
		<dc:creator>Robin Hanson</dc:creator>
		<pubDate>Sat, 28 Mar 2009 19:32:14 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385651</guid>
		<description>Barkley, I am not trying to improve the journal system here; I&#039;m trying to understand human nature.  Why do some forums where folks compete for awards hide the identities of losers, while other forums reveal them?  This seems an interesting and potentially insightful data point.  Yes, high refunded-if-accepted submission fees are a similar phenomena.

</description>
		<content:encoded><![CDATA[<p>Barkley, I am not trying to improve the journal system here; I&#8217;m trying to understand human nature.  Why do some forums where folks compete for awards hide the identities of losers, while other forums reveal them?  This seems an interesting and potentially insightful data point.  Yes, high refunded-if-accepted submission fees are a similar phenomena.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Barkley Rosser</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385650</link>
		<dc:creator>Barkley Rosser</dc:creator>
		<pubDate>Fri, 27 Mar 2009 16:15:00 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385650</guid>
		<description>Although no one has commented on my comment, I am going to add a bit more here.  It is fine for the Mason lunch crowd that likes to bet with itself and blog hard until the food falls off the table to view research and acceptances and rejections as a sports contest, and certainly some parts of it can look like that, especially the tournament aspect of Nobel prizes or who gets the top chairs in the top departments and some other things of that sort.  But I do not think journal publishing should resemble that, or at least certainly not as much.

Let me note that there are many people who find it hard even to submit papers to journals at all because they suffer great personal pain from the rejections they receive, which although usually the editors are more or less kindly, sometimes the referee reports can be harsh and insulting and demeaning, not to mention unfair and inaccurate.  So, adding the stress of possibly being publicly reported on as having been rejected simply adds to this problem.

Now, Robin poses the idea of the possible efficiency of &quot;raising the bar.&quot;  This argument has been floating around for a number of other proposals (or realities) as well.  Thus, Ofer Azar has published papers arguing that the steadily increasing average time of first response by journals in economics is potentially efficient on these grounds, raising the bar for authors who misjudge the appropriate level of journal to which they should be submitting.  Of course, this is the opposite of what is done in the natural sciences, where rapid turnarounds and publicatin are strongly emphasized, and most authors do not like such slow turnarounds (just as I suspect most would not like public revelation of their rejections).

Another way this is done is that certain journals have very high submission fees that are then refunded to authors whose papers are accepted, $650 at the Journal of Financial Economics and a high one also at the Journal of Monetary Economics (both dominated by the sorts of models that are now looking very stupid and useless in the wake of the recent crises in the financial markets).  They make this efficiency argument, although it tends to weed out anybody submitting who does not buy into their somewhat distorted view of things (and in the natural sciences one tends to find just the opposite, with there being no submission fees, but then often page fees for authors who are getting published).
</description>
		<content:encoded><![CDATA[<p>Although no one has commented on my comment, I am going to add a bit more here.  It is fine for the Mason lunch crowd that likes to bet with itself and blog hard until the food falls off the table to view research and acceptances and rejections as a sports contest, and certainly some parts of it can look like that, especially the tournament aspect of Nobel prizes or who gets the top chairs in the top departments and some other things of that sort.  But I do not think journal publishing should resemble that, or at least certainly not as much.</p>
<p>Let me note that there are many people who find it hard even to submit papers to journals at all because they suffer great personal pain from the rejections they receive, which although usually the editors are more or less kindly, sometimes the referee reports can be harsh and insulting and demeaning, not to mention unfair and inaccurate.  So, adding the stress of possibly being publicly reported on as having been rejected simply adds to this problem.</p>
<p>Now, Robin poses the idea of the possible efficiency of &#8220;raising the bar.&#8221;  This argument has been floating around for a number of other proposals (or realities) as well.  Thus, Ofer Azar has published papers arguing that the steadily increasing average time of first response by journals in economics is potentially efficient on these grounds, raising the bar for authors who misjudge the appropriate level of journal to which they should be submitting.  Of course, this is the opposite of what is done in the natural sciences, where rapid turnarounds and publicatin are strongly emphasized, and most authors do not like such slow turnarounds (just as I suspect most would not like public revelation of their rejections).</p>
<p>Another way this is done is that certain journals have very high submission fees that are then refunded to authors whose papers are accepted, $650 at the Journal of Financial Economics and a high one also at the Journal of Monetary Economics (both dominated by the sorts of models that are now looking very stupid and useless in the wake of the recent crises in the financial markets).  They make this efficiency argument, although it tends to weed out anybody submitting who does not buy into their somewhat distorted view of things (and in the natural sciences one tends to find just the opposite, with there being no submission fees, but then often page fees for authors who are getting published).</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Barkley Rosser</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385649</link>
		<dc:creator>Barkley Rosser</dc:creator>
		<pubDate>Thu, 26 Mar 2009 21:46:51 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385649</guid>
		<description>As a journal editor, let me throw in a few observations.  One, unmentioned so far, is that, believe it or not, most of us editors actually have sympathy for those who submit their papers to us.  However, in economics, unlike perhaps in astrophysics, the majority of papers get rejected.  At my journal the rejection rate is around 85%.  Furthermore, about half the papers submitted get &quot;desk rejected&quot; by me upfront without being sent out for reviews (refereeing time is indeed a very scarce commodity).

So, we would be talking about huge numbers of papers, and talking about humiliating large numbers of people.  For what purpose?  Research is not a sports contest.

I also note that there is a rather famous history of famous papers that were rejected, some of them by many journals.  There just is not much of a gain for the journals in doing this sort of thing.
</description>
		<content:encoded><![CDATA[<p>As a journal editor, let me throw in a few observations.  One, unmentioned so far, is that, believe it or not, most of us editors actually have sympathy for those who submit their papers to us.  However, in economics, unlike perhaps in astrophysics, the majority of papers get rejected.  At my journal the rejection rate is around 85%.  Furthermore, about half the papers submitted get &#8220;desk rejected&#8221; by me upfront without being sent out for reviews (refereeing time is indeed a very scarce commodity).</p>
<p>So, we would be talking about huge numbers of papers, and talking about humiliating large numbers of people.  For what purpose?  Research is not a sports contest.</p>
<p>I also note that there is a rather famous history of famous papers that were rejected, some of them by many journals.  There just is not much of a gain for the journals in doing this sort of thing.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: dclayh</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385648</link>
		<dc:creator>dclayh</dc:creator>
		<pubDate>Thu, 26 Mar 2009 20:25:42 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385648</guid>
		<description>&lt;i&gt;Most academic papers are rejected by several journals before some journal finally accepts them.&lt;/i&gt;

Is this really true?  It is certainly not the case in my field (astrophysics) where being rejected by a journal is unheard-of unless you are a crackpot operating from some dank basement (in which case no other journal would accept the paper either).
</description>
		<content:encoded><![CDATA[<p><i>Most academic papers are rejected by several journals before some journal finally accepts them.</i></p>
<p>Is this really true?  It is certainly not the case in my field (astrophysics) where being rejected by a journal is unheard-of unless you are a crackpot operating from some dank basement (in which case no other journal would accept the paper either).</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Tim Tyler</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385647</link>
		<dc:creator>Tim Tyler</dc:creator>
		<pubDate>Wed, 25 Mar 2009 13:18:00 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385647</guid>
		<description>Re: A journal could simply make rejected papers available online

Any journal that published &quot;rejected&quot; papers in a corner of its web site would screw up the author&#039;s chances of getting their then &quot;previously-published&quot; papers accepted elsewhere.  Authors would not stand for that - and would avoid that publication like the plague.  So, journals do not do this through simple self-interest.

</description>
		<content:encoded><![CDATA[<p>Re: A journal could simply make rejected papers available online</p>
<p>Any journal that published &#8220;rejected&#8221; papers in a corner of its web site would screw up the author&#8217;s chances of getting their then &#8220;previously-published&#8221; papers accepted elsewhere.  Authors would not stand for that &#8211; and would avoid that publication like the plague.  So, journals do not do this through simple self-interest.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mikeh</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385646</link>
		<dc:creator>Mikeh</dc:creator>
		<pubDate>Wed, 25 Mar 2009 00:08:31 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385646</guid>
		<description>Assuming that no author is absolutely confident in his ability to be accepted, and that public rejection hurts publishing prospects at other papers; it seems that any one journal publishing the rejections would drive away papers of its prestige level (as they would approach a journal a rung lower rather than take the risk of dropping several rungs) on the other hand it would attract submissions of a much lower level hoping for a long shot. It seems that there are stable equilibriums at universal publication or non-publication). Perhaps a journal which is at the top of its pyramid such that it doesn&#039;t face significant competition for its paper submissions (not certain about relative prestige, but perhaps Nature would be at that level) might be able to do it unilaterally, but that type of journal can only lose from publishing rejection.
</description>
		<content:encoded><![CDATA[<p>Assuming that no author is absolutely confident in his ability to be accepted, and that public rejection hurts publishing prospects at other papers; it seems that any one journal publishing the rejections would drive away papers of its prestige level (as they would approach a journal a rung lower rather than take the risk of dropping several rungs) on the other hand it would attract submissions of a much lower level hoping for a long shot. It seems that there are stable equilibriums at universal publication or non-publication). Perhaps a journal which is at the top of its pyramid such that it doesn&#8217;t face significant competition for its paper submissions (not certain about relative prestige, but perhaps Nature would be at that level) might be able to do it unilaterally, but that type of journal can only lose from publishing rejection.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: frelkins</title>
		<link>http://www.overcomingbias.com/2009/03/reveal-rejects.html#comment-385645</link>
		<dc:creator>frelkins</dc:creator>
		<pubDate>Tue, 24 Mar 2009 23:18:32 +0000</pubDate>
		<guid isPermaLink="false">http://prod.ob.trike.com.au/2009/03/reveal-rejects.html#comment-385645</guid>
		<description>As more and more journals become for-profit, do we know that papers are rejected based on quality?  I  mean, a for-profit has tight space constraints. They may like a paper &quot;enough&quot; but just run out of space! Why not just pop everything onto the &#039;net that meets a certain quality, even if it doesn&#039;t have room for the paper version? Not being an academic, I don&#039;t know if this would work. . .
</description>
		<content:encoded><![CDATA[<p>As more and more journals become for-profit, do we know that papers are rejected based on quality?  I  mean, a for-profit has tight space constraints. They may like a paper &#8220;enough&#8221; but just run out of space! Why not just pop everything onto the &#8216;net that meets a certain quality, even if it doesn&#8217;t have room for the paper version? Not being an academic, I don&#8217;t know if this would work. . .</p>
]]></content:encoded>
	</item>
</channel>
</rss>

<!-- Performance optimized by W3 Total Cache. Learn more: http://www.w3-edge.com/wordpress-plugins/

Page Caching using disk (enhanced)
Database Caching using disk
Object Caching 438/455 objects using disk
Content Delivery Network via Amazon Web Services: S3: overcomingbias-assets.s3.amazonaws.com

Served from: www.overcomingbias.com @ 2012-02-11 18:22:39 -->
