For our academic "knights in shining armor," do we care more that their suits shine, than that they are armor? From a recent Science: [Journal peer] reviewers make two common mistakes. The first mistake is to reflexively demand that more be done. Do not require experiments beyond the scope of the paper, unless the scope is too narrow. Avoid demanding that further work apply new techniques and approaches, unless the approaches and techniques used are insufficient to support the conclusions. …The second mistake … Do not reject a manuscript simply because its ideas are not original, if it offers the first strong evidence for an old but important idea. Do not reject a paper with a brilliant new idea simply because the evidence was not as comprehensive as could be imagined. Do not reject a paper simply because it is not of the highest significance, if it is beautifully executed and offers fresh ideas with strong evidence.
Hmmm. I guess there is an issue here as to what constitutes "polish," especially fundamentally empty polish. You seem to include being well written as polish. I would say that being well written is a positive serious attribute of an article. Indeed, it goes against the other part of your "polish," bringing in obscure and overly high-powered math that may nevertheless be faddish or impressive. So, one kind of polish makes a paper more readable and the other kind makes it less so. I certainly would agree that there is a lot of puffing and use of faddish math, including econometric techniques. However, it remains the case, that more elegant presentations (hence more comprehensible and readable) of a particular kind of math are more likely to get published, and deservedly so, even if the particular kind of math itself is inappropriately obscure and unnecessary.
Nick, high polish signals a high ability to produce polish, and if the abilities to produce polish and content are correlated, that indicates a higher ability to produce content. But high polish would only then signal high content if authors actually used that high ability to produce high content.
Psycho, it is clearly easier to evaluate polish than content, but it is not clear that polish is easier to produce.
Barkley, it would be nice to believe some journals focus on content over polish and flourish.
Silas, yes, polish can be used to make content less readable.
First off, RH does not take note of the fact that increasing polish seems marginally much, much cheaper than increasing content. So long as increasing content is unduly expensive, people will focus on polish.
For the rest of this I'm making some assumptions about the selection process for articles that I believe are justified. If they are not, my point is likely invalid.
Basically, when people judge nearly anything, they rarely take the time or effort to reserve judgement about the entire piece until they have completed it. They make judgements early on and then marginally adjust them they read more.
If I'm deciding whether to put something in a journal, and I believe the first 10% of it is either boring (relatively speaking) or hard to understand (again, relatively), then my evaluation of the remaining 90% will probably be similarly colored. Besides, given that readers of any publication have limited time, many will elect to not read an article that starts out boring or incomprehensible unless they have a highly specific need to read that article.
It's also easier to spot an article with good polish than with good content.
Polish may serve to signal quality. When it is hard to evaluate content directly, readers may rely on polish as a proxy. If somebody can explain their ideas clearly, shows knowledge of the literature through appropriate referencing, and puts in enough time and care to avoid minor errors, typos, etc., the chances increase that it is worth paying attention to what they say. It's a hard-to-fake indicator that says "I can think clearly, I have extensive knowledge of the area, I'm aware of the reigning schools and fashions, and I deem this idea important enough that I've been willing to spend one month polishing the paper".
If polish serves this (potentially very useful) function, it's not so obvious that it would be better if academics invested less in polish (although of course that might still be the case on the margin.)
Also, in some areas there might be great benefits to approximating perfection more closely. For example, given that there are already so many books on Shakespeare, maybe the world is better served by getting one additional great book on Shakespeare than by getting another fifty (or five hundred) fairly good books on Shakespeare.
I don't think "care" is the right word here; I guess it would be more accurate to say that even scientific referees are often easily impressed by "polish". Which is sad enough.
Not so long ago, I read an article that reported a simple perceptual deterrence study* of which there are dozens. The only unusual thing about it was that the authors started out by discussing Hobbes and Rousseau. It was in the ASR, one of sociology's top two journals.
*Basically, you ask people how likely they think it is that they'll get punished if they do X and later look whether that influenced their behaviour.
I mostly agree with the arguments in the quote you provide, especially the first part. Regarding the second part, well, these are matters of judgment. There are limits on publishing space in journals, which means at least in economics that a majority of papers submitted are rejected. Most papers are not really terrible and can be argued to maybe fit some of those pieces of the second part up to some point. But, especially if a journal is emphasizing novelty of ideas, it is not unreasonable to reject those that are merely "flourishes," unless they are mighty spectacular ones.
Benoit,
I am not going to claim that intellectual "markets" are efficient or perfect or anything of the sort. They are clearly subject to all kinds of ridiculous fads and bubbles and biases and whatnot. However, I would note that at the higher levels it is not publications that matter, and certainly not just the sheer number of publications. Where one publishes certainly matters, but ultimately more important than pubs is citations. Now, of course, lots of games get played with that, including the scam of referees demanding that authors cite their glorious work. As an editor, I find such stuff often nauseating and very unprofessional, although sometimes it is justified, as authors often fail to cite appropriate material.
Silas,
I can assure you that editors are not impressed in general by efforts at greater obscurantism. There is indeed evidence that well written papers are more likely to be accepted than badly written ones. And in math, elegant proofs are greatly admired. This does not mean, of course, that there is not lots of overly obscure and unnecessarily incomprehensible stuff getting published.
Robin_Hanson: I thought academics tended to write in a way that makes their work *less* accessible. That has been my consistent experience. Anything well-written enough to look easy, almost by definition looks insigificant. ("That's it?") I took a class outside of my discipline in grad school (engineering), and in that class (math) the professor explained his experience of reading proofs in journals where the authors do their best to make their proofs hard to follow, and the same philosophy prevailing ("it you make it look simple, you make it look insigificant").
So, while they might add polish, they don't add it in that manner.
Scientific publishing has become a game where the goal is for academics to gather publication capital which they can then put on their cv and get money in the form of jobs and tenure. The more publication they get the better. It doesn't matter if their research has no practical value and usually also very little intellectual value. As long as they can convince reviewers through ,as you say, "polish and flourish" and often borderline fraudulent statistical manipulations and misinterpretations, they get points on their cv. Plus, there's a viscous circle where those who play the game then become those who rate others and therefore are able to keep real scientists out of science positions.I think this guy makes a very valid point: http://www.lambdassociates....
Of course there are exceptions, I've seen good research come out of some groups but my impression is that the good scientists are being more and more smothered out of the academic community. If you read literature written by historically significant scientists you will see that most of the great scientists would not be able to get a position in todays universities because they would be considered too marginal and I'm pretty sure none would stoop so low as to play the publication game for their job.
People always complain that we can't trust what we read in the media about science anymore because the media distorts the facts. Although the media should share a part of the blame for the misinformation, I think the "scientific community" should get most of it. Once a professor has published dozens of papers exaggerating the significance of his work, he's not going to turn around and tell how very little his results have to do with the real world when the media comes at his door. Also some professors will say _anything_ to be recognized by the media.
If one wants to get a good picture of the poor state of academia one only has to read the Piled Higher and Deeper comic series(http://www.phdcomics.com/co....
Robin,
Hmmm. I guess there is an issue here as to what constitutes "polish," especially fundamentally empty polish. You seem to include being well written as polish. I would say that being well written is a positive serious attribute of an article. Indeed, it goes against the other part of your "polish," bringing in obscure and overly high-powered math that may nevertheless be faddish or impressive. So, one kind of polish makes a paper more readable and the other kind makes it less so. I certainly would agree that there is a lot of puffing and use of faddish math, including econometric techniques. However, it remains the case, that more elegant presentations (hence more comprehensible and readable) of a particular kind of math are more likely to get published, and deservedly so, even if the particular kind of math itself is inappropriately obscure and unnecessary.
Nick, high polish signals a high ability to produce polish, and if the abilities to produce polish and content are correlated, that indicates a higher ability to produce content. But high polish would only then signal high content if authors actually used that high ability to produce high content.
Psycho, it is clearly easier to evaluate polish than content, but it is not clear that polish is easier to produce.
Barkley, it would be nice to believe some journals focus on content over polish and flourish.
Silas, yes, polish can be used to make content less readable.
First off, RH does not take note of the fact that increasing polish seems marginally much, much cheaper than increasing content. So long as increasing content is unduly expensive, people will focus on polish.
For the rest of this I'm making some assumptions about the selection process for articles that I believe are justified. If they are not, my point is likely invalid.
Basically, when people judge nearly anything, they rarely take the time or effort to reserve judgement about the entire piece until they have completed it. They make judgements early on and then marginally adjust them they read more.
If I'm deciding whether to put something in a journal, and I believe the first 10% of it is either boring (relatively speaking) or hard to understand (again, relatively), then my evaluation of the remaining 90% will probably be similarly colored. Besides, given that readers of any publication have limited time, many will elect to not read an article that starts out boring or incomprehensible unless they have a highly specific need to read that article.
It's also easier to spot an article with good polish than with good content.
Polish may serve to signal quality. When it is hard to evaluate content directly, readers may rely on polish as a proxy. If somebody can explain their ideas clearly, shows knowledge of the literature through appropriate referencing, and puts in enough time and care to avoid minor errors, typos, etc., the chances increase that it is worth paying attention to what they say. It's a hard-to-fake indicator that says "I can think clearly, I have extensive knowledge of the area, I'm aware of the reigning schools and fashions, and I deem this idea important enough that I've been willing to spend one month polishing the paper".
If polish serves this (potentially very useful) function, it's not so obvious that it would be better if academics invested less in polish (although of course that might still be the case on the margin.)
Also, in some areas there might be great benefits to approximating perfection more closely. For example, given that there are already so many books on Shakespeare, maybe the world is better served by getting one additional great book on Shakespeare than by getting another fifty (or five hundred) fairly good books on Shakespeare.
I don't think "care" is the right word here; I guess it would be more accurate to say that even scientific referees are often easily impressed by "polish". Which is sad enough.
Not so long ago, I read an article that reported a simple perceptual deterrence study* of which there are dozens. The only unusual thing about it was that the authors started out by discussing Hobbes and Rousseau. It was in the ASR, one of sociology's top two journals.
*Basically, you ask people how likely they think it is that they'll get punished if they do X and later look whether that influenced their behaviour.
Robin,
I mostly agree with the arguments in the quote you provide, especially the first part. Regarding the second part, well, these are matters of judgment. There are limits on publishing space in journals, which means at least in economics that a majority of papers submitted are rejected. Most papers are not really terrible and can be argued to maybe fit some of those pieces of the second part up to some point. But, especially if a journal is emphasizing novelty of ideas, it is not unreasonable to reject those that are merely "flourishes," unless they are mighty spectacular ones.
Benoit,
I am not going to claim that intellectual "markets" are efficient or perfect or anything of the sort. They are clearly subject to all kinds of ridiculous fads and bubbles and biases and whatnot. However, I would note that at the higher levels it is not publications that matter, and certainly not just the sheer number of publications. Where one publishes certainly matters, but ultimately more important than pubs is citations. Now, of course, lots of games get played with that, including the scam of referees demanding that authors cite their glorious work. As an editor, I find such stuff often nauseating and very unprofessional, although sometimes it is justified, as authors often fail to cite appropriate material.
Silas,
I can assure you that editors are not impressed in general by efforts at greater obscurantism. There is indeed evidence that well written papers are more likely to be accepted than badly written ones. And in math, elegant proofs are greatly admired. This does not mean, of course, that there is not lots of overly obscure and unnecessarily incomprehensible stuff getting published.
Ha, the title of this post made me do a double take. I think maybe polish/Polish is the only English word that changes its meaning when capitalized.
@ Benoit Essiambre
I think you mean vicious, not viscous.
Robin_Hanson: I thought academics tended to write in a way that makes their work *less* accessible. That has been my consistent experience. Anything well-written enough to look easy, almost by definition looks insigificant. ("That's it?") I took a class outside of my discipline in grad school (engineering), and in that class (math) the professor explained his experience of reading proofs in journals where the authors do their best to make their proofs hard to follow, and the same philosophy prevailing ("it you make it look simple, you make it look insigificant").
So, while they might add polish, they don't add it in that manner.
Scientific publishing has become a game where the goal is for academics to gather publication capital which they can then put on their cv and get money in the form of jobs and tenure. The more publication they get the better. It doesn't matter if their research has no practical value and usually also very little intellectual value. As long as they can convince reviewers through ,as you say, "polish and flourish" and often borderline fraudulent statistical manipulations and misinterpretations, they get points on their cv. Plus, there's a viscous circle where those who play the game then become those who rate others and therefore are able to keep real scientists out of science positions.I think this guy makes a very valid point: http://www.lambdassociates....
Of course there are exceptions, I've seen good research come out of some groups but my impression is that the good scientists are being more and more smothered out of the academic community. If you read literature written by historically significant scientists you will see that most of the great scientists would not be able to get a position in todays universities because they would be considered too marginal and I'm pretty sure none would stoop so low as to play the publication game for their job.
People always complain that we can't trust what we read in the media about science anymore because the media distorts the facts. Although the media should share a part of the blame for the misinformation, I think the "scientific community" should get most of it. Once a professor has published dozens of papers exaggerating the significance of his work, he's not going to turn around and tell how very little his results have to do with the real world when the media comes at his door. Also some professors will say _anything_ to be recognized by the media.
If one wants to get a good picture of the poor state of academia one only has to read the Piled Higher and Deeper comic series(http://www.phdcomics.com/co....