Martin Weller considers academic credit for blogs: The answer to … whether new approaches such as blogging constitute scholarly activity, is an emphatic yes. Which leads us to a more problematic question: How should we recognize it? …
The solution is already in place and widely used. You use it every time you blog about someone else's blog post. You are actually reviewing it openly and allow further interaction and improvement. Your blogging platform gives you identity (important so no one can forge a review) and also the technical means to track references (trackbacks and such). The only thing to do is to have a system that navigates the web of reports, comments, links and update a score every time an article is identified on someone's blog that is known as a trusted reviewer. Efforts are underway to completely automate this process.
How about the same way that Credit Ratings Agencies get paid. How do S&P and Moody's get paid? Who pays to have investments rated?
We have seen how well that system works. If it is good enough to rate trillions of dollars of assets, surely it must be good enough to rate science papers.
Even more to the point, citation indices do not discriminate between positive and negative citation - that is, if you can write an absolutely terrible article and get it published, you could have hundreds of citations to it as everyone points it out as an example of what not to do - but that is still a lot of citations!
Robin, you might be interested to know that there is an explicit mechanism for academic recognition of blogging, at least in the hard sciences. All of us whose research is funded by the National Science Foundation have to include a discussion of "broader impact" in our grant proposals. And being able to show that you have a widely read blog on which science takes place counts for this, and helps you get funded.
I guess an additional idea is just paying for evaluation of your blog. I think I've heard of journal submission fees in the hundreds of dollars. You're essentially paying for respected evaluation of the quality of your paper.
What if you could pay a thousand, or a few thousand, for an evaluation of the current quality of your blog, A through E, and no ranking, from the AEA, or just AEA certified or not AEA certified, or another level AEA certified top blog, say? For those with good blogs it might be worth paying even $5,000 or more, and if the blog is good enough it might be worth it for the department to pay.
Again, there's probably something good (or especially bad) they're doing if they're being cited by quality people, and it's not due to being prestigious or famous already.
But yeah, there's some vagueness, unless you evaluate the content of their blog directly and not with just simple "objective" criteria. But this happens to a large extent anyway in separating an average AER pub from a very important or seminal one, the pub is read and evaluated directly, and not judged just on simple "objective" criteria for its quality.
Or they could just be good, in a given post, at understanding and applying to the current problems and arguments ideas that aren't original. But again, would you nonetheless want to reward that in academia?
Of course, it could just be that the blogger's a good writer, a good teacher, a good communicator, of underlying ideas that aren't original, and that's why they're getting all of the cites in the quality blogosphere. But would you nonetheless want to reward that in academia?
I'm not sure the level of originality you have in mind (unless it's 100% plagiarized, it's original to some extent).
I thought of this because I have little prestige; I never finished my dissertation. So when Brad DeLong or Rajiv Sethi blog about a post or comment of mine, it's not because I'm famous; it's only because they think it's good in at least some important way (or especially bad, but that's easily determined).
In the case where someone is already prestigious, or famous, then all the citing in the quality blogosphere could just be because someone prestigious or famous said it, but in that case academic evaluation is not as important.
If someone doesn't have tenure, is not famous, and especially if they're not at a top school, but is still getting lots of positive citing from top blogs, that probably says something good that's not that hard to see and quantify. And for such a person, having that evaluated in academic reward is important.
The peer review comes very directly from top bloggers like Paul Krugman, Simon Wren-Lewis, yourself, etc. deciding to quote your post, or post on it, or link to it.
In academia currently certain journals have come to be considered more prestigious than others, and pubs in those journals carry all the weight for reward and punishment.
In academia, certain blogs are already considered more prestigious than others, so quality/contribution could be rated based on posts on your posts in top blogs, as well as quotes from your posts, and links to them.
Some respected academics could even rank blogs in top journal articles, say A through E, and you could calculate statistics based on that.
Vaniver, merely knowing that someone was a professor someone in something would not be remotely enough topic-specific expertise.
How specific are citation indices? I thought it was the total number of times you had been cited in any journal, not just specific journals. I agree that it takes more effort to publish a paper that cites an unrelated paper than it is to Like a post I have no expertise about, but this is again something you can track and discourage.
In my information science masters program, we had an assignment that tackled this very topic. Basically, we compared the results of google scholar's ranking algorithm to the impact factors (or other available bibliometric data) from established, credential laden, sources (web of science, etc.) It still requires a good deal of manual effort (eg google reported the same article, published in different journals, as separate), but the basic method of using web analytics on academic content works rather well.
Semantic web technology may one day be able to do all of the work, but in the interim, I agree with the comments: I like a part computational/part human review, web based system.
The solution is already in place and widely used. You use it every time you blog about someone else's blog post. You are actually reviewing it openly and allow further interaction and improvement. Your blogging platform gives you identity (important so no one can forge a review) and also the technical means to track references (trackbacks and such). The only thing to do is to have a system that navigates the web of reports, comments, links and update a score every time an article is identified on someone's blog that is known as a trusted reviewer. Efforts are underway to completely automate this process.
How about the same way that Credit Ratings Agencies get paid. How do S&P and Moody's get paid? Who pays to have investments rated?
We have seen how well that system works. If it is good enough to rate trillions of dollars of assets, surely it must be good enough to rate science papers.
Even more to the point, citation indices do not discriminate between positive and negative citation - that is, if you can write an absolutely terrible article and get it published, you could have hundreds of citations to it as everyone points it out as an example of what not to do - but that is still a lot of citations!
Robin, you might be interested to know that there is an explicit mechanism for academic recognition of blogging, at least in the hard sciences. All of us whose research is funded by the National Science Foundation have to include a discussion of "broader impact" in our grant proposals. And being able to show that you have a widely read blog on which science takes place counts for this, and helps you get funded.
I guess an additional idea is just paying for evaluation of your blog. I think I've heard of journal submission fees in the hundreds of dollars. You're essentially paying for respected evaluation of the quality of your paper.
What if you could pay a thousand, or a few thousand, for an evaluation of the current quality of your blog, A through E, and no ranking, from the AEA, or just AEA certified or not AEA certified, or another level AEA certified top blog, say? For those with good blogs it might be worth paying even $5,000 or more, and if the blog is good enough it might be worth it for the department to pay.
Again, there's probably something good (or especially bad) they're doing if they're being cited by quality people, and it's not due to being prestigious or famous already.
But yeah, there's some vagueness, unless you evaluate the content of their blog directly and not with just simple "objective" criteria. But this happens to a large extent anyway in separating an average AER pub from a very important or seminal one, the pub is read and evaluated directly, and not judged just on simple "objective" criteria for its quality.
Or they could just be good, in a given post, at understanding and applying to the current problems and arguments ideas that aren't original. But again, would you nonetheless want to reward that in academia?
Of course, it could just be that the blogger's a good writer, a good teacher, a good communicator, of underlying ideas that aren't original, and that's why they're getting all of the cites in the quality blogosphere. But would you nonetheless want to reward that in academia?
I'm not sure the level of originality you have in mind (unless it's 100% plagiarized, it's original to some extent).
I thought of this because I have little prestige; I never finished my dissertation. So when Brad DeLong or Rajiv Sethi blog about a post or comment of mine, it's not because I'm famous; it's only because they think it's good in at least some important way (or especially bad, but that's easily determined).
In the case where someone is already prestigious, or famous, then all the citing in the quality blogosphere could just be because someone prestigious or famous said it, but in that case academic evaluation is not as important.
If someone doesn't have tenure, is not famous, and especially if they're not at a top school, but is still getting lots of positive citing from top blogs, that probably says something good that's not that hard to see and quantify. And for such a person, having that evaluated in academic reward is important.
That is attention, but not evaluation for an original intellectual contribution.
The Romans had a useful tradition, to keep pride tamed, during victory parades.
One must ALWAYS be careful not to over-credit someone simply because of longevity, seniority or popularity: that, is called American Politics.
We see, how that functions.
The peer review comes very directly from top bloggers like Paul Krugman, Simon Wren-Lewis, yourself, etc. deciding to quote your post, or post on it, or link to it.
In academia currently certain journals have come to be considered more prestigious than others, and pubs in those journals carry all the weight for reward and punishment.
In academia, certain blogs are already considered more prestigious than others, so quality/contribution could be rated based on posts on your posts in top blogs, as well as quotes from your posts, and links to them.
Some respected academics could even rank blogs in top journal articles, say A through E, and you could calculate statistics based on that.
Vaniver, merely knowing that someone was a professor someone in something would not be remotely enough topic-specific expertise.
How specific are citation indices? I thought it was the total number of times you had been cited in any journal, not just specific journals. I agree that it takes more effort to publish a paper that cites an unrelated paper than it is to Like a post I have no expertise about, but this is again something you can track and discourage.
Inevitably our views of this are coloured by our views of existing academia and its social value.
I do find it interesting that RH leaves out an alternative; that academic status becomes less important.
In my information science masters program, we had an assignment that tackled this very topic. Basically, we compared the results of google scholar's ranking algorithm to the impact factors (or other available bibliometric data) from established, credential laden, sources (web of science, etc.) It still requires a good deal of manual effort (eg google reported the same article, published in different journals, as separate), but the basic method of using web analytics on academic content works rather well.
Semantic web technology may one day be able to do all of the work, but in the interim, I agree with the comments: I like a part computational/part human review, web based system.