2006/06/14

 

Nature Open Peer Review Watch

In the kitchens, lunchrooms, and halls of academic departments around the world, even as I write this, postdocs, grad students, and junior faculty are daring each other to be the first to post a critical comment on Nature's new open peer review trial.

In the event you haven't heard -- and if you've managed to dredge down this deep in the Mariana Trench of the blogosphere, you almost certainly have -- Nature, about a week and a half ago, started a three-month experiment in which the journal would for certain papers, in parallel with its normal, anonymous peer review process, put the papers up on a preprint server for very public comments. Basically, it's an instance of the blogging software Movable Type, with comments enabled; commenters would be required to include their name and institution with the comment, and supposedly the comments would be considered along side the reviews received through normal channels in assessing the paper. It's not a completely open forum; Nature's editors will assess the comments received and determine which ones will be posted -- assuming they're not too busy screening out comment spam with titles like "Hot new pix!" and "Cheaper v1agra."

The first two papers were posted on the preprint server last week, both of them in immunology; one of them has since rather mysteriously disappeared -- perhaps the paper was rejected. Today, there are four manuscripts posted -- one of the original two in immunology, plus one in astrophysics, one in structural biology, and one in paleoclimatology. None of them have attracted any comments yet. (There's also an RSS feed if you're itching to keep track of when a new paper is posted; unfortunately, though, it doesn't appear to notify you of new comments to existing papers.)

Don't get me wrong; I think this is a terrific, very clever idea -- blogging the peer review process. But I must admit that I've found myself wondering (1) who would post their paper in such a forum, and (2) who would comment.
On point (1), the FAQ on this experiment includes the following revealing item (answered with admirable forthrightness):
Whether or not the preprint is eventually published in Nature, isn't there a chance of the results being scooped by competitors?

Yes.

Even for those who profess to want a more open peer review process from journals practicing anonymous review, I wonder if this process might seem a bit too open.

And with respect to item (2), who is actually going to respond? If I were a young scientist in career-building mode (arguably the most likely person to be interested in a feature like this), I'd be very reluctant to put my name on anything that might come back to haunt me later -- particularly if the author of the manuscript under review were, say, a dyspeptic Nobel laureate with a known vindictive streak (I'm sure there's one or two of them out there). That's an extreme case, of course, but the principle remains. That could just serve to make criticism in these open comments more polite and fruitful than it sometimes is in anonymous review. But it could also select for a preponderance of positive comments and against negative or critical ones.

So it will be very interesting to see what happens when that initial comment is posted -- and who it is that breaks the silence. It's an interesting experiment, and I suspect we'll all learn something from it.

Technorati Tags : , , ,


# posted by ahab @ 8:02 AM
Comments: Post a Comment



<< Home

This page is powered by Blogger. Isn't yours?