The quality of scientific research is maintained by the peer review process. Or at least that’s the idea of peer review. But that process is expensive, time consuming and anything but fail-safe. It puts a considerable strain on academics’ time. Some that I know, claim to get multiple requests to review papers every day—though I’m not sure I believe them. In any case, it takes time to review papers properly and you are expected to do it as part of your service to the scientific community. In other words, you do it for free. There are serious doubts about the utility of this system of self policing both in the natural and social sciences. There are plenty of cases of good or even great papers that get rejected and mediocre papers that get published in elite journals. It’s a capricious process.
Now Nature, as part of an ongoing debate on peer review, has launched an interesting experiment. Authors can choose to participate in a trial online peer review where they post papers to a website and academics in the field can provide comments and review the manuscripts. Submitted papers will still go through the traditional review process, but the editors will be able to take both sets of comments into account when making publication decisions.
This seems like a pretty good idea to me. It happens all the time that papers get rejected/published because editors don’t have enough information, or rely too heavily on the judgement of a bad reviewer or ignore that of a good reviewer. Hopefully the Nature trial will work in such a way that bad reviews will stand out from the crowd as just that.