There are [still] good reasons to be suspicious of metric-based research assessment, concludes the first editorial of the year in Nature (457, 7-8; 2009; free to access online) in the wake of the sixth and final UK Research Assessment Exercise (RAE). The RAE depended heavily on the peer-reviewed literature; unsurprisingly, attention is now being focused on what system will replace it (it already has a title, the Research Excellence Framework, or REF).
Publication statistics taken alone are a poor measure of research quality, according to the editorial and many others in the scientific community. “An example from this journal illustrates the point. Our third most highly cited paper in 2007, with 272 citations at the time of inspection, was of a pilot study in screening for functional elements of the human genome. The importance lay primarily in the technique. In contrast, a paper from the same year revealing key biological insights into the workings of a proton pump, which moves protons across cell membranes, had received 10 citations.”
The editorial concludes: “The signs are that, after several false starts and delays, the final proposals for the REF, due in autumn 2009, are unlikely to be the radical departure from the RAE that the government first envisaged in 2006. Expert review is far from a problem-free method of assessment, but policy-makers have no option but to recognize its indispensable and central role.”
Would most scientists agree with that view? Let us know what you think – should scientific output of institutions and individuals be assessed, and if so, how best to do it?