Furore over journal impact factors
William Burns
Tuesday, 03 February 2009 02:16 UTC
For any of you who don’t live in the small world of history and philosophy of science, you might have missed this furore about impact factors. I think it raises a number of questions about our own reading habits in science. And because reading is the yang to writing’s yin, it raises questions about our writing habits too.
The background to this story is that the European Science Foundation want to develop impact factor measurements for the quality of various history and philosophy of science journals.
This has not gone down well. In fact, the editors of all the major journals in the field have said a resounding ‘no’ to the whole idea.
One argument against the ESF’s impact factor scheme is that such calculations don’t take account of excellent work in minor publications – or mediocre work that gets into the more widely read journals. Another argument being made is that the calculations are biased towards English-language journals (which get more widely cited not neccesarily because their content is the best but because more people have the language skills to read them).
In essence, the argument is that impact factors encourage us to assess arguments not by their quality but by what journal they’re in. We stop reading articles and instead put our blind faith in the metrics.
Well, we seem to have gone a long way along this road in science. How many fellowship/grant/recruitment committees read all the publications listed on CVs – or at least the most recent ones? Even if they read them, do ‘the powers that be’ really engage with the arguments and data in these papers, and judge candidates based on them? I doubt it, although you may tell me otherwise. We simply look at the impact factor and make a judgement based on that.
Isn’t this a bad thing? Have we got to a stage where the writing is important only in as much as it gets the paper past the foibles of a few referees (and therefore into a top journal)? Does it matter after that if no one reads it any more? Does it matter if the paper’s central argument is even true?
-
Replies
This topic has been locked by the forum moderators.
Jump to resultsResults
-
I think the problem is ultimately one of trying to define quality in a rigorous way. It’s a bit like pornography – we know it when we see it. If we can define quality, then I think we’ll be able to estimate it. At the moment we’re devising measures in one or another odd way, and using them to define quality. It’s backwards.
-
Bob,
I think you try to create the difficulties there where they are absent. Pornography (I am forced to use comparison, which you proposed) can be defined/determined very simply. Unlike normal sexual relations, it contains no love and respect to human, pursues dirty aims and in addition to that – is obtrusively demonstrative. Analogically, those non-best investigations which I mentioned, don’t contain real Science despite of their publication in high-impact-factored journals. What does this mean – “don’t contain real Science”?
It means that they contains no love and respect to human reason, pursues dirty aims and in addition to that – are obtrusively demonstrative.
I can explain more particularly. For example, such investigation is not independent and merely imitates results, approaches and achievement of other, more talented, scientists. Certainly I don’t mean experiments for testing the repeatability of other’s results – such experiments are normal practice of science. I mean just aforethought imitation of results and/or approaches (sometimes even accompanied by sham of data). Dirty aims in case of scientific research are any aims, except scientific cognition of world, i.e. discovery of new truths about the world (e.g. the writing of high-impact-factored papers for the nice record in university’s bureaucratic bodies is dirty aim).
And it is sadly that sometimes even high-quality journals take part in such “scientific pornography”.
I think that David Colquhoun, who raises here difficult questions, and Maxine Clarke, who is an editor in Nature, can understand me best of all.
Results
-
