Sorting pubmed articles on the impact factor
Pierre Lindenbaum
Wednesday, 18 June 2008 14:03 UTC
Hi all,
We recently had an interesting discussion on friendfeed about how it would be possible to order the articles in pubmed using the impact factor of their journals as a key of sorting
Lars Juhl Jensen and Deepak Singh suggested me to have a look at http://www.eigenfactor.org where the Eigenfactor is a measure of the journal’s total importance to the scientific community. I then wrote a tool sorting the articles in pubmed using this score.
See here for more information.
Pierre
-
Replies
Jump to resultsResults
-
Case in point – I’m about to submit a funding application in which my career achievements/track record account for 50% of the final ranking. I am asked to provide complete citation metrics for all publications, and then given less than 1/2 page to give “other information” on career achievements.
I think this is a fair reflection of how heavily we are judged on the IF of our publications. Surely, in order to make a thourough judgement of career achievements/potential, particularly early in career, referees/committee members would want to know a little more than the IF of the papers I’ve published… awards, patents, invited seminars, peer review activity etc etc
In reality, they don’t look far beyond publication lists.
-
Despite all the flaws and “ridiculousness”, the system of impact factors, eigenfactors and all sorts of metrics has many effects with respect to knowledge production. It all has to do with credibility.
Scientists citing papers judge the credibility of the results presented and the conclusions reached. The journal in which a paper is published is a relevant variable when it comes to judging the credibility of the science reported in it. That credibility is derived from a high impact factor, a grand history of the journal, the eigen factor or familiarity with the editors of, or authors who have published in, that journal and much more.
The Journal of Animal Science or the Slovak Journal of Animal Science are different in many ways. Ignoring that is unwise. That does not have to say that the SJAS publishes nonsense. It conveys less credibility.
Scientists build careers on the credibility of their science, which is derived from the journals or books in which they report it and the sources of the support they get for it (e.g. funding). The more credible science is, the more citations it potentially attracts, the more relevant it becomes.
Whether we like them or not, metrics need to be taken seriously. They shape science like never before.
-
That’s odd. I judge the credibility of a paper by reading the methods section and looking at the results. The journal in which it is published is irrelevant Now everything is on line, it is quite possible (and indeed preferable) to not even notice what journal a paper has appeared in.
-
@David, I take the same approach to judging papers.
Unfortunately, however, those in positions of influence over my career progression (many of whom no doubt got there on the back of so-called “high-impact” papers) don’t see it that way. It’s simple, if I want to have any chance of getting a job that will allow me to continue publishing papers, I have to play the IF game.
-
Although I believe that Impact Factors are widely misused and misunderstood, and I agree with David that a “journal Impact Factor” is insufficiently granular to say anything meaningful about an individual paper in that journal, I also know that some journals are higher-quality than other journals for various reasons. For example, some journals operate a fairer peer-review process than others; or employ highly qualified and insightful editors compared with minimal editorial values in others. Therefore, the journal you publish in certainly means something, particularly if you look at your publication record as a whole and the range of journals in which you’ve published.
-
By the way, I thought I’d log in this thread that the eigenfactor which Pierre mentions in his initial post is now “embedded” in the Thomson-Reuters figures. This year (2009), the company provided eigenfactors as well as impact factors (and some other metrics) when they announced the 2008 figures.
-
How very sad to hear Derren Brown say
“It’s simple, if I want to have any chance of getting a job that will allow me to continue publishing papers, I have to play the IF game.”
If it is actually true (and I sometimes wonder if the perception is worse than the reality) it is a truly appalling reflection on the quality of those people who control his promotion. Perhaps he should think of moving to a place that is run by more sensible people.
-
In listening to the discontent with the current system of scientific promotion which relies on citation statistics, I was reminded on the story behind the creation of the H-Index where Jorge E. Hirsch, a physicist at UC San Diego, was unhappy with the way he was being evaluated and as a result came up with what he thought was a better way. In 2005, he published a paper introducing the H-index and included a data analysis to validate his new metric. In the intervening years, the H-Index has changed the landscape of how many institutions measure the scientific productivity and impact of their scientific communities. The irony of Hirsch’s work is that, according to Google Scholar, it has become his most cited paper (i.e., 650 citations) which in the end adds significantly to his scientific achievement using whatever measure of impact you chose. Perhaps, as a result of this thread, a new H-Index-like paper will be written thereby killing two birds with one stone – helping to change a flawed system and at the same time helping the authors in question to move along their career trajectory.
-
There are flaws in the H index as there are in any of the metrics currently being used. As David says, the only way really to evaluate someone’s output is to read their papers. But many organisations seem to not have time for that, and to be looking for shortcuts.
-
Hmm…
David, as a rule perception is adequate to reality (on the average).
But don’t it seem to you, that your own reality is not better than Darren’s reality? If you look at your reality more attentively you will notice that you are forced to play in IF game too. And what to do? Will you seek better people for you too? And will you find them? Doubtfully… It is such age now – you wrote the article about it yourself! Probably it is necessary simply to change your behaviour and attitude to this problem. Try to intrude your own game upon these people and upon their epoch…
Results
-
