Seminar on the evolution of impact metrics, June 2009
Maxine Clarke
Monday, 29 September 2008 13:09 UTC
Received via e-mail today, from ALPSP (Association of Learned and Professional Society Publishers):
One-day seminar: MEASURING UP – THE EVOLUTION OF IMPACT METRICS – programme available shortly
Chair: Ian Craig, Wiley-Blackwell
Monday 15 June, London (full day) 2009
This seminar will explore the evolution of Impact Metrics, examining strengths and weaknesses and the implications in a society increasingly focussed on measuring research quality. Moving forward from the Impact Factor, it will cover usage (download) measurements from the MESUR project; author level indices such as the Hirsch index and derivatives; and eigenvector (PageRank) style metrics such as the EigenFactor and the SCImago Journal Rank Indicator.
I will post further details of the programme when I receive them. I shall probably go to this seminar, and if so I hope perhaps to see other members of this group there.
Updated 29 September 2008 13:11 UTC
-
Replies
Jump to resultsResults
-
That does look like an interesting seminar!
-
Maxine, that looks interesting – I’ll try to be there.
-
I think it looks thoroughly mindless. We now have a breed of people who, instead of doing science, spend their time dreaming up more useless “metrics”. Of course there is a strong commercial interest in doing this -publishers see money in it.
The people who do this sort of thing are free to waste their time if that is what they want to do. The problem for the rest of us is that their activity is not neutral. It does active harm to the progress of good and innovative science.
It may not be too far-fetched to see an analogy between the emphasis on short-term gain produced by a metrics culture and the emphasis on short-term gain that is, at the moment, bringing the financial system to its knees. Both activities encourage the spiv mentality.
-
Well I hope you will go, David, they should invite you to speak!
-
Agree with Maxine, David, you should be there to represent the Metric-o-phobic.
I’m interested in what people who run journals think about the indices. It seems to me that there is a kind of Doublethink where there is a public line:
“IFs and similar determine which journal people send their paper too, so we are stuck with them and we need ours to improve, and be better than the Poppleton Journal of XYZology’s IF”
And simultaneously the private after-three-beers view:
“Metrics are a complete and utter load of cr*p”
Some editors even put the latter view in print. The incoming editor of the Journal of Neurophysiology_, David Linden, recently penned an editorial on his plans for the journal, which was fairly forcefully scornful of impact factors. On a blog recentlylaw.php#comment-1019932 he was even more trenchant, describing the different publication metrics as “various flavors of bullshit".
Personally I think it is another of those ubiquitous idiot statistics / targets things, like the RAE, the NHS Trust star systems, the A&E 4 hr wait stats or the newspapers’ University League tables. Everyone involved in the activities being rated readily admits that it is all a bunch of nonsense, but once the statistic exists, and is a target for something, the gaming and competition nature of modern life means that no-one can afford to be seen to be losing.
-
Quite seriously, I think one (of several) issues here is that journals end up “talking to themselves” on these issues. It would be a breath of fresh air if David or Austin or other articulate scientist would sign up (funded by their institution, naturally) so that these views can be heard. It’s important.
-
Hmm…
Well, I have to admit to a conflict of interest as I once calculated my own H-index. It was suitably depressing.
David Colquhoun was chiding me for doing it when we were talking about this over on Brian Derby’s blog a little while ago.
Actually, Wiley-Blackwell publish the Phys Soc’s journals, so I’m sure I could go along in my capacity as the man from Physiology News and heckle from the back. I suspect David might regard the meeting as a bit of a waste of time, but as a far lowlier scientist than him I am readier to be distracted, especially in my amateur journalistic capacity.
-
Impact and h-factors are certainly a favourite topic for conversation. I had just such a conversation this morning over breakfast at the conference I am at (Brazil-MRS). The eminent Prof from ETH Zurich was lamenting the Hirsch index and generally bemoaning metrics. However, he did mention ETH’s position in international league tables during his plenary!
I suspect we all secretly love metrics that are in our favour and bemoan them when they are not. In terms of David Colquhoun’s comments, I sympathise but I suspect that those academics that compile impact statistics in their “spare” time would probably be doing something like cricket scores instead and I don’t believe it necessarily reduces their productive time.
Publishers, of course, see impact factors as a marketing tool (or am I wrong). It doesn’t matter if they are flawed, if they are believed they must be utilised to promote the product. Before impact factors editors used other metrics such as papers published/submitted or subscription numbers to gauge the virility of a publication.
I am sure the meeting will be interesting but it is a publishing forum and I cannot justify the time no matter how interesting.
-
Well, being quite new to this topic, I can say that I still have a lot to catch up on, hence my interest in the meeting. And yes, I would love to hear David’s standpoint, I wouldn’t expect this to be an all-pro-metrics meeting – will it be?
Regardless, Austin and Brian – of course you are probably right about publisher’s attitude on metrics being somewhat of a ‘necessary evil’, and of course we’d be dumb not to use metrics that are in our favour for advertising our journals.
Having said that, if I was on the science side, I would not think twice about it. I have to add that my background is in oceanography, where the IF of a journal still doesn’t seem to play such a big role in where one submits a manuscript. Unless you can get one accepted by Nature…
-
ALPSP don’t seem to be giving any details about the meeting yet, but it sounds as though it’s more about journal rankings than science, and journal ranking is of no interest whatsoever to anyone but journal publishers. If they invited me to talk (unlikely), I’d go, but otherwise I can think of better ways of spending a day.
Results
-
