Citation in Science - Don't Quote Me on That
Allan Sudlow
Monday, 19 May 2008 15:15 UTC
Following a debate on Citation in Science on 27th May at TalkScience
We invite you to continue the discussion. Here are a few topics to get things started:
1. ‘Tools for the Job’: does use of a single citation search tool (e.g. PubMed, UKPMC, Google Scholar, Web of Science) bias the results? Is there a call for the use of mutiple tools?
2. ‘Pick n’ Mix’: selective citation to support a particular argument/hypothesis. Are people only citing portions of an article and thereby deliberately ignoring conflicting evidence elsewhere within the same article?
3. ‘Don’t Quote Me on That’: Even when the “original” paper is cited it is often misquoted. Do those citing not always fully understand the meaning behind a paper? Is this form of mis-citation more a case of misinterpretation rather than misrepresentation?
4. ‘It’s all Just Greek to Me’: is there a citation bias against non-English language papers or papers from “non-English-speaking” countries?
5. ‘Return to Nature’: is there a preference for citing known/higher impact factor paper?
6. ‘Measure for Measure’: are citations and bibliometric measures in general an accurate reflection of research excellence?
-
Replies
Jump to resultsResults
-
Thanks to everyone who came along to the TalkScience debate on Citation in Science last night at the British Library. In particular, a big thank you to Prof. Tim Birkhead for a stimulating and provocative introduction. We had a lively exchange of views involving around 40 or so science researchers, publishers, funders, information specialists and other intrested parties. If you have move to say, please do contribute to the forum….
-
I’d like to see a bit more discussion about what we want to use the metrics for. If you liken scientific progress to the building of a wall, then it would be nice to think that the metrics could tell us who had contributed significantly to advancing particular bits of the structure. Does this necessarily correlate with citation volumes ?
The ‘wall’ analogy would imply the existence of ‘consensus views’ telling us where particular fields were at, and why, and what the next research priorities were. They would reflect mechanisms, causal relationships and such, rather than being simply Review-like. Perhaps this anticipates too much structure in the funding process ?
As an aside, if a lot of the citation volume is spurious, can we estimate just how much noise there is in the signal ? What’s the correlation between ‘celebrity status’ and historical scientific impact ?
-
Thank you, Allan, Lee-Ann and colleagues for a most stimulating and pleasant evening. I very much enjoyed it.
The discussion covered many aspects of citations, ranging from:- authorship – conventions and credit
- “misconduct” (ignoring or mis-citation, plagiarism, conflict of interest in peer-review and so on);
- whether the metrics in themselves are robust;
- whether they are in fact measuring “creativity and innovation” or are too susceptible to gaming or misuse (by which I mean "are being used for purposes other than those for which they were created);
- and who these measures are for — funders, journal publishers, academic institutions, individual reasearchers?;
to name but five.
Perhaps it would be useful to focus the discussion on one of these aspects at a time to make some progress, or at least agree some ground rules of what this group is aiming to achieve. Otherwise, the scope of the debate is vast.
I see that Ian Mulvaney (who was also present at the citation evening) has posted a forum topic in this group about the various metrics used and the basis for how they are calculated. This is a really useful benchmark post which will help a lot to focus thinking (mine, certainly). (See forum index page in this group.)
-
As Maxine says, a thoughtful piece from Ian here
-
David, your comment reminds me of a story I heard about an English climber. Climbing shares a little something with science in that the first person to climb a route gets the honour of naming the route, and it gets written up in a guide book. This climber used to love guide books, because they showed him where the routes weren’t on a crag and he could go and forge out new paths. That would be an interesting application to citation analysis, data mine to find the holes in scientific knowledge!
-
data mine to find the holes
Interesting. That’s one step ahead of the emerging fields one can track in Sciencewatch .
-
Or, as someone recently said to me “The only reason we need to mine data is because we bury it in the first place”
-
I think I posted my comment on the wrong thread. It is here
(and I shoulf have written augur, not auger: pedants unite).
-
I have made David’s comments into a separate forum post for further discussion.
The question is whether publication metrics are appropriate to measure people and/or institutions.
Ian’s forum thread, The art of counting is about the basis for and accuracy of various types of metrics.Please add your views to both of these threads.
-
Simon has started another discussion topic for those who want to discuss citation practices (specifically referencing habits) but do not want to get wrapped up in the bibliometrics. He raises some interesting questions here
Results
-
