Referencing. The do's and the dont's.
Simon Hughes
Thursday, 05 June 2008 11:52 UTC
On a wider topic of citation. When writing papers I always attempted to cite only those papers that: a) correctly, and in their entirety, support my own findings; b) point out the most relevant and useful studies to the reader; and c) acknowledge only relevant sources of ideas and techniques. On the other side of the fence, when refereeing a paper I always read a number of the article’s references to help set the scene for what I was being asked to assess. However, many papers seem to have a never ending list of references, making this very much a pick and mix process. I appreciate that it is important to correctly cite the literature, but there appears to be a growing tendency for researchers to go slightly over board on the number of references they include in a paper, which raises the questions:
1) Have they really read them all?
2) Are they second-hand citing?
3) Are they citing papers for which they have only read the abstracts?
4) Are they citing their own or colleagues work?
5) Are they trying to deliberately muddy the waters?
-
Replies
Jump to resultsResults
-
I wonder if it is therefore easier for referees if journals restrict the number of references they allow an author to cite? At Nature, for example, it is 30 for Letters and 50 for Articles.
Typically, when a manuscript is submitted to Nature, the editors search/scan the literature as well as looking at the reference list and our own databases when they are looking for referees. I hope that part of the benefit of this process is to “flush out” hidden lacunae in the authors’ reference list (eg deliberate omission).
I also think that if two or three referees are assessing a paper, and they have been well-chosen, they will between them know whether the authors have appropriately cited (or not) relevant work, but perhaps in view of your post, my assumption is wrong.
-
I think 2, 3 and 4 are quite prevalent.
-
I also believe that #2, #3 and #4 are common. Restricting the references to 30 or 50 is a good idea. I also like the commented references, common with some review journals.
References eventually form the basis for almost all citation metrics, including impact factor and h-index. I guess that most paper authors don’t think about that when collecting their references.
-
I think there is a cultural issue here. Looking at my most recent papers and excluding a paper which used a summary of lots of previous data for a modelling exercise, my average numbers of references per paper is about 20. My field is materials science and my impression is that typically about 20 references is the norm. I believe that the culture in biology is to have much longer bibliographies. This explains why impact factors of biological and medical journals are much greater than those in the physical sciences and, I expect, peoples’ corresponding h-indices/citations per published paper.
Having said that there are a reduced number of papers cited in each publication, I still believe points 2-4 apply very strongly in the physical sciences.
-
I suspect that there might be a rough inverse correlation between time spent as an academic and number of citations per paper. That is, less experienced people tend to cite things more than they need. This occurred to me not with my Nature hat on, but as editor of Mallorn, the journal of the Tolkien Society, which receives a lot of articles from amateur scholars who, to be charitable, try much to hard to come over as scholarly. As an arts journal it has a much fussier style of citation than a science magazine, using no fewer than THREE different systems, simultaneously:
- footnotes
- references
- a separate bibliography
Needless to say I am whipping them into line, and impressing on people that you don’t need to give a full citation every time you mention (say) The Lord of the Rings, as the people who read Mallorn will know this anyway.
An interesting phenomenon that’s happened to me is that people who submit papers to Nature on vertebrate origins will often cite my eccentric 1996 book Before the Backbone: Views on the Origin of the Vertebrates, which is nice of them, but it’s long out of date and there are now other sources. I suspect they are apple-polishing and I often ask people to excise that reference on the grounds that it is a scurrilous, secondary source.
-
Martin writes: I also like the commented references, common with some review journals. I agree. The Nature Review journals have done this since they were first published, and now Nature has adopted the practice for the Review, Progress and Insights articles we publish. We ask for the comments to be a brief summary sentence, however, to be maximally useful to readers.
-
Interesting. We had a very closely related discussion in this post on social versus scientific citations. The point being that I am pretty sure many people do cite other papers not because they are relevant to the presented work, but because politeness demands it or because it is politically smart. E.g. they might cite people who have cited them, or who they hope to be cited by, or people who rank high in the hierarchy and so on. Nothing of this should play a role scientifically, but it very practically does.
A tale-telling sign of useless citations is a clustered appearance in the text with a very vague description. Like something of the type “Interesting results have been derived recently in [4,5,6,7,8,9,10,11,12,13,14,15].” (For examples, see above mentioned post).
I think restricting the number of papers isn’t a good option. It depends too much on the paper length and the topic.
-
Typo: I meant restricting the number of citations.
Come to think of it, maybe it would actually be a good idea to restrict the number of papers people can publish per time. It might actually convince them to put more effort into it, instead of taking stuff apart into the least publishable units and just submit it until it goes through somewhere.
-
I also found that commented references (as mentioned here by Martin and Maxine) can be very useful indeed.
I am in the process of finishing up a manuscript and we (my co-authors and I) should decide whether sending it out to Journal A or Journal B. Journal A poses no limits to the number of references, while Journal B requires no more than 50. We are at about 60 now in our draft.
Is the paper going to get better if we reduced the number of citations by 10, regardless of the journal we are sending the paper out to for review? May be the answer is yes. -
One suggestion we make sometimes to authors is that they cite a recent review that rounds-up relevant primary references.
This is convenient for the author and the reader, in reducing the length of reference lists.
However, the “citation game” has made it seem unfair to some, ie those whose work is discussed in the review but who do not get a citation if the review is cited instead.
However, this method can save on half a dozen individual citations.
Results
-
