Repairing Research Integrity
Brendan Maher
Tuesday, 17 June 2008 18:51 UTC
Update (4 August): Nature just published a handful of letters in response to the commentary from Sandra Titus James Wells and Lawrence Rhoades.
Sandra Titus, James Wells and Lawrence Rhoades provide a stirring indictment of the research community in a commentary this week. Research integrity can’t be maintained if misconduct goes unreported, they say. And a survey they commissioned through the Office of Research Ingegrity indicates there could be as many as a thousand unreported instances of misconduct a year.
The authors promote a zero tolerance policy among institutions and make other recommendations. An editorial this week makes a broader call for change, recognizing that not all incidents require full on investigations with punitive action, it calls instead for enhanced structures to use instances of mistake and misconduct as learning experiences that might inform better policies.
How could better policies push back against this seeming flood of misconduct?
Updated 04 August 2008 19:22 UTC
-
Replies
Jump to resultsResults
-
I concur with Lynn Howard Ehrle’s concern about the effect of the broader institutional environment on research integrity, including University leaders who accept “posts on corporate boards of directors where they have a primary legal fiduciary responsibility to their stockholders that is in conflict with the mission of the university, their students and patients.” Indeed, consider the case of a president of both the University and its affiliated health system who also serves on the Board of Directors of a tobacco company.
I likewise concur that self-policing (and simple disclosure) will not address these problems. As evidenced by their recent report on the same, the US Dept of Health & Human Services Office of Inspector General was not especially impressed with the NIH’s oversight of conflict of interest COI) among its grantee institutions, and the recent New York Times article on Senator Grassley’s investigation of the significant and significantly incomplete disclosure by two Harvard Faculty (also see Nature coverage) hints at the gravity of the situation.
However, I feel he is a bit hard on ORI. They can and do step up their penalties as needed, as demonstrated by the Eric Poehlman case. Further, ORI focuses on scientific integrity related to work funded by the Public Health Service, which would not include policing upper administration personnel not engaged in actual research nor broader issues of COI. In the same vein, ORI cannot revoke medical licenses or impose penalties related to other health professions and cannot require an institution to dismiss a researcher. Indeed, it would be inappropriate for them to seek (much less wield) such sweeping powers.
Perhaps accrediting bodies or some other appropriate regulatory agency could begin to take on such a role (i.e., oversight of institutional COI) given the proliferation of increasingly complex agreements among industry, universities, and academic health centers and their potential impact on education, training, patient care – and research integrity.
-
Anonymous
I think if there was a way to punish institutions for the actions of their faculty, the institutions would better self police themselves. I wish ORI could do this, but I understand the complexities that Michelle elaborated on.
I had posted my story which is written about in plagiary.org. What helped me was U. S. Copyright Law. I wish I had known more about it at the time, then I would have registered my dissertation sooner, giving it more protection. Perhaps if articles about plagiarism included more about copyright law, then folks could step and take measures to protect themselves better?
-
Anonymous
Is research fraud simply the result of some “bad apples"? I’ve presented a few papers showing how data were misrepresented by some rather clever bad apples. These papers might well have been titled, “How to misrepresent your data and get away with it.”
Let’s say you formulate your hypothesis, you conduct your research, and you find that your data do not support your hypothesis. Can you now publish findings that support your hypothesis? Yes, you can. Here’s how.
1. The Peer Review Process
There’s peer review, then there is “peer review.” Submit your paper to a journal editor you know. Editors who know you, or know of you, won’t believe you would ever misrepresent your findings. They’ll stand by you.2. The Raw Data
Scientists are supposed to respond to requests for their raw data. However, there is no negative consequence if you don’t hand them over. Some say that all researchers should submit their raw data with their paper. Don’t worry. The current consensus is that it is too expensive to check your raw data.3. The Abstract
Focus on making your abstract convincing. The academic community is too busy to read your article.4. Critics
When colleagues question your work, attack them personally in print. Journal editors will publish your attacks. Eventually, most of your critics will back off, deciding it’s just not worth the trouble.Finally, be aware that the scientific method has a built-in reporting system, which eventually could lead to your misrepresentations being discovered. Still, few will ever know unless the media covers the story.
-
It looks like the University of Toronto’s Institute of Medical Science will be requiring a Hippocratic Oath for graduate students in the life sciences. Check out The Chronicle of Higher Ed for more.
-
There is a relevant discussion going on currently at the Nature Precedings forum on Nature Network, specifically about plagiarism.
-
Anonymous
I agree with Lynn Howard Ehrle that the Office of Research Integrity does very little. Thanks for helping me improve my vocabluary and learn what a “Faustian pact,” means (I looked it up on Wikipedia).
I learned that registering your copyright for $45 is one way to protect yourself. My full story is posted at http://www.plagiary.org/responses.htm. I wrote about my experience when my former PhD advisor published parts of my dissertation, incorrectly and without giving me credit. I did contact the Office of Research Integrity, and John Dahlberg from that office basically said they couldn’t do much because we were former collaborators, and if the material had been plagiarized incorrectly, it was probably an honest mistake. The university acted, ensuring the final version of the paper removed all my content. However, the draft version remains on the journal’s website, and they refused to take it down.
My advice to people facing these dilemnas. First, make sure you save all your records. Second, write down everything you feel BY HAND (to keep you from going overboard). Then, take a DEEP breath. Come up with a story that a non-technical person can understand. Mention specific facts, but don’t go into details (something we do when we’re anxious). Then, write letters to Nature, write letters to journals like Plagiary, write letters to the Office of Research Integrity, or start your own blog on blogspot. Also, read my story about copyrights and see if you can register a copyright. If it is registered, your legal fees are covered if you go after someone.
I think Anonymous at 24 June 2008 03:53 also makes some good points about what happens. It has happened to me.
When my issue happened, the guest editor for that journal had been a coworker on almost the same paper published 6 months earlier. It also involved a compound and he had founded a company related to that compound.
In the article, I could also tell the data had been “combined” to make it look better. I knew, because I had generated some of the data, and knew what parts had not been generated. However, it was all presented in one paragraph (actually, my paragraph) as if it was done together. My p values were still there, but the “n” numbers had been removed when my paragraph was copied. I too was also attacked personally when I brought it up.
-
Relatively speaking, academic misconduct is aggressively dealt with when it involves powerless undergraduates. Less so in graduate school, even less when it involves faculty, and essentially not at all once it becomes institutionalized and government funded. Repairing scientific integrity will eventually require punishing the powerful, which few if any seem willing to do.
See IJOEH paper posted at sludgefacts.org (Snyder, 2005) as a case in point.
The U. S. Environmental Protection Agency’s regulation on land application of sewage sludge (biosolids) failed to pass an internal peer-review in the Office of Research & Development (ORD) where I worked in 1992. To solve this problem, EPA’s Office of Water (OW) decided to create an extensive body of scientific literature to support the regulation.
To this end, OW put together a cooperative agreement with an industry trade association to build a network of supportive scientists, called “gatekeepers,” at land grant universities across the country funded with congressional earmarks. To ensure success, the agreement established a network of college deans to report any scientists working in the area at their institutions and stressed the importance of discrediting opponents to EPA’s regulation.
Sixteen years and many tens of millions of dollars later, EPA is the proud owner of an extensive body of peer-reviewed scientific literature, including two EPA-funded National Academy of Sciences (NAS) National Research Council reports.
According to the NAS (2002), over half of America’s sewage sludge is land applied and there is no documented scientific evidence that EPA’s regulation has ever failed to protect public health or the environment. Two peer-reviewed scientific articles linking land applied sewage sludge to gastrointestinal, respiratory and skin-related illnesses were cited in drafts but deleted from the final report.
These would be my papers, which I published as a research microbiologist when I worked at EPA-ORD and was assigned to the University of Georgia.
To get the research done, I had to personally fund all of the work myself at the Univesity of Georgia over a period of five years. Senior EPA officials cut off all of my in-house EPA funding. They also attempted to remove my laboratory director for approving a research article raising concerns about sewage sludge, which I published in Nature in 1999. In return for allowing me to continue this research on my own at the University of Georgia, EPA headquarters required that I agree to resign my position at EPA afterwards, which I had held for over thirty years.
On May 15, 2008, Nature published a superb news article and editorial (pp. 258; 262-3) questioning some of the data used by the NAS to support EPA’s sludge regulation. A federal judge variously described these data as “incomplete” “fudged” “fabricated” and “invented.”
Nature editors referred to EPA’s sewage sludge program as “an institutional failure spanning more than three decades – and presidential administrations of both parties.” Editors commended the NAS for its report issued in 2002, and (erroneously) praised the NAS for citing our work. Editors mentioned that our findings were independently confirmed last year by several universities in Ohio.
On May 28, Nature ran a brief correction quoting an NAS spokesperson who said (erroneously) that the NAS panel voted to delete our research from the final version of their report because it was not relevant. Fortunately, an NAS panel member submitted a correction to Nature’s “correction” (E. Harrison to correspondence@nature.com, June 17), stating that the NAS panel – in violation of NAS policies – was never informed that our research was being removed from the final report and that she would never have supported such an action.
The half of this story, which goes to show how hopeless it can be for the system to correct itself, has never been told.
The National Academy of Sciences panel extensively borrowed from my research presentations, pre-publication manuscripts, and in-press journal articles throughout its report and used our information to reach a number of its overarching conclusions. Initially, the NAS properly cited all of our work in draft versions. However, someone with access to the final report removed all references to our work, thus giving the NAS credit for the entirety of our original published and unpublished works on sewage sludge.
The rub came when we submitted our work to Environmental Health Perspectives shortly after the NAS published its report. Our manuscript was rejected for not crediting the NAS. EHP Science Editor James Burkhart wrote: “A major shortcoming of the manuscript is the lack of any reference to the very recent National Research Council report, Biosolids Applied to Land, which addresses virtually all of the issues raised by the authors. A Commentary on that report would give the authors ample scope to present their views, either criticism or support, in a contemporary context.”
After we provided extensive documentation proving that we were the original source of all of the information in our submission, EHP accepted our work. The journal let us reference our published papers and presentations and just credit the NAS with “echoing” our findings and recommendations (Gattie DK, Lewis DL. Environ. Health Perspect. 112:126-31 (2004)).
Nevertheless, EPA headquarters terminated me in 2003 at the peak of my scientific career when I was publishing first-authored articles in such journals as Nature, Lancet, and Nature Medicine. The University of Georgia informed my department chairman where I have temporary status as an unpaid visiting scientist that it would not support employing me because that would “hurt EPA funding.”
In the end, I spent my family’s savings to finish the research EPA tried to stop – only to have the NAS take credit for it all. For the past five years, I have remained unemployed in Georgia. I have no reason to believe that EPA and its network of “gatekeepers” would not do the same to me again if I tried to start over again elsewhere. They ended up robbing me of more than just academic credit – they took away my life’s work and future as a scientist.
The point, however, is not what was done to me personally. What’s important is that the scientific community realizes the full extent to which academic misconduct is becoming institutionalized at levels beyond which the system is capable of self-correcting. I personally believe that our best hope is that, one day, publishers and editors at Nature and other leading science journals will decide to tackle institutional, government-funded scientific misconduct on their own.
Every scientific institution that permits academic misconduct to invade its top management levels depends on science journals to publish their data and give their scientists credibility. Publishers and editors simply need to become better educated on how scientific misconduct gets institutionalized in government and academia and then develop effective ways to hold these institutions accountable when it is not corrected.
-
Reports such as those by Titus et al. highlight the fact that misconduct occurs with higher frequency than anyone would like to think. In my personal experience, the prevalence is probably much higher than that reported recently.
In my opinion, scientific and medical research, as a career, is susceptible to similar pressures as those applied on any member of any other profession. The combination of the expected pressure and an individual’s “susceptibility”, whether psychological, ethical, or otherwise, leads to misconduct.
I believe that all institutions, regardless of funding source, should adopt a zero tolerance policy. Students, faculty and staff should be educated, as a requirement of training/funding/employment, on the definition of scientific misconduct, the infrastructure in place to deal with concerns of misconduct, the protocol to follow when one suspects misconduct and the potential penalties one can expect to suffer if he/she engages in scientific misconduct.
Beyond the necessity of education and awareness on the part of institutions, I fundamentally believe that every person involved in the scientific process should be committed to the pursuit of truth. These individuals should also be prepared to defend the truth when and if necessary. Perhaps an oath, as has been recently implemented in Toronto, might bolster such a sense of responsibility-although I am unconvinced that it will not soon become nothing more than a ceremonial rite of passage.
Most importantly, more frequent and informed discussion on the topic of misconduct, and how to handle/report it, is needed. Such dialogue will lead to greater cognizance and, ultimately, deterrence.
-
David Lewis makes a real good point about the role of journals. I had always thought of journals as just reporting it – not getting involved. However, as Lewis points out, journals publish the work. They are the final step. So if they step up their enforcement, it could be major stride to reducing misconduct.
Plagiarized Grad Student
plagiarizedgradstudent@gmail.com -
Anonymous
CNN reported today that a professor at Columbia Univ who was in the midst of scandal for a number of things was let go due to students claiming plagiarism. Hopefully Nature and other news outlets will have more details later
Results
-
