CSE's White Paper on Promoting Integrity in Scientific Journal Publications

2.0 ROLES AND RESPONSIBILITIES IN PUBLISHING

2.3 Reviewer Roles and Responsibilities

Because the number of scientific articles published each year continues to grow, the peer-review process, together with the quality of the editorial board, is cited as the primary influence on a journal's reputation, impact factor, and standing in the field.

Scientific journals publishing peer-reviewed articles depend heavily on the scientific referees or reviewers who, in most cases, volunteer their time and expertise to participate in the peer-review process. In most circumstances, at least 2 reviewers are solicited to evaluate a manuscript; some journals request 3 reviews. In cases of controversy or strong disagreement regarding the merits of the work, an additional review might be solicited or one of the journal's editors might give an evaluation. Also, more than 3 reviewers are sometimes used if reviewers from several fields appear to be needed to obtain a thorough evaluation of a paper.

This important process requires both fairness in judgment and expertise in the field. Peer reviewers also have significant responsibilities toward authors, editors, and readers.

Some peer reviewer responsibilities to authors include:

• Providing written unbiased feedback in a timely manner on the scholarly merits and the scientific value of the work, together with a documented basis for the reviewer's opinion.
• Indicating whether the writing is clear, concise, and relevant and rating the work's composition, scientific accuracy, originality, and interest to readers.
• Avoiding personal comments or criticism.
• Refraining from direct author contact without the editor's permission.

Some peer reviewer responsibilities to editors include:

• Notifying editor immediately if unable to review in a timely manner and providing the names of potential other reviewers.
• Determining scientific merit, originality, and scope of the work; indicating ways to improve it; and recommending acceptance or rejection while using a rating scale.
• Noting any ethical concerns, such as substantial similarity between the reviewed manuscript and any published paper or any manuscript concurrently submitted to another journal. Any violation of accepted norms of ethical treatment of animal or human subjects should also be pointed out.
• Alerting the editor about any potential personal or financial conflict of interest and declining to review when a possibility of a conflict exists (see 2.3.2).

Some peer reviewer responsibilities to readers include:

• Ensuring that the published articles adhere to the journal's standards.
• Protecting readers from incorrect or flawed research or studies that cannot be validated by others.
• Being alert to any failure to cite relevant work by other scientists.

2.3.1 Reviewer Selection

Editors, frequently with the assistance of electronic databases of reviewers kept by their journal's offices, choose reviewers whose expertise most closely matches the manuscript's topic and invite them to review the paper. The editors also consider the number of manuscripts sent to a reviewer so as not to overburden any one expert.

Frequently the reviewer selection process and the journal's internal policies address the issue of potential bias by eliminating reviewers from the same institution as that of the author(s) and by asking the reviewer to disclose any potential conflict of interest. Reviewers might also be asked to disclose to the editor any personal or professional connection to the author(s) and decline the assignment if they feel unqualified to do the review or cannot review in a timely manner. This "bias screening" at the point of reviewer selection may be incorporated into an online submission system or posted on the journal site as a policy.

2.3.2 Ethical Responsibilities of Reviewers

Confidentiality. Material under review should not be shared or discussed with anyone outside the designated review process unless necessary and approved by the editor. Material submitted for peer review is a privileged communication that should be treated in confidence, taking care to guard the author's identity and work. Reviewers should not retain copies of submitted manuscripts and should not use the knowledge of their content for any purpose unrelated to the peer-review process. Although it is expected that the editor and/or reviewers will have access to the material submitted, authors have a reasonable expectation that the review process will remain strictly confidential. If a reviewer is unsure about the policies for enlisting the help of others in the review process, he or she should ask the editor.

Constructive critique. Reviewer comments should acknowledge positive aspects of the material under review, identify negative aspects constructively, and indicate the improvements needed. Anything less leaves the author with no insight into the deficiencies in the submitted work. A reviewer should explain and support his or her judgment adequately so that editors and authors may understand the basis of the comments. Any statement that an observation or argument has been previously reported must be accompanied by a relevant citation. Knowledge of duplicate publication should also be shared.

The purpose of peer review is not to demonstrate the reviewer's proficiency in identifying flaws. Reviewers have the responsibility to identify strengths and provide constructive comments to help the author resolve weaknesses in the work. A reviewer should respect the intellectual independence of the author.

Although reviews are confidential, all comments should be courteous and capable of withstanding public scrutiny.

Competence. Reviewers who realize that their expertise is limited have a responsibility to make their degree of competence clear to the editor. Although a reviewer may not be an expert in every aspect of the content, the assignment should be accepted only if he or she has adequate expertise to provide an authoritative assessment. A reviewer without the requisite expertise is at risk of recommending acceptance of a submission with substantial deficiencies or rejection of a meritorious paper. In such cases, a reviewer should decline the review.

Impartiality and integrity. Reviewer comments and conclusions should be based on an objective and impartial consideration of the facts, exclusive of personal or professional bias. All comments by reviewers should be based solely on the paper's scientific merit, originality, and quality of writing as well as on the relevance to the journal's scope and mission, without regard to race, ethnic origin, sex, religion, or citizenship of the authors.

A reviewer should not take scientific, financial, personal, or other advantage of material available through the privileged communication of peer review, and every effort should be made to avoid even the appearance of taking advantage of information obtained through the review process. Potential reviewers who are concerned that they have a substantial conflict of interest should decline the request to review or discuss their concerns with the editor.

Disclosure of conflict of interest. To the extent possible, the system of review should be designed to minimize actual or perceived bias on the reviewer's part. If reviewers have any interest that might interfere with an objective review, they should either decline a role as reviewer or disclose the conflict of interest to the editor and ask how best to address it. Some journals require reviewers to sign disclosure forms that are similar to those signed by authors.

Timeliness and responsiveness. Reviewers are responsible for acting promptly, adhering to the instructions for completing a review, and submitting it in a timely manner. Failure to do so undermines the review process. Every effort should be made to complete the review in the time requested. If it is not possible to meet the deadline for the review, then the reviewer should promptly decline to perform the review or should inquire whether some accommodation can be made to resolve the problem.

2.3.3 Examples of Reviewer Impropriety

• Misrepresenting facts in a review.
• Unreasonably delaying the review process.
• Unfairly criticizing a competitor's work.
• Breaching the confidentiality of the review.
• Proposing changes that appear to support the reviewer's own work or hypotheses.
• Making use of confidential information to achieve personal or professional gain.
• Using ideas or text from a manuscript under review.
• Including personal or ad hominem criticism of the author(s).
• Failing to disclose a conflict of interest that would have excluded the reviewer from the process.

2.3.4. Using Anonymous Reviewers: Critique of the Process

For many scientific journals, the peer review is performed as a "partially masked" system where the names of the reviewers are unknown to the authors, but the names of the authors are known to reviewers and editors. Other journals use a double-masked system, where the reviewers do not know the identity of the authors or their affiliation.

There is an ongoing discussion on whether the popular model of the partially masked peer-review is optimal, and some journals and editors¹ propose a fully open system in which all participants know each other's identities. There are strong arguments for and against each model, but most journal editors consider anonymity of the reviewer a norm that they are not willing to change.

The strongest criticism of the partially masked peer-review process has to do with the fact that, even when all precautions are taken, the process remains highly subjective and relies on reviewers who may take advantage of ideas they find in yet-unpublished manuscripts; show bias in favor or against a researcher, an institution, or an idea; be insufficiently qualified to provide an authoritative review; or abuse their position because they do not feel accountable.

The open peer-review concept (where all parties' identities are fully disclosed) offers its own dilemmas, however. Knowledge of reviewers' names could make them objects of animosity or vengeful behavior and consequently reviewers could become less critical and impartial, especially when judging their colleagues' work. This can also occur with the partially masked system, particularly within small specialties where researchers can easily guess who reviewed the manuscript.

(Authorship: Anna Trudget took the lead in writing this section of the white paper on behalf of the CSE Policy Committee. Members of the Policy Committee and the CSE Board of Directors reviewed and commented on it. This section was formally approved by the CSE Board of Directors on September 13, 2006.)

1. Rennie D. Freedom and responsibility in medical publication: setting the balance right. JAMA. 1998;280:300-302.

Return to Top

Resources and Case Studies

Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA. 1998;280:231-233.

Committee on Publication Ethics. Reviewers competing interests: the COPE cases. Available at: http://www.publicationethics.org.uk/cases/onethreetwo. Accessed May 5, 2005.

Committee on Publication Ethics. Reviewer confidentiality: the COPE cases. Available at: http://www.publicationethics.org.uk/cases/onethreethree. Accessed May 5, 2005.

The Committee on Publication Ethics (COPE) Report 1999. Available at: http://www.publicationethics.org.uk/reports/1999/1999pdfcomplete.pdf. Accessed January 18, 2005.

The COPE Report 2000. Case 00/24. Reviewer submitting for publication material that had been removed from a paper he had reviewed. Available at: http://www.publicationethics.org.uk/reports/2000/12i.pdf/download. Accessed May 5, 2005.

The COPE Report 2000. Case 00/25. A paper which discloses confidential material. Available at: http://www.publicationethics.org.uk/reports/2000/12j.pdf. Accessed May 5, 2005.

The COPE Report 2001. Case 01/24. Submission of a paper by a reviewer. Available at: http://www.publicationethics.org.uk/reports/2001/14t.pdf/download. Accessed May 5, 2005.

The COPE Report 2002. Case 01/34. Allegation of reviewer malpractice. Available at: http://www.publicationethics.org.uk/reports/2002/9h.pdf/download. Accessed May 5, 2005.

The COPE report 2002. Case 01/39. Referee with a conflict of interest. Available at: http://www.publicationethics.org.uk/reports/2002/9l.pdf/download. Accessed May 5, 2005.

Ethical Guidelines to Publication of Chemical Research. Washington, DC: American Chemical Society; January 2000. Available at: https://paragon.acs.org/paragon/ShowDocServlet?contentId=paragon/menu_content/newtothissite/eg_ethic2000.pdf. Accessed January 18, 2005.

Evans AT, McNutt RA, Fletcher SW, Fletcher RH. The characteristics of peer reviewers who produce good quality reviews. J Gen Intern Med. 1993;8:422-428.

Guidelines for Reviewers. Available at: http://www.bath.ac.uk/lispring/journal/reviewgd.htm. Accessed January 18, 2005.

Lock S. A Difficult Balance: Editorial Peer Review in Medicine. New York, NY: Lippincott Williams and Wilkins; 1985.

Office of Research Integrity, Office of Public Health and Science, US Department of Health and Human Services. Managing Allegations of Scientific Misconduct—a Guidance Document for Editors. Available at: http://ori.dhhs.gov/documents/masm_2000.pdf. Accessed February 24, 2006.

Rennie D. Freedom and responsibility in medical publication: setting the balance right. JAMA. 1998;280:300-302. Available at: http://www.ama-assn.org/public/peer/7_15_98/pv71038x.htm. Accessed January 18, 2005.

Research Ethics Program, University of California, San Diego. Responsible Conduct of Research (RCR) Internet Instruction. Available at: http://ethics.ucsd.edu/courses/integrity/assignments/review.html. Accessed January 18, 2005.

van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998;280:234-237.

World Association of Medical Editors. Reviewer conflict of interest. Available at: http://www.wame.org/conflict.htm. Accessed May 5, 2005.

Return to the Promoting Integrity in Scientific Journals table of contents page.