Peer review is the principal mechanism by which the quality of research is judged. Most funding decisions in science and the academic advancement of scientists are based on peer-reviewed publications.
Because the number of scientific articles published each year continues to grow, the quality of the peer-review process and the quality of the editorial board are cited as primary influences on a journal’s reputation, impact factor, and standing in the field.
Scientific journals publishing peer-reviewed articles depend heavily on the scientific referees or reviewers who typically volunteer their time and expertise. In most circumstances, at least 2 reviewers are solicited to evaluate a manuscript; some journals request 3 reviews. This may be required in situations where review by a statistician is needed. In cases of controversy or strong disagreement regarding the merits of the work, an additional review may also be solicited or one of the journal’s editors might give an evaluation. More than 3 reviewers are sometimes used if reviewers from several fields are needed to obtain a thorough evaluation of a paper.
In addition to fairness in judgment and expertise in the field, peer reviewers have significant responsibilities toward authors, editors, and readers.
Peer-reviewer responsibilities toward authors
- Providing written, unbiased feedback in a timely manner on the scholarly merits and the scientific value of the work, together with the documented basis for the reviewer’s opinion
- Indicating whether the writing is clear, concise, and relevant and rating the work’s composition, scientific accuracy, originality, and interest to the journal’s readers
- Avoiding personal comments or criticism
- Maintaining the confidentiality of the review process: not sharing, discussing with third parties, or disclosing information from the reviewed paper
- Notifying the editor immediately if unable to review in a timely manner and providing the names of potential other reviewers
- Alerting the editor about any potential personal or financial conflict of interest and declining to review when a possibility of a conflict exists (see section 2.3.2)
- Complying with the editor’s written instructions on the journal’s expectations for the scope, content, and quality of the review
- Providing a thoughtful, fair, constructive, and informative critique of the submitted work, which may include supplementary material provided to the journal by the author
- Determining scientific merit, originality, and scope of the work; indicating ways to improve it; and recommending acceptance or rejection using whatever rating scale the editor deems most useful
- Noting any ethical concerns, such as any violation of accepted norms of ethical treatment of animal or human subjects or substantial similarity between the reviewed manuscript and any published paper or any manuscript concurrently submitted to another journal which may be known to the reviewer
- Refraining from direct author contact
Sample correspondence related to this topic is available on the CSE website.5
Peer-reviewer responsibilities toward readers
- Ensuring that the methods are adequately detailed to allow the reader to judge the scientific merit of the study design and be able to replicate the study, if desired
- Ensuring that the article cites all relevant work by other scientists
Editors, frequently with the assistance of electronic databases of reviewers kept by their journal’s offices, choose reviewers whose expertise most closely matches the manuscript’s topic and invite them to review the paper. The editors also consider the number of manuscripts sent to a reviewer so as not to overburden any one expert. Some journals encourage authors to suggest preferred reviewers and reviewers they would prefer to be excluded.
Ideally, the reviewer selection process and the journal’s internal policies address the issue of potential bias by excluding reviewers from the same institution as that of the author(s) and by asking reviewers to disclose any potential conflict of interest. Reviewers may also be asked to disclose to the editor any personal or professional connection to the author(s) and decline the assignment if they believe there is a potential conflict of interest, feel unqualified to do the review, or cannot review in a timely manner. This “bias screening” at the point of reviewer selection may be incorporated into the forms in an online submission system, the e-mail sent to request the review, or posted on the journal site as a policy.
Confidentiality. Material under review should not be shared or discussed with anyone outside the review process unless necessary and approved by the editor.6–7 Sample correspondence related to this topic is available on the CSE website.5 Material submitted for peer-review is a privileged communication that should be treated in confidence, taking care to guard the author’s identity and work. Reviewers should not retain copies of submitted manuscripts and should not use the knowledge of their content for any purpose unrelated to the peer review process.
Although it is expected that the editor and reviewers will have access to the material submitted, authors have a reasonable expectation that the review process will remain strictly confidential. If a reviewer is unsure about the policies for enlisting the help of others in the review process, he or she should ask the editor.
Constructive critique. Reviewer comments should acknowledge positive aspects of the material under review, identify negative aspects constructively, and indicate the improvements needed. Anything less leaves the author with no insight into the deficiencies in the submitted work. A reviewer should explain and support his or her judgment clearly enough that editors and authors can understand the basis of the comments. The reviewer should ensure that an observation or argument that has been previously reported be accompanied by a relevant citation and should immediately alert the editor when he or she becomes aware of duplicate publication.
The purpose of peer review is not to demonstrate the reviewer’s proficiency in identifying flaws. Reviewers have the responsibility to identify strengths and provide constructive comments to help the author resolve weaknesses in the work. A reviewer should respect the intellectual independence of the author.
Although reviews are confidential, all anonymous comments should be courteous and capable of withstanding public scrutiny. Some journals ask reviewers to provide two sets of comments: one for the author and the other for the editor only. The latter can sometimes be more candid and can recommend that the manuscript be accepted or rejected (something that arguably should not be part of comments to the author).
Competence. Reviewers who realize that their expertise is limited have a responsibility to make their degree of competence clear to the editor. Reviewers need not be expert in every aspect of an article’s content, but they should accept an assignment only if they have adequate expertise to provide an authoritative assessment. A reviewer without the requisite expertise is at risk of recommending acceptance of a submission with substantial deficiencies or rejection of a meritorious paper. In such cases, the reviewer should decline the review.
Impartiality and integrity. Reviewer comments and conclusions should be based on an objective and impartial consideration of the facts, exclusive of personal or professional bias. All comments by reviewers should be based solely on the paper’s scientific merit, originality, and quality of writing as well as on the relevance to the journal’s scope and mission, without regard to race, ethnic origin, sex, religion, or citizenship of the authors.
A reviewer should not take scientific, financial, personal, or other advantage of material available through the privileged communication of peer review, and every effort should be made to avoid even the appearance of taking advantage of information obtained through the review process. Potential reviewers who are concerned that they have a substantial conflict of interest should decline the request to review and/or discuss their concerns with the editor.
Disclosure of conflict of interest. To the extent possible, the review system should be designed to minimize actual or perceived bias on the reviewer’s part. If reviewers have any interest that might interfere with an objective review, they should either decline the role of reviewer or disclose the conflict of interest to the editor and ask how best to address it. Some journals require reviewers to sign disclosure forms that are similar to those signed by authors.
Timeliness and responsiveness.Reviewers are responsible for acting promptly, adhering to the instructions for completing a review, and submitting it in a timely manner. Failure to do so undermines the review process. Every effort should be made to complete the review within the time requested. If it is not possible to meet the deadline for the review, then the reviewer should promptly decline to perform the review or should inquire whether some accommodation can be made to resolve the problem.
- Misrepresenting facts in a review
- Unreasonably delaying the review process
- Unfairly criticizing a competitor’s work
- Breaching the confidentiality of the review
- Proposing changes that appear to merely support the reviewer’s own work or hypotheses7
- Making use of confidential information to achieve personal or professional gain
- Using ideas or text from a manuscript under review
- Including personal or ad hominem criticism of the author(s)
- Failing to disclose a conflict of interest that would have excluded the reviewer from the process
For many scientific journals, the peer review is performed as a single masked, or single blind, system in which the names of the reviewers are unknown to the authors, but the names of the authors are known to reviewers and editors. Other journals use a double masked, or double blind, system, in which the reviewers do not know the identity of the authors or their affiliation.
There is an ongoing discussion about whether the popular model of partially masked peer review is optimal, and some journals and editors8 propose a fully open system in which all participants know the others’ identities. There are strong arguments for and against each model, but most journal editors consider anonymity of the reviewer a norm that they are not willing to change.
The strongest criticism of the partially masked peer-review process is that, even when all precautions are taken, the process remains highly subjective and relies on reviewers who may take advantage of ideas they find in yet-unpublished manuscripts; show bias in favor of or against a researcher, an institution, or an idea; be insufficiently qualified to provide an authoritative review; or abuse their position because they do not feel accountable.
The open peer-review concept (in which all parties’ identities are fully disclosed) offers its own dilemmas, however. Knowledge of reviewers’ names could make them objects of animosity or vengeful behavior, and consequently reviewers could become less critical and impartial, especially when judging their colleagues’ work. This can also occur with the partially masked system, particularly within small specialties where researchers can easily guess who reviewed the manuscript.
Some journals find it useful to publicly thank reviewers for their generous volunteer efforts. This may take the form of a published list of reviewers that appears in the journal on a regular (annually, semiannually) basis. Journals may also offer continuing medical education credits for completed reviews.
- Committee on Publication Ethics (COPE). What to do if a reviewer suspects undisclosed conflict of interest (CoI) in a submitted manuscript (flowchart). Available at: https://publicationethics.org/resources/flowcharts-new/conflict-interest (Accessed April 29, 2018).
- Committee on Publication Ethics (COPE). What to do if you suspect redundant (duplicate) publication – Suspected redundant publication in a submitted manuscript (flowchart). Available at: http://publicationethics.org/files/u2/01A_Redundant_Submitted.pdf (Accessed March 9, 2012).
- Committee on Publication Ethics (COPE). What to do if you suspect plagiarism – Suspected plagiarism in a submitted manuscript (flowchart). Available at: http://publicationethics.org/files/u2/02A_Plagiarism_Submitted.pdf (Accessed March 9, 2012).
- Committee on Publication Ethics (COPE). What to do if you suspect fabricated data – Suspected fabricated data in a submitted manuscript (flowchart). Available at: http://publicationethics.org/files/u7140/Flowchart%20Fabricated%20A%20revised%20May%202011.pdf (Accessed March 9, 2012).
- Council of Science Editors. Sample correspondence. Available at: http://www.cseditors.wpengine.com/i4a/pages/index.cfm?pageid=3335 (Accessed March 9, 2012).
- Committee on Publication Ethics (COPE). What to do if you suspect an ethical problem with a submitted manuscript (flowchart). Available at: http://publicationethics.org/files/u2/06_Ethics_Submitted.pdf (Accessed March 9, 2012).
- Committee on Publication Ethics (COPE). What to do if you suspect a reviewer has appropriated an author’s ideas or data (flowchart). Available at: http://publicationethics.org/files/u2/07_Reviewer_misconduct.pdf (Accessed March 9, 2012).
- Rennie D. Freedom and responsibility in medical publication: setting the balance right. JAMA. 1998;280:300-302. Available at: https://jamanetwork.com/journals/jama/fullarticle/187765 (Accessed April 29, 2018).
American Society of Plant Biologists (ASPB). ASPB ethics in publishing: ASPB policies and procedures for handling allegations of editorial misconduct. Available at:http://www.aspb.org/publications/editorialethics.cfm (Accessed March 9, 2012).
Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA. 1998;280:231-233.
Committee on Publication Ethics (COPE). Available at: http://publicationethics.org/ (Accessed March 9, 2012).
American Chemical Society. Ethical guidelines to publication of chemical research, January 2000. Available at:http://pubs.acs.org/userimages/ContentEditor/1218054468605/ethics.pdf (Accessed March 9, 2012).
Evans AT, McNutt RA, Fletcher SW, Fletcher RH. The characteristics of peer reviewers who produce good quality reviews. J Gen Intern Med. 1993;8:422-428.
House of Commons Science and Technology Committee. Peer Review in Scientific Publications. Eighth Reports of Session 2010-12. Available at:http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/85602.htm (Accessed March 9, 2012)
International Congress on Peer Review and Biomedical Publication. Available at: http://www.ama-assn.org/public/peer/peerhome.htm (Accessed March 9, 2012).
Living Spring Journal. Guidelines for reviewers. Available at: http://people.bath.ac.uk/liskmj/living-spring/journal/reviewgd.htm (Accessed April 29, 2018).
Lock S. A Difficult Balance: Editorial Peer Review in Medicine. New York, NY: Lippincott Williams & Wilkins; 1985.
Office of Research Integrity, Office of Public Health and Science, US Department of Health and Human Services. Managing allegations of scientific misconduct: a guidance document for editors. Available at: https://ori.hhs.gov/images/ddblock/masm_2000.pdf (Accessed April 29, 2018).
Research Ethics Program, University of California, San Diego. Responsible conduct of research (RCR) internet instruction. Available at: http://blink.ucsd.edu/sponsor/ora/respcond.html (Accessed April 29, 2018).
van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998;280:234-237.
World Association of Medical Editors (WAME). Reviewer conflict of interest. Available at: http://www.wame.org/ethics-resources/reviewer-conflict-of-interest/ (Accessed March 9, 2012).
(Authorship: Anna Trudgett took the lead in writing this section of the white paper on behalf of the CSE Editorial Policy Committee. Anna Trudgett and Robert Edsall revised this section for the 2009 Update. Stephen Morrissey and Kristi Overgaard revised this section for the 2012 Update. Members of the Editorial Policy Committee and the CSE Board of Directors reviewed and commented on it. This section was formally approved by the CSE Board of Directors on March 30, 2012.)