Rachel T. Santiago, PhD, University of Missouri, and Garret J. Hall, PhD, Florida State University
Academia’s running joke about the hypercritical Reviewer 2 is somewhat exaggerated, but it has a grain of truth. Publishing your scholarship is, at best, a difficult process that often results in negative feedback, rejection, and a potential existential crisis. Yet you are committed to your work because the (highly) variable reinforcement schedule of publishing is enough to keep you going, including the payoff of that first job or faculty promotion.
As early-career scholars and associate editors at different journals in the field (RS: Journal of School Psychology; GH: Assessment for Effective Intervention), we experience the joy and exasperation of publishing our own work while devoting substantial effort to making recommendations on others’ work. The peer-review process has its flaws, and manuscript decisions are often very nuanced. It isn’t easy to find an effective way to respond to feedback (especially when your indignation spikes after reading your feedback for the first time), even when the feedback is constructive (and sometimes it’s very unconstructive). Your reviewer response approach depends largely on the specific feedback you receive and the nature of your paper. Unfortunately, this is often not something graduate students or early career professionals are formally taught; much of the learning comes from personal experience and consultation with colleagues and mentors. In 2019, the ECF offered step-by-step guidance to navigating Revise and Resubmit decisions (see here: https://www.sssp-research.org/revise-and-resubmit-now-what/). The current post builds on that prior work by highlighting some of the review response processes to help develop an effective response strategy.
General Tips
Read, Process, then Read Again
A Revise and Resubmit (R&R) or Major Revision is still a type of rejection. They’re willing to see a revision, but it is a rejection nonetheless, and it is perfectly normal to have feelings about that and to feel frustrated about the feedback you received. Therefore, it can be helpful to give yourself some breathing space in between your first read-through of the decision letter and your second planning-oriented read-through. (And if you want to print out a particularly rude review and throw it into a fireplace, no judgment here.)
Importantly, you don’t want to get in your own way—don’t count your work out before receiving a final decision. On the flipside, don’t be overconfident that they will accept the revision. Read the review, sort out your reactions to it, then read it again so you can make a plan for submitting the best possible revision and maximizing your chances of acceptance.
Make a Plan
Your review timeline is a difficult part of the process to manage, especially when working with multiple co-authors. It is important to develop a plan to finalize your reviewer responses and manuscript edits with your co-authors as soon as possible after initially receiving a manuscript decision. There is no one way to do this, and it depends on the dynamics of the authorship team, but it’s best to have clear author roles at the revision stage. In our experience, manuscript revision timelines have ranged from a couple of weeks to several months. Journals have default timelines that are communicated within manuscript decision letters. Completing a revision within the standard timeline is ideal, and your initial revision plan should be based on that. However, it’s acceptable to ask the AE or editor assigned to your paper for an extension, if needed.
When you develop a revision plan, you should prioritize how each piece of feedback will be addressed, especially if it is not clear how to address highly variable (or outright discrepant) comments from reviewers. AEs will often provide guidance on which revisions to prioritize. Feedback about core assumptions of your paper, like your theoretical framework or statistical modeling choices, will probably take more time to both revise in the paper and develop a persuasive reviewer response. Communication and collaboration with co-authors are essential for articulating a cohesive response to complex feedback.
Also, find the big picture of the feedback you received, in addition to paying attention to small details. Often, the whole response is greater than the sum of the individual responses to comments. Give yourself time to evaluate both the finished reviewer response and your revised manuscript before you resubmit.
Documenting Your Responses
There are different ways to document your responses to reviewer feedback. A good first step is to read the decision letter carefully to see if the editor or AE specifies how they want you to do this, because there can be variability in what a journal expects. Sometimes an editor will ask for a document with your response to each comment, in addition to highlighting changes (or using tracked changes) in the resubmitted manuscript; sometimes they ask for your responses to each comment without specifying whether you should indicate changes in the revised manuscript; and sometimes they request something else. If the letter or the author guidelines on the journal website do not give specific instructions, you can safely assume that you should include a deidentified document with a response to each editorial and reviewer comment. There are different ways to format this document (e.g., a table or a list). Just know that you may be asked to enter it into a text box in the submission portal or to upload it as an attachment (or both!). You can also ask co-authors or colleagues if they are willing to share an example of how they have formatted their responses to reviewers in the past.
Regardless of the approach you use, aim to be detailed and clear to make it as easy as possible for the editor, AE, and reviewers to find the changes you made. The easier they can find the changes, the clearer it will be that you were thoughtful and comprehensive in the revision process.
Handling Tricky Situations
Sometimes reviewers say or suggest things that are hard to navigate, and there isn’t a handbook for what to do about them. Below, we offer some tips for handling more difficult situations.
Rude, Harsh, or Unnecessarily Critical Reviewers
Reviewers are only able to give constructive feedback to the extent that their own characteristics, egos, and biases allow. It doesn’t make it okay, but it means that sometimes reviewers give problematic feedback. In general, AEs are required not to redact or edit a reviewer’s comments; sometimes an AE explicitly states in their decision letter that a piece of feedback is not necessary to consider; sometimes they do not. In general, it is a good practice to respond to ideas, not tone (even if it is tempting to tell them exactly what you think of their review). However, sometimes reviewers’ comments can be bigoted, racist, xenophobic, or prejudiced (e.g., uncalled-for comments about authors’ language proficiency, unsubstantiated assumptions about culturally relevant practices). In these situations, a good first step is to consult with your co-authors or with other colleagues you trust to identify how to best move forward in a way that protects both your scholarship and your well-being.
You Already Did What They Asked You to Do
A reviewer asks, “Why didn’t you do XYZ?” even when you clearly stated you did on page 15 of your manuscript. Don’t be snarky in your response, even if you think that detail was impossible to miss. Acknowledge the concern the reviewer is raising and respectfully note the location of the detail in question (page number, specific sentence, etc.). At some point, as a reviewer, you will also miss an obvious detail.
Sometimes a reviewer may question something you already did because they didn’t understand it in the first place. In this case, first reflect on whether your initial explanation adequately communicated the detail the reviewer misunderstood. After you do that, determine whether you need to revise the manuscript itself as well as provide further explanation in the response letter. You may only need to do one or the other, but we advise that you revise the manuscript for clarity AND provide more context in the reviewer response.
Misalignment Between Feedback and Methodology
There might be multiple valid ways to conduct a specific method (assessment strategy, data analysis, research design, etc.), and reviewers may offer legitimate criticism and alternatives/improvements to your methods. Sometimes, however, the reviewers are plain wrong. Sometimes, you and your coauthors are too. It’s important to recognize when your methods demonstrate legitimate flaws and assess whether this can be improved upon in revision, either through changes to the methods or revisions to the explanation. However, it’s also important to be able to identify when a reviewer’s methodological suggestion is incorrect or unhelpful to the goals of your study. When a review recommends something that you believe is incorrect, it’s important to communicate a respectful and thorough response to reviewers that demonstrates you have a clear methodological justification for your design or analytic choices and why their recommendation may be inappropriate for your study. It may also help to discuss with the AE what changes are necessary. For example, feedback that criticizes the methods of single-case designs or qualitative studies by comparing them to group design standards of statistical power or sample size is often invalid. This could come from a place of reviewers’ unfamiliarity with the designs/methods or just a bias against them (or both). Either way, providing a clear and thorough justification will demonstrate that you have a strong theoretical, methodological, and/or practical basis for your choices. At the same time, it’s not your job to teach reviewers a method they are unfamiliar with; they need to recognize the limits of their own knowledge before offering constructive feedback.
When You Aren’t Going to Implement a Suggestion
Like the last section, when you decide that you won’t make recommended changes to your study or manuscript, it’s important to provide a clear rationale, especially if the suggestion you are not implementing is a key part of their review. Logically justified responses can provide a compelling basis for saying “We ignored this.” Even when reviewer feedback is clearly not possible to implement (e.g., a reviewer asks why you didn’t collect measures on something when you are using secondary data), remain collegial in your response and just note that the change is not possible for a clear reason. As noted earlier, this also depends on the scope of the changes. A major change you decide against will require more explanation and more nuance. A small change may require only a brief response, like, “We appreciate the attention to detail. We have decided to do XYZ because of XYZ” (i.e., “thanks but no thanks”).
Thoughts from an Associate Editor’s Perspective
As early-career researchers who also serve in reviewing and AE capacities, we currently navigate the review process from the author, reviewer, and AE perspectives. Based on our experience, we recommend asking the AE assigned to your manuscript any questions or clarifications you may have. Personally, we would rather an author ask us to clarify something than have them experience difficulties in the revising process.
It’s important to recognize that different AEs take different approaches, and it can be hard to reach an editorial decision. We are balancing methodological expertise and content knowledge across reviewers, and sometimes reviewers’ recommendations don’t neatly align. Some AEs treat the decision as a vote among reviewers (e.g., two recommend R&R and one recommends Reject, so the AE decides R&R), while others base their decision on the overall picture of the reviews. In addition, some AEs need approval from the journal’s editor-in-chief before finalizing a decision, whereas others do not; this varies by journal policy and editors’ styles. Overall, there will be variability across journals and AEs, and multiple factors go into the decision-making process. In short, it is helpful to be informed about the process while recognizing what is within your control: doing the best work you can and communicating clearly with your AE and in your response to reviewers.
Final Thoughts
Getting reviews on a piece of writing can be hard, both logistically and emotionally. It is even harder when you are an early-career scholar, when your scholarship is underrepresented in school psychology, or if your work challenges preconceived notions of what scholarship should look like. And if you hold marginalized identities, receiving a difficult review could also activate feelings of impostor syndrome or contribute to your mental and emotional labor. Systems-level problems in peer review will require systems-level solutions, but we feel the impacts at the individual level and don’t often get clear guidance on how to address them. Our goal with this post is to offer general guidance that can supplement the support and advice you get from mentors and colleagues. Ultimately, keeping your focus on the scholarship will mean your work has the best possible chance of finding its home.