How NOT to Write a Response to Reviewers

Last year I outlined my strategy for writing a response to reviewers.  It was intended primarily for early-career scientists, and the strategy I outlined was most relevant for a paper that had generally positive reviews.

One piece of my advice was to try to view every comment as constructive, even if you disagree with it. Reviewers are often mistaken on some points; indeed, one of the major benefits of the review process is that it calls attention to where we, as authors, have not explained ourselves clearly to the reader.

In my experience as an author and editor, it is pretty rare for a reviewer to say things that are truly hostile or otherwise inappropriate. However, it does occasionally happen that reviewers are unfair. 

I’ve blogged previously about one particularly aggressive and unconstructive review that my coauthors and I received. It was a harsh critique of the very first paper on the long-term evolution experiment with E. coli.  Fortunately, the other reviewer was very positive, and the editor requested a revision.

For some time I’ve thought about posting my response to that negative review. However, I thought the response was perhaps somewhat ill-tempered and overly long. Now, more than 30 years later, if I were advising a young scientist facing a similar review, I’d probably say: “Forget revising it for that journal. Just move on and try again elsewhere.”  But I didn’t do that myself, and I guess it worked out alright in the end.

Without further ado, here’s the response to that reviewer. (You can click on the image for each of the 4 pages to enlarge it.)

---

How to Write a Response to Reviewers in Ten Easy Steps

This post is intended for early-career scientists who have just received the reviews for one of their first journal submissions.  It’s probably most relevant for papers that received generally positive reviews, and which require only minor or moderate revisions.*

1. Copy all of the reviewers’ comments into a new document.  If the editor has substantive comments—especially ones that guide you to which reviewer comments are most important—then copy those, too.
2. Write a short note of thanks to the reviewers and the editor at the top, above all of their comments.  See, you’re already making progress!
3. Use a different font (bold or color) for your responses, to make them easy for the editor (and reviewers, if the paper is returned to them for re-review) to find.
4. Draft a quick response to each comment.  Some responses will be easy, while others will take more time.  I suggest tackling the easy ones first, because it helps you see that you can make good progress even when there are several pages of single-spaced comments (as there often are).  You can polish your responses or change them later, as needed.  Each quick response is a way of recording your initial reaction and the difficulty (small or large) of addressing a specific comment or suggestion.  However, do NOT start editing the actual paper. That should wait until you have got a complete draft of your response letter, which will serve as a “road map” for revising the paper.
5. Reviewers often begin by providing a synopsis of your paper. Thank the reviewers (again, after the synopsis) for their summary and kind remarks, when appropriate. There’s no need to write anything more after the synopsis, unless there’s a substantive misunderstanding of an important point that comes up again in a later comment.  In that case, you might say something like “Thank you for this summary of my/our paper. However, you may have misunderstood one point, about such and such, that I/we address in response to your comment below.”
6. Try to view every comment as constructive. Reviewers may well be mistaken on some points, but rarely (in my own experience) do they say things that are clearly inappropriate. (However, it does occasionally happen.)  That means that you should accept their suggestions, if they improve the paper.  At the same time, you can push back (gently) against a suggestion that you don’t accept, because either (a) it’s not feasible and/or beyond the scope of your study to address it, or (b) you have a different opinion on the issue at hand.  In any case, your response should explain why you don’t accept a particular suggestion.  If (a), then that probably only needs to be said in the response, rather than requiring a change to your paper. Or perhaps you could add something to the paper about the value of future work to examine that issue. If (b), use your response to explain why you disagree and add something like this: “In the revised text on page NN, we have clarified our reasoning on this point.”  Of course, that means you really do need to clarify the issue in the text — but not yet (see point 4 above).
7. Pay special attention to comments where two (or more) reviewers comment on the same issue.  If they both agree (for example, they tell you to clarify or delete some passage), then that is almost certainly something you should do. You should cross-reference the reviewers’ comments when appropriate. You can simplify your responses by saying, for example, “Reviewer #2 made a similar point …” and “We addressed this point in our response to the related comment from Reviewer #1.” If two reviewers made opposing suggestions, then you should also state that fact. Explain how and why you’ve followed one recommendation or the other; if possible, find a way to strike an appropriate balance. For example: “While Reviewer #1 thought we should delete this passage entirely, Reviewer #2 suggested we emphasize the point and provide more context from the literature. After considering both possibilities, we made the following changes: First, we added a sentence with historical context and several references. We then clarify that the relevance of this earlier work to our own study is speculative, and that it is therefore an issue worth exploring in future work.”
8. Once you’ve got a draft response, share it with your co-authors (if any) to see whether they are on board with your resonses or have other suggestions.  Once everyone is in agreement, then use your response letter as a “road map” to edit your paper.  Track your edits, so your co-authors (and the editor and reviewers, if requested in the editor’s instructions) can focus their attention on the relevant sections.
9. You’ll probably find that some text is trickier to edit than you thought in your draft response.  For example, new sentences to address a reviewer’s concern might disrupt the existing flow.  That’s the scholarly life: careful working and reworking are needed at every stage. Also, be cognizant that changes in one section might require (or at least suggest) changes elsewhere to maintain consistency.  For example, if your paper’s Introduction says there are two scenarios (or models or hypotheses or possible outcomes), and a reviewer then suggests adding a third scenario to the Discussion, then maybe you need to edit the Introduction as well. Or perhaps you decide to leave the Introduction as it is, but then clarify in the Discussion that, while you presented two alternatives in the Introduction, your new results (or whatever) raise a third possibility. The point is that changes to one part of a paper may require additional changes elsewhere. Here’s one obvious, and simple, case: If you delete a passage that cites references, you may need to remove them from the Literature Cited (unless you cite them elsewhere) and/or renumber other references, depending on the journal format.
10. Once you and coauthors are satisfied with the revised text, then go back and edit your responses to reviewers. Clarify the changes that you actually made (as opposed to those you initially imagined you would make). When appropriate, add page numbers (from the revised paper) so the reviewer and/or editor can see how you rewrote important passages.
11. Bonus advice! Check out the journal’s instructions for authors.  Make sure all of your paper’s formatting, references, figures, and such conform to those instructions before submitting the revised version.  This will save you from having to submit a re-revised version.

There, you’re done!  Congratulations and good luck!!

*If your paper was rejected, I’m sorry for that.  If it’s any comfort, it happens to everyone.  And yes, it’s often upsetting, even to senior scientists, though many of us have gotten pretty used to it.  Take some time to “blow off steam” by going for a long walk or whatever works for you. Then step away from the paper and reviews for a few days. When you return, you might wonder whether it’s worth appealing the decision to the journal. My general advice would be that it’s rarely worth appealing. I’ve tried it only once or twice, without success. I realize now there are so many fine journals, and so many ways to share papers (preprints, Twitter announcements, etc.), that it’s better to move on and try somewhere else. Of course, you may well want to revise your paper based on the reviews that led to its rejection. Even those reviewers who recommend rejection often have useful comments and advice.

 

A Blast from the Past

Sometimes you need a thick skin to be a scientist or scholar. Almost everyone, it seems, has encountered a reviewer who didn’t bother to read what you wrote or badly misunderstood what you said.

In other cases, you realize on reflection that a reviewer’s criticisms, although annoying and even painful at first, are justified in whole or in part. Addressing the reviewer’s criticisms helps you improve your paper or grant. That’s been my experience in most cases.

Sometimes, though, a reviewer just doesn’t like your work. And occasionally they can be pretty nasty about it. Here’s a case that I experienced on submission of the first paper about the Long-Term Evolution Experiment.

{You can click on the image of the review to enlarge it.}

A few choice lines:

“This paper has merit and no errors, but I do not like it …”

“I feel like a professor giving a poor grade to a good student …”

“The experiment is incomplete and the paper seriously premature …”

“I am upset because continued reliance on statistics and untested models by population geneticists will only hasten the demise of the field.”

“Since the Deans of Science at most universities can only count and not read, I can fully appreciate the reasons for trying to publish this part of the work alone.”

“Molecular biology … should be used whenever possible because molecular biologists control the funding and most of the faculty appointments.”

I’ve occasionally shared this with members of my lab when they get difficult reviews to remind them that it’s not the end of the world or their career, or even the paper that has been scorched.

PS The revised paper was accepted by The American Naturalist. In fact, it won the best-paper award there for the year in which it was published. It has also been cited well over a thousand times.

EDIT:  You can read my response to this review here.

The Ten Commandments of Statistical Inference

As Handed Down to Lenski by Sir Ronald Fisher

1.  Remember the type II error, for therein is reflected the power if not the glory.

2.  Thou shalt not pseudoreplicate or otherwise worship false degrees of freedom.

3.  Respect the one-tailed test, for it can make thine inferences strong.

4.  Forget not the difference between fixed treatments and random effects.

5.  Thou shalt not commit unplanned comparisons without adjusting the rate of type I error for thy transgressions.

6.  Honor both thy parametric and thy nonparametric methods.

7.  Consider not the probability of a particular set of data, but rather the probability of all those sets as or more extreme than thine own.

8.  Thou shalt confuse neither manipulation and observation, nor causation and correlation.

9.  Thou shalt not presume statistical significance to be of scientific importance.

10.  Thou shalt not be fearful of paying homage to a Statistician or His Holy Book, especially before planning an experiment; neither shalt thou be fearful of ignoring the Word of a Statistician when it is damnable; for thou art alone responsible for thine acceptance or rejection of the hypothesis, be it ever so false or true.

The Golden Rule:  Review unto others as you would have them review unto you.

***

Notes:  I wrote this for a graduate course on quantitative methods in ecology and evolutionary biology that I taught in Spring, 1989, at UC-Irvine.  The course focused on experimental design and frequentist methods for drawing inferences.