How NOT to Write a Response to Reviewers

Last year I outlined my strategy for writing a response to reviewers.  It was intended primarily for early-career scientists, and the strategy I outlined was most relevant for a paper that had generally positive reviews.

One piece of my advice was to try to view every comment as constructive, even if you disagree with it. Reviewers are often mistaken on some points; indeed, one of the major benefits of the review process is that it calls attention to where we, as authors, have not explained ourselves clearly to the reader.

In my experience as an author and editor, it is pretty rare for a reviewer to say things that are truly hostile or otherwise inappropriate. However, it does occasionally happen that reviewers are unfair. 

I’ve blogged previously about one particularly aggressive and unconstructive review that my coauthors and I received. It was a harsh critique of the very first paper on the long-term evolution experiment with E. coli.  Fortunately, the other reviewer was very positive, and the editor requested a revision.

For some time I’ve thought about posting my response to that negative review. However, I thought the response was perhaps somewhat ill-tempered and overly long. Now, more than 30 years later, if I were advising a young scientist facing a similar review, I’d probably say: “Forget revising it for that journal. Just move on and try again elsewhere.”  But I didn’t do that myself, and I guess it worked out alright in the end.

Without further ado, here’s the response to that reviewer. (You can click on the image for each of the 4 pages to enlarge it.)

---

Advertisement

The Ten Commandments of Statistical Inference

As Handed Down to Lenski by Sir Ronald Fisher

1.  Remember the type II error, for therein is reflected the power if not the glory.

2.  Thou shalt not pseudoreplicate or otherwise worship false degrees of freedom.

3.  Respect the one-tailed test, for it can make thine inferences strong.

4.  Forget not the difference between fixed treatments and random effects.

5.  Thou shalt not commit unplanned comparisons without adjusting the rate of type I error for thy transgressions.

6.  Honor both thy parametric and thy nonparametric methods.

7.  Consider not the probability of a particular set of data, but rather the probability of all those sets as or more extreme than thine own.

8.  Thou shalt confuse neither manipulation and observation, nor causation and correlation.

9.  Thou shalt not presume statistical significance to be of scientific importance.

10.  Thou shalt not be fearful of paying homage to a Statistician or His Holy Book, especially before planning an experiment; neither shalt thou be fearful of ignoring the Word of a Statistician when it is damnable; for thou art alone responsible for thine acceptance or rejection of the hypothesis, be it ever so false or true.

The Golden Rule:  Review unto others as you would have them review unto you.

***

Notes:  I wrote this for a graduate course on quantitative methods in ecology and evolutionary biology that I taught in Spring, 1989, at UC-Irvine.  The course focused on experimental design and frequentist methods for drawing inferences.