A few years ago I started signing my manuscript reviews. My major motivation was to give the authors an opportunity to contact me to clarify what I meant or to ask how strongly I felt about statements without the delays of multiple rounds of review mediated by an editor. I figured that the positives for increasing the clarity, transparency, and speed of scientific communication would outweigh any potential negative side effects, such as “retribution”. Another positive was that I had to write each review thoroughly and fairly.

Despite some weird dynamics with editors, authors, and other reviewers throughout the years, I’ve found this to be a positive experience overall. But still had two nagging issues with peer review. First, I’ve had experiences where my reviews have been ignored after rejection at Journal #1 and papers have ultimately appeared in Journal #2 (or #3….) without ever responding to the substantive issues that I or other reviewers raised. Second, I was still bothered that we lose a tremendous amount of the written scientific record because reviews are only ever seen by the editors, authors, and reviewers themselves. This second issue is somewhat mitigated at journals like EMBO, Nature Communications, and eLife, which have more transparent review processes for accepted papers. And as part of ASAPbio, we are working on spreading this idea further in our initiatives on peer review

But, in the background of these larger initiatives and reflecting back on my own reasons for preprinting our lab’s work, I think there is more that we can do as individual scientists. Earlier this year, I decided to only review papers that have preprints associated with them so that I could post the reviews as comments on the preprints. I’ve openly reviewed 6 papers in the last year:

• Distinct metabolic states of a cell guide alternate fates of mutational buffering through altered proteostasis
• RNA tertiary structure energetics predicted by an ensemble model of the RNA double helix
• Stimulus-responsive Self-Assembly of Enzymatic Fractals by Computational Design
• Collection of continuous rotation MicroED Data from Ion Beam Milled Crystals of Any Size
• A microtubule RELION-based pipeline for cryo-EM image processing
• Identifying dynamic, partially occupied residues using anomalous scattering

By far the best interaction has been with Sagar Khare’s lab, where they responded to our review on BioRxiv (and through Nature Chemistry, which eventually invited us to write a News and Views on their accepted article). It’s also been instructive to see that sometimes the authors interpret a remark very differently than the way I intended it and we can clarify the point directly. I also like the idea that the authors can start thinking about our comments immediately, instead of waiting for all the reviewers and the editor to collate things together. Another side benefit is that I can work more efficiently: the proliferation of Elsevier Editorial System accounts overwhelms my ability to use a password manager and now I can just email the editors a link to my comment.

The weirdest interactions I’ve had has been with some journals who worry that I’m breaking the confidentiality of the review process (for example). The authors have already posted a preprint - so I’m not sure about my responsibility to keep their manuscript confidential as it is already public. Similarly, I’m confused as to whether posting a public comment on a public manuscript is the same as corresponding directly with the author (presumably the editors are worried about backroom deals, not public comments)?

Since adopting this stance, my reviewing load has been substantially lower. I’m considering just picking some preprints to review with lab members (even when not asked by a journal to review them) and posting our reviews. Hopefully the authors can incorporate the feedback and add it to their portfolio when they are seeking journal sanctioned peer review.

The etiquette of commenting on preprints is still developing. By posting manuscripts on a preprint server, are authors inviting the same type of scholarly feedback that now occurs behind closed doors? I’m operating under the assumption that authors would welcome this type of feedback. As a reviewer, posting our reviews gives us the same benefits that drew us to posting preprints in the first place: transparency, speed, and scientist-control of disclosure!

Finally, I recognize that I can do this because of my secure position at UCSF. We need mechanisms to help folks who aren’t in as secure a position to be able to contribute in an anonymous, but somehow credited way, as well. My hope is that by moving the scientific communication culture towards something that is more open and transparent, potential for retribution will be reduced and the need for anonymity will be decreased.