tocBased on the available literature, feedback from the community, and the discussions of the Working Group, it is our recommendation that the Institute move toward a dual-anonymous proposal process beginning with Cycle 26 HST in late 2018. We understand that a fully anonymous process requires active participation from community, and that there is notable apprehension as to what the effect of anonymizing will do to the scientific productivity of the observatory. We therefore recommend a phased approach, in which most of review is done anonymously with a sensibility check done at the very end of the review.
Guidelines and FAQ
- Proposer Guidelines in Anonymous Reviews
- Reviewer Guidelines in Anonymous Reviews
- FAQ on Anonymizing Proposal Reviews
Purpose of the Working Group
We're working on a plan for implementing anonymous proposal reviews beginning with the Cycle 26 HST TAC process. This includes,
- a plan for proposal reviewsreview and possible revision of the proposal process, from phase I submissions to TAC selection.
- instructions to proposers on how to write anonymous proposals
- instructions to the TAC , panels , and chairs on how to review anonymous proposals
- information to for the community on the issues with single-blinded singly anonymous peer reviews, and the solutions double-blinded dual anonymous reviews should address.
The document with our charge, Working Group on Anonymous Proposing v1.pdf
Chair: Lou Strolger (STScI)
Members: Peter Garnavich (Notre Dame), Stefanie Johnson (Leeds Business School, U. Colorado, Boulder), Mercedes Lopez- Morales (CfA, STUC), Andrea Prestwich (CfA), Christina Richey (JPL), Paule Sonnentrucker (STScI), Michael Strauss (Princeton), and Brian Williams (STScI)
Ex-officio: Tom Brown, Neill Reid (STScI)
The following presentations were given to the WGAPR:
- On the statistics on HST proposal success rates,
- (N. Reid)
- On Gender Bias in Hubble Proposal Ratings, Hubble Presentation.pptx (S. Johnson)
The HST Proposal Process
The HST Peer Review Information site has detailed information from the Science Policies Group on HST peer review, including more detailed some history on the evaluation of the review, more specific guidelines to reviewers, presentations provided at orientation on the observatory status and science activities, lists of previous panelists and chairs, and the proposal processing procedures.
Dual Anonymous Reviews
The STScI Chief Librarian, Jenny Novacescu, has complied a few articles on doubledual-blind anonymous peer reviews that should be have been useful in our discussions. They are available on https://stsci.box.com/s/ls9n3etdbcnwbukmdq3nclpzrlf7kx85
Before See the document Anonymous-Double Blind Review Annotated Bibliography.docx, but before diving into these articles, a place to start might be this article in Science Magazine. A question I had in reading this is if we had an optional anonymous submission option, as a compromise, would anyone use it?
In the document “Anonymous-Double Blind Review Annotated Bibliography.docx” Jenny has thankfully provided some crib notes on each of the articles.