Why are you moving to double-anonymous reviews?
STScI places a high value on the equity and integrity of the proposal review process. As with similar reviews, the goal is to give each reviewer an unbiased look at the proposal. Several studies have shown that a reviewer's attitude toward a submission may be affected, even unconsciously, by the identity of the lead author or principal investigator (see reference papers). We have noted that over the last 15 cycles, HST proposals led by women have had systematically and successively lower success rates than those lead by men (Reid 2014). While the exact cause is unknown, independent studies on our reviews suggest a double-anonymous process would resolve this inequity, and may balance out other areas of potential bias including affiliation and country of origin.
Are truly anonymous submissions even possible?
Even in our relatively small community, it's not as likely as one might believe that one would correctly guess the authorship of a proposal. While it is possible to correctly guess the authorship, studies from other fields suggest the principle investigator's identity would remain unknown 50% to 75% of the time. So, while a system that provided perfectly unbreakable anonymity would be ideal, our goal is simply to obscure identity, to discourage guessing, and to reduce unconscious bias, and not make authorship a focus of the discussion and evaluation of a proposal.
How will you know if the experiment was successful or not?
It is not correct to consider the move to a double-anonymous process an experiment. It is one in a progression of changes that have been enacted over the years in improve the equity and integrity of the proposal review process. We continuously evaluate the review process, with attention to fairness and balance over several factors, some of which are programmatic (e.g., are there more disproportionately more extragalactic programs than galactic?), and others demographic. We then make changes accordingly to address these issues.
How difficult will the changes be on proposers?
How do the reviewers assess the proposers' responsible use of the telescope, or likelihood of scientific return?
How do the reviewers assess the proposers' responsible use of funds that are allocated with each science program?
What will the implementation process be?
What will happen to proposals that are not sufficiently anonymized?
in the long-term? in the interim?
How do we deal with continuing programs?
demonstrating the knowledge of what's been done, discuss work in progress by the community or by reference. Justify the science each time they propose.
