Inclusive Astronomy 2 Banner


A response to the community on IA2 participant selections

Please see our letter in response to some concerns communicated in the community with regards to the limited space and our participation selection process.

Inclusive Astronomy 2 Participant Selection Procedure

As an inclusivity conference, we understand the importance of transparency in our method for participant selection for Inclusive Astronomy 2 (IA2). This report, put together by the Organizing Committees of IA2, details how this was done, the options we considered, and the method we ultimately adopted to select people from a pool of pre-registrants. If you have any questions regarding anything in this document, please get in contact at inclusion2@stsci.edu.  For an overview of the numbers and demographic proportions of the pre-registrant pool and selected participants, please head to the tables at the end of this document.

Summary

In this report, we describe the methods of selection employed for filling the available spaces in IA2 that were not filled by those asked to give a talk or participate in a panel.  

Since this conference aims to discuss the status and path forward for all traditionally marginalized groups in astronomy, the goals of the selection method were the following:

  • Be as equitable as possible
  • Ensure that every decision made can be justified
  • Critically evaluate whether any of our own biases are informing a decision
  • Strive for multiple people of each minoritized identity to attend, to ensure that voices from multiple perspectives within that identity can be heard


We outline the limitations of the conference and the efforts made to address or negate these.  We briefly discuss the selection of talks/panel members. We describe how we chose the selection method we used and what considerations were made.  Finally, we describe the use of the Entrofy code to make our selections and give table summaries of the numbers and aggregate demographics of pre-registrants and selected participants.

Conference limitations: venue size and conference duration

The most significant factor in this selection process, and the reason we had to choose participants at all, was the limited space.  The venue is the 180-seat Bahcall Auditorium, the largest conference space at STScI. The space is available to the conference at no cost, and STScI provides event support staff, further reducing costs.  The auditorium is well matched to the number of participants in IA1. When it became clear that community interest is significantly larger than we had anticipated, it was, in effect, too late to change the venue given the anticipated budget.

Additional (or supplementary) venues in the area are also not available.  The STScI conference rooms are in a secured area, and spaces at Johns Hopkins are inaccessible to those with physical disabilities and those who use mobility aids. Additional venues within Baltimore were considered; however, booking fees and the lack of appropriate event infrastructure would have driven costs up and created a substantial financial barrier to participants.

We also considered delaying the conference until the spring or summer.  The intention behind scheduling the conference in October is to provide additional community momentum for addressing inclusivity in the Astro 2020 Decadal Survey.  While a larger space that can accommodate everyone is absolutely necessary for further conversations, we did not want to reschedule at the expense of this once-per-decade opportunity.  Instead, we hope that the incredible community enthusiasm for inclusion will support people who want to invest the time and energy to make IA3 happen as soon as possible.

Pre-registration

We adopted pre-registration for IA2 as it has been shown to enable a wider cross-section of people to apply for conferences than a first-come first-served method.  A total of 266 people, not including the organizing committees, pre-registered for the conference. In our pre-registration form, we asked pre-registrants to provide answers (where comfortable) to the following optional demographic questions.  The LOC recognizes the sensitivity of asking such questions, and these optional questions were intentionally vague so that this sensitive information was not easily personally identifiable once anonymized. Pre-registrants were informed that the information would be used for participant selection with Entrofy.  We will explain how we used the answers to these questions in a later section.  

  1. Country (and State, if US) 
  2. Do you self-identify as a person of color or otherwise a racial or ethnic minority? [Yes/No]
  3. Do you self-identify as a gender minority in astronomy? [Yes/No]
  4. Do you self-identify as trans and/or nonbinary? [Yes/No]
  5. Do you self-identify as a minority with respect to sexual orientation? [Yes/No]
  6. Do you self-identify as a minority with respect to disability, Deaf, and/or neurodiverse status? [Yes/No]
  7. Do you self-identify as a minority in astronomy with respect to a category we did not list above? If so, please describe here.  [No, Other:]
  8. What stage are you at in your career, with respect to your highest level of education? [Early (Student or 0-10 years out)/Middle (10-30 years out)/Senior (30+ years out)]


We realize in making these questions broad that we did not identify specific underrepresented and marginalized groups within each category covered by these questions.  We considered asking for more specific identity information (e.g., detailed race/ethnicity), however, we did not want people to feel exposed by these questions. Moreover, there are known issues associated with specificity, particularly related to non-binary gender (Rasmussen et al., 2019, https://arxiv.org/abs/1907.04893).  The broadness of the questions, we believe, circumvents the concerns of misuse expressed by the creators of Entrofy (Huppenkothen et al, 2019; https://arxiv.org/abs/1905.03314).  We also wanted to avoid anyone feeling required or pressured to give over potentially sensitive, personally-identifiable information in order to be selected.  We recognize that there is the potential for marginalized groups, within each minority area identified, to be underrepresented in the final selection as a result.  Although anonymization and protection of confidentiality was a key driver of the non-specific questions, we note that this is a possible limitation of the adopted method.  

Selection for talks/panels

The overall goals for the meeting are to first present the white-paper recommendations submitted to the State of the Profession for the Astro2020 Decadal Survey, and secondly revisit and renew recommendations to the community from the Nashville meeting. In preliminary planning, it was decided that the two-day duration of the meeting would support approximately 15 talks of 20-minute duration, with sufficient time for questions and discussion both immediately following presentations, and in post-session moderator-led discussions. 

The SOC focused on topics and discussions that highlight recommendations, implementable policies, and best practices that could have a large impact on the state of the field in the next decade. From the pool of 266 pre-registrants, a total of 92 abstracts were submitted having selected ‘talk or poster” from the submission choices. There were a few submissions of the same abstracts by two or more submitters; with such duplications only one of the submissions was (randomly) considered. Each submission was read and ranked by at least two SOC members. The SOC also considered if related talks could be combined to create panel discussions. Following a discussion on the top ranked submissions, the SOC chose a total of 21 speakers for 12 presentations and 2 panels.

Demographic data was not considered in the selection of talks, rather the content of what was to be presented. That said, 16 of the 21 invitees identified as minorities with respect to the questions provided. 

In response to concerns that the window of opportunity to submit abstracts was too short for some in the community, the organizers opened a call for late abstracts to select up to two additional talks for the meeting. That call will close on September 1, 2019. If selected, the additional talks will be accommodated by shortening some of the breaks and scheduled discussion time.

We recognize that there were many meaningful and significant equity and inclusion topics that were not selected. We could not hope to do justice to the entire topic of equity and inclusion in a two day conference, and we hope that future IA meetings will be organized to address other aspects of equity and inclusion.

Number of Attendees

The Bahcall Auditorium at STScI holds approximately 180 people. Approximately 30 spots are held in reserve for participation by the organizing committees, and for strategic audience members, which include the soon-to-be-announced members of the APC/State of the profession and societal impacts panel, and the steering committee for the decadal survey. The SOC selected 21 speakers and panelists for presentations. The LOC was tasked with filling 130 remaining spots; the use of Entrofy to select these participants is described in detail below.  

Why Entrofy?

Entrofy is an optimization tool developed to select participants for meetings and conferences, in a less-biased way than hand selection, according to a set of user-defined constraints.  The decision to use Entrofy as a method of participant selection for IA2 was motivated by its successful application to a number of past astronomy meetings and conferences. The code is flexible and can be used subject to the method chosen by each set of organizers.  The number and type of constraints is left to the user, and the target percentages of different constraints is user-defined. For these reasons of flexibility, repeatability, and past successful applications, we chose Entrofy for selecting participants for IA2. Links to more information on the code, our application of it, and the code we used for IA2 with a sample data set is provided at the end of this report.

Anonymization of data

To protect the privacy of those that pre-registered for the conference, only three of the LOC members had access to the full responses of pre-registrants. Responses to the pre-registration questions were associated with a unique numeric ID, which was used in communications with the LOC and in the selection via the Entrofy software.  Only the eight demographic questions were used for anonymized selection via the Entrofy software. The financial support information was only retained to keep track of how many pre-registrants would need support and what kind. The need for support was not used to make selections.  

Selection Criteria

In seeking to make the conference more affordable to early-career attendees, we decided upon a sliding scale of registration fee: $20 for early-career attendees and $170 for middle- and late-career attendees.  Therefore, to balance our budget, no more than 53% of our attendees could be in an early career stage. We recognize that by reducing the fee for early career attendees, we limit the number of early-career attendees that we can support.  However, we felt that a monetary barrier would make it difficult for the early-career attendees to be able to attend at all. This decision dictated how we were able to make further selections.

To maximize equitability, the LOC explored two options:

A) Accept all persons who self-identified as being a member of any of the minority groups identified on the pre-registration form.

B) Calculate the percentage of pre-registrants who self-identified as being a member of any of the minority groups on the pre-registration form (Q2-Q7).  Accept, at a minimum, the same percentage of participants as pre-registrants in these groups.

We found that Option A would be impossible to implement given the number of pre-registrants who self-identified as being a member of any of the minority groups listed on the pre-registration form.  170 people identified as having a gender identity that is minoritized in astronomy (Q3), which is, alone, more than the number of people we could admit through the selection process. 155 people answered “yes” to any of Q2, Q4, Q5, and Q6 (i.e., not including gender minority).  Of these 155 people, 70% identify as being in an early career stage. Due to the limitations previously mentioned on financial support of early-career participants, admitting all of these people would not be possible. However, even if we had opted to use a flat registration fee, choosing these 155 people without considering Q1, Q3, or Q8 would also be arbitrarily exclusive.

To evaluate Option B, we first compared the diversity of the pre-registrant pool to that of the US population.  The percentages of minority persons represented in each of the groups that we identified are larger than the reported percentages of these same groups in the US population.  That being said, we recognize that several of the identities can be invisible and that the recorded numbers for the US population are not accurate. Therefore, lacking any obvious independent criteria for representation, we determined that the applicant pool itself would be a good indicator of what our target fractions should be.  

We felt that Option B provided the best opportunity to maximize representation across all of the groups that we identified.  It also minimized potential biases that would arise from a more targeted ‘hand-picked’ selection of participants. Although, as a result of using Option B, some key community leaders were not identified by Entrofy.  This selection does mean, however, that new voices within each demographic group can have the opportunity to be part of the discussion. Future IA conferences may choose to approach this question differently. Our choice to include people from all demographic groups that pre-registered was so that a cross-section of the whole community could be present to listen to the voices that have self-identified as minorities.  This can help move this discussion forward and ensure this conference has further reaching impact.

Detailed Entrofy Selection Procedure

In this section, we describe every step we took from anonymized demographics to Entrofy-selected participants.

1. Go through “other” demographic question (Q7) and re-categorize if appropriate

Q7 above allowed pre-registrants to add additional ways in which they identify as a minority in the field.  We found that five pre-registrants answered this question by self-identifying as one of the minority groups we had already identified, so we used those responses to update the answers to the relevant question(s).  There were several other things mentioned by participants, such as first generation American (or immigrant in their country of origin) or first generation college student. Since we did not ask all pre-registrants to address these categories, we did not feel that it would be fair or equitable to use these added categories in the selection criteria.  Future conference organizers may wish to include these, and other, additional categories.

2. Remove members of LOC/SOC and consulting members

Our LOC membership changed during the course of our planning, so some people that had pre-registered for the conference ended up later joining the LOC as members or consulting members.  We did not include LOC/SOC demographics in our selection criteria, primarily because they would be difficult to truly anonymize, so we didn’t want to require members to disclose this information.  Therefore, LOC/SOC members were removed from the selection list.  

3. Remove duplicates 

Some pre-registrants sent in multiple applications, either because they applied to give more than one presentation or they edited their response.  These duplicates were removed to make sure the demographic fractions were accurate and so that no person would have a higher probability of being selected.

4. Set targets for each demographic question

For a given category, when setting the target fractions (e.g., for “Yes” and “No” answers) in Entrofy, the fractions can be set in two ways:

  • fractions add up to 1: We are aiming for these exact proportions.
  • fractions add up to <1: The values are the minimum fraction we want for a given category.  After the target fraction is reached, the rest are effectively randomly selected.


For questions Q2-Q6, we set the target of Entrofy selected participants within each minority group to be the percentage of pre-registrants that answered “yes” to that question (i.e.  64% of our pre-registrants identified as being a gender minority in astronomy, therefore we aim to have at least 64% of our selected participants identify as being a gender minority in astronomy).  We set a target of 0% for any “no” responses to these demographic questions (Q2-Q6).  As noted above, once the target is reached by Entrofy, the rest of the attendees are randomly selected, which means that the actual percentage of attendees that answered “yes” to each question will be larger than our target.  

For the following questions, we used a different method for setting our targets.

  • Country (and State, if US) 
  • What stage are you at in your career, with respect to your highest level of education? [Early (Student or 0-10 years out)/Middle (10-30 years out)/Senior (30+ years out)]


For location, we grouped the countries into US/Canada, Western Europe, and Other location.  We wanted to ensure geographic diversity at the conference given the small number of pre-registrants not located in US/Canada (3.75%).  We set a target at 2% for each of Western Europe and Other location, which was to the nearest percent of those that pre-registered within each of these categories.

In the case of career stage, we were limited in the number of people we could select from different career stages due to the registration fee differences and our grants to support attendees.  We received grant funding from STScI to support 20 early-career attendees and funding from AAS for 10 additional early-career attendees. Unfortunately, 69% of pre-registrants were from early-career stages and we determined that we could, at a maximum (with regards to budget and travel support grants), only support 85 participants in their early career.  This comes to 53% of 150 participants (including those allocated talks). We then used the relative percentage of middle and senior career pre-registrants, to adjust the targets for middle and senior careers groups accordingly. This is the only case where we set the targets for more than one answer to the same question.

Note on multiply-minoritized pre-registrants

We recognize that an individual can have multiple marginalized identities within the groups that we identified.  We note that this is why the percentages from the 3rd or 6th columns in Table 1 (see end of document) add up to larger than 100%.  We do not explicitly take these multiply-minoritized identities into consideration when setting our targets in Entrofy. However, as shown in Table 3, the selection process results in a higher fraction of participants with multiple-minority identities than in the original applicant pool.

5. Run Entrofy

We run Entrofy through a Jupyter notebook.  You can find the notebook that follows the method that we used to select participants from a simulated sample that has similar (though not identical) demographics to our actual sample, on our website at: https://outerspace.stsci.edu/display/IA2/Participant+Selection

To install the Entrofy code, head to the GitHub repository: https://github.com/dhuppenkothen/entrofy

You can read more about the Entrofy code in this publication (Huppenkothen et al. 2019): https://arxiv.org/pdf/1905.03314.pdf

Final Results of the Selection Procedure

Our final selection for the conference aims to focus on those that are often and traditionally marginalized in the field of astronomy.  Due to the subject of the conference, we found that the large majority of people that pre-registered for this conference identify themselves as part of the groups we aim to serve with the conference.  We therefore opted to use the demographics of the diverse pre-registrant pool as a starting point in an effort to reduce biases of the Organizing Committees in the participant selection process for ~130 remaining spots.

Please see Table 1 for the numbers of pre-registrant demographics and percentages, the targets set in Entrofy, and the selected participant demographic numbers and percentage breakdown.  Table 2 shows all of the pre-registrant and selected participant demographics that were not used in selection, but that resulted from the Entrofy selections.  

Table 3 gives insight into how people with multiple-minoritized identities were represented in our applicant and selected pools.  The proportion of people who did not identify as a minority along any of these axes, or did not answer these optional demographic questions, was 43 of 266 pre-registrants (16%).  The number that ended up being selected to participate in the conference as a result of the constraints outlined above was 12 of 152 selected participants (8%).  

Summary of Pre-Registrant and Selected Participant Demographics

Table 1: Categories that influenced selection.

Category

Number of pre-registrants

Percent of pre-registrants

Target percentage set in Entrofy

Number selected

Percent of selected

Total 

266

--

57%

152

--

POC, or otherwise a racial or ethnic minority

89

33.3%

33.3%

57

37.5%

Gender minority

171

64.0%

64.0%

116

76.3%

Trans and/or nonbinary

20

7.5%

7.5%

13

8.6%

Sexual orientation minority

73

27.3%

27.3%

50

32.9%

Deaf, neurodiverse, and/or person with a disability

44

16.5%

16.5%

31

20.4%

Career stage: early*

166

62.2%

53%

80

52.6%

Career stage: middle + late

79 + 20 

= 99

29.6% + 7.5% = 37.1%

38% + 9% 

= 47%

58 + 14 

= 72

38.2% + 9.2% 

= 47.4%

Location: Western Europe

4

1.5%

2%

3

2.0%

Location: outside US/Canada/Western Europe

6

2.25%

2%

4

2.6%

*This cap was set to ensure early-career participants could be financially supported by supplementing their costs with fees from the mid- to late-career participants and from grants received.

Table 2: Other categories that did not influence selection.

Category

Number of pre-registrants

Percent of pre-registrants 

Target percentage set in Entrofy

Number Selected

Percent of selected

Total 

266

--

57%

152

--

Career path: academic

194

73.0%

N/A

109

71.7%

Career path: non-academic

69

25.8%

N/A

41

27.0%

Childcare support

6

2.5%

N/A

3

2.0%

Childcare support (maybe)

9

3.4%

N/A

7

4.6%

Full financial support

45

16.9%

N/A

26

17.1%

Partial financial support

50

18.8%

N/A

28

18.4%

Require accommodations

10

3.8%

N/A

6

3.9%

Location: US/Canada

257

96.25%

N/A

145

95.4%


Table 3: Representation and minoritized identities summary.

Category

Number of pre-registrants

Percent of pre-registrants 

Target percentage set in Entrofy

Number Selected

Percent of selected

Did not identify with any minoritized identity (includes not answering demographic questions)

43

16.2%

N/A

12

7.9%

Identified with 1 minoritized identity

106

40.0%

N/A

61

40.1%

Identified with 2+ minoritized identities

117

44.0%

N/A

79

52.0%




Example Entrofy Notebooks

Simulated sample of data, generated for Entrofy selection for this conference.

Entrofy for IA2 - simulated sample.ipynb – a Jupyter notebook example of the Entrofy selection for this conference. 

If you have any questions regarding the participant selection prior to this report being distributed, please contact the committee (inclusion2@stsci.edu ).