Page History
| Excerpt | ||
|---|---|---|
| ||
Strategies for checking JWST proposals for duplicating observations. |
| Panel | ||||||
|---|---|---|---|---|---|---|
| ||||||
The JWST Duplicate Observations Policy requires proposers for new JWST observing time to justify observations that duplicate any existing or planned observations in other programs. This article describes tools and strategies for identifying and understanding potential duplications. It is the proposer's responsibility to determine if matching observations are genuine duplicates according to the policy. On this page...
|
Tools for Identifying Duplicate Observations
MAST provides tools to discover existing and planned JWST observations that match attributes of those you may be planning for an observing proposal. The information from these searches will help you determine if your observation would be a genuine duplication. This article describes the JWST Duplicate Observations Search, which is optimized for this purpose, and mentions other possible approaches. Later sections of this article summarize strategies for effective searches, as well as caveats that apply to the range of available tools.
JWST Duplicate Observation Search
This custom tool, shown in Fig. 1, allows you to compare the parameters of your intended observations with existing or planned observations in the JWST program database. Any matches the tool finds are potential duplicates that require some careful analysis to determine if they are genuine. Detailed comparisons are best done with the program specifications in the APT files of accepted programs with similar observations, but this can be a lengthy and tedious process. This tool will help you identify observations where a potential duplication exists.
| Tip |
|---|
Unlike most other MAST tools, this search does not return links to data. Rather, it returns metadata of executed or approved observations. |
The search interface (Fig. 1) is similar to other MAST Mission web applications. Use the boxes and pull-down menus to specify the parameters of potential observations, and then execute the search to see if your observations match any that exist in the JWST program database.
| borderColor | #00617E |
|---|---|
| bgColor | #E3EEF1 |
| borderWidth | 2 |
| borderStyle | solid |
Figure 1 — Interface for searching for JWST Observations that match the parameters of your intended observations. Many elements of the interface are similar to those in other MAST Mission search interfaces. Various elements of the interface (numbered boxes) are described in the text.
To use the tool, enter a fixed target by name or coordinates (box 1) and specify the search parameters (box 2) to align with your intended configuration. Then specify additional constraints (box 3). You may specify more or fewer output fields in the search results (box 4). See Moving Target Search for strategies to find observations of Solar System targets.
Search parameters
The constraint fields provide the means to specify the observing configuration that matches your intended observation.
Object/Coordinates: Specify the name or coordinates plus the search radius for a fixed target or field of interest, or upload a list of targets/coordinates. These elements and their function are described on the Target Name Search page.
Note that uploading a large list of targets (or specifying no target at all) could make the results table unwieldy if there are a lot of matches. In this case it can be helpful to add additional constrains using the Column Name/Condition boxes.
APT observation template: A compact way of specifying an instrument, its configuration, and data-taking mode. All templates are matched by default, but this pull-down menu allows for specific selections or combinations of selections.
Visit Status: One of: Completed, Failed, Flight_ready, Implementation. Scheduled, Skipped, Withdrawn, Note that Skipped and Failed visits may have been re-planned for a future date. Matches all statuses by default.
Column Name/Condition: Enables setting values or ranges on other field values. These elements and their function are described on the Additional Search Parameters page of the Mission Search Guide.
The
requires proposers for new JWST observing time to justify observations that duplicate any existing or planned observations in other programs. This article describes tools and strategies for identifying and understanding potential duplications. It is the proposer's responsibility to determine if matching observations are genuine duplicates according to the policy. On this page...
|
Tools for Identifying Duplicate Observations
MAST provides tools to discover existing and planned JWST observations that match attributes of those you may be planning for an observing proposal. The information from these searches will help you determine if your observation would be a genuine duplication. This article describes the
| Html-bobswift | ||
|---|---|---|
| ||
<a href="https://mast.stsci.edu/duplication/#/search" target="_blank" rel="noopener noreferrer">JWST Duplicate Observation Search</a> |
, which is optimized for this purpose, and mentions other possible approaches. Later sections of this article summarize strategies for effective searches, as well as caveats that apply to the range of available tools.
JWST Duplicate Observation Search
This
| Html-bobswift | ||
|---|---|---|
| ||
<a href="https://mast.stsci.edu/duplication/#/search" target="_blank" rel="noopener noreferrer">custom tool</a> |
, shown in Fig. 1, allows you to compare the parameters of your intended observations with existing or planned observations in the JWST program database. Any matches the tool finds are potential duplicates that require some careful analysis to determine if they are genuine. Detailed comparisons are best done with the program specifications in the APT files of accepted programs with similar observations, but this can be a lengthy and tedious process. This tool will help you identify observations where a potential duplication exists.
| Tip |
|---|
Unlike most other MAST tools, this search does not return links to data. Rather, it returns metadata of executed or approved observations. |
The search interface (Fig. 1) is similar to other MAST Mission web applications. Use the boxes and pull-down menus to specify the parameters of potential observations, and then execute the search to see if your observations match any that exist in the JWST program database.
| Panel | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
Figure 1 — Interface for searching for JWST Observations that match the parameters of your intended observations. Many elements of the interface are similar to those in other MAST Mission search interfaces. Various elements of the interface (numbered boxes) are described in the text. |
To use the tool, enter a fixed target by name or coordinates (box 1) and specify the search parameters (box 2) to align with your intended configuration. Then specify additional constraints (box 3). You may specify more or fewer output fields in the search results (box 4). See Moving Target Search for strategies to find observations of Solar System targets.
Search parameters
The constraint fields provide the means to specify the observing configuration that matches your intended observation.
Object/Coordinates: Specify the name or coordinates plus the search radius for a fixed target or field of interest, or upload a list of targets/coordinates. These elements and their function are described on the Target Name Search page.
Note that uploading a large list of targets (or specifying no target at all) could make the results table unwieldy if there are a lot of matches. In this case it can be helpful to add additional constrains using the Column Name/Condition boxes.
APT observation template: A compact way of specifying an instrument, its configuration, and data-taking mode. All templates are matched by default, but this pull-down menu allows for specific selections or combinations of selections.
Visit Status: One of: Completed, Failed, Flight_ready, Implementation. Scheduled, Skipped, Withdrawn, Note that Skipped and Failed visits may have been re-planned for a future date. Matches all statuses by default.
Column Name/Condition: Enables setting values or ranges on other field values. These elements and their function are described on the Additional Search Parameters page of the Mission Search Guide.
Results table
Upon executing a search, a table of results will be displayed (Fig. 2). The table rows are matches from the JWST program database to your search query parameters, and the fields (columns) contain values of metadata that pertain to each matched exposure. These metadata will help you assess whether each given match constitutes a genuine duplication of your intended observation.
| Note |
|---|
The individual table rows aggregate exposures that are identical in: Program ID, Observation, Visit, Target ID, Template, Optical elements, and Visit status The values of other columns not in the above list are taken from one of the aggregated rows. This includes the Region shape, which would correspond to only one of the aperture positions in a dithered or tiled sequence. |
| Panel | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
Figure 2 — Search results table, showing 8 selected rows (blue background) from a GO program. The rows may be sorted by clicking a field (i.e., table column) name. |
Elements of the interface, such as displayed table size and navigation, are similar to those in other MAST Mission search interfaces, except that there are no options to download data. The table (in its entirety, or only selected rows) may be downloaded for detailed analysis. The names and descriptions of the essential results fields are described below.
| Search Result Essential Field Descriptions | |
|---|---|
| Field(s) | Description |
| Dec Computed | Declination in decimal degrees at the specified epoch. Nominally in the International Celestial Reference System (ICRS). Blank when target_type is MOVING or GENERIC. Blank when primacy is PURE. |
| Dec Proper Motion | Declination component of proper motion in units of arcsec/year provided by the observer. Can be blank. Positional searches do not take into account proper motion, so increase search radius for targets with large proper motion. |
| Epoch | Date when the target was at the coordinates in ra_computed and dec_computed, provided by the observer. Can be blank if ra_proper_motion and dec_proper_motion are not specified. |
| Level 1, Level 2 | Top-level hierarchical identifier for a solar system target. Can be a standard name (for example, JUPITER), "COMET", or "ASTEROID". Only populated when target_type is MOVING. Second-level hierarchical description of a solar system target. Only standard names (for example EUROPA) are shown here. APT may show additional constraints in one of six Target Reference Systems. Only populated when target_type is MOVING. |
| Num Exposures | The number of exposures that were aggregated for that row, with a common Program ID, Observation, Visit, Target ID, Template, Optical Elements, and Visit Status. |
| Observation | Observation number in the range 1 to 999, specified by the observer in APT. This is a link to the corresponding Program Status Information page. |
| Optical Config | Instrument optical configuration. Depending on which APT template the observer used, the optical configuration encapsulates the filter, pupil, coronagraphic mask, and/or disperser. A forward slash separates different optical elements. |
| Program | Observing program identification number. Click the program number to visit the Program Information web site and retrieve the program in APT to get detailed information about a program. |
| Visit | Visit number within an observation. |
| RA Computed | Right ascension in decimal degrees at the specified epoch. Nominally in the International Celestial Reference System (ICRS). Blank when target_type is MOVING or GENERIC. Blank when primacy is PURE. |
| RA Proper Motion | Right ascension component of proper motion in units of arcsec/year provided by the observer. Can be blank. Positional searches do not take into account proper motion, so increase search radius for targets with large proper motion. |
Target Name, Standard Target Name | Convenient target name in the proposal provided by the observer. The Standard Target Name is the target name for the archive provided by the observer, ideally resolvable by a standard name resolver (e.g., NED, SIMBAD). Can be blank. |
| Target Type | FIXED for fixed targets, MOVING for solar system targets, GENERIC for generic targets that have not been specified explicitly (e.g., targets of opportunity), or GROUP for target groups. |
| Template | APT template that the observer used to specify the exposure(s). The 17 main science templates are
There are other calibration and engineering templates, some of which are not relevant for duplication checking. |
| Other fields are information-only. View the descriptions by hovering your mouse over the field name, as shown at left. |
Alternative tools
Other tools may also help you determine if your observation will duplicate. These include the MAST Portal and custom notebooks. Here is a comparison.
| Tool | Strengths | Weaknesses | |||||
|---|---|---|---|---|---|---|---|
|
|
| |||||
|
|
| |||||
|
|
| |||||
|
|
|
The most accurate means of determining if potential duplicates are genuine is to examine the detailed observation specifications (i.e., the APT file) for programs where a potential duplication is identified.
Search Strategies
The following strategies and tips will help make duplication checking more efficient.
- Limit the initial query to a particular fixed source or set of coordinates, but without specifying other criteria.
- If there are no matches, there may be no potential duplicates (except for moving targets)
- If there are too many matches, try searching for one target at a time
- Use a generous search radius (for fixed targets), at least as large as recommended in the Duplication checking and review procedures.
- Increase the search radius for small-aperture observations of high proper-motion targets
- Initially use the default of all templates, all program types, and all status values.
- If the search yields matching results, refine your search by selecting parameter values that match your observation in detail.
| Anchor | ||||
|---|---|---|---|---|
|
Moving object names are not resolvable through the object field. To search for solar system observations add a Condition, as in the following examples.
| Column Name | Condition | Result |
|---|---|---|
| Level 1 | Jupiter | Get observations of anything in the Jupiter system |
| Target Name | jup* | Get observations where one of the target names specified by the observer begins with jup |
| Target Type | MOVING | Get all moving target observations |
| Standard Target Name | *didymos* | Get observations of this small solar-system body |
| Tip | ||
|---|---|---|
| ||
For small bodies, the user-provided Target Name is often nonstandard. The Standard Target Name field, which is the name string used in the archive, follows more predictable naming conventions. Nonetheless, adding wildcard characters before and after the search string is recommended for the most reliable querying. See below for general guidelines on naming conventions. |
Caveats
There are limitations to identifying genuine duplications with search tools, particularly for special circumstances. See also the weakness listed in the Alternative Tools section.
Target names
It is possible to search MAST for individual sources by name, in one of three ways: Object Name, by Target Name, or (with the Duplicate Observation Search tool) the Standard Target Name.
- Object Name: MAST invokes an astrophysical name resolver (e.g., NED or Simbad) to look up source coordinates. Not all object names are recognized by resolvers. Also, different resolvers sometimes return substantially different coordinates; MAST will pick one of them (but not tell you which one). Sometimes the coordinates returned by the resolvers differ by substantial amounts, often because they refer to different sources.
- A classic case is the bright star μ Eri from the Bayer Greek letter system, and the star MU Eri in the general catalog of variable stars. In MAST,
- Searching for
* mu. eri(or just plainmu erimatches μ Eri (RA = 4:45:30.15, Dec = -3:15:16.777) - Searching for
V* mu erimatches MU Eri (RA = 2:48:10.566, Dec = -15:18:04.03)
- Searching for
- A classic case is the bright star μ Eri from the Bayer Greek letter system, and the star MU Eri in the general catalog of variable stars. In MAST,
- Target Name: This search matches to target names specified by Investigators in observing proposals to refer to sources. These are not guaranteed to match standard names for astrophysical sources, and often do not.
- Standard Target Name: This name is also provided by proposers and is intended to be a name that can be resolved to coordinates by SIMBAD, NED, or other services. The values are not independently vetted, and are often blank.
For small solar system objects (e.g., asteroids, comets, KBOs), the Standard Target Name field (case-insensitive) follows several naming conventions, which are listed below. For objects with multiple designations, it is recommended that users query all possible designations with wildcard characters using both the Target Name and Standard Target Name conditions.
- Named small bodies: <number> <name> (NOTE: Beginning in Cycle 2, the number is no longer included in the Standard Target Name for named asteroids. The use of wildcard characters is recommended (e.g., "*didymos*")
- Numbered small bodies: <number> (<provisional designation>) For provisional designations, a space is placed between the year and the first letter (e.g., "459865 (2013 XZ8)")
- Unnumbered small bodies: (<provisional designation>)
- Named comets: <comet name> (e.g., "Schwassman-Wachmann 1")
- Unnamed comets: <discoverer> (<provisional designation>) (e.g., "PANSTARRS (C/2014 OG392)")
Results table
Upon executing a search, a table of results will be displayed (Fig. 2). The table rows are matches from the JWST program database to your search query parameters, and the fields (columns) contain values of metadata that pertain to each matched exposure. These metadata will help you assess whether each given match constitutes a genuine duplication of your intended observation.
| Note |
|---|
The individual table rows aggregate exposures that are identical in: Program ID, Observation, Visit, Target ID, Template, Optical elements, and Visit status The values of other columns not in the above list are taken from one of the aggregated rows. This includes the Region shape, which would correspond to only one of the aperture positions in a dithered or tiled sequence. |
| borderColor | #00617E |
|---|---|
| bgColor | #E3EEF1 |
| borderWidth | 2 |
| borderStyle | solid |
Figure 2 — Search results table, showing 8 selected rows (blue background) from a GO program. The rows may be sorted by clicking a field (i.e., table column) name.
Elements of the interface, such as displayed table size and navigation, are similar to those in other MAST Mission search interfaces, except that there are no options to download data. The table (in its entirety, or only selected rows) may be downloaded for detailed analysis. The names and descriptions of the essential results fields are described below.
Top-level hierarchical identifier for a solar system target. Can be a standard name (for example, JUPITER), "COMET", or "ASTEROID". Only populated when target_type is MOVING.
Second-level hierarchical description of a solar system target. Only standard names (for example EUROPA) are shown here. APT may show additional constraints in one of six Target Reference Systems. Only populated when target_type is MOVING.
The number of exposures that were aggregated for that row, with a common Program ID, Observation, Visit, Target ID, Template, Optical Elements, and Visit Status.
Right ascension in decimal degrees at the specified epoch. Nominally in the International Celestial Reference System (ICRS). Blank when target_type is MOVING or GENERIC. Blank when primacy is PURE.
Right ascension component of proper motion in units of arcsec/year provided by the observer. Can be blank. Positional searches do not take into account proper motion, so increase search radius for targets with large proper motion.
Target Name,
Standard Target Name
APT template that the observer used to specify the exposure(s). The 17 main science templates are
- MIRI Coron, MIRI Imaging, MIRI LRS, MIRI MRS,
- NIRCam Coron, NIRCam Grism TS, NIRCam Imaging, NIRCam TS, NIRCam WFSS,
- NIRISS AMI, NIRISS Imaging, NIRISS SOSS, NIRISS WFSS,
- NIRSpec BOTS, NIRSpec FSS, NIRSpec IFU, and NIRSpec MOS.
There are other calibration and engineering templates, some of which are not relevant for duplication checking.
Alternative tools
Other tools may also help you determine if your observation will duplicate. These include the MAST Portal and custom notebooks. Here is a comparison.
- accesses metadata from the database of observation specifications
- option for specifying observing templates to ensure a precise match to instrument configuration and data-taking mode
- does not provide total exposure duration for an observation
- does not show footprints of matched results
- matches at the exposure level, so there may be many entries per single observation
- shows spatial footprints of observations against a sky survey background
- can show observations from other MAST missions for planning purposes
- displays correct exposure duration
- harder to match precise instrument configuration; searches by observing template are not possible
- additional work required to determine if matches to TSO observations are genuine duplications
- may not show observations that are being re-planned
- exposure duration for planned observations do not account for combined products
- no special accounting for targets with high proper motion
- shows a wide variety of observation metadata
- searches can include observing template
- does not show results for planned observations
- does not show footprints
- exposure duration is only useful for combined (L-3) products
- very efficient for querying against many targets at once, and analyzing the results
- does not show footprints of matched results
- additional work required to determine if matches to TSO observations are genuine duplications
The most accurate means of determining if potential duplicates are genuine is to examine the detailed observation specifications (i.e., the APT file) for programs where a potential duplication is identified.
Search Strategies
The following strategies and tips will help make duplication checking more efficient.
- Limit the initial query to a particular fixed source or set of coordinates, but without specifying other criteria.
- If there are no matches, there may be no potential duplicates (except for moving targets)
- If there are too many matches, try searching for one target at a time
- Use a generous search radius (for fixed targets), at least as large as recommended in the Duplication checking and review procedures.
- Increase the search radius for small-aperture observations of high proper-motion targets
- Initially use the default of all templates, all program types, and all status values.
- If the search yields matching results, refine your search by selecting parameter values that match your observation in detail.
Searching for moving targets is slightly more involved. To search for solar system observations add a Condition, as in the following examples.
jupCaveats
There are limitations to identifying genuine duplications with search tools, particularly for special circumstances. See also the weakness listed in the Alternative Tools section.
Target names
It is possible to search MAST for individual sources by name, in one of three ways: Object Name, by Target Name, or (with the Duplicate Observation Search tool) the Standard Target Name.
- Object Name: MAST invokes an astrophysical name resolver (e.g., NED or Simbad) to look up source coordinates. Not all object names are recognized by resolvers. Also, different resolvers sometimes return substantially different coordinates; MAST will pick one of them (but not tell you which one). Sometimes the coordinates returned by the resolvers differ by substantial amounts, often because they refer to different sources.
- A classic case is the bright star μ Eri from the Bayer Greek letter system, and the star MU Eri in the general catalog of variable stars. In MAST,
- Searching for
* mu. eri(or just plainmu erimatches μ Eri (RA = 4:45:30.15, Dec = -3:15:16.777) - Searching for
V* mu erimatches MU Eri (RA = 2:48:10.566, Dec = -15:18:04.03)
- Searching for
- A classic case is the bright star μ Eri from the Bayer Greek letter system, and the star MU Eri in the general catalog of variable stars. In MAST,
- Target Name: This search matches to target names specified by Investigators in observing proposals to refer to sources. These are not guaranteed to match standard names for astrophysical sources, and often do not.
- Standard Target Name: This name is also provided by proposers and is intended to be a name that can be resolved to coordinates by SIMBAD, NED, or other services. The values are not independently vetted, and are often blank.
Because of the limitations noted above, matching observations by target name Target Name alone may produce misleading results (false positives and false negatives), so it should not be the sole criterion for duplication checking. Entering coordinate pairs is the best approach to searching for matching fixed targets.
Targets with high proper motion
The fields containing the computed RA and Dec correspond to the celestial coordinates in ICRS (J2000) for fixed targets, or to the coordinates at the specified EPOCH if the proper motion is non-zero. But there are caveats to keep in mind:
- The computed coordinates for sources with significant proper motion will not (by construction) be the same as the ICRS coordinates of a resolved source name, nor of the coordinates of your future observation.
- The Region Shape field, which contains coordinate pairs that define the observation aperture, must be used with care. The region shapes correspond to the epoch of past observations, not the epoch of your future observation. Keep this in mind when deciding if there is spatial overlap.
- If you conduct a search with the MAST Portal to visualize the footprints (i.e., region shapes), be aware that the background survey, on which the footprints are superimposed, correspond roughly to the mean epoch of the sky survey: DSS (roughly 1990) and Pan-STARRS (roughly 2012.5).
Exposure duration
The JWST Duplicate Observations tools does not compute a total exposure duration. This is normally computed by the CAL pipeline, which knows the distinctions between, e.g., dithered vs. tiled images, dithered and nodded spectra, etc.