Relationships Among Data Products
Many different data products, often hundreds or even thousands, contribute to the construction of a small set of high-level, most-combined JWST science products. They are all connected within MAST to one or more primary Observations, which are displayed as single rows in the MAST Portal results table. Users should be aware how all of these data products are related to the final Science products. This article describes the nature of these relationships; the article Linkages in the Portal provides a visual representation of how they are represented in the Portal user interface.
The term association is used within the JWST data management system to describe formally the connections among data that are essential in order to produce the highest-level products. See JWST Data Associations for details. Here are informal descriptions of the types of relationships that are included in associations:
The products in this relationship support the proper execution of the science observations on the spacecraft, or provide evidence that observations were executed as planned. They do not contribute to science data processing.
- Pre-image (if obtained as a part of the program)
- Confirmation image
The input products for a stage of the pipeline are related to output products by a parent-child relationship. That is, Level-1b products for a given program, observation, visit, detector, and exposure are antecedents for their corresponding Level-2a products; L-2a are antecedents for L-2b, and so on. (See the Science Data Products article for information about product levels.) In addition, the association definition files are themselves required to specify the output child products.
Level-2b products, or the output of Stage 2 processing, are the final, calibrated science products per-exposure. Further processing by the Stage-3 pipeline produces L-3 products per source or target by combining L-2b products in a way that depends on the instrument configuration. That is, L-3 products for a given source or target are composed of L-2b exposure-level products. Here are examples of types of exposures from a given instrument, filter/grating that are associated by composition to form L-3 products:
- All dither positions
- All detectors in a camera
- All mosaic tiles within or among observations in a single program
- All segments of a (lengthy) time-series observation
- All orientations (for coronagraphy)
See the section Types of Associations in the JWST Data Associations article for an exhaustive list. See the article Linkages in the Portal for a visual representation of associations as viewed from the Portal.
Some L-3 spectroscopic products are processed after combination to produce multiple derived products, e.g., one extracted spectrum per target.
Some types of calibration observations are obtained as a part of an observation, and near in time to specific science exposures. Contemporaneous calibrations are specific to the instrument configuration and to the specific target, and are used in science calibration processing:
- Arc lamp and flat-field exposures
- Astrophysical background exposures
- Reference PSF exposures in the same program for coronagraphy
Some anciliary data products are associated with the Observation, but are not used in the science processing to create the highest-level science products. These include:
- Intermediate products generated during calibration processing
- Guide-star products obtained during guided observations
Linked Data Products
Other products are also related to the science observations, though less directly than associated products. They are represented in MAST as linked data, and include:
- Contemporaneous guide-star observations
- Micro-shutter Array (MSA) Plan source catalog (NIRSpec only)
- Previews and image cut-outs generated to aid in data visualization within the Portal
- Links to data from other missions (applies to High-Level Science Products, but not currently to standard JWST products)
The following kinds of data products are neither associated nor linked within MAST, although they do contribute to constructing the highest-level products in an Observation. They can be accessed from the MAST Portal via special mechanisms.
Calibration Reference Files
Static calibration products are used during pipeline calibration processing to remove some element of the instrumental signature, or are used to specify how output products are to be created from input products. These products are updated periodically to reflect some improvement in the calibration, or to capture a temporal change in a calibration.
Engineering data are incorporated as header metadata in science data processing. The engineering data related to a specific exposure or observation are available in the Engineering Database interface.
Pre-images of a field from an external source (e.g., an HST program) may be contributed by an observer as an astrometric reference for a multi-object spectroscopic observation.
Minimum Recommended Data Products
Of the many different data products produced by the calibration pipeline, a subset has been identified as essential for extracting the intended science from the data. These are identified in the MAST Portal as "minimum recommended products" (MRP). The data products included in this set depends upon the the highest level product that is produced. Generally, products in the MRP include the most calibrated science products (i.e., those for which the instrumental signature has been removed, plus the combined products), but exclude anciliary products.
The MRP checkbox in the Download Manager must be de-selected in order to see and select raw or intermediate-level data products, and ancillary products, for retrieval. Note: Some L-2b products may also be appropriate for science analysis, but will not be visible if MRP is checked and L-3 products exist.