Key elements of MAST interfaces are summarized here, and include Portal tutorials for users new to MAST.

On this page...

Getting Started with MAST

Certain operations require that you log into MAST. This includes access to most data from active JWST observing programs. See MAST Accounts for details.

If you have a MyST account and login to MAST you may subscribe to notifications about JWST program data availability, even for observations that have not yet executed.

Exclusive Access Data

All data from JWST will eventually become public, and anyone may conduct searches for JWST data. However, when data fall under an exclusive access period (EAP) the files may only be retrieved by authorized persons. To retrieve EAP data from MAST:

You must have a MyST account. See this URL to create one: https://proper.stsci.edu/proper/authentication/auth
Login to MAST at any point prior to retrieving data. See MAST Accounts for details.
Those with MyST accounts must be authorized by the PI of the program to retrieve EAP data.
- PIs of observing programs may grant access to Co-Is (or anyone with a MyST account) by visiting MyST and selecting Registered Users→Update→Manage Access to Exclusive Access Science Data.

MAST Interfaces

Web Applications

MAST offers a multiple web-based tools to search for and retrieve science and engineering data, or for other information. Links to these tools appear in the table below.

Search Type	Select a collection...	URL
Observations of a target or field	MAST Observations by Object Name or RA/Dec	https://mast.stsci.edu/portal/Mashup/Clients/Mast/Portal.html
Observations Matching a Program ID	Observations by Program ID
Files matching Instrument Keyword Values	JWST Instrument Keywords
Wavefront Sensing products	JWST WSS
DOI Portal		https://mast.stsci.edu/portal/Mashup/Clients/DOI/DOIPortal.html
Calibrated Engineering Data		https://mast.stsci.edu/portal/Mashup/Clients/jwstedb/jwstedb.html
Keyword Dictionary		https://mast.stsci.edu/portal/Mashup/Clients/jwkeywords/index.html

Browser Issues

Note that the MAST Portal has incompatibilities with certain browsers and settings. In particular, pop-up blockers must grant an exception for the mast.stsci.edu domain. For details see Introduction to the MAST Portal in the Portal Guide.

API

Many users find scripted access to MAST searches and data retrievals to suit their needs better. The most popular interface offered is the Python package astroquery.mast. A brief tutorial is appears near the end of this article, and more extensive documentation is offered in the JWST Archive Manual in the Using MAST APIs chapter.

Portal Tutorials

The MAST Portal search panel is shown below. It is a starting point for your interactions with MAST.

Search results are displayed below the search panel, as shown in the example below.

Results consist of Filter dialogs (left); a table of matched results (center) showing rows for public (white background), Exclusive Access (yellow background, with the padlock icon), and planned but not yet executed observations (orange background); and the AstroView tool (right) which shows the spatial extent (colored footprints) of the exposures superimposed on an image of the sky. For details see Field Guide to the Portal.

Searches with the Portal

Many types of Portal search return Observations, which are really collections of all available data products, including uncalibrated, intermediate, fully calibrated, and combined science data. See Linkages in the Portal to learn more about how various kinds of data products are associated in the Portal. See Download Basket to learn how to select and download one or more of these types of products.

Target Name or Coordinates

A basic search will return Observations from all hosted MAST missions. This search will provide access to all levels of JWST data products.

	Instruction	Notes
1	Select 'MAST Observations by Object Name or RA/Dec' from the 'Select a collection…’ pull-down menu at the left of the search panel.
2	Enter a target name and (optionally) a search radius, which will be resolved to coordinates. See Basic Search for syntax rules. `m57 r=1m`
3	Click the Search button to the right of the target dialog.
4	Optional: Filter the results in the left panel by selecting Mission: `JWST` and the Instrument(s)/configuration(s). For example: `MIRI/IFU` and `NIRSPEC/IFU`
5	Evaluate the search results grid. It will look something like the example.

An advanced search allows you to customize the query by additional attributes, and can return Observations for all hosted MAST missions. This search will provide access for access to all levels of JWST data products (unless you choose to restrict the calibration level).

	Instruction	Notes
1	Select MAST Observations by Object Name or RA/Dec from the 'Select a collection…’ pull-down menu on the upper-left.
2	To search using more criteria than target name or position, click the Advanced Search link below the target dialog.
3	Search by multiple criteria with filter dialogs. In this case: Mission: `JWST` will greatly accelerate the query Instrument: MIRI* (i.e., all configurations of MIRI). Note typing the name of the instrument in the dialog box will accelerate the selection. Target Name: NGC6720*. Including the asterisk (wildcard) at the end will select the target name and variants, which may include offset positions.
4	Click the search button at upper-left.
5	The search results grid will look similar to the example. Select one or more Observations by ticking the box(es) at the left of the results table, then choose a download method.

Program ID

This type of search will return all Observations for a given observing program (a.k.a. proposal number). This is a quick way for Investigator Teams to search for all data related to their science program.

This search will match program IDs for both JWST and HST, even though the results for one mission are not scientifically related to the other. It is easy to filter the results table for the mission of interest.

	Instruction	Notes
1	Select MAST Observations by Proposal ID from the 'Select a collection…’ pull-down menu on the upper-left.
2	Enter one or more proposal numbers, separated by commas, then click the Search button.
3	Filter the results to select the common target `VV191`. (Note the variations in the spelling.)
4	The search results grid will look something like the example. Select one or more Observations by ticking the box(es) at the left of the results table, then choose a download method.

JWST Instrument Keywords

This type of search for JWST data provides a much larger set of criteria, including important metadata that are not available in standard searches such as:

the flag for time-series observations (tsovisit)
exposure type (exp_type, which could indicate, e.g., coronographic observations)
observation sequence identifier (observtn)
program category (category, e.g., COM and ERS where all data are instantly public)

Beware that this type of search matches data for only one selected JWST instrument, and only the highest-level calibrated science FITS files.

Highest-level products only!

This kind of search will not provide access to ancillary files, nor to uncalibrated (e.g., Level-1b) files if higher-level products exist. It will also not match calibrated files that are not in FITS format, such as source catalogs or light curves.

	Instruction	Notes
1	Select JWST Instrument Keywords from the 'Select a collection…’ pull-down menu on the upper-left, and select the instrument of interest from the pull-down menu below that.
2	Click the Advanced Search button.
3	Open filter dialogs of interest and enter search parameters. In this example we search COM and ERS programs for Single-Object Slitless Spectra taken in Time-series mode.
4	Click the Search button.
5	Optional: Filter the results table for Level-3 (fully calibrated and combined) data products.
6	The search results grid will look something like the example. Select one or more Observations by ticking the box(es) at the left of the results table, then choose a download method.

There is a way, using scripted queries, to search by values of instrument keywords and return Observations, that is, complete collections of all data products that match the query criteria. See API Tutorials for details.

Data Retrieval Options

There are multiple options for downloading data products from MAST. Each option has its strengths, as noted below.

Searches that return Observations to the results table contain links to all available JWST products associated with each Observation. Place one or more results in the Download Basket to view and select exactly which products (e.g., uncalibrated data) to download.

Direct Download

This is the simplest form of download, and works for individual entries in the results table. The download bundle is streamed to your machine through your browser.

MRP files only

This option does not download all available products, but rather the most highly calibrated and combined science products. It is similar to downloading from the basket when the Minimum Recommended Products option is selected.

	Instruction	Notes
1	Click the file icon in one of the rows in the results table.
2	After a moment a pop-up dialog may appear, asking if the file should be saved. Note that the files are zipped for transfer efficiency. Whether a dialog appears, and the specific wording in the dialog, depend upon browser settings. The location where the files will be written also depends upon your browser settings.

Download Basket

This form of download is intended for retrieving multiple observations at once, and for retrieving all (or only selected) types of products. There are three options: immediate streaming download (which imposes a size limit), and retrieval by curl script or ftp staging (which do not).

Streaming Download

This example applies to results that contain Observations, rather than individual files (as you would have after an Instrument Keyword search).

	Instruction	Notes
1	Select one or more rows in the results grid.
2	Click the basket button just above the grid to add the selected files to your download basket.
3	The Download Manager window may look something like that below. Note that the number of files associated with JWST Observations can be very large (nearly 7500 in this case).
4	The next task is to select which files to include in the download bundle, which consists of two steps:
4a	Recommended: use one or more filter dialogs to select types of products to include or exclude. Note that selecting Minimum Recommended Products will hide lower-level (uncalibrated) products from the file selector tree. Selecting `SCIENCE` and `INFO` categories will include all files necessary for re-running the calibration pipeline on your own machine. Not including the `AUXILIARY` category will have the effect of excluding guide-star files. This may greatly decrease the volume of data (7282 files in this example) Selecting the `UNCAL` group will include uncalibrated (Level 1-b) files. Your selection for the MRP checkbox will be remembered for subsequent downloads during your Portal session.
4b	Required: Select specific files or sub-directories of files to include. check the boxes of observations and/or files you wish to download. Note: checking the box of a parent observation automatically checks the boxes of all child files in the dataset. optional: click the triangle on the left to review the detailed contents of the sub-directory.
6	Optional: After selecting exposures of interest, click the Retrieve References button to bring up a dialog where you can choose one or more of all the calibration reference files that were used by the CAL pipeline to produce the selected data products.
7	Click the Download button.
8	Select the format for the download package from the pull-down, then click the Download but Optional: Check the "Remove completed files from basket" box. This will purge download basket, which may help avoid confusion the next time you prepare to download files using the basket.

Download via cURL Script

This type of retrieval is asynchronous and has no limits to the volume of data to download. It is more robust against internet interruptions, and can be resumed if interrupted.

	Instruction	Notes
1	Follow instructions 1–7 for Streaming Download (above).
2	Select Curl from the file Format pull-down menu.
3	Save the cURL shell script in a directory where you wish the data files to be stored. Whether a dialog appears, and the specific wording in the dialog, depend upon browser settings. The location where the files will be written also depends upon your browser settings.
4	Bring up a shell and execute the downloaded script. Note: If you are retrieving Exclusive Access Protected data, you will need to either provide your MyST login credentials, or have set an environment varialbe with your Auth.MAST token, in order for the shell to complete the retrieval successfully. The cURL script is targeted to a bash shell. Specifying bash on the command line allows it to be invoked from any unix shell.	bash MAST_2018-10-31T2028.sh

MAST Auth Tokens

Authentication for access to EAP data via a cURL or other scripts is managed in MAST via tokens. See MAST API Tokens to learn how to create or update a MAST.auth token.

Batch Download

Staging files for ftps retrieval is currently supported for a few MAST missions (including JWST and HST), but not all. It is highly recommended to instead retrieve large numbers of files via cURL scripts.

	Instruction	Notes
1	Follow instructions 1–6 for Immediate Download (above).
2	Click the Batch Retrieval button.
3	Select staging from the file Delivery Method pull-down menu and (if you are not logged in) provide the email address where you wish the notification to be sent. Note: If you are logged in to MAST, the e-mail address on record will be pre-filled.
4	Acknowledge the email pop-up notification, and check your email at the delivery address you specified.
5	The email notification will give the location of the files that have been staged.
6	Use a third-party client such as Cyberduck or wget to retrieve the files.

API Tutorial

The following tutorial introduces the basics of MAST queries with the astroquery.mast package. More extensive documentation is offered in the JWST Archive Manual in the Using MAST APIs chapter.

The set of all available parameters for this type of query is summarized here. Consider a search for JWST/NIRCam observations in observing program 1073, using the F277W filter. Use the .query_criteria() method.

matched_obs = Observations.query_criteria(
        obs_collection = 'JWST'
        , proposal_id = '1073'
        , instrument_name = 'Nircam'
        , filters = 'F277W'
        )

The result is an astropy Table object, with one row per matched observation.

If for a particular query you are not interested in observations from any mission except JWST, specify it with the parameter obs_collection='JWST'. This will narrow the list of possible matches considerably, and speed up your query.

Having found Observations that match your criteria, the next step is to fetch a table of data products associated with each Observation.

Some observations contain huge numbers of associated or linked files, sometimes in excess of 10,000 products. This is particularly true for NIRSpec MSA, or MIRI and NIRCam slitless spectroscopy, but may also be true for large mosaics of images. Each observation is likely to contain many files in common, such as guide-star files and ancillary products.

It is strongly recommended to retrieve product lists one or a few at a time from each observation to avoid server timeouts, and then to construct a set of unique products from the combined observations to avoid large numbers of duplicated products.

The following retrieves a list of tables of data products for each observation, and returns combined table containing unique data products.

if (matched_obs > 0):
    t = [Observations.get_product_list(obs) for obs in matched_obs]
    files = unique(vstack(t), keys='productFilename')

If at least one observation matched the search criteria the above call returns a table of unique products, one per row, which could number in the hundreds or thousands. You may wish to filter the results by masking all but a limited number of file suffixes and excluding certain sub-strings.

The selected products may now be downloaded to your local machine. Note that you will need to login with a valid Auth.MAST token to download exclusive access (EAP) data. If the token is needed but not supplied, the you will be prompted to enter one.

You may download all the products in the files table, or select a subset if you prefer. The following will select L-1b products (i.e. the raw, uncalibrated, *_uncal.fits) from the data product list for retrieval. Other types of products may instead be selected, either by matching product name sub-strings or selecting named products in the productSubGroupDescription.

manifest = Observations.download_products(
           files, 
           productSubGroupDescription='UNCAL',
           curl_flag=True
           )

Setting the optional curl_flag parameter to True will instead download a bash script that contains cURL commands to fetch the files at a later time. This approach is highly recommended for large numbers of files. The name of the download script will be something like: mastDownload_YYYYMMDDhhmmss.sh, where the latter part of the name is a numeric timestamp. What remains is to invoke the downloaded script on your machine to retrieve the files.

What's in the Box

All data products for all selected observations will be bundled together for delivery. When the Zip or tar file is unpacked, data for each observation/visit/exposure will appear in a separate sub-directory.

For each sub-directory, the data bundle includes by default the highest-level data products, plus all parent data. For example, if an observation/visit/exposure combination resulted in Level-2 data products, all Level-1 products would automatically be included unless the user explicitly chooses otherwise.

The Download Manifest

The zip (or tar) file will include a file called MANIFEST.HTML which lists each file name, a short description, and whether access is restricted. It will also note any files that could not be downloaded and the reason why (e.g., if you do not have permission to retrieve them).

File Names and Content

The semantic content of science files may be inferred from the file suffix, and the filename signature. For example, files ending in _uncal.fits are Level-1b uncalibrated science products. See the JWST Data Product Types documentation for details.

For Further Reading...

Reference Documents

MAST Portal Guide
JWST Archive Manual
JWST User Documentation Home (JDox)
JWST Pipeline Documentation (ReadTheDocs)
astroquery.mast documentation

User Support

Archive Help Desk: archive@stsci.edu
JWST Help Desk

Content

Space Tools

MAST Primer for JWST