This article summarizes multiple options for retrieving data products of interest from MAST via the Portal.
On this page...
Context
Retrieving data from MAST generally requires having executed a successful search to identify Observations of interest (see Data Search). Often users select some specific products (see Data Selection) from the search results, and optionally add supplementary products, such as calibration reference files. What remains is to choose a method to download the selected data files.
Before attempting to download data, be aware of various issues related to web browsers: see Introduction to the Portal in the Portal Guide for details.
Download Options
The Portal offers multiple means to retrieve data, which are summarized briefly below, and described fully in the Retrieving Data chapter of the Portal Guide. Which method is best is largely a matter of personal preference, but some choices are better if you are only interested in select data products, or if the volume of data is large (which is often true for JWST datasets), or your internet connectivity is poor.
Direct Download
This method features one-click convenience for single Observations. However it can be tedious if your goal is to retrieve more than one Observation. The Observation will be streamed directly to your local system through your browser. See One-Click Download in the Portal Guide for details.
This method will retrieve only Minimum Recommended Products within an Observation. Use the Download Basket if other products are also desired.
Download Basket Bundle
This method offers the most flexibility for retrieving multiple products, even from different missions (e.g., including HST Observations). The selected files can be packaged as a tar or zip file, and streamed to your local system through the browser. See the Retrieval Methods article in the Portal Guide for details.
The data transfer speed for this method is not especially high, and can be very slow for large bundles. The size of the bundle is limited to no larger than about 50 GB. Use a curl script or stage the data for larger bundles.
Curl Script
This method is one of the options for transferring a download basket bundle. When selected, a bash script will be auto-generated and streamed to your local system. The downloaded script can be executed on your local system. It uses cURL commands (with authentication, when needed) to actually transfer files. This method is fairly fast, is not volume limited, and is robust against spotty internet connectivity. See the Curl Script explanation in the Portal Guide for details.
To retrieve data with this method before the Exclusive Access Period (EAP) expires you must have a valid Auth.MAST token, and either pass the token to the script on the command line, or set it in an environment variable.
Batch Retrieval
With this method, the selected files are staged on an ftp site at STScI. An email notice will contain the server name and the path to the directory where the data were staged. If you requested EAP data (while logged in to the Portal), you must log on to the ftp server using your MyST credentials. See Batch Retrieval in the Portal Guide for details.
Downloaded Data Products
All selected data products will be included in a download bundle (a zip or tar ball), provided you are authorized to retrieve them. The data products for each observation will appear in a directory structure similar to that displayed in the Portal Download Basket. See The Download Bundle in the Portal Guide for details.
For Further Reading...
See other retrieval methods using one of the APIs:
- Using MAST APIs
- Beyond The Portal article in the Portal Guide