This article summarizes multiple options for retrieving data products of interest from MAST via the Portal.

On this page...

Context

Retrieving data from MAST generally requires having executed a successful search to identify Observations of interest (see Data Search). Often users select  some specific products (see Data Selection) from the search results, and optionally add supplementary products, such as calibration reference files. What remains is to choose a method to download the selected data files.

Before attempting to download data, be aware of various issues related to web browsers: see Introduction to the Portal in the Portal Guide for details.

Download Options

The Portal offers multiple means to retrieve data, which are summarized briefly below, and described fully in the Retrieving Data chapter of the Portal Guide. Which method is best is largely a matter of personal preference, but some choices are better if you are only interested in select data products, or if the volume of data is large (which is often true for JWST datasets), or your internet connectivity is poor.

Direct Download

This method features one-click convenience for single Observations. However it can be tedious if your goal is to retrieve more than one Observation. The Observation will be streamed directly to your local system through your browser. See One-Click Download in the Portal Guide for details.

This method will retrieve only Minimum Recommended Products within an Observation. Use the Download Basket if other products are also desired.

Download Basket Bundle

This method offers the most flexibility for retrieving multiple products, even from different missions (e.g., including HST Observations). The selected files can be packaged as a tar or zip file, and streamed to your local system through the browser. See the Retrieval Methods article in the Portal Guide for details.

The data transfer speed for this method is not especially high, and can be very slow for large bundles. The size of the bundle is limited to no larger than about 50 GB. Use a curl script or stage the data for larger bundles.

Curl Script

This method is one of the options for transferring a download basket bundle. When selected, a bash script will be auto-generated and streamed to your local system. The downloaded script can be executed on your local system. It uses cURL commands (with authentication, when needed) to actually transfer files. This method is fairly fast, is not volume limited, and is robust against spotty internet connectivity. See the Curl Script explanation in the Portal Guide for details.

To retrieve data with this method before the Exclusive Access Period (EAP) expires you must have a valid Auth.MAST token, and either pass the token to the script on the command line, or set it in an environment variable.

Batch Retrieval

With this method, the selected files are staged on an ftp site at STScI. An email notice will contain the server name and the path to the directory where the data were staged. If you requested EAP data (while logged in to the Portal), you must log on to the ftp server using your MyST credentials. See Batch Retrieval in the Portal Guide for details.

Mac OS X users may need to install a third-party file-transfer application, such as Cyberduck or wget. If using CyberDuck, be sure to open your connection as FTP-SSL, since MAST now uses secure ftp (ftps). 

Downloaded Data Products

All selected data products will be included in a download bundle (a zip or tar ball), provided you are authorized to retrieve them. The data products for each observation will appear in a directory structure similar to that displayed in the Portal Download Basket. See The Download Bundle in the Portal Guide for details.

For Further Reading...

See other retrieval methods using one of the APIs:

Data Use | Acknowledgements | DOI | Privacy

Send comments & corrections on this MAST document to: archive@stsci.edu