On this page...

Direct from Search Results

Retrieving data from MAST in the Portal context requires first executing a successful search (see the chapter on Searching). Each of the rows in the search results includes a File icon which enables a direct download of a single file or observation (see One-Click Download for details).

Using the Download Manager

In the Portal Download Manager you must check one of more directories or file boxes in the File panel before download methods become available to you (see Download Basket).

Portal Log-on

You must be logged in to the Portal to download data that is marked as Exclusive Access. Such observations appear with a yellow background in the Search Results table.

Browser Pop-ups

The Portal requires that pop-up blockers be disabled for the domain: https://mast.stsci.edu/ in order to download data. Check the documentation for your browser to determine how this is done, and be alert for notifications from the browser that popups are being blocked.  In many cases it simply requires giving approval for pop-ups when you are asked.

Options

Click one of the buttons described below to choose the details of your download.

Browser Download

Follow the steps below to initiate a download of the selected files. The files will be bundled into a .zip or a .tar (optionally compressed) file.


InstructionNotes
1Click the Download button.

Download products button

2Select your preferred form for the download bundle and click Download.

Download bundle format dialog

3Save the file bundle to your local storage. The appearance of the file-save dialog depends upon your browser settings.

Download to local disk dialog (browser specific)

Curl Script

You may elect to download a custom Bash script, which when run on your host system will retrieve your selected data files using cURL commands.


InstructionNotes
1Click the Download button.

Download products button

2Select Curl from the file Format pull-down menu.

Download bundle format dialog

3Save the Curl shell script in a directory where you wish the data files to be stored.

Download to local disk dialog (browser specific)

4

Bring up a shell and execute the script.

Note: If you are retrieving Exclusive Access Protected data, you will need to either provide your MyST login credentials, or have established a Auth.MAST token, in order for the shell to complete the retrieval successfully.

bash MAST_2018-10-31T2028.sh

Choosing Shippable Media

Requesting data via shippable media will involve considerable (i.e., days) delay. Note that you will need to certify that you are unable to use another means of data retrieval.

MAST Auth Tokens

Authentication for access to Exclusive Access data via a cURL or other scripts is managed in MAST via tokens. See MAST API Tokens to learn how to create or update a MAST.auth token.

Batch Retrieval

The Batch Retrieval option allows the selected files to be written to external media for subsequent retrieval. Follow the steps below to stage data on the archive.stsci.edu ftp server. Note that requesting data via shippable media will incur a substantial delay, and requires that you certify that you cannot retrieve data via staging.

Staging files for ftp retrieval is currently supported for a few MAST missions (including JWST and HST), but not all. It is highly recommended to instead retrieve large numbers of files via cURL.


InstructionNotes
1Click the Batch Retrieval button.

Batch retrieval button

2

Select Staging from the Delivery Method pull-down menu and (if you are not logged in) provide the email address where you wish the notification to be sent.

Note: If you are logged in to MAST, the e-mail address on record will be pre-filled.

Batch retrieval popup

3Acknowledge the pop-up notification, and check your messages at the delivery email address you specified.

Operation completed popup

4The email notification will give the location of the files that have been staged.

Delivery status request email message

5

Access the staging disk using an encrypted FTPS connection. See below for suggestions for modern, third-party clients.

The MAST FTP server (archive.stsci.edu) no longer supports unencrypted FTP connections. Command-line ftp clients, as in Linux or MacOS, may not work with our server.

Please see https://archive.stsci.edu/ftp.html for more information.


Retrieval via third-party clients

Please see the Archive Support Knowledge Base for more details on each of these clients, and to explore other options.

To download the app or for more help, please see the Cyberduck site.

Launch the app and select Open Connection.

  • Under the FTP dropdown select FTP-SSL (Explicit AUTH TLS)
  • Server: archive.stsci.edu
  • Select Anonymous Login OR enter your MyST login credentials

  • Select Connect

Cyberduck FTP Panel

You will see two folders: "pub" and "stage":

  • pub contains public data available on MAST
  • stage contains data that have staged by MAST users. It is purged periodically.

Under "stage", navigate to the directory specified in the email notification and you should see the folder(s) containing your download.

  • To download the entire directory highlight the folder, then under Action on the menu bar select Download.
  • If you only want a few files from the directory, expand the desired directory, and then select the files you want to download.

To download the app or for more help, please see the FileZilla site.

Launch the app.

In the Host field enter ftps://archive.stsci.edu, then your MyST email address and password in the Username and Password fields, respectively. Select Quickconnect.

  • If TLS connection is not set as the default, Quickconnect may be unsuccessful. If so, navigate to the Site Manager.
  • Select New Site

  • Protocol: FTP - File Transfer Protocol

  • Host: ftps://archive.stsci.edu
  • Encryption: Require explicit FTP over TLS

  • Logon Type: Select Anonymous OR enter your MyST credentials

  • Select Connect

Filezilla FTP Panel

Under Remote site, you will see two folders: "pub" and "stage"

  • Pub is all of the public data available on MAST
  • Stage is the data that has been added to the staging area

Under "stage", navigate to the directory specified in the email notification and you should see the folder(s) containing your download.

Once you locate the folder(s) or file(s) you want to download, right click and select Download.

To install wget or for more help, please see the wget site.

Bring up a shell.

If you requested a retrieval anonymously, run the following command in the shell:

wget -r --ftp-user='anonymous' --ask-password ftps://archive.stsci.edu/stage/anonymous/<anonymous requestID>/

If you were logged into MAST when you requested that your data be staged, run:

wget -r --ftp-user='<your full MyST email>' --ask-password ftps://archive.stsci.edu/stage/<userID>/<requestID>/

Note, for users internal to STScI there may still be a certificate problem; if you encounter an error add '--no-check-certificate'  to the command.

To install curl or for more help, please see the curl site. Curl is likely already installed on many machines, and if not, is freely available as open source software.


Bring up a shell.

If you requested a retrieval anonymously, run:

curl --ssl-reqd -O ftp://archive.stsci.edu/stage/anonymous/<anonymous requestID>/<filename>

If you were logged into MAST when you requested that your data be staged, run:

curl --ssl-reqd -O ftp://archive.stsci.edu/stage/<userID>/<requestID>/<filename> -u '<your full MyST email>'

Note that you must include the filename in these commands. If you do not know the name of the file you'd like to retrieve, or want to download an entire directory, we recommend wget or one of the other methods detailed here.

The MAST FTP server (archive.stsci.edu) no longer supports unencrypted FTP connections. Command-line ftp clients may not work with our server.

Bring up a shell (see note below) and initiate an ftp session on archive.stsci.edu.

% ftp archive.stsci.edu
Connected to archive.stsci.edu

The appropriate login credentials will be one of the following:

  • If you were logged into MAST when you requested that your data be staged, use your MyST login credentials.

  • If you requested a retrieval anonymously (for non-exclusive access data), use:
    • Name: anonymous
    • Password: <your full MyST email address>
Name: anonymous
331 Guest login ok, type your email address as
the password
Password:

Navigate to the directory specified in the email notification and initiate the transfer of data files.

ftp> cd /stage/anonymous/<anonymous requestID>
# turn off interactive prompt:
ftp> prompt
ftp> mget *

MAC OS X 10.14+ Users

Apple has removed the native ftp application for Mac OS 10.14 (Mojave) and higher. A solution is to install a third-party file-transfer application, such as Cyberduck or wget.

Other Download Methods

It is possible to search for and retrieve data using MAST web services, or the Python astroquery.mast library. These methods are described in Beyond the Portal.

For Further Reading...

Send comments & corrections on this MAST document to: archive@stsci.edu