On this page...
Download Manager Window
Once observations have been placed in the Portal Download Basket (see Marking Results), select which files to retrieve and the method of retrieval using the Download Manager window, as shown in the figure below. The workflow is:
- Use filters to select subsets of files
- Use checkboxes in the file selector to fine-tune the set of files for download
- Optionally click Retrieve References to include applicable calibration reference files in the download
- Select a download method from the Action Bar.
The Download Manager window is divided into 4 sections: the Action Bar (top), the Filters panel (left), the Files selector panel (middle), and a Details panel (right) to view information about each file.
Download History
The Portal preserves a history of your downloads; this history will persist across sessions if you are logged in. Click the Download History tab at the upper left to revisit downloads you have requested in the past. You may filter the selected files just as with the Download Basket window.
Filters
Only a few observations can result in hundreds of primary and associated data files in the download basket. Using one or more filters provide a quick way to exclude from view large numbers of files that may be of little interest to a user, and make it easier to select data products of interest. The filters are grouped on the left-hand side of the screen, and are described below.
Recommended Products
When selecting products for retrieval, by default only a subset of available products are exposed: those deemed to be most essential. These Minimum Recommended Products (MRP) are selected with a Filter checkbox. This selection:
- Exposes data products that are most relevant for science analysis
- Reduces the volume of (often large) files to be transferred to your machine.
Users who wish to select all available products may do so after un-checking the MRP box. This selection will be remembered for subsequent downloads during the same Portal session.
Data Product Category
Data products are categorized in the MAST database according to their intended use. The set of categories displayed depends somewhat on the specific mission.
| Category | Semantic Content |
|---|---|
| Auxiliary | Guide-Star products, processing logs, pre-images for target selection, Micro-shutter array metadata (for JWST), processing logs, telemetry files, etc. |
| Science | Science data products: the most calibrated* products (if the MRP box is checked), plus raw and some intermediate products (if the MRP box is un-checked) |
| Catalog | Source catalogs containing position, brightness, and other information. |
| Calibration | If calibration files have been added to the download basket for one or more observations, they will appear in this category. |
| Preview | These thumbnail images are mostly useful for quick visualizations of available products in MAST. They are generated for all FITS image-type science products. |
| Info | For JWST, these products are essential for users to re-process data on their own compute platforms, or for understanding in detail how high-level products were produced from contributing low-level products. |
*The term "most calibrated" means an endpoint of the calibration pipeline, exclusive of intermediate products.
Data Product Type
This filter distinguishes between primary science products and the calibration reference files that were used to produce one or more of them.
File Extensions
The file extension indicates the internal file organization and provides a clue about what type of software is necessary to understand the file format. Most types will be familiar to users:
| Extension | Description |
|---|---|
.fits | Scientific data products in FITS format. |
.jpg | Preview image, generally of a FITS data product, in JPEG format |
.png | Preview/thumbnail in PNG format |
.csv | Comma-separated variable structure in ASCII |
.ecsv | Enhanced CSV format (i.e., also contains metadata) |
.json | Structured ASCII data in JSON format |
Batch Download Availability
Files from certain missions (including HST and JWST) can be downloaded in batch (i.e., by staging to an ftp area for later retrieval). Files from other missions must be fetched with a different retrieval method.
Data Product Group
The file group indicates the semantic content of the file. It is a suffix to the filename and for most missions is preceded by an underscore, and followed by a period and the file extension. There are dozens of groups; the set of group names presented depends upon the mission, instrument, and the processing pipeline used to produce the data.
File Selector
The file selector presents only those data products in the download basket that match the filter selections (see above). The files are organized in a hierarchical directory structure where clicking the little triangles on the left exposes the contents of each directory. Click the check-boxes to select individual files, or to select all files within a given directory, as shown at left. Only selected files will be packaged for download.' Click a file name to display detailed information about that file.
File Details
Click any file name in the file selector to display details such as a brief description of the product, including the proposal/program ID, the version of the calibration pipeline used to produce the file, and the size in bytes.
Action Bar
The action bar displays the total number of files in the basket, as well as the approximate size of the selected files. Once at least one file is selected, the action bar includes buttons for:
- retrieving calibration reference files. This button is only enabled if the Minimum Recommended Products box is unchecked. Clicking it brings up another file panel to select applicable calibration files; then you must add them explicitly to your cart.
- choosing a method for downloading the selected files (see Retrieval Methods for details)
- an option for "batch" downloads (i.e., by staging them on an ftp server for later retrieval)
You can also remove some or all items from the basket.
Bundle Size Limit
File bundles larger than approximately 50 GB are too large to manage through a browser, and must be downloaded using the batch method or a bash (curl) script. Very large bundles typically transfer very slowly, so non-browser options are encouraged in these cases.