Page History
| Excerpt | ||
|---|---|---|
| ||
Landing page for AWS download scripts |
| Panel | ||||||
|---|---|---|---|---|---|---|
| ||||||
To facilitate high demand during the first few weeks, MAST has copied all the pipeline-produced products to the Amazon cloud from both the "Webb First Images" (aka, "ERO") and the Commissioning programs. On this page...
|
Bulk Downloads from AWS
Copies of the early JWST data products have been hosted in Amazon Web Services (AWS). Using these scripts:
- Retrieves the data straight from Amazon cloud to your computer. This avoids all network traffic in/out of STScI, and is limited only by your own internet connection speed to AWS US East.
- Ensures you'll be able to download the released data products immediately, even while the files at MAST migrate from "exclusive access" to "anonymous access" over the first couple days.
The scripts only require that curl be installed on your machine. When you invoke a script, the data will be downloaded from Amazon cloud at no cost to you. You do not require an account of any kind to retrieve the data.
| Tip |
|---|
Each line in the script corresponds to a single data file. So if you want to make edits to the products you retrieve, you can simply delete or comment out (add '#' as the first character) lines for files you do not want to download. |
Three scripts are available for each program ID: "Calibrated" contain Stage 2+3 products, "Uncalibrated" contain Stage 0+1 products, and "Guide Star" contain guide star files. See https://jwst-pipeline.readthedocs.io/en/latest/jwst/data_products/product_types.html for a description of the products contained within each Stage.
| Warning | ||
|---|---|---|
| ||
The products linked here were produced by the standard calibration pipeline. At this early stage of the mission, with the current operational pipeline and available reference files, calibrated products may show a variety of known issues. Please check the JWST Calibration Pipeline Caveats (link not yet available) page in JDox for more information about limitations of the current data and suggested options. |
Table of Scripts - Webb First Images ("ERO" Programs)
| Anchor | ||||
|---|---|---|---|---|
|
| Info |
|---|
The Bulk Download Scripts links will be made available starting . |
The links in each row will download a script for a particular category of products related to a given Program ID. Execute the script on your own platform to retrieve the associated data from AWS. Note that these scripts will download all pipeline products for the program ID, even if the press release images used a subset of the observations within a program.
| JWST Program ID | Description | Bulk Download Scripts |
|---|---|---|
| 2731 | Calibrated | Uncalibrated | Guide Star | |
| 2732 | Calibrated | Uncalibrated | Guide Star | |
| 2733 | Calibrated | Uncalibrated | Guide Star | |
| 2734 | Calibrated | Uncalibrated | Guide Star | |
| 2736 | Calibrated | Uncalibrated | Guide Star |
Table of Scripts - Commissioning Programs
| Anchor | ||||
|---|---|---|---|---|
|
Many datasets were collected during the six months of commissioning, including some before the telescope was aligned or had reached operating temperature. Certain highlighted datasets represent those that the mission team have identified as being most useful to researchers, and are available at: JWST Commissioning Data Highlights. For completeness, we provide in the table below bulk download scripts for every program executed during commissioning.
| Info |
|---|
The Bulk Download Scripts links will be made available starting . The table will be augmented as programs become available. Prior to the release date, the table below is just a place-holder. |
The links in each row will download a script for a particular category of products related to a given Program ID. Execute the script on your own platform to retrieve the associated data from AWS.
| JWST Program ID | Bulk Download Scripts |
|---|---|
| Calibrated | Uncalibrated | Guide Star | |
| Calibrated | Uncalibrated | Guide Star | |
| Calibrated | Uncalibrated | Guide Star |