On this page...

Contributing a New Collection

Hosting HLSP collections is a collaborative effort between the contributing science team and MAST staff. For carefully prepared data collections, contributing the ensemble to MAST is straightforward. Just follow these steps:

  1. Fill out the on-line Contribution Request Form for your project. A MAST staff member will contact you about your request.
    1. Consult with MAST staff early about the contribution process, ideally before the details of your collection have been finalized.
    2. Contributors who wish to deliver a new version of their collection, or who are adding to an existing collection, should not fill out the form again. Instead, see Updating an Existing Collection below, and contact MAST through the Archive Help Desk.
  2. Ensure that your HLSP collection contains all of the required files.
  3. Ensure that the science data files:
    1. are provided in an accepted format, and are well formed
    2. contain the required metadata
    3. are properly named
  4. Place your data products and supporting documents on an appropriate platform for electronic transfer to STScI.
    1. Your MAST contact will provide specific instructions for transferring the data.
    2. If electronic transfer is not possible for your collection, please consult your MAST contact or e-mail the Archive Help Desk for alternatives.
    3. (optional) Compress the files with a non-proprietary tool (e.g., gzip, bzip2, fpack) to minimize the transfer time to MAST
  5. Work with your MAST contact to resolve problems with the collection, if any.
  6. Prior to data publication, review for accuracy the new MAST web page that describes your project.

Contact MAST Early!

MAST staff will gladly help you formulate and prepare your HLSP collection. It is best to contact MAST as soon as possible if you think you want to contribute a science product collection. This will provide time to work with MAST staff to understand and comply with the contribution requirements, as well as time to prepare your collection for publication when your descriptive paper is accepted for publication. Many HLSP collections are added each year. If you wish to offer an HLSP collection to MAST, please contact MAST via email or the submission form at least 6 weeks in advance of the date you wish for the collection to be available in MAST.

Uploading Files to MAST

Please follow these instructions after you have worked with MAST staff to setup your upload directory.

You can use curl and the FTPS protocol to upload files to the MAST archive, as shown in the example below:

curl --ssl-reqd -u USERNAME
	 -T 'hlsp_{dogs.txt,cats.fits,birds-and-worms.fits}'
	 ftp://archive.stsci.edu/pub/hlsp/MY_DIR/

Note the following:

  1. We use --ssl-reqd to create an explicit FTPS connection. MAST no longer supports FTP connections.
  2. Your username is your email address. You will be prompted for your MyST password.
  3. Wildcard characters in filenames are not supported due to curl URL globbing. You'll have to input filenames as shown above, where the example will upload hlsp_dogs.txt, hlsp_cats.fits, and hlsp_birds-and-worms.fits in a single command.
  4. A MAST staff member will inform you of the value of 'MY_DIR' once they create the area for your HLSP. Generally, this is a a shortened name of your HLSP. The resulting destination URL should, as in the example, end with a "/". When uploading multiple files, this will fail without the ending "/".

Another option is to use an FTP Client, such as Cybderduck or Filezilla

To download the app or for more help, please see the Cyberduck site.

Launch the app and select Open Connection.

  • Under the FTP dropdown select FTP-SSL (Explicit AUTH TLS)
  • Server: archive.stsci.edu
  • Uncheck the 'Anonymous Login' option and enter your MyST login credentials (your full email address and password)

  • Select Connect

Cyberduck FTP Panel

You will see two folders: "pub" and "stage":

  • Pub is the location for uploading your HLSP
  • Stage is used for data retrieval and is not relevant to HLSP contributions

Under "Pub", navigate to the directory specified by MAST staff (e.g. /pub/hlsp/<your-HLSP-name>). You can then drag-and-drop files directly into the window, or use 'File → Upload' to select from your computer. 

Tools

A small set of tools is being developed to help contributors prepare and verify their collection contents. Until they are available, contributors may benefit from FITS tools in the astropy package, as well as the fitsverify tool from HEASARC, to validate FITS files against the v4.0 Standard.

Updating an Existing Collection

Depending upon the project, data may be delivered in stages, or re-delivered if, for instance:

  • new products become available, e.g., from
    • another observing program
    • the continuation of a large survey program
  • the software used to process and create the HLSP is revised and results in improved products

If you find that your HLSP data products or documentation need to be revised or supplemented, please contact MAST staff through the Archive Help Desk. Do not fill out a new request form.

Although MAST can discover literature references to papers related to HLSP products, it is helpful for the HLSP originators to provide updates to the list of refereed papers as they become available. Please email links to these papers to archive@stsci.edu.

Data Use | Acknowledgements | DOI | Privacy

Send comments & corrections on this MAST document to: archive@stsci.edu