On this page...
Preparing Your HLSP Collection
MAST staff will work with HLSP contributors to prepare their collection to be hosted in the Archive, and prepare the public-facing description that will help investigators use your data for new science. This article describes the process of preparing an HLSP collection for publication, and the respective responsibilities of the contributors and MAST staff. Actions the contributing team can take to facilitate the process are given in the last subsection.
Timeline
The process of preparing an HLSP collection involves several steps, and typically takes from several weeks to a few months to complete. For updates to already-existing HLSP products, the turn-around time will likely be much shorter.
The timeline is driven by a number of factors, including the number of data files and the complexity of the data collection, but mostly it depends upon how quickly the contributing team can respond to requests for fine-tuning the data products and for generating the descriptive material. Below is a schematic timeline for the various steps. Steps with a green border indicate an iterative feedback–response process; the duration of these steps depends to a large degree on how promptly the HLSP team responds to feedback requests. Note that responses from the MAST staff take place during business days, and exclude holidays.
Step 1: < 1 day
HLSP Team submits request form. MAST staff evaluate the request. The HLSP team may or may not have created products at this stage.
Step 2: 1 day
If approved, MAST staff send file upload instructions to the HLSP Team.
Step 3
HLSP team uploads a sample of each kind of product in their collection.
Step 4: ~1 month
MAST staff provide feedback on form, organization of content, and file naming to the HLSP team, who then revise products as necessary. The timeline is driven primarily by the number and complexity of the products.
Step 5: ~1 day
HLSP team should upload all data files plus a draft of the README. For extremely large datasets the upload could take considerably longer than a day.
Step 6: weeks to 2 months
MAST staff review data and metadata in detail, and provide feedback about any needed revisions. The review ensures that the products can be integrated with MAST infrastructure. The timeline is driven largely by the number and types of files in the collection.
Step 7: days to weeks
The HLSP team provides a description of the collection for potential science users, plus a compelling graphic. MAST staff will draft a web page and request feedback. The timeline is driven by the number of products in the collection and the timeliness of the HLSP team response.
Step 8: days to weeks
For very large collections (e.g., with millions of products) it can take days to more than a few weeks to ingest them into MAST. 1
Step 9
Announcements may be delayed if the HLSP team wishes to coordinate the release with some external event, such as a meeting or a press release.
The Fine Print
Recent experience shows that HLSP collections are typically published within 3 months of MAST receiving the initial sample products, though some collections have taken as long as 6 months. Note that the availability of scientists at MAST to support HLSPs varies throughout the year, and our capacity is dependent upon a number of factors. Certain times of the year, such as close to JWST/HST proposal deadlines and days between Thanksgiving and New Years Day, MAST capacity is limited. In addition, we tend to receive a larger number of requests to publish HLSP collections close to or during the winter AAS meeting, which may not be possible unless the collection is ready to publish some weeks beforehand.
The metadata we require in the contributor guidelines is intended to make your data widely searchable across the various MAST interfaces and Virtual Observatory tools. We try to update the guidelines regularly as we learn ways to make the process easier for both you and those of us at MAST.
Accelerating the Timeline
What can you do to help the process go as quickly as possible?
- Submit the contribution request form as soon as possible, even before you finish creating the intended products.
- Be sure that your data products conform to the MAST requirements for content
- Be sure all files are correctly named
- Be sure that all required metadata (keywords, for FITS files) are present and have accurate values
- Verify that the format of FITS files is valid (e.g., using fitsverify)
- Respond quickly to requests from MAST for feedback
For Further Reading...
- HLSP Required Contents
- HLSP Required Metadata