Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DraftMASTDOCS and version 23.04
Excerpt
hiddentrue
Learn which file types and formats are accepted
Panel
borderColor#00617E
borderWidth2px
borderStylesolid

On this page...

Table of Contents
maxLevel2
indent24px

Accepted File and Content Types

The types and formats of products that are accepted for HLSP collections are given in the following tables. Note that science products in FITS format should conform to the latest version of the Standard, and may have multiple extensions (MEF).

Science Products

ProductFormatsData OrganizationNotes
ImageFITSSimple, or MEF with one or more IMAGE extensionsOne MEF file should include science and concomitant pixel data, in separate extensions (variance, data quality, exposure map, etc.), where appropriate.
Spectra: 2-D or higherFITSSimple, or MEF with one or more IMAGE extensionsOne MEF file should include science and concomitant pixel data, in separate extensions (variance, data quality, etc.), where appropriate. One-dimensional specta contained in images must express the WCS using standard header keywords.
Spectra: 1-DSimple, or MEF with one or more IMAGE or BINTABLE extensions
CatalogFITS, csv, DB dumpBINTABLE (FITS), or ASCII (csv, DB dump)CSV files should be parsable with common software. Database dumps from e.g. MySQL or PostgresSQL may be acceptable; SQLite files are certainly acceptable.
Light curveFITSBINTABLETime coordinate information may be represented in keywords or as an explicit table column.
Model or Simulation

Consult with MAST staff. Simulations of observational data would ordinarily be represented in the format appropriate for the product being simulated.

Ancillary Products

ProductFormatsData OrganizationNotes
README

ASCII, HTML,markdown,sphinx

Single flat file, or file with markup; see Example README file, or a simpler ASCII README fileDocument that describes the collection contents. It must provide a description of the semantic content and organization of data in the files, and bibliographic information for the published paper(s) that describes the creation and use of the collection. Put details of the science goals, data processing methodology, etc. in the Project Summary file.
Manifestjson, yaml, csv

Structured; see example file in yaml format: MANIFEST.yml

Document that describes the product delivery manifest. It must provide the organization of data among the files. Details of the science goals, data processing methodology, etc. must be included in the Project Summary file.
GraphicsPDF, PostScript, gif, jpeg, png
Plots and illustrations to be used as previews for image, light curve, spectra, atlases, (possibly) catalogs. Also appropriate for elements of the project description.
AnimationMP4, WebM
Appropriate for some data product previews, particularly for models or simulations.
Project SummaryOutline-oriented structure: ASCII, HTML, sphinx
One or two paragraph description of the science aims of the data collection, for incorporation into a web site for your collection. May also include a description of the observing program (including sky coverage). Must include literature reference(s) to the methodology used to create the HLSPs. For small HLSP collections this content may instead be folded into the README file.

For all data files be sure to:

For science data files be sure to:

  • include the required HLSP metadata (i.e., keywords) in the headers

Non-Accepted File and Content Types

Proprietary Formats and Certain Content Types

Content that cannot be archived includes: publications, project tar files, or any files in a proprietary format (e.g., Microsoft Office).

Software

It can be useful for contributors to associate software with their data products. Reasons include:

  • Documenting the processing software
  • Visualizing the data products
  • Providing an analysis tool or Jupyter notebook that is tuned to the HLSP collection
  • Providing back-end services that operate on the collection data

MAST embraces the idea of associating software with data, but does not accept or support source code.

Tip

The staff at MAST encourage contributors to use third-party software repositories such as GitHub, and to register your software with the Astrophysics Source Code Library (ASCL). A link to your software repository can be included on the MAST website, and the link will be added to our master list of astronomical software that is tied to HLSP collections.

Please note: links to a repository must point to a version of the resource (software, notebook, or documentation) that corresponds to the delivered version of the HLSP data products. 

Only very limited support is presently available for back-end (analytical) services connected to HLSP collections, but more support is expected in the future.

Certain FITS File Organization

FITS is a fairly general format, and allows for multiple ways to organize data within a single file. However, some potential choices of data organization within FITS files have not seen wide use and are presently not supported for HLSP collections, including:

  • Binary tables containing images (any dimensionality) within table cells
    • Table cells may contain simple 1-D arrays, however
  • ASCII table extensions (however, BINTABLE extensions are supported)