Versions Compared

Old Version 4

changes.mady.by.user Jennifer Mack

Saved on

compared with

New Version 5

changes.mady.by.user Jennifer Mack

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The key to implementing improvements to the astrometry is the use of headerlets, self-contained FITS extensions containing a WCS transformation which can be attached to a FITS file and applied to the primary WCS. An observation can have multiple headerlets, each of which may have astrometry derived by differing methods. As HST data is processed/reprocessed, all available headerlets will be present as FITS extensions in the archived image with the best solution applied to the primary WCS.  A full description of headerlets and how the WCS information is stored may be found in the Headerlet Primer for the DrizzlePac software.

A database was has been created to contain all the headerlets that is used by the xxxtask ? task to save and retrieve headerlets during pipeline processing. This database is accessed using a restful web service as well as a simple web-form. Prior to enabling this database in the HST pipeline, it was pre-populated prepopulated with headerlets derived from:

  • The the original pipeline astrometry based on original guide star positions at the time of observation
  • a priori corrected astrometry based on updating the guide star positions to the GAIA DR1 reference frame
  • a posteriori corrected astrometry based on sources in the Hubble Legacy Archive (HLA) images fitted to the Hubble Source Catalog (HSC). These positions are primarily based on the Pan-STARRS catalog, which is on the GAIA reference frame but with larger errors. These are only available for a subset of the ACS and WFC3 images that were public prior to October 2017.

Pipeline processing now includes a source finding step and cross-identification to a limited set of external reference catalogs; namely, GAIA DR1 and GAIA DR2.  Additional astrometric catalogs (like Pan-STARRS, SDSS, 2MASS, GSC 2.4 etc) can may be added to the pipeline in order to obtain the best a posteriori solution, only after they have been determined to provide high quality alignment. In addition to loading this headerlet to the database, it will append all other available headerlets to the image before it gets is archived and available to users.

WCS

...

Naming Conventions

Successfully aligning an observation to GAIA using the a posteriori processing will result in an update of the 'active' WCS of the image with the new solution and the new headerlet extension. This headerlet not only includes the WCS keywords which define the transformation from pixels to GAIA-aligned positions on the sky, but it also contains information about how this solution was derived along with the errors to be expected based on the fit. 

The various The different WCS solutions are identified by the WCSNAME keyword found in each FITS headerlet and use the following naming convention:  wcsName = OriginalSolution - CorrectionType

...

and CorrectionType may have several forms

  •    GSC240 : Guide a priori WCS where guide star coordinates are corrected from the original reference frame (e.g. GSC1.1 or GSC2.3) to the GAIA DR1-based GSC2.4.0
  •    HSC30 : Image a priori WCS corrected from the original reference frame to the Hubble Source Catalog (HSC v3) frame which is based on GAIA DR1
  •    FIT-IMG-ReferenceCatalog : Image a posteriori WCS derived from matching to a reference catalog, where 'IMG' implies individual image catalog fits
  •    FIT-REL-ReferenceCatalog : Image a posteriori WCS derived from matching to a reference catalog, where 'REL' implies images were aligned to each other before a catalog alignment

...

WCSNAME

WCSTYPE

OPUS

‘distorted not aligned’

IDC_11d1433lj

‘undistorted not aligned'                                                                   

IDC_w3m18525i-HSC30

‘undistorted a priori solution based on HSC30’

IDC_q692007bj-GSC240

‘undistorted a priori solution based on GSC240'

IDC_0461802dj-FIT_REL_NONE

‘undistorted a posteriori solution relatively aligned to NONE’

IDC_w3m18525i-FIT_REL_GAIADR1

‘undistorted a posteriori solution relatively aligned to GAIADR1’

IDC_w3m18525i-FIT_REL_GAIADR2

‘undistorted a posteriori solution relatively aligned to GAIADR1’

IDC_0461802ej-FIT_IMG_NONE

‘undistorted a posteriori solution aligned image-by-image to NONE’

IDC_0461802ej-FIT_IMG_GAIADR1

‘undistorted a posteriori solution aligned image-by-image to GAIADR1’

IDC_0461802ej-FIT_IMG_GAIADR2

‘undistorted a posteriori solution aligned image-by-image to GAIADR2’

...

Usage 

(Still needs revision)

Images downloaded from the HST archive after reprocessing with the new Enhanced Pipeline Products code will already have headerlets added as extra extensions to the FITS file. The primary WCS may be changed to any other preferred solution using the xxx 'apply_headerlet_as_primary' ? task. Alternately, any of the available headerlets may be downloaded from MAST/STScI and applied to existing data.

If you run your own version of the pipeline it (?) should automatically connect to the database and retrieve/apply the headerlets.

...

→ LINK to JUPYTER NOTEBOOK (Varun)

Some suggested basic examples for looking at the data:

____

from astropy.io import fits
fits.info('/internal/hladata/ENVS_OUTPUT/ALIGNDEV_12Oct19/popen-gw1/test_alignpipe_randomlist_J8C020/j8c041sdq_flc.fits')

No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 279 ()
1 SCI 1 ImageHDU 253 (4096, 2048) float32
2 ERR 1 ImageHDU 57 (4096, 2048) float32
3 DQ 1 ImageHDU 49 (4096, 2048) int16
4 SCI 2 ImageHDU 249 (4096, 2048) float32
5 ERR 2 ImageHDU 57 (4096, 2048) float32
6 DQ 2 ImageHDU 49 (4096, 2048) int16
7 WCSCORR 1 BinTableHDU 59 14R x 24C [40A, I, A, 24A, 24A, 24A, 24A, D, D, D, D, D, D, D, D, 24A, 24A, D, D, D, D, J, 40A, 128A]
8 HDRLET 1 NonstandardExtHDU 22 (60480,)
9 HDRLET 2 NonstandardExtHDU 26 (112320,)
10 HDRLET 3 NonstandardExtHDU 26 (112320,)
11 HDRLET 4 NonstandardExtHDU 26 (112320,)
12 HDRLET 5 NonstandardExtHDU 26 (112320,)
13 HDRLET 6 NonstandardExtHDU 26 (112320,)
14 WCSDVARR 1 ImageHDU 15 (64, 32) float32
15 WCSDVARR 2 ImageHDU 15 (64, 32) float32
16 D2IMARR 1 ImageHDU 15 (64, 32) float32
17 D2IMARR 2 ImageHDU 15 (64, 32) float32
18 WCSDVARR 3 ImageHDU 15 (64, 32) float32
19 WCSDVARR 4 ImageHDU 15 (64, 32) float32
20 D2IMARR 3 ImageHDU 15 (64, 32) float32
21 D2IMARR 4 ImageHDU 15 (64, 32) float32
22 HDRLET 7 NonstandardExtHDU 26 (112320,)
23 HDRLET 8 NonstandardExtHDU 26 (112320,)

...

EXTN              HDRNAME                                                   WCSNAME                                                  WCSNAME                            DESCRIP
8         j8c041sdq_flt_OPUS-hlet.fits                                 OPUS
9        OPUS2019-06-04                                                    IDC_0461802ej 0461802ej                     Default WCS from Pipeline Calibration
10       j8c041sdq_flt_OPUS-GSC240-hlet.fits                 OPUS-GSC240 GSC240                     Guide Stars updated to GAIA coordinates
11        j8c041sdq_flt_IDC_0461802ej-GSC240-hlet.fits IDC_0461802ej-GSC240 GSC240     Guide Stars updated to GAIA coordinates
12       j8c041sdq_flt_OPUS-HSC30-hlet.fits                    OPUS-HSC30 HSC30                      Guide Stars updated to GAIA coordinates
13       j8c041sdq_flt_IDC_0461802ej-HSC30-hlet.fits    IDC_0461802ej-HSC30 HSC30       Guide Stars updated to GAIA coordinates
22      IDC_0461802ej                                                          IDC_0461802ej
23      IDC_0461802ej-FIT_REL_GAIADR2                          IDC_0461802ej-FIT_REL_GAIADR2

...

from stwcs.wcsutil import headerlet
headerlet.get_headerlet_kw_names('/internal/hladata/ENVS_OUTPUT/ALIGNDEV_12Oct19/popen-gw1/test_alignpipe_randomlist_J8C020/j8c041sdq_flc.fits',kw='WCSNAME')

...