Page History

Text modified revised from HST Astrometry Project Overview

...

The key to implementing improvements to the astrometry is the use of headerlets, self-contained FITS extensions containing a WCS transformation which can be attached to a FITS file and applied to the primary WCS. An observation can have multiple headerlets, each of which may have astrometry derived by differing methods. As HST data is processed/reprocessed, all available headerlets will be present as FITS extensions in the archived image with the best solution applied to the primary WCS.  A full description of headerlets and More details on how the WCS information is stored in headerlets may be found in the Headerlet Primer for the DrizzlePac softwareon the page Astrometry in Drizzled Products.

A database has been created to contain all the headerlets that is used by the xxxtask  to save and retrieve headerlets during pipeline processing. (Do we need this part?) This database is accessed using a restful web service as well as a simple web-form. Prior to enabling this database in the HST pipeline, it was prepopulated pre-populated with headerlets derived from:

...

Pipeline processing now includes a source finding step and to create additional a posteroiri solutions via cross-identification to with a limited set of external reference catalogs; namely, GAIA DR1 and DR2.  Additional Alternate astrometric catalogs (like Pan-STARRS, SDSS, 2MASS, GSC 2.4 etc) may eventually be added to the pipeline in order to obtain the best a posteriori solution WCS, but only after they have been determined to provide high quality alignment. In addition to loading this headerlet to the database, it will append all other available headerlets to the image before it is archived and available to users.

WCS Naming Conventions

Successfully aligning an observation to GAIA using the a posteriori processing will result in an update of the 'active' WCS of the image with the new solution and the new headerlet extension. This headerlet not only includes the WCS keywords which define the transformation from pixels to GAIA-aligned positions on the sky, but it also contains information about how this solution was derived along with the errors to be expected based on the fit. 

...

Images downloaded from the HST archive after reprocessing with the new Enhanced Pipeline Products code will already have headerlets added as extra extensions to the FITS file. The A new python notebook <insert LINK> has been developed to familiarize users with the structure of the new FITS images and to demonstrate how the primary WCS may be changed to any other preferred solution using the xxx 'apply_headerlet_as_primary' ? task. Alternately, any of the available headerlets may be downloaded from MAST/STScI and applied to existing data.If you run your own version of the pipeline it (?) should   For users who wish to manually reprocess existing data, the software will automatically connect to the database and retrieve/apply the headerlets.

→ LINK to JUPYTER NOTEBOOK (Varun)

Some suggested basic examples for looking at the data:

Future Improvements

While the majority of calibrated data products are now aligned to a common absolute reference frame, there are still some improvements in the alignment which will be made in the next data release.  For example, exposures in different filters that were obtained in the same visit may no longer be aligned to one another, for example, if each filter had a different number of matches to GAIA. Additionally, grism images may be now offset from their direct image counterparts, where only the later of which may be aligned to an external reference catalog. These refinements in the alignment process will be coming soon in the upcoming release for Hubble Advanced Products: Single Visit Mosaics. These new products will likely be able to improve the alignment for exposures which were obtained in the same visit, but which had guide star lock issues. They will also correct for small errors in the pointing of the telescope when very large dithers are commanded withing a visit (eg. half the detector FOV).

____Some suggested very basic examples for the Jupyter notebook___:

1.) Show the extra extensions____

from astropy.io import fits
fits.info('/internal/hladata/ENVS_OUTPUT/ALIGNDEV_12Oct19/popen-gw1/test_alignpipe_randomlist_J8C020/j8c041sdq_flc.fits')

No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 279 ()
1 SCI 1 ImageHDU 253 (4096, 2048) float32
2 ERR 1 ImageHDU 57 (4096, 2048) float32
3 DQ 1 ImageHDU 49 (4096, 2048) int16
4 SCI 2 ImageHDU 249 (4096, 2048) float32
5 ERR 2 ImageHDU 57 (4096, 2048) float32
6 DQ 2 ImageHDU 49 (4096, 2048) int16
7 WCSCORR 1 BinTableHDU 59 14R x 24C [40A, I, A, 24A, 24A, 24A, 24A, D, D, D, D, D, D, D, D, 24A, 24A, D, D, D, D, J, 40A, 128A]
8 HDRLET 1 NonstandardExtHDU 22 (60480,)
9 HDRLET 2 NonstandardExtHDU 26 (112320,)
10 HDRLET 3 NonstandardExtHDU 26 (112320,)
11 HDRLET 4 NonstandardExtHDU 26 (112320,)
12 HDRLET 5 NonstandardExtHDU 26 (112320,)
13 HDRLET 6 NonstandardExtHDU 26 (112320,)
14 WCSDVARR 1 ImageHDU 15 (64, 32) float32
15 WCSDVARR 2 ImageHDU 15 (64, 32) float32
16 D2IMARR 1 ImageHDU 15 (64, 32) float32
17 D2IMARR 2 ImageHDU 15 (64, 32) float32
18 WCSDVARR 3 ImageHDU 15 (64, 32) float32
19 WCSDVARR 4 ImageHDU 15 (64, 32) float32
20 D2IMARR 3 ImageHDU 15 (64, 32) float32
21 D2IMARR 4 ImageHDU 15 (64, 32) float32
22 HDRLET 7 NonstandardExtHDU 26 (112320,)
23 HDRLET 8 NonstandardExtHDU 26 (112320,)

___2.) Show the different headerlets and the corresponding WCSNAMEs.  Explain how to know which WCS is primary.

from stwcs.wcsutil.headerlet import headerlet_summary
headerlet_summary('/internal/hladata/ENVS_OUTPUT/ALIGNDEV_12Oct19/popen-gw1/test_alignpipe_randomlist_J8C020/j8c041sdq_flc.fits',columns='HDRNAME,WCSNAME')

EXTN              HDRNAME                                                   WCSNAME                            DESCRIP
8         j8c041sdq_flt_OPUS-hlet.fits                                 OPUS
9        OPUS2019-06-04                                                    IDC_0461802ej                     Default WCS from Pipeline Calibration
10       j8c041sdq_flt_OPUS-GSC240-hlet.fits                 OPUS-GSC240                     Guide Stars updated to GAIA coordinates GSC240                   
11        j8c041sdq_flt_IDC_0461802ej-GSC240-hlet.fits IDC_0461802ej-GSC240     Guide Stars updated to GAIA coordinates
12       j8c041sdq_flt_OPUS-HSC30-hlet.fits                    OPUS-HSC30                      Guide Stars updated to GAIA coordinates
13       j8c041sdq_flt_IDC_0461802ej-HSC30-hlet.fits    IDC_0461802ej-HSC30       Guide Stars updated to GAIA coordinates
22      IDC_0461802ej                                                          IDC_0461802ej
23      IDC_0461802ej-FIT_REL_GAIADR2                          IDC_0461802ej-FIT_REL_GAIADR2____

3.) Alternate way to get the keyword names. Is the last one always primary?

from stwcs.wcsutil import headerlet
headerlet.get_headerlet_kw_names('/internal/hladata/ENVS_OUTPUT/ALIGNDEV_12Oct19/popen-gw1/test_alignpipe_randomlist_J8C020/j8c041sdq_flc.fits',kw='WCSNAME')

['OPUS', 'IDC_0461802ej', 'OPUS-GSC240', 'IDC_0461802ej-GSC240', 'OPUS-HSC30', 'IDC_0461802ej-HSC30', 'IDC_0461802ej', 'IDC_0461802ej-FIT_REL_GAIADR2']

headerlet.get_headerlet_kw_names('/internal/hladata/ENVS_OUTPUT/ALIGNDEV_12Oct19/popen-gw1/test_alignpipe_randomlist_J8C020/j8c041051_drc.fits',kw='WCSNAME')

>This doesnt seem to work for the DRC image...