Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Pipeline stepNIRISS statusNIRISS commentsNIRCam statusNIRCam commentsMIRI statusMIRI commentsNIRSpec statusNIRSpec comments
assign_wcs

Status
colourGreen
titleDone

Assign WCS indeed adds information to the ASDF name of the files. Tested that ranges of the WCS information made sense.

Status
colourGreen
titleDONE

Runs without a crash and assigns WCS correctly.

Status
colourGreen
titleDONE

done

Status
colourGreen
titleDone

Build 7.5: Though we do not have a validation test, this is presumably ok since FS passes the test.

This test is not directly performed for BOTS since we do not have ESA intermediary products to compare against.  

Image AddedJP-1577 - Assign_wcs in spec 2 passes validation test OPEN

extract_2d

Status
colourGreen
titleDone

This step is not supported for NIRISS/SOSS, and indeed running this steps shows the "not-supported" message. Still, saves an output equal to the input.

Status
colourGreen
titledoNE

Runs without a crash and extracts 2d spectra.

Status
colourGreen
titleDONE

n/a

Status
colourGreen
titleDone

Build 7.5: Though at present we do not have a validation test, this is presumably ok since FS passes the test.

Image AddedJP-1578 - Extract_2d step in spec2 passes the validation test OPEN

flat_field

Status
colourGreen
titleDone

Flat field is correctly applied to NIRISS/SOSS data. 

Status
colourGreen
titledoNE

Runs without a crash and performs flat field correction.

Status
colourYellow
titleONGOING


Status
colourGreen
titleDone

Build 7.5: Validated/verified


JP-1071 -NIRSpec: flat_field calspec2 step failing validation for FS ALLSLITS, BOTS, and MOS data 

  • 31/Aug/20 BOTS component has been fixed and delivered


srctype

Status
colourGreen
titleDone

As stated in the documentation, TSO always defines SRCTYPE to POINT. The pipeline indeed does this for NIRISS/SOSS TSO data.

Status
colourGreen
titleDONE

Runs without a crash. SRCTYPE correctly set to POINT.

Status
titletodo


Status
colourGreen
titleDONE

Build 7.5: 

Validated/verified

Logic is now correct

JP-1576 - source_type logic in spec2 is not correct for NIRSpec FS and MOS modes  CLOSED

photom

Status
colourGreen
titleDone

Photometric calibration is performed according to the NIRISS reference files.

Status
colourGreen
titleDONE

Runs without a crash and performs as expected.

Status
colourYellow
titleONGOING


Status
colourGreen
titleDONE

Build 7.5: N/A 

tso_photometry is skipped for NIRSpec. 

Jul 2020: Logic for the output units was coded in NPTT and tested. All modes passed.

extract_1d

Status
colourRed
titleDone

This step currently extracts a big chunk of the whole subarray as a spectrum, which produced order overlap, which makes this step produce an unusable product for science right now. An algorithm is in development for this step. For details, see 

Jira
serverSTScI JIRA
serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
keyJP-201
.

Status
colourGreen
titleDONE

Runs without a crash and extracts spectra. Still need to enable column-by-column background subtraction with a Jira ticked issued 

Jira
serverSTScI JIRA
serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
keyJP-1478

Status
colourYellow
titleONGOING


Status
titletodo

Build 7.5:                methodology needs to be vetted by TSOWG

This step is currently crashing with BOTS data; additional comments pending

Extended comments by instrument branch:

...

Stage 3: Time-Series Observations (TSO) Processing testing

white_light
Pipeline stepNIRISS statusNIRISS commentsNIRCam statusNIRCam commentsMIRI statusMIRI commentsNIRSpec statusNIRSpec comments
outlier_detection

Status
colourGreen
titleDone

Testing and validation done. Found some problems both with the documentation (

Jira
serverSTScI JIRA
serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
keyJP-1647
) and with the effectiveness of the algorithm itself. On this latter point, the algorithm marks as outliers points that are predominantly on the edges of the SOSS profile (
Jira
serverSTScI JIRA
serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
keyJP-1654
). The algorithm also doesn't use time-series information at all, and only uses the median image to catch outliers (
Jira
serverSTScI JIRA
serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
keyJP-1655
). In my opinion, algorithm currently not optimal for NIRISS/SOSS TSOs at least.

Status
colourGreen
titleDONE


Status
titletodo


Status
titletodo

Build 7.5: Validation test needs to be written.  


tso_photometry

N/A


Status
colourGreen
titleDONE

Runs without a crash and performs as expected.

Status
titletodo

have successfully run TSO imaging simulations through Image2 and TSO3 but no notebooks yet for individual steps. high level checks looked good.

Status
colourGreen
titleDONE

N/A

tso_photometry is not performed by the pipeline for spectroscopy (NIRSpec). 
extract_1d

Status
colourRed
titleDone

This step currently extracts a big chunk of the whole subarray as a spectrum, which produced order overlap, which makes this step produce an unusable product for science right now. An algorithm is in development for this step. For details, see 

Jira
serverSTScI JIRA
serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
keyJP-201
.

Status
colourGreen
titleDONE

Runs without a crash and extracts spectra. Still need to enable column-by-column background subtraction with a Jira ticked issued 

Jira
serverSTScI JIRA
serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
keyJP-1478

Status
titletodo


Status
titletodo

Build 7.5: Validation test needs to be written

.  Sept 2019: For now, use the same test as for spec2.  

Image AddedJP-1728 - NIRSpec extract_1d testing OPEN

Image AddedJP-1729 - labeling of NIRSpec BOTS integrations OPENThe level 3 "x1dints" product for NIRSpec BOTS data contains separate extensions for each integration and detector obtained in the exposure.  Currently, there is no obvious way to distinguish between each extension in terms of its corresponding detector and integration, which is necessary in order for a user to be able to combine the data on both detectors for a given integration. 


white_light

Status
titletodo


Status
colourGreen
titleDONE

Runs without a crash. Jira ticket issued for defining min/max wavelength for white light curve: 

Jira
serverSTScI JIRA
serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
keyJP-1469

Status
titletodo


Status
titletodo

Build 7.5: 

Validation test needs to be written

Sept 2019: Need to design tests; start with verification


Extended comments by instrument branch:

...