One of the main tasks of the TSO WG on FY2020 is to perform testing of the steps made by the JWST Calibration Pipeline for TSO-like data. This tracking sheet contains all the information regarding pipeline testing by the members of the TSO WG. Nestor Espinoza and Sarah Kendrew will be populating this page heavily in the weeks of July 20 and 27, 2020, for NIRISS and MIRI TSO testing, respectively. Nikolay Nikolov and Tony Keyes will be populating at their own pace for NIRCam and NIRSpec, respectively.
In general, if bugs are found, the best would be to discuss them first in one of our meetings before going ahead and submitting them (unless this is high-priority, in which case we could schedule a special meeting for this). This is left at the discretion of each TSO WG member.
Before jumping to read the below table, some important comments to keep in mind while reading them:
Pipeline step | NIRISS status | NIRISS comments | NIRCam status | NIRCam comments | MIRI status | MIRI comments | NIRSpec status | NIRSpec comments |
---|---|---|---|---|---|---|---|---|
group_scale | The step is supossed to do nothing for NIRISS/SOSS (here, a group = 1 frame). Indeed, nothing changes when running this step, except for the length of the ASDF element in the HDU list object, which is normal. | This step runs well with run command, but produces an error with .call: Error determining best reference for 'pars-groupscalestep' = Unknown reference type 'pars-groupscalestep' | N/A | |||||
dq_init | This step correctly initializes the data-quality flags, as it is supposed to do, according to the reference mask in CRDS. | |||||||
saturation | This step does what is expected to do: it captures saturated pixels and flags them in the GROUPDQ masks. The simulations used for this validation had saturated pixels and they were correctly identified (flagged) by the pipeline. | done for imager. only high level check for LRS. | ||||||
superbias | Tested my own bias substraction using reference files and got exact same results. | N/A | ||||||
reset | N/A | N/A | This was tested on a TSO imaging file separately since this step does not currently run by default. The test passed with no issues. | N/A | ||||
linearity | Tested linearity correction using reference files and got exact same results. | sufficiently covered by unit testing. | ||||||
rscd | N/A | N/A | Step currently flags frames in integrations higher than the first as 'DO_NOT_USE' in the groupdq array. This has been tested successfully. | N/A | ||||
dark_current | Dark frames perform expected corrections on the data. | sufficiently covered by unit testing | ||||||
refpix | Currently, does not do anything to NIRISS/SOSS simulations, as reference pixels there are set to zero. When injecting a fake strain of reference pixels got some deviations that most likely have to do with DMS-detector frame rotations. | This step is sufficiently tested in unit tests. It is skipped for MIRI subarray data so only applies to FULL array imaging TSOs, and note that MIRISim's implementation of Refpix is different from the pipeline step, so may give unexpected results. | ||||||
jump | Step produced no errors. However, currently detecting more jumps than it should — will revisit once some errors on the SOSS simulator are addressed (see below). | Runs without a crash, but identifies too many pixels as cosmic rays or bad with the default threshold of 4 sigma. Results improve if the threshold is increased to >50 sigma. NOTE: This step names files 'jump', i.e. without the root 'step'. This differs to all of the remaining steps in stage 1. | high level testing is done. Inserted cosmic rays of varying strengths were detected (based on threshold) and flagged neighboring pixels (based on level above background) | |||||
ramp_fitting | The pipeline was tested against an OLS ramp fit, and pipeline does exactly as intended. | produced rateints with 7.5 and independent linear fits; substantial difference in a subset of data. | Well covered by unit tests. May be a candidate for more detailed testing when time. | |||||
gain_scale | This step only applies to NIRSpec data. Because no GAINFACT is found in the headers, this step is properly skipped by the pipeline. | Documentation states 'This currently only applies to NIRSpec exposures that are read out using a subarray pattern, in which case a gain setting of 2 is used instead of the standard setting of 1.' This step is properly skipped for MIRI data. | ||||||
Extended comments by instrument branch:
Pipeline step | NIRISS status | NIRISS comments | NIRCam status | NIRCam comments | MIRI status | MIRI comments | NIRSpec status | NIRSpec comments |
---|---|---|---|---|---|---|---|---|
assign_wcs | Assign WCS indeed adds information to the ASDF name of the files. Tested that ranges of the WCS information made sense. | some loose ends to tie up | ||||||
extract_2d | This step is not supported for NIRISS/SOSS, and indeed running this steps shows the "not-supported" message. Still, saves an output equal to the input. | |||||||
flat_field | Flat field is correctly applied to NIRISS/SOSS data. | Build 7.5: Failed current validation test | passes for NRS2, but fails for NRS1: JP-1071 -NIRSpec: flat_field calspec2 step failing validation for FS ALLSLITS, BOTS, and MOS data OPEN | |||||
srctype | As stated in the documentation, TSO always defines SRCTYPE to POINT. The pipeline indeed does this for NIRISS/SOSS TSO data. | Build 7.5: Logic is not correct, always defaults to POINT, even when provided EXTENDED | JP-1576 - source_type logic in spec2 is not correct for NIRSpec FS and MOS modes OPEN | |||||
photom | Photometric calibration is performed according to the NIRISS reference files. | |||||||
extract_1d | This step currently extracts a big chunk of the whole subarray as a spectrum, which produced order overlap, which makes this step produce an unusable product for science right now. An algorithm is in development for this step. For details, see . | Build 7.5: methodology needs to be vetted by TSOWG | additional comments pending |
Extended comments by instrument branch:
Pipeline step | NIRISS status | NIRISS comments | NIRCam status | NIRCam comments | MIRI status | MIRI comments | NIRSpec status | NIRSpec comments |
---|---|---|---|---|---|---|---|---|
assign_wcs | N/A | N/A | ||||||
flat_field | N/A | N/A | ||||||
photom | N/A | N/A |
Extended comments by instrument branch:
Pipeline step | NIRISS status | NIRISS comments | NIRCam status | NIRCam comments | MIRI status | MIRI comments | NIRSpec status | NIRSpec comments |
---|---|---|---|---|---|---|---|---|
outlier_detection | Testing and validation done. Found some problems both with the documentation () and with the effectiveness of the algorithm itself. On this latter point, the algorithm marks as outliers points that are predominantly on the edges of the SOSS profile (). The algorithm also doesn't use time-series information at all, and only uses the median image to catch outliers (). In my opinion, algorithm currently not optimal for NIRISS/SOSS TSOs at least. | |||||||
tso_photometry | N/A | |||||||
extract_1d | This step currently extracts a big chunk of the whole subarray as a spectrum, which produced order overlap, which makes this step produce an unusable product for science right now. An algorithm is in development for this step. For details, see . | |||||||
white_light |
Extended comments by instrument branch: