Page History
...
| Pipeline step | NIRISS status | NIRISS comments | NIRCam status | NIRCam comments | MIRI status | MIRI comments | NIRSpec status | NIRSpec comments | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| group_scale |
| The step is supossed to do nothing for NIRISS/SOSS (here, a group = 1 frame). Indeed, nothing changes when running this step, except for the length of the ASDF element in the HDU list object, which is normal. |
| Runs well without a crash with the run() and call() commands for both imaging and spectroscopy TS. An CRDS - ERROR - Error determining best reference for 'pars-groupscalestep' = Unknown reference type 'pars-groupscalestep'. This seems to be a warning, rather than an error. | N/A |
| ||||||||||||||||||||||||||||
| dq_init |
| This step correctly initializes the data-quality flags, as it is supposed to do, according to the reference mask in CRDS. |
| Runs well without a crash with the run() and call() commands for both imaging and spectroscopy TS. The data quality flags are initialized correctly. This step has sufficiently been tested by unit testing. |
|
| ||||||||||||||||||||||||||||
| saturation |
| This step does what is expected to do: it captures saturated pixels and flags them in the GROUPDQ masks. The simulations used for this validation had saturated pixels and they were correctly identified (flagged) by the pipeline. |
| Runs well without a crash with the run() and call() commands for both imaging and spectroscopy TS. Saturated pixels are identified and flagged in the GQ masks correctly. This step has sufficiently been tested by unit testing. |
| done for imager. only high level check for LRS. |
| |||||||||||||||||||||||||||
| superbias |
| Tested my own bias substraction using reference files and got exact same results. |
| Runs well without a crash with the run() and call() commands for both imaging and spectroscopy TS. This step has sufficiently been tested by unit testing. | N/A |
| ||||||||||||||||||||||||||||
| reset | N/A | N/A |
| This was tested on a TSO imaging file separately since this step does not currently run by default. The test passed with no issues. | N/A | |||||||||||||||||||||||||||||
| linearity |
| Tested linearity correction using reference files and got exact same results. |
| Runs well without a crash with the run() and call() commands for both imaging and spectroscopy TS. The step performs what is expected and has sufficiently been tested by unit testing. |
| sufficiently covered by unit testing. |
| |||||||||||||||||||||||||||
| rscd | N/A | N/A |
| Step currently flags frames in integrations higher than the first as 'DO_NOT_USE' in the groupdq array. This has been tested successfully. | N/A | |||||||||||||||||||||||||||||
| dark_current |
| Dark frames perform expected corrections on the data. |
| Runs well without a crash with the run() and call() commands for both imaging and spectroscopy TS. Dark subtraction works well. |
| sufficiently covered by unit testing |
| |||||||||||||||||||||||||||
| refpix |
| Currently, does not do anything to NIRISS/SOSS simulations, as reference pixels there are set to zero. When injecting a fake strain of reference pixels got some deviations that most likely have to do with DMS-detector frame rotations. |
| Runs well without a crash with the run() and call() commands for both imaging and spectroscopy TS. The step performs what is supposed to do and has sufficiently been tested by unit testing. Still, I find this step to correct much less for subarrays (i.e. using half of the reference pixels) compared to a subtraction of the median value. This is not a suggestion to improve this for exoplanets, as the 1d cbc spectral extraction would do a better correction. Would be good for the imaging data though. |
| This step is sufficiently tested in unit tests. It is skipped for MIRI subarray data so only applies to FULL array imaging TSOs, and note that MIRISim's implementation of Refpix is different from the pipeline step, so may give unexpected results. |
| |||||||||||||||||||||||||||
| jump |
| Step produced no errors. However, currently detecting more jumps than it should — will revisit once some errors on the SOSS simulator are addressed (see below). |
| Runs without a crash, but identifies too many pixels as cosmic rays or bad with the default threshold of 4 sigma. Results improve if the threshold is increased to >50 >15 sigma. NOTE: This step names files 'jump', i.e. without the root 'step'. This differs to all of the remaining steps in stage 1. |
| high level testing is done. Inserted cosmic rays of varying strengths were detected (based on threshold) and flagged neighboring pixels (based on level above background) |
| |||||||||||||||||||||||||||
| ramp_fitting |
| The pipeline was tested against an OLS ramp fit, and pipeline does exactly as intended. |
| Runs well without a crash with the run() and call() commands for both imaging and spectroscopy TS. Ramp fitting (including OLS) has sufficiently been tested by unit testing and performs as expected. The run() command accepts weighting = "unweighted", despite the optimal is the only option and produces different results. The call() command has no options for weighting. |
| Well covered by unit tests. May be a candidate for more detailed testing when time. |
| |||||||||||||||||||||||||||
| gain_scale |
| This step only applies to NIRSpec data. Because no GAINFACT is found in the headers, this step is properly skipped by the pipeline. |
| Only applies to NIRSpec and skipped for NIRCam. |
| Documentation states 'This currently only applies to NIRSpec exposures that are read out using a subarray pattern, in which case a gain setting of 2 is used instead of the standard setting of 1.' This step is properly skipped for MIRI data. |
| |||||||||||||||||||||||||||
...
| Pipeline step | NIRISS status | NIRISS comments | NIRCam status | NIRCam comments | MIRI status | MIRI comments | NIRSpec status | NIRSpec comments | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| assign_wcs | N/A |
| Runs without a crash and assigns WCS correctly. |
| have successfully run imaging TSO simulations through Image2 and TSO3 but no notebooks yet for individual steps. high level checks looked good. | N/A | |||||||||||||||
| flat_field | N/A |
| Runs without a crash and performs flat field correction. |
| N/A | ||||||||||||||||
| photom | N/A |
| Runs without a crash and performs as expected. |
| N/A |
Extended comments by instrument branch:
...
| Pipeline step | NIRISS status | NIRISS comments | NIRCam status | NIRCam comments | MIRI status | MIRI comments | NIRSpec status | NIRSpec comments | ||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| outlier_detection |
| Testing and validation done. Found some problems both with the documentation (
|
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||
| tso_photometry | N/A |
| Runs without a crash and performs as expected. |
| have successfully run TSO imaging simulations through Image2 and TSO3 but no notebooks yet for individual steps. high level checks looked good. |
| ||||||||||||||||||||||||||||||||||||||||||||||||
| extract_1d |
| This step currently extracts a big chunk of the whole subarray as a spectrum, which produced order overlap, which makes this step produce an unusable product for science right now. An algorithm is in development for this step. For details, see
|
| Runs without a crash and extracts spectra. Still need to enable column-by-column background subtraction with a Jira ticked issued
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||
| white_light |
|
| Runs without a crash. Jira ticket issued for defining min/max wavelength for white light curve:
|
|
|
...
