You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

The JWST (STScI) Pipeline, and in particular, the steps used by its TSO-mode are in constant development. In this page, we will keep track of the proposed enhancements to the pipeline by the TSO WG. Status of each of the proposed enhancements can be:

IDEA: the step is currently an idea; might or might not have an associated Jira ticket.
PROPOSED: the enhancement has been officially proposed to the CalWebb WG. Discussion ongoing as to whether move it to further study or not. 
ONGOING: step currently under development by the Pipeline Development Team.
DONE: step has been implemented in the JWST pipeline.

Priorities are defined for the TSO WG:

1: Critical — will produce unusable results for TSOs.
2: High — will produce sub-optimal results for TSOs.
3: Low — will produce slightly offset results for TSOS.

Proposed enhancementBrief descriptionTODOsRelevant jira ticket (s)Meeting links where this was discussedStatusPriorityUpdate date
TSO3 Outlier Detection

Currently, the JWST pipeline does not properly use TSO information to reject outliers (e.g., jump step or current tso3 outlier rejection — see Jira tickets). Proposed to implement HST/STIS algorithm developed by Nikolay Nikolov on JWST TSO pipeline. 

  • Algorithm currently tried only on NIRCam data. Need to try it on NIRISS/SOSS, MIRI/LRS and NIRSpec/BOTs.

Problems with current TSO3 step: JP-1285 - Getting issue details... STATUS JP-1654 - Getting issue details... STATUS JP-1647 - Getting issue details... STATUS

CalWebb WG: 2021-05-04 Meeting Notes; 2021-01-05 Meeting Notes.

TSO WG: 2021-02-24 TSO WG Meeting notes.


3December 2, 2021
Jump detection using TSO information

Similar to the above, but do it at the group-level; detect jumps using TSO information. This is an idea proposed by Michael Regan. Pros: you don't rely on the reference files.

  • Need to do simulations to test algorithm.


Correct jitter spectral movement prior to photom stepIf jitter is an issue during JWST observations, then the photom step will incorrectly apply photometric flux standarization to the wrong wavelengths, introducing time-dependant systematics. TSO WG reps think turning off the photom step would be ideal, but not generic solution. Correcting those jitters prior to the photom step would be a solution.
  • Need to write cross-correlation function to correct spectra.
  • Apply it to spectra; and see if this can correct photom step jitter issues.

JP-2082 - Getting issue details... STATUS

CalWebb WG: 2021-06-01 Meeting Notes.

TSO WG: 2021-06-02 TSO WG Meeting notes.


Spectral tracingPipeline needs to perform spectral tracing of TSOs in order to (a) record trace movements (useful as external parameters to decorrelate lightcurves) and (b) extract the same portion of the spectrum at each time.
  • Define tracing algorithms for each instrument.
  • Test those.
  • Implement on the pipeline.


2December 15, 2021
Pre-amp reset correction ("reference pixels for subarrays without reference pixels")
  • Pipeline needs to take care of detector effects not taken care of by, e.g., superbias in the case where there are no reference pixels.
  • Everett Schlawin proposes a "fast-by-fast", "slow-by-slow" substraction; give pipeline pixels that it can use on each mode (= "unilluminated pixels")
  • Various codes to do this already at hand. Significant improvement when doing this for NIRISS, NIRSpec and NIRCam.
  • No tests for MIRI on this yet.


1December 15, 2021
Background substraction 
  • Background substraction is embedded on extract1d. But this is unintuitive.
  • This is particularly troublesome for MIRI/LRS. 


2December 15, 2021
Optimal extraction
  • Pipeline currently does box-extractions. Should perform extraction by weighting pixels by their SNRs/noise levels.

  • Optimal extraction could include likelihood information on 1/f noise, for instance.
  • Code for uncorrelated gaussian likelihoods is known.


2December 15, 2021
Pixel timing
  • Due to the sequential nature of the pixel readout modes, every pixel does not have the same time-stamp. The pipeline should either report this, or have an utility function that corrects for this.
  • Timing of the pixels is known, but uncertainties are not.

Discussion on this:
JP-2330 - Getting issue details... STATUS


1December 15, 2021

  • No labels