2021-12-01 TSO WG Meeting notes

Nestor Espinoza 

• TSO JWebbinar. First session took place yesterday, led by Sarah Kendrew and Nestor Espinoza (30 Nov.); very positive. Produced some good comprehensive notebooks. Next session this Friday.
  
  
• High efficiency modes:
  • First draft of the report is being circulated, can be shared amongst the group (Everett Schlawin, Knicole Colon  & Sarah Kendrew would like to read it). Nestor Espinoza will share amongst the group.
  • Some highlights: clear improvement in precision can be achieved - see plots in slides.
  • Provides pretty strong justification for implementation.
  • Sarah Kendrew asks whether data volume considered/discussed? Not so far - but should factor in there as higher efficiency will produce more data.
  • Michael Regan suggests that it would also be be useful to add info on ngroups to the summary table.
  • Everett Schlawin mentions that a compelling message to really convey is that a good deal of detections will be marginal so higher efficiency would make a difference to the most challenging proposals. In particular, this is important for the TRAPPIST-1 proposals. Nestor Espinoza did include a citation to the work of Natasha Batalha which goes in detail on this exact same point, but might be good to reiterate it on the docuement.
  • For MIRI we can in principle actually do Ngroups = 1  if we don't care what it looks like. NIRISS will be doing this as well. 100% efficiency mode thus would be a huge improvement to those prospects.
    
    
• NIRSpec pipeline validation: the gift that keeps on giving
  • Analyses led by Leonardo Ubeda indicate that jump step seems to behave weirdly on CV3 data. Finds 10s of thousands of jumps in a small subarray (basically all the background pixels). Found this was due to negative background counts, produced by group-to-group flux variations. 
  • This was fixed by removing "background" levels studying "non-iluminated" pixels. Simply took the median of first 10 and last 10 rows, and that made the algorithm to behave better. However, Nestor Espinoza mentioned jumps are predominantly detected around 1/f banding and on the center of the trace, as small trace position variations make small intra-pixel variations to produce larger-than-threshold flux variations. This latter might be a particularity of the NIRSpec CV3 data (which had larger-than-expected jitter).
  • Michael Regan is suspicious of this 1/f banding effect. He suggests it is most likely a reference file problem rather than an algorithmic problem. Nestor Espinoza mentions he is pretty sure the reference file is being generated "correctly", i.e., taking the combined variance as the read-out-noise of everything (1/f + white-noise) in the darks. He will investigate further.
  • Michael Regan shows reservations on using "background" pixels as pseudo-reference pixels, because it's hard to identify in general where "non-iluminated pixels" are located. Nestor Espinoza suggests this is relatively easy to do in general for TSOs. Perhaps that could be implemented there. Thomas Beatty mentions this is what they do in their own NIRCam data analysis pipeline. 
  • Michael Regan suggests maybe one should just drop the jump step for subarrays for which there are no reference pixels. Nestor Espinoza is not very keen on this idea; the jump step is a good step, it just needs a bit of "help". Michael Regan shares that he's been working on an algorithm that uses TSO information to compute the jumps; Nestor Espinoza shared that this is exactly what he does on his own pipeline, and this should be the way to go.
  • Nestor Espinoza will open a ticket (or multiple ones) to address this. 

• First-group behavior/analysis (JP-2071)
  • Michael Regan introduces the fact that currently, the pipeline "forces" the usage of the superbias as a zeroth-group in order to calculate ramps for single-group integrations. This might be problematic first because it might be inaccurate if there are temperature variations, but also because of a lack of consistency: if this method for obtaining ramps is good, why not use it for all ramps? An extra group-time would be gained (more details on the discussion here).
  • He suggests that this should be optional. And that the possibility of doing single-group integrations using the superbias as the zeroth-group should be studied with multi-group integrations (in order to have some ground truth).
  • Nestor Espinoza suggests a good action item for the TSO WG could be to study this with the TSO commissioning datasets. This way, the stability across exposures of doing this could be studied in detail. Enabling it or "suggesting" it for use for exoplanet science, for instance, might be dangerous to do before this is actually tested.

• Pixel-level time-stamps (JP-2330)
  • Nestor Espinoza points to this ticket and introduces the problem: given the readout of JWST detectors is sequential, certain pixels have a different time-stamp than others. This needs to either be (a) fixed/provided by the pipeline or (b) coded up so users can fix this in their datasets. He asks whether we know precisely the clock timing of pixels.
  • Michael Regan answers that yes, it is known to be 10 micro-seconds; the row/column wait time is based on those clock-times. Nestor Espinoza asks if this has any tiny uncertainty associated with it; Michael Regan suggests that this is not the case.
  • Being that the case, this will be brought to the CalWebb WG on January then as an item to work on pipeline-wise. Discussion will be tracked in the Jira ticket.