2022-01-12 TSO WG Meeting notes

Launch delay to no earlier than 24 Dec. 

Brian Brooks new Pandeia alpha release. helping people on other development teams get used to the software - has a lot of API changes. could be useful for anyone working on PandExo. Important in particular for ExoCTK.

Nestor Espinoza 

• JWebbinar is all done! The sessions were quite easy, preparations were hard work. Now have good notebooks in place that people can use, they are available in STScI repositories. From TSO WG Everett and Knicole were present - feedback welcome.
  
  
  • Everett Schlawin asks whether these notebooks will receive long term support. Answer: no provision for maintenance right now.
    
    
• Last day to comment on the high-efficiency modes report. Please send feedback or comments asap. 
  
  
• 1/f noise update from Nestor Espinoza:
   
  
  • Given we have fits for 1/f power spectral densities (PSDs), we can now generate "fake" darks and perform analyses on those. The first experiment Nestor Espinoza tried was to perform column-to-column (a.k.a., "fast-to-fast" read direction) substraction on simulated NIRISS SOSS/SUBSTRIP256 darks with this model. This, of course, doesn't correct for all the 1/f signal, but significantly decreases its power at length-scales larger than about 10 columns. Interestingly, doing this correction at the group-level is much better than doing it at the ramp-level (see slides).
    
    
  • Everett Schlawin asks why the corrected PSDs of both methods do not meet at long time/length-scales (i.e., short frequencies). Nestor Espinoza has some hunches, the main one being the fact that these are the PSDs of the ramps — so it's not as simple as the PSD of a time-series after a filtering procedure.
    
    
  • Diane Karakla asks why the noise level at low frequencies in the corrected PSD does not match the white-noise level at high frequencies in the uncorrected PSD. Nestor Espinoza's hunch on this is similar to the response to Everett; he will run some experiments to double check this is the case. She also asks why the spikes are gone on the corrected PSDs; he's also not sure. However, his hunch is that given a ton of power has been removed from low frequencies, not much signal is leaked at higher ones and hence the peaks height diminishes. He will double check if this is indeed true.
    
    
• Nestor Espinoza also ran some end-to-end experiments of 1/f noise during NIRISS commissioning rehearsals. Loic Albert generated simulated observations with & w/o 1/f noise (not with MIRAGE); Nestor Espinoza ran the pipeline end to end and looked at the difference (see slides for plots). Saw significant decrease in the (scatter/errorbars) plot as a function of wavelength. This was measured using the wavelength-dependant lightcurves. This either implies the scatter is smaller or the errorbars are better determined. Will check which one is the case (probably both).
  
  
  • Michael Regan : have to be careful about how noise is put into the simulations. Simulators are not fully validated, they are not "truth". So we need to keep that in mind when using simulated data to investigate this issue. Nestor Espinoza  agrees with the statement!
    
    
• Fits for 1/f noise look good for full frames but subarrays still have peaks that we don't understand. Nestor Espinoza  went back to darks data to try and understand where this power is coming from. 
  
  
  • Exploring this further, but would be best if detector experts could take a look at this as well.
  • Michael Regan will ask Eddie to look at these data; Nestor Espinoza will send a paragraph and some scripts their way so they can have a look at it.

• Nestor Espinoza has created a page (TSO Pipeline Proposed Enhancements) with proposed enhancements for the pipeline, as we're on the hook to present those in the CalWebb WG meeting in January. Want to collect the proposals there in a table before submitting actual JIRA tickets. This started a discussion on possible enhancements:
  
  
• Regarding TSO3 Outlier detection:
  • Michael Regan this algorithm needs to be tested on data that has run through the jump step (even for the NIRCam data); there have been enhancements to the jump step. MR proposed additional change to jump step that is specific for TSO data. MR has proposed but not yet demonstrated with data. (uses sigma clipping rather than work with reference files).
    
    
• Additional items to add to this table:
  • pre-amp reset corrections (ref pixels for subarrays; "pseudo-reference pixels")
  • IRS2 in subarray for NIRCam
  • background subtraction for slitless LRS → action on SK 
    
    
• Action on all to add further ideas for enhancements to the table. will be discussed in the Jan CalWG meeting.