Versions Compared

Old Version 1

changes.mady.by.user Nestor Espinoza

Saved on

compared with

New Version Current

changes.mady.by.user Sarah Kendrew

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Apologies:


Meeting agenda:

...

JWebbinar is all done! The sessions were quite easy, preparations were hard work. Now have good notebooks in place that people can use, they are available in STScI repositories. From TSO WG Everett and Knicole were present - feedback welcome.
Everett Schlawin asks whether these notebooks will receive long term support. Answer: no provision for maintenance right now.
Last day to comment on the high-efficiency modes report. Please send feedback or comments asap. 
1/f noise update from Nestor Espinoza:
 
Given we have fits for 1/f power spectral densities (PSDs), we can now generate "fake" darks and perform analyses on those. The first experiment Nestor Espinoza tried was to perform column-to-column (a.k.a., "fast-to-fast" read direction) substraction on simulated NIRISS SOSS/SUBSTRIP256 darks with this model. This, of course, doesn't correct for all the 1/f signal, but significantly decreases its power at length-scales larger than about 10 columns. Interestingly, doing this correction at the group-level is much better than doing it at the ramp-level (see slides).
Everett Schlawin asks why the corrected PSDs of both methods do not meet at long time/length-scales (i.e., short frequencies). Nestor Espinoza has some hunches, the main one being the fact that these are the PSDs of the ramps — so it's not as simple as the PSD of a time-series after a filtering procedure.
Diane Karakla asks why the noise level at low frequencies in the corrected PSD does not match the white-noise level at high frequencies in the uncorrected PSD. Nestor Espinoza's hunch on this is similar to the response to Everett; he will run some experiments to double check this is the case. She also asks why the spikes are gone on the corrected PSDs; he's also not sure. However, his hunch is that given a ton of power has been removed from low frequencies, not much signal is leaked at higher ones and hence the peaks height diminishes. He will double check if this is indeed true.
Nestor Espinoza also ran some end-to-end experiments of 1/f noise during NIRISS commissioning rehearsals. Loic Albert generated simulated observations with & w/o 1/f noise (not with MIRAGE); Nestor Espinoza ran the pipeline end to end and looked at the difference (see slides for plots). Saw significant decrease in the (scatter/errorbars) plot as a function of wavelength. This was measured using the wavelength-dependant lightcurves. This either implies the scatter is smaller or the errorbars are better determined. Will check which one is the case (probably both).
Michael Regan : have to be careful about how noise is put into the simulations. Simulators are not fully validated, they are not "truth". So we need to keep that in mind when using simulated data to investigate this issue. Nestor Espinoza  agrees with the statement!
Fits for 1/f noise look good for full frames but subarrays still have peaks that we don't understand. Nestor Espinoza  went back to darks data to try and understand where this power is coming from. 
  • Exploring this further, but would be best if detector experts could take a look at this as well.
  • Michael Regan will ask Eddie to look at these data; Nestor Espinoza will send a paragraph and some scripts their way so they can have a look at it.
Nestor Espinoza has created a page (TSO Pipeline Proposed Enhancements) with proposed enhancements for the pipeline, as we're on the hook to present those in the CalWebb WG meeting in January. Want to collect the proposals there in a table before submitting actual JIRA tickets. This started a discussion on possible enhancements:
Regarding TSO3 Outlier detection:Michael Regan this algorithm needs to be tested on data that has run through the jump step (even for the NIRCam data); there have been enhancements to the jump step. MR proposed additional change to jump step that is specific for TSO data. MR has proposed but not yet demonstrated with data. (uses sigma clipping rather than work with reference files).
Additional items to add to this table:
  • pre-amp reset corrections (ref pixels for subarrays; "pseudo-reference pixels")
  • IRS2 in subarray for NIRCam
    • background subtraction for slitless LRS → action on SK 
    Action on all to add further ideas for enhancements to the table. will be discussed in the Jan CalWG meeting.
    TimeItemWhoNotes
    5 mins

    1. News & announcements

    		Everyone

    Launch delay to no earlier than 24 Dec. 

    Brian Brooks new Pandeia alpha release. helping people on other development teams get used to the software - has a lot of API changes. could be useful for anyone working on PandExo. Important in particular for ExoCTK.

    25min2. Current task updates

    Nestor Espinoza 

    25min3. Pipeline enhancements
    Nestor Espinoza 
    •  release candidate for B7.9 of the JWST pipeline is out (v1.4.0)
    • relevant changes for TSOs:
      • optional Generalised Least Squares (GLS) ramp fitting; runs more slowly but returns a covariance matrix. 
        Michael Regan  does not recommend using the GLS method as it is not working properly at the moment. we also know the noise reference files are wrong.
      • some updates to the jump step: these are fixes of fixes, no new functionality. 
      • Loic Albert : the SOSS extraction step was delivered but ran into a last minute issue. The code that cleans contomination between orders was included but it is not the default option as it assumes the photom step is turned off, and that is not the case in the default pipeline parameters. 
    25min2. PIpeline priorities presented to the CalWG, 4 Jan 2022


    Nestor Espinoza 

    See notes of the CalWG meeting, 4 Jan 2022.

    Enhancements were positively received but need some more work from us. 

    Critical priority items:

    • pre-amp reset correction (det1)
    • pixel timing & time-stamp accuracy (tso3)

    High priority:

    • jump detection (det1)
    • centroiding/tracing (spec2)
    • optimal extraction (spec2)
    • jitter correction (spec2)
    • background subtraction (spec2)

    Low priority:

    • integration-to-integration aperture positions (spec2)
    • outlier detection (tso3)

    Lots of discussion on the best way to do the jitter correction for photom. Different modes may have different ideal implementations - either in 2D space or post-extraction in 1D? This will need further thinking & work.

    Pixel timing: how do we best capture the info? via CRDS reference files, or with header keywords? Or a combination of both? Also needs further thought.

    Time stamping: this is quite a complex systems engineering issue. Nikolay looked into the details and read. reports but was all v technical. Nestor has created a JIRA ticket for this issue.


    		3. TSO Task updates


    		Nestor Espinoza 
    • JDox (Sarah Kendrew ): there has been no progress on this task. SK asked for instrument-specific input before the holidays, but has not received anything yet. Spoke with Alaina and there is no rush to get these new pages published, so it is up to us to determine when to get this done. NE proposes deadline for the instrument-specific information for the next meeting in 2 weeks' time.
    • NIRSPec pipeline validation: Leonardo on leave.
    • The high-efficiency modes tech report has been submitted to SOCCER
    2 mins4. Closing RemarksNestor EspinozaNext meeting: 29 Dec26 January.