Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Date

05 May  

Attendees

Apologies:

Meeting agenda:

  1. News & announcements
  2. High efficiency mode
  3. Outlier detection update
  4. 1/f noise update
  5. Closing remarksUpdates on labor schedule for 2021 (Néstor & Sarah).
  6. JDox documentation proposal plan (all).
  7. Discussion on error estimation by the pipeline (Michael).
  8. Closing remarks.


Meeting slides


Discussion items

TimeItemWhoNotes
5 mins

1. News & announcements

Everyone
  • Nestor created a JIRA ticket re. the visit planning problem for long TSOs (see notes April 7th): 
    Jira
    serverSTScI JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverIdaac911eb-9c55-3f14-8368-77b436fb6dc2
    keyAPT-92525
30min2. High efficiency modes
  • We held a meeting on Thursday 29 April, see Sarah's notes here
5min3. Outlier detection update
  • Nikolay presented his work at the CalWG meeting on Tuesday 3 May. We discussed in a small internal meeting on Monday 2 May. 
5 mins4. 1/f noiseNestor Espinoza
  • Photom step

MR: worried that the jitter correction is not as trivial as people think it is. These are not CCD pixels. Requires extensive validation and testing on flight data.

NE: Agreed and people have found quite a lot of additional complexity doing this for HST. can only really work on this properly when we have flight data so right now safer to just turn the step off. 

ES: the OTIS tests do have some jitter built in. we could compare 2D and 1D method to see. 

doesn't necessarily have to be 2D, it could be a 1D correction to a 2D image

TB: agree with the summary. agree that doing the correction not as simple as initially seems. have to be careful about interpolation in the photom step. would need to run a few tests and demonstrate to the calWG before moving to the implementation.  

MR: less sophisticated solution to the problem: have a flag that indicates to the automated pipeline whether the step should be run or not. 

NN: hard to advise to people how to use that as is currently entirely based on previous experience from other telescopes. It's up to TSO WG to really prioritise this discussion. it would be good to make a plan, maybe have a few options lined up that we can then test with flight data. 

NE: the point is that right now the photom step will give wrong or non-optional answers and people will have to turn it off. 

NIRISS will study the pointing jitter and impact on TSOs as part of a commissioning project but will do all this analysis without the photom step

ES: 

TB: channeling Karl - issue with photom is a symptom of an underlying issue that we are not addressing. if we can perfectly adjust the WCS then this would fix the issue. so Karl woudl say why are we not trying to fix the underlying issues. Can imagine a science case for having flux calibrated data for TSOs. 

MR: has come around on the issue, we should provide the best pipeline for the general case and the general (majority) case for TSOs is to not run the step then it should really be turned off. The argument that the previous TSO WG asked it to be in should not be a reason for not making this change. 

ES: FYI, Anton Koekemoer is now Cal WG lead so he is technically the one that decides on the photom step.

SK: recommend we get feedback from teh community on this, and encourage people to contact the JSTUC if they have strong feelings. this is another way to get things moving on this.

KC: seconds that suggestion









2. Updated Labor schedule for FY 2021


  • See slides. 
  • New task added for documentation added for FY 2021. Probably of order 6 new articles. Approx 5 full days of writing → publication time. (see next agenda item)




5min3. JDox Documentation Proposal plan


  • See slides
  • New sections:
    • Under methods & roadmaps: create a new "specific guidance" (a kind of "known issues" section), where we talk about timing contraints, and also document issues like the max ngroups limitation, the APT phase contraints bugs 
    • In the Data section: have similar "methods and roadmaps" section where we can give TSO-specific guidance on data and pipeline methods. Includes also: timing information; guidelines & known limitations (sp extarction, noise sources cf Schlawin et al 2020,2021, photom step, outlier detection, MIRI MRS etc)

KC: where do we capture the overall best practices - like the Spitzer IRAC page? 

yes this is what we are aiming for but have to see where to best place it. have the Methods & roadmaps section, and also "recommended strategies"  in the instrument sections. Avoid duplicating too much content, ensure taht we link to other relevant pages as much as possible. 

5 mins4. Error estimation in the pipelineMichael Regan

detectors do not count electrons, the measure voltages. have to convert.

photon transfer issues:

  • have non-linearity. errors on non-linearity are squared on gain - cyrrently NL corrections have errors on the order of 1%
  • MIRI detectors cannot use the standard methods on the gain as in the mid-IR the non-linearity is due to a change in QE, not a change in gain. MIRI gain is an estimate and there's some evidence it is off by >20%

  • exact time spent accumulating charge is not actually returned (with the actual  number of frames used to calculate the slope). CRs during the integration affect the actual time. only read noise and Poisson noise uncertainty are returned. 
  • feel like none of the teams have put sufficient effort into the gain characterization. this is why the jump detection step doesn't work very well - as the read noise and gain assumptions are not correct. Most observations don't care! 
  • especially hard for MIRI
  • MIRI's QE also changes with wavelength

NE: can use self-calibration with long time series?

MR will be presenting on non-linearity mistakes in TIPS in 2 weeks. Will hopefully start the discussion on what the way forward should be. 

NE: can host a meeting specifically on this after TIPS, should not be TSO specific. 










5 mins5. Final comments or updates



Sarah KendrewNone!