Versions Compared

Old Version 1

changes.mady.by.user Sarah Kendrew

Saved on

compared with

New Version Current

changes.mady.by.user Sarah Kendrew

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Meeting agenda

...

See slides below!

...


Meeting slides

TSOWG-May5-2021.key

N&A
NN - outlier algorithm presented along w tests yesterday on the CalWG meeting: https://outerspace.stsci.edu/pages/viewpage.action?spaceKey=JWSTCC&title=2021-05-04+Meeting+notesS.K. asks what are his thoughts regarding the overall opinion of the CalWG following the presentation yesterday?
NN mentions that CalWebb has outliers removal in Stages 1 and 3 and that is likely that the Stage 3 outliers removal will likely be switched off the Cycle 1. He also mentions that DIMON could be a complementary algorithm to clean CaRs missed for some reason by the jump detection in Stage 1 and only for exoplanets. However, this wont be an official part of the pipeline. A side by side comparison of DIMON v the jump detection step will be needed on real data (or perhaps some simulations) to ensure DIMON is an improvement. There is the possibility to convert the IDL codes from Nikolov into Python and be released to the community.S.K. the CR algorithm (DIMON) could be made available to the community in a Jupiter Notebook style; She spoke with Ori Fox about JupiterNotebooks and there is an opportunity to include algorithms which are considered useful for the community.L. A. expresses interest toward DIMON as his group in MontReal is charged with delivery of spectra and extraction. He would like to include DIMON within their Ghost pipeline
NN will provide his code to LABB mentions notebooks that a he prepared and released to the community; links:
https://innerspace.stsci.edu/display/DATB/Timeline
from Brian Brooks (internal) to Everyone:    11:19  AM
https://spacetelescope.github.io/jdat_notebooks/S.K a fair amount of work to provide notebooks, but the Notebook group would be happy to include; not neededS.K. mentions an issue brought by J.Valenti: JSOCINT-488 exposure limitations 192,000 nints; 8 programs are affected by this and several of them are phase curves; MIRI LRS and NIRSpec; JV asked to assess the impact and would want to see if there is a problem for those programsS.B. asks if we have the possibility to allow this in all instruments? Otherwise we end up with one observation having a split exposure for one instrument but not for another. SK. Will need to check this to see if all instruments are allowed.S.K. Mentions an example of a phase curve observation that will break in 3 exposures instead of a single as a solution
NN discusses the impact on the exposure interruptions and the data loss. It is alright to avoid gaps during transit and phase curve events, but also data loss could have significant impact during coverage of hot spots, which are short in time, so data loss could potentially be a problem.
S.K. the interruptions should be on the order of minutes. She proposed a list to be created with all the limitations; and make them available to the users; data volumes are another problem; not sure if all the problems are taken into account in APT;
S.B. we have time for addressing the data volume issue
NN Political problem about trimming the data; will we compensate PIs about this, as the PIs weren’t informed that 1 continuous exposure will need to be trimmed in a few with gaps prior to Cycle 1?
SK It is not in APT nor JDOX, nobody complained; We will resolve on a case by case basis.
BB mentions that JV ran the APT 92473 to a
TK asks if this change has been implemented in the APT
Puzzled by the 6-7 visits that are affected. Who is going to contact these PIs
SK assumes that the TR will address this and will be brought to the PIs; the CS will ask if splitting will be mentioning;
TK Doesn’t recall a time when OSS will implement the multiple exposures; OSS will not change; will took 2 activities instead of one; clean the exposure and do a new one with a clean of a few minutes;
D.K. thinks it is implemented for NIRSpec
If it is new activity, it will not move a filter;2nd point; APT-92525 unable to plan scheduling accurately when you have a long exposure
We asked the users to double the orbital period; need to work with APT folks to resolve the issue; important for our reviews; link provided; need to be included in JDox; S.B. rises as a part of their program reviews; thought it is a problem of the APT features; as soon as you put a phase constraint; the phase constraint is not imposed; all phase curves that want an overlap; the observation will start anywhere; APT should just enforce the constraints regardless of the length;TK agrees;
S.K. we should consider this in our reviews as long as total length;Nikolov’s presentation - discussedUpdates from 1/f
Individual spectrum; one paragraph from all of us -
D.K. no problem to provide this; worked together to clear things; A.U. found an issue in how the padding was done and fixed it; the row by row subtraction was changed with the minimum instead of median; but saw more noise in the PSD; 1^-3 additional noise spikes;T.K. the rationale for adding this algorithm is to secure it works on subarrays and using the edge pixels would be justified there;Sub512 fabricated to get subarray data; padded the original data with padded pixels on the edges;ES new reference pixel correction from Burnie Rauscher in software; paper is available link from ESHigh Efficiency Mode meeting - SK mentions the meeting last week; looking at an exercise of the impact/gain/down sides of including the HEM for each instruments for TSOs - what would be the benefits; notes link here; for the NIR it is good as we can include efficiency to 70%-80%, it is understood how this wcan be done in ASIC; looks promising; next step to work with exoplanet community and how this would be seen and what needs to be done; in the MIR is more difficult; inserting an extra reset, so we are loosing efficiency now; for others this would reduce detector systematics; there are bias voltage adjustments, which are not trivial to perform. Next steps would be to establish a small WG to establish the technicalities and reach out the community to discuss in an easy to digest style and seek feedback. Will provide updates here in 2 weeks

spacetelescope.github.io
Spacetelescope dat_pyinthesky Index
A space for Astronomy Notebooks


Discussion items

TimeItemWhoNotes
5 mins

1. News & announcements

Everyone
  • Jeff Valenti sent round a list of program IDs that currently violate the total ngroups limitation. 6 GO and 2 GTO programs are currently affects; 7 programs use MIRI LRS and 1 NIRSpec BOTS. JV requested an assessment of the workaround for these and SK provided the required feedback.
    • SB: is aplitting into multiple exposures allowed for all instruments? otherwise different workarounds are needed for different instruments.
    • NN discusses the impact on the exposure interruptions and the data loss. It is alright to avoid gaps during transit and phase curve events, but also data loss could have significant impact during coverage of hot spots, which are short in time, so data loss could potentially be a problem.
    • S.K. the interruptions should be on the order of minutes. She proposed a list to be created with all the limitations; and make them available to the users; data volumes are another problem; not sure if all the problems are taken into account in APT;
    • S.B. we have time for addressing the data volume issue
    • NN Political problem about trimming the data; will we compensate PIs about this, as the PIs weren’t informed that 1 continuous exposure will need to be trimmed in a few with gaps prior to Cycle 1?
    • SK It is not in APT nor JDOX, nobody complained; We will resolve on a case by case basis.
    • SK assumes that the technical review will address this and will be brought to the PIs; the contact scientist will communicate with the PIs
    • TK Doesn’t recall a time when OSS will implement the multiple exposures; OSS will not change; will took 2 activities instead of one; clean the exposure and do a new one with a clean of a few minutes;
    • D.K. thinks it is implemented for NIRSpec


  • APT has deferred “solving” phase-constraint problem for long exposures 
 for Cycle 2 (see APT-92525). Please, note this in your instrument reviews (consider adding this to the IS 
 Checklist). This is important; it is not merely a display issue. Users need 
 to fix this during reviews. Need to work with APT folks to update JDox 
 (perhaps add a section on TSO Strategies?)







30min2. Outlier detection updates


Nikolay Nikolov
    • NN - outlier algorithm presented along w tests yesterday on the CalWG meeting: 2021-05-04 Meeting Notes
    • NN mentions that CalWebb has outliers removal in Stages 1 and 3 and that is likely that the Stage 3 outliers removal will likely be switched off for Cycle 1. He also mentions that DIMON could be a complementary algorithm to clean CRs missed for some reason by the jump detection in Stage 1 and only for exoplanets. However, this wont be an official part of the pipeline.
    • A side by side comparison of DIMON v the jump detection step will be needed on real data (or perhaps some simulations) to ensure DIMON is an improvement. There is the possibility to convert the IDL codes from Nikolov into Python and be released to the community.
    • S.K. the CR algorithm (DIMON) could be made available to the community in a Jupiter Notebook style; She spoke with Ori Fox about JupiterNotebooks and there is an opportunity to include algorithms which are considered useful for the community. It is fair amount of work to provide notebooks, but the Notebook group would probably be happy to include one for this. (see https://innerspace.stsci.edu/display/DATB/Timeline, https://spacetelescope.github.io/jdat_notebooks/
    • L. A. expresses interest toward DIMON as his group in MontReal is charged with delivery of spectra and extraction. He would like to include DIMON within their Ghost pipeline. NN will provide his code to LABB mentions notebooks that a he prepared and released to the community




5min3. 1/f noise


Sarah Kendrew 

  • Néstor has working code to fit PSD profiles. Needs from you: Individual time-series and PSDs of each group you have at hand; A one-paragraph description on how this was obtained; He’ll work on putting all together

    • DK happy to provide this
    • A.U. found an issue in how the padding was done and fixed it; the row by row subtraction was changed with the minimum instead of median; but saw more noise in the PSD; 1^-3 additional noise spikes;
    • T.K. the rationale for adding this algorithm is to secure it works on subarrays and using the edge pixels would be justified there;Sub512 fabricated to get subarray data; padded the original data with padded pixels on the edges;
    • ES new reference pixel correction from Burnie Rauscher in software; paper is available




5 min4. High-efficiency modes



Sarah Kendrew

  • Meeting took place last week. Notes are here.
  • We are looking at an exercise of the impact/gain/down sides of including the HEM for each instruments for TSOs - what would be the benefits & trade-offs?
  • for the NIR it is good as we can include efficiency to 70%-80%, it is understood how this can be done in ASIC. next step is to work with the exoplanet community and how this would be seen and what needs to be done;
  • in the MIR is more difficult; inserting an extra reset, so we are loosing efficiency now; for others this would reduce detector systematics; there are bias voltage adjustments, which are not trivial to perform. Next steps would be to establish a small WG to establish the technicalities and reach out the community to discuss in an easy to digest style and seek feedback. Will provide updates here in 2 weeks
5 mins5. Final comments or updates



Sarah Kendrew
  • Let's meet again in 2 weeks!

...