Date

Attendees

Meeting agenda:

  1. News & announcements.
  2. Plan for build 7.5 pipeline testing throughout the summer (all).
  3. TSO activities on each instrument branch (all).
  4. Discussion/prioritization of TSO Jira tickets.

Discussion items

TimeItemWhoNotes

1. News & announcements

Everyone

2. Plan for build 7.5 pipeline testing throughout the summer



Everyone

Nestor Espinoza introduces the fact that we have to provide feedback to the CalWebb WG on the JWST Calibration Pipeline Build 7.5 (mainly through opening of Jira tickets). The hope is that this also serves as input to the community to know what works, what doesn't — and of course, for us. He notes that we should not only provide feedback on the final TSO3 Stage, but also on Stage 1 and Stage 2 (both for imaging and spectroscopy), following the TSO steps that are taken in those stages. Sarah Kendrew notices the importance of doing this for imaging, where they found a bug in the MIRI imaging mode (see below); Nikolay Nikolov also mentions this is valuable based on his experience (see below); Nestor Espinoza has been having a similar experience with NIRISS/SOSS testing on his side. The team agrees that this is the way to go.

The proposal Nestor Espinoza gives to the TSO WG is to do this in a (2-week) sprint: basically, work together in the problem in order to have quick feedback about possible steps that might fail (which might be useful to test if this is due to problems in the simulations or an actual bug on the pipeline). Sarah Kendrew likes this idea as well. Nikolay Nikolov and Tony Keyes would prefer to do the testing on their own, at their own pace. On the one hand, Nikolay doesn't think he can commit to a 2-week effort, whereas Tony has to consult with the experts testing the pipeline in NIRSpec (James M. and Maria P.). The team thus agreed for Nestor Espinoza and Sarah Kendrew to do a testing sprint, report that to the team, and then for Nikolay Nikolov and Tony Keyes to perform their own testing and report back to the team. 

To organize the work, Nestor Espinoza will create a spreadsheet where we will leave our feedback to each of the steps of the pipeline. Sarah Kendrew mentions to be careful between validation and verification. We want to provide feedback to the CalWebb WG via Jira tickets by testing that the algorithms do what they are expected to do. If we want to provide enhancements, however, that most likely involves a direct e-mail or document sent to the WG. Both are extremely useful, but use very different channels to reach the CalWebb WG. Tony Keyes also mentions to be aware of the Jira tickets in the TSOWG DMS Dashboard, and Sarah Kendrew notes that useful information is also available on the dashboards of each of the instruments (MIRI DMS Dashboard, NIRCam DMS Dashboard, NIRISS DMS Dashboard, NIRSpec DMS Dashboard). If you encounter a bug/problem with a step, make sure you check those before opening a new one — use those also as general information; a quick read through them before starting testing is very, very useful.

  • Nestor Espinoza will create and share a spreadsheet to track progress on JWST Pipeline testing.
  • Nestor Espinoza & Sarah Kendrew will have a 2-week sprint testing every step of the pipeline for MIRI and NIRISS for TSOs.
  • Tony Keyes will report back on what will be the plans of NIRSpec for TSO testing. Will define a meeting before the end of August to present results on this front.
  • Nikolay Nikolov define a meeting before the end of August to present results on Pipeline testing for NIRcam.
25min3. TSO activities on each instrument branch
5minNIRISS activities/updates

Nestor Espinoza

  • Nestor has been working in the past two weeks on a JDAT notebook for transiting exoplanets. He has both worked on the actual simulations, calibration and data reduction, doing effectively end-to-end simulations for NIRISS/SOSS. Has notebooks for those three steps. He showed part of his notebook today, and David Sing asked if he could share how he did the transit fits on his notebook, as his notebook was a bit slow on this side. Nestor Espinoza mentions that he used the library he wrote (juliet; http://juliet.readthedocs.io/) for this, which in his case fits the data in a couple lines of code. Will share this with David offline.
5minNIRCam activities/updates

Brian Brooks Nikolay Nikolov

  • Nikolay has been summarizing all the results of his testing of the JWST pipeline that he did based on this ticket. He has a docs document for this.
  • Also looking into the TSO Quicklook tool details, as well as the time-stamps problem definition. Will have a meeting tomorrow about this.
  • Brian also mentions that the Synphot refactoring for the ETC release is done (thanks to everyone that helped out with this!).  He further mentioned that the ETC & JWST MO would, in general, like to have better coordination with ExoCTK. They will be defining how to do this in practice in the next weeks. Nestor Espinoza agrees with this, and is happy this is happening — they have had problems in the past because of this.
5minNIRSpec activities/updatesTony Keyes
  • Worked on reviewing TSO Jira tickets, and had meetings with James M. to get up to speed on this. Of interest is the fact that the JWST Pipeline for NIRSpec is a bit stuck in the rectification step. Nestor Espinoza mentions that he has been providing feedback on this, because although it is understandable that NIRSpec uses rectification in the other modes, for BOTS/transit spectroscopy this is very sensible to do and probably not recommended. For example, an immediate problem with this is the fact that if you rectificate/resample the data to account for the non-vertical wavelength solution, you also pass information on background signals which actually is vertical into a non-vertical orientation in the rectified image (e.g., 1/f noise which is column-by-column in NIRSpec). For more information on this discussion, see this ticket.
  • David also chimed in on his analysis of CV3 data. Basically done (as Nestor's notebook above); CV3 data is very rich in terms of systematics, so tons of information embedded in the notebook which will most likely be very useful for users.
5minMIRI activities/updates

Sarah Kendrew

  • Sarah has been working with Katie M. working on general LRS pipeline testing, which is a very good start for the TSO sprint on pipeline testing defined above. Plan is for Sarah to use those scripts for the Pipeline testing of LRS slitless mode.
  • For imaging Misty has been generating new imaging TSO data with some scripts from Sarah. They found some issues in the early steps of the pipeline, which was due to header problems. Turns out this is not a problem for NIRCam, but only for MIRI.
  • Worked on defining what needs to be implemented in the JWST Pipeline for MIRI/MRS TSO mode. Discussion currently ongoing (see this and this ticket). 





4. Closing remarks of the meeting
  • Tony mentioned that within NIRSpec they have their own program to test each of the individual modes NPTT (NIRSpec Python Testing Tool). This is automated; nothing in it for TSO at this point, but they'll be trying to flesh this out now.