Page History

2. Plan for build 7.5 pipeline testing throughout the summer

Nestor Espinoza introduces the fact that we have to provide feedback to the CalWebb WG on the JWST Calibration Pipeline Build 7.5 (mainly through opening of Jira tickets). The hope is that this also serves as input to the community to know what works, what doesn't — and of course, for us. He notes that we should not only provide feedback on the final TSO3 Stage, but also on Stage 1 and Stage 2 (both for imaging and spectroscopy), following the TSO steps that are taken in those stages. Sarah Kendrew notices the importance of doing this for imaging, where they found a bug in the MIRI imaging mode (see below); Nikolay Nikolov also mentions this is valuable based on his experience (see below); Nestor Espinoza has been having a similar experience with NIRISS/SOSS testing on his side. The team agrees that this is the way to go.

The proposal Nestor Espinoza gives to the TSO WG is to do this in a (2-week) sprint: basically, work together in the problem in order to have quick feedback about possible steps that might fail (which might be useful to test if this is due to problems in the simulations or an actual bug on the pipeline). Sarah Kendrew likes this idea as well. Nikolay Nikolov and Tony Keyes would prefer to do the testing on their own, at their own pace. On the one hand, Nikolay doesn't think he can commit to a 2-week effort, whereas Tony has to consult with the experts testing the pipeline in NIRSpec (James M. and Maria P.). The team thus agreed for Nestor Espinoza and Sarah Kendrew to do a testing sprint, report that to the team, and then for Nikolay Nikolov and Tony Keyes to perform their own testing and report back to the team. 

Nestor Espinoza

• Nestor has been working in the past two weeks on a JDAT notebook for transiting exoplanets. He has both worked on the actual simulations, calibration and data reduction, doing effectively end-to-end simulations for NIRISS/SOSS. Has notebooks for those three steps. He showed part of his notebook today, and David Sing asked if he could share how he did the transit fits on his notebook, as his notebook was a bit slow on this side. Nestor Espinoza mentions that he used the library he wrote (juliet; http://juliet.readthedocs.io/) for this, which in his case fits the data in a couple lines of code. Will share this with David offline.

Brian Brooks Nikolay Nikolov

• Nikolay has been summarizing all the results of his testing of the JWST pipeline that he did based on this ticket. He has a docs document for this.
• Also looking into the TSO Quicklook tool details, as well as the time-stamps problem definition. Will have a meeting tomorrow about this.
• Brian also mentions that the Synphot refactoring for the ETC release is done (thanks to everyone that helped out with this!).  He further mentioned that the ETC & JWST MO would, in general, like to have better coordination with ExoCTK. They will be defining how to do this in practice in the next weeks. Nestor Espinoza agrees with this, and is happy this is happening — they have had problems in the past because of this.

• Worked on reviewing TSO Jira tickets, and had meetings with James M. to get up to speed on this. Of interest is the fact that the JWST Pipeline for NIRSpec is a bit stuck in the rectification step. Nestor Espinoza mentions that he has been providing feedback on this, because although it is understandable that NIRSpec uses rectification in the other modes, for BOTS/transit spectroscopy this is very sensible to do and probably not recommended. For example, an immediate problem with this is the fact that if you rectificate/resample the data to account for the non-vertical wavelength solution, you also pass information on background signals which actually is vertical into a non-vertical orientation in the rectified image (e.g., 1/f noise which is column-by-column in NIRSpec). For more information on this discussion, see this ticket.
• David also chimed in on his analysis of CV3 data. Basically done (as Nestor's notebook above); CV3 data is very rich in terms of systematics, so tons of information embedded in the notebook which will most likely be very useful for users.

Sarah Kendrew