Page History

• Thanks everyone for sending feedback on our TSO WG schedule! Seems all is approved; Nestor Espinoza will work on putting that on the WIC tasks.
  
  
• Sarah Kendrew mentions the fact that we discussed before christmas about a student from Paris that developed a tool called exonoodle — this basically "hacks" MIRI-sim in order to produce time-series observations. She's giving a talk next Tuesday (February 2nd) in the Science Coffee slot (11:30 AM ET). If you are interested on hearing about this — please join the Science Coffee!
  
  
• Brian Brooks makes an announcement on NIRCam rehearsals. Happening from Wednesday, March 24th to Sunday, March 28th. Important as it will happen during TSO WG meeting. People will be at the MOC.
  
  
• Nikolay Nikolov also updates on the development of the NIRCam TSO data challenge; meetings were moved to Fridays, at 13:30 (ET). If interested, meeting link will be shared in the Slack by Nikolay Nikolov. Currently, this is focused on differences between GRISMC and GRISMR (i.e., read on the "1/f" direction or perpendicular to it), through MIRAGE simulations.  
  
  
• Nikolay Nikolov also mentions that work is ramping up on the instrument reviews for JWST+HST. To keep in mind for our TSO WG tasks.

2. TSO oulier detection algorithm updates

 Nikolay Nikolov Sarah Kendrew

• Sarah Kendrew reminds everyone of the problem of this algorithm: idea is to flag pixels not captured by the Jump detection step in Stage 1. This algorithm is at Stage 3. Current algorithm is not good for TSOs; currently implementing and testing an algorithm implemented by Nikolay Nikolov. See 

  Jira
  server STScI JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId aac911eb-9c55-3f14-8368-77b436fb6dc2 key JP-1655

   and slides from previous meeting for an illustration of the problem and description of the proposed algorithm.
  
  
• Nikolay Nikolov has performed some testing on JWST-like data already with his algorithm. (see slides). The original versus corrected for one of the images looks great! Results are encouraging. The data used are MIRAGE simulations of a transiting exoplanet, for NIRCAM on GRISMR. Using the rateints images. Sarah Kendrew mentions that really, the calints outputs will be used — but Nikolay Nikolov mentions he used this because he was really worried about the 1/f patterns. Cosmic rays injected in the MIRAGE simulation by Brian Hilbert. Used 4 images before, 4 images after — basically scan each line of the detector, and checks everything that is up and below of 3.5-sigma. Did the same using a box to identify hot/cold pixel. For the correection, used 3 columns before and after, median combining those, and used that to obtain the PSF of the column of the problematic pixel. The correction is also pretty good!
  
  The difference images are interesting, as they do show (small, but observable) residuals of the trace. However, this is accounted for by the algorithm. What remains to do is to extract the spectra/time-series from the non-corrected and corrected image to see the differences. 
  
  
• Sarah Kendrew asks how the cosmic rays are injected. Nikolay Nikolov mentions these are obtained from a library and they are done internally by MIRAGE. Sarah Kendrew mentions it would be good to check the positions of those cosmic rays to see the efficiency of the algorithm on detecting these outlier pixels. She suggests this might help out to define the sigma thresholds, also determine the false-positive rate, etc. 
  
  
• Sarah Kendrew mentions that we are not aiming to put this algorithm in the flight build, so we have time to keep doing analyses. Basically, we need to find the parameters/methods that work best for NIRCam first; then do the same for the other instruments. This will take time, and Nikolay Nikolov will work on this — however, we have to define in the meantime what we should do with the currently implemented outlier_detection method. She mentions, for instance, that for MIRI it might be better to skip it. Nikolay Nikolov mentions that an alternative would be to put a very high threshold on the current algorithm.
  
  
• Diane Karakla asks if the algorithm would work even better perhaps if the 1/f noise is removed before combining the difference images. Nikolay Nikolov will work on this by the next meeting — was in his plans as well! His feeling is that this will have to be done on a more-or-less iterative way because of the cosmic rays themselves (i.e., do a first pass without flagging cosmic rays, then come back at removing 1/f noise flagging the previously flagged pixels).
  
  
• Nestor Espinoza reminds everyone on the fact that this algorithm is only supposed to flag pixels. What to do with them will be the subject of each of the spectral extraction algorithms. 
  
  
• Nikolay Nikolov will present more updates on this analysis on the next TSO WG meeting.

Nestor Espinoza

• Nestor Espinoza gave a presentation on the first part of our analysis, done on the NIRCam darks, which can be found in this repository. The idea is that with this notebook, folks can reproduce the analyses on NIRSpec and NIRISS.
  
  He reproduces Figure 4 on Everett's paper. The only change he proposes to that analysis is that instead of cutting outliers by using a threshold of 80 counts, one uses some outlier-resistant metric to estimate the dispersion of the flux counts like the median-absolute deviation, and uses that to perform n-sigma rejection of outlier pixel values. He compared that in his notebook and obtained the same results as Everett's. He reminds everyone that the full description of that paper and the analysis we are performing can be found on our December 6th meeting notes.
  
  
• Nikolay Nikolov mentions it would be good to plot the power spectrum of the residuals, once column-to-column or row-to-row substractions are made, in order to see the best-case scenario of the removal of these 1/f components. Nestor Espinoza mentions that this is indeed what needs to be done after producing these initial power spectra; perhaps also varying the number of pixels used in this removal in order to emulate "real" spectral extraction algorithms.
  
  
• Diane Karakla asks about the rest of the peaks in the periodogram. Are they important also to model/analyze? Nestor Espinoza suggests that most of those are aliases and harmonics of a "master" frequency, due to the 120 microseconds "jumps". Diane Karakla suggests it would be worth analyzing those, as there might be effects besides 1/f noise which we might want to take care of. Nestor Espinoza agrees.
  
  
• Loic Albert asks how this power spectral density analysis could help with the actual removal of 1/f noise. Nestor Espinoza mentions that there are various objectives on doing this power spectral density (PSD) analysis. First, to verify that the slope, shape and peaks overall on the PSD are similar between instruments and readout modes (the answer should be yes, but we need to show that). Second, this very same analysis can be used to figure out optimal "length/time"-scales to remove 1/f noise, allowing us to figure out which subarrays would be the most impacted by this. Third, with this information, the smallest subarrays, on which there is little-to-no non-iluminated pixels, would be greatly benefited either with extraction algorithms that take this frequency information into account or perhaps doing removals that are not only column-to-column (or row-by-row in the case of NIRCam), but that take several columns as samples to remove the effect from individual columns.
  
  
• Loic Albert mentions he did do the analyses a while ago on SUBSTRIP256, and it seemed the overall shape agreed with what we show here. Unknown User (aroy) is currently working on getting this same PSD in order to plot one on top of the other to check this; will repeat analysis on SUBSTRIP96 as well.

• Sarah Kendrew mentions that she had an ESA meeting on the morning, where they are working on a JWST TSO visualization tool. She'll be testing this environment, but they need some data to test it. She asks if there's maybe MIRAGE simulations they could use to test those. Nestor Espinoza mentions that in principle this could be done because MIRAGE simulations, for instance, are open source. Nikolay Nikolov mentions that the NIRCam simulations are public; he'll coordinate with Sarah Kendrew on this. Loic Albert asks what the tool is about (i.e., lightcurve fitting?). Sarah Kendrew mentions this is pure visualization.
  
  
• Loic Albert asks if there's any group or team at ESA or STScI working on lightcurve analysis tools. Nestor Espinoza mentions that ExoCTK has this under its radar; in particular, exploring how these could be made in a cloud computing environment. He'll touch base on this with him offline. Sarah Kendrew suggests that this could also be presented in a future meeting — Nestor Espinoza would be happy to!