This page archives Slack comments from the splinter session on spectral extraction of the Improving JWST Data Products Workshop (IJDPW).


Jeff Valenti

Thu and Fri of the workshop this week will be a hands-on exercise comparing data products generated by the community and the STScI pipeline. The organizing committee has suggested that we meet (virtually) on Thursday (1:30-4:30 EST)  to discuss spectral extraction. We can work asynchronously before and after that.Some questions we may want to address:

  • What are the key differences between spectra you extract and spectra extracted by the STScI pipeline?
  • Which steps should be handled automatically during data reduction and which should be handled by the user during data analysis?
  • What changes should be made to the STScI pipeline over the next year? On a longer timescale? Never?


To answer the first question and inform discussion of the remaining questions, it would be useful to have and compare your extracted spectra with spectra generated by the latest version of the STScI pipeline and reference data.

  • Please suggest one or more data sets for which you already have products or for which you can create products this week. Public datasets are preferred, but not required. I suggest NIRSpec "fixed slit" spectra (G140H, G235H, G395H) of an L7 brown dwarf from program 2288, observation 3. I am Co-PI on this program, but the data are public.
  • Please comment on the agenda described above for the breakout session.


This will be our primary means of asynchronous communication before and after our breakout session.

Sarah Kendrew

If anyone would like to work with MIRI LRS fixed-slit data,  we created a demo on spectral extraction (with the JWST pipeline extract_1d step) for this mode here: https://github.com/STScI-MIRI/LRS_ExampleNB/blob/main/miri_lrs_pipeline_extraction.ipynb and I will be around on Thursday. In addition, MIRI LRS lead @Greg Sloan is attending virtually.

Dan Coe

let's make sure to discuss this today:

Anna de Graaf posted in #advise-us

some tweaks for the extraction of MSA spectra that I think would be great to have:
1) as discussed in another channel, a wavelength correction for the fact that sources almost never fall in the centre of the shutter can really help with the extraction of faint emission lines of compact sources. This wavecorr is currently already computed to perform the flat fielding, but then not propagated further in the pipeline.
2) the "bounding box" used for the extraction is very tight, e.g. when opening 3 shutters the extraction box is only 15 pixels, meaning that at long wavelengths some flux is missing. As a user it would be nice to have a free parameter that you can set to extract a slightly larger aperture in the cross-dispersion direction, rather than having to hack the *msa.fits file to add fake shutters in order to trick the pipeline. Looking at the source code, I think this should be possible, I just don't know how I would do it as a high-level user. Same goes for the dispersion direction - occasionally we see lines at 5.3-5.4 micron in NIRSpec, but the pipeline can't extract it (I know this will be harder to implement than the spatial direction).

Jean-Baptiste Ruffio

FYI: Continuum normalized spectrum extraction we use to very accurately identify bad pixels



Jeff Valenti

The spectral extraction breakout session came up with the following list of possible improvements:

  • Study PSF variations for NIRSpec within shutter (diffraction) and across FOV
  • Add optimal extraction option(s). Avoid resampling. (default for point sources of bright enough sources)
  • Document the extraction aperture that was actually used (ticket exists)
  • Create notebook showing how to calculate shutter footbprints and spectral bounding boxes on original pixels. Also, expose MSA slitlet mapping to sky, planned source location, etc.
  • When doing box extraction, pixel replacement should happen either during extraction or resampling, so that we don't add interpolated values to spec2 cal files.
  • Consider documenting how to create NIRSpec data products to longer wavelengths, even though we can't calibrate the longer wavelengths now (it's hard).
  • No labels