This page archives Slack comments from Day 1, Session 4 of the Improving JWST Data Products Workshop (IJDPW).
Jeff Valenti - Welcome to day 1, session 4!
Jeff Valenti - @David Nidever Links to prometheus and spyderwebb.
David Law - @David Nidever Do the wiggles have a frequency variation that correlates with the pixel-crossing frequency of the trace on the detector?
Jeff Valenti - During Q&A immediately after the talk, @David Nidever answered that the wiggles do not correlate with the spectrum crossing rows. Later during the spectral extraction breakout session on day 3, I suggested that the wiggles may be caused by improper removal of throughput variations caused by the antireflection coating on the order-sorting filter (F100LP).
David Nidever - Here’s the filter throughput on top of the wiggles in the data. I think this could be the cause!!
Jeff Valenti - Indeed, the amplitude and period look similar. The question is why they don't line up. Different temperature and off axis are possibilities that come to mind. The latter can be tested by looking at behavior across the FOV.
David Nidever - I was wondering the same thing. I thought the throughput values might not be super exact, but it could be something else as well. I think looking at the flux calibration standards would also help.
Jeff Valenti - @David Nidever - Links to doppler
Jeff Valenti - Motivated by @David Nidever's talk, should we add an RV correction to the pipeline, based on predicted location in the MSA shutter? The correlation looked quite tight. What about extended sources?
David Nidever - I think that’s a great idea. At least to make that an option. It was difficult to implement this because you have to work with the internal WCS object. People need to dig deep into the code to make this themselves.
Howard Bushouse - Isn't this already handled by the Spec2 wavecorr step: wavecorr description or am I missing something?
David Nidever - That looks like the right thing, but I remember I tried using wavecorr and it didn’t produce the correction that was needed but I should look at it again. It’s been almost a year since I last worked on this in detail.
Anna de Graff - Just to chime in here to say that @James Davies flagged this issue on github a while ago. The wavecorr solution is computed for the flat-fielding but subsequently not used later in the pipeline and combination of spectra. Having this correction makes a significant improvement in the SNR of (often narrow) emission lines of compact high-z galaxies that are commonly observed in multiple visits/dithers. (In the NIRSpec GTO team the ESA/NIPS pipeline does do this correction, and we see substantial changes when turning this feature on or off)
David Nidever - Great. Hopefully that can be fixed in the near future.
James Davies - The issue that @Anna de Graaff mentions is issue 7794 which only concerns the offset in wavelength due to the dispersion-direction offset of the source in the MSA shutter. Basically the correction is done and a wavelength array is written out, but the dispersion in the WCS object is not updated. The effect is that the _cal files have corrected wavelengths, but the _s2d files do not.
Dan Coe - @David Nidever Can you say a few words about dithering in the dispersion direction to improve wavelength resolution?
Marco Sirianni - In this article some discussion on the dithering strategy also to improve spectral sampling: nirspec-dithering-recommended-strategies
Harry Ferguson - @Jeff Valenti Why use the predicted location? Is it impossible to recover the actual location using info from the acquisition?
Tony Sohn - If I understand it correctly, the GWA rotation from grating to mirror causes shifts in positions of stars?
Jeff Valenti - Perhaps also all the same reasons image WCS are sometimes off, e.g., errors in GS coordinates, focal plane knowledge, etc.?
Harry Ferguson - @Tony Sohn Not sure how the GWA rotation would affect the positioning of the targets in the slits if you aren't moving the telescope. (I can see how it would affect the projection of the spectra on the detector.)
Charles Proffitt - The shift in the position on the detector is pretty well calibrated by the sensor on the grating wheel. But there are residual errors in the catalog and distortion solutions that can add up. One thing to check is the post TA reference image which can be used to check how well MSATA worked. Also if you took confirmation images of the targets in the slits that would also be useful. Ask the help desk for an evaluation of your TA reference images.
David Nidever - @Harry Ferguson are there acquisition images at the final position? We had hoped to take images without the grating at the end of the visit, but were told we couldn’t do that.
James Davies - Relatedly, it would be really nice to see a notebook showing how to use the confirmation images to be able to assess source location in the slit.
Sarah Kendrew - Yes @James Davies we would like to do this for MIRI LRS!
Tony Sohn - @Charles Proffitt For stellar RVs, we need very precise locations within each slit. How good is the repeatability of GWA rotation? I thought the uncertainties were comparable to the level we are trying to measure? Also, yes residual distortion uncertainties will cause issues...
Harry Ferguson - @David Nidever confirmation images are optional. Even without them, I wonder if there is some other information available in addition to the traces of the spectra, either from the guider, or from parallel imaging observations, or from the TA metadata.
David Nidever - I forgot to mention the parallel imaging idea/option. All Cycle 3 proposals that my team put in included those to try to improve nailing down the stars in their shutters.
Taylor Bell - To what extent does the described LRS SLIT wavelength calibration issue impact LRS SLITLESS data?
Jeff Valenti - I have this same question.
Sarah Kendrew - It’s a long story but the tl;dr is not at all. our wavelength calibrators in commissioning weren’t the best; for slitless we had a better target, and the calibration there was better from the start. we did make a small improvement to that last year, but the fixed slit mode was the one where we really struggled (because so many known calibrators are too bright for the slit mode).
The longer backstory is that we didn’t realise the field dependence of the LRS dispersion is as strong as it is, because when we did all the ground testing we hadn’t defined the location of the slitlessprism subarray yet. so that was a bit of a surprise and we suddenly realised we had less redundancy in our wave cal plans.
Everett Sclawin - Looks neat @Ori Fox ! Does the Jdaviz + Imviz tool have many of the capabilities of ds9?
Harry Ferguson - Many of the most common capabilities of ds9 are implemented in imviz. (Performance has improved to the level where it is also very similar to ds9.
Ori Fox - Thanks @Harry Ferguson for answering this. The answer is definitely yes, although the layout is a little different. Just some time to get used to.
Jeff Valenti - @Larry Bradley - psf
Savannah Gramze - @Justin Pierel Could we have the links to the github/docs linked here?
Jeff Valenti - space-phot docs
Avinash Ck - @Larry Bradley Hi! How does EPSFBuilder compare to existing routines like PSFEx, especially for crowded images. Also, does photutils also have any automated routine for sample selection for EPSF building.
Larry Bradley - @Avinash Ck I've never run PSFEx to compare. The algorithm in photutils EPSFBuilder is described in Anderson and King (2000; PASP 112, 1360) and subsequent enhancements detailed mainly in Anderson (2016; WFC3 ISR 2016-12). Jay's fortran version was used to generate the STDPSF library PSFs for HST.You could use any of the star finder tools in photutils to define a sample of stars. Ideally you'd want to include only "isolated" stars to input to EPSFBuilder. The IRAFStarFinder has minsep_fwhm keyword to enforce a minimum separation.
Avinash Ck - Great! Thank you for the info. I had attempted building PSF for F200W image of NGC 628 using EPSFBuilder but had no success as the image doesn't have a lot of isolated stars. I will explore it further. Thanks again.
Larry Bradley - For JWST, instead of building your own ePSF model, I would highly recommend using webbpsf. The most recent webbpsf version (v1.2, released in August) produces models that are much better matched to the data.
Avinash Ck - I only have access to stage 3 data from MAST. I performed PSF photometry using DOLPHOT with PSF from WebbPSF, but the residual image was not very good, since stage 3 data is drizzled I guess. The empirical PSF obtained using PSFEx gave much better results than the simulated PSFs.
Gabe Brammer - @Justin Pierel @Larry Bradley Is there a plan from you or Jay Anderson to have a "standard" set of ePSFs for JWST instruments?
Varun Bajaj - Jay and I have done some digging, and it seems its not entirely necessary to make separate ePSFs for NIRCam at least. One of the features I'm hoping to implement in the future will be a perturbation utility, to perturb the WebbPSF models to look more like the actual image PSF. That said, they are pretty similar, and from a PSF fitting perspective, you can do quite well. The main thing is that the WebbPSF models are a bit too sharp.
Gabe Brammer - I guess what I was after is the speed of the precomputed ePSFs.
Varun Bajaj - In terms of not having to generate them from webbpsf?
Justin Pierel - @Gabe Brammer since my stuff is fully focused on single source photometry, I pre-compute the webbpsf psf model at the SN position, or if I’m fitting for position I create a gridded model (equivalent to a gridded psf model from HST) and use that, so it doesn’t have to be generated on each fit iteration or anything like that.
Varun Bajaj - I forget if it was Larry or Marshall that ended up writing a thing for it, but you can save out the webbpsf models to fits files. So it should just be run it once and you have a library of the psf files.
Marshall Perrin - @Varun Bajaj are you finding the models are still a bit too sharp, even after the updates in version 1.2?
Varun Bajaj - I will have to double check, as I hadn't looked at that in a while. I think I had thought it was just Miri that had the charge diffusion fix. I believe I find nircam PSFs have a pretty good q fit value of ~.03 for nircam f150w, meaning the PSF model is very close to the actual data.
Marshall Perrin - Fixes added for all the imaging instruments; NIRCam, NIRISS & MIRI. I tuned the charge diffusion model coefficients to match ePSFs provided by Mattia and Jay, so it should be by construction pretty good, but I fully expect there’s still room to further improve, particularly in terms of wavelength dependence and models for the slight variations over the FOV.
Varun Bajaj - I will make it a point to check it in the hack day!
Larry Bradley - @Varun Bajaj The webbpsf grid models can be precomputed and saved to FITS file. Use these keywords options in webbpsf.psf_grid: save=False, outdir=None, outfile=None, overwrite=True .The next Photutils release (1.10.0; in a few days) can read the webbpsf FITS files directly into a GriddedPSFModel : psf_model = GriddedPSFModel.read('<file>.fits'). It can already read STDPSF FITS files.
Varun Bajaj - That is great Larry, thank you
Sarah Kendrew - @Gabe Brammer MIRI plans to release a set of ePSFs (generated from data, not webbPSF) in the coming weeks.
Everett Sclawin - @Alessandro Savino, do the time-dependent effects decrease for large apertures?
Alessandro Savino - I have not checked that directly. We use an aperture only to define the region over which we perform PSF fit, so I do not think that would change things. I would imagine that, for aperture photometry, this time-dependent PSF variations could have a lesser impact, provided the aperture is large enough to capture most of the flux redistribution.
Marshall Perrin - For detector effects in ePSFs, please note that the most recent version of webbPSF (v1.2, released in August) added models for detector charge diffusion and PSF broadening, which should significantly improve agreement between models and data. Feedback on those very much welcome to @Marcio Melendez and myself. See this docs page for more detail on those models.
Mario Gennaro - Hi @Marcio Melendez, @Marshall Perrin, one thing that I have been pondering in the last couple of months (related to the stellar pops ERS) is the effect of these effects (charge diffusion, brighter fatter, IPC, etc.) on the "level-2" PSF. In general all codes that do photometry using a psf model are run on level-2 or even level-3 products. These codes could either use webbpsf model directly and/or build their own ePSF. Since effects such as charge diffusion are non necessarely linear with time (or I would at least naively think so) I am curious of what would happen if you applied the detector effects at the group stage (or even the frame stage) and then simulated an up-the ramp fit to obtain a level-2 webbpsf. Obviously things would strongly depend on the readout pattern and the source flux, so in practice this may not be a viable approach for psf fitting photometry, but possibly a study of the magnitude of the level-1-to-level-2 differences is warranted?
Marshall Perrin - The nonlinearity of charge diffusion with time (or more specifically with accumulated charge in a pixel) is the brighter-fatter effect. So yes certainly there are differences that will occur depending on readout pattern and source flux, and could be modeled this way. But that level of detector modeling is way beyond the scope of what has been contemplated in webbpsf.
To be clear the model of charge diffusion that we put into webbpsf is an ad hoc simple Gaussian convolution kernel, not anything remotely like a proper ab initio physical simulation of charge transfer within the detector. There do exist codes for doing that rigorously - see for instance Argyriou et al. 2023 for MIRI - but it’s outside the scope of webbpsf.
On a practical level, I think a plausible but probably-lower-effort approach would be to build sets of ePSFs for different flux levels to empirically calibrate the brighter fatter effect (i.e. one set of ePSFs for faint stars, one for near-full-well, one for highly saturated, etc).
Marcio Melendez - I think there is also an effort to include the IPC effect directly into the pipeline. There is already an IPC step that’s inactive due to other changes that needs to happen in other references files because of the IPC kernels. Also, to reiterate on Marshall’s comments, there is a wavelength dependance on all these effect that we don’t include, at least not explicitly as we can fine tune some parameters to match observations with different filters.
