This page archives Slack comments from the splinter session on WFSS of the Improving JWST Data Products Workshop (IJDPW).
Jeff Valenti
Set the channel description: Discussion about wide-field slitless spectroscopy, one of the topics for days 3 and 4 of the workshop
Thu and Fri of the workshop this week will be a hands-on exercise comparing data products generated by the community (Gabe in the case of WFSS) and the STScI pipeline. In the case of WFSS, our in-person breakout session is Thu 9:30-12:30 in TBD room. I will coordinate. Some questions we may want to address:
- What are the key differences between WFSS data products generated by Gabe and the STScI pipeline?
- Which steps should be handled automatically during data reduction and which should be handled by the user during data analysis?
- What changes should be made to the STScI pipeline over the next year? On a longer timescale? Never?
To answer the first question and inform discussion of the remaining questions, it would be useful to have and compare data products generated by Gabe with products generated by the latest version of the STScI pipeline and reference data.- Please suggest one or more data sets for which you already have products or for which you can create products this week. Public datasets are preferred, but not required.
- Please comment on the agenda described above for the breakout session.
This channel will be our primary means of asynchronous communication before and after our in-person breakout session.
Jo Taylor
Program 1571 (PASSAGE) would serve as a good test case. If we want to really probe the worst-case calibration scenario, I would suggest using F200W, otherwise any wide-band filter would do. If others agree, I can find some interesting fields in this program. (edited)
I would also suggest using 2079 (NGDEEP) since that may streamline comparisons between jwst pipeline and grizli.
Of course, this is just for NIRISS, I can’t speak to NIRCam\
Rachel Plesha
I was looking at the distribution of filters used by GO/GTO for NIRISS, and the only three filters used so far are F200W, F150W, F115W, with F200W being used the most, so I agree that using that filter would be really useful. I would be interested to see another filter though, perhaps F150W because we kA note that we got moved from the Auditorium to the CafeConnow there are some calibration issues that appear to be worst for F200W.Additionally, I think it would be insightful to have a field that is relatively sparse and one that is more crowded, which in particular might really highlight the contamination correction grizli is doing better if I understand correctly among other things. @Jo Taylor (she/her) do you have any of these in your back pocket from all of your analysis?
Jo Taylor
I can find some F150W data as well 
If we’re strapped for time, it may be more efficient to actually use the worst-case scenario, to showcase the difference in calibration but good to have both sides of the coin for sure!Most non-cal programs are relatively sparse, at least compared to our cal fields… “crowded” is in the eye of the beholder 
I can certainly share some where contamination is an issue though!
Jeff Valenti
@Alicia Canipe - Adding you to this channel in hopes that you will participate in the breakout session tomorrow morning.
Jo Taylor
Also tagging @Gabe Brammer since I see he is not in this channel already. You’re planning on attending this breakout tomorrow, I assume?
Steph LaMassa
for a crowded field WFSS observation, one possibility is the NIS-10 commissioning observation when we took confirmation images. Program ID 1085, Observation 12
Jo Taylor
I’m not convinced we actually need to do such a thing in this session, since this is basically only done in CAL programs and not a typical WFSS field. I would like to hear from external folks if this would be useful given the limited time we have.
Alicia Canipe
For NIRCam, our typical WFSS test data programs are 1076 (NIRCam LW Grism Baseline Characterization), or the CEERS survey (program 1345)
Jeff Valenti
@Nor Pirzkal - Adding you to this channel in hopes that you will participate in the breakout session tomorrow morning.
Jo Taylor
@Gabe Brammer Are there any observations you would want to look at together tomorrow?
Jo Taylor
For NIRISS, I would suggest using program 1571 obs 304 for GR150R, and obs 302 for direct image (both F200W). This is a typical WFSS field with some contamination. It would be helpful if folks can download products for these observations in advance:
https://mast.stsci.edu/search/ui/#/jwst/results?resolve=true&dataset_id=JW01571302%2a,%20JW01571304%2a&useStore=false&search_key=5b67e3cea25e8
Jo Taylor
I have no idea who is “organizing” this session, but I have downloaded the above data and calibrated it using jwst v1.12.5. If you are interested in calibrating data too, perhaps you should use this version as well. You can do so by following these instructions (edited)
Good morning WFSS friends 
Suggested agenda for today’s session (modifications most welcome!):
- @Gabe Brammer can give a
grizlitutorial using the same WFSS dataset that @Jo Taylor (she/her) has already processed with the latest version of the pipeline (see above). - Results from pipeline and
grizlican be compared. - Discussion topics:
- What updates to calibrations are needed to improve results?
- What updates to pipeline processing are needed to improve results?
- What are the timescale of updates to calibration and pipeline?
- Are there any “quick wins” we can implement to better support the community while developing a long-range strategy for better support?
I encourage folks to take notes about the discussion in this channel. I’ll take notes too, but since I’m remote and the audio in the auditorium for remote folks isn’t super duper unless someone is speaking into a mic, there may be really great points during the discussion that I might miss.
After the session ends, I’ll create an organized summary of the discussion from this channel in either a Word Doc or slides so we’ll have it for future reference.
Hi folks, if anyone would like some NIRCam data to use without getting it from MAST, I put some 1076 data on Box here (I’m not sure if the permissions will work right, I’m bad at Box): It includes calwebb_spec2 and spec3 associations, the direct image and catalog, and uncal, .rate, and .cal WFSS fits files processed with Build 10 of the pipeline. I think everything should all be there, but haven’t had a chance to run the associations to make sure I grabbed all the files 
(edited)
Gabe noticed that astrometry for pure parallel WFSS programs is off. Dither offsets from prime mode seem to not be propagated to WCS for parallel WFSS exposures.
- Can be corrected off line, but better for it to be done by pipeline
- May already be a ticket for this, if not, we’ll file one.
- Sounds like ticket should be boosted to high priority given community feedback Action item - quick win
See Gabe’s notebook for steps that
grizlidoes that isn’t yet default in pipeline (Run preprocessingsection of notebook). High level summary: manual 1/f correction, snowball correction, and treating grism exposures as “images” for updating WCS to go from pixels to sky coordinates.- Can update SIP WCS to be more precise (i.e., higher order). Requested update - quick win
- Max order for SIP is 5 - works well for NIRISS. Can pick 5th order as default (rather than trying lower orders and accepting those if residuals are low, and then increase order if needed)
Jeff Valenti
Gabe uses maximum of 5th order for SIP.
Gabe says Legacy Survey is a good source for catalogs to use for computing astrometric corrections.
ALicia Canipe
Resample was rewritten to allow things to process on disk rather than in memory. Default resample step is still set to do everything in memory, so we should consider turning that off (just a parameter file update)
Jeff Valenti
Gabe suggests pre-defining a grid for the entire sky and then making level-3 products on that grid. This greatly simplifies building larger mosaics and making cutouts.
Gabe starts by assuming contaminating spectra are "flat" (in f_lambda, I think). Then he extracts spectra starting with the brightest, fitting a polynomial to flux vs. wavelength. Two or three iterations does improve contamination correction. One could use the actual extracted specturm, but Gabe does not do that.
Steph LaMassa
Gabe’s derived *.conf files seem to work better than the specwcs reference file. Maybe we could use Gabe’s *.conf files as a starting point for updating the NIRISS calibration?
Rachel Plesha
there hasn’t been a lot of checking yet how much better the conf files are doing compared to our derived conf files (for NIRISS). It would be easy to use the .conf files to create a specwcs to compare to the current default pipeline
Howard Bushouse
I think "we" may have tools available to convert the .conf file info into the equivalent specwcs format.
Rachel Plesha
Yes we do. That is part of the analysis I have been doing, Howard
Steph LaMassa
Speed up contamination modeling in pipeline. Other codes (like grizli) run quicker.
Jeff Valenti
Gabe assumes that all pixels in an image segment have the same relative mapping to wavelength.
Alicia Canipe
STScI’s wfss_contam step is off by default due to discrepancies that have been observed between the modeling and the simulated spectra (flux discrepancies, models are offset — e.g. on the order of a few pixels). wfss_contam is based on Nor’s code. Issues are seen in both NIRISS and NIRCam data. (edited)
Are there updates to default parameters that can improve spectral extraction in the pipeline?
Howard Bushouse
I think the params used to create the source catalog and segmentation map from the direct image could use tweaking. In the past we've seen segmentation maps that cover ridiculously large regions for some sources (i.e. the thresholds could be set higher, so that the segmentation map is more confined for each source).
Jo Taylor
Files are now in /user/jotaylor/NIRISS/jwst_workshop
Howard Bushouse
Can you supply one of the rate files for a grism exposure, so that we could attempt to run it up through the wfss_contam step in calspec2? The contam step gets applied before photom, so the cal file won't work as input to that step. We have to start back further in the flow (after extract_2d and srctype).
Jo Taylor
You can try this one: /user/jotaylor/NIRISS/jwst_workshop/jw01571304001_08201_00001_nis_rate.fits
Howard Bushouse
The jwst_workshop directory is set to group "wit", so I can't read it. Resetting to "science" should do the trick.
Jo Taylor
will do try now
Gabe Brammer
The updated configuration files Jasleen Matharu created are at https://github.com/gbrammer/grizli/pull/128
Gabe Brammer
It's pretty ugly, but here is the function that "disperses" the pixels within an object segment: https://github.com/gbrammer/grizli/blob/master/grizli/utils_c/disperse.pyx#L26
def disperse_grism_object(np.ndarray[FTYPE_t, ndim=2] flam,
np.ndarray[FTYPE_t, ndim=2] segm,
FTYPE_t seg_id,
np.ndarray[LINT_t, ndim=1] idxl,
np.ndarray[DTYPE_t, ndim=1] yfrac,
np.ndarray[DTYPE_t, ndim=1] ysens,
np.ndarray[FTYPE_t, ndim=1] full,
np.ndarray[LINT_t, ndim=1] x0,
np.ndarray[LINT_t, ndim=1] shd,
np.ndarray[LINT_t, ndim=1] sh_thumb,
np.ndarray[LINT_t, ndim=1] shg):
"""Compute a dispersed 2D spectrum
Parameters
----------
flam : 2D array
Direct image cutout in units of cgs f-lambda
segm : 2D array
Segmentation cutout
seg_id : float
Object ID, which should match some pixel(s) in ``segm``
idxl : 1D int array
Pixel indices of the dispersed pixel spectrum in the grism exposure relative to a pixel index in the direct image
yfrac : 1D float array
The pixel indices are two cross dispersion pixels per wavelength pixel, centered on the trace. This array indicates what
how the flux is distributed in those two spatial pixels at a particular wavelength, accounting for the trace centering
ysens : 1D float array
The model spectrum to disperse, e.g., a template multiplied by the sensitivity curve
full : array
Container for the dispersed spectrum
x0 : int
some pixel index....
shd, sh_thumb, shg : tuple
shapes of the direct image, cutout and grism image
Returns
-------
The dispersed spectrum is stored in-place in ``full``
"""
Jo Taylor
My single-extracted source to match Gabe’s notebook is here: /user/jotaylor/NIRISS/jwst_workshop/improved_1.12.5/single_object
If anyone wants it on Box, let me know
Gabe Brammer
I think the dispersed spectrum ``full`` is a flattened version of the 2D grism cutout
Jo Taylor
Here is a different object to extract: 265.7746515029047 67.09450625380478
Howard Bushouse
Not directly relevant to comparing grism results, but if anyone is interested in the issues we currently have with WFSS taken in parallel: JWSTDMS-787 - Getting issue details... STATUS
Jeff Valenti
The only tickets I could find relevant to WCS of pure parallels is JP-3200 - Getting issue details... STATUS and linked tickets.
a = datamodels.open('/Users/tpauly/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0049.asdf')
>>> a.dispx
[[<Polynomial2D(2, c0_0=-0.2840988, c1_0=-0.00000119, c2_0=0., c0_1=-0.00000032, c0_2=0., c1_1=0.)>, <Polynomial2D(2, c0_0=-0.08561026, c1_0=0.00000346, c2_0=-0., c0_1=-0.00000007, c0_2=-0., c1_1=-0.)>, <Polynomial2D(2, c0_0=-0.00146268, c1_0=-0.00000217, c2_0=0., c0_1=-0.00000012, c0_2=0., c1_1=-0.)>], [<Polynomial2D(2, c0_0=-0.4121956, c1_0=0.00000065, c2_0=-0., c0_1=-0.0000001, c0_2=-0., c1_1=0.)>, <Polynomial2D(2, c0_0=-0.08613369, c1_0=-0.00000104, c2_0=0., c0_1=-0.00000015, c0_2=-0., c1_1=-0.)>, <Polynomial2D(2, c0_0=-0.00013161, c1_0=0.00000172, c2_0=-0., c0_1=-0.00000014, c0_2=0., c1_1=-0.)>], [<Polynomial2D(2, c0_0=0.1040224, c1_0=-0.0000138, c2_0=0.00000001, c0_1=-0.00000355, c0_2=0., c1_1=-0.)>, <Polynomial2D(2, c0_0=-0.5755887, c1_0=0.00004502, c2_0=-0.00000002, c0_1=0.00000591, c0_2=-0.00000001, c1_1=-0.)>, <Polynomial2D(2, c0_0=0.01514786, c1_0=-0.00003239, c2
Bryan Hilbert
Proposed verification test: choose several locations across the detector, and check the value of the position offset between the dispersed source and the 2D model of the source, keeping track of which specwcs files are used each time. Howard reports offsets in the current pipeline contam step of ~2 pixels. Gabe finds offsets on the order of 0.1 pixel. It's not clear what is causing the 2 pixel offset in the pipeline currently.
Howard Bushouse
Regarding the possibility of a more useful intermediate product, the hook that's in the Spec2 processing of grism images to save an image in units of e/sec does in fact happen right after the flatfield step (so it's still a full-frame image, before cutouts are created). I'm guessing that right now those e/sec intermediate products are not created/saved by default. Could be activated with a param ref file update.
But that spot in the flow does mean that the master grism background has already been subtracted, so if that's not useful or desired, the flow would need to change.
Alicia Canipe
From Tyler: spec2 param ‘save_wfss_esec’
Gabe Brammer
The trace comparison I showed does seem to correctly use all coefficients of the trace polynomial.
the dotted lines are evaluated straight from the "dispx" and "dispy" polynomial models, transformed to the +x dispersed frame
import grizli.grismconf
grizli_file = grizli.grismconf.get_config_filename(instrume='NIRISS', filter='F200W', grism='GR150R', use_jwst_crds=False)
conf = grizli.grismconf.load_grism_config(grizli_file)
conf.get_beams()
crds_file = grizli.grismconf.get_config_filename(instrume='NIRISS', filter='F200W', grism='GR150R', use_jwst_crds=True)
cc = grizli.grismconf.load_grism_config(crds_file)
cc.get_beams()
import matplotlib.pyplot as plt
x0, y0 = 1024, 1024.
fig, ax = plt.subplots(1,1,figsize=(10,5))
for k in cc.beam_names:
b = cc.beam_names[k]
if b not in cc.dxlam:
continue
dx = cc.dxlam[b]
crds_dy, crds_lam = cc.get_beam_trace(x=x0, y=y0, dx=dx, beam=b)
grizli_dy, grizli_lam = conf.get_beam_trace(x=x0, y=y0, dx=dx, beam=b)
pl = ax.plot(dx, crds_dy, label=f'CRDS {b}', alpha=0.5)
ax.plot(dx, grizli_dy, alpha=0.5, linestyle='--', color=pl[0].get_color())
# Show directly from the datamodel
import jwst.datamodels
import numpy as np
t = np.linspace(0,1,128)
dm = jwst.datamodels.open(crds_file)
for i, o in enumerate(dm.orders[:-1]):
cdx = np.array([d(x0, y0) for d in dm.dispx[i]])
cdy = np.array([d(x0, y0) for d in dm.dispy[i]])
#clam = np.array([d(x0, y0) for d in dm.displ[i]])
dxi = np.polyval(cdx[::-1], t)
dyi = np.polyval(cdy[::-1], t)
#lami = np.polyval(clam[::-1], t)
#plt.plot(dxi, dyi, label=f'Order {o}')
ccx, ccy = cc.transform.forward(x0+dxi, y0+dyi)
plt.plot(ccx-x0, ccy-y0 + 0.1, label=f'Order {o}', linestyle=':')
plt.legend()
ax.grid()
ax.legend()
Jeff Valenti
The WFSS breakout session came up with the following list of desired improvements:
- Change default flag value to have one source image at a time in memory when resampling
- Correct WCS for pure parallel images (dither offsets not included, issue for all modes)
- Use 5th order polynomial description in SIP header
- Create test that measures offset between actual trace and trace in the 2D model at a few representative points in the field of view
- Add spatial dependence to trace location model for NIRISS (in progress)
- Astro_query should return coordinates of object in cutout or spectrum, not coordinates of full image.
- Optimize source extraction parameters (in progress)
- Reduce time required to add one source to contamination model (from 1 minute to 10 ms).
- The WFSS breakout session came up with some more strategic ideas:
- Leave level 3 to the science users
- Provide rate image with WCS and flat field (which is what users want)
Gabe Brammer
@Jo Taylor (she/her) @Rachel Plesha Reminder if you could please send me the "conf" version of your trace configuration before it gets turned into a specwcs file
Rachel Plesha
I’m double checking that I’m sending you the right version, but I’ll send it very soon
This is still very much a development version, but should be an improvement for what is in the pipeline. I attached an example plot I have too of the change in the cutouts (best case example, there are still some that are still off by several pixels).It will be interesting to see how these compare, although I think @Jo Taylor (she/her) found notes that say that Nor is fitting each spectral order independently. Her fits are used as data points and then refit taking into account the wavelength. It will be informative to know if this approach really does skew what’s in the final specwcs file.
R_Plesha_NIRISS_F200W_GR150R.conf
R_Plesha_GR150R_mosaic8_paterson_new_ref_vs_16Nov23_workshop_cutouts.html
We are in the boardroom (which appears to not be occupied by a splinter) if you are not in another session now and would like to talk more today.
And here is the one that created what is in the pipeline now, if that’s helpful. It has improved wavelengths, but not traces.
R_Plasha_current_pipeline_NIRISS_F200W_GR150R.conf
Gabe Brammer
I'm in the extractions session but will come by soon
I'll hang in the cafe for a little while still this afternoon, otherwise we can connect tomorrow whenever works for you. I never did get an ID card to get upstairs.
Jo Taylor
sounds good, how about tomorrow after lunch? Does 1:30 work? We can meet in the cafecon again
Gabe Brammer
Yes, that should be fine
Hi. I'm in the debrief in the auditorium and will come meet you when we finish
Rachel Plesha
Sounds Good !
Gabe Brammer
Could it be that the "t" polynomials are just not calculated correctly given your trace measurements? Here is a version of the plot from yesterday that includes the file you sent above with the thick lines. They're closer to my traces in the dashed lines, but the curvature doesn't appear to be quite correct in that the different orders are different segments of a single curve.
Another position:
Jo Taylor
what is the difference between the thick and thin lines ?
Gabe Brammer
the thin lines is the CRDS specwcs reference file jwst_niriss_specwcs_0049.asdf, thick is NIRISS_F200W_GR150R.conf and dashed is GR150R.F200W.221215.conf from me and Jasleen (edited)
Gabe Brammer
I think you'd like to have a single set of DISPX and DISPY values and implement the orders with different DISPL, where the opposite is what is currently implemented
Jo Taylor
@Gabe Brammer @Rachel Plesha @Paul Goudfrooij Did you want to meet tomorrow afternoon to talk WFSS calibration? Happy to meet up at Muller
Gabe Brammer
@Paul Goudfrooij I was hopeful that your suggestion to just use RA_V1, DEC_V1 and PA_V3 to set adjust the pure-parallel pointing, but it seems like those values are themselves not populated correctly. Here is an example of a Prime + PurePar exposure pair. If things were correct shouldn't RA_V1 and DEC_V1 be the same in both files?
FILE DATE-OBS TIME-OBS INSTRUME APERNAME jw01571084001_02201_00001_nis_rate.fits 2023-01-07 06:42:27.098 NIRISS NIS_CEN jw01933001002_02101_00001_nrca1_rate.fits 2023-01-07 06:42:37.786 NIRCAM NRCA1_FULL FILE RA_V1 DEC_V1 PA_V3 RA_REF DEC_REF PA_APER jw01571084001_02201_00001_nis_rate.fits 150.51780208841888 2.2362378291515537 295.0310352069688 150.6593799676183 2.0809023947009675 295.59749808603055 jw01933001002_02101_00001_nrca1_rate.fits 150.5178173814962 2.236239113629607 295.0310354611394 150.66487886501164 2.2045885348272405 294.4903337070566
Kevin Wolk
There is the instrument to instrument offset in the focal plane
Gabe Brammer
I just verified that RA_V1 and DEC_V1 are consistent with the sciaf apertures and attitudes set from RA_REF, DEC_REF, ROLL_REF, i.e., that they're both wrong for the PP exposure.
Jo Taylor
@Gabe Brammer I got an error running your WFSS grizli example notebook, I’ll post it in thread
sh: aws: command not found
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[6], line 11
2 prep_args = {'oneoverf_kwargs': {'deg_pix':2048, 'dilate_iterations':3,
3 'thresholds':[5,4,3],
4 'other_axis':False},
5 'align_clip':32,
6 # 'reference_catalogs':['LS_DR9'],
7 }
9 os.chdir(path)
---> 11 visit_processor.process_visit(assoc, clean=False, sync=False,
12 with_db=False, # run without DB credentials
13 do_make_visit_mosaic=False,
14 prep_args=prep_args,
15 )
File ~/miniconda3/envs/grizli39/lib/python3.9/site-packages/grizli/aws/visit_processor.py:1327, in process_visit(assoc, clean, sync, max_dt, combine_same_pa, visit_split_shift, blue_align_params, ref_catalogs, filters, prep_args, fetch_args, get_wcs_guess_from_table, master_radec, align_guess, with_db, global_miri_skyflat, tab, **kwargs)
1324 if os.path.exists(assoc) & (clean > 0):
1325 os.system(f'rm -rf {assoc}*')
-> 1327 os.environ['orig_iref'] = os.environ.get('iref')
1328 os.environ['orig_jref'] = os.environ.get('jref')
1329 set_private_iref(assoc)
File ~/miniconda3/envs/grizli39/lib/python3.9/os.py:684, in _Environ.__setitem__(self, key, value)
682 def __setitem__(self, key, value):
683 key = self.encodekey(key)
--> 684 value = self.encodevalue(value)
685 putenv(key, value)
686 self._data[key] = value
File ~/miniconda3/envs/grizli39/lib/python3.9/os.py:756, in _createenviron.<locals>.encode(value)
754 def encode(value):
755 if not isinstance(value, str):
--> 756 raise TypeError("str expected, not %s" % type(value).__name__)
757 return value.encode(encoding, 'surrogateescape')
TypeError: str expected, not NoneType
Jo Taylor (she/her)
12 days ago
Rachel pointed out that I needed to do the aws dependency install. I tried that but got this error at the end of pip install "grizli[aws]" :
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
aiobotocore 2.7.0 requires botocore<1.31.65,>=1.31.16, but you have botocore 1.32.2 which is incompatible.
Rachel Plesha
I appear to have gotten it successfully running past where Jo got an error by doing a pip install grizli[jwst, aws, test] however pip install[jwst, aws] gives the error Jo pointed out above
t heard me say it was running and decided to crash now. I think this is a slightly different error than what Jo got (and mine was running for the past 20 minutes or so where Jo’s failed much more quickly).
Gabe Brammer
Yikes. I'm not quite sure how to diagnose that error, but we can look at it together after lunch.The other issue is that the code needs to have iref and jref environment variables set (from the old HST days) even though those aren't used for anything in JWST.
Rachel Plesha
I think that might be the issue. I was trying to reproduce your other plot and was running into that issue, too, but I think I got it fixed now. I’ll give it a try and see what happens.
Paul Goudfrooij
@Gabe Brammer I just started looking into this myself (after picking up son), just having confirmed that RA_V1 and DEC_V1 are indeed consistent with RA_REF and DEC_REF in the parallel exposures but … wow. There is supposed to be only one V1 for the telescope, namely the boresight. I’ll have to look into this in more detail.
Paul Goudfrooij
OK @Gabe Brammer - I just derived the RA_V1, DEC_V1 values using pysiaf under the assumption that RA_REF, DEC_REF are correct (for the prime NIRCam data). They come out to be different from the values in the headers (from your table above) by about 0.054 arcsec (roughly 1 pixel), which is also roughly the difference in RA_V1, DEC_V1 between the NIRCam and NIRISS headers. I think we’re looking at slight differences between the calculations done by PPS (the proposal planning system, which uses Java) vs. those done with the pysiaf package. I’ll look more into this tomorrow
Howard Bushouse
You're right @Gabe Brammer in that RA_V1, DEC_V1 should be the same for both instruments, since the V1 axis is pointed at the same place. It's RA_REF, DEC_REF that should differ for the different instruments (and detectors), because that's the sky coord at the aperture reference point. RA_REF and DEC_REF should essentially be equivalent to RA_V1, DEC_V1 + V2_REF, V3_REF, where V2_REF, V3_REF are the V2/V3 locations of the aperture reference point for each instrument/detector.
Paul Goudfrooij
@Gabe Brammer - could you also list the ROLL_REF value for the NIRCam and NIRISS data of your example above? Wanted to check one more thing. (I see PA_APER, but ROLL_REF is slightly different).
Gabe Brammer
Sorry, here's the extra keyword:
FILE RA_V1 DEC_V1 PA_V3 RA_REF DEC_REF ROLL_REF PA_APER /tmp/jw01571084001_02201_00001_nis_rate.fits 150.51780208841888 2.2362378291515537 295.0310352069688 150.6593799676183 2.0809023947009675 295.03623091603055 295.59749808603055 /tmp/jw01933001002_02101_00001_nrca1_rate.fits 150.5178173814962 2.236239113629607 295.0310354611394 150.66487886501164 2.2045885348272405 295.0367760370566 294.4903337070566
I just pushed an update to my workaround in grizli that tries to get the correct ra_v1, dec_v1 and pa_v3 values from a query to the FGS astroquery table, which worked very well. Some of the shift of the grism model of the test dataset I was showing yesterday is actually itself significantly improved with the update, which is more robust for getting the pointing attitude.
Paul Goudfrooij
FWIW, I just checked (RA_V1, DEC_V1) for another (NIRCam) prime exposure vs. the associated pure parallel (NIRISS) exposure, and they are off by (-0.007, 0.004) arcsec, while the (RA_V1, DEC_V1) in the NIRCam header is equal to the calculated values using pysiaf to within 1e-10 arcsec. So for this dataset, the APT/PPS calculations are consistent with the pysiaf ones, and the NIRCam and NIRISS pointing values are only off by a very small amount.
We’ll have to figure this out, but yes, using the FGS data as the “reference” is a good idea. (edited)
Rachel Plesha
sending the link to the main chanel for screen sharing: https://stsci.webex.com/stsci/j.php?MTID=m32a5051092e4803d1115f49cba3d0b20We have it set up (we think) in the cafe con for people in person
James Davies
Note that jwst.lib.set_telescope_pointing is the code that populates the pointing header information from the telescope telemetry database mnemonics. We’ve had bugs here before.
Jo Taylor
https://jwst-pipeline.readthedocs.io/en/latest/_modules/jwst/lib/set_telescope_pointing.html
for posterity
Rachel Plesha
A : +1,
B : 0,
C : +2,
D : +3,
E : -1
Gabe Brammer
Here's the updated function that pulls the attitude information for pure parallels from the FGS logs:
https://github.com/gbrammer/grizli/blob/master/grizli/jwst_utils.py#L2960
which uses this query
https://github.com/gbrammer/mastquery/blob/master/mastquery/jwst.py#L623
Paul Goudfrooij
Just to clarify: This is currently needed because the WCS in pure parallel data is “off” by of order 1 pix, and it is off by different amounts in each dither (!), so that spectral extractions from the dithered grism exposures are off when “blindly” using the WCS info in the header.
