Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Query

Code Block
languagesql
SELECT o.objID,

...

 
o.raMean, o.decMean, o.raMeanErr, o.decMeanErr,

...


o.qualityFlag,

...


o.gMeanPSFMag, o.gMeanPSFMagErr, o.gMeanPSFMagNpt,

...


o.rMeanPSFMag, o.rMeanPSFMagErr, o.rMeanPSFMagNpt,

...


o.iMeanPSFMag, o.iMeanPSFMagErr, o.iMeanPSFMagNpt,

...


o.zMeanPSFMag, o.zMeanPSFMagErr, o.zMeanPSFMagNpt,

...


o.yMeanPSFMag, o.yMeanPSFMagErr, o.yMeanPSFMagNpt,

...


o.rMeanKronMag, o.rMeanKronMagErr,

...


o.nDetections, o.ng, o.nr, o.ni, o.nz,o.ny,

...


o.gFlags, o.gQfPerfect,

...


o.rFlags, o.rQfPerfect,

...


o.iFlags, o.iQfPerfect,

...


o.zFlags, o.zQfPerfect,

...


o.yFlags, o.yQfPerfect,

...


soa.primaryDetection, soa.bestDetection

...


INTO mydb.[HighFidelityStarsDR2]

...


FROM dbo.fGetNearbyObjEq(334, 0.0, 0.2*60.0) as x

...


JOIN MeanObjectView o on o.ObjID=x.ObjId

...


LEFT JOIN StackObjectAttributes AS soa ON soa.objID = x.objID

...


WHERE o.

...

nDetections>5
AND soa.

...

primaryDetection>0
AND o.

...

gQfPerfect>0.85 and o.

...

rQfPerfect>0.85 and o.

...

iQfPerfect>0.85 and o.

...

zQfPerfect>0.85

...


AND (o.rmeanpsfmag - o.rmeankronmag < 0.05)


Results

696 objects, here are the first 10:

...

The Kepler Extra-Galactic Survey (KEGS) is a program using the Kepler telescope to search for supernovae, active galactic nuclei, and other transients in galaxies. We have to identify galaxies in a suitable redshift range (z<=0.12) a priori, which will be monitored by K2. Here is an example to get galaxies for Campaign 14. We only select objects with r<=19.5, and we make a cut on (rmeanpsfmag - rmeankronmag)>=0.05 in order to remove stars (More info about Star-Galaxy Separation is here). We only want to use objects for which the majority of pixels were not masked, thus the cut on QFperfect>=0.95. We also obtain the petrosian radii in order to be able to select galaxies by size.

Query

...

Code Block
languagesql
SELECT  o.objID,

...


ot.raStack, ot.decStack, ot.raMean, ot.decMean,

...


ot.ng,

...

  o.gMeanPSFMag,o.gMeanPSFMagErr,o.gMeanKronMag,o.gMeanKronMagErr,

...


ot.nr,

...

  o.rMeanPSFMag,o.rMeanPSFMagErr,o.rMeanKronMag,o.rMeanKronMagErr,

...


ot.ni,

...

  o.iMeanPSFMag,o.iMeanPSFMagErr,o.iMeanKronMag,o.iMeanKronMagErr,

...


ot.nz,

...

  o.zMeanPSFMag,o.zMeanPSFMagErr,o.zMeanKronMag,o.zMeanKronMagErr,

...


ot.ny,

...

  o.yMeanPSFMag,o.yMeanPSFMagErr,o.yMeanKronMag,o.yMeanKronMagErr,

...


o.gQfPerfect,o.rQfPerfect,o.iQfPerfect,o.zQfPerfect,o.yQfPerfect,

...


ot.qualityFlag,ot.objInfoFlag,

...


sp.gpetRadius,sp.rpetRadius,sp.ipetRadius,sp.zpetRadius,sp.ypetRadius,

...


sp.gpetR50,sp.rpetR50,sp.ipetR50,sp.zpetR50,sp.ypetR50,

...


soa.primaryDetection, soa.bestDetection

...


       INTO mydb.[C14]

...


FROM MeanObject AS o

...


JOIN fgetNearbyObjEq(160.68333, 6.85167 , 8.5*60.0) cone ON cone.objid = o.objID

...


JOIN ObjectThin AS ot ON ot.objID = o.objID

...


LEFT JOIN StackPetrosian AS sp ON sp.objID = o.objID

...


LEFT JOIN StackObjectAttributes AS soa ON soa.objID = o.objID

...


 WHERE ot.ni >=

...

 3
     AND ot.ng >=

...

 3
     AND ot.nr >=

...

 3
     AND soa.primaryDetection>0
     AND (o.rMeanKronMag > 0 AND o.rMeanKronMag <= 19.5

...

 )
     AND (o.gQfPerfect >= 0.95)

...


     AND (o.rQfPerfect >= 0.95)

...


     AND (o.iQfPerfect >= 0.95)

...


     AND (o.zQfPerfect >= 0.95)

...


     AND (o.rmeanpsfmag - o.rmeankronmag > 0.05)


Results

First 10 out of 161,920 Rows of MyDB Table C14

...

We search for objects in the PS1 DB that are within 3 arcsec of the Catalina positions. Most of the parameters (mean mags, # of detections, etc) we get from the MeanObjectView table. We also join the StackObjectAttributes table in order to get primaryDetection and bestDetection (BestDetection is corrupted in DR2, but will be fixed in DR2.1, see the description of StackObjectThin table for a more detailed description). These RR Lyrae stars are bright, thus we can expect many detections, and we therefore require  o.nDetections>5. We only want to use objects for which the majority of pixels were not masked, thus the cut on QFperfect>=0.95. In order to select only stars (and exclude galaxies), we require (o.rmeanpsfmag - o.rmeankronmag)< 0.05, as described here

Code Block
languagesql
SELECT d.CSS_ID, d.RA as CSSRA, d.Dec as CSSDec, d.V, d.Period, d.Amp, d.Npts, d.Dist, d.Red, d.CSIDnum,

...


o.objID,

...

  
o.raMean, o.decMean, o.raMeanErr, o.decMeanErr,

...

 
o.qualityFlag,

...


o.gMeanPSFMag, o.gMeanPSFMagErr, o.gMeanPSFMagNpt,

...


o.rMeanPSFMag, o.rMeanPSFMagErr, o.rMeanPSFMagNpt,

...


o.iMeanPSFMag, o.iMeanPSFMagErr, o.iMeanPSFMagNpt,

...


o.zMeanPSFMag, o.zMeanPSFMagErr, o.zMeanPSFMagNpt,

...


o.yMeanPSFMag, o.yMeanPSFMagErr, o.yMeanPSFMagNpt,

...

 
o.rMeanKronMag, o.rMeanKronMagErr,

...


o.nDetections, o.ng, o.nr, o.ni, o.nz,o.ny,

...


o.gFlags, o.gQfPerfect,

...


o.rFlags, o.rQfPerfect,

...


o.iFlags, o.iQfPerfect,

...


o.zFlags, o.zQfPerfect,

...


o.yFlags, o.yQfPerfect,

...


soa.primaryDetection, soa.bestDetection

...


 INTO mydb.[RRLPS1]

...


 FROM mydb.[RRLcatalina] d

...


CROSS APPLY dbo.fGetNearbyObjEq(d.RA, d.Dec, 3.0/60.0) as x

...


JOIN MeanObjectView o on o.ObjID=x.ObjId

...


LEFT JOIN StackObjectAttributes AS soa ON soa.objID = x.objID

...


WHERE o.

...

nDetections>5 
AND soa.primaryDetection>0 
AND o.gQfPerfect>0.85 and o.rQfPerfect>0.85 and o.iQfPerfect>0.85 and o.zQfPerfect>0.85 
AND (o.rmeanpsfmag - o.rmeankronmag < 0.05)


Results (25 minutes)

There are 12 out of 13 matches, and below are the first 10 entries

...

Now we get all detections associated with these objIDs

Code Block
languagesql
SELECT

...


o.objID, o.raMean, o.decMean,

...


d.ra, d.dec, d.raErr, d.decErr,

...

 
d.detectID, d.obstime, d.exptime, d.airmass, d.psfflux, d.psffluxErr, d.psfQf, d.psfQfPerfect, d.psfLikelihood, d.psfChiSq, d.extNSigma, d.zp, d.apFlux, d.apFluxErr,

...


d.imageID, d.filterID,

...


d.sky, d.skyerr, d.infoflag, d.infoflag2, d.infoflag3,

...


o.qualityFlag,

...


o.gMeanPSFMag, o.gMeanPSFMagErr, o.gMeanPSFMagNpt,

...


o.rMeanPSFMag, o.rMeanPSFMagErr, o.rMeanPSFMagNpt,

...


o.iMeanPSFMag, o.iMeanPSFMagErr, o.iMeanPSFMagNpt,

...


o.zMeanPSFMag, o.zMeanPSFMagErr, o.zMeanPSFMagNpt,

...


o.yMeanPSFMag, o.yMeanPSFMagErr, o.yMeanPSFMagNpt,

...

 
o.rMeanKronMag, o.rMeanKronMagErr,

...


o.nDetections, o.ng, o.nr, o.ni, o.nz,o.ny,

...


o.gFlags, o.gQfPerfect,

...


o.rFlags, o.rQfPerfect,

...


o.iFlags, o.iQfPerfect,

...


o.zFlags, o.zQfPerfect,

...


o.yFlags, o.yQfPerfect,

...


o.primaryDetection, o.bestDetection

...


 INTO mydb.[RRLPS1det]

...


 FROM mydb.[RRLPS1] o

...


JOIN Detection d on d.ObjID = o.ObjID


Results (30 seconds)

There are 946 detection entries for the 12 objects, and below are the first 10 entries

...

Now we get the forced photometry

...

Code Block
languagesql
SELECT
o.objID, o.raMean, o.decMean,

...


fwm.detectID, fwm.obstime, fwm.exptime, fwm.airmass, fwm.Fpsfflux, fwm.FpsffluxErr, fwm.FpsfQf, fwm.FpsfQfPerfect, fwm.FpsfChiSq, fwm.zp, fwm.FapFlux, fwm.FapFluxErr,

...


fwm.forcedWarpID, fwm.filterID,

...


fwm.Fsky, fwm.Fskyerr, fwm.Finfoflag, fwm.Finfoflag2, fwm.Finfoflag3,

...


o.qualityFlag,

...


o.gMeanPSFMag, o.gMeanPSFMagErr, o.gMeanPSFMagNpt,

...


o.rMeanPSFMag, o.rMeanPSFMagErr, o.rMeanPSFMagNpt,

...


o.iMeanPSFMag, o.iMeanPSFMagErr, o.iMeanPSFMagNpt,

...


o.zMeanPSFMag, o.zMeanPSFMagErr, o.zMeanPSFMagNpt,

...


o.yMeanPSFMag, o.yMeanPSFMagErr, o.yMeanPSFMagNpt,

...

 
o.rMeanKronMag, o.rMeanKronMagErr,

...


o.nDetections, o.ng, o.nr, o.ni, o.nz,o.ny,

...


o.gFlags, o.gQfPerfect,

...


o.rFlags, o.rQfPerfect,

...


o.iFlags, o.iQfPerfect,

...


o.zFlags, o.zQfPerfect,

...


o.yFlags, o.yQfPerfect,

...


o.primaryDetection, o.bestDetection

...


 INTO mydb.[RRLPS1forceddet]

...


 FROM mydb.[RRLPS1] o

...


JOIN ForcedWarpMeasurement fwm on fwm.ObjID = o.ObjID


Results (3:45 minutes)

There are 824 forced detection entries for the 12 objects, and below are the first 10 entries

...

Query #1: Get the ObjID for the star

Code Block
languagesql
SELECT

...

 
o.objID,

...

  
o.raMean, o.decMean, o.raMeanErr, o.decMeanErr,

...

 
o.qualityFlag,

...


o.gMeanPSFMag, o.gMeanPSFMagErr, o.gMeanPSFMagNpt,

...


o.rMeanPSFMag, o.rMeanPSFMagErr, o.rMeanPSFMagNpt,

...


o.iMeanPSFMag, o.iMeanPSFMagErr, o.iMeanPSFMagNpt,

...


o.zMeanPSFMag, o.zMeanPSFMagErr, o.zMeanPSFMagNpt,

...


o.yMeanPSFMag, o.yMeanPSFMagErr, o.yMeanPSFMagNpt,

...

 
o.rMeanKronMag, o.rMeanKronMagErr,

...


o.nDetections, o.ng, o.nr, o.ni, o.nz,o.ny,

...


o.gFlags, o.gQfPerfect,

...


o.rFlags, o.rQfPerfect,

...


o.iFlags, o.iQfPerfect,

...


o.zFlags, o.zQfPerfect,

...


o.yFlags, o.yQfPerfect

...


 INTO mydb.[RRL_584630948352256_PS1]

...


FROM dbo.fGetNearbyObjEq(46.341468915923, 1.54199810825252, 1.0/60.0) as x

...


JOIN MeanObjectView o on o.ObjID=x.ObjId


Results (1 entry, < 1 min)

...

Query #2: Get the detections

Code Block
languagesql
SELECT

...

 
o.objID, o.raMean, o.decMean,

...


d.ra, d.dec, d.raErr, d.decErr,

...

 
d.detectID,
 d.obstime, d.exptime, d.airmass, d.psfflux, d.psffluxErr, d.psfQf, 
d.psfQfPerfect, d.psfLikelihood, d.psfChiSq, d.extNSigma, d.zp, 
d.apFlux, d.apFluxErr,

...


d.imageID, d.filterID,

...


d.sky, d.skyerr, d.infoflag, d.infoflag2, d.infoflag3,

...


o.qualityFlag,

...


o.gMeanPSFMag, o.gMeanPSFMagErr, o.gMeanPSFMagNpt,

...


o.rMeanPSFMag, o.rMeanPSFMagErr, o.rMeanPSFMagNpt,

...


o.iMeanPSFMag, o.iMeanPSFMagErr, o.iMeanPSFMagNpt,

...


o.zMeanPSFMag, o.zMeanPSFMagErr, o.zMeanPSFMagNpt,

...


o.yMeanPSFMag, o.yMeanPSFMagErr, o.yMeanPSFMagNpt,

...

 
o.rMeanKronMag, o.rMeanKronMagErr,

...


o.nDetections, o.ng, o.nr, o.ni, o.nz,o.ny,

...


o.gFlags, o.gQfPerfect,

...


o.rFlags, o.rQfPerfect,

...


o.iFlags, o.iQfPerfect,

...


o.zFlags, o.zQfPerfect,

...


o.yFlags, o.

...

yQfPerfect
 INTO mydb.[RRL_584630948352256_PS1det]

...


 FROM mydb.[RRL_584630948352256_PS1] o

...


JOIN Detection d on d.ObjID = o.ObjID


Results (92 entries, < 1 min)

...

Query #3: Get the forced detections

...

Code Block
languagesql
SELECT
o.objID, o.raMean, o.decMean,

...


fwm.detectID,
 fwm.obstime, fwm.exptime, fwm.airmass, fwm.Fpsfflux, fwm.FpsffluxErr, 
fwm.FpsfQf, fwm.FpsfQfPerfect, fwm.FpsfChiSq, fwm.zp, fwm.FapFlux, 
fwm.FapFluxErr,

...


fwm.forcedWarpID, fwm.filterID,

...


fwm.Fsky, fwm.Fskyerr, fwm.Finfoflag, fwm.Finfoflag2, fwm.Finfoflag3,

...


o.qualityFlag,

...


o.gMeanPSFMag, o.gMeanPSFMagErr, o.gMeanPSFMagNpt,

...


o.rMeanPSFMag, o.rMeanPSFMagErr, o.rMeanPSFMagNpt,

...


o.iMeanPSFMag, o.iMeanPSFMagErr, o.iMeanPSFMagNpt,

...


o.zMeanPSFMag, o.zMeanPSFMagErr, o.zMeanPSFMagNpt,

...


o.yMeanPSFMag, o.yMeanPSFMagErr, o.yMeanPSFMagNpt,

...


o.rMeanKronMag, o.rMeanKronMagErr,

...


o.nDetections, o.ng, o.nr, o.ni, o.nz,o.ny,

...


o.gFlags, o.gQfPerfect,

...


o.rFlags, o.rQfPerfect,

...


o.iFlags, o.iQfPerfect,

...


o.zFlags, o.zQfPerfect,

...


o.yFlags, o.yQfPerfect

...


INTO mydb.[RRL__584630948352256_PS1forceddet]

...

 
FROM mydb.[RRL_584630948352256_PS1] o

...


JOIN ForcedWarpMeasurement fwm on fwm.ObjID = o.ObjID


Results (84 entries, <1 min)

...

As a comparison, the Catalina Sky Survey V-band light curve:

Image RemovedImage Added

...

Select mean objects and associated detections within 3 deg of HDF; compare photometry and astrometry with Gaia DR2

...

We start with a cone search of ObjectThin within a 3 degree radius of the nominal HDF position (169.2,61.3).  We select only mean objects with at least three detections and mean PSF magnitude r < 21.0 (Gaia sources are sparse at fainter magnitudes).  The following search was run in CasJobs and took 8:29, returning 192,611 objects.

Code Block
languagesql
select o.objID, o.raMean, o.decMean, o.raMeanErr, o.decMeanErr,

...

 
   o.raStack, o.decStack, o.raStackErr, o.decStackErr, o.epochMean,

...

 
   o.nDetections, o.ng, o.nr, o.ni, o.nz, o.ny, o.objInfoFlag, o.qualityFlag,

...


   m.gMeanPSFMag, m.rMeanPSFMag, m.iMeanPSFMag, m.zMeanPSFMag, m.yMeanPSFMag,

...


   m.gMeanPSFMagErr, m.rMeanPSFMagErr, m.iMeanPSFMagErr, m.zMeanPSFMagErr, m.yMeanPSFMagErr,

...


   m.gMeanKronMag, m.rMeanKronMag, m.iMeanKronMag, m.zMeanKronMag, m.yMeanKronMag,

...


   m.gFlags, m.rFlags, m.iFlags, m.zFlags, m.yFlags

...

 
   into mydb.MyTable_HDF_3deg

...

 from fGetNearbyObjEq(169.20,62.30,180.0) nb

...


inner join ObjectThin o on o.objid=nb.objid and o.

...

nDetections>3
inner join MeanObject m on o.objid=m.objid and o.uniquePspsOBid=m.uniquePspsOBid and m.

...

rMeanPSFMag<21.0


The output table is 48,200 kb; here is a sample of the first few rows:

...

The following query identifies all detections associated (i.e., having the same ObjID) as the mean objects obtained in Step 1, and extracts relevant parameters for each.  The query ran for 3:18 and returned 8,560,632 rows (detections), or an average of over 40 per mean object. 

Code Block
languagesql
select d.detectID, d.objid, d.obstime, d.exptime, d.airmass, d.psfflux, d.psffluxErr, d.psfQF,

...


d.imageID, d.filterID, d.sky, d.skyerr, d.infoflag, d.infoflag2, d.infoflag3, d.ra, d.dec, d.raerr, d.decerr

...

 
into mydb.MyTable_HDF_det2

...


from Detection d

...


inner join MyDB.MyTable_HDF_3deg mm on d.objid=mm.objid


The resulting table is about 1.3 GB, exceeding the standard limit in CasJobs.  A sample of the first several rows follows.

...

The same region of the sky contains 86,685 Gaia DR2 sources, of which 77448 have a valid five-parameter astrometric solution, and 68638 have also a valid G magnitude.  For the first matching stage, we do not require valid magnitude or five-parameter astrometry; the direct positional match with a generous 10" radius finds 82,398 matches, with the vast majority well within 1":

Image RemovedImage Added

In this comparison, it is useful to distinguish point sources from extended sources.  The latter wuill likely be treated differently in Gaia vs. Pan_STARRS, and their astrometric and photometric accuracy is likely poorer.  One way to distinguish point from extended sources is to use the difference between Kron and PSF magnitudes; another is encoded in the QF_OBJ_EXT bit (bit 1) from the qualityFlag parameter in ObjectThin (extracted from the query in Step 1):

ExtendedFlag = (qualityFlag AND 1)

(this is a bitwise AND).  The ExtendedFlag thus defined correlates well with the Kron vs. PSF magnitude difference (which presumably is part of the definition):

Image Removed PDFImage Added

Red dots in this plot have ExtendedFlag == true.  In the following, we use ExtendedFlag to identify extended objects.

...

The match distance distribution shows clearly that "true" matches likely have match distances << 1".  This plot shows that PanSTARRS r-band PSF magnitudes and Gaia G band magnitudes match very well for ``true'' point source matches (separation < 0.2"):

Image Removed PDFImage Added

The plot shows two populations of point sources: a narrow band with r-G ~ 0, containing a majority of the matches, and a broader distribution at positive r-G values (brighter in G than in r).  Extended sources are mostly brighter in PanSTARRS r, as expected since Gaia, with its higher angular resolution, would only measure the core of the object.  (We will neglect extended sources henceforth.)  It turns out that the second population of point sources, at positive r-G, is constituted of very red stars, for which the extremely broad G passband is dominated by red photons.  This becomes clear if the r-G magnitude difference is plotted against PanSTARRS colors, e.g., r-i:

Image Removed PDFImage Added

where the region of positive r-G (Gaia brighter) corresponds with positive r-i (red stars).  The narrowness of the sequence testifies to the quality of Gaia and PanSTARRS magnitudes. 

We can take one additional step and attempt to define a color transformation that maps Gaia magnitudes (G, B, and R) into PanSTARRS r.  A degree 4 polynomial results in a very good match, albeit with large scatter at faint magnitudes, where Gaia B and R magnitudes lose precision:

Image Removed PDFImage Added

It would also be possible to define a combination of PanSTARRS magnitudes to map into Gaia G, but the above plots clealy show that magnitudes are well matched, so this has not been pursued further. 

...

Finally, we compare the astrometric match between Gaia and PanSTARRS using both mean object quantities and the average of detection positions.  Mean object astrometry is obtained from the DR2 table ObjectThin and has been adjusted to Gaia.  Detection positions are taken from the DR2 Detection table, and have not been adjusted to Gaia.  Accordingly, if we assign to each matched object a ``mean detection'' position corresponding to the average of its detections (as indicated by objID), this position is a true indicator of the underlying quality of PanSTARRS astrometry before Gaia matching:

Image Removed PDFImage Added

The black histogram shows the match distance for mean object (typically Gaia-adjusted) positions; it peaks at 5 mas, with a significant tail beyond 20 mas.  The red histogram uses the mean position of detections; it peaks at 20 mas, with a significant tail beyond 50 mas.  More analysis is in progress to better understand how proper motions might impact the relative astrometry for both matched and unmatched PanSTARRS sources.  In particular, we plan to use detection positions to obtain proper motion estimates or bounds for sources that do not have Gaia proper motions.  Note that separations based on mean detection positions have not been adjusted for the (known) proper motion of the corresponding Gaia source; this is ongoing.

...