You are here: Home / Surveys & Projects / WFCAM / Technical / Catalogue generation

Catalogue generation

Stacked and/or interleaved frames will have full catalogue generation carried out on them and an accurate WCS computed and back-propagated through the component images. This is done in order to provide quality control, and astrometric and photometric calibration information, it is necessary to generate detected object (i.e. stars, galaxies) catalogues for each target frame.

Objects are extracted from the frames and catalogued into FITS binary tables. Standard object descriptors include assorted aperture flux measurements, intensity weighted centroid estimates and shape information such as intensity-weighted second moments to encode the equivalent elliptical Gaussian light distribution. These values are then used for morphological object classification.

The catalogue generation software makes direct use of the confidence maps for object detection and parameterisation, and produces the requisite information via the use of standard object descriptors.

The derived object catalogues are stored in multi-extension FITS files as FITS binary tables, one for each image extension with a dummy primary header unit. Each catalogue header contains a copy of the relevant telescope FITS header content in addition to detector-specific information. Each detected object has an attached set of descriptors, forming the columns of the binary table and summarising derived position, shape and intensity information. During further processing stages ancilliary information such as the sky properties, seeing and so on are derived from the catalogues and stored in the FITS headers attached to each catalogue extension.

The following tables cover the WFCAM standard and further processing pipeline output catalogues, where for simplicity all derived parameters are stored as floating point numbers even though some of them are more naturally integers.

No.    Name                            Description
1 Sequence number Running number for ease of reference, in strict order of image detections
2 Isophotal flux Standard definition of summed flux within detection isophote, apart from detection filter is used to define pixel connectivity and hence which pixels to include. This helps to reduce edge effects for all isophotally derived parameters.
3 X coord Intensity-weighted isophotal centre-of-gravity in X
4 Error in X Estimate of centroid error
5 Y coord Intensity-weighted isophotal centre-of-gravity in Y
6 Error in Y Estimate of centroid error
7 Gaussian sigma These are derived from the three general intensity-weighted second moments the equivalence between them and a generalised elliptical Gaussian distribution is used to derive:
Gaussian sigma = (σa2b2)1/2
ellipticity = 1.0-σab
position angle = angle of ellipse major axis wrt x axis

8 Ellipticity
9 Position angle
10 Areal profile 1 Number of pixels above a series of threshold levels relative to local sky. levels are set at T, 2T, 4T, 8T ... 128T where T is the threshold. These can be thought of as a sort of poor man's radial profile. Note that for now deblended, i.e. overlapping images, only the first areal profile is computed and the rest are set to -1 flagging the difficulty of computing accurate profiles.
11 Areal profile 2
12 Areal profile 3
13 Areal profile 4
14 Areal profile 5
15 Areal profile 6
16 Areal profile 7
17 Areal profile 8 For blended images this parameter is used to flag the start of the sequence of the deblended components by setting the first in the sequence to 0
18 Peak height In counts relative to local value of sky - also zeroth order aperture flux
19 Error in peak height
20 Aperture flux 1 These are a series of different radii soft-edged apertures designed to adequately sample the curve-of-growth of the majority of images and to provide fixed-sized aperture fluxes for all images. The scale size for these apertures is selected by defining a scale radius ∼<FWHM> for site+instrument. In the case of WFCAM this "core" radius (rcore) has been fixed at 1.0 arcsec for convenience in inter-comparison with other datasets. A 1.0 arcsec radius is equivalent to 2.5 pixels for non-interleaved data, 5.0 pixels for 2x2 interleaved data, and 7.5 pixels for 3x3 interleaved data. In ∼1 arcsec seeing an rcore-radius aperture contains roughly 2/3 of the total flux of stellar images. [In general the rcore parameter is user specifiable and hence is recorded in the output catalogue FITS header.]

The aperture fluxes are sky-corrected integrals (summations) with a soft-edge (ie. pro-rata flux division for boundary pixels). However, for overlapping images they are more subtle than this since they are in practice simultaneously fitted top-hat functions, to minimise the effects of crowding. Images external to the blend are also flagged and not included in the large radius summations.

Aperture flux 3 is recommended if a single number is required to represent the flux for ALL images - this aperture has a radius of rcore.

Starting with parameter 20 the radii are: (1) 1/2×rcore, (2) 1/√2×rcore, (3) rcore, (4) √2×rcore, (5) 2×rcore, (6) 2√2×rcore, (7) 4×rcore, (8) 5×rcore, (9) 6×rcore, (10) 7×rcore, (11) 8×rcore, (12) 10×rcore, (13) 12×rcore

Note 4×rcore contains ~99% of PSF flux

The apertures beyond this are for generalised galaxy photometry and are further spaced in radius to ensure reasonable sampling further out.

Note these are all corrected for pixels from overlapping neighbouring images

The biggest aperture has a radius 12×rcore ie. ∼24 arcsec diameter

The aperture fluxes can be combined with later-derived aperture corrections for general purpose photometry and together with parameter 18 (the peak flux) give a simple curve-of-growth measurement which forms the basis of the morphological classification scheme

21 Error in flux
22 Aperture flux 2
23 Error in flux
24 Aperture flux 3
25 Error in flux
26 Aperture flux 4
27 Error in flux
28 Aperture flux 5
29 Error in flux
30 Aperture flux 6
31 Error in flux
32 Aperture flux 7
33 Error in flux
34 Aperture flux 8
35 Error in flux
36 Aperture flux 9
37 Error in flux
38 Aperture flux 10
39 Error in flux
40 Aperture flux 11
41 Error in flux
42 Aperture flux 12
43 Error in flux
44 Aperture flux 13
45 Error in flux
46 Petrosian radius rp as defined in Yasuda et al. 2001 AJ 112 1104
47 Kron radius rk as defined in Bertin and Arnouts 1996 A&A Supp 117 393
48 Hall radius rh image scale radius eg. Hall & Mackay 1984 MNRAS 210 979
49 Petrosian flux Flux within circular aperture to k×rp; k = 2
50 Error in flux
51 Kron flux Flux within circular aperture to k×rk,; k = 2
52 Error in flux
53 Hall flux Flux within circular aperture to k×rh; k = 5; alternative total flux
54 Error in flux
55 Error bit flag Bit pattern listing various processing error flags; initially set to no. of bad pixels within aperture of radius "rcore" - note that this can be fractional due to soft-edged apertures
56 Sky level Local interpolated sky level from background tracker
57 Sky rms local estimate of variation in sky level around image
58 Child/parent Flag for parent or part of deblended deconstruct (redundant since only deblended images are kept)

The following are accreted after standard catalogue generation

59 RA RA and Dec explicitly put in columns for overlay programs that cannot, in general, understand astrometric solution coefficients - note r*4 storage precision accurate only to ∼50mas. Astrometry can be derived more precisely from WCS in header and XY in parameters 5 & 6

60 Dec
61 Classification Flag indicating most probable morphological classification: eg. -1 stellar, +1 non-stellar, 0 noise, -2 borderline stellar, -9 saturated
62 Statistic
An equivalent N(0,1) measure of how stellar-like an image is, used in deriving parameter 61 in a "necessary but not sufficient" sense. Derived mainly from the curve-of-growth of flux using the well-defined stellar locus as a function of magnitude as a benchmark (see Irwin et al. 1994 SPIE 5493 411 for more details).

From the further processing pipeline after deriving a suitable PSF

63 PSF flux Fitted flux from derived PSF
64 Error in PSF flux
65 PSF X coord Updated PSF-fitted X centre
66 Error in X coord
67 PSF Y coord Updated PSF-fitted Y centre
68 Error in Y coord
69 PSF fit χ2 Standard normalised variance of fit
70 nPSF Number of degrees of freedom for PSF fit
71 1D Sersic flux Fitted flux for Sersic profile
72 Scale length Scale factor of fit
73 Power index Power law index of fit
74 Error in fit Standard normalised variance of fit
75 nS1 Number of degrees of freedom for 1D Sersic fit
76 2D Sersic flux Fitted flux for PSF-deconvolved 2D Sersic fit
77 Scale length Scale factor of fit
78 Power index Power law index of fit
79 Error in fit Standard normalised variance of fit
80 nS2 Number of degrees of freedom for 1D Sersic fit

Note:- a more formal mathematical definition of many of these parameters, together with a corresponding definition of the image processing steps is given in the VDFS Data Reduction Library Design document VIS-SPE-IOA-20000-0010, and references therein. Although this particular document is specific to VISTA, the formalism is generally applicable and the processing steps and catalogue parameters are expected to be almost identical.

Derived catalogue FITS header contents

The following are examples of extra FITS header items derived from the catalogues or during catalogue creation, one set for each extension:

SKYLEVEL Median sky brightness (counts/pixel)
An automatic 2D background-following algorithm is used to track and "remove" slowly varying background features such as image gradients etc. The default scale size for background tracking (NBSIZE) is currently set to 64 pixels, coupled with a smidge of non-linear filtering this gives a background tracking scale of order 100 pixels. (A bilinear interpolator is used to generate pixel resolution background maps internally).
SKYNOISE Pixel noise at sky level (counts)
Robust MAD estimator for noise scaled to equivalent Gaussian rms value ie. = MAD x 1.48 after removing large scale sky background variations. MAD = Median of the Absolute Deviations about the median
THRESHOL Isophotal analysis threshold (counts)
User-selectable parameter, the default is to set this to 1.5×skynoise as a compromise between detecting close to the limit of the data and not being swamped by spurious sources. It is possible to push the data limit fainter but at the expense of a large increase in spurious sources. LSBG detection requires a more subtle add-on to the catalogue and could be added given suitable demand.
MINPIX Minimum size for images (pixels)
User-selectable parameter, in conjunction with the threshold above this determines how deep and how small "real" images can be. This default precludes many of the few pixel-hit cosmic rays from being considered since "real" images must have 4 contiguous simply-connected pixels in the union of the detection filter and data domains.
CROWDED Crowded field analysis flag (0 none, 1 active)
User-selectable parameter, detection algorithm tried to disentangle overlapping images or images supperposed on the "slowly" varying background of other large images (default) otherwise just straighforward isophotal detection.
RCORE Core radius for default profile fit (pixels)
User-selectable parameter, aperture flux designed to match median seeing of survey data. It is straighforward to show that if rcore = FWHM then for typical profiles encountered the rcore flux estimate has between 80-90% of the accuracy of an idealised perfectly known PSF model method.
OPTFILT FWHM of Gaussian detection filter (pixels)
User-selectable parameter, should be chosen to match the average FWHM of stellar images in the data. This defines the Gaussian matched detection filter to use.
SEEING Average FWHM (pixels)
An average realistic FWHM estimated directly from the stellar images on the frame. Mutiply by pixel scale size to convert to arcsec (eg. ×0.400 WFCAM; ×0.333 INT WFC).
ELLIPTIC Average stellar ellipticity (1-b/a)
A direct estimate of the average stellar ellipticity, useful for spotting trailed frames usw.. Should not average much above 0.15 for "normal" frames.
CLASSIFD Classified (true/false)
Has image morphological classifier been run? If so, an object classification flag and a stellarness index is included in the binary table columns.
SATURATE Average saturation level in frame
An estimate directly from saturated images on the frame at what level image saturation occurs, including sky. This varies from detector to detector depending on the relative gains applied to bring them to a uniform flatfield response etc.

Photometric Information

APCORPK
APCOR1
APCOR2
APCOR3
APCOR4
APCOR5
APCOR6
APCOR7
Stellar aperture correction - peak height, core1-7 flux
Aperture corrections in magnitudes needed to correct the assorted aperture-like measures produced in the catalogues onto the equivalent of a total flux stellar system. These constitute the components of a curve-of-growth analysis contained within the catalogues with radii defined in the previous table; to be used in the sense that
corrected photometry = 2.5×log10(flux) + apcor
They also work well as a first-order seeing correction for faint galaxies. Larger aperture corrections are not included.


PERCORR





Sky calibration correction (mags)
This is a correction based on the median dark sky recorded in science frames compared to the median for all the detectors and as such is an ancillary correction to the gain correction derived from the flatfield (usually twilight flats) data. This correction is to be used in the same sense as before in that

corrected photometry = 2.5×log10(flux) + apcor + percorr
MAGZPT












Photometric ZP (mags) for default extinction
Derived detector zero-point in the sense of what magnitude object gives a total (corrected) flux of 1 count/s. These ZPs are appropriate for generating magnitudes in the natural detector+filter system based on Vega, see the photometry pagefor more details on colour equations etc. The ZPs have been derived from a robust average of all photometric standards observed on any particular set of frames, corrected for airmass but assuming the default extinction values listed later. For other airmass or other values of the extinction use
ZP ⇒ ZP - [sec(z)-1]×extinct + default extinct - extinct

You can then make use of any of the assorted flux estimators to produce magnitudes via

Mag = ZP - 2.5×log10(flux/exptime) - apcor - percorr

Note that for the so-called total and isophotal flux options it is not possible to have a single-valued aperture correction
MAGZRR

Photometric ZP error (mags)
If this was a good photometric night the error in the zero-point will be at the level of a few percent. Values of 0.05 and above indicate correspondingly non-photometric nights and worse.
EXTINCT










Default extinction in passband
For WFCAM these are currently set to a constant clear night level, which within the current measuring error is the same for all passbands. Note that the frame-by-frame derived ZP from 2MASS automatically corrects for extinction variations, assuming they are uniform across the field of view.

To compute approximate errors in the fluxes you can also use the following:-

error2 = flux/gain + npixels×skynoise2
where npixels is either the effective area ie. π×rcore2 for the "core" measures or the number of pixels above the detection isophote ie. areal profile1; gain is the final overall detector system gain. (see the Science Verification report for values); flux is whichever measure you are using but note that for "total" this formula is not accurate since "total" fluxes are derived using a rather convoluted curve-of-growth technique; average skynoise can be obtained from the catalogue fits header.
NUMZPT
Number of standards used
The actual number of standards used in the photometric calibration for the pointing
NIGHTZPT 
Average photometric ZP (mags) for night
A robust estimate of the average ZP in a particualar passband for the night
NIGHTZRR Photometric ZP sigma for night (mags)


A robust estimate of the scatter in this ZP to give an indication of how photometric or otherwise the night was (0.03 or better is good; worse than 0.05 is usually indicative that the majority of the night was non-photometric).



Astrometric Information

The astrometric calibration uses the World Coordinate System (WCS). See Calabretta & Greisen 2002 A&A 395 1077 and Greisen & Calabretta 2002 A&A 395 1061 for a description. The following is an example of such a calibration. Each extension has its own calibration.
CTYPE1  = 'RA---ZPN'           / Zenithal polynomial projection
CTYPE2  = 'DEC--ZPN'           / Zenithal polynomial projection
CRPIX1  =              2999.40 / Reference pixel X on axis 1
CRPIX2  =              -940.32 / Reference pixel Y on axis 2
CRVAL1  =         112.45449231 / [deg] Right ascension at the reference pixel
CRVAL2  =          30.07196251 / [deg] Declination at the reference pixel
CRUNIT1 = 'deg     '           / Unit of right ascension co-ordinates   
CRUNIT2 = 'deg     '           / Unit of declination co-ordinates       
CD1_1   =      -7.11999928E-08 / Transformation matrix element         
CD1_2   =       1.11427136E-04 / Transformation matrix element         
CD2_1   =       1.11433289E-04 / Transformation matrix element          
CD2_2   =      -1.73270884E-07 / Transformation matrix element         
PV2_1   =                  1.0 / Pol.coeff. for pixel -> celestial coord
PV2_3   =                -50.0 / Pol.coeff. for pixel -> celestial coord
PROJP1  =                  1.0 / Old style ZPN projection keyword r term
PROJP3  =                -50.0 / Old style ZPN projection keyword r**3 term
NUMBRMS =                   87 / Number of standards used              
STDCRMS =                0.089 / Astrometric fit error (arcsec)

Example Code

Here is an example (in perl) of how to implement the WCS to convert from pixels to RA, DEC. Use this at your own risk ! It hasn't been tested in anger and is posted here just to give you an example.

################################################################################
# SUBROUTINE -CONVERT X, Y to RA, DEC
#
sub xytoradec{
    my($x,$y,$crpix1,$crpix2,$crval1,$crval2,$cd1_1,$cd1_2,
       $cd2_1,$cd2_2,$pv2_1,pv2_3)=@_;
    my($pi)=4*atan(1.0);
    my($dtor)=$pi/180.0;
    my($tpi)=$pi*2.0;
    my($tpa)=$crval1*$dtor;
    my($tpd)=$crval2*$dtor;
    my($secd)=1.0/cos($tpd);    
    my($tand)=tan($tpd);
    my($a)=$cd1_1*$dtor;
    my($b)=$cd1_2*$dtor;
    my($d)=$cd2_1*$dtor;
    my($e)=$cd2_2*$dtor;
    my($c)=$crpix1;
    my($f)=$crpix2;
    my($xi)=$a*($x-$c)+$b*($y-$f);
    my($xn)=$d*($x-$c)+$e*($y-$f);
    my($rp)=sqrt($xi**2+$xn**2);
    my($rfac)=$pv2_1+($pv2_3*$rp**2);
    $rp=$rp/$rfac;
    $rfac=$pv2_1+($pv2_3*$rp**2);
    $xi=$xi/$rfac;
    $xn=$xn/$rfac;
#    print "xi $xi xn $xn\n";
    my($angle)=atan($xi*$secd/(1.0-$xn*$tand));
    my($alpha)=$tpa+$angle;
    if($xi!=0.0){
        $delta=atan(($xn+$tand)*sin($angle)/($xi*$secd));
    }
    else{
        $delta=atan(($xn+$tand)/(1.0-$xn*$tand));
    }
    if($alpha>$tpi){
        $alpha=$alpha-$tpi;
    }
    if($alpha<0.0){
        $alpha=$alpha+$tpi;
    }
#    print "alpha $alpha delta $delta\n";
    return ($alpha,$delta);
}