Data Flow

Data Processing

The data are processed by the Cambridge Astronomical Survey Unit (CASU) and follows the same steps as for the processing of the Wide Field Survey data. We give here a summary of the process; more details are available from the INT WFS pages.The data are first debiassed (full 2D bias removal is necessary). Bad pixels and columns are then flagged and recorded in confidence maps, which are used during catalogue generation. Linearity tests using sequences of dome flats revealed that the CCDs have significant non linearities so a correction using look-up-tables is then applied to all data. Flatfield images in each band are constructed by combining several sky flats obtained in bright sky conditions during the twilight.

Finally an astrometric solution starts with a rough WCS based on the known telescope and camera geometry and is the progressively refined using the Guide Star Catalogue for a first pass and the 2MASS catalogues for a final pass. The WFC field distortion is modelled using a zenithal equidistant projection (ZPN). The resulting internal astrometric precision is better than 100 mas over the whole WFC array (based on intercomparison of overlap regions). The object detection is performed in each band separately using a standard APM-style object detection and parametrisation algorithm using apertures of radius 1.2 arcsec. 

Object Extraction

The derived object catalogues are stored in multi-extension FITS files as FITS binary tables, one for each image extension with a dummy primary header unit. Each catalogue header contains a copy of the relevant telescope FITS header content in addition to detector-specific information.

Each detected object has an attached set of descriptors, forming the columns of the binary table and summarising derived position, shape and intensity information. During further processing stages ancilliary information such as the sky properties, seeing, average stellar image ellipticity, are derived from the catalogues and stored in the FITS headers attached to each catalogue extension. In addition to being the primary astronomical products from the pipeline processing, the catalogues and associated derived summary information form the basis for astrometric and photometric calibration and quality control monitoring.

The standard catalogue generation software makes direct use of the confidence maps previously generated for object detection and parametrisation producing quality control information, standard object descriptors and detected object overlay files. The possibly varying sky background is estimated automatically, prior to object detection, using a combination of robust iteratively clipped estimators. The image catalogues are then further processed to yield morphological classification for detected objects and used to generate astrometric and photometric calibration information.

Morphological Classification

For classification all detector-level catalogues for each pointing and/or passband are processed independently. Objects are classified based on their overall morphological properties, specifically the curve-of-growth of their flux distribution, and their ellipticity as derived from intensity-weighted second moments. The average stellar locus on each detector in these parameter spaces is generally well-defined and is used as the basis for a null hypothesis stellarness test for use in morphological classification

The classification is primarily based on comparing the curve-of-growth of the flux for each detected object with the well-defined curve-of-growth for the general stellar locus. This latter is a direct measure of the integral of the point spread function out to various radii and is independent of magnitude if the data are properly linearised, and if saturated images are excluded. In using this property the classifier further assumes that the effective PSF for stellar objects is constant over each detector, although individual detectors are allowed to have different PSFs.

The reference stellar loci are defined from the discrete curve-of-growth of the aperture fluxes by analysing the difference in magnitude (or flux ratio) between different pairs of apertures as a function of magnitude. In practice, the aperture with radius 1.2 arcsec is used as the fixed reference and also defines the internal magnitude (flux) scale. The linearity of the system implies that the position of the stellar locus for any function of the aperture fluxes is independent of magnitude (at least until images saturate). Therefore marginalising the flux ratios over magnitude yields one-dimensional distributions that can be used to greatly simplify automatically locating the stellar locus. With the location fixed, the median of the absolute deviation from the median (MAD) provides a solid measure of the scatter about this locus as a function of magnitude, at least until galaxies dominate in number. This process is repeated iteratively for each distribution, using 3-sigma clipping to remove non-stellar outliers, until satisfactory convergence is reached.

The discrete curve-of-growth of the flux for each object is then compared to that derived from the (self-defining) locus of stellar objects, and combined with information on the ellipticity of each object, to generate the overall detector-level classification statistic. The combination (essentially a weighted sum of the normalised signed distributions) is designed to preserve information on the ``sharpness'' of the object profile and is finally renormalised, as a function of magnitude, to produce the equivalent of an overall N(0,1) measure.

In practice measures derived from real images do not exactly follow Gaussian distributions. However, by combining multiple normalised distributions (with well-defined 1st and 2nd moments), the Central Limit Theorem works in our favour such that the resulting overall statistic is Gaussian-like to a reasonable approximation and hence can be used with due care as the likelihood component of a Bayesian Classification scheme, making optional use of prior knowledge.

Objects lying within 2-3 sigma of the stellar locus (i.e. of zero) are generally flagged as stellar images, those below -3 to -5 sigma (i.e. sharper) as noise-like, and those above 2-3 sigma (i.e. more diffuse) as non-stellar.

Although the discrete classification scheme is based on the N(0,1) measure of stellarness it has several overrides built in to attempt to make it more reliable. For example, adjustments to the boundaries at the faint-end (to cope with increased RMS noise in the statistic) and at the bright-end (to cope with saturation effects) are also made, while the overall image ellipticity provides a further check.

A by-product of the curve-of-growth analysis and the classification is an estimate of the average PSF aperture correction for each detector for those apertures (up to and including 4r , which includes typically about 99%, or more, of the total stellar flux) used in deriving the classification statistic. Accurate assessment of the aperture correction to place the (stellar) fluxes on a total flux scale is a crucial component of the overall calibration. We find that this method of deriving aperture corrections contributes about 1% to the overall photometry error budget and also provides a useful first order seeing correction for non-stellar sources. Further by-products of the morphological classification process are improved estimates of the seeing and average PSF ellipticity from making better use of well-defined stellar-only sources. These parameters are required for quality control monitoring of telescope performance and ``atmospheric'' seeing.

Photometric Calibration

Photometric calibration is done using series of Landoldt standard stars (Landoldt 1992) with photometry in the SDSS system. For each night a zero point in each filter is derived. For photometric nights the calibration over the whole mosaic has an accuracy of 1-2%. For the purpose of the photometric calibration, standard stars observations have been obtained each night at an interval of 2h and have been used to calibrate the r' and i' frames. The Ha frames have been calibrated using a fixed offset of 3.14 magnitudes with respect to r', corresponding to the magnitude difference in r'-Ha for a Vega-type star.

All calibration is by default corrected for the mean atmospheric extinction at La Palma during pipeline processing 0.09 in r' and Ha and 0.05 in i' ).

During non-photometric nights, in otherwise acceptable observing conditions, we find that the derived zeropoint systematic errors can be up to 10% or more. Although the pipeline usually successfully flags such nights as non-photometric it still leaves open the problem of what to do about tracking the varying extinction during these nights. For this Early Data Release we have not attempted a global photometric solution for the while survey. Data from non-photometric nights have been included in the release but flagged as such so that they do not appear in out "Best" catalogue.

Astrometric Calibration

Astrometric calibration is a multi-stage process and aims to provide each image, and any derived catalogues, with a World Coordinate System (WCS) to convert between pixel and celestial coordinates. This happens in the pipeline in two generic stages.

An initial WCS based on knowledge of the instrument, e.g. orientation, field-scale, telescope pointing, is embedded in the FITS headers, with telescope-specific information in the primary header and detector-specific information in the secondary headers. This serves to locate each detector image to within a few to several arcsec, depending on the pointing accuracy of the telescope and model parameters. The essential information required is the RA and Dec of the pointing, a (stable) reference point on the detector grid for those coordinates (e.g. the optical axis of the instrument), the central pixel scale, the rotation of the camera, the relative orientation of each detector and the geometrical distortion of the telescope and camera optics, which defines the astrometric projection to use.

Given a rough WCS for the processed frames, a more accurate WCS can be defined using astrometric standards. We have based our calibration on the 2MASS point source catalog (Skrutskie et al. 2006) for several reasons: it is an all-sky NIR survey; it is calibrated on the International Celestial Reference System (ICRS); it provides at least 100 or more suitable standards per pointing; it is a relatively recent epoch (mid-1990s) minimising proper motion problems; the global systematics are better than 100mas over the entire sky; and for 2MASS point sources with signal:to:noise  10:1 the RMS accuracy per source is about 100mas.