—
Plain Text,
16 KB (16889 bytes)
File contents
27th November 2008 APM meeting tables 11:30-12:50
Present: MJI, JRL, STH, MR, DWE, EGS, RGM, RWA
Apologies: NAW, PSB
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. Recent and upcoming meetings
4. Data archives update - AAT, ING, WFCAM, UKIRT
5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru
6. WFCAM update
7. VISTA update
8. AOB
Minutes
-------
1. Actions from last meeting
STH has not finished checking that the Plone astrometry technical pages
are correct, however, the photometry pages have been updated and
the astrometry ones are "mid-edit" - ongoing <<<<
STH has included the graphic showing the size of the photometric radial
distortion correction in the Plone technical pages
JRL has not produced a web page of HawkI publicity for CASU - ongoing <<<<
MJI have not finished off the stage-I software release pages - ongoing <<<<
EGS
JRL attended the Cambridge VOTC meeting
MR
STH has not made the photometric calibration paper available on the Plone
and VDFS sites since he is waiting for the referee's response on the
re-submitted paper. Apart from holding off placing this on astro-ph
it was not thought necessary to wait further - make it so <<<<
MJI has not finished off the technical paper - ongoing <<<<
MJI has passed on feedback regarding CASU contentment with proposed directory
structure for transferring data on USB disks
MJI has informed ESO that CASU would like to acquire the first few nights
of SV data as soon as possible after it is taken
PSB has made some processed VIRCAM test data available to Edinburgh
STH acquired some decent calibration data for VIRCAM
MJI made sure JPE/WJS are aware of the Ndit=1 issue for twilight flats
2. Comments on WFAU minutes
There were no specifc comments other than to note the plans for the next
UKIDSS data release, DR5.
3. Recent and upcoming meetings
EGS was asked to check if the CASU presentation from the ESO Public Surveys
meeting in September is available on their website. <<<<
ESO meetings:
MJI was over at ESO for the LP workshop and had a meeting with the ESO
members of the VISTA SV teams. Among other things discussed were action
items on fast SV data access and the format (apparently Rice-tile compressed)
of the 500 Gbyte USB disks which ESO will deliver VISTA data on. He also
confirmed that PIs of VISTA surveys will get access to the raw data, if
required, direct from ESO. The ESO Garching pipeline will run on all
calibration frames to derive QC information but will apparently only sample
a subset of the science frames for QC purposes !
QC0 run at Paranal is mainly concerned with conditions of the observations
and visual checks. Real time OB grades are only stored locally in logs/DBs
which are heavily firewalled => summit OB grade will not make it into the
FITS headers at least initially. A processing status web page similar
to that used for WFCAM could be used for feedback of OB information derived
at CASU, but there is still an open question of how to close the loop on
bad OBs, though most should get spotted at Paranal ? We need to emphasise
that the onus is on the survey PIs and ESO to access and make use of the
information on our web pages.
MJI was also over at ESO for the OPC meeting in November (17th-20th) and
discussed further VISTA narrow-band NB118 processing and assessment issues
(bandpass dependence on radius, sky fringing, calibration etc.) with Palle
Moller and Wolfram Freudling. Our default scheme for calibrating
the NB118 data, i.e. treat it like any other band and calibrate on 2MASS,
may work well enough for their immediate science needs. A simple test is
to monitor the stability of the difference between ZPs derived from
consecutive J and NB118 data. So far this looks promising.
Estimating the likely size of illumination corrections from existing VIRCAM
data may be possible but no firm promises have been made since this is still
TBD. To ease their worries about NB118 and ZY calibration STH has made the
WFCAM calibration paper available to them. We also need to check what
the Calibration Plan document currently says about these issues. <<<<
One further outcome of these meetings was the need to arrange another meeting
~February with the SV teams to discuss results from first pass processing
and their initial assessment of the products. It was agreed that we would
try and provide rapid pawprint level access to these data products from the
first two nights to facilitate the assessment process. It was agreed that
access via a passwd-protected web page would be fine for this, but we also
need to ensure that there is easy access to the DP description linked from
this access point.
We also discussed deep stacking the N4945 SV data, which includes NB118,
and examining band-merged catalogues across the Orion SV region to check
for lower level artefacts and other gotchas. A further possiblity is to use
the many repeat observations in the N4945 SV data to examine time series
analysis problems in conjuntion with NJC. Tiling will come second, since
it is more important to iterate quickly rather than wait for fully polished
products.
WFCAM meeting:
MJI observed with WFCAM at UKIRT on 3rd-6th October and whilst there
had a meeting with Andy and Chris to finalise the format and content
of the MSBTid logs. These are now generated after the first stage
nightly data processing and provide a more rapid feedback on MSB
quality with respect to cloud cover, seeing and sky brightness, if these
are specified by the PI. The Cl-range was clarified by defining 3 categories.
As the nominal ZP varies by ~+/-2% due to a mixture of measuring and
natural variation due to mirror cleaning cycles, these are now:
<5% variation in the nominal ZP represents photometric; 5-20% is
thin cirrus, and >20% represents thick cirrus. The MSBTid log also
flags those MSBs rejected at the summit.
The hopefully final iteration on the offset sky strategy was also discussed
and a full set of recipes are now in place (JITTER_FLIP12, 13, and
JITTER_SKY), and, as data subsequently taken shows, working.
ADASS:
MJI gave a talk at ADASS and JRL talked.
MJI also noted that there is an upcoming UKIRT board meeting and that
the submitted report is available in the usual place on the internal
web pages.
Likely attendance at the December Science with UKIDSS RAS meeting was also
discussed. MJI will attempt to provide JRL with results from the new
and improved bug-fixed deblended Petrosian et al. measures prior to the
meeting to pre-empt expected criticism from the unusual suspects. >>>>
4. Data archives update - AAT, ING, WFCAM, UKIRT
RWA reported on an email from a satisfied customer using the ING archives.
A taxidermist has been contacted.
All archives are up to date though JRL noted that he is still sorting old
UKIRT CASS data header problems.
5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru, ESO WFI
EGS reported that he has been processing data from the ESO Spitzer Imaging
Survey (P.I. Berta) obtained with WFI (ESO). This is a survey over 5.5 sq.deg.
of the Spitzer/SWIRE S1 area obtained in B, V and R from 2002 to 2007 in 155
nights in total. Only the central part has been processed before and the idea
is to process everything with the same pipeline used also to process
the optical data in the rest of SWIRE (i.e. WFS and Mosaic I and II).
MJI noted that it was time to get some mileage out of our processing of
HAWKI SV data and asked STH if he could contact Mark McCaughrean and see
what the status was. JRL commented that he was going to revisit the sky
subtraction before we produce the next iteration of the catalogues.
There was also some discussion about the usual radial distortion model that
we generated to do the astrometry. Apparently this is newsworthy !
MJI will contact Mark Casali and see if it really is. <<<<
RGM noted that there was some HAWKI PI data that he would like processing. <<<<
EGS has taken over finishing off the EGAPS UVEX data products from 07B now
that the U-band and HeI-band offset calibration has been sorted out.
MJI noted that various people including Google Sky and MicroSoft are
interested in IPHAS mosaics of the Galactic Plane. This is currently a
no work on our part project, which is fine if it stays that way.
Assorted new panoramic M31 survey data from MegaCam for 08B has arrived and
is being processed and the first data with the new Hamamatsu deep depletion
CCDs on SuprimeCam has been processed. These detectors are early production
versions of those for HyperSuperimeCam and give a much improved i and z-band
response whilst maintaining the good blue passband sensitivity. The cosmic-
ray tracks are also much more entertaining.
6. WFCAM update
The photometric calibration paper has been updated to incorporate the
responses to the referee comments and has been resubmitted to MNRAS.
MR has been beavering away running the nightly pipeline for WFCAM data as
it comes in and is comfortably keeping up with the data flow. The raw data
arrives and is checked, MEF'd and ingested within 24 hours of it being taken.
The nightly processing is started shortly thereafter. Checking the products,
fixing the parquet floor pattern and bundling up the monthly photometry
updates takes the rest of the time.
The new data transfer setup from JAC now allows us to offer a (limited)
fast track processing for urgent data requests where residual channel
offsets are unimportant and for which photometric calibration to 2%
accuracy is sufficient. This option has already been trialled and can
deliver data products to the PI within 48 hours of the data being observed.
The service will be necessarily limited in scope and, after discussion with
JAC, will probably only be offered through the PATT form via a tick box
requiring additional justification for its use. The extra load imposed by
this is expected to be marginal, but will be monitored and re-asssessed as
necessary.
Assorted reprocessing issues have cropped up or been stalled since our last
get together. Three years downstream a long standing bug with estimated
Petrosian radii and fluxes (also affecting Kron and Hall measures) for
deblended objects has been reported to us. This affects all processed WFCAM
data. Fortunately, due to the rather large number of flux measures in the
catalogues, a fix has been found for this that can be run on the existing
catalogues. Although the update can potentially be generated over a w/e run,
progressing it all the way through the various data products in the middle
of DR5 preparations did not seem like a good idea. This update will therefore
be rolled out for DR6 with the added benefit of allowing much more time for
testing.
The UDS team came in with an unexpected request to reprocess all the UDS
data for 05B as this semester currently has a high attrition rate due
to background variations. A few tests on typical nights unfortunately
confirmed that the latest "pawprint" sky estimation strategy, with iterative
object masking, does appear to do a much better job. This reprocessing is
scheduled after completion of 08b sometime mid-December.
A much shorter request for reprocessing a limited amount of Orion data taken
for U/05a/100 recently came in. The original sky estimation and subtraction
for this was so bad that this was pushed through quickly. This data also
provides an interesting test case for the nebulosity filter developed for
GPS photometry in regions of high nebulosity.
This leaves us with the extant GPS reprocessing in nebulous regions to improve
both detection and photometry, also on hold due to lack of available effort.
MJI commented that there still appeared to be a problem with scaling for
nexp > 1 data. Although not irrepairable, this has been noted many time
previously and he asked JRL to check with Brad at JAC to find out why this
issue still persists. <<<<
7. VISTA update
STH gave a brief report on his "observing" trip to VISTA and summarised
the data he had taken whilst there. As a result we now have available a
series of pointings (pawprints) in Y,J,H,Ks,NB118 on several 2MASS
touchstone fields and a few assorted other science fields plus the
corresponding series of twilight flats, darks and linearity sequences. Apart
from a few problems due a TCS dither offset bug, this data has been processed
with a preliminary Cambridge pipeline and has yielded some useful science
products.
A first pass analysis of these and some earlier data has provided a good
first stab at the astrometric distortion, including the need for, and
definition of, an r**5 term (PV2_5). It has also yielded estimates of the
system zero-points in these passbands and will be used to check the overall
system throughput, sky brightness in various bands and provide a first pass
estimate of the colour equations with respect to 2MASS. STH was asked to
look at this available VISTA data and further investigate this data. <<<<
To make the data easier to digest JRL and STH will also sort out the object
names of the various OBs according to their RA and Dec (all currently have
the same name). <<<<
MJI noted that all VIRCAM data products use the ESO HIERARCH system for
recording processing information (seeing, sky level, noise etc.) and also
use the "legal" table form of the WCS for recording the astrometry transform
parameters in the catalogues. This is currently incompatible with all the
WFCAM post-processing analysis software operating on the catalogues. As an
interim solution this has been sidestepped by running a translating script to
convert them to WFCAM style keyword names, but will need updating properly
later on.
JRL noted that the latest release v0.8.4 of the VIRCAM pipeline software and
updated docs were delivered to ESO the other day. Ironically some requested
CPL updates related to FITS header problems and efficient MEF file writing
that have been incorporated by ESO recently, have caused problems with
running the VISTA summit pipeline at Paranal due to them not being
incorporated there yet.
Apparently a USB disk with all the VISTA data on up to early November is
en route to us via the correct channels, and currently is being ingested
(slowly) into the ESO archive in Garching. It hasn't arrived here yet but
watch this space.
PSB reports that he has acquired a VISTA Trac account and also that against
the odds network transfers from Paranal are making good progress. So we
already have access to the new data on the USB disk before it got here.
MJI reported that a sample of narrow band NB118 and J-band data has been
made available to the UDS narrow-banders to enable them to check assorted
properties. This had been previously cleared with JPE and WJS.
8. AOB
MJI gave a brief update on the (lack of) news on the rolling grant application
of 18 months ago. Various parties with vested interests in the data
processing have been made aware of the issues and asked to press the case
for some action.
RWA has done a sterling job organising the APM and friends Christmas party,
lunchtime 9th December, consequently a good afternoon for inspired software
updates.
Finally, MJI reported on the recent changes proposed, nay happening, in the
IoA computer support group and the impact it will have on CASU.
Continuing actions
------------------
STH check that Plone astrometry technical pages are correct
JRL produce web page of HawkI publicity for CASU
MJI finish off stage-I software release pages
EGS
STH make photometric calibration paper available on Plone and VDFS sites
MJI finish off technical paper
New actions
-----------
PSB compare Calibration Plan with current expectation for calibration
EGS check if the CASU presentation from the ESO Public Surveys meeting in
September is available on ESO website.
MJI pre-empt expected criticism re: deblended Petrosian fluxes
STH contact Mark McCaughrean about HAWKI Eta Carina data status
MJI contact Mark Casali about HawkI radial distortion
RGM liaise with JRL about processing HAWKI PI data
JRL contact Brad at JAC regarding the Nexp > 1 issue
STH further investigate photometric features in available VISTA data
(throughput, colour equations, sky brightness)
JRL sort out project names of VISTA MSBs taken by STH
STH
