—
Plain Text,
14 KB (14838 bytes)
File contents
4th October 2007 Observatory Meeting Room 10:30-13:00
Present: JRL, STH, EGS, PSB, MR, MJI, DWE
Apologies: NAW, RGM, RWA
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. Recent meetings
4. Data archives update - AAT, ING, WFCAM, UKIRT
5. Optical processing update
6. WFCAM news - raw data transfer status
- WFCAM network transfers status
- processing status 07B
- reprocessing status
- photometric calibration status
7. VISTA news - general news
- cold camera test data report
- letter to PIs [and ESO]
8. ADASS 2007 - post mortem
9. AOB - retired cass36
- system management
Minutes
-------
1. Actions from last meeting
RGM have not completed the work on the classification aspect of cross-talk
EGS weeding and classification generally - ongoing <<<<
MJI
MJI is putting the OMP log comments into the raw directories, though he
commented he was still occassionally having problems with the mirrored
OMP database, which Jim was in the process of fixing.
JRL are still working out a scheme for doing header updates for ING archive
EGS - ongoing <<<<
MR found out when the first 07B tapes were shipped
MR has contacted Jim Geach regarding comparing his processed data with
pipeline output (see later)
EGS has helped with the assessment of VIRCAM lab data
MJI have found out about CASU system management from JMI
PSB
2. Comments on WFAU minutes
DWE liked the joke.
STH asked if the weekly mails from Andy Adamson could be distributed round
all of CASU. MJI noted that at the moment he forwarded these emails to JRL
and MR and would forward them to anyone else who requested. <<<<
3. Recent meetings
There was a VDMT meeting on 13th September. MJI and STH reported on the new
timetable for VISTA presented at the VDMT meeting and the subsequent email
sent out to PIs by JPE relating the new schedule. On the same day, several
members of CASU held a meeting with DES representatives to discuss data
processing issues for the new 3 sq deg FOV optical camera for the CTIO 4m.
4. Data archives update
It was noted that AAT data was still sputtering in over the network,
likewise ING data. Some complete nights of data for 07B had been received
and were available in the WFCAM raw data archive.
RWA sent in a report noting the following:
The ING CD cabinets are being re-arranged to make room for those discs
previously stored in SPO20 and including the AAT archive disks which are
now all in jewel cases.
The DAT tapes in SPO20 will be dumped into the skip next week leaving
3 large empty cupboards with sliding drawers. We need to decide if we have
a future use for these for offline tape storage or whether we should dump
them as well. Given the delay in starting the Kavli building construction
this can be a leisurely decision.
The corner cupboard in the Obs meeting room has had the lock changed and
now stores backup DLT, LTO and other tapes. It is about half full at present
and MR has a large batch of WFCAM tapes to go in later. Keys are held by RWA,
MR and Mark Hurn.
Requests for the oldest (~20 years old) ING data have almost dried up - 19
over the last 6 months and 4 in the last 4 months but they take longer to
fill as the old system no longer works.
5. Optical processing update
STH is processing INT WFC data for the Lund group of whom Daniel Aden is
currently visiting the IoA.
MJI noted that Megacam had an operational problem on June 17th destroying the
i' filter and was off telescope and unavailable until July 27th - missing the
i' filter, naturally. Several of the programmes MJI is involved in require
the i' filter and are now on hold.
MJI noted that representatives from Google Sky had contacted the IPHAS
consortium and held a telecon about the possibility of using IPHAS data to
make colour images of the Galactic Plane to serve through Google Sky,
negotiations are ongoing.
Gavin Ramsay from Armagh observatory has contacted us about the possibility
of collaborating on a Galactic Plane variability project. A recent INT run
will be processed and assessed to determine the feasibility of CASU doing
the data processing, and possibly light curve generation, using software
already developed for the Monitor project.
6. WFCAM news
Tapes up to and including complete nights of data from 4th-13th August and
25th-26th August have been received and assembled into MEFS and the
processing of these is almost complete. There is a delay with other
intermediate nights due to a missing tape. This is hopefully about to be
remedied and should enable us to catch up with the backlog.
MJI noted that the backlog was alleviated somewhat by the fact that UKIRT
was closed for the nights of 15th-17th and 29th August, and 1st-11th
September due to weather and cold head problems respectively. He also
reported that he had received good news about the channel-edge problem from
Andy Adamsom as a result of Derek Ives' recent visit to JAC.
MR pointed out that he had added a new column to the processed status web
pages which directly linked to the CFH weather pages for each night. This
was to encourage PIs to check the weather information before asking for
reprocessing of not-so-good data.
He and MJI have been comparing the pipeline processed data with some COSMOS
fields processed from scratch by Jim Geach (Durham). This involved stacking
the pipeline science products of two consecutive deep stack MSBs by us and
then directly comparing with the same data (images only) processed "manually".
This was done both by visual inspection and by generating catalogues from
both sets of products and comparing depths, number of images and so on.
Although Geach's processed data has flatter backgrounds the CASU version
goes significantly deeper and the shapes of faint images are much "smoother".
There is also a curious non-linearity in the difference of derived magnitudes
which may be related to the rather "spiky" faint image appearance in the
Geach version.
Siddhartha Ghosh (Sussex) has been comparing his own methods of deriving
bad pixel masks with the bad pixels implicitly flagged in the CASU-derived
confidence maps. So far the comparison has revealed no problems.
MJI gave a brief summary of the various odds and ends of reprocessing that
have been occurring over the last few months. Apart from a UDS query/request
that came in a few days ago and is being assessed, all of the reprocessing
is now complete. When finished MJI will check the status of all the
reprocessed UDS data <<<<
STH asked for an update on the status of internet transfer of data from
UKIRT. PSB reported that a Gbit/s fibre connection from summit down to sea
level is being installed and that when this is completed tests of
transferring data directly from the summit will proceed. This will include
some procedure refinement and network transfer optimisation. We do not have
a definitive timescale for this yet but are quietly optimistic about prospects.
This should help mitigate against some of the current data transfer problems
we are experiencing with tapes. PSB will contact Brad Cavanagh at JAC for
an update. <<<<
STH and MJI reported on the status of the WFCAM photometric calibration.
Z and Y-band offsets (with respect to Vega) have been measured (using a
robust maximum-likelihood fitting technique) from LAS and GCS datasets and
found to be consistent with zero (within the errors of measuring it) for the
Z-band and 0.08+/-0.01 for the Y-band. MJI has now implemented per-detector
zeropoints in the calibration which removes a significant fraction of the
apparent spatial systematics. These new calibrations have been applied to all
processed WFCAM data (05a,05b,06a,06b,07a) on a monthly basis (following the
master flat updates) and the resulting updated FITS header keywords have been
made available to WFAU for incorporation in DR3. MJI discussed the systematic
chip-to-chip offsets, most noticeable in the Z-band, which seem
likely to arise from intrinsic differences in QE between the detectors and
possible pedestal offsets in the flatfields. In addition, he has generated
monthly tables and plots of the residual photometric spatial systematics for
all broad bands. It is not yet clear what is causing these remaining
~1-2% systematics, but possible effects include: contributions from PSF
distortions; scattered light illumination corrections; and residual
non-linearity in the detector responses. These, and differences between
the colour equations for each chip, will be investigated for future
releases. STH and MJI continue to work on the accompanying paper describing
this in more details and plan to circulate a draft within 2 weeks.
MJI asked MR to investigate how to make the correction tables and systematics
plots available on the processed status web pages to enable users to assess
the remaining size of the problem and, if they want, have the wherewithall
to implement a correction. <<<<
7. VISTA news
MJI noted an email from JPE commenting on his discussions with ESO about the
(successful) bid to get EC funding to lay a 1 Gbps optical fibre link between
Paranal and Antofagasta. When installed this opens up the possibility of
demonstrating its utility by transferring VISTA data to Garching and then
to Cambridge.
There was some discussion about the cold camera test data report compiled
by JRL and PSB and the related upcoming telecon to discuss this. There
was some worry expressed about the problems having two different (and
incompatible) readout modes might engender. However, without further
tests it is not clear if this will be the case. JRL is still working on
further self-flatting tests on this VIRCAM data and will email a summary of
points of note from VIRCAM tests to participants of the upcoming telecon <<<<
JMS had sent an email around about another cold run at the end of October
and asked for input on what data tests should ensue. PSB and JRL will
compile a wish list and relay it. <<<<
PSB noted that the VISTA QC development project is underway: MR is dealing
with the data-ingestion DB (raw & processed) and EGS is working on the
interactive visualisation tool. PSB is playing the role of user-in-chief
and making the demands.
EGS sent out a letter to PI's about the design of the VISTA QC pipeline/survey
progress user interace based on the WFCAM version as a starting point. So far
he has had two replies: one was positive noting that the WFCAM system was good
already, The other was neutral in the sense that the PI planned to design
and implement their own survey progress monitoring tool.
There was further discussion concerning data quality for current WFCAM and
planned VISTA processing, centering on how to identify MSBs/OBs as pass/fail
on the basis of DQC criteria, and how to communicate these flags to PIs and
survey managers i.e. closing the circle. (PIs will likely have stricter
criteria based also upon their ability to do science with the data, whereas
our criteria will be on the ability to process and calibrate the data as well
as comparing target QC criteria with observed.)
PSB reminded us all that we still need the filter transmission data for
the NB118 and Z filters in order to finish off the pre-commissioning
ETC technical details.
8. ADASS 2007
JRL presented a rundown of lessons learnt from organising the ADASS 2007
meeting. All in all it was a successful meeting and helped raise the profile
of CASU in the ADASS community.
There are still several loose-ends to tidy up.
JRL needs to reacquire the helpdesk laptop to obtain the ADASS presentations
and PDF'ify them, and also needs to email other contributors for PDF versions
of their talks to post on a web site. <<<<
RWA will be editor in chief of the proceedings when the contributions start
to roll in, with help from JRL and MR on latex++ details.
DWE raved on about the chocolate fountain and wants one permanently installed
at the IoA.
9. AOB
MJI noted that cass36, and all its disks, had finally been retired and indeed
"has left the building". The disk contents (suitably relinked as well) are
all still online elsewhere.
DWE volunteered to do the system management updates for the windows machine
that still gets occassional use. <<<<
STH noted the upcoming UKIDSS meeting in December and suggested interested
parties should register as soon as possible. MJI, MR and JRL already are. <<<<
There is so much muck clogging up the air intakes on the computer kit that
MJI and PSB volunteered to sort out buying a handheld vacuum to temporarily
reverse the appropriate sucking polarity. <<<<
PSB finally retired after 15 years as a starlink manager and now works full
time on CASU projects. His first one is to investigate getting wireless
installed in the APM building. <<<<
Finally, congratulations to Marco and Francesca on the birth of Martino.
And really finally, a mysteriously acquired bottle of champagne was consumed
in honour of these latter two momentous events.
Continuing Actions
------------------
RGM report back on classification aspect of cross-talk weeding
EGS
MJI
JRL work out scheme for doing header updates for ING archive
EGS
New Actions
-----------
MJI forward weekly email JAC reports from AJA to a wider audience
MJI check status of the reprocessed UDS data
PSB contact Brad Cavanagh at JAC for an update regarding the WFCAM
network transfer
MR figure out best to way to advertise illumination correction information
JRL continue with self-flatting tests on VIRCAM data and email summary of
points to note from VIRCAM tests to participants of telecon
JRL compile wishlist for late October VIRCAM cold camera tests
PSB
JRL reacquire the helpdesk laptop and PDF'ify ADASS presentations
DWE start duty of updating apm1 Windows machine
PSB investigate getting wireless for the APM building
MJI buy handheld vacuum for APM building
PSB
ALL consider attending UKIDSS meeting 17-19 December at ESO
