File contents

6th July 2004  Obs meeting room 10.15 - 13:00

Present:   JRL, PSB, MJI, DWE, WJS, STH, JMI
Apologies: RGM

Agenda
------

1.  Actions from last meeting
2.  Comments on WFAU minutes
3.  JAC/WFCAM news inc. Derek's visit
4.  Report from SPIE meeting
5.  MSB design issues
6.  Status of WFCAM processing hardware + JRLs Sybase IQ kit
7.  Status of UK VDFS URD
8.  Results of CASU job interviews
9.  Q3 work plans Q2 progress
10. AOB
    - RIX responses
    - APM building news update
    - next grant round bid

Minutes
-------

1.  Actions from last meeting

STH  has finished studying the spatial systematics in the UFTI standards 
     data. Photometry is taken at 5 positions so that a sky flat can be 
     generated. However, fringing or dust emission is potentially a 
     problem. The data from each of these positions were stacked 
     separately. No photometric systematics were detected at the ~1% level. 
     This was carried out for the J and H data. With this limited spatial
     coverage (+/- 15 arcsec) no effect can be seen, however this is not an
     extensive test but it was felt that further tests should await WFCAM 
     commissioning data.
     
MJI  no progress on comparison of FIRES project reduced data with ours 
     (catalogues for the FIRES data are needed) - ongoing                  <<<<

JRL  reported that QFITS 5.0.0 has been released and progress has been made.
     Almost all items on our wish-list are there or can be outmanouvered.
     For example, a single extension from a multi-extension FITS file can 
     now be extracted (although not by using the usual [] naming convention).
     The main thing missing is image compression. JRL was asked to draw up
     up a short (2 page) document summarizing our wish-list for CPL/QFITS.
     PSB was asked to help draft this and then circulate it first within
     CASU and then around the VDMT so that it could be sent to Michelle 
     Peron by the end of July.                                             <<<<
     
ALL  the job interviews for the CASU post have taken place (see item 8)

MJI  has sorted out the building name. The name agreed was the Astronomical
     Survey Unit - APM building.

PSB  has placed the VISTA Survey Planning document on our internal web site.
     It was noted that Martin Folger will be visiting ESO to discuss
     survey issues with them.
     
JRL  gave a presentation at the PPARC Data Curation Review, which appeared
     to be well received. The main unique observatory archives stored on 
     the Cambridge server are AAO, UKIRT and ING.


2.  Comments on WFAU minutes

It was noted that PSB has also done some soak testing, of keyboards, and 
has found that Adnam's Broadside is particularly effective.

There was general consensus with Nigel's viewpoint that we should carefully
consider the cost-benefits of attending meetings ($700+ for a name badge 
at a recent meeting being a good example of part of the cost).

However, JRL noted that an exception to this was the ADASS meeting in
Pasadena (October 24-27). The deadline for abstracts is 2nd August. It is
probable that JRL will be attending from CASU.  MJI suggested making a
start on the abstract and contents of talk using the SPIE abstract and 
submitted paper.


3.  JAC/WFCAM news inc. Derek Ives's visit

JRL reported on the meeting he had with Derek Ives.  The main points of
the meeting were to gain a better understanding of the read-out modes of 
the camera and the detailed orientation of the readout sector pattern. 
Currently all 4 science arrays are in the test cryostat and are being 
characterised.

The cross-talk characteristics seem to only occur within a quadrant due to
some nifty wiring. The persistence seems to have a very large time constant
of about 1 minute (not an RC value) and appears to be independent of the
number of resets.  If this persists *groan* this may have some interesting
consequences.  Derek will do some flat field exposures for us to test the 
reset anomaly. JRL will obtain a copy of this data and of the report of 
the recent detector tests on a few week timescale.                         <<<<

Due to corrector plate properties, the seeing will be much more stable 
and WFCAM will start out well-sampled with 2-3 pixel FWHM images. The 
current plan is to ship without the corrector, carry out a series of 
commissioning tests as scheduled and then eventually install a correctly-
figured corrector plate.

In a nutshell, the outcome of the acceptance tests were that the 
electronics and detectors have passed, but there were problems with the 
optics. 

The next telecon will be on the 14th July at which these and other issues 
will be discussed.


4.  Report from SPIE meeting

Nigel Hambly, Jim Emerson and MJI gave presentations at the SPIE meeting in
Glasgow. No new issues were raised at the meeting. JPE pointed out in his
presentation that ESO ought to consider the possibility of doing the first
stage (ie. standard) pipeline processing, since they were carrying out most
of the work for calibration purposes anyway (as we have been recommending
all along).


5.  MSB design issues

All UKIDSS survey heads have now submitted plans. UDS will be using a
similar plan to that of DXS.  This appears to be a standard dither/
interleave pattern (fixed but order randomised) such that no image pixel 
resampling will be required (ie. small differential astrometric distortion). 

Offset skies (GPS) must be within an MSB in order to ensure that they are
carried out contemporaneously with the main programme data.

The LAS MSB selection will be dependant on the observing conditions.

MSBs will range in duration from ~10 minutes to ~1 hour and will be
interwoven with broadband (all) photometry MSBs roughly every hour.

The minutes of the last UKIDSS Implementation Meeting (9/6/04) are available
on the UKIDSS web pages.

Since the last update for this was a few weeks ago STH volunteered to 
contact Simon Dye to find out what is happening.                           <<<<

The current set of MSB designs will be processable with relatively simple
stacking algorithms ie. no resampling. This will mean that the nightly 
MSB processing can be implemented using only the standard pipeline 
components.


6.  Status of WFCAM processing hardware + JRLs Sybase IQ kit

MJI reported that some of this was ordered about 2 weeks ago (after 
notification from PPARC that the extra money would be forthcoming) and has 
already arrived. The processing hardware consists of 5 rack-mountable twin 
Xeon processor 3GHz PCs each with a 3ware 1 Tbyte RAID array and 2 Gbytes 
of memory.  They all have hot swappable everything including twin redundant 
power supplies (yes even these).

JMI has installed Debian on these machines and they are all up and running
in a single rack.  This completes the standard processing hardware for
WFCAM.   Together with the earlier machine (similar spec.) we now have in 
place: an ingest server, 4 processing drones and a data products server.  
External mass storage devices are still needed and another backup LTO-II 
tape loader.  Purchase of these items will be phased in over the next few 
months as required.

The current plan is to test the machines by processing a 250 Gbyte chunk of
recently received INT WFC data on them.  For this test the data for each 
filter will be fed into a separate machine with the ingest server doing 
the bossing.  WJS asked what would happen for WFCAM if all of one night 
was in a particular passband, could the processing cope since only one
machine would be being used.  There might also be problems with this scheme
with eg. persistence.  At this stage it is not 100% clear how linear the 
processing chain will be and what bits can be parallelized so this is more
a test of machine inter-control and buck passing.

It was noted the master processing software (/usr and /home) on the data 
ingest server is mirrored on the data product server (via rsync) every 24 
hours, as a backup and spare.  DWE/PSB suggested that it would be wise to 
also use a machine in another building to store a backup. JMI will 
investigate doing this on a 24 hour cycle                                  <<<<

JRL reported that he had been testing a version of Sybase IQ. This is a
large read-only (ie. only update occasionally) database. The results have
shown that it is very fast (PostgreSQL a transactional system is very slow 
in comparison). The remaining question is which version of RedHat Linux to 
use.  According to the blurb IQ has to be run under RedHat (Enterprise 
expensive, Fedora hmmm.....), we don't know if we can run it under Debian.  
The database will be run on a dedicated fast PC with the same hardware spec.
as the WFCAM processing machines.


7.  Status of UK VDFS URD

JPE has circulated a version of the UK VDFS User Requirements Document. 
This has supposedly only had restructuring carried out since the previous 
version, so it was suggested that it would be more ergonomic if WJS did a 
quick check of the document first for obvious typos before the rest of
us comment on it.                                                          <<<<

WJS said that some useful input has come from the VDUC members and a section
on variable stars has been added. There was also some discussion regarding
the Z filter. PSB said that even though the current finances do not stretch
to buying a Z filter, the URD should task the pipeline with the ability to
process such data. 

After WJS's checks we should all read the latest version of the URD.       <<<<


8.  Results of CASU job interviews

The advertising for this post was deliberately low key both to test the
water and to ensure we were not swamped with applications.  However, as
it happened, we received several promising applications (and several not
so promising).  Two candidates clearly stood out above the others, a 
post-doc from the IoA and a final year PhD student from ESO.  Both were 
short-listed and interviewed and were unambiguously acceptable for the job.

To cut a long story short, the ESO candidate was offered, and accepted, the 
advertised post, and will start working on VDFS-related issues from January 
next year.  The IoA post-doc will work part time from October this year 
mainly on other CASU projects and potentially later on WFCAM processing/
operations.

Longer term funding arrangments will be reviewed after the results of the
next grant round bid.


9.  Q3 work plans Q2 progress

The next VDMT meeting with be on Thursday 15th July in Edinburgh. Progress
reports for the last month, earned values data (days or %) for the last
quarter and next quarter tasks to STH by Friday.                           <<<<

10. AOB

    - RIX responses
    
It was decided that it would be more efficient for all those who attended 
the PDR (starting with PSB) to comment in turn on a hand-me-down copy of 
the recently received RIX response document. After it has done a complete 
round, PSB will collate and forward the collective wisdom to JPE.          <<<<

    - APM building news update

This is still on schedule (sic!). By 1st September the APM building has to 
be cleared of ALL junk.  There was some discussion regarding where to 
move all the good junk to (the active computer equipment has an earmarked 
temporary home already).  One suggestion was that we recolonize part of 
Greenwich House again.  MJI said he and Peter Weeks (PW) from the 
workshop had already discussed this with Paul Aslin who saw possible 
interesting political ramifications.  MJI and PW will do a bit more
negotiating and report back at the next meeting.                           <<<<

MJI pointed out that most of the newly acquired hardware would probably
not be running while the building work was being carried out (because
of the heat load in the office it was temporarily being relocated to). 
Probably only the ingest and data product servers would be powered up,
but this should be enough to assess any WFCAM commissioning data and 
help fine tune the pipeline. 

    - next grant round bid

The next grant bid will need to be in by the end of September. This will
cover the ramp down in the current CASU grant from 1st April 2005 and will 
include WFCAM operations, the rest of the Data Archive activities etc...,
but not VDFS development or GAIA activities.

    - ad hoc

MJI noted that Paul Hirst's current User Guide has WFCAM with a J-band
sensitivity ~1 magnitude worse than those from previous estimates based on
UFTI predictions and original WFCAM estimates.  H-band has changed for the 
worse by ~0.6 magnitudes and K-band by ~0.3 magnitudes.  MJI is currently 
investigating WTFIGO in conjunction with PH.


Continuing Actions
------------------

MJI  compare FIRES project reduced data catalogues with ours
     (generate catalogues for the FIRES data)

New Actions
-----------

PSB  draft a short document summarizing our CPL/QFITS wish-list. Circulate 
JRL  this so that it can be sent to Michelle Peron by the end of July.

JRL  obtain data and copy of the report of the recent tests from Derek Ives.

STH  contact Simon Dye to see what is happening regarding the MSB design.

JMI  investigate adding an additional backup to a machine outside the APM
     building.

WJS  check UK VDFS URD for faux pas and correct then send to ALL

ALL  then read and comment on the latest version of the UK VDFS URD

ALL  monthly reports and quarterly data to STH by Friday

PSB  comment on and circulate RIX response document
WJS
MJI
JRL
STH

PSB  collate RIX response document comments and forward to JPE

MJI  raise issue of storage requirements for detritus from APM building