File contents

26th July 2005  Hoyle committee room 11.30 - 13:25

Present:    MJI, RGM, PSB, DWE, MR, JRL, WJS, EGS, NAW
Apologies:  STH

Agenda
------

1.  Actions from last meeting
2.  Comments on WFAU minutes
3.  Data archives update - UKIRT, WFCAM, AAO, ING
4.  Optical processing update
5.  WFCAM JAC telecons and processing status
6.  VDFS update
7.  AstroGrid liaison update
8.  AOB - ADASS 2005

Minutes
-------

1.  Actions from last meeting

JRL  has still to include access to "observing" log index files in the
     raw WFCAM archive. It was decided to transfer this action to 
     MR as part of the interface update

MJI  are still in the process of writing a new report on the detector 
JRL  properties as they currently are and the processing strategy. This
MR   is needed for the SV meeting in Edinburgh (18 August) and should be
     finished about a week before this. It was decided that EGS's report   <<<<
     should be separate since it deals with the internal science 
     verification. EGS, MR, MJI and STH are the likely attendees.

JRL  AAO have not replied yet regarding the connectivity issues to AAO. It
     was suggested that in order to progress this an additional email 
     (cc'd to various other interested parties) or a telephone call was
     required - ongoing                                                    <<<<

WJS  was unsure whether the Nov 2004 version was the latest SADT Definition 
     Document. He said that he would find out what is happening - ongoing  <<<<

MJI  raised the issue of observing efficiency at the JAC telecon (see item 5)

RGM  said that the AstroGrid hardware money was still there

RGM  sent PSB's comments on the document covering the service support for 
     VO to Tony Linde

STH  has yet to trial the test version of the SADT available for both VISTA
     and VST - ongoing                                                     <<<<

ALL  only DWE sent in a Q3 plan.  MJI will draw up a quarterly draft plan
     and circulate it for comments. The finalized plan needs to be done 
     in advance of the next VDMT meeting (11 August) - ongoing             <<<<

MR   has not yet improved the look of the front page of the WFCAM raw data 
     archive - ongoing                                                     <<<<

JRL  is half-way to sorting out local scripts for easy access to the mirrored
     OMP data base tables - ongoing                                        <<<<

MR   has investigated Aladin for parts of the CASU WFCAM processed DQC 
     archive. He said that although it was in Java (painful), it would be
     useful because of its generality. It was decided this should now go 
     into the Q3 plan rather than remaining an action.
     
JRL  has run his summit pipeline timing tests on other nights, in particular
     those containing DXS and GPS observations where the pipeline has to
     work hardest. On average, the pipeline only gets a 5 minutes backlog
     when processing this type of data.  He will report on this at the
     next JAC telecon.                                                     <<<<

MR   has investigated the properties of all flats taken during 05A and found
     them to be globally stable to better than 1% for all filters. The noise 
     properties were also OK. More detail will go in the aforementioned 
     report. JRL was asked to look at the sky calibration frames to see if 
     there is any correlation between Moon angle and scattered light.      <<<<

DWE  has put in a link to the processing progress web page and the raw data
     archive off the main VDMT web page. 

MR   has started to put some explanatory text in the processing progress web 
     page but has not finished - ongoing                                   <<<<

STH  has not finished the 3 remaining RIXs in the ETC document. This has to
     be finished by 5 August - ongoing                                     <<<<

PSB  forwarded the Michele Peron email to JPE and JMS

PSB  has checked the status of the various VISTA documents. Not all are
     up-to-date - ongoing                                                  <<<<

WJS  has not yet checked the VISTA template definitions  - ongoing         <<<<

MJI  has not yet raised the calibration/science frame issue at the VDMT
     since there hasn't been one since the last meeting. The general feeling
     was that it would be simpler to classify the observations of the 
     standards as calibration frames - ongoing                             <<<<

PSB  has contacted JPE about the status of the SADT with respect to the
     RIXs - ongoing                                                        <<<<


2.  Comments on WFAU minutes

RGM said that he had managed to access the WFAU archive. There was a short
discussion regarding Petrosian magnitudes and other flux measures.


3.  Data archives update - UKIRT, WFCAM, AAO, ING

UKIRT: about 20 more requests have come in. We are about 1 year behind the
observations ie. up-to-date with the proprietary period.

WFCAM: all the raw data has been ingested. About 2 more weeks worth of data
await transfer to ESO.

AAO: awaiting resolution of connectivity issue.

ING: over the last month data has been delivered over the Internet (~150
Gbyte/month). We are currently keeping up with the observations. This method
requires much less effort that reading in DVDs. JRL suggested that Rice
compressing this data might be useful. MJI noted that the data was gzipped
prior to transfer and suggested that we could wait another month for the 
dust to settle before starting on this.


4.  Optical processing update

EGS said that he had been processing INT WFC data for Alan Fitzsimmons and 
Hans Deeg. There were no problems at the moment in keeping up with this.

RGM said that an XMM user wanted to stack the processed products where
applicable.  MJI said that the simplest solution was to use our stacking 
software. EGS said that he had been using this for various WFCAM tests and
would make this available.                                                 <<<<


5.  WFCAM JAC telecons and processing status

The next JAC telecon will be tomorrow evening. The main topic of discussion
will be the pipeline requirements and how far they have been met.  Problems
with 3x3 interleaving and also the chip 2 dropouts will be rasied.

At the last JAC telecon (15 July) the main topic was observing efficiency
and the design of the MSBs for the UKIDSS surveys.

RGM asked what fraction of time is spent observing standards. This did not
seem to be factored into the various survey observing plans since the old
calibration plan was to use 2MASS. It was not clear though when this earlier 
plan was abandoned (2003?) and whether or not the impact of the new 
calibration strategy had been assessed.

NAW asked how would the summit pipeline be affected if the observing
strategy for the MSBs were to be changed to be more efficient by using
less interleaving, longer exposures or more coadds. It was pointed out that 
there generally would be less frames to process, the summit processing is
quite flexible, and the load on the summit pipeline would therefore 
generally be less.

DWE gave some more details regarding the observing efficiency derived
from the timing information in the FITS headers.  For each observation 
(integration) there is about 1.2s of reset-read-read overhead per
exposure. If there is a filter change or there is a large movement of the 
telescope there is an overhead of ~30-45s. The remaining overhead is 2-3s per
observation. There doesn't seem to be any correlation between this overhead
and the amount of dither movement that has been made.  Further more detailed
information on the current and expected overheads are being sent out by JAC
for discussion at the Survery Heads meeting in Durham.

MR said that the reductions were progressing well. Almost all have been
finished apart from some frames that need some manual intervention. Data 
up to 6 June is ready and waiting for transfer to Edinburgh. Transfer of 
data has been completed up to 10 May. He said that the catalogue generation 
can sometimes take a very long time in the Galactic plane survey. RGM asked 
if this would cause problems for PSF fitting. DWE said that with the current 
version of the software, all the hard work had been done by the catalogue 
generation programme in performing the deblending and that the PSF fitting 
itself was quite quick.

MJI said that we need to consider what lessons we have learnt from these
reductions and factor in various improvements before the next batch of
data arrives.  He also noted that several aspects of the pipeline had
improved since the first batch of data (April) had been processed and 
that the catalogues will also all need regenerating once the parameters
have been finalised.  This will eventually include PSF V1,2 information.

DWE asked what happened to badly interleaved data. MJI said that the grouped
frames are thrown away, but that the individual frames are kept. Similar
questions were asked regarding frames where chip 2 has failed. The data is
kept, but the data quality control will indicate that the frame requires
reobserving.

NAW asked about whether the CASU DQC will be fed back to JAC. This will be
the case, but the JAC DQC should pick up most of the problems.

EGS reported on the analysis he has been carrying out on the SV1 DXS data.
The relative astrometry has an accuracy of 0.03" and the photometry 3%
(for 13-17 mag). Comparisons have been carried out with other surveys. The
DXS stack goes down to 20.6 mag in K (5 sigma limit in 2 hours). It was 
noted that the UDS data overlaps with some of this, but there is a problem 
with the 3x3 interleaving. EGS said that the report wasn't quite ready for 
general release but that he would distribute the web location to the group 
for comments.                                                              <<<<

MJI said that he would be attending the UKIDSS survey heads meeting at
Durham tomorrow.


6.  VDFS update

PSB reviewed the latest observing templates, one issue of note was that a
single on/off command for the flat lamps had been provided, but after
discussion will now be a 3-bit code to control the three sets.  Some other
minor discrepancies were passed on.

MJI reiterated that STH needs to finish the ETC document.

PSB noted that SMB has said that "what's important is the list of parameters
associated with each template. It would be good to reach a general agreement
on those parameters before submitting them to ESO". ESO will then release a
VISTA-enabled version of P2PP (ESO Phase-2 Preparation tool) which can be
used to generate OBs.

The ESO DRS (Data-Reduction Specific) keyword proposal, originally due
in the spring, has been acquired. PSB remarked on the extremely concise
wording in the proposal.

WJS reported on a number of VISTA hardware issues. A test set of detectors 
being cooled together with an autoguider CCD so that tests including pickup 
and cross-talk can be carried out on them.  One of the three camera corrector 
lenses is still in the process of being polished.  QEs at Z and Y of ~60-70%
have been measured.

Issues regarding inter-pixel capacitance have been raised at recent meetings.
These cause an small coherence in noise properties between pixels such that
the noise covariance matrix has a modest off-diagonal power (~20% for Hawaii
II, unknown? for the Raytheon detectors for VISTA).  This causes, for example,
the photon noise to be underestimated and hence overestimates things like 
the gain by similar amounts.

WJS noted that VISTA should be on the sky by the second half of next year.

There was some concern that there may be pressure to cutback in the testing 
phase in order to save money. This was regarded as a false economy.


7.  AstroGrid liaison update

No news is ........


8.  AOB - ADASS 2005

The next ADASS will be in Madrid on 2-5 October. JRL has not quite         <<<<
finished writing the abstract yet. It is expected that JRL, PSB and EGS 
will attend.

We were reminded to keep our .away files up-to-date so that we can work out,
among other thngs, when to have the next meeting.                          <<<<


Continuing Actions
------------------

MJI  finish off report on the detector properties as they currently are 
JRL  include section on processing strategy and illustrate with 
MR   processing examples. Also finish EGS's SV report.
EGS  Deadline: a week before the SV meeting in Edinburgh (18 August).

JRL  finish off download side of AAO archive scripts (further email
     prodding or a telephone call required)

WJS  find out status of new version of the SADT Definition Document

STH  drive test version of the SADT available for both VISTA and VST 

MJI  draw up a draft Q3 plan and circulate for comments

ALL  review Q3 plan by the next VDMT meeting (11 August)

MR   improve look of front page of WFCAM raw data archive

JRL  sort out scripts for access to the summit comments database and
     make available

MR   put some explanatory text in the processing progress web page

STH  finish the 3 remaining RIXs in the ETC document (by 5th August)

PSB  check the status of the various VISTA documents and ensure they are
     up-to-date

WJS  check that VISTA template definitions are sensible

MJI  raise calibration/science frame issue for VISTA at next VDMT

PSB  wait for response from JPE regarding the status of the SADT with 
     respect to the RIXs


New Actions
-----------

MR   include access to "observing" log index files in raw WFCAM archive

JRL  investigate the sky calibration frames to see if there is any
     correlation between Moon angle and scattered light

EGS  make stacking software available for specific requests

EGS  finish and distribute location of SV1 report

ALL  send back comments on SV1 report to EGS

JRL  finish writing ADASS abstract

ALL  keep .away file up-to-date