File contents

Wednesday 15th July 2009   APM Meeting tables    11:30-12:30

Present:  MJI, STH, RWA, NAW, MR, EGS, JRL, SC, RGM (later), DWE (later) 

Agenda
------
1.  Actions from last meeting
2.  Comments on WFAU minutes
3.  Recent and upcoming meetings
4.  Data archives update     - AAT, ING, WFCAM, UKIRT
5.  Optical/NIR processing   - HAWKI, INT WFC, MegaCam, Subaru, VST.....
6.  WFCAM update
7.  VISTA update
8.  AOB - rolling grant status etc..

Minutes
-------

1. Actions from the last meeting

RWA    VISTA technical pages are now being populated with technical 
EGS    information derived from various commissioning data and from drafts 
       of reports from JRL.  Strange formatting is being eliminated in
       parallel and a link to the pipeline progress web section is being
       added.  Since this will be an ongoing activity for at least the
       next several months and the information is also needed as input for 
       the VISTA User Manual that ESO are preparing, this item has now been
       taken off the action list and will in future appear in the VISTA
       update section.

MJI    stage I software release - still ongoing                            <<<<

MJI    technical paper - still ongoing                                     <<<<

MJI    utility to convert binary tables plus corrections - still ongoing   <<<<

EGS    UVEX support - program sent

MJI    A/C unit fixed after nearly 2 weeks - a stunning response time.  MJI 
       & SC have re-configured apm35_a to be the raw VISTA data suppository 
       and are in the middle of converting apm29_a,b,c to be the VISTA 
       processed data dump.  JRL will look at PSB's stuff on apm29_b before 
       we get rid of the partition and see if we need to keep any of it. 
       The processed data on apm29_c will be moved to apm29_a prior to
       getting rid of the apm29_c partition and tidying up the rest of the
       detritus.  Related to this STH suggested we construct a guide to what 
       data is on what disk (mainly related to raw ING archive it turned out)
       and felt so strongly about it that he was volunteered to update the
       relevant web page and also to keep it up to date.                   <<<<

MJI    the Petrosian Fix (sounds like a techno-thriller) still needs to be 
       carried through but has been held up by assorted VISTA commissioning 
       tasks and the finite number of hours in a week.                     <<<<

JRL    has submitted an abstract for ADASS XIX and thereby gets to keep his 
       Silver Star and his horse

MJI    made the data for the assessment of the need for filter flushes on
       WFCAM available, but does not know if any subsequent decisions have
       been made.  He suspects it will be left up to the individual survey 
       heads and PIs of other WFCAM programmes.

STH    have compared the ETC throughput estimates with those derived by STH 
MJI    by hand and found good agreement.  VISTA appears to have nearly a 
       factor of 3 higher DQE compared to WFCAM.  These throughput values
       will shortly be included on the technical web pages.  The ETC estimates
       of sky brightness also seem very good.  Todos still are the magnitude
       limits/signal:to:noise estimates and an update of the overheads.
       There is now enough data to get a good estimate of the average
       sky brightness in Z,Y,J,H,Ks for the ~month of data in hand since
       VISTA restarted on-sky observation on 19th May.   MJI will generate
       an overview summary plot of the sky brightness levels over this one 
       month period and hand it over for the technical web pages.          <<<<


2. Comment on WFAU minutes

No significant comments so we had a resume of status of data transfers, DR6 
checking and timescales.


3. Recent and upcoming meetings

The last VISTA IOT on June 30 produced some interesting numbers and dates.
The minutes are on the CASU internal web pages for those of a curious bent.
The next IOT telecon will be July 29+/-1.

The deadline for abstract submission for the UKIRT workshop (14-16th 
September in Edinburgh) is the end of July. STH said he intended to go to 
give his invited talk but just in case he is observing then he has a backup
plan to delegate to David Pinfield.  MJI, in collaboration with Annette 
Ferguson, will be submitting an abstract on the LG work with WFCAM.

There will be a VDMT this afternoon. 


4. Raw data archives

AAT,ING,WFCAM,VISTA everything ok and up to date.
Now that a lot of the dust has settled MJI asked JRL and RWA to look into
creating an offline tape backup of the raw VISTA data using the LTO4 library
unit attached to apm29.                                                    <<<<
 
UKIRT - JRL said he was back on the case and promises to finish by late 
summer, though which year wasn't specified.


5. Optical/NIR processing : HAWKI, INT WFC, MegaCam, Subaru, VST......

Nothing much happening for a change mainly due to hols and effort dedicated
to processing VISTA commissioning data.

EGS is processing some Subaru data and STH is tackling some CFHT and 
`bizarre' Russian data - whatever that means.


6. WFCAM update.  

Processing:
MR noted that apart from the odd hiccup in raw data transfers due to knock-on
effects from a power failure at JAC end on 20090623, everything is running
smoothly.  Raw data transfers are up to date as is the processing.  June
data has just been flagged as ready for transfer.

Reprocessing:
All the GPS nebulosity reprocessed (more accurately optional extra) data for
DR5 was flagged as ready to copy and has been slurped up.

Likewise the UDS K-band data earmarked for the UDS DR5 release has also been
flagged and transferred up by Edinburgh.

DR6:
The Petrosian Fix is still next in the queue, it got bumped by VISTA data
temporarily, but should be out of the door by the end of July.

Various UKIDSS checkers have reported back:
LAS jpeg checking looks good - only 1.1% FU's and these are virtually all 
trailed images;

GPS noted a small bunch of moon reflection issues on one particular night
20081113.  The "shadows" change rapidly with frame sequence number making 
them effectively unfixable, but given the nature of the shadowing it 
probably doesn't affect the catalogues much anyway.

MR cunningly spotted some ZP funnies with some UDS data on 2080920.  It 
transpired that the pipeline had obviously got bored with interleaving first 
and then stacking and instead went straight for stacking.  All the data
passed QC checks since there was nothing wrong with it per se.  This problem
occurred about half way through the night. A few GCS frames were also affected 
in the same way.  It looks like this occurred after the pipeline was restarted 
mid-way through the processing.  Impossible to reproduce the problem, never 
been seen before and hopefully won't be again.  We'll write that one down to 
the "fat electron" theory.  The affected frames have already been reprocessed 
and reflagged to go.


7. VISTA update

Raw data has been received via Garching for up to and including June 24. It 
includes some daytime calibration data with good linearity sequences from 
which JRL has also derived some good bad pixel masks. JRL also exploited the 
recent linearity data to investigate low light level linearity.  Some 
of this data was taken to explicitly probe the low-count regime and indeed 
shows non-linearities exist at the 1 - 3% level even below 1000 ADUs.

STH commented he needs to revisit the linearity check in the catalogue
photometry -v- 2MASS to ensure that the non-linearity has indeed been
corrected for.  He also needs to revisit the colour equations now that we 
have about twice as much data as last time we computed them.               <<<<

As noted earlier, MJI will supplement this with the sky brightness estimates
and plots.

EGS independently discovered that the standard for describing WCS keywords 
in FITS binary table catalogues is different to what we have been using so 
far.  Some of our post-processing software has not been adapted to the 
different standard yet.

JRL, as always, is taking a broom to the pipeline and tidying things up and 
fixing things faster than he breaks them.  A pernicious buggette with
the pipeline astrometry which generally manifested itself in crowded fields
may have been tracked down and fixed, much to the post-pipeline astrometry
fixer's relief.  This may also have been the cause of some of the ESO
tickets about the stability of the photometric calibration results from
the summit and Garching pipelines.  In contrast the Cambridge photometric 
zeropoints on the summary pages look very well behaved.

JRL released via the VPO a new version of the summit pipeline to fix a couple 
of problems but does not know if it is being used yet. He reported that 
Sandra Castro, who was dealing with the VISTA pipeline has left ESO and is 
being replaced by Lander de Bilbao.

During processing recently apm29_a,b,c hung up a couple of times. It may
be a software issue and SC has now applied the most recent software patches 
to see if that fixes the problem.

MJI noted that since the upgrade the remote temperature monitoring web
page for apm29 has flatlined.  SC will take a look see why.                <<<<

Remaining commissioning tasks include investigating the efficacy of deep
stacking on a 16hour set of J-band pawprint data and completing the loop   <<<<
for the offsetting accuracy when/if we get more data.  The other previously
outstanding issue relating to stability of the pin cushion distortion model,
including centre coordinates, was booted out the door last week.

Having been asked to look at science data, MJI examined the VISTA Y and H 
tile OBs for the globular cluster M55 and produced an appealing colour 
image from the tiles together with a plausible CMD from the individual
detector stacks.  He noted that the passbands used had been cunningly 
chosen to avoid all published theoretical isochrones so it was difficult 
to analyse the rather nice M55 locus quantitatively.

Feeling upbeat he then tried to repeat the trick on test tile OB J,Ks data
taken of the Sculptor dwarf (to compare with CIRSI observations) at airmass 
of ~1.5 and found that subtly varying seeing throughout the tile exposures
caused some interesting problems when using catalogues generated from the 
resulting tiles.  This was not completely unexpected since each sub-region 
in the tile is made up of different partial exposures potentially leading to 
nasty disjoint PSF variations across the tile.  The effect was worse than
anticipated even using aperture photometry, which is not as susceptible to
systematics as PSF fitting.  This highlights a generic problem in doing 
accurate photometry on VISTA tiles and will in general require a 
VISTA-specific solution.  He then outlined a couple of possible solutions
neither of which is particularly elegant.  This requires some further work.


8. AOB

MJI reported on the current grant status and noted the recent joint letter
that had been sent out by WFAU & CASU to alert users to the situation.

SC & MJI are looking at ways of adding new A/C capability to deal with the 
proposed increased amount of Gaia hardware and also in the near future at 
least to provide some redundancy in cooling capability.  This posed the 
ancillary problem of what to do with the large number of (mainly) ING 
archive CD/DVD storage cabinets which would need to be moved out to make 
room for new computing kit.


Continuing actions
------------------

EGS  add catalogue software and binary table conversion software to software 
MJI  release pages  

MJI  finish off technical paper updates

MJI  write utility to convert binary tables with all corrections

MJI  do the Petrosian fix

STH  further compare ETC fact with fiction
MJI


New actions
-----------

STH  update data storage web page particularly with respect to ING archive

MJI  generate an overview summary plot of the sky brightness levels using
     recent VISTA data and hand it over for the technical web pages

RWA  generate LTO4 backups of VISTA raw data received to date

STH  revisit average zero-points and colour equations for VISTA and check
     non-linearity correction is working by cf. with 2MASS data

SC   fix the flatlined temperature monitoring web page for apm29

EGS  investigate stacking on the 16hour set of J-band pawprint data