Wednesday 15th July 2009 APM Meeting tables 11:30-12:30
Present: MJI, STH, RWA, NAW, MR, EGS, JRL, SC, RGM (later), DWE (later)
1. Actions from last meeting
2. Comments on WFAU minutes
3. Recent and upcoming meetings
4. Data archives update - AAT, ING, WFCAM, UKIRT
5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru, VST.....
6. WFCAM update
7. VISTA update
8. AOB - rolling grant status etc..
1. Actions from the last meeting
RWA VISTA technical pages are now being populated with technical
EGS information derived from various commissioning data and from drafts
of reports from JRL. Strange formatting is being eliminated in
parallel and a link to the pipeline progress web section is being
added. Since this will be an ongoing activity for at least the
next several months and the information is also needed as input for
the VISTA User Manual that ESO are preparing, this item has now been
taken off the action list and will in future appear in the VISTA
MJI stage I software release - still ongoing <<<<
MJI technical paper - still ongoing <<<<
MJI utility to convert binary tables plus corrections - still ongoing <<<<
EGS UVEX support - program sent
MJI A/C unit fixed after nearly 2 weeks - a stunning response time. MJI
& SC have re-configured apm35_a to be the raw VISTA data suppository
and are in the middle of converting apm29_a,b,c to be the VISTA
processed data dump. JRL will look at PSB's stuff on apm29_b before
we get rid of the partition and see if we need to keep any of it.
The processed data on apm29_c will be moved to apm29_a prior to
getting rid of the apm29_c partition and tidying up the rest of the
detritus. Related to this STH suggested we construct a guide to what
data is on what disk (mainly related to raw ING archive it turned out)
and felt so strongly about it that he was volunteered to update the
relevant web page and also to keep it up to date. <<<<
MJI the Petrosian Fix (sounds like a techno-thriller) still needs to be
carried through but has been held up by assorted VISTA commissioning
tasks and the finite number of hours in a week. <<<<
JRL has submitted an abstract for ADASS XIX and thereby gets to keep his
Silver Star and his horse
MJI made the data for the assessment of the need for filter flushes on
WFCAM available, but does not know if any subsequent decisions have
been made. He suspects it will be left up to the individual survey
heads and PIs of other WFCAM programmes.
STH have compared the ETC throughput estimates with those derived by STH
MJI by hand and found good agreement. VISTA appears to have nearly a
factor of 3 higher DQE compared to WFCAM. These throughput values
will shortly be included on the technical web pages. The ETC estimates
of sky brightness also seem very good. Todos still are the magnitude
limits/signal:to:noise estimates and an update of the overheads.
There is now enough data to get a good estimate of the average
sky brightness in Z,Y,J,H,Ks for the ~month of data in hand since
VISTA restarted on-sky observation on 19th May. MJI will generate
an overview summary plot of the sky brightness levels over this one
month period and hand it over for the technical web pages. <<<<
2. Comment on WFAU minutes
No significant comments so we had a resume of status of data transfers, DR6
checking and timescales.
3. Recent and upcoming meetings
The last VISTA IOT on June 30 produced some interesting numbers and dates.
The minutes are on the CASU internal web pages for those of a curious bent.
The next IOT telecon will be July 29+/-1.
The deadline for abstract submission for the UKIRT workshop (14-16th
September in Edinburgh) is the end of July. STH said he intended to go to
give his invited talk but just in case he is observing then he has a backup
plan to delegate to David Pinfield. MJI, in collaboration with Annette
Ferguson, will be submitting an abstract on the LG work with WFCAM.
There will be a VDMT this afternoon.
4. Raw data archives
AAT,ING,WFCAM,VISTA everything ok and up to date.
Now that a lot of the dust has settled MJI asked JRL and RWA to look into
creating an offline tape backup of the raw VISTA data using the LTO4 library
unit attached to apm29. <<<<
UKIRT - JRL said he was back on the case and promises to finish by late
summer, though which year wasn't specified.
5. Optical/NIR processing : HAWKI, INT WFC, MegaCam, Subaru, VST......
Nothing much happening for a change mainly due to hols and effort dedicated
to processing VISTA commissioning data.
EGS is processing some Subaru data and STH is tackling some CFHT and
`bizarre' Russian data - whatever that means.
6. WFCAM update.
MR noted that apart from the odd hiccup in raw data transfers due to knock-on
effects from a power failure at JAC end on 20090623, everything is running
smoothly. Raw data transfers are up to date as is the processing. June
data has just been flagged as ready for transfer.
All the GPS nebulosity reprocessed (more accurately optional extra) data for
DR5 was flagged as ready to copy and has been slurped up.
Likewise the UDS K-band data earmarked for the UDS DR5 release has also been
flagged and transferred up by Edinburgh.
The Petrosian Fix is still next in the queue, it got bumped by VISTA data
temporarily, but should be out of the door by the end of July.
Various UKIDSS checkers have reported back:
LAS jpeg checking looks good - only 1.1% FU's and these are virtually all
GPS noted a small bunch of moon reflection issues on one particular night
20081113. The "shadows" change rapidly with frame sequence number making
them effectively unfixable, but given the nature of the shadowing it
probably doesn't affect the catalogues much anyway.
MR cunningly spotted some ZP funnies with some UDS data on 2080920. It
transpired that the pipeline had obviously got bored with interleaving first
and then stacking and instead went straight for stacking. All the data
passed QC checks since there was nothing wrong with it per se. This problem
occurred about half way through the night. A few GCS frames were also affected
in the same way. It looks like this occurred after the pipeline was restarted
mid-way through the processing. Impossible to reproduce the problem, never
been seen before and hopefully won't be again. We'll write that one down to
the "fat electron" theory. The affected frames have already been reprocessed
and reflagged to go.
7. VISTA update
Raw data has been received via Garching for up to and including June 24. It
includes some daytime calibration data with good linearity sequences from
which JRL has also derived some good bad pixel masks. JRL also exploited the
recent linearity data to investigate low light level linearity. Some
of this data was taken to explicitly probe the low-count regime and indeed
shows non-linearities exist at the 1 - 3% level even below 1000 ADUs.
STH commented he needs to revisit the linearity check in the catalogue
photometry -v- 2MASS to ensure that the non-linearity has indeed been
corrected for. He also needs to revisit the colour equations now that we
have about twice as much data as last time we computed them. <<<<
As noted earlier, MJI will supplement this with the sky brightness estimates
EGS independently discovered that the standard for describing WCS keywords
in FITS binary table catalogues is different to what we have been using so
far. Some of our post-processing software has not been adapted to the
different standard yet.
JRL, as always, is taking a broom to the pipeline and tidying things up and
fixing things faster than he breaks them. A pernicious buggette with
the pipeline astrometry which generally manifested itself in crowded fields
may have been tracked down and fixed, much to the post-pipeline astrometry
fixer's relief. This may also have been the cause of some of the ESO
tickets about the stability of the photometric calibration results from
the summit and Garching pipelines. In contrast the Cambridge photometric
zeropoints on the summary pages look very well behaved.
JRL released via the VPO a new version of the summit pipeline to fix a couple
of problems but does not know if it is being used yet. He reported that
Sandra Castro, who was dealing with the VISTA pipeline has left ESO and is
being replaced by Lander de Bilbao.
During processing recently apm29_a,b,c hung up a couple of times. It may
be a software issue and SC has now applied the most recent software patches
to see if that fixes the problem.
MJI noted that since the upgrade the remote temperature monitoring web
page for apm29 has flatlined. SC will take a look see why. <<<<
Remaining commissioning tasks include investigating the efficacy of deep
stacking on a 16hour set of J-band pawprint data and completing the loop <<<<
for the offsetting accuracy when/if we get more data. The other previously
outstanding issue relating to stability of the pin cushion distortion model,
including centre coordinates, was booted out the door last week.
Having been asked to look at science data, MJI examined the VISTA Y and H
tile OBs for the globular cluster M55 and produced an appealing colour
image from the tiles together with a plausible CMD from the individual
detector stacks. He noted that the passbands used had been cunningly
chosen to avoid all published theoretical isochrones so it was difficult
to analyse the rather nice M55 locus quantitatively.
Feeling upbeat he then tried to repeat the trick on test tile OB J,Ks data
taken of the Sculptor dwarf (to compare with CIRSI observations) at airmass
of ~1.5 and found that subtly varying seeing throughout the tile exposures
caused some interesting problems when using catalogues generated from the
resulting tiles. This was not completely unexpected since each sub-region
in the tile is made up of different partial exposures potentially leading to
nasty disjoint PSF variations across the tile. The effect was worse than
anticipated even using aperture photometry, which is not as susceptible to
systematics as PSF fitting. This highlights a generic problem in doing
accurate photometry on VISTA tiles and will in general require a
VISTA-specific solution. He then outlined a couple of possible solutions
neither of which is particularly elegant. This requires some further work.
MJI reported on the current grant status and noted the recent joint letter
that had been sent out by WFAU & CASU to alert users to the situation.
SC & MJI are looking at ways of adding new A/C capability to deal with the
proposed increased amount of Gaia hardware and also in the near future at
least to provide some redundancy in cooling capability. This posed the
ancillary problem of what to do with the large number of (mainly) ING
archive CD/DVD storage cabinets which would need to be moved out to make
room for new computing kit.
EGS add catalogue software and binary table conversion software to software
MJI release pages
MJI finish off technical paper updates
MJI write utility to convert binary tables with all corrections
MJI do the Petrosian fix
STH further compare ETC fact with fiction
STH update data storage web page particularly with respect to ING archive
MJI generate an overview summary plot of the sky brightness levels using
recent VISTA data and hand it over for the technical web pages
RWA generate LTO4 backups of VISTA raw data received to date
STH revisit average zero-points and colour equations for VISTA and check
non-linearity correction is working by cf. with 2MASS data
SC fix the flatlined temperature monitoring web page for apm29
EGS investigate stacking on the 16hour set of J-band pawprint data