You are here: Home / Documents / Minutes of CASU Meetings / minutes_091119.txt

Plain Text icon — Plain Text, 13 KB (14089 bytes)

File contents

Thursday 19th November 2009   APM Meeting tables    11:40-13:00

Present:        MJI, RWA, EGS, JRL, JPE(for item 7), MR, SC, STH, RGM 
Apologies:      NAW

1.  Actions from last meeting
2.  Comments on WFAU minutes
3.  Recent and upcoming meetings
4.  Data archives update     - AAT, ING, WFCAM, UKIRT
5.  Optical/NIR processing   - HAWKI, INT WFC, MegaCam, Subaru, VST .....
6.  WFCAM update
7.  VISTA update
8.  AOB 


1. Actions from the last meeting

MJI  stage I software release on hold due to other pressures - ongoing     <<<<

MJI  ditto technical paper - ongoing                                       <<<<

MJI  ditto utility to convert binary tables plus corrections - ongoing     <<<<

MJI  still comparing WFCAM dome flats with twilight sky flats. Low light   <<<<
     level flats have been taken for both WFCAM and VISTA and it will be
     interesting to see how the low light level linearity compares.  JRL
     is investigating this aspect                                          <<<<

SC   two high temperature monitoring alarms have been set up in the A/C rooms.
     They will alert security if either room temperature goes above 30C; who 
     will in turn contact Paul Aslin or MJI as necessary. The temperature
     sensors on some of the computer kit in those rooms can also be monitored
     remotely via webpages to relieve, or possibly cause, paranoia
EGS  made a night of VISTA data available for WFAU slurping purposes

MJI  contacted WFAU about the mosaic tile file formats and provided a current
     example for scrutiny and further possible iteration on content

MJI  duly wrote and sent in a report to the UKIRT Board two weeks ago. It is 
     available on the usual internal web pages.
JRL  has reduced the August data testing the stability of the proposed new 
     VISTA dome light source with promising results.  All we need to do now
     is get it wired up properly and take some linearity sequences.

MJI  investigated the wavelength dependence of the astrometric distortion
     and could find no measureable effect.  In doing this he has updated
     the distortion model coefficients using all the data from 20090927 to 
     20091027.  Residual systematics are now less than 5 mas over the
     entire array (see later item for more detail)

JPE  sent the outstanding NB filter transmission curves to MJI via EXCEL 
     spread sheets

EGS  investigated the stability of repetition of filter position by examining
     sky frames and confirmed there was a brief spell at the beginning of the
     PR run where this was an issue.  No further occurrences of the filter 
     wheel position being lost in the middle of the night have occurred yet.
     The SV data is OK possibly because re-datuming was carried out at regular

MJI  measured the VISTA SIQ directly from oversampled PSFs constructed from 
     some of the best seeing data on the SMC (20091005). The profiles meet 
     the 50% SIQ criterion but not the 80% one which assumed Gaussian profiles.
     Real PSFs are closer to Moffat.  In this case one with beta=2.5 fits
     rather well, as it also does for WFCAM data taken in good seeing.  With a
     realistic profile shape the 80% SIQ criterion would be met.

JPE  said that trial OBs for some of the PS's had already been taken during
     the SV run

JRL  noted that he had not yet arranged for a new anti-surge unit for his 
     office nor bought any beer to propel it - ongoing                     <<<<

SC   is still figuring out a practical back-up stategy for some of the 
     machines such as apm25, 26, 28 etc.  - ongoing                        <<<<

2. Comments on the WFAU minutes

It was noted that WFAU had asked the TERAPIX team to incorporate Rice 
compression in SExtractor but were politely fobbed off. 

3. Recent and upcoming meetings

There was a VISTA IOT on 3rd November including some discussion of the recent 
SV run and what to do next in terms of comparing results.  The last SV night
was 2nd November.  In addition to the main SV programmes some PS 2 hour trial
OBs were also completed (VVV, VMC, and a combined UltraVISTA/VIDEO OB).  Some 
of these 'extra' OBs used an SV project ID making identifying them slightly 
more challenging.  All of this data is still embargoed so apart from the
SV data not much soul searching on product data quality has yet taken place.

The SV meeting at ESO on 25/26 November was confirmed during the IOT and 
shortly thereafter a provisional agenda for this was drawn up and iterated

There have also been several Euclid and PLATO telecons recently to discuss
among other things data processing and the ground segment design for both 

4. Data archives update

JRL reiterated his promise to finish off pootling around with the UKIRT 
CASS raw data archive by this Xmas.

All the others, AAT, ING, and WFCAM are motoring along as usual.

Meanwhile, we now have enough VISTA data to need a raw data archive for
this to help navigate and JRL has set up a password-protected one for
internal use.

5. Optical /NIR processing

JRL said that he was doing some HawkI processing for RGM but not much else -
other than processing VISTA data, writing pipelines, updating ESO deliverables

MJI took 2 weeks to FTP over the latest batch of PAndAS MegaCam data for 09B
(a mere 250 GB from the CADC thereby demonstrating how good our raw WFCAM
transfers are) and has started processing it in the background.

6. WFCAM update

WFCAM processing is up to speed with processing commencing within a few
days of the data being taken.  [Note for JAC: if there are any special
requests to deal with data in a particular way the best way to notify
us is via a direct email to MJI and MR since we need them more or less
at the same time we get the data i.e. within 24 hours of the final bit
of data being taken.  Although we interrogate the nightly OMP entries we 
don't always read them.  It is often too late by the time we get the weekly 
summary since some of the data is already processed.  This has rarely been a 
problem but we noticed one example on the 20th Nov which we will follow up 

MJI reported that there have been some intermittent problems with camera #2 
which occassionally went AWOL for large chunks of a night.  These (few)
nights are currently flagged as 'unprocessed' for the time being and will
be addressed during a lull in other work.

The nebulous GPS data needed for DR6 was reprocessed and WFAU (and Phil) 
have been informed.  This just leaves the UDS J,H in the queue.

7. VISTA update

Data is (almost) regularly arriving about 2 weeks after the observations.
Ingest and verification is usually relatively routine and we currently
have data up to 4th November on disk.  This has all been processed to
the pawprint level.  Tiling tests are still ongoing using the SV data 
as a scapegoat for different strategies.

MJI has investigated wavelength dependence of the distortion model for the 
VISTA focal plane using a month of stacked 2MASS residual data from 27th 
September to 27th October. There are around 1 million stellar measurements
to make use of in most passbands albeit taken at a range of sky PAs.  Detailed
tests projecting the residuals onto the detector grid reveal no measureable
wavelength dependence of the distortion model at levels of < 5 mas.  Likewise 
projecting residuals onto a sky grid (i.e. not rotating back to a fixed PA) 
also show similar results.  During these tests the distortion coefficient
values were also investigated and were updated (slightly) before producing 
the final stacked residual maps (PV2_3  42 -> 44 ; PV2_5  -10000 -> -10300).

In practice this means that the systematic astrometric error over the whole 
detector array is better than 5 mas and hence is already about a factor 4
better than the goal.  It also implies that doing accurate proper motion,
and even parallax, work with VISTA over large areas is feasible and could
lead to some interesting nearby cluster/stellar population projects.

JRL added a few more words on the VISTA raw data archive he has created.  This
is searchable by night, run number, filter, sky position etc.. and will remain 
strictly in house for the moment. Keen as always, STH asked how he could 
access it and was told that he would need a username and password, though
he might want to consider the processed data equivalent that EGS is 
constructing.  EGS noted that he was working on a graphical interface for this
which he thinks will be ready in another week.

The SV people at ESO sent back favourable comments on the data processing web 
pages and a few suggestions for discussion at the upcoming SV meeting at ESO.
Agenda items will include technical processing details and a preliminary 
discussion of the science products for the NGC 253 and Orion surveys.  EGS and 
STH will handle the latter and MJI the former.  

MJI and STH are going to revisit the subject of the illumination corrections,
colour equations and assorted other generic odds and ends in time for the 
meeting. JPE suggested that since the Z filters were made in two different 
batches then we may need two different colour equations. STH will tell us on
Monday if he can see this (he did and the answer was thankfully no).

Meanwhile MJI and EGS computed the relative QE of all the detectors using the
internal gains from the flats and the measured gains (e-/ADU) of each
detector.  The results have been posted on the VISTA technical web pages
and will be used to update the overall throughput computations in the ETC. <<<<

MJI noted that ROE has the first week of SV data and was actioned to check 
that the rest was ready to send on.                                        <<<<

JRL noted that there were several interesting gotchas with the raw data WCS 
and array position angle which had involved a workaround in the processing 
pipeline.  The PA of the array was not always accurate in the header (at the 
level of ~1/100th of a degree) and the corresponding rotation matrix in the 
FITS header was similarly off.  This meant that it had been diffcult to use 
the computed jitter offsets correctly leading to smearing of images by a few
pixels near the edges of the field at large -ve declinations.  The pipeline 
workaround now deals with this issue but it would be good to get to the bottom 
of why it is occuring in the acquisition system. Steven Beard and Dave Terrett 
are investigating.

JRL noted that many observations contained interrupted OBs, whereby the 
subsequent OB was restarted from where the previous one failed.  This was
initially a surprise, but then Murphy's law was applied, aka if it is at all
possible to restart an OB half-way through it will indeed happen. Its a pity
the FITS header information doesn't reflect Mr. Murphy's flexibility.  So, as 
usual, the nightly processing preview script has acquired a few more if's 
butt's and elsie's.

JPE updated us briefly on the latest installment in the PR date and handover
saga.  MJI said that we had done all we can for the PR colour pictures and
have generated far more than are going to be used in the first PR release. 
Some of these images are quite stunning particularly those of M42 and the
Flame nebula in Orion.

JRL said he is preparing a new release of the pipeline and an update of the
DRLD over the next few days, hopefully in plenty of time for any issues raised
to be discussed during the SV meeting at ESO.                              <<<<

[This subsequently reminded us that the Calibration Plan document also needs
updating with some urgency.]                                               <<<<

In answer to a question from RGM, JPE and MJI said a meeting of the survey 
PIs would be arranged in Cambridge sometime in the New Year to discuss the 
processing of, and results from, dry run data. After the PR date and acceptance
JPE will talk to ESO about the release of data already taken. When all this has
finally happened we will have a better idea of when the Cambridge meeting
with the PIs is likely to occur. 

8. Any other business

SC raised the spectre of increased power consumption and its effect on the 
University electricity bill. As an example, the APM building has doubled its 
power consumption in the last two years due mainly to the installation of a 
lot of computer kit and extra air conditioning systems.  MJI has always been 
strongly in favour of using natural cooling where possible and wanted to 
progress this now that the University has finally woken up to the huge 
increase in electricity bills.

[Corollary: it turns out we are probably going to be asked to pay the
difference in the CASU electricity bill from two years ago out of our grants
income making it even more urgent to come up with a greener solution !]

The more important task of arranging the CASU Christmas Party fell to STH, 
SC and DWE who will form a sub-committee to investigate dates and organise
a smidge of food and drink.  Everyone was asked to make sure that their 
online calendar files were up to date for December to help with this. JRL 
said he would only come if there was a chocolate fountain like 2 years back
(yuk !).

Continuing actions

EGS  add catalogue software and binary table conversion software to 
MJI  software release pages  

MJI  finish off technical paper updates

MJI  write utility to convert binary tables with all corrections

MJI  compare WFCAM dome flats with twilight sky flats

JRL  quit dithering, buy the beer, and enjoy the anti-surge

SC   is still figuring out a practical back-up stategy for some of the 
     machines such as apm25, 26, 28 etc...

New actions

JRL  compare WFCAM and VISTA low light level linearity measures

MJI  update the overall througput figure in ETC using average detector QEs

MJI  check whether rest of SV data is ready to go to WFAU

JRL  release next update of pipeline and DRLD

JRL  update the Calibration Plan document

SC   decide date for Yuletide celebration.