You are here: Home / Documents / Minutes of CASU Meetings / minutes_100532.txt


Plain Text icon minutes_100532.txt — Plain Text, 11 KB (12227 bytes)

File contents

Wednesday 1st June 2010   APM Meeting tables    11:30 - 13:00

Present:     MJI, EGS, JRL, RGM, SC, JPE
Apologies:   NAW, MR, JCR, STH


1.  Actions from last meeting
2.  Comments on WFAU minutes
3.  Recent & upcoming meets
4.  Data archives update     - AAT, ING, WFCAM, UKIRT, VISTA
5.  Optical/NIR processing   - HAWKI, INT WFC, MegaCam, Subaru, VST
6.  WFCAM update
7.  VISTA update
8.  AOB


1. Actions from the last meeting

EGS  stage I software release partially done but now awaiting JRL to
JRL  to investigate building a release distrubtion more robustly           <<<<

STH  updating the Calibration Plan document is still ongoing               <<<<

JPE  raised the issue of filter keywords for the not_the NB980/990 filters
     (975/985) at the VISTA IOT.  As expected just a single keyword in
     the PHU is present and don't hold your breath for propagation down
     the SHUs.  Gavin "remembered" which set are which - honest ! and 
     for the record 1-8 are NB975 and 9-16 are NB985.  JPE cunningly
     split the difference by labelling the whole lot NB980 in the only
     filter keyword value.  All we need to do is find a redshift 7 object
     to confirm which is which, on second thoughts better make that OII 
     or Halpha at lower redshift.

RGM  is still testing the catalogue conversion software to destruction     <<<<

EGS  text for our own versions of ESO-released PR images on the CASU web
ALL  pages needed. EGS still needs to put in links/cross references to 
     the ESO versions all ongoing                                          <<<<

EGS  finished off the SV reprocessing and delivered the goods in time
STH  for the Orion meeting.

JPE  raised the VISTA CVRAL issues at the IOT telecon and JRL sent in a 
     minimum opus re: his thoughts on the subject.

MJI  constructed and sent off the UKIRT Board report.  He will attend the
     meeting next week.

JRL  fixed it -> apm29_a disk space and minor IAMFUD FUDs in the catalogues

2. WFAU minutes

Not many, other than RGM asked what the schedule for the next UKIDSS 
release was.

3. Recent and Upcoming Meetings

Way too many:

EGS and JRL partook of a VVV group meeting via telecon.  Most things
seem fine.  There were a few grumbles about missing files and supposedly
the CASU server was unreliable at times, neither of which we can find any
clear evidence for.  MJI was entertained by the contortions (seven separate 
steps) that one of the VVV members was going through to generate (nice 
looking) tiles.  It would be interesting to do a comparison against our 
single step procedure tiles; EGS will contact them and make it so.         <<<<

VISTA IOT telecons: they do drag on a bit.  At 3+ hours the last one has 
to be one of the biggest wastes of time on the planet.  There are a few
perennial problems which no matter how much chin wagging always seem to
come back   The CRVAL problem + associated other features; and the
persistent use of sparse sampled non-deblended cataloguing to estimate 
ellipticity (and seeing) QC parameters and infer misleading trends.
The biennial detector zero-point issue raised at previous meetings also
seems to be a facet of the QC pipelines being used rather than a genuine
problem.  MJI produced and sent round a bunch of histograms from the 
science pipelines from April data that look fine.  Persistence is futile 
cropped up again for discussion but the conclusion (pushed hard by CASU) 
was that if a sensible observing sequence is used persistence is not a 
problem and does not need specifically correcting for since any remnant 
images dither stack out.  Finally, JPE undertook to politely inquire if 
we can have more focussed discussions at future IOTs.                      <<<<
NAW, STH and MJI (intermittently) took part in the PLATO Data Centre
meeting at the IoA 27/28th May. Good progress was made in defining the 
nature of the processing tasks and organisational structure. IoA leads the 
Exoplanet Analysis System development. Fingers are crossed for next hurdle.

Meanwhile RGM attended the EUCLID consortium meeting in Paris on 25/26th May. 
There was discussion about Ground Segment Data Processing which includes the 
processing of imaging data from the Optical imager and NIR imagers and 
spectra from the NIR slitless spectrograph. Mellier from IAP and Pasian 
from INAF led the discussions which were mainly high level. There will be 
further discussion with ESA about the division of responsibilities and 
deliverables between ESA and the Euclid Consortium. The next stage has a 
deadline in early September, 2010.

The second CASU release of the SV data products based on v0.8 processing
occured a few weeks ago. EGS and STH attended an SV Orion meeting at ESO to 
explain the processing, discuss v0.8 issues and even see what sort of 
science is being done.  The SV NGC253 team maintained a respectful silence
other than grumbling about persistence problems in their SV data, cunningly
sequenced such that even in deep stacks all the remnant images line up
perfectly.  EGS and STH produced a set of notes made during the meeting
that are included as an Appendix (for internal access only).

There was an IoA review of computer needs a couple of weeks ago at which
MJI and SC made sure the WFCAM and VISTA computer requirements were safely
accounted for.

MJI is off to ESO again this week to further the progress, or otherwise, 
of the ongoing negotiation with ESO re:in-kind payments.  Colin Vincent,
Bob Mann and Keith Noddle are the other UK reps.  Michele Peron,
Fernando Comeron, Pascal Ballaster, Andreas Kaufer and Martino Romaniello 
turned out for ESO.

MJI pointed out the upcoming meeting on Galactic near-IR survey science 
being organised by Melvin Hoare, in Leeds on Thursday 8th July 11am-6pm
"UKIDSS and VISTA Galactic surveys meeting".  JPE and he are going.

4. Data Archive

WFCAM, ING, AAO, UKIRT, VISTA  all up to date and trucking .......

5. Optical / NIR Processing

Not a lot going on other than a pile of Subaru PF Hamamatsu detector data 
being processed.

VST - there is an upcoming "Review of VST Public Surveys" September 28-29
2010 in Garching.  Though the letter quite carefully states that the
"ESO STC has asked us to revisit the scientific goals of the approved public 
 surveys with VST" rather than review.  The cast list includes the public
survey and GTO PI's, the original PSP, and the corresponding "experts" 
of STC (Y. Mellier and R. Nichol) hmmm......  MJI and EGS will probably
partake for CASU to support Janet and Tom's endeavours.

6. WFCAM Update

Processing and raw data ingest is up to date.  WFCAM went on vacation
from 24th May, back on the 11th June we believe, for a well-earned spot of
surfing while its host is having its mirror re-aluminised.  The May data is
being checked and flagged off even as we speak.

No news from enterprising PhD student after he got all the raw WFCAM data 
for CMP/3.  

The last of the parallax measures for T720 were ticked off and made available
to the eager team.  

CASU are glad to see JAC humour persists with the new "FOC_TOFF" keyword.

MJI noted the new arrangements for UKIRT with Tom Kerr becoming Head of 
UKIRT Operations and that Chris Davis will remain our point of contact for
WFCAM matters.

The reprocessing queue has grown with Phil Lucas' usual request for nebulous
things in selected regions.  We will add that to the small pile of incumbents.

7. VISTA Update

Ingestion and processing data to pawprint level (version 0.8) up to date and 
done and dusted up to 23rd April VISTA intervention.  A modest amount of
v0.9 has been included due to the need to reprocess parts of a few selected 
October and November nights where either the sky subtraction did not get
done or detectors #5 and #6 were swapped (19th November) and needed 
unswapping prior to processing.

As noted earlier the v0.8 SV reprocessing was finished on time and the
science products distributed to the SV teams.  This includes all the
standard flat files including tiles and is intended as a thorough external
shakedown of the system before releasing v1.0 of everything shortly.

Version1.0 reprocessing has begun with the 1st month of data from 15th October
in progress.  This includes, as part of a throrough shakedown test, the 
automatic production of tiled data products.  After yet more discussion 
on this we have opted for a slight sea change again: the catalogues will
be produced from the nebulosity filtered tiles but the images will be
the unfiltered tiles - reasoning behind this on application.

RGM pointed out that there was a web page on the ESO website that
specified required keywords for processed imaging data products.

e.g.  Photometric magnitude system    (AB, VEGA)      PHOTSYS

Whilst these are Survey PI requirements and can be added later or existing 
keywords values copied, he thought it might be more convenient if they were 
included in the CASU products. 

The consensus was that after these have stabilised, or indeed been added to,
we should revisit this issue since it is also not yet clear how ESO will 
deal with catalogue data product FITS binary tables.  Though having said 
that, interestingly, the DICB, who oversee FITS issues at ESO have not 
complained about either image or catalogue keywords produced by the pipeline 
(other than some totally trivial things - see previous minutes).

JRL tried out the algorithm that uses externally generated sky masks on the
Fynbo January NB118 data with improved sky subtraction results.  
Unfortunately this whole sky masking issue involves knowing an accurate
WCS much earlier in the pipeline and has required a rewrite (gulp!) of
the dither stacking front-end.  On the +ve side this also helps with the 
CRVAL/rotation matrix problem.  Let's hope its all +ve since there are some 
potential gotchas with sharp teeth lying deep in left-field.

The short wave suicidal mania of #16 has been assessed further in making 
tiles for the SV programmes.  Currently we have not altered the confidence 
maps prior to tiling since in practice we found it better so far to deal with 
the problem at either tile catalogue generation or interpretation phase.
Tests on this are ongoing with more datasets.

JPE noted that despite ESO being happy with the delivered VISTA pipelines 
they have still not been formally aacepted.  He will contact Michele and 
Fernando and attempt to get this signed off.                               <<<<

The data transfer test is still on the agenda and will likely involve the
transfer of a week of compressed data from Garching to Cambridge (in addition
to transfer tests from Paranal to Garching).  Oddly, ESO seem pleasantly
surprised by the 4x Rice compression obtained.

JPE also raised the need to reprocess the PR data from 28th September to 
14th October since assorted bods have asked for access to the FITS images.
This is not completely straightforward since various bodges with sky
subtraction were needed due to the interesting way observations proceeded.

8. AOB

Not much for a change other than MJI finally got his wish with some fitted
external fans for the main computer room to supplement/replace some of
the unnecessary cooling - energy consumption spies please note.

EGS sent an email around earlier about the recent Plone website upgrade
and asked for any faux pas to be reported.  JPE was first on the draw,
long may it be so.

Continuing Actions:

EGS  - finish adding catalogue software to release pages

STH  - finish off last bits of calibration plan document update

RGM  - test catalogue conversion software

ALL  - contribute text to go with CASU versions of VISTA PR images
EGS    add links to ESO versions

New Actions:

EGS  - contact VVV team and get an example tile for comparison

JPE  - politely inquire about more focussed discussions at future IOTs

JPE  - contact ESO re: signing off pipeline deliverables