minutes_100532.txt
minutes_100532.txt
—
Plain Text,
11 KB (12227 bytes)
File contents
Wednesday 1st June 2010 APM Meeting tables 11:30 - 13:00
Present: MJI, EGS, JRL, RGM, SC, JPE
Apologies: NAW, MR, JCR, STH
Agenda
=======
1. Actions from last meeting
2. Comments on WFAU minutes
3. Recent & upcoming meets
4. Data archives update - AAT, ING, WFCAM, UKIRT, VISTA
5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru, VST
6. WFCAM update
7. VISTA update
8. AOB
Minutes
========
1. Actions from the last meeting
--------------------------------
EGS stage I software release partially done but now awaiting JRL to
JRL to investigate building a release distrubtion more robustly <<<<
STH updating the Calibration Plan document is still ongoing <<<<
JPE raised the issue of filter keywords for the not_the NB980/990 filters
(975/985) at the VISTA IOT. As expected just a single keyword in
the PHU is present and don't hold your breath for propagation down
the SHUs. Gavin "remembered" which set are which - honest ! and
for the record 1-8 are NB975 and 9-16 are NB985. JPE cunningly
split the difference by labelling the whole lot NB980 in the only
filter keyword value. All we need to do is find a redshift 7 object
to confirm which is which, on second thoughts better make that OII
or Halpha at lower redshift.
RGM is still testing the catalogue conversion software to destruction <<<<
EGS text for our own versions of ESO-released PR images on the CASU web
ALL pages needed. EGS still needs to put in links/cross references to
the ESO versions all ongoing <<<<
EGS finished off the SV reprocessing and delivered the goods in time
STH for the Orion meeting.
MJI
JPE raised the VISTA CVRAL issues at the IOT telecon and JRL sent in a
minimum opus re: his thoughts on the subject.
MJI constructed and sent off the UKIRT Board report. He will attend the
meeting next week.
JRL fixed it -> apm29_a disk space and minor IAMFUD FUDs in the catalogues
2. WFAU minutes
----------------
Not many, other than RGM asked what the schedule for the next UKIDSS
release was.
3. Recent and Upcoming Meetings
-------------------------------
Way too many:
EGS and JRL partook of a VVV group meeting via telecon. Most things
seem fine. There were a few grumbles about missing files and supposedly
the CASU server was unreliable at times, neither of which we can find any
clear evidence for. MJI was entertained by the contortions (seven separate
steps) that one of the VVV members was going through to generate (nice
looking) tiles. It would be interesting to do a comparison against our
single step procedure tiles; EGS will contact them and make it so. <<<<
VISTA IOT telecons: they do drag on a bit. At 3+ hours the last one has
to be one of the biggest wastes of time on the planet. There are a few
perennial problems which no matter how much chin wagging always seem to
come back The CRVAL problem + associated other features; and the
persistent use of sparse sampled non-deblended cataloguing to estimate
ellipticity (and seeing) QC parameters and infer misleading trends.
The biennial detector zero-point issue raised at previous meetings also
seems to be a facet of the QC pipelines being used rather than a genuine
problem. MJI produced and sent round a bunch of histograms from the
science pipelines from April data that look fine. Persistence is futile
cropped up again for discussion but the conclusion (pushed hard by CASU)
was that if a sensible observing sequence is used persistence is not a
problem and does not need specifically correcting for since any remnant
images dither stack out. Finally, JPE undertook to politely inquire if
we can have more focussed discussions at future IOTs. <<<<
NAW, STH and MJI (intermittently) took part in the PLATO Data Centre
meeting at the IoA 27/28th May. Good progress was made in defining the
nature of the processing tasks and organisational structure. IoA leads the
Exoplanet Analysis System development. Fingers are crossed for next hurdle.
Meanwhile RGM attended the EUCLID consortium meeting in Paris on 25/26th May.
There was discussion about Ground Segment Data Processing which includes the
processing of imaging data from the Optical imager and NIR imagers and
spectra from the NIR slitless spectrograph. Mellier from IAP and Pasian
from INAF led the discussions which were mainly high level. There will be
further discussion with ESA about the division of responsibilities and
deliverables between ESA and the Euclid Consortium. The next stage has a
deadline in early September, 2010.
The second CASU release of the SV data products based on v0.8 processing
occured a few weeks ago. EGS and STH attended an SV Orion meeting at ESO to
explain the processing, discuss v0.8 issues and even see what sort of
science is being done. The SV NGC253 team maintained a respectful silence
other than grumbling about persistence problems in their SV data, cunningly
sequenced such that even in deep stacks all the remnant images line up
perfectly. EGS and STH produced a set of notes made during the meeting
that are included as an Appendix (for internal access only).
There was an IoA review of computer needs a couple of weeks ago at which
MJI and SC made sure the WFCAM and VISTA computer requirements were safely
accounted for.
MJI is off to ESO again this week to further the progress, or otherwise,
of the ongoing negotiation with ESO re:in-kind payments. Colin Vincent,
Bob Mann and Keith Noddle are the other UK reps. Michele Peron,
Fernando Comeron, Pascal Ballaster, Andreas Kaufer and Martino Romaniello
turned out for ESO.
MJI pointed out the upcoming meeting on Galactic near-IR survey science
being organised by Melvin Hoare, in Leeds on Thursday 8th July 11am-6pm
"UKIDSS and VISTA Galactic surveys meeting". JPE and he are going.
4. Data Archive
---------------
WFCAM, ING, AAO, UKIRT, VISTA all up to date and trucking .......
5. Optical / NIR Processing
---------------------------
Not a lot going on other than a pile of Subaru PF Hamamatsu detector data
being processed.
VST - there is an upcoming "Review of VST Public Surveys" September 28-29
2010 in Garching. Though the letter quite carefully states that the
"ESO STC has asked us to revisit the scientific goals of the approved public
surveys with VST" rather than review. The cast list includes the public
survey and GTO PI's, the original PSP, and the corresponding "experts"
of STC (Y. Mellier and R. Nichol) hmmm...... MJI and EGS will probably
partake for CASU to support Janet and Tom's endeavours.
6. WFCAM Update
---------------
Processing and raw data ingest is up to date. WFCAM went on vacation
from 24th May, back on the 11th June we believe, for a well-earned spot of
surfing while its host is having its mirror re-aluminised. The May data is
being checked and flagged off even as we speak.
No news from enterprising PhD student after he got all the raw WFCAM data
for CMP/3.
The last of the parallax measures for T720 were ticked off and made available
to the eager team.
CASU are glad to see JAC humour persists with the new "FOC_TOFF" keyword.
MJI noted the new arrangements for UKIRT with Tom Kerr becoming Head of
UKIRT Operations and that Chris Davis will remain our point of contact for
WFCAM matters.
The reprocessing queue has grown with Phil Lucas' usual request for nebulous
things in selected regions. We will add that to the small pile of incumbents.
7. VISTA Update
---------------
Ingestion and processing data to pawprint level (version 0.8) up to date and
done and dusted up to 23rd April VISTA intervention. A modest amount of
v0.9 has been included due to the need to reprocess parts of a few selected
October and November nights where either the sky subtraction did not get
done or detectors #5 and #6 were swapped (19th November) and needed
unswapping prior to processing.
As noted earlier the v0.8 SV reprocessing was finished on time and the
science products distributed to the SV teams. This includes all the
standard flat files including tiles and is intended as a thorough external
shakedown of the system before releasing v1.0 of everything shortly.
Version1.0 reprocessing has begun with the 1st month of data from 15th October
in progress. This includes, as part of a throrough shakedown test, the
automatic production of tiled data products. After yet more discussion
on this we have opted for a slight sea change again: the catalogues will
be produced from the nebulosity filtered tiles but the images will be
the unfiltered tiles - reasoning behind this on application.
RGM pointed out that there was a web page on the ESO website that
specified required keywords for processed imaging data products.
http://archive.eso.org/cms/eso-data/data-submission/image-guide
e.g. Photometric magnitude system (AB, VEGA) PHOTSYS
Whilst these are Survey PI requirements and can be added later or existing
keywords values copied, he thought it might be more convenient if they were
included in the CASU products.
The consensus was that after these have stabilised, or indeed been added to,
we should revisit this issue since it is also not yet clear how ESO will
deal with catalogue data product FITS binary tables. Though having said
that, interestingly, the DICB, who oversee FITS issues at ESO have not
complained about either image or catalogue keywords produced by the pipeline
(other than some totally trivial things - see previous minutes).
JRL tried out the algorithm that uses externally generated sky masks on the
Fynbo January NB118 data with improved sky subtraction results.
Unfortunately this whole sky masking issue involves knowing an accurate
WCS much earlier in the pipeline and has required a rewrite (gulp!) of
the dither stacking front-end. On the +ve side this also helps with the
CRVAL/rotation matrix problem. Let's hope its all +ve since there are some
potential gotchas with sharp teeth lying deep in left-field.
The short wave suicidal mania of #16 has been assessed further in making
tiles for the SV programmes. Currently we have not altered the confidence
maps prior to tiling since in practice we found it better so far to deal with
the problem at either tile catalogue generation or interpretation phase.
Tests on this are ongoing with more datasets.
JPE noted that despite ESO being happy with the delivered VISTA pipelines
they have still not been formally aacepted. He will contact Michele and
Fernando and attempt to get this signed off. <<<<
The data transfer test is still on the agenda and will likely involve the
transfer of a week of compressed data from Garching to Cambridge (in addition
to transfer tests from Paranal to Garching). Oddly, ESO seem pleasantly
surprised by the 4x Rice compression obtained.
JPE also raised the need to reprocess the PR data from 28th September to
14th October since assorted bods have asked for access to the FITS images.
This is not completely straightforward since various bodges with sky
subtraction were needed due to the interesting way observations proceeded.
8. AOB
-------
Not much for a change other than MJI finally got his wish with some fitted
external fans for the main computer room to supplement/replace some of
the unnecessary cooling - energy consumption spies please note.
EGS sent an email around earlier about the recent Plone website upgrade
and asked for any faux pas to be reported. JPE was first on the draw,
long may it be so.
Continuing Actions:
===================
EGS - finish adding catalogue software to release pages
JRL
STH - finish off last bits of calibration plan document update
JRL
RGM - test catalogue conversion software
ALL - contribute text to go with CASU versions of VISTA PR images
EGS add links to ESO versions
New Actions:
============
EGS - contact VVV team and get an example tile for comparison
JPE - politely inquire about more focussed discussions at future IOTs
JPE - contact ESO re: signing off pipeline deliverables
