You are here: Home / Documents / Minutes of CASU Meetings / minutes_031024.txt

Plain Text icon — Plain Text, 13 KB (14237 bytes)

File contents

24 October 2003 

Present:   MJI, JRL, PSB, DWE, STH

Agenda
------

1.   Actions from last meeting
2.   Comments on WFAU minutes
3.   Report on AGOC visit
4.   Report on ADASS
5.   Comments on V1.0 WFCAM FITS header doc
6.   Comments on SMB's Observation Software Detailed Design document
7.   UKIDSS CSS and CFT positions and pipeline acceptance
8.   AOB

Minutes
-------

1.   Actions from last meeting

STH  has investigated Sandy Legget's data. Although good quality, it is
     probably not enough to do extensive extinction investigations on since
     it only covers 3 hours of one night (0.5 in airmass). Sandy has pointed
     STH to an archive containing photometry files (reduced data) for every
     standard (also humidity etc. data). The investigation of this data will
     be done by the UKIDSS meeting (21 Nov).                               <<<<
     
STH  no progress has been made with the extension of the photometric
     standards to the south for VISTA, ongoing                             <<<<

STH  has almost finished sorting out document numbers for our released
DWE  documents. It was decided that we would follow VISTA's lead 
     regarding version numbering, with the version number separate 
     to the document number.  Given the variety of methods (html, latex
     word) no formal template will be used, but each document will be 
     headed by Title, Doc. No., Author(s), Date (centred).  For now
     no logos will be added other than possibly the CASU one.  STH has
     asked Richard Sword to slightly modify the current CASU logo since     
     on b/w printers it currently reads "ASU"   :-)
     Finish off numbering and email authors with the appropriate numbers.  <<<<

PSB  the first draft version of the DID is done, and a copy was sent to 
MJI  Steven Beard

MJI  has contacted Mark Casali regarding the QE variations of the 4 
     science-grade detectors, and is awaiting a reply, ongoing             <<<<

MJI  needs to contact JPE regarding a list of the sections missing from the 
     User Requirements Document and Calibration Plan document.             <<<<

MJI  have sorted out their visit to the Astrowise group in Groningen. This
JRL  will be during the first week of December (3/4?). It was noted that
     members of WFAU would also be going.

JRL  has discussed the summit pipeline software plans with JAC staff at
     ADASS and resolved any outstanding issues.

JRL  has informed MJI of the things that need updating in the milestones 
     document.  The on-line version is now up to date.

MJI  has not contacted SJW regarding adding MSB design to the agenda at the 
     forthcoming UKIDSS meeting because it is implicit in the memo that
     AL sent round (item 7.) and there is no draft agenda yet anyway.

JRL  is in the process of reducing the FIRES data.  This will be useful
     for several purposes: supplying test stacking data for the Subaru 
     group; demonstrating a different perspective for NIR data reduction
     cf. UFTI; and can also be used for "manual pipeline" comparisons.
     The results of this are need for the upcoming UKIDSS meeting.         <<<<

PSB  met with JMS to discuss camera templates, nomenclature and other issues 
     and had a constructive meeting.

PSB  met up with ESO folk at ADASS (see later).
JRL

MJI  checked the on-sky characterization document - all ok for now


2.   Comments on WFAU minutes

It was noted that WFAU were investigating GridFTP.

MJI noted that the summary of the AGOC meeting given in the WFAU minutes
was very accurate.


3.   Report on AGOC visit

MJI reported that the pipeline progress, WSA updates and SSA demo (prototype
UKIDSS archive interface) were well received at the AGOC meeting.  The panel
seemed to be satisfied that good progress was being made on these aspects 
of the programme and that both the CASU and WFAU parts of the project were 
generally on track.  This appears to be the last meeting of this special
advisory group and it is likely that future oversight will come from the
full Grid Steering Committee (GSC).

The next meeting of the GSC is likley to be in January where, among other
things, an "enhanced" VEGA bid will be (re)considered.   A meeting of
the extended VEGA consortium will take place at the IoA on Tuesday 28th 
October.

One clear message from this (and the previous) AGOC meeting was that a
User Committee should be set up for VDFS (cf. UKIDSS).  This should 
provide scientific and technical oversight and act as an interface to 
the user community.  JPE is looking into this.

Simple summary/overviews such as single page GANTT progress charts were 
requested by the committee.  STH has produced a summary GANTT chart
and updated the more detailed GANTT spreadsheet at the same time.  These
have been forwarded to JPE and placed (more prominently) on the web pages.

The committee also suggested revisiting the issues of risk assessment and
mitigation strategies in the light of WFCAM delays, and to look at 
prioritising VDFS deliverables for possible descoping in case of future 
funding or timeline constraints.


4.   Report on ADASS

PSB reported the following on his and JRL's ADASS trip:

"Jim 'n' Pete met with Michele Peron, Benoit Pirenne and Adam Dobrzycki.

We discussed pipeline infrastructure with Michele (as head of software
development) and she reported that version 1.0 of the Common-Pipeline 
Library (CPL) was due to be issued in the New Year.  We agreed that it 
would be beneficial to visit ESO and discuss CPL and the VLT pipeline 
infrastructure, at a date to be determined, but before Christmas.  ESO 
would give us the release of CPL at that time.

Benoit is concerned with archiving WFCAM data, and Adam is chairman of
the Data Interface Control Board (DICB).  It was agreed informally that

o WFCAM should add the MJD-OBS keyword at source

o CASU will, en passant, add a number of ESO-friendly archive keywords 
  during assembly and transfer of the raw archive data to ESO.  (For the
  record these will also be added as comment statements in the raw online
  MEFs)

o ESO would provide a module, in C, to use networking protocol (HTTP
  probably) which would run in Cambridge to transfer WFCAM data to ESO
  via the internet

o ESO will consider urgently and sympathetically accepting lossless
  FITS/TILE/Rice compressed data."

PSB needs to arrange this ESO visit with Malcolm Stewart and Michele Peron.
This would probably be 13/14 November.                                     <<<<


5.   Comments on V1.0 WFCAM FITS header doc

Various comments had been sent in by email:  a couple of typos 
(RA---ZN -> RA---ZPN and PV3_3 -> PV2_3) and a few misunderstandings 
about the structure of the WCS.

MJI will collate these comments, add his own and forward them on. 
When this has been done the FITS header document can be signed off, as 
can the JAC CASU ICD.                                                      <<<<


6.   Comments on SMB's Observation Software Detailed Design document

It was noted that PSB will go to the Camera FDR during the week of
December 15th.

Various points were discussed.  Since many of these were technical
detail it was agreed to email them to MJI (by the end of this afternoon) 
rather than record them in the minutes.  These points would then be 
assembled into a coherent whole, sent off to SMB and copied to the rest 
of the group, by the end of the day.                                       <<<<


7.   UKIDSS CSS and CFT positions and pipeline acceptance

As a general comment, CASU agreed with the AGOC that there is an urgent 
need to set up a VDFS user committee to provide scientific and technical 
oversight and act as an interface to the user community.  Both of these 
consortium positions should come within the remit of that committee.
The CSS should be a member of the committee and the CFT should report to
it.

CSS task list: 

(1) It was unclear what the purpose of a review of these pipelines and
archive procedures might be.  The main comment was that such a review 
seemed far too late in the project.

(2) The relevance of UFTI data to WFCAM and VISTA was questioned. The 
field of view is much smaller, the camera controller is different, and 
there are known problems with the stability of the results from the 
instrument.  Practical strategies to deal with UFTI data are not 
necessarily optimal for dealing with WFCAM or VISTA data.

(3)-(4) Fine

(5) There was some general discussion on the desirability that MSBs 
should not have filter changes within them.  It was also noted that the 
optimum MSB strategy to adopt for each survey depends on the (unknown)
performance and stablity of the camera.

(6)-(15) Fine

CFT job description: 

It was thought that the job title "Consortium Food Taster" was not going 
to win any prizes and that an alternative title incorporating words 
like "data quality" or "science verification" would be more appropriate.

(1) Fine

(2) Some of the tasks eg. repeat data, seemed to overlap with what
the Survey Manager is supposed to do.

(3) Setting up the "sample science questions" seems like a task for the
heads of the various survey groups.

(4)-(9) Fine

(10) CASU are not happy with the CFT authorizing data releases to ESO.
This will introduce an extra layer of unnecessary bureaucracy into the 
system.  And anyway how does (10) differ from (11), if the ESO community 
have the same access rights as the UK community ?  However, this does
raise the question of who will have authority over data releases ?

(11) Regarding science verification the main question should be - are the
pipeline and archive meeting the science requirements ?

(12) Can this be partly equivalent to a user support role ?  It was felt
that the best use of this person would be to act in a support role for 
users and to help WFAU and CASU deal with queries from users.

The CFT ought to be involved in the fine tuning of the system while the 
on-sky characterization and commissioning is being carried out (without 
necessarily having to be on Hawaii).

The general consensus was that this sounded very much like a full-time post
rather than 50% as described. A question that needs to be answered is what
is the relationship between this post and the Survey Manager?

Pipeline acceptance:

There was then some discussion regarding the process of pipeline acceptance.
This was also in response to a separate email from SJW. 

The overall comment was that it was difficult to see how this could be 
completed before commissioning and instrument characterisation since no 
battle-plan survives contact with the enemy.  However, as part of a pipeline 
acceptance schedule, STH in discussions with SJW have suggested comparing 
our pipeline processing of (in order of preference) FIRES, UFTI and
CIRSI data with "manual" processing.  SJW has also suggested comparing EIS
processing of ???? with the CASU pipeline.

It was pointed out that JRL has already installed an UFTI version of the 
summit pipeline at JAC running under ORAC-DR and successfully compared the
CASU pipeline output with the normal JAC ORAC-DR reduction.

Further UFTI comparisons will be undertaken using the CASU pipeline in 
Cambridge to process the 6 sets of UFTI service data obtained by SJW.
STH said that he would find the 6 sets of service data and pass it over 
to JRL.                                                                    <<<<

The FIRES data is in the process of being reduced - see earlier action.

A lot of CIRSI data has already been processed.  STH will ask SJW if he 
wants to pursue this aspect of the comparisons further.                    <<<<

It was decided that there was no point in CASU processing further CIRSI 
data unless there was a compelleing science reason for so doing.

Between them STH and JRL need to organise the FIRES and UFTI reductions 
by the November UKIDSS meeting.                                            <<<<

We all need to think about this issue further. How should the process of
pipeline acceptance be carried out?                                        <<<<


8.   AOB

MJI asked STH to look into the spatial systematics in UFTI data using 
Sandy Legget's photometry data.                                            <<<<

MJI noted that there have been a few top level alterations to the 
WFCAM/VISTA VDFS web pages to emphasise the VDFS aspect more.  These 
pages can now reached from either the current /~wfcam or the new 
http://www.ast.cam.ac.uk/vdfs/ link.  STH reminded MJI that the CASU 
entry web page has been updated and that there is a new description
of the CASU activities for the IoA's graduate prospectus.

PSB pointed out that with the upturn in VISTA activity we will have to be
more precise in referring to activities in the minutes to avoid possible
confusion.


Continuing Actions
------------------

STH  to finish investigating the supplied UFTI photometric standards files 
     by the UKIDSS meeting (21 Nov)

STH  extend the number of fields that he has established for WFCAM to the
     south for use by VISTA.  This is now a Q4 task. 
	  
STH  finish off numbering of documents and email authors with the appropriate 
DWE  information
     
MJI  acquire details about the QE variations of the 4 science-grade detectors

MJI  contact JPE regarding a list of the sections missing from the 
     User Requirements Document and Calibration Plan document.

JRL  finish reducing FIRES data in time for November 21st UKIDSS meeting


New Actions
-----------

PSB  arrange ESO visit with Malcolm Stewart and Michele Peron

ALL  email comments concerning V1.0 FITS header document to MJI (by
     end of this afternoon)

MJI  collate these comments and send them off by end of today

ALL  email comments to MJI regarding Steven Beard's document (by the end of 
     this afternoon)

MJI  collate these comments and send them off by end of today

STH  acquire (find?) the 6 sets of UFTI service data and pass it on to JRL
JRL  for reduction in time for the UKIDSS meeting

STH  to ask SJW if he wants to pursue the issue of CIRSI comparisons

ALL  think about how the process of pipeline acceptance should be organised

STH  look at the spatial systematics in the UFTI standards data