You are here: Home / Documents / Minutes of CASU Meetings / minutes_050916.txt

Plain Text icon — Plain Text, 12 KB (12650 bytes)

File contents

16th September 2005  Hoyle meeting room 14.00 - 16:00

Present:    MJI, PSB, STH, MR, JRL, RGM, WJS (RGM & WJS partially)
Apologies:  DWE, NAW, EGS

Agenda
------

1.  Actions from last meeting
2.  Comments on WFAU minutes
3.  Review of VDUC and VDMT meetings
4.  VDFS VISTA update
5.  Data archives update - UKIRT, WFCAM, AAO, ING
6.  WFCAM news and JAC telecons
7.  Upcoming UKIDSS SV meeting
8.  Optical processing update 
9.  AstroGrid liaison update
10. AOB - hardware
        - ADASS
        - PD33s

Minutes
-------

1.  Actions from last meeting

STH  has yet to trial test version of SADT, PSB said he would help out
PSB  by contacting Martin Folger re: the software and installing it 
     here - ongoing                                                        <<<<

JRL  has still not finished simple access scripts to mirrored OMP 
     database tables - ongoing                                             <<<<

ALL  assorted presentations were placed in the appropriate web pages

MR   is still writing up the flat field properties and sky frames 
     characteristics report - ongoing                                      <<<<

MR   has put a "known features" page on the WFCAM status pages             
     (note EGS to check that this is publically accessible since both
      MJI and RGM had not been able to access it remotely)                 <<<<

WJS  has checked the VISTA template definitions and pointed out to SMB
     a few areas where enhancements are desirable (eg. filter loop and
     different exposure times)

PSB  has contacted JPE about the status of the SADT with respect to the
     RIXs and is still awaiting a response

EGS  has written a test script to remove duplicate entries in the ING
     DVD and CD databases, he and RWA will test this out shortly

MJI  produced a PDF version of the catalogue parameter document and
     DWE linked it off the web pages            

MJI  produced a draft UK VISTA deliverables document and distributed it
STH  for comment (see also item 3.)

ALL  examine new catalogues for bugs, errors on the errors and so on.
     Several tests were made and a few interesting features noted that
     lead to several minor enhancements

MJI  is still waiting to confirm AAO plans for AAOmega data processing
     which he will progress at the AAT Board meeting next week - ongoing   <<<<


2.  Comments on WFAU minutes

There were several comments on the comments about the issue of how to deal 
with transferring new processed data versions.  MJI had been out of the 
loop on this due to travelling back from observing but said he would 
investigate the recent emails with WFAU and sort it out.  He also
noted that he had thought that an agreed policy was already in place.      <<<<


3.  Review of VDUC and VDMT meetings

MJI, STH, PSB and WJS gave their impressions of the recent VDUC meeting
and the substantive issues that were raised.  WJS noted that he had sent in
a list of actions from the meeting to the Chair but noted that there was 
still some internal debate on the exact wording of the actions.

Regarding data releases: the consensus was that a release to the UKIDSS 
community (NB. also ESO) of Semester 05A data ought to take place by
the end of the year since it would have then been 6 months after the last
data for that semester was taken.  Version 1 of the pipeline processing
for this semester is has been complete since the end of July.  Reprocessing
for Version 2 of the 2D images is complete and a test Version 2 of the
catalogues for SV1 has been available for the last 3 weeks.  However,
since there seemed to be no SV impetus for a release of this to the SV1
testers at the last SV meeting we have taken advantage of the hiatus to
carry out further tests on these catalogues.  A few minor improvements 
have subsequently been made and the SV1 catalogues will be reprocessed
to generate the official Version 2 by the end of next week (~22nd September).
This will then be extended to the whole of 05A over the following two
weeks.                                                                     <<<<

Simon Dye again presented the puzzling differences in photometry he was
finding, but during the meeting the problem that he was having was resolved
by STH and his photometry is now in agreement with the CASU pipeline version.

A further substantive point discussed, but not resolved, was the transfer
of data for VISTA from Garching to Cambridge.  The only sensible long
term (ie. economic) option is to use internet transfers.  However, this
raises the tricky issue of the bandwidth  bottleneck into and out of ESO
Garching.  This will be followed up at VPO level.

There was some discussion about the details of the URD and MJI is to 
provide some further feedback on several items to the next VDMT meeting   <<<<

The day after the VDUC meeting there was a VDMT meeting held via telecon.
STH reported on the actions from that meeting.   There was considerable 
overlap on topics with the previous days VDUC meeting but the main items
to resolve from CASU's point of view are the diverging dates and version 
contents of the planned deliverable releases.


4.  VDFS VISTA update

PSB reported on the progress with VDFS deliverables.  A prototype DRL
recipe for dark frame combining has been produced and used as a trial to
shakedown our understanding of some of the interface issues.  Delivery of
DRL v0.1 is on track but will soon require delivery to CASU of simulated 
FITS files with full (pseudo) header information present to fully end-to-end
test the various components.  

JRL noted that he had almost finished porting the CASU image detection and 
cataloguing code to run within the CPL environment.  

PSB noted that the HOWFS FITS specification was looking good and had
welcomed the chance to feedback comments.


5.  Data archives update - UKIRT, WFCAM, AAO, ING

UKIRT - up to date no further changes:

WFCAM - MR noted that the first batch of tapes for semester 05B covering
the period 25th August - 4th September had arrived and had been ingested
into the raw archive system.  He also noted that there were no missing files.
MR then gave an impressive demonstration of the DQC products database and
query system that he and EGS have been developing.  This was well received
and provides a lot of the missing functionality in chasing up progress,
feedback from external users and so on.  MJI noted that a survey progress
web page run off the backend of this database could provide automatic
updates of the progress of the UKIDSS WFCAM surveys and would be an excellent
prototype for the required VISTA survey progress databases.  MR and EGS
will investigate this extra appendage.                                     <<<<

AAO   - JRL reported that he had almost finished setting up the data archive
access and would attempt to complete the remaining scripts by next week.   <<<<
He also noted that the AAO are planning to changeover from Sybase to 
MySQL sometime in the near future which will require some reworking of
the archive content mirroring scripts.

ING   - MJI reported on a fruitful discussion that he and RWA had held with 
their ING counterparts during last week's final CMT management meeting in 
La Palma.  At this meeting it was agreed that internet transfers would now
be the sole means of shipping data as of the end of June 2005.  We currently
have in Cambridge DVDs covering the period up to mid-February 2005 and the 
remaining period from February to June, plus some overlap, will be shipped 
on hard (UBS2) disk.  ING have the last ~15 months of data on RAID disks
and prior to that local tapes going back to January 2000.  RWA will finish
off the ingest of the DVDs (the exact period covered contingent on the hard 
disk delivery) for the INT on one half of the new disk server which will 
also be the final resting place for the internet transferred data, currently 
buffered through a RAID disk on another PC.  The final step in placing all
of the ING archive online will be the ingest of the remaining CDs for the INT,
although alternative strategies for the latter will also be considered.


6.  WFCAM news and JAC telecons

As noted earlier, the first batch of WFCAM data for 05B has been received 
and ingested.  The image quality for WFCAM was reportedly much improved with
the whole focal plane now in alignment.  We do not currently have actual
average figures but MJI noted that the data taken for the UDS 2x2 -v- 3x3
tests averaged 0.7 arcsec image quality across the entire array.  

The data for one detector (#3) was transfer directly from JAC via internet and
is being used to assess the merits of doing deep surveys with alternative 
MSB strategies.  Initial results on image quality, seeing, enclosed energy,
detection, suggest (unsurprisingly) little difference in the final data 
products at this level of average seeing.  MJI noted that he had also used
this data to test drizzling the interleaved 3x3 data, at 0.133 arcsec/pixel, 
onto a 0.267 arcsec grid and had obtained good results.  Further tests are 
still ongoing in conjunction with Omar Almaini the UDS PI.


7.  Upcoming UKIDSS SV meeting

STH noted that in addition to the SV meeting in Cambridge on the 21st, there 
would also be a WFCAM calibration working group meeting in Cambridge on
the 20th.  STH presented the draft report that will form the basis for
discussion at the calibration meeting (and will no doubt also be discussed 
at the SV meeting).  

He noted that generally the agreement between the default 2MASS calibration 
and the UKIRT FS standard system was excellent but that there was still a 
puzzling offset, particularly in the H-band of ~3.6% between the calibrations.
He showed that colour equations from WFCAM to MKO could not account for this 
since they are essentially zero and he also noted that the nightly extinction 
on "good" nights (eg. 8th and 18th April where many standards were taken) was 
very close to the default values of 0.05 mags being used in the pipeline.

PSB suggested directly comparing the mags of UKIRT FS stars in the 2MASS 
catalogue, thereby eliminating WFCAM from the equation, as a direct check on 
the integrity of the different systems and the colour equations adopted.   <<<<

STH noted that he had done some more work on the residual photometric 
spatial systematics and was only finding an overall effect at the +/-1% level
over the whole array.  MJI noted that this is already within the systematic 
photometric error budget.

STH said that of order 20 external people would be coming to the SV meeting
and MJI asked if other CASU members could attend to help answer questions
about WFCAM data processing since he would not be there.


8.  Optical processing update

No further news on this.

9.  AstroGrid liaison update

Nothing to report.

10. AOB - hardware

MJI noted that 2 more disk servers have been delivered and the first of these
has been installed.  One of these 4 Tbyte partitions will be used for ING 
archive storage, the other will be for raw WFCAM data storage.

        - ADASS

MJI asked for any further news of the upcoming ADASS meeting and if any 
preparations were required it.  JRL noted that his spot had been upgraded 
to a talk and that he had plenty of WFCAM material to base it on.

        - PD33s

PSB asked if MJI could read the draft PD33s before they were sent off to
IoA admin for ingestion into the system.  MJI said, yes, if you are quick, <<<<
but noted the Paul Aslin had indicated that the drafts were generally fine
and covered what was wanted.

        - Other

STH raised issue of Plone and Solaris web pages for discussion at next 
CASU meeting.                                                              <<<<



Continuing Actions
------------------

STH  trial test version of SADT with PSB's help to install software
PSB  

JRL  finish access scripts to mirrored OMP database tables             

MR   finish writing up the flat field properties and sky frames report 

MR   ensure features web link is publically accessible
EGS

MJI  find out about AAO plans for AAOmega data processing


New Actions
-----------

MJI  investigate problems regarding new processed data versions with WFAU

MR   reprocess the catalogues for V2 release
JRL

MJI  append more details to the URD feedback for the next VDMT 

MR   investigate adding survey progress web pages to DQC database
EGS

JRL  finish AAO archive access scripts

STH  do a direct comparison of UKIRT FS and 2MASS photometry

MJI  read and comment on the PD33s ASAP

ALL  think about issue of Plone and Solaris web pages for discussion at next 
     CASU meeting