—
Plain Text,
17 KB (17449 bytes)
File contents
12th July 2005 Hoyle committee room 10.00 - 12:55
Present: MJI, RGM, PSB, DWE, STH, MR, JRL
Apologies: WJS, EGS
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. Report on VST meeting
4. VDMT report - Q3 plans
5. Data archives update - UKIRT, WFCAM, AAO, ING
6. Optical processing update
7. WFCAM JAC telecons and processing update
8. VDFS update
9. AstroGrid liaison update
10. AOB
Minutes
-------
1. Actions from last meeting
JRL has still to include access to "observing" log index files in the
raw WFCAM archive. There was some discussion if this was required
since they are now available in the processing status web page but
the general opinion was that it was worth having in more than one
place - ongoing <<<<
MJI are still in the process of writing a new report on the detector
JRL properties as they currently are and the processing strategy,
MR ongoing and needed in time for the SV meeting <<<<
MJI collated the comments on the WSA Photometric Recalibration Design
EGS properties and passed them on (this was acknowledged in the WFAU
minutes)
ALL sent in our quarterly reports to STH
JRL has encountered problems while working on the download side of the
AAO archive scripts due to connectivity issues to AAO - ongoing <<<<
WJS the latest SADT Definition Document may have been acquired,
unfortunately it is the Nov 2004 version. There was some discussion
regarding whether there was a new version available at all - ongoing <<<<
DWE wrote an observing efficiency script. The general result was that all
surveys were about as efficient as each other ie. there was not much
difference between the deep and shallow surveys. For 5s exposures the
telescope is usefully integrating on sky for about 55% of the time
(for 10s it is 65% and for 20s 83%). The observations for the
University of Hawaii and other non-UKIDSS observations followed a
similar pattern. MJI will raise this at the JAC telecon on Friday. <<<<
JRL has removed the remnants of the old UKIRT data archive from the data
discs on cass36. Some scripts remain, but the data has all been removed.
DWE updated the hardware web page to reflect the latest changes
MJI see item 3 for the report on the VST meeting
MJI raised the issue of flatfields (twilight and dome) at the WFCAM telecon.
He noted that in June many more successful flats were taken.
MJI asked about the WFCAM schedule at the WFCAM telecon. WFCAM will return
to the telescope on 26 August and will start science observing again
~a few days later. We should therefore start receiving data again for
processing ~mid-September.
RGM distributed the email from Steve Warren regarding the next UKIDSS
meeting. The dates of the next WFCAM related meetings were then
discussed:
UKIDSS Survey Heads 27th July to discuss MSB design
Science Verification meeting 3-4 weeks after data release from WFAU
- the pencilled in date is 18 August in Edinburgh
Workshop on 'Panoramic near-Infrared Astronomy' in Edinburgh 9-10th
November, followed by a UKIDSS consortium meeting 11th November
The next VDUC meeting is pencilled in for the first week of September.
MR took over development of the WFCAM progress DQC database (see item 5)
MJI discussed date of next VDUC/UKIDSS meeting at the VDMT (see above)
RGM has not yet found out more detail about the AstroGrid hardware money.
However, he assured us that it was still there - ongoing <<<<
RGM has not yet sent PSB's comments on the document covering the service
support for VO to Tony Linde - ongoing <<<<
2. Comments on WFAU minutes
The air-conditioning failure and resulting dampness was noted. Good luck
with the recovery and hardware move.
The transfer of data between Cambridge and Edinburgh is working well.
Around 2 Tbytes of data was transfered over one weekend with an average
transfer rate of ~10 Mbytes/s, to the apoplexy of the University network
monitoring service.
Noting the comment regarding transfer of data via LTO tape, MJI pointed out
that this was discussed at the VDMT meeting, which recommended that
offering this service was probably not a good idea and that network transfer
should be the goal.
3. Report on VST meeting
MJI reported on the VST meeting held at Garching on 22 & 23 June. This was a
very useful and productive meeting discussing the 3 core and 4 secondary
public survey proposals. Also discussed was what data products would be
produced and their verification.
It was noted that ESO will serve the data products of VST directly to the
public and that these have to be delivered to VO compliant standards
(whatever that will mean).
There was some reorganization of the proposals:
a) the two Galactic plane proposals - VPHAS (Drew et al.) and UVEX
(Groot et al.) are going to be merged;
b) KIDS (Kuijken et al.) should absorb the the remaining UKIDSS LAS/GCS
programme (Warren et al.) and Dalton et al. proposal.
c) z<2 census (Oliver et al.) trimmed but to be resubmitted
d) ATLAS (Shanks et al.) to be resubmitted in its entirety
The end result is that these 4 proposals will be resubmitted late September
for consideration by the PSP who will then forward them with recommendations
to the OPC for final deliberation. For a number of the above proposals the
originally requested Z observations will probably be dropped with the
intention of using VISTA to obtain these.
The time allocation for the first 10 years is 40% GTO time (ie. to the
consortia who paid for and built the camera and telescope, 10% to Chile and
50% to ESO (this probably being split ~40% to public surveys and ~10% to
private proposals).
Considering the proposals that have been chosen it looks like CASU will be
processing around ~20% of the data from VST, which is about what we expected.
It is likely that a similar model to the above will be used for VISTA and
there will be a VISTA call for proposals in the autumn.
First light for VST is planned for early 2006 and public surveys may start
around autumn 2006.
(Note that ESO Messenger#120 contains a status report on VST and a list of
the GTO proposals.)
4. VDMT report - Q3 plans
MJI and STH reported on the VDMT held on 8 July.
The UK URD was signed off and there is a copy (without all the signatures)
in the usual place.
The issue regarding keywords in the VISTA FITS files was discussed and
will be brought to a controlled conclusion.
For the next VDUC meeting what is required for the VDFS UK URD review is
an itemised list of work packages/tasks with dates and some priority ranking,
similar to the development plan in the VEGA proposal. This will be
submitted to the next VDMT meeting for review and discussion prior to
further release.
In parallel with this the VDMT felt that it was now timely to reassess
the dates and timings for all the work packages and version releases in
light of the delays to WFCAM commissioning since the original plan, drawn
up in 2003.
STH pointed out that there is a test version of the SADT available for
both VISTA and VST and that he would be investigating it. <<<<
For the Q3 plan it was noted that most of this would be continuing what
was current in Q2. JRL said that an additional item would be the
preparation for the ADASS in Spain. This is during the first week in
October and the abstract deadline is 1 August.
All were asked to send in proposed plans for Q3 for review. <<<<
5. Data archives update - UKIRT, WFCAM, AAO, ING
UKIRT: another tape of non-WFCAM data has arrived. JRL noted that we have
had 44 requests submitted and met by the online system since it was
re-instated.
WFCAM: raw data archive now has full authentication with individual
password password and the running and maintenance of this has been
handed over to MR. MR was asked to smarten up the front page. <<<<
JRL pointed out that there was a (known) problem with all calibration frames
in the period 5th to 23rd May whereby the FITS PHU contains the wrong
jitter and microstep information (in fact inherited from the previous
frame. This has also affected some science frames - those taken without
interleaving.
JRL was asked if the summit comments (OMP) database was easily accessible.
MJI reminded him that he had produced a simple perl script for local use
that he had made available to Simon Dye via the web. He will put links on
our web pages to this. <<<<
AAO: at the moment there is a connectivity problem with the AAO DB
which is slowing up progress but it is being sorted out.
ING: RWA and MJI are carrying out Internet transfer tests with Don Carlos
Abrams and Luis Hernandez at LPO to find an alternative to transferring
data to Cambridge via DVD, which has been found to be extremely unreliable.
These tests showed that reliable <transfer> rates of ~3 Mbits/s are possible
- no prizes for speed but gets here in the end. This is sufficiently
promising in that network transfers will currently just about keep up with
the average data taking rate (and the bottleneck may lessen in the future).
The need for a local backup of this data was raised and will probably be
on LTO.
RGM said that WFCAM raw data can now be obtained direct from the ESO
archive via one of two routes (he has tested this).
MJI pointed out that it is now ESO policy to make public survey raw data
world accessible immediately it is available in the ESO archive (WFCAM is
a special case).
WFCAM processed DQC archive: MR noted that this was not a simple matter
of cloning the INT WFC DQC (for various reasons). MR said that there
was a need to make it more homogeneous since this will be used for VST and
VISTA as well. MR said that he would have a chat with JMI in order to
progress this. It was noted that some of the design features from Aladin
would be useful and using this could take some of the strain out of
maintenance. This will be investigated further after MR has finished the
current processing. <<<<
6. Optical processing update
EGS has carried out some processing of XMM INT WFC data following a
request from RGM.
MJI and Nicolas Martin are processing several hundred frames of MegaCam
data.
7. WFCAM JAC telecons and processing update
Two telecons have been held since the last meeting (24 June and 6 July). The
four main issues were speed of the summit pipeline, problems with the FITS
headers, twilight flats and observing strategy.
Speed: JRL has speeded up the summit pipeline and carried out tests to see
how much of a logjam builds up. Using a night of data taken in June which
previously gave problems, he noted that using a simple script to mimic the
incoming data rate, the delay is now on average 38s with a maximum of 117s.
This compares favourably with the up to 1+ hour delay previously incurred.
A few bits of fine tuning are still in hand to improve the speed even
further. STH asked if these tests were carried out with Galactic plane
data, since this is another survey where the data processing has to
work hard. JRL said he would investigate alternative nights <<<<
One of the problems for the summit pipeline is that JAC also want to use
it to generate catalogues for each science frame (e-star project for
transient alert science). Fulfilling real-time DQC and this simultaneously
is tricky.
There was some discussion about hardware infrastructure at the summit and
how difficult it may be to test possibly network-dependent aspects outside
of a real observing night.
Twilight flats & FITS header problem: these are carried out at dawn and
have tended to either be overexposed or have a dither problem. However,
although this has limited the number of independent flats obtained it
appears to have been sorted, and there are 7 nights in June with reasonable
flats. MR will continue his investigation of these. <<<<
Observing strategy: currently only one strategy is being employed (shock
and awe - take as many independent frames as possible) making it difficult
to assess alternatives (eg. coaveraging with fewer dithers). MJI will
attempt some tests on the existing data to mimic this latter strategy
and cf. science achieved.
MR has made a web page displaying the progress of the WFCAM processing
(http://apm15.ast.cam.ac.uk/wfcam/). This is currently linked off the diary
page and provides links to the DQC summary info plots and index files.
DWE was tasked to link off the main web page and also include a link
to the rwa WFCAM archive. <<<<
MR was asked to put some brief explanatory text in the web page on the
various information available (with MJI's help if needed) <<<<
8. VDFS update
PSB said that good progress has been made recently. The issue of what goes
into the FITS headers appears to have been resolved at the last VDMT
meeting eg. status of the flat fielding lamps. We have also had feedback
from ESO regarding the VDFS FDR documents.
Michele Peron has informed PSB that the re-submitted FDR documents are now
ok. The remaining FDR documents are the SADT and the ETC. There are a
few minor changes to be made to the textual part of the ETC and the C
modules need inserting in place of the perl modules in the appendix. STH
will complete these changes and the updated ETC will be sent off. <<<<
MJI asked PSB to forward the email from Michele Peron to JPE and JMS. <<<<
PSB said that he would check the status of the various VISTA documents to
make sure that the version that we have is the final version containing all
the signatures. It was thought that this could be done at the same time as
sorting out the ETC document. <<<<
A number of relevant issues have been decided at a recent DICB meeting.
It was agreed that the number of jitters can additionally go into the
primary header using a non-ESO hierarchical keyword. Sometime in future
the astrometric reference system used will include the ICRS, for now
it remains FK5.
PSB raised the issue of whether VISTA calibration frames (domeflat,dark,...)
need to have WCS keywords in their headers. The general opinion was that
their absence (or presence) would not cause the pipeline software to barff.
It was thought that further detailed checking of SMB's template definitions
would be a good idea and that WJS would be the best person to check that
these are sensible. <<<<
PSB said that it was still unclear whether calibration frames would be
classified as science or calibration. This issue needs resolving by project
management. MJI will raise issue at next VDMT. <<<<
We were reminded that some DRL software will be required to be delivered
by the end of the year. However, this is dependent on realistic simulated
data being available in good time.
PSB noted that if the RIX responses are now agreed and the Board disposition
disposed, signing off and releasing the RR document would be good news.
However, there still may be outstanding issues relating to the SADT.
He will contact JPE about its status. <<<<
9. AstroGrid liaison update
It was reported that Astrogrid 1.1 is still about to come out.
RGM said that he had had discussions with other Astrogrid sites and all have
said that their installations have had to be carried out by Astrogrid staff.
10. AOB
None
Continuing Actions
------------------
JRL include access to "observing" log index files in raw WFCAM archive
MJI finish off report on the detector properties as they currently are
JRL include section on processing strategy and illustrate with
MR processing examples
JRL finish off download side of AAO archive scripts
WJS find out if there is a new version of the SADT Definition Document
RGM find out more detail about the AstroGrid hardware money
RGM send PSB's comments on the document covering the service support for
VO to Tony Linde
New Actions
-----------
MJI raise issue of observing efficiency at JAC telecon on Friday
ALL send in our proposed plans for Q3
MR improve look of front page of WFCAM raw data archive
JRL inform where scripts are for access to the summit comments data base
MR contact CDS regarding Aladin for CASU WFCAM processed DQC archive
MR investigate properties of all flats taken during 05A
DWE link processing progress web page off main VDMT web page (also a link
to the raw data archive)
MR put some explanatory text in the processing progress web page
STH finish the 3 remaining RIXs in the ETC document
MJI raise calibration frame issue at next VDMT
STH test test version of the SADT available for both VISTA and VST
PSB forward the Michele Peron email to JPE and JMS
PSB check the status of the various VISTA documents and ensure they are
up-to-date
WJS check that VISTA template definitions are sensible
