—
Plain Text,
14 KB (14601 bytes)
File contents
12th June 2007 Observatory Meeting Room 10:30 - 13:00
Present: DWE, STH, JRL, PSB, MR, MJI, RGM, RWA
Apologies: NAW, EGS, RG
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. Rolling grant submission
4. News from meetings - VDMTs, VDUC, UKIRT board, UKIDSS
5. Data archives update - AAT, ING, WFCAM, UKIRT
6. Optical processing update- VST delays
- IPHAS release
7. WFCAM news - raw data transfer status
- processing status
- photometric calibration update
8. VISTA news - pipeline progress
- summit processing hardware
- cold camera tests lab and on telescope
- draft milestones
9. ADASS 2007 - latest
10. AOB - retiring cass36, cass123
- attempted break in
- plates and tapes in SPO
Minutes
-------
1. Actions from last meeting
RGM have not completed the work on the classification aspect of cross-talk
EGS weeding and classification generally - ongoing <<<<
MJI
MR has sorted out the credit card payments for ADASS 2007. Ian Bell has
JRL access to a credit card machine and the information for this (entered
manually) will come from the web form.
MJI raised the issue of field rotation flat field tests and VISTA integration
at a VDMT meeting (see later)
MJI raised the issue of updating the VDFS UK User Requirements Document
at a VDMT meeting. This document now has been updated.
NAW emailed Tim Naylor (probably) to obtain agenda for VDUC meeting and
minutes from previous meeting. These were then distributed.
MR contacted the Durham group (Jim Geach) about their progress in
processing raw WFCAM data. They are happy with their reduction. MR
was asked to obtain some example images from them for comparison. <<<<
MJI asked about the possibility of obtaining more VISTA test data at
the next VDMT meeting (see later)
NAW have discussed the possibility of fast links to ESAC (for Gaia).
PSB This item will be continued offline.
EGS has not investigated photometric redshifts in DXS - ongoing <<<<
PSB has sent his FTP test results to Brad (CC'd to Andy Adamson) and this
has shown that the transfer of (possibly) all WFCAM data is feasible.
PSB was asked to prod JAC to see if anything was going to come out of
this eg. transferring the missing NDR data. <<<<
MR inquired about the causes of the intermittent nature of WFCAM transfers
to ESO. Jens Knudstrup replied and said that the software had been
checked and was OK. The most significant reason is that there is disk
downtime caused by having to take various machines down to change
disks. The nominal bandwidth for transfers is 2.0 Mbytes/s, but what
is achieved is 0.4 Mbytes/s due partly to the ESO-end ingestion process.
JRL has been in touch with Kognitio regarding further tests involving their
data base software. Although a test doing a cross federation of IPHAS
and 2MASS data has not been carried out, a simple calibration test on
IPHAS data has been very successful. While the test took 20 days using
a single-thread Sybase system, the Kognitio system took 8 minutes! (to
be fair they had many more processors, but it does show that this is a
scalable task).
Kognitio have agreed to give us a demonstration copy of their software
and JRL and PSB will be carrying out various tests on it. <<<<
EGS have sorted out the use of the spare 750 Gb disks as backups for the
MJI IPHAS data base. MJI has double checked that there are two extra spares.
MJI has still not put the OMP log comments into the processed directories
- ongoing <<<<
MJI have looked into the light leaks problems in dark frames. A consequence
MR of this is that the generated darks for each night processed have to
be checked manually. These light leaks have been detected as far back
as 06A.
STH has updated his secret webpage on the recent photometric calibration
developments.
MJI emailed NCH to remind him about the photometric corrections needed to
account for the astrometric distortion
MJI did not have to raise the issue of what is meant by exposure time value
contained within VISTA FITS headers at the VDMT meeting since it was
picked up directly from the minutes. After a flurry of emails an SPR has
now been issued.
ALL provided inputs for the rolling grant application which was submitted on
Friday.
2. Comments on WFAU minutes
It was noted that UKLight is working, but it is thought to be a bit slow (3
Mbytes/s). This is fine for the current workload, but may be a problem for
when VISTA and WFCAM data need to be transfered.
STH drew attention to the problems GPS is causing at the archive end of
operations.
There was a short discussion about the DR3 release. No date can really be
given for this until CASU receives the missing NDR data. Once that has
arrived, it should be processed within a month.
3. Rolling grant submission
MJI reported on the recently submitted rolling grant application.
4. News from meetings
There have been 2 VDMT meetings (27 April & 8 June) since the last CASU
meeting. Various issues arising from our actions were raised. The main
gripe was the release of the pipeline software version 0.6 to ESO without
going through the VISTA project office. In mitigation 0.6 was a minor
upgrade to fix "tickets" arising from the major release 0.5.
PSB and MJI attended the VDUC meeting in Edinburgh 10 May. After this meeting
a new UK URD was produced (much cheering). There has also been a lot of useful
feedback from the PIs and more to follow after the release of the updated
URD to them for comment.
STH attended the UKIDSS Survey Heads meeting in Oxford 9 May. Topics
discussed: date for DR3; new scheduling log for surveys (allocations of time);
award of time to 3 new campaign surveys (including STH's Transit Survey).
STH noted that the world release for EDR would be 10 August and for DR1 21
January next year. Omar Almaini commented at the meeting that some of the
UDS data for 06B needs to be reprocessed. MJI said that he had an email about
this yesterday and will investigate.
The next Survey Heads meeting is supposed to be at the end of June, and will
be in Cambridge.
Following the Oxford meeting (10 May), STH attended a calibration meeting with
Steve Warren and Paul Hewett in which they spent half a day going through
the calibration paper and progress. Both seemed happy with it and the next
meeting is scheduled for the first week of July.
MJI reported on the UKIRT board meeting he attended on 17 May in Durham. All
the reports to the UKIRT board are available on our web site
http://casu.ast.cam.ac.uk/documents/other-meetings/ukirt/
and the presentation can be found at
http://www.ast.cam.ac.uk/~wfcam/talks/ukirt_may2007.ppt
The board seemed happy with the data processing and were particularly taken
by the diagram showing the growth of data volumes compared to other surveys.
5. Data archives update
ING:
RWA reported that 300 VMS backup tapes were taken to La Palma in order to
extract the data from them. 60% of the tapes were processed resulting in
7000 files in SDF format that need to be converted to FITS. The transfer was
quite slow since data also has to be copied across to a different machine
afterwards. Most of the data is from WHT Taurus, FOS or direct imaging. The
tapes have been left out there to be processed at a future date.
JRL said that no progress had been made regarding the header updates for
the ING archive. He and EGS will work out a scheme which RWA can use. <<<<
The store room in the SPO containing our tapes and plates is being gradually
emptied. The backup tapes will be stored in the annex between the Obs
building and old RGO building. Many other tapes (and CDs) will be thrown
away since they overlap with other media. DWE expressed a concern that the
annex will get very hot in the summer and that this would not be good for
tapes. RWA said that he would ask Peter Weeks if blinds can be fitted. <<<<
AAT:
JRL reported that the AAT archive is now working including the
synchronization between their data base with the observing logs and ours.
http://archive.ast.cam.ac.uk/aat/
This archive contains data from 1990 to the present. There was a discussion
on how to advertize this. There already is a news item on the Plone site,
but STH felt that more was needed. The consensus was that the advertising
should come from the AAO side and JRL said that he would email Stuart Ryder
to expedite this. <<<<
UKIRT:
This is up-to-date.
WFCAM:
Tapes have been received up to 21 May ie. we have up to all of 07A and 06B
except for the NDR files from the end of October to 20th December.
The copying of UKIDSS and calibration files over to ESO for the raw archive
is complete up to 23 April.
6. Optical processing update
MJI reported on the VST delays. It is unlikely that any public surveys
will start until well into 2009.
A large part of the band-merged IPHAS products have been released within the
IPHAS consortium for limited testing using AstroGrid software. RG is currently
working on moving the remaining IPHAS web pages at ING over to Cambridge.
CASU are also still carrying out the extinction calculation derived from the
Carlsberg Meridian Telescope data. DWE said that there is still a payment
due for this work which needs chasing up. MJI said that he would ask Paul
Aslin about this. <<<<
7. WFCAM news
MR said that processing was complete up to 20 April except for a small chunk
of January data that is still being processed.
MJI gave an update on the photometric calibration. Half of the illumination
correction required is at the detector level. This probably arises from
pedestal offsets within the flat fields since the offset patterns are stable
over a flat field period (ie. roughly monthly). It is simple to remove this
part of the systematic effects using the individual detector FITS header
keywords.
The remaining systematic patterns appear unrelated to the flat fields and
seem fairly stable over semester-length periods suggesting that a correction
table could be used to further reduce their impact.
CASU have been asked about how asteroids are dealt with during the data
processing. The short answer is that they are ignored since this was always
felt to be an end-user/archive issue.
A further 20 Tbytes of WFCAM data storage have been purchased (apm24_c/d)
and installed and a bit of data shuffling has started populating (and
testing) them.
8. VISTA news
Maintenance issue 0.6 of the pipeline software has been sent to ESO and
deals with the gripes about the 0.5 version. In general, things are looking
good.
There was a brief discussion of a draft milestones document.
The cold lab tests of the camera are being carried out at Paranal and it is
planned that camera will go on to the telescope for EMC tests in July. CASU
have been asked to help investigate the results. Related to this is the
issue of how to get access to the data there. PSB has requested (via JMS)
access to an account on the VIRCAM system.
There was a brief discussion about what further tests could be done.
PSB is already working on this. One extra possibility is to attempt
flat field rotation tests even though the illumination will not be uniform
it should still be possible to assess possible problems.
PSB reported that the summit processing hardware may only have 1 or 2
processors, rather than the 16 processor blade machine originally foreseen.
If this goes ahead it will limit the amount of real-time DQC that can be
carried out.
MJI said that the revised SMPs (even RGM's) have been submitted.
Documentation was then discussed, in particular the Calibration Plan. MJI
will email JPE regarding version 1.5pre1 of this and will request the
pending v1.4 be released in the meantime. <<<<
9. ADASS 2007
JRL said that there was no real news. The hall cost has gone up by 2%, but
this was anticipated.
A decision was made that there will be a nominal charge for the tutorials.
10. AOB
cass36 and cass123 will be the next machines to be retired. cass123 is just
coming off maintenance and will be naturally retired. cass36 should only
be serving access to disks. A similar piecewise removal operation will be
carried out as for cass38.
There was an attempted break in at the back of the APM building last week.
Steel plates will be placed over the damaged woodwork. MJI noted that we
should ensure the blinds are closed in the evening and at weekends and also
make sure that laptops or other visible goodies are not left lying around.
STH raised the issue of office space in July. He has many visitors and some
of his visitors have invited their own visitors (d'er!) It is unclear where
these folk will go and STH will liaise with MJI offline to solve this.
PSB pointed out that there are next to no spare slots in our network switch
gear. He was therefore asked to order another Gbit switch for the APM
building to solve the problem. <<<<
Continuing Actions
------------------
RGM report back on classification aspect of cross-talk weeding
EGS
MJI
EGS investigate photometric redshifts in DXS
MJI place OMP log comments into processed directories
New Actions
-----------
MR obtain some example images from Durham group (Jim Geach) in order
to do a comparison
PSB contact JAC regarding FTP transfers eg. transferring missing NDR data
JRL carry out tests with demonstration copy of Kognitio software
PSB
JRL work out scheme for doing header updates for ING archive
EGS
RWA ask Peter Weeks if blinds can be fitted to interconnecting annex
JRL email Stuart Ryder about advertising the AAO archive
MJI ask Paul Aslin about the Carlsberg extinction payments
PSB email JPE regarding version 1.5pre1 of Calibration Plan
PSB order another Gbit switch for APM building
