—
Plain Text,
13 KB (14136 bytes)
File contents
8th March 2006 Hoyle committee room 10.30 - 13:35
Present: DWE, MJI, PSB, STH, MR, JRL, WJS, (RGM after 12:15)
Apologies: NAW, EGS
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. VDFS VISTA update
4. WFCAM processing
5. Prototype Plone website
6. Data archives update
7. Optical processing update
8. AstroGrid liaison update
9. AOB
Minutes
-------
1. Actions from last meeting
JRL has not provided an MSB quality script which MJI will use to update
the FITS header during pipeline operations. This is waiting on the
OMP database mirror being fixed - ongoing <<<<
MJI has carried out further investigations into the merged classification
statistics - ongoing <<<<
WJS AIT3a has been and gone. A visit is likely for the next set of tests
AIT3b (see item 3)
MR has made the science plots of IC4665 and M17 and the LAS tile visible
from the Plone reduction status web pages
PSB have discussed a suitable strategy for jittering VISTA twilight flats
JRL a summary of this has been emailed to SMB
MJI
JRL has not finished the mods to AAT archive but will by the end of month <<<<
JRL has asked if there is any more UKIRT (non-WFCAM) data that needs
ingesting (but has not received a reply)
STH has not raised various VDFS issues at the VDMT meeting since there
hasn't been one since this year. He will initiate discussions. <<<<
DWE has carried out further investigations into persistence (see item 4)
MJI forwarded the EDR paper CASU coauthor list to Simon Dye
JRL processed the publicity frames for the BBC, but found that that the
data quality was very poor and that no sensible reduction was possible
MJI has not had any feedback from the UKIRT board meeting so everything
must be fine
ALL sent in reports to STH, however there hasn't been a VDMT meeting since
then. The lack of VDMT meeting and format of our reports were discussed.
MJI said that he and STH needed to revisit the work packages for 06Q1
and reassess progress <<<<
RGM investigated the CASU presence at NAM 2006. EGS submitted an abstract
MJI emailed Andy Lawrence (CC Andy Adamson) asking for clarification
regarding CASU direct access data rights on the processed data, but has
not received a reply.
2. Comments on WFAU minutes
MJI noted that there had been some activity regarding UKLight. PSB reported
that some UKLight hardware has been installed by the University Computing
Service. Many emails have been exchanged over the last week and much of it has
been centred on the issue of transfer speed. PSB pointed out that this is
not the central issue for CASU. The current worry is over the charged costs of
data transfer roughly, 1k per TByte. Thus the benefit of UKLight should be
judged largely by cost and not data transfer speed.
PSB said that Andy Batey has looked at what proportion of the IoA's
network traffic is caused by CASU. The split seems to be 80:20 between CASU
and SEGUE. Note that the CASU part includes ESO as well as WFAU transfers.
STH noted that there had been a comment regarding the lack of dynamic range
within deep stacked images and catalogue generation due to the way data was
handled internally. MJI said that he had replied to this query (while he was
away on holiday) suggesting simple interim fixes.
3. VDFS VISTA update
- Telescope status:
WJS reported on the current status of the telescope and mirror schedule.
- VIRCAM Status - AIT:
The next set of camera tests to occur will be AIT3b, which will occur after
AIT4b! AIT3b will be in ~2 weeks time and a CASU visit during this to carry
out and acquire some test data is being negotiated (PSB and WJS to sort out
details). Real VIRCAM test data will be useful for creating a test harness
for later ESO deliverables relating to the summit and Garching pipelines. <<<<
PSB sent a request to RAL that the TCS simulator be available and that
generally the current software be installed to ensure up-to-date data format;
there is some difficulty with the use of a very old version of the DCS
subsystem to control IRACE to work around the random FITS ordering problem.
PSB noted that flat-fields would be one of the things required. These do not
have to be perfect so an even illumination is not critical. MJI asked PSB and
WJS to make a list of the desired data and forward to RAL. <<<<
- VDFS docs status:
PSB reported on the status of these documents which is summarised below.
Documents completing FDR:
RIX: v1.0 2005-05-05 ??
DFS Impact: v1.2 2005-05-09 awaiting release
Calibration Plan: v1.2 2005-08-12 awaiting release
DRL Design: v1.2 2005-08-12 awaiting signing
ETC: v1.1 2005-09-01 released to ESO?
SADT: ?? ??
Documents complementing DRL release v0.1:
DFS Impact: v1.3 2005-12-16 awaiting approval
Calibration Plan: v1.3 2005-12-19 awaiting approval
DRL Design: v1.3 2005-12-22 released
Documents for UK review:
UK URD v3.0 needs updating?, signing
- DRL status:
JRL said that excellent progress was being made on the DRL plus documentation
and noted that a few more recipes would be needed before a 0.5 version is
ready.
JRL,PSB and MJI will have a telecon with ESO tomorrow to discuss the status
of the DRL and schedule for delivering incremental releases.
- Workstation for summit pipeline:
There was some discussion regarding where this would come from. The equipment
would first go to ESO Garching for software installation, before being
shipped to the summit. Both PSB and JRL stressed the advantage of being
able to test the pipeline within the ESO software environment before
delivering the VDFS summit and Garching DRL modules to ESO.
- Commissioning:
It is unclear what requirements there are on CASU presence and use of CASU
VDFS pipeline software during commissioning. This needs to be discussed at
the next VDMT meeting. <<<<
- ETC status:
PSB reported that the most recent issues were regarding overheads. There
has also been some recent discussion as to whether the calibration files
should be made available to users. MJI's opinion was that this could be
done most easily via the help pages but PSB pointed out that this would
still involve the production of more user-friendly calibration files. <<<<
It was noted that there had been 2804 hits on the ETC (presumably by people
designing surveys). There followed some discussion regarding whether slew
and dump-to-disc time should be included in the calculations - the general
feeling was that they should be, but there was still some uncertainty
regarding parallelising the latter with moving to the next position on sky.
PSB noted that he had removed the slew overhead from the reported overhead
time on request from JPE.
PSB should check that the ETC uses the updated numbers that Will suggested.<<<<
- News on VISTA survey proposal front:
WJS said that he knew of 9 surveys that had been proposed. STH went to the
twiki page http://wiki.astrogrid.org/bin/view/VISTA (there were 11 there).
These were discussed along with what CASU's contribution might be to these.
MJI suggested that a table summarizing these surveys would be useful.
We were all asked to read the proposals to see if there were any we would
like to be involved with over and above the standard processing. <<<<
MJI volunteered to update JPE's data management plan template suggesting
that for standard VDFS processing tasks and data quality control, we should
have a single CASU entry. More project-specific processing tasks would
depend on CASU individuals wanting to be directly involved in the science
and have true Co-I status. In this case the relevant individual's names
should be used. <<<<
Higher level deliverables within these surveys need to be clearly flagged
and the resources to carry out any additional work need to be identified.
CASU have been asked about our preferred method of data transfer
between ESO and Cambridge. MJI said that, given Internet transfer is not an
option with ESO, then LTO tape would be preferable to discs. MJI said that
he would contact JPE regarding this. <<<<
4. WFCAM processing
- photometric calibration status
STH reported on a single Cohen spectrophotometric calibrator star (G type)
test that had been carried out. In JHK the photometry was off by 2-4% and in
Y and Z it was different by 10-15%. Internally the scatter between different
WFCAM measurements of the same stars was at the level of 1-2%. It is
unclear whether this reflects a real problem or single star shot-noise.
STH noted that further investigation of the Z-band calibation were
planned using SDSS comparison and both Y and Z using updated Z and Y
pseudo-standards from Sandy Leggett.
- persistence investigation
DWE described his current work on persistence. A practical robust scheme
to correct the image data does not seem possible. The error on the
correction would be of order the size of the correction. The source of the
scatter may be systematic in some way rather than random eg. dependent on
spatial location, but is proving difficult to measure and hence quantify.
The impact on the UKIDSS surveys will probably be small because most frames
are interleaved or are part of deeper stacks. Due to moving the frame
between each interleave exposure, persistence images get even further
smeared out and they are thus not detected as a source by the cataloguing
software. For non-interleaved data, eg. LAS, it is estimated that around
0.1% of catalogued images may be caused by persistence.
Flagging options were discussed as an alternative. This could be via the
confidence maps (a lot of extra computation) or more practically via the
catalogues, using 2MASS stars and a simple persistence model as a predictor
of possibly suspicious images.
There is still a need for assessing how serious a problem it is and RGM
said that he will use a list of 1000 known spurious outliers from 20 sq deg
of LAS data to assess this. These have already been inspected visually but
will now be characterised into different causes (including cross-talk and
persistence) to assess the need for possible remedial action. <<<<
- progress on PSF fitting
DWE said that no progress has been made on PSF fitting since the last
meeting, due to investigating persistence taking precedence.
- UKIDSS EDR Sci and CASU papers
It was noted that we were still waiting to see a new version of the science
paper. However, a new version of the EDR paper had been received and all
named CASU authors were encouraged to read and comment on it. <<<<
5. Prototype Plone website
Due to time considerations and the lack of EGS, the demonstration of the
new Plone web site was postploned. Meanwhile, we were all asked to look at
the new site and send in comments to DWE and EGS. <<<<
6. Data archives update
- AAT, ING, WFCAM, UKIRT:
All archives are up-to-date. ING is now all on-line bar a gap of 6 months
between the end of DVD ingest and start of internet transfers. This is
planned to be filled by internet transfers.
MR pointed out that several nights of the last of the WFCAM data did not
have any data from chip 3 due to a problem with the controller. JRL needs
to upgrade the pipeline to cope with this. This has caused a slight delay
in finishing off the last week or so of 05B data processing. <<<<
7. Optical processing update
- VST, IPHAS
MJI noted that the VST public survey updated management plan deadline is at
the end of the month and that some input is required from him. <<<<
8. AstroGrid liaison update
STH said that there had been an AstroGrid Implementation meeting in
Liverpool on 1 March. Only NAW and FvL from CASU were able to attend. STH
said that he would find out what happened and report at the next meeting. <<<<
9. AOB
EGS and MR are preparing a data reduction paper from the various reports
and papers that have already been written. MJI will review progress after
the VISTA PS deadline passes.
STH is writing a photometry paper.
Continuing Actions
------------------
JRL once the OMP database mirror has been fixed, provide a script to add
JAC MSB quality flag to the FITS headers
MJI finish investigating merged classification statistics
JRL finish mods to AAT archive interface by end of month
STH liaise with rest of VDMT team to organise next meeting and make sure
assorted issues are on the agenda
New Actions
-----------
STH reassess the 06Q1 work packages and progress in advance of VDMT meeting
MJI
WJS sort out CASU visit to RAL during AIT3b
PSB
PSB make a list of desired data from the AIT3b visit and send this to RAL
WJS
MJI raise issue of requirements on CASU during VISTA commissioning at
next VDMT
MJI include pointers to tables within the help pages
PSB
PSB check that the ETC uses updated overhead numbers
ALL read VISTA survey proposals to check for possible science involvement
MJI update JPE's data management plan template
MJI contact JPE regarding CASU preferred method of data transfer from ESO
RGM quantify foul-up causes in LAS outliers list
ALL read and comment on new version of the EDR paper
ALL look at the new Plone site and send comments to DWE and EGS
JRL upgrade WFCAM pipeline so that it will cope with completely missing chips
MJI provide input for VST PS management plans
STH find out what happened at the AstroGrid Implementation meeting
