—
Plain Text,
14 KB (15269 bytes)
File contents
23th March 2006 Observatory meeting room 10.00 - 13:15
Present: DWE, MJI, PSB, STH, MR, JRL, RGM, EGS
Apologies: NAW, WJS
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. VDMT and Q1 reporting and WPs
4. Data archives update
5. Optical processing
6. WFCAM processing
7. VDFS VISTA update
8. ProtoPlone website
9. AstroGrid liaison update
10. AOB
Minutes
-------
1. Actions from last meeting
JRL has not received any news about the OMP database mirror and its repair.
He has emailed Andy Adamson about this. The script which updates the
MSB quality flag has been written. JRL will send this to MJI for
inclusion in the pipeline. <<<<
MJI has made no further progress investigating the merging of
classification statistics - ongoing <<<<
JRL has not finished the mods to the AAT archive but will by the end of
month. It was pointed out that the AAT board will meet 3/4 April. <<<<
STH has circulated a set of dates for the next VDMT meeting which is in the
process of being organized.
STH have reassessed the 06Q1 work packages and progress in advance of the
MJI VDMT meeting. MJI said that he would circulate the ASCII version of <<<<
this and we should all send back comments to him. <<<<
WJS tried to sort out a CASU visit to RAL during AIT3b (see item 7)
PSB
PSB made a list of desired AIT3b data and sent this to RAL (see item 7)
WJS
MJI has raised the issue of requirements on CASU during VISTA commissioning
and is awaiting some documentation regarding the plans for this
MJI were unable to include pointers to tables within the ETC help pages
PSB since they are still waiting for digital data. This is subsumed into
the ETC workpackage. Meanwhile it was suggested that WJS expedite
acquiring the necessary tables. <<<<
PSB checked the ETC and it uses the current set of numbers that WJS has
provided
ALL have read the VISTA survey proposals to check for possible science
involvement and unexpected processing requirements. A small meeting
was held to discuss this.
MJI updated JPE's data management plan template. CASU as a group is
normally listed and only where appropriate are named individuals
mentioned as Co-Is.
MJI has not contacted JPE regarding CASU preferred method of data transfer
from ESO - ongoing <<<<
RGM is working on quantifying the foul-up causes in LAS outliers list
- ongoing <<<<
DWE added that 95% of detected persistent images are classified as
galaxies and 5% as stars.
ALL had read the new version of the EDR paper and sent in any appropriate
comments (see item 6)
ALL had a look at the new Plone site and sent in comments to DWE and EGS
(see item 8)
JRL has upgraded the WFCAM pipeline so that it will cope with completely
missing detector data
MJI is providing input for the VST SMPs
STH has not found out what happened at the AstroGrid Implementation
meeting. He pointed out that no minutes had been published - ongoing <<<<
2. Comments on WFAU minutes
The main topic was transfer speed between CASU and WFAU. In general this
is 10 Mbytes/s, but seems to vary between 3-14 Mbytes/s. CASU are still
puzzled by the requirements on transfer speed. All of WFCAM 05A and 05B
processed data occupies only 16 Tbytes of disk space. A 10 Mbytes/s
sustained transfer speed would enable the complete set to be copied in
less than 3 weeks. CASU are still concerned where the bottleneck is
and PSB was asked to get MRTG set up on apm7 to monitor the traffic. <<<<
It would also be useful if WFAU could provide some information about
where they feel the bottlneck in network traffic is when they detect
low transfer rates eg. some traceroute stats.
The costs of UKLight were also discussed. MJI is concerned about the
recurrent costs of UKLight for which there appear to be no figures available.
It was noted that all of the relevant WFCAM O5A and 05B internet transfers
to ESO are complete and that as this is controlled from a different machine
it would have had negligible impact on transfers to WFAU.
There was also some discussion regarding the blitz nature of the transfers
which renders apm7 almost unusable when they are in progress.
3. VDMT and Q1 reporting and WPs
MJI said that the forward plans for Q2 have been produced and will be
circulated for comments (see above actions).
STH requested that we all send him reports covering the whole of Q1 by
the end of the month, even though several people have already sent earlier
monthly reports. <<<<
4. Data archives update
AAT: JRL said progress had been made on the AAT archive interface. He was
reminded that it was important to get this finished by the end of the month.
WFCAM: up to date
UKIRT: up to date.
ING: although most of the data (apart from small gaps) is on disc, the
current archive setup only knows about which CDs to be read since the
archive does not know the location of the online data. This is not completely
straightforward due to duplicate entries which need culling. There is also
the issue of reordering the earlier data using date subdirectories. JRL
pointed out that this will only work back to about 1990 since earlier data
often had no date information in the header. EGS has made a start sorting
out the duplicate entries and he, JRL and RWA need to get together urgently
to sort this out. <<<<
5. Optical processing
MJI reported that a draft of the VST public surveys Survey Management Plan
for the 2 UK-led VST surveys was being written. The VPHAS+ plan should be
finished by the end of the month and will be used as a starting point for
the ATLAS plan which is due two weeks later (sic!).
6. WFCAM processing
Photometric calibration status:
STH said that since the last meeting a preliminary investigation into the
Y and Z calibration has been carried out using the Cohen standard. MJI
was concerned that in the random check he did for this object it was flagged
as saturated in the Y and Z catalogues and therefore BAU. Alternatives
are being investigated including spectrophotometric standards from Sandy
Leggett, some UFTI data, and a comparison with SDSS z' data.
The updated WFCAM calibration will use a restricted 2MASS J-K colour range
of 0.0-1.0 to help exclude late-type giants, weird objects and heavily
reddened stars. This should produce a more stable calibration for the Y and
Z bands (but have little impact on the JHK calibration).
MJI said that before starting the recalibration he wanted one final look
at the colour equations currently in use to see if any small final tweaks
were needed.
MJI initiated some discussion on what was needed to describe the error on
the photometric zeropoint and the nightly variation in extinction. Was
more than a single number needed ? Opinion was divided on this.
Persistent investigation:
DWE was asked to get the persistence report ready for external release. <<<<
He was also asked to add a few examples of persistence to the examples page
and include frame number and X,Y position on chip. <<<<
UKIDSS EDR Sci and CASU papers:
STH said that the EDR paper was already on astroph. The group thought that
Simon Dye should be congratulated on doing a good job in producing the
paper.
MR reported that a good start on the draft (13 pages and rising) of the
CASU technical paper had been made. MJI thanked MR and EGS for progressing
this and noted that he would work on the draft after the ESO deadlines.
Problems eg. UDS:
MJI mentioned that he had received an email from Omar about problems with
the EDR UDS catalogue data and asked if anyone was aware of other issues
with the EDR data.
RGM showed some examples of LAS outliers. Some of the problems seemed to be
caused by the deblending algorithm. JRL said that he would look into some of
these examples. <<<<
Reprocessing 05A:
The reprocessing of the 05A data will take place soon. JRL said that the
main change was in the determination of the sky (and also cross-talk
correction would be applied). The selection of which frames are used has
changed and the object rejection within each contributing frame has
changed.
During the discussion it became apparent that avoiding the list of bad
frames forwarded by Nigel (those not to be used for sky determination due
to Moon problems) had not yet been implemented. MJI said that he would <<<<
email this (again) to JRL who would then implement it. <<<<
There was also a raw data header problem list of files that had slipped
through the fixing net. This also needs dealing with. MJI noted that <<<<
he had already fixed the DXS (and a few UDS) QUICK LOOK recipe keyword
problems.
Extra quality control of the reprocessing was discussed and it was agreed
that MR and MJI would look hard at the data from 8/9 April to confirm that
the reprocessing was satisfactory. <<<<
News on science with WFCAM data:
STH said that a paper on alpha Sco was in production. This is a GCS target
and the pipeline has worked very well with this region.
RGM mentioned that he had been doing work on high redshift QSOs and also
very red objects.
7. VDFS VISTA update
VIRCAM Status AIT3b:
PSB reported that he had forwarded a list of requested observations to be
carried out prior to VIRCAM shipping. Gavin Dalton noted that he thought
that they could do the AIT3b relevant ones at the end of the week, but then
having had a quick look at the TCS simulation setup viz-a-viz the camera
setup, felt that this was going to be difficult to achieve by the 24th March.
He also requested that CASU not attend AIT3b, and thinks that the best time
to actually visit is definitely going to be during all-up testing.
The tests that have been requested can be at
http://www.ast.cam.ac.uk/vdfs/docs/vista/cambridge_ait_request.txt
VDFS docs status:
PSB said that there had been no movement on the status of the VDFS
documentation since the last meeting.
DRL status:
JRL has been working on the DRL and supporting documentation. A telecon
with Sandra Castro and Pascal Ballester was held on 9 March to discuss a
dry run of the pipeline recipes on ESO's new infrastructure (Condor). To
progress this a meeting in ESO will be held on 26 April with Sandra and
Pascal.
ETC status:
PSB said that there had been 3776 hits on the ETC and people still seem to
be using it. Jakob Vinther has been assigned to write the ESO version of the
ETC. PSB has sent him the most recent version (v1.1) of the ETC document
together with the associated data tables and C code.
Summary of VISTA survey proposals:
RGM said that there were 14 proposals (known to us) and that this was about
a 2-3 times oversubscription factor. He circulated a summary of the known
proposals.
8. ProtoPlone website
There was much discussion on this, but apart from minor comments (failure of
print button and font size was too small), the main topic was the dumping
of documents. STH said that the dumping of documents tended to create a
mess. At least with Plone, these would be indexed and it would be possible
to find them. PSB said that he still wanted an ordinary file system that
could be accessed over the web. RGM and MJI did not see why we couldn't have
a hybrid system. STH was very much against this since it would lead to
confusion as to the location of documents. JRL said that we should give
Plone a go in order to see where the problems were etc.........
DWE was concerned about trying to maintain two systems. MJI suggested that
a script be written to check what files had been recently added to the
file based system and automatically load it onto the Plone system. DWE said
that doing it automatically wouldn't work due to different organizational
structures, but he would look further into this. <<<<
RGM and STH clarified a lot of the discussion by stating that there were two
fundamental issues 1) how do we handle our internal documents 2) how do we
organize our public interface. The latter was probably the more important.
MJI asked DWE to reorganize the internal file system on the same basis as
the internal Plone system. <<<<
9. AstroGrid liaison update
Nothing to report.
10. AOB
NAM:
EGS said that he had not been allocated an oral presentation for the "New
Techniques & Future Facilities" session at NAM. He will now contact Steve
Warren to try for the "Wide-Field Surveys: First results from UKIDSS" <<<<
session. It was felt that if we can't get an oral presentation then we
would eschew presenting a poster.
MJI drew our attention to an image processing workshop in Croatia. The
deadline is the end of March. He said that he would distribute the details
by email. <<<<
IoA Open Day: we will not be opening up the APM building.
Continuing Actions
------------------
JRL provide a script to add JAC MSB quality flag to the FITS headers
MJI finish investigating merged classification statistics
JRL finish mods to AAT archive interface by end of March
MJI contact JPE regarding CASU preferred method of data transfer from ESO
RGM finish quantifying foul-up causes in LAS outliers list
STH find out what happened at the AstroGrid Implementation meeting
New Actions
-----------
MJI circulate the ASCII version of the 06Q1 work packages
ALL feedback comments regarding the 06Q1 work packages
WJS contact JPE regarding acquiring the numeric (rather than graphic)
versions of the data that is required by the ETC
PSB set up MRTG on apm7 to monitor network traffic
ALL send reports to STH covering the entire quarter
JRL have a meeting with RWA to expedite progress with the ING archive
EGS
MJI recalibrate the colour terms of 05A
STH check if the colour terms need updating
DWE get the persistence report ready for release
DWE add a few examples of persistence to the examples web page
(including frame number and X,Y position on chip)
JRL investigate deblending problems identified from LAS outliers list
MJI email Nigel's list of bad sky frames to JRL again
JRL implement bad frame list in sky determination procedure
JRL check and if possible fix any remaining FU'd headers
MR check reprocessed data from 8/9 April for quality control
MJI
DWE write script to check for newly added files to internal file system
DWE reorganize the internal file system cf. Plone internal system
EGS contact Steve Warren regarding giving a presentation at the Wide-Field
Surveys session at NAM
MJI distribute the details of Croatia workshop
