—
Plain Text,
15 KB (15469 bytes)
File contents
Wednesday 23rd December 2009 APM Meeting tables 11:35-13:15
Present: MJI, RWA, EGS, JRL, JPE, MR, SC (for select items), RGM
Apologies: NAW, STH
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. Recent and upcoming meetings
4. Data archives update - AAT, ING, WFCAM, UKIRT, VISTA
5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru, VST .....
6. WFCAM update
7. VISTA update
8. AOB
Minutes
-------
1. Actions from the last meeting
MJI stage I software release still on hold - ongoing <<<<
EGS
MJI technical paper - no progress <<<<
MJI however did report that he finished writing the utility to convert
binary tables in Sydney airport. Among other things it shows how to
compute accurate astrometry and fluxes with all the corrections and
it also attempts to calculate correct astronomical position angles
for the equivalent Gaussian second moment ellipses rather than the
usual internal PAs. MJI noted he has so far only done limited testing
on it, however, RGM offered to test it further. If it works for VISTA
and WFCAM it should be reasonably foolproof. <<<<
MJI reported that the dome flat comparisons has not yet been finished <<<<
JRL likewise has not finished the low light level linearity tests <<<<
JRL has not yet supplied any beer so does not win the anti-surge unit for
his office.
SC has begun investigating the backup plan for remaining kit and meanwhile
noted that tape backups are part of the disaster recovery plan and all
tape backups are (or should be) stored in the Observatory building
(meeting room cupboard or link room) - ongoing <<<<
MJI The relative QEs of each VIRCAM detector have now been put on the
EGS technical web pages and the average QE factored into the latest
Version (1.5) of the ETC. While on this topic JRL and EGS have a nice
example of persistence a star of magnitude H = -0.4 and were actioned
to fill in the persistence section of the technical web pages with this
as an example and at the same time educate users with examples of the
strange 'holes' aka 'stars' that appear in the calibration frames for
several of the detectors. <<<<
MJI flagged the second batch of SV data for copying by WFAU. This includes
data up to and including 4th Nov i.e. some dry run data too. He noted
that the ESO SV team has released two examples of processed SV data.
JRL has issued a new release of the pipeline software and DLRD
JRL have made slow progress on a calibration plan update. JPE asked whether
STH we needed more calibration data (e.g. Y and Z cf. with WFCAM) since
there was still an allocation of VISTA time for this purpose. MJI said
in general this is not needed but would verify this with STH on his
return <<<<
DWE decided on a date for Yuletide celebrations and indeed enacted it
SC
STH
2. Comments on the WFAU minutes
The problems with reading the FITS binary table WCS were noted as were the
issues raised by switching to the newer 16-bit Rice compression. No doubt
a whole new swathe of users will end up being confused by the new flat file
formats. To migitate some of this MJI asked EGS to check our own web pages
for VISTA to ensure users are made aware of these issues and how to handle
them. <<<<
WFAU have made available access to VDFS processed VISTA science verification
and survey dry-run products through the VISTA Science Archive; though
JPE stated that currently all SV and dry-run data has been released to PIs
without distinction and it is important to maintain the propietary rights on
data. There is some confusion over IDs of OBs partly due to PI dry run data
being taken with SV IDs. This problem will vanish after ESO'isation but
meanwhile JPE and EGS were actioned to drawn up a list of project IDs and
circulate them and MJI will check the status of flagged data transfers to
WFAU to see how much PI dry run data is in there. <<<<
It was noted that ESO policy for all calibration data, including photometric
standard fields, is that it should all be in the public domain asap.
3. Recent and upcoming meetings
There was a VISTA IOT on 1st December for which the minutes have been received.
JRL reports there was not much of import to note other than the minor
detail of the timetable for VISTA acceptance by ESO.
There was an SV meeting at ESO at which much detail of the VISTA processing
and SV results were discussed. The talks (well most of them) are online at
http://www.eso.org/sci/observing/policies/PublicSurveys/docs/. The Orion
SV data science products have been handled by EGS and STH while MJI did the
honours for the NGC253 component.
Among other things a staged release of the science data products was
discussed. Stage I would be the flat file products but due to the limited
bandwidth to ESO, internet transfer of this was taking too long. CASU
agreed to dump all the images and confidence maps to disk and send directly
to the SV team. This was done by the middle of December using a 1TB external
USB drive. The catalogues were copied directly.
There was a recent general email sent out by the ESO SV team covering the URL
http://www.eso.org/sci/activities/vistasv/VISTA_SV.html
where links to the raw data packages, a preview of some pawprint-level data
products, and a report describing observations, tests, and results are all
available.
The next stage is the generation of tiles, production and release of
tile-level band-merged catalogues and deep stacks for NGC253. This will
be available for a planned PSPI meeting at the IoA late January 2010.
JPE said there would be a VISTA session at NAM in Glasgow (12-16 April 2010)
and that he was arranging an RAS Specialist Discussion Meeting on VISTA and
would find out details for the next meeting. <<<<
4. Data archives update
JRL said that he would have completed the UKIRT archive updates before now,
honest, but the power glitches on the 17th took out so much kit that he has
had to pause his final lap.
The AAT, ING and WFCAM archives are up to date after a minor hiatus, but
transfers to ESO are paused awaiting a repair on apm6. MR said that he had
been notified of a minor problem with the RA units for an extra keyword
inserted prior to file transfer to ESO and that this has now been fixed.
VISTA raw data has been ingested up to 2nd December and backed up onto tape
as far as 19th November. JPE warned of another customs strike in Chile on
Dec 21/22 so we may be a month behind observations by early January.
[Added later: in fact a disk from ESO arrived shortly after the meeting,
extending the raw data here to 9th December.]
5. Optical /NIR processing
Not much to report. MJI noted an email from Janet Drew to say that VST would
apparently start operations in the last quarter of 2010. We will see, since
rumour has it that the M1 unit which had corroded on its outward trip to Chile
corroded even more on its way back. MJI asked about the latest estimate for
the dark energy camera upgrade on the Blanco 4m and RGM commented that this
is currently slated to start observing in the 3rd quarter of 2011.
6. WFCAM update
Processing of semester 09B is on track with the usual mid-winter variation in
amounts of data received. MJI will take over processing whilst overseeing
progressive installation of RAID controller and disk upgrades on the rest of
the WFCAM processing kit. Two machines were upgraded earlier in the year as
a test. The benefits include: enough on-board storage to process two large
nights of data simultaneously (always a space juggling struggle before);
a 3 times improvement in disk I/O performance; replacement of some
reliable but now 5 year old disks nearing the end of their useful life;
low cost; and no extra power consumption !
The "new" disks for the upgrade are coming from progressive WFCAM data
storage upgrades enabled by swapping sets of 12 x 400GB SATA disks for
12 x 1TBs in the current WFCAM storage RAIDs which is also a (necessary) low
cost upgrade and likewise has no impact on power consumption (nor cooling
requirements).
MJI reported that he has made some inroads in reprocessing UDS J-band data
for 05B-07B and has completed 46 out of 59 nights. He noted (again) that
from 09A this exercise should not need repeating as it is already built into
the pipeline. When the J are done early in the New Year the UDS folk will be
notified and when they are happy the data will be made available for transfer
to WFAU. The final step will be to replace the earlier versions in our own
on-line product storage system ...... and then begin on the H-band !
A few frames of ancient WFCAM GPS data had their astrometry improved. These
must have escaped the earlier trawl since although satisfying the standard
checks they still needed some more TLC. Running the toolkit astrometry
update software cured the problem without further manual intervention.
7. VISTA update
VISTA preliminary acceptance by ESO happened on the 10th December and an
ESO press release occured the day after. The ESO press release is at
http://www.eso.org/public/outreach/press-rel/pr-2009/pr-49-09.html
and contains some rather stunning colour images.
Receipt of raw data and processing proceeds apace. With a fair range of
PI dry run data now in the can JRL had noted a few liberties with OB
strategy that were causing pipeline problems in sky estimation. In particular
some tile OBs are taking an hour to observe. For these cases he is
experimenting with skies formed from subsets of the tile data with a strategy
chosen according to the observing loop nesting e.g. FJP, FPJ etc..
Otherwise processing to pawprint level is proceeding smoothly.
JPE said that PIPS Phase 2 material was needed by 22 January and suggested
that useful comments or advice for OB design could be fedback to the PIs
though quite how is another matter.
There was discussion with Phil Lucas and the VVV team about how to deal with
sky estimation on the Galactic Plane (sound familiar?). Tests on single tile
sky estimation strategies show the expected residual lumpiness due to
the impossiblity of completely removing stars using 6 (paws) x 2 (jitters).
Concatenating tiles for sky estimation is a possibility and is being trialled.
MJI noted he is a bit worried that the sky estimation strategy is becoming
too complex and debugging it in full pipeline mode might be challenging.
JRL had spotted an interesting faux pas on the night of 19/20th November where
it transpired that the the cables of detectors#5 and #6 had been switched to
see if channel#14 problems on detector#6 was caused by a connector problem. In
fact it wasn't and the problem was traced to a video card, replaced on the
20th November. More worryingly CASU were not informed that the cables had
been switched, and there is unsurprisingly no information in the FITS headers;
furthermore it is quite likely that switching video cards may have altered the
non-linearity. We now urgently need to do a new good linearity sequence
to assess this.
There was another night with seriously chewed up data (9th November).
Herewith extract from log:
"-- NOTE ADDED NEXT AFTERNOON (Nov. 10): During the day it was found
that the instrument cryocooler had been off since practically the
beginning of the night (PPRS 033405), resulting in detector
temperature increase by almost 30 K during the course of the
night. The corresponding increase in noise affects both science data
and the morning darks (more the latter than the former). All
science OBs observed during the night have been reclassified as "Must
Repeat". Unfortunately, no alarm had been triggered (PPRS 033413)."
JPE has asked for an audible alarm to be fitted otherwise it is difficult to
notice these problems in real time.
JRL said that he and Steven Beard had been discussing a problem with jitter
offsets repeating when making sequential standard observations. SB was
unable to now log in remotely so it is proving difficult to replicate
and fix the problem.
Several changes have recently been made to the catalogue software to fix
various problems specific to VISTA stacked data due to correlated noise and
to upgrade some of the parameters. These are now being tested in the latest
pipeline products and will be backpropagated to the earlier catalogues from
27th September if ok.
EGS said that quality control information is ingested on a regular basis and
a password-protected web-based browser available.
By the way it was JRL who spotted the org/asm feature of the ESO web pages
so don't blame us.
8. Any other business
The apm colour printer is deceased, indeed it is no more. SC will investigate
nailing it to its perch and the purchase of a new colour printer along the
lines of colour5 in the Hoyle. The cost of this will be shared between
various CASU grants and DWE. <<<<
SC noted that there will be a general lockdown on the Institute computers over
Christmas to counter hacking attempts. There was some discussion on the
ramnifications of this but as it doesn't directly affect the CASU kit
this may not be much of a problem.
The consequences of recent severe power glitches are still being sorted out
- 5 PSUs and several RAIDs needed attention if not downright replacing.
The web pages for remote monitoring of network activity, temperature monitoring
etc.. need some TLC after the recent hiatii. SC will investigate. <<<<
MJI noted that interviews for the PSB replacement post will take place at the
IoA on 2nd February.
There was some discussion about the increased costs of electricity consumption
and the change in charging policy that the University has now adopted.
The extra GAIA hardware and air conditioning units have been the main
contributors to a 20K per year cost increase from the baseline comparison of
2 years ago. CASU/IoA will have to foot this bill. MJI has been banging
on about using natural cooling especially in the Winter months for aeons and
if he gets a cut of the savings will do something about it.
Continuing actions
------------------
EGS - add catalogue software and binary table conversion software to
MJI software release pages.
MJI - finish off technical paper updates
MJI - finish off dome flat comparison
JRL - finish off low light level comparison
SC - continue backup plan investigations
JRL - carry on with calibration plan doc updates and check if we need any
STH specific calibration data
New actions
-----------
RGM - soak test the catalogue conversion software
JRL - add some meat to the persistence web pages and add something about
EGS "holes"
EGS - add Rice compression information to VISTA web pages and make sure
binary table WCS information is readily visible
MJI - check the dates of the VISTA data that WFAU have copied
JPE - find out about the details of the RAS meeting on VISTA
SC - investigate purchase of a new colour printer.
