Tuesday 16th February 2010 APM Meeting tables 12:00-13:00 Present: MJI, RWA, EGS, NAW, RGM, MR, SC, STH Apologies: JRL, JPE Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Recent and upcoming meetings - VISTA IOTs - VISTA PI meeting in Cambridge 4. Data archives update - AAT, ING, WFCAM, UKIRT 5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru, VST ..... 6. WFCAM update 7. VISTA update 8. AOB Minutes ------- 1. Actions from the last meeting MJI stage I software release still on hold - ongoing <<<< EGS JRL has produced a standalone catalogue software tarball which will be incorporated together with the example piece of code for applying all the corrections from the catalogue binary FITS tables. MJI technical paper - no further progress but there is a cunning plan ........ MJI reported that the dome flat comparison has not yet been finished <<<< JRL likewise has not finished the low light level linearity tests <<<< He has done some related work with the new waveform test data but the linearity tests require further effort and/or better data. SC reviewed the backup plan for the main cluster and outlined a possible solution for the remaining kit. MJI noted that most backup tapes are now housed separately in the Observatory building so other than for a major catastrophe (in which case we wouldn't care anyway) we are sorted. EGS confirmed that the example of persistence in VISTA, and other recently derived information discussed during the PSPI meeting in Cambridge, have been put on the technical web pages. STH said that he and JRL are working on the updates to the calibration JRL document. JRL managed to find the latest copy from a backup of PSB's and STH will update the plan to include 2MASS calibration details. He also wanted to know how much information to include on the more subtle aspects of this. MJI suggested that he includes a brief description of these and then refers the interested reader to the WFCAM MNRAS paper (Hodgkin et al. 2009) for more details. This would also be a good opportunity to compute and publish the latest colour equations including any Vega zero-point offsets, AB magnitude conversions etc.. We would like to be close to signing this one off by the next meeting <<<< RGM has not yet destructively tested the catalogue conversion software <<<< EGS added the Rice compression information to the VISTA web pages for those users who are not familiar with the WFCAM ones. There was a brief discussion about which other software packages seamlessly supported this and the, now standard, table WCS format. MJI checked the status of the VISTA transfer to WFAU and JPE and EGS collated and supplied the list of project IDs SC reported that apmcolour has now been replaced with ... err ... apmcolour. The cost would be split between VDFS and GAIA, and Dafydd. JPE forwarded details of the VISTA session at NAM and succeeded in confusing everybody about the timing of the session. Fortunately the website is now online and in real time RGM checked that the VISTA session is during the afternoon of the 13th April. SC still needs to dish out some TLC to the assorted web pages for remote monitoring. Whilst on this topic she noted that there had recently been a call from security to say that one of the high temperature alarms was going off. This was great apart from the fact that the temperature was fine and nobody told us (until this meeting). Clearly a few wrinkles to iron out then ! <<<< 2. Comments on the WFAU minutes It was noted that PMW is due to retire ~end of April and we thought it would be a good excuse to mark this occasion with an appropriate knees-up. MJI will contact NCH and see what plans there are afoot. <<<< MJI noted that NJC's WFCAM standards star timeseries database is now available and encouraged STH, among others, to take a peek. JRL reported a comment from the minutes of 20100115 suggesting that provenance keywords are missing in VISTA confidence maps. He will look into this, but would appreciate being directly informed about it. RGM noted that the date for the next DR data release (DR7) was a tad vague and wondered if there was a target date set. 3. Recent and upcoming meetings MJI said that there was nothing of import to report from the last IOT and JRL concurred (offline). The next one will be in two weeks or so. MJI commented that the PSPI meeting in Cambridge seemed to be very successful and offered up the following list of items that needed investigating and/or acting upon. In no particular order and by no means complete: versioning in each FITS file to avoid ambiguities half-light radius and flux also useful as classification check the CASU ETC seems to give erroneous Z-band predictions other differences between ESO and CASU ETC most likely aperture corrections FITS header items missing/desirable e.g. containers, concatenations, recipes conversion to AB mags cf.Blanton et al. 2005 2MASS; Hewett et al. 2006 WFCAM colour equations and extinction corrections (for latter see zpts) how to deal with the "ears" of tiles what to do with the top 1/3 of detector #16 seeing -v- sec(z); sky SB -v- moon phase/distance -v- twilight is resampling within dither stacks to finer grid worthwhile completeness estimates - scaling relations ?? tiling strategy - nebulosity filter, tile, then catalogue how many PI surveys really need "naked" tiles ? There will be a session on VISTA (status and science) at the upcoming NAM in Glasgow in April. It would be good to have a CASU presence as this even though the main focus will be on early survey science results. Maybe the SV results which CASU are involved in would be a contender. <<<< 4. Data archives update AAT, ING, WFCAM, VISTA: all going smoothly UKIRT: JRL reported that he was still awaiting the last 20 or so nights of missing data. The rest of the port is in fact done (consumed?) and being backed up even as we speak. Once JAC have come through with the last bits of missing data he will dot the i's and cross the t's and underline the z's. 5. Optical /NIR processing JRL reported he has just done some restacking of Hawki data for RGM + student. Apparently there are a lot more waiting to be done - que? MJI relayed some news about the VST received via Janet Drew. Apparently the next attempt to get the M1 mirror cell shipped to Chile is due to be in May, and the stated cunning plan is to have the telescope assembled and pretty much commissioned by the end of 2010. The summoning of survey representatives to ESO Garching is presently being pencilled in for June/July ! A pile of PAndAS project MegaCam data has been slowly winding its way through the system and has now been stacked, pummelled, catalogued, and otherwise processed out. Further IPHAS and UVEX processing is on hold until we get a breather. 6. WFCAM update MJI and JRL report that JAC have been experimenting with subtle changes to waveforms for the the WFCAM detectors in an attempt to minimise channel-edge effects, parquet floor patterns etc.... A set of test data from 20100129 was fast-track processed by MJI and JRL spent a while straining his eyeballs to assess the effects. Of the 10 different waveforms, a mixture of 4 CDS and 6 NDR, there were only two sets (current and a new one) that were worth pursuing further. Further darks and linearity sequences were done with these and reported back on. We are currently waiting now to see whether/when these new waveforms will be implemented. In parallel a small slew of other hardware work has also been carried out in an attempt to correct some of the long-standing quirks of the detector arrays. It is not yet clear what, if any, the knock-on effects of these hardware changes will be but it will be worth keeping an eye out for any changes as of 1st February processed data. WFCAM is currently running on the "old" waveforms, the implementation of the new has been deferred for now (note added in writing up: now = 20100218). MR reports slight delays, partly because of lack of available disk space, partly due to the large amounts of data being received - recent nights have contained more than 3000 exposures, and partly due to time spent looking at a lot of extraneous interrupted LAS files. MJI added that there had been a slight upus-cockus with delivery of some 1TB disks which are going to be used as a cost-effective (and Green) measure to swap out 2 sets of 12 x 400 GByte disks to increase WFCAM storage space. [As usual when reporting these things, the disks turned up shortly after the meeting concluded]. Meanwhile a chunk of VISTA storage space has been purloined temporarily and normal business will be resumed as soon as possible. There have also been the odd few problems with the recipe keyword in the fits headers but nothing untoward otherwise. MR added that Brad, who among other things, had been handling the data transfers at the JAC end, had now left and his replacement Anubhav Agarwal (of the philosophical email musings) has taken over this aspect. MJI noted the STFC intent to withdraw from UKIRT at the end of 2010 but given both the difficulties of achieving this and the community desire to continue the UKIDSS surveys, somehow doubted that the end of 2010 would be the last we see of WFCAM data. Current CASU planning, both financial and staff effort are in place to deal with WFCAM data processing and user support until 31st March 2012. MJI added that the J-band UDS reprocessing for 05B-07B was finished. It just needs some of his CPU time to finish of the monthly calibration updates. There remains the equivalent 56 nights of H-band data to do but this is a long way down the pecking order. DR7 jpeg checking is appparently proceeding apace. The only feedback received so far has been from SJW who noted that 1.5% of LAS data were being deprecated, but the vast majority of this was due to trailed images. 7. VISTA update EGS reported that we have now added a new keyword in the primary header of all fits products, CASUVERS, which stores the CASU product release number. Since the release of version 0.6 to WFAU and to the PIs we have made several improvements and fixes: the sky rms is now corrected for the average noise covariance properties; a small problem with the astrometry of stacked VISTA images has been fixed; the Hall flux and radius have been replaced by the half-light flux and radius; we have added the average confidence in the default aperture for each object (column #58) to the tables; and we have fixed a problem with the rejection algorithm on the stacks. Unfortunately this took two iterations (version 0.7 was deprecated for internal use only) and we are currently (re)reprocessing all the data from Oct 15 2009 and when finished it will be tagged as version 0.8. JRL sent in a short report which after removing all the expletives was even shorter: we now have raw data upto and including 20100203 but because of the reprocessing are a couple of weeks behind with the normal processing. The ever prescient MJI had already asked SC to get a quote for more Sun Blade CPU's to fill the remaining 5 slots in the VISTA processing kit. SC noted that as Oracle have now taken over Sun, all the educational discounts have to be (re)checked which is delaying receipt of a quote. At some (TBD) future stage we will also need to conjure up some more VISTA disk space but will delay making that purchase as long as possible. JRL noted that he has also received an update from ESO on how the information about linked/concatenated/cojoined OBs will be propagated via the FITS headers and will hopefully enlighten the rest of us. He also reported that version 1.0 of the summit and Garching pipelines and version 2.0 of the DRLD have been released. All of the important ARS tickets have been resolved (good job really). RWA reconfirmed that data up to Feb 3 had been received and ingested and that data up to Feb 2 has been backed up onto tape. ESO apparently want the disks returned more quickly since they have only bought a small no. of them (d'er!) We had, of course, been sending them back ~3 at a time to save posting costs. There have been quite a large number of upgrades to the technical pages and MJI urged the assembled hordes to check that the right data is available in the right place e.g. there was some discussion during the meeting regarding the whereabouts of the colour equations and zero-points. MJI noted that chip #14 has a marginally dodgy channel #11 amounting to a few ADU, but although it is being monitored, it is not considered a significant issue at this stage. MJI said that the UltraVISTA team are worried about residual haloes around bright stars, among other things, and want to trial external sky object masks. JRL has been working on this and will send then a "template" for how this can be most easily achieved. MJI is still skeptical about the utiliity of this, particularly for halos of bright stars done using a pawprint sky method, since we will end up with regions that are not sky-corrected at all. As a glance at virtually any sky correction map will reveal, this is generally not a good idea. <<<< MJI and Henry McCracken have been discussing AB magnitude conversions. As a starting point, M sent H the conversions for 2MASS and WFCAM since we expect only minor differences for VISTA. Henry worked out the VISTA Ks offset to AB using the filter and detector QE curves, which turned out to be 0.01 mag different to the 2MASS published Ks conversion - yup minor. STH will revisit this and incorporate all of it in the VISTA photometric calibration paper MJI reported verbally about an ongoing STFC-ESO exchange relating to contributions in kind in lieu of dosh for VISTA late delivery. Anybody fancy living in Garching ? ESO had another PR release of the CASU-generated colour images, this one was the M42 beaut. Presumably this means we can now make our colour versions of this and the other released PR images available. EGS will make it so. <<<< 8. AOB MJI gave a summary of the news about the various grants that CASU people are involved in this round and those that have been reviewed as part of the STFC hatchet job. The only ones vaguely smelling of roses are the CASU roller and the VISTA exploitation grant, and even these are subject to 15% cuts - though quite what that means in practice is still unclear. After the job interviews in early Februrary an offer has been made to fill PSB's post and we hope to have more definitive news next week. There was some further discussion about backup strategies for software on assorted machines and SC will continue looking into various options for this. In an attempt to be even greener than usual, and to forestall ever spiralling electricity costs, Peter Weeks has been asked to investigate installing a a louvred fan to externally cool the main computer room in cooler weather. If this works the ~5K/yr likely savings should provide enough beer money even for us. Continuing actions ------------------ EGS - add catalogue software and binary table conversion software to MJI software release pages. MJI - finish off dome flat comparison JRL - finish off low light level comparison and linearity assessment JRL - carry on with calibration plan doc updates and produce a new draft STH for discussion at next meeting RGM - soak test the catalogue conversion software SC - provide a bit of TLC for the remote monitoring web pages SC - report at next meeting on cunning backup plan New actions ----------- MJI - contact NCH and see what plans are afoot for Peredur's retirement ALL - consider suitable talks for VISTA session at NAM and discuss with MJI well before abstract submission deadline JRL - make a pre-emptive strike to fix the format of external object masks EGS - plonk CASU versions of VISTA PR (released) colour images on web pages