—
Plain Text,
15 KB (15473 bytes)
File contents
16th April 2009 APM Meeting tables 11:30-13:00
Present: MJI, JRL, STH, MR, EGS, RGM, SC, RWA
Apologies: NAW, DWE
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. Recent and upcoming meetings
4. Data archives update - AAT, ING, WFCAM, UKIRT
5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru
6. WFCAM update
7. VISTA update
8. AOB
Minutes
-------
RWA has taken over responsibility for arranging monthly group meetings and
has also taken over as minute-taker extraordinaire. Thus .......
1. Actions from the last meeting
STH check that Plone technical description pages are up to date <<<<
MJI RWA will help edit the content, needs edit access sorting out.
JRL reported that he had a beautiful image from HAWKI and STH said he
STH is designing a website to put it (and others like it) on. He <<<<
will also add something about the HAWKI photometry. There is a
zoomable browser used for Spitzer images which seems to work very
well and which emphasizes the splendour of the HAWKI images which
are of very high quality. JRL has re-processed all the data now and
MJI said he would look at the photometry again. <<<<
MJI finish off stage-I software release pages. <<<<
EGS
MJI finish off technical paper. MJI said he was not going anywhere <<<<
in May and would devote a week of time to this task.
RGM liaise with JRL about processing HAWKI PI data - continues <<<<
JRL redid the HAWKI NB image data processing to improve sky subtraction
JRL are still sorting out remaining VISTA documentation set and making
STH sure latest copies are available on internal web pages. JRL has all
RWA but one current document in place, the ETC. The current version cannot
be found online. However, the changes from the previous version are
minor and will be added. MJI asked for all the docs and pdfs to be
placed on the internal web pages. JRL exhorted everyone not to modify
them and noted that the lead author should be responsible for any
updating. There is still a lot of stuff in PSB's home directory not
completely sorted out - ongoing <<<<
MJI write utility to convert binary tables with all corrections
applied and dump as ASCII file - ongoing <<<<
JRL send latest version of cataloguing software to WFAU - not yet done <<<<
MJI contacted JPE about CASU partaking in VISTA IOT March telecon
NAW submitted abstract for CASU talk at JENAM. EGS is also giving a talk
JRL thinks he implemented the sky method FITS keyword in current pipeline
and just had to back propagate it to all the 09A data <<<<
EGS inquired what language UVEX team are using for coding. The answer is
C and he will provide an example demonstrating reading from merge <<<<
binary table.
NAW send polite but firm email to IPHAS about VIZIER data release ?? <<<<
MR is chasing up wherewithall to produce duplicate WFCAM transfer script,
nice though it is to still get emails from PSB. <<<<
STH will inquire about possibility/status of new WFCAM linearity measures
when he goes out observing next week or so <<<<
MJI forwarded UDS test results to JRL
STH figured out the problem causing the apparent non-linearity when
comparing VISTA with 2MASS: pawprint skies from limited dithers +
object masking + big aperture = FU
2. WFAU minutes
MJI noted there have been two sets of minutes since the last CASU meeting.
DR5 happened on April 8th, GPS & UDS are not yet included but will be
at some date still TBD. WFAU have transferred and ingested all the 08B
data. Prior to DR6 the assorted Petrosian et al. fluxes and radii for
blended objects need fixing for 05A-08A, 08B and 09A -> already have the
fix applied. Timescales for DR6 are still uncertain.
RGM noted no LAS objects appear to be flagged as saturated (i.e. with class
= -9) in the DR5 data release. MJI commented that this was an issue raised
well over a year ago and the solution put forward then was to use the
detector-level saturation keyword/value in the FITS headers, together with
the peak intensity of an object and the local sky level to decide which
detected objects are likely to be saturated. This is deliberately not done
in the catalogue table entries since the processed image saturation level is
not uniquely defined, making this a "fuzzy" decision. CASU set this flag
during band-merging using a simple conservative recipe. MJI recalled some
email discussion with WFAU along these lines a few years ago but couldn't
recall the outcome.
3. Recent and upcoming meetings
There was a VISTA Instrument Operations Team (IOT) telecon on March 26th.
MJI and STH partly attended while JRL sat through the whole thing. CASU
were asked to produce a brief report on their analysis of the VIRCAM data
taken prior to the end of January and JRL was asked about his advice on how
to proceed with enabling an offset sky strategy. Ideally this report should
be put together prior to the next IOT meeting scheduled for the 28th. MJI
will mainly deal with astrometry, STH photometry and JRL image processing
aspects. This needs to go be sent to JPE/VPO in plenty of time for the IOT
meeting, which means in practice by Friday 24th if possible. <<<<
The minutes of these IOT telecons will be placed on the VDFS internal
web pages (VISTA/misc). <<<<
STH and NAW attended a PLATO meeting in Aarhus, but with beer at 8 quid a
pint there the travel budget has shrunk considerably.
The same duo will attend a PLATO consortium meeting on June 5th in Rome
and while tpying these minutes news came in of a PLATO Science Consortium
meeting in Belfast on the 12th June, a busy time for Platonics.
EGS reported nothing of note happened at the Euro VO workshop last week held
at ESO, Garching.
STH had a week-long meeting with folks from ASTEP who seem happy with
our contribution = astrometry and photometry sorted - not much then !
They have 8 x 8 degree fields taken unfiltered in the optical with exposures
every 30 seconds, even with the Sun above the horizon, and also a nice movie
of the Aurora Australis. Used TYCHO2 for the astrometry. Summary of this,
including movie, needs putting on a webpage, somewhere near you. <<<<
4. Data archives
AAT: data trickles in
INT: more of a stream - usual occasional header problems, legal ASCII
characters set, moi!
WFCAM: data transfers (in and UKIDSS/CAL out to ESO) and ingests continue
apace. MR noted an uncommon recent request for raw data, from Jim Geach,
again.
UKIRT: 95% of DLTs re-read - just a few composite DLTs and LTOs to
absorb. JRL noted the latest batch of CASS data has yet to arrive.
5. Optical/NIR processing : HAWKI, INT WFC, MegaCam, Subaru
HAWKI - reprocessed, recatalogued and awaiting magic from MJI
INT WFC - not much to report, processed some SNR data for Albert Zijlstra
Megacam - plus ca change
SUBARU - ditto
IPHAS - ditto
UVEX - processed and ingesting to database. EGS is dealing with export of
data products to UVEXers.
6. WFCAM update
Semester 09A has been dominated by rubbish weather but at least this
means that it was easier than expected to keep abreast of the processing.
At the time of this meeting we were still waiting for nbH sky flats but
had decided to press on with the reductions using ones from last Autumn.
[By the time these minutes were written, 20090416! produced a good set of
nbH sky flats which have subsequently been used to update the cal files
for most of the April data.] MR noted that one earlier night, 20090312,
had been on hold for a while due to a rather odd set of channel#4 FITS
headers. Apart from the 1st 8 header keywords, the rest were all missing.
This was eventually bodged by a suitable workaround. MR will check
with JAC if the same problem was apparent on the original SDF files. <<<<
[e.g. w20090312_00245.fit see README in processed data directory for full set]
MJI noted he is compiling a UKIRT Board report for the upcoming May
meeting and STH also noted up that he is submitting an update on his
WFCAM campaign project. <<<<
MJI asked about the SDF->FITS conversion/compression feature - the raw
files still crash ds9 though have no other deleterious effects. JRL will
check with JAC if the version of CFITSIO used in the conversion has been
updated. <<<<
MJI reported that WFAU had ingested both 08A and 08B without problems
and that he has been talking to Nigel Hambly about encoding in the average
confidence levels (for aperture 3) for objects in catalogue flat files,
in addition to the Petrosian et al. flux fix for blended objects noted
earlier.
On the reprocessing front, MJI reported that we have now finished UDS K-band
reprocessing up to and including 07B with the plan that this will be stacked
by the UDSer's, by sometime in May and make it into the UDS DR5 release.
J and H are on hold until the GPS nebulosity filtering exercise, required
for GPS DR5, is sorted.
MJI verified with JRL that the overall computed offsets per pointing
were now routinely being used in the pipeline when dither stacking.
This is to eliminate the odd problem noted previously where one detector
goes awol while the rest are fine.
MJI and JRL noted that Thomas Robitaille (CfA) had rediscovered the not
quite complete back propagation of the WCS for ....sf_st'd file component
parts (it only goes back one layer). It turned out Thomas wanted to restack
(essentially drizzle) the single processed images to get rid of the "hedgehog"
PSFs arising in 2x2 interleave stacks. MJI pointed out that there was a
simpler solution namely dribbling the final stacks, which works just as
well and of course is much faster and less trouble. For the sake of
posterity the solution implemented, and used quite frequently on request,
in a CASU toolkit prog (fitsio_dribble) is outlined here: ................
If interleaving has been used, we optionally use a drizzle-like algorithm
to correct the PSF mismatches that occur in interleaved data, making use
of the fact that the real pixels are 0.4 arcsec in size and overlap the
neighbouring pixels in a 2x2 (or 3x3 if used) interleaved 0.2 (or 0.133)
arcsec grid. This runs directly on the pipeline output interleaved stacks
and gives much better behaved PSFs and, of course, correlated noise. Or put
another way, each pixel in a stacked set of interleave images (e.g. 0.2 x 0.2
arcsec) actually contains the flux of a pixel that is 0.4 x 0.4 arcsec
centered on that position. So in effect, neighbouring pixels actually overlap,
and we "dribble" these "big" pixels onto the underlying grid - comprenez ?
7. VISTA update
No new VISTA data has been received since the last CASU meeting since VIRCAM
has not been up and running on the telescope. Nonetheless disks still
relentlessly flow from ESO each containing about 50 Mbyte of log files (of
what?) at a transport cost of about 70 pounds per disk.
It transpires that the summit pipeline at Paranal will look at the QC
info for all the detectors. But due to real time speed constraints it will
instead randomly sample the observation sequence rather than examine them
all. This is not dissimilar to what is planned for the Garching pipeline.
JRL gave us an update on the status of detector#16 and the assorted trials
and tribulations he has been through to demonstrate its properties.
He has also being doing some extended mosaicing tests with MONTAGE which
could form the basis for at least a benchmark tiling algorithm for VIRCAM
(still TBD). Related to this is the still open issue of what extra tiling
keywords (and inheritance of keywords) will go in the tile files.
Whilst he had the floor, he also reminded us of the ancient, and as of
yet, still unresolved question of RECIPE keywords for the different
surveys/eventual PI programmes, e.g. which of the three on offer
interpolation schemes ? At this rate these will be hard-wired in for
each of the surveys after consultation with the PIs.
VISTA parameters webpage. JRL said this is currently stalled and needs some
action from MJI who will look at the VISTA reduced data and make sure
that standard summaries a la WFCAM can be generated. <<<<
MJI noted a date for our diaries, JPE has booked the RAS Specialist
Discussion Meeting slot on 12th November 2010 for VISTA science.
8. AOB
MJI said that he anticipated no further news about the grant until after the
Budget and is expecting a Healey tax 'em until the pips squeak. He noted
that it will be time to write the next rolling grant in 10 months time !
Interviews for the advertised posts have now been concluded and suitable
candidates chosen. Start dates etc. are still being negotiated but are
unlikely to be for at least another 3 months.
Two additional RAID arrays have been installed. One fell off the back of
the usual lorry and is otherwise known as /data/apm22_e (14 TByte of
actual RAID 6 level storage). The other will be affectionately called
/data/apm35_a (30 TByte of actual RAID z2) and is the other half
of the i/o for VISTA data for this decade. This one will, after some
shuffling be configured to hold the raw data, the other /data/apm29_a
after similar concommitant shuffling will house the reduced VISTA data.
MJI will sort out the shuffling <<<<
Continuing actions
------------------
STH with RWA's help check that the Plone technical description pages are
MJI up to date
JRL produce web page of HawkI publicity for CASU
STH
MJI finish off stage-I software release pages
EGS
MJI finish off technical paper
RGM liaise with JRL about processing HawkI PI data
JRL sort out remaining VISTA documentation set and make sure latest
STH copies of docs and pdfs are available on internal VDFS web pages
RWA
MJI write utility to convert binary tables with all corrections
JRL send latest version of cataloguing software to WFAU
JRL check if you have implemented the sky method keyword for 09A
NAW send polite but firm email to IPHAS about VIZIER data release
EGS will provide an example C program demonstrating reading from merge
binary tables
MR finish chasing up wherewithall to produce duplicate WFCAM transfer script
STH inquire about possibility/status of new WFCAM linearity measures during
WFCAM visit
Actions
-------
MJI finish off HAWKI reprocessed data by nebulosity filtering and
recataloguing
JRL prepare report for Vista Project Office in advance of next IOT
STH telecon on April 28th
MJI
MJI put IOT minutes on internal web pages
STH put ASTEP info on webpage
MR check with JAC to see if SDF files have same detector#4 header problem
MJI prepare a report for UKIRT Board meeting by end of April
JRL check if JAC version of CFITSIO has been updated.
MJI produce summaries a la WFCAM of reduced VISTA data
MJI sort out VISTA i/o disks
