minutes_100922.txt
minutes_100922.txt
—
Plain Text,
12 KB (12921 bytes)
File contents
Wednesday 22th September 2010 APM Meeting tables 11:30 - 13:00
Present: JCR, MJI, STH, EGS, JRL, AKY, MR, NAW
Apologies: RGM, JPE
Agenda
======
1. Actions from last meeting
2. Comments on WFAU minutes
3. Meetings and Telecons
4. Data archives update - AAT, ING, WFCAM, UKIRT, VISTA
5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru, VST
6. WFCAM update
7. VISTA update
8. CASUhelps
9. AOB
Minutes
=======
1. Actions from the last meeting
--------------------------------
JRL CASU tool releases are up to date and at v1.06
JRL NBSIZE parameters have gone into the FITS header
JRL has done all the updates he can on the calibration plan
STH needs to revise the colour equations offline and finish off <<<<
JRL the latest DRLD has been sent out
MJI has added an illumination table option, action passed to
EGS to link WFCAM tables to web pages <<<<
RGM still testing catalogue listing software but sent in a note
regarding astrometric differences at the 10mas level. MJI noted
that these are undoubtedly caused by use of 32-bit arithmetic <<<<
JPE text to go with CASU versions of VISTA PR images - ongoing <<<<
JPE contact ESO re: signing off pipeline deliverables - ongoing <<<<
though he is signing off the DRLD and sending it to ESO
EGS have made good progress with the SV v1.0 reprocessing and will
MJI discuss with the 3M's ++ at ESO as part of the VST meeting visit <<<<
JRL has tied up his loose ends re: simplesky -> tilesky
AKY is investigating various aspects of the ellipticity trends as
part of an IOT action item (see below) - ongoing <<<<
STH investigated the issue of CASU authorship on the Carina papers.
It was decided to not be appropriate except on the initial papers.
He will compose and send off a suitable email also commenting on
the calibration errors <<<<
JPE didn't quite manage to find out the date of the VISTA PSPI PhaseIII
workshop but this was divulged during the VST meeting (30th November)
and hence no longer an action item
JRL found missing HAWKI data - where is it then ?
MJI WFCAM calibrations in regions of high extinction investigation is
STH still ongoing <<<<
AKY have provided feedback to PIs on derived QC information etc .....
EGS in the form of FITS tables in addition to the usual sql dooberrys
2. Comments on WFAU minutes
---------------------------
WFAU have been radio quiet for last week as a result of a power outage
on Blackford Hill caused by some workmen with a drill and a bad aim.
RGM informed us before the meeting that WFAU apparently need the CASU tiling
software to make their band-merged catalogues, quite how and/or why we can't
figure out, but the basic tiling software has now been made available anyway.
3. Meetings and Telecons
------------------------
On 07/09/10 there was a meeting to address GAIA computing requirements at the
IoA. The upshot is that in ~a year there will (may) be 2x4 racks of
computing space available in the old dog kennels at the Vet School
(as opposed to the 2 X 16 racks available for finance !!). It will have a
halon gas fire extinguishing system which means people will have to
enter the building through the dog flap in pairs for 'elf & safety.
STH said he had a very nice time in Portugal and enlightened JENAM'ers
about VISTA processing, nebulisers, calibration, lightcurve generation
and no doubt much else.
On 28/29 September there is a VST review at ESO. EGS and MJI will be
attending to talk about the CASU VST pipeline and processing experience
en general, as well as talk to the VISTA SV teams.
NAW pointed out that due to several upcoming large spectral surveys
(e.g. FLAMES GIRAFFE/UVES, possible new wide-field MOS's etc....) there is a
growing need for a half-decent spectral reduction pipeline. This is something
that we have done in the past, at least to the extraction and wavelength
calibration stage; while more advanced spectral products for GA are an active
area of research within CASU. Let us see.
4. Data Archive
---------------
ING, AAO, UKIRT, VISTA all up to date and motoring. JRL sorted out the
delayed (at JAC request) few nights of CASS UKIRT archive and they are
now also available.
5. Optical / NIR Processing
---------------------------
Everything ticking along nicely apart from the IMACS data which has stalled
due to advanced boredom with having to solve yet another astrometry setup.
MJI has been investigating a scattered light issue in MegaCam, triggered by
being a rapporteur for a SNLS PhD but also as part of the preparation for a
PAndAS (M31 survey) meeting. He now understands the problem well enough to
figure out a post-processing correction table to apply to the catalogues.
EGS and Janet Drew have been exchanging some fascinating e-mails about
which VEGAs system IPHAS is calibrated to.
JRL has finished processing RGM's HAWKI data on the GOODS fields.
Action on MJI and EGS to develop a VST pipeline plan before VST meeting, <<<<
or at least be amazingly persuasive at same.
6. WFCAM Update
---------------
All available 10B data has been received as expected and has been ingested
into the raw data archive. Transfers to ESO are also up to date.
September processing is a tad behind due to delays in getting a complete set
of flats. Otherwise everything has been routinely pipelining and progressing
as expected.
The August data was flagged as ready to copy on September 16th and despite
the best efforts of the workmen in Blackford Hill turned up at the WFAU
archive end just over a week later.
No requests for raw data and the only early access request for processed
data was from Sandy Leggett.
Omar and the UDS team noticed an interesting stripey pattern in J & K-band
deep stacks in the UDS data prior to DR8 release. MJI investigated the
problem, ruled out decurtaining due to the symmetry being wrong and tracked
it down to a less than 1 part in 10000 effect visible in individual MSB
stacks if you know what to look for. This looks like a (very minor) hardware
feature whereby brightish stars in each quadrant cause a slight depression, or
sometimes a slight rise, in the background level along nearby rows and/or
columns. The effect is isolated in quadrants suggesting it is a readout
feature. Mind you at this level if only becomes really noticeable when you
make deep stacks. In the K-band, for example, the surface brightness of
these stripes is between 23.5 and 24 mag/sq arcsec. Unfortunately as the
effect is "attached" to the stars it does not go away on normal stacking
and is probably best corrected during intermediate depth stacks (to minimise
quadrant overlaps). This has resulted in a delay to UDS DR8 although begs
the question of why this effect was not spotted before.
7. VISTA Update
---------------
VISTA data up to the end of August has been received and processed data up
to 7th August has been released. All data up to 20100901 has been processed
and we are now up to the last couple of nights of post-processing.
MJI noted that apm29a is fast running out of disk space and asked for a tidy
up of non-VISTA processed data from there.
The SV NGC253 data that was mistakenly processed with the 'simplesky'
algorithm has now been reprocessed yet again, but this time with the
original 'tilesky' option. This is hopefully it, and the latest latest
version has been made available to ESO (Magda) and WFAU. Meanwhile EGS
and MJI have been finishing off the rest of the SV post-pipeline processing
and will discuss results with the 3M's during the ESO VST meeting.
Matt Jarvis has observed 5 nights with the dual narrow band (975/985) filters.
The data is expected at our end anytime soon.
The odd few NaN/infinity/negative CRVAL issues++ have been investigated.
Negative CRVALs crop up from the code not checking for offset sky
exposures. The infinite values were caused by a bug in the overlapping
routine. JRL has fixed both of these.
The QC tables are now available on the CASU website with notes explaining
column headers and the FITS files. We just need to tell the PIs where all
this bounty is.
AKY has been investigating why VVV and VHS sometimes have larger than
acceptable ellipticities. It turns out that these occur most frequently during
camera reshuffle periods (October last year, May this year..). Outside
of this problem the other main culprits are trailed images due to poor
tracking, primarily because the A/G hadn't settled before the OB sequence
began. Meanwhile AKY will contact Marina and see what further information
ESO want in this regard.
VVV have obtained w/e use of a dedicated "large" bandwidth line from
Santiago, Chile, during which they downloaded (or not) the newer tiles and
catalogues not included on the shipped USB disks.
It was decided that the data releases to PIs will be monthly from here on
to simplify bookkeeping at both ends.
JRL has been doing further work on the issue of apparent scattering in the
detector zero-points raised by WHU at ESO. We are confident this is a
non-issue and MJI sent a detailed explanation of why sometime ago.
There followed a brief discussion about processing or otherwise of incomplete
OBs and how to keep track of the process. Currently incomplete OBs are
processed as usual providing there are enough frames to compute a reliable
sky estimate. Grossly incomplete OBs are not processed. This raises two
issues: how to keep track in our QC database of which ones are not processed;
and flagging more directly in the science products that only "m" out of "n"
jitters are present in the stack. Although the provenance information in the
headers allows you to deduce this, it was felt that a specific keyword should
be added to make it easier to "grep" for. <<<<
Finally MJI suggested that a 'known features' page on the website
might be useful for noting minor gremlins/gotchas and pre-empting some of
the CASUhelp e-mails. EGS with helpers will make it so <<<<
8. CASUhelps
------------
Somehow MR mysteriously acquired exclusive ownership of CASUhelp and
CASUreply email folders recently. This has been sorted but we have no idea
how it happened.
The other mr. confused everybody (CASU+WFAU) by managing to be browsing our
web pages here as some June data was flagged (red -> green) as ready to go
and then getting confused about dates as it turned out to be a repeat OB of
earlier (Feb) data.
ESO seem very keen on the PA of stellar ellipticities so we might have to
give in and compute an average PA for each detector and plonk the results
in the FITS headers.
...... and the contender for query of the month
"it seems that the astrometric solution was only applied to the WCS positions
(ra, dec) and not to the pixel positions. Would this be correct?"
yup.
9. AOB
--------
The new VISTA storage space disk server est arrivee (see pile of cardboard
in the APM foyer). It should be accessible shortly and with any luck well
before we run out of disk space for VISTA data products. A recalcitrant
bit of extra metal bracket in the rack has now been removed to make way
for it.
One of the WFCAM RAID arrays (raw data) has lost one of its power supplies
(careless of it). MJI is looking into getting it repaired as these seem to
be no longer available. Meanwhile the data has been copied to apm19e as a
precaution even though the RAID is limping along fine on one power supply
module.
MJI noted that home disk space was getting a bit tight again and asked for
a bit of rm'ing.
Continuing Actions:
===================
STH - produce "final" VISTA colour equations and finish revising
calibration plan document
MJI - link WFCAM illumination correction tables and figs. to plone web pages
EGS
RGM - finish off testing catalogue conversion software
JPE - produce some text to go with the VISTA PR images
JPE - contact ESO re: signing off pipeline deliverables
MJI - see if it is possible to finish off the SV processing saga
EGS
AKY - iterate a bit more with ESO about what ellipticity information they
require, to be also further discussed at next IOT.
MJI - finish off investigating WFCAM high extinction calibration issue
STH
New Actions:
============
STH - investigate large photometric errors quoted in Thomas' Carina paper
(and others in the series) + compose a suitable email re: author lists
MJI - produce cunning VST pipeline plan for VST meeting
EGS
ALL - move all non-VISTA processed data off apm29a
JRL - add yet another keyword to the FITS headers (MJITTER?)
EGS - to lead compiling a 'known features' web page
ALL to help make it so
