You are here: Home / Documents / Minutes of CASU Meetings / minutes_100922.txt


Plain Text icon minutes_100922.txt — Plain Text, 12 KB (12921 bytes)

File contents

Wednesday 22th September 2010  APM Meeting tables 11:30 - 13:00

Present:   JCR, MJI, STH, EGS, JRL, AKY, MR, NAW
Apologies: RGM, JPE


1.  Actions from last meeting
2.  Comments on WFAU minutes
3.  Meetings and Telecons
4.  Data archives update     - AAT, ING, WFCAM, UKIRT, VISTA
5.  Optical/NIR processing   - HAWKI, INT WFC, MegaCam, Subaru, VST
6.  WFCAM update
7.  VISTA update
8.  CASUhelps
9.  AOB


1. Actions from the last meeting

JRL	CASU tool releases are up to date and at v1.06

JRL	NBSIZE parameters have gone into the FITS header

JRL	has done all the updates he can on the calibration plan
STH	needs to revise the colour equations offline and finish off       <<<<

JRL	the latest DRLD has been sent out

MJI	has added an illumination table option, action passed to
EGS     to link WFCAM tables to web pages                                 <<<<

RGM	still testing catalogue listing software but sent in a note
        regarding astrometric differences at the 10mas level. MJI noted
        that these are undoubtedly caused by use of 32-bit arithmetic     <<<< 

JPE	text to go with CASU versions of VISTA PR images - ongoing        <<<<

JPE	contact ESO re: signing off pipeline deliverables - ongoing       <<<<
        though he is signing off the DRLD and sending it to ESO

EGS     have made good progress with the SV v1.0 reprocessing and will
MJI     discuss with the 3M's ++ at ESO as part of the VST meeting visit  <<<<

JRL	has tied up his loose ends re: simplesky -> tilesky

AKY	is investigating various aspects of the ellipticity trends as 
        part of an IOT action item (see below) - ongoing                  <<<<

STH	investigated the issue of CASU authorship on the Carina papers.
	It was decided to not be appropriate except on the initial papers.
        He will compose and send off a suitable email also commenting on
        the calibration errors                                            <<<<

JPE     didn't quite manage to find out the date of the VISTA PSPI PhaseIII
        workshop but this was divulged during the VST meeting (30th November)
        and hence no longer an action item

JRL	found missing HAWKI data - where is it then ?

MJI	WFCAM calibrations in regions of high extinction investigation is
STH     still ongoing                                                     <<<<

AKY	have provided feedback to PIs on derived QC information etc .....
EGS     in the form of FITS tables in addition to the usual sql dooberrys

2. Comments on WFAU minutes

WFAU have been radio quiet for last week as a result of a power outage
on Blackford Hill caused by some workmen with a drill and a bad aim.

RGM informed us before the meeting that WFAU apparently need the CASU tiling 
software to make their band-merged catalogues, quite how and/or why we can't 
figure out, but the basic tiling software has now been made available anyway. 

3. Meetings and Telecons

On 07/09/10 there was a meeting to address GAIA computing requirements at the
IoA.  The upshot is that in ~a year there will (may) be 2x4 racks of
computing space available in the old dog kennels at the Vet School
(as opposed to the 2 X 16 racks available for finance !!). It will have a
halon gas fire extinguishing system which means people will have to
enter the building through the dog flap in pairs for 'elf & safety.

STH said he had a very nice time in Portugal and enlightened JENAM'ers
about VISTA processing, nebulisers, calibration, lightcurve generation
and no doubt much else.

On 28/29 September there is a VST review at ESO. EGS and MJI will be
attending to talk about the CASU VST pipeline and processing experience
en general, as well as talk to the VISTA SV teams.

NAW pointed out that due to several upcoming large spectral surveys 
(e.g. FLAMES GIRAFFE/UVES, possible new wide-field MOS's etc....) there is a 
growing need for a half-decent spectral reduction pipeline.  This is something
that we have done in the past, at least to the extraction and wavelength 
calibration stage; while more advanced spectral products for GA are an active
area of research within CASU.  Let us see.

4. Data Archive

ING, AAO, UKIRT, VISTA  all up to date and motoring.  JRL sorted out the
delayed (at JAC request) few nights of CASS UKIRT archive and they are
now also available.

5. Optical / NIR Processing

Everything ticking along nicely apart from the IMACS data which has stalled
due to advanced boredom with having to solve yet another astrometry setup.

MJI has been investigating a scattered light issue in MegaCam, triggered by
being a rapporteur for a SNLS PhD but also as part of the preparation for a 
PAndAS (M31 survey) meeting.  He now understands the problem well enough to 
figure out a post-processing correction table to apply to the catalogues.

EGS and Janet Drew have been exchanging some fascinating e-mails about 
which VEGAs system IPHAS is calibrated to.

JRL has finished processing RGM's HAWKI data on the GOODS fields.

Action on MJI and EGS to develop a VST pipeline plan before VST meeting,  <<<<
or at least be amazingly persuasive at same.

6. WFCAM Update

All available 10B data has been received as expected and has been ingested
into the raw data archive. Transfers to ESO are also up to date.

September processing is a tad behind due to delays in getting a complete set 
of flats. Otherwise everything has been routinely pipelining and progressing 
as expected.

The August data was flagged as ready to copy on September 16th and despite  
the best efforts of the workmen in Blackford Hill turned up at the WFAU
archive end just over a week later. 

No requests for raw data and the only early access request for processed 
data was from Sandy Leggett.

Omar and the UDS team noticed an interesting stripey pattern in J & K-band
deep stacks in the UDS data prior to DR8 release.  MJI investigated the
problem, ruled out decurtaining due to the symmetry being wrong and tracked
it down to a less than 1 part in 10000 effect visible in individual MSB 
stacks if you know what to look for. This looks like a (very minor) hardware 
feature whereby brightish stars in each quadrant cause a slight depression, or 
sometimes a slight rise, in the background level along nearby rows and/or 
columns.  The effect is isolated in quadrants suggesting it is a readout
feature.  Mind you at this level if only becomes really noticeable when you 
make deep stacks.  In the K-band, for example, the surface brightness of 
these stripes is between 23.5 and 24 mag/sq arcsec.  Unfortunately as the
effect is "attached" to the stars it does not go away on normal stacking
and is probably best corrected during intermediate depth stacks (to minimise
quadrant overlaps).  This has resulted in a delay to UDS DR8 although begs
the question of why this effect was not spotted before.

7. VISTA Update

VISTA data up to the end of August has been received and processed data up 
to 7th August has been released.  All data up to 20100901 has been processed
and we are now up to the last couple of nights of post-processing.

MJI noted that apm29a is fast running out of disk space and asked for a tidy
up of non-VISTA processed data from there.

The SV NGC253 data that was mistakenly processed with the 'simplesky'
algorithm has now been reprocessed yet again, but this time with the 
original 'tilesky' option.  This is hopefully it, and the latest latest
version has been made available to ESO (Magda) and WFAU.  Meanwhile EGS
and MJI have been finishing off the rest of the SV post-pipeline processing
and will discuss results with the 3M's during the ESO VST meeting.

Matt Jarvis has observed 5 nights with the dual narrow band (975/985) filters. 
The data is expected at our end anytime soon.

The odd few NaN/infinity/negative CRVAL issues++ have been investigated.
Negative CRVALs crop up from the code not checking for offset sky
exposures.  The infinite values were caused by a bug in the overlapping
routine. JRL has fixed both of these.

The QC tables are now available on the CASU website with notes explaining 
column headers and the FITS files. We just need to tell the PIs where all 
this bounty is.

AKY has been investigating why VVV and VHS sometimes have larger than 
acceptable ellipticities. It turns out that these occur most frequently during
camera reshuffle periods (October last year, May this year..).  Outside
of this problem the other main culprits are trailed images due to poor 
tracking, primarily because the A/G hadn't settled before the OB sequence
began.  Meanwhile AKY will contact Marina and see what further information 
ESO want in this regard. 

VVV have obtained w/e use of a dedicated "large" bandwidth line from 
Santiago, Chile, during which they downloaded (or not) the newer tiles and 
catalogues not included on the shipped USB disks.

It was decided that the data releases to PIs will be monthly from here on
to simplify bookkeeping at both ends.

JRL has been doing further work on the issue of apparent scattering in the 
detector zero-points raised by WHU at ESO.  We are confident this is a 
non-issue and MJI sent a detailed explanation of why sometime ago.

There followed a brief discussion about processing or otherwise of incomplete
OBs and how to keep track of the process.  Currently incomplete OBs are
processed as usual providing there are enough frames to compute a reliable
sky estimate. Grossly incomplete OBs are not processed.  This raises two
issues: how to keep track in our QC database of which ones are not processed;
and flagging more directly in the science products that only "m" out of "n"
jitters are present in the stack.  Although the provenance information in the
headers allows you to deduce this, it was felt that a specific keyword should
be added to make it easier to "grep" for.                                  <<<<

Finally MJI suggested that a 'known features' page on the website
might be useful for noting minor gremlins/gotchas and pre-empting some of 
the CASUhelp e-mails.  EGS with helpers will make it so                    <<<<

8. CASUhelps

Somehow MR mysteriously acquired exclusive ownership of CASUhelp and 
CASUreply email folders recently.  This has been sorted but we have no idea
how it happened.

The other mr. confused everybody (CASU+WFAU) by managing to be browsing our 
web pages here as some June data was flagged (red -> green) as ready to go 
and then getting confused about dates as it turned out to be a repeat OB of 
earlier (Feb) data.

ESO seem very keen on the PA of stellar ellipticities so we might have to
give in and compute an average PA for each detector and plonk the results
in the FITS headers.

...... and the contender for query of the month
"it seems that the astrometric solution was only applied to the WCS positions
(ra, dec) and not to the pixel positions.  Would this be correct?" 


9. AOB

The new VISTA storage space disk server est arrivee (see pile of cardboard 
in the APM foyer). It should be accessible shortly and with any luck well
before we run out of disk space for VISTA data products.  A recalcitrant
bit of extra metal bracket in the rack has now been removed to make way 
for it.

One of the WFCAM RAID arrays (raw data) has lost one of its power supplies
(careless of it). MJI is looking into getting it repaired as these seem to
be no longer available.  Meanwhile the data has been copied to apm19e as a 
precaution even though the RAID is limping along fine on one power supply 

MJI noted that home disk space was getting a bit tight again and asked for
a bit of rm'ing.

Continuing Actions:

STH  - produce "final" VISTA colour equations and finish revising 
       calibration plan document

MJI  - link WFCAM illumination correction tables and figs. to plone web pages

RGM  - finish off testing catalogue conversion software

JPE  - produce some text to go with the VISTA PR images

JPE  - contact ESO re: signing off pipeline deliverables

MJI  - see if it is possible to finish off the SV processing saga

AKY  - iterate a bit more with ESO about what ellipticity information they
       require, to be also further discussed at next IOT.

MJI  - finish off investigating WFCAM high extinction calibration issue 

New Actions:

STH - investigate large photometric errors quoted in Thomas' Carina paper
      (and others in the series) + compose a suitable email re: author lists

MJI - produce cunning VST pipeline plan for VST meeting

ALL - move all non-VISTA processed data off apm29a

JRL - add yet another keyword to the FITS headers (MJITTER?)

EGS - to lead compiling a 'known features' web page
ALL   to help make it so