Docstoc

downloading

Document Sample
downloading Powered By Docstoc
					HSC Data Challenge1 - Software Test Items

Listed based on the DC1 target description on the wiki hsc-gw1:8000/wiki/HscDataChallenge1




Stage                   Function
Input Data              CCD data




DataManager             Data location

                        Data Manager

MKFLAT                  CollectFrame

                        CreateFlat
                        UploadFlatFrame


ISR                     CollectFrame

                        Debias
                        Flatfielding
                        InterpolateBadpix
                        SimpleObjectDet

                        SimplePSFMeasure

                        Remove cosmic ray

                        Mask & variance

                        Skysubtraction?


MeasurePSF              FindBrightObjects
             MeasurePSF


             FindBrightObjectsWithPSF




Astrometry   AstrometryMatch




             AstrometryWcsSolve




FluxCalib    FluxCalib




Mosaic       CollectFrame


             MosaicFrame:
             Starsel


             ExternalAstrometrySolve
             (InternalAstrometrySolve)
                       (FluxScaling)
                       (FeedbackToCcd)


Stack                  SkySubtraction



                       WarpFluxScaling




                       Stack




CatalogMaking          FaintObjectFinding

                       MeasureObjects




Framework              parallelization




                       logging




Output Products        MosaicedImages
                       Catalogs




Sharing One Code Set   Codes
                     v. 1.0 - 8.10.2010
                     v. 2.0 - 9.07.2010
t description on the wiki hsc-gw1:8000/wiki/HscDataChallenge1




                   Requirement                                                                 Assigned Member
                   104 CCDs arranged in the HSC FOV format                                  Furusawa
                   Includes i<26 galaxies?
                   Includes USNO-B1.0 or SDSS stars?
                   Observing conditions are correct? (seeing=0.7", expt=300sec, readout=10,
                   gain=2.5-3.5, PA=0, EL=0)
                   Disortion and airmass are correctly added?
                   Vignetting included?
                   Flat pattern + bad pixels included?
                   Data format the same as Scam
                   sequential DET-ID is assigned 0 to 103
                   Initial WCS (10 arcsec offset from the real position)
                   No cosmic rays, no difference CCD size, focus, PSF

                   HSC simulated data are well structured in a defined directory trees, which
                                                                                              Loomis, Furusawa
                   are accessed by dataManager
                   Can read policy files and derive a file path, camera/ccd/amp information
                                                                                              Loomis, Bick, Furusawa
                   necessary for analysis
                   Can collect frames to stack into a flat frame. Human hands in DC1?          Yasuda, Fursawa, Bick

                   Can create flat frames by combining os-subtracted images by taking
                   median
                   Can put the resultant flat into the defined directory

                   Can collect frames to reducebased on the progId/frameID. Human hands
                                                                                               Loomis, Bick
                   in DC1?
                   overscan counts can be subtracted line by line, and os regions and blanks
                                                                                               Furusawa
                   are trimmed?
                                                                                               Yasuda, Furusawa
                   Can identify distinct bad pixels and mask them?
                   Can derive objects for PSF measurement                                      Lupton
                   Can determine crude PSF across the CCD area, in a way the PSF can be
                                                                                               Lupton
                   used in the next object detection and centroiding for astrometry stage
                   minimal implementation in DC1                                               Lupton
                   Mask and variance images are calculated in "debias and flatfielding (also
                   skysubtraction?)" and properly attached to exposure object
                   Does not affect photometry of bright objects                                Lupton

                   Good enough centroiding accuracy; much better than positional accuracy
                                                                                          Lupton
                   of reference catalog sources.
Good enough accuacy in flux counting; much better than reference
catalog sources.
Enough number of objects in each CCD (> 50 reasonable reference
sources)
Can use crude PSF information for detection and measurements at the
source positions
Can provide point sources for precise PSF determination
Can derive good PSF information across the CCD area                           Lupton
Resultant PSF can be obtained by specifying (x, y) positions
Can provide good source lists for the astrometry stage with good
                                                                              Lupton
centroiding by using thbe MeasurePSF results
The number of detections for typical dataset, Position and flux accuracy to
be tested and reported
Can provide matching Lists of matched pairs with reference sources
                                                                              Loomis, Bick, Furusawa
containing (x,y, RA,Dec, SDSS mags, flags)
Well determined setting available for matching to succeed (at least >50
matched stars output per CCD) for every simulated data in DC1 (5shot x 5
dataset)
The number of matched objects and robustness/quality of matching to be
tested and reported for typical simulated data and also for Suprime-Cam
data
Can fit a single TAN-SIP at a time for 104 CCDs assuming the CCD
                                                                              Okura
geometrical arrangement is known, obtain WCS of each CCD

Fitting error of WCS solve is tested and reported


Can derive photometric zeropoint of each CCD by assuming mag of
                                                                          Furusawa, Bick
reference sources are in HSC band system
Prepared for determination of zeropoint with color term by using SDSS
multi-band magnitudes?
Fitting error in zeropoint and how magnitudes of objects can be recovered
to be tested and reported
Can collect FITS frames (+WCS) to be mosaiced by specifying
                                                                              Yasuda, Loomis?
progId/filter/time range etc. Human hands also acceptable for DC1
Can collect necessary matchlist and source lists of each CCD                  Mineo?

sourceSet of objects in each CCDs can be available                       Lupton
(x,y,flux) with good enough accuracy of centroiding and flux measurement
to do mosaic
matched lists with reference sources available?                          Yasuda, Okura

Good WCS for each CCD provided by AstrometryWcsSolve available?               Yasuda
Can do matching of sources across CCDs from different shots by using
WCS from astrometry stage?
Can determine the best geometrical alignment of each CCD by
determining TAN-SIP WCS keywords of each CCD
Fitting error and rms in geometrical alignment of CCDs to be tested and
reported for typical simulated dataset.
Processing speed is also to be investigated.
zeropoints from fluxCalib stage are to be used in DC1.                      Yasuda
Is WCS and zeropoint can be refined in Mosaic stage? If so, those results
can be re-distributed to each CCD?

Does not affect profiles and fluxes of bright sources (S/N>10)            Lupton
Can subtract typical smooth sky bkg of simulated data and the resultant
sky-bkg level goes around 0 ADU?
Can transform each CCD into the final xy plane by decoding TAN-SIP
                                                                          Yasuda, Lupton
keywords?
(Can conserve fluxes of objects? To What level?) flux conservation should
be better than 5% level in systematic error.
(Can conserve shapes of objects? To What level?)
Can provide mask and variance images, too?
Can stack transformed pixel data by taking median?                        Yasuda, Lupton
Clipping outlying values available?

Can provide mask and variance images, too?




Can detect any object above 5sigma significance level                       Lupton

Object parameters to be derived: id, [ x,y, ra, dec (based on TAN-SIP) +
                                                                             Lupton
those errors ], flag, [ psfMag + error], [ApertureMag 1",2",3" + errors ],
[petroMag + errors (elliptical apertures if possible), petroRad ], 2nd-order




Can run 104 CCD reduction and calibration on multiple cores based on a
                                                                            Mineo
user-defined configuration
Can collect matchList for astrometry stage onto a node
Can run the astrometry stage by using the match lists
Can re-distribute the result WCS to each node for each CCD
Can detect error events from a pipeline and log any messages from the
                                                                            Mineo
pipeline




To be compared with the original simulated input pixel data
To be tested for number of objects (n(m)), object positions, fluxes, and
size



Making push all the necessary latest sources to repository and Tag them
Status   Comment AI
D
HD       i=25        Should go 1-2 mag deeper in the next challenges
HD                   Should add mag >22
D
HD                   Orientation for gravity should be correctly included in the next stage
D
NotYet
D
D
HD                   No pointing offset is tested
D


D

HD                   Corner 4 CCDs should be also handled

D

D
HD                   Resultant flat files should be written into the proper directory


D

D
D
D
D

D

HD                   Implemented but not yet tested with real CR's

HD                   Only mechanism is done. In each analysis stage, algorithms to propagate noises should

D        Not fully tested. DC1 is ok.


D
    D

    D

    D
    D
    D        ok for DC1
    D        available by detection & measurements code
    D

    HD                  to be finalized with the tagged version codes

    HD                  5mags are postponed


    HD                  4
             good for DC1 frames got failure in matching of 3000 frames


    HD                  to be investigated with the tagged version codes


    D


    HD                  under investigation



    D                   Should update the codes with clean simulated data with clean reference sources

    NotYet

    D        1% rms in (PSFmag - refmag) for DR8 data. 3% rms for DR7 r', and 5-6% rms for DR7 I' data. basica


    D

Mineo?

Lupton


    D
    D

    D        testing

    HD                  Improvement in astrometric accuracy is not verified. Need investigation of how to imp
HD                 Not yet tested
NotYet


HD                 envelopes of bright stars seem to be too much subtracted
D

D        working   making it fast

NotYet             To be tested
NotYet
HD                 Algorithms to propagate variances should be investigated
D
?

HD                 same as above




HD                 Not applied to stacked images,yet.

HD                 Currently, the following parameters are available: id, [ x,y, ra, dec (based on TAN-SIP) +




D
D
D
D
NotYet             will work on this from now on




HD
NotYet             to be derived soon




HD
ectly included in the next stage




nto the proper directory




ysis stage, algorithms to propagate noises should be investigated
mulated data with clean reference sources




s for DR7 r', and 5-6% rms for DR7 I' data. basically robust. The cause of systematic errors should be understood with good simulated data




is not verified. Need investigation of how to improve astrometric accuracy in mosaicking stage.
oo much subtracted




e available: id, [ x,y, ra, dec (based on TAN-SIP) + those errors ], flag, [ psfMag + error], [ApertureMag 1",2",3" + errors ]
rstood with good simulated data

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:5/7/2013
language:Unknown
pages:14
gegouzhen12 gegouzhen12
About