2001_10_TERAPIX_DAO by yaoyufang

VIEWS: 1 PAGES: 20

									The TERAPIX pipeline




    DAO 2001-10-05
                       1
          The TERAPIX team at IAP



    Yannick Mellier (PI)        Mario Radovich (OAC)
    Gilles Missonnier           François Bonnarel (CDS)
    Pierre Didelon (80%)        Jean-Charles Cuillandre (CFHT)
    Bertrand Morin (20%)        Henry-Joy McCracken (VIRMOS)
    Arnaud Thernisien (?)




E.BERTIN / TERAPIX          DAO 2001-10-05                   2
                     TERAPIX products
• Large, “clean” and well-calibrated science
  images
     Astrometric and photometric solution
     Image warping
         • Effects of resampling and geometrical corrections on the
           scientific content of images?
     Image co-addition
         • Background subtraction needed for wide dithering patterns
           (to fill the gaps between CCDs)
• “Standard” source catalogs
E.BERTIN / TERAPIX             DAO 2001-10-05                          3
Overview of the TERAPIX pipeline




E.BERTIN / TERAPIX   DAO 2001-10-05   4
                            Software parts
• Data reduction modules: POSIX-compliant C/C++
  and/or PDL+PERL-Tk
     All of them have been developped and are maintained on-site :
         • Perfectly matched to MEGACAM images
         • Perfect control; fast response in case of problems
         • A lot of things are done within each module
               Efficiency (minimizes inputs/outputs)
               Simplifies the pipeline at higher levels
               Possible thanks to permanent, dedicated, team members
     Distributed to the general community A.S.A.D
         • Tremendously improves robustness and portability (code, instruments, file
           formats)
         • Brings up new ideas
         • Helps in forcing people to prepare a detailed documentation
         • Released under GPL, but…



E.BERTIN / TERAPIX                      DAO 2001-10-05                                 5
                     Software distribution
                                             http://terapix.iap.fr/soft




E.BERTIN / TERAPIX          DAO 2001-10-05                          6
                     Software parts (cont.)
• O.O. pipeline database system based on
  Objectivity
     Provides fast and direct access to image
      parameters and individual detections
     C/C++ interface + shell commands
     Currently running under True64 on ???-Alpha
      workstations
     Interface with software modules shall start in a
      few weeks
E.BERTIN / TERAPIX           DAO 2001-10-05              7
                     Software parts (cont.)

• Misc simulation and visualisation tools:
  C/C++ stand-alone programs
     Stuff: Simulation of galaxy populations
     SkyMaker: Image simulation
     Panorapix: Large image visualisation
     Stiff: image compositing and conversion to TIFF
• Development essentially completed
E.BERTIN / TERAPIX           DAO 2001-10-05         8
           Global astrometric solution
• Uses both overlapping
  detections and a
  reference astrometric
  catalog (USNO)
• Works best with a wide
  dithering pattern
• Current method
  (Astrometrix): iterative                                  ( xik  x jk ) 2  ( yik  y jk ) 2
                                  2                                 ik   2
  solution                               i  j ,ki  j
                                                                        2
                                                                               jk

     No need to bring                      ( xik  x0 k ) 2  ( yik  y0 k ) 2
      coordinates to a common      
      projection                    i ,k0            ik   0 k
                                                          2       2



E.BERTIN / TERAPIX         DAO 2001-10-05                                                     9
                     SWarp: overview




E.BERTIN / TERAPIX       DAO 2001-10-05   10
  Some projections supported in Swarp




E.BERTIN / TERAPIX   DAO 2001-10-05     11
      Resampling astronomical images
     “standard” conditions: photometry




                     Bilinear                    Lanczos3

E.BERTIN / TERAPIX              DAO 2001-10-05              12
Main remaining data processing issues
• PSF homogenization
     Robust astrometric+photometric solution on large
      scales implies
         Wide dithering pattern
         Fragmented distribution of PSF shapes in the co-added image
         Standard PSF-mapping impossible
              Crowded field photometry, source classification, shape analyses and
               weak-lensing studies compromised 
• Optical ghosts
     CCD camera with the wides field ever
     Concern about camera design
     Consequences on the observing strategy
E.BERTIN / TERAPIX                 DAO 2001-10-05                                    13
         PSF homogenization (cont.)




E.BERTIN / TERAPIX   DAO 2001-10-05   14
         PSF homogenization (cont.)




                Degraded                    “Improved”


E.BERTIN / TERAPIX         DAO 2001-10-05                15
                Optical ghosts on WFI




E.BERTIN / TERAPIX     DAO 2001-10-05   16
                     Current Hardware




E.BERTIN / TERAPIX        DAO 2001-10-05   17
                      Hardware issues
• Processing (CPU)
     Demise of Alpha
     Rise of fast, low-cost 32bit PC-Linux servers
         • Cheap 64bit machines should appear in 2002 (AMD Hammer)
     Coarse grain parallele processing (as at CFHT)
         400Go+ IDE-RAID0 array at each node
         Constraint on network bandwidth
         First batch of machines shall be acquired before the end of 2001
• Data storage
     Go for FC-IDE RAID5 arrays: ~5kEuro/TB
     Tape reader: juke-box/format?

E.BERTIN / TERAPIX                 DAO 2001-10-05                            18
       TERAPIX “pipeline” network




E.BERTIN / TERAPIX   DAO 2001-10-05   19
                             Data transfers
• Transfers over the internet
  exceedingly preferred
      Price, latency, reliability, manpower
      IAP is currently connected to the
       French research backbone at
       20MB/s, increasing to 155MB/s in
       December 2001
          • Current bottleneck at CADC
            (50kB/s) and CFHT (30kB/s)?
          • Local reliability is good (link up
            >99% of the time)
          • Necessary bandwidth: 400kB/s typ.
            (24/7 if no compression)
      IAP, CADC and U. of Hawaii part of
       the STAR-TAP program
          • V.O. context




E.BERTIN / TERAPIX                      DAO 2001-10-05   20

								
To top