Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Image Quality Evaluation of QuickBird Super Resolution and Revisit by nne25858

VIEWS: 66 PAGES: 31

									          Know the Earth…Show the Way




Image Quality Evaluation of QuickBird
Super Resolution and Revisit of IKONOS
Civil And Commercial Application Project (CCAP)




Edward C. Jones
8 November 2004
High Spatial Resolution Commercial Imagery Workshop
703.735.3107, jonesec@nga.mil
                                    NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Study Objectives


   • Determine the National Imagery
     Interpretability Rating Scale (NIIRS) for:
       1. Space Imaging IKONOS Revisit
       2. DigitalGlobe QuickBird Super Resolution




                                                                    2
                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Evaluation Process

   •   Prepare plan
   •   Define variables
   •   Collect materials
   •   Develop evaluation capabilities
   •   Perform pilot evaluation
   •   Perform evaluation
   •   Analyze data
   •   Prepare briefing
   •   Publish brief
                                                                    3
                                          Know the Earth…Show the Way
   Know the Earth…Show the Way




IKONOS Revisit




                             NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Background
   • IQ&U assessed IKONOS pan imagery in the
     fall of 2000
        – 46 chips from 24 images
        – Average GSD: 0.93 meters (36.6 inches)
        – Average NIIRS 4.65
   • Data was re-analyzed because new outlier
     procedures had been established
        – Two IAs were removed as outliers that were not
          removed in the previous analysis
        – Average NIIRS under this new analysis is 4.49 ±
          0.19 (95% confidence)
        – Confidence bounds assumes IAs are random
          effects as opposed to fixed effects
                                                                       5
                                             Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Objective of Revisit


   • To re-assess the quality of IKONOS
     imagery to determine if there had been a
     change over time




                                                                    6
                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Experimental Design

   • 10 Imagery Analysts
   • 22 Images
        – One to five sub-scenes extracted from each
          image
        – Total of 68 chips
        – GSD from 0.82 (best) to 0.995 meters (32 to
          39 inches)
            • Average GSD = 0.89
        – Various OBs (A,C, G, I, N)
   • Absolute NIIRS Ratings
                                                                    7
                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Outlier Analysis


   • Calculated Bias and RMSE for each IA
        – Remove IAs that exceed thresholds
   • Examine individual ratings to identify
     possible outliers
   • Results
        – Two IAs removed from the analysis and one
          additional rating was identified as an outlier
          and also removed from the analysis

                                                                    8
                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Average NIIRS by Image Scene
   n=22

                          5.0
          Average NIIRS




                          4.5



                          4.0



                          3.5
                             0   5   10           15   20      25
                                          Image

                                                                                      9
                                                            Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Average NIIRS for Image Chips
   n=68
                         5.0
         Average NIIRS




                         4.5



                         4.0



                         3.5
                            0   10   20   30   40   50   60   70
                                           Scene

                                                                                   10
                                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Average NIIRS by OB

                   5.0




                   4.5
            AVE8




                   4.0
                                                  A - Air
                                                  C - Cultural
                                                  G - GOB
                                                  I - Industrial
                                                  N - NOB
                   3.5
                         A   C    G       I   N
                                  OB

                                                                            11
                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Comparison with Previous Evaluation


   • Average NIIRS
        – 1st Evaluation (2000): 4.49 ± 0.187
        – 2nd Evaluation (2003): 4.30 ± 0.155


   • Difference not significant at the 0.05 level



                                                                   12
                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Conclusions


   • While average NIIRS rating was smaller
     than previous evaluation, difference not
     statistically significant
   • Conclude that IKONOS panchromatic
     image quality has not changed between
     2001 and 2003



                                                                   13
                                          Know the Earth…Show the Way
    Know the Earth…Show the Way




QuickBird Super Resolution




                              NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Purpose of Evaluation


   • DigitalGlobe developed an algorithm for
     processing a Super Resolution imagery
     product
   • NGA Commercial Imagery Office
     requested NGA IQ&U evaluate the Super
     Resolution product



                                                                   15
                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Background of Original QuickBird
   Evaluation
   • 10 Imagery Analysts (IAs)
   • 53 Images
        –   28 Pan (Basic 1B)
        –   10 MSI (Basic 1B)
        –   15 Pan Sharpened MSI (Standard 2A)
        –   One to five sub-scenes extracted from each image
   • Total of 213 chips
   • All images at near-nadir collection geometry
        – Pan and Pan-Sharpened: GSD between 0.62 and
          0.7 meters (24.4 – 27.6 inches)
        – MSI: GSD = 2.54 meters (100 inches)
                                                                       16
                                              Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Super Resolution Experimental Design

   • 9 IAs
   • 8 scenes re-processed two ways
        – Original
        – Super Resolution
   • Each IA
        – Absolute NIIRS rated the regular scenes
        – Delta NIIRS rated the original against the
          Super Resolution product

                                                                   17
                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Average Absolute NIIRS by Image
   (IA #4 removed from analysis)

            7


                                                         2 standard deviations
            6
    NIIRS




            5
                                                          Average NIIRS = 4.57 (± 0.10)

            4



            3                                            95% confidence interval for
                     .tif .tif p1 p2 d1 chi d2      s.
                 a n1 an3 _air _air hda d1_ hda orge     the Average is ± 0.10 NIIRS
            H ain Hain ghd ghd bag hda bag reeg
                         ba ba     ba
                                     g      th
                               IMAGE
                                                                                            18
                                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Outlier Analysis
   Absolute NIIRS

   • IA NIIRS ratings examined for outliers by
        – Correlation analysis
        – Bias
           • Needs to be less than 0.45 in absolute value
        – RMSE
           • Needs to be less than 0.75


   • Based upon this analysis, IA #4 responses
     were identified as being different from the
     remaining IAs and these data were removed
     from the analysis
                                                                       19
                                              Know the Earth…Show the Way
   NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

               Delta NIIRS by Image
               (IAs #6 and #8 removed)

              1.2

              1.0
                                                                   2 standard deviations
              0.8
Delta NIIRS




              0.6

              0.4                                                 Average Delta NIIRS = 0.27 ± 0.063

              0.2

              0.0                                                   95% confidence interval for
              -0.2
                                                                   the Average is ± 0.063 NIIRS
                       .tif .tif p1  p2     d1   hi      d2  s.
                     n1 an3 _air _air hda 1_c hda orge
                  ina ain      d   d   g       d    g      g
                Ha    H bagh bagh    ba hda      ba ree
                                          g           th
                                        ba
                                       IMAGE
                                                                                                         20
                                                                                Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

       Average Delta NIIRS by IA
       (IAs #6 and #8 removed)

                 1.5

                 1.2

                 0.9
   Delta NIIRS




                 0.6                                           IA #6 is only IA to not
                                                                  prefer the super
                 0.3
                                                                  resolution image
                 0.0

                 -0.3

                 -0.6
                     0   1   2   3   4    5 6   7   8   9 10
                                         IA

                                                                                              21
                                                                     Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Outlier Analysis
   Delta NIIRS

   • IA Delta NIIRS ratings examined for outliers by:
        –   Average Delta NIIRS
        –   Correlation analysis
        –   Bias (No threshold established)
        –   RMSE (No threshold established)


   • Based upon this analysis, IAs #6 and #8
     responses were identified as being different
     from the remaining IAs and these data were
     removed from the analysis

                                                                       22
                                              Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   Conclusions

   • All but one IA preferred the Super
     Resolution product when compared to
     the original product
   • Average Delta NIIRS improvement (after
     outlier analysis) was 0.27
   • Average Delta NIIRS is statistically
     different from zero at the 0.01 level of
     significance

                                                                   23
                                          Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   QB Super Resolution
                            Three Gorges




       Standard Product                   Super Resolution Product          24
                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   QB Super Resolution
                            Three Gorges




       Standard Product                   Super Resolution Product          25
                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   QB Super Resolution
                         Baghdad Airport1




       Original Product                   Super Resolution Product          26
                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   QB Super Resolution
                         Baghdad Airport1




        Original Product                  Super Resolution Product          27
                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   QB Super Resolution
                                Hainan1




        Original Product                  Super Resolution Product
                                                                            28
                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   QB Super Resolution
                                Hainan1




        Original Product                  Super Resolution Product
                                                                            29
                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

   CCAP Team

 • Government Lead                        • Additional Support
      – Ken Peterman, 301.227.3520          – Image Quality & Utility
 • Technical Lead                             Branch (AEAI)
      – Terry Lehman, 703.735.3869          – Leigh Harrington,
 • Imagery Analyst                            Statistician (AEAI)
      – Ed Jones, 703.735.3107              – Community IAs
 • Photogrammetry Lead
      – Paul Bresnahan, 703.735.3565
 • Geospatial Analyst
      – Jonathan Koch, 703.735.3890



                                                                            30
                                                   Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY




    NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY




                Know the Earth…Show the Way




                                                                       31
                                              Know the Earth…Show the Way

								
To top