Document Sample
lecture01 Powered By Docstoc
					CSci 6971: Image Registration
   Lecture 1: Introduction
      January 13, 2004

     Prof. Chuck Stewart, RPI
     Dr. Luis Ibanez, Kitware
      Syllabus
      Registration problem
      Applications of registration
      Components of a solution
      Thematic questions underlying registration
      Software toolkits

Image Registration         Lecture 1                 2
                                           Syllabus - Topic
      Image registration:
         Determining the mapping between two images of
           the same object, similar objects, the same region
           or similar regions
      All aspects of the problem will be covered:
         Underlying mathematics
            Images
            Algorithms
            Implementations
         Applications
      Special emphasizis on software toolkits

Image Registration             Lecture 1                       3
                                   Syllabus - Instructors

          Prof. Chuck Stewart                         Dr. Luis Ibanez
           107 Amos Eaton                          Kitware Corporation
     Rensselaer Polytechnic Institute             (518)-371-3971 x 112

Image Registration                 Lecture 1                             4
                     Syllabus - Distributed Course
      NSF Center for Subsurface Sensing and Imaging
       Systems (CenSSIS) Course
      Four universities:
         Boston University,
         Northeastern University,
         Rensselaer Polytechnic Institute,
         University of Puerto Rico Mayaguez
      Lectures live at Rensselear
      Lectures recorded in voice-annotated powerpoint for
       remote students
      Lectures missed due to weather or travel will also be
       available via voice-annotated powerpoint

Image Registration            Lecture 1                        5
                           Syllabus - Office Hours
      At Rensselaer:
            Tuesday and Friday immediately following
      Distributed office hours
            Web-meeting
            Anyone can participate

Image Registration           Lecture 1                  6
                          Syllabus - Prerequisites
      Data structures
      Calculus
      Linear algebra:
            Vectors and matrices
      Experience working with images
      C++ programming experience
            Templates!

Image Registration           Lecture 1               7
                        Syllabus - Requirements
      Weekly homework assignments and
       programming projects
      Extended programming project (due Tuesday,
       April 6)
      10-page research paper (due Tuesday, May 4)
      Each is worth 30% of the semester grade
      Late assignments will not be accepted without
       prior arrangement or a verified personal
      Last 10% is for acting as a scribe during
       lecture. This is a pass/fail requirement that off-
       campus students automatically pass.
Image Registration         Lecture 1                        8
                     Syllabus - Course Materials
      Voice-annotated powerpoint lectures will be
       placed on CenSSIS website

      Software toolkits will include tutorials
      Reading materials will also be placed on the
       CenSSIS website

Image Registration          Lecture 1                 9
                                          Syllabus - Topics
      Introduction
      Mathematical background
      First examples
      Intensity-based registration and ITK
      Feature-based registration and the CenSSIS/RPI toolkit
        Initialization techniques
        Multiresolution techniques
        Mutual information
        Deformable registration
        Video registration and image mosaics
        Trends and research questions

Image Registration            Lecture 1                         10
                     Syllabus - Academic Integrity
      Students may discuss homework and
       programming assignments
         Solutions must be written in students own
      Extended programming project and research
       paper must be individual work with
       appropriate citations
      A serious incident will result in failing the

Image Registration          Lecture 1                  11
                     Registration Problem Definition

                                                              q = (912,632)
                     p = (825,856)   q = T(p;a)

    Pixel location in first image                 Homologous pixel location in
                                                  second image

                           Pixel location mapping function

Image Registration                    Lecture 1                                  12
                            Example Mapping Function

                                                  q = (912,632)
                     p = (825,856)

                                                 Pixel scaling and
Image Registration                   Lecture 1                    13
                     Registration Problem Definition

                                                        q = (912,632)
                     p = (825,856)   q = T(p;q)

        • Form of mapping function T
        • Unknown mapping parameters q            “Chicken-and-egg”
        • Unknown correspondences, p,q             problem
Image Registration                    Lecture 1                         14
                 Applications: Multimodal Integration
      Two or more different sensors view same region or
      Different viewpoints
         (Some specialized sensors have two or more
          coincident modalities, so registration is not needed.)
      Different information is prominent in each image
      The images may even have different dimensions!
         Range images vs. intensity images
         CT volumes vs. fluoro images

Image Registration             Lecture 1                           15
             Example: MR-CT Brain Registration

                                            MR                       CT
     MR (magnetic
      resonance) measures
      water content
     CT measures x-ray
     Bone is brightest in CT
      and darkest in MR
     Both images are 3d


Image Registration              Lecture 1                                                  16
                       MR-CT Registration Results

      Aligned images        Superimposed images, with bone
                              structures from CT in green

Image Registration          Lecture 1                    17
                     Retinal Angiogram and Color Image

Image Registration             Lecture 1                 18
                     Applications: Image Mosaics

      Many, partially overlapping images
      No one gives a complete view
      Goal: “stitch” images together
      Requires:
         Limited camera viewpoint such as rotation about
          optical center
         Simple surface geometry such as plane or

Image Registration           Lecture 1                      19
                     Retinal Image Mosaics

Image Registration   Lecture 1               20
                                                 Sea-Floor Mosaics

 Courtesy Woods Hole Oceanographic Institution

Image Registration                      Lecture 1                    21
                            Spherical Mosaics

                                 Images from Sarnoff Corporation

Image Registration   Lecture 1                                     22
                     Applications: Building 3d Models
      Range scanners store an (x,y,z)
       measurement at each pixel location
      Each “range image” gives a partial view
      Must register range images and texture map
      Applications:
            Reverse engineering
            Digital architecture and archaeology

Image Registration              Lecture 1               23

                QuickTime™ an d a                                QuickTime™ an d a
           YUV420 codec decompressor                        YUV420 codec decompressor
          are need ed to see this p icture .               are need ed to see this p icture .


Image Registration                             Lecture 1                                        24
                     Applications: Change Detection
      Images taken at different times
      Following registration, the differences
       between the images may be indicative of
      Deciding if the change is really there may be
       quite difficult

Image Registration            Lecture 1                25
                     Retinal Change Example

Image Registration     Lecture 1              26
                     Regions Showing Change

Image Registration      Lecture 1             27
      Applications: Video Super-Imposed on 3d Model

                        Taken from Sarnoff Corporation research

Image Registration      Lecture 1                                 28
                                   Other Applications
      Multi-subject registration to develop organ
       variation atlases.
            Used as the basis for detecting abnormal
      Object recognition - alignment of object model
       instance and image of unknown object
      Industrial inspection
            Compare CAD model to instance of part to
             determine errors in manufacturing process

Image Registration           Lecture 1                   29
                          Steps Toward a Solution
      Analyze the images
      Determine the appropriate image primitives
      Determine the transformation model
            Geometric and intensity
      Design an initialization technique
      Develop constraints and an error metric on
       the transformation estimate
      Design a minimization algorithm
      Develop a convergence criteria

Image Registration           Lecture 1              30
                                         Software Toolkits
      ITK
         Medical image processing, segmentation, and
          registration toolkit
         C++, heavily templated, data flow architecture
         Registration stresses intensity-based approaches
      VXL
         Computer vision applications
         C++, moderate templating
         Registration stresses feature-based approaches
      CenSSIS tool suite is a hybrid of these
         They can be used together
         Programming styles are different

Image Registration           Lecture 1                       31
                     Summary: Pervasive Questions
      Three questions to consider in approaching
       any registration problem:
            What intensity information or image
             structures is/are consistent between the
             images to be registered?
            What is the geometric relationship between
             the image coordinate systems?
            What prior information can be used to
             constrain the domain of possible

Image Registration            Lecture 1                   32
           Looking Ahead: Lecture 2 - Friday, January 16

      Mathematical background, part 1:
            Vectors and matrices
      No meeting on Rensselaer campus
      Voice-annotated lecture will be posted
       Thursday night, January 15

Image Registration           Lecture 1               33
                               Homework Problem
      Due Tuesday, January 20 at 12 noon (via
       email to Professor Stewart)
      Problem:
         Find an application of registration,
          preferably in a research area of interest to
          you. In a short write-up (less than a full
          page), describe the problem and attempt to
          sketch answers to the three “Pervasive
          Questions” posed.

Image Registration         Lecture 1                     34

Shared By: