Image Processing - Intro by ewghwehws

VIEWS: 3 PAGES: 70

									     Image Processing #1
   Introduction to Digital Images


           Thomas Moeslund
Computer Vision and Media Technology lab.
           Aalborg University
            tbm@imi.aau.dk


                                            1
  Why are digital images interesting?

• Humans are visual creatures in a
  visual world
• Images are (often) the primary sense
   – Imagine you could only keep one sense…
• ”A picture is worth a 1000 words”
• Words are many times ambiguous
• So, if we want to build systems capable of human
  skills, then they should be capable of
  understanding images (many applications)
• Images in a computer are DIGITAL, as opposed to
  analog images in an (old) photo-camera
                                                     2
   Why should MED-students learn
       about digital images?
• Because images are fun 
• Understand the media: Images
• Understand how images can be manipulated in order to create
  visual effects
   – Rotation, scaling, blurring, etc.
       • As done in standard image manipulation tools
       • Remove parts of an image
   – Combine graphics with real images
   – Combine (part of) one image with another
• Generate control signals for an application (project)
   – Understand how to find and follow (well defined) objects in an image
• Recognize objects (many industrial applications)

                                                                     3
The (rough) plan for the 3 ECTS
• Tuesdays (12.30 – 16.15)
• Fridays: (12.30 – 16.15)
• Content
  –   Basic image processing (6)
  –   Video processing (2)
  –   Summary (1)
  –   Mini-project (6)
• PE-course: Let me know your needs!


                                       4
              General information
• IP lectures
   – Timing
       • Lecture, break, lecture, exercises
   – Feel free to interrupt!
• Literature
   – Image and Video Processing, 2nd edition
     Thomas B. Moeslund
      • Comments welcome!
• Web
   – Moodle: Literature, slides, PE-questions
• Exercises
   – Slides
• SW
   – Basic image processing: ImageJ
   – Video processing: EyesWeb
   – Mini-project: C++                          5
              Plan for today
• Digital images
  – Definitions, HW, etc.
• Exercises




                               6
Image definitions




                    7
Where does an image come from?




                                 8
Where does an image come from?
      Electromagnetic spectrum




                                 9
Where does an image come from?

                     Charged coupled device
                     CCD-chip




                                          10
   Where does an image come from?




• Integration over time
   – Exposure time
      • Shutter time (DK: lukketid)
   – Motion blur
   – Maximum charge
      • Saturation
      • Blooming                      11
Where does an image come from?




                                          A/D



    • Image elements, picture elements, pels, pixels   12
            Imaging system
• Image acquisition

• Illumination
  – Passive: sun
  – Active: ordinary lamp, X-ray, radar, IR

• Camera lens
  – Focus the light on the CCD chip
                                              13
   Digital Image Representation
      Origin
                                  x



               f(x,y)



  y
• Image is seen as a discrete function f(x,y) as
  opposed to a continuous function (show)
• x and y cannot take on any value!
                                                   14
    Discrete image coordinate system
     Origin                    f(0,0)
                                          x
                       x




              f(x,y)

                                        f(?,?)
                           y
                               f(2,6)
y


                                           15
Digital Image Representation




                               16
   Digital Image Representation
                                                 Width
• An image f(x,y) is represented
                                                  ROI




                                        Height
  as an 2D Array, matrix
• Width =
  number of pixels in x-direction
• Height = number of pixels in y-direction
• Size (width x height, width > height)
• ROI = region of interest
   – To reduce the amount of data
                                                         17
       Spatial Image Resolution:
• Resolution
   – The size of an area in a scene that is represented
     by one pixel in the image
• Different Resolutions are possible (256x256….16x16)




• Lower resolution leads to data reduction!


                                                          18
    Digital Image Representation
• Pixel representation (bits)
   – A few words on bits and bytes: One bit: {0,1}
        • One byte = eight bits
   –   One pixel: one byte = eight bits = one number: [0,255] (show)
   –   Grey-scale, intensity, black/white: 8 bits = [0,255]
   –   Binary image: 1 bit {0,1}. Black and white: visualized as: 8 bit {0,255}
   –   Colors: next time




• Image representation (2D image versus 3D data)
                                                                         19
   – (show: 2D-gel: crop lower left corner, surface plot)
           Gray-level Resolution:
               Quantization




• Different gray-level resolutions: 256, 128, …, 2
• Less gray-levels leads to data reduction.
• For 256, 128, 64 gray-levels: Difference hardly visible

                                                            20
        Working with images….
• Image compression
• Image manipulation
   – Simple operations, e.g., scale image, improve quality
• Image processing
   – Improve the image, e.g., remove noise
• Image analysis
   – Analyze the image, e.g., find the person in the image
• Machine vision
   – Industry, e.g., Quality control, Robot control
• Computer vision
   – Everything: multiple cameras, video-processing, etc.
                                                             21
Fundamental Steps in Computer Vision
                                        Point 1: 22,33
                                        Point 2: 24,39
                                        …..


                                       Representation
                        Segmentation   and description

                                                         Actor sitting



        Preprocessing
                                                         Recognition
                              Knowledge base             and
  Problem                                                Interpretation
                                                                          Result
  domain Image
                                                                    22
          acquisition
                     Image file types
• image.jpg, image.tif, image.gif, image.png, image.ppm, ….
• Raw:
    –   No data is lost
    –   Header + data (234 235 32 21…)
    –   For example: image.pgm
    –   The file can be viewed
• Lossless compression:
    – No data is lost, but the file cannot be viewed
    – For example: image.gif
• Lossy compression:
    –   Better compression
    –   Some data is lost (optimized from the HVS’ point of view)
    –   The file cannot be viewed
    –   For example: image.jpg

                                                                    23
                Image file types
• Normally you don’t care about the file type
   – The application will take care of it for you:
   – For example: rotate
      • Application
          – image.x => raw
          – Rotate the raw image
          – Rotated raw => rotated_image.x
• But to write your own programs from scratch the
  images need to be in the raw format (without a
  header)

                                                     24
The camera




             25
              Sensor Chip Formats

Number of Pixels
from 500x500
to 5000x5000



Pixel size
from 4m x 4 m
to 16 m x 16 m


                                    26
Image formation - the lens




                             27
The lens




           28
                         The lens




• A lens focuses a bundle of rays to one point
• Parallel rays pass through a focal point F at a distance f beyond
  the plane of the lens. f is the focal length
• O is the optical center
• F and O span the optical axis                                  29
The lens




1 1 1
  
g b f
           30
           Focus

                                  1 1 1
                                    
                                  g b f




In focus           Not in focus




                                                  31
                                    (vis på oh)
        Focus and depth-of-field
• DK: Depth-of-field = dybteskarphed.
• Distance range in which the blur does not exceed a certain value




1 1 1
                                                             32
g b f
                   Aperture (DK: blænde)




Depth-of-field


 • More aperture => better depth-of-field
 • Downside: less light enters => increase exposure time =>
        risk of blur due to motion                            33
                Field-of-view




• Field-of-view, v, depends on size of the sensor and focal length
• ”Fisheye” lens => small focal length and large field-of-view
• Horizontal FOV and Veetical FOV                              34
                            Zoom

• Digital zoom (later)
• Optical zoom
  – The optics (lens) is
    changed => focal
    length, f, is changed




                                   35
                 A good image…

• Issues to consider:
   –   Dist. to object
   –   Motion of object
   –   Zoom
   –   Focus
   –   Depth-of-fields
   –   Focal length
   –   Shutter
   –   Field-of-view
   –   Aperture (DK: blænde)
   –   Sensor (size and type)

                                 36
Lighting in computer vision




                              37
Levels of Natural Light




                          [Burke]



                               38
39
40
41
                Lighting
• ”In machine vision lighting is more than 50%”
• Use controlled lighting!




• Avoid direct/indirect sunlight

• Avoid highlights
                                            42
Backlighting




               43
           Spherical Marker
• Viewpoint invariant
• High reflectance in illumination direction




                                               44
           Infrared Illumination
• Robust
                       Visually Opaque IR pass filter




                                                45
                What to remember
• Definitions
   – Images are discrete as opposed to continuous
   – Pixel: Grey-scale, intensity: 8 bits = [0,255]
• Computer vision system
   – Image acquisition, preprocessing, segmentation, representation,
     recognition
        • No clear definitions!
• Image file types: raw, compressed (lossy, lossless)
• Camera
   –   Focus
   –   Depth-of-field
   –   Zoom
   –   Field-of-view
   –   Etc.
                                                                       46
                      Exercises (1/3)
• Discuss the PE-questions (and write down the answers)
• Look at the x-tra slides and see if you can find other
  situations where image processing is used
• Given a 512 x 512 x 8bit image. How many different
  images can be made?
• We want to photograph an object, which is 1m tall and
  10m away from the camera. The height of the object in the
  image should be 1mm.
   – What should the focal length (f) be ?
   – (we assume that the object is in focus at the focal point, hence f=b)
                                                                   47
               Exercises (2/3)
• Mick is 2m tall and standing 5m from a camera.
  The camera’s focal length is 5mm.
   – A focused image of Mick is formed inside the camera. At
     which distance from the lens?
• How tall (in mm) will Mick be on the CCD-chip?
• How tall (in pixels) will Mick be on the CCD-chip?
   – The camera has a 1/2” CCD chip:


   – The camera image has a size of: 640x480 pixels
• What is field-of-view of the camera?
                                                         48
              Exercises (3/3)
• What is a CCD-chip and how does it operate?
• What is Depth-of-field (DK:dybteskarphed)?
• Given a 512 x 512 x 8bit image. How is the
  memory size reduced when you:
  – Decrease the grayscale resolution repeatedly by 2
  – Decrease the x-size and y-size of the image
    repeatedly by 2
                                     1 1 1
• Show that the following is true:     
                                     g b f        49
Xtra




       50
               Digital Image
• Why digital ?
• Before 1920: Image transmission from USA to
  Europe: more than a week: by ship!
• Early 1920s: Bartline cable picture transmission
  system: Transmission in three hours!
• Transmission via telegraph/wire, radio
  signals for newspapers

                                                     51
             Small Progress
• Small progress in digital imaging until 1964
• Jet Propulsion Lab (JPL) in Pasadena, CA
  – Transmission and correction of lunar images
    from Ranger 7.
• Not so good quality so the images had to be
  processed before they could be viewed
• Since then many applications…

                                                  52
Applications




               53
   Examples: Image Correction




• Needed when image data is erroneous:
  – Bad transmission
  – Bits are missing: Salt & Pepper Noise

                                            54
 Image Deblurring: Motion Blur




• Can be used when a camera or object is
  moved during exposure
                                           55
              Deblurring




• Can be used when the camera was not
  focused properly!!
                                        56
         Image manipulation
• Image improvement, e.g. too dark image




• Rotate + scale

                                           57
    Medical Image Processing




• Image Processing is widely used
• E.g. Analysis of microscopic images
                                        58
    Medical Image Processing




• MR/CT Imaging of a human body
• Use for Brain Surgery
                                  59
    Machine vision applications
• Classification     • Pose estimation
                        – Pick and place applications

 THOR       HOF
                    Conveyor belt

• Quality control       – Bin-picking



     THOR

                                                60
Machine vision




                 61
                        Biometrics
• Recognizing/verifying the identity of a person by
  analyzing one or more characteristics of the human body
• Characteristics:
   – Fingerprint, eye (retina, iris), ear, face, heat profile, shape (3D
     face, hand), motion (gait, writing), …
• Applications:
   – Verifying: Access control (bio-passports)
   – Recognizing: Surveillance: 9/11




                                                                   62
Chroma keying




                63
       Analysis of Sport Motions




•   Here: Analysis of motion of Sarah Hughes
•   3D Tracking of body parts
•   Motion interpretation
•   Action recognition
                                               64
           Motion Capture
• Special effects
  – Advertising
  – Movies




                        Andy Serkis   65
Motion Capture




                 66
67
               Illumination Setups
Directed
illumination                         Diffuse
                                     illumination




Rear                                  Light field
illumination                          illumination



                                           68
69
70

								
To top