Learning Center
Plans & pricing Sign in
Sign Out
Get this document free

An approach to a pseudo real-time image processing engine for hyperspectral imaging


Vol. 8 No. 7 October 2010 International Journal of Computer Science and Information Security

More Info
									                                                            (IJCSIS) International Journal of Computer Science and Information Security,
                                                            Vol. 8, No. 7, October 2010


   An approach to a pseudo real-time image processing engine for
                      hyperspectral imaging

        Sahar Sabbaghi Mahmouei*                           Prof.Dr.Shattri Mansor                               Abed Abedniya
     Smart Technology and Robotics Programme              Remote Sensing and GIS Programme,                     MBA Programme,
      Institute of Advanced Technology (ITMA),              Department of Civil Engineering,              Faculty of Management (FOM),
     Universiti Putra Malaysia, Serdang, Malaysia      Universiti Putra Malaysia, Serdang, Malaysia       Multimedia University, Malaysia

Hyperspectral imaging provides an alternative way of increasing             reflectance data such as extraction of various vegetation
the     accuracy    by    adding    another     dimension:      the         spectral features. Satellite-based remote sensing provides a
wavelength. Recently, hyperspectral imaging is also finding its             unique opportunity to obtain characteristics over large
way into many more applications, ranging from medical imaging               areas, whereas airborne remote sensing provides remotely
in endoscopy for cancer detection to quality control in the sorting
of fruit and vegetables. But effective use of hyperspectral
                                                                            sensed data over the medium scale, such as farms and
imaging requires an understanding of the nature and limitations             small watersheds [4]. However, these studies largely
of the data and of various strategies for processing and                    depend on the availability of spectral images that are
interpreting it. Also, the breakthrough of this technology is               usually quite expensive and need to be acquired by
limited by its cost, speed and complicated image interpretation.            professional image providers. Ground based hyperspectral
We have therefore initiated work on designing real-time                     imaging has been used as a cheap tool to acquire remotely
hyperspectral image processing to tackle these problems by using            sensed data from individual part of proposed area [4].
a combination of smart system design, and pseudo-real time                            In this paper, we propose an approach to pseudo
image processing software.  The main focus of this paper is the
                                                                            real-time image processing engine for hyperspectral
development of a camera-based hyperspectral imaging system for
stationary remote sensing applications. The system consists of a
                                                                            imaging to increase mission flexibility for environmental
high performance digital CCD camera, an intelligent processing              planning, medical diagnostics, remote sensing, and natural
unit, an imaging spectrograph, an optional focal plane scanner              resources     applications.     All   processes     in    the
and a laptop computer equipped with a frame grabbing card.  In              implementation of hyperspectral imagery and remote
addition, special software has been developed to synchronize                sensing apply near real time image processing done at the
between the frame grabber (video capture card), and the digital             spatial and numerical modeling laboratory (SNML) at the
camera with different image processing techniques for both                  University of Putra Malaysia. The main focus of this
digital and hyperspectral data.                                             research is the development of a camera-based
                                                                            hyperspectral imaging system for stationary remote
Keywords: Remote sensing, image processing, Real-Time,                      sensing applications.  Hyperspectral imaging provides an
frame grabber, hyperspectral, Hardware/Software Design.
                                                                            alternative way of increasing the accuracy by adding
                                                                            another      dimension:       the     wavelength. Recently,
1. Introduction                                                             hyperspectral imaging is also finding its way into many
                                                                            more applications, ranging from medical imaging in
Digital and Remote sensing image processing is nowadays                     endoscopy for cancer detection to quality control in the
a mature research area. Use of hyperspectral remote                         sorting of fruit and vegetables.
sensing in both research and operational applications has                             The impetus in performing this research was
been steadily increasing in the last decade. Hyperspectral                  given by existing snags and problems faced by workers in
imaging systems can capture imagery from tens to                            the field. So far, many of the image processing software
hundreds of narrow bands in the visible to infrared spectral                available in the market do not process images in real time.
regions. These systems offer new opportunities for better                   The software has to download and read the images first
differentiation and estimation of biophysical attributes and                and then prepare image-processing functionalities on them.
have the potential for identification of optimal bands                      In this paper, we attempt to show that it is possible to have
and/or band combinations for a variety of remote sensing                    pseudo-real image processing. This means that processing
applications [1-3],[11]. Different remote sensing                           is done on the fly: as soon as the camera captures the
applications have proven to be potential sources of                         image, the image processing algorithm comes into play
                                                                            immediately in all embedded applications.

 * Responsible author

                                                                                                        ISSN 1947-5500
                                                     (IJCSIS) International Journal of Computer Science and Information Security,
                                                     Vol. 8, No. 7, October 2010


          The Hyperspectral imaging system consists of               systems allow greater resolution of data to be assimilated
four components:                                                     than do line scanner systems.
     • A sensing component: a hyperspectral sensor
         (high performance digital CCD camera now
         known as the ImSpector manufactured by                      3. Real-Time Hyperspectral Imaging System
         SPECIM systems) for acquiring data or images.               Requirement
     • An optional focal plane scanner
     • A video capture (frame grabber) card connected                3.1. The sensor
         to the CPU from the camera helps in data capture.
                                                                     The hyperspectral sensor used in this study was a ground
     • Acer Extensa Notebook 4630z, which is a 2.0                   based user-friendly line sensor ImSpector (V10) See
         GHz Intel notebook, computer manufactured by                (Fig.1). The new ImSpector Fast10 is a high intensity
         Acer Inc., has been used as the CPU on the sensor           imaging spectrograph, and makes spectral imaging
         part.                                                       possible at hundreds and even up to 1500 images per
          The remainder of this article is structured as             second. ImSpector Fast10 imaging spectrograph
follows. Section 2 presents essential characteristics and            provides[5]:
concepts in the scope of the work. Section 3 presents
System Requirement. We will describe developing
                                                                         •    high light throughput
software in section 4. Description of proposed method,
                                                                         •    superior image quality
design and a relative technique are discussed in section 5.
Section 6 shows the experimental result and discussions.                 •    good spectral resolution of 15 nm
Section 7 presents the conclusion of this paper.                         •    full VNIR spectrum of 400 - 1000 nm over a
                                                                              narrow dimension, allowing short read out times
                                                                         •    maximum light intensity on the camera
2. Concepts and characteristics
                                                                              pixels,allowing short integration times
In order to draw a clear picture of fundamental concepts                 •    high speed acquisition in many low cost industrial
and characteristics of hyperspectral imaging, it is                           CCD and CMOS cameras
important to recap some key concepts and definitions
which are accepted by experts in this field.                                  The ImSpector imaging spectrograph is a
                                                                     component that can be combined with a broad range of
        Real-time image processing: operating systems                monochrome matrix cameras to form a spectral imaging
serve application requests nearly real-time. In the other            device. Equipping the instrument with an objective lens
word manipulation of live images, typically within 50 to             coupled with a monochrome area camera, converts
100 milliseconds, so the human user perceives them as                ImSpector to a spectral line imaging camera. Operation is
instantaneous.                                                       based on the direct sight imaging spectrograph technology
        Embedded systems: An embedded system is                      of the Spectral Imaging Ltd. (SPECIM), Oulu, Finland [6].
a computer system designed to perform one or a few                   ImSpector captures a line image of a target and disperses
dedicated          functions often          with real-time           light from each line image pixel to spectrum. Each spectral
computing constraints. It is embedded as part of a                   image then contains line pixels in the spatial axis and
complete device often including hardware and mechanical              spectral pixels in the spectral axis (Fig. 2) [4]. It is possible
parts.                                                               to acquire full spectral information for each line image
                                                                     acquired from the target. Since ImSpector captures
        Engine: The image processing engine, or image                sequential images of the moving target (or the sensor itself
processor, is an important component of a digital                    moves), a 2D spectral image can be formed. This
camera and plays a vital role in creating the digital image.         technology allows diverse opportunities to analyze the
The image processing engine comprises a combination of               target accurately based on its spectral features.
hardware processors and software algorithms. The image
processor gathers the luminance and chrominance
information from the individual pixels and uses it to
compute/interpolate the correct color and brightness values
for each pixel.

        Pushbroom: In remote sensing, an imaging device
consisting of a linear array of sensors (CCD camera)
which is swept across the area of observation. Pushbroom                     Fig. 1– hyperspectral sensor (ImSpector V10).

                                                                                                     ISSN 1947-5500
                                                                (IJCSIS) International Journal of Computer Science and Information Security,
                                                                Vol. 8, No. 7, October 2010


3.1.1 Advantages of hyperspectral imaging system                               and projects a two-dimensional image profile (line image)
                                                                               onto the CCD surface. This configuration allows image
Hyperspectral imaging is extremely advantageous in terms                       acquisition under stationary or laboratory settings [2], [10].
of its data, presenting the information in the spatial
direction which is useful for extracting information with                      3.3 A video capture (frame grabber)
less loss of data. Some advantages of hyperspectral
imaging over conventional techniques such as: NIRS                             The FrameLink frame grabber is a TYPE II PC Card with
(Near-infrared spectroscopy), RGB, and hyperspectral                           both a Camera Link and Card bus interface. It provides the
imaging are shown in Table 1 [7, 8].                                           ability to capture digital video data from a ‘base
                                                                               configuration’ Camera Link interface and transfer that data
Feature            RGB             NIRS           MSI           HSI            to host memory via a Card bus (PCI) interface. The frame
                   imaging                                                     link is a professional state of the art PCMCIA card bus
                                                                               digital video capture card, allowing user to display,
Spatial                 √                             √          √             capture, store and preview mega pixel video image (up to
information                                                                    16 mega pixels) on the notebook computer [9]. The
                                                                               Imperx FrameLink video capture card is as shown in
                                                                               (Fig.3) below.
Spectral                               √           Limited
information                                                      √

constituent         Limited            √           Limited

Sensitivity to
minor                                              Limited
Components                                                       √
                                                                               Fig.3 – The IMPERX FrameLink Fast CardBus video capture (frame
                                                                               grabber) card. This picture has been taken from the official website of
      Table.1 Advantages of hyperspectral imaging system                       Imperx Inc.

                                                                               3.4 The computer system

                                                                               The computer is an Intel Pentium III (800 MHz) processor
                                                                               based system with 250 GB hard drive. The operating
                                                                               system on the computer is Microsoft Windows XP. A PCI
                                                                               interface board provided with the imaging system is
                                                                               installed in a master PCI slot in the computer. The utility
                                                                               software is installed in the computer for complete camera
                                                                               control, image acquisition and applies image processing
                                                                               technique. The Acer Notebook computer is as shown in
                                                                               (Fig. 4) below.

              Fig. 2 – The operating principles of ImSpector.

3.2 An optional focal plane scanner

The focal plane scanner performs line scanning across an                              Fig.4 – Different views of the Acer Extensa 4630z Notebook
input imaging area within the focal plane of the front lens                    computer. This has been used as the CPU on our hyperspectral imaging
and the spectrograph disperses each line into a spectrum                       system. These pictures have been obtained from Acer Inc.

                                                                                                               ISSN 1947-5500
                                                             (IJCSIS) International Journal of Computer Science and Information Security,
                                                             Vol. 8, No. 7, October 2010


                                                                                     Furthermore, the functions embedded in the
3.5 Acquiring ground-based hyperspectral images                             popup menu endow the interface with many other image
                                                                            processing and parameter extraction capabilities.
The ground-based hyperspectral line imaging systems is
shown in (Fig. 5). The hyperspectral sensor ImSpector
captures the scene. ImSpector captures a line image of the
scene and disperses it into a spectrum. By moving the
sensor up and down or left and right by means of a battery-
powered movable tripod base, the whole scene is captured.
The rate of image acquisition can be up to 30 fps, and data
can saved in an audio–video interleave (avi) file format.
The raw spectral data obtained by the sensor and the image
generated by applying a line pixel assembly algorithm to
the raw data in an image [4]. Each frame represents the
spectral data corresponding to a spatial line. The x-axis of
each frame is the spatial axis and the y-axis is the spectral
axis. Each frame is composed of 480 spectral lines, each
representing spectral data at a particular wavelength. In
order to facilitate comprehension of these spectral data, an                               Fig. 6 – Software interface
image is generated by applying a line pixel assembly
algorithm to every frame. Assembly of spectral lines with                              The Image Processing Toolbox provides a
an equivalent wavelength from all frames makes one                          comprehensive set of standard algorithms and graphical
image, and thus the procedure can generate a total of 480                   tools for image processing, analysis, visualization, and
images, each displaying the scene captured with a different                 algorithm development. You can restore noisy or degraded
wavelength [6].                                                             images, enhance images for improved intelligibility,
                                                                            extract features, analyze shapes and textures, and register
                                                                            two images. Most toolbox functions are written in the C++
                                                                            language. A schematic diagram of the interface design and
                                                                            its utilities is shown in (Fig. 6, 9).

                                                                            4.1 Some key features of image acquisition toolbox

                                                                                •   Image enhancement, including linear and
                                                                                    nonlinear filtering, filter design, and automatic
                                                                                    contrast enhancement;
                                                                                •   Binarization filters (threshold, threshold with
                                                                                    carry, ordered dithering, Bayer dithering, Floyd-
                                                                                    Steinberg, Burkes;
          Fig. 5 – Hyperspectral image acquisition system.                      •   Color image processing, including color space
                                                                                    conversions and channel replacing, channel
4. Developing the software                                                      •   Spatial transformations and image registration,
                                                                                    including a graphical tool for control-point
Once the hyperspectral images are generated, they would                             selection;
appear as a stack of continuous images. Manipulation of                         •   Fourier transformation (low pass and high pass
hyperspectral images and extraction of useful spectral                              filters;
information from these multidimensional data requires the                       •   Mathematical morphology filters (erosion,
development of intelligent software. For this purpose,                              dilatation, opening, closing, hit & miss, thinning,
software with many high level computing and                                         thickening);
visualization functions embedded in a number of useful                          •   Edge detectors (homogeneity, difference, sobel,
toolboxes. (Fig.6). illustrates the main menu and its user                          canny);
interfaces for image processing and data extraction.
                                                                                •   Median filter, Adaptive smoothing, Conservative

                                                                                                         ISSN 1947-5500
                                                            (IJCSIS) International Journal of Computer Science and Information Security,
                                                            Vol. 8, No. 7, October 2010


5. Methods
                                                                            Preliminary image acquisition testing trials indicate that
In this technique only hyperspectral data for areas of                      this CCD camera-based hyperspectral imaging system has
interest captured. The principal component analysis                         potential for agricultural and natural resources
method is used to hyperspectral imaging systems shown in                    applications. (Fig.8) shows the architecture of the ground-
(Fig.8.)                                                                    based hyperspectral line imaging systems based on
                                                                            architecture at SMNL laboratory in UPM.
         Based on the raw data, the software developed is
used to generate images for a three-dimensional, including
two spatial axes and one spectral axis that can be produced
in one of four ways: Fourier transform imaging, point-to-
point spectral scan in a spatial grid pattern, line by line
spatial scan i.e. the pushbroom technique, and wavelength
tuning with filters. The line by line spatial scan and
wavelength methods are more suitable. In the pushbroom
method spectral data acquire across the full spectral range
of single spatial lines consecutively to reconstruct the
hyper spectral tube.

          The CCD camera provides 1280(h) x 1024(v)
pixel resolution and true 12-bit dynamic range. The
imaging spectrograph is attached to the camera via an                         Fig. 8 – architecture of the ground-based hyperspectral imaging
adapter to disperse radiation into a range of spectral bands.               systems
The effective spectral range resulting from this integration
is from 400 nm to 1000 nm. Diffuse illumination of the
sample is made possible by a florescent-halogen or LED                      6. Experimental result and discussion
source [6]. The light reflected from the target enters the
objective lens and then spread into its component
                                                                            In order to test the sensor design concept and to integrate
wavelengths as shown in (Fig7).
                                                                            software design, we simulate a realistic scene. The Digital
                                                                            Imaging and hyperspectral software, developed at Institute
          The optional focal plane scanner can be attached
                                                                            of Advance Technology (ITMA) in Malaysia. By scene
to the front of the spectrograph via another adapter for
                                                                            simulation and sensor modeling, we hope to reduce the
stationary image acquisition. The camera and the frame
                                                                            cost and development time in new sensor designs, together
grabbing card are connected via a double coaxial cable,
                                                                            with the support of the algorithm and the techniques
and the utility software allows for complete camera control
                                                                            method. The image processing algorithms are designed
and image acquisition. The imaging system captures one
                                                                            only to demonstrate the idea of effectively capturing
line image for all the bands at a time and the focal plane
                                                                            hyperspectral data. Needless to say, more sophisticated
scanner serves as a mobile platform to carry out
                                                                            algorithms need to be developed for more challenging
pushbroom scanning in the along direction.
                                                                                    After software executed, the main window will
                                                                            appear. The main window provides the primary area for
                                                                            viewing real-time images received from the camera. When
                                                                            image viewing is active, pull-down menu with two options
                                                                            reveals: ‘Player’ and ‘Exit’. Player button will toggle
                                                                            between ‘Start Grab’ and ‘Stop Grab’ every time the user
                                                                            clicks on it. By clicking on ‘Start Grab’ enables the engine
                                                                            and causes the main window to display live images
                                                                            received from the camera. Clicking on ‘Stop Grab’
                                                                            disables the engine and causes the display to freeze. When
                                                                            recording images to disk, Image Format option selects the
                                                                            format, ‘BMP’, ‘JPEG’ or ‘TIFF’ that the image will be
                                                                            saved in. Selecting ‘JPEG’ activates a compression slider.
                                                                            ‘Best Quality’ provides the least compression while
                                                                            ‘Smallest File’ provides the most compression.
Fig 7– A Scheme diagram the of current Hyperspectral imaging system

                                                                                                          ISSN 1947-5500
                                                              (IJCSIS) International Journal of Computer Science and Information Security,
                                                              Vol. 8, No. 7, October 2010


           In order to evaluate the system and simulate a
realistic scene we pluck the leaf from the tree in the fields
near our campus and around University of Putra Malaysia.
In hyperspectral imaging system design, different portion
of bandwidth can be selected and determined by analyzing
model spectral profile combined to a single image profile
and a binary decision was made using a threshold found by
experience. Thus, object can be demonstrated in real time.
In (Fig .9) we show a single image snapshot captured, and
the result was combined to produce a co-registered
composite image. After capture the scene raw data save as
‘JPEG’ format then we apply some image processing
technique in order to assess our software.
                                                                                          Fig.10 – Apply Thresholding Filter
Here we select histogram filter to determine the overall
intensity of the image that is suitable for our inspection                            The hyperspectral line sensor captures the raw
task. Based on the histogram data, you can adjust your                       reflectance data in the approach that illustrated above. As
image acquisition conditions to acquire higher quality                       this is a ground-based system, the cost is much lower than
images see (Fig.9).                                                          for airborne- or satellite-based remotely sensed data. The
                                                                             nominal spectral resolution of 1.5–2nm within the
                                                                             wavelength range of 400–1000nm is sufficient for most
                                                                             application studies. The software developed serves a
                                                                             pivotal role in dealing with the spectral data that are
                                                                             captured. It can generate images from the raw spectral data
                                                                             in an audio–video interleave format or image format.
                                                                             Useful image analysis algorithms are included, such as;
                                                                             Thresholing and other functions determine whether an
                                                                             image meets certain criteria for inclusion in an analysis.

                                                                             7. Conclusions

                                                                             This paper reviews the recent developments in ground-
                                                                             based hyperspectral imaging system for acquisition of
      Fig.9 – captures the scene and apply Histogram Filter
                                                                             reflectance data that is useful for many real-life
                                                                             applications such as; environmental planning and natural
Another filter that applied for evaluate our system work is
                                                                             resources applications. The hyperspectral imaging
thresholding (Fig. 10). Objective of thresholding filter is
                                                                             technique described in this article provides a new
converting the image into binary objects. Thresholding is
                                                                             opportunity for determining the optical properties and
the simplest method of image segmentation. From
                                                                             quality of product such as food and agricultural products.
a grayscale image, so we could apply the basic
morphology processes, Image analysis capability can also
                                                                                    Compared to other techniques, the hyperspectral
be expanded to include other types of analytical techniques
                                                                             imaging technique is simpler, faster and easier to use, and
for a particular image analysis purpose.
                                                                             more importantly it is capable of determining optical
                                                                             properties over a broad spectral range simultaneously. The
                                                                             technique also is useful for measuring the optical
                                                                             properties of turbid food and agricultural products.
                                                                             Moreover the hyperspectral imaging technique is
                                                                             potentially useful in assessing, sorting, and grading fruit

                                                                                                           ISSN 1947-5500
                                                          (IJCSIS) International Journal of Computer Science and Information Security,
                                                          Vol. 8, No. 7, October 2010


[1] D.Tuia, and G.Camps-Valls, "Recent advances in remote
sensing image processing", IEEE International Conference
on Image Processing (ICIP), 2009 , 3705 – 3708.                            AUTHORS PROFILE
[2] H. James Everitt, and R. Michael Davis , and Chenghai
Yang, "A CCD Camera-based Hyperspectral Imaging System for
Stationary and Airborne Applications" , Geocarto International,
Vol. 18, No. 2, June 2003.                                                                 Sahar Sabbaghi Mahmouei is currently
                                                                                           doing her Master degree in Institute of
[3] H. James Everitt, Chenghai Yang, Joe M. Bradford and Dale                              Advanced Technology and Research
Murden, Airborne Hyperspectral Imagery and Yield Monitor                                   (ITMA), Universiti Putra Malaysia, UPM.
Data for Mapping Cotton Yield Variability, Volume 5, Number                                Sahar has received her B.Sc in software
5, 445-461,2004.                                                                           computer engineering field in 2006 from Iran
                                                                                           Azad University. Her research interest
                                                                                           includes image processing, machine vision,
[4] Xujun Yea, Kenshi Sakaib, Hiroshi Okamotoc, Leroy O.                                   artificial Intelligence and e-commerce.
Garcianod," A ground-based hyperspectral imaging system for
characterizing vegetation spectral features",Elsevier Science
Publishers B. V, Volume 63, NO 1, 13-21 , August 2008.

[5] Official website of Spectral Imaging Ltd, Finland.

[6] Users Manual for Imspector spectrograph Ver.2.0 from
SPECIM website.

[7] A. A., O'Donnell, C. P., Cullen, P. J., Downey, G., and Frias,
J. M. 2007. "Hyperspectral Imaging - an Emerging Process
Analytical Tool for Food Quality and Safety Control",Volume 18,
Issue 12, 2007, 18(12); 590-598.

[8] Osama M. Ben Saaed, Abdul Rashid Mohamed Shariff,
Helmi Zulhaidi Mohd Shafri, Ahmad Rodzi Mahmud,Meftah
Salem M Alfatni, "Hyperspectral Technique System for Fruit
Quality Determin ", Map Asia 2010 and ISG 2010 Conference,

[9] Official website of Imperx Inc,USA .

[10] C. Mao., "Hyperspectral imaging systems with digital CCD
cameras for both airborne and laboratory application", 17th
Biennial Workshop on Videography and Color Photography in
Resource Assessment, American Society for Photogrammetry
and Remote sensing, Bethesda, MD. pp. 31-40,1999.

[11] C. Mao., " Hyperspectral focal plane scanning-an innovative
approach to airborne and laboratory pushbroom hyperspectral
imaging. Proc. 2nd International Conference on Geospatial
Information in Agriculture and Forestry", ERIM International,
Inc.,Ann Arbor, MI. Vol. 1, pp. 424-428, 2000.

                                                                                                     ISSN 1947-5500

To top