A. Binocular Vision Theory

Document Sample
A. Binocular Vision Theory Powered By Docstoc
					          Wood Stand Positioning and Ranging System
              Design Based on Binocular Vision
                                            Yixuan Wang, Jiangming Kan, Junmei Zhang

 Abstract—Forestry is one of the economic lifelines in
                                                                       Yixuan Wang, Beijing Forestry University.100083,Qinghua
national development, forestry robots’ modernization and
                                                                    Donglu35, No.1072 Mailbox, China
intelligent is the requirements of the progress of the times.
According to inefficient status of our forestry robot technology
                                                                       Jiangming Kan, Beijing Forestry University, 100083 China.
and equipment, we propose a DSP + ARM platform to study a
set of positioning and ranging system used in forestry robot
                                                                       Junmei Zhang, Beijing Forestry University, 100083 China.
obstacle avoidance system, based on the binocular vision
theory. Calculate the three-dimensional information by
acquiring the images through binocular cameras, output
targeting and ranging results and ensure that the measurement      using the feature point extraction and matching to finish
error is less than 5%. The final experimental results show that    the completion of the 3D model is widely used among
the system design is feasible.                                     the three-dimensional vision technology workers. To this
 Keywords—binocular vision; target recognition; positioning;       end, the image feature point extraction and matching
ranging                                                            results will directly affect the accuracy of the subsequent
                                                                   three-dimensional reconstruction; it is also a key
                          I. INTRODUCTION
                                                                   technology in the study as a multi-sensor image fusion,
      With the further development of the computer,                as discussed by [1].
intelligent   control,     sensor    technology,    IT,   SCM           According to the more artificial and intelligent need
technology     etc.,     people’s   requirement     for   robot    in our artificial intelligence industry development
performance is also increasing. Computer vision is one of          process, research the targeting and ranging system based
the hot issues of today's researches; the aim is to make           on binocular vision applied in the field of forestry
machines with visual perception similar to the human.              machines, and thus greatly reduce the labor intensity,
Binocular vision is one important form of machine                  improve work efficiency, with a very important practical
vision; it is based on the parallax principle and obtains          significance, as discussed by [2].
two images of the detected object from different
                                                                                          II. SYSTEM DESIGN
locations, then get the 3D geometry information by
calculating the positional deviation between the points.                In this paper, we propose to research a set of
The research results are widely used in the field of robot         forestry robot visual targeting and ranging system basing
navigation, industrial measurement precision, object               the binocular vision theory on the TI platform. Firstly the
recognition, virtual reality scene roaming, site survey, as        system acquire the images by two CCD digital cameras,
well as military. In image-based 3D reconstruction                 DSP subsystem process and calculate the images,
techniques,                                                        ARM subsystem control the whole system, and finally
                                                                   output the targeting and ranging results to the VGA
                                                                   display.   Applying binocular vision theory to simulate
                                                                   the human eyes in targeting and ranging process can
                                                                   make the output more accurate. A complete targeting and
                                                                   ranging system should include vision acquiring and
                                                                   processing platform and software development platform,
                                                                   the overall flow chart is shown in Figure 1.
Images Acquisition By        Image         Pixel        Target Positioning     Display
                                                                                           the power supply circuit, the USB OTG circuit, network
  Binocular cameras      Preprocessing    Matching         and Ranging       the results
                                                                                           circuit, display circuit etc., as discussed by [3].
                          Figure 1 Overall flow chart
                                                                                             B. Camera Calibration

                        3 BUILD THE HARDWARE PLATFORM                                        Camera calibration of stereo vision system means to
                                                                                           determine the mapping relationship between the left and
       This study is based on the theory of binocular vision
                                                                                           right camera image to location in three-dimensional
and designs the hardware system on the DSP + ARM
                                                                                           scene, is a crucial step to achieve the reconstruction of
platform; achieve the goal of positioning and ranging
                                                                                           three-dimensional model. Binocular vision can calculate
functions in the process of forestry work. The entire
                                                                                           the relationship between the 3D and 2D by the
system we studied includes image acquisition module,
                                                                                           corresponding pixels between two dimensional images,
DSP high-speed computing module, ARM system
                                                                                           and get the three-dimensional coordinates of the object,
control module; the image encoding output module, the
                                                                                           such correspondence in three-dimensional reconstruction
peripheral circuit module, etc., the hardware system
                                                                                           is particularly important. In this experiment, we select the
design flow is shown in Figure 2.
                                                                                           exactly same two cameras to set up the binocular stereo
  Left Video    Video Port 0    Video
 Input Signal                  Decoder               EMIF                                  vision image acquisition platform to minimize the error
                                              DM3730                          VGA output
                                                               Encoder                     of the calibration results. And then refer to Zhang
  Right Video   Video Port 1     Video               EMIF
 Input Signal                   Decoder                                                    Zhengyou calibration method constantly modify the
                                                                                           relative position between the two cameras, as discussed
                   Fig. 2 hardware system design flow
                                                                                           by [4].
   A. Determine the TI Core Board
                                                                                             C. Build the Development Platform
   Texas Instruments Incorporated (TI) as the world's
                                                                                             Firstly we should install the Linux platform, there are
leading semiconductor company prove innovative digital
                                                                                           a lot of versions of the Linux operating system on the PC
signal processing (DSP) and analog technologies for the
                                                                                           machine, it is recommended to use the Red Hat
real world. TI's products involve a number of areas, in
                                                                                           Enterprise Linux v4 and Fedora Core v9. It is proved that
addition to the semiconductor business, it also provides
                                                                                           the two versions are more convenient operation, and
sensing and control, education products and digital light
                                                                                           better compatibility in the process of experiment.
processing solutions. TI is headquartered in Dallas,
                                                                                           Secondly, DVSDK suite provides a complete software
Texas, USA, and has manufacturing, design or sales
                                                                                           development environment for the processors to use the TI
operations in several countries. Referring many video
                                                                                           DM37x family, can be used in WinCE, Linux, and
applications processors, We choose TI’s DaVinc DM37x
                                                                                           Android three operating systems. Containing the
system with the experimental requests, DM37x processor
                                                                                           underlying drivers, it is user-friendly to evaluate DaVinc
combines the ARM Cortex-A8 and TMS320C64x + ™
                                                                                           series processors, design the system drives and software
DSP core, the kernel proves a wide range of digital video
                                                                                           applications for the users. Users are convenient to
equipment with optimization.
                                                                                           exercise DaVinc processor’s powerful data computing
   The system has a modular hardware structure to meet
                                                                                           quickly and can reduce product development cycle,
the requirements of real-time, accuracy and reliability.
                                                                                           achieve the rapid products market launch, such as
Each device is working in parallel to achieve the high
                                                                                           consumer       electronics,    medical       instrumentation,
efficiency of processing. DSP chips compared to
                                                                                           multimedia processing, video surveillance, industrial
traditional PC image processing significantly faster, with
                                                                                           control products. DaVinc processor is a feature-rich
the advantage of the other processors cannot replace. In
                                                                                           development board and provides a quick and easy
this experiment, DSP subsystem is used for image
                                                                                           development platform for embedded designers. Finally,
preprocessing and calculates the positioning and ranging
                                                                                           the terminal program allows the user to control the
information of target. The ARM subsystem is responsible
                                                                                           development board and run the program through the
for controlling the reset, call of all the modulus,
                                                                                           serial port of the host side. In the experiment using the
coordinate the workflow of the entire system and full use
                                                                                           ZOC Terminal, you can also choose the Windows
of its advantages. The peripheral circuit design includes
                                                                                           HyperTerminal or MINICOM under Linux, as discussed
by [5].                                                                         TUSB1210BRHBT and Upd7720110 chip to extent the
                                                                                core board to the external ports, form a set of four USB
                                                                                differential networks. In this experiment, the binocular
     D. Modular Design of Hardware System
                                                                                cameras are connected via the USB ports.
     The system has the modular hardware structure to
                                                                                6. Ethernet
meet the requirements of real-time, accuracy and
                                                                                   In the experiment, we use LAN9221i-ABZJ chip to
reliability. Each device is working in parallel to achieve
                                                                                provide 16bit 10M/100M high-performance Ethernet
the high efficiency of processing. DSP chips’ processing
                                                                                RJ45 interface; be used for communicating between the
speed is significantly faster compared to traditional PC
                                                                                host and network,
image, with the advantages of other processors cannot
                                                                                7. Display
replace. In this experiment, DSP subsystem was applied
                                                                                   Display module converts the output signals of the
in image preprocessing, and positions the target and
                                                                                DM3730's by two 74ALVC164245DGG to LCD signal
calculates    the     ranging.        The       ARM            subsystem is
                                                                                firstly,   and   then   transfer   to   ADV7125KSTZ140
responsible for the reset, call control of each module and
                                                                                converted to VGA signal, finally packaged into the
coordinate the workflow of the entire system, full use of
                                                                                standard VGA output interface, users can change the
its advantages. The design of peripheral circuits include
                                                                                proportion of pixels displayed by a final control terminal.
the power circuit, USB circuit, Ethernet circuit and
display circuit etc., the hardware system block diagram is                                       4   BUILD SOFTWARE PLATFORM
shown in Figure 3.
                                                                                A. Binocular Vision Theory
                              Power and                   Real-time
                                          VGA Display
                                 Set                        clock
                                                                                   Binocular stereo vision technology is an important
             Series Ports
                                                                                branch of the field of computer vision. Two cameras
                                                                                observe the same scene from different angles and get a
                                                                                set of images, observe the differences combing the two
             Host network         FLASH                   SDRAM
                              Program Memory           Data Memory              eyes images, make us to get the obvious sense of the
             Fig. 3 hardware system block diagram                               depth and build the corresponding relationship between

1. B2B interface                                                                the characteristics, compare the physical points of the

     This module was designed to connect the core board                         same space in the different images, this difference, we

with the peripheral circuits.                                                   call it Disparity. Binocular stereo vision technology is

2. Power supply                                                                 familiar to the human visual, the widely broad

     The DM37x system integrates the TPS65951 power                             application prospects attract the widely attention. With

management module to meet the complex system power                              the level of research development, binocular stereo vision

requirements. In the power supply module design, we                             technology has increasingly wide range of applications

have adopted the TPS54286PWP power chip to provide                              such as robot visual navigation, medical image analysis,

power for TPS65951.                                                             product testing and measurement, analysis aerial images

3. RAM FLASH                                                                    and 3D map restructure. In the current video surveillance

     There is 32bit 256MByte DDR SDRAM and 16bit                                study, mostly researches are based on the monocular

256MByte NAND Flash memory module inside the                                    vision. Each time Monocular vision only takes an angle

DM37x system.                                                                   image, although the fast calculation speed, short reaction

4. Series ports                                                                 time, but lost the three-dimensional depth information of

     This    experiment            uses         SP3203EEY-L              and    the scene. From this perspective, the binocular stereo

SN74LVC2G07DCK chip to provide the system with a                                vision has irreplaceable advantages. Therefore, the

5-wire RS232 interface, 3-wire system UART1-3 ports,                            binocular stereo vision is becoming the focus of the

and two I2C ports for clock and data signals.                                   study, with great significance in the accurate tracking and

5. USB                                                                          three-dimensional       measurement.    Binocular    vision

     USB module provides users with four USB 2.0 ports,                         measurement method has the advantages of high

which can be used for mounting mobile devices with up                           efficiency, appropriate accuracy, simple system structure

to     480Mbit/sec          port    speeds.        The          system   uses   and low cost, very suitable for manufacturing site online,
non-contact product testing and quality control. In the              Information extraction of image features is the premise
measurements of moving objects (including animal and                 and foundation of calibrating of the computer vision
human body), as the image acquisition is completed in an             system model parameters and practicing application of
instant, so the three-dimensional visual method is a more            computer vision. The basic task of feature extraction is
effective measurement method. The binocular vision                   identifying the most effective features from a number of
system is one of the key technologies of computer vision;            characteristics in order to achieve compression of the
Achieve the distance information in three-dimensional                feature space. It contains special points (such as corner
scene is the basis in computer vision research, as                   points), line, boundary, regional and other values or
discussed by [6].                                                    symbols description. The extraction of feature points is
                                                                     the main task in this experiment. According the target
                                                                     shape features, we extracted the feature points (corners)
                                                                     and the edge as the main features in the binocular vision
                                                                     feature points matching, as discussed by [8.9].
                                                                       Corner point has two versions of the definition: First,
                                                                     the corner point is the intersection point of two edges, the
                                                                     points change greatly in the horizontal and vertical
                   Fig. 4 binocular vision system                    direction; second, corner point is the characteristic points
  The           triangular           geometric            relation   with two main directions in neighborhood. Since the
is: u1  f
              x ,        ( x c  b) ,             yc                 1970s, a lot of corner detection algorithms appeared. In
                c u2  f              v1  v 2  f c .
              z               z c
                                                   z                 our experiment we choose the template-based corner
                                                                     detection method, mainly the Harris and SUSAN corner
(x c ,y c ,z c ) is the coordinate in the left camera
                                                                     detections to complete the feature point extraction, these
coordinate system. b is the baseline distance, f is the              two methods corner detection methods are both based on
                                                                     gray-scale change. The images processed are shown in
focal length of the two cameras. (u 1 ,v 1 )and(u 2 ,v 2 )are
                                                                     Figure 5
point P’s coordinates in the left image and right image
respectively. The vision defines it as the alternate
position     spike     as    a    point    in       two   images:
                     f  b , then we can get point P’s
d  (u1  u 2 ) 
coordinate in the left camera coordinate system:                                          Fig.5 processed images
       b  u1 c   b v c   b  f , as discussed by [7].
xc          ,y      ,z 
        zc         d         d                                                                5 CONCLUSIONS

  B. Image Feature Points Extraction and Matching                         3D reconstruction is one hot difficulty point in
                                                                     computer vision, the image feature point extraction and
  Image feature extraction and matching is a very
                                                                     matching    result     will   impact     the   accuracy   of
important part of computer vision, including the point,
                                                                     three-dimensional reconstruction directly, it is also a key
line, surface characteristics of the image, feature points
                                                                     technology in the study as a multi-sensor image fusion.
are the image features studied in this paper. Feature
                                                                     The system has the DSP + ARM core modules, each
points is the most common form of image features,
                                                                     module circuits are integrated on a board. The system
including object edge points, corner points, lines and
                                                                     capture the image data by the imaging technology of
cross points. The common feature point extraction
                                                                     high-speed space CCD camera, then converted to
methods include the interest operator method, edge
                                                                     receivable signals, lastly sent to the DSP subsystem. The
points extraction based on wavelet transform, corner
                                                                     DSP subsystem is responsible for image processing and
detection method etc.. Corner detection method is a
                                                                     calculation and the ARM system controls and calls for
gray-based test method; it can calculate the curvature and
                                                                     the total system. The advantages are structural flexibility,
gradient of the point to detect the corner points by
                                                                     versatility, operation and processing efficiency; at the
considering the change of the pixels in intensity field
same time the development cycle is short, easy to
maintain and extend the strong ability to adapt to
different algorithms. Image acquisition technology based
on two video cameras can position the target and
measure the distance according the anthropomorphic
binocular visual imaging principle, it is more practicality
and accurate compared with the single-camera image
acquisition, this technology will have a promising
development. Combing embedded system development,
binocular visual theory with the forestry robot to apply in
obstacle avoidance system is significant to the forestry
development. Although this article has basically reached
the expected results, but still there is a great distance
from the practical application. Especially in terms of
accuracy, there is a big gap compared with other
advanced countries. I believe the project will be more
in-depth and maturity with the development of computer
image processing technology,

[1]   Haijun Li, Fengsheng Xu, Jianchen Zhang. The application
      research of automatic identification system based on binocular
      vision[J]. Changchun University Technology (Natural Science),
      2007.9: 30.3.
[2] Jiao Song, Anxin Fu. Embedded robot target recognition systems
      based on the DM642 [J]. Electrical and Mechanical Engineering,
      2008.1: 25.1.
[3] Texas Instruments Incorporated TM320DM3730 Technical
      Overview, July 2010.
[4] De Xu, Min Tan, Yuan Li. Robot vision measurement and control
      (2nd edition) [M]. Defense industry Press, 2011.5.
[5] Texas Instruments Incorporated TM320DM6446 Technical
      Overview, August 2009.
[6]   Zheng Yang, Fuxin Zhou. The structure design of mirrored
      single-camera     vision   sensor[J].   Mechanical   Engineering
      Journal, 2011.11: 47.22.
[7] Yanbin Hou, Yushan Li, Jing Zhang. The mathematical image
      processing system based on DSP and binocular CMOS camera
      [J]. Electronic Application Technique, 2006.1
[8] Zhanguo Wei, Jiangming Kan, Jinhao Liu. Large-scale forestry
      equipment design based 3D virtual technology [J]. Computer
      2012: 26, 10-1.
[9]   Lei Li, Wenbin Yang, Jiangming Kan. Stumpage information
      extraction method based on the mathematical morphology [J].
      2008.5: 24.3

Shared By: