Research on a novel system of optical image processing
Lei Liu, Hongmin Yu*, Zhimin Zhao, Lei Ji
Nanjing University of Aeronautics and Astronautics
29, Yudao str., Nanjing, 210016, China
Tel.: + 86 84892011 Fax: +86  84892011 *E-mail: firstname.lastname@example.org
Traditional optical image processing systems are mostly based on PC and image
acquisition cards are often needed. So they are restricted in many fields for their volume and
high cost. A novel system of optical image processing based on FPGA and DSP is advanced.
It consists of three parts: image acquisition system, image processing system and image
display system. Image acquisition system is made up of CMOS image sensor, FPGA and
image buffer memory. DSP is adopted as the key element of the image processing system. An
image memory is also used for storing image data. Program of optical image processing can
be written into DSP and can be executed. The optical images processed can be transmitted to
a small LCD for display of the results. The system can operate conveniently, smoothly and
inerrably with high speed and precision by experiments and will have more significant
application in more optical measurement fields.
Keywords: Optical image processing, measurement, FPGA, DSP
As an important part of optical measurement technology, the technology of optical image
processing attracts more attention for its non-contact and accurate measurement . Optical
images, such as infrared image, laser interference pattern, holographic image and speckle
image, are widely applied in civil and military fields. For example, we cannot see the objects
by our eyes at late night, but we can detect and recognize the objects by analyzing their
infrared images. We can measure the size of small hole or work piece by laser interference
pattern, which cannot be measured by common graduated scale. We can also detect the
deformation and micro-displacement of the work piece by laser speckle image . At present,
most systems of optical image processing are based on PC and operate with image acquisition
cards. So they are restricted in many fields for their volume and high cost. So, to design a
novel system of optical image processing with excellent performance for many kinds of fields
has become the problem urgent to be solved.
In this paper, image sensor and FPGA (Field Programmable Gate Arrays) are adopted to
design an image acquisition system, and DSP (Digital Signal Processor) is adopted to design
an image processing system. At present, the signal processing systems usually take the
structure of FPGA and DSP, because FPGA is suitable for the design of interfaces and some
preprocessing, especially for some work which has mass data and needs high speed of
processing, but has simpler algorithm structure, while DSP is suitable for the work which has
less data but has more complex algorithm structure. Accordingly, this structure has been
applied in the domains of signal processing and image processing .
The optical images such as holographic image and speckle image can be acquired
separately for their specific characteristics. The images acquired by image sensor can be sent
into the image processing system by interfaces and data buses. The results can be displayed
by a small LCD which is adopted in the image display system. So we can realize the function
of acquisition, processing and display of optical images by integration of the above three
subsystems. The system of optical image processing we designed can realize miniaturization
because it can operate without PC and image acquisition card. The system has the merits such
as short developing period, small volume, low cost and stable operation. The design of the
novel system of optical image processing is mainly introduced in the paper.
2. Hardware design of the system
2.1 System composition
FPGA produced by Xilinx and Altera is the most popular in the electrical design at
present. Considering the performance, power consumption and cost, we choose XC3S200,
Spartan 3 series of Xilinx, which has prolific logic gates and I/O ports and can work as the
main control device.
DSP is often used in abundant digital signal calculation. In the same clock sequence, DSP
can realize FFT algorithm 2-3 order of magnitude higher than general CPU. DSP usually has
high processing speed for its unique structure.
In the domain of image processing, DSPs of TMS320C6000 series produced by TI are
the most popular. In this paper, we choose TMS320C6713 produced by TI. TMS320C6713
has high speed operation ability and its dominant frequency can reach 225MHz.
TMS320C6713 has many resources such as 32-bit external memory interface and DMA
communication mode. So it can communicate with SRAM, SDRAM, dual-port RAM, FIFO
and FLASH in high speed.
3) Image sensor
CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide
Semiconductor) image sensor are the two kinds of common devices used in obtaining the
images. CMOS image sensor has its advantages such as high integration, low power
consumption, strong anti-interference ability and flexible mode of data-readout. At present,
CMOS image sensor has become the new focus of image sensor [4, 5].
In this paper, we select OV7649 produced by OmniVision as the imaging device.
OV7649 is a CMOS image sensor which integrates function modules of image sensor array,
clock generator, analog signal processing, AD conversion, format control of data output, port
of digital video and interface of SCCB (Serial Camera Control Bus). Though 8-bit parallel
data port, OV7649 can export digital video signal in RGB, Raw RGB, YUV or YCbCr.
The system also includes image buffer memory and image memory. A piece of dual-port
RAM, CY7C007A produced by Cypress Company is selected as image buffer memory, which
can store the image data acquired by image sensor temporarily. CY7C007A is a low power
CMOS 32K 8bit dual-port static RAM and has two groups of completely symmetric address
buses, data bus and control bus. It is possible that two CPUs can access CY7C007A at the
A piece of SDRAM is selected as image memory for storing plenty of image data. So we
choose HY57V643220, a high speed SDRAM, which can provide capacity of 64Mbit and has
the ability to transmit data in a high speed for its synchronized interface and internal structure
of pipeline. It also has four memory bodies and we can access the memory unit of different
memory bodies by time-sharing address bus of system.
2.2 System structure
As shown in Fig. 1, the system of optical image processing includes the above-mentioned
devices. FPGA can control the work of CMOS image sensor. The optical image data can be
transmitted to dual-port RAM, from which DSP can also read image data. The optical image
data can be stored in SDRAM. So DSP can process it conveniently. The result can be
transmitted to the display through interface of VGA.
Image data Dual-port Image data
CMOS Image sensor Display
Fig. 1. Structure of the optical image processing system.
2.3 Image acquisition system
Image acquisition system mainly consists of three parts: image sensor, FPGA and image
buffer memory. The structure of image acquisition system is shown in Fig. 2.
CMOS FPGA RD port
Fig. 2. Structure of image acquisition system.
FPGA is selected as the main device to control the operation of CMOS image sensor. In
this system, FPGA is responsible for decoding synchronous signal from CMOS image sensor
and controlling operation of reading and writing of image buffer memory and processing of
image data. As an important device, the roles that FPGA plays in the system are as follows:
1) Initialization of SCCB
Image sensors of OmniVision have programmable characteristic. As a serial bus, SCCB
has been defined and adopted by OmniVision to control the operation of image sensors
produced by OmniVision . Initialization of SCCB is the start of the system and some
registers in OV7649 should be set. We can set the frame frequency of images, exposure time,
the location and size of fenestration. Then the image sensor can start to collect images. In this
paper, we use FPGA to control and initialize SCCB.
2) Realization of time sequence signal
When clock signals of VSYNC, HREF, PCLK enter FPGA, image acquisition system
generates address signal, writing signal and chip selection signal of image frame memory.
Then image data can be transmitted into image buffer memory.
2.4 Image processing system
Image processing system mainly consists of three parts: DSP, image buffer memory and
image memory. The structure of image processing system is shown in Fig. 3.
Memory Image Data
Acquisition Control CPU
Memory Image Data
Fig. 3. Structure of image processing system.
The image processing system can work in the next four steps:
1) The system is reset by power up. DSP is initialized and waits for interrupt signal.
2) When DSP receives interrupt signal from image acquisition system, DSP will read the
image data through the image buffer memory.
3) Image data is transmitted into image memeory and processed in algorithms of optical
image processing by DSP.
4) The result of image processing is transmitted to a samll LCD.
The clock signal of image acquisition system are not consistent with DSP. So image
cannot be read from image acquisition system directly. Thus, an image buffer memory is
designed to restore image data temporarily, and DSP can read image data from image buffer
memory in its own clock.
Large-capacity memory is always needed in digital image processing. In this paper,
optical images we collect are often larger than 300KB. The internal memory capacity of DSP
is not enough. So we choose a high speed SDRAM, which can provide capacity of 64Mbit.
2.5 Image display system
In the image display system, a small LCD is adopted as the display. We designed the
interface of VGA. Through the interface, optical images processed by the image processing
system can be displayed in the small LCD. So we can see the result clearly. It can help us
make the correct judgment.
3. Software design of the system
In this paper, we develope algorithms for classical optical image processing and write the
program in DSP. In interferometry, fringe spacing can be used in measuring displacement and
deformation of objects. Fringe images should be processed to get fringe spacing. Image
enhancement, space domain filtering, image binarization are the algorithms we adopt in this
1) Image enhancement
Many low quality images have low contrast for the reason that the gray value is
concentrated. So it is absolutely necessary to enhance the contrast of images. Gray level
transformation and histogram modification are the two common methods to enhance the
contrast of images.
Gray level transformation can increase the dynamic range of the optical image and make
the optical image more distinct. Gray histogram can describe the general view of image,
including gray range, frequency of every pixel, gray distribution. Gray distribution of the
optical image processed by histogram modification will become more even.
2) Space domain filtering
Noise often exists in optical images we collect. It can worsen the quality of images and
make images fuzzy. In order to decrease the noise, filtering is often adopted. In this system,
space domain filtering are adopted.
3) Image binarization
Image binarization can transform the original image into the image that only includes
background and characteristic substances by solving the threshold. The principle of threshold
selection is that image information should be retained as much as possible and disturbance of
background and noise should be reduced as much as possible.
Optical images we collect are usually of poor quality, so the fringes cannot be extracted
easily. First an optical image is processed by image enhancement to improve the image
contrast and then processed by medium filtering algorithm to eliminate the noise of image.
Finally the image is processed by image binarization and we can get the binary image of
In this paper, we have designed a novel system of optical image processing based on
FPGA and DSP. The image acquisition system, the image processing system and the image
display system are designed separately. We can integrate the above three subsystems in
together, so the system of optical image processing can be small in volume. Not using PC, the
system has also decreased the cost. The system can realize the function of image acquisition,
image processing and image display by debugging. Compared with traditional optical image
processing system, the system realizes miniaturization and can operate conveniently,
smoothly and inerrably with high speed and precision by experiments. To improve the image
clarity, decrease disturbance of circuits and optimize the algorithm of image processing will
be the work to be continued for us. We believe that the novel system of optical image
processing will be a great help to researchers in optical measurement and other relative fields.
This study was supported by research funds from the National Natural Science
Foundation of China (NO. 10172043), the Aeronautics Science Foundation of China (NO.
05G52047) and International Science and Technology Cooperation Program (NO.
1. W.Y. Liu. Opto-electric Image Processing. Publishing House of Electronics Industry.
2. L.B. Tang, B.J. Zhao. Design of a new type of universal image processing system. Optical
Technique. 2008, 34(3), pp. 114, 214, 314 (in Chinese).
3. L. Yan, H.Y. Zhao, M.X. Luo. Digital Imaging Fundamental, System & Technology.
Publishing House of Electronics Industry. Beijing, 2007 (in Chinese).
4. J.H. Ni, Q.Y. Huang. CMOS Image Sensor and Its Development Trend. OME Information.
2008, 5, pp. 33-38 (in Chinese).
5. W. Li, Y. H. Qiu, J. Dong. Design of Video Accessing System Based on CMOS Image
Sensor. Microcomputer Information. 2008, 24(11), pp. 277, 278, 159 (in Chinese).
6. OmniVision Serial Camera Control Bus (SCCB) Functional Specification. Document 2.1