nokia n900 by moradhammer88


									Tramchan: a mobile robot to clean tables
   based on the Nokia N900 phone
                          Nicolau Werneck


Tramchan is a mobile robot that sweeps dining tables autonomously,
built over a traditional hand-held rotary brush. A Nokia N900 mobile
computer provides the camera, used for localization and mapping, and
performs all necessary computations to control the robot. Control sig-
nals are transmitted from the audio output jack and received by a sim-
ple electronic circuit that interfaces the N900 with a pair of engines con-
nected to wheels in differential drive configuration. The robot exhibits
three behaviors: roaming to perform localization and mapping, moving
over autonomously planned paths to clean the table, and following di-
rections provided by a remote user.

1    Introduction
One of my fondest childhood memories is of my grandfather cleaning
bread crumbs from the dining table after breakfast. For that reason my
favorite object chosen for this Nokia PUSH N900 project is a table brush,
the kind that has a rotary brush and collects the dirt inside its case.
More specifically we will use a brush from the very traditional Feiticeira
brand, in black and white plastic. Figure 1 shows a Feiticeira being
used. The brush will be supplemented with motors and a N900 to create
a small mobile robot. The word tramchan picked as a name to the robot
is an old-time Brazilian slang that means something very pleasant, of
high quality, a well-built object or a well-done repair to something.
    Mobile robotics is a thriving research field that has archived great
successes in the last decades, such as the NASA Mars rovers, the vehi-
cles at the DARPA challenges and the iRobot Roomba, the best-selling

          Figure 1: A traditional Feiticeira hand-held rotary brush.

robot. Recent advances on the use of monocular vision for localization
and mapping are making it easier to develop small applications where
the traditional but large and costly ultrasound and laser range-finders
cannot be employed. The application described here is a great example
of this modern robotics trend, for it is a small and simple indoors mobile
robot implemented with an off-the-shelf portable computer.
    Many electronics and computer enthusiasts have been devoting them-
selves to mobile robot projects lately. Sometimes they build robots from
spare parts and general-purpose processing units acquired from special-
ized shops, but platforms such as LEGO Mindstorms, the Roomba and
the Surveyor SRV-1 have also attracted numerous interested hackers
and allowed them to develop applications that wouldn’t be easy to do
without technology that has only recently become available to them.
    The N900 phone is such a revolutionary technology becoming avail-
able right now, and the potential of similar products to mobile robotics
has been recognized before — a N770 and later N800 controlled dog
puppy robot has hit the news in previous years [1]. The N900 has a great
potential as a platform for the development of robotic applications, offer-
ing similar or better resources than not only the previous Nokia tablets,
but also popular products such as the CMUCam and the SRV-1 Blackfin
board. Tramchan will explore this potential, showcasing the N900 as a
platform for sophisticated autonomous portable robots.
    Many hobbyist robots are merely self-propelled devices that are re-
motly controlled by users or have no control at all, and simply perform
pre-programmed movement sequences. These applications are not even
considered robots by those who argue that autonomy is a necessary char-
acteristic for any robot. These applications have little or no ability to
adapt to their surrounding environment. Tramchan will not only be

a self-propelled device that might be remotely controlled, but will ex-
hibit autonomous behavior: both a simple and reactive form of obstacle
avoidance, and a more complex and deliberative task planning. These
control processes will be based on visual information gathered from the
N900 camera, and all processing will happen inside this same mobile
computer. This not only explores the potential of the N900 for Mobile
Robotics, but for other applications of Computer Vision and Artificial
Intelligence in general.

2     Description
The project goal is to build a mobile robot that can move autonomously
over a dining table. The robot will be able to:
    • Acquire and integrate data from the camera and accelerometer to
      determine its location.
    • Avoid to fall from table borders or collide with table objects.
    • Search autonomously for bread crumbs on the table and wipe them
    • Have a basic idea of its location and a map of the environment,
      enabling it to do route planning and improve safety.
    • Follow directions provided from a remote desktop or smartphone.
Figure 2 illustrates the robot in action. It moves over a table avoiding
obstacles while it maps the environment and estimates its location. The
path is planned autonomously, either to follow directions provided by a
remote user, or to fulfill the task of cleaning the table by moving over the
dirt. A video of a paper-built prototype moving over a table — pushed
by hand, of course — is available on the Tramchan blog1 , or directly at
YouTube 2 .
   The development of Tramchan will depend on building fundamental
tools that can be reused by other mobile robot applications in the fu-
ture. Building this application means laying out a foundation for other
applications that make use of the same principles, and also applications
from other areas related to mapping and localization. Some examples
are the usual navigational aid already performed today with cell phones
and also Augmented Reality (AR) applications. Today these applica-
tions rely only on A-GPS, electronic compasses and accelerometers, but

Figure 2: Illustration of the robot in action, with references to its main features.

the most advanced AR applications require the more precise localization
provided by computer vision. Location through computer vision is also
the best alternative in indoor environments where the usual localization
technologies may not work well enough for any purpose.
    The robot will be constructively simple. The Feiticeira will be mod-
ified to host two DC gear motors directly connected to wheels, in coax-
ial configuration to allow the differential driving of the base. It will
also hold extra electronics, a battery, and the N900 standing vertical to
the ground, pointing to the direction of movement and held by a simple
metallic rig. The basic sense-act cycle works as follows:
   • Images and accelerator data are captured.
   • Data is processed to determine the state of the robot and environ-
   • Task planner decides the path that should be followed for next few
   • Path follower calculates velocity control to be sent to engines.
   • Signal generator creates 2 PWM signals that are amplified and
     buffered by an electronic circuit connected to the motors.
   Although the aim of the project is to build this sophisticated system
able to plan routes over the table, the robot can be made very simple at
first. It can simply locate at the pictures objects that are different from
the expected table color, and move towards them with constant velocity

and a direction control that centers the targets on the picture. It can be
first built with a control system similar to the one in Roomba, closer to a
reactive design, and just later grown into a system with more reasoning
and planning.

3    Implementation details
The Feiticeira will be modified to host two dc motors connected to wheels,
in coaxial configuration to allow the differential driving of the base. It
will also hold extra electronics, a battery, and the N900 standing ver-
tical to the ground, pointing to the direction of movement. The control
of the engines will be performed by audio signals output from the N900
audio jack outlet. Figure 3 shows four pictures of the paper-built model
of the robot. The actual Feiticeira was used to build this model, while
paper, cardboard and masking tape were used to replace the wheels,
motor, and the N900. Figure 4 is a picture of Tramchan in its habitat.

        Figure 3: Photographs of a cardboard model of Tramchan.

   The control system is set of interoperating processes, as usual in
robot design. Figure 5 presents a block diagram that describes the robot
architecture. It can be seen as a flow chart: data is input to the system
from the environment surrounding the robot, represented by the dashed

             Figure 4: Picture Tramcham spotted in the wild.

arrow labeled World. This input data is analyzed by the IP and SLAM
blocks. These two blocks are the input stage of the system, and they
produce in real time a map of the environment, a location estimate and
all data needed to control the robot.
    The Planner block is a process that operates in the highest level of
abstraction. It takes in consideration the given task and the current
state of the robot and the environment to decide the next actions to
be performed. The output of this block is a series of instructions to a
pipeline of lower-level blocks that end up activating the engines that
make the robot move. The robot motion causes changes to its state in
the world and also to other elements of the environment (e.g. removal
of dirt). These changes are eventually seen in the state estimate of the
robot as it continues to acquire information. The whole process has a
cyclic nature, reflected by the loops closed in the diagram by the World
    Each processing block corresponds roughly to a program that will
be running on top of the Maemo operational system in the N900. The
communication between them will be performed mostly via DBUS.

3.1   Input and localization
The IP block in Figure 5 refers to an Image Processing program that
analyzes images taken in real time with the robot camera — the camera
at the back of the N900 phone. This is the most important input data
the robot will rely on to perform all of his tasks. This program extracts
multiple measures from the images it receives, and makes them avail-
able to the other blocks. The Motion Control block receives from IP a
visual hodometry reading, which is a short-term but imprecise location

                         User            Location
                       Commands          and Map

                        Planner            SLAM



                         Audio           Electronics
                       Synthesizer       & Engines

              Figure 5: Flow chart of the robot architecture.

information to complement the more accurate but slower information
provided by the SLAM process, and can also receive a signal to stop due
to immanent danger. On the other hand, the SLAM block receives all
the necessary data to perform its task of producing a map of the envi-
ronment and a real-time location estimate. The accelerometer data can
be regarded as being acquired by this block too.
    The double arrow between the SLAM block and the data indicates
the fact that SLAM performs an update on the available information
given new evidence, so the current data must be considered. This is
different from the most simple GPS applications, for example, where a
complete new estimate is repeatedly calculated from a received signal.
The double arrow to the IP block refers to the fact that the processing
is largely directed by the context. In traditional image processing for
robotics and computer vision usually the whole area of input images are
swept in pursuit of useful data. This usually represents an exceedingly
large computational load for mobile hand-held devices operating in real
time. Modern techniques seek to tailor the analysis to the presented
situation, so a minimum of calculations is needed to acquire the infor-
mation needed to perform the SLAM.

3.1.1   Image processing
The IP block is based on traditional image analysis tools to locate and
track visual features, such as contour finding, interest point detection,
image registration and color segmentation. Some of the measurements
produced by this block take a longer time to be calculated, and are inte-
grated with larger delays by the SLAM block. Some other quicker calcu-
lations are produced more frequently and with less delay, and are con-
sumed by the Motion Control block. This image processing block might
be programmed with the Python PIL module, or with C programs.
    It is important in this application to avoid obstacles, and specially
to avoid dropping from the table tops the robot will be typically moving
over. One technique that can be used to quickly track the borders from
the table and from large obstacles in the environment is to restrict the
necessary matching analyzes and disparity calculations only to the cen-
tral horizontal pixel column of the images. This can be performed very
quickly, and provides enough information to allow a robot to move quick
and safely over a table [6].
    Figure 6(a) shows two example images taken from a robots moving
in straight line over a table. The central pixel column is highlighted in
the images. The color intensity in the three channels from the pixels of
this column in images taken along the way is displayed in Figure 6(b).


Figure 6: Example images taken from a robot moving over a table (a), and
          intensities at the middle column from successive images (b).

   The technique is based on the matching of points from the one-dimensional
functions displayed at Figure 6(b). The location relative to the robot of

the table borders, obstacles and other horizontal visual landmarks are
calculated from the disparity in the location of the matched points be-
tween the images. When the robot moves at constant speed, matching
points are found over hyperbolas in Figure 6(b).
   Figure 7(a) presents a frame captured from a computer graphics sim-
ulation of a robot moving over a table inside a room. The result of apply-
ing the technique to the image sequence is the graphic at Figure 7(b).
The point clouds indicate the location of borders in the environment,
either the table border or the bottom end of the wall. This is a very pre-
liminary test of the technique, and the results can be much improved
with probabilistic models and the tuning of parameters, specially the
distance travelled between each picture taken.

                   (a)                                 (b)

Figure 7: Test of the obstacle detection by Werneck et al. [6]. Frame from CG
          simulation (a), and detection readings produced over time (b).

    For more precise localization this program must also look for visual
landmarks using feature detectors, and match predicted landmarks pro-
vided by the SLAM algorithm [2]. These procedures require only classic
and easily implemented image processing operations, and are performed
in restricted areas of the image, minimizing the computational load.

3.1.2   SLAM
The Simultaneous Localization and Mapping (SLAM) problem has be-
come a major concern in robot research, and multiple techniques have
been developed to solve it for different applications. Performing SLAM
means to determine the location of the robot over time, while simulta-
neously building a map of the environment. The SLAM block in the pro-

posed system implements such a solution based on recent research on
monocular visual SLAM [2, 3, 4]. Simple visual landmarks are tracked
as the robot moves. They are sought by the system as more landmarks
are needed, and eventually dropped if they turn out to be not very use-
ful. Figure 8 displays a example images from this system.

                  (a)                                 (b)

Figure 8: Example of input image with located landmarks (a), and an estimated
          3D map (b). Taken from the work of Davison et al. [2].

    The process is based on a probabilistic approach, the most reliable in
contemporary robotics. The classical Unscented Kalman Filter (UKF) is
the model over which the whole process will be built. The robot pose (lo-
cation and heading) is a vector that is updated by this filter, and the map
is a set of vectors with the coordinates of the different landmarks, asso-
ciated with information related to its visual appearance. The Kalman
Filter (KF) maintains a probability estimate of these vectors, their mean
and variance, and update them as new information is provided [5]. Tech-
niques such as the UKF are extensions of the classic linear KF to non-
linear systems. Figure 9 presents a block diagram of how the Kalman
filter works.

3.2   Control and output
The location and mapping information produced by the SLAM block
from the visual input provided by the phone camera can now be used
by other parallel processes running in the system to control the move-
ments of the robots, and autonomously plan actions to reach goals. The
Planning is the most important block in the control section. It takes
in consideration the given task and the current state of the robot and
its surrounding environment, and decides the next actions of the robot.

          Figure 9: Block diagram of the standard Kalman Filter.

Outside of communication acts, all the robot is able to do is move, so
the most relevant output produced by the Planner is a trajectory to be
followed by the robot.
    The trajectory planning first considers if the user has made any re-
quests for the robot. If it is free to choose its destination, it traces a
route to move over dirty areas present in the map. If no dirt is known
to be over the table, the robot then plans a path intended to explore the
area and update the map.
    As the planned path is updated it is delivered to the Motion Con-
trol block. The main function of this process is to act as a path follower,
calculating how the robot should move to follow the desired path given
its current location estimate and the motion restrictions of the robot.
The simplest way to do this is to repeatedly rotate the robot so it points
to the path, then walk straight ahead a fixed distance, and then check
the state again. More sophisticated algorithms can be created to make
the robot move more gracefully. The Motion Control process can also
implement reactive behaviors, such as stopping the robot because of im-
manent danger not previously considered by the Planning process.

3.3   Power electronics
The Motion control process calculates the force that should be applied
to each motor so the robot moves in the desired direction and speed.
The motors used in the robot are two small electric, brushed DC motors.
Their torques are controlled by the voltage applied to their terminals.
Varying this voltage is the most usual way to control the motor torque
and speed, which are related by the load connected to the shaft of the
motor, caused by the inertia, friction and other sources.
   Since the control is performed by a digital computer, an analog-to-
digital chip must be employed to generate the proper analog levels re-
lated to the digital values calculated by the computer. The audio output

jack in a computer, such as the Nokia N900, is as such a converter, and
can be used for system control as long as its parameters such as the
sampling rate and resolution satisfy the application requirements.
    This kind of control by the variation of the input voltage has some
disadvantages, though. The gain between the output jack and the mo-
tor is a critical parameter, and the circuit must be very approximately
linear. Also, the whole circuit between the motor and the audio out-
put must work in the DC frequency band, what is a troublesome project
requirement. Although all these problems can be solved, a better alter-
native is to rely on a different kind of control signal.
    Instead of applying a continuous voltage to the motor terminals, a
popular alternative is to perform a pulse width modulation (PWM) in
the motor input. A PWM signal varies only between two levels, generat-
ing a periodic signal with a constant frequency selected by the designer,
and a variable duty cycle. The frequency is usually the double of the
maximum speed of the motor, and the duty cycle varies the mean volt-
age of this signal. This mean voltage is then set to the same level that
would be directly applied in the previously mentioned control technique.
    Using a PWM signal to control a DC motor greatly simplifies the
design of the necessary amplifier circuit. The signal is immune to non-
linearities, and as long as there is no significant filtering, the relation
between the mean applied voltage and the duty cycle is quite robust.
Figure 10 shows the electronic schematics of a such a simple amplifier.




Figure 10: Example electronic circuit for interfacing the N900 audio output to
           the electric motors to perform PWM control.

4     Conclusion
The project proposed here is an entry for the Nokia PUSH N900 contest.
The loved object selected to be hacked is a classic Feiticeira hand-held
rotary brush to clean dining tables. The idea is quite straightforward:
tie a couple of wheels and a N900 to the brush, and transform it into a
mobile robot so it can do its job autonomously.
    The proposal explores many interesting aspects of the N900 mobile
computer. The control system is composed by separate applications that
work in parallel and communicate with each other, and both the cam-
era and audio output are heavily used. This is all made possible by the
unique characteristics of the Maemo OS. It offers to developers easy ac-
cess to multitasking and to system resources such as the camera, audio
and network interfaces.
    The use of the camera and audio output in this application are some-
what unconventional. This project may serve as inspiration to the cre-
ation of other kinds of monocular computer visual applications such as
Augmented Reality ones, or to use the audio output as a general analog
signal synthesizer to interface the computer with electronic circuits, and
not just as an output for audio signals to be heard.
    The creation of an autonomous robot with a N900, specially such
a small and simple robot, may help people in general to see the great
potential of this device. It can also help this peculiar mobile computer
to present its own personality to the world. We will make the N900 act
by itself and move about in our homes, working at large. This project
will definitely show that this device is much more than a cell phone,
which are just passive tools that are always either lying inert over tables
waiting to be picked up, or crying obnoxiously when someone calls. The
N900 is more than that, it is a mobile computer, but it can be even more.
It can be a mobile and autonomous computer. Tramcham will show how
we can make the N900 fulfill this destiny of his.

4.1   Material needed
The N900 mobile computer necessary for this project will be provided
by Nokia. The electronics and structural parts are easily found in lo-
cal shops in Sao Paulo and on the Internet. Mechanical parts such as
motors and wheels are also available on specialized shops on the Inter-
net. One of the best manufacturers of reduced size electric motors is the
Swiss Faulhaber, and their products can be acquired both from the In-
ternet and from a local dealer in Brazil. The contestant has also already
acquired a Feiticeira.

4.2   Responsible hacker
Nicolau Werneck (28, Brazilian, Linux user #460716) is an Electric En-
gineer graduate student specialized on computing, signal processing
and, more recently, mobile robotics. He is currently working at the de-
velopment of visual mobile robotics techniques suited for small indoor
robots and hand-held devices. Apart from the academic background, he
also has experience with building small electronic projects. He owns
a brace, a hacksaw and a bottle of ferric acid, and is not afraid to use

[1] Evan Blass. Nokia decapitates tablet puppy, upgrades to n800,
    October 2007. URL
[2] Andrew J. Davison, Ian D. Reid, Nicholas Molton, and Olivier
    Stasse. MonoSLAM: Real-time single camera SLAM. IEEE Trans-
    actions on Pattern Analysis and Machine Intelligence, 29(6):1052–
    1067, 2007. URL doi://10.1109/TPAMI.2007.1049.
[3] Ethan Eade and Tom Drummond. Edge landmarks in monocular
    SLAM. Image Vision Computing, 27(5):588–596, 2009. ISSN 0262-
    8856. URL
[4] Andrew P. Gee, Denis Chekhlov, Andrew Calway, and Walterio
    Mayol-Cuevas. Discovering higher level structure in visual SLAM.
    IEEE Transactions on Robotics, 24(5):980–990, October 2008. ISSN
    1552-3098. URL
[5] Greg Welch and Gary Bishop. An introduction to the Kalman Filter.
    Technical Report TR 95-041, University of North Carolina at Chapel
    Hill, Chapel Hill, NC, 27599-3175, USA, July 2006. URL http:
[6] Nicolau Leal Werneck, Anna Helena Reali Costa, and Flavio Sales
    Truzzi.         ¸˜        ˆ
              Medicao de distancia e altura de bordas horizon-
                  ˜                              o     o
    tais com visao monocular linear para robˆ s m´ veis.         In
    Anais do V Workshop de Visao Computacional, 2009.         URL˜nwerneck/almoxarifado/


To top