CSCi 5980 Project Report
December 10, 2008
Vanishing points based navigation
One of the key requirements for successfully deploying a robotic sensor net-
work is the ability of the robot to navigate and localize within its environment.
The sensor network for our project was to be deployed indoors, in the EECS
corridors. On-board vision and bump sensors were the two options available
for navigation within the corridors. We decided to use vision for its obvious
advantages such as the speed of navigation.
Using the vanishing point is one of the most widely used technique for indoor
vision-based navigation. For a perspective camera, lines parallel in the world are
seen to intersect at a common point in the image plane, called as the vanishing
point. A lot of such lines are present in the corridors, such as the edges of the
side-walls with the ﬂoor and ceiling. Using the vanishing point determined from
these lines, heading of the robot can be estimated .
The ﬁrst step in estimating the vanishing point is the extraction of the lines
parallel in the world, within the image. Canny edge detector was used to ﬁrst
extract the edges in the image. Change in the illumination conditions is one
of the biggest problems associated with vision-based techniques. To reduce the
eﬀect of this problem, histogram equalisation was performed on the grayscale
image. This ensured suﬃcient contrast within the image, to extract the required
edges using the Canny edge detector. Since its very diﬃcult to extract the right
lines under diﬀerent conditions, a layered ﬁltering approach was used, where
spurious lines or edges are eliminated at each level. The Canny edge detector
thresholds are set low so that it always extracts more than the required edges.
A Hough line transform was then performed to extract the straight lines
within these edges. Since the lines used to compute the vanishing point associ-
ated with the heading of the robot are parallel to the camera optical axis, these
lines in the image plane have a ﬁnite non-zero slope. Hence, vertical and hor-
izontal lines extracted by Hough transform are trivially eliminated. All other
lines are considered for estimating the vanishing point. Not all of these lines
are the parallel lines we seek to ﬁnd. Hence, for this technique to perform well
in the real world with obstacles and people moving about (contributing to the
lines), further ﬁltering is necessary. After this step we have a set of lines, each
represented by its slope and intercept. Some of the lines are parallel in the
world hence ideally intersect at a common point, while others are spurious lines
detected which intersect at other points. In practice, due to the inherent noise
in detection, the parallel lines do not intersect at a single point but rather form
a cluster of closed space points.
With the lines in the image represented by their slopes and intercepts, a
Cascaded Hough Transform  can be used to determine their common point
Figure 1: Vanishing point as estimated by RANSAC (red) and naive averaging
of intersection. The following equations present this idea clearly.
y = mx + c (1)
c = (−x)m + y (2)
where (−x) is the slope and y is the intercept in the parametric space for each
line (m, c). Hence, the point of intersection of various lines can be found by
linear regression in the parametric space to get the coordinates (x, y)
Since the set of lines found by the Hough transform consists of lines not
belonging to the set of parallel lines, we cannot directly apply a least squares
linear regression over the entire set. Hence, we use RANSAC to get the inliers
from amongst this set. Any two lines are randomly selected from this set to give
the two parametric points and a straight line is ﬁt through these points. Then
this model is ﬁt to all other points in the given set and points with an error
within a threshold are marked as inliers. This step is performed for a given
number of iterations yeilding a diﬀerent number of inliers at each iteration.
Then, the instance with the highest number of inliers found is selected and a
least squares line is ﬁt through these inliers. The output of this least squares
regression is a line in the parametric space corresponding to a point in the image
plane. This point is then chosen as the vanishing point.
Inspite of these ﬁltering, there is possibly a chance of incorrect estimation
of vanishing point in cases of severe occlusion or in presence of many cluttered
objects. Hence another check is performed to check the validity of the vanishing
point estimated. Since the pitch of the camera does not change during the robot
motion and the optical axis is perpendicular to the vertical axis in the world
frame, the vanishing point in the image must always have its y-coordinate close
to half of the image height. By performing this simple check, the validity of
the vanishing point can be established. A median ﬁlter is then applied over
the computed vanishing points to smoothen the eﬀects of non-linear projec-
tion changes.With these layered ﬁltering and rejection approach, the resultant
estimation of the vanishing point is very robust. During the entire phase of test-
ing, there were hardly any instances when the algorithm produced an incorrect
Figure 1 shows the comparison of the RANSAC based vanishing point esti-
mation results with naive estimation using average of all point of intersections.
As can be clearly seen, RANSAC based technique gives the expected solution
in this case.
When the robot is moving in a straight line within the corridor parallel to the
walls, the x-coordinate of the vanishing point within the image is close to the
image center Cx . As the heading of the robot changes within the corridor, the
x-coordinate of the vanishing point changes. The robot then employs a simple
controller to maintain a straight heading for the robot. As the robot deviates
from its straight line heading, the controller appropriately varies the two wheel
velocities to bring back the robot to its required heading. Navigation using this
technique was successfully demonstrated during the ﬁnal presentation.
Localization of the robot within the environment is another essential task re-
quired for a robotic sensor network. We chose vision as a primary means for
robot localization which was to be used for further RSSI-based localization tasks.
For accurate estimation of robot pose, distance to landmarks, artiﬁcial or other-
wise, needs to be found. For obtaining distance information to landmarks, it is
necessary to know the intrinsic parameters i.e. the focal lengths and the image
center locations, for the camera. The camera calibration was performed with a
8x6 checkerboard pattern using tools present in OpenCV . The calibration
was repeated a number to times to further reﬁne the values of intrinsic param-
eters obtained. These values were later employed in ﬁducial based localization
on the robot.
 X. Lebegue, J. Aggarwal, “Signiﬁcant line segments for an indoor mobile
robot,” IEEE Transactions on Robotics and Automation vol. 9, issue 6, pp.
801-815, Dec 1993
 T. Tuytelaars, M. Proesmans, L. Van Gool, “The cascaded Hough trans-
form,”Proceedings of International Conference on Image Processing, vol. 2,
pp. 736-739, Oct 1997
 OpenCV [Online] Available: http://opencv.willowgarage.com/