A Study on the Performance of Classical Clustering Algorithms with Uncertain Moving Object Data Sets by ijcsiseditor


More Info
									                                                               (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                   Vol. 9, No. 4,April, 2011
             A Study on the Performance of Classical Clustering Algorithms with
                             Uncertain Moving Object Data Sets

                 Angeline Christobel . Y
               College of Computer Studies                                                    Dr. Sivaprakasam
              AMA International University                                             Department of Computer Science
              Salmabad, Kingdom of Bahrain                                                   Sri Vasavi College
             angeline_christobel@yahoo.com                                                       Erode, India

Abstract— In recent years, real world application domains are                      arises out of the limitations of data collection
generating data with uncertainty, incomplete and probabilistic in                  equipment. In such cases, different features of
nature. Examples of such data include location based services,                     observation may be collected to a different level of
sensor networks, scientific and biological databases. Data mining                  approximation.
is widely used to extract interesting patterns in the large amount
of data generated by such applications.                                        •   The imputation procedures can be used to estimate
In this paper, we addressed the classical mining and data-analysis                 the missing values in the case of missing data. The
algorithms, particularly clustering algorithms, for clustering                     statistical error of imputation for a given entry is
uncertain and probabilistic data. To model uncertain database,                     often known a-priori, if such procedures are used.
we simulated a moving object database with two states: one
contains real location and another contains outdated recorded                  •    Data mining methods are applied to derived data sets
location. We evaluated the performance and compared the                            that are generated by statistical methods such as
results of clustering the two states of location data with k-means,                forecasting. In such cases, the error of the data can be
DBSCAN and SOM.                                                                    derived from the methodology used to construct the
    Key Words: Data Mining, Uncertain Data, Moving Objects
                                                                               •   The data is available only on a partially aggregated
    Database, Clustering.                                                          basis in many applications such as demographic data
                                                                                   sets. Each aggregated record is actually a probability
    I.        INTRODUCTION                                                     •   The trajectory of the objects may be unknown in
Data uncertainty naturally arises in many real world                               many mobile applications. In fact, many
applications due to reasons such as outdated sources or                            spatiotemporal applications are inherently uncertain,
imprecise measurement. This is true for applications such as                       since the future behavior of the data can be predicted
location based services [12] and sensor monitoring [6] that                        only approximately.
needs interaction with the physical world. For example, in the
                                                                           This paper will neither address the existing techniques for
case of moving objects, it is impossible for the database to
                                                                           uncertain data clustering nor propose a new one. Instead, it
track the exact locations of all objects at all time. So the
                                                                           will address the impact of uncertain data in clustering results
location of each object is associated with uncertainty between
                                                                           using a primitive model of a moving object database.
updates [7]. In order to produce good mining results, their
uncertainties have to be considered.
In recent years, there has been much research on the                        II. CLUSTERING ALGORITHMS
management of uncertain data in databases, such as the                     Clustering is a data mining technique used to identify clusters
representation of uncertainty in databases and querying data               based on the similarity between data objects. Traditionally,
with uncertainty but only little research work has addressed               clustering is applied to unclassified data objects with the
the issue of mining uncertain data. Many scientific methods                objective to maximize the distance between clusters and
for data collection are known to have error-estimation                     minimize the distance inside each cluster. Clustering is widely
methodologies built into the data collection and feature                   used in many applications including pattern recognition, dense
extraction process. In[2],[13], a number of real applications, in          region identification, customer purchase pattern analysis, web
which such error information can be known or estimated a-                  pages grouping, information retrieval, and scientific and
priori has been summarized as follows:                                     engineering analysis. Clustering algorithms deal with a set of
                                                                           objects whose positions are accurately known [3].
    •    The statistical error of data collection can be
         estimated by prior experimentation, if the inaccuracy

                                                                      11                              http://sites.google.com/site/ijcsis/
                                                                                                      ISSN 1947-5500
                                                            (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                Vol. 9, No. 4,April, 2011
To study the performance of the clustering algorithms with              points and on the use of density relations between points
uncertain moving object date sets, we have chosen K-means,              directly density reachable, density reachable, density
DBSCAN, SOM algorithms and it is discussed below.                       connected[Ester 1996] to form the clusters.

A) K-Mean Clustering algorithm                                          Core points:
                                                                        The points that are at the interior of a cluster are called core
One of the best known and most popular clustering algorithms            points. A point is an interior point if there are enough points in
is the k-means algorithm. K-means clustering involves search            its neighborhood.
and optimization.
                                                                        Border points:
K-means is a partition based clustering algorithm. K-means’
                                                                        Points on the border of a cluster are called border points.
goal is to partition data D into K parts, where there is little
                                                                        NEps(p): {q belongs to D | dist(p,q) <= Eps}
similarity across groups, but great similarity within a group.
More specifically, K-means aims to minimize the mean square
                                                                        Noise points:
error of each point in a cluster, with respect to its cluster
                                                                        A noise point is any point that not a core point or a border
Formula for Square Error:
                                                                        Directly Density-Reachable:
Square Error (SE)= ∑ ∑| ci |  x − M  ,                                A point p is directly density-reachable from a point q with
                       j = 1 j
                                   ci 
                                                                       respect to Eps, MinPts if p belongs to NEps(q) |NEps (q)| >=
                  i =1
where k is the number of clusters, |ci| is the number of
elements in cluster ci, and Mci is the mean for cluster ci.             Density-Reachable:
                                                                        A point p is density-reachable from a point q with respect to
Steps of K-Means Algorithm                                              Eps, MinPts if there is a chain of points p1, …, pn, p1 = q, pn
The k Means algorithm is explained in the following steps.              = p such that pi+1 is directly density-reachable from pi
The algorithm normally converges in short iterations. But will
take considerably long time for iteration if the number of data         Density-Connected:
points and the dimension of each data are high.                         A point p is density-connected to a point q with respect to Eps,
                                                                        MinPts if there is a point o such that both, p and q are density-
Step 1: Choose k random points as the cluster centroids.                reachable from o with respect to Eps and MinPts.
Step 2: For every point p in the data, assign it to the closest
centroid. That is compute d(p, Mci) for all clusters, and assign        Algorithm: The algorithm of DBSCAN is as follows (M. Ester,
                                                                        H. P. Kriegel, J. Sander, 1996)
p to cluster C* where distance
                                                                             •    Arbitrary select a point p
(d(P, Mc*) <= d(P, Mci))                                                     •     Retrieve all points density-reachable from p with
                                                                                  respect to Eps and MinPts.
Step 3: Recompute the center point of each cluster based on all
points assigned to the said cluster.                                         •     If p is a core point, a cluster is formed.
                                                                             •     If p is a border point, no points are density-reachable
Step 4: Repeat steps 2 & 3 until there is convergence. (Note:                     from p and DBSCAN visits the next point of the
Convergence can mean repeating for a fixed number of times,                       database.
or until SEnew - SEold <= ε, where ε is some small constant, the
meaning being that we stop the clustering if the new SE                      •     Continue the process until all of the points have been
objective is sufficiently close to the old SE.)                                   processed.

B) DBSCAN Algorithm                                                     C) The Self-Organizing Map SOM
Density based spatial clustering of applications with noise rely        The Self Organizing Map (SOM) is developed by Professor
on a density-based notion of clusters, which is designed to             Teuvo Kohonen in the early 1980's. It is a computational
discover clusters of arbitrary shape and also have ability to           method for the visualization and analysis of high dimensional
handle noise.                                                           data.
DBSCAN requires two parameters                                          A self organizing map consists of components called nodes.
    • Eps: Maximum radius of the neighborhood                           The nodes of the network are connected to each other, so that
    • MinPts: Minimum number of points in an Eps-                       it becomes possible to determine the neighborhood of a node.
         neighborhood .                                                 Each node receives all elements of the training set, one at a
The clustering process is based on the classification of the            time, in vector format. For each element, Euclidean distance is
points in the dataset as core points, border points and noise           calculated to determine the fit between that element and the

                                                                   12                                 http://sites.google.com/site/ijcsis/
                                                                                                      ISSN 1947-5500
                                                             (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                 Vol. 9, No. 4,April, 2011
weight of the node. The weight is a vector of the same                   b is the number of elements in S that are not in the same
dimension as the input vectors. This allows to determine the             partition in X and not in the same partition in Y,
“winning node”, that is the node that represents the best                c is the number of elements in S that are in the same partition
training element. Once the winning node is found, the                    in X and not in the same partition in Y,
neighbors of the winning node are identified. The winning                d is the number of elements in S that are not in the same
node and these neighbors are then updated to reflect the new             partition in X but are in the same partition in Y.
training element.                                                        Intuitively, one can think of a + b as the number of agreements
It appears to be customary that both the neighborhood function           between X and Y and c + d the number of disagreements
and the learning rate are a decreasing function of time. This            between X and Y. The Rand index, R, then becomes,
means that as more training elements are learned, the
neighborhood is smaller and the nodes are less affected by the
new elements.
We express this change as the following function: for a node
                                                                         The Rand index has a value between 0 and 1 with 0 indicating
x, the update is equal to
                                                                         that the two data clusters do not agree on any pair of points
x(t+1) = x(t) + N(x,t)α(t)(ξ(t) – x(t))
                                                                         and 1 indicating that the data clusters are exactly the same.
x(t+1) is the next value of the weight vector
x(t) is the current value of the weight vector                           III. MODELING      MOVING OBJECT DATABASE                         WITH
N(x,t) is the neighborhood function, which decreases the                 UNCERTAINTY
         size of the neighbourhood as a function of time                  The following figure from [1] illustrates the problem
α(t) is the learning rate, which decreases as a function of               when a clustering algorithm is applied to moving objects
    time                                                                  with location uncertainty. Figure 4(a) shows the actual
ξ(t) is the vector representing the input document                        locations of a set of objects, Figure 4(b) shows the
Based on this information, the algorithm is given below.                  recorded location of these objects, which are already
Algorithm                                                                 outdated and Figure4(c) shows the uncertain data
      1. Initialize the weights of the nodes, either to random            locations. The clusters obtained from these outdated
           or pre computed values                                         values could be significantly different from those obtained
    2.   For all input elements:                                          as if the actual locations were available (Figure 4(b)). If we
                                                                          solely rely on the recorded values, many objects could
             •    Take the input, get its vector
                                                                          possibly be put into wrong clusters. Even worse, each
             •    For each node in the map: Compare the node              member of a cluster would change the cluster centroids,
                  with the input’s vector                                 thus resulting in more errors.
             •    The node with the vector closest to the input
                  vector is the winning node.
             •    For the winning node and its neighbors,
                  update them according to the formula above.

The Metric Used to Measure the Performance
In order to compare clustering results against external criteria,
a measure of agreement is needed. Since we assume that each
record is assigned to only one class in the external criterion
and to only one cluster, measures of agreement between two
partitions can be used.
The Rand index or Rand measure is a commonly used                            Figure 4: The Uncertain Data Clustering Scenario
technique for measure of such similarity between two data
clusters. This measure was found by W. M. Rand and                       We have modeled a moving object database to resemble the
explained in his paper "Objective criteria for the evaluation of         previously explained scenario. Here we present an example
clustering methods" in Journal of the American Statistical               case of the model under consideration. The Attributes of the
Association (1971).                                                      Simulated Moving Object Database presented here are:
Given a set of n objects S = {O1, ..., On} and two data clusters
of S which we want to compare: X = {x1, ..., xR} and Y =                 The Number of Groups                 : 5
{y1, ..., yS} where the different partitions of X and Y are              The Number of Dimensions             : 2
disjoint and their union is equal to S; we can compute the
following values:                                                        Number of Objects per Groups         : 50
a is the number of elements in S that are in the same partition          The Standard Deviation               : 0.6
in X and in the same partition in Y,

                                                                    13                              http://sites.google.com/site/ijcsis/
                                                                                                    ISSN 1947-5500
                                                                         (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                             Vol. 9, No. 4,April, 2011
                                                                                      algorithm, then certainly cluster centers in the two case will
                                                                                      slightly different from one another.
Total Area                                 : 2000 Sq. Units
Max Possible Mobility in unit time : 200 Units
                                                                                      IV. EXPERIMENTAL RESULTS
Total Number of Locations                   : 250
                                                                                      We have implemented the three clustering algorithms K-
Percentage of Uncertain Locations : 10 % (25 locations)                               means, DBSCAN and SOM in Matlab and performed the
                                                                                      experiments on a normal desktop computer.
                                                                                      We have kept some parameters of the simulation as constant
    The following plot of locations represents the real location of the object        and vary few parameters and measured the performance. The
    at time t.
                                                                                      following are the Constant and variable parameters of the

                                                                                      The Number Of Groups/Clusters : 3,4,5,6,7
                                                                                      The Number Of Dimensions           : 2
                                                                                      Number Of Objects Per Groups       : 50
                                                                                      The Standard Deviation             : 0.4-0.6
                                                                                      Total Area                         :2000 Sq. Units
                                                                                      Max Possible Mobility in unit time :200 Units
                                                                                      Total Number of Locations          : 250
                                                                                      Percentage of Uncertain Locations :10 % (25 locations)
                                                                                      The Number of Groups/Clusters was changed and in each case
                                                                                      the Rand index was measured with real data as well as the
                                                                                      recorded data with uncertainty. During creating synthetic
                                                                                      moving object database, the parameter, the standard deviation
                                                                                      is only used to attain non overlapping and well distributed
                                                                                      clusters.    To simulate uncertainty, 10% of locations
                                                                                      (uncertainty) were randomly altered from 0 to 200 units of
    Figure 5: Real Object Locations at Time t                                         In the following table(Table 1), we summarized the results
                                                                                      arrived in several iterations.
    The following plot of locations represents the recorded location of the
    object at the same time t.                                                                               Table 1: Summary of results

                                                                                                                  Accuracy of Classification (Rand Index)
                                                                                            Number of Clusters

                                                                                                                                                    With         Recorded
                                                                                                                  With Real Data
                                                                                                                                                    Uncertain Data




                                                                                      1    3                      0.94       1.00       0.99        0.86      0.96         0.93
                                                                                      2    4                      0.89       0.99       0.98        0.84      1.00         0.97
                                                                                      3    5                      0.84       0.92       0.92        0.88      0.99         0.83
                                                                                      4    6                      0.83       0.99       0.94        0.79      0.93         0.75
                                                                                      5    7                      0.79       0.99       0.82        0.83      0.97         0.76
                                                                                           Avg                    0.86       0.98       0.93        0.84      0.97         0.85
Figure 6: Recorded Object Locations at Time t
Since there are approximately 10% of un-updated objects in                            The following graph (Figure 7) shows the accuracy of
the database (intentionally introduced to simulate uncertainty),                      classification of real data. The Rand Index was measured
this plot is slightly different from the previous one. Due to the                     between the original and calculated class labels of real data.
uncertainty in the data, if we apply any classical clustering

                                                                                 14                                                 http://sites.google.com/site/ijcsis/
                                                                                                                                    ISSN 1947-5500
                                                              (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                  Vol. 9, No. 4,April, 2011
                                                                       V. CONCLUSION             AND     SCOPE         FOR      FURTHER
                   Clustering Accuracy with Real Locations
                  1                     0.978                          Traditional clustering algorithms do not consider
                                                       0.93            uncertainty inherent in a data item and can produce
   Rand Index

                                                                       incorrect mining results that do not correspond to the
                 0.9      0.858                                        real-world data. All the three algorithms produced little bit
                0.85                                                   poor result with uncertain data. But, while comparing the
                 0.8                                                   results with one another, it was observed that, the SOM
                                                                       based clustering algorithm has some ability to produce
                                                                       meaningful results even with the presence of uncertain
                         k-mean         SOM          DBSCAN
                                                                       records in the data. The reason for better results in the case
                                      Algorithm                        of SOM may be the aspect of unsupervised training involved
                                                                       in the clustering process which is approximating the
Figure 7: Accuracy of clustering with real locations                   uncertain data in a meaningful way.
The following graph (Figure 8) shows the accuracy of                   DBSCAN clustering algorithm and K-mean clustering
classification of Recorded data. The Rand Index was measured           algorithm were produced comparatively poor results than
between the original and calculated class labels of recorded           SOM. Particularly, the density based clustering algorithm
data.                                                                  DBSCAN produced little bit poor result than k-means. The
                                                                       main reason for this poor result is the nature of distribution
                Clustering Accuracy with Recorded Locations            of data (sphere/spheroid shaped distribution) under
                                                                       consideration. Generally all the density based clustering
                                                                       algorithms will try to do clustering in spatial data sets with
                  1                     0.97
                                                                       clusters of widely varying shapes; varying densities; and very
                0.95                                                   large data sets. With such kind of data, we may expect good
   Rand Index

                 0.9                                                   results with DBSCAN
                          0.84                        0.848
                0.85                                                   Future works may address the methods for handling the
                 0.8                                                   uncertainty along with other attributed during the clustering
                0.75                                                   process. In fact, there are few already available solutions for
                         k-mean         SOM          DBSCAN            uncertain data clustering with modified or improved k-means
                                                                       algorithm and DBSCAN algorithm. One may address new
                                      Algorithm                        ideas to improve the existing algorithms. Further, the issues
                                                                       involved in improving the performance of the algorithm in
Figure 8: Accuracy of Clustering with Recorded Locations               terms of speed as well as accuracy may be addressed in future
The following graph (Figure 9) shows the difference in                 works.
accuracy of classification between Real and Recorded data.
                                                                                     VI.      REFERENCES
                                                                           1.   Chau, M., Cheng, R., and Kao, B., "Uncertain Data
                                                                                Mining: A New Research Direction," in Proceedings
                                                                                of the Workshop on the Sciences of the Artificial,
                                                                                Hualien, Taiwan, 2005.
                                                                           2.   Charu C. Aggarwal, "On Density Based Transforms
                                                                                for Uncertain Data Mining", IBM T. J. Watson
                                                                                Research Center, 19 Skyline Drive, Hawthorne, NY
                                                                           3.   Ben Kao Sau, Dan Lee, David W. Cheung, Wai-Shing
                                                                                Ho, K. F. Chan, "Clustering Uncertain Data using
                                                                                Voronoi Diagrams", Eighth IEEE International
                                                                                Conference on Data Mining,2008
Figure 9: The difference in clustering accuracy                            4.   Barbara, D., Garcia-Molina, H. and Porter, D. "The
                                                                                Management of Probabilistic Data," IEEE
                                                                                Transactions on Knowledge and Data Engineering,

                                                                  15                               http://sites.google.com/site/ijcsis/
                                                                                                   ISSN 1947-5500
                                                    (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                        Vol. 9, No. 4,April, 2011
5.   Bezdek, J. C. Pattern Recognition with Fuzzy
     Objective Function Algorithms. Plenum Press, New
                                                                                  AUTHOR’S PROFILE
     York (1981).
6.   Cheng, R., Kalashnikov, D., and Prabhakar, S.
     "Evaluating Probabilistic Queries over Imprecise                             Ms.Angeline Christobel, Asst. Professor,
     Data," Proceedings of the ACM SIGMOD                                         AMA International University, Bahrain
     International Conference on Management of Data,                              is currently pursuing her research in
     June 2003.                                                                   Karpagam University, Coimbatore,
                                                                                  Tamil Nadu, India. Her research interest
7.   Cheng, R., Kalashnikov, D., and Prabhakar, S.
                                                                                  is in Data mining.
     "Querying Imprecise Data in Moving Object
     Environments," IEEE Transactions on Knowledge
     and Data Engineering, 2004
8.   Cheng, R., Xia, X., Prabhakar, S., Shah, R. and
     Vitter, J.    "Efficient Indexing Methods    for                             Dr. Sivaprakasam is working as a
     Probabilistic Threshold Queries over Uncertain                               Professor in Sri Vasavi College, Erode,
     Data," Proceedings of VLDB, 2004.                                            Tamil Nadu, India. His research
                                                                                  interests include Data mining, Internet
9.   Hamdan, H. and Govaert, G. "Mixture Model                                    Technology,         Web & Caching
     Clustering of Uncertain Data," IEEE International                            Technology,             Communication
     Conference on Fuzzy Systems, 2005.                                           Networks and Protocols, Content
10. Ruspini, E. H. "A New Approach to Clustering,"                                Distributing Networks.
    Information Control, 1969.
11. Sato, M., Sato, Y., and Jain, L. “Fuzzy Clustering
    Models     and   Applications”,    Physica-Verlag,
    Heidelberg 1997.
12. Wolfson, O., Sistla, P., Chamberlain, S. and Yesha,
    Y. "Updating and Querying Databases that Track
    Mobile Units," Distributed and Parallel Databases,
13. Charu C. Aggarwal and Philip S. Yu “A Survey of
    Uncertain Data Algorithms and Applications” IEEE
    transactions on knowledge and data Engineering,
14. Martin Ester, Hans Peter Kriegel, Jorg Sander,
    Xiaowei Xu “ A Density based Algorithm for
    Discovering Clusters in Large Spatial Databases with
    Noise” Proceedings of 2nd International Conference
    on Knowledge Discovery and Data mining(KDD-96)
15. H.P.Kriegel and M.Pfeifle, “Density based clustering
    of uncertain data:, ACM KDD Conference,2005
16. Charu C. Aggarwal and Philip S. Yu “On Indexing
    High Dimensional Data With Uncertainty”, IBM T. J.
    Watson Research Center
17. Rustum R, Adeloye AJ. "Replacing outliers and
    missing values from activated sludge data using
    Kohonen Self Organizing Map". Journal of
    Environmental Engineering,2007

                                                           16                            http://sites.google.com/site/ijcsis/
                                                                                         ISSN 1947-5500

To top