From Wikipedia, the free encyclopedia Cluster-weighted modeling
Cluster-weighted modeling
In statistics, cluster-weighted modeling (CWM) is an functions between the clusters determines
algorithm-based approach to non-linear prediction of whether a particular value of x is associated
outputs (dependent variables) from inputs (independent with any given cluster-center. This density
variables) based on density estimation using a set of mod- might be a Gaussian function centered at a
els (clusters) that are each notionally appropriate in a parameter representing the cluster-center.
sub-region of the input space. The overall approach
works in jointly input-output space and an initial version In the same way as for regression analysis, it will be im-
was proposed by Neil Gershenfeld.[1][2] portant to consider preliminary data transformations as
part of the overall modeling strategy if the core compo-
nents of the model are to be simple regression models for
Basic form of model the cluster-wise condition densities, and normal distrib-
The procedure for cluster-weighted modeling of an utions for the cluster-weighting densities pj(x).
input-output problem can be outlined as follows.[2] In or-
der to construct predicted values for an output variable
y from an input variable x, the modeling and calibration
General versions
procedure arrives at a joint probability density function, The basic CWM algorithm gives a single output cluster
p(y,x). Here the "variables" might be uni-variate, multi- for each input cluster. However, CWM can be extended to
variate or time-series. For convenience, any model para- multiple clusters which are still associated with the same
meters are not indicated in the notation here and sev- input cluster.[3] Each cluster in CWM is localized to a
eral different treatments of these are possible, including Gaussian input region, and this contains its own trainable
setting them to fixed values as a step in the calibration local model.[4] It is recognized as a versatile inference al-
or treating them using a Bayesian analysis. The required gorithm which provides simplicity, generality, and flex-
predicted values are obtained by constructing the con- ibility; even when a feedforward layered network might
ditional probability density p(y|x) from which the pre- be preferred, it is sometimes used as a "second opinion"
diction using the conditional expected value can be ob- on the nature of the training problem.[5]
tained, with the conditional variance providing an indi- The original form proposed by Gershenfeld describes
cation of uncertainty. two innovations:
The important step of the modeling is that p(y|x) is as- • Enabling CWM to work with continuous streams of
sumed to take the following form, as a mixture model: data
• Addressing the problem of local minima
encountered by the CWM parameter adjustment
process[5]
CWM can be used to classify media in printer applica-
where n is the number of clusters and {wj} are weights tions, using at least two parameters to generate an out-
that sum to one. The functions pj(y,x) are joint probability put that has a joint dependency on the input parame-
density functions that relate to each of the n clusters. Th- ters.[6]
ese functions are modeled using a decomposition into a
conditional and a marginal density:
pj(y,x) = pj(y | x)pj(x),
References
[1] Gershenfeld, N. (1997) "Nonlinear Inference and
where: Cluster-Weighted Modeling", Annals of the New York
• pj(y|x) is a model for predicting y given x, and Academy of Sciences, 808, 18–24. doi:10.1111/
given that the input-output pair should be j.1749-6632.1997.tb51651.x
associated with cluster j on the basis of the [2] ^ Gershenfeld, N., Schoner, B.* & Metois, E. (1999)
value of x. This model might be a regression Cluster-weighted modelling for time-series
model in the simplest cases. analysis, Nature, 397 (28 Jan. 1999), 329–332
[3] Feldkamp, L.A.; Prokhorov, D.V.; Feldkamp, T.M.
• pj(x) is formally a density for values of x, given (2001). "Cluster-weighted modeling with
that the input-output pair should be associated multiclusters". International Joint Conference on
with cluster j. The relative sizes of these Neural Networks 3 (1): 1710–1714.
1
From Wikipedia, the free encyclopedia Cluster-weighted modeling
http://ieeexplore.ieee.org/Xplore/login.jsp?url=/ M. Feldkamp. A New Approach to Cluster-Weighted
iel5/7474/20319/00938419.pdf?temp=x. Modeling. Dearborn, MI: Ford Research Laboratory.
[4] Boyden, Edward S.. Tree-based Cluster Weighted http://home.comcast.net/~dvp/cwm.pdf.
Modeling: Towards A Massively Parallel Real-Time [6] Gao, Jun; Ross R. Allen (2003-07-24). CLUSTER-
Digital Stradivarius. Cambridge, MA: MIT Media Lab. WEIGHTED MODELING FOR MEDIA CLASSIFICATION.
http://edboyden.org/violin.pdf. Palo Alto, CA: World Intellectual Property
[5] ^ Prokhorov, A New Approach to Cluster-Weighted Organization. http://www.wipo.int/pctdb/en/
Modeling Danil V.; Lee A. Feldkamp, and Timothy wo.jsp?wo=2003059630.
Retrieved from "http://en.wikipedia.org/w/index.php?title=Cluster-weighted_modeling&oldid=457670318"
Categories:
• Multivariate statistics
• Data clustering algorithms
• Estimation of densities
This page was last modified on 27 October 2011 at 15:32. Text is available under the Creative Commons Attribution-
ShareAlike License; additional terms may apply. See Terms of use for details. Wikipedia® is a registered trademark of
the Wikimedia Foundation, Inc., a non-profit organization.Contact us
Privacy policy About Wikipedia Disclaimers Mobile view
2