Least Squares Best Fit Line

Document Sample
Least Squares Best Fit Line Powered By Docstoc
					               Least Squares Best Fit Line
Least Squares Best Fit Line

The method of least squares is a standard approach to the approximate solution of
overdetermined systems, i.e., sets of equations in which there are more equations
than unknowns.

"Least squares" means that the overall solution minimizes the sum of the squares of
the errors made in the results of every single equation.

The most important application is in data fitting. The best fit in the least-squares
sense minimizes the sum of squared residuals, a residual being the difference
between an observed value and the fitted value provided by a model.

When the problem has substantial uncertainties in the independent variable (the 'x'
variable), then simple regression and least squares methods have problems;

in such cases, the methodology required for fitting errors-in-variables models may be
considered instead of that for least squares.
                  Know More About Point Slope and Standard Form Worksheet


  Tutorcircle.com                                              Page No. : ­ 1/4
Least squares problems fall into two categories: linear or ordinary least squares and
non-linear least squares, depending on whether or not the residuals are linear in all
unknowns.

The linear least-squares problem occurs in statistical regression analysis; it has a
closed-form solution.

A closed-form solution (or closed-form expression) is any formula that can be
evaluated in a finite number of standard operations.

The non-linear problem has no closed-form solution and is usually solved by iterative
refinement; at each iteration the system is approximated by a linear one, thus the
core calculation is similar in both cases.

The least-squares method was first described by Carl Friedrich Gauss around 1794.
Least squares corresponds to the maximum likelihood criterion if the experimental
errors have a normal distribution and can also be derived as a method of moments
estimator.

The following discussion is mostly presented in terms of linear functions but the use
of least-squares is valid and practical for more general families of functions.

Also, by iteratively applying local quadratic approximation to the likelihood (through
the Fisher information), the least-squares method may be used to fit a generalized
linear model.

For the topic of approximating a function by a sum of others using an objective
function based on squared distances, see least squares (function approximation).
  Read  More About Solving Systems by Elimination using Multiplication Worksheet



  Tutorcircle.com                                              Page No. : ­ 2/4
Regression for prediction. Here a model is fitted to provide a prediction rule for
application in a similar situation to which the data used for fitting apply.

Here the dependent variables corresponding to such future application would be
subject to the same types of observation error as those in the data used for fitting. It is
therefore logically consistent to use the least-squares prediction rule for such data.

Regression for fitting a "true relationship". In standard regression analysis, that leads
to fitting by least squares, there is an implicit assumption that errors in the
independent variable are zero or strictly controlled so as to be negligible. When errors
in the independent variable are non-negligible, models of measurement error can be
used; such methods can lead to parameter estimates, hypothesis testing and
confidence intervals that take into account the presence of observation errors in the
independent variables.

An alternative approach is to fit a model by total least squares; this can be viewed as
taking a pragmatic approach to balancing the effects of the different sources of error
in formulating an objective function for use in model-fitting.




   Tutorcircle.com                                                 Page No. : ­ 3/4
                                                                   Page No. : ­ 2/3
Thank You For Watching




     Presentation

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:6/15/2012
language:
pages:4