# Numerical Linear Algebra

Document Sample

```					Numerical Linear Algebra
Chris Rambicure
Guojin Chen
Christopher Cprek
WHY USE LINEAR ALGEBRA?
1) Because it is applicable in many
problems….

2)…And it’s usually easier than calculus
TRUE

“Linear algebra has become as basic and as applicable as
calculus,and fortunately it is easier.”
-Gilbert Strang

Calculus
HERE COME THE BASICS…
SCALARS
What you’re used to dealing with
Have magnitude, but no direction
VECTORS
Represent both a magnitude and a
direction
Can add or subtract, multiply by scalars,
or do dot or cross products
THE MATRIX
It’s an mxn array
Holds a set of numerical values
Especially useful in solving certain types
of equations
Operations: Transpose, Scalar Multiply,
EIGENVALUES
You can choose a matrix A, a vector x,
and a scalar x so that Ax = sx, meaning
the matrix just scales the vector
X in this case is called an eigenvector,
and s is its eigenvalue
CHARACTERISTIC EQUATION
det(M-tI) = 0
M: the matrix
I: the identity
t: eigenvalues
CAYLEY-HAMILTON THEOREM
IF

AND

THEN p(A) = 0, meaning A satisfies its
characteristic equation
A Couple Names, A Couple
Algorithms
IN THE BEGINNING…
(Grassmann’s Linear Algebra)

Grassmann is considered to be the
“father” of linear algebra
Developed the idea of a linear algebra
in which the symbols representing
geometric objects can be manipulated
Several of his operations: the interior
product, the exterior product, and the
multiproduct
What’s a Multiproduct
Equation Look Like?
d1d2 + d1d2 = 0
The multiproduct has many uses,
including scientific, mathematic,
and industrial
Got updated by William Clifford
CLIFFORD’S MODIFICATION
TO GRASSMAN’S EQUATION
d1d2 + d1d2 = 2kij
The 2kij is what’s referred to as
Kronecker’s Symbol
Both of these equations are used for
Quantum Theory Math
VECTOR SPACE
Another idea which is kind of tied with
Grassman
Vector Space refers to some set of
vectors that contains the origin
It is usually infinite
Subspace is a subset of vector space.
It, of course, is also vector space
Cholesky Decomposition
Algorithm developed by Arthur Cayley
Takes a matrix and factors it into a
triangular matrix times its transpose
A=R’R
Useful for matrix applications
Becomes even more worthwhile in
parallel
HOW TO USE LINEAR
ALGEBRA FOR PDE’S
You can use matrices and vectors to
solve partial differential equations
For equations with lots of variables,
you’ll wind up with really sparse
matrices
Hence, the project we’ve been working
on all year
BIBLIOGRAPHY
“Hermann Grassmann.” Online.
http://members.fortunecity.com/johnhays/grassmann
.htm
“Abstract Linear Spaces. Online. http://www-
groups.dcs.stand.ac.uk/~history/HistTopics/Abstract_
linear_spaces.html
Liberman, M. “Linear Algebra Review.” Online.
http://www.ling.upenn.edu/courses/ling525/linear_al
gebra_review.html
“Cholesky Factorization.” Online.
http://www.netlib.org/utk/papers/factor/node9.html
Numerical Linear Algebra
Guojin Chen
Christopher Cprek
Chris Rambicure
Johann Carl Friedrich Gauss

Born: April 30, 1777 (Germany)
Died: Feb 23, 1855 (Germany)
Gaussian Elimination
LU Factorization
Operation Count
Instability of Gaussian Elimination
without Pivoting
Gaussian Elimination with Partial
Pivoting
Linear systems
A linear system of equations (n equations with n unknowns) can be
written:

a11 x1 + a12 x2 + ... + a1n xn = b1
a21 x1 + a22 x2 + ... + a2n xn = b2
...
an1 x1 + an2 x2 + ... + ann xn = bn

Using matrices, the above system of linear equations can be
written:
Gauss Elimination and Back Substitution

Convert this to triangular form:

Then solve the system by Back Substitution.
LU Factorization
Gaussian elimination transforms a full
linear system into an upper-triangular
one by applying simple linear
transformations on the left.
Let A be a square matrix. The idea is to
transform A into upper-triangular matrix
U by introducing zeros below the
diagonal.
LU Factorization
This “elimination” process is equivalent
to multiplying by a sequence of lower-
triangular matrices Lk on the left:

Lm-1 … L2L1A = U
LU Factorization
Setting L = (Lm-1 )-1… (L2)-1(L1)-1
We obtain an LU factorization of A
A = LU
In order to find a general solution of a system of equations, it is
helpful to simplify the system as much as possible.
Gauss elimination is a standard method (which has the advantage
of being easy to implement on a computer) for doing this.
Gauss elimination uses elementary operations. We can:
interchange any two equations
multiply an equation by a (nonzero) constant
add a multiple of one equation to any other one
and aim to reduce the system to triangular form. The system
obtained after each operation is equivalent to the original one,
meaning that they have the same solutions.
Algorithm of Gaussian Elimination without Pivoting

U = A, L = I
For k = 1 to m-1
for j = k +1 to m
ljk = ujk/ukk
uj,k:m = uj,k:m – ljkuk,k:m
Operation Count
There are 3 loops in the previous
algorithm
There are 2 flops per entry
For each value of k, the inner loop is
repeated for rows k+1, …, m.
Work for Gaussian elimination is

~ 2    m3 flops
3
Instability of Gaussian
Elimination without Pivoting
Consider the following matrices:

A1 = 0        1
1     1
       
1020
A2 =          1

 1       1
Pivoting
Pivots
Partial Pivoting
Example
Complete Pivoting
Pivot
Partial Pivoting
Example
2                1       1        0
4                3       3        1
A   =8
                 7       9

5
                                   
6                7       9        8

             1    
2       1       1   0        8   7   9   5
4                   1        4

    1             
           3       3        =       3   3   1

1                    8       7       9   5        2   1   1   0
                                                               
                 1
6       7       9   8
6   7   9   8

P1
L1
 1                             8   7        9   5
 1/ 2        1                4   3        3   1
                                                 
 1/ 4                 1       2   1        1   0
                                                 
 3 / 4                    1   6   7        9   8

8            7         9              5 
0       1/ 2         3/ 2          3 / 2
=                                                  
0       3/ 4         5/ 4          5 / 4
                                           
0        7/4          9/4           17 / 4 
Reference:
http://www.maths.soton.ac.uk/teaching/units/ma273/node8.html
http://www.maths.soton.ac.uk/teaching/units/ma273/node9.html
Numerical Linear Algebra by Lloyd Trefethen and David Bau, III
http://www.sosmath.com/matrix/system1/system1.html
Numerical Linear Algebra:
The Computer Age
Christopher Cprek
Chris Rambicure
Guojin Chen
What I’ll Be Covering
Algebra relevant.
LAPACK
Solving Dense Matrices on Parallel
Computers.
Why All the Sudden Interest?
Gregory Moore regards the
axiomatization of abstract vector spaces
to have been completed in the 1920s.
Linear Algebra wasn’t offered as a
separate mathematics course at major
universities until the 1950’s and 60’s.
Interest in linear algebra skyrocketed.
Before computers, solving a system of
100 equations with 100 unknowns was
unheard of.
The brute mathematical force of
incredibly useful for all kinds of
applications involving linear algebra.
Computers and Linear Algebra
The computer software Matlab provides
a good example: it is among the most
popular in engineering applications and
at its core it treats every problem as a
linear algebra problem.
A need for more advanced large matrix
operations resulted in LAPACK.
What is LAPACK?
Linear Algebra PACKage
Software package designed specifically
for linear algebra applications.
The original goal of the LAPACK
project was to make the widely used
EISPACK and LINPACK libraries run
efficiently on shared-memory vector
and parallel processors.
LAPACK continued…
LAPACK is written in Fortran77 and provides
routines for solving systems of simultaneous
linear equations, least-squares solutions of
linear systems of equations, eigenvalue
problems, and singular value problems.
Dense and banded matrices are handled, but
not general sparse matrices. In all areas,
similar functionality is provided for real and
complex matrices, in both single and double
precision.
Parallel Dense Matrix
Partitioning
Parallel computers are well suited for
processing large matrices.
In order to process a matrix in parallel,
it is necessary to partition the matrix so
that the different partitions can be
mapped to different processors.
Partitioning Dense Matrices
Striped Partitioning
   Block-Striped
   Cyclic-Striped
   Block-Cyclic-Striped
Checkerboard Partitioning
   Block-Checkerboard
   Cyclic-Checkerboard
   Block-Cyclic-Checkerboard
Striped Partitioning
Matrix is divided into groups of
complete rows or columns, and each
processor is assigned one such group.
Striped Partitioning cont…
Block-striped Partitioning is when
contiguous rows or columns are
assigned to each processor together.
Cyclic-striped Partitioning is when rows
or columns are sequentially assigned to
processors in a wraparound manner.
Block-Cyclic-Striped is a combination of
the two.
Striped Partitioning cont…
In a column-wise block striping of an
n*n matrix on p processors (labeled
P(0), P(1), …, P(P-1):
   P(I) contains columns with indices (n/p)I,
(n/p)I + 1, … , (n/p)(I+1) – 1.
In row-wise striping:
   P(I) contains rows with indices I, I+p,
I+2p, … , I+n-p.
Checkerboard Partitioning
The matrix is divided into smaller square or
rectangular blocks or submatrices that are
distributed among processors.
Checkerboard Partitioning
cont…

Much like striped-partitioning, checkerboard
partitioning may use block, cyclic, or a
combination.
A checkerboard-partitioned square matrix
maps naturally onto a two-dimensional
square mesh of processors. An n*n matrix
onto a p processor mesh divides the blocks
into size (n/p)*(n/p).
Matrix Transposition on a Mesh
Assume that an n*n matrix is stored in an
n*n mesh of processors, so each processor
holds a single element.
A diagonal runs down the mesh.
An element above the diagonal moves down
to the diagonal and then to the left to its
destination processor.
An element below the diagonal moves up to
the diagonal and then to the right to its
destination processor.
Matrix Transposition cont…
Matrix Tranposition cont…
An element at initial p8 moves to p4,
p0, p1, and finally to p2.
If p<n*n, then the tranpose can be
computed in two phases.
   Square matrix blocks are treated as
indivisible units, and whole blocks are
elements.
   Then do a local rearrangement within the
blocks.
Matrix Transposition cont…
Communication and the Local Rearrangement
Matrix Transposition cont…
The total parallel run-time of the
procedure for transposition of matrix on
a parallel computer:
Parallelization of Linear Algebra
Transposition is just an example of how
numerical linear algebra can be easily
and effectively parallelized.
The same techniques and principles can
be applied to operations like
This explains their current popularity.
Conclusion
Linear algebra is flourishing in an age of
computers, where there are limitless
applications.
LAPACK exists as an efficient code
library for processing large systems of
equations on parallel processing
computers.
Parallel Computers are very well suited
to these kinds of problems.
http://www.crpc.rice.edu/CRPC/brochure/res_la.html
http://citeseer.nj.nec.com/26050.html
http://www.maa.org/features/cowen.html
http://www.nersc.gov/~dhbailey/cs267/Lectures/Lect_10_2000.
pdf
http://www.cacr.caltech.edu/ASAP/news/specialevents/tutorialnl
a.htm
http://www.netlib.org/scalapack/
http://citeseer.nj.nec.com/125513.html
http://discolab.rutgers.edu/classes/cs528/lectures/lecture7/
http://www.cse.uiuc.edu/cse302/lec20/lec-matrix/lec-
matrix.html

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 8 posted: 11/26/2011 language: English pages: 65
How are you planning on using Docstoc?