Introduction to Probability, Statistics, and Random Processes

							                        CNS 286C – 3-0-6 Units - Spring 2008
                         MWF 4-5, Beckman Institute 115

          “Introduction to Probability, Statistics, and Random Processes”

Syllabus (date/topic associations are approximate!)

           Week 1 Mar. 31 – Apr 4
           A. Organization of Course
           B. Introduction and Mathematical Background:
The Basic Terminology of Probability Theory
Sample Spaces
Probability Measures
           C. Permutations and Combinations:
Binomial Coefficients
The Matching Problem
The Birthday Problem
The Girl-Named-Florida Problem
           D. Intro to Conditional Probability:
Definition of Conditional Probability
Relations between Intersections and Conditional Probability
Inverse Probability
Conditioning
Bayes’ Rule


           Week 2 Apr 7 – Apr 11
           A. More Conditional Probability:
Medical Screening Tests
Neuronal Sparseness Model
Prior and Posterior Probability
Bayesian Updating
Tree Diagrams
The Monty Hall Problem
Independence
           B. Introduction to Random Variables:
Set Functions
Concept of a random variable
Discrete Random Variables
Continuous Random Variables
The Definition of Continuity
Probability Distributions and Densities
Cumulative Distribution Functions




                                                                            1
            Week 3 Apr 14 – Apr 18
                A. More Random Variables:
Changes of Variable
Joint and Marginal Distributions
                B. Expectations:
Expectation of a Random Variable
Expectations of Sums
Bernoulli trials
Indicator Variables
Expectations in Occupancy Problems
Solving Matching Problems Employing Indicator Variables
The Coupon Collector’s Problem
Expectations of Products
                C. Intro to Moments
Moments of Probability Distributions
Moment Generating Functions
The Second Moment and the Variance
Linear Transformations of a Random Variable


           Week 4 Apr 21 – Apr 25
                  A. More on Moments:
The Covariance of Two Random Variables
The Probability Density for a sum of Random Variables
The Variance in Matching Problems
                  B. Special Probability Distributions:
The Discrete and Continuous Uniform Distribution
A Gambler’s Paradox
The Binomial Distribution
The Relation of Binomial Distribution and Bernoulli Trials
The Poisson Approximation to the Binomial Distribution
The Poisson Distribution and Applications
The Geometric Distribution and Applications
                  C. Intro to The Normal Distribution:
Moments of Probability Distributions
The Gaussian (normal) density
The Normal Cumulative Distribution Function
The Standard Normal Density and Distribution Function


           Week 5 Apr 28 – May 2
                    A. More on the Normal Distribution:
Percentiles
The Normal Moment Generating Function
Applications to Waiting Times, Inventory Management, and Electronic Noise.
The Chi Square as the Distribution for the Sum of Squared Normal Random Variables



                                                                                    2
Properties of the Chi Square Distribution
The Weak and strong law of large numbers
The Central Limit Theorem
Application to Risk in Insurance and Civil Engineering
The Normal Approximation to the Binomial Distribution
                   B. Intro to Statistical Sampling:
Defining a random sample from a distribution of a random variable
Inference Problems
What is meant by a Statistic
The Sample Mean and its Expectation
The Probability Distribution of the Sample Mean


            Week 6 May 5 – May 9
                            More Statistical Sampling:
The Median and its Distribution, Expectation and Variance
The Sample Variance and its Expectation
Using the Normal Curve
Intro to signal detection theory
Relating Mean and Variance of Sample Means and Sample Variances to the Population
Distribution of the Sample Mean and Variance for Normal Populations
The t-distribution
Some Applications
Sampling from a finite population
Application to Polling Data


           Week 7 May 12 – May 16
                         A. Point Estimation:
Biased and Unbiased Estimators
Minimum Variance Estimators
Consistent Estimators
Point estimation
Method of Maximum Liklihood
Applications to Normal, Binomial and Poisson Distributions

Wed. May 14: QUIZ #1 on weeks 1 – 5 (probability)

                         B. Interval Estimation:
Interval estimation
Confidence Intervals
Interval Estimates for a Normal Population
Comparing Normal Populations
Confidence Intervals for Bernoulli Random Variables
Bayesian Intervals




                                                                                    3
             Week 8: May 19 – May 23
                           A. Hypothesis Testing
Idea Behind Hypothesis testing
Losses, Errors and Risks
Null and Alternative hypotheses
Simple and Composite Hypotheses
Significance Levels for means and variance
The F-test
Power of a test
Likelihood ratio
Type I and II errors
Testing and Comparing Normal Populations
Case of Unknown Variance: The t-test
Analysis of variance (ANOVA)
                           B. Regression and Correlation:
Dependent and Independent Variables
Linear Regression
The Method of Least Squares
Distribution of the Estimators
Statistical Inferences about the Regression Parameters
Regression to the Mean
The Correlation Coefficient
                           C. Intro to Random Processes
What is a random process?
Definition and Tests for Random Numbers and Sequences
Coin Tossing Models
Changes in leads
Hot hand fallacies


         Week 9: May 26 – May 30
Mon May 26 = No Class - Memorial Day
Wed May 28 = QUIZ #2 on weeks 6-8 (statistics).

                                 Introduction to Random Processes
Poisson processes
Applications of Poisson processes
Definition of Markov Processes
Basic Concepts of random walks
Multi-dimensional Walks
Random Walks with Barriers




                                                                    4
The Ballot theorem
Probability of return
Last visit and long leads
First passages
Classic ruin problem
Expected durations
Brownian Motion
Diffusion

JUNE 6: Final Exam

Class Web Site: http://www.klab.caltech.edu/cns286/

Instructor: Leonard Mlodinow
email: len@caltech.edu
ext: 8969
off: BBB room 200B
Office hours: MWF 5-6 (i.e., immediately after class)

TA: Alex Huth
email:alex.huth@gmail.com
off: BBB 200A
Office hours: 4-5 the day preceding all quizzes and homework due dates, or by
appointment (feel free to e-mail), at any time.

Class Requirements: you are responsible for attending the lectures, and doing the
homework and quizzes. There will be 5 homework assignments and 2 quizzes, each
counting 10% of your grade. The final exam will count 30%.

Texts: No text is required for the course, however the following will be on reserve at
SFL:
An Introduction to Probability Theory by W. Feller (1971)
Introduction to Probability Models by S.M. Ross (2002)
Introduction to Probability and Statistics for Scientists and Engineers (2004)




                                                                                         5