How do I know the answer if I’m not sure of the question?
Putting robustness into estimation
K. E. Schubert 11/7/00
Familiar Picture?
Basic Problem
Picture of something that has been blurred If I know how it was blurred then I should be able to clean it up If system is invertible then I can get the original
A
x b
A
†
x
b
Familiar Picture
Encountering Resistance
Consider a simpler problem.
– –
–
–
Unknown resistor. Take current and voltage measurements. Plot them out. Want to fit a line to the points.
No measurement is perfect.
–
–
No exact fit to all the points. Want “best” fit.
Measured Values
12 Unknown Resistor 10
8
6
4
2
0
0
2
4
6
8 10 voltage [V]
12
14
16
18
Gauss’ Stellar Problem
Orbit of Ceres. Errors were in people’s measurements Consider distance from the measurements to the equation to fit minimize the square of this distance
–
min ||Ax-b||
T -1 T
2
x=(A A) A b=A b
†
Understanding Solution
In our problem A, b are vectors Finding nearest scaled A to b Projection
b
Ax-b A
Ax
Resistor Solved
Want to find slope, 1/R i=(1/R)v Ax=b A vector of voltages b vector of currents x is slope † 1/R=v i
Best line
12 Unknown Resistor 10
8
6
4
2
0
0
2
4
6
8 10 voltage [V]
12
14
16
18
Reasonable Question
What if I considered v=iR? Errors assumed in v now! † R=i v How do the measured resistances compare?
Comparison of Methods
12 Unknown Resistor 10
8
6
4
2
0
0
2
4
6
8 10 voltage [V]
12
14
16
18
Errors in Both
A has errors (actual is A+dA) Want to minimize distance
–
min ||(A+dA)x-b||
2
Need to know something about dA Worst dA in bounded region Best dA in bounded region The dA that makes Ax=b consistent
Worst in a Bounded Region
Keep worst case ok, rest will be fine
Projection to farthest A+dA
b (A+dA)x-b dA
||dA||< (bounded region)
A
(A+dA)x
Best in a Bounded Region
Pick best dA but limit options
Projection to nearest A+dA
b (A+dA)x-b dA
||dA||< (bounded region)
(A+dA)x
A
Consistent Equation (TLS)
Called Total Least Squares Projection nearest to A and b in new space No bound on dA, as big as need!
b
(A+dA)x
A
General Regression Problems
All of the techniques mentioned so far fall into the general category of regression (including least squares) Find a solution for most by taking the gradient and setting it equal to zero T -1 T x=(A A+I) A b Equation for , which is solved by finding the roots of the equation (Newton’s or bisection)
Resistor by TLS
12 Unknown Resistor 10
8
6
4
2
0
0
2
4
6
8 10 voltage [V]
12
14
16
18
Simple Picture
Consider a city skyline.
– –
Only consider outline of buildings. Height is a function of horizontal distance.
Nice one dimensional picture.
Hazy Day
Smog and haze blur the image.
– –
3.5 Actual Measur ed 3
Rounds the corners off. Want to get the corners back.
2.5
2
1.5
1
0.5
0
0
10
20
30
40
50
60
70
80
90
100
Least Squares Fails!
Blurring works like a Gaussian distribution Don’t know the exact blur
2000 Measur ed Least Squares 1500 1000
500
0
-5 00
-1 000
-1 500
-2 000 0
10
20
30
40 50 60 Least Squares Solution
70
80
90
100
TLS Too Optimistic!
TLS assumes things are consistent Allows dA to be large
60 Measur ed TLS 40 20
0
-2 0
-4 0
-6 0 0
10
20
30
40
50
60
70
80
90
100
More Robust Solutions
Picking a solution with some restrictions yields good results.
3.5 Actual Measur ed MinMin MinBE 3 2.5
2
Signal
1.5
1
0.5
0
-0 .5 0
10
20
30
40
50 Sample
60
70
80
90
100
Conclusions
Least Squares has nice properties and generally works well. Problems can arise in simple problems.
–
Fundamental errors
Must account for errors in basic system. Robust ~ works well for all nearby systems
–
Can’t do as well or as bad (compromise)