Get documents about "02 Signal Detection Theory
Document Sample
2. SIGNAL DETECTION THEORY
J. Elder PSYC 6256 Principles of Neural Coding
Signal Detection Theory
2 Probability & Bayesian Inference
Provides a method for characterizing human
performance in detecting, discriminating and
estimating signals.
For noisy signals, provides a method for identifying
the optimal detector (the ideal observer) and for
expressing human performance relative to this.
Origins in radar detection theory
Developed through the 1950s and on by Peterson,
Birdsall, Fox, Tanner, Green & Swets
PSYC 6256 Principles of Neural Coding J. Elder
Example 1
3 Probability & Bayesian Inference
The observer sits in a dark room
On every trial, a dim light will be flashed with 50%
probability.
The observer indicates whether she believes the
light was flashed or not.
This is a yes-no detection task.
PSYC 6256 Principles of Neural Coding J. Elder
Noise
4 Probability & Bayesian Inference
In this example, the information useful for the task is the light
energy of the stimulus.
By the time the stimulus information is received by decision
centres in the brain, it will be corrupted by many sources of
noise:
photon noise
isomerization noise
neural noise
Many of these noise sources are Poisson in nature: the
dispersion increases with the mean.
PSYC 6256 Principles of Neural Coding J. Elder
Equal-Variance Gaussian Case
5 Probability & Bayesian Inference
It is often possible to approximate this noise as
Gaussian-distributed, with the same variance for
both stimulus conditions.
Then the noise is independent of the signal state.
PSYC 6256 Principles of Neural Coding J. Elder
Discriminability d’
6 Probability & Bayesian Inference
(
⎛ x−µ
) ⎞
2
1
( )
p x | S = sH = exp ⎜ −
⎜ 2σ
H
2
⎟
⎟
2πσ ⎝ ⎠
⎛ x−µ
( ) ⎞
2
1
( )
p x | S = sL = exp ⎜ −
⎜ 2σ 2
L
⎟
⎟
2πσ ⎝ ⎠
signal separation µH − µL µH − µL
d'= =
signal dispersion σ
σ
PSYC 6256 Principles of Neural Coding J. Elder
Criterion Threshold
7 Probability & Bayesian Inference
The internal response is often approximated as a continuous
variable, called the decision variable.
But to yield an actual decision, this has to be converted to a
binary variable (yes/no).
A reasonable way to do this is to define a criterion threshold z:
z
x ≥ z → ' yes'
x < z → 'no'
x
x
PSYC 6256 Principles of Neural Coding J. Elder
Effect of Shifting the Criterion
8 Probability & Bayesian Inference
PSYC 6256 Principles of Neural Coding J. Elder
How did we calculate these numbers?
9 Probability & Bayesian Inference
(
⎛ x−µ
) ⎞
2
1
( )
p x | S = sH = exp ⎜ −
⎜ 2σ
H
2
⎟
⎟
2πσ ⎝ ⎠
⎛ x−µ
( ) ⎞
2
1
( )
p x | S = sL = exp ⎜ −
⎜ 2σ 2
L
⎟
⎟
2πσ ⎝ ⎠
µH − µL
d ' = zFA − zHIT
σ
PSYC 6256 Principles of Neural Coding J. Elder
What is the right criterion to use?
10 Probability & Bayesian Inference
Suppose the observer wants to maximize the expected number
of times they are right.
Then the optimal decision rule is to always select the state s
with higher probability for the observed internal response x:
( ) ≥ 1→ ' yes '
p x | sH
p (x | s )
L
The ‘likelihood ratio test’
p (x | s )
H
< 1→ ' no '
p (x | s )
L
This is the maximum likelihood detector.
For the equal-variance case, this means that the criterion is the
average of the two signal levels:
z
1
2
(
z = µL + µH )
PSYC 6256 Principles of Neural Coding J. Elder
Optimal Performance
11 Probability & Bayesian Inference
The performance of the maximum likelihood
observer for this yes/no task is given by
⎛ d′ ⎞
p(correct) = p(HIT) = p(CORRECT REJECT) = erfc ⎜ −
⎝ 2 2⎟
⎠
PSYC 6256 Principles of Neural Coding J. Elder
Bias
12 Probability & Bayesian Inference
For this optimal decision rule, the different types of
errors are balanced: p(FA) = p(MISS)
For observers that use a different criterion, the
different types of errors will be unbalanced.
Such observers have lower p(correct) and are said
to be biased.
z
PSYC 6256 Principles of Neural Coding J. Elder
ROC Curves
13 Probability & Bayesian Inference
Suppose the experiment is repeated many times
under different instructions.
The first time, the observer is instructed to be
extremely stringent in their criterion, only reporting
‘yes’ when they are 100% sure the light was flashed.
On subsequent repetitions, the observer is instructed
to gradually relax their criterion.
PSYC 6256 Principles of Neural Coding J. Elder
ROC Curves
14 Probability & Bayesian Inference
As the criterion threshold is swept from right to left, p(HIT)
increases, but p(FA) also increases.
The resulting plot of p(HIT) vs p(FA) is called a receiver-
operating characteristic (ROC).
Increasing d ′
d′ = 0
PSYC 6256 Principles of Neural Coding J. Elder
ROC Curves
15 Probability & Bayesian Inference
Note that d’ remains fixed as the criterion is varied!
Thus d’ is criterion-invariant, and is thus a pure
reflection of the signal-to-noise ratio.
PSYC 6256 Principles of Neural Coding J. Elder
Example 2: Motion Direction Discrimination
16 Probability & Bayesian Inference
Random dot kinematogram Britten et al (1992)
Signal dots are either all moving up or all moving down
Noise dots are moving in random directions
PSYC 6256 Principles of Neural Coding J. Elder
100% Coherence
17 Probability & Bayesian Inference
PSYC 6256 Principles of Neural Coding J. Elder
30% Coherence
18 Probability & Bayesian Inference
PSYC 6256 Principles of Neural Coding J. Elder
5% Coherence
19 Probability & Bayesian Inference
PSYC 6256 Principles of Neural Coding J. Elder
0% Coherence
20 Probability & Bayesian Inference
PSYC 6256 Principles of Neural Coding J. Elder
The Medial Temporal Area (V5)
21 Probability & Bayesian Inference
PSYC 6256 Principles of Neural Coding www.thebrain.mcgill.ca J. Elder
Experimental Details
22 Probability & Bayesian Inference
Signal direction always in preferred or anti-
preferred direction for cell.
What kind of task is this?
Note that now there is external noise as well as
internal noise.
To calculate neural discrimination performance,
assumed neuron paired with identical neuron, tuned
to opposite direction of motion.
PSYC 6256 Principles of Neural Coding J. Elder
Anti-Preferred Preferred
Behaviour Direction Neuron Direction
Hit Rate
False Alarm Rate
Priors
25 Probability & Bayesian Inference
Note that if the probabilities of the two signal
states are not equal, the maximum likelihood
observer will be suboptimal.
In this case we must make use of the posterior ratio.
( ) ≥ 1→ ' yes '
p sH | x
p (s | x )
L
Maximum a posteriori (MAP) rule
p (s | x )
H
< 1→ ' no '
p (s | x )
L
PSYC 6256 Principles of Neural Coding J. Elder
MAP Inference
26 Probability & Bayesian Inference
Using Bayes’ rule, we obtain:
( ) = p ( x | s ) p (s )
p sH | x H H
p (s | x ) p ( x | s ) p (s )
L L L
Thus we simply scale the likelihoods by the priors.
PSYC 6256 Principles of Neural Coding J. Elder
Loss and Risk
27 Probability & Bayesian Inference
Maximizing p(correct) is not always the best thing to
do.
How would you adjust your criterion if you were
A venture capitalist trying to detect the next Google?
A pilot looking for obstacles on a runway?
PSYC 6256 Principles of Neural Coding J. Elder
Loss Function
28 Probability & Bayesian Inference
In general, different types of correct decision or action will
yield different payoffs, and different types of errors will yield
different costs.
These differences can be accounted for through a loss function:
Let a(x) represent the action of the observer, given internal response x.
( )
Then L s,a(x) represents the cost of taking action a, given world state s.
PSYC 6256 Principles of Neural Coding J. Elder
The Ideal Observer
29 Probability & Bayesian Inference
The Ideal Observer uses the decision rule that
minimizes the Expected Loss, aka the Risk R(a|x):
( ) (
R(a | x) = ∑ L s,a(x) p(s, x) = ∑ L s,a(x) p(x | s)p(s) )
s s
PSYC 6256 Principles of Neural Coding J. Elder
Example 3: Slant Estimation
30 Probability & Bayesian Inference
PSYC 6256 Principles of Neural Coding J. Elder
