# Reasoning Under Uncertainty Marginal and Conditional Independence by nyut545e2

VIEWS: 14 PAGES: 16

• pg 1
```									   Recap                              Marginal Independence          Conditional Independence

Reasoning Under Uncertainty: Marginal and
Conditional Independence

CPSC 322 Lecture 25

March 21, 2007
Textbook §9.2 – §9.3

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 1
Recap                              Marginal Independence          Conditional Independence

Lecture Overview

1    Recap

2    Marginal Independence

3    Conditional Independence

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 2
Recap                              Marginal Independence          Conditional Independence

Conditioning

Probabilistic conditioning speciﬁes how to revise beliefs based
on new information.
You build a probabilistic model taking all background
information into account. This gives the prior probability.
All other information must be conditioned on.
If evidence e is all of the information obtained subsequently,
the conditional probability P (h|e) of h given e is the posterior
probability of h.

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 3
Recap                              Marginal Independence                          Conditional Independence

Conditional Probability

The conditional probability of formula h given evidence e is

P (h ∧ e)
P (h|e) =
P (e)

Chain rule:
n
P (f1 ∧ f2 ∧ . . . ∧ fn ) =                  P (fi |f1 ∧ · · · ∧ fi−1 )
i=1

Bayes’ theorem:

P (e|h) × P (h)
P (h|e) =                       .
P (e)

Reasoning Under Uncertainty: Marginal and Conditional Independence                   CPSC 322 Lecture 25, Slide 4
Recap                              Marginal Independence          Conditional Independence

Lecture Overview

1    Recap

2    Marginal Independence

3    Conditional Independence

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 5
Recap                              Marginal Independence          Conditional Independence

Marginal independence

Deﬁnition (marginal independence)
Random variable X is marginally independent of random variable
Y if, for all xi ∈ dom(X), yj ∈ dom(Y ) and yk ∈ dom(Y ),

P (X = xi |Y = yj )
= P (X = xi |Y = yk )
= P (X = xi ).

That is, knowledge of Y ’s value doesn’t aﬀect your belief in the
value of X.

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 6
Recap                              Marginal Independence          Conditional Independence

Examples of marginal independence

The probability that the Canucks will win the Stanley Cup is
independent of whether light l1 is lit.
remember the diagnostic assistant domain—the picture will
recur in a minute!
Whether there is someone in a room is independent of
whether a light l2 is lit.
Whether light l1 is lit is not independent of the position of
switch s2.

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 7
Recap                              Marginal Independence          Conditional Independence

Lecture Overview

1    Recap

2    Marginal Independence

3    Conditional Independence

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 8
Recap                              Marginal Independence          Conditional Independence

Conditional Independence
Sometimes, two random variables might not be marginally
independent. However, they can become independent after we
observe some third variable.

Deﬁnition
Random variable X is conditionally independent of random
variable Y given random variable Z if, for all xi ∈ dom(X),
yj ∈ dom(Y ), yk ∈ dom(Y ) and zm ∈ dom(Z),

P (X = xi |Y = yj ∧ Z = zm )
= P (X = xi |Y = yk ∧ Z = zm )
= P (X = xi |Z = zm ).

That is, knowledge of Y ’s value doesn’t aﬀect your belief in
the value of X, given a value of Z.
Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 9
Recap                              Marginal Independence           Conditional Independence

Conditional Independence Example

Kevin separately phones two students, Alice and Bob.
To each, he tells the same number, nk ∈ {1, . . . , 10}.
Due to the noise in the phone, Alice and Bob each imperfectly
(and independently) draw a conclusion about what number
Kevin said.
Let the numbers Alice and Bob think they heard be na and nb
respectively.
Are na and nb marginally independent?

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 10
Recap                              Marginal Independence           Conditional Independence

Conditional Independence Example

Kevin separately phones two students, Alice and Bob.
To each, he tells the same number, nk ∈ {1, . . . , 10}.
Due to the noise in the phone, Alice and Bob each imperfectly
(and independently) draw a conclusion about what number
Kevin said.
Let the numbers Alice and Bob think they heard be na and nb
respectively.
Are na and nb marginally independent?
No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1).

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 10
Recap                              Marginal Independence           Conditional Independence

Conditional Independence Example

Kevin separately phones two students, Alice and Bob.
To each, he tells the same number, nk ∈ {1, . . . , 10}.
Due to the noise in the phone, Alice and Bob each imperfectly
(and independently) draw a conclusion about what number
Kevin said.
Let the numbers Alice and Bob think they heard be na and nb
respectively.
Are na and nb marginally independent?
No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1).
Why are na and nb conditionally independent given nk ?

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 10
Recap                              Marginal Independence           Conditional Independence

Conditional Independence Example

Kevin separately phones two students, Alice and Bob.
To each, he tells the same number, nk ∈ {1, . . . , 10}.
Due to the noise in the phone, Alice and Bob each imperfectly
(and independently) draw a conclusion about what number
Kevin said.
Let the numbers Alice and Bob think they heard be na and nb
respectively.
Are na and nb marginally independent?
No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1).
Why are na and nb conditionally independent given nk ?
Because if we know the number that Kevin actually said, the
two variables are no longer correlated.
e.g., P (na = 1|nb = 1, nk = 2) = P (na = 1|nk = 2)

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 10
Recap                                Marginal Independence                       Conditional Independence

Example domain (diagnostic assistant)

outside power
cb1
s1              w5
circuit
w1                                                      breaker
s2                         w3     cb2
w2                                           off
s3
w0                                                                   switch
on
w4                   w6
two-way
switch
l1
light
l2                   p2

p1                              power
outlet

Reasoning Under Uncertainty: Marginal and Conditional Independence                 CPSC 322 Lecture 25, Slide 11
Recap                              Marginal Independence           Conditional Independence

More examples of conditional independence

Whether light l1 is lit is independent of the position of light
switch s2 given whether there is power in wire w0 .
two random variables that are not marginally independent can
still be conditionally independent
Every other variable may be independent of whether light l1 is
lit given whether there is power in wire w0 and the status of
light l1 (if it’s ok, or if not, how it’s broken).

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 12
Recap                              Marginal Independence           Conditional Independence

More examples of conditional independence

The probability that the Canucks will win the Stanley Cup is
independent of whether light l1 is lit given whether there is
outside power.
sometimes, when two random variables are marginally
independent, they’re also conditionally independent given a
third variable.
But not always...
Let C1 be the proposition that coin 1 is heads; let C2 be the
proposition that coin 2 is heads; let B be the proposition that
coin 1 and coin 2 are both either heads or tails.
P (C1 |C2 ) = P (C1 ): C1 and C2 are marginally independent.
But P (C1 |C2 , B) = P (C1 |B): if I know both C2 and B, I
know C1 exactly, but if I only know B I know nothing.
Hence C1 and C2 are not conditionally independent given B.

Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 13

```
To top