VIEWS: 14 PAGES: 16 POSTED ON: 5/28/2011 Public Domain
Recap Marginal Independence Conditional Independence Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25 March 21, 2007 Textbook §9.2 – §9.3 Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 1 Recap Marginal Independence Conditional Independence Lecture Overview 1 Recap 2 Marginal Independence 3 Conditional Independence Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 2 Recap Marginal Independence Conditional Independence Conditioning Probabilistic conditioning speciﬁes how to revise beliefs based on new information. You build a probabilistic model taking all background information into account. This gives the prior probability. All other information must be conditioned on. If evidence e is all of the information obtained subsequently, the conditional probability P (h|e) of h given e is the posterior probability of h. Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 3 Recap Marginal Independence Conditional Independence Conditional Probability The conditional probability of formula h given evidence e is P (h ∧ e) P (h|e) = P (e) Chain rule: n P (f1 ∧ f2 ∧ . . . ∧ fn ) = P (fi |f1 ∧ · · · ∧ fi−1 ) i=1 Bayes’ theorem: P (e|h) × P (h) P (h|e) = . P (e) Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 4 Recap Marginal Independence Conditional Independence Lecture Overview 1 Recap 2 Marginal Independence 3 Conditional Independence Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 5 Recap Marginal Independence Conditional Independence Marginal independence Deﬁnition (marginal independence) Random variable X is marginally independent of random variable Y if, for all xi ∈ dom(X), yj ∈ dom(Y ) and yk ∈ dom(Y ), P (X = xi |Y = yj ) = P (X = xi |Y = yk ) = P (X = xi ). That is, knowledge of Y ’s value doesn’t aﬀect your belief in the value of X. Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 6 Recap Marginal Independence Conditional Independence Examples of marginal independence The probability that the Canucks will win the Stanley Cup is independent of whether light l1 is lit. remember the diagnostic assistant domain—the picture will recur in a minute! Whether there is someone in a room is independent of whether a light l2 is lit. Whether light l1 is lit is not independent of the position of switch s2. Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 7 Recap Marginal Independence Conditional Independence Lecture Overview 1 Recap 2 Marginal Independence 3 Conditional Independence Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 8 Recap Marginal Independence Conditional Independence Conditional Independence Sometimes, two random variables might not be marginally independent. However, they can become independent after we observe some third variable. Deﬁnition Random variable X is conditionally independent of random variable Y given random variable Z if, for all xi ∈ dom(X), yj ∈ dom(Y ), yk ∈ dom(Y ) and zm ∈ dom(Z), P (X = xi |Y = yj ∧ Z = zm ) = P (X = xi |Y = yk ∧ Z = zm ) = P (X = xi |Z = zm ). That is, knowledge of Y ’s value doesn’t aﬀect your belief in the value of X, given a value of Z. Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 9 Recap Marginal Independence Conditional Independence Conditional Independence Example Kevin separately phones two students, Alice and Bob. To each, he tells the same number, nk ∈ {1, . . . , 10}. Due to the noise in the phone, Alice and Bob each imperfectly (and independently) draw a conclusion about what number Kevin said. Let the numbers Alice and Bob think they heard be na and nb respectively. Are na and nb marginally independent? Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 10 Recap Marginal Independence Conditional Independence Conditional Independence Example Kevin separately phones two students, Alice and Bob. To each, he tells the same number, nk ∈ {1, . . . , 10}. Due to the noise in the phone, Alice and Bob each imperfectly (and independently) draw a conclusion about what number Kevin said. Let the numbers Alice and Bob think they heard be na and nb respectively. Are na and nb marginally independent? No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1). Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 10 Recap Marginal Independence Conditional Independence Conditional Independence Example Kevin separately phones two students, Alice and Bob. To each, he tells the same number, nk ∈ {1, . . . , 10}. Due to the noise in the phone, Alice and Bob each imperfectly (and independently) draw a conclusion about what number Kevin said. Let the numbers Alice and Bob think they heard be na and nb respectively. Are na and nb marginally independent? No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1). Why are na and nb conditionally independent given nk ? Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 10 Recap Marginal Independence Conditional Independence Conditional Independence Example Kevin separately phones two students, Alice and Bob. To each, he tells the same number, nk ∈ {1, . . . , 10}. Due to the noise in the phone, Alice and Bob each imperfectly (and independently) draw a conclusion about what number Kevin said. Let the numbers Alice and Bob think they heard be na and nb respectively. Are na and nb marginally independent? No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1). Why are na and nb conditionally independent given nk ? Because if we know the number that Kevin actually said, the two variables are no longer correlated. e.g., P (na = 1|nb = 1, nk = 2) = P (na = 1|nk = 2) Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 10 Recap Marginal Independence Conditional Independence Example domain (diagnostic assistant) outside power cb1 s1 w5 circuit w1 breaker s2 w3 cb2 w2 off s3 w0 switch on w4 w6 two-way switch l1 light l2 p2 p1 power outlet Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 11 Recap Marginal Independence Conditional Independence More examples of conditional independence Whether light l1 is lit is independent of the position of light switch s2 given whether there is power in wire w0 . two random variables that are not marginally independent can still be conditionally independent Every other variable may be independent of whether light l1 is lit given whether there is power in wire w0 and the status of light l1 (if it’s ok, or if not, how it’s broken). Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 12 Recap Marginal Independence Conditional Independence More examples of conditional independence The probability that the Canucks will win the Stanley Cup is independent of whether light l1 is lit given whether there is outside power. sometimes, when two random variables are marginally independent, they’re also conditionally independent given a third variable. But not always... Let C1 be the proposition that coin 1 is heads; let C2 be the proposition that coin 2 is heads; let B be the proposition that coin 1 and coin 2 are both either heads or tails. P (C1 |C2 ) = P (C1 ): C1 and C2 are marginally independent. But P (C1 |C2 , B) = P (C1 |B): if I know both C2 and B, I know C1 exactly, but if I only know B I know nothing. Hence C1 and C2 are not conditionally independent given B. Reasoning Under Uncertainty: Marginal and Conditional Independence CPSC 322 Lecture 25, Slide 13