Reasoning Under Uncertainty Marginal and Conditional Independence

Document Sample
Reasoning Under Uncertainty Marginal and Conditional Independence Powered By Docstoc
					   Recap                              Marginal Independence          Conditional Independence




              Reasoning Under Uncertainty: Marginal and
                      Conditional Independence

                                          CPSC 322 Lecture 25


                                            March 21, 2007
                                          Textbook §9.2 – §9.3




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 1
   Recap                              Marginal Independence          Conditional Independence


Lecture Overview




      1    Recap


      2    Marginal Independence


      3    Conditional Independence




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 2
   Recap                              Marginal Independence          Conditional Independence


Conditioning



              Probabilistic conditioning specifies how to revise beliefs based
              on new information.
              You build a probabilistic model taking all background
              information into account. This gives the prior probability.
              All other information must be conditioned on.
              If evidence e is all of the information obtained subsequently,
              the conditional probability P (h|e) of h given e is the posterior
              probability of h.




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 3
   Recap                              Marginal Independence                          Conditional Independence


Conditional Probability

      The conditional probability of formula h given evidence e is

                                                           P (h ∧ e)
                                           P (h|e) =
                                                             P (e)

      Chain rule:
                                                            n
                    P (f1 ∧ f2 ∧ . . . ∧ fn ) =                  P (fi |f1 ∧ · · · ∧ fi−1 )
                                                           i=1

      Bayes’ theorem:

                                                      P (e|h) × P (h)
                                      P (h|e) =                       .
                                                           P (e)


Reasoning Under Uncertainty: Marginal and Conditional Independence                   CPSC 322 Lecture 25, Slide 4
   Recap                              Marginal Independence          Conditional Independence


Lecture Overview




      1    Recap


      2    Marginal Independence


      3    Conditional Independence




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 5
   Recap                              Marginal Independence          Conditional Independence


Marginal independence



      Definition (marginal independence)
      Random variable X is marginally independent of random variable
      Y if, for all xi ∈ dom(X), yj ∈ dom(Y ) and yk ∈ dom(Y ),

                                       P (X = xi |Y = yj )
                                          = P (X = xi |Y = yk )
                                          = P (X = xi ).

      That is, knowledge of Y ’s value doesn’t affect your belief in the
      value of X.



Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 6
   Recap                              Marginal Independence          Conditional Independence


Examples of marginal independence



              The probability that the Canucks will win the Stanley Cup is
              independent of whether light l1 is lit.
                     remember the diagnostic assistant domain—the picture will
                     recur in a minute!
              Whether there is someone in a room is independent of
              whether a light l2 is lit.
              Whether light l1 is lit is not independent of the position of
              switch s2.




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 7
   Recap                              Marginal Independence          Conditional Independence


Lecture Overview




      1    Recap


      2    Marginal Independence


      3    Conditional Independence




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 8
   Recap                              Marginal Independence          Conditional Independence


Conditional Independence
              Sometimes, two random variables might not be marginally
              independent. However, they can become independent after we
              observe some third variable.

      Definition
      Random variable X is conditionally independent of random
      variable Y given random variable Z if, for all xi ∈ dom(X),
      yj ∈ dom(Y ), yk ∈ dom(Y ) and zm ∈ dom(Z),

                              P (X = xi |Y = yj ∧ Z = zm )
                                  = P (X = xi |Y = yk ∧ Z = zm )
                                  = P (X = xi |Z = zm ).

              That is, knowledge of Y ’s value doesn’t affect your belief in
              the value of X, given a value of Z.
Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 9
   Recap                              Marginal Independence           Conditional Independence


Conditional Independence Example

              Kevin separately phones two students, Alice and Bob.
              To each, he tells the same number, nk ∈ {1, . . . , 10}.
              Due to the noise in the phone, Alice and Bob each imperfectly
              (and independently) draw a conclusion about what number
              Kevin said.
              Let the numbers Alice and Bob think they heard be na and nb
              respectively.
              Are na and nb marginally independent?




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 10
   Recap                              Marginal Independence           Conditional Independence


Conditional Independence Example

              Kevin separately phones two students, Alice and Bob.
              To each, he tells the same number, nk ∈ {1, . . . , 10}.
              Due to the noise in the phone, Alice and Bob each imperfectly
              (and independently) draw a conclusion about what number
              Kevin said.
              Let the numbers Alice and Bob think they heard be na and nb
              respectively.
              Are na and nb marginally independent?
                     No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1).




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 10
   Recap                              Marginal Independence           Conditional Independence


Conditional Independence Example

              Kevin separately phones two students, Alice and Bob.
              To each, he tells the same number, nk ∈ {1, . . . , 10}.
              Due to the noise in the phone, Alice and Bob each imperfectly
              (and independently) draw a conclusion about what number
              Kevin said.
              Let the numbers Alice and Bob think they heard be na and nb
              respectively.
              Are na and nb marginally independent?
                     No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1).
              Why are na and nb conditionally independent given nk ?




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 10
   Recap                              Marginal Independence           Conditional Independence


Conditional Independence Example

              Kevin separately phones two students, Alice and Bob.
              To each, he tells the same number, nk ∈ {1, . . . , 10}.
              Due to the noise in the phone, Alice and Bob each imperfectly
              (and independently) draw a conclusion about what number
              Kevin said.
              Let the numbers Alice and Bob think they heard be na and nb
              respectively.
              Are na and nb marginally independent?
                     No: we’d expect (e.g.) P (na = 1|nb = 1) > P (na = 1).
              Why are na and nb conditionally independent given nk ?
                     Because if we know the number that Kevin actually said, the
                     two variables are no longer correlated.
                     e.g., P (na = 1|nb = 1, nk = 2) = P (na = 1|nk = 2)


Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 10
   Recap                                Marginal Independence                       Conditional Independence


Example domain (diagnostic assistant)

                                                                     outside power
                                             cb1
                                        s1              w5
                                                                                   circuit
                           w1                                                      breaker
                   s2                         w3     cb2
                                w2                                           off
                                        s3
              w0                                                                   switch
                                                                             on
                              w4                   w6
                                                                                   two-way
                                                                                   switch
                  l1
                                                                                   light
                                   l2                   p2

                                                   p1                              power
                                                                                   outlet



Reasoning Under Uncertainty: Marginal and Conditional Independence                 CPSC 322 Lecture 25, Slide 11
   Recap                              Marginal Independence           Conditional Independence


More examples of conditional independence




              Whether light l1 is lit is independent of the position of light
              switch s2 given whether there is power in wire w0 .
                     two random variables that are not marginally independent can
                     still be conditionally independent
              Every other variable may be independent of whether light l1 is
              lit given whether there is power in wire w0 and the status of
              light l1 (if it’s ok, or if not, how it’s broken).




Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 12
   Recap                              Marginal Independence           Conditional Independence


More examples of conditional independence

              The probability that the Canucks will win the Stanley Cup is
              independent of whether light l1 is lit given whether there is
              outside power.
                     sometimes, when two random variables are marginally
                     independent, they’re also conditionally independent given a
                     third variable.
              But not always...
                     Let C1 be the proposition that coin 1 is heads; let C2 be the
                     proposition that coin 2 is heads; let B be the proposition that
                     coin 1 and coin 2 are both either heads or tails.
                     P (C1 |C2 ) = P (C1 ): C1 and C2 are marginally independent.
                     But P (C1 |C2 , B) = P (C1 |B): if I know both C2 and B, I
                     know C1 exactly, but if I only know B I know nothing.
                     Hence C1 and C2 are not conditionally independent given B.


Reasoning Under Uncertainty: Marginal and Conditional Independence   CPSC 322 Lecture 25, Slide 13

				
DOCUMENT INFO