Docstoc

rhuang Miccl ai ai

Document Sample
rhuang Miccl ai ai Powered By Docstoc
					• Bayes nets
• Computing conditional probability
• Polytrees
• Probability Inferences


                                      1
                    Formulas to remember
• Conditional probability

               P(B|A) =     P(A, B)
                             P(A
                             )
• Production rule
               P(A , B)=P(A|B)P(B)
• Bayes rule

                P(B|A) =    P(A|B)P(B)
                               P(A
                               )
                                    P(A|B)P(B)
                 P(B|A) =
                            P(A|B)P(B) + P(A| B)P(B)




                                                         2
                        Bayes Nets

• It is also called “Causal nets”, “belief networks”, and
  “influence diagrams”.
• Bayes nets provide a general technique for computing
  probabilities of causally related random variables given
  evidence for some of them.
• For example,               True/False

Causal link               Cold


        Sore-throat                       Runny-nose
 True/False                                        True/False


• ? Joint distribution: P(Cold, Sore-throat, Runny-nose)        3
               Some “query”examples?

• How likely is it that Cold, Sore-throat and Runny-nose are
 all true?
  compute P(Cold, Sore-throat, Runny-nose)
• How likely is it that I have a sore throat given that I have a
 cold?
  compute P(Sore-throat|Cold)
• How likely is it that I have a cold given that I have a sore
 throat?
  compute P(Cold| Sore-throat)
• How likely is it that I have a cold given that I have a sore
 throat and a runny nose?
  compute P(Cold| Sore-throat, Runny-nose)                        4
                For nets with a unique root
? Joint distribution: P(Cold, Sore-throat, Runny-nose)
  The joint probability distribution of all the variables in the
  net equals the probability of the root times the probability of
   each non-root node given its parents.

                             Cold


             Sore-throat                 Runny-nose
  P(Cold, Sore-throat, Runny-nose) =
       P(Cold)P(Sore-throat|Cold)P(Runny-nose|Cold)
? Prove it
                                                                    5
                               Proof
For the “Cold” example, from the bayes nets we can assume
that Sore-throat and Runny-nose are irrelevant, thus we can
apply conditional independence.
 P(Sore-throat | Cold, Runny-nose) = P(Sore-throat | Cold)
 P(Runny-nose | Cold, Sore-throat) = P(Cold | Sore-throat)
compute
 P(Cold, Sore-throat, Runny-nose)
   = P(Runny-nose | Sore-throat, Cold) P(Sore-throat | Cold)P(Cold)
   = P(Runny-nose | Cold) P(Sore-throat | Cold)P(Cold)




                                                                      6
                        Further observations
• If there is no path that connects 2 nodes by a sequence of
 causal links, the nodes are conditionally independent with
 respect to root. For example, Sore-throat, Runny-nose
• Since Bayes nets assumption is equivalent to conditional
 independence assumptions, posterior probabilities in a
 Bayes net can be computed using standard formulas from
 probability theory

                                          P(Sore-throat | Cold) P(Cold)
  P(Cold | Sore-throat) =
                            P(Sore-throat | Cold) P(Cold) + P(Sore-throat | Cold) P(Cold)




                                                                                       7
                         An example

                          P(S) = 0.3
0.3   Habitual smoking    P(L|S) = 0.5, P(L|S) = 0.05
                          P(C|L) = 0.7, P(C| S) = 0.06
       0.3, 0.05
                          Joint probability distribution:
                           P(S, L, C) = P(S) P(L|S)P(C|L)
        Lung cancer                    = 0.3*0.5 *0.7 = 0.105


        0.7, 0.06
                          ? P(L|C)
       Chronic cough




                                                                8
                                 Compute P(L|C)
                                        P(S) = 0.3
0.3   Habitual smoking                  P(L|S) = 0.5, P(L|S) = 0.05
                                        P(C|L) = 0.7, P(C| L) = 0.06
         0.3, 0.05
                                        Joint probability distribution:
                                          P(S, L, C) = P(S) P(L|S)P(C|L)
          Lung cancer                                  = 0.3*0.5 *0.7 = 0.105

                                        P(L|C) = (P(C|L)P(L)) / (P(C))
          0.7, 0.06
                                        P(C) = P(C/L)P(L) + P(C/L)P(L)
                                        P(L) = P(L/S)P(S) + P(L/ S)P(S) = 0.5*0.3 + 0.05*(1-0.3) = 0.185
         Chronic cough                  P(L) = (1-0.185) = 0.815
                                        P(C) = 0.7*0.185 + 0.06*0.815 = 0.1784
                                        P(L|C) = 0.7*0.185 / 0.1784 = 0.7258968

General way of computing any conditional probability:
1.    Express the conditional probabilities for all the nodes
2.    Use the Bayes net assumption to evaluate the joint probabilities.                     9
3.    P(A) + P(A) = 1
                            Better methods

• A general method is not efficient
• Better methods depend on systematic use of the independence
 assumptions implicit in the Bayes net assumption:
 A set of nodes X is independent of a set of nodes Y given nodes E
                      iff
  every undirected path connecting a node in Y is directly or indirectly blocked by E
                                              direct blockage

                                  E                                      Y

   X



                                              indirect blockage                  10
                                              (no descendants in
        Examples (1)

                       direct
              E        blockage


X                                 Y




                       direct
P(X|Y, E) = P(X|E)     blockage



                                      11
        Examples (2)

                       unblocked path
                       through here


X                               Y




      E
                       unblocked path
P(X|Y, E)  P(X|E)
                       through here


                                        12
                  Inference in Polytrees
• Singly connected networks are called Polytrees.
• Algorithm that works on Polytrees are derived in the following
 three steps.
                                          Express P(X|E) in terms of
                                              P(E-x|X) and P(X|E+x)
                                          Where,
                                          P(E-x|X) is likelihood of “evidential support” given X
                                          P(X|E+x) is likelihood of X given its “causal support”
                                          E-x is E-nodes connected to X via X’s children
                                          E+x is E-nodes connected to X via X’s parents




                                                                           13
     • Express P(X|E+x) recursively in terms of
             P(Ui| E+x)
      where Ui are the parents of X.




       • Express P(E-x|X) recursively in terms of
              P(E-yi|Yi) and P(Zij|E-Zij Yi)
         where Zij are the parents of Yi,
                Yi are X’s children,
                E-Zij Yi are the E-nodes connected
                to Zij except via Yi.



                                  14
Yi
               The nature of probability inferences

    • (a) Diagnose inferences (from effects to causes)
      e.g. Given that JohnCalls, infer that P(Burglary|JohnCalls) = 0.016

    • (b) Causal inferences (from causes to effects)
      e.g. Given Burglary, P(JohnCalls|Burglary)=0.86 and P(MaryCalls|Burglary) = 0.67

    • (c) Inter-causal inferences (between causes of a common effect)
     e.g. Given Alarm, we have P(Burglary|Alarm) = 0.376. But if we add the evidence
          that Earthquake is true, then P(Burglary|Alarm  Earthquake) goes down to 0.003.
          Even though burglaries and earthquakes are independent, the presence of one
          makes the other less likely.

    • (d) Mixed inferences (combining two or more of the above)
     Q                E                                              E
                                         Q         E
a              b                   c                          d      Q
                                                                                   15
      E               Q                                              E
                Applications of Bayes nets
• Calculating the belief in query variables given define values for
 evidence variables,
• Making decisions based on probabilities in the network and on
 agent’s utilities,
• Deciding which additional evidence variables should be
 observed in order to gain useful information,
• Performing sensitive analysis to understand which aspects of the
 model have greatest impact on the probabilities of the query
 variables,
• Explaining the results of probabilistic inference to the user.


                                                                      16
Exercises

Ex1.
Could you calculate P(S|L) in page 8.

Ex2.
Could you write down the formulas to compute
  P(Cold, Sore-throat, Runny-nose), P(Sore-throat|Cold)
  P(Cold| Sore-throat), P(Cold| Sore-throat, Runny-nose)




                           Cold


         Sore-throat                  Runny-nose
                                                           17

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:6
posted:7/28/2011
language:
pages:17