# rhuang Miccl ai ai

Document Sample

```					• Bayes nets
• Computing conditional probability
• Polytrees
• Probability Inferences

1
Formulas to remember
• Conditional probability

P(B|A) =     P(A, B)
P(A
)
• Production rule
P(A , B)=P(A|B)P(B)
• Bayes rule

P(B|A) =    P(A|B)P(B)
P(A
)
P(A|B)P(B)
P(B|A) =
P(A|B)P(B) + P(A| B)P(B)

2
Bayes Nets

• It is also called “Causal nets”, “belief networks”, and
“influence diagrams”.
• Bayes nets provide a general technique for computing
probabilities of causally related random variables given
evidence for some of them.
• For example,               True/False

Sore-throat                       Runny-nose
True/False                                        True/False

• ? Joint distribution: P(Cold, Sore-throat, Runny-nose)        3
Some “query”examples?

• How likely is it that Cold, Sore-throat and Runny-nose are
all true?
 compute P(Cold, Sore-throat, Runny-nose)
• How likely is it that I have a sore throat given that I have a
cold?
 compute P(Sore-throat|Cold)
• How likely is it that I have a cold given that I have a sore
throat?
 compute P(Cold| Sore-throat)
• How likely is it that I have a cold given that I have a sore
throat and a runny nose?
 compute P(Cold| Sore-throat, Runny-nose)                        4
For nets with a unique root
? Joint distribution: P(Cold, Sore-throat, Runny-nose)
The joint probability distribution of all the variables in the
net equals the probability of the root times the probability of
each non-root node given its parents.

Cold

Sore-throat                 Runny-nose
P(Cold, Sore-throat, Runny-nose) =
P(Cold)P(Sore-throat|Cold)P(Runny-nose|Cold)
? Prove it
5
Proof
For the “Cold” example, from the bayes nets we can assume
that Sore-throat and Runny-nose are irrelevant, thus we can
apply conditional independence.
P(Sore-throat | Cold, Runny-nose) = P(Sore-throat | Cold)
P(Runny-nose | Cold, Sore-throat) = P(Cold | Sore-throat)
compute
P(Cold, Sore-throat, Runny-nose)
= P(Runny-nose | Sore-throat, Cold) P(Sore-throat | Cold)P(Cold)
= P(Runny-nose | Cold) P(Sore-throat | Cold)P(Cold)

6
Further observations
• If there is no path that connects 2 nodes by a sequence of
causal links, the nodes are conditionally independent with
respect to root. For example, Sore-throat, Runny-nose
• Since Bayes nets assumption is equivalent to conditional
independence assumptions, posterior probabilities in a
Bayes net can be computed using standard formulas from
probability theory

P(Sore-throat | Cold) P(Cold)
P(Cold | Sore-throat) =
P(Sore-throat | Cold) P(Cold) + P(Sore-throat | Cold) P(Cold)

7
An example

P(S) = 0.3
0.3   Habitual smoking    P(L|S) = 0.5, P(L|S) = 0.05
P(C|L) = 0.7, P(C| S) = 0.06
0.3, 0.05
Joint probability distribution:
P(S, L, C) = P(S) P(L|S)P(C|L)
Lung cancer                    = 0.3*0.5 *0.7 = 0.105

0.7, 0.06
? P(L|C)
Chronic cough

8
Compute P(L|C)
P(S) = 0.3
0.3   Habitual smoking                  P(L|S) = 0.5, P(L|S) = 0.05
P(C|L) = 0.7, P(C| L) = 0.06
0.3, 0.05
Joint probability distribution:
P(S, L, C) = P(S) P(L|S)P(C|L)
Lung cancer                                  = 0.3*0.5 *0.7 = 0.105

P(L|C) = (P(C|L)P(L)) / (P(C))
0.7, 0.06
P(C) = P(C/L)P(L) + P(C/L)P(L)
P(L) = P(L/S)P(S) + P(L/ S)P(S) = 0.5*0.3 + 0.05*(1-0.3) = 0.185
Chronic cough                  P(L) = (1-0.185) = 0.815
P(C) = 0.7*0.185 + 0.06*0.815 = 0.1784
P(L|C) = 0.7*0.185 / 0.1784 = 0.7258968

General way of computing any conditional probability:
1.    Express the conditional probabilities for all the nodes
2.    Use the Bayes net assumption to evaluate the joint probabilities.                     9
3.    P(A) + P(A) = 1
Better methods

• A general method is not efficient
• Better methods depend on systematic use of the independence
assumptions implicit in the Bayes net assumption:
A set of nodes X is independent of a set of nodes Y given nodes E
iff
every undirected path connecting a node in Y is directly or indirectly blocked by E
direct blockage

E                                      Y

X

indirect blockage                  10
(no descendants in
Examples (1)

direct
E        blockage

X                                 Y

direct
P(X|Y, E) = P(X|E)     blockage

11
Examples (2)

unblocked path
through here

X                               Y

E
unblocked path
P(X|Y, E)  P(X|E)
through here

12
Inference in Polytrees
• Singly connected networks are called Polytrees.
• Algorithm that works on Polytrees are derived in the following
three steps.
Express P(X|E) in terms of
P(E-x|X) and P(X|E+x)
Where,
P(E-x|X) is likelihood of “evidential support” given X
P(X|E+x) is likelihood of X given its “causal support”
E-x is E-nodes connected to X via X’s children
E+x is E-nodes connected to X via X’s parents

13
• Express P(X|E+x) recursively in terms of
P(Ui| E+x)
where Ui are the parents of X.

• Express P(E-x|X) recursively in terms of
P(E-yi|Yi) and P(Zij|E-Zij Yi)
where Zij are the parents of Yi,
Yi are X’s children,
E-Zij Yi are the E-nodes connected
to Zij except via Yi.

14
Yi
The nature of probability inferences

• (a) Diagnose inferences (from effects to causes)
e.g. Given that JohnCalls, infer that P(Burglary|JohnCalls) = 0.016

• (b) Causal inferences (from causes to effects)
e.g. Given Burglary, P(JohnCalls|Burglary)=0.86 and P(MaryCalls|Burglary) = 0.67

• (c) Inter-causal inferences (between causes of a common effect)
e.g. Given Alarm, we have P(Burglary|Alarm) = 0.376. But if we add the evidence
that Earthquake is true, then P(Burglary|Alarm  Earthquake) goes down to 0.003.
Even though burglaries and earthquakes are independent, the presence of one
makes the other less likely.

• (d) Mixed inferences (combining two or more of the above)
Q                E                                              E
Q         E
a              b                   c                          d      Q
15
E               Q                                              E
Applications of Bayes nets
• Calculating the belief in query variables given define values for
evidence variables,
• Making decisions based on probabilities in the network and on
agent’s utilities,
• Deciding which additional evidence variables should be
observed in order to gain useful information,
• Performing sensitive analysis to understand which aspects of the
model have greatest impact on the probabilities of the query
variables,
• Explaining the results of probabilistic inference to the user.

16
Exercises

Ex1.
Could you calculate P(S|L) in page 8.

Ex2.
Could you write down the formulas to compute
P(Cold, Sore-throat, Runny-nose), P(Sore-throat|Cold)
P(Cold| Sore-throat), P(Cold| Sore-throat, Runny-nose)

Cold

Sore-throat                  Runny-nose
17

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 6 posted: 7/28/2011 language: pages: 17