Document Sample

• Bayes nets • Computing conditional probability • Polytrees • Probability Inferences 1 Formulas to remember • Conditional probability P(B|A) = P(A, B) P(A ) • Production rule P(A , B)=P(A|B)P(B) • Bayes rule P(B|A) = P(A|B)P(B) P(A ) P(A|B)P(B) P(B|A) = P(A|B)P(B) + P(A| B)P(B) 2 Bayes Nets • It is also called “Causal nets”, “belief networks”, and “influence diagrams”. • Bayes nets provide a general technique for computing probabilities of causally related random variables given evidence for some of them. • For example, True/False Causal link Cold Sore-throat Runny-nose True/False True/False • ? Joint distribution: P(Cold, Sore-throat, Runny-nose) 3 Some “query”examples? • How likely is it that Cold, Sore-throat and Runny-nose are all true? compute P(Cold, Sore-throat, Runny-nose) • How likely is it that I have a sore throat given that I have a cold? compute P(Sore-throat|Cold) • How likely is it that I have a cold given that I have a sore throat? compute P(Cold| Sore-throat) • How likely is it that I have a cold given that I have a sore throat and a runny nose? compute P(Cold| Sore-throat, Runny-nose) 4 For nets with a unique root ? Joint distribution: P(Cold, Sore-throat, Runny-nose) The joint probability distribution of all the variables in the net equals the probability of the root times the probability of each non-root node given its parents. Cold Sore-throat Runny-nose P(Cold, Sore-throat, Runny-nose) = P(Cold)P(Sore-throat|Cold)P(Runny-nose|Cold) ? Prove it 5 Proof For the “Cold” example, from the bayes nets we can assume that Sore-throat and Runny-nose are irrelevant, thus we can apply conditional independence. P(Sore-throat | Cold, Runny-nose) = P(Sore-throat | Cold) P(Runny-nose | Cold, Sore-throat) = P(Cold | Sore-throat) compute P(Cold, Sore-throat, Runny-nose) = P(Runny-nose | Sore-throat, Cold) P(Sore-throat | Cold)P(Cold) = P(Runny-nose | Cold) P(Sore-throat | Cold)P(Cold) 6 Further observations • If there is no path that connects 2 nodes by a sequence of causal links, the nodes are conditionally independent with respect to root. For example, Sore-throat, Runny-nose • Since Bayes nets assumption is equivalent to conditional independence assumptions, posterior probabilities in a Bayes net can be computed using standard formulas from probability theory P(Sore-throat | Cold) P(Cold) P(Cold | Sore-throat) = P(Sore-throat | Cold) P(Cold) + P(Sore-throat | Cold) P(Cold) 7 An example P(S) = 0.3 0.3 Habitual smoking P(L|S) = 0.5, P(L|S) = 0.05 P(C|L) = 0.7, P(C| S) = 0.06 0.3, 0.05 Joint probability distribution: P(S, L, C) = P(S) P(L|S)P(C|L) Lung cancer = 0.3*0.5 *0.7 = 0.105 0.7, 0.06 ? P(L|C) Chronic cough 8 Compute P(L|C) P(S) = 0.3 0.3 Habitual smoking P(L|S) = 0.5, P(L|S) = 0.05 P(C|L) = 0.7, P(C| L) = 0.06 0.3, 0.05 Joint probability distribution: P(S, L, C) = P(S) P(L|S)P(C|L) Lung cancer = 0.3*0.5 *0.7 = 0.105 P(L|C) = (P(C|L)P(L)) / (P(C)) 0.7, 0.06 P(C) = P(C/L)P(L) + P(C/L)P(L) P(L) = P(L/S)P(S) + P(L/ S)P(S) = 0.5*0.3 + 0.05*(1-0.3) = 0.185 Chronic cough P(L) = (1-0.185) = 0.815 P(C) = 0.7*0.185 + 0.06*0.815 = 0.1784 P(L|C) = 0.7*0.185 / 0.1784 = 0.7258968 General way of computing any conditional probability: 1. Express the conditional probabilities for all the nodes 2. Use the Bayes net assumption to evaluate the joint probabilities. 9 3. P(A) + P(A) = 1 Better methods • A general method is not efficient • Better methods depend on systematic use of the independence assumptions implicit in the Bayes net assumption: A set of nodes X is independent of a set of nodes Y given nodes E iff every undirected path connecting a node in Y is directly or indirectly blocked by E direct blockage E Y X indirect blockage 10 (no descendants in Examples (1) direct E blockage X Y direct P(X|Y, E) = P(X|E) blockage 11 Examples (2) unblocked path through here X Y E unblocked path P(X|Y, E) P(X|E) through here 12 Inference in Polytrees • Singly connected networks are called Polytrees. • Algorithm that works on Polytrees are derived in the following three steps. Express P(X|E) in terms of P(E-x|X) and P(X|E+x) Where, P(E-x|X) is likelihood of “evidential support” given X P(X|E+x) is likelihood of X given its “causal support” E-x is E-nodes connected to X via X’s children E+x is E-nodes connected to X via X’s parents 13 • Express P(X|E+x) recursively in terms of P(Ui| E+x) where Ui are the parents of X. • Express P(E-x|X) recursively in terms of P(E-yi|Yi) and P(Zij|E-Zij Yi) where Zij are the parents of Yi, Yi are X’s children, E-Zij Yi are the E-nodes connected to Zij except via Yi. 14 Yi The nature of probability inferences • (a) Diagnose inferences (from effects to causes) e.g. Given that JohnCalls, infer that P(Burglary|JohnCalls) = 0.016 • (b) Causal inferences (from causes to effects) e.g. Given Burglary, P(JohnCalls|Burglary)=0.86 and P(MaryCalls|Burglary) = 0.67 • (c) Inter-causal inferences (between causes of a common effect) e.g. Given Alarm, we have P(Burglary|Alarm) = 0.376. But if we add the evidence that Earthquake is true, then P(Burglary|Alarm Earthquake) goes down to 0.003. Even though burglaries and earthquakes are independent, the presence of one makes the other less likely. • (d) Mixed inferences (combining two or more of the above) Q E E Q E a b c d Q 15 E Q E Applications of Bayes nets • Calculating the belief in query variables given define values for evidence variables, • Making decisions based on probabilities in the network and on agent’s utilities, • Deciding which additional evidence variables should be observed in order to gain useful information, • Performing sensitive analysis to understand which aspects of the model have greatest impact on the probabilities of the query variables, • Explaining the results of probabilistic inference to the user. 16 Exercises Ex1. Could you calculate P(S|L) in page 8. Ex2. Could you write down the formulas to compute P(Cold, Sore-throat, Runny-nose), P(Sore-throat|Cold) P(Cold| Sore-throat), P(Cold| Sore-throat, Runny-nose) Cold Sore-throat Runny-nose 17

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 6 |

posted: | 7/28/2011 |

language: | |

pages: | 17 |

OTHER DOCS BY MikeJenny

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.