# Introduction to Bayesian Networks What is it all about

Document Sample

```					                                                                   What is it all about?

Method to reasoning under uncertainty
Introduction to Bayesian Networks

Where we reason using probabilities
28th of september 2009

Tina Birk Jensen

Reasoning under uncertainty: An example                         Reasoning under uncertainty: An example

Gastro                                                          Gastro
Intestinal                   Pneumonia                          Intestinal                 Pneumonia
disorders                                                       disorders

Reduced                                                        Reduced
Diarrhoea                                            Coughing   Diarrhoea                                          Coughing
Weight gain                                                     Weight gain

Reasoning under uncertainty: An example                         Reasoning under uncertainty: An example

Gastro                                                          Gastro
Intestinal                   Pneumonia                          Intestinal                 Pneumonia
disorders                                                       disorders

Reduced                                                        Reduced
Diarrhoea                                            Coughing   Diarrhoea                                          Coughing
Weight gain                                                     Weight gain

1
Where is Bayesian networks placed in AHM?               Text books/literature

1. Bayesian Networks and Decision Graphs

A general textbook on Bayesian networks and decision
graphs.

Written by professor Finn Verner Jensen from Ålborg
University – one of the leading research centers for
Bayesian networks.

2. Bayesian Networks without Tears

Article written by Eugene Charniak

Software                                                Outline
Esthauge LIMID Software System                          Today (28th of september)
General introduction to Bayesian networks:
www.esthauge.dk
What is a Bayesian network?
homepage                                                    Transmission of evidence
Exercise 11.2

Tuesday (29th of september)
Building Bayesian network models

Friday (2nd of October)
Case example

Bayesian networks (in general)                          Bayesian networks - definition

Graphical model with some restrictions (next slide)   Bayesian networks consist of:

Basically a static method (“here and now” imagine)       A set of variables and a set of directed edges between
variables
A static version of data filtering
Each variable has a finite set of mutually exclusive states

All parameters are probabilities

2
”Each variable has a definte set of mutually excusive states”              Bayesian networks - definition

Bayesian networks consist of:
Yes                            Yes
No                             No                           A set of variables and a set of directed edges between
variables
Gastro
Intestinal                    Pneumonia                         Each variable has a finite set of mutually exclusive states

disorders
The variables and directed edges form a directed acyclic
0-100g/day                    No coughing        graph
No diarrhea
101-200 g/day               Some coughing
Some diarrhoea
>200 g/day                 A lot of coughing
A lot of diarrhoea
Reduced
Diarrhoea                                                     Coughing
Weight gain

”The variables and directed edges form a directed acyclic graph”            ”The variables and directed edges form a directed acyclic graph”

A                                                                           A

B                                                                           B

C                    D                                                   C                        D

F                       E                    G                             F                       E                         G

”The variables and directed edges form a directed acyclic graph”            Bayesian networks - definition

Bayesian networks consist of:
A
A set of variables and a set of directed edges between
variables

Each variable has a finite set of mutually exclusive states
B

The variables and directed edges form a directed acyclic
graph

C                    D                                To each variables A with parents B1, B2 to Bn there is
attached the probability table P(A| B1, B2 …. Bn )

F                       E                    G

3
Baye’s Theorem                                                                          Now an example!!

A small Bayesian network: Pregnancy and heat
A            A1, A2,….,An                       detection in cows

Yes
Not observable                    Pregnant
B            B1, B2,….,Bm                                                                                No

P( Bj | Ai ) P ( Ai )
P( Ai | Bj ) =                                                                                                                                          Yes
P( Bj | A1) p( A1) + P( Bj | A2) P( A2) + ..... + P( Bj | An) P ( An)            Observable                       Heat
No
n
P ( Bj ) = ∑ P( Bj | Ak ) P ( Ak )
k =1

Pregnancy and heat detection in cows                                                  Conditional probabilities

What is the probability that a farmer observes a
particular cow in heat during a 3-week period?                                      Now, assume that the cow is pregnant.
What is the conditional probability that the farmer observes it in heat?
P(Heat = ”yes” | Pregnant = ”yes”) = ap+
P(Heat = ”yes”) = a                                                                 P(Heat = ”no” | Pregnant = ”yes”) = bp+
P(Heat = ”no”) = b                                                                  Again, ap+ + bp+ = 1
a + b = 1 (no other options)
Now, assume that the cow is not pregnant.
What is the probability that the cow is pregnant?                                     Accordingly:
P(Heat = ”yes” | Pregnant = ”no”) = ap-
P(Heat = ”no” | Pregnant = ”no”) = bp-
P(Pregnant = ”yes”) = c
Again, ap- + bp- = 1
P(Pregnant = ”no”) = d
c + d = 1 (no other options)                                                  Each value of Pregnant defines a full probability distribution for Heat.
Such a distribution is called conditional

A small Bayesian net                                                                    Experience with the net: Evidence

Pregnant = ”yes” Pregnant = ”no”                  By entering information on an observed value of Heat
Pregnant                                                                           we can revise our belief in the value of the
unobservable variable Pregnant.
c = 0.5               d = 0.5
The observed value of a variable is called evidence.

Heat = ”yes”    Heat = ”no”
The revision of beliefs is done by use of Baye’s
Theorem:
Heat              Pregnant =”yes”            ap+ = 0.02      bp+ = 0.98
Pregnant = ”no”            ap- = 0.60      bp- = 0.40
P ( Bj | Ai ) P ( Ai )
P( Ai | Bj ) =
P ( Bj | A1) p ( A1) + P ( Bj | A2) P ( A2) + ..... + P ( Bj | An) P ( An)
Let us build the net!

4
Baye’s Theorem for our net
Baye’s Theorem for our net

How do we use Bayes formula to calculate:                                                         How do we use Bayes formula to calculate:
P(Pregnant=”yes”|Heat=”yes”)                                                                      P(Pregnant=”yes”|Heat=”no”)

P ( Bj | Ai ) P ( Ai )
P ( Ai | Bj ) =
P ( Bj | A1) p ( A1) + P ( Bj | A2) P ( A2) + ..... + P ( Bj | An) P ( An)

Extension of the net

Now time for exercise 11.1!                                                                  Info. variables
Insem.                 Prior probability

Hypothesis variable          Pregnant

Info. variables    Heat1       Heat2         Heat3        Test

Advantages of Bayesian networks                                                                     Herd diagnostics

Consistent combination of information from various                                        Risk factors         Herd        SPF         Purchase
sources
size       status         policy

Can estimate certainties for the values of variables
that are not observable (or very costly to observe).
These variables are called ”hypothesis variables”.
Mycoplasma
Hypothesis variable
These estimates are obtained by entering evidence in                                                                   pneumonia
”information variables” that

Influence the hypothesis variable
Depend on the hypothesis variable

Symptoms          ↓DWG          Temp    ↑   Coughing

5
Transmission of evidence                                             Transmission of evidence

Serial connections                                                   Diverging connection

Breed
Feed                   Colic               Death

Litter
Color
size
If ”Colic” is observed, there will be no connection
between ”Feed” and ”Death”
If ”Breed” is observed, there will be no influence
”Feed” and ”Death” are d-separated given                          of ”Color” on ”Litter size”
”Colic”
”Litter size” and ”Color” are d-separated given
Evidence may be transmitted through a serial                      ”Breed”
connection unless, the state of the intermediate
variable is known                                                 Evidence may be transmitted through a diverging
connection unless, the state of the intermediate
variable is known

Transmission of evidence                                             The previous example – d-separation

Converging connection

Mastitis              Heat

Temp                                                   Gastro
Intestinal                 Pneumonia
If ”Temp” is observed, the information that a cow                     disorders
is not in heat will influence the belief that the cow
has mastitis
Evidence may only be transmitted through a
converging connection if a connecting variable (or
descendant is observed)                                                             Reduced
Diarrhea                                          Coughing
Weight gain

The previous example – d-separation                                  The previous example – d-separation

Age                      Season                                    Age                       Season

Gastro                                                             Gastro
Intestinal                 Pneumonia                                Intestinal                 Pneumonia
disorders                                                           disorders

Reduced                                                             Reduced
Diarrhea                                          Coughing          Diarrhea                                          Coughing
Weight gain                                                         Weight gain

6
Exercise: Mastitis detection                                      Compilation of Bayesian networks
Cursory

Compilation:
Previous case              Milk yield         Mastitis index
Create a moral graph
Add edges between all pairs of nodes having a
common child.
Remove all directions
Triangulate the moral graph
Heat
Mastitis                                               Add edges until all cycles of more than 3
nodes have a chord
Identify the cliques of the triangulated graph and
organize them into a junction tree.

The software system does it automatically (and
Conductivity                Temperature                         can show all intermediate stages).

Sum up                                                          Next time (29th of september)

Bayesian networks                                               Building Bayesian networks

Reasoning under uncertainty                                    Determining the graphical structure

Graphical model with some restrictions                         Determining the conditional probabilities
Variables and nodes form a DAG
All interpendencies are descibed using conditional
probabilty distributions                                   Modeling tricks and tips
Can reason against the causal direction

Consistent combination of information from various
sources

Can estimates certainties for hypothesis variables

7

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 35 posted: 10/8/2009 language: English pages: 7
How are you planning on using Docstoc?