Example Bayes’ Net
CS 188: Artificial Intelligence
Fall 2006
Lecture 16: Bayes Nets II
10/24/2006
Dan Klein – UC Berkeley
Bayes’ Net Semantics
Building the (Entire) Joint
ƒ A Bayes’ net:
ƒ A set of nodes, one per variable X
ƒ A directed, acyclic graph
ƒ A conditional distribution of each variable
conditioned on its parents (the parameters θ)
A1
An
ƒ We can take a Bayes’ net and build any entry
from the full joint distribution it encodes
X
ƒ Semantics:
ƒ A BN defines a joint probability distribution
over its variables:
Example: Alarm Network
ƒ Typically, there’s no reason to build ALL of it
ƒ We build what we need on the fly
ƒ To emphasize: every BN over a domain implicitly
represents some joint distribution over that
domain, but is specified by local probabilities
Bayes’ Nets
ƒ So far: how a Bayes’ net encodes a joint distribution
ƒ Next: how to answer queries about that distribution
ƒ Key idea: conditional independence
ƒ Last class: assembled BNs using an intuitive notion of
conditional independence as causality
ƒ Today: formalize these ideas
ƒ Main goal: answer queries about conditional independence and
influence
ƒ After that: how to answer numerical queries (inference)
1
Conditional Independence
Example: Independence
ƒ Reminder: independence
ƒ For this graph, you can fiddle with θ (the CPTs) all you
want, but you won’t be able to represent any distribution
in which the flips are dependent!
ƒ X and Y are independent if
ƒ X and Y are conditionally independent given Z
X1
ƒ (Conditional) independence is a property of a
distribution
X2
h
0.5
h
0.5
t
0.5
t
0.5
All distributions
Topology Limits Distributions
Independence in a BN
Y
ƒ Given some graph
topology G, only certain
joint distributions can
be encoded
ƒ The graph structure
guarantees certain
(conditional)
independences
ƒ (There might be more
independence)
ƒ Adding arcs increases
the set of distributions,
but has several costs
X
ƒ Important question about a BN:
Z
Y
X
Z
Are two nodes independent given certain evidence?
If yes, can calculate using algebra (really tedious)
If no, can prove with a counter example
Example:
X
Y
Z
ƒ Question: are X and Z independent?
ƒ Answer: not necessarily, we’ve seen examples otherwise:
low pressure causes rain which causes traffic.
ƒ X can influence Z, Z can influence X (via Y)
ƒ Addendum: they could be independent: how?
Y
X
Z
Causal Chains
Common Cause
ƒ This configuration is a “causal chain”
X: Low pressure
X
ƒ
ƒ
ƒ
ƒ
Y
Z
Y: Rain
Z: Traffic
ƒ Another basic configuration: two
effects of the same cause
Y
ƒ Are X and Z independent?
ƒ No, remember the “project due” example
ƒ Are X and Z independent given Y?
ƒ Is X independent of Z given Y?
X
Z
Y: Project due
X: Newsgroup
busy
Yes!
Yes!
ƒ Evidence along the chain “blocks” the influence
Z: Lab full
ƒ Observing the cause blocks
influence between effects.
2
Common Effect
ƒ Last configuration: two causes of
one effect (v- structures)
The General Case
X
ƒ Are X and Z independent?
ƒ Yes: remember the ballgame and the rain
causing traffic, no correlation?
ƒ Still need to prove they must be (homework)
Z
ƒ General question: in a given BN, are two
variables independent (given evidence)?
Y
ƒ Are X and Z independent given Y?
ƒ Any complex example can be analyzed
using these three canonical cases
X: Raining
ƒ No: remember that seeing traffic put the rain
and the ballgame in competition?
Z: Ballgame
ƒ This is backwards from the other cases
Y: Traffic
ƒ Solution: graph search!
ƒ Observing the effect enables influence
between effects.
Reachability
Reachability (the Bayes’ Ball)
ƒ Recipe: shade evidence nodes
ƒ Correct algorithm:
L
ƒ Attempt 1: if two nodes are
connected by an undirected path
not blocked by a shaded node,
they are conditionally independent
S
ƒ Shade in evidence
ƒ Start at source node
ƒ Try to reach target by search
R
B
ƒ States: pair of (node X, previous
state S)
X
X
S
ƒ Successor function:
ƒ X unobserved:
ƒ Almost works, but not quite
ƒ Where does it break?
ƒ Answer: the v-structure at T doesn’t
count as a link in a path unless shaded
D
T
S
ƒ X observed:
ƒ From parent to parent
T’
Example
ƒ To any child
ƒ To any parent if coming from a
child
ƒ If you can’t reach a node, it’s
conditionally independent of the
start node given evidence
X
X
S
Example
L
Yes
Yes
R
Yes
D
B
T
Yes
T’
3
Example
Causality?
ƒ When Bayes’ nets reflect the true causal patterns:
ƒ Variables:
ƒ
ƒ
ƒ
ƒ
R: Raining
T: Traffic
D: Roof drips
S: I’m sad
ƒ Often simpler (nodes have fewer parents)
ƒ Often easier to think about
ƒ Often easier to elicit from experts
R
T
ƒ BNs need not actually be causal
D
ƒ Questions:
ƒ Sometimes no causal net exists over the domain
ƒ E.g. consider the variables Traffic and Drips
ƒ End up with arrows that reflect correlation, not causation
S
ƒ What do the arrows really mean?
ƒ Topology may happen to encode causal structure
ƒ Topology only guaranteed to encode conditional independencies
Yes
Example: Traffic
Example: Reverse Traffic
ƒ Basic traffic net
ƒ Let’s multiply out the joint
R
r
1/4
¬r
3/4
r
T
t
3/4
¬t
1/4
t
1/2
¬t
1/2
¬r
ƒ Reverse causality?
r
t
3/16
r
¬t
1/16
¬r
t
6/16
¬r
¬t
6/16
T
t
R
¬t
Example: Coins
t
9/16
¬t
7/16
r
1/3
¬r
2/3
r
1/7
¬r
6/7
r
t
3/16
r
¬t
1/16
¬r
t
6/16
¬r
¬t
6/16
Alternate BNs
ƒ Extra arcs don’t prevent representing
independence, just allow non
- independence
X1
X2
X1
X2
h
0.5
h
0.5
h
0.5
h|h
0.5
t
0.5
t
0.5
t
0.5
t|h
0.5
h|t
0.5
t|t
0.5
4
Summary
ƒ Bayes nets compactly encode joint distributions
ƒ Guaranteed independencies of distributions can
be deduced from BN graph structure
ƒ The Bayes’ ball algorithm (aka d
- separation)
ƒ A Bayes net may have other independencies
that are not detectable until you inspect its
specific distribution
5