Bayesian Networks

A Bayesian Network, also known as Directed Graphical Model is a particular DAG (Directed Acyclic Graph) in which each node is an event and the parent is the event on which it conditionally depends on. In this Network we can see that the probability of $S$ (Slip) depends on $W$ (Wet ground) and $R$ (Rain), but $W$ (Win lottery) is completely independent from the others.

Because of this relationship, we can trivially see why the graph has to be acyclic.

Given that, we can say that:

• Each variable is conditionally independent (definition of Conditional Independence) of its non-descendents given its parents. In this figure, if we consider the variable $C$, and we observe $A$, we can say that $B$ and $D$ are not influenced by $A$. Also here, $E$ is independent from $A$ and $B$, since they are it non-descendents, and so it’s only dependent on its parent.
  • Each variable is conditionally independent of any other variable given it Markov blanket. (Meaning that each other variable outside a variable’s Markov blanket doesn’t influence the variable.) This is very useful for efficiency.

Inference

Let’s see how, given a Bayesian Network that describes a joint probability $P(X,Y,Z)$ we can compute the Marginal Probability $P(Y)$ (this is inference).

The most naive approach is just enumeration, meaning computing the exact marginal probability.

Let’s see an example with the first graph we have seen: $P(R,W,S,C)=P(R)P(C)P(W|C,R)P(S|W)$ From this, we want to compute $P(R)$ (is it raining or not?)

In order to do so, we have to iterate over all the possibilities of all the other variables $W,S,C$, which for a large graph with more than 3 variable is very expensive. A solution to this problem is to use variable elimination.

---

tags: probability-theory - graph-theory references:

• 17 Probabilistic Graphical Models and Bayesian Networks - YouTube