Talk:Bayesian network

From Wikipedia, the free encyclopedia

Contents 1 history 2 learning 3 Dynamic Bayesian network 4 Way, way, wayyyyy too technical 5 Graphical? 6 Bayesian aspect 7 reference? 8 more incorrect information 9 looming edit war? 10 N or n in "Definition" section? 11 Good start AnAj

[edit] history

Wasnt there some big history to bayesian networks? They were one of the first statisical processing methods invented, no?

---

The article as it stands (2003/12/26) limits the definition unnecessarily. I'm going to edit the article to address these points: (1) a node can represent any kind of variable, not just discrete random variables; variables need not be discrete, and they need not be random. (2) the arcs don't represent correlation; correlation in probability theory has a certain well-defined meaning which is not applicable here. What arcs do represent is conditional dependence. (3) "Conditional probability table" assumes that the variables involved are discrete; need to allow for continuous variables. (4) The list of applications can be expanded.

---

I've addressed (or tried to) items (1) through (3) above. Wile E. Heresiarch 07:41, 27 Dec 2003 (UTC)

An example would be very helpful in this article. Banno 01:05, Jul 7, 2004 (UTC)

[edit] learning

It might be interesting to put some comments about the learning of the BNs

--Response: I did this implicitly by saying that distributions could be parameterized and discussing the use of EM (expectation-maximization) in estimating these parameters from data. Also, there is already a paragraph on learning the structure of BN's. I think this covers the fundamental "learning" areas, though the section on parameters and EM could be expanded a bit.

[edit] Dynamic Bayesian network

I was disappointed to see the DBN link redirect back to BN. In this case someone needs to write a section discussing the specialization to DBN's, and the special cases of hidden Markov models, Kalman filters, and switching state-space models, and their applications in tracking and segmentation problems

[edit] Way, way, wayyyyy too technical

No doubt it's a wonderfully accurate and concise explanation exactly what a Bayesian network is but it's useless to those of us who aren't striving for that PhD in math. At the very least it needs an introduction stating, in plain english, what such a network is. I can't be the one to write that introduction because after reading the article I have less of a clue about what they are then before I read it. The sort of detail in this article is great as a stand-alone webpage or reference, but it's not an encyclopedia entry. Frustrating.

• I added a small hopefully more human readable 3 sentences to the introduction... --UmassThrower 04:59, 23 March 2007 (UTC)

[edit] Graphical?

Seems like a graphical tool should use some graphics... It would be a lot easier to understand with one of the example networks from the reference material. For example, Murphy's Fig. 1.

[edit] Bayesian aspect

Currently the article says

[Variables] are not restricted to representing random variables; which forms the "Bayesian" aspect of a Bayesian network.

I don't think that this is the cause for the network being "Bayesian". Surely also non-Bayesian models can have non-random variables (random variables whose probability distribution is concentrated on one value)? I would say that the name comes from the fact the Bayes' theorem is used to update the node probabilities after observing some values. AnAj 20:22, 24 March 2007 (UTC)

I'd actually like to hear what the general opinion is on this. Often, BNs are used to calculate probabilities of what a frequentist would call hypotheses. This is a no-go in the frequentist world, and the hallmark of the Bayesian way of looking at things, hence leading to the name 'Bayesian network'. Does this make sense? If not, why not? Tomixdf 21:23, 24 March 2007 (UTC)

Here's a quote from the Wikipedia entry on Bayesian probability that might clarify things:

The difference between Bayesian and Frequentist interpretations of probability has important consequences in statistical practice. For example, when comparing two hypotheses using the same data, the theory of hypothesis tests, which is based on the frequency interpretation of probability, allows the rejection or non-rejection of one model/hypothesis (the 'null' hypothesis) based on the probability of mistakenly inferring that the data support the other model/hypothesis more. The probability of making such a mistake, called a Type I error, requires the consideration of hypothetical data sets derived from the same data source that are more extreme than the data actually observed. This approach allows the inference that 'either the two hypotheses are different or the observed data are a misleading set'. In contrast, Bayesian methods condition on the data actually observed, and are therefore able to assign posterior probabilities to any number of hypotheses directly. The requirement to assign probabilities to the parameters of models representing each hypothesis is the cost of this more direct approach.

Tomixdf 21:29, 24 March 2007 (UTC)

[edit] reference?

do we have a reference for this? It directly contradicts Pearl. "A Bayesian network need not to represent causal relationships. However, if knowledge about causality is available it is natural to use it for selecting the parent variables when building the network, thus resulting in causal Bayesian networks." MisterSheik 21:18, 27 March 2007 (UTC)

In addition to Pearl's causal model Bayesian networks can be interpreted as (non-causal) models of probabilistic relationships. See Heckerman: A Bayesian approach to learning causal networks for a discussion of the two interpretations. AnAj 17:47, 29 March 2007 (UTC)

[edit] more incorrect information

"Because the joint distribution can be decomposed into product of local distributions in any order, there is no unique Bayesian network for a given distribution."

I simplified this (check page history for original version) but it's not true either. That's not the reason that there's no unique Bayesian network, because you cannot, for example, reverse a causal link just by calculating the conditional probability backwards. I think the person writing this has gotten this idea from Bishop, but he doesn't explicitly say that, and anyway it's not true (due to re-decomposition of products of local distributions-- it is true that you can add in more nodes and reorganize the network.) MisterSheik 21:39, 27 March 2007 (UTC)

Suppose we have a joint probability distribution p(x,y). It can be written either as p(x,y) = p(x|y)p(y) or p(x,y) = p(y|x)p(x). The first correspondes to a Bayesian network, where y is the parent of x, and the second corresponds to a network, where x is the parent of y. AnAj 17:53, 29 March 2007 (UTC)

I did some research, and I realize that there are formulations of Bayesian networks that aren't causal, but Pearl invented these networks, and he defines them as being causal. In that case, the directed arrows are not just conditional probabilities, but rather direct causal links, which cannot usually be reversed. Pearl devotes a whole chapter to determining the directions of the links in prob. reas. in int. sys.

In other words, the directed arrows do not just store conditional probability information, but primarily they are used to reason about independence.

My suggestion is that we have a different page for the other directed graphical models that you are describing. MisterSheik 22:17, 29 March 2007 (UTC)

I think both viewpoints can be discussed in the same article. See my recent edits. AnAj 17:11, 1 April 2007 (UTC)

[edit] looming edit war?

I can see from the recent history that an edit war may be looming. In defence of my recent changes to the definition, I want to reaffirm on the talk page that links don't represent statistical independences, but direct causal relationships. Two nodes can be dependent without being connected (for example, if they are results of the same cause.) These nodes become independent once that causal variable is known. Similarly, they are made dependent by an effect that becomes known (or if its effect is know, and so on). This is what should be explained under d-separation, which really shouldn't have its own article, but should be explained here.

d-separation is the whole point of Bayesian networks--not efficient computational properties. (Perhaps the computations are easier for students to grasp and so presented first by instructors?)

Tractability is also not an important point because the conditional probability of a variable on its parents could be intractable.

MisterSheik 21:54, 27 March 2007 (UTC)

[edit] N or n in "Definition" section?

• In the Definition section, are "N" and "n" used interchangeably?--Skoch3 16:51, 29 March 2007 (UTC)

Yes. It's fixed now. AnAj 17:58, 29 March 2007 (UTC)

[edit] Good start AnAj

I think the article is getting better, but I still think we need to cut it right in two. The definition makes it sound like there the causal thing is some funky requirement that you can drop. The reality is that the causal version is completely different: the operations that you can perform on it work differently. For example, checking to see if two nodes are independent. In the Pearl's "version", d-separation (a path, with specific requirements--this is the magic of causal relationships) is the procedure that determines independence, but without the notion of causality, you're back to m-separation (any path). The belief updating (what's been labeled inference) is done with likelihoods and priors in pearls version; i'm not sure how it's done otherwise. Also, the notion of rewiring the network-- in fact, the whole meaning of network has changed. The introduction should really reflect the reality that there are in fact two separate concepts. They are as different as markov networks and bayesian network, pearl's bayesian networks and the other "directed graphical models". I'm okay with including them in the same article since that reflects the reality of how people use the term, but I think they should be presented in a more side-by-side fashion. MisterSheik 21:11, 1 April 2007 (UTC)