Talk:Bayesian probability

From Wikipedia, the free encyclopedia

Contents 1 Reference Classes 2 Types of probability: Bayesian, subjective, personal, epistemic 3 Crow Paradox 4 Observation and question 5 problem of information 6 Disputed: Paradoxes in Bayesian inference 7 Another dispute: What Laplace meant 8 Need a better example of frequentist probability 9 Beta Prior Distribution at Bayesian data analysis example 10 A good overview for "dummies" would be nice. 11 Bogus Portrait? 12 Bayesian and frequentist probability 13 Credence: is it a synonym for subjective probability 14 subject matter

[edit] Reference Classes

It's true that the remarks on the absence of reference classes are confusing, mostly because it's incorrect (now fixed in the controversy section at least). There must always be a reference class for any application of probability theory under anybody's interpretation. The basic point is that uncertainty is all expressed probabilistically. Whether the uncertainty comes from known empirical sampling variation in repeated trials, or just brute subjective states, is irrelevant. (Viz)

---

Hey authors, I'm sure the maths in this article is great, but as a non mathamatician, I can't get past the second paragraph in this article without being completely confused

"The Bayesian interpretation of probability allows probabilities to be assigned to all propositions (or, in some formulations, to the events signified by those propositions) independently of any reference class within which purported facts can be thought to have a relative frequency. "

Reference class? What is that? There is no link to what that means... Just an idea: a bit of a rewrite, so that non-mathematicians (or logical philosophers) can understand this article would be great.--Bilz0r 21:03, 28 March 2006 (UTC)

---

[edit] Types of probability: Bayesian, subjective, personal, epistemic

"Bayesian probability is also known as subjective probability, personal probability, or epistemic probability." Is this true? I'm no expert on this, but I thought subjective probability was more or less the "feel" for how probable something was, that personal probability was a very close synonym of subjective probability, and epistemic probability was, well, something different yet: the degree to which a person's belief is probably true given the total evidence that the person has. Again, I could easily be wrong, but wouldn't it be an at least somewhat controversial theory about subjective, personal, and epistemic probability to say that they are each reducible to Bayesian probability? --LMS

The way you describe it, they all seem like the same thing. Cox's theorem suggests that either they must be the same thing or that one or more of the theories contains inconsistencies or violates "common sense".

Bayesian probability are subjective in the sense that two different people can assign different probabilities to the same event because they each have different information available to them. Probabilities are subjective because they depend on what you know. In the frequentist world view probabilities are properties of the system, and don't vary with the observer. Jaynes talked about this a lot.-- [[User:Gavin Crooks|]

I definitely agree with Larry (i.e. "LMS") on this one. The fact that different probabilities may be assigned given different information certainly is not enough to mean they're subjective. Michael Hardy 30 June 2005 23:50 (UTC)

---

They are not the same thing. The mathematics involved is the same thing according to what Richard Cox is saying, but what the mathematics is applied to is the difference. Mathematicians like to think probability theory is only about mathematics and nothing else, but that's just narrowness of mathematicians. (Shameless plug: See my paper on pages 243-292 of the August 2002 issue of Advances in Applied Mathematics. I assert there that I think Cox's assumptions are too strong, although I don't really say why. I do say what I would replace them with.) -- Mike Hardy

I agree with what Larry Sanger said above. The article was wrong to say the following (which I will alter once I've figured out what to replace it with): "The term 'subjective probability' arises since people with different 'prior information' could apply the theory correctly and obtain different probabilities for any particular statement. However given a particular set of prior information, the theory will always lead to the same conclusions." That people with the SAME prior information could assign DIFFERENT probabilities is what subjectivity suggests; if people with the same prior information were always led to the same epistemic probability assignments when they correctly applied the theory, then those would be epistemically OBJECTIVE assessments of probability. They would not be "objective" in the sense in which that word unfortunately gets used most often; i.e., the probabilities would not be "in the object"; they would not be relative frequencies of successes in independent trials, nor proportions of populations, etc. I distinguish between "logical" epistemic probabilities, which are epistemically objectively assigned (and nobody has any idea how to do that except in very simple cases) and "subjective" epistemic probabilities, which measure how sure someone is of something --- the "feel", as Larry puts it. I have been known to say in public that the words "random" and "event" should be banished from the theory of probability, but we're getting into gigantic Augean stables when we say that. Michael Hardy 22:27 Jan 18, 2003 (UTC)

It's hard to overstate the difficulties in understanding the foundations of probability. A quick web search turns up [1] which mentions several schools of thought in a few paragraphs, inevitably including several different schools within the "subjective probability" camp! User:(

---

It's not strictly correct to say that no one knows how to assign logical probabilities in non-trivial cases. For example, the empirical fact of Benford's Law can be logically explained using scale invariance, which is a special case of E.T. Jaynes's Principle of transformation groups. Another non-trivial case which can solved with this principle is Bertrand's Problem. Jaynes's articles are available here. Cyan 22:42 Mar 27, 2003 (UTC)

Jaynes' arguments are very interesting, but I don't think they have yet reached the stage of proof-beyond-all-reasonable-doubt, and there's a lot of work to be done before many statisticians can reliably apply them in typical applied statistics problems. Michael Hardy 00:31 Mar 28, 2003 (UTC)

But you must admit that while Jayes' position is not complete, it has wider scope and greater consistancy than the frequentist approach (muddle of ad-hoc methods)? For me, Jaynes' recent book makes this case, and does it by focusing on comparison of results (i've added an ext-link to it on his page). (I fear that you may be interpreted as implying 'even the best cars can get stuck in the mud - so you should always walk...'. While researchers rightly focus on what is left to be added, someone reading an encyclopedia is looking to learn what is) 193.116.20.220 16:58 16 May 2003 (UTC)

---

It is also at times an attempt to describe the scientific method of starting with an initial set of beliefs about the relative plausiblity of various hypotheses, collecting new information (for example by conducting an experiment), and adjusting the original set of beliefs in the light of the new information to produce a more refined set of beliefs on the plausibility of the different hypotheses.

This sentence and the paragraph "Applications of Bayesian Probability" should be removed or moved to Bayes Theorem. In order to avoid confusion it is crucial to distinguish the philosophical interpretation of probability from the mathematical formula developed by Bayes. These are not the same, they are often misunderstood, and the current version of the article makes it easy to get it wrong. Bayes' Theorem is a mathematical formula whos truth cannot be reasonably disputed. Bayes probability is the interpretation of mathematical construct (probability) and there was significant dispute in the past. I suggest we discuss the philosophy and the historical dispute in this article and the math with its applications in Bayes Theorem. 134.155.53.23 14:28, 23 Dec 2003 (UTC)

[edit] Crow Paradox

Isn't this related to the All Crows are Black Paradox?

Hempel's paradox says that "All Crows are Black" (uncontroversial) logically implies "All not-Crows are not-Black" (manifestly untrue). What's your point? 217.42.117.73 14:46, 6 Mar 2005 (UTC)

No, it's equivalent to "all non-black things are not crows". Banno 19:44, Mar 6, 2005 (UTC)

Could it be that the paradox intended here is that a brown cow counterintuitively becomes a confirming instance of the hypothesis that all crows are black. PJTraill 00:08, 6 November 2006 (UTC)

[edit] Observation and question

I don't know how or if this can be incorprated, but it's been my experience from comparison of frequentist multiple-range tests (e.g., Ryan-Einot-Gabriel-Welsch) with a Bayesian test (e.g., Waller-Duncan) that the former are more subject to perturbation by the overall distribution of the dataset. Specifically, if one mean is very much greater magnitude than all the other means, the frequentist test will assign the extreme mean to one group while assigning all other means to a second group, no matter how much difference there may be among the remaining means and no matter how tightly each treatment's results group around their respective means. The Bayesian test, on the other hand, does not do this. While none of us poor molecular biologists in our lab have our heads around the math, the Bayesian outcome "smells better" given our overall understanding of the specific system, developed over multiple experiments. Since we're cautious, we just report both analyses in our papers. Dogface 04:07, 13 Oct 2004 (UTC)

[edit] problem of information

I just added a section about (well-known) problem of conveying information with probabilities; I think it should be better integrated with the rest, but I am not an expert in the rest. :) Samohyl Jan 19:59, 11 Feb 2005 (UTC)

Looks interesting, but can it be revised to either use "information" in the technical sense, or some other word if not? It seems like we should omit the disclaimer note that the use of "information" here is not a strict mathematical one if possible. Regards & happy editing, Wile E. Heresiarch 00:37, 12 Feb 2005 (UTC)

You're right. I have changed "information" to "evidence", and it looks much better now. Samohyl Jan 10:16, 12 Feb 2005 (UTC)

Oh, man. People really shouldn't write sections when they have no clue what they are talking about. A bayesian would assign the same probability of picking a head, however the prior probability over hypotheses (what is the probability of the coin landing heads), would most likely be represented as a Beta distribution whose distribution is

P(theta;a,b) = Gamma(a+b)/Gamma(a)Gamma(b) * theta^a-1 (1-theta)^b-1

I'm going to place a NPOV message on this article until these errors are sorted out.

why don't you just fix it? Why the fuss?Banno 06:06, August 27, 2005 (UTC)

I think the section is about that there is other uncertainty - ignorance - than probability, and as such it cannot be represented by the probability distribution itself. You could of course represent both as a probability over the space of hypotheses, but, imho, this is exactly the core of the frequentist/bayesian debate about what can/should probabilities represent. You chose beta distribution for technical reasons (conjugate prior), but in fact, you're in this case only interested in the mean and variance of the probability distribution, so many other distributions would do. This notion is exactly what Dempster-Shafer theory or upper probability tries to model. On the other hand, you're right, from a practical POV, bayesians commonly do it this way, and it (probably) yields the same results as using Dempster-Shafer theory (except I am not sure if you won't run into computational difficulties in more complex cases than coin flipping). But I am not expert, and will happily read opinion of the others. Samohyl Jan 13:21, 27 August 2005 (UTC)

I'm not solely interested in the mean and variance. In fact, if I were going to determine P(something), where something depended on theta then I would treat theta as a nuisance parameter and integrate it out, taking every piece of information conveyed in the distribution into account. The inappropriate thing about the section (and the reason I fixed it) is that this page is about Bayesian probability, and in Bayesian probability, you can express uncertainty in hypothesis using probability theory. I think that, eventually, these pages on Bayesian probability and inference should be rearranged and that a new page on the war between frequentists and bayesians be separated out. It's important to know that there is controversy, but the back-and-forth "this is what bayesians think but..." makes an already difficult subject matter even more difficult.

Now, Samohyl. You say that "this is exactly the core of the debate." I agree. However, we cannot have the articles flopping back and forth. Bayesians choose to interpret probabilities as degrees of beliefs. Frequentists don't. So your section on how frequentists point out some shortfall of Bayesian analysis, when in fact, that's NOT how bayesians DO IT, is absurd. Frequentists think its a shortfall because they don't allow the solution Bayesians use.

I think we should hear some suggestions about what might go on a "controversy page"... My suggestion would be a page on "Frequentists paradoxes", "Bayesian paradoxes", "Bayesian vs Frequentist", and then change the Bayesian pages to cite that there is controversy but then explain clearly the point of view of Bayesians (and similarly on the frequentist pages). roydanroy 13:48, 27 August 2005 (EST)

Fine. I just found probability interpretations, which seems to be the right page. Samohyl Jan 20:36, 27 August 2005 (UTC)

[edit] Disputed: Paradoxes in Bayesian inference

I would dispute the following:

This can lead to paradox situations such as false positives in bayesian inference, where your prior probability can be based on more evidence (contains more information) than your posterior probability, if the conditional probability is not based on enough evidence.

What evidence is there that paradoxes occur? User:Blaise 21:11, 2 Mar 2005 (UTC)

You are right, I thought there was a connection, but it isn't, so I will remove the sentence. Thanks for correction. But some example where this is a problem would be fine, if there is any. Samohyl Jan 19:54, 3 Mar 2005 (UTC)

[edit] Another dispute: What Laplace meant

In the history section, this apparent paraphrasing of Laplace bugs me:

'It is a bet of 11000 to 1 that the error in this result is not within 1/100th of its value'

Am I reading this wrong, or is that saying exactly the opposite of what is mean? Shouldn't "not within" be replaced by either "within" or "not more than" in this context? Or am I reading the stating of the odds backwards? I traced it all the way back to the edit which added that whole section, so it's not just some random minor vandal sticking a "not" in there, at least. --John Owens (talk) 23:28, 2005 Mar 14 (UTC)

I am morally certain that you are correct. The sentence doesn't make sense as written, and doesn't comport with my memory of what Laplace wrote. However, I cannot find the original quote in my library. Stigler doesn't have it, nor does J. O. Berger (at least, not so I can find it from the index). I had thought that Jaynes might have it, but the index in his finally printed version is not very good. I would be comfortable in making the change, but think that if someone can find the quote we would be on more solid ground. Perhaps the original contributor of this quotation can verify it? Bill Jefferys 20:34, 29 August 2005 (UTC)

I believe I have found where the original contributor got this. The quotation from the Wiki article is

For instance, Laplace estimated the mass of Saturn, given orbital data that were available to him from various astronomical observations. He presented the result together with an indication of its uncertainty, stating it like this: 'It is a bet of 11000 to 1 that the error in this result is not within 1/100th of its value'. Based on current estimates, he would have won the bet, as another 150 years' accumulation of data has changed the estimate by only 0.63%.

D. S. Sivia, in Data Analysis: A Bayesian Tutorial (Oxford: Clarendon Press 1996, pp. 7-8) writes

Laplace stated that '...it is a bet of 11,000 to 1 that the error of this result is not 1/100 of its value'. He would have won the bet, as another 150 years' accumulation of data has changed the estimate by only 0.63%!

The only significant difference is the inclusion of the word 'within' in the Wiki article, which appears to be a mistake. Accordingly, I will remove that word from the article.

However, the second sentence now becomes problematic, as it appears to have been lifted from Sivia's book without attribution and only minor change. I do not know what the Wiki convention for this would be. It seems to me that a paraphrase is in order, or else an attribution to Sivia. Can someone with more experience with Wikipedia comment? Bill Jefferys 23:05, 29 August 2005 (UTC)

Hello Bill. I'm glad to see you're here. I am familiar with your posts to sci.stat.math (mine appear under the name "Robert Dodier"). Anyway, about the unattributed quotation, we do try to keep our collective noses clean wrt copyrights. I see two solutions: (1) "put it in quote marks" and attribute the remark to Sivia. (2) replace it with a paraphrase. Maybe someone has already done one or the other. Thanks for your contributions and keep up the good work! Wile E. Heresiarch 01:12, 30 August 2005 (UTC)

Hello Robert, I remember you well (but have stopped reading UseNet as it was taking too much of my time). Thanks for your remarks. I have modified the article to explicitly credit Sivia with the comment, and have added his book in the references section.

Meantime, if you (or anyone else) wishes to look at the article I wrote on admissible decision rules or the additions I made to the section on uninformative priors, I would be grateful for any additions or corrections. Also, articles are needed on reference priors, complete class theorems and Stein effect. Is anyone willing to take a crack at some of these? Bill Jefferys 20:43, 2 September 2005 (UTC)

[edit] Need a better example of frequentist probability

In the section titled Bayesian and Frequentist probability, the statement is:

'For example, Laplace estimated the mass of Saturn (described above) in this way. According to the frequency probability definition, however, the laws of probability are not applicable to this problem. This is because the mass of Saturn is a constant and not a random variable, therefore, it has no frequency distribution and so the laws of probability cannot be used.'

My reading of this is that the statement refers to the value for the mass arrived at experimentally, not the absolute mass of Saturn. The experimental results will have a frequency distribution, depending on the degree of error associated with the method and apparatus used. I don't have an alternative example, but I find this one less than ideal.

WRONG WRONG WRONG WRONG!!!!!! This is a good example precisely because it's about the absolute mass and that is NOT a random variable. The whole point of Bayesianism is to assign probability distributions NOT to things that vary randomly, but to things that are uncertain. Michael Hardy 23:17, 28 August 2005 (UTC)

Note that some authorities say that the essence of Bayesianism is that quantities such as the mass of Saturn are considered to be random variables, with the data fixed and not themselves random variables (although a particular data set does arise from a random process) whereas under frequentism it is the other way around. Much of the Bayesian literature talks in this way. But I am in agreement with Michael Hardy that the Saturn mass example is an excellent one (besides being a classical example that is commonly given as such). Bill Jefferys 20:16, 29 August 2005 (UTC)

Indeed, James Berger is one of the Bayesian authorities from whom you may hear that, and he will tell you explicitly that he really doesn't care about this particular issue, so at least on that point he is not an authority. Other "Bayesian authorities" use the language of "random variables" merely because it's conventional in probability theory, and it takes considerable effort to extensively revise conventional nomeclature to be consistent with one's philosophical position. Bottom line: those "Bayesian authorities" who say things like that are not necessarily taking a well-thought-out philosophical position; they're just being conventional. But see Edwin Jaynes's much-discussed book Probability Theory: the Logic of Science. Michael Hardy 01:17, 30 August 2005 (UTC)

I asked Jim Berger about exactly this point not too long ago. He agreed that from the Bayesian POV, parameters are properly considered random variables. He didn't volunteer that he doesn't particularly care about the issue, and I didn't ask. I'm puzzled why you would think that his not caring (if this is indeed the case) would make him not an authority! But in any case, from an axiomatic point of view, parameters in Bayesian statistics possess all of the characteristics of random variables, so it seems hard to avoid concluding that they are. Of course, if you are using the phrase 'random variable' in a vernacular sense ("things that vary randomly") then they aren't, but I don't see a good reason to abandon a precise mathematical definition just because the general public uses 'random' in an imprecise way.

I have the published version of Jaynes' book (Cambridge Press, 2003)...can you cite a page in particular? I don't want to have to search the entire book, and the index isn't very good. Bill Jefferys 19:38, 2 September 2005 (UTC)

From a mathematical point of view, they are "random variables". I'll try to find an appropriate page in the Jaynes book, but until then, look in the index under "mind projection fallacy". Michael Hardy 00:00, 3 September 2005 (UTC)

The word random is avoided by Bayesians because most things are not random (and some think nothing is). Jaynes goes through a lot of trouble explaining that a coin flip's result is uncertain not because coin flips are random and unpredictable, but instead, that the initial conditions of the coin flip (which 100% characterize its final head/tail state) are uncertain and that this uncertainty propagates and results in uncertainty in its final resting state. People who do bayesian analysis but still use the frequentist language (random variables, IID, etc) sometimes are still confused about this issue, falling prey to Jaynes' mind projection fallacy as Mr. Hardy cited. Roydanroy

[edit] Beta Prior Distribution at Bayesian data analysis example

Hi,

When "probability that the next ball is black is p" is calculated under situation 1 (no knowledge as to the quantities) a Beta prior distribution is calculated "B(alfa_B=1,alfa_W=1)". Nevertheless when I checked Beta distribution definition at "http://en.wikipedia.org/wiki/Beta_distribution" it doesn't correspond. Fita should be powered to "alfa_B-1" and "1-fita" should be powered to "alfa_W-1" which shoud be in our case "0" for both cases. This is not what is shown at the example where both values are powered to "1".

I also think there is a type mistake at same point. Later it refers to the case where m black balls and n white balls are drawed and it is defined "alfa_B=1+m" and "alfa_B=1+n". I assume that second case should be "alfa_W=1+n".

Besides that it would help if same notation is used everywhere ("alfa_B" should be "alfa" and "alfa_W" should be "beta").

I hope it is clear enough. Sorry for my English.

Regards

Alex

You're right. Although some authorities define the beta distribution as in this page, the most standard definition incorporates a '-1' in the exponents of both α and β. Also, the Wiki article on the beta distribution defines it in this way. So for uniformity, the article on Bayesian probability should be revised to conform. Note that the normalizing constant also needs correction as per the beta distribution article. Are there any disagreements? Bill Jefferys 17:08, 4 September 2005 (UTC)

Thanks (good eye). I've fixed it.

Hi again,

I'm not so sure that this new equation "P(fita)=1" regardless fita value is correct. It implies that statement that the probability that the next ball is black is ex. 30% is 100%, also that statement that the probability that the next ball is black is 60% is also 100%, ... it would make more sense to me if only P(0.5)=1 and all other values 0, what do you think?

Alex

I assume by 'fita' you mean 'theta' or in symbols either θ or Θ. But nowhere in the article can I find that anyone is asserting that θ = 1. In example three it is suggested that when you know how many balls are in the box (and that the number of black balls equals the number of white balls), that an appropriate prior is , though I think the writer meant since Θ isn't defined anywhere (someone check me on this). I think this isn't quite right...rather, it should be written , which puts a unit mass on θ = 1 / 2 and no mass anywhere else. The lower case p is appropriate since the Dirac delta function is a (limiting) probability density rather than a distribution on a finite space. Bill Jefferys 16:54, 9 September 2005 (UTC)

Hi Bill, You are right, I meant theta "θ" Sorry :). My question was on example 1 "You have a box with white and black balls, but no knowledge as to the quantities". After last editorial modification it states that P(θ) = 1 letting θ = p represent the statement "the probability that the next ball is black is p". From that, I conclude that independently to θ value (from 0 to 1) probability of θ is always 1. So, regardless the probability value (ex. p=0.3, p=0.9, ...) a Bayesian will assign a 100% probability to the sentence. That doesn't make sense to me ... It makes sense to me that all sentences (with different "p" values") may have the same probability value since all can be equally right, except for values p=0 (since we know there are black balls in the box) and p=1 (since we know there are white balls in the box), but, to me that probability value cannot be 1. Which are your thoughts? Besides that I agree with you, is more correct. Bet regards, Alex

I am missing something here. I do not see where it states, after example 1, after the last editorial modification, where it says that P(θ) = 1. If it were to say this, it would certainly be wrong in the context of the example. But I cannot find this statement. Even when I search the source code for the article for 'P(\theta)=1', I cannot find it.

Please show me specifically where the statement is. Bill Jefferys 01:52, 10 September 2005 (UTC)

Example 1 states , so, at the end it is P(θ) = 1

OK, now I see the confusion. The statement indeed says that P(θ) = 1, but that is not a statement that θ = 1 dogmatically. Rather, it is a statement that θ can take on any value in its range, which for the beta distribution is the interval [0,1]. Equal probability is assigned to each value in the range. Recall that P(θ) is a probability density function. The value 1 is just the normalization (since the integral over all of the range of θ has to equal 1). In another case, say where θ ranges from -1 to 1 (for example, a correlation coefficient), one would have range [-1,1] and P(θ) = 1 / 2 so that the integral would be 1.

So, there is nothing wrong with this equation. It is exactly correct.

Does this clarify things for you? Bill Jefferys 13:54, 10 September 2005 (UTC)

Hi, I've got to the same conclusion this afternoon while I was driving. P(θ) = 1 means that all statements (all θ values) have the same probability. But it doesn't mean that they have 100% probability. I only would like to emphasize that only θ = 0 and θ = 1 values should have 0% probability since we know for sure that in the box there are black and white balls. Regards. Alex

Technically, if you know that there are both black balls and white balls in the box, the probability is zero for . This would mean that P(θ) = 1 only on the open interval (0,1) but not on the closed interval [0,1]. However, this is only a technicality, since probabilities are derived by integration over intervals and generally speaking (for well-behaved functions) the integrals are the same regardless of whether one includes or excludes the endpoints.

To make things even more complicated, probabilities are defined on a sigma-algebra of sets, and for the real line these are usually defined as sets that are half-open and half-closed, e.g., [a,b). The technicalities of constructing such a sigma-algebra may lead to odd looking things happening at the endpoints of intervals. But, as I say, these technicalities need not concern anyone but the most fastidious, since they don't arise under most well-behaved circumstances. Bill Jefferys 22:31, 10 September 2005 (UTC)

Do last editing modifications on example 1 affect earlier sentence "...they would choose probability 1/2 in all three cases..." refering to Bayesian? Alex

No Bill Jefferys 02:54, 12 September 2005 (UTC)

Hi, at example 1, "...α_B = 1 + m, α_B = 1 + n...." should read "...α_B = 1 + m, α_w = 1 + n", shoudn't it? Alex

You're right. I've fixed this. Bill Jefferys 15:14, 19 September 2005 (UTC)

[edit] A good overview for "dummies" would be nice.

I read this article and had a little trouble deciphering exactly what Bayesian probability is. This article seems like a good body of knowledge for people who already know about this topic. I found a good overview on the web at http://www.bayesian.org/openpage.html (Overview Section) and then it follows up with http://www.bayesian.org/bayesexp/bayesexp.htm . This type of simplified explanation is exactly what I think is missing from this article. —The preceding unsigned comment was added by A67676767 (talk • contribs) 18 December 2005.

I agree. I'm doing an assignment about errors in information systems and I need a paragraph summing up Bayesian probability. This article was useless to me.--195.194.178.251 16:07, 6 March 2007 (UTC)

Technical tag added, this is something only an expert can understand SeanWDP 20:27, 24 March 2007 (UTC)

[edit] Bogus Portrait?

Note that the portrait of Thomas Bayes in the article is believed by most authorities not to be of Thomas Bayes, although the portrait is widely used. Don't know what to do about this. At the very least, the fact that the authenticity of the portrait is disputed should be noted. Bill Jefferys 23:28, 30 January 2006 (UTC)

[edit] Bayesian and frequentist probability

I find this comment somewhat unclear: The Bayesian approach is in contrast to the concept of frequency probability where probability is held to be derived from observed or imagined frequency distributions or proportions of populations. If you allow someone to invent some imagined frequency distribution, then how can you rule out them imagining a multiverse, each member of which has a Saturn of different mass (distributed according to some prior)? Jdannan 13:19, 2 February 2006 (UTC)

I have changed "imagined" distributions to "predicted" frequency distributions in the article; but I don't know whether that helps.

I've changed it to "defined". It seems to me that a very clear division can be made over whether the prior is explicitly (implicitly) provided as part of the problem, or must be chosen by the analyst. If someone asks me for the probability of rainfall on a random February day, it would be natural to use an (essentially frequentist) analysis of historical data. If someone asks me for the probability of rain tomorrow, there is no obvious prior and I have to make some judgements.Jdannan 00:16, 4 February 2006 (UTC)

Such imagined distributions, or statistical ensembles as they are called, are widely used by both Bayesians and frequentists to mentally visualise probabilities. The basic issue is that frequentists deny that you can meaningfully form a prior for the mass of Saturn in any objective way; on the other hand, if you know the (observable) pattern of random errors introduced by your instruments, you can form an objective law of probability for the results of your observations for a given fixed unknown parameters. And this set of probabilities would be falsifiable: it would reflect the predicted spread of repeated observations, which you could check by performing the experiment again and again. On the other hand there is only one Saturn, so (from a frequentist's point of view), any probability distribution for it is unobjective, unfalsifiable, and therefore meaningless. For the frequentist it is only probabilities which correspond to predicted frequencies of observations that you could in principle perform that can have any meaning. -- Jheald 14:32, 2 February 2006 (UTC).

FOUNDATIONS As is known, the idea of a probabilistic logic of inductive inference based on some form of the Principle of Indifference always retained a powerful appeal. Keynes recommended a modified version of the principle in order to achieve this aim. Carnap followed Keynes in this attempt of creating a purely logical theory of induction. However, up to now all modifications of the principle failed. A modified version of the Principle of Indifference may be provided without generating paradoxes and inconsistencies. Besides, a general criterion of assignment of prior probabilities in case of initial ignorance may be suggested, thus providing a reply to the objections to the so-called objective Bayesianism. The new (or modified) principle of indifference prescribes a uniform distribution over a partition of hypotheses where there is no reason to believe one more likely to be true than any other, in the sense of both irrelevance of prior information and impartiality of design d (or method of inquiry). In order to ensure the impartiality of d with respect to the parameter theta, it is sufficient that, for all possible likelihood functions that can be obtained from d, the respective maxima of likelihood remain constant with respect to theta itself. In other words, the prior is uniform if the maximum of each curve is constant (or it is situated on the same level of any other). Besides, we can assume the prior for a parameter proportional to the corresponding maximum value of likelihood for all possible likelihood functions obtainable from the projected design. cf. de Cristofaro, Foundations of the Objective Bayesian Inference This paper was presented to the First Symposium on Philosophy, History and Methodology of ERROR held to Virginia Tech in Blacksburg (Virginia) on 1-5 June 2006. Available on INTERNET. —The preceding unsigned comment was added by 150.217.32.23 (talk • contribs) 20 June 2006., possibly Rodolfo de Cristofaro (signed that way elsewhere).

[edit] Credence: is it a synonym for subjective probability

The classic statement of the Sleeping Beauty problem employs the term credence, apparently as a synonym for subjective probability. I have been unable to find convincing examples of the word in this sense on the internet. The Oxford English Dictionary of 1933 offers 8 senses, of which only the first means belief, and does not havve a shade of meaning of degree of belief. Do other contributors regard credence as a technical term, or was the formulation of the problem unfortunate? PJTraill 00:50, 6 November 2006 (UTC)

For some reason this problem (Sleeping Beauty) is usually stated using the word "credence." Maybe Adam Elga (the inventor) wanted to emphasize the subjective nature of his question in a theory-neutral manner—thus avoiding theory-loaded expressions like "subjective probability" and "Bayesianism"? INic 04:03, 6 November 2006 (UTC)

[edit] subject matter

The article doesn't clearly state what "Bayesian probability" is. It should probably be contrasted to frequency-based probability. This is not done anywhere, not even in the section claiming to do it in its name. --MarSch 12:49, 18 February 2007 (UTC)