An Essay Towards Solving a Problem in the Doctrine of Chances
Template:Short description "An Essay Towards Solving a Problem in the Doctrine of Chances" is a work in the mathematical theory of probability by Thomas Bayes, published in 1763,[1] two years after its author's death. The essay was prepared for publication, and supplemented with amendments and additions, by Bayes's friend Richard Price.
The title reflects the contemporary use of "doctrine of chances" to mean probability theory, a usage introduced by the title of a book by Abraham de Moivre. Contemporary reprints of the essay instead carry the more descriptive title A Method of Calculating the Exact Probability of All Conclusions Founded on Induction.[2]
The essay contains theorems of conditional probability that form the basis of what is now known as Bayes' theorem, together with a detailed treatment of the problem of setting a prior probability.
Overview
Bayes considered a sequence of independent trials, each resulting in either success or failure, with the probability of success on any given trial equal to some fixed but unknown quantity p between 0 and 1. He treated p itself as uncertain, assigning it a probability of falling in any subinterval of [0, 1] equal to the length of that subinterval — in modern language, a uniform distribution on [0, 1].
Conditional on the value of p, the outcomes of the trials are independent; unconditionally, they are not, since observing many successes makes a larger value of p more probable, which in turn makes success on the next trial more probable. Bayes's question was: given the number of successes and failures observed so far, what is the conditional probability distribution of p?
The answer is that the probability density function of p, given k successes in n trials, is
(and f(p) = 0 outside that interval). This is the Beta distribution with parameters k + 1 and n − k + 1.
Outline
Bayes's preliminary results on conditional probability — particularly Propositions 3, 4, and 5 — imply the theorem now named for him. In his own words: Template:Blockquote Symbolically (see Stigler 1982[3]), this is:
which leads to Bayes's theorem for conditional probabilities:
Bayes does not appear to have emphasized this identity itself. His real target was a broader inferential problem: Template:Blockquote The essay illustrates this with a man trying to estimate the ratio of "blanks" to "prizes" in a lottery. Having watched the lottery produce ten blanks and one prize, Bayes shows how to compute the probability that the true ratio of blanks to prizes lies between 9:1 and 11:1 — about 7.7%. He then extends the calculation as more draws accumulate: twenty blanks and two prizes, forty blanks and four prizes, and so on, until, after 10,000 blanks and 1,000 prizes, the probability rises to about 97%.[1]
Bayes's central result, Proposition 9, is, in modern notation:
- Assuming a uniform prior distribution on the binomial parameter p, after observing m successes and n failures,
Whether Bayes himself was "Bayesian" in the modern sense — that is, whether his interest lay in Bayesian inference specifically, rather than probability more generally — is unclear. Proposition 9 reads as "Bayesian" in framing a probability statement about the parameter p itself. Yet Bayes poses the underlying problem in frequentist-sounding physical terms: a ball is thrown at random onto a square table, and further balls are then thrown, landing to the left or right of the first with probabilities p and 1 − p respectively. (The table is often described in retellings as a billiard table and the ball as a billiard ball, though Bayes himself never calls them that.) The algebra is identical regardless of which interpretation is taken.
Richard Price and the existence of God
Richard Price discovered the essay and its theorem among Bayes's papers after Bayes's death, and believed it offered support for the existence of God. In his introduction to the essay, Price wrote: Template:Blockquote In modern terms, this is an instance of the teleological argument.