I've done a dreadful job of reading The Theory That Would Not Die, but several weeks ago I somehow managed to read the appendix. Here the author gives a short explanation of Bayes' theorem using statistics related to breast cancer and mammogram results. This is the same real world example (one of several) used by Nate Silver. It's profound in its simplicity and- for an idiot like me- a powerful gateway drug. Possibly related to this is my recent epiphany that when we're talking about Bayesian analysis, we're really talking about multivariate probability. The breast cancer/mammogram example is the simplest form of multivariate analysis available. What does it all mean, how can we extend it and what does it have to do with an underlying philosophy of Bayesian analysis (if such a thing exists)?
(This article was first published on PirateGrunt » R, and kindly contributed to R-bloggers)
The Theory That Would Not Die is sitting at my desk at work, so I'm going to refer to the figures quoted by Nate Silver on page 246. Odds for cancer are read across the columns, odds for a positive mammogram are read down the rows.
Before I go any further, I have to point out that the positioning of the tables is dreadful. WordPress experts are invited to help me sort this out.
From this table, the joint probabilities are easy to read. What is the chance that a person has breast cancer and received a negative mammogram? 3 in 1000. What is the chance that a person does not have cancer, but received a positive mammogram? 99 in 1000, or roughly 10%. It's a trivial thing to determine the marginal probabilities.
The context of this information is what matters to the authors. Each presents the result that the likelihood that a patient has cancer- even with a positive mammogram- is still rather low (10% in this case). This is consistent with advice from some areas of the medical establishment that women not get routine mammograms before a particular age. This (slightly) surprising result is driven by the fact that the positive predictive value (number of true positives divided by the number of predicted positives) is very low as is the likelihood of a positive. Put differently, a mammogram does not appear to have a good success rate at predicting cancer (for this data) and the overall rate of cancer is quite low. How would things look if the numbers changed?
How do we do that? In order to hold the cancer probability fixed, we can't change the marginal totals. So, we can move numbers in the same column from one row to another. Or, if we move from one column to another, we must offset that in the other row. As an extreme, we could assume that the test is perfectly predictive. This would move the 3 false negatives into the true positive cell and the 99 false positives to the true negative cell. In this case, there is no probability in the upper right or lower left corner of the matrix. From another perspective, it is impossible to distinguish the two marginal distributions.
But that's a bit boring, so let's create something more interesting. We'll not alter the number of false negatives, but reduce the false positives so that the positive predictive value is close to 80%.
The chance that a person has cancer, conditional on a positive mammogram is now 44.0%. Before I look at another scenario, I'm going to scrap the tables in favor of something graphical. Here's what the first matrix looks like:
And the second matrix:
In the second plot, we continue to have a large concentration of the probability in the bottom right corner, but the the top half is now more balanced. This balance comes from a shift away from top right corner. All of this means that the information about a mammogram becomes more predictive.
What happens when we increase the likelihood of cancer? In graphical terms, this would mean giving the left side a more yellow color. We'll hold the original positive predictive value (roughly 10%) fixed, but raise the likelihood of cancer to 25%.
This is interesting. The highest probability remains at the lower right hand corner (no cancer, clean mammogram) but there is now a greater concentration at the upper right and lower left corner. So, if one has a positive mammogram result, what is the posterior probability that they have cancer? The same 10% as before. And if the test showed negative? It's now 27%. This is higher than the probability if one got a positive result. Of course, this is because we've held the positive predictive value fixed, while raising the probability of the event. The efficacy of the test and the prevalence of the disease are now anti-correlated. Not the sort of thing one wants in a diagnostic tool. How would things look if the PPV were 50%?
So what makes this Bayesian? The simple answer is that I don't know. I have trouble reconciling Silver and McGrayne's simple (though very accessible) examples of Bayesian inference with what I read in Gelman and Albert. Untangling the math takes me away from the philosophy, so I'll list three quick notions about what Bayesian analysis means to me:
In this example, I look at altering the joint probability distribution. I'm free to do that, if evidence warrants it. If mammography improves- or there is a provable difference in physicians' interpretations of the results- then I may alter the probabilities. If environment and lifestyle changes yield an alteration in disease prevalence, that also affects the joint distribution. It's a great toy example to begin to explore more varied problems. That's what I'll do next as I expand the example from a very simple 2Ã2 matrix to something more complicated.
Before I forget, my understanding of the definition of positive predictive value is taken from An Introduction to Statistical Learning, which is a great book. That value is one component of the fascinating subject of binary classification. I first heard about this in a great talk given by Dan Kelly at a meeting of the Research Triangle Analysts
Session info:
To leave a comment for the author, please follow the link and comment on their blog: PirateGrunt » R.
Bayesian Analysis In Ai VideoR-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more..
If you got this far, why not subscribe for updatesfrom the site? Choose your flavor: e-mail, twitter, RSS, or facebook..
Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Bayesian inference is an important technique in statistics, and especially in mathematical statistics. Bayesian updating is particularly important in the dynamic analysis of a sequence of data. Bayesian inference has found application in a wide range of activities, including science, engineering, philosophy, medicine, sport, and law. In the philosophy of decision theory, Bayesian inference is closely related to subjective probability, often called 'Bayesian probability'.
Introduction to Bayes' rule[edit]
A geometric visualisation of Bayes' theorem. In the table, the values 2, 3, 6 and 9 give the relative weights of each corresponding condition and case. The figures denote the cells of the table involved in each metric, the probability being the fraction of each figure that is shaded. This shows that P(A|B) P(B) = P(B|A) P(A) i.e. P(A|B) = P(B|A) P(A)/P(B) . Similar reasoning can be used to show that P(Ä|B) = P(B|Ä) P(Ä)/P(B) etc.
Formal explanation[edit]
Bayesian inference derives the posterior probability as a consequence of two antecedents: a prior probability and a 'likelihood function' derived from a statistical model for the observed data. Bayesian inference computes the posterior probability according to Bayes' theorem:
where
For different values of H{displaystyle textstyle H}, only the factors P(H){displaystyle textstyle P(H)} and P(Eâ£H){displaystyle textstyle P(Emid H)}, both in the numerator, affect the value of P(Hâ£E){displaystyle textstyle P(Hmid E)} â the posterior probability of a hypothesis is proportional to its prior probability (its inherent likeliness) and the newly acquired likelihood (its compatibility with the new observed evidence).
Bayes' rule can also be written as follows:
where the factor P(Eâ£H)P(E){displaystyle textstyle {frac {P(Emid H)}{P(E)}}} can be interpreted as the impact of E{displaystyle E} on the probability of H{displaystyle H}.
Alternatives to Bayesian updating[edit]
Bayesian updating is widely used and computationally convenient. However, it is not the only updating rule that might be considered rational.
Ian Hacking noted that traditional 'Dutch book' arguments did not specify Bayesian updating: they left open the possibility that non-Bayesian updating rules could avoid Dutch books. Hacking wrote[1][2] 'And neither the Dutch book argument nor any other in the personalist arsenal of proofs of the probability axioms entails the dynamic assumption. Not one entails Bayesianism. So the personalist requires the dynamic assumption to be Bayesian. It is true that in consistency a personalist could abandon the Bayesian model of learning from experience. Salt could lose its savour.'
Indeed, there are non-Bayesian updating rules that also avoid Dutch books (as discussed in the literature on 'probability kinematics') following the publication of Richard C. Jeffrey's rule, which applies Bayes' rule to the case where the evidence itself is assigned a probability.[3] The additional hypotheses needed to uniquely require Bayesian updating have been deemed to be substantial, complicated, and unsatisfactory.[4]
Formal description of Bayesian inference[edit]Definitions[edit]
Bayesian inference[edit]
Note that this is expressed in words as 'posterior is proportional to likelihood times prior', or sometimes as 'posterior = likelihood times prior, over evidence'.
Bayesian prediction[edit]
Bayesian theory calls for the use of the posterior predictive distribution to do predictive inference, i.e., to predict the distribution of a new, unobserved data point. That is, instead of a fixed point as a prediction, a distribution over possible points is returned. Only this way is the entire posterior distribution of the parameter(s) used. By comparison, prediction in frequentist statistics often involves finding an optimum point estimate of the parameter(s)âe.g., by maximum likelihood or maximum a posteriori estimation (MAP)âand then plugging this estimate into the formula for the distribution of a data point. This has the disadvantage that it does not account for any uncertainty in the value of the parameter, and hence will underestimate the variance of the predictive distribution.
(In some instances, frequentist statistics can work around this problem. For example, confidence intervals and prediction intervals in frequentist statistics when constructed from a normal distribution with unknown mean and variance are constructed using a Student's t-distribution. This correctly estimates the variance, due to the fact that (1) the average of normally distributed random variables is also normally distributed; (2) the predictive distribution of a normally distributed data point with unknown mean and variance, using conjugate or uninformative priors, has a student's t-distribution. In Bayesian statistics, however, the posterior predictive distribution can always be determined exactlyâor at least, to an arbitrary level of precision, when numerical methods are used.)
Note that both types of predictive distributions have the form of a compound probability distribution (as does the marginal likelihood). In fact, if the prior distribution is a conjugate prior, and hence the prior and posterior distributions come from the same family, it can easily be seen that both prior and posterior predictive distributions also come from the same family of compound distributions. The only difference is that the posterior predictive distribution uses the updated values of the hyperparameters (applying the Bayesian update rules given in the conjugate prior article), while the prior predictive distribution uses the values of the hyperparameters that appear in the prior distribution.
Inference over exclusive and exhaustive possibilities[edit]
If evidence is simultaneously used to update belief over a set of exclusive and exhaustive propositions, Bayesian inference may be thought of as acting on this belief distribution as a whole.
General formulation[edit]
Diagram illustrating event space Ω{displaystyle Omega } in general formulation of Bayesian inference. Although this diagram shows discrete models and events, the continuous case may be visualized similarly using probability densities.
Suppose a process is generating independent and identically distributed events En,n=1,2,3,â¦{displaystyle E_{n},n=1,2,3,ldots }, but the probability distribution is unknown. Let the event space Ω{displaystyle Omega } represent the current state of belief for this process. Each model is represented by event Mm{displaystyle M_{m}} (what is 'm' here? its range and its meaning are not obvious). The conditional probabilities P(Enâ£Mm){displaystyle P(E_{n}mid M_{m})} are specified to define the models. P(Mm){displaystyle P(M_{m})} is the degree of belief in Mm{displaystyle M_{m}}. Before the first inference step, {P(Mm)}{displaystyle {P(M_{m})}} is a set of initial prior probabilities. These must sum to 1, but are otherwise arbitrary.
Suppose that the process is observed to generate Eâ{En}{displaystyle textstyle Ein {E_{n}}}. For each Mâ{Mm}{displaystyle Min {M_{m}}}, the prior P(M){displaystyle P(M)} is updated to the posterior P(Mâ£E){displaystyle P(Mmid E)}. From Bayes' theorem:[5]
Upon observation of further evidence, this procedure may be repeated.
Multiple observations[edit]
For a sequence of independent and identically distributed observations E=(e1,â¦,en){displaystyle mathbf {E} =(e_{1},dots ,e_{n})}, it can be shown by induction that repeated application of the above is equivalent to
Where
Currently the one I use most because it can stretch a 4:3 input to 16:9, which I find very helpful. Easycap viewer pro apk. Would be nice if it had the option to get rid of thewatermark in recorded video. I'd pay a buck to.not. One of my favorite apps for this stuff.
Parametric formulation[edit]
By parameterizing the space of models, the belief in all models may be updated in a single step. The distribution of belief over the model space may then be thought of as a distribution of belief over the parameter space. The distributions in this section are expressed as continuous, represented by probability densities, as this is the usual situation. The technique is however equally applicable to discrete distributions.
Let the vector θ{displaystyle mathbf {theta } } span the parameter space. Let the initial prior distribution over θ{displaystyle mathbf {theta } } be p(θâ£Î±){displaystyle p(mathbf {theta } mid mathbf {alpha } )}, where α{displaystyle mathbf {alpha } } is a set of parameters to the prior itself, or hyperparameters. Let E=(e1,â¦,en){displaystyle mathbf {E} =(e_{1},dots ,e_{n})} be a sequence of independent and identically distributed event observations, where all ei{displaystyle e_{i}} are distributed as p(eâ£Î¸){displaystyle p(emid mathbf {theta } )} for some θ{displaystyle mathbf {theta } }. Bayes' theorem is applied to find the posterior distribution over θ{displaystyle mathbf {theta } }:
Where
Mathematical properties[edit]Interpretation of factor[edit]
P(Eâ£M)P(E)>1âP(Eâ£M)>P(E){displaystyle textstyle {frac {P(Emid M)}{P(E)}}>1Rightarrow textstyle P(Emid M)>P(E)}. That is, if the model were true, the evidence would be more likely than is predicted by the current state of belief. The reverse applies for a decrease in belief. If the belief does not change, P(Eâ£M)P(E)=1âP(Eâ£M)=P(E){displaystyle textstyle {frac {P(Emid M)}{P(E)}}=1Rightarrow textstyle P(Emid M)=P(E)}. That is, the evidence is independent of the model. If the model were true, the evidence would be exactly as likely as predicted by the current state of belief.
Cromwell's rule[edit]
If P(M)=0{displaystyle P(M)=0} then P(Mâ£E)=0{displaystyle P(Mmid E)=0}. If P(M)=1{displaystyle P(M)=1}, then P(M|E)=1{displaystyle P(M|E)=1}. This can be interpreted to mean that hard convictions are insensitive to counter-evidence.
The former follows directly from Bayes' theorem. The latter can be derived by applying the first rule to the event 'not M{displaystyle M}' in place of 'M{displaystyle M}', yielding 'if 1âP(M)=0{displaystyle 1-P(M)=0}, then 1âP(Mâ£E)=0{displaystyle 1-P(Mmid E)=0}', from which the result immediately follows.
Asymptotic behaviour of posterior[edit]
Consider the behaviour of a belief distribution as it is updated a large number of times with independent and identically distributed trials. For sufficiently nice prior probabilities, the Bernstein-von Mises theorem gives that in the limit of infinite trials, the posterior converges to a Gaussian distribution independent of the initial prior under some conditions firstly outlined and rigorously proven by Joseph L. Doob in 1948, namely if the random variable in consideration has a finite probability space. The more general results were obtained later by the statistician David A. Freedman who published in two seminal research papers in 1963 [6] and 1965 [7] when and under what circumstances the asymptotic behaviour of posterior is guaranteed. His 1963 paper treats, like Doob (1949), the finite case and comes to a satisfactory conclusion. However, if the random variable has an infinite but countable probability space (i.e., corresponding to a die with infinite many faces) the 1965 paper demonstrates that for a dense subset of priors the Bernstein-von Mises theorem is not applicable. In this case there is almost surely no asymptotic convergence. Later in the 1980s and 1990s Freedman and Persi Diaconis continued to work on the case of infinite countable probability spaces.[8] To summarise, there may be insufficient trials to suppress the effects of the initial choice, and especially for large (but finite) systems the convergence might be very slow.
Conjugate priors[edit]
In parameterized form, the prior distribution is often assumed to come from a family of distributions called conjugate priors. The usefulness of a conjugate prior is that the corresponding posterior distribution will be in the same family, and the calculation may be expressed in closed form.
Estimates of parameters and predictions[edit]
It is often desired to use a posterior distribution to estimate a parameter or variable. Several methods of Bayesian estimation select measurements of central tendency from the posterior distribution.
For one-dimensional problems, a unique median exists for practical continuous problems. The posterior median is attractive as a robust estimator.[9]
If there exists a finite mean for the posterior distribution, then the posterior mean is a method of estimation.[10]
Taking a value with the greatest probability defines maximum a posteriori (MAP) estimates:[11]
There are examples where no maximum is attained, in which case the set of MAP estimates is empty.
There are other methods of estimation that minimize the posterior risk (expected-posterior loss) with respect to a loss function, and these are of interest to statistical decision theory using the sampling distribution ('frequentist statistics').[12]
The posterior predictive distribution of a new observation x~{displaystyle {tilde {x}}} (that is independent of previous observations) is determined by[13]
Examples[edit]Probability of a hypothesis[edit]
Suppose there are two full bowls of cookies. Bowl #1 has 10 chocolate chip and 30 plain cookies, while bowl #2 has 20 of each. Our friend Fred picks a bowl at random, and then picks a cookie at random. We may assume there is no reason to believe Fred treats one bowl differently from another, likewise for the cookies. The cookie turns out to be a plain one. How probable is it that Fred picked it out of bowl #1?
Intuitively, it seems clear that the answer should be more than a half, since there are more plain cookies in bowl #1. The precise answer is given by Bayes' theorem. Let H1{displaystyle H_{1}} correspond to bowl #1, and H2{displaystyle H_{2}} to bowl #2.It is given that the bowls are identical from Fred's point of view, thus P(H1)=P(H2){displaystyle P(H_{1})=P(H_{2})}, and the two must add up to 1, so both are equal to 0.5.The event E{displaystyle E} is the observation of a plain cookie. From the contents of the bowls, we know that P(Eâ£H1)=30/40=0.75{displaystyle P(Emid H_{1})=30/40=0.75} and P(Eâ£H2)=20/40=0.5.{displaystyle P(Emid H_{2})=20/40=0.5.} Bayes' formula then yields
Before we observed the cookie, the probability we assigned for Fred having chosen bowl #1 was the prior probability, P(H1){displaystyle P(H_{1})}, which was 0.5. After observing the cookie, we must revise the probability to P(H1â£E){displaystyle P(H_{1}mid E)}, which is 0.6.
Making a prediction[edit]
Example results for archaeology example. This simulation was generated using c=15.2.
An archaeologist is working at a site thought to be from the medieval period, between the 11th century to the 16th century. However, it is uncertain exactly when in this period the site was inhabited. Fragments of pottery are found, some of which are glazed and some of which are decorated. It is expected that if the site were inhabited during the early medieval period, then 1% of the pottery would be glazed and 50% of its area decorated, whereas if it had been inhabited in the late medieval period then 81% would be glazed and 5% of its area decorated. How confident can the archaeologist be in the date of inhabitation as fragments are unearthed?
The degree of belief in the continuous variable C{displaystyle C} (century) is to be calculated, with the discrete set of events {GD,GD¯,G¯D,G¯D¯}{displaystyle {GD,G{bar {D}},{bar {G}}D,{bar {G}}{bar {D}}}} as evidence. Assuming linear variation of glaze and decoration with time, and that these variables are independent,
Assume a uniform prior of fC(c)=0.2{displaystyle textstyle f_{C}(c)=0.2}, and that trials are independent and identically distributed. When a new fragment of type e{displaystyle e} is discovered, Bayes' theorem is applied to update the degree of belief for each c{displaystyle c}:
fC(câ£E=e)=P(E=eâ£C=c)P(E=e)fC(c)=P(E=eâ£C=c)â«1116P(E=eâ£C=c)fC(c)dcfC(c){displaystyle f_{C}(cmid E=e)={frac {P(E=emid C=c)}{P(E=e)}}f_{C}(c)={frac {P(E=emid C=c)}{int _{11}^{16}{P(E=emid C=c)f_{C}(c)dc}}}f_{C}(c)}
A computer simulation of the changing belief as 50 fragments are unearthed is shown on the graph. In the simulation, the site was inhabited around 1420, or c=15.2{displaystyle c=15.2}. By calculating the area under the relevant portion of the graph for 50 trials, the archaeologist can say that there is practically no chance the site was inhabited in the 11th and 12th centuries, about 1% chance that it was inhabited during the 13th century, 63% chance during the 14th century and 36% during the 15th century. Note that the Bernstein-von Mises theorem asserts here the asymptotic convergence to the 'true' distribution because the probability space corresponding to the discrete set of events {GD,GD¯,G¯D,G¯D¯}{displaystyle {GD,G{bar {D}},{bar {G}}D,{bar {G}}{bar {D}}}} is finite (see above section on asymptotic behaviour of the posterior).
In frequentist statistics and decision theory[edit]
A decision-theoretic justification of the use of Bayesian inference was given by Abraham Wald, who proved that every unique Bayesian procedure is admissible. Conversely, every admissible statistical procedure is either a Bayesian procedure or a limit of Bayesian procedures.[14]
Wald characterized admissible procedures as Bayesian procedures (and limits of Bayesian procedures), making the Bayesian formalism a central technique in such areas of frequentist inference as parameter estimation, hypothesis testing, and computing confidence intervals.[15][16][17] For example:
Model selection[edit]Applications[edit]Computer applications[edit]
Bayesian inference has applications in artificial intelligence and expert systems. Bayesian inference techniques have been a fundamental part of computerized pattern recognition techniques since the late 1950s. There is also an ever-growing connection between Bayesian methods and simulation-based Monte Carlo techniques since complex models cannot be processed in closed form by a Bayesian analysis, while a graphical model structure may allow for efficient simulation algorithms like the Gibbs sampling and other MetropolisâHastings algorithm schemes.[22] Recently[when?] Bayesian inference has gained popularity among the phylogenetics community for these reasons; a number of applications allow many demographic and evolutionary parameters to be estimated simultaneously.
As applied to statistical classification, Bayesian inference has been used in recent years to develop algorithms for identifying e-mail spam. Applications which make use of Bayesian inference for spam filtering include CRM114, DSPAM, Bogofilter, SpamAssassin, SpamBayes, Mozilla, XEAMS, and others. Spam classification is treated in more detail in the article on the naive Bayes classifier.
Solomonoff's Inductive inference is the theory of prediction based on observations; for example, predicting the next symbol based upon a given series of symbols. The only assumption is that the environment follows some unknown but computable probability distribution. It is a formal inductive framework that combines two well-studied principles of inductive inference: Bayesian statistics and Occamâs Razor.[23][unreliable source?] Solomonoff's universal prior probability of any prefix p of a computable sequence x is the sum of the probabilities of all programs (for a universal computer) that compute something starting with p. Given some p and any computable but unknown probability distribution from which x is sampled, the universal prior and Bayes' theorem can be used to predict the yet unseen parts of x in optimal fashion.[24][25]
Bioinformatic applications[edit]
Bayesian inference has been applied in different Bioinformatics applications, including differentially gene expression analysis[26][27], single-cell classification[28], cancer subtyping[29], and etc.
In the courtroom[edit]
Bayesian inference can be used by jurors to coherently accumulate the evidence for and against a defendant, and to see whether, in totality, it meets their personal threshold for 'beyond a reasonable doubt'.[30][31][32] Bayes' theorem is applied successively to all evidence presented, with the posterior from one stage becoming the prior for the next. The benefit of a Bayesian approach is that it gives the juror an unbiased, rational mechanism for combining evidence. It may be appropriate to explain Bayes' theorem to jurors in odds form, as betting odds are more widely understood than probabilities. Alternatively, a logarithmic approach, replacing multiplication with addition, might be easier for a jury to handle.
If the existence of the crime is not in doubt, only the identity of the culprit, it has been suggested that the prior should be uniform over the qualifying population.[33] For example, if 1,000 people could have committed the crime, the prior probability of guilt would be 1/1000.
The use of Bayes' theorem by jurors is controversial. In the United Kingdom, a defence expert witness explained Bayes' theorem to the jury in R v Adams. The jury convicted, but the case went to appeal on the basis that no means of accumulating evidence had been provided for jurors who did not wish to use Bayes' theorem. The Court of Appeal upheld the conviction, but it also gave the opinion that 'To introduce Bayes' Theorem, or any similar method, into a criminal trial plunges the jury into inappropriate and unnecessary realms of theory and complexity, deflecting them from their proper task.'
Gardner-Medwin[34] argues that the criterion on which a verdict in a criminal trial should be based is not the probability of guilt, but rather the probability of the evidence, given that the defendant is innocent (akin to a frequentistp-value). He argues that if the posterior probability of guilt is to be computed by Bayes' theorem, the prior probability of guilt must be known. This will depend on the incidence of the crime, which is an unusual piece of evidence to consider in a criminal trial. Consider the following three propositions:
Gardner-Medwin argues that the jury should believe both A and not-B in order to convict. A and not-B implies the truth of C, but the reverse is not true. It is possible that B and C are both true, but in this case he argues that a jury should acquit, even though they know that they will be letting some guilty people go free. See also Lindley's paradox.
Bayesian epistemology[edit]
Bayesian epistemology is a movement that advocates for Bayesian inference as a means of justifying the rules of inductive logic.
Karl Popper and David Miller have rejected the idea of Bayesian rationalism, i.e. using Bayes rule to make epistemological inferences:[35] It is prone to the same vicious circle as any other justificationist epistemology, because it presupposes what it attempts to justify. According to this view, a rational interpretation of Bayesian inference would see it merely as a probabilistic version of falsification, rejecting the belief, commonly held by Bayesians, that high likelihood achieved by a series of Bayesian updates would prove the hypothesis beyond any reasonable doubt, or even with likelihood greater than 0.
Other[edit]
Bayes and Bayesian inference[edit]
The problem considered by Bayes in Proposition 9 of his essay, 'An Essay towards solving a Problem in the Doctrine of Chances', is the posterior distribution for the parameter a (the success rate) of the binomial distribution.[citation needed]
History[edit]
The term Bayesian refers to Thomas Bayes (1702â1761), who proved a special case of what is now called Bayes' theorem. However, it was Pierre-Simon Laplace (1749â1827) who introduced a general version of the theorem and used it to approach problems in celestial mechanics, medical statistics, reliability, and jurisprudence.[41] Early Bayesian inference, which used uniform priors following Laplace's principle of insufficient reason, was called 'inverse probability' (because it infers backwards from observations to parameters, or from effects to causes[42]). After the 1920s, 'inverse probability' was largely supplanted by a collection of methods that came to be called frequentist statistics.[42]
In the 20th century, the ideas of Laplace were further developed in two different directions, giving rise to objective and subjective currents in Bayesian practice. In the objective or 'non-informative' current, the statistical analysis depends on only the model assumed, the data analyzed,[43] and the method assigning the prior, which differs from one objective Bayesian to another objective Bayesian. In the subjective or 'informative' current, the specification of the prior depends on the belief (that is, propositions on which the analysis is prepared to act), which can summarize information from experts, previous studies, etc.
In the 1980s, there was a dramatic growth in research and applications of Bayesian methods, mostly attributed to the discovery of Markov chain Monte Carlo methods, which removed many of the computational problems, and an increasing interest in nonstandard, complex applications.[44] Despite growth of Bayesian research, most undergraduate teaching is still based on frequentist statistics.[45] Nonetheless, Bayesian methods are widely accepted and used, such as for example in the field of machine learning.[46]
See also[edit]
References[edit]Citations[edit]
Sources[edit]
Further reading[edit]
Elementary[edit]
The following books are listed in ascending order of probabilistic sophistication:
Intermediate or advanced[edit]
External links[edit]
Retrieved from 'https://en.wikipedia.org/w/index.php?title=Bayesian_inference&oldid=904457488'
You don't have to know a lot about probability theory to use a Bayesian probability model for financial forecasting. The Bayesian method can help you refine probability estimates using an intuitive process.
Any mathematically-based topic can be taken to complex depths, but this one doesn't have to be.
How It's Used
The way that Bayesian probability is used in corporate America is dependent on a degree of belief rather than historical frequencies of identical or similar events. The model is versatile, though. You can incorporate your beliefs based on frequency into the model.
The following uses the rules and assertions of the school of thought within Bayesian probability that pertains to frequency rather than subjectivity. The measurement of knowledge that is being quantified is based on historical data. This view is particularly helpful in financial modeling.
About Bayes' Theorem
The particular formula from Bayesian probability we are going to use is called Bayes' Theorem, sometimes called Bayes' formula or Bayes' rule. This rule is most often used to calculate what is called the posterior probability. The posterior probability is the conditional probability of a future uncertain event that is based upon relevant evidence relating to it historically.
In other words, if you gain new information or evidence and you need to update the probability of an event occurring, you can use Bayes' Theorem to estimate this new probability.
The formula is:
P(Aâ£B)=P(Aâ©B)P(B)=P(A)ÃP(Bâ£A)P(B)where:P(A)=Probability of A occurring, called theprior probabilityP(Aâ£B)=Conditional probability of A giventhat B occursP(Bâ£A)=Conditional probability of B giventhat A occursP(B)=Probability of B occurringbegin{aligned} &P (A | B) = frac{ P ( A cap B ) }{ P ( B ) } = frac{ P ( A ) times P ( B | A ) }{ P ( B ) } &textbf{where:} &P(A) = text{Probability of A occurring, called the} &text{prior probability} &P(A|B) = text{Conditional probability of A given} &text{that B occurs} &P(B|A) = text{Conditional probability of B given} &text{that A occurs} &P(B) = text{Probability of B occurring} end{aligned}âP(Aâ£B)=P(B)P(Aâ©B)â=P(B)P(A)ÃP(Bâ£A)âwhere:P(A)=Probability of A occurring, called theprior probabilityP(Aâ£B)=Conditional probability of A giventhat B occursP(Bâ£A)=Conditional probability of B giventhat A occursP(B)=Probability of B occurringâ
P(A|B) is the posterior probability due to its variable dependency on B. This assumes that A is not independent of B.
If we are interested in the probability of an event of which we have prior observations; we call this the prior probability. We'll deem this event A, and its probability P(A). If there is a second event that affects P(A), which we'll call event B, then we want to know what the probability of A is given that B has occurred.
In probabilistic notation, this is P(A|B) and is known as posterior probability or revised probability. This is because it has occurred after the original event, hence the post in posterior.
This is how Bayes' theorem uniquely allows us to update our previous beliefs with new information. The example below will help you see how it works in a concept that is related to an equity market.
An Example
Let's say we want to know how a change in interest rates would affect the value of a stock market index.
A vast trove of historical data is available for all the major stock market indexes, so you should have no problem finding the outcomes for these events. For our example, we will use the data below to find out how a stock market index will react to a rise in interest rates.
Here:
P(SI) = the probability of the stock index increasing
P(SD) = the probability of the stock index decreasing P(ID) = the probability of interest rates decreasing P(II) = the probability of interest rates increasing
So the equation will be:
P(SDâ£II)=P(SD)ÃP(IIâ£SD)P(II)begin{aligned} &P (SD | II) = frac{ P ( SD ) times P ( II | SD ) }{ P ( II ) } end{aligned}âP(SDâ£II)=P(II)P(SD)ÃP(IIâ£SD)ââ
Plugging in our numbers we get the following:
P(SDâ£II)=(1,1502,000)Ã(9501,150)(1,0002,000)=0.575Ã0.8260.5=0.474950.5=0.9499â95%begin{aligned} P (SD | II) &= frac{ left ( frac{ 1,150 }{ 2,000 } right ) times left ( frac { 950 }{ 1,150 } right ) }{ left ( frac { 1,000 }{ 2,000 } right ) } &= frac{ 0.575 times 0.826 }{ 0.5 } &= frac{ 0.47495 }{ 0.5 } &= 0.9499 approx 95% end{aligned}P(SDâ£II)â=(2,0001,000â)(2,0001,150â)Ã(1,150950â)â=0.50.575Ã0.826â=0.50.47495â=0.9499â95%â
The table shows, the stock index decreased in 1,150 out of 2,000 observations. This is the prior probability based on historical data, which in this example is 57.5% (1150/2000).
This probability doesn't take into account any information about interest rates and is the one we wish to update. After updating this prior probability with information that interest rates have risen leads us to update the probability of the stock market decreasing from 57.5% to 95%. Therefore, 95% is the posterior probability.
Modeling with Bayes' Theorem
As seen above, we can use the outcome of historical data to base the beliefs we use to derive newly updated probabilities.
This example can be extrapolated to individual companies by using changes within their own balance sheets, bonds given changes in credit rating, and many other examples.
So, what if one does not know the exact probabilities but has only estimates? This is where the subjective view comes strongly into play.
Many people put great emphasis on the estimates and simplified probabilities given by experts in their field. This also gives us the ability to confidently produce new estimates for new and more complicated questions introduced by the inevitable roadblocks in financial forecasting.
Instead of guessing, we can now use Bayes' Theorem if we have the right information with which to start.
When to Apply Bayes' Theorem
Changing interest rates can greatly affect the value of particular assets. The changing value of assets can therefore greatly affect the value of particular profitability and efficiency ratios used to proxy a company's performance. Estimated probabilities are widely found relating to systematic changes in interest rates and thus can be used effectively in Bayes' Theorem.
We can also apply the process to a company's net income stream. Lawsuits, changes in the prices of raw materials, and many other things can influence a company's net income.
By using probability estimates relating to these factors, we can apply Bayes' Theorem to figure out what is important to us. Once we find the deduced probabilities that we are looking for, it is a simple application of mathematical expectancy and result forecasting to quantify the financial probabilities.
Using a myriad of related probabilities, we can deduce the answer to rather complex questions with one simple formula. These methods are well accepted and time-tested. Their use in financial modeling can be helpful if applied properly.
Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |