Bayes' Theorem: Conditional Probability with Priors and Likelihood

Concept Overview

This question tests the application of Bayes' Theorem, a fundamental concept in probability theory that allows us to update the probability of a hypothesis based on new evidence. It's particularly useful when we have prior beliefs about an event and want to calculate the posterior probability after observing some data. The theorem elegantly relates conditional probabilities and is crucial for understanding how to incorporate likelihoods and prior distributions.

Step 1: Understand Bayes' Theorem Bayes' Theorem provides a way to calculate a conditional probability $P(A|B)$ given the probabilities $P(B|A)$, $P(A)$, and $P(B)$. The formula is:

Here, $P(A)$ is the prior probability of event A, $P(B|A)$ is the likelihood of observing event B given that A has occurred, $P(B|A^c)$ is the likelihood of observing event B given that A has not occurred, and $P(B)$ is the total probability of event B. $P(A|B)$ is the posterior probability of A given B.

Step 2: Express the denominator $P(B)$ using the Law of Total Probability The probability of event B, $P(B)$, can be expanded using the law of total probability. If we have a partition of the sample space (e.g., events A and $A^c$, where $A^c$ is the complement of A), then $P(B)$ can be written as:

This step is crucial because often $P(B)$ is not directly given and must be calculated from the conditional probabilities and prior probabilities of the events in the partition.

Step 3: Substitute the expression for $P(B)$ into Bayes' Theorem By substituting the expanded form of $P(B)$ from Step 2 into the Bayes' Theorem formula from Step 1, we get the most commonly used form for problems involving a binary partition (like A and $A^c$):

This form directly uses the prior probabilities $P(A)$ and $P(A^c)$ and the likelihoods $P(B|A)$ and $P(B|A^c)$ to find the posterior probability $P(A|B)$.

Step 4: Identify the given probabilities in a problem When solving a problem, carefully identify what each given probability represents:

• Prior Probability ($P(A)$): The initial belief in event A before any new evidence is considered.
• Likelihood ($P(B|A)$): The probability of observing the evidence (event B) given that the hypothesis (event A) is true.
• Complementary Prior ($P(A^c)$): The probability that event A does not occur, which is $1 - P(A)$.
• Complementary Likelihood ($P(B|A^c)$): The probability of observing the evidence (event B) given that the hypothesis (event A) is false.

Step 5: Calculate the posterior probability Plug the identified values into the formula derived in Step 3 and perform the arithmetic calculation to find the posterior probability $P(A|B)$. This value represents the updated belief in event A after considering the evidence B.

Let's consider a hypothetical example: Suppose a factory produces items, and 1% are defective (event A). A new testing machine (event B) correctly identifies 95% of defective items (likelihood $P(B|A) = 0.95$) but also incorrectly flags 2% of non-defective items as defective (likelihood $P(B|A^c) = 0.02$). If an item is flagged as defective by the machine, what is the probability it is actually defective?

Here, $P(A) = 0.01$ (prior probability of being defective). $P(A^c) = 1 - P(A) = 1 - 0.01 = 0.99$ (prior probability of not being defective). $P(B|A) = 0.95$ (likelihood of machine flagging a defective item). $P(B|A^c) = 0.02$ (likelihood of machine flagging a non-defective item).

Using the formula from Step 3:

So, even though the machine flags an item, there's only about a 32.42% chance it's actually defective, due to the low prior probability of defects and the machine's false positive rate.

Key Takeaways:

• Bayes' Theorem updates prior probabilities into posterior probabilities using new evidence (likelihoods).
• The Law of Total Probability is essential for calculating the probability of the evidence, $P(B)$, when it's not directly given.
• Carefully identify prior probabilities ($P(A)$) and likelihoods ($P(B|A)$, $P(B|A^c)$) from the problem statement.
• The formula $P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B|A) \cdot P(A) + P(B|A^c) \cdot P(A^c)}$ is a practical form for binary hypotheses.

Answer: The application of Bayes' Theorem involves updating prior beliefs with observed evidence using the formula $P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$, where $P(B)$ is often expanded using the Law of Total Probability.