Introduction to Bayesian Inference

Duration: 5 min

This module introduces Bayesian Inference, a powerful statistical method used in machine learning to update the probability estimate for a hypothesis as more evidence or information becomes available. Understanding Bayesian Inference is crucial for developing robust machine learning models that can adapt to new data.

Bayes' Theorem

Bayes' Theorem provides a way to update the probability of a hypothesis based on new evidence. It is mathematically expressed as P(H|E) = [P(E|H) * P(H)] / P(E), where P(H|E) is the posterior probability, P(E|H) is the likelihood, P(H) is the prior probability, and P(E) is the marginal likelihood.

from scipy.stats import binom

# Prior probability of a coin being fair
prior = 0.5

# Likelihood of observing 6 heads in 10 flips if the coin is fair
likelihood = binom.pmf(6, 10, 0.5)

# Marginal likelihood (normalizing constant)
marginal_likelihood = sum(binom.pmf(i, 10, 0.5) for i in range(11))

# Posterior probability
posterior = (likelihood * prior) / marginal_likelihood

print(f'Posterior probability: {posterior}')

Try it in Google Colab:

Posterior probability: 0.205078125

Bayesian Updating

Bayesian updating is the process of revising the probability of a hypothesis as new data is observed. This is done by applying Bayes' Theorem iteratively, using the posterior probability from one update as the prior for the next.

import numpy as np

# Initial prior
prior = np.array([0.1, 0.2, 0.3, 0.4])

# Likelihood of observing data given each hypothesis
likelihood = np.array([0.1, 0.2, 0.5, 0.2])

# Normalize to get the posterior
posterior = (prior * likelihood) / np.sum(prior * likelihood)

print(f'Posterior probabilities: {posterior}')

💡 Tip: When performing Bayesian updating, ensure that the likelihoods are correctly calculated and that the priors are reasonable to avoid skewed results.

Key Concepts

Check Your Understanding