Markov Chain Monte Carlo Methods

Duration: 5 min

This module delves into Markov Chain Monte Carlo (MCMC) methods, a class of algorithms for sampling from probability distributions based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. Understanding MCMC is crucial for machine learning applications, especially in Bayesian inference and complex model simulations.

Metropolis-Hastings Algorithm

The Metropolis-Hastings algorithm is a widely used MCMC method that allows us to generate samples from a probability distribution that we cannot sample from directly. It works by proposing moves in the state space and accepting or rejecting these moves based on a certain acceptance probability, ensuring that the Markov chain converges to the target distribution.

import numpy as np

def metropolis_hastings(target_dist, proposal_dist, initial_state, num_samples):
    samples = [initial_state]
    current_state = initial_state
    for _ in range(num_samples):
        proposed_state = proposal_dist(current_state)
        acceptance_ratio = target_dist(proposed_state) / target_dist(current_state)
        if np.random.rand() < min(1, acceptance_ratio):
            current_state = proposed_state
        samples.append(current_state)
    return samples

# Example usage
target_dist = lambda x: np.exp(-x**2 / 2)  # Gaussian distribution
proposal_dist = lambda x: np.random.normal(x, 1)  # Normal proposal distribution
samples = metropolis_hastings(target_dist, proposal_dist, 0, 1000)
print(samples[:10])

Try it in Google Colab:

[0, -0.616519148992323, -0.764896698864358, -1.011643618582294, -0.746348992567007, -0.633764476835605, -0.662136050399051, -0.942686501969756, -0.735845451644559, -0.584100369892346]

Gibbs Sampling

Gibbs sampling is another MCMC method that is particularly useful when the target distribution is multivariate. It works by iteratively sampling each variable from its conditional distribution given the current values of all other variables. This method is efficient when the conditional distributions are easy to sample from.

import numpy as np

def gibbs_sampling(num_samples):
    samples = np.zeros((num_samples, 2))
    x, y = 0, 0
    for i in range(num_samples):
        x = np.random.normal(y, 1)
        y = np.random.normal(x, 1)
        samples[i] = [x, y]
    return samples

# Example usage
samples = gibbs_sampling(1000)
print(samples[:10])

💡 Tip: Ensure that the proposal distribution in Metropolis-Hastings and the conditional distributions in Gibbs sampling are well-tuned to the target distribution to achieve efficient convergence.

Key Concepts

Check Your Understanding