Expectation and Variance

Duration: 5 min

This module delves into the concepts of expectation and variance, which are fundamental in understanding the behavior of random variables. These concepts are crucial for assessing the performance of machine learning models and making informed decisions based on data.

Expectation

Expectation, or the expected value, is a measure of the central tendency of a random variable. It represents the average outcome if an experiment is repeated many times. Mathematically, for a discrete random variable X with possible values x_1, x_2,..., x_n and corresponding probabilities p_1, p_2,..., p_n, the expectation E(X) is given by the sum of each value multiplied by its probability: E(X) = Σ(x_i * p_i).

import numpy as np

# Define the values and probabilities
values = np.array([1, 2, 3, 4, 5])
probabilities = np.array([0.1, 0.2, 0.3, 0.2, 0.2])

# Calculate the expectation
expectation = np.sum(values * probabilities)
print('Expectation:', expectation)

Try it in Google Colab:

Expectation: 3.0

Variance

Variance measures the spread of a random variable's possible values. It quantifies how much the values deviate from the expected value. For a random variable X with expectation E(X), the variance Var(X) is defined as E[(X - E(X))^2]. A higher variance indicates that the data points are more spread out from the mean.

import numpy as np

# Define the values and probabilities
values = np.array([1, 2, 3, 4, 5])
probabilities = np.array([0.1, 0.2, 0.3, 0.2, 0.2])

# Calculate the expectation
expectation = np.sum(values * probabilities)

# Calculate the variance
variance = np.sum(probabilities * (values - expectation)**2)
print('Variance:', variance)

💡 Tip: When calculating variance, ensure that the probabilities sum to 1 to avoid incorrect results.

Key Concepts

Check Your Understanding