Probability Theory Basics

Duration: 15 min

This module delves into the fundamental principles of probability theory, which is essential for understanding and developing artificial intelligence systems. Probability theory provides the mathematical framework for dealing with uncertainty, which is a key aspect of AI. It enables us to make informed decisions under uncertain conditions, which is crucial for tasks like prediction, decision-making, and pattern recognition in AI.

Visual: Probability Concepts

Sample Space (All Outcomes)
┌──────────────────────────────┐
│  Coin Flip: {H, T}           │
│  Probability: P(H) = 0.5     │
│                              │
│  Dice Roll: {1,2,3,4,5,6}    │
│  Probability: P(3) = 1/6     │
└──────────────────────────────┘

Conditional Probability: P(A|B)
Event A given Event B occurred

Key Concepts Table

Understanding Events and Sample Spaces

In probability theory, an event is a set of outcomes of an experiment, and the sample space is the set of all possible outcomes. For example, when flipping a coin, the sample space is {Heads, Tails}, and getting Heads is an event. Understanding how to define and work with these concepts is crucial for calculating probabilities and making probabilistic inferences.

import itertools

# Define the sample space for a single coin flip
coin_flip = ['Heads', 'Tails']

# Define the sample space for two coin flips
sample_space = list(itertools.product(coin_flip, repeat=2))

# Print the sample space
print('Sample Space:', sample_space)

Try it in Google Colab:

Sample Space: [('Heads', 'Heads'), ('Heads', 'Tails'), ('Tails', 'Heads'), ('Tails', 'Tails')]

Calculating Probabilities

The probability of an event is a measure of the likelihood that the event will occur. It is calculated as the number of favorable outcomes divided by the total number of possible outcomes. For example, the probability of getting Heads in a single coin flip is 0.5, since there is one favorable outcome (Heads) out of two possible outcomes (Heads or Tails).

import itertools

# Define the sample space for a single coin flip
coin_flip = ['Heads', 'Tails']

# Define the sample space for two coin flips
sample_space = list(itertools.product(coin_flip, repeat=2))

# Count the number of favorable outcomes for getting at least one Heads
favorable_outcomes = [outcome for outcome in sample_space if 'Heads' in outcome]

# Calculate the probability
probability = len(favorable_outcomes) / len(sample_space)

# Print the probability
print('Probability of getting at least one Heads:', probability)

Probability of getting at least one Heads: 0.75

💡 Tip: When calculating probabilities, ensure that the sample space is correctly defined and that all outcomes are equally likely. A common pitfall is to overlook some outcomes or to assume unequal probabilities without justification.

Practice Quizzes

Quiz 1: If P(A) = 0.3 and P(B) = 0.4, and A and B are independent, what is P(A∩B)?

• 0.7
• [✓] 0.12
• 0.1
• 0.35

Quiz 2: What does Bayes' Theorem allow us to calculate?

• The probability of an event
• [✓] The inverse probability given new evidence
• The sum of probabilities
• The variance of a distribution

Quiz 3: What is conditional probability P(A|B)?

• Probability of A or B
• [✓] Probability of A given B has occurred
• Probability of A and B
• Probability of B given A