introduction-to-deep-learning

Duration: 6 min

This module provides an introduction to deep learning, a subset of machine learning, which is revolutionizing the field of artificial intelligence. We will explore the fundamental concepts, architectures, and practical applications of deep learning using PyTorch, a powerful and flexible deep learning framework.

Visual: Neural Network Layers

Input Layer    Hidden Layers    Output Layer
    ●              ●                ●
    ●              ●                ●
    ●              ●                ●
    ●              ●                ●
    
    x₁            h₁,h₂,h₃          y
    x₂            (neurons)      (prediction)
    x₃
    x₄
    
Forward Pass: x → h → y
Backward Pass: ∂L/∂w ← gradients

Key Concepts Table

Concept	Definition	Role
Neuron	Computational unit	Processes input
Weight	Connection strength	Learned parameter
Bias	Offset term	Learned parameter
Activation	Non-linearity	Introduces complexity
Forward Pass	Input → Output	Prediction
Loss	Error measure	Optimization target
Backprop	Gradient computation	Weight updates

Understanding Neural Networks

Neural Network Architecture

Neural networks are the core of deep learning, mimicking the way the human brain processes information. They consist of layers of interconnected nodes (neurons) that can learn from data. Each neuron takes input, applies a weight, sums it, and passes it through an activation function to produce an output. This process is repeated across multiple layers, allowing the network to learn complex patterns.

import torch
import torch.nn as nn
import torch.optim as optim

# Define a simple neural network
class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(10, 5)
        self.fc2 = nn.Linear(5, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

# Instantiate the model
model = SimpleNN()

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Example input
input_data = torch.randn(1, 10)

# Forward pass
output = model(input_data)
print(output)

Try it in Google Colab:

tensor([-0.0123], grad_fn=<ThAddmmBackward>)

Training a Neural Network

Training a neural network involves adjusting the weights of the network to minimize the error between the predicted output and the actual output. This is achieved using an optimization algorithm like Stochastic Gradient Descent (SGD). The process includes forward propagation (computing the output), calculating the loss, and backward propagation (updating the weights).

import torch
import torch.nn as nn
import torch.optim as optim

# Define a simple neural network
class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(10, 5)
        self.fc2 = nn.Linear(5, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

# Instantiate the model
model = SimpleNN()

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Example input and target
input_data = torch.randn(1, 10)
target = torch.tensor([0.5])

# Forward pass
output = model(input_data)

# Calculate loss
loss = criterion(output, target)
print(f'Loss: {loss.item()}')

# Backward pass and optimization
optimizer.zero_grad()
loss.backward()
optimizer.step()

💡 Tip: Ensure that your input data is properly normalized and preprocessed before training to improve convergence speed and model performance.

❓ What is the primary function of the activation function in a neural network?

To compute the weighted sum of inputs To introduce non-linearity into the model To update the weights during backpropagation To normalize the input data

❓ Which of the following is a common optimization algorithm used in training neural networks?

Gradient Descent Random Walk Mean Squared Error Principal Component Analysis

Practice Quizzes

Quiz 1: What is the purpose of activation functions?

Store data
[✓] Introduce non-linearity
Compute loss
Normalize weights

Quiz 2: What does backpropagation compute?

Predictions
[✓] Gradients for weight updates
Loss values
Activations

Quiz 3: What is a neuron?

A biological cell
[✓] A computational unit in NN
A weight matrix
An activation function

Quiz 4: Why do we need hidden layers?

For storage
[✓] To learn complex patterns
To reduce computation
To normalize data