pytorch-fundamentals

Duration: 6 min

This module covers the essential concepts and tools needed to get started with deep learning using PyTorch. Understanding these fundamentals is crucial as they form the building blocks for more advanced topics in deep learning. By mastering these basics, you will be well-equipped to build, train, and deploy neural networks efficiently.

Visual: PyTorch Workflow

┌─────────────────────────────────┐
│  1. Create Tensors              │
│     torch.tensor([1,2,3])       │
└────────────┬────────────────────┘
             │
┌────────────▼────────────────────┐
│  2. Define Model                │
│     nn.Linear, nn.Conv2d        │
└────────────┬────────────────────┘
             │
┌────────────▼────────────────────┐
│  3. Forward Pass                │
│     output = model(input)       │
└────────────┬────────────────────┘
             │
┌────────────▼────────────────────┐
│  4. Compute Loss                │
│     loss = criterion(out, tgt)  │
└────────────┬────────────────────┘
             │
┌────────────▼────────────────────┐
│  5. Backward Pass               │
│     loss.backward()             │
└────────────┬────────────────────┘
             │
┌────────────▼────────────────────┐
│  6. Update Weights              │
│     optimizer.step()            │
└─────────────────────────────────┘

Key Concepts Table

Component	Purpose	Example
Tensor	Multi-dimensional array	torch.tensor([1,2,3])
Device	CPU or GPU	torch.device("cuda")
Autograd	Automatic differentiation	requires_grad=True
Module	Neural network layer	nn.Linear(10, 5)
Optimizer	Weight updater	torch.optim.Adam()
Loss Function	Error measure	nn.CrossEntropyLoss()
DataLoader	Batch iterator	DataLoader(dataset)

Understanding Tensors

Tensors are the core data structure in PyTorch, similar to NumPy arrays but with additional capabilities. They are used to represent and manipulate data in deep learning models. Tensors can be created with various methods, including from data, random numbers, or existing data structures. They support a wide range of mathematical operations, which are essential for building neural networks.

import torch

# Creating a tensor from a list
tensor_from_list = torch.tensor([1.0, 2.0, 3.0])
print('Tensor from list:', tensor_from_list)

# Creating a tensor with random numbers
random_tensor = torch.rand(3, 3)
print('Random tensor:', random_tensor)

# Creating a tensor with zeros
zeros_tensor = torch.zeros(2, 3)
print('Zeros tensor:', zeros_tensor)

Try it in Google Colab:

Tensor from list: tensor([1., 2., 3.])
Random tensor: 
tensor([[0.0962, 0.3682, 0.9647],
        [0.7917, 0.1463, 0.7895],
        [0.6308, 0.7215, 0.5449]])
Zeros tensor: 
tensor([[0., 0., 0.],
        [0., 0., 0.]])

Building Neural Networks

In PyTorch, neural networks are built using the torch.nn module, which provides a variety of layers and loss functions. The nn.Module class is used to define the architecture of the network. By subclassing nn.Module and implementing the forward method, you can define how the data flows through the network. This modular approach allows for easy experimentation and customization of network architectures.

import torch
import torch.nn as nn
import torch.optim as optim

# Define a simple neural network
class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(10, 5)
        self.fc2 = nn.Linear(5, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

# Instantiate the network
net = SimpleNN()

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(net.parameters(), lr=0.01)

print('Network architecture:', net)

💡 Tip: When defining your network, ensure that the dimensions of your layers match. Mismatched dimensions can lead to errors during training.

❓ What is the primary function of the `torch.nn` module in PyTorch?

Creating tensors Defining neural network architectures Optimizing neural networks Visualizing data

❓ What does the `forward` method in a PyTorch `nn.Module` subclass do?

Initializes the network Defines the forward pass of the network Computes gradients Optimizes the network parameters

Practice Quizzes

Quiz 1: What is a PyTorch tensor?

A weight matrix
[✓] A multi-dimensional array
A loss function
An optimizer

Quiz 2: What does autograd do?

Loads data
[✓] Computes gradients automatically
Trains model
Saves checkpoints

Quiz 3: What is the purpose of an optimizer?

Compute loss
[✓] Update weights based on gradients
Load data
Evaluate model

Quiz 4: How do you move a tensor to GPU?

tensor.gpu()
[✓] tensor.to('cuda')
tensor.device('cuda')
tensor.cuda()