Autoencoders

Duration: 8 min

This module delves into the world of autoencoders, a type of neural network used for unsupervised learning. Autoencoders are essential for tasks like dimensionality reduction, denoising, and generating new data. Understanding autoencoders will enhance your ability to work with complex datasets and improve model performance.

Understanding Autoencoders

An autoencoder consists of two main parts: an encoder and a decoder. The encoder compresses the input into a latent-space representation, while the decoder reconstructs the input from this representation. The goal is to minimize the difference between the input and the reconstruction, making autoencoders powerful tools for learning efficient data codings.

import torch
import torch.nn as nn
import torch.optim as optim

# Define the Autoencoder
class Autoencoder(nn.Module):
    def __init__(self):
        super(Autoencoder, self).__init__()
        self.encoder = nn.Sequential(
            nn.Linear(784, 128),
            nn.ReLU(),
            nn.Linear(128, 64),
            nn.ReLU(),
            nn.Linear(64, 12)
        )
        self.decoder = nn.Sequential(
            nn.Linear(12, 64),
            nn.ReLU(),
            nn.Linear(64, 128),
            nn.ReLU(),
            nn.Linear(128, 784),
            nn.Sigmoid()
        )

    def forward(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded

# Initialize the autoencoder, loss function, and optimizer
autoencoder = Autoencoder()
criterion = nn.MSELoss()
optimizer = optim.Adam(autoencoder.parameters(), lr=0.001)

# Example input (flattened 28x28 image)
input_data = torch.randn(1, 784)

# Forward pass
output = autoencoder(input_data)

# Compute loss
loss = criterion(output, input_data)
print(f'Loss: {loss.item()}')

Try it in Google Colab:

Loss: 0.123456789

Training an Autoencoder

Training an autoencoder involves feeding it input data and adjusting the weights to minimize the reconstruction error. This process typically uses a loss function like Mean Squared Error (MSE) to measure the difference between the input and the reconstructed output. Through backpropagation, the autoencoder learns to produce a more accurate reconstruction.

import torch
import torch.nn as nn
import torch.optim as optim

# Assume autoencoder is defined as in example1.py

# Training loop
num_epochs = 10
for epoch in range(num_epochs):
    optimizer.zero_grad()
    output = autoencoder(input_data)
    loss = criterion(output, input_data)
    loss.backward()
    optimizer.step()
    print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')

💡 Tip: Ensure your input data is normalized before training the autoencoder to achieve better convergence and performance.