Autoencoders Fundamentals

Duration: 5 min

This module delves into the fundamentals of autoencoders, a type of neural network used for unsupervised learning. We'll explore the architecture, training process, and applications of autoencoders, understanding why they are crucial for tasks like dimensionality reduction and anomaly detection.

Understanding Autoencoders

Autoencoders are neural networks designed to reproduce their input at the output after passing through a bottleneck layer. This forces the network to learn a compressed representation of the input data. They consist of an encoder that reduces the input to a lower-dimensional representation and a decoder that reconstructs the input from this representation.

import numpy as np
import tensorflow as tf
from tensorflow.keras import layers, models

# Define a simple autoencoder
input_dim = 784
encoding_dim = 32

input_img = layers.Input(shape=(input_dim,))
encoded = layers.Dense(encoding_dim, activation='relu')(input_img)
decoded = layers.Dense(input_dim, activation='sigmoid')(encoded)

autoencoder = models.Model(input_img, decoded)
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')

# Example usage
# x_train and x_test should be your data
# autoencoder.fit(x_train, x_train, epochs=50, batch_size=256, shuffle=True, validation_data=(x_test, x_test))

Try it in Google Colab:

Model is compiled and ready for training. Actual output will depend on the dataset used for training and validation.

Training and Evaluating Autoencoders

Training an autoencoder involves feeding it input data and adjusting the weights to minimize the difference between the input and the reconstructed output. Evaluation can be done by measuring the reconstruction error or by using the encoded representations for downstream tasks like clustering or classification.

import matplotlib.pyplot as plt

# Assuming autoencoder is trained and x_test is available
encoded_imgs = autoencoder.encoder(x_test).numpy()
decoded_imgs = autoencoder.decoder(encoded_imgs).numpy()

# Plot original and reconstructed images
num_images = 10
plt.figure(figsize=(20, 4))
for i in range(num_images):
    ax = plt.subplot(2, num_images, i + 1)
    plt.imshow(x_test[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    ax = plt.subplot(2, num_images, i + 1 + num_images)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()

💡 Tip: When training autoencoders, ensure your loss function aligns with the data distribution. For instance, use 'binary_crossentropy' for binary data and'mse' for continuous data.