Autoencoders

Duration: 7 min

This module delves into the world of autoencoders, a type of neural network designed for unsupervised learning. Autoencoders are particularly useful for tasks like dimensionality reduction, anomaly detection, and data denoising. Understanding autoencoders is crucial for leveraging neural networks in various applications where labeled data is scarce or unavailable.

Understanding Autoencoders

Autoencoders consist of an encoder and a decoder. The encoder compresses the input into a latent-space representation, and the decoder reconstructs the input from this representation. The goal is to minimize the difference between the input and the reconstruction, typically using a loss function like mean squared error.

import tensorflow as tf
from tensorflow.keras import layers

# Define a simple autoencoder
input_dim = 784
encoding_dim = 32

input_img = tf.keras.Input(shape=(input_dim,))
encoded = layers.Dense(encoding_dim, activation='relu')(input_img)
decoded = layers.Dense(input_dim, activation='sigmoid')(encoded)

autoencoder = tf.keras.Model(input_img, decoded)
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')

# Example usage
import numpy as np
x_train = np.random.random((1000, 784))
x_test = np.random.random((100, 784))

autoencoder.fit(x_train, x_train, epochs=50, batch_size=256, shuffle=True, validation_data=(x_test, x_test))

Try it in Google Colab:

Output will be the training and validation loss over epochs. Exact values will vary due to randomness in the data and model initialization.

Convolutional Autoencoders

Convolutional autoencoders use convolutional layers in both the encoder and decoder. They are particularly effective for image data, as convolutional layers can capture spatial hierarchies in the data. The encoder typically consists of convolutional and pooling layers, while the decoder uses convolutional and upsampling layers.

import tensorflow as tf
from tensorflow.keras import layers

input_img = tf.keras.Input(shape=(28, 28, 1))

# Encoder
x = layers.Conv2D(16, (3, 3), activation='relu', padding='same')(input_img)
x = layers.MaxPooling2D((2, 2), padding='same')(x)
encoded = layers.Conv2D(8, (3, 3), activation='relu', padding='same')(x)

# Decoder
x = layers.Conv2DTranspose(8, (3, 3), strides=2, activation='relu', padding='same')(encoded)
x = layers.Conv2DTranspose(16, (3, 3), activation='relu', padding='same')(x)
decoded = layers.Conv2DTranspose(1, (3, 3), activation='sigmoid', padding='same')(x)

autoencoder = tf.keras.Model(input_img, decoded)
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')

# Example usage
import numpy as np
x_train = np.random.random((1000, 28, 28, 1))
x_test = np.random.random((100, 28, 28, 1))

autoencoder.fit(x_train, x_train, epochs=50, batch_size=128, shuffle=True, validation_data=(x_test, x_test))

💡 Tip: When training autoencoders, ensure that the loss function aligns with the data type. For binary data, use 'binary_crossentropy', and for continuous data, use'mean_squared_error'.