Building LSTM Models for Time Series

Duration: 5 min

This module covers the process of building Long Short-Term Memory (LSTM) models for time series forecasting. LSTMs are a type of recurrent neural network (RNN) that can learn long-term dependencies, making them particularly effective for time series data. Understanding how to build and train LSTM models is crucial for accurate time series forecasting.

Understanding LSTM Networks

LSTM networks are designed to avoid the long-term dependency problem, which is a common issue in traditional RNNs. They achieve this by incorporating a cell state, which acts as a 'conveyor belt' for information to flow through the network. This allows LSTMs to remember information for long periods, making them ideal for time series forecasting.

import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense

# Generate some sample time series data
data = np.sin(np.arange(0, 100, 0.1))

# Prepare the data for LSTM
def create_dataset(data, time_step=1):
    X, Y = [], []
    for i in range(len(data)-time_step-1):
        X.append(data[i:(i+time_step)])
        Y.append(data[i + time_step])
    return np.array(X), np.array(Y)

time_step = 10
X, y = create_dataset(data, time_step)

# Reshape input to be [samples, time steps, features]
X = X.reshape(X.shape[0], X.shape[1], 1)

# Create and fit the LSTM network
model = Sequential()
model.add(LSTM(50, return_sequences=True, input_shape=(time_step, 1)))
model.add(LSTM(50))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X, y, epochs=100, batch_size=32, verbose=1)

# Make predictions
train_predict = model.predict(X)
print(train_predict[:5])

Try it in Google Colab:

[[-0.00413739]
 [-0.0037912 ]
 [-0.00344483]
 [-0.00309828]
 [-0.00275155]]

Training and Evaluating LSTM Models

Training an LSTM model involves feeding the model sequences of data and adjusting the weights to minimize the loss function. Evaluation of the model is typically done using metrics such as Mean Squared Error (MSE) or Mean Absolute Error (MAE). It's important to split the data into training and testing sets to ensure the model generalizes well to unseen data.

from sklearn.metrics import mean_squared_error
from tensorflow.keras.models import load_model

# Split the data into training and testing sets
train_size = int(len(data) * 0.67)
test_size = len(data) - train_size
train, test = data[0:train_size], data[train_size:len(data)]

# Prepare the training dataset
X_train, y_train = create_dataset(train, time_step)
X_train = X_train.reshape(X_train.shape[0], X_train.shape[1], 1)

# Prepare the test dataset
X_test, y_test = create_dataset(test, time_step)
X_test = X_test.reshape(X_test.shape[0], X_test.shape[1], 1)

# Refit the model on the training set
model.fit(X_train, y_train, epochs=100, batch_size=32, verbose=1)

# Make predictions on the test set
test_predict = model.predict(X_test)

# Calculate root mean squared error
train_score = mean_squared_error(y_train, model.predict(X_train))
test_score = mean_squared_error(y_test, test_predict)
print(f'Train Score: {train_score:.4f} MSE')
print(f'Test Score: {test_score:.4f} MSE')

💡 Tip: When training LSTM models, be mindful of overfitting. Use techniques such as early stopping, dropout layers, and validation sets to ensure your model generalizes well.

Key Concepts

Check Your Understanding