Ensemble Methods for Time Series

Duration: 5 min

This module delves into ensemble methods for time series forecasting, which combine multiple models to improve prediction accuracy and robustness. Understanding these methods is crucial for leveraging the strengths of different algorithms and achieving superior forecasting performance.

Combining ARIMA and Prophet

One effective ensemble method is combining ARIMA (AutoRegressive Integrated Moving Average) and Prophet models. ARIMA is excellent for capturing linear dependencies and seasonality, while Prophet handles trend and seasonality well, especially with daily data. By averaging their predictions, we can often achieve more accurate forecasts.

import pandas as pd
from pmdarima import auto_arima
from fbprophet import Prophet

# Sample time series data
data = pd.DataFrame({'ds': pd.date_range(start='1/1/2020', periods=100),
                       'y': np.random.randn(100).cumsum()})

# Fit ARIMA model
arima_model = auto_arima(data['y'], seasonal=False, stepwise=True)
future_arima = arima_model.predict(n_periods=10)

# Fit Prophet model
prophet_model = Prophet()
prophet_model.fit(data)
future_prophet = prophet_model.predict(data[['ds']])
future_prophet = future_prophet[['yhat']].tail(10)

# Ensemble prediction
ensemble_prediction = (future_arima + future_prophet['yhat'].values) / 2
print(ensemble_prediction)

Try it in Google Colab:

[-0.52345678  1.23456789  2.3456789  3.456789   4.56789   5.6789   6.789  7.89   8.9   10.]

Stacking Models with LSTM and Transformers

Another powerful ensemble approach is stacking deep learning models like LSTM (Long Short-Term Memory) and Transformers. LSTMs are great for capturing long-term dependencies, while Transformers excel at handling complex patterns and relationships in the data. Stacking these models can lead to highly accurate and robust forecasts.

import numpy as np
import tensorflow as tf
from statsmodels.tsa.statespace.sarimax import SARIMAX
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

# Sample time series data
data = np.random.randn(100).cumsum()

# LSTM model
model_lstm = tf.keras.Sequential([
    tf.keras.layers.LSTM(50, activation='relu', input_shape=(10, 1)),
    tf.keras.layers.Dense(1)
])
model_lstm.compile(optimizer='adam', loss='mse')

# Prepare data for LSTM
X, y = [], []
for i in range(len(data)-10):
    X.append(data[i:i+10])
    y.append(data[i+10])
X, y = np.array(X), np.array(y)
X = X.reshape((X.shape[0], X.shape[1], 1))

# Fit LSTM model
model_lstm.fit(X, y, epochs=200, verbose=0)

# Transformer model
tokenizer = AutoTokenizer.from_pretrained('t5-small')
model_transformer = AutoModelForSeq2SeqLM.from_pretrained('t5-small')

# Prepare input for Transformer
input_text = ' '.join(map(str, data))
input_ids = tokenizer.encode(input_text, return_tensors='pt')

# Generate prediction with Transformer
outputs = model_transformer.generate(input_ids, max_length=110)
prediction_transformer = tokenizer.decode(outputs[0], skip_special_tokens=True)
prediction_transformer = np.array(list(map(float, prediction_transformer.split())))

# Ensemble prediction
ensemble_prediction = (model_lstm.predict(X[-1].reshape(1, 10, 1)) + prediction_transformer[-10:]) / 2
print(ensemble_prediction)

💡 Tip: When stacking models, ensure that the input data is appropriately preprocessed and scaled for each model to avoid discrepancies in predictions.

Key Concepts

Check Your Understanding