Model Training and Hyperparameter Tuning

Duration: 5 min

This module delves into the essential practices of model training and hyperparameter tuning, critical components in the machine learning lifecycle. Understanding these processes is vital for developing robust, high-performing models that generalize well to new data.

Model Training

Model training involves feeding data into a machine learning algorithm to learn the parameters that best map inputs to outputs. This process requires careful preparation of the dataset, selection of an appropriate model, and configuration of training parameters. The goal is to minimize the loss function, which measures the difference between predicted and actual values.

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

# Generate synthetic data
np.random.seed(0)
X = 2 * np.random.rand(100, 1)
y = 4 + 3 * X.squeeze() + np.random.randn(100)

# Split data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train the model
model = LinearRegression()
model.fit(X_train, y_train)

# Make predictions
y_pred = model.predict(X_test)

# Evaluate the model
mse = mean_squared_error(y_test, y_pred)
print(f'Mean Squared Error: {mse:.2f}')

Try it in Google Colab:

Mean Squared Error: 1.12

Hyperparameter Tuning

Hyperparameter tuning is the process of finding the optimal set of hyperparameters for a machine learning model. These are parameters that are set prior to the learning process and cannot be learned from the data. Techniques like Grid Search and Random Search are commonly used to explore different combinations of hyperparameters to find the best performing model.

from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestRegressor

# Define the parameter grid
param_grid = {
    'n_estimators': [50, 100, 200],
   'max_depth': [None, 10, 20, 30],
    'min_samples_split': [2, 5, 10]
}

# Initialize the model
model = RandomForestRegressor(random_state=42)

# Set up the grid search
grid_search = GridSearchCV(estimator=model, param_grid=param_grid, cv=3, scoring='neg_mean_squared_error', verbose=2, n_jobs=-1)

# Fit the grid search to the data
grid_search.fit(X_train, y_train)

# Get the best parameters and the best score
best_params = grid_search.best_params_
best_score = grid_search.best_score_
print(f'Best Parameters: {best_params}')
print(f'Best Score: {best_score:.2f}')

💡 Tip: When performing hyperparameter tuning, it's crucial to use a validation set or cross-validation to avoid overfitting the model to the training data.

Key Concepts

Check Your Understanding