Neural Networks with Scikit-Learn

Duration: 5 min

This module delves into the implementation of neural networks using Scikit-Learn, a powerful library for machine learning in Python. Neural networks are a class of models inspired by the human brain, capable of learning complex patterns in data. Understanding how to implement and tune neural networks is crucial for tackling a wide range of machine learning problems, from image recognition to natural language processing.

Understanding MLPClassifier

The MLPClassifier in Scikit-Learn is a implementation of a Multi-Layer Perceptron (MLP), a type of feedforward neural network. It consists of an input layer, one or more hidden layers, and an output layer. Each layer is fully connected to the next one. MLPs are versatile and can be used for both regression and classification tasks. The key parameters to tune include the number of hidden layers, the number of neurons in each layer, and the activation function.

from sklearn.neural_network import MLPClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

# Generate a random n-class classification problem
X, y = make_classification(n_samples=100, n_features=20, n_informative=2, n_redundant=10, n_classes=2, random_state=1)

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)

# Create an MLPClassifier with 1 hidden layer of 50 neurons and 'relu' activation function
mlp = MLPClassifier(hidden_layer_sizes=(50,), activation='relu', solver='adam', max_iter=300, random_state=1)

# Train the model
mlp.fit(X_train, y_train)

# Predict on the test set
y_pred = mlp.predict(X_test)

# Print the accuracy
print(f'Accuracy: {mlp.score(X_test, y_test):.2f}')

Try it in Google Colab:

Accuracy: 0.95

Hyperparameter Tuning and Cross-Validation

Hyperparameter tuning is critical for achieving good performance with neural networks. Common hyperparameters include the number of hidden layers, the number of neurons in each layer, the activation function, the solver (optimization algorithm), and the learning rate. Cross-validation is a technique used to evaluate the model's performance on different subsets of the data, ensuring that the model generalizes well. Scikit-Learn provides tools like GridSearchCV to automate the process of hyperparameter tuning with cross-validation.

from sklearn.neural_network import MLPClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import GridSearchCV, train_test_split

# Generate a random n-class classification problem
X, y = make_classification(n_samples=100, n_features=20, n_informative=2, n_redundant=10, n_classes=2, random_state=1)

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)

# Define the parameter grid
param_grid = {
    'hidden_layer_sizes': [(50,), (100,), (50, 50)],
    'activation': ['relu', 'tanh', 'logistic'],
   'solver': ['adam','sgd']
}

# Create an MLPClassifier
mlp = MLPClassifier(max_iter=300, random_state=1)

# Set up the grid search
grid_search = GridSearchCV(mlp, param_grid, cv=5, scoring='accuracy')

# Perform the grid search
grid_search.fit(X_train, y_train)

# Print the best parameters and the best score
print(f'Best parameters: {grid_search.best_params_}')
print(f'Best score: {grid_search.best_score_:.2f}')

# Evaluate the best model on the test set
best_mlp = grid_search.best_estimator_
print(f'Test set score: {best_mlp.score(X_test, y_test):.2f}')

💡 Tip: When tuning hyperparameters, start with a coarse grid to identify promising regions of the parameter space, then refine your search in those regions. This approach is more efficient than performing a fine-grained search over the entire parameter space from the start.

Key Concepts

Check Your Understanding