Support Vector Machines (SVM)

Duration: 5 min

This module delves into Support Vector Machines (SVM), a powerful supervised learning model used for classification and regression. SVMs are particularly effective in high dimensional spaces and are known for their robustness in handling various types of data. Understanding SVMs is crucial for developing accurate and efficient machine learning models.

Understanding SVMs

Support Vector Machines (SVMs) are a set of supervised learning methods used for classification, regression, and outliers detection. The aim of an SVM is to find the optimal boundary (hyperplane) that separates data points of different classes with the maximum margin. This boundary is determined by support vectors, which are the data points closest to the hyperplane.

from sklearn import datasets, svm
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load the iris dataset
iris = datasets.load_iris()
X = iris.data
y = iris.target

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create a SVM classifier
clf = svm.SVC(kernel='linear')

# Train the model using the training sets
clf.fit(X_train, y_train)

# Predict the response for the test dataset
y_pred = clf.predict(X_test)

# Calculate and print the accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy}')

Try it in Google Colab:

Accuracy: 1.0

Kernel Trick in SVMs

The kernel trick is a technique that allows SVMs to solve non-linear classification problems by mapping the input features into a higher-dimensional space where a linear separation is possible. Common kernels include linear, polynomial, radial basis function (RBF), and sigmoid. The choice of kernel and its parameters can significantly affect the performance of the SVM.

from sklearn import datasets, svm
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load the iris dataset
iris = datasets.load_iris()
X = iris.data
y = iris.target

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create a SVM classifier with RBF kernel
clf = svm.SVC(kernel='rbf', gamma='scale')

# Train the model using the training sets
clf.fit(X_train, y_train)

# Predict the response for the test dataset
y_pred = clf.predict(X_test)

# Calculate and print the accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy}')

💡 Tip: When using SVMs, it's important to scale your data before training, as SVMs are sensitive to the scaling of the data. Use StandardScaler or MinMaxScaler from sklearn.preprocessing to normalize your features.

Key Concepts

Check Your Understanding