Support Vector Machines (SVM) Basics

Duration: 5 min

This module provides a comprehensive introduction to Support Vector Machines (SVM), a powerful supervised learning algorithm used for classification and regression. We will explore the fundamental concepts, mathematical underpinnings, and practical applications of SVMs, highlighting their importance in machine learning tasks.

Understanding SVMs

Support Vector Machines (SVMs) are a set of supervised learning methods used for classification and regression. The core idea behind SVMs is to find the optimal hyperplane that separates data points into different classes with the maximum margin. This hyperplane is determined by support vectors, which are the data points closest to the decision boundary.

from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

# Load the iris dataset
iris = datasets.load_iris()
X = iris.data
y = iris.target

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create an SVM classifier
svm_classifier = SVC(kernel='linear')

# Train the classifier
svm_classifier.fit(X_train, y_train)

# Make predictions
y_pred = svm_classifier.predict(X_test)

# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy:.2f}')

Try it in Google Colab:

Accuracy: 0.97

Kernel Trick in SVMs

The kernel trick is a technique used in SVMs to transform data into a higher-dimensional space where it becomes easier to find a linear separating hyperplane. Common kernels include linear, polynomial, and radial basis function (RBF). The choice of kernel can significantly impact the performance of the SVM.

from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

# Load the iris dataset
iris = datasets.load_iris()
X = iris.data
y = iris.target

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create an SVM classifier with RBF kernel
svm_classifier = SVC(kernel='rbf')

# Train the classifier
svm_classifier.fit(X_train, y_train)

# Make predictions
y_pred = svm_classifier.predict(X_test)

# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy:.2f}')

💡 Tip: When using SVMs, it's important to scale your data before training, as SVMs are sensitive to the scale of the data. Use StandardScaler from sklearn.preprocessing to standardize features.

Key Concepts

Check Your Understanding