Project: Implementing SVM

Duration: 5 min

This module focuses on implementing Support Vector Machines (SVM) in Python. SVM is a powerful supervised learning algorithm used for classification and regression tasks. Understanding SVM is crucial for developing robust machine learning models, especially when dealing with high-dimensional data.

Understanding SVM

Support Vector Machines (SVM) are supervised learning models used for classification and regression. The algorithm aims to find the optimal hyperplane that separates data points of different classes with the maximum margin. SVMs are particularly effective in high-dimensional spaces and are versatile for various applications.

import numpy as np
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

# Load the iris dataset
iris = datasets.load_iris()
X = iris.data
y = iris.target

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create an SVM classifier
svm_classifier = SVC(kernel='linear')

# Train the model
svm_classifier.fit(X_train, y_train)

# Make predictions
y_pred = svm_classifier.predict(X_test)

# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy * 100:.2f}%')

Try it in Google Colab:

Accuracy: 96.67%

Kernel Trick in SVM

The kernel trick is a technique that allows SVM to solve non-linear classification problems by transforming the data into a higher-dimensional space where a linear separator can be found. Common kernels include linear, polynomial, and radial basis function (RBF).

import numpy as np
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

# Load the iris dataset
iris = datasets.load_iris()
X = iris.data
y = iris.target

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create an SVM classifier with RBF kernel
svm_classifier = SVC(kernel='rbf')

# Train the model
svm_classifier.fit(X_train, y_train)

# Make predictions
y_pred = svm_classifier.predict(X_test)

# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy * 100:.2f}%')

💡 Tip: When using SVM with different kernels, it's important to tune hyperparameters like C (regularization parameter) and gamma (kernel coefficient for RBF) to achieve the best performance.

Key Concepts

Check Your Understanding