SVM Advanced Techniques

Duration: 5 min

This module delves into advanced techniques for Support Vector Machines (SVM), a powerful supervised learning algorithm used for classification and regression. We will explore kernel tricks, parameter tuning, and handling multi-class problems to enhance the performance and applicability of SVMs in complex datasets.

Kernel Tricks in SVM

Kernel tricks allow SVMs to solve non-linear problems by mapping input features into higher-dimensional spaces where a linear separator can be found. Common kernels include linear, polynomial, radial basis function (RBF), and sigmoid. Choosing the right kernel and its parameters is crucial for model performance.

from sklearn import datasets, svm
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load dataset
iris = datasets.load_iris()
X = iris.data
y = iris.target

# Split dataset into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create a SVM classifier using RBF kernel
clf = svm.SVC(kernel='rbf')

# Train the model using the training sets
clf.fit(X_train, y_train)

# Predict the response for test dataset
y_pred = clf.predict(X_test)

# Model Accuracy
print("Accuracy:", accuracy_score(y_test, y_pred))

Try it in Google Colab:

Accuracy: 1.0

Parameter Tuning for SVM

Parameter tuning is essential to optimize SVM performance. Key parameters include C (regularization parameter), gamma (kernel coefficient for RBF), and degree (for polynomial kernel). Grid search and cross-validation are commonly used techniques to find the best parameters.

from sklearn.model_selection import GridSearchCV

# Define parameter range
param_grid = {'C': [0.1, 1, 10, 100], 'gamma': [1, 0.1, 0.01, 0.001], 'kernel': ['rbf', 'linear']}

# Create a SVM classifier
grid = GridSearchCV(svm.SVC(), param_grid, refit=True, verbose=2)

# Fit the model
grid.fit(X_train, y_train)

# Print best parameters
print(grid.best_params_)

# Predict the response for test dataset
y_pred = grid.predict(X_test)

# Model Accuracy
print("Accuracy:", accuracy_score(y_test, y_pred))

💡 Tip: When tuning SVM parameters, be cautious of overfitting. Use cross-validation to ensure that the model generalizes well to unseen data.

Key Concepts

Check Your Understanding