Feature Engineering

Duration: 5 min

This module delves into the critical process of feature engineering, a crucial step in the machine learning pipeline that involves creating new features or transforming existing ones to improve model performance. Understanding and mastering feature engineering can significantly enhance the accuracy and efficiency of supervised learning algorithms like Linear Regression, Logistic Regression, Decision Trees, Random Forests, SVM, and Gradient Boosting.

Creating New Features

Creating new features involves generating additional columns in your dataset that can provide more information to the model. This can be done by combining existing features, applying mathematical transformations, or extracting domain-specific insights. New features can help capture complex relationships and improve the model's ability to generalize from the training data to unseen data.

import pandas as pd

# Sample dataset
data = {'feature1': [1, 2, 3, 4], 'feature2': [5, 6, 7, 8]}
df = pd.DataFrame(data)

# Creating a new feature by combining existing features
df['feature3'] = df['feature1'] * df['feature2']

print(df)

Try it in Google Colab:

   feature1  feature2  feature3
0        1        5         5
1        2        6        12
2        3        7        21
3        4        8        32

Feature Transformation

Feature transformation involves applying mathematical functions to existing features to make them more suitable for the model. Common transformations include log transformation for skewed data, polynomial features for capturing non-linear relationships, and scaling features to have a mean of zero and a standard deviation of one. These transformations can help stabilize variance, reduce skewness, and improve the model's performance.

import pandas as pd
from sklearn.preprocessing import StandardScaler

# Sample dataset
data = {'feature1': [1, 2, 3, 4], 'feature2': [5, 6, 7, 8]}
df = pd.DataFrame(data)

# Standardizing features
scaler = StandardScaler()
df[['feature1', 'feature2']] = scaler.fit_transform(df[['feature1', 'feature2']])

print(df)

💡 Tip: Always check the distribution of your features before and after transformation to ensure that the transformation has the desired effect.

Key Concepts

Check Your Understanding