Data Preprocessing & Feature Engineering

Duration: 25 min

Overview

Raw data is messy. Preprocessing turns it into something models can learn from.

Handling Missing Values

from sklearn.impute import SimpleImputer
import numpy as np
imputer = SimpleImputer(strategy='median')  # or 'mean', 'most_frequent'
X_clean = imputer.fit_transform(X_train)
X_test_clean = imputer.transform(X_test)  # use training stats!

Scaling Features

from sklearn.preprocessing import StandardScaler, MinMaxScaler
StandardScaler: mean=0, std=1 (best for most models)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
MinMaxScaler: range [0, 1] (good for neural nets)
scaler = MinMaxScaler()

Encoding Categorical Variables

from sklearn.preprocessing import OneHotEncoder, LabelEncoder
import pandas as pd
One-hot encoding (for features)
encoder = OneHotEncoder(sparse_output=False, handle_unknown='ignore')
X_encoded = encoder.fit_transform(df[['color', 'size']])
Or use pandas
df_encoded = pd.get_dummies(df, columns=['color', 'size'])

Pipelines (Do It Right)

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
numeric_features = ['age', 'income', 'score']
categorical_features = ['city', 'gender']
preprocessor = ColumnTransformer(transformers=[
    ('num', Pipeline([
        ('imputer', SimpleImputer(strategy='median')),
        ('scaler', StandardScaler())
    ]), numeric_features),
    ('cat', Pipeline([
        ('imputer', SimpleImputer(strategy='most_frequent')),
        ('encoder', OneHotEncoder(handle_unknown='ignore'))
    ]), categorical_features)
])
Full pipeline: preprocess + model
pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('classifier', RandomForestClassifier())
])
pipeline.fit(X_train, y_train)
print(f"Score: {pipeline.score(X_test, y_test):.3f}")

Quiz

Q1: What is the key concept here?

• A) An optional technique

• B) A fundamental sklearn pattern for building reliable ML systems ✓

• C) Only for deep learning

• D) Deprecated in newer versions

Q2: What should you always do before evaluating a model?

• A) Train on all available data

• B) Hold out a test set the model has never seen ✓

• C) Use the largest model

• D) Remove all features

Q3: When does this technique matter most?

• A) Only for toy datasets

• B) For real-world problems where data is messy and performance matters ✓

• C) Only during research

• D) Never in production