Data Preprocessing Hyperparameters

Introduction

Data preprocessing hyperparameters control how raw data is transformed before model training. These parameters significantly impact model performance, training efficiency, and generalization ability. Understanding how to tune preprocessing parameters is crucial for building effective machine learning pipelines.

Scaling Parameters

StandardScaler

Standardizes features to zero mean and unit variance:

• No hyperparameters
• Assumes normal distribution
• Sensitive to outliers

MinMaxScaler

Scales features to [0, 1] range:

• Feature range: (min, max)
• Default: (0, 1)
• Preserves zero entries

RobustScaler

Uses median and IQR for scaling:

• No hyperparameters
• Robust to outliers
• Uses quartiles

Encoding Parameters

One-Hot Encoding

Creates binary columns for categorical variables:

• Drop parameter: 'first', 'if_binary', None
• Sparse parameter: True/False
• Handle unknown: 'error', 'ignore'

Label Encoding

Maps categories to integers:

• No hyperparameters
• Order matters
• Use for ordinal data

Target Encoding

Encodes categories using target statistics:

• Smoothing parameter: controls overfitting
• Min samples leaf: minimum samples per category
• Noise level: adds random noise

Feature Engineering Parameters

Polynomial Features

Creates polynomial combinations:

• Degree: polynomial order
• Interaction only: no powers
• Include bias: add constant term

Feature Selection

Selects most important features:

• K best: number of features to select
• Score function: f_classif, mutual_info_classif
• Percentile: percentage of features to keep

Imputation Parameters

Simple Imputer

Fills missing values:

• Strategy: imputation method
• Fill value: for constant strategy
• Missing values: what to consider missing

KNN Imputer

Uses k-nearest neighbors for imputation:

• N neighbors: number of neighbors
• Weights: 'uniform', 'distance'
• Metric: distance metric

Dimensionality Reduction Parameters

PCA

Principal Component Analysis:

• N components: number or variance ratio
• Whiten: normalize components
• Random state: for reproducibility

LDA

Linear Discriminant Analysis:

• N components: number of components
• Solver: 'svd', 'lsqr', 'eigen'
• Shrinkage: regularization parameter

Text Preprocessing Parameters

TF-IDF

Term Frequency-Inverse Document Frequency:

• Max features: vocabulary size
• N-gram range: word combinations
• Min/max DF: document frequency limits

Count Vectorizer

Counts word occurrences:

• Max features: vocabulary size
• N-gram range: word combinations
• Binary: binary counts

Image Preprocessing Parameters

Resize

Resizes images to fixed dimensions:

• Size: target dimensions
• Interpolation: resampling method
• Keep aspect ratio: maintain proportions

Normalization

Normalizes pixel values:

• Mean: channel means
• Std: channel standard deviations
• Per channel: normalize each channel

Parameter Tuning Strategies

Pipeline Search

Validation Strategy

• Use cross-validation
• Prevent data leakage
• Fit transformers on training data
• Transform test data with fitted transformers