Scaling and normalization: StandardScaler vs MinMaxScaler vs RobustScaler, and the models that don’t care

Why feature scale changes a model

Suppose a dataset contains two useful features:

• Age: roughly 18–80

• Annual income: roughly 20,000–250,000

A person can judge both as meaningful. A machine-learning algorithm sees numbers. In a Euclidean distance, dot product, kernel, variance calculation, or regularization penalty, the larger numeric range can dominate unless the model or preprocessing compensates for it.

Feature scaling changes each column’s numeric representation while preserving which value belongs to which observation. It is especially important for:

• k-nearest neighbors and other distance-based methods

• RBF support vector machines

• regularized linear and logistic regression

• k-means and related clustering methods

• PCA, which centers features but does not scale them automatically

It is less important for ordinary decision trees and tree ensembles because their splits depend mainly on value ordering rather than Euclidean geometry. “Tree models ignore scaling” is still too absolute: finite precision, tied values, histogram binning, and implementation details can occasionally produce small differences.

Feature scaling vs normalization in scikit-learn

The word normalization is often used loosely for any rescaling. In scikit-learn, Normalizer has a specific meaning: it rescales each sample row to unit norm. StandardScaler, MinMaxScaler, and RobustScaler instead transform each feature column independently.

Scaler	Transformation	Good starting use
StandardScaler	Mean near 0, variance near 1	General-purpose scaling
MinMaxScaler	Training values mapped to a chosen range	Bounded inputs with stable limits
RobustScaler	Median centering and quantile-based scaling	Long-tailed features or influential outliers

None of these removes outliers, fixes measurement errors, or makes a distribution Gaussian.

Reproducible environment

As of July 30, 2026, the current stable scikit-learn release is 1.9.0 and requires Python 3.11 or newer. This lesson pins current compatible releases of NumPy, SciPy, pandas, and Matplotlib.

python -m pip install \
    "scikit-learn==1.9.0" \
    "numpy==2.5.1" \
    "scipy==1.18.0" \
    "pandas==3.0.5" \
    "matplotlib==3.11.1"

See all three scalers on one outlier

import numpy as np
from sklearn.preprocessing import MinMaxScaler, RobustScaler, StandardScaler

X = np.array([
    [10.0],
    [11.0],
    [12.0],
    [13.0],
    [500.0],
])

scalers = {
    "StandardScaler": StandardScaler(),
    "MinMaxScaler": MinMaxScaler(),
    "RobustScaler": RobustScaler(),
}

for name, scaler in scalers.items():
    transformed = scaler.fit_transform(X).ravel()
    print(f"{name:16s}: {np.round(transformed, 3)}")

StandardScaler  : [-0.508 -0.503 -0.497 -0.492  2.   ]
MinMaxScaler    : [0.    0.002 0.004 0.006 1.   ]
RobustScaler    : [ -1.   -0.5   0.    0.5 244. ]

The outlier remains an outlier under every transformation. The difference is how much leverage it has over the statistics used to position the ordinary values.

StandardScaler: the usual default

StandardScaler applies this transformation independently to each feature:

z = (x − training mean) / training standard deviation

A nonconstant training feature normally ends with mean near 0 and variance near 1. The feature does not need to be normally distributed; standardization changes its center and scale, not its shape.

This is a strong first candidate for kNN, SVMs, logistic regression, PCA, and many optimization-based estimators. Its weakness is outlier sensitivity because both the mean and standard deviation can be pulled by extreme observations. For sparse CSR or CSC matrices, use StandardScaler(with_mean=False) because centering would destroy sparsity.

MinMaxScaler: bounded training values

With the default range (0, 1), MinMaxScaler applies:

x′ = (x − training minimum) / (training maximum − training minimum)

The smallest training value becomes 0 and the largest becomes 1. Future values are not guaranteed to stay inside that range. A value above the training maximum can transform above 1; clip=True forces held-out values into the configured range but discards information.

Use min-max scaling when a downstream component expects bounded inputs, physical limits are stable, and severe outliers are absent or already handled.

RobustScaler: reduce outlier leverage

RobustScaler centers each feature using its training median. By default, it divides by the interquartile range:

IQR = 75th percentile − 25th percentile

Its approximate transformation is:

x′ = (x − training median) / training IQR

The median and middle quantiles are less influenced by a small number of extreme values. That often leaves the ordinary region more spread out than StandardScaler or MinMaxScaler. The outliers are not removed; they can still transform to very large magnitudes.

You can tune the quantile range, but validate it with the complete model rather than assuming wider is better.

from sklearn.preprocessing import RobustScaler

scaler = RobustScaler(quantile_range=(10.0, 90.0))

Cherry on the cake: one outlier steals 99% of MinMaxScaler’s range

Five ordinary training values from 10 to 14 fill the whole [0, 1] range. Add a single value of 500 and those same five values are squeezed into approximately [0, 0.0082]—less than one percent of the available scale.

import matplotlib.pyplot as plt
import numpy as np
from sklearn.preprocessing import MinMaxScaler

ordinary = np.arange(10.0, 15.0).reshape(-1, 1)
with_outlier = np.vstack([ordinary, [[500.0]]])

scaled_without_outlier = MinMaxScaler().fit_transform(ordinary).ravel()
scaled_with_outlier = MinMaxScaler().fit_transform(with_outlier).ravel()[:-1]

print(np.round(scaled_without_outlier, 4))
print(np.round(scaled_with_outlier, 4))

fig, axes = plt.subplots(1, 2, sharey=True, figsize=(10, 4))

axes[0].plot(ordinary.ravel(), scaled_without_outlier, marker="o")
axes[0].set_title("Training range: 10 to 14")
axes[0].set_xlabel("Original value")
axes[0].set_ylabel("Min-max scaled value")

axes[1].plot(ordinary.ravel(), scaled_with_outlier, marker="o")
axes[1].set_title("Training range after adding 500")
axes[1].set_xlabel("Original value")

for axis in axes:
    axis.set_ylim(-0.05, 1.05)
    axis.grid(True)

fig.tight_layout()
plt.show()

[0.   0.25 0.5  0.75 1.  ]
[0.     0.002  0.0041 0.0061 0.0082]

This is why a stable training maximum matters. Scikit-learn’s own scaler comparison likewise shows that MinMaxScaler can compress inliers into a tiny interval when outliers define the range.

Runnable benchmark: which models care?

The following experiment creates two equally useful features, multiplies one by 1,000, adds several extreme values, and compares four models under four preprocessing choices.

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import MinMaxScaler, RobustScaler, StandardScaler
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier

rng = np.random.default_rng(42)
n_samples = 2_500

feature_1 = rng.normal(size=n_samples)
feature_2 = rng.normal(size=n_samples)
signal = feature_1 + feature_2 + rng.normal(scale=0.6, size=n_samples)
y = (signal > 0).astype(int)

X = np.column_stack([feature_1, feature_2 * 1_000.0])
outlier_rows = rng.choice(n_samples, size=25, replace=False)
X[outlier_rows, 1] *= 20.0

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.30,
    stratify=y,
    random_state=42,
)

scalers = {
    "None": "passthrough",
    "Standard": StandardScaler(),
    "Min-max": MinMaxScaler(),
    "Robust": RobustScaler(),
}

models = {
    "kNN": KNeighborsClassifier(n_neighbors=15),
    "RBF SVM": SVC(C=2.0, gamma="scale"),
    "Logistic regression": LogisticRegression(max_iter=2_000),
    "Decision tree": DecisionTreeClassifier(max_depth=5, random_state=42),
}

rows = []
for model_name, model in models.items():
    for scaler_name, scaler in scalers.items():
        pipeline = make_pipeline(scaler, model)
        pipeline.fit(X_train, y_train)
        rows.append({
            "model": model_name,
            "scaler": scaler_name,
            "test_accuracy": pipeline.score(X_test, y_test),
        })

results = pd.DataFrame(rows)
comparison = results.pivot(
    index="model",
    columns="scaler",
    values="test_accuracy",
)
comparison = comparison[["None", "Standard", "Min-max", "Robust"]]
print(comparison.round(3))

Run the script and focus on the pattern rather than a magic score:

Model	Expected scaling behavior
kNN	Strongly scale-sensitive because neighbors are selected by distance
RBF SVM	Strongly scale-sensitive because the kernel uses squared distances
Logistic regression	Scaling affects conditioning, convergence, coefficient penalties, and the useful range of C
Decision tree	Usually nearly unchanged under positive affine rescaling

For SVC(gamma="scale"), scikit-learn computes one global gamma from 1 / (n_features × X.var()); it does not balance every feature independently.

The fit-on-train-only rule

Never fit a scaler on the entire dataset before evaluation. Doing so lets validation or test values influence the training mean, minimum, maximum, median, or quantiles. That is data leakage.

The safe sequence is:

1. Split first.

2. Put preprocessing and the estimator in one Pipeline.

3. Cross-validate the complete pipeline so each fold fits its own scaler using only that fold’s training partition.

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler, RobustScaler, StandardScaler

pipeline = Pipeline([
    ("scale", StandardScaler()),
    ("model", LogisticRegression(max_iter=2_000)),
])

parameter_grid = {
    "scale": [
        "passthrough",
        StandardScaler(),
        MinMaxScaler(),
        RobustScaler(),
    ],
    "model__C": [0.1, 1.0, 10.0],
}

search = GridSearchCV(
    estimator=pipeline,
    param_grid=parameter_grid,
    scoring="accuracy",
    cv=5,
    n_jobs=-1,
)
search.fit(X_train, y_train)

print(search.best_params_)
print(search.score(X_test, y_test))

A pipeline makes leakage prevention the default behavior and lets you tune the scaler together with model hyperparameters. That joint tuning matters particularly for regularized models.

Practical model-by-model recommendation

• kNN and distance-based models: scale by default; start with StandardScaler, then test RobustScaler when outliers are legitimate.

• RBF SVM: scale almost always; tune the scaler, C, and gamma together.

• Logistic and linear models: usually scale when regularization or iterative solvers are involved; tune regularization after scaling.

• PCA: scale when feature units should contribute comparably because PCA centers but does not scale columns.

• k-means and distance-based clustering: scale unless different units are intentionally meant to receive different weights.

• Decision trees, random forests, and boosted trees: scaling is usually unnecessary, but verify the complete implementation and pipeline rather than relying on an absolute rule.

Final takeaway

StandardScaler is the best general starting point. MinMaxScaler is useful when bounded training values are meaningful and stable. RobustScaler is valuable when extreme but legitimate observations would otherwise control the scale. Tree models are largely insensitive to these positive affine transformations, while distance-, kernel-, variance-, and regularization-sensitive models can change dramatically.

Run the benchmark, replace the synthetic data with your own numeric features, and compare cross-validated pipelines—not scalers in isolation—before choosing your production preprocessing strategy.