-
Regularizing a boosted model: reg_lambda, reg_alpha, gamma, and the constraints that matter
XGBoost does not regularize a tree in quite the same way that ridge regression or Lasso regularizes a linear model. A standard tree booster does not have one coefficient per input feature. Instead, it
-
Regularization in practice: telling underfitting from overfitting and choosing the strength
A machine learning model can fail in two opposite ways.
-
Overfitting on purpose: dropout, weight decay, and early stopping in one runnable lab
Machine learning tutorials often introduce overfitting as a failure mode: the model memorizes the training data, performs brilliantly on examples it has already seen, and then collapses on new data.
-
Expert: model capacity and overfitting trade-offs
Understanding model capacity and overfitting trade-offs is essential for designing high-performing, generalizable machine learning systems. This article explores how model capacity affects bias and variance, how overfitting manifests in deep models, and what strategies expert practitioners use to optimize complexity without sacrificing generalization.