Categorical features and missing values in XGBoost: the native path vs one-hot

XGBoost can learn from missing numeric values and categorical columns without forcing every project through an impute-then-one-hot pipeline. The real decision is not whether native support exists; it is which representation gives the best validation quality, training cost, memory footprint, and production contract for your data.

This guide compares three practical paths on one synthetic e-commerce conversion dataset:

• sparse one-hot encoding;

• pandas categorical columns with enable_categorical=True;

• leakage-safe target encoding.

The examples target XGBoost 3.3.0 and scikit-learn 1.9.0, the stable releases available in July 2026. PyPI lists XGBoost 3.3.0 and scikit-learn 1.9.0.

The three available paths

1. Sparse one-hot encoding

One-hot encoding creates one binary column per observed category. Its strengths are familiar feature names, compatibility with many estimators, and a straightforward deployment contract. OneHotEncoder(sparse_output=True) avoids materializing a dense matrix full of zeros.

The trade-offs are predictable:

• feature count grows with cardinality;

• the encoder becomes part of the saved pipeline;

• categories are separated, so trees need additional depth to reconstruct useful category groups;

• wide sparse matrices can increase training and inference overhead.

One subtle XGBoost rule matters: absent entries in a SciPy sparse matrix are treated as missing by tree boosters. Converting the same matrix to dense fills absent entries with numeric zero, which can change model semantics. Keep genuinely meaningful numeric zeros explicit, or avoid mixing them into a sparse representation accidentally. The XGBoost FAQ documents this sparse-versus-dense behavior.

2. Native categorical handling

For the native path, keep each categorical column as pandas category and enable categorical support:

for column in categorical_columns:
    X[column] = X[column].astype("category")

model = xgb.XGBClassifier(
    tree_method="hist",
    enable_categorical=True,
)

For low-cardinality features, XGBoost can use one-category-versus-rest splits. Above the max_cat_to_onehot threshold, it sorts category gradient statistics and evaluates contiguous partitions, grouping categories that have similar estimated leaf outputs. max_cat_threshold limits how many categories are considered for partition-based splits.

The exact tree method does not support categorical features; use hist or approx. See the current categorical-data guide and categorical parameters.

Native categories keep the matrix narrow and remove a separate encoder. They are a strong default baseline when XGBoost is the primary model and several columns have medium or high cardinality.

3. Leakage-safe target encoding

Target encoding replaces each category with a smoothed statistic derived from the target. For binary classification, the encoded value is a shrunk estimate of that category’s positive rate.

The danger is leakage. Encoding a training row with a statistic calculated from that same row can make rare categories look artificially predictive. In scikit-learn 1.9, TargetEncoder.fit_transform() uses internal cross-fitting; fit(X, y).transform(X) is deliberately not equivalent.

Missing categorical values are encoded as a category, while unseen categories at transform time receive the global target mean. The current TargetEncoder API explains these semantics.

Target encoding is compact and often attractive for high-cardinality data, but it adds supervised preprocessing that must be reproduced exactly during retraining and inference.

Why leaving NaN is usually better than filling with -999

XGBoost’s tree booster learns a default direction for missing values at every split node. For a candidate such as cart_value < 75, it evaluates the gain with the missing group sent left and again with the missing group sent right. The better direction is stored in that node. Another node can learn the opposite direction.

That is more expressive than replacing every gap with -999 and treating it as an ordinary number. A sentinel:

• imposes a fake ordering;

• creates artificial thresholds;

• can collide with future legitimate values;

• encourages the model to spend splits isolating the sentinel;

• assumes missingness behaves like an extremely small measurement.

Prefer keeping numeric gaps as np.nan:

model = xgb.XGBClassifier(tree_method="hist")
model.fit(X_train, y_train)

A legacy pipeline may set missing=-999, which tells XGBoost to remove that value from split scans rather than treat it numerically. That is valid, but NaN is usually clearer.

For categorical data, an explicit "unknown" label is a real category, not a missing value; use it only when that state has deliberate business meaning. XGBoost’s FAQ confirms that NaN is the default missing marker and that tree models learn missing-value branches.

A focused, reproducible comparison

Use Python 3.13 and pin a current environment rather than broad ranges that silently install older releases:

python -m pip install \
  "xgboost==3.3.0" \
  "scikit-learn==1.9.0" \
  "pandas==3.0.5" \
  "numpy==2.5.1" \
  "scipy==1.18.0"

Those versions were current in July 2026.

The script below generates 60,000 neutral e-commerce sessions, inserts numeric and categorical gaps, and reports:

• test AUC;

• preprocessing time;

• model-fitting time;

• prepared training-input memory;

• feature count and best iteration;

• the complete runtime environment.

Preprocessing timing includes only training and validation transformations for every method; test transformation is excluded. Memory means the prepared training input, not peak process RSS or model size.

from pathlib import Path
import platform
import sys
import time

import numpy as np
import pandas as pd
import scipy
import sklearn
import xgboost as xgb
from scipy import sparse
from sklearn.compose import ColumnTransformer
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import StratifiedKFold, train_test_split
from sklearn.preprocessing import OneHotEncoder, TargetEncoder

SEED = 42
ROWS = 60_000
THREADS = 8
rng = np.random.default_rng(SEED)

levels = {
    "channel": np.array(
        ["search", "social", "email", "direct", "affiliate"]
    ),
    "region": np.array([f"region_{i:02d}" for i in range(20)]),
    "product": np.array([f"product_{i:03d}" for i in range(60)]),
    "campaign": np.array([f"campaign_{i:03d}" for i in range(180)]),
    "device": np.array(["mobile", "desktop", "tablet", "other"]),
}

X = pd.DataFrame(
    {
        column: rng.choice(values, ROWS).astype(object)
        for column, values in levels.items()
    }
)
X["cart_value"] = rng.lognormal(3.7, 0.75, ROWS).astype("float32")
X["discount_pct"] = (
    rng.beta(2, 8, ROWS) * 0.5
).astype("float32")
X["items"] = (rng.poisson(2.2, ROWS) + 1).astype("float32")
X["tenure_days"] = (
    rng.exponential(420, ROWS)
    .clip(0.1, 3000)
    .astype("float32")
)
X["session_hour"] = (
    rng.integers(0, 24, ROWS) + 1
).astype("float32")

scales = {
    "channel": 0.35,
    "region": 0.25,
    "product": 0.45,
    "campaign": 0.55,
    "device": 0.20,
}
effects = {
    column: dict(
        zip(
            values,
            rng.normal(0, scales[column], len(values)),
        )
    )
    for column, values in levels.items()
}

logit = (
    -2.2
    + 0.012 * np.minimum(X["cart_value"], 180)
    + 1.8 * X["discount_pct"]
    + 0.10 * np.minimum(X["items"], 8)
    - 0.00025 * X["tenure_days"]
    + 0.25 * X["session_hour"].between(19, 23)
)
for column in levels:
    logit += X[column].map(effects[column]).to_numpy()

missing = {
    "cart_value": rng.random(ROWS) < 0.08,
    "discount_pct": rng.random(ROWS) < 0.12,
    "tenure_days": rng.random(ROWS) < 0.05,
    "campaign": rng.random(ROWS) < 0.15,
    "region": rng.random(ROWS) < 0.03,
}

# Missingness carries synthetic operational signal.
logit += (
    0.35 * missing["campaign"]
    - 0.25 * missing["cart_value"]
)

probability = 1 / (1 + np.exp(-logit))
y = rng.binomial(1, probability).astype("int8")

for column, mask in missing.items():
    X.loc[mask, column] = np.nan

categorical = list(levels)
numeric = [
    "cart_value",
    "discount_pct",
    "items",
    "tenure_days",
    "session_hour",
]

X_full, X_test, y_full, y_test = train_test_split(
    X,
    y,
    test_size=0.20,
    stratify=y,
    random_state=SEED,
)
X_train, X_valid, y_train, y_valid = train_test_split(
    X_full,
    y_full,
    test_size=0.20,
    stratify=y_full,
    random_state=SEED,
)

base_params = {
    "objective": "binary:logistic",
    "n_estimators": 500,
    "learning_rate": 0.06,
    "max_depth": 7,
    "min_child_weight": 4,
    "subsample": 0.85,
    "colsample_bytree": 0.85,
    "reg_lambda": 1.5,
    "tree_method": "hist",
    "eval_metric": "auc",
    "early_stopping_rounds": 30,
    "n_jobs": THREADS,
    "random_state": SEED,
}


def sparse_mb(matrix):
    matrix = matrix.tocsr()
    total = (
        matrix.data.nbytes
        + matrix.indices.nbytes
        + matrix.indptr.nbytes
    )
    return total / 2**20


def frame_mb(frame):
    return (
        frame.memory_usage(index=True, deep=True).sum()
        / 2**20
    )


def benchmark(
    name,
    prepare,
    prepare_test,
    memory_mb,
    extra_params=None,
):
    started = time.perf_counter()
    train_ready, valid_ready = prepare()
    preprocessing_seconds = time.perf_counter() - started

    model = xgb.XGBClassifier(
        **base_params,
        **(extra_params or {}),
    )

    started = time.perf_counter()
    model.fit(
        train_ready,
        y_train,
        eval_set=[(valid_ready, y_valid)],
        verbose=False,
    )
    fitting_seconds = time.perf_counter() - started

    # Test transformation is intentionally outside the timer.
    test_ready = prepare_test()
    probabilities = model.predict_proba(test_ready)[:, 1]

    return {
        "method": name,
        "test_auc": roc_auc_score(
            y_test,
            probabilities,
        ),
        "preprocess_s": preprocessing_seconds,
        "fit_s": fitting_seconds,
        "train_input_mb": memory_mb(train_ready),
        "features": train_ready.shape[1],
        "best_iteration": model.best_iteration,
    }


results = []

# Path 1: sparse one-hot encoding.
one_hot = ColumnTransformer(
    [
        (
            "categorical",
            OneHotEncoder(
                handle_unknown="ignore",
                sparse_output=True,
                dtype=np.float32,
            ),
            categorical,
        ),
        ("numeric", "passthrough", numeric),
    ],
    sparse_threshold=1.0,
)

results.append(
    benchmark(
        "one-hot",
        lambda: (
            one_hot.fit_transform(X_train),
            one_hot.transform(X_valid),
        ),
        lambda: one_hot.transform(X_test),
        sparse_mb,
    )
)

# Path 2: native pandas categorical columns.
category_levels = {
    column: pd.Index(
        X_train[column].dropna().unique()
    )
    for column in categorical
}


def native_frame(frame):
    converted = frame.copy()
    for column in categorical:
        converted[column] = pd.Categorical(
            converted[column],
            categories=category_levels[column],
        )
    return converted


results.append(
    benchmark(
        "native category",
        lambda: (
            native_frame(X_train),
            native_frame(X_valid),
        ),
        lambda: native_frame(X_test),
        frame_mb,
        {
            "enable_categorical": True,
            "max_cat_to_onehot": 4,
            "max_cat_threshold": 64,
        },
    )
)

# Path 3: cross-fitted target encoding.
cross_folds = StratifiedKFold(
    n_splits=5,
    shuffle=True,
    random_state=SEED,
)
target_encoder = TargetEncoder(
    target_type="binary",
    smooth="auto",
    cv=cross_folds,
)


def target_frame(frame, training=False):
    if training:
        encoded = target_encoder.fit_transform(
            frame[categorical],
            y_train,
        )
    else:
        encoded = target_encoder.transform(
            frame[categorical]
        )

    return np.column_stack(
        [
            encoded.astype("float32"),
            frame[numeric].to_numpy(dtype="float32"),
        ]
    )


results.append(
    benchmark(
        "target encoding",
        lambda: (
            target_frame(X_train, training=True),
            target_frame(X_valid),
        ),
        lambda: target_frame(X_test),
        lambda matrix: matrix.nbytes / 2**20,
    )
)

summary = pd.DataFrame(results)
print(
    summary.to_string(
        index=False,
        float_format=lambda value: f"{value:.4f}",
    )
)

cpu = platform.processor() or platform.machine()
cpuinfo = Path("/proc/cpuinfo")
if sys.platform.startswith("linux") and cpuinfo.exists():
    for line in cpuinfo.read_text().splitlines():
        if line.startswith("model name"):
            cpu = line.split(":", 1)[1].strip()
            break

print("\nEnvironment")
print(f"Python: {platform.python_version()}")
print(f"OS: {platform.platform()}")
print(f"CPU: {cpu}")
print(f"XGBoost threads: {THREADS}")
print(f"xgboost: {xgb.__version__}")
print(f"scikit-learn: {sklearn.__version__}")
print(f"pandas: {pd.__version__}")
print(f"numpy: {np.__version__}")
print(f"scipy: {scipy.__version__}")

Benchmark integrity note: The available execution environment could not run XGBoost 3.3.0, so older 3.1.3 timings are not relabeled here. Run the pinned script on the publication machine and keep its environment block beside the result table.

The useful comparison is usually the pattern, not a universal winner:

• one-hot encoding tends to produce the widest input;

• native categories remain compact without supervised preprocessing;

• target encoding can perform strongly when stable category-level effects dominate.

Results can change with cardinality, interactions, class balance, missingness, unseen categories, thread count, and hardware.

How sparsity-aware split finding works

The missing-value mechanism is part of XGBoost’s sparsity-aware split finder. For each feature and node, the algorithm scans only present entries, evaluates both default directions for the missing group, and records the direction with maximum gain.

Work therefore scales with the number of non-missing entries rather than blindly visiting every gap or implicit sparse element. This unifies missing measurements, one-hot sparsity, and other sparse feature-engineering outputs.

It also explains why sparse and dense representations are not interchangeable: an absent sparse entry is missing, while a dense zero is a valid numeric value.

Cherry on the cake: the original 50× result

In the original XGBoost paper, the Allstate-10K experiment was sparse mainly because of one-hot encoding. The sparsity-aware algorithm was reported as more than 50 times faster than a naive implementation that ignored sparsity.

The speedup came from visiting only non-missing entries while learning the best default branch—not merely from accepting NaN. Read Section 3.4 and Figure 5 in the paper.

XGBClassifier or native Booster and DMatrix?

Use XGBClassifier for most supervised projects. It integrates with scikit-learn pipelines, cross-validation, metrics, parameter search, predict_proba(), and familiar model-management code. You can still access the underlying booster through model.get_booster().

Use the native API when you need:

• explicit DMatrix or QuantileDMatrix lifecycle control;

• iterator or external-memory workflows;

• custom training loops and callbacks;

• training continuation;

• prediction modes such as pred_contribs, pred_interactions, and pred_leaf.

dtrain = xgb.QuantileDMatrix(
    X_train_native,
    label=y_train,
    enable_categorical=True,
)

booster = xgb.train(
    {
        "objective": "binary:logistic",
        "tree_method": "hist",
    },
    dtrain,
    num_boost_round=300,
)

contributions = booster.predict(
    dtrain,
    pred_contribs=True,
)

The categorical-data guide shows the native route for categorical DMatrix inputs and contribution or interaction predictions. The Python package introduction covers the native, scikit-learn, and Dask interfaces.

Production rules that prevent surprises

Keep these decisions in the model contract:

• preserve column names, order, and dtypes;

• save categorical models as JSON or UBJSON;

• fit one-hot and target encoders only on training data;

• use cross-fitting for training-time target encodings;

• keep numeric gaps as NaN unless a documented legacy marker is required;

• monitor missingness rates and category drift separately;

• define an explicit policy for genuinely unseen categories.

Since XGBoost 3.1, the Python dataframe interface can remember category encodings and recode compatible dataframe inputs. It can reconcile changed integer codes and handle an inference batch that omits categories known during training.

It cannot invent a valid code for a genuinely new category. Retrain, map it through a documented domain rule, or reserve an explicit unknown category during training. Category-name types must also remain compatible. The auto-recoding section documents these boundaries precisely.

Which path should you choose?

Choose sparse one-hot encoding when estimator portability, simple inspection, or compatibility with several model families matters most.

Choose native categorical handling as the first XGBoost baseline when you have multiple medium- or high-cardinality columns and want a compact pipeline with no supervised encoder.

Choose target encoding when matrix width and speed matter, category effects are stable, and you can guarantee cross-fitting plus identical inference transformations.

Run the benchmark on your own split, then record validation quality, preprocessing and fit time, prepared-input memory, prediction latency, and unseen-category behavior. Promote the simplest representation that satisfies those production constraints—not the one that wins by convention.