Learning to rank with XGBoost: pairwise and listwise objectives for search and recommendations

Ranking systems do not predict items in isolation. They choose an order inside a query, session, user slate, or recommendation list.

That distinction changes the data shape, objective, validation split, and metrics. A regressor can predict relevance labels reasonably well yet still place the best product below several mediocre ones. Learning-to-rank objectives spend their optimization budget on within-query ordering instead.

This tutorial builds a neutral product-search experiment with XGBoost 3.x and compares:

• Pointwise relevance regression with XGBRegressor

• Pairwise ranking with rank:pairwise

• NDCG-aware LambdaMART with rank:ndcg

• Grouped evaluation with NDCG@10 and MAP@10

• Position-biased click training without pretending clicks are unbiased labels

Ranking is a grouped prediction problem

Suppose a search for “quiet mechanical keyboard” retrieves 50 products. The useful output is not a universally calibrated score for each catalog item. It is an ordering among those 50 candidates.

The three practical formulations are:

• Pointwise: predict every item’s relevance independently with regression or classification.

• Pairwise: learn that item A should score above item B within the same query.

• Listwise-aware: scale pairwise updates by their effect on a list metric such as NDCG.

That last wording matters. XGBoost’s rank:ndcg is LambdaMART: it constructs document pairs, then scales gradients according to NDCG impact. It is listwise-aware, not a full permutation-probability model. rank:pairwise uses the unscaled RankNet-style pairwise loss. Current XGBoost 3.3 documentation lists rank:ndcg as the default ranking objective and supports mean and topk pair construction.

Install the current stable XGBoost release:

python -m pip install "xgboost==3.3.0" pandas numpy

XGBoost 3.3.0 was released on June 17, 2026.

Build a neutral product-search dataset

The generator below creates query-product features such as semantic match, price fit, stock status, and shipping speed. Relevance is ordinal from 0 to 4 and assigned from each query’s internal utility ordering.

from __future__ import annotations

import numpy as np
import pandas as pd
import xgboost as xgb

SEED = 7
FEATURES = [
    "semantic_match",
    "lexical_match",
    "price_fit",
    "brand_affinity",
    "popularity",
    "fast_shipping",
    "in_stock",
    "query_price_sensitivity",
    "query_popularity_bias",
]


def make_product_search_data(
    n_queries: int = 360,
    min_items: int = 35,
    max_items: int = 70,
    seed: int = SEED,
) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    rows: list[dict[str, float | int]] = []

    for qid in range(n_queries):
        n_items = int(rng.integers(min_items, max_items + 1))
        price_sensitivity = rng.uniform(0.2, 1.8)
        popularity_bias = rng.uniform(-0.5, 1.1)

        semantic = rng.beta(2.0, 2.2, n_items)
        lexical = np.clip(
            0.65 * semantic + rng.normal(0, 0.22, n_items),
            0,
            1,
        )
        price_fit = rng.beta(2.2, 2.2, n_items)
        brand = rng.beta(1.8, 2.5, n_items)
        popularity = rng.beta(1.5, 3.0, n_items)
        shipping = rng.binomial(1, 0.55, n_items)
        stock = rng.binomial(1, 0.93, n_items)

        utility = (
            2.6 * semantic
            + 1.5 * lexical
            + price_sensitivity
            * (1.2 * price_fit + 0.9 * semantic * price_fit)
            + 0.8 * brand
            + popularity_bias * popularity
            + 0.35 * shipping
            + 0.6 * stock
            - 0.9 * (1 - stock) * semantic
            + rng.normal(0, 0.40, n_items)
        )

        order = np.argsort(utility)[::-1]
        relevance = np.zeros(n_items, dtype=np.int32)
        relevance[order[:1]] = 4
        relevance[order[1:3]] = 3
        relevance[order[3:7]] = 2
        relevance[order[7:15]] = 1

        for i in range(n_items):
            rows.append(
                {
                    "qid": qid,
                    "semantic_match": semantic[i],
                    "lexical_match": lexical[i],
                    "price_fit": price_fit[i],
                    "brand_affinity": brand[i],
                    "popularity": popularity[i],
                    "fast_shipping": shipping[i],
                    "in_stock": stock[i],
                    "query_price_sensitivity": price_sensitivity,
                    "query_popularity_bias": popularity_bias,
                    "relevance": int(relevance[i]),
                }
            )

    return pd.DataFrame(rows)


def split_by_query(
    df: pd.DataFrame,
    test_fraction: float = 0.25,
    seed: int = SEED,
) -> tuple[pd.DataFrame, pd.DataFrame]:
    rng = np.random.default_rng(seed)
    qids = df["qid"].unique().copy()
    rng.shuffle(qids)

    n_test = int(np.ceil(len(qids) * test_fraction))
    test_qids = set(qids[:n_test])

    train = (
        df.loc[~df["qid"].isin(test_qids)]
        .sort_values("qid", kind="stable")
        .reset_index(drop=True)
    )
    test = (
        df.loc[df["qid"].isin(test_qids)]
        .sort_values("qid", kind="stable")
        .reset_index(drop=True)
    )
    return train, test


data = make_product_search_data()
train, test = split_by_query(data)

Always split by query, not by row. A row-level split leaks products from the same result list across training and testing, making evaluation unrealistically easy.

Evaluate ordering, not accuracy

NDCG rewards graded relevance near the top. Its gain is 2ʳᵉˡ − 1, lower ranks receive logarithmically less credit, and the score is normalized by the ideal ordering.

MAP requires binary relevance. Here, labels 2, 3, and 4 count as relevant; that threshold must be documented.

def dcg_at_k(y_true: np.ndarray, y_score: np.ndarray, k: int) -> float:
    order = np.argsort(-y_score, kind="stable")[:k]
    gains = np.power(2.0, y_true[order]) - 1.0
    discounts = np.log2(np.arange(2, len(order) + 2))
    return float(np.sum(gains / discounts))


def ndcg_at_k(y_true: np.ndarray, y_score: np.ndarray, k: int) -> float:
    ideal = dcg_at_k(y_true, y_true.astype(float), k)
    return 0.0 if ideal == 0.0 else dcg_at_k(y_true, y_score, k) / ideal


def average_precision_at_k(
    y_true: np.ndarray,
    y_score: np.ndarray,
    k: int,
    relevance_threshold: int = 2,
) -> float:
    relevant = y_true >= relevance_threshold
    order = np.argsort(-y_score, kind="stable")[:k]
    hits = relevant[order].astype(float)
    denominator = min(int(relevant.sum()), k)

    if denominator == 0:
        return 0.0

    precision = np.cumsum(hits) / np.arange(1, len(hits) + 1)
    return float(np.sum(precision * hits) / denominator)


def evaluate_ranker(
    df: pd.DataFrame,
    scores: np.ndarray,
    k: int = 10,
) -> dict[str, float]:
    scored = df.assign(predicted_score=scores)
    ndcgs: list[float] = []
    aps: list[float] = []

    for _, group in scored.groupby("qid", sort=False):
        labels = group["relevance"].to_numpy()
        predictions = group["predicted_score"].to_numpy()
        ndcgs.append(ndcg_at_k(labels, predictions, k))
        aps.append(average_precision_at_k(labels, predictions, k))

    return {
        f"NDCG@{k}": float(np.mean(ndcgs)),
        f"MAP@{k}": float(np.mean(aps)),
    }

Train pointwise, pairwise, and NDCG-aware models

With mean, XGBoost samples a configured number of pairs per document. With topk, pair construction concentrates on the head of the ranked list. The documentation recommends matching the objective and pair strategy to the serving metric and available effective pairs.

X_train = train[FEATURES]
y_train = train["relevance"]
qid_train = train["qid"].to_numpy()
X_test = test[FEATURES]

common = {
    "n_estimators": 700,
    "max_depth": 4,
    "learning_rate": 0.03,
    "subsample": 0.9,
    "colsample_bytree": 0.9,
    "min_child_weight": 1,
    "reg_lambda": 1.0,
    "tree_method": "hist",
    "random_state": SEED,
    "n_jobs": 4,
}

models = {
    "Pointwise regression": xgb.XGBRegressor(
        objective="reg:squarederror",
        **common,
    ),
    "Pairwise ranking": xgb.XGBRanker(
        objective="rank:pairwise",
        lambdarank_pair_method="mean",
        lambdarank_num_pair_per_sample=32,
        **common,
    ),
    "NDCG-aware ranking": xgb.XGBRanker(
        objective="rank:ndcg",
        eval_metric="ndcg@10",
        lambdarank_pair_method="topk",
        lambdarank_num_pair_per_sample=32,
        **common,
    ),
}

models["Pointwise regression"].fit(X_train, y_train)
models["Pairwise ranking"].fit(X_train, y_train, qid=qid_train)
models["NDCG-aware ranking"].fit(X_train, y_train, qid=qid_train)

results = {
    name: evaluate_ranker(test, model.predict(X_test))
    for name, model in models.items()
}
print(pd.DataFrame(results).T.round(4))

A representative seeded run produced:

Model	NDCG@10	MAP@10
Pointwise regression	0.8597	0.7954
Pairwise ranking	0.8673	0.8000
NDCG-aware ranking	0.8594	0.7933

The pairwise objective wins because it directly trains on within-query preferences. Regression spends much of its loss budget reducing absolute label error across many low-relevance rows.

The NDCG-aware model does not automatically win every synthetic benchmark. Objective alignment improves what gradients emphasize; it does not remove sensitivity to pair sampling, cutoff, noise, label density, and tree capacity.

The classic silent group-boundary bug

XGBoost expects ranking rows to be sorted by qid in nondecreasing order. Features, labels, and query IDs must all use the same permutation.

This is wrong:

order = np.argsort(qid)
qid = qid[order]

# BUG: X and y still use their original row order.
ranker.fit(X, y, qid=qid)

This is correct:

order = np.argsort(qid, kind="stable")
X = X.iloc[order].reset_index(drop=True)
y = y.iloc[order].reset_index(drop=True)
qid = qid[order]

assert len(X) == len(y) == len(qid)
assert np.all(qid[:-1] <= qid[1:])

ranker.fit(X, y, qid=qid)

The corrupted version can still train and emit plausible scores. It simply learns pair relationships across the wrong rows, which makes this bug especially dangerous.

Feed click data honestly

A click is not a relevance judgment. Higher positions receive more attention, and clicks also reflect presentation, price, brand familiarity, accidental taps, and the previous ranker’s policy.

XGBoost provides experimental Unbiased LambdaMART through lambdarank_unbiased=True. The feature estimates position bias for click labels; current documentation notes that position debiasing is not supported by the distributed ranking interfaces.

def simulate_click_log(df: pd.DataFrame, seed: int = 19) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    log = df.copy()

    logging_score = (
        1.8 * log["semantic_match"]
        + 0.8 * log["popularity"]
        + 0.2 * log["fast_shipping"]
        + rng.normal(0, 0.25, len(log))
    )
    log["shown_position"] = (
        logging_score.groupby(log["qid"])
        .rank(method="first", ascending=False)
        .astype(np.int32)
        - 1
    )

    examination = 1.0 / np.log2(log["shown_position"].to_numpy() + 2.0)
    attractiveness = np.clip(
        0.03 + 0.18 * log["relevance"].to_numpy(),
        0.0,
        0.95,
    )
    log["clicked"] = rng.binomial(
        1,
        examination * attractiveness,
    ).astype(np.int32)

    return log.sort_values(
        ["qid", "shown_position"],
        kind="stable",
    ).reset_index(drop=True)


click_logs = simulate_click_log(train)
click_features = [
    "semantic_match",
    "lexical_match",
    "price_fit",
    "brand_affinity",
    "popularity",
    "fast_shipping",
    "in_stock",
]

click_ranker = xgb.XGBRanker(
    objective="rank:ndcg",
    eval_metric="ndcg@10",
    tree_method="hist",
    n_estimators=400,
    max_depth=4,
    learning_rate=0.05,
    lambdarank_pair_method="topk",
    lambdarank_num_pair_per_sample=12,
    lambdarank_unbiased=True,
    lambdarank_bias_norm=2.0,
    random_state=SEED,
)

click_ranker.fit(
    click_logs[click_features],
    click_logs["clicked"],
    qid=click_logs["qid"].to_numpy(),
)

Keep shown_position and the logging-policy version for auditing, but do not casually use position as a feature. Otherwise the replacement model can learn to reproduce the old ranker. Preserve unclicked impressions, candidate-set boundaries, exploration traffic, and a separate human-judged evaluation set.

Cherry on the cake: Airbnb’s production relevance jump

In a 2025 CIKM paper, Airbnb described a co-trained pointwise initial ranker and setwise Transformer re-ranker. The deployed system re-ranked the top 40 listings and trained on roughly 360 million booking-labeled examples.

The paper reports a 1.7% offline NDCG improvement. In a three-week online experiment, the system increased booking conversion by 0.6% and reduced the number of listing-detail pages viewed before booking by 0.88%.

The lesson is not that every search stack needs a deep Transformer. It is that moving beyond isolated pointwise scoring—and aligning training with list context—can produce measurable relevance and business gains.

Production checklist

Before shipping a ranker:

• Split and cross-validate by query, user, or session.

• Sort rows and qid with one shared permutation.

• Compute metrics independently per query, then average.

• Report NDCG and MAP at the actual serving cutoff.

• Define the binary relevance threshold used for MAP.

• Tune pair construction alongside tree parameters.

• Segment head, tail, new, navigational, and ambiguous queries.

• Validate click-trained models against less-biased labels.

• Run shadow traffic and an online experiment before replacement.

Run the seeded experiment, replace the synthetic table with one real query-candidate dataset, add the group-alignment assertions, and make NDCG@k—not regression loss—the release gate for your next search or recommendation model.