Rule learning: RIPPER, decision lists, and models that output human-readable rules

Why learn rules instead of another black box?

Most classifiers return a label or probability. A rule learner returns logic that a person can inspect and execute:

Predict malignant when worst_radius > 17.8 AND worst_texture > 19.4.
Otherwise, continue checking the remaining rules.

That difference matters when a model must be reviewed by domain experts, translated into operational policy, audited for problematic conditions, or implemented without a large inference stack. Rule learning is not post-hoc explanation: the readable rules are the predictive model.

As of July 2026, the practical Python choices include wittgenstein 0.3.5 for IREP and RIPPER-style rulesets, imodels 2.0.4 for rule lists, rule sets, scoring systems, and compact trees, and scikit-learn 1.9.0 for a strong decision-tree baseline. wittgenstein 0.3.5 added per-rule precision, recall, F1, and coverage statistics, plus minimum-sample controls for rules.

Rule sets, decision lists, and decision trees

These families all use conditions, but their prediction semantics differ.

Rule set: any matching rule is enough

A binary ruleset usually describes the positive class as an OR of AND clauses:

positive = rule_1 OR rule_2 OR rule_3
rule_1 = condition_a AND condition_b

In wittgenstein output, ^ means AND and V means OR. Multiple rules may match one row; any match produces the positive prediction. If no rule matches, the model returns the default negative class.

Decision list: the first match wins

A decision list is ordered:

IF rule_1 matches: return positive
ELSE IF rule_2 matches: return negative
ELSE IF rule_3 matches: return positive
ELSE: return the default

Reordering two entries can change predictions. This structure fits policies with exceptions, escalation steps, or explicit precedence. Current imodels releases expose GreedyRuleListClassifier, BayesianRuleListClassifier, and OneRClassifier.

Decision tree: follow one path

A tree repeatedly branches until a leaf is reached. Every root-to-leaf path can be rewritten as a rule, but the complete model remains hierarchical rather than flat. Trees can represent interactions efficiently, yet even a modest tree may require users to trace several branches and compare many leaves.

Scikit-learn’s current export_text function prints a fitted tree as text and supports feature and class names. Its documentation also warns that the textual representation is not guaranteed to remain backward compatible, so store the fitted model and version metadata rather than treating exported text as a serialization format.

Question	RIPPER ruleset	Decision list	Decision tree
How is a prediction selected?	Any positive rule matches	First matching rule wins	One path reaches a leaf
Is order important?	Usually no	Yes	Branch structure is essential
Default outcome?	Negative class	Final fallback	Every path has a leaf
Can several rules explain one row?	Yes	No, only the first match	Normally one path
Main complexity budget	Rules and conditions	Entries and conditions	Depth and leaves

How RIPPER learns a compact ruleset

RIPPER stands for Repeated Incremental Pruning to Produce Error Reduction. The implementation is more involved than a simple greedy loop, but the useful mental model is:

1. Choose a positive class to describe explicitly.

2. Grow a rule by adding conditions that separate positive examples from negatives.

3. Prune trailing conditions using held-out pruning data.

4. Add the rule and remove examples it covers.

5. Repeat until stopping criteria are reached.

6. Revisit rules through replacement and revision passes.

7. Remove rules that do not improve the description-length objective.

The current wittgenstein source separates initial growth, optimization iterations, coverage of remaining positives, and a final description-length reduction stage. Its constructor exposes limits such as max_rules, max_rule_conds, max_total_conds, min_rule_samples, and min_ruleset_samples.

These controls create an explicit interpretability budget. Instead of merely asking for an interpretable model, you can require, for example:

• No more than eight rules

• No more than three conditions per rule

• At least five covered training rows before rule growth continues

• No more than 18 conditions across the complete ruleset

Runnable demo: RIPPER versus a compact tree

Install the current versions used here:

python -m pip install \
    "wittgenstein==0.3.5" \
    "scikit-learn==1.9.0"

The program below learns rules for malignant tumors in scikit-learn’s breast-cancer dataset, prints per-rule statistics, and compares the result with a constrained decision tree.

import numpy as np
import pandas as pd
import wittgenstein as lw

from sklearn.datasets import load_breast_cancer
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, export_text


def main() -> None:
    dataset = load_breast_cancer(as_frame=True)

    X = dataset.data.copy()
    X.columns = [
        column.lower().replace(" ", "_")
        for column in X.columns
    ]

    # The source dataset labels malignant as 0 and benign as 1.
    # Re-map the target so RIPPER's positive class means malignant.
    y = pd.Series(
        (dataset.target == 0).astype(int),
        index=X.index,
        name="malignant",
    )

    X_train, X_test, y_train, y_test = train_test_split(
        X,
        y,
        test_size=0.25,
        random_state=42,
        stratify=y,
    )

    ripper = lw.RIPPER(
        k=2,
        prune_size=0.33,
        n_discretize_bins=8,
        max_rules=8,
        max_rule_conds=3,
        max_total_conds=18,
        min_rule_samples=5,
        random_state=42,
    )
    ripper.fit(X_train, y_train, pos_class=1)

    ripper_predictions = np.asarray(
        ripper.predict(X_test),
        dtype=int,
    )

    print("RIPPER rules")
    print("------------")
    ripper.out_model()

    print("\nRIPPER test metrics")
    print(
        classification_report(
            y_test,
            ripper_predictions,
            target_names=["benign", "malignant"],
            digits=3,
            zero_division=0,
        )
    )

    rule_statistics, _ = ripper.rule_stats(
        X_train,
        y_train,
        n_examples=2,
    )

    print("Per-rule training statistics")
    print(
        rule_statistics[
            [
                "rule_idx",
                "rule_str",
                "coverage",
                "precision",
                "recall",
                "f1",
            ]
        ].to_string(index=False)
    )

    tree = DecisionTreeClassifier(
        max_depth=4,
        max_leaf_nodes=8,
        min_samples_leaf=12,
        random_state=42,
    )
    tree.fit(X_train, y_train)
    tree_predictions = tree.predict(X_test)

    print("\nDecision-tree rules")
    print("-------------------")
    print(
        export_text(
            tree,
            feature_names=list(X.columns),
            class_names=["benign", "malignant"],
            decimals=2,
            show_weights=True,
        )
    )

    print("Decision-tree test metrics")
    print(
        classification_report(
            y_test,
            tree_predictions,
            target_names=["benign", "malignant"],
            digits=3,
            zero_division=0,
        )
    )

    rule_count = len(ripper.ruleset_.rules)
    condition_count = sum(
        len(rule.conds)
        for rule in ripper.ruleset_.rules
    )

    print("Complexity summary")
    print(f"RIPPER rules: {rule_count}")
    print(f"RIPPER conditions: {condition_count}")
    print(f"Tree leaves: {tree.get_n_leaves()}")
    print(f"Tree depth: {tree.get_depth()}")


if __name__ == "__main__":
    main()

The exact thresholds can vary with the split, pruning sample, and discretization settings. Focus on the model’s shape: RIPPER produces sufficient conditions for malignancy plus an implicit benign default; the tree produces nested branches for both classes.

The rule_stats call is especially useful. In version 0.3.5 it returns rule-level coverage, covered-row count, precision, recall, and F1, with optional example sampling.

Read rule output correctly

Suppose the fitted model looks like this:

[[condition_a ^ condition_b] V
 [condition_c] V
 [condition_d ^ condition_e]]

Read it as:

• Predict positive when both A and B hold

• OR when C holds

• OR when both D and E hold

• Otherwise predict negative

Two cautions matter.

First, a rule is predictive, not causal. Changing a feature named in a rule does not necessarily change the real-world outcome.

Second, a binary ruleset is asymmetric. It explicitly describes the positive class; the negative result means only that no positive rule matched. It is not an equally detailed explanation of why the example is negative.

Evaluate accuracy and cognitive cost together

Do not select a rule model from accuracy alone. Report:

• Precision, recall, and F1 for the operationally important class

• Number of rules and total conditions

• Coverage and precision for every rule

• Stability across repeated splits or bootstrap samples

• Results for relevant demographic or operational subgroups

• Sensitivity near numeric bin boundaries

• Whether a domain expert can accurately restate the model

A rule with perfect training precision but 1% coverage may be correct yet unimportant. A broad rule with 70% precision may create too many false alarms. Per-rule metrics reveal that trade-off more clearly than a single global score.

Where readable rules still fail

Readable syntax does not guarantee a trustworthy system.

• Rule explosion: Twenty short rules can be harder to review than one small tree.

• Threshold instability: Small measurement changes near a bin boundary can flip a decision.

• Proxy discrimination: A rule may omit a protected attribute while using a strong proxy.

• Weak probability calibration: Crisp rules naturally express decisions; their probability estimates may need separate validation.

• Distribution shift: A beautifully documented rule becomes wrong when populations, sensors, prices, or procedures change.

• False confidence: Human-readable logic can look authoritative even when learned from biased or insufficient data.

Cherry on the cake: credit decisions need specific reasons

U.S. Regulation B currently requires adverse-action notices to state specific principal reasons. Saying that an applicant failed an institution’s internal policy or did not reach a qualifying score is insufficient. The rule does not mandate RIPPER, decision lists, or any particular model family—but it creates a practical problem for systems that cannot reliably map a decision to accurate reasons.

That is where inherently readable models become more than a visualization preference. A compact ruleset can be reviewed, versioned, monitored, and mapped to reason codes using the same logic that makes the prediction. It still requires legal review, fairness testing, validation, and governance; readability simply reduces the gap between model behavior and the explanation supplied to people.

Put rule learning into practice

Take one binary tabular problem you already understand. Train RIPPER and a constrained decision tree on the same splits, then compare predictive metrics, model size, rule coverage, threshold sensitivity, and stability across at least five seeds.

Set the interpretability budget before tuning. Then run the demo, replace the dataset with your own, and choose the model your team can defend, monitor, and safely operate—not merely the one with the highest score.