Chi-squared tests for categorical data: independence and goodness-of-fit

Why categorical data needs a different statistical test

Many important questions in data analysis involve categories rather than numerical measurements:

• Do customers with different account types prefer different notification channels?

• Does a survey’s distribution of product preferences match an established benchmark?

• Is payment method associated with a customer’s region?

• Do different age groups prefer different product features?

• Are support tickets distributed across categories in the proportions a company expects?

These questions cannot always be answered by comparing averages. Categories such as email, push notifications, and SMS do not have meaningful numerical averages. Instead, we count how often each category occurs and compare those counts with what we would expect under a particular statistical assumption.

The chi-squared test, commonly written as the χ² test, provides a way to evaluate whether differences between observed and expected categorical counts are consistent with random variation or provide evidence against a specified hypothesis.

There are two main applications:

• Chi-squared test of independence: examines whether two categorical variables are associated.

• Chi-squared goodness-of-fit test: examines whether one categorical variable follows a specified distribution.

Both tests use the Pearson chi-squared statistic, but they answer different questions. Choosing the correct test is the first step toward a defensible statistical conclusion.

This tutorial covers both tests using runnable Python examples and SciPy, explains how to interpret p-values, examines expected-count limitations, and demonstrates when Fisher’s exact test is a better choice.

1. Understanding the chi-squared statistic

The Pearson chi-squared statistic measures how far observed counts differ from expected counts.

Its formula is:

χ² = Σ (Observed − Expected)² / Expected

For each category or table cell, the test performs three operations:

1. Subtracts the expected count from the observed count.

2. Squares the difference and divides it by the expected count.

3. Adds the contributions from all categories or cells.

This calculation has two important properties.

First, differences are evaluated relative to expected frequencies. A discrepancy of 10 observations matters more when 10 were expected than when 1,000 were expected.

Second, squaring the differences prevents positive and negative deviations from canceling each other out.

A larger chi-squared statistic generally indicates a greater discrepancy between the observed data and the null hypothesis. However, the statistic alone does not tell us whether the result is statistically significant. We must also consider the degrees of freedom and the corresponding p-value.

The degrees of freedom depend on the structure of the test and the constraints imposed by the null hypothesis. SciPy uses the relevant chi-squared distribution to calculate the p-value for the conventional Pearson test.

2. Set up a reproducible Python environment

We will use NumPy to represent frequency counts and SciPy to perform the statistical tests.

For this tutorial, use Python 3.12 or newer, NumPy 2.5.4, and SciPy 1.18.1. These versions are current reference points as of October 2026; SciPy 1.18.1 was released in August 2026, and NumPy 2.5.4 was released in October 2026. You can verify the releases through the official SciPy package page and NumPy package page.

On macOS or Linux, create an isolated environment and install the pinned dependencies:

python -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install numpy==2.5.4 scipy==1.18.1

On Windows, use the equivalent environment and interpreter paths:

py -m venv .venv
.venv\Scripts\python.exe -m pip install --upgrade pip
.venv\Scripts\python.exe -m pip install numpy==2.5.4 scipy==1.18.1

The version pins make the examples easier to reproduce. For a production project, record and maintain the dependency versions in your project’s own environment or lock file.

The examples use three current SciPy functions:

• scipy.stats.chi2_contingency for contingency-table analysis.

• scipy.stats.chisquare for goodness-of-fit testing.

• scipy.stats.fisher_exact for Fisher’s exact test.

One important preparation step is to understand what your numbers represent. These tests operate on observed frequency counts, not arbitrary scores, averages, or percentages.

If your dataset contains one row per customer, for example, you generally need to aggregate those rows into a contingency table before performing a chi-squared test of independence.

3. The chi-squared test of independence

What question does it answer?

The chi-squared test of independence asks whether two categorical variables are statistically associated.

Consider a fictional online retailer investigating the relationship between customer type and preferred notification channel. The retailer surveys 150 customers, asking whether they are new or returning customers and whether they prefer email, push notifications, or SMS.

The observed counts are:

Customer type	Email	Push	SMS	Total
New	30	20	10	60
Returning	20	30	40	90
Total	50	50	50	150

The pattern appears substantial. Half of the new customers prefer email, while returning customers appear more likely to prefer SMS.

But is this pattern strong enough to provide statistical evidence of an association, rather than simply reflecting sampling variation?

The chi-squared test of independence addresses that question.

Define the hypotheses

Before running the test, specify the null and alternative hypotheses.

• Null hypothesis (H₀): Customer type and preferred notification channel are independent.

• Alternative hypothesis (H₁): Customer type and preferred notification channel are associated.

The null hypothesis does not claim that every category must have identical counts. Different group sizes naturally produce different totals.

Instead, independence means that knowing a customer’s type does not change the distribution of notification preferences in the population being studied.

Calculate the expected counts

If customer type and notification preference were independent, we could estimate the expected count for each cell using its row and column totals.

Expected count = (Row total × Column total) / Grand total

For new customers who prefer email:

Expected count = (60 × 50) / 150 = 20

For returning customers who prefer email:

Expected count = (90 × 50) / 150 = 30

Applying the same calculation to every cell gives:

Customer type	Email	Push	SMS
New	20	20	20
Returning	30	30	30

These are the frequencies expected under the independence hypothesis, given the observed marginal totals.

Notice that the expected values preserve the different row totals. Independence does not require the two customer groups to contain the same number of people. It requires their category proportions to follow the same distribution.

Run the test with SciPy

The chi2_contingency function calculates the chi-squared statistic, p-value, degrees of freedom, and expected frequencies directly from the observed table.

The following example is complete and runnable:

import numpy as np
from scipy.stats import chi2_contingency

observed = np.array([
    [30, 20, 10],
    [20, 30, 40],
])

result = chi2_contingency(observed, correction=False)

print(f"Chi-squared statistic: {result.statistic:.3f}")
print(f"Degrees of freedom: {result.dof}")
print(f"P-value: {result.pvalue:.6f}")
print("Expected frequencies:")
print(result.expected_freq)

n = observed.sum()
rows, columns = observed.shape

cramers_v = np.sqrt(
    result.statistic / (n * min(rows - 1, columns - 1))
)

print(f"Cramer's V: {cramers_v:.3f}")

The output is approximately:

Chi-squared statistic: 16.667
Degrees of freedom: 2
P-value: 0.000240
Expected frequencies:
[[20. 20. 20.]
 [30. 30. 30.]]
Cramer's V: 0.333

The correction=False argument explicitly requests the uncorrected Pearson statistic. Yates’ continuity correction is relevant to tables with one degree of freedom, such as a 2 × 2 table, rather than this 2 × 3 example.

Interpret the result

The test produces a chi-squared statistic of approximately 16.667, with two degrees of freedom and a p-value of approximately 0.000240.

Suppose we selected a significance threshold of 0.05 before running the test.

Because the p-value is below that threshold, we reject the null hypothesis. The data provide evidence of an association between customer type and notification preference.

The p-value means that, if the null hypothesis and the test assumptions held, a chi-squared statistic at least as large as the observed statistic would occur with a probability of approximately 0.024%.

It does not mean there is a 99.976% probability that the alternative hypothesis is true. A p-value is calculated under the null hypothesis; it is not the probability that a hypothesis is correct.

The original table helps explain the nature of the association:

• New customers selected email in 50% of cases.

• Returning customers selected email in approximately 22.2% of cases.

• New customers selected SMS in approximately 16.7% of cases.

• Returning customers selected SMS in approximately 44.4% of cases.

These percentages reveal the pattern more clearly than the p-value alone.

The example also calculates Cramér’s V, an effect-size measure for categorical association. Its value of approximately 0.333 adds context about the strength of the observed pattern. There is no universal effect-size threshold suitable for every application, so interpret Cramér’s V alongside the table dimensions, business context, and consequences of the decision.

Finally, statistical association does not establish causation. This survey cannot demonstrate that becoming a returning customer causes someone to prefer SMS.

4. The chi-squared goodness-of-fit test

The goodness-of-fit test answers a different question.

Instead of comparing two categorical variables, it compares the observed distribution of one categorical variable with a distribution specified by the analyst.

Imagine that a retailer has an established benchmark for how shoppers should distribute their preferences across three purchasing channels:

• Online: 40%

• In-store: 35%

• Marketplace: 25%

These proportions might come from a historical benchmark or a business target specified before the new survey begins.

The retailer surveys 100 randomly selected shoppers and records the following responses:

Preferred channel	Observed count	Expected proportion	Expected count
Online	55	40%	40
In-store	25	35%	35
Marketplace	20	25%	25
Total	100	100%	100

The question is whether the observed distribution differs significantly from the specified benchmark.

Define the hypotheses

• Null hypothesis (H₀): The population distribution follows the specified proportions of 40%, 35%, and 25%.

• Alternative hypothesis (H₁): The population distribution differs from those proportions in at least one category.

Unlike the independence test, this test requires a reference distribution. The expected proportions must come from an independently specified benchmark or a model whose parameter estimation is handled appropriately.

Run the goodness-of-fit test

SciPy’s chisquare function accepts observed and expected frequency arrays.

import numpy as np
from scipy.stats import chisquare

observed = np.array([55, 25, 20])

expected_proportions = np.array([0.40, 0.35, 0.25])
expected = observed.sum() * expected_proportions

result = chisquare(
    f_obs=observed,
    f_exp=expected,
)

print(f"Chi-squared statistic: {result.statistic:.3f}")
print(f"Degrees of freedom: {len(observed) - 1}")
print(f"P-value: {result.pvalue:.6f}")
print(f"Expected counts: {expected}")

The result is approximately:

Chi-squared statistic: 9.482
Degrees of freedom: 2
P-value: 0.008729
Expected counts: [40. 35. 25.]

Because the p-value is below 0.05, the observed distribution provides evidence against the specified benchmark.

The online preference count is 15 higher than expected, while the in-store and marketplace counts are lower than expected. These deviations contribute to the overall chi-squared statistic.

The test does not automatically identify which categories differ significantly from their expected values when considered individually. If you investigate individual categories in follow-up tests, use an appropriate procedure and account for multiple comparisons.

Understand the degrees of freedom

For a goodness-of-fit test with k categories and fixed, prespecified probabilities, the usual degrees of freedom are:

Degrees of freedom = k − 1

One degree of freedom is lost because the observed counts must sum to the total sample size.

If parameters of the reference distribution are estimated from the same observations, the degrees of freedom may need to be adjusted to account for those estimated parameters. SciPy’s chisquare function supports this through its ddof argument.

This distinction matters because testing against a prespecified benchmark is not statistically identical to testing against a distribution fitted using the same observations.

5. Independence versus goodness of fit: which test should you use?

Both tests use the chi-squared statistic, but their hypotheses and inputs differ.

Research question	Appropriate test	Typical input
Is customer type associated with notification preference?	Chi-squared test of independence	Two-dimensional contingency table
Does the distribution of channel preferences match a specified benchmark?	Chi-squared goodness-of-fit test	Observed counts and expected counts
Does a small 2 × 2 table provide evidence of an association?	Fisher’s exact test may be appropriate	Small 2 × 2 contingency table

A useful decision rule is to identify the structure of your question before writing code.

If you have two categorical variables and want to test their association, start with a contingency table and consider chi2_contingency.

If you have one categorical variable and want to compare its distribution with specified probabilities, consider chisquare.

If the counts are small, do not assume that the standard chi-squared approximation will automatically be reliable. Examine the expected counts and consider an exact or suitable resampling-based method.

6. The expected-count caveat: when the standard test becomes unreliable

One of the most important considerations in chi-squared testing concerns expected cell frequencies.

A commonly cited conservative guideline is that expected frequencies should be at least five in every cell. This is a rule of thumb for assessing the accuracy of the chi-squared approximation, not an absolute law that automatically invalidates every table containing a smaller expected count.

The concern is that the standard test uses a theoretical chi-squared distribution to approximate the sampling distribution of the statistic. When expected counts are very small, that approximation can be inaccurate, making the reported p-value unreliable.

Pay particular attention when:

• The sample size is small.

• The table contains many categories relative to the sample size.

• One or more categories are rare.

• Some expected cell frequencies are close to zero.

• The table is highly unbalanced.

Always inspect the expected frequencies returned by chi2_contingency. A large total sample does not guarantee that every cell has enough expected observations.

Use Fisher’s exact test for a small 2 × 2 table

Consider a small survey that compares two groups and a binary preference.

Group	Prefers A	Prefers B
Group 1	4	1
Group 2	1	4

The total sample is only 10, and the expected count in each cell is 2.5. This is a situation in which the usual large-sample approximation deserves caution.

Fisher’s exact test calculates a p-value using an exact conditional distribution rather than relying on the standard chi-squared approximation.

from scipy.stats import fisher_exact

small_table = [
    [4, 1],
    [1, 4],
]

result = fisher_exact(
    small_table,
    alternative="two-sided",
)

print(f"Odds ratio: {result.statistic:.3f}")
print(f"P-value: {result.pvalue:.6f}")

The output is:

Odds ratio: 16.000
P-value: 0.206349

Despite the visible difference between the groups, the two-sided p-value is approximately 0.206. At the 0.05 significance level, this sample does not provide sufficient evidence to reject the null hypothesis.

That conclusion does not establish that the groups are identical. With only 10 observations, the data provide limited information, and uncertainty remains substantial.

Fisher’s exact test is a common choice for small 2 × 2 tables. For larger tables, the appropriate method depends on the table dimensions, sampling design, and computational requirements. Current SciPy documentation also describes resampling-based approaches for supported contingency-table tests.

Do not respond to small expected counts by automatically merging categories. Combining categories may make a test more suitable computationally, but it can also erase meaningful distinctions. Only combine categories when there is a defensible substantive reason to do so.

For implementation details, consult the current SciPy documentation for Fisher’s exact test.

7. Other assumptions that deserve attention

Checking expected counts is necessary, but it is not sufficient. A statistically valid test also depends on how the data were collected.

Independent observations

The standard tests assume that observations satisfy the relevant independence conditions.

If the same customer answers a survey multiple times, several observations come from the same household, or responses are strongly clustered by location, treating every response as independent may underestimate uncertainty.

Repeated measurements, matched observations, and clustered survey designs can require different statistical methods.

Mutually exclusive categories

Each observation should be assigned consistently to the relevant categories.

In a simple preference survey, each respondent might choose exactly one notification channel. If respondents can select multiple options, the resulting counts do not automatically form the kind of mutually exclusive categorical table assumed by the standard test.

The analysis must reflect the actual response structure.

Appropriate sampling and measurement

A significant p-value cannot correct a biased sample, ambiguous survey questions, nonresponse bias, or poorly defined categories.

If a survey only reaches highly engaged customers, its results might not represent all customers, regardless of how carefully the chi-squared statistic is calculated.

Multiple testing

Suppose an analyst tests 40 customer attributes against notification preference and reports only the variables with p-values below 0.05.

Even if all the null hypotheses are true, running many tests increases the chance of obtaining at least one apparently significant result.

Predefine hypotheses where possible, and consider a suitable multiple-testing correction when conducting many comparisons.

Statistical significance should be interpreted as one part of an analysis, not as automatic authorization to act.

8. The cherry on the cake: a recent survey where the apparent story needed to change

A useful real-world example comes from a 2026 study published in Business Strategy and the Environment, From Awareness to Action: Consumer Behaviour, Attitudes and Business Strategy in Sustainable and Circular Electronic Devices.

The researchers surveyed 407 Italian consumers about their purchasing habits, electronic-device disposal practices, environmental awareness, and preferred strategies for encouraging the collection of waste electrical and electronic equipment.

Consider one specific question: which interventions would respondents find most effective for encouraging electronic-waste collection?

The reported percentages appear to tell a generational story:

• Respondents aged 18–24 were more likely to favor information campaigns, at 19.17%.

• The corresponding proportions were 12.03% among respondents aged 25–40 and 11.11% among those over 40.

• Preferences for economic incentives ranged from 29.02% among the youngest respondents to 35.80% among those over 40.

A chart-first interpretation might be that younger consumers want educational campaigns while older consumers prefer financial incentives, meaning that each age group needs a different strategy.

That interpretation sounds plausible. But do the observed differences provide sufficient statistical evidence to support it?

The study reported a chi-squared test of independence for age group and preferred collection strategy:

• Chi-squared statistic: 7.55

• Degrees of freedom: 8

• p-value: 0.478

The p-value is well above the conventional 0.05 significance threshold. The test therefore does not provide sufficient evidence of an association between age group and preferred strategy in this sample.

The conclusion needs to be narrowed. The percentages describe differences in the surveyed respondents, but they do not establish that age groups have statistically different distributions of preferred interventions. A claim that businesses should choose strategies based specifically on age would require stronger supporting evidence.

This does not prove that every age group has identical preferences. It means the observed differences were not statistically convincing under this test.

The same study reported significant age associations for other outcomes, including electronic-device disposal practices, with χ²(8) = 34.22 and p less than 0.001, and awareness of collection points, with χ²(2) = 28.42 and p less than 0.001.

That distinction is important: age may be associated with actual disposal behavior and awareness without providing sufficient evidence that age predicts which collection strategy respondents prefer.

The lesson is not that chi-squared tests are infallible. It is that a statistical test can prevent an analyst from turning an appealing descriptive pattern into a stronger claim than the evidence supports.

9. A practical checklist for your next analysis

Before publishing a chi-squared result or using it to guide a product decision, work through the following checklist.

• Define the question. Are you testing an association between two categorical variables or comparing one distribution with a benchmark?

• State the hypotheses. Specify the null hypothesis before examining the result.

• Use counts. Construct the contingency table from observed frequencies rather than feeding percentages into the test as if they were counts.

• Calculate expected frequencies. Verify that the table supports the chosen approximation.

• Check the sampling design. Investigate repeated responses, clustering, weighting, and other sources of dependence.

• Choose the appropriate method. Consider Fisher’s exact test or a suitable resampling method when the usual approximation is questionable.

• Interpret the p-value correctly. Compare it with a prespecified significance level without treating it as the probability that the null hypothesis is true.

• Inspect the pattern. Examine the observed counts, row or column percentages, and relevant effect sizes.

• Avoid causal claims. An association alone does not demonstrate a causal relationship.

• Report limitations. Explain what the sample and study design allow you to conclude.

For reproducible reporting, include the observed table, test statistic, degrees of freedom where applicable, p-value, and a plain-language conclusion. If the result is statistically significant, explain what the observed pattern looks like and whether its magnitude is practically meaningful.

10. References and further reading

The following current resources provide implementation details and further guidance:

• SciPy: chi2_contingency — contingency tables, expected frequencies, degrees of freedom, and supported resampling options.

• SciPy: chisquare — Pearson’s goodness-of-fit test and degrees-of-freedom adjustments.

• SciPy: fisher_exact — exact testing for contingency tables.

• SciPy 1.18.1 on PyPI — release information and supported Python versions.

• NumPy 2.5.4 on PyPI — current package release information.

• D’Adamo and colleagues’ 2026 survey study — a real-world example of interpreting categorical survey results.

Your next step

Run a chi-squared test on a categorical dataset from your own project today. Choose a dataset involving customer preferences, retail orders, support-ticket categories, or another everyday domain; build the appropriate contingency table; run the test in SciPy; and inspect the expected counts before interpreting the p-value.

Write down your null hypothesis before you run the code, then record the test statistic, p-value, and a plain-language conclusion that distinguishes statistical evidence from practical importance.

Once you have completed both examples, continue to the next lesson and explore how effect sizes, confidence intervals, and more advanced categorical-data methods can help turn statistical results into better decisions.