Statistical inference · Lesson 10 of 13

Chi-square tests: goodness of fit and independence

The tests so far compare means. A great many questions are about counts: how many households fall in each category, and whether that pattern is what a theory predicts or what independence would produce. One statistic handles both. This lesson shows where it comes from, why the degrees of freedom are what they are, and why a significant chi-square can still mean a weak association.

← Statistical inference moduleLesson 10 of 13 · about 50 minutes

A question to hold on to

A die is rolled 120 times and the faces come up 25, 17, 15, 23, 24 and 16 times. A fair die would give 20 of each. The most common face has appeared 25 times and the least common 15: is that evidence the die is loaded, or does a fair die often scatter this much?

A classic, with a dieSolved at the end ↓

New here?What is Ashanagar?The invented district behind this page's examples: 60 households, with incomes you can inspect.

Ashanagar is an invented district. The statistics lessons use it for every worked example, so you meet the same households again as the ideas build: first as a list of numbers, then as a spread, then as a population to sample from.

It has 60 households, each with a monthly income in rupees. Unless a page says otherwise, those 60 are the whole population, and a “sample” means a few of them drawn at random. Where a page needs two variables, it uses a smaller set of 12 households with the head’s years of schooling beside income.

The figures are illustrative, made up once and fixed. They are not official statistics.

Households: 60
Mean income: ₹28,783 a month
Median income: ₹25,000 a month
Standard deviation: ₹13,366
Lowest and highest: ₹10,500 and ₹72,000

Meet the full data set →

The idea

Tests for counts

Suppose you count how many items fall into each of k categories. A theory, or the assumption of independence, tells you how many should fall in each: the expected counts E. The chi-square statistic measures the total disagreement between what you observed, O, and what you expected, in a way that treats every cell fairly.

The chi-square statistic

χ^{2} = cells \sum \frac{( O - E ) ^{2}}{E}

Add over every cell. Large values mean the data are far from what the null hypothesis expects, so the test rejects in the upper tail only.

Three features are worth noticing at once. It is never negative. It is zero only when every observed count equals its expectation. And each cell’s gap is divided by E, because being 5 off in a cell that expected 20 is a serious miss and being 5 off in a cell that expected 2,000 is nothing. Two uses cover almost every exam question: goodness of fit (one variable against a stated distribution) and independence (two variables in a table).

Interactive

Goodness of fit

The null hypothesis names the probability of each category, so the expected count is E = n × p for each. The statistic is compared with a chi-square distribution on k − 1 degrees of freedom, where k is the number of categories (CUET 2019 asks which parameter fixes the shape of the chi-square distribution: it is the degrees of freedom).

Number of rolls, n120How loaded the die is towards sixfairAt 0 the die is fair and the null hypothesis is true.

Statistic, Σ(O − E)²/E4.7

Critical value, 5 df11.07

p-value0.454

observed countexpected if fair, 20

2,000 fair-die experimentsChi-square curve, 5 df5% cut-offBlack line: this experiment

What it shows

The die is fair, yet the counts are not all 20: that scatter is exactly what chance produces. The statistic 4.7 falls well inside the bulk of what a fair die gives. The gold bars, built by actually rolling fair dice, trace the chi-square curve.

The lab is the die of the opening question. With the die fair, press Roll the die again many times and watch the black line wander about inside the gold bars: those 2,000 gold bars are fair-die statistics, and they follow the blue curve. A statistic beyond the red 5% line happens about one time in twenty even when nothing is wrong. Then load the die towards six and raise n: a bias that no one could see at 30 rolls is rejected at 600.

Derive it

Why the sum of (O − E)²/E is a chi-square, with one degree of freedom lost

We want to show

For counts in k categories,

\sum_{i = 1}^{k} \frac{( O _{i} - E _{i} ) ^{2}}{E _{i}}

behaves like a chi-square with k − 1 degrees of freedom, and with k − 1 − m after m parameters are estimated from the data.

Start with two categories, where everything can be checked.
A coin is tossed n times and X heads appear. The two categories are heads and tails, with observed counts X and n − X and expected counts np and n(1 − p), where p is the chance of heads under H₀. A fair coin tossed 100 times has expected counts 50 and 50. Suppose 60 heads and 40 tails appear. The statistic is
$\frac{( 60 - 50 ) ^{2}}{50} + \frac{( 40 - 50 ) ^{2}}{50} = 2 + 2 = 4$
The two cells carry one piece of information, not two.
Because the counts must add to n, the shortfall in one cell is exactly the excess in the other: O₂ − E₂ = −(O₁ − E₁). Both squared deviations are the same number d² = (X − np)². Then
$\frac{d ^{2}}{n p} + \frac{d ^{2}}{n ( 1 - p )} = d^{2} \cdot \frac{( 1 - p ) + p}{n p ( 1 - p )} = \frac{( X - n p ) ^{2}}{n p ( 1 - p )}$
That is the square of a z statistic.
X has mean np and variance np(1 − p), so (X − np) ÷ √(np(1 − p)) is approximately standard normal for large n. Our two-cell statistic is the square of it. Here z = (60 − 50) ÷ 5 = 2 and z² = 4. A squared standard normal is a chi-square with one degree of freedom (the chi-square derivation). Two categories, one degree of freedom: k − 1.
A check on the bookkeeping for k categories: the count in cell i has variance np_i(1 − p_i) = E_i(1 − p_i), so each term (O_i − E_i)² ÷ E_i has expected value 1 − p_i. Adding over the cells gives Σ(1 − p_i) = k − 1, the mean of a chi-square with k − 1 degrees of freedom. A fair die has k = 6, so the statistic averages 5.
With k categories, count the free quantities.
Each standardised deviation (O_i − E_i) ÷ √E_i is roughly normal. If the k of them were independent, the sum of their squares would be a chi-square with k degrees of freedom. They are not: the counts add to n, so the deviations add to zero, which ties the last one to the others. One linear constraint takes away one degree of freedom, leaving k − 1. Each parameter you estimate from the same data, such as a Poisson mean, fits the expected counts to the observed ones and takes away one more.

Step 1 of 4

Mendel's peas

In a cross of pea plants, 556 offspring show the four types 315, 108, 101 and 32. Mendel’s theory predicts the ratio 9 : 3 : 3 : 1. Does the data fit?

Expected counts.
556 × 9/16 = 312.75, 556 × 3/16 = 104.25 (twice) and 556 × 1/16 = 34.75.
Add up (O − E)²/E.
0.016 + 0.135 + 0.101 + 0.218 = 0.47.
Compare.
Four categories give 3 degrees of freedom, and the 5% critical value is 7.815. The statistic is far below it (p = 0.925).

Answer

The data are entirely consistent with 9 : 3 : 3 : 1. A large p-value means the data do not contradict the theory, which is not the same as proving it.

When you estimate a parameter first

A theory does not always name the probabilities. “The number of power cuts per week is Poisson” leaves the mean open. In Ashanagar, 100 weeks give 19, 36, 24, 13, 6 and 2 weeks with 0, 1, 2, 3, 4 and 5 cuts. The mean is 1.57 cuts a week, and the Poisson probabilities for that mean give expected counts of 20.8, 32.66, 25.64, 13.42, 7.47 for 0, 1, 2, 3 and “4 or more”. The last two observed cells (4 and 5 cuts) were joined because their expected counts were below 5. The statistic is 0.653.

Now the degrees of freedom. There are 5 categories, so k − 1 = 4, but the mean was estimated from the same data, which takes one more: 5 − 1 − 1 = 3. The 5% critical value is 7.815, and 0.65 is far inside it, so a Poisson fits well. Forgetting the extra degree of freedom makes you compare with 9.49 instead of 7.81 and accept fits that should be questioned.

Watch out for

The mistakes examiners are fishing for

Using percentages instead of counts. The statistic needs the actual counts. Percentages give a number that is not chi-square at all.
Forgetting to lose a degree of freedom for each estimated parameter. A fully specified null has k − 1. An estimated mean or proportion costs one more each.
Small expected counts. The chi-square approximation fails when expected counts fall below about 5. Join neighbouring categories first, as the power-cut example did.
Rejecting in both tails. Only a large statistic signals a bad fit. A very small one is suspiciously good, which is a matter for an auditor and not a rejection.

Interactive

Independence in a table

For two categorical variables arranged in a table of counts, the null hypothesis is that they are unrelated. If they are, each cell should hold the count E = (row total × column total) ÷ grand total. Compute the statistic cell by cell, and compare it with a chi-square on (r − 1)(c − 1) degrees of freedom, where r and c count the rows and the columns (Hyderabad 2015 asks it outright: the chi-square test is the one for association between attributes).

Households by head’s schooling and whether the household holds a bank account (invented). Type any counts.

	Has an account	No account	Total
No schooling	expected 21contributes 3.86	expected 29contributes 2.79	50
Primary	expected 21contributes 0.05	expected 29contributes 0.03	50
Secondary or more	expected 21contributes 4.76	expected 29contributes 3.45	50
Total	63	87	150

χ² = Σ (O − E)² ÷ E14.943

Degrees of freedom (r − 1)(c − 1)2

5% critical value5.991

p-value0.0006

χ² = 14.94 exceeds 5.99, so at the 5% level we reject independence: schooling and account-holding are associated. The shaded cells, with the longest bars, are where the table departs most from what independence predicts. Account-holding rises with schooling, far more than chance would explain.

Derive it

Where the expected count and the degrees of freedom of a table come from

We want to show

Under independence the expected count in cell (i, j) is

E_{ij} = \frac{R _{i} C _{j}}{N}

, and a table with r rows and c columns has (r − 1)(c − 1) degrees of freedom.

Independence says the cell probability is a product.
If the row variable and the column variable are unrelated, the chance that a household lands in cell (i, j) is the chance of row i times the chance of column j: P(i, j) = P(i) × P(j). This is the same product rule as independent events in the conditional probability lesson.
Estimate the two chances from the margins.
We do not know P(i) or P(j), but the table’s margins estimate them: P(i) ≈ R_i ÷ N and P(j) ≈ C_j ÷ N, where R_i is the row total, C_j the column total and N the grand total. The expected count is N times the cell probability:
$E_{ij} = N \cdot \frac{R _{i}}{N} \cdot \frac{C _{j}}{N} = \frac{R _{i} C _{j}}{N}$
In a table with row totals 50 and 50, column totals 50 and 50 and N = 100, every cell expects 50 × 50 ÷ 100 = 25. The observed table 30, 20, 20, 30 gives a statistic of 4 × 25 ÷ 25 = 4.
Count the free cells.
The table has rc cells, but the expected counts were built from the data, so the observed counts are tied to them. The N total is fixed, which removes one. The row margins are fixed, but they add to N already, so they remove another r − 1. The column margins remove c − 1. What is free:
$r c - 1 - (r - 1) - (c - 1) = (r - 1) (c - 1)$
Equivalently, we estimated r − 1 row chances and c − 1 column chances, and the previous derivation takes one degree of freedom off for each estimated parameter. For a 3 × 2 table that is 2 × 1 = 2, and for a 2 × 2 table it is 1.
A shortcut for two by two.
For the table [[a, b], [c, d]] the statistic collapses to
$χ^{2} = \frac{N ( a d - b c ) ^{2}}{( a + b ) ( c + d ) ( a + c ) ( b + d )}$
With 30, 20, 20, 30: 100 × (900 − 400)² ÷ (50 × 50 × 50 × 50) = 4, the same 4. The quantity ad − bc is zero exactly when the rows are proportional, which is independence.

Step 1 of 4

Independence or homogeneity?

Two designs give the same arithmetic. In a test of independence one sample is classified two ways (households by schooling and by account-holding). In a test of homogeneity separate samples are drawn from each group (500 men and 500 women), and the question is whether the groups share one distribution. The expected counts, the statistic and the degrees of freedom are identical. For a 2 × 2 table the statistic is the square of the two-proportion z of Lesson 9, which is why the two tests always agree.

Interactive

Significant is not strong

The statistic grows in proportion to the sample size. Hold the shares in a table fixed and multiply every count by ten, and the statistic multiplies by ten. So with enough households any association, however faint, becomes “significant”. To measure how strong an association is, rescale the statistic by the sample size. For a table with r rows and c columns,

Cramér's V

V = \frac{χ ^{2}}{N ( min ( r , c ) - 1 )}

Lies between 0 (independent) and 1 (each category of one variable fixes the other). For 2 × 2 it equals the absolute correlation of the two 0/1 variables.

Households surveyed, N200The shares in the four cells stay fixed at 26%, 24%, 24%, 26%.

	Has an account	No account	Total
Schooled	52	48	100
Not schooled	48	52	100

Chi-square, 1 df0.32

5% critical value3.841

p-value0.5716

Cramér’s V0.04

What it shows

With 200 households the statistic 0.32 is below the cut-off, so independence is not rejected, although the association in the table is exactly the same, V = 0.04. Slide N upward: the chi-square statistic grows in proportion, and V does not move.

In the lab, schooled households hold an account at a rate of 52% against 48% for the unschooled, whatever N is. At N = 200 the statistic is 0.32, nowhere near 3.84, and the test finds nothing. At N = 5,000 the statistic is 8.00 and the p-value is below 0.005, yet V is 0.04 throughout, a very weak association. Report the effect size alongside the p-value.

Pause and predict

The same table of shares is observed in a survey ten times as large. The chi-square statistic and Cramér's V

One more caution on assumptions. The chi-square approximation needs every expected count to be at least about 5. In a 2 × 2 table with small counts, statisticians apply Yates’ continuity correction, or use Fisher’s exact test, which does not rely on the approximation. Entrance papers rarely go beyond naming the rule.

A different use

One more use: a variance

The same distribution tests a single variance. For a normal sample, (n − 1)s² ÷ σ² follows a chi-square with n − 1 degrees of freedom (the result behind the interval of Lesson 6). To test H₀: σ² = σ₀², plug in the claimed value. A machine is meant to produce items with variance 20, and 16 items give s² = 30. The statistic is 15 × 30 ÷ 20 = 22.5, on 15 degrees of freedom. The two-sided 5% acceptance band is 6.26 to 27.49, so a variance of 30 against 20 is not rejected here (p = 0.191).

Do not confuse this with the F test of Lesson 9. A chi-square tests one variance against a number. Two variances against each other make a ratio of chi-squares, which is an F.

Drill

Which chi-square? A drill

Which test, and how many degrees of freedom? · 1 of 6

Births in a hospital are counted for each day of the week. Are births equally likely on every day?

Solving the opening question

The opening question, solved

A die · the puzzle

A die rolled 120 times gives 25, 17, 15, 23, 24 and 16. Is it loaded?

Expected counts.
A fair die gives every face 120 ÷ 6 = 20.
Add up (O − E)²/E.
(25 − 20)²/20 = 1.25, (17 − 20)²/20 = 0.45, (15 − 20)²/20 = 1.25, (23 − 20)²/20 = 0.45, (24 − 20)²/20 = 0.80, (16 − 20)²/20 = 0.80. The sum is 5.
Degrees of freedom and cut-off.
Six faces give 5 degrees of freedom, and the 5% critical value is 11.07. The statistic 5 is well below it, with p = 0.416.

Answer

There is no evidence the die is loaded. The spread from 15 to 25 looks alarming, but it is smaller than a fair die usually produces: the statistic of a fair die averages k − 1 = 5, and this one is exactly 5.0. The lab’s gold bars show how often a fair die does worse.

Revise

The lesson on one screen

Statistic: χ² = Σ (O − E)² ÷ E, summed over cells. Never negative. Reject only for large values.
Goodness of fit: E = n × p. df = k − 1, minus one for each parameter estimated from the data.
Independence: E = row total × column total ÷ grand total. df = (r − 1)(c − 1). Same arithmetic for homogeneity.
Why divide by E: A cell's variance is about E, so each term is about a squared z. One constraint, that the counts add to n, costs one degree of freedom.
2 × 2 shortcut: χ² = N(ad − bc)² ÷ (product of the four margins). Equals the square of the two-proportion z.
Strength: The statistic grows with N. Cramér's V = √(χ² ÷ (N(min(r, c) − 1))) does not.
Conditions: Expected counts of at least about 5: join categories, or use Fisher's exact test.
One variance: (n − 1)s² ÷ σ₀² on n − 1 df. Two variances make an F.

Practice

Check yourself

0 of 14 correct so far

Compute

A die

A die is rolled 60 times and the faces appear 6, 14, 9, 13, 8 and 10 times. Find the goodness-of-fit statistic against a fair die.

χ²

A 2 × 2 table

Of 200 households, the counts are 40 and 60 in the first row and 60 and 40 in the second. Find the chi-square statistic for independence.

χ²

An expected count

In a table with grand total 400, one row totals 80 and one column totals 120. What count is expected in the cell where they meet?

Degrees of freedom

How many degrees of freedom does a chi-square test of independence have for a table with 4 rows and 3 columns?

A 2 × 3 table

Rows: 20, 30, 50 and 30, 30, 40. Find the chi-square statistic for independence. (Row totals 100 and 100, column totals 50, 60, 90.)

χ²

An estimated parameter

A Poisson distribution is fitted to counts grouped into 6 categories after the mean was estimated from the same data. How many degrees of freedom does the goodness-of-fit test have?

A variance

n = 16 and s² = 30. Test H₀: σ² = 20. Find the chi-square statistic.

χ²

Cramér's V

For the table in problem 2 (chi-square 8, N = 200, 2 × 2), find Cramér’s V.

Quick questions

1The chi-square test is used to test

2A test of independence on a 4 × 3 table has how many degrees of freedom?

3Several expected counts in a goodness-of-fit test are below 2. The usual remedy is to

4The same table of shares is observed in a survey four times as large. The chi-square statistic

5A normal distribution is fitted to grouped data after estimating both its mean and its standard deviation, leaving 8 groups. The goodness-of-fit test has how many degrees of freedom?

6A chi-square statistic of 0.02 on 5 degrees of freedom suggests

Real questions

Past papers, by level

Foundation

Definitions and one-step calculations. Most CUET and Hyderabad questions sit here.

1 questionPractise →

Standard

Joint tables, standard errors and statement questions, up to JNU and IIT JAM level.

2 questionsPractise →

Delhi School level

This topic’s hardest questions, from the DSE entrance, are worked through in Lesson 13: Tests on means and proportions.

4 questionsLesson 13 →

All 3 past-paper questions for this lesson, easiest first: practise the full set →

These are questions from the TestStop bank. A free account answers 10 a day, each with its worked solution; All-Access opens every set with no daily limit.

← Lesson 9Comparing two groups: means, proportions and spreads Lesson 11 →Analysis of variance and multiple comparisons

Chi-square tests: goodness of fit and independence

Tests for counts

Goodness of fit

When you estimate a parameter first

The mistakes examiners are fishing for

Independence in a table

Independence or homogeneity?

Significant is not strong

One more use: a variance

Which chi-square? A drill

The opening question, solved

The lesson on one screen

Check yourself

Compute

Quick questions

Past papers, by level

Foundation

Standard

Delhi School level

Read next

Chi-square tests: goodness of fit and independence

Tests for counts

Goodness of fit

When you estimate a parameter first

The mistakes examiners are fishing for

Independence in a table

Independence or homogeneity?

Significant is not strong

One more use: a variance

Which chi-square? A drill

The opening question, solved

The lesson on one screen

Check yourself

Compute

Quick questions

Past papers, by level

Foundation

Standard

Delhi School level

Read next