Derivation · Statistics

The chi-square, t and F distributions, derived

Q: What is the chi-square distribution?

The chi-square distribution with k degrees of freedom is the distribution of the sum of k independent squared standard normal variables. It has mean k and variance 2k, and it is a gamma distribution with shape k/2 and rate 1/2.

Q: Why is (n − 1)S²/σ² chi-square with n − 1 degrees of freedom?

The sum of squares Σ(xᵢ − μ)²/σ² around the true mean is chi-square with n degrees of freedom. It splits into Σ(xᵢ − x̄)²/σ² plus n(x̄ − μ)²/σ², and the second piece is a single squared standard normal, independent of the first. Degrees of freedom add, so the first piece has n − 1.

Q: What is the t distribution?

A t variable with k degrees of freedom is a standard normal Z divided by the square root of an independent chi-square V over its degrees of freedom: T = Z / √(V/k). It arises when a sample mean is standardised with the sample standard deviation instead of the unknown population value.

Q: Why does the t distribution have heavier tails than the normal?

Its denominator is random. When the sample standard deviation happens to come out small, the ratio is large. The variance of t is k/(k − 2), which exceeds 1 and falls to 1 as k grows, so t approaches the standard normal for large k.

Q: What is the F distribution?

An F variable with d₁ and d₂ degrees of freedom is the ratio (V₁/d₁) / (V₂/d₂) of two independent chi-square variables, each divided by its degrees of freedom. The ratio of two sample variances from independent normal samples, scaled by the population variances, has this distribution.

Q: How are t and F related?

The square of a t variable with k degrees of freedom is an F variable with 1 and k degrees of freedom. So the two-sided t test and the F test with one numerator degree of freedom give the same answer.

The three distributions that fill the tables at the back of an inference course are not three new ideas. They are one idea, the standard normal, squared, divided, and divided again. This page builds each from that single ingredient, so the degrees of freedom stop being a number you look up and become a count of the squares that went in.

SymbolsReading the maths on this pageΣ, x̄, E[ ] and friends: how to say them and what they tell you to do.

Tap a symbol.

The idea

One ingredient, three distributions

Start with Z, a standard normal. From it you can make three things, and each is one of the standard tables.

Square and add: Z₁² + … + Z_k² is chi-square with k degrees of freedom.
Divide a normal by a chi-square: Z ÷ √(V ÷ k) is t with k degrees of freedom.
Divide a chi-square by a chi-square: (V₁ ÷ d₁) ÷ (V₂ ÷ d₂) is F with d₁ and d₂ degrees of freedom.

Why would anyone build these? Because a sample gives you only estimates. A sample variance is a sum of squares, so it is chi-square. A sample mean standardised by an estimated spread is a normal over a chi-square, so it is t. And two sample variances compared with each other are a chi-square over a chi-square, so they are F.

Pause and predict

A sample of 10 from a normal population has variance s². Compared with the true variance σ², the quantity 9s² ÷ σ² is chi-square with

Derivation

Chi-square, derived

Derive it

The chi-square distribution, from squared normals

We want to show

The sum of k independent squared standard normals has density

\frac{y ^{k /2 - 1} e ^{- y /2}}{2 ^{k /2} Γ ( k /2 )}

, mean k and variance 2k. It is the gamma distribution with shape k ÷ 2 and rate ½.

Square one standard normal.
Let Z be standard normal and Y = Z². Squaring wipes out the sign, so Y is never negative, and Y is small when Z is near 0. We want the density of Y.
Find its distribution function, then differentiate.
For y > 0, Y ≤ y exactly when −√y ≤ Z ≤ √y, so
$P (Y \leq y) = Φ (y) - Φ (- y) = 2Φ (y) - 1$
The density is the slope. The slope of Φ is φ, and the slope of √y is 1 ÷ (2√y), so
$f (y) = 2 φ (y) \cdot \frac{1}{2 y} = \frac{1}{2 π} y^{- 1/2} e^{- y /2}$
Recognise a gamma.
The gamma density with shape α and rate λ is λ^αy^α−1e^−λy ÷ Γ(α). With α = ½ and λ = ½, and Γ(½) = √π, this is (½)^½ y^−½ e^−y/2 ÷ √π = y^−½ e^−y/2 ÷ √(2π). That is exactly f(y). So a squared normal is a gamma with shape ½ and rate ½.
Add k independent squares.
Gamma variables with the same rate add by adding their shapes, the same additivity that made the gamma a sum of exponentials. Adding k of them gives shape k ÷ 2 and rate ½. This is the chi-square distribution with k degrees of freedom, and the mean and variance follow from the gamma formulas:
$E [χ_{k}^{2}] = \frac{k /2}{1/2} = k, Var (χ_{k}^{2}) = \frac{k /2}{( 1/2 ) ^{2}} = 2 k$
A direct check: E[Z²] = 1, and Var(Z²) = E[Z⁴] − 1 = 3 − 1 = 2. Add k independent copies and you get k and 2k. The lab simulates sums of squared normals and sets them under the curve.
See why it appears in statistics: the sample variance.
For a normal sample, each (x_i − μ) ÷ σ is a standard normal, so Σ(x_i − μ)² ÷ σ² is chi-square with n degrees of freedom. Split the sum around the sample mean:
$\sum (x_{i} - μ)^{2} = \sum (x_{i} - x ˉ)^{2} + n (x ˉ - μ)^{2}$
Divide by σ². The last term is Z², where Z = (x̄ − μ) ÷ (σ ÷ √n) is standard normal, so it is chi-square with 1 degree of freedom, and for normal samples it is independent of the first term. Degrees of freedom add, so the first term has n − 1:
$\frac{( n - 1 ) S ^{2}}{σ ^{2}} \sim χ_{n - 1}^{2}$
That is the “n − 1” of the sample variance, seen as one degree of freedom spent on estimating the mean.
Say it in words
“Chi-square with k degrees of freedom is the sum of k independent squared standard normals.”
Turn it into an interval for the variance.
With n = 16 and s² = 25, the quantity 15 s² ÷ σ² is chi-square with 15 degrees of freedom. Its 2.5% and 97.5% points are 6.262 and 27.488. Solve the inequality for σ²:
$\frac{( n - 1 ) s ^{2}}{χ _{0.975}^{2}} < σ^{2} < \frac{( n - 1 ) s ^{2}}{χ _{0.025}^{2}} \Rightarrow \frac{375}{27.488} < σ^{2} < \frac{375}{6.262}$
That is 13.64 to 59.88. The interval is not symmetric about s² = 25, and the larger chi-square value gives the lower limit, because σ² sits in the denominator.

Step 1 of 6

The chi-square distribution

f (y) = \frac{y ^{k /2 - 1} e ^{- y /2}}{2 ^{k /2} Γ ( k /2 )}, E = k, Var = 2 k

k is the number of independent squared standard normals. It is a gamma with shape k ÷ 2 and rate ½.

Derivation

The t distribution, derived

Derive it

The t distribution, from a normal over a chi-square

We want to show

If Z is standard normal and V is an independent chi-square with k degrees of freedom, then

T = \frac{Z}{V / k}

has the t distribution with k degrees of freedom, with variance k ÷ (k − 2), heavier tails than the normal, and the normal as its limit.

Start with the problem the t solves.
For a normal sample the mean standardises to a standard normal: Z = (x̄ − μ) ÷ (σ ÷ √n). But σ is rarely known. Replace it with the sample standard deviation s and the ratio is no longer exactly normal, because s changes from sample to sample.
Rewrite the ratio as a normal over a chi-square.
Divide the top and bottom of (x̄ − μ) ÷ (s ÷ √n) by σ ÷ √n:
$\frac{x ˉ - μ}{s / n} = \frac{Z}{S ^{2} / σ ^{2}} = \frac{Z}{V / ( n - 1 )}, V = \frac{( n - 1 ) S ^{2}}{σ ^{2}} \sim χ_{n - 1}^{2}$
For a normal sample x̄ and S² are independent, so Z and V are independent. The definition above, with k = n − 1, is exactly this ratio.
Find the spread: why the tails are heavy.
T² = Z² · k ÷ V, and Z and V are independent, so E[T²] = E[Z²] · k · E[1 ÷ V] = k · E[1 ÷ V]. For the chi-square, a gamma with shape k ÷ 2 and rate ½, the gamma integral gives E[1 ÷ V] = (½) ÷ (k ÷ 2 − 1) = 1 ÷ (k − 2), for k > 2. So
$Var (T) = \frac{k}{k - 2} > 1$
The variance is above 1 because the denominator is itself random: when V happens to be small, the ratio is large. With k = 4 the variance is 2; with k = 30 it is 1.071. Test it in the lab.
See the normal as the limit.
V is a sum of k squares with mean 1 each, so V ÷ k settles at 1 as k grows, and √(V ÷ k) goes to 1. Then T goes to Z. The 97.5th percentile of t shows the closing-in: 2.571 for 5 degrees of freedom, 2.228 for 10, 2.042 for 30, and 1.960 for the normal. At the other end, with k = 1 the ratio is a normal over the absolute value of another, a Cauchy distribution, whose tails are so heavy that the mean does not exist.
Use it for an interval.
A sample of 10 has mean 50 and standard deviation 6. With 9 degrees of freedom, t_0.975 = 2.262, so the 95% interval is 50 ± 2.262 × 6 ÷ √10 = 50 ± 4.29. With the normal value 1.96 it would be 50 ± 3.72: too narrow, because it ignores that s is itself an estimate.
Say it in words
“A t variable is a standard normal divided by the square root of an independent chi-square over its degrees of freedom.”

Step 1 of 5

The t variable

T = \frac{Z}{V / k}, Var (T) = \frac{k}{k - 2} (k > 2)

Z standard normal, V chi-square with k degrees of freedom, independent.

Derivation

The F distribution, derived

Derive it

The F distribution, from a ratio of chi-squares

We want to show

If V₁ and V₂ are independent chi-squares with d₁ and d₂ degrees of freedom,

F = \frac{V _{1} / d _{1}}{V _{2} / d _{2}}

has the F distribution with mean d₂ ÷ (d₂ − 2), and 1 ÷ F is F with the degrees of freedom swapped.

Start with comparing two variances.
Two independent normal samples have sample variances S₁² and S₂², with d₁ = n₁ − 1 and d₂ = n₂ − 1. From the chi-square derivation, S₁² ÷ σ₁² = V₁ ÷ d₁ and S₂² ÷ σ₂² = V₂ ÷ d₂, each with V a chi-square. Divide one by the other:
$\frac{S _{1}^{2} / σ _{1}^{2}}{S _{2}^{2} / σ _{2}^{2}} = \frac{V _{1} / d _{1}}{V _{2} / d _{2}} \sim F (d_{1}, d_{2})$
If the two population variances are equal, the σ² cancel and the plain ratio S₁² ÷ S₂² has this distribution. That is the variance-ratio test.
Say what it looks like.
Each of V₁ ÷ d₁ and V₂ ÷ d₂ is an average of squared normals, so it is near 1. Their ratio is usually near 1 too, never negative, and can be large when the denominator is small: a hump close to 1 with a long right tail. The lab simulates the ratio from sums of squared normals.
Find the mean.
V₁ and V₂ are independent, so the mean of the ratio is the product of the means: E[V₁ ÷ d₁] = d₁ ÷ d₁ = 1, and E[d₂ ÷ V₂] = d₂ · E[1 ÷ V₂] = d₂ ÷ (d₂ − 2), using the same E[1 ÷ V] = 1 ÷ (d₂ − 2) as in the t derivation. So
$E [F] = \frac{d _{2}}{d _{2} - 2} (d_{2} > 2)$
The mean depends only on the denominator’s degrees of freedom, and is a little above 1.
Turn it upside down.
1 ÷ F = (V₂ ÷ d₂) ÷ (V₁ ÷ d₁), which is the same construction with the roles swapped, so it is F(d₂, d₁). It follows that P(F < c) = P(1 ÷ F > 1 ÷ c), and the lower cut-off of F(d₁, d₂) is the reciprocal of the upper cut-off of F(d₂, d₁). Tables list only upper points for this reason. For F(10, 8) the 95% point is 3.347.
Connect it to t.
Square a t variable: T² = Z² ÷ (V ÷ k) = (Z² ÷ 1) ÷ (V ÷ k). Here Z² is a chi-square with 1 degree of freedom, so T² is F(1, k). The numbers agree: t₀.₉₇₅ for 10 degrees of freedom is 2.228, whose square 4.965 is the 95% point of F(1, 10) = 4.965. A two-sided t test and a one-sided F test with one numerator degree of freedom are the same test.
Say it in words
“F is a ratio of two independent chi-squares, each divided by its own degrees of freedom.”

Step 1 of 5

The F variable

F = \frac{V _{1} / d _{1}}{V _{2} / d _{2}}, E [F] = \frac{d _{2}}{d _{2} - 2}, T^{2} \sim F (1, k)

V₁ and V₂ independent chi-squares. The lower cut-off of F(d₁, d₂) is 1 ÷ the upper cut-off of F(d₂, d₁).

Putting it together

Which one, and where

Built from the standard normal
Distribution	Built as	Mean	Variance	Where it is used
Chi-square, k	Z₁² + … + Z_k²	k	2k	Interval and test for a variance, goodness of fit, independence in a table
t, k	Z ÷ √(V ÷ k)	0 (k > 1)	k ÷ (k − 2) (k > 2)	Mean with unknown σ, a regression coefficient, a difference of means
F, d₁ and d₂	(V₁ ÷ d₁) ÷ (V₂ ÷ d₂)	d₂ ÷ (d₂ − 2) (d₂ > 2)	depends on both	Ratio of variances, ANOVA, the overall F test of a regression

A quick decision rule. Testing or estimating a variance: chi-square. Testing or estimating a mean with σ unknown: t. Comparing two variances, or comparing the variation between groups with the variation within them: F. The lessons that use them are confidence intervals and tests in practice. The normal derivation gives the ingredient, and the gamma derivation explains why chi-square is a gamma.

Degrees of freedom

For chi-square, degrees of freedom count the independent squares added. For t, they are those of the chi-square in the denominator. For F, there are two sets, one for the top and one for the bottom, and the order matters.

Exam focus

The mistakes examiners are fishing for

Watch out for

The mistakes examiners are fishing for

Using n instead of n − 1. (n − 1)S² ÷ σ² has n − 1 degrees of freedom, because the sample mean was estimated from the same data.
Putting the chi-square points the wrong way round. For a variance interval, the larger chi-square point gives the lower limit: 13.64 to 59.88 for 15 degrees of freedom, not the other way.
Expecting a symmetric interval for a variance. The chi-square curve is lopsided, so the interval is not centred on s².
Using 1.96 when σ is estimated. With 9 degrees of freedom the t value is 2.262, which gives a wider and honest interval.
Swapping the F degrees of freedom. F(10, 8) and F(8, 10) are different distributions: 95% points 3.347 and 3.072.
Forgetting that t² is F. A two-sided t test and an F test with one numerator degree of freedom are the same test.

Practice

Worked examples

A variance interval

A sample of 16 from a normal population has s² = 25. Give a 95% interval for σ².

The pivot
15 s² ÷ σ² is chi-square with 15 degrees of freedom.
The points
The 2.5% and 97.5% points are 6.262 and 27.488.
Solve for σ²
Lower limit 375 ÷ 27.488 = 13.64; upper limit 375 ÷ 6.262 = 59.88.

Answer

σ² lies between 13.64 and 59.88.

A mean with σ unknown

A sample of 10 has mean 50 and standard deviation 6. Give a 95% interval for the population mean.

Choose t
σ is estimated, so use t with 9 degrees of freedom: 2.262.
The margin
2.262 × 6 ÷ √10 = 4.29.

Answer

45.71 to 54.29.

Two variances

Samples of 11 and 9 have variances 40 and 16. Test equal variances against the first being larger, at 5%.

The statistic
F = 40 ÷ 16 = 2.5 with 10 and 8 degrees of freedom.
The cut-off
The 95% point of F(10, 8) is 3.347.
Decide
2.5 < 3.347, so there is not enough evidence that the first variance is larger.

Answer

Do not reject: F = 2.5 is below 3.347.

Revision

The page on one screen

The lesson on one screen

Chi-square: The sum of k independent squared standard normals. Gamma with shape k ÷ 2, rate ½. Mean k, variance 2k.
Why n − 1: Σ(x − μ)² splits into Σ(x − x̄)² and n(x̄ − μ)². The second is one square, so the first has n − 1 degrees of freedom.
Variance interval: (n − 1)s² ÷ χ²_upper to (n − 1)s² ÷ χ²_lower. Larger point, lower limit.
t: Z ÷ √(V ÷ k). Variance k ÷ (k − 2). Heavier tails than the normal, and the normal as k grows.
F: (V₁ ÷ d₁) ÷ (V₂ ÷ d₂). Mean d₂ ÷ (d₂ − 2). 1 ÷ F is F with the degrees of freedom swapped.
t and F: t_k² is F(1, k).

Practice

Check yourself

0 of 13 correct so far

Numbers

Use the values quoted on this page for the percentage points.

Chi-square moments

X is chi-square with 7 degrees of freedom. What is its variance?

Variance

Counting the squares

A chi-square variable has mean 12. How many independent squared standard normals were added?

A variance interval

A sample of 16 has s² = 25, and the chi-square points for 15 degrees of freedom are 6.262 and 27.488. What is the lower limit of the 95% interval for σ²? (two decimals)

Lower limit

A t interval

A sample of 10 has mean 50 and standard deviation 6, and the t value for 9 degrees of freedom is 2.262. What is the margin of error of the 95% interval? (two decimals)

Margin

The variance of t

What is the variance of a t variable with 6 degrees of freedom? (three decimals)

Variance

A variance-ratio test

Samples of 11 and 9 have variances 40 and 16. What is the F statistic S₁² ÷ S₂²?

The lower cut-off

The 95% point of F(8, 10) is 3.072. What is the 5% point of F(10, 8)? (three decimals)

5% point

The mean of F

What is the mean of an F distribution with 5 and 10 degrees of freedom?

Mean

Concepts

1The sum of squares of n independent standard normal variables follows

2A standard normal divided by the square root of an independent chi-square over its degrees of freedom follows

3The ratio of two independent chi-squares, each divided by its degrees of freedom, follows

4As the degrees of freedom increase, the t distribution approaches

5The square of a t variable with k degrees of freedom follows

Past papers

Past papers, by level

Foundation

Definitions, directions and one-step sums. Most CUET, UGC NET and Hyderabad questions sit here.

1 questionPractise →

Standard

Classification traps, multi-step numericals and statement questions, up to the hardest JNU, JAM and GATE items.

1 questionPractise →

Delhi School level

The Delhi School entrance asks this topic harder than any other exam. These questions are kept apart from the ordinary sets.

1 questionTry it →

All 2 past-paper questions on the chi-square, t and F distributions, easiest first: practise the full set →

These are questions from the TestStop bank. A free account answers 10 a day, each with its worked solution; All-Access opens every set with no daily limit.

Quick answers

What is the chi-square distribution?

The chi-square distribution with k degrees of freedom is the distribution of the sum of k independent squared standard normal variables. It has mean k and variance 2k, and it is a gamma distribution with shape k/2 and rate 1/2.

Why is (n − 1)S²/σ² chi-square with n − 1 degrees of freedom?

The sum of squares Σ(xᵢ − μ)²/σ² around the true mean is chi-square with n degrees of freedom. It splits into Σ(xᵢ − x̄)²/σ² plus n(x̄ − μ)²/σ², and the second piece is a single squared standard normal, independent of the first. Degrees of freedom add, so the first piece has n − 1.

What is the t distribution?

A t variable with k degrees of freedom is a standard normal Z divided by the square root of an independent chi-square V over its degrees of freedom: T = Z / √(V/k). It arises when a sample mean is standardised with the sample standard deviation instead of the unknown population value.

Why does the t distribution have heavier tails than the normal?

Its denominator is random. When the sample standard deviation happens to come out small, the ratio is large. The variance of t is k/(k − 2), which exceeds 1 and falls to 1 as k grows, so t approaches the standard normal for large k.

What is the F distribution?

An F variable with d₁ and d₂ degrees of freedom is the ratio (V₁/d₁) / (V₂/d₂) of two independent chi-square variables, each divided by its degrees of freedom. The ratio of two sample variances from independent normal samples, scaled by the population variances, has this distribution.

How are t and F related?

The square of a t variable with k degrees of freedom is an F variable with 1 and k degrees of freedom. So the two-sided t test and the F test with one numerator degree of freedom give the same answer.

The chi-square, t and F distributions, derived

One ingredient, three distributions

Chi-square, derived

The t distribution, derived

The F distribution, derived

Which one, and where

The mistakes examiners are fishing for

The mistakes examiners are fishing for

Worked examples

The page on one screen

Check yourself

Numbers

Concepts

Past papers, by level

Foundation

Standard

Delhi School level

Quick answers

What is the chi-square distribution?

Why is (n − 1)S²/σ² chi-square with n − 1 degrees of freedom?

What is the t distribution?

Why does the t distribution have heavier tails than the normal?

What is the F distribution?

How are t and F related?

Read next

The chi-square, t and F distributions, derived

One ingredient, three distributions

Chi-square, derived

The t distribution, derived

The F distribution, derived

Which one, and where

The mistakes examiners are fishing for

The mistakes examiners are fishing for

Worked examples

The page on one screen

Check yourself

Numbers

Concepts

Past papers, by level

Foundation

Standard

Delhi School level

Quick answers

What is the chi-square distribution?

Why is (n − 1)S²/σ² chi-square with n − 1 degrees of freedom?

What is the t distribution?

Why does the t distribution have heavier tails than the normal?

What is the F distribution?

How are t and F related?

Read next