Derivation · Statistics

The exponential and gamma distributions, derived

The Poisson counts how many events arrive. This page times them. The wait for the next event is exponential, the wait for the r-th event is gamma, and both fall out of the Poisson formula you already have. Along the way you meet the most testable fact about the exponential, that it has no memory. 4 past-paper questions at the end.

A question to hold on to

A random variable X follows an exponential distribution with mean 50. The conditional probability P(X > 70 | X > 60) is e^(−7/5), e^(−6/5), e^(−1/5), or e^(−7/6)?

JAM 2024Solved at the end ↓

SymbolsReading the maths on this pageΣ, x̄, E[ ] and friends: how to say them and what they tell you to do.

Tap a symbol.

The idea

From counting events to timing them

Imagine events arriving at random, at an average of λ per hour. Two different questions can be asked of the same process. How many events in the next hour? That is the Poisson of the previous page. How long until the next one? That is the exponential. And how long until the third? That is the gamma.

The two questions are tied together by one observation: you wait longer than t hours exactly when the count over those t hours is zero. Every formula below comes from that sentence, or from adding independent waits.

Pause and predict

Buses arrive at random, 4 an hour on average. You have already waited 10 minutes. Compared with someone who has just arrived, your chance of waiting another 5 minutes is

Derivation

The exponential, derived

Derive it

The exponential distribution, from a Poisson process

We want to show

The waiting time T to the first event of a rate-λ process has

P (T > t) = e^{- λ t}

, density

λ e^{- λ t}

, mean 1 ÷ λ and variance 1 ÷ λ², and it has no memory.

Start from the Poisson count.
Suppose events arrive at random at an average rate of λ per hour (buses at a stop, calls to a helpline). From the previous page, the number of events in t hours is Poisson with mean λt, so P(N_t = k) = e^−λt(λt)^k ÷ k!.
Turn a wait into a count of zero.
Let T be the time until the first event. The wait lasts longer than t hours exactly when no event has happened in the first t hours. So
$P (T > t) = P (N_{t} = 0) = \frac{e ^{- λ t} ( λ t ) ^{0}}{0 !} = e^{- λ t}$
With λ = 0.5 per hour, the chance of waiting more than 2 hours is e⁻¹ = 0.3679.
Read off the distribution function and the density.
The chance of the event arriving by time t is what is left over, and the density is its slope:
$F (t) = 1 - e^{- λ t}, f (t) = F^{'} (t) = λ e^{- λ t} (t \geq 0)$
The density is highest at t = 0 and falls steadily: short waits are the most likely, and very long ones are rare but possible.
Say it in words
“The chance of waiting longer than t is e to the minus lambda t.”
Find the mean and the variance from the survival curve.
For a non-negative variable, the mean is the area under its survival curve, and E[T²] is the area under 2t times it. Both integrals are easy here:
$E [T] = \int_{0}^{\infty} e^{- λ t} d t = \frac{1}{λ}, E [T^{2}] = \int_{0}^{\infty} 2 t e^{- λ t} d t = \frac{2}{λ ^{2}}$
So Var(T) = 2 ÷ λ² − 1 ÷ λ² = 1 ÷ λ². The standard deviation equals the mean. The median m solves e^−λm = ½, so m = ln 2 ÷ λ ≈ 0.69 ÷ λ, which is less than the mean 1 ÷ λ: a few long waits pull the mean up (DSE 2021).
Check that it has no memory.
Suppose you have already waited s hours. The chance of waiting at least t more is a ratio of survival chances:
$P (T > s + t ∣ T > s) = \frac{e ^{- λ (s + t)}}{e ^{- λ s}} = e^{- λ t} = P (T > t)$
The wait so far cancels out. Drag s in the lab and the two curves stay on top of each other. JAM 2024: for a mean of 50, λ = 1/50, and P(X > 70 | X > 60) = P(X > 10) = e^−10/50 = e^−1/5.
Estimate the rate from data.
For n observed waits x₁, …, x_n the likelihood is λⁿe^−λΣx. Its log is n ln λ − λΣx, with derivative n ÷ λ − Σx. Setting it to zero gives the maximum-likelihood estimate
$λ^= \frac{n}{\sum x _{i}} = \frac{1}{x ˉ}$
The sample 1.75, 2.25, 2.50, 2.75, 3.25 has mean 2.5, so λ̂ = 0.4 = 2/5 (JAM 2023).

Step 1 of 6

The exponential distribution

f (t) = λ e^{- λ t}, P (T > t) = e^{- λ t}, E [T] = \frac{1}{λ}, Var (T) = \frac{1}{λ ^{2}}

Rate λ: events per unit of time. The mean wait is 1 ÷ λ.

The memoryless property

P(T > s + t | T > s) = P(T > t). It holds for the exponential, and for no other continuous waiting-time distribution.

Derivation

The gamma, derived

Derive it

The gamma distribution, from waiting for the r-th event

We want to show

The time to the r-th event has density

\frac{λ ^{r} t ^{r - 1} e ^{- λ t}}{( r - 1 )!}

, mean r ÷ λ and variance r ÷ λ². Allowing r to be any positive number gives the gamma family.

A wait for r events is r exponential waits in a row.
After each event, the process starts afresh (the exponential has no memory), so the gaps between events are independent exponentials with the same rate. The time of the r-th event is their sum, T_r = T₁ + … + T_r. Independence lets means and variances add:
$E [T_{r}] = \frac{r}{λ}, Var (T_{r}) = \frac{r}{λ ^{2}}$
Say what it means to be still waiting.
The r-th event has not happened by time t exactly when fewer than r events have happened. So P(T_r > t) is a sum of Poisson probabilities:
$P (T_{r} > t) = k = 0 \sum r - 1 \frac{e ^{- λ t} ( λ t ) ^{k}}{k !}$
For r = 2 and λ = 1 at t = 2 this is e⁻²(1 + 2) = 0.406.
Differentiate to get the density.
The density is minus the slope of the survival function. Each term has slope
$\frac{d}{d t} \frac{e ^{- λ t} ( λ t ) ^{k}}{k !} = λ \frac{e ^{- λ t} ( λ t ) ^{k - 1}}{( k - 1 )!} - λ \frac{e ^{- λ t} ( λ t ) ^{k}}{k !}$
When you add the terms for k = 0, …, r − 1, each positive piece cancels the negative piece of the term before it. Only the last negative piece survives, so
$f (t) = \frac{λ e ^{- λ t} ( λ t ) ^{r - 1}}{( r - 1 )!} = \frac{λ ^{r} t ^{r - 1} e ^{- λ t}}{( r - 1 )!}$
For r = 2, f(t) = λ²te^−λt, which you can confirm by differentiating e^−λt(1 + λt) directly.
Let the shape be any positive number.
The only piece that needs a whole number is (r − 1)!. Replace it with the gamma function, Γ(α) = ∫ x^α−1e^−x dx. Integrating by parts gives Γ(α + 1) = αΓ(α), and Γ(1) = 1, so Γ(n) = (n − 1)! for whole n. The gamma density with shape α and rate λ is
$f (t) = \frac{λ ^{α} t ^{α - 1} e ^{- λ t}}{Γ ( α )}, E [T] = \frac{α}{λ}, Var (T) = \frac{α}{λ ^{2}}$
With α = 1 it is the exponential. For α ≥ 1 it peaks at (α − 1) ÷ λ. The lab shows the family, and, for whole shapes, simulated sums of exponentials landing on the curve.
Recognise it in a past paper.
JNU 2018 gives F(x) = 1 − e^−x − xe^−x = 1 − e^−x(1 + x). That is exactly the survival function above with r = 2 and λ = 1. So the variable is gamma with shape 2 and rate 1: mean 2 ÷ 1 = 2, mode (2 − 1) ÷ 1 = 1. The density is te^−t, and 0.3679 at its peak t = 1.

Step 1 of 5

The gamma distribution

f (t) = \frac{λ ^{α} t ^{α - 1} e ^{- λ t}}{Γ ( α )}, E [T] = \frac{α}{λ}, Var (T) = \frac{α}{λ ^{2}}

Shape α (the number of events waited for, when whole) and rate λ.

Putting it together

How the distributions fit together

One process, three questions
Question	Distribution	Mean	Variance
How many events in time t?	Poisson, mean λt	λt	λt
How long until the first event?	Exponential, rate λ	1 ÷ λ	1 ÷ λ²
How long until the r-th event?	Gamma, shape r, rate λ	r ÷ λ	r ÷ λ²

Two warnings in that table. The Poisson and the exponential look as if they share the symbol λ, but λ is a rate in the exponential (events per hour, so the mean wait is its reciprocal) and a mean count in the Poisson. And the Poisson’s mean equals its variance, while the exponential’s mean equals its standard deviation. Back on the previous page, the Poisson derivation is the source of everything here, and the continuous distributions lesson applies the exponential to the exam questions.

Exam focus

The mistakes examiners are fishing for

Watch out for

The mistakes examiners are fishing for

Taking λ as the mean. For the exponential, λ is a rate and the mean is 1 ÷ λ. Check whether the question gives you the mean or the rate before you substitute.
Using the time already waited. In P(T > s + t | T > s) the s cancels. Do not compute the conditional probability from scratch, and do not shift the interval.
Saying the mean equals the median. The median is (ln 2) ÷ λ, below the mean (DSE 2021).
Adding standard deviations. The variance of a sum of independent waits adds, the standard deviation does not: the gamma has variance r ÷ λ² and standard deviation √r ÷ λ.
Getting the gamma peak wrong. It is at (α − 1) ÷ λ for α ≥ 1, not at the mean α ÷ λ (JNU 2018).
Forgetting the Poisson link. To find the chance the r-th event is still to come, add the Poisson chances of 0, 1, …, r − 1 events.

Back to the puzzle

The opening question, solved

JAM 2024 · moderate

X is exponential with mean 50. Find P(X > 70 | X > 60).

Remember the property
The exponential has no memory, so P(X > 70 | X > 60) = P(X > 10).
Find the rate
The mean is 50, so λ = 1/50.
Apply the survival formula
P(X > 10) = e^(−10/50) = e^(−1/5).

Answer

e^(−1/5), which is option C. Option A, e^(−7/5), is what you get by forgetting the conditioning and using 70 ÷ 50.

JNU 2018 · moderate

F(x) = 1 − e^(−x) − xe^(−x) for x ≥ 0. Find the mean and the mode.

Spot the form
1 − e^(−x)(1 + x) is P(T₂ ≤ x) for the second event of a rate-1 process: a gamma with shape 2 and rate 1.
Read off the mean
α ÷ λ = 2.
Read off the mode
The density is the slope, xe^(−x), which peaks where 1 − x = 0, at x = 1.

Answer

Mean 2 and mode 1.

JAM 2023 · moderate

f(x) = λe^(−λx), x ≥ 0. The sample is 1.75, 2.25, 2.50, 2.75, 3.25. What is the maximum-likelihood estimate of λ?

Use λ̂ = 1 ÷ x̄
The sample total is 12.5, so x̄ = 12.5 ÷ 5 = 2.5.

Answer

λ̂ = 1 ÷ 2.5 = 0.4 = 2/5.

Revision

The page on one screen

The lesson on one screen

The key sentence: Waiting longer than t means no event in the first t. So P(T > t) = P(Poisson(λt) = 0) = e^(−λt).
Exponential: Density λe^(−λt), mean 1 ÷ λ, variance 1 ÷ λ², median (ln 2) ÷ λ.
Memoryless: P(T > s + t | T > s) = P(T > t): the wait so far cancels.
Rate from data: λ̂ = 1 ÷ x̄ (maximum likelihood).
Gamma: The sum of r independent exponentials: mean r ÷ λ, variance r ÷ λ², peak (r − 1) ÷ λ.
Survival of a gamma: P(T_r > t) = Σ e^(−λt)(λt)^k/k! for k from 0 to r − 1.
Any shape: Replace (r − 1)! by Γ(α), with Γ(n) = (n − 1)!. The chi-square is gamma with shape k ÷ 2, rate ½.

Practice

Check yourself

0 of 12 correct so far

Numbers

Give answers to four decimal places where they are not whole numbers.

A survival chance

Calls arrive at a rate of 0.5 a minute, so the wait for the next one is exponential with λ = 0.5. What is the chance the wait exceeds 2 minutes?

P(T > 2)

The mean

An exponential waiting time has rate 0.25 per hour. What is its mean wait, in hours?

Mean

No memory

The wait is exponential with mean 50. Given you have already waited 60, what is the chance of waiting more than 10 further? (four decimals)

Probability

The median

For an exponential with mean 10, what is the median? (two decimals)

Median

A gamma mean

The time to the 3rd event, with events arriving at rate 2 an hour. What is the mean time, in hours?

Mean

A gamma survival chance

The time to the 2nd event at rate 1. What is the chance it exceeds 2? (four decimals)

P(T₂ > 2)

Estimating the rate

Five observed waits have a mean of 2.5. What is the maximum-likelihood estimate of λ for an exponential?

λ̂

Concepts

1For an exponential distribution, the relation between mean and median is

2X is exponential with mean 50. P(X > 70 | X > 60) equals

3The mean of a Poisson with parameter λ and the mean of an exponential with parameter λ are equal

4If F(x) = 1 − e^(−x)(1 + x) for x ≥ 0, the mean and the mode of the distribution are

5The sum of 4 independent exponential waits, each with rate 2, has mean and variance

Past papers

Past papers, by level

Standard

Classification traps, multi-step numericals and statement questions, up to the hardest JNU, JAM and GATE items.

4 questionsPractise →

Delhi School level

The Delhi School entrance asks this topic harder than any other exam. These questions are kept apart from the ordinary sets.

1 questionTry it →

All 4 past-paper questions on the exponential and gamma distributions, easiest first: practise the full set →

These are questions from the TestStop bank. A free account answers 10 a day, each with its worked solution; All-Access opens every set with no daily limit.

Quick answers

What is the exponential distribution?

The exponential distribution describes the waiting time until the first event of a process in which events arrive at random at a constant rate λ. Its density is λe^(−λt) for t ≥ 0, the chance of waiting longer than t is e^(−λt), the mean is 1/λ and the variance is 1/λ².

What does memoryless mean?

A waiting time is memoryless if the time already waited does not change the distribution of the time still to wait: P(T > s + t | T > s) = P(T > t). The exponential has this property because e^(−λ(s+t)) divided by e^(−λs) is e^(−λt). It is the only continuous distribution on the positive axis with the property.

Why is the median of an exponential smaller than its mean?

The median m solves e^(−λm) = 1/2, so m = ln 2 / λ, about 0.69/λ. The mean is 1/λ. The distribution has a long right tail of rare, very long waits, which pull the mean above the median.

How are the exponential and Poisson distributions related?

They describe the same process from two sides. In a Poisson process with rate λ, the number of events in a time t is Poisson with mean λt, and the waiting time to the first event is exponential with rate λ. The Poisson counts, and the exponential times.

What is the gamma distribution?

The gamma distribution with whole shape r and rate λ is the waiting time for the r-th event of a Poisson process, the sum of r independent exponential waits. Its density is λ^r t^(r−1) e^(−λt) / (r − 1)!, its mean is r/λ and its variance is r/λ². With a non-integer shape, the factorial is replaced by the gamma function.

How is the gamma related to the chi-square distribution?

A chi-square distribution with k degrees of freedom is a gamma distribution with shape k/2 and rate 1/2. The chi-square is built from squared normal variables, and the derivations library covers it separately.

The exponential and gamma distributions, derived

From counting events to timing them

The exponential, derived

The gamma, derived

How the distributions fit together

The mistakes examiners are fishing for

The mistakes examiners are fishing for

The opening question, solved

The page on one screen

Check yourself

Numbers

Concepts

Past papers, by level

Standard

Delhi School level

Quick answers

What is the exponential distribution?

What does memoryless mean?

Why is the median of an exponential smaller than its mean?

How are the exponential and Poisson distributions related?

What is the gamma distribution?

How is the gamma related to the chi-square distribution?

Read next

The exponential and gamma distributions, derived

From counting events to timing them

The exponential, derived

The gamma, derived

How the distributions fit together

The mistakes examiners are fishing for

The mistakes examiners are fishing for

The opening question, solved

The page on one screen

Check yourself

Numbers

Concepts

Past papers, by level

Standard

Delhi School level

Quick answers

What is the exponential distribution?

What does memoryless mean?

Why is the median of an exponential smaller than its mean?

How are the exponential and Poisson distributions related?

What is the gamma distribution?

How is the gamma related to the chi-square distribution?

Read next