Probability · Lesson 8 of 9

Sums of random variables, and the inequalities

Almost nothing in economics is a single random variable. A portfolio is a sum, a sample mean is a sum, total demand is a sum. This lesson shows what happens to the average and the spread when random variables are added and weighted, why that makes spreading your money around a good idea, and then proves, in a line each, three bounds that hold for every distribution: Markov, Chebyshev and the law of large numbers. It ends with Jensen’s inequality, the reason a person who dislikes risk turns down a fair gamble.

← Probability moduleLesson 8 of 9 · about 50 minutes

A question to hold on to

You split your savings equally between two assets whose returns are independent. Each has a standard deviation of 10 percentage points a year. What is the standard deviation of the return on your whole portfolio? (A) 10 (B) 7.07 (C) 5 (D) 14.1

Reasoning, inventedSolved at the end ↓

The rules

The mean and variance of a combination

Take two random variables X and Y and form the weighted sum aX + bY. Two rules cover everything, and they behave very differently.

A weighted sum of two random variables

E (a X + bY) = a E (X) + b E (Y), Var (a X + bY) = a^{2} Var (X) + b^{2} Var (Y) + 2 ab Cov (X, Y)

The mean always adds. The variance needs the covariance, which is zero when the variables are independent.

The mean rule holds without any condition on X and Y. The variance rule has a cross term, and for independent variables that term is zero, leaving Var(aX + bY) = a²Var(X) + b²Var(Y). Note what the squares do: the variance of a difference X − Y of independent variables is Var(X) + Var(Y), not the difference of the variances. Subtracting a random quantity never makes the total steadier. For n independent draws with common mean μ and variance σ², the sum has mean nμ and variance nσ², and the mean has mean μ and variance σ² ÷ n.

In a past paper (CUET 2023): if X has mean 10 and variance 100, then Var(5X + 10) = 25 × 100 = 2,500. The shift of 10 has no effect on the spread, and the multiplier is squared. The claim that Var(aX + b) = Var(X) is the mistake the question sets as a trap.

A worked combination

X has mean 4 and variance 9, and Y has mean 1 and variance 4. They are independent. Find the mean and variance of 3X − 2Y. What changes if Cov(X, Y) = 3?

Mean.
3 × 4 − 2 × 1 = 10.
Variance, independent.
3² × 9 + (−2)² × 4 = 81 + 16 = 97.
With Cov = 3.
Add 2 × 3 × (−2) × 3 = −36, so the variance is 97 − 36 = 61. A positive covariance between X and Y reduces the variance of their difference.

Answer

Mean 10, variance 97 for independent variables and 61 when the covariance is 3. The mean did not depend on the covariance, and the variance did.

Interactive

Diversification: a portfolio of two assets

A portfolio that puts weight w in one asset and 1 − w in another is a weighted sum of two returns, so the rule above applies, and the covariance between the two assets decides how much risk the mix removes. Perfectly correlated assets give no relief. Independent ones give a lot. This is the mathematics behind “do not put all your eggs in one basket”.

Derive it

The variance of a portfolio, and the weight that minimises it

We want to show

A portfolio with weight w in asset 1 and 1 − w in asset 2 has

Var = w^{2} σ_{1}^{2} + (1 - w)^{2} σ_{2}^{2} + 2 w (1 - w) ρ σ_{1} σ_{2}

, minimised at

w^{*} = \frac{σ _{2}^{2} - ρ σ _{1} σ _{2}}{σ _{1}^{2} + σ _{2}^{2} - 2 ρ σ _{1} σ _{2}}

The variance of a weighted sum.
The portfolio return is R = wR₁ + (1 − w)R₂. From the rule for combinations, Var(aX + bY) = a²Var X + b²Var Y + 2ab Cov(X, Y), with a = w, b = 1 − w and Cov = ρσ₁σ₂. Expectations always add, but variances add only after the covariance term.
Three cases.
If the returns are uncorrelated (ρ = 0) and both assets have σ = 10, the 50-50 mix has variance 2 × (½)² × 100 = 50, so its standard deviation is 10 ÷ √2 = 7.07: risk falls with no sacrifice of average return.
If they are perfectly correlated (ρ = 1), the variance is (wσ₁ + (1 − w)σ₂)², the standard deviation is the plain weighted average (10 here), and mixing buys nothing. If they are perfectly opposed (ρ = −1), the standard deviation is |wσ₁ − (1 − w)σ₂|, which is zero when w = σ₂ ÷ (σ₁ + σ₂): risk is removed entirely (0 at 50-50 here).
Minimise over w.
Differentiate the variance with respect to w and set the slope to zero:
$2 w σ_{1}^{2} - 2 (1 - w) σ_{2}^{2} + 2 (1 - 2 w) ρ σ_{1} σ_{2} = 0 ⟹ w^{*} = \frac{σ _{2}^{2} - ρ σ _{1} σ _{2}}{σ _{1}^{2} + σ _{2}^{2} - 2 ρ σ _{1} σ _{2}}$
In terms of the covariance c = ρσ₁σ₂ this is w* = (σ₂² − c) ÷ (σ₁² + σ₂² − 2c). The denominator is Var(R₁ − R₂), which is positive, so the second derivative is positive and this is a minimum. For variances 0.07 and 0.05 and covariance 0.04 (IIT JAM 2023): w* = (0.05 − 0.04) ÷ (0.07 + 0.05 − 0.08) = 0.25.
Look at it.
Move the correlation from 1 down to −1 and watch the curve sag away from the dashed line. For σ₁ = 20 and σ₂ = 10 and ρ = 0 the least-risk weight on the riskier asset is 20%, and with ρ = 0.5 it is 0%: when the assets move together the safe one has to carry more of the load.

Step 1 of 4

Watch out for

The mistakes examiners are fishing for

Averaging the standard deviations. The weighted average of σ₁ and σ₂ is the portfolio risk only when ρ = 1. For anything less, the portfolio is safer.
Averaging the variances with the weights. That drops both the squares on the weights and the covariance term.
Forgetting the average return. The mix has the weighted average return, so risk falls without any loss of expected return. That is why diversification is called the one free lunch in finance.

Sums

Sums that stay in the family

For a few distributions, the sum of independent variables from the family is again in the family, so its whole distribution is known, not just its mean and variance. These are the reproductive properties, and they all follow from the product rule for moment generating functions in the MGF derivation.

Independent variables	Their sum
Normal N(μ₁, σ₁²) and N(μ₂, σ₂²)	N(μ₁ + μ₂, σ₁² + σ₂²). Also for any weighted combination.
Poisson with means λ₁ and λ₂	Poisson with mean λ₁ + λ₂
Binomial (n₁, p) and (n₂, p), the same p	Binomial (n₁ + n₂, p). Not closed if the p’s differ.
Gamma with shapes α₁, α₂ and the same rate	Gamma with shape α₁ + α₂. Exponentials add up to a gamma.
Chi-square with k₁ and k₂ degrees of freedom	Chi-square with k₁ + k₂ degrees of freedom

IIT JAM 2024 · easy

X is N(5, 4) and Y is N(3, 2), independent. What is the distribution of 2X + 3Y?

Mean.
2 × 5 + 3 × 3 = 19.
Variance.
2² × 4 + 3² × 2 = 16 + 18 = 34.
Family.
A weighted combination of independent normals is normal.

Answer

N(19, 34).

Pause and predict

X and Y are independent and each uniform on (0, 1). Is X + Y uniform on (0, 2)?

Interactive

Markov, Chebyshev and the law of large numbers

Lesson 5 stated Chebyshev’s inequality. Here it is proved, in the order that makes it easy: a bound from the mean alone, then one from the mean and variance, then a statement about sample means.

Derive it

Markov, Chebyshev and the weak law of large numbers, each from the one before

We want to show

For X ≥ 0,

P (X \geq a) \leq \frac{E ( X )}{a}

. Applying it to (X − μ)² gives

P (∣ X - μ ∣ \geq k σ) \leq \frac{1}{k ^{2}}

, and applying that to the sample mean gives

P (∣ X ˉ - μ ∣ \geq ε) \leq \frac{σ ^{2}}{n ε ^{2}}

Markov: the mean limits the tail.
Split E(X) into the part where X is below a and the part where it is at least a. The first part is at least 0 because X ≥ 0. In the second, every value of X is at least a:
$E (X) \geq 0 + a \cdot P (X \geq a) ⟹ P (X \geq a) \leq \frac{E ( X )}{a}$
If mean household income is ₹30,000, then at most 1/3 of households can earn ₹90,000 or more, because otherwise they alone would push the mean above ₹30,000. No assumption about the shape of the income distribution was needed. The bound is met exactly by a variable that is 0 with probability 0.9 and 10 with probability 0.1 (mean 1, a = 10).
Chebyshev: Markov applied to a squared deviation.
(X − μ)² is never negative and has mean σ². The event |X − μ| ≥ kσ is the same event as (X − μ)² ≥ k²σ². Apply Markov with a = k²σ²:
$P (∣ X - μ ∣ \geq k σ) = P ((X - μ)^{2} \geq k^{2} σ^{2}) \leq \frac{σ ^{2}}{k ^{2} σ ^{2}} = \frac{1}{k ^{2}}$
At least 75% of any distribution lies within two standard deviations of its mean, and at least 8/9 within three, whatever its shape.
The weak law of large numbers.
The sample mean has mean μ and variance σ² ÷ n (the standard error lesson). Apply Chebyshev to X̄ with deviation ε = k·σ/√n:
$P (∣ X ˉ - μ ∣ \geq ε) \leq \frac{σ ^{2}}{n ε ^{2}} ⟶ 0 (n \to \infty)$
With σ = 20, ε = 2 and n = 1,000 the bound is 400 ÷ (1,000 × 4) = 0.1. For any fixed ε, the chance that the sample mean is off by that much falls to zero as n grows: this is what it means for X̄ to be a consistent estimator of μ.
Valid, and loose.
These bounds use only a mean, or a mean and a variance, so they are safe for every distribution and sharp for none. With n = 100 and σ = 20, Chebyshev bounds P(|X̄ − μ| ≥ 4) by 400 ÷ (100 × 16) = 0.25, while the central limit theorem says it is about 2(1 − Φ(2)) = 0.046. Use the bounds when the distribution is unknown, and the normal approximation when it is reasonable. The lab shows the gap for five distributions.

Step 1 of 4

The pattern

Each bound trades information for safety. With only a mean and X ≥ 0, Markov gives a weak but certain limit. Add a variance and Chebyshev tightens it. Add the whole distribution, and you can compute the tail exactly or approximate it with the normal curve. The less you know, the more conservative the guarantee.

Watch out for

The mistakes examiners are fishing for

Using Markov on a variable that can be negative. The proof needs X ≥ 0.
Reading the bound as the probability. Chebyshev says at most 1/k², not equal to it. At k = 2 it permits up to 25% in the tails, and a normal variable has 4.6%.
Using Chebyshev with k ≤ 1. The bound 1/k² is then at least 1, which says nothing.

Interactive

Jensen: the curve against the average

Average first and then apply a function, or apply the function and then average? For a straight line the answers agree. For a curve they differ in a fixed direction, and that direction is the content of Jensen’s inequality. It explains why a risk-averse person refuses a fair gamble, and why the sample standard deviation underestimates σ.

Derive it

Jensen's inequality: the average of a curve against the curve of the average

We want to show

If g is convex,

E g (X) \geq g (E X)

. If g is concave, the inequality reverses.

A convex curve lies above its tangent.
Let μ = E(X) and draw the tangent to g at μ. A convex function never dips below it, so for every x
$g (x) \geq g (μ) + g^{'} (μ) (x - μ)$
Take expectations.
The inequality holds at every value of X, so it holds on average. The expectation of the right side is g(μ) + g′(μ)(E X − μ) = g(μ) + 0:
$E g (X) \geq g (μ) + g^{'} (μ) \cdot E (X - μ) = g (μ)$
3
What it gives you.
- g(x) = x² is convex, so E(X²) ≥ (E X)², which says Var(X) ≥ 0.
- √x is concave, so E(√X) ≤ √(E X). With X = S², this says E(S) ≤ σ: the sample standard deviation is biased downward even though S² is unbiased (Lesson 4).
- 1/x is convex for x > 0, so E(1/X) ≥ 1 ÷ E(X). The average of reciprocals is above the reciprocal of the average.
- A concave utility u gives E u(W) ≤ u(E W): a person with diminishing marginal utility prefers a sure income to a fair gamble with the same average. That is risk aversion (UOH 2012). The gap, in rupees, is the risk premium.
For the gamble of ₹0 or ₹100 thousand with equal chance and u(x) = √x, E u = 0.5 × 0 + 0.5 × 10 = 5, while u(50) = 7.07. A sure ₹25 thousand gives √25 = 5, so the gamble is worth only ₹25 thousand to this person, a premium of ₹25 thousand.

Step 1 of 3

Solving the opening question

The opening question, solved

Reasoning · the puzzle

Equal weights on two independent assets with σ = 10 each. What is the standard deviation of the portfolio?

Variance of the weighted sum.
Var = (½)² × 100 + (½)² × 100 + 2 × ½ × ½ × 0 = 25 + 25 = 50.
Root.
√50 = 7.07.
The traps.
Option A (10) ignores diversification. Option C (5) halves the standard deviation but forgets there are two assets. Option D (14.1) adds the variances without the weights.

Answer

(B) 7.07, that is 10 ÷ √2. Independence removes the covariance term, and the squared weights shrink each variance to a quarter. With a correlation of 1 the answer would have been 10, and with a correlation of −1 it would have been zero.

Revise

The lesson on one screen

Mean of a combination: E(aX + bY) = aE(X) + bE(Y). Always.
Variance of a combination: a²Var X + b²Var Y + 2ab Cov. Independent: no cross term. The variance of X − Y is Var X + Var Y.
Sums of n independent draws: Mean nμ, variance nσ². The mean of the draws has variance σ²/n.
Portfolio: Var = w²σ₁² + (1 − w)²σ₂² + 2w(1 − w)ρσ₁σ₂. Least risk at w* = (σ₂² − c)/(σ₁² + σ₂² − 2c), c = Cov. Diversification helps whenever ρ < 1.
Stays in the family: Normal + normal, Poisson + Poisson, binomial + binomial (same p), gamma + gamma (same rate), chi-square + chi-square. Uniform + uniform does not.
Markov: X ≥ 0: P(X ≥ a) ≤ E(X)/a.
Chebyshev: P(|X − μ| ≥ kσ) ≤ 1/k². The weak law: P(|X̄ − μ| ≥ ε) ≤ σ²/(nε²) → 0.
Jensen: Convex g: E g(X) ≥ g(E X). Concave u: E u(W) ≤ u(E W), which is risk aversion.

Practice

Check yourself

0 of 14 correct so far

Compute

A combination

X has mean 4 and variance 9, Y has mean 1 and variance 4, and they are independent. Find Var(3X − 2Y).

Variance

With a covariance

Var(X) = 4, Var(Y) = 9 and Cov(X, Y) = 3. Find Var(X + Y).

Variance

A portfolio

40% is held in an asset with σ = 20 and 60% in one with σ = 10, and the correlation is 0.5. Find the standard deviation of the portfolio.

Standard deviation

Least risk

Two assets have variances 0.09 and 0.04 and covariance 0.02. Find the weight on the first asset that minimises the variance of the portfolio.

Markov

The mean household income is ₹40,000 and incomes cannot be negative. At most what share of households can earn ₹120,000 or more? Give a fraction as a decimal.

Bound

Chebyshev

A score has mean 100 and standard deviation 15. At least what probability does Chebyshev guarantee for a score between 70 and 130?

Probability

The weak law

σ = 10 and n = 400. By Chebyshev’s inequality applied to the sample mean, what is the upper bound on P(|X̄ − μ| ≥ 1)?

Bound

A certainty equivalent

A gamble pays ₹0 or ₹64 thousand with equal chance. Utility is u(x) = √x. What sure income gives the same utility? (₹ thousand)

Certainty equivalent

Quick questions

1X and Y are independent with variances 4 and 9. The variance of X − Y is

2The sum of two independent Poisson variables with means 2 and 3 is

3Markov's inequality P(X ≥ a) ≤ E(X)/a requires that

4By Chebyshev's inequality, the probability that a random variable lies within three standard deviations of its mean is at least

5An individual's utility is concave in income. This implies that the individual is

6Two assets with equal risk are perfectly negatively correlated. Equal weights give a portfolio with standard deviation

Real questions

Past papers, by level

Foundation

Definitions and one-step problems. Most CUET and UGC NET questions sit here.

1 questionPractise →

Standard

Two or three steps, up to the hardest JAM, JNU and Hyderabad questions.

1 questionPractise →

All 2 past-paper questions for this lesson, easiest first: practise the full set →

These are questions from the TestStop bank. A free account answers 10 a day, each with its worked solution; All-Access opens every set with no daily limit.

← Lesson 7Continuous distributions Lesson 9 →The hard problems

Sums of random variables, and the inequalities

The mean and variance of a combination

Diversification: a portfolio of two assets

The variance of a portfolio, and the weight that minimises it

The mistakes examiners are fishing for

Sums that stay in the family

Markov, Chebyshev and the law of large numbers

Markov, Chebyshev and the weak law of large numbers, each from the one before

The mistakes examiners are fishing for

Jensen: the curve against the average

Jensen's inequality: the average of a curve against the curve of the average

The opening question, solved

The lesson on one screen

Check yourself

Compute

Quick questions

Past papers, by level

Foundation

Standard

Read next

Sums of random variables, and the inequalities

The mean and variance of a combination

Diversification: a portfolio of two assets

The variance of a portfolio, and the weight that minimises it

The mistakes examiners are fishing for

Sums that stay in the family

Markov, Chebyshev and the law of large numbers

Markov, Chebyshev and the weak law of large numbers, each from the one before

The mistakes examiners are fishing for

Jensen: the curve against the average

Jensen's inequality: the average of a curve against the curve of the average

The opening question, solved

The lesson on one screen

Check yourself

Compute

Quick questions

Past papers, by level

Foundation

Standard

Read next