Basic statistics · Lesson 7 of 10

A first look at regression: the line of best fit

Correlation tells you how closely two variables move together. Regression goes one step further and answers a question economists ask all the time: if x is this much higher, how much higher do we expect y to be? This lesson builds the least-squares line from scratch, shows how it connects to r and to R², and ends with what a fitted line can and cannot tell you. It is a primer; the detailed treatment comes in the econometrics module.

← Basic statistics moduleLesson 7 of 10 · about 50 minutes

A question to hold on to

The two regression lines are 3X + 2Y = 26 and 6X + Y = 31. What is the correlation coefficient r?

CUET 2021Solved at the end ↓

New here?What is Ashanagar?The invented district behind this page's examples: 60 households, with incomes you can inspect.

Ashanagar is an invented district. The statistics lessons use it for every worked example, so you meet the same households again as the ideas build: first as a list of numbers, then as a spread, then as a population to sample from.

It has 60 households, each with a monthly income in rupees. Unless a page says otherwise, those 60 are the whole population, and a “sample” means a few of them drawn at random. Where a page needs two variables, it uses a smaller set of 12 households with the head’s years of schooling beside income.

The figures are illustrative, made up once and fixed. They are not official statistics.

Households: 60
Mean income: ₹28,783 a month
Median income: ₹25,000 a month
Standard deviation: ₹13,366
Lowest and highest: ₹10,500 and ₹72,000

Meet the full data set →

The idea

From moving together to predicting

Lesson 6 left a question open. The twelve Ashanagar households showed a correlation of 0.87 between years of schooling and monthly income. That says the two rise together. It does not say how much extra income one more year of schooling goes with, and that is usually what we want to know. A regression line answers it: it is the straight line that comes closest to the points, written as

$y^= a + b x$

Here x is the variable we start from (the independent or explanatory variable), y is the one we want to explain (the dependent variable), b is the slope and a is the intercept, the predicted y when x is zero. The hat over y means a predicted value, as opposed to an observed one.

This is a different job from correlation. Correlation treats the two variables alike. Regression picks a side: it predicts y from x, and the line you get changes if you swap them, as the fourth section shows.

Interactive

The line of best fit

Any line leaves a gap between what it predicts and what actually happened. That gap is the residual: actual minus estimated, ê = y − ŷ (CUET 2020, CUET 2023). If the fitted line is Ŷ = −15 + 0.75X, then at X = 30 the line predicts −15 + 22.5 = 7.5. A household with Y = 50 there has a residual of 50 − 7.5 = 42.5 (CUET 2018): the line badly underestimates it.

Which line is the best? Residuals above and below the line would cancel if you just added them, so we square them first, exactly as we did for the variance. The least-squares line is the one that makes the sum of the squared residuals as small as possible. It is the only line that does.

Least squares

choose a and b to minimise i = 1 \sum n (y_{i} - a - b x_{i})^{2}

Σe² is called the residual sum of squares, or SSE.

Try it yourself. The two sliders set a line through the twelve households. The red segments are the residuals, and the box on the left adds up their squares. Slide until you think you have the smallest total, then snap to the real least-squares line to see how close you came.

Intercept: income when schooling is 0 (₹ thousand)10Slope: extra income per year of schooling (₹ thousand)1

Your sum of squared residuals373

The smallest any line can reach167.6

R² of your line0.455

Least-squares line2.47 + 1.97x

Your line leaves 205.4 more squared miss than the best line. Tilt it and move it up or down until the red segments are as short as you can make them together. A bad line can even give a negative R², because it fits worse than a flat line at the mean income.

Why squares

Squaring does three jobs. It stops positive and negative misses cancelling, it punishes one big miss more than several small ones, and it gives formulas for the best a and b that can be written down exactly, with no trial and error. It is the same square that built the variance.

The result

The slope and the intercept

Minimising the squared residuals gives two clean formulas:

The least-squares line

b = \frac{Cov ( X , Y )}{σ _{X}^{2}} = r \frac{σ _{Y}}{σ _{X}}, a = y ˉ - b x ˉ

The slope is also written b_yx, the regression coefficient of y on x.

Three things follow, and each appears in the exams:

The slope is the covariance over the variance of x. With a covariance of 100 and standard deviations of 5 and 20, the regression of Y on X has slope 100 ÷ 5² = 4 (CUET 2023). The standard deviation of Y is not used at all.
The line passes through the point of means (x̄, ȳ). That is where the intercept formula comes from. With X̄ = 2, Ȳ = 4.67 and a slope of 1.5, the intercept is 4.67 − 1.5 × 2 = 1.67 (CUET 2023).
The slope has the sign of r. It is positive when the correlation is positive and negative when it is negative, since the covariance sets both.

For the twelve households, the covariance is 21.83 and the variance of schooling is 3.33² = 11.08, so b = 1.97 and a = 23 − 1.97 × 10.42 = 2.47. The line is income ≈ 2.47 + 1.97 × schooling, in ₹ thousand. Read it as: households whose head has one more year of schooling have, on average in this sample, about ₹1,970 a month more.

Derive it

Where the slope and the intercept come from

We want to show

The line ŷ = a + bx that makes the squared misses smallest has

b = \frac{S _{x y}}{S _{xx}}

and

a = y ˉ - b x ˉ

, where

S_{xx} = \sum (x_{i} - x ˉ)^{2}

and

S_{x y} = \sum (x_{i} - x ˉ) (y_{i} - y ˉ)

Say what is being minimised.
Take any line ŷ = a + bx. Household i misses by $y_{i} - a - b x_{i}$ . Misses above and below would cancel, so square them and add:
$S (a, b) = i = 1 \sum n (y_{i} - a - b x_{i})^{2}$
S depends on two numbers, a and b. Drawn against both it is a bowl, and the lowest point is where the bowl stops sloping in either direction. So we look for the place where both slopes are zero.
Slope in the a direction: the line goes through the means.
$\frac{\partial S}{\partial a} ⟹ \sum y_{i} - na - b \sum x_{i} = - 2 \sum (y_{i} - a - b x_{i}) = 0 = 0 ⟹ y ˉ = a + b x ˉ$
Dividing by n says the average miss is zero, which is the same as saying the line passes through the point of means (x̄, ȳ). Whatever slope we end up with, the intercept is a = ȳ − b x̄. For the twelve households, x̄ = 125 ÷ 12 = 10.42 and ȳ = 23.
Put that intercept back and one unknown is left.
Write u_i = x_i − x̄ and v_i = y_i − ȳ. With a = ȳ − b x̄ the miss at household i is v_i − b u_i, so
$S (b) = \sum (v_{i} - b u_{i})^{2} = S_{y y} \sum v_{i}^{2} - 2 b S_{x y} \sum u_{i} v_{i} + b^{2} S_{xx} \sum u_{i}^{2}$
For the twelve households S_xx = 132.92, S_xy = 262 and S_yy = 684, so S(b) = 684 − 524b + 132.92b². Slide b along the bowl and watch the line it stands for.
Find the bottom of the bowl.
$\frac{d S}{d b} = - 2 S_{x y} + 2 b S_{xx} = 0 ⟹ b = \frac{S _{x y}}{S _{xx}}$
It is a minimum, not a maximum, because the second derivative 2S_xx is positive. There is also a way to see it without calculus. Complete the square:
$S (b) = S_{xx} (b - \frac{S _{x y}}{S _{xx}})^{2} + (S_{y y} - \frac{S _{x y}^{2}}{S _{xx}})$
The first term is never negative and is zero at exactly one slope. The second term does not involve b at all, so nothing can beat it.
Read the result.
b = S_xy ÷ S_xx = 262 ÷ 132.92 = 1.971 and a = 23 − 1.971 × 10.42 = 2.47. Divide the top and bottom by n and the slope becomes covariance over variance, 21.83 ÷ 11.08, the form the lesson uses. Since S_xy = r √(S_xxS_yy), it is also $r σ_{Y} / σ_{X}$ .
The smallest possible total miss is the second bracket: 684 − 262² ÷ 132.92 = 167.56, which is S_yy(1 − r²). That small fact is where R² comes from, in the last derivation of this lesson.

Step 1 of 5

IIT JAM 2026 · hard

n = 100, ΣX² = 460,000, ΣY² = 320,000, ΣX = 6,000, ΣY = 500 and ΣXY = 40,000. Find the estimated intercept α.

Means.
X̄ = 6,000 ÷ 100 = 60 and Ȳ = 500 ÷ 100 = 5.
Sums of squares and products about the means.
Sₓₓ = ΣX² − nX̄² = 460,000 − 100 × 3,600 = 100,000. Sₓᵧ = ΣXY − nX̄Ȳ = 40,000 − 100 × 60 × 5 = 10,000.
Slope.
b = Sₓᵧ ÷ Sₓₓ = 10,000 ÷ 100,000 = 0.1.
Intercept.
a = Ȳ − bX̄ = 5 − 0.1 × 60 = −1.

Answer

α = −1. The sum ΣY² is not needed for the line. It would be needed for R². Questions often give more than you need.

Pause and predict

A fitted line is ŷ = 60 + 5x. A student says 'multiply the slope 5 by x and you get the predicted y'. Is that right?

Rescaling

What happens when the units change

Regression slopes carry units, so they respond to changes of units in a pattern worth learning once. Suppose the fitted line is y = 10 + 3x.

Add a constant to x or y. The slope does not change, because the line is only shifted. If the slope of Y on X is 15, the slope of Y + 10 on X − 5 is still 15 (CUET 2023). Only the intercept moves.
Multiply y by a number. Everything about y scales, so both the intercept and the slope are multiplied by it. Multiply y by 0.9 and y = 10 + 3x becomes y = 9 + 2.7x (DSE 2018).
Multiply x by a number c. The slope is divided by c, since a unit of the new x is c old units. The intercept does not change, because x = 0 is still x = 0.
R² and r do not change when either variable is shifted or rescaled by a positive number, for the same reason as in Lesson 6.

Models that are linear in the parameters are linear regressions even when they are not linear in the variables: y = a + b ln x, or y = a + bx + cx² (CUET 2023, CUET 2025). A model such as y = a·x^b is not, until you take logarithms to turn it into ln y = ln a + b ln x.

Logs and growth rates

A regression of ln Y on time t, ln Y_t = a + bt, says Y grows by a roughly constant percentage each year. The slope is the growth rate in logs, so the compound annual rate is e^b − 1. With b = 0.1512 that is e^0.1512 − 1 = 16.32% (JAM 2026). The slope read as a percentage, 15.12%, is only an approximation, and it gets worse as b grows. This is the same continuous-versus-compound point as in Lesson 9.

Interactive

Two regression lines

You can regress y on x, minimising vertical misses, or x on y, minimising horizontal misses. These are different lines. Their slopes are

The two regression coefficients

b_{y x} = r \frac{σ _{Y}}{σ _{X}}, b_{x y} = r \frac{σ _{X}}{σ _{Y}}, b_{y x} b_{x y} = r^{2}

Multiply them and the standard deviations cancel, leaving r². So r = ±√(b_yx b_xy), with the sign taken from the slopes, and b_yx and b_xy always have the same sign. If b_yx = 1.6 and b_xy = 0.4, then r = √0.64 = 0.8, positive because both are (CUET 2021). Use the sliders to see how the two lines behave as r changes.

Correlation r0.6Spread of y compared with spread of x× 1

Slope of y on x0.6

Slope of x on y0.6

Product of the two slopes0.36 = r²

Angle between the lines (as drawn)28°

y on x: minimises vertical missesx on y: minimises horizontal misses

The weaker the correlation, the further apart the lines swing. Both slopes carry the sign of r, and both lines pass through the point of means. Change the spread of y and both slopes change, since they depend on units, while the picture and the product of the slopes do not.

The facts the exams ask, all visible in the picture:

Both lines pass through the point of means (x̄, ȳ), so that is where they cross (Hyderabad 2011, JNU 2021).
If one coefficient is greater than 1, the other is less than 1. Their product is r², which cannot exceed 1 (CUET 2022, JNU 2014, JNU 2021).
The lines are at right angles exactly when r = 0 (CUET 2022). Then both slopes are zero, so b_yx + b_xy = 0 (JNU 2020), and the lines are horizontal and vertical.
The lines coincide exactly when r = ±1, so the correlation is perfect (Hyderabad 2019).

Why two lines? Because the predictions differ. Take very tall parents: their children tend to be tall, but on average not as tall as the parents. Predicting child from parent pulls the prediction back toward the average. That pull, called regression to the mean, is what the weaker slope shows. Francis Galton noticed it in heights in the 1880s, and the word “regression” comes from it.

A question that hands you the two lines and asks for r needs one decision first: which line is y on x, and which is x on y? Try an assignment and multiply the two slopes. If the product exceeds 1 the assignment was wrong, because r² cannot. The opening question works through exactly this, at the end of the lesson.

Goodness of fit

How much the line explains: R²

A line can be the best possible and still fit badly. The coefficient of determination, R², measures how well it fits. The total variation in y is SST, the sum of squared deviations of y from its mean. The line explains part of it, SSR, and leaves the rest as residual, SSE:

R-squared

R^{2} = \frac{explained variation}{total variation} = 1 - \frac{SSE}{SST} = r^{2}

With an intercept, R² in simple regression equals r², the square of the correlation.

If the correlation is 0.6, then 36% of the variation in Y is explained by X (CUET 2023). If r = 0.8, then 64% (UGC NET 2024). The 8% and 80% options forget to square.
The coefficient of non-determination is the unexplained share, 1 − R² (CUET 2022).
R² is a number from 0 to 1. It is 0 when the slope is zero, so the line is flat at ȳ and explains nothing (DSE 2013). It is 1 when every point is on the line. Coefficient of determination is therefore both the explained-over-total share and a measure of how well the sample regression fits the data (CUET 2025).

A small worked case: the line Y = −2.5 + 2X is fitted to Y = 2, 5, 9, 14 and X = 2, 4, 6, 8. It predicts 1.5, 5.5, 9.5, 13.5, so the residuals are 0.5, −0.5, −0.5, 0.5 and SSE = 1. The mean of Y is 7.5, so SST = 30.25 + 6.25 + 2.25 + 42.25 = 81. Then R² = 1 − 1 ÷ 81 = 0.988 (JAM 2024).

For the schooling data R² = 0.87² = 0.755: about three-quarters of the variation in income among these households is accounted for by schooling in a straight-line fit. It does not say the other quarter is noise; it says schooling alone does not account for it.

Swapping the variables

If you regress x on y instead, the slope changes (it is b_xy, not 1 ÷ b_yx) and so does the sum of squared errors, since it now measures horizontal misses in different units. But r and R² do not change, and the slope keeps its sign (UGC NET 2024).

Derive it

Why R² is the share of variation the line explains, and why it equals r²

We want to show

For the least-squares line, SST = SSR + SSE, and the explained share SSR ÷ SST equals r².

Split each household's distance from the mean.
The line predicts ŷ_i. The distance of the actual income from the mean income splits into a part the line accounts for and a part it misses:
$y_{i} - y ˉ = (y^_{i} - y ˉ) + (y_{i} - y^_{i})$
Explained plus left over, drawn as the blue and red strokes in the picture below.
Square and add.
$SST \sum (y_{i} - y ˉ)^{2} = SSR \sum (y^_{i} - y ˉ)^{2} + SSE \sum (y_{i} - y^_{i})^{2} + 2 \sum (y^_{i} - y ˉ) (y_{i} - y^_{i})$
The last term has to go if the pieces are to add up. Does it?
For the least-squares line the cross term is zero.
With a = ȳ − b x̄ we have ŷ_i − ȳ = b u_i, and the miss is e_i = v_i − b u_i (u and v as in the slope derivation). So
$\sum (y^_{i} - y ˉ) e_{i} = b \sum u_{i} (v_{i} - b u_{i}) = b (S_{x y} - b S_{xx}) = 0$
because b = S_xy ÷ S_xx. Try it in the picture. At the least-squares slope SSR + SSE equals SST exactly. Tilt the line and the pieces stop adding up, by the amount 2b(bS_xx − S_xy).
The explained share is r².
$SSR = \sum b^{2} u_{i}^{2} = b^{2} S_{xx} = \frac{S _{x y}^{2}}{S _{xx}} ⟹ \frac{SSR}{SST} = \frac{S _{x y}^{2}}{S _{xx} S _{y y}} = r^{2}$
It also follows that SSE = SST − SSR = SST(1 − r²), the smallest total miss found in the slope derivation.
Check with the twelve households.
SST = 684, SSR = 262² ÷ 132.92 = 516.4 and SSE = 167.6, and 516.4 + 167.6 = 684. The explained share is 516.4 ÷ 684 = 0.755, which is 0.869². About three quarters of the variation in income across these households goes with differences in schooling.

Step 1 of 5

Reading it carefully

What a fitted line does not show

A regression line is a summary of how y moves with x in the data. It does not, on its own, say that x causes y. Regression analysis does not necessarily imply causation (Hyderabad 2015), and the line represents a quantitative relationship between variables, not causality and not an economic identity (Hyderabad 2011). It is the tool used when we have a reason to treat one variable as depending on another, which is why CUET 2019 names regression as the method for analysing a dependence between two variables. Whether the dependence is causal is a question about the data and the design of the study, which the line cannot answer.

Some cautions that apply every time you read a fitted line:

Do not predict far outside the data. The twelve Ashanagar households have 5 to 16 years of schooling. The line says nothing reliable about 25 years, or about 0.
One point can tilt the line. Drag a household far from the rest in the first lab and watch the fitted line follow, as r did in Lesson 6.
A line can be fitted to anything. Check the scatter plot first. The four sets in Anscombe’s quartet all have the same line.
Is the slope really different from zero? With a sample, even a useless variable produces some slope. Testing whether the slope is real uses its standard error and a p-value. A low p-value is evidence that the slope is not zero in the population. JNU 2013 asks which statement about the slope is incorrect, and it is the one that says a slope may be treated as zero when its p-value is below 0.01, which is the wrong way round. That belongs to the next module; the residual sum of squares has n − 2 degrees of freedom in a simple regression, so 98 with 100 observations (CUET 2023).

Watch out for

The mistakes examiners are fishing for

Regression is not symmetric. The line of y on x is not the line of x on y, and you cannot invert one slope to get the other.
r² needs squaring. A correlation of 0.8 explains 64% of the variation, not 80%.
Slope times x is not ŷ. Add the intercept.
Rescaling. Adding to x or y leaves the slope alone. Multiplying y multiplies slope and intercept. Multiplying x divides the slope.
High R² is not causation, and a low R² does not mean the slope is unimportant.

Solving the opening question

The opening question, solved

CUET 2021 · hard

The regression lines are 3X + 2Y = 26 and 6X + Y = 31. Find r.

Slopes of the two lines.
Line 1 as Y on X: slope −1.5. Line 2 as X on Y: slope −1/6.
r².
(−1.5)(−1/6) = 0.25.
Sign.
Both slopes are negative, so r is negative: r = −0.5.

Answer

−0.5, which is option B. The other options fail the check: ±0.4, 0.6 and 0.7 do not square to 0.25. The two lines also give you the means, since they cross there: substituting Y = 31 − 6X into the first line gives 3X + 62 − 12X = 26, so X̄ = 4 and Ȳ = 7. Check: 3 × 4 + 2 × 7 = 26 and 6 × 4 + 7 = 31.

Revise

The lesson on one screen

The line: ŷ = a + bx, with a and b chosen to minimise Σ(y − a − bx)². Residual = actual − estimated.
Slope and intercept: b = Cov(X, Y) ÷ σₓ² = r σᵧ ÷ σₓ. a = ȳ − b x̄. The line passes through (x̄, ȳ).
Two lines: b_yx = r σᵧ/σₓ, b_xy = r σₓ/σᵧ. b_yx × b_xy = r². Same sign. One above 1 means the other below 1. Cross at the means. r = 0: perpendicular. r = ±1: coincide.
R²: Explained ÷ total = 1 − SSE ÷ SST = r². 1 − R² is the coefficient of non-determination. Same for y on x and x on y.
Units: Shift x or y: slope unchanged. Multiply y by c: a and b × c. Multiply x by c: b ÷ c. R² unchanged.
Logs: ln Y on t: growth rate e^b − 1. Linear regression means linear in parameters.
Limits: Not causation. Not outside the data. One outlier can tilt it. Plot first.

Practice

Check yourself

0 of 14 correct so far

Compute

The intercept

x̄ = 6, ȳ = 20 and the slope of y on x is 1.5. Find the intercept.

Slope from r

r = 0.6, σ_Y = 10 and σ_X = 5. Find the slope of the regression of Y on X.

r from the two slopes

b_yx = 0.9 and b_xy = 0.4. Find r.

Negative slopes

b_yx = −1.2 and b_xy = −0.3. Find r.

R² from r

The correlation between x and y is −0.7. What proportion of the variation in y does the regression of y on x explain?

R²

A residual

The fitted line is Ŷ = 5 + 2X. A household has X = 4 and Y = 10. Find its residual.

Residual

From the sums

n = 10, ΣX = 50, ΣY = 80, ΣX² = 290 and ΣXY = 480. Find the slope of the regression of Y on X.

Two lines, one point

The regression lines are 4X − 5Y + 33 = 0 and 20X − 9Y = 107. Find the mean of X.

Mean of X

Log growth rate

A regression of ln Y on the year t has slope 0.07. Find the compound annual growth rate, in percent, to two decimal places.

Growth (%)

Quick questions

1A constant is added to every value of x. The slope of the regression of y on x

2You swap the dependent and independent variables and refit. Which of these is unchanged?

3The two regression lines coincide. Then

4One regression coefficient is 1.4. The other must be

5A regression of y on x finds a strong fit. This shows that

Real questions

Past papers, by level

Foundation

Definitions, one-step arithmetic and which-measure questions. Most CUET and Hyderabad questions sit here.

15 questionsPractise →

Standard

Grouped data, combined means, effects of shifting and scaling, and statement questions, up to JNU level.

20 questionsPractise →

Delhi School level

This topic’s hardest questions, from the DSE entrance, are worked through in Lesson 10: Regression.

17 questionsLesson 10 →

All 35 past-paper questions for this lesson, easiest first: practise the full set →

These are questions from the TestStop bank. A free account answers 10 a day, each with its worked solution; All-Access opens every set with no daily limit.

← Lesson 6Correlation: covariance, r and causation Lesson 8 →Index numbers: Laspeyres, Paasche and Fisher

A first look at regression: the line of best fit

From moving together to predicting

The line of best fit

The slope and the intercept

What happens when the units change

Logs and growth rates

Two regression lines

How much the line explains: R²

Swapping the variables

What a fitted line does not show

The mistakes examiners are fishing for

The opening question, solved

The lesson on one screen

Check yourself

Compute

Quick questions

Past papers, by level

Foundation

Standard

Delhi School level

Read next

A first look at regression: the line of best fit

From moving together to predicting

The line of best fit

The slope and the intercept

What happens when the units change

Logs and growth rates

Two regression lines

How much the line explains: R²

Swapping the variables

What a fitted line does not show

The mistakes examiners are fishing for

The opening question, solved

The lesson on one screen

Check yourself

Compute

Quick questions

Past papers, by level

Foundation

Standard

Delhi School level

Read next