High School
SAT
ACT
University
MCAT
LSAT
DAT
Wize University Statistics Textbook > Simple Linear Regression

Solving for the Regression Line

Coefficient of DeterminationPrevious Section
Predictions and Residual PlotsNext Section
Popular Courses
Find My Course
0:00 / 0:00

Solving for the Regression Line (r, Sy, Sx Method)



Given a bunch of data coordinates (X,Y), we can generate a scatterplot. Then, we determine the best-fit line to represent the relationship between two quantitative variables: the explanatory variable XX and the response variable YY.

y^=bo+b1x\boxed{\hat{y}=b_o+b_1x}

The slope b1\colorFour{b_1}, tells us how much yy changes for every one unit increase in xx.

PAGE BREAK

Let’s prove this using calculus:

dy^dx=b1\displaystyle\boxed{\frac{d\hat{y}}{dx}=b_1}

“A unit increase in xx will change yy by b1b_1.”


Wize Concept
The slope b1b_1 and correlation coefficient rr always have the same sign!


PAGE BREAK
The intercept bo\colorFour{b_o}, tells us the value of yy when x=0x=0. It is the point where the line crosses the y-axis.



0:00 / 0:00

Example: Solving for the Regression Line (r, Sy, Sx Method)


We want to see if there is a relationship between the number of hours a student studies the day before the exam and the exam grade. We randomly sample 34 students:



The explanatory variable (X) is:
Study (hours)

The response variable (Y) is:
Grade

PAGE BREAK

Scatterplot


We see a positive correlation. In fact, r=0.54r=0.54.

This means that there is a weak, positive correlation between hours studied and exam grade.

What does r2 tell us?

r2=(0.54)2=0.2916r^2=\left(0.54\right)^2=0.2916
About 29% of exam grade is explained by how many hours you study.


Portions of information contained in this publication/book are printed with permission of Minitab, LLC. All such material remains the exclusive property and copyright of Minitab, LLC. All rights reserved.

PAGE BREAK

Suppose you are given r, sx, sy, x‾, y‾r,\ s_x,\ s_y,\ \overline{x},\ \overline{y}:

x‾=7.794, y‾=77.824, sx=3.906, sy=18.235, r=0.54\overline{x}=7.794,\ \overline{y}=77.824,\ s_x=3.906,\ s_y=18.235,\ r=0.54

Step 1: Find the slope
b1=r(sysx)\displaystyle\boxed{b_1=r\left(\frac{s_y}{s_x}\right)}

b1=0.54(18.2353.906) =2.52\displaystyle{b_1=0.54\left(\frac{18.235}{3.906}\right)\ =2.52}



Step 2: Find the intercept
bo=y‾−b1x‾\displaystyle\boxed{b_o=\overline{y}-b_1\overline{x}}

bo=77.824−(2.52)(7.794)=58.18b_o=77.824-\left(2.52\right)\left(7.794\right)=58.18



Step 3: Show the full linear equation

y^=bo+b1x\displaystyle\boxed{\hat{y}=b_o+b_1x}
y^=58.18+2.52x\hat{y}=58.18+2.52x



Using the data set below, determine the correlation, slope, and intercept of the least squares regression line.



xˉ\bar{x}= 60 yˉ\bar{y}= 4.8

sxs_{x}= 38.08 sys_{y}= 2.59

r=???r=???

0:00 / 0:00

Solving for the Regression Line (Least Squares Method)


Given a bunch of data coordinates (X,Y), we can generate a scatterplot. Then, we determine the best-fit line to represent the relationship between two quantitative variables: the explanatory variable XX and the response variable YY.

y^=bo+b1x\boxed{\hat{y}=b_o+b_1x}

The slope b1\colorFour{b_1}, tells us how much yy changes for every one unit increase in xx.

Let’s prove this using calculus:

dy^dx=b1\displaystyle\boxed{\frac{d\hat{y}}{dx}=b_1}

“A unit increase in xx will change yy by b1b_1.”

Wize Concept
The slope b1b_1 and correlation coefficient rr always have the same sign!

PAGE BREAK
The intercept bo\colorFour{b_o}, tells us the value of yy when x=0x=0. It is the point where the line crosses the y-axis.




0:00 / 0:00

Example: Solving for the Regression Line (Sxy, Sxx Method)


We want to see if there is a relationship between the number of hours a student studies the day before the exam (XX) and the exam grade (YY). We randomly sample 34 students:

Note: raw data has been truncated


n=34n=34

Important: (∑xi)(∑yi)≠∑xiyi\left(\sum_{ }^{ }x_i\right)\left(\sum_{ }^{ }y_i\right)\neq\sum x_iy_i


PAGE BREAK
Scatterplot


Portions of information contained in this publication/book are printed with permission of Minitab, LLC. All such material remains the exclusive property and copyright of Minitab, LLC. All rights reserved.


PAGE BREAK



Step 1: Find the slope

b1=SSxySSxx\displaystyle\boxed{b_1=\frac{SS_{xy}}{SS_{xx}}}

SSxy=∑xiyi−(∑xi)(∑yi)n\displaystyle{SS_{xy}=\sum_{ }^{ }x_iy_i-\frac{\left(\sum_{ }^{ }x_i\right)\left(\sum_{ }^{ }y_i\right)}{n}}

SSxy=21,892−(265)(2,646)34=1,268.76\displaystyle{SS_{xy}=21,892-\frac{\left(265\right)\left(2,646\right)}{34}=1,268.76}


SSxx=∑xi2−(∑xi)2n\displaystyle{SS_{xx}=\sum_{ }^{ }x_i^2-\frac{\left(\sum_{ }^{ }x_i\right)^2}{n}}


SSxx=2,569−(265)234=503.56\displaystyle{SS_{xx}=2,569-\frac{\left(265\right)^2}{34}=503.56}

Therefore:

b1=SSxySSxx=1,268.76503.56=2.52\displaystyle{b_1=\frac{SS_{xy}}{SS_{xx}}=\frac{1,268.76}{503.56}=2.52}


PAGE BREAK

Step 2: Find the intercept
bo=y‾−b1x‾\displaystyle\boxed{b_o=\overline{y}-b_1\overline{x}}

y‾=∑yin\displaystyle{\overline{y}=\frac{\sum_{ }^{ }y_i}{n}}

y‾=2,64634=77.824\displaystyle{\overline{y}=\frac{2,646}{34}=77.824}

x‾=∑xin\displaystyle{\overline{x}=\frac{\sum_{ }^{ }x_i}{n}}

x‾=26534=7.794\overline{x}=\frac{265}{34}=7.794

Therefore:

bo=77.824−(2.52)(7.794)=58.18b_o=77.824-\left(2.52\right)\left(7.794\right)=58.18


Step 3: Show the full linear equation

y^=bo+b1x\boxed{\hat{y}=b_o+b_1x}
y^=58.18+2.52x\hat{y}=58.18+2.52x
Using the information provided below, solve for the slope and intercept of the linear regression equation.

∑x=702∑y=461∑xy=19,871∑x2=31396∑y2=13,449\begin{array}{ll}\sum x=702\\\sum y=461\\\sum xy=19,871\\\sum x^2=31396\\\sum y^2=13,449\end{array}
n=18n=18
Click on 'HINT' if you are stuck!
Extra Practice
Linear Regression
xˉ\bar{x}= 60 yˉ\bar{y}= 4.8
sxs_{x}= 38.08 sys_{y}= 2.59 r=0.71r=0.71
Regression line and Residual plots
Using the data set below, determine the correlation, slope, and intercept of the least squares regression line.
xˉ\bar{x}= 60 yˉ\bar{y}= 4.8
sxs_{x}= 38.08 sys_{y}= 2.59
Practice (Version 1)
Using the data set below, determine the correlation and intercept of the least squares regression line.
∑x=300\sum{x}=300 ∑y=24\sum{y}=24
sx=38.08s_{x}=38.08 sy=2.59s_{y}=2.59
Correlation and Regression Line
If r=−1.00r=-1.00, then 100% of the data points fall exactly on the regression line and the slope cannot be equal or greater than 0.
Regression Line
The following data is obtained to assess the relationship between height in cm and shoe size.
The correlation coefficient is 0.93
Which of the following is the equation of the regression line?
Regression Line
One day George wanted to examine the linear relationship between how many hours a student studies (xx) the day before the exam and the exam grade (yy).
He asks 13 random students and collects the following data
r=0.819  sx=3.24  sy=14.24  x‾=8.25  y‾=78.25r=0.819\ \ s_x=3.24\ \ s_y=14.24\ \ \overline{x}=8.25\ \ \overline{y}=78.25
Regression Line
In investigating the relationship between number of people on a person's phone list (xx) and the number of text messages per day (yy), 40 individuals were surveyed and the following information was recorded:
∑i=140xi=700sx=3.5\sum_{i=1}^{40}x_i=700 \quad s_x=3.5
∑i=140yi=3200sy=18.9r=0.67\sum_{i=1}^{40}y_i=3200 \quad s_y=18.9 \quad r=0.67
Regression Line
The equation y^=3+0.025x\hat y=3+0.025x is used to predict the amount of weight lost yy (in kg) by a group of individuals in a study based on the amount of their daily exercise xx (in minutes). Given that 1kg≈2.2lb1kg\approx2.2lb, suppose we want to change the amount of weight lost to pounds (lb) in this equation, which of the following will be the correct linear regression line?
Slope of a Regression Line
Suppose we want to see if there is a relationship between the number of hours a student studies the day before their exam (x) and their exam grade (y). We randomly sample 8 students and record our results, the correlation coefficient is 0.819.
If a student studies an additional 2 hours the day before their final exam, by how much can they expect their mark to increase?
Linear Regression
Here's the data and summary of the weekly earnings and amount of coffee purchased by 5 different students:
xˉ\bar{x}= 60 yˉ\bar{y}= 4.8
sxs_{x}= 38.08 sys_{y}= 2.59 r=0.71r=0.71
Regression Line: Changing Units of $x$
The equation y^=35+1.50x\hat y=35+1.50x is used to predict the hourly wage y of someone with x years of experience.
Suppose that x is changed to months of experience. Which of the following will be the correct regression line?
Regression line
Suppose we want to see if there is a relationship between the size of a condo unit and the selling price of it. We randomly sampled 32 units:
r=0.762sx=278.87sy=60350.59xˉ=1034.84yˉ=445734.38\begin{array}{c}r=0.762\\s_x=278.87\\s_y=60350.59\\\bar{x}=1034.84\\\bar{y}=445734.38\end{array}
With this linear regression plot
Regression Line
Suppose we want to see if there is a relationship between the size of a condo unit and the selling price of it. We randomly sampled 32 units:
r=0.762sx=278.87sy=60350.59xˉ=1034.84yˉ=445734.38\begin{array}{c}r=0.762\\s_x=278.87\\s_y=60350.59\\\bar{x}=1034.84\\\bar{y}=445734.38\end{array}
With this linear regression plot
Regression Line
Suppose we want to see if there is a relationship between the size of a condo unit and the selling price of it. We randomly sampled 32 units:
r=0.762sx=278.87sy=60350.59xˉ=1034.84yˉ=445734.38\begin{array}{c}r=0.762\\s_x=278.87\\s_y=60350.59\\\bar{x}=1034.84\\\bar{y}=445734.38\end{array}
With this linear regression plot
Regression Line: Calculation
One day George wanted to examine the linear relationship between how many hours a student studies (xx) the day before the exam and the exam grade (yy).
He asks 13 random students and collects the following data
r=0.819 sx=3.24 sy=14.24 x‾=8.25  y‾=78.25r=0.819\ \quad s_x=3.24\ \quad s_y=14.24\ \quad\overline{x}=8.25\ \ \overline{y}=78.25
Regression Line: Challenging Use of Slope
Suppose we want to see if there is a relationship between the number of hours a student studies the day before their exam (x) and their exam grade (y). We randomly sample 8 students and record our results. The correlation coefficient is 0.819.
If a student studies an additional 2 hours the day before their final exam, by how much can they expect their mark to increase?
Regression Line: Changing Units of X
The equation y^\hat{y}=35+1.50xx is used to predict the hourly wage y of someone with x years of experience. Suppose that x is changed to months of experience. Which of the following will be the correct regression line?
Regression Line: Changing Units of Y Example
Problem
EThe equation weightLost^\widehat{weight Lost}=3+0.025 (exercise^\widehat{exercise}) is used to predict the amount of weight lost, in kilograms, by a group of individuals in a study based on the average amount of daily exercise, in minutes. Each kilogram is approximately 2.2 pounds. Suppose that we change weight lost to pounds. Which of the following describes the corresponding linear regression line?
Linear Regression
xˉ\bar{x}= 60 yˉ\bar{y}= 4.8
sxs_{x}= 38.08 sys_{y}= 2.59 r=0.71r=0.71
Regression Line
Refer to Question 19.
What is the equation of the regression line?
Regression line and Residual plots
Using the data set below, determine the correlation, slope, and intercept of the least squares regression line.
xˉ\bar{x}= 60 yˉ\bar{y}= 4.8
sxs_{x}= 38.08 sys_{y}= 2.59
Solving for the Regression Line
The equation y^=3+0.025x\hat{y}=3+0.025x is used to predict the amount of weight lost yy, in kilograms, by a group of individuals in a study based on the average amount of daily exercise xx, in minutes. Each kilogram is approximately 2.2 pounds.
Suppose that we change weight lost to pounds. Determine the new linear regression line.
Regression Line
What is the intercept of the regression line?
Regression Line
Suppose Big Joe is added to the scatterplot below. He is 13 years old and has 50% body fat. What will happen to the slope and correlation coefficient?
Correlation and Regression Line
Suppose we want to see if there is a relationship between the number of times a student attended their labs in the entire semester (x) and their final exam grade (y). We randomly sample 8 students and recorded our results.
∑(xi−x‾)(yi−y‾)=264.6\sum_{ }^{ }\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)=264.6
Regression line: Predictions and Residual Plots
True or false?
If the sum of the residuals is equal to 0, then the best-fitting regression line goes through all the data points.