An Introduction

Author: Adrian Tan

Date: 6/4/2024

Simple Linear Regression

• What is Simple Linear Regression?
• Applications
• Key Concepts
• Example

Definition: Simple Linear Regression is a statistical method that allows us to summarize and study relationships between two continuous (quantitative) variables: - One variable, denoted \(Y\), is regarded as the dependent variable. - The other variable, denoted \(X\), is regarded as the independent variable.

Applications of Simple Linear Regression: - Predicting house prices based on size - Estimating sales based on advertising spend - Predicting student scores based on study hours

Key Concepts:

- Dependent Variable (Y): The variable we are trying to predict or explain.

- Independent Variable (X): The variable we are using to make predictions.

- Regression Line: The line that best fits the data points on a scatter plot.

- Equation: \(Y = \beta_0 + \beta_1X + \epsilon\)

The Regression Equation:

• \(Y = \beta_0 + \beta_1X + \epsilon\)
• \(\beta_0\): Intercept (value of Y when X = 0) - \(\beta_1\): Slope (change in Y for a one-unit change in X)
• \(\epsilon\): Error term (difference between observed and predicted Y)

Example Problem: Suppose we have data on the number of hours studied (X) and the test scores (Y) of students.
-Data:

Calculating the Regression Line:

To find the regression line, we need to calculate the slope (\(\beta_1\)) and the intercept (\(\beta_0\)).

Slope (\(\beta_1\)):

\[ \beta_1 = \frac{n(\sum XY) - (\sum X)(\sum Y)}{n(\sum X^2) - (\sum X)^2} \] Intercept (\(\beta_0\)): \[ \beta_0 = \frac{\sum Y - \beta_1 \sum X}{n} \] Where: - \(n\) is the number of observations - \(\sum XY\) is the sum of the product of X and Y values - \(\sum X\) and \(\sum Y\) are the sums of X and Y values respectively - \(\sum X^2\) is the sum of the squares of X values Let’s use these formulas to calculate the regression line for our example data.

Example Solution:

Using the data provided: - \(\sum X = 1 + 2 + 3 + 4 + 5 = 15\) - \(\sum Y = 50 + 55 + 60 + 65 + 70 = 300\) - \(\sum XY = (1 \cdot 50) + (2 \cdot 55) + (3 \cdot 60) + (4 \cdot 65) + (5 \cdot 70) = 1100\) - \(\sum X^2 = 1^2 + 2^2 + 3^2 + 4^2 + 5^2 = 55\) - \(n = 5\) Calculate the slope (\(\beta_1\)): \[ \beta_1 = \frac{n(\sum XY) - (\sum X)(\sum Y)}{n(\sum X^2) - (\sum X)^2} = \frac{5(1100) - (15)(300)}{5(55) - (15)^2} = \frac{5500 - 4500}{275 - 225} = \frac{1000}{50} = 20 \] Calculate the intercept (\(\beta_0\)): \[ \beta_0 = \frac{\sum Y - \beta_1 \sum X}{n} = \frac{300 - 20 \cdot 15}{5} = \frac{300 - 300}{5} = 0 \] Thus, the regression equation is: \[ Y = 0 + 20X \] This equation can be used to predict the test score based on the number of hours studied.

Prediction: Using the regression equation \(Y = 0 + 20X\): - If a student studies for 6 hours, their predicted test score is: \[ Y = 0 + 20 \cdot 6 = 120 \] Thus, by inputting the number of hours studied into our regression equation, we can predict the test score for any given number of study hours.

## 
## Attaching package: 'plotly'

## The following object is masked from 'package:ggplot2':
## 
##     last_plot

## The following object is masked from 'package:stats':
## 
##     filter

## The following object is masked from 'package:graphics':
## 
##     layout

R Code: This R code creates a scatter plot with a regression line using the ggplot2 library.

# Load the ggplot2 library
library(ggplot2)

# Example data
data <- data.frame(
  Hours_Studied = c(1, 2, 3, 4, 5),
  Test_Score = c(50, 55, 60, 65, 70)
)

# Fit a linear model
model <- lm(Test_Score ~ Hours_Studied, data = data)

# Create a scatter plot with a regression line
ggplot(data, aes(x = Hours_Studied, y = Test_Score)) +
  geom_point() +
  geom_smooth(method = "lm", col = "blue") +
  ggtitle("Regression Line: Hours Studied vs Test Score") +
  xlab("Hours Studied") +
  ylab("Test Score")

## `geom_smooth()` using formula = 'y ~ x'

Simple Linear Regression is a powerful tool for predicting outcomes. Key components include: Dependent Variable (Y): The variable we are trying to predict or explain. Independent Variable (X): The variable we are using to make predictions. Regression Line: The line that best fits the data points on a scatter plot. - Slope (\(\beta_1\)): Change in Y for a one-unit change in X. Intercept (\(\beta_0\)): Value of Y when X = 0. Example Problem: Demonstrated how to calculate the regression line and make predictions using given data on hours studied and test scores. Visualizations: Included scatter plots, regression lines, and a 3D plot to illustrate the relationship between variables.

Understanding these concepts allows us to use simple linear regression effectively for various real-world applications.

References:

• https://openstax.org/details/books/introductory-statistics/
• Wikipedia: Linear Regression