2024-03-18

Linear Regression

  • Linear Regression is a statistical concept that is used to predict a certain value, based on other data points.
  • A common example of Linear Regression is known as “The Line of Best Fit”.
  • Linear Regression uses Exploratory Variables to find the relationship between all variables

Exploratory Variables

  • Exploratory Variables are known by other names, such as Dependent and Independent Variables
  • If one Exploratory Variable exists, then the model is referred to as “Simple Linear Regression”
  • If multiple Exploratory Variables exist, then the model is referred to as “Multiple Linear Regression”

Mathematics of Linear Regression

Linear Regression Equation:
\(y_i = x_i^T \beta + \epsilon_i\) with \(i=1,...,n\)

\(x =\) represents the vector of regressors
\(T\) represents the Transpose
\(x_i^T \beta\) represents the _inner product between vectors \(x_i\) and \(\beta\)

More Math!

Another form of the equation for Linear Regression is:
- \(y_i = \beta_0 + \beta_1 x_i_1 + ... + \beta_p x_i_p + \epsilon_i\)
- This form is equivalent to the equation from the previous slide

An example of Linear Regression

  • To illustrate an example of how Linear Regression is used we will be using a public dataset called “Salary Dataset - Simple Linear Regression” to illustrate this
  • Below we import the dataset, called “Salary_dataset.csv” and show the head of the dataset
  • Link to Source: Kaggle Salary Dataset
head(salaryDataset)
##   X YearsExperience Salary
## 1 0             1.2  39344
## 2 1             1.4  46206
## 3 2             1.6  37732
## 4 3             2.1  43526
## 5 4             2.3  39892
## 6 5             3.0  56643
## [1] 0

Visualizing the Data

First let’s graph the dataset using the plotly function

plot_ly(data = salaryDataset, x = salaryDataset$YearsExperience, y = salaryDataset$Salary, type = 'scatter', mode = 'markers')

Finding the Correlation

Now we’ll use the lm() function to use Linear Regression to find the correlation between “Years of Experience” and “Salary”

linearRegressionModel = lm(salaryDataset$YearsExperience ~ salaryDataset$Salary, data = salaryDataset)
summary(linearRegressionModel)
## 
## Call:
## lm(formula = salaryDataset$YearsExperience ~ salaryDataset$Salary, 
##     data = salaryDataset)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.12974 -0.46457  0.04105  0.54311  0.79669 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)          -2.283e+00  3.273e-01  -6.976 1.38e-07 ***
## salaryDataset$Salary  1.013e-04  4.059e-06  24.950  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5992 on 28 degrees of freedom
## Multiple R-squared:  0.957,  Adjusted R-squared:  0.9554 
## F-statistic: 622.5 on 1 and 28 DF,  p-value: < 2.2e-16

Visualizing the Linear Regression

Below is a plot of our model onto the dataset

dataPlot = ggplot(salaryDataset, aes(x = YearsExperience, y = Salary)) + geom_point()
modelPlot = dataPlot + geom_smooth(method = "lm", se = FALSE)
print(modelPlot)
## `geom_smooth()` using formula = 'y ~ x'

Visualizing the Linear Regression - Continued

Now let’s add labels and colors to visualize the model better

dataPlot = ggplot(salaryDataset, aes(x = YearsExperience, y = Salary)) + geom_point() + labs(x = "Years of Experience", y = "Salary in Dollars", title = "Linear Regression Model between Experience & Salary")
modelPlot = dataPlot + geom_smooth(method = "lm", se = FALSE, color = "green")
print(modelPlot)
## `geom_smooth()` using formula = 'y ~ x'

Summary

  • As we can see in the previous two slides, we can see a strong correlation between “Years of Experience” and “Salary in Dollars”
  • As years of experience increases, so does salary
  • As I have shown, Linear Regression is an incredible statistical tool to be able to see if a correlation exists between two characteristics in a dataset
  • Of course, Linear Regression is one of many tools that a Data Scientist has, but it is a useful tool nonetheless