Simple Linear Regression

Introduction

In statistics, linear regression is an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables (or independent variables) denoted X. In simple linear regression, there are only one independent variable, denoted x.

In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data. Such models are called linear models, which takes this form:

\[ y = b_0 + b_1x \] where \(x\) is the independent variable and \(y\) is the dependent variable.

Simple Linear Regression in R

To build the simple linear regression model in R, first we will import the dataset from a CSV file.

dataset = read.csv("Salary_Data.csv")

Then we split the dataset into training set and test set.

library(caTools)
split = sample.split(dataset$Salary, SplitRatio = 2/3)
training_set = subset(dataset, split == TRUE)
test_set = subset(dataset, split == FALSE)

The sample.split function takes a vector and based on a split ratio, it produces a split vector which contains two logical value TRUE and FALSE. Next we use the subset function to split the dataset based on the TRUE, FALSE value in split vector.

Now we can build the simple linear regression model based on the training set.

regressor = lm(formula = Salary ~ YearsExperience, data = training_set)

The lm function is used to fit linear models. The formula argument accepts a formula class object that describe the model to be fitted. Salary ~ YearsExperience means a linear model while the Salary and YearsExperience are dependent variable and independent variable.

Once we have the model, we can get the summary of the model by typing

summary(regressor)

Call:
lm(formula = Salary ~ YearsExperience, data = training_set)

Residuals:
   Min     1Q Median     3Q    Max 
 -7338  -4418   -225   2959  11803 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)      26105.9     2687.2   9.715 1.39e-08 ***
YearsExperience   9256.5      423.3  21.867 2.05e-14 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 5850 on 18 degrees of freedom
Multiple R-squared:  0.9637,    Adjusted R-squared:  0.9617 
F-statistic: 478.2 on 1 and 18 DF,  p-value: 2.053e-14

And we can apply this model to the test set to predict salary in test set.

salary_predict = predict(regressor, newdata = test_set)

Then we can compare the predicted value and the real value.

print(data.frame(Salary = test_set$Salary, Salary_Predict = salary_predict))

We can also plot our data to visualize the linear model we have built. We will use ggplot2 to plot our data.

library(ggplot2)
ggplot() +
  geom_point(aes(x = training_set$YearsExperience, y = training_set$Salary), colour = 'green') +
  geom_point(aes(x = test_set$YearsExperience, y = test_set$Salary), colour = 'red') +
  geom_line(aes(x = training_set$YearsExperience, y = predict(regressor, newdata = training_set)), colour = 'blue') +
  ggtitle('Salary vs Experience (Green: Training Set, Red: Test Set)') +
  xlab('Years of experience') +
  ylab('Salary')