Simple_regression
Mohammed Al haqbani
2022-10-02
- Overview of simple linear regression
- Linear equation of the simple regression model
- Simple linear regression with categorical predictor
- Explore the data
- Question 1: Can we predict the absence days using sex column (binary predictor)
- Question 2: Can we predict the absence days using marital status column (categorical predictor with more than 2 categories)
- Question 3: Can we predict the absence days using salary column (continuous or numerical predictor)
Overview of simple linear regression
Simple linear regression is a general framework for relating any continuous (numerical) outcome to a single predictor by a linear equation.
We called it simple because we only consider one predictor.
Regression models can be extended to allow using multiple predictors, and so called multiple regression, so we can better predict the numerical outcomes using more than one predictor.
Linear equation of the simple regression model
\(y = \beta_o + \beta_1 x_1\)
\(x_1\) is the predictor of interest which can be categorical or continuous.
\(y\) is the mean of outcome, \(\overline{y}\).
Simple linear regression with categorical predictor
- The categorical predictor can be:
Binary predictor with only two values.
Categorical predictor with more than two values.
Linear regression is fitted using the lm() function.
lm() function has 2 main arguments, formula takes the form \(outcome \sim predictor\) and data takes the dataset that contains the outcome and predictor of interest.
Explore the data
library(tidyverse)
dat<-read_csv("/Users/haqbani/Library/CloudStorage/Dropbox/5 _courses/20 R-programing/course material/project 1/HR_data.csv")
glimpse(dat)## Rows: 311
## Columns: 5
## $ EmpID <dbl> 10026, 10084, 10196, 10088, 10069, 10002, 10194, 10062, 10…
## $ Salary <dbl> 62506, 104437, 64955, 64991, 50825, 57568, 95660, 59365, 4…
## $ Sex <chr> "M", "M", "F", "F", "F", "F", "F", "M", "F", "M", "F", "M"…
## $ MaritalDesc <chr> "Single", "Married", "Married", "Married", "Divorced", "Si…
## $ Absences <dbl> 1, 17, 3, 15, 2, 15, 19, 19, 4, 16, 12, 15, 9, 7, 1, 20, 1…- Our data contains 311 rows (employees) and 5 columns:
EmpID: the employee ID (numeric).
Salary: the employee’s salary in dollars (numeric).
Sex: the employee’s gender, male or female (character).
MaritalDesc: the employee’s marital status (character).
Absences: the employee’s absences in days (numeric).
Question 1: Can we predict the absence days using sex column (binary predictor)
Frequency table of sex
table(dat$Sex)##
## F M
## 176 135- We have 176 females and 135 males in our data.
Plot the relation between sex and absence days
ggplot(data = dat, aes(x = Sex, y = Absences, col = Sex))+ geom_boxplot()
- Males tend to have lower absence than females.
Create the model
mod_sex<-lm(formula= Absences ~ Sex, data = dat)
mod_sex##
## Call:
## lm(formula = Absences ~ Sex, data = dat)
##
## Coefficients:
## (Intercept) SexM
## 10.26136 -0.05396- The linear regression equation is estimated as
\(\hat{y} = \hat{\beta_o} + \hat{\beta_1} x_1\)
\(\hat{y} = \hat{\beta_o} + \hat{\beta_1} SexM\)
\(\hat{y} = 10.26 - 0.05 SexM\)
We put hats as this regression equation with \(\hat{y}\),\(\hat{\beta_o}\),\(\hat{\beta_1}\) are estimated from our sample data.
\(SexM\) = 1 for males and =0 for females
This equation estimates 2 values:
mean absences for males \(\hat{y} = 10.26 - 0.05 SexM\) = 10.21 days since \(SexM\) = 1 for males.
mean absences for females \(\hat{y} = 10.26\) = 10.26 days since \(SexM\) = 0 for females.
- So in simple regression with binary predictor:
\(\hat{\beta_o}\) or intercept is the mean of continuous outcome for the reference category (females).
\(\hat{\beta_1}\) or slope is the mean difference between the two groups who differ by only one unit of predictor or sex \(x_1\).
- Since for binary predictor (as sex), \(x_1\) can only takes 2 values, 0 for females and 1 for males, so -0.05 day is the mean absence difference between females and males in this population. In other words, the absence days of females is higher than the absence days of males in this population by 0.05 day on the average.
Question 2: Can we predict the absence days using marital status column (categorical predictor with more than 2 categories)
Frequency table of marital status
table(dat$MaritalDesc)##
## Divorced Married Separated Single Widowed
## 30 124 12 137 8- We have 30 divorced, 124 married, 12 separated, 137 single, and 8 widowed employees in our data.
Plot the relation between marital status and absence days
ggplot(data = dat, aes(x = MaritalDesc, y = Absences, col = MaritalDesc))+ geom_boxplot()
- Separated employees tend to have lower absence than other employees.
Create the model
mod_marital<-lm(formula= Absences ~ MaritalDesc, data = dat)
mod_marital##
## Call:
## lm(formula = Absences ~ MaritalDesc, data = dat)
##
## Coefficients:
## (Intercept) MaritalDescMarried MaritalDescSeparated
## 10.0000 0.9274 -1.4167
## MaritalDescSingle MaritalDescWidowed
## -0.2263 0.8750- The linear regression equation is estimated as
\(\hat{y} = \hat{\beta_o} + \hat{\beta_1} Married+ \hat{\beta_2} Separated+ \hat{\beta_3} Single+ \hat{\beta_4} Widowed\)
\(\hat{y} = 10.00 + 0.93Married - 1.42Separated - 0.23Single + 0.88Widowed\)
The equation estimated \(\beta\) for each marital status and set divorced as the reference level.
\(Married\) = 1 for married employees and =0 for other marital status.
\(Separated\) = 1 for separated employees and =0 for other marital status, and so on.
This equation estimates 5 mean absences:
mean absence for divorced employees, \(\hat{y} = \hat{\beta_o}\) = 10 days since Married, Separated, Single, and Widowed= 0.
mean absence for married employees, \(\hat{y} = \hat{\beta_o} + \hat{\beta_1} Married\) = 10.00+0.93 = 10.93 days since Married = 1 for married employees and = 0 for other marital status.
mean absence for separated employees, \(\hat{y} = \hat{\beta_o} + \hat{\beta_2} Separated\) = 10.00-1.42 = 8.58 days.
mean absence for single employees, \(\hat{y} = \hat{\beta_o} + \hat{\beta_3} Single\) = 10.00-0.23 = 9.77 days.
mean absence for widowed employees, \(\hat{y} = \hat{\beta_o} + \hat{\beta_4} Widowed\) = 10.00+ 0.88= 10.88 days.
- We conclude from this simple regression:
Married employees have higher absence than divorced employees by 0.93 day on the average.
Separated employees have lower absence than divorced employees by 1.42 day on the average.
Single employees have lower absence than divorced employees by 0.23 day on the average.
Widowed employees have higher absence than divorced employees by 0.88 day on the average.
Question 3: Can we predict the absence days using salary column (continuous or numerical predictor)
Frequency table of salaries
table(dat$Salary)##
## 45046 45069 45115 45395 45433 45998 46120 46335 46428 46430 46654
## 1 1 1 1 1 1 1 1 1 1 1
## 46664 46738 46799 46837 46998 47001 47211 47414 47434 47750 47837
## 1 1 1 1 1 1 1 1 1 1 1
## 47961 48285 48413 48495 48513 48888 49256 49773 49920 50178 50274
## 1 1 1 1 1 1 1 1 1 1 1
## 50373 50428 50470 50482 50750 50825 50923 51044 51259 51337 51505
## 1 1 1 1 1 1 1 1 1 1 1
## 51777 51908 51920 52057 52087 52177 52249 52505 52599 52624 52674
## 1 1 1 1 1 1 1 1 1 1 1
## 52788 52846 52984 53018 53060 53171 53180 53189 53250 53366 53492
## 1 1 1 1 1 1 1 1 1 1 1
## 53564 54005 54132 54237 54285 54670 54828 54933 55000 55140 55315
## 1 1 1 1 1 1 1 1 1 1 1
## 55425 55578 55688 55722 55800 55875 55965 56147 56149 56294 56339
## 1 1 1 1 1 1 1 1 1 1 1
## 56847 56991 57568 57575 57583 57748 57815 57834 57859 57954 57975
## 1 1 1 1 1 1 2 1 1 1 1
## 58062 58207 58273 58275 58370 58371 58523 58530 58709 58939 59026
## 1 1 1 1 1 1 1 1 1 1 1
## 59124 59144 59231 59238 59365 59369 59370 59472 59728 59892 60070
## 1 1 1 1 1 1 1 1 1 1 1
## 60120 60270 60340 60380 60436 60446 60627 60656 60724 60754 61154
## 1 1 1 1 1 1 1 1 1 1 1
## 61242 61349 61355 61422 61555 61568 61584 61656 61729 61809 61844
## 2 1 1 1 1 1 1 1 1 1 1
## 61962 62061 62065 62068 62162 62385 62425 62506 62514 62659 62810
## 1 1 1 1 1 1 1 1 1 1 1
## 62910 62957 63000 63003 63025 63051 63108 63291 63322 63353 63381
## 1 1 1 1 2 1 1 1 1 1 1
## 63430 63450 63478 63515 63676 63682 63695 63763 63813 63878 63973
## 1 1 1 1 1 1 1 1 1 1 1
## 64021 64057 64066 64246 64375 64397 64520 64724 64738 64786 64816
## 1 1 1 1 1 1 1 1 1 1 1
## 64919 64955 64971 64991 64995 65288 65310 65707 65714 65729 65893
## 1 1 1 1 1 1 1 1 1 1 1
## 65902 66074 66149 66441 66541 66593 66738 66808 66825 67176 67237
## 1 1 1 1 1 1 1 1 1 1 1
## 67251 68051 68099 68182 68407 68678 68829 68898 68999 69340 70131
## 1 1 1 1 1 1 1 1 1 1 1
## 70187 70468 70507 70545 70621 71339 71707 71776 71860 71966 72106
## 1 1 1 1 1 1 1 1 1 1 1
## 72202 72460 72609 72640 72992 73330 74226 74241 74312 74326 74417
## 1 1 1 1 1 1 1 1 1 1 1
## 74669 74679 74813 75188 75281 76029 77692 77915 80512 81584 82758
## 1 1 1 1 1 1 1 1 1 1 1
## 83082 83363 83552 83667 84903 85028 86214 87565 87826 87921 88527
## 1 1 1 1 1 1 1 1 1 1 1
## 88976 89292 89883 90100 92328 92329 92989 93046 93093 93206 93396
## 1 1 1 1 1 1 1 1 1 1 1
## 93554 95660 95920 96820 97999 99020 99280 99351 100031 100416 101199
## 1 1 1 1 1 1 1 1 1 1 1
## 103613 104437 105688 105700 106367 107226 108987 110000 110929 113999 114800
## 1 1 1 1 1 1 1 1 1 1 1
## 120000 138888 140920 148999 150290 157000 170500 178000 180000 220450 250000
## 1 1 1 1 1 1 1 1 1 1 1- We have different salaries from 45,046 to 250,000 dollars in our data.
Plot the relation between salary and absence days
ggplot(data = dat, aes(x = Salary, y = Absences))+ geom_point()+ geom_smooth(method = "lm")
- High salary employees tend to have higher absence than low salary employees.
Create the model
mod_salary<-lm(formula= Absences ~ Salary, data = dat)
mod_salary##
## Call:
## lm(formula = Absences ~ Salary, data = dat)
##
## Coefficients:
## (Intercept) Salary
## 8.915e+00 1.917e-05- The linear regression equation is estimated as
\(\hat{y} = \hat{\beta_o} + \hat{\beta_1} x_1\)
\(\hat{y} = \hat{\beta_o} + \hat{\beta_1} Salary\)
\(\hat{y} = 8.915 + 0.000019 Salary\)
This equation estimates 2 values:
mean absence for employees with 0 salary, \(\hat{y} = \hat{\beta_o} + \hat{\beta_1} Salary\) = \(\hat{\beta_o}\) = 8.915 dollar since \(Salary\) = 0
Slope, \(\hat{\beta_1}\)= 0.000019 , has many meanings:
Positive number meaning that linear regression line is of positive slope and as the value of salary increases, the absence increase on the average.
The mean difference in absence for groups of employees who differ by 1 dollar in salary. In other words, each 1 dollar increase in salary is associated with 0.000019 day increase in absence.
This estimate is constant across all values of salaries in our data (45,046 to 250,000) but we cannot extrapolate this relationship to higher or lower values.
Some continuous predictors, as salary, do not have a zero value naturally. In that case, R will also estimate an intercept but this intercept is scientifically meaningless and it is just a placeholder to define the linear equation.
Examples
The mean absence difference between employees with salary 45,046 compared to employees with 45,047 = 0.000019 day.
The mean absence difference between employees with salary 46,000 compared to employees with 46,001 = 0.000019 day.
The mean absence difference between employees with salary 46,000 dollars compared to employees with 47,000 = 1000 X 0.000019 = 0.019 day.
The mean absence difference between employees with salary 46,000 dollars compared to employees with 100,000 = 54000 X 0.000019 = 1.026 day.
The mean absence difference between employees with salary 46,000 dollars compared to employees with 250,000 = 204000 X 0.000019 = 3.876 days.