/– title: “Business Analytics Lab Worksheet 04” author: “Ava Risch” date: “09/29/2026 output: word_document: default html_notebook: default pdf_document: default html_document: default subtitle: Data Preparation Activities - Outliers —
About
The focus of this lab is on data outliers.
Setup
Remember to always set your working directory to the source file location. Go to ‘Session’, scroll down to ‘Set Working Directory’, and click ‘To Source File Location’. Read carefully the below and follow the instructions to complete the tasks and answer any questions. Submit your work to RPubs as detailed in previous notes.
Important Note
For your assignment you may be using different data sets than what is included in this worksheet demo. Make sure to read carefully the instructions on Sakai.
Task 1: Data Outliers
First, we must calculate the mean, standard deviation, maximum, and minimum for the Age column using R.
In R, we must read in the file again, extract the column and find the values that are asked for.
#Read File
mydata = read.csv(file="data/creditrisk.csv")
head(mydata)## Loan.Purpose Checking Savings Months.Customer Months.Employed Gender
## 1 Small Appliance 0 739 13 12 M
## 2 Furniture 0 1230 25 0 M
## 3 New Car 0 389 19 119 M
## 4 Furniture 638 347 13 14 M
## 5 Education 963 4754 40 45 M
## 6 Furniture 2827 0 11 13 M
## Marital.Status Age Housing Years Job Credit.Risk
## 1 Single 23 Own 3 Unskilled Low
## 2 Divorced 32 Own 1 Skilled High
## 3 Single 38 Own 4 Management High
## 4 Single 36 Own 2 Unskilled High
## 5 Single 31 Rent 3 Skilled Low
## 6 Married 25 Own 1 Skilled Low#Name the extracted variable
age = mydata$Age
head(age)## [1] 23 32 38 36 31 25#Calculate the average age below. Refer to Worksheet 2 for the correct command.
meanage = mean(mydata$Age)
meanage## [1] 34.39765#Calculate standard deviation of age below. Refer to Worksheet 2 for the correct command.
spreadage=sd(age)
spreadage## [1] 11.04513Next, use the formula from class to detect any outliers. An outlier
is value that “lies outside” most of the other values in a set of data.
A common way to estimate the upper and lower threshold is to take the
mean (+ or -) 3 * standard deviation. Try using this
formula to find the upper and lower limit for age.
#Use the formula above to calculate the upper and lower threshold
#UpperThreshold = meanage + 3*spreadage
UpperThreshold = mean(mydata$Age) + 3*sd(mydata$Age)
UpperThreshold## [1] 67.53302LowerThreshold = meanage - 3*spreadage
LowerThreshold## [1] 1.262269Do you think we have outliers? Yes, there is an outlier.
#Calculate the maximum of age below. The command to find the maximum is max(variable) where variable is the extracted variable.
MaxAge = max(age)
MaxAge## [1] 73upoutlier = age > UpperThreshold
which(upoutlier == TRUE)## [1] 183#or
subset(mydata, age > UpperThreshold)## Loan.Purpose Checking Savings Months.Customer Months.Employed Gender
## 183 Business 0 406 6 35 M
## Marital.Status Age Housing Years Job Credit.Risk
## 183 Single 73 Own 4 Unskilled Low#Calculate the minimum of age below. The command to find the minimum is min(variable) where variable is the extracted variable.
MinAge = min(age)
MinAge## [1] 18Do you expect to have outliers? No, there are no lower-end outliers.
Another similar method to find the upper and lower thresholds involves finding the interquartile range. Follow along below to see how we first calculate the interquartile range..
quantile(age) ## 0% 25% 50% 75% 100%
## 18 26 32 41 73lowerq = quantile(age)[2]
upperq = quantile(age)[4]
iqr = upperq - lowerq
iqr## 75%
## 15#or
iqrange=IQR(age)
iqrange## [1] 15The formula below calculates the threshold. The threshold is the boundaries that determine if a value is an outlier. If the value falls above the upper threshold or below the lower threshold, it is an outlier.
Below is the upper threshold:
upperthreshold = (iqr * 1.5) + upperq
upperthreshold## 75%
## 63.5upoutlier = age > upperthreshold
which(upoutlier == TRUE)## [1] 79 144 156 179 183 214 318 363 412#or
subset(mydata, age>upperthreshold)## Loan.Purpose Checking Savings Months.Customer Months.Employed Gender
## 79 Repairs 0 626 43 0 M
## 144 Small Appliance 317 10980 13 17 M
## 156 New Car 3329 0 19 15 M
## 179 New Car 0 886 22 96 M
## 183 Business 0 406 6 35 M
## 214 Business 8948 110 31 90 M
## 318 Repairs 271 759 19 0 F
## 363 New Car 0 272 7 90 M
## 412 Small Appliance 765 10406 10 24 F
## Marital.Status Age Housing Years Job Credit.Risk
## 79 Single 64 Own 4 Unemployed Low
## 144 Single 65 Own 3 Unskilled High
## 156 Single 67 Rent 4 Skilled High
## 179 Single 64 Own 4 Skilled Low
## 183 Single 73 Own 4 Unskilled Low
## 214 Single 65 Own 4 Management High
## 318 Divorced 66 Own 4 Skilled Low
## 363 Single 67 Own 4 Management High
## 412 Divorced 65 Own 3 Unskilled LowBelow is the lower threshold:
lowerthreshold = lowerq - (iqr * 1.5)
lowerthreshold## 25%
## 3.5loweroutlier = age < lowerthreshold
which(loweroutlier == TRUE)## integer(0)#or
subset(mydata, age<lowerthreshold)## [1] Loan.Purpose Checking Savings Months.Customer
## [5] Months.Employed Gender Marital.Status Age
## [9] Housing Years Job Credit.Risk
## <0 rows> (or 0-length row.names)Are there any outliers? How many? Yes, there are 9 outliers. visualieta using a box and whisker plot. The boxplot below supports the IQR we found of 15 and upper and lower threshold.
boxplot(age) 
There are at least 5 outliers as seen on the boxplot.
Part 2: Another Eample to Check for possible outliers
Task 1: Standard Approach
library(readxl)
myengdata <- read_excel("data/Engineers.xlsx")
dim(myengdata)## [1] 100 5head(myengdata)## # A tibble: 6 × 5
## BirthDate Personality Salary Level Certficates
## <dttm> <chr> <dbl> <chr> <dbl>
## 1 1973-07-31 00:00:00 Explorer 48000 Engineer I 0
## 2 1967-08-29 00:00:00 Diplomat 44000 Engineer I 5
## 3 1969-05-28 00:00:00 Explorer 42000 Engineer I 2
## 4 1961-02-22 00:00:00 Explorer 54000 Engineer II 3
## 5 1957-04-06 00:00:00 Analyst 59000 Engineer II 5
## 6 1974-02-06 00:00:00 Diplomat 102000 Senior Engineer 3#Calculate mean&standard deviation
Mean = mean(myengdata$Salary)
Spread = sd(myengdata$Salary)
Mean## [1] 60020Spread## [1] 17552.79#Calculate upper threshold for Salary
UpperThr = Mean + 3*Spread
UpperThr## [1] 112678.4#Calculate lower threshold for ambience
LowerThr = Mean - 3*Spread
LowerThr## [1] 7361.616Do you think we have outliers? Yes, there are possible outliers at the upper-end.
#Calculate the maximum of age below. The command to find the maximum is max(variable) where variable is the extracted variable.
Max = max(myengdata$Salary)
Min = min(myengdata$Salary)
Max## [1] 126000Min## [1] 32000Possible outliers at the upper end of the Salary data, but still we can check through the following chunk
upoutlier = myengdata$Salary > UpperThr
which(upoutlier == TRUE)## [1] 80#or
subset(myengdata, myengdata$Salary > UpperThr)## # A tibble: 1 × 5
## BirthDate Personality Salary Level Certficates
## <dttm> <chr> <dbl> <chr> <dbl>
## 1 1979-09-11 00:00:00 Diplomat 126000 Senior Engineer 5You can run the same analysis if we have outliers at the lower end.
Task 2: IQR approach
Another similar method to find the upper and lower thresholds involves finding the interquartile range. Follow along below to see how we first calculate the interquartile range..
quantile(myengdata$Salary) ## 0% 25% 50% 75% 100%
## 32000 47000 57000 68000 126000lowerq = quantile(myengdata$Salary,0.25)
upperq = quantile(myengdata$Salary,0.75)
iqr = upperq - lowerq
iqr## 75%
## 21000#or
iqrange=IQR(myengdata$Salary)
iqrange## [1] 21000The formula below calculates the threshold. The threshold is the boundaries that determine if a value is an outlier. If the value falls above the upper threshold or below the lower threshold, it is an outlier.
Below is the upper threshold:
upperthreshold = (iqr * 1.5) + upperq
upperthreshold## 75%
## 99500upoutlier = myengdata$Salary > upperthreshold
which(upoutlier == TRUE)## [1] 6 14 80#or
subset(myengdata, myengdata$Salary>upperthreshold)## # A tibble: 3 × 5
## BirthDate Personality Salary Level Certficates
## <dttm> <chr> <dbl> <chr> <dbl>
## 1 1974-02-06 00:00:00 Diplomat 102000 Senior Engineer 3
## 2 1974-04-29 00:00:00 Explorer 109000 Senior Engineer 6
## 3 1979-09-11 00:00:00 Diplomat 126000 Senior Engineer 5Below is the lower threshold:
lowerthreshold = lowerq - (iqr * 1.5)
lowerthreshold## 25%
## 15500loweroutlier = myengdata$Salary < lowerthreshold
which(loweroutlier == TRUE)## integer(0)#or
subset(myengdata, myengdata$Salary<lowerthreshold)## # A tibble: 0 × 5
## # ℹ 5 variables: BirthDate <dttm>, Personality <chr>, Salary <dbl>,
## # Level <chr>, Certficates <dbl>Are there any outliers? How many? Yes, there are 3 outliers. It can also be useful to visualize the data using a box and whisker plot. The boxplot below supports the IQR we found of 15 and upper and lower threshold.
boxplot(myengdata$Salary) 
There are at least 3 outliers as seen on the boxplot.