Modeling
Sean Marshelle
1. Modeling
Modeling is a process to provide low-dimensional summary of a dataset. The goal is to partition data into patterns and residuals. There are two parts to modeling defining a family of models and generating a fitted model.
Statistical Machine Learning used to modeling a dataset, for estimate and inference. Example a model Y=f(x)+e, estimate is to knowthe dimensions of an X, inference is to conclude how X can explain Y.
# package
library(ggplot2)
library(caret)## Warning: package 'caret' was built under R version 4.3.2## Loading required package: latticelibrary(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.3 ✔ readr 2.1.4
## ✔ forcats 1.0.0 ✔ stringr 1.5.0
## ✔ lubridate 1.9.3 ✔ tibble 3.2.1
## ✔ purrr 1.0.2 ✔ tidyr 1.3.0## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ✖ purrr::lift() masks caret::lift()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorslibrary(AER)## Warning: package 'AER' was built under R version 4.3.2## Loading required package: car
## Loading required package: carData
##
## Attaching package: 'car'
##
## The following object is masked from 'package:dplyr':
##
## recode
##
## The following object is masked from 'package:purrr':
##
## some
##
## Loading required package: lmtest## Warning: package 'lmtest' was built under R version 4.3.2## Loading required package: zoo## Warning: package 'zoo' was built under R version 4.3.2##
## Attaching package: 'zoo'
##
## The following objects are masked from 'package:base':
##
## as.Date, as.Date.numeric
##
## Loading required package: sandwich## Warning: package 'sandwich' was built under R version 4.3.2## Loading required package: survival
##
## Attaching package: 'survival'
##
## The following object is masked from 'package:caret':
##
## clusterlibrary(corrplot)## Warning: package 'corrplot' was built under R version 4.3.2## corrplot 0.92 loadedlibrary(randomForest)## Warning: package 'randomForest' was built under R version 4.3.2## randomForest 4.7-1.1
## Type rfNews() to see new features/changes/bug fixes.
##
## Attaching package: 'randomForest'
##
## The following object is masked from 'package:dplyr':
##
## combine
##
## The following object is masked from 'package:ggplot2':
##
## margin2. Data
data(Affairs, package="AER")
head(Affairs, 10) summary(Affairs) ## affairs gender age yearsmarried children
## Min. : 0.000 female:315 Min. :17.50 Min. : 0.125 no :171
## 1st Qu.: 0.000 male :286 1st Qu.:27.00 1st Qu.: 4.000 yes:430
## Median : 0.000 Median :32.00 Median : 7.000
## Mean : 1.456 Mean :32.49 Mean : 8.178
## 3rd Qu.: 0.000 3rd Qu.:37.00 3rd Qu.:15.000
## Max. :12.000 Max. :57.00 Max. :15.000
## religiousness education occupation rating
## Min. :1.000 Min. : 9.00 Min. :1.000 Min. :1.000
## 1st Qu.:2.000 1st Qu.:14.00 1st Qu.:3.000 1st Qu.:3.000
## Median :3.000 Median :16.00 Median :5.000 Median :4.000
## Mean :3.116 Mean :16.17 Mean :4.195 Mean :3.932
## 3rd Qu.:4.000 3rd Qu.:18.00 3rd Qu.:6.000 3rd Qu.:5.000
## Max. :5.000 Max. :20.00 Max. :7.000 Max. :5.000Data named Affairs have 9 variables such as affairs, gender, age, years married, children, religiousness, education, occupation, and rating. From 9 variables, children is gonna be used as Y variable and the others as X variables.
3. Data Exploration
# Using bar chart to see numbers for Yes and No
ggplot(Affairs, aes(x = children)) +
geom_bar(fill = "lightblue") +
labs(x = "Y Respond", y = "Frekuensi", title = "Bar Chart Y Respond") +
theme_minimal()
Affairs.2 <- Affairs %>%
mutate(gender = ifelse(gender == "male", 1, 0)) %>%
mutate(children = ifelse(children == "no", 0, 1))
head(Affairs.2)multicol <- cor(Affairs.2)
corrplot(multicol, method = "number")
The variable respond show, >150 “No” or didn’t have children and >400 “Yes” or have children.After checking multicolinearity there are variables that indicate have multicolinearity such as years married and age; and education and occupation.Because some variables have multicolinearity the variables years married and age gonna exclude it from the data.
Affairs <- subset(Affairs, select = c("affairs","gender","children","religiousness","rating","education","occupation"))
head(Affairs,10)4. Modeling
Affairs$children <- as.factor(Affairs$children)
Affairs$gender <- as.factor(Affairs$gender)
Affairs$religiousness <- as.double(Affairs$religiousness)
Affairs$rating <- as.double(Affairs$rating)
Affairs$occupation <- as.double(Affairs$occupation)# Data Split
set.seed(1910)
split.data <- createDataPartition(Affairs$children, p = 0.8, list=FALSE)
trainData <- Affairs[split.data,]
testData <- Affairs[-split.data,]# Logistic Regression
log.reg <- glm(children~ ., data = trainData, family = binomial)
summary(log.reg)##
## Call:
## glm(formula = children ~ ., family = binomial, data = trainData)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.04092 0.93596 1.112 0.266077
## affairs 0.08941 0.04002 2.234 0.025486 *
## gendermale 0.63746 0.24556 2.596 0.009434 **
## religiousness 0.34322 0.09348 3.671 0.000241 ***
## rating -0.42409 0.11308 -3.750 0.000177 ***
## education 0.06388 0.05546 1.152 0.249346
## occupation -0.20984 0.07561 -2.775 0.005517 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 574.75 on 480 degrees of freedom
## Residual deviance: 525.33 on 474 degrees of freedom
## AIC: 539.33
##
## Number of Fisher Scoring iterations: 4rob.reg.log<-predict(log.reg, testData, type="response")
prediksi<-as.factor(ifelse(rob.reg.log>0.5,"yes","no"))
pred.aktual<-data.frame(prediksi,testData$children)
head(pred.aktual)confusionMatrix(prediksi, testData$children)## Confusion Matrix and Statistics
##
## Reference
## Prediction no yes
## no 9 5
## yes 25 81
##
## Accuracy : 0.75
## 95% CI : (0.6627, 0.8245)
## No Information Rate : 0.7167
## P-Value [Acc > NIR] : 0.2415025
##
## Kappa : 0.2512
##
## Mcnemar's Test P-Value : 0.0005226
##
## Sensitivity : 0.2647
## Specificity : 0.9419
## Pos Pred Value : 0.6429
## Neg Pred Value : 0.7642
## Prevalence : 0.2833
## Detection Rate : 0.0750
## Detection Prevalence : 0.1167
## Balanced Accuracy : 0.6033
##
## 'Positive' Class : no
## # Random Forest
rf_model <- randomForest(children ~ ., data = trainData)
summary(rf_model)## Length Class Mode
## call 3 -none- call
## type 1 -none- character
## predicted 481 factor numeric
## err.rate 1500 -none- numeric
## confusion 6 -none- numeric
## votes 962 matrix numeric
## oob.times 481 -none- numeric
## classes 2 -none- character
## importance 6 -none- numeric
## importanceSD 0 -none- NULL
## localImportance 0 -none- NULL
## proximity 0 -none- NULL
## ntree 1 -none- numeric
## mtry 1 -none- numeric
## forest 14 -none- list
## y 481 factor numeric
## test 0 -none- NULL
## inbag 0 -none- NULL
## terms 3 terms callrf_predictions <- predict(rf_model, newdata = testData)
summary(rf_predictions)## no yes
## 24 96confusionMatrix(as.factor(rf_predictions), as.factor(testData$children))## Confusion Matrix and Statistics
##
## Reference
## Prediction no yes
## no 15 9
## yes 19 77
##
## Accuracy : 0.7667
## 95% CI : (0.6807, 0.839)
## No Information Rate : 0.7167
## P-Value [Acc > NIR] : 0.13169
##
## Kappa : 0.3694
##
## Mcnemar's Test P-Value : 0.08897
##
## Sensitivity : 0.4412
## Specificity : 0.8953
## Pos Pred Value : 0.6250
## Neg Pred Value : 0.8021
## Prevalence : 0.2833
## Detection Rate : 0.1250
## Detection Prevalence : 0.2000
## Balanced Accuracy : 0.6683
##
## 'Positive' Class : no
## Logistic regression and random forest modeling predicted people have affairs even they are have children with accuracy of logistic regression 75% and random forest 77%, from the accuracy value, random forest is the best method to predict the data. Sensitivity and specificity value need to evaluate. Sensitivity is indicates the model is good at capturing positive instances and specificity is indicates the model is good at avoiding false positives. Based on accuracy, sensitivity, and specificity random forest method is better than logistic regression.