In this assignment, you’ll explore both numeric and categorical variables and do regression analysis.
You will use the dataset D8 described on DMR (online access via Pitt
network) Appendix A (data8) for this assignment. The data is
audit.csv and already included in the repository. The
objective is to predict the binary (TARGET_Adjusted) and continuous
(RISK_Adjustment) target variables. Create RMarkdown report using this
template.
hint: treat the given binary values “1” as “positive”
and “0” as “negative”
hint: be careful about missing values in data, try to
remove them if there are only a few instances. You will loose 2 points
if you directly using missing values without processing them.
The audit.csv data set is an artificially constructed data set that contains the characteristics of n = 2000 individual tax returns. The data set includes the following variables:
Read the data description above, and identify and report response variable and predictors.
Explore the dataset, and generate both statistical and graphical summary with respect to the numerical and categorical variables.
Apply logistic regression analysis to predict TARGET_Adjusted. Evaluate the models through cross-validation and on holdout samples. Interpret the effect of the predictors.
Apply linear and non-linear regression analysis to predict RISK_Adjustment. Evaluate the models through 10-fold cross-validation.
YOUR ANSWER for Problem 1.
# Load necessary library
library(readr)
# Read the data description from the given source
data_description <- "
The audit.csv data set is an artificially constructed data set that contains the characteristics of n = 2000 individual tax returns. The data set includes the following variables:
- ID: Unique identifier for each person.
- Age: Age of person.
- Employment: Type of employment.
- Education: Highest level of education.
- Marital: Current marital status.
- Occupation: Type of occupation.
- Income: Amount of income declared.
- Gender: Gender of person.
- Deductions: Total amount of expenses that a person claims in their financial statement.
- Hours: Average hours worked on a weekly basis.
- RISK_Adjustment: The continuous target variable; this variable records the monetary amount of any adjustment to the person’s financial claims as a result of a productive audit. This variable is a measure of the size of the risk associated with the person.
- TARGET_Adjusted: The binary target variable for classification modeling (0/1), indicating nonproductive and productive audits, respectively. Productive audits are those that result in an adjustment being made to a client’s financial statement.
"
# Print the data description
cat(data_description)
##
## The audit.csv data set is an artificially constructed data set that contains the characteristics of n = 2000 individual tax returns. The data set includes the following variables:
##
## - ID: Unique identifier for each person.
## - Age: Age of person.
## - Employment: Type of employment.
## - Education: Highest level of education.
## - Marital: Current marital status.
## - Occupation: Type of occupation.
## - Income: Amount of income declared.
## - Gender: Gender of person.
## - Deductions: Total amount of expenses that a person claims in their financial statement.
## - Hours: Average hours worked on a weekly basis.
## - RISK_Adjustment: The continuous target variable; this variable records the monetary amount of any adjustment to the person’s financial claims as a result of a productive audit. This variable is a measure of the size of the risk associated with the person.
## - TARGET_Adjusted: The binary target variable for classification modeling (0/1), indicating nonproductive and productive audits, respectively. Productive audits are those that result in an adjustment being made to a client’s financial statement.
# Identify and report response variable and predictors
response_variable <- "TARGET_Adjusted"
predictors <- c("ID", "Age", "Employment", "Education", "Marital", "Occupation", "Income", "Gender", "Deductions", "Hours", "RISK_Adjustment")
# Print response variable and predictors
cat("\nResponse Variable:", response_variable, "\n")
##
## Response Variable: TARGET_Adjusted
cat("Predictors:", paste(predictors, collapse = ", "), "\n")
## Predictors: ID, Age, Employment, Education, Marital, Occupation, Income, Gender, Deductions, Hours, RISK_Adjustment
# Read the CSV file
data <- read_csv("audit.csv")
# Print the first few rows of the data
head(data)
## # A tibble: 6 × 12
## ID Age Employment Education Marital Occupation Income Gender Deductions
## <dbl> <dbl> <chr> <chr> <chr> <chr> <dbl> <chr> <dbl>
## 1 1004641 38 Private College Unmarr… Service 8.18e4 Female 0
## 2 1010229 35 Private Associate Absent Transport 7.21e4 Male 0
## 3 1024587 32 Private HSgrad Divorc… Clerical 1.55e5 Male 0
## 4 1038288 45 Private Bachelor Married Repair 2.77e4 Male 0
## 5 1044221 60 Private College Married Executive 7.57e3 Male 0
## 6 1047095 74 Private HSgrad Married Service 3.31e4 Male 0
## # ℹ 3 more variables: Hours <dbl>, RISK_Adjustment <dbl>, TARGET_Adjusted <dbl>
YOUR ANSWER for Problem 2a. # Problem 2a. Data exploration: statistical and graphical summary
# Load necessary libraries
library(readr)
library(dplyr)
library(tidyr)
# Read the CSV file with explicit column types specification
data <- read_csv("audit.csv", col_types = cols(
ID = col_double(),
Age = col_double(),
Employment = col_character(),
Education = col_character(),
Marital = col_character(),
Occupation = col_character(),
Income = col_double(),
Gender = col_character(),
Deductions = col_double(),
Hours = col_double(),
RISK_Adjustment = col_double(),
TARGET_Adjusted = col_double()
))
# Filter out rows with missing or unexpected values
data <- na.omit(data)
# Generate a summary table for numerical variables
summary_table <- data %>%
summarise(across(where(is.numeric),
list(mean = ~mean(., na.rm = TRUE),
median = ~median(., na.rm = TRUE),
q1 = ~quantile(., 0.25, na.rm = TRUE),
q3 = ~quantile(., 0.75, na.rm = TRUE),
sd = ~sd(., na.rm = TRUE))))
# Print the summary table
print("Summary Table for Numerical Variables:")
## [1] "Summary Table for Numerical Variables:"
print(summary_table)
## # A tibble: 1 × 35
## ID_mean ID_median ID_q1 ID_q3 ID_sd Age_mean Age_median Age_q1 Age_q3
## <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 5597921. 5629974 3396063 7855862. 2589774. 38.3 37 28 47
## # ℹ 26 more variables: Age_sd <dbl>, Income_mean <dbl>, Income_median <dbl>,
## # Income_q1 <dbl>, Income_q3 <dbl>, Income_sd <dbl>, Deductions_mean <dbl>,
## # Deductions_median <dbl>, Deductions_q1 <dbl>, Deductions_q3 <dbl>,
## # Deductions_sd <dbl>, Hours_mean <dbl>, Hours_median <dbl>, Hours_q1 <dbl>,
## # Hours_q3 <dbl>, Hours_sd <dbl>, RISK_Adjustment_mean <dbl>,
## # RISK_Adjustment_median <dbl>, RISK_Adjustment_q1 <dbl>,
## # RISK_Adjustment_q3 <dbl>, RISK_Adjustment_sd <dbl>, …
YOUR ANSWER for Problem 2b. # Problem 2b. Density Distribution and Normality Test for Numerical Variables
# Load necessary libraries
library(ggplot2)
# Plot the density distribution for numerical variables
numerical_vars <- select_if(data, is.numeric)
for (col in names(numerical_vars)) {
plot <- ggplot(data, aes(x = !!sym(col))) +
geom_density(fill = "skyblue", color = "blue") +
ggtitle(paste("Density Distribution of", col)) +
theme_minimal() +
labs(x = col, y = "Density") +
stat_function(fun = dnorm, color = "red", linetype = "dashed", args = list(mean = mean(data[[col]]), sd = sd(data[[col]])))
print(plot)
# Normality test
shapiro_test <- shapiro.test(data[[col]])
print(paste("Shapiro-Wilk Normality Test for", col, "p-value:", shapiro_test$p.value))
# Save the plot to a file
ggsave(paste("density_", col, ".png", sep = ""), plot)
}
## [1] "Shapiro-Wilk Normality Test for ID p-value: 1.31134727255062e-23"
## [1] "Shapiro-Wilk Normality Test for Age p-value: 1.61406142897257e-19"
## [1] "Shapiro-Wilk Normality Test for Income p-value: 2.75617874418494e-39"
## [1] "Shapiro-Wilk Normality Test for Deductions p-value: 3.39672735402962e-67"
## [1] "Shapiro-Wilk Normality Test for Hours p-value: 6.17166471175914e-35"
## [1] "Shapiro-Wilk Normality Test for RISK_Adjustment p-value: 2.18020275088603e-66"
## [1] "Shapiro-Wilk Normality Test for TARGET_Adjusted p-value: 1.24609867101701e-57"
YOUR ANSWER for Problem 2c. # Problem 2c. Conditional Histogram Plot of Response Variable for Categorical Predictors
# Load necessary libraries
library(ggplot2)
# Conditional histogram plot of response variable for categorical predictors
categorical_vars <- select_if(data, is.character)
for (col in names(categorical_vars)) {
plot <- ggplot(data, aes(x = !!sym(col), fill = factor(TARGET_Adjusted))) +
geom_bar(position = "dodge", alpha = 0.7) +
ggtitle(paste("Conditional Histogram of TARGET_Adjusted by", col)) +
theme_minimal() +
labs(x = col, y = "Frequency", fill = "TARGET_Adjusted")
print(plot)
# Save the plot to a file
ggsave(paste("histogram_", col, ".png", sep = ""), plot)
}
YOUR ANSWER for Problem 3A.
# Load necessary libraries
library(caret)
library(pROC)
library(ggplot2)
library(ROCR)
# Convert response variable to factor with consistent levels
data$TARGET_Adjusted <- factor(data$TARGET_Adjusted)
# Create training and testing sets
set.seed(123) # for reproducibility
index <- createDataPartition(data$TARGET_Adjusted, p = 0.7, list = FALSE)
train_data <- data[index, ]
test_data <- data[-index, ]
# Ensure both training and testing datasets have the same levels for the response variable
if (!identical(levels(train_data$TARGET_Adjusted), levels(test_data$TARGET_Adjusted))) {
stop("Levels of response variable 'TARGET_Adjusted' are not consistent between training and testing datasets.")
}
# Train logistic regression model
model <- train(TARGET_Adjusted ~ ., data = train_data, method = "glm", family = "binomial")
# Make predictions on test data
probabilities <- predict(model, newdata = test_data, type = "prob")[, "1"]
# Calculate AUC
roc_obj <- roc(ifelse(test_data$TARGET_Adjusted == 1, 1, 0), probabilities)
auc <- auc(roc_obj)
# Plot ROC curve
plot(roc_obj, main = "ROC Curve", col = "blue")
legend("bottomright", legend = paste("AUC =", round(auc, 2)), col = "blue", lty = 1, cex = 0.8)
# Calculate Lift
perf <- performance(prediction(probabilities, test_data$TARGET_Adjusted), "lift", "rpp")
plot(perf, main = "Lift Curve", col = "red")
YOUR ANSWER for Problem 3B.
# Extract coefficients from the best logistic regression model
coefficients <- coef(model$finalModel)
# Compute odds ratios
odds_ratios <- exp(coefficients)
# Create a data frame to display the results
odds_ratios_df <- data.frame(
Predictor = names(coefficients),
Coefficient = coefficients,
Odds_Ratio = odds_ratios
)
# Print the odds ratios
print(odds_ratios_df)
## Predictor Coefficient
## (Intercept) (Intercept) -7.430790e+00
## ID ID 4.002059e-08
## Age Age 2.711809e-02
## EmploymentPrivate EmploymentPrivate 7.518512e-01
## EmploymentPSFederal EmploymentPSFederal 6.666913e-01
## EmploymentPSLocal EmploymentPSLocal 7.314289e-01
## EmploymentPSState EmploymentPSState 1.255457e-01
## EmploymentSelfEmp EmploymentSelfEmp -2.475495e-01
## EmploymentVolunteer EmploymentVolunteer -1.627244e+01
## EducationBachelor EducationBachelor -4.474297e-01
## EducationCollege EducationCollege -1.710670e+00
## EducationDoctorate EducationDoctorate -8.233002e-01
## EducationHSgrad EducationHSgrad -1.603451e+00
## EducationMaster EducationMaster -1.248325e+00
## EducationPreschool EducationPreschool -1.677007e+01
## EducationProfessional EducationProfessional 2.004511e+00
## EducationVocational EducationVocational -1.573812e+00
## EducationYr10 EducationYr10 -1.725673e+01
## EducationYr11 EducationYr11 -1.865984e-02
## EducationYr12 EducationYr12 -1.669572e+01
## EducationYr1t4 EducationYr1t4 -1.795218e+01
## EducationYr5t6 EducationYr5t6 -2.112761e+00
## EducationYr7t8 EducationYr7t8 -1.726043e+01
## EducationYr9 EducationYr9 -1.789224e+01
## MaritalDivorced MaritalDivorced -6.463136e-01
## MaritalMarried MaritalMarried 2.384201e+00
## `MaritalMarried-spouse-absent` `MaritalMarried-spouse-absent` 1.728296e+00
## MaritalUnmarried MaritalUnmarried 1.344718e+00
## MaritalWidowed MaritalWidowed -2.192567e+00
## OccupationClerical OccupationClerical 9.144261e-01
## OccupationExecutive OccupationExecutive 2.361707e+00
## OccupationFarming OccupationFarming -1.214356e+00
## OccupationHome OccupationHome -1.377886e+01
## OccupationMachinist OccupationMachinist 9.740476e-02
## OccupationMilitary OccupationMilitary -1.303580e+01
## OccupationProfessional OccupationProfessional 1.818852e+00
## OccupationProtective OccupationProtective 1.847228e+00
## OccupationRepair OccupationRepair 7.956120e-01
## OccupationSales OccupationSales 9.554832e-01
## OccupationService OccupationService -2.798745e-01
## OccupationSupport OccupationSupport 1.688044e+00
## OccupationTransport OccupationTransport 8.737200e-01
## Income Income 1.769370e-06
## GenderMale GenderMale -1.830898e-02
## Deductions Deductions 1.041769e-03
## Hours Hours 2.325425e-02
## RISK_Adjustment RISK_Adjustment 4.210229e-03
## Odds_Ratio
## (Intercept) 5.927192e-04
## ID 1.000000e+00
## Age 1.027489e+00
## EmploymentPrivate 2.120923e+00
## EmploymentPSFederal 1.947782e+00
## EmploymentPSLocal 2.078048e+00
## EmploymentPSState 1.133767e+00
## EmploymentSelfEmp 7.807116e-01
## EmploymentVolunteer 8.569735e-08
## EducationBachelor 6.392691e-01
## EducationCollege 1.807446e-01
## EducationDoctorate 4.389805e-01
## EducationHSgrad 2.012009e-01
## EducationMaster 2.869851e-01
## EducationPreschool 5.210180e-08
## EducationProfessional 7.422462e+00
## EducationVocational 2.072537e-01
## EducationYr10 3.202556e-08
## EducationYr11 9.815132e-01
## EducationYr12 5.612313e-08
## EducationYr1t4 1.597599e-08
## EducationYr5t6 1.209036e-01
## EducationYr7t8 3.190722e-08
## EducationYr9 1.696292e-08
## MaritalDivorced 5.239738e-01
## MaritalMarried 1.085039e+01
## `MaritalMarried-spouse-absent` 5.631048e+00
## MaritalUnmarried 3.837103e+00
## MaritalWidowed 1.116299e-01
## OccupationClerical 2.495343e+00
## OccupationExecutive 1.060905e+01
## OccupationFarming 2.969010e-01
## OccupationHome 1.037331e-06
## OccupationMachinist 1.102306e+00
## OccupationMilitary 2.180837e-06
## OccupationProfessional 6.164776e+00
## OccupationProtective 6.342214e+00
## OccupationRepair 2.215797e+00
## OccupationSales 2.599927e+00
## OccupationService 7.558786e-01
## OccupationSupport 5.408890e+00
## OccupationTransport 2.395807e+00
## Income 1.000002e+00
## GenderMale 9.818576e-01
## Deductions 1.001042e+00
## Hours 1.023527e+00
## RISK_Adjustment 1.004219e+00
# Print the summary of the best logistic regression model
summary(model)
##
## Call:
## NULL
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -7.431e+00 1.932e+00 -3.847 0.00012 ***
## ID 4.002e-08 6.077e-08 0.659 0.51017
## Age 2.712e-02 1.416e-02 1.916 0.05540 .
## EmploymentPrivate 7.519e-01 5.846e-01 1.286 0.19842
## EmploymentPSFederal 6.667e-01 8.999e-01 0.741 0.45876
## EmploymentPSLocal 7.314e-01 8.589e-01 0.852 0.39442
## EmploymentPSState 1.255e-01 1.023e+00 0.123 0.90231
## EmploymentSelfEmp -2.475e-01 9.799e-01 -0.253 0.80056
## EmploymentVolunteer -1.627e+01 1.075e+04 -0.002 0.99879
## EducationBachelor -4.474e-01 7.557e-01 -0.592 0.55379
## EducationCollege -1.711e+00 8.056e-01 -2.124 0.03370 *
## EducationDoctorate -8.233e-01 1.448e+00 -0.569 0.56964
## EducationHSgrad -1.603e+00 7.662e-01 -2.093 0.03637 *
## EducationMaster -1.248e+00 1.019e+00 -1.225 0.22049
## EducationPreschool -1.677e+01 4.860e+03 -0.003 0.99725
## EducationProfessional 2.005e+00 1.346e+00 1.489 0.13643
## EducationVocational -1.574e+00 1.068e+00 -1.474 0.14053
## EducationYr10 -1.726e+01 1.508e+03 -0.011 0.99087
## EducationYr11 -1.866e-02 1.114e+00 -0.017 0.98663
## EducationYr12 -1.670e+01 2.871e+03 -0.006 0.99536
## EducationYr1t4 -1.795e+01 4.656e+03 -0.004 0.99692
## EducationYr5t6 -2.113e+00 1.574e+00 -1.342 0.17948
## EducationYr7t8 -1.726e+01 1.872e+03 -0.009 0.99264
## EducationYr9 -1.789e+01 2.307e+03 -0.008 0.99381
## MaritalDivorced -6.463e-01 8.239e-01 -0.784 0.43275
## MaritalMarried 2.384e+00 5.351e-01 4.456 8.36e-06 ***
## `MaritalMarried-spouse-absent` 1.728e+00 1.233e+00 1.402 0.16093
## MaritalUnmarried 1.345e+00 9.243e-01 1.455 0.14573
## MaritalWidowed -2.193e+00 2.981e+00 -0.736 0.46198
## OccupationClerical 9.144e-01 1.290e+00 0.709 0.47838
## OccupationExecutive 2.362e+00 1.135e+00 2.081 0.03742 *
## OccupationFarming -1.214e+00 2.586e+00 -0.470 0.63859
## OccupationHome -1.378e+01 4.705e+03 -0.003 0.99766
## OccupationMachinist 9.740e-02 1.322e+00 0.074 0.94125
## OccupationMilitary -1.304e+01 1.075e+04 -0.001 0.99903
## OccupationProfessional 1.819e+00 1.189e+00 1.529 0.12623
## OccupationProtective 1.847e+00 1.482e+00 1.247 0.21257
## OccupationRepair 7.956e-01 1.160e+00 0.686 0.49296
## OccupationSales 9.555e-01 1.209e+00 0.790 0.42953
## OccupationService -2.799e-01 1.437e+00 -0.195 0.84562
## OccupationSupport 1.688e+00 1.312e+00 1.287 0.19808
## OccupationTransport 8.737e-01 1.255e+00 0.696 0.48643
## Income 1.769e-06 3.146e-06 0.562 0.57382
## GenderMale -1.831e-02 5.596e-01 -0.033 0.97390
## Deductions 1.042e-03 4.695e-04 2.219 0.02651 *
## Hours 2.325e-02 1.357e-02 1.714 0.08655 .
## RISK_Adjustment 4.210e-03 8.350e-04 5.042 4.61e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 1451.42 on 1329 degrees of freedom
## Residual deviance: 315.33 on 1283 degrees of freedom
## AIC: 409.33
##
## Number of Fisher Scoring iterations: 18
YOUR ANSWER for Problem 4A.
# Load necessary libraries
library(caret)
library(MASS)
# Define training control
train_control <- trainControl(method = "cv", number = 10)
# Train linear regression model using all predictors
lm_model <- train(RISK_Adjustment ~ ., data = data, method = "lm", trControl = train_control)
# Evaluate model performance using RMSE through cross-validation
rmse <- sqrt(lm_model$results$RMSE)
# Print RMSE
print(paste("RMSE:", mean(rmse)))
## [1] "RMSE: 84.256430720626"
# Check if residuals contain missing values
if (any(is.na(lm_model$residuals))) {
print("Residuals contain missing values.")
} else if (length(lm_model$residuals) == 0) {
print("Residuals are empty.")
} else {
# Generate Q-Q plot for residuals
qqnorm(lm_model$residuals)
qqline(lm_model$residuals)
}
## [1] "Residuals are empty."
YOUR ANSWER for Problem 4B.
# Load necessary libraries
library(caret)
library(mgcv)
# Define your dataset and response variable
data <- read_csv("audit.csv")
response_variable <- "RISK_Adjustment"
# Define a list of different combinations of predictors
predictors_list <- list(
c("Age", "Employment", "Education"),
c("Marital", "Occupation", "Income"),
c("Gender", "Deductions", "Hours")
# Add more combinations as needed
)
# Initialize vectors to store RMSE values for linear and non-linear models
rmse_linear <- numeric(length(predictors_list))
rmse_nonlinear <- numeric(length(predictors_list))
# Define cross-validation control
train_control <- trainControl(method = "cv", number = 10)
# Remove rows with missing values
data <- na.omit(data)
# Train and evaluate linear regression models
for (i in seq_along(predictors_list)) {
predictors <- predictors_list[[i]]
formula <- as.formula(paste(response_variable, "~", paste(predictors, collapse = "+")))
model_linear <- train(formula, data = data, method = "lm", trControl = train_control)
rmse_linear[i] <- sqrt(model_linear$results$RMSE)
}
# Train and evaluate non-linear regression models (e.g., using GAM)
for (i in seq_along(predictors_list)) {
predictors <- predictors_list[[i]]
formula <- as.formula(paste(response_variable, "~", paste(predictors, collapse = "+")))
model_nonlinear <- train(formula, data = data, method = "gam", trControl = train_control)
rmse_nonlinear[i] <- sqrt(model_nonlinear$results$RMSE)
}
# Compare RMSE values of linear and non-linear models
comparison <- data.frame(
Combination = paste("Combination", 1:length(predictors_list)),
Linear_RMSE = rmse_linear,
Nonlinear_RMSE = rmse_nonlinear,
Better_Model = ifelse(rmse_linear < rmse_nonlinear, "Linear", "Nonlinear")
)
print(comparison)
## Combination Linear_RMSE Nonlinear_RMSE Better_Model
## 1 Combination 1 86.54572 86.80157 Linear
## 2 Combination 2 87.23154 84.90962 Nonlinear
## 3 Combination 3 87.60832 88.35960 Linear
YOUR ANSWER for Problem 4C.
Identify your best model: Based on the comparison of RMSE values, I identify the best model by selecting the model with the lower RMSE. In this case, it appears that the non-linear model (Combination 2) has a lower RMSE compared to the linear models (Combination 1 and Combination 3).
Identify the most important predictor in the model: Hours
Explain how you determine the importance of the predictors: The importance of the predictor “Hours” is determined by considering its coefficient magnitude, statistical significance, effect size (in non-linear models), degree of freedom, ETC.
# Summary of the non-linear model to identify important predictors
summary(model_nonlinear)
##
## Family: gaussian
## Link function: identity
##
## Formula:
## .outcome ~ GenderMale + s(Hours)
##
## Parametric coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1469.2 342.9 4.284 1.93e-05 ***
## GenderMale 807.9 417.0 1.937 0.0529 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Approximate significance of smooth terms:
## edf Ref.df F p-value
## s(Hours) 1.538 9 1.567 7.6e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## R-sq.(adj) = 0.0118 Deviance explained = 1.31%
## GCV = 6.7014e+07 Scale est. = 6.689e+07 n = 1899