Overview

In this assignment, you’ll explore both numeric and categorical variables and do regression analysis.

You will use the dataset D8 described on DMR (online access via Pitt network) Appendix A (data8) for this assignment. The data is audit.csv and already included in the repository. The objective is to predict the binary (TARGET_Adjusted) and continuous (RISK_Adjustment) target variables. Create RMarkdown report using this template.
hint: treat the given binary values “1” as “positive” and “0” as “negative”
hint: be careful about missing values in data, try to remove them if there are only a few instances. You will loose 2 points if you directly using missing values without processing them.

The audit.csv data set is an artificially constructed data set that contains the characteristics of n = 2000 individual tax returns. The data set includes the following variables:

  • ID: Unique identifier for each person.
  • Age: Age of person.
  • Employment: Type of employment.
  • Education: Highest level of education.
  • Marital: Current marital status.
  • Occupation: Type of occupation.
  • Income: Amount of income declared.
  • Gender: Gender of person.
  • Deductions: Total amount of expenses that a person claims in their financial statement.
  • Hours: Average hours worked on a weekly basis.
  • RISK_Adjustment: The continuous target variable; this variable records the monetary amount of any adjustment to the person’s financial claims as a result of a productive audit. This variable is a measure of the size of the risk associated with the person.
  • TARGET_Adjusted: The binary target variable for classification modeling (0/1), indicating nonproductive and productive audits, respectively. Productive audits are those that result in an adjustment being made to a client’s financial statement.
  1. Read the data description above, and identify and report response variable and predictors.

  2. Explore the dataset, and generate both statistical and graphical summary with respect to the numerical and categorical variables.

    1. Generate a summary table for the data. For each numerical variable, list: variable name, mean, median, 1st quartile, 3rd quartile, and standard deviation.
    2. For numerical variables, plot the density distribution. Describe whether the variable has a normal distribution or certain type of skew distribution. hint: you can use Shapiro-Wilk normality test to test for normal distribution
    3. For each categorical predictor, generate the conditional histogram plot of response variable.
  3. Apply logistic regression analysis to predict TARGET_Adjusted. Evaluate the models through cross-validation and on holdout samples. Interpret the effect of the predictors.

    1. Implement a 10-fold cross-validation scheme by splitting the data into training and testing sets. Use the training set to train a logistic regression model to predict the response variable. Examine the performance of different models by varing the number of predictors. Report the performance of the models on testing set using proper measures (accuracy, precision, recall, F1, AUC) and plots (ROC, lift).
    2. For the best model, compute the odds ratio and interpret the effect of each predictors.
  4. Apply linear and non-linear regression analysis to predict RISK_Adjustment. Evaluate the models through 10-fold cross-validation.

    1. Use all predictors in a standard linear regression model to predict the response variable. Report the model performance using RMSE through cross-validation. Use Q-Q plot to check the diagnostics and examine whether the normality is violated. hint: You can log transform the response variable if the normality is violated.
    2. Use different combination of predictors in standard linear and non-linear regression models to predict the response variable. (Here we don’t consider interaction terms.) Evaluate which model performs better using RMSE through cross-validation.
    3. From the previous answers,identify your best model, and identify the most important predictor in the model. Explain how you determine the importance of the predictors.

Problem 1: Identify and report response variable and predictors

YOUR ANSWER for Problem 1.

# Load necessary library
library(readr)

# Read the data description from the given source
data_description <- "
The audit.csv data set is an artificially constructed data set that contains the characteristics of n = 2000 individual tax returns. The data set includes the following variables:

- ID: Unique identifier for each person.
- Age: Age of person.
- Employment: Type of employment.
- Education: Highest level of education.
- Marital: Current marital status.
- Occupation: Type of occupation.
- Income: Amount of income declared.
- Gender: Gender of person.
- Deductions: Total amount of expenses that a person claims in their financial statement.
- Hours: Average hours worked on a weekly basis.
- RISK_Adjustment: The continuous target variable; this variable records the monetary amount of any adjustment to the person’s financial claims as a result of a productive audit. This variable is a measure of the size of the risk associated with the person.
- TARGET_Adjusted: The binary target variable for classification modeling (0/1), indicating nonproductive and productive audits, respectively. Productive audits are those that result in an adjustment being made to a client’s financial statement.
"

# Print the data description
cat(data_description)
## 
## The audit.csv data set is an artificially constructed data set that contains the characteristics of n = 2000 individual tax returns. The data set includes the following variables:
## 
## - ID: Unique identifier for each person.
## - Age: Age of person.
## - Employment: Type of employment.
## - Education: Highest level of education.
## - Marital: Current marital status.
## - Occupation: Type of occupation.
## - Income: Amount of income declared.
## - Gender: Gender of person.
## - Deductions: Total amount of expenses that a person claims in their financial statement.
## - Hours: Average hours worked on a weekly basis.
## - RISK_Adjustment: The continuous target variable; this variable records the monetary amount of any adjustment to the person’s financial claims as a result of a productive audit. This variable is a measure of the size of the risk associated with the person.
## - TARGET_Adjusted: The binary target variable for classification modeling (0/1), indicating nonproductive and productive audits, respectively. Productive audits are those that result in an adjustment being made to a client’s financial statement.
# Identify and report response variable and predictors
response_variable <- "TARGET_Adjusted"
predictors <- c("ID", "Age", "Employment", "Education", "Marital", "Occupation", "Income", "Gender", "Deductions", "Hours", "RISK_Adjustment")


# Print response variable and predictors
cat("\nResponse Variable:", response_variable, "\n")
## 
## Response Variable: TARGET_Adjusted
cat("Predictors:", paste(predictors, collapse = ", "), "\n")
## Predictors: ID, Age, Employment, Education, Marital, Occupation, Income, Gender, Deductions, Hours, RISK_Adjustment
# Read the CSV file
data <- read_csv("audit.csv")

# Print the first few rows of the data
head(data)
## # A tibble: 6 × 12
##        ID   Age Employment Education Marital Occupation Income Gender Deductions
##     <dbl> <dbl> <chr>      <chr>     <chr>   <chr>       <dbl> <chr>       <dbl>
## 1 1004641    38 Private    College   Unmarr… Service    8.18e4 Female          0
## 2 1010229    35 Private    Associate Absent  Transport  7.21e4 Male            0
## 3 1024587    32 Private    HSgrad    Divorc… Clerical   1.55e5 Male            0
## 4 1038288    45 Private    Bachelor  Married Repair     2.77e4 Male            0
## 5 1044221    60 Private    College   Married Executive  7.57e3 Male            0
## 6 1047095    74 Private    HSgrad    Married Service    3.31e4 Male            0
## # ℹ 3 more variables: Hours <dbl>, RISK_Adjustment <dbl>, TARGET_Adjusted <dbl>

YOUR ANSWER for Problem 2a. # Problem 2a. Data exploration: statistical and graphical summary

# Load necessary libraries
library(readr)
library(dplyr)
library(tidyr)

# Read the CSV file with explicit column types specification
data <- read_csv("audit.csv", col_types = cols(
  ID = col_double(),
  Age = col_double(),
  Employment = col_character(),
  Education = col_character(),
  Marital = col_character(),
  Occupation = col_character(),
  Income = col_double(),
  Gender = col_character(),
  Deductions = col_double(),
  Hours = col_double(),
  RISK_Adjustment = col_double(),
  TARGET_Adjusted = col_double()
))

# Filter out rows with missing or unexpected values
data <- na.omit(data)

# Generate a summary table for numerical variables
summary_table <- data %>%
  summarise(across(where(is.numeric), 
                   list(mean = ~mean(., na.rm = TRUE),
                        median = ~median(., na.rm = TRUE),
                        q1 = ~quantile(., 0.25, na.rm = TRUE),
                        q3 = ~quantile(., 0.75, na.rm = TRUE),
                        sd = ~sd(., na.rm = TRUE))))

# Print the summary table
print("Summary Table for Numerical Variables:")
## [1] "Summary Table for Numerical Variables:"
print(summary_table)
## # A tibble: 1 × 35
##    ID_mean ID_median   ID_q1    ID_q3    ID_sd Age_mean Age_median Age_q1 Age_q3
##      <dbl>     <dbl>   <dbl>    <dbl>    <dbl>    <dbl>      <dbl>  <dbl>  <dbl>
## 1 5597921.   5629974 3396063 7855862. 2589774.     38.3         37     28     47
## # ℹ 26 more variables: Age_sd <dbl>, Income_mean <dbl>, Income_median <dbl>,
## #   Income_q1 <dbl>, Income_q3 <dbl>, Income_sd <dbl>, Deductions_mean <dbl>,
## #   Deductions_median <dbl>, Deductions_q1 <dbl>, Deductions_q3 <dbl>,
## #   Deductions_sd <dbl>, Hours_mean <dbl>, Hours_median <dbl>, Hours_q1 <dbl>,
## #   Hours_q3 <dbl>, Hours_sd <dbl>, RISK_Adjustment_mean <dbl>,
## #   RISK_Adjustment_median <dbl>, RISK_Adjustment_q1 <dbl>,
## #   RISK_Adjustment_q3 <dbl>, RISK_Adjustment_sd <dbl>, …

YOUR ANSWER for Problem 2b. # Problem 2b. Density Distribution and Normality Test for Numerical Variables

# Load necessary libraries
library(ggplot2)

# Plot the density distribution for numerical variables
numerical_vars <- select_if(data, is.numeric)

for (col in names(numerical_vars)) {
  plot <- ggplot(data, aes(x = !!sym(col))) +
    geom_density(fill = "skyblue", color = "blue") +
    ggtitle(paste("Density Distribution of", col)) +
    theme_minimal() +
    labs(x = col, y = "Density") +
    stat_function(fun = dnorm, color = "red", linetype = "dashed", args = list(mean = mean(data[[col]]), sd = sd(data[[col]])))
  
  print(plot)
  
  # Normality test
  shapiro_test <- shapiro.test(data[[col]])
  print(paste("Shapiro-Wilk Normality Test for", col, "p-value:", shapiro_test$p.value))
  
  # Save the plot to a file
  ggsave(paste("density_", col, ".png", sep = ""), plot)
}
## [1] "Shapiro-Wilk Normality Test for ID p-value: 1.31134727255062e-23"

## [1] "Shapiro-Wilk Normality Test for Age p-value: 1.61406142897257e-19"

## [1] "Shapiro-Wilk Normality Test for Income p-value: 2.75617874418494e-39"

## [1] "Shapiro-Wilk Normality Test for Deductions p-value: 3.39672735402962e-67"

## [1] "Shapiro-Wilk Normality Test for Hours p-value: 6.17166471175914e-35"

## [1] "Shapiro-Wilk Normality Test for RISK_Adjustment p-value: 2.18020275088603e-66"

## [1] "Shapiro-Wilk Normality Test for TARGET_Adjusted p-value: 1.24609867101701e-57"

YOUR ANSWER for Problem 2c. # Problem 2c. Conditional Histogram Plot of Response Variable for Categorical Predictors

# Load necessary libraries
library(ggplot2)

# Conditional histogram plot of response variable for categorical predictors
categorical_vars <- select_if(data, is.character)

for (col in names(categorical_vars)) {
  plot <- ggplot(data, aes(x = !!sym(col), fill = factor(TARGET_Adjusted))) +
    geom_bar(position = "dodge", alpha = 0.7) +
    ggtitle(paste("Conditional Histogram of TARGET_Adjusted by", col)) +
    theme_minimal() +
    labs(x = col, y = "Frequency", fill = "TARGET_Adjusted")
  
  print(plot)
  
  # Save the plot to a file
  ggsave(paste("histogram_", col, ".png", sep = ""), plot)
}

YOUR ANSWER for Problem 3A.

Problem 3A. 10-fold Cross-validation and Model Performance Evaluation

# Load necessary libraries
library(caret)
library(pROC)
library(ggplot2)
library(ROCR)

# Convert response variable to factor with consistent levels
data$TARGET_Adjusted <- factor(data$TARGET_Adjusted)

# Create training and testing sets
set.seed(123) # for reproducibility
index <- createDataPartition(data$TARGET_Adjusted, p = 0.7, list = FALSE)
train_data <- data[index, ]
test_data <- data[-index, ]

# Ensure both training and testing datasets have the same levels for the response variable
if (!identical(levels(train_data$TARGET_Adjusted), levels(test_data$TARGET_Adjusted))) {
  stop("Levels of response variable 'TARGET_Adjusted' are not consistent between training and testing datasets.")
}

# Train logistic regression model
model <- train(TARGET_Adjusted ~ ., data = train_data, method = "glm", family = "binomial")

# Make predictions on test data
probabilities <- predict(model, newdata = test_data, type = "prob")[, "1"]

# Calculate AUC
roc_obj <- roc(ifelse(test_data$TARGET_Adjusted == 1, 1, 0), probabilities)
auc <- auc(roc_obj)

# Plot ROC curve
plot(roc_obj, main = "ROC Curve", col = "blue")
legend("bottomright", legend = paste("AUC =", round(auc, 2)), col = "blue", lty = 1, cex = 0.8)

# Calculate Lift
perf <- performance(prediction(probabilities, test_data$TARGET_Adjusted), "lift", "rpp")
plot(perf, main = "Lift Curve", col = "red")

YOUR ANSWER for Problem 3B.

Problem 3B. Odds Ratio and Interpretation of Predictors for Best Model

# Extract coefficients from the best logistic regression model
coefficients <- coef(model$finalModel)

# Compute odds ratios
odds_ratios <- exp(coefficients)

# Create a data frame to display the results
odds_ratios_df <- data.frame(
  Predictor = names(coefficients),
  Coefficient = coefficients,
  Odds_Ratio = odds_ratios
)

# Print the odds ratios
print(odds_ratios_df)
##                                                     Predictor   Coefficient
## (Intercept)                                       (Intercept) -7.430790e+00
## ID                                                         ID  4.002059e-08
## Age                                                       Age  2.711809e-02
## EmploymentPrivate                           EmploymentPrivate  7.518512e-01
## EmploymentPSFederal                       EmploymentPSFederal  6.666913e-01
## EmploymentPSLocal                           EmploymentPSLocal  7.314289e-01
## EmploymentPSState                           EmploymentPSState  1.255457e-01
## EmploymentSelfEmp                           EmploymentSelfEmp -2.475495e-01
## EmploymentVolunteer                       EmploymentVolunteer -1.627244e+01
## EducationBachelor                           EducationBachelor -4.474297e-01
## EducationCollege                             EducationCollege -1.710670e+00
## EducationDoctorate                         EducationDoctorate -8.233002e-01
## EducationHSgrad                               EducationHSgrad -1.603451e+00
## EducationMaster                               EducationMaster -1.248325e+00
## EducationPreschool                         EducationPreschool -1.677007e+01
## EducationProfessional                   EducationProfessional  2.004511e+00
## EducationVocational                       EducationVocational -1.573812e+00
## EducationYr10                                   EducationYr10 -1.725673e+01
## EducationYr11                                   EducationYr11 -1.865984e-02
## EducationYr12                                   EducationYr12 -1.669572e+01
## EducationYr1t4                                 EducationYr1t4 -1.795218e+01
## EducationYr5t6                                 EducationYr5t6 -2.112761e+00
## EducationYr7t8                                 EducationYr7t8 -1.726043e+01
## EducationYr9                                     EducationYr9 -1.789224e+01
## MaritalDivorced                               MaritalDivorced -6.463136e-01
## MaritalMarried                                 MaritalMarried  2.384201e+00
## `MaritalMarried-spouse-absent` `MaritalMarried-spouse-absent`  1.728296e+00
## MaritalUnmarried                             MaritalUnmarried  1.344718e+00
## MaritalWidowed                                 MaritalWidowed -2.192567e+00
## OccupationClerical                         OccupationClerical  9.144261e-01
## OccupationExecutive                       OccupationExecutive  2.361707e+00
## OccupationFarming                           OccupationFarming -1.214356e+00
## OccupationHome                                 OccupationHome -1.377886e+01
## OccupationMachinist                       OccupationMachinist  9.740476e-02
## OccupationMilitary                         OccupationMilitary -1.303580e+01
## OccupationProfessional                 OccupationProfessional  1.818852e+00
## OccupationProtective                     OccupationProtective  1.847228e+00
## OccupationRepair                             OccupationRepair  7.956120e-01
## OccupationSales                               OccupationSales  9.554832e-01
## OccupationService                           OccupationService -2.798745e-01
## OccupationSupport                           OccupationSupport  1.688044e+00
## OccupationTransport                       OccupationTransport  8.737200e-01
## Income                                                 Income  1.769370e-06
## GenderMale                                         GenderMale -1.830898e-02
## Deductions                                         Deductions  1.041769e-03
## Hours                                                   Hours  2.325425e-02
## RISK_Adjustment                               RISK_Adjustment  4.210229e-03
##                                  Odds_Ratio
## (Intercept)                    5.927192e-04
## ID                             1.000000e+00
## Age                            1.027489e+00
## EmploymentPrivate              2.120923e+00
## EmploymentPSFederal            1.947782e+00
## EmploymentPSLocal              2.078048e+00
## EmploymentPSState              1.133767e+00
## EmploymentSelfEmp              7.807116e-01
## EmploymentVolunteer            8.569735e-08
## EducationBachelor              6.392691e-01
## EducationCollege               1.807446e-01
## EducationDoctorate             4.389805e-01
## EducationHSgrad                2.012009e-01
## EducationMaster                2.869851e-01
## EducationPreschool             5.210180e-08
## EducationProfessional          7.422462e+00
## EducationVocational            2.072537e-01
## EducationYr10                  3.202556e-08
## EducationYr11                  9.815132e-01
## EducationYr12                  5.612313e-08
## EducationYr1t4                 1.597599e-08
## EducationYr5t6                 1.209036e-01
## EducationYr7t8                 3.190722e-08
## EducationYr9                   1.696292e-08
## MaritalDivorced                5.239738e-01
## MaritalMarried                 1.085039e+01
## `MaritalMarried-spouse-absent` 5.631048e+00
## MaritalUnmarried               3.837103e+00
## MaritalWidowed                 1.116299e-01
## OccupationClerical             2.495343e+00
## OccupationExecutive            1.060905e+01
## OccupationFarming              2.969010e-01
## OccupationHome                 1.037331e-06
## OccupationMachinist            1.102306e+00
## OccupationMilitary             2.180837e-06
## OccupationProfessional         6.164776e+00
## OccupationProtective           6.342214e+00
## OccupationRepair               2.215797e+00
## OccupationSales                2.599927e+00
## OccupationService              7.558786e-01
## OccupationSupport              5.408890e+00
## OccupationTransport            2.395807e+00
## Income                         1.000002e+00
## GenderMale                     9.818576e-01
## Deductions                     1.001042e+00
## Hours                          1.023527e+00
## RISK_Adjustment                1.004219e+00
# Print the summary of the best logistic regression model
summary(model)
## 
## Call:
## NULL
## 
## Coefficients:
##                                  Estimate Std. Error z value Pr(>|z|)    
## (Intercept)                    -7.431e+00  1.932e+00  -3.847  0.00012 ***
## ID                              4.002e-08  6.077e-08   0.659  0.51017    
## Age                             2.712e-02  1.416e-02   1.916  0.05540 .  
## EmploymentPrivate               7.519e-01  5.846e-01   1.286  0.19842    
## EmploymentPSFederal             6.667e-01  8.999e-01   0.741  0.45876    
## EmploymentPSLocal               7.314e-01  8.589e-01   0.852  0.39442    
## EmploymentPSState               1.255e-01  1.023e+00   0.123  0.90231    
## EmploymentSelfEmp              -2.475e-01  9.799e-01  -0.253  0.80056    
## EmploymentVolunteer            -1.627e+01  1.075e+04  -0.002  0.99879    
## EducationBachelor              -4.474e-01  7.557e-01  -0.592  0.55379    
## EducationCollege               -1.711e+00  8.056e-01  -2.124  0.03370 *  
## EducationDoctorate             -8.233e-01  1.448e+00  -0.569  0.56964    
## EducationHSgrad                -1.603e+00  7.662e-01  -2.093  0.03637 *  
## EducationMaster                -1.248e+00  1.019e+00  -1.225  0.22049    
## EducationPreschool             -1.677e+01  4.860e+03  -0.003  0.99725    
## EducationProfessional           2.005e+00  1.346e+00   1.489  0.13643    
## EducationVocational            -1.574e+00  1.068e+00  -1.474  0.14053    
## EducationYr10                  -1.726e+01  1.508e+03  -0.011  0.99087    
## EducationYr11                  -1.866e-02  1.114e+00  -0.017  0.98663    
## EducationYr12                  -1.670e+01  2.871e+03  -0.006  0.99536    
## EducationYr1t4                 -1.795e+01  4.656e+03  -0.004  0.99692    
## EducationYr5t6                 -2.113e+00  1.574e+00  -1.342  0.17948    
## EducationYr7t8                 -1.726e+01  1.872e+03  -0.009  0.99264    
## EducationYr9                   -1.789e+01  2.307e+03  -0.008  0.99381    
## MaritalDivorced                -6.463e-01  8.239e-01  -0.784  0.43275    
## MaritalMarried                  2.384e+00  5.351e-01   4.456 8.36e-06 ***
## `MaritalMarried-spouse-absent`  1.728e+00  1.233e+00   1.402  0.16093    
## MaritalUnmarried                1.345e+00  9.243e-01   1.455  0.14573    
## MaritalWidowed                 -2.193e+00  2.981e+00  -0.736  0.46198    
## OccupationClerical              9.144e-01  1.290e+00   0.709  0.47838    
## OccupationExecutive             2.362e+00  1.135e+00   2.081  0.03742 *  
## OccupationFarming              -1.214e+00  2.586e+00  -0.470  0.63859    
## OccupationHome                 -1.378e+01  4.705e+03  -0.003  0.99766    
## OccupationMachinist             9.740e-02  1.322e+00   0.074  0.94125    
## OccupationMilitary             -1.304e+01  1.075e+04  -0.001  0.99903    
## OccupationProfessional          1.819e+00  1.189e+00   1.529  0.12623    
## OccupationProtective            1.847e+00  1.482e+00   1.247  0.21257    
## OccupationRepair                7.956e-01  1.160e+00   0.686  0.49296    
## OccupationSales                 9.555e-01  1.209e+00   0.790  0.42953    
## OccupationService              -2.799e-01  1.437e+00  -0.195  0.84562    
## OccupationSupport               1.688e+00  1.312e+00   1.287  0.19808    
## OccupationTransport             8.737e-01  1.255e+00   0.696  0.48643    
## Income                          1.769e-06  3.146e-06   0.562  0.57382    
## GenderMale                     -1.831e-02  5.596e-01  -0.033  0.97390    
## Deductions                      1.042e-03  4.695e-04   2.219  0.02651 *  
## Hours                           2.325e-02  1.357e-02   1.714  0.08655 .  
## RISK_Adjustment                 4.210e-03  8.350e-04   5.042 4.61e-07 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 1451.42  on 1329  degrees of freedom
## Residual deviance:  315.33  on 1283  degrees of freedom
## AIC: 409.33
## 
## Number of Fisher Scoring iterations: 18

YOUR ANSWER for Problem 4A.

Problem 4A. Linear Regression Analysis with All Predictors

# Load necessary libraries
library(caret)
library(MASS)

# Define training control
train_control <- trainControl(method = "cv", number = 10)

# Train linear regression model using all predictors
lm_model <- train(RISK_Adjustment ~ ., data = data, method = "lm", trControl = train_control)

# Evaluate model performance using RMSE through cross-validation
rmse <- sqrt(lm_model$results$RMSE)

# Print RMSE
print(paste("RMSE:", mean(rmse)))
## [1] "RMSE: 84.256430720626"
# Check if residuals contain missing values
if (any(is.na(lm_model$residuals))) {
  print("Residuals contain missing values.")
} else if (length(lm_model$residuals) == 0) {
  print("Residuals are empty.")
} else {
  # Generate Q-Q plot for residuals
  qqnorm(lm_model$residuals)
  qqline(lm_model$residuals)
}
## [1] "Residuals are empty."

YOUR ANSWER for Problem 4B.

Problem 4B. Different Combination of Predictors in Linear and Non-linear Regression Models

# Load necessary libraries
library(caret)
library(mgcv)

# Define your dataset and response variable

data <- read_csv("audit.csv")
response_variable <- "RISK_Adjustment"

# Define a list of different combinations of predictors
predictors_list <- list(
  c("Age", "Employment", "Education"),
  c("Marital", "Occupation", "Income"),
  c("Gender", "Deductions", "Hours")
  # Add more combinations as needed
)

# Initialize vectors to store RMSE values for linear and non-linear models
rmse_linear <- numeric(length(predictors_list))
rmse_nonlinear <- numeric(length(predictors_list))

# Define cross-validation control
train_control <- trainControl(method = "cv", number = 10)

# Remove rows with missing values
data <- na.omit(data)

# Train and evaluate linear regression models
for (i in seq_along(predictors_list)) {
  predictors <- predictors_list[[i]]
  formula <- as.formula(paste(response_variable, "~", paste(predictors, collapse = "+")))
  model_linear <- train(formula, data = data, method = "lm", trControl = train_control)
  rmse_linear[i] <- sqrt(model_linear$results$RMSE)
}


# Train and evaluate non-linear regression models (e.g., using GAM)
for (i in seq_along(predictors_list)) {
  predictors <- predictors_list[[i]]
  formula <- as.formula(paste(response_variable, "~", paste(predictors, collapse = "+")))
  model_nonlinear <- train(formula, data = data, method = "gam", trControl = train_control)
  rmse_nonlinear[i] <- sqrt(model_nonlinear$results$RMSE)
}

# Compare RMSE values of linear and non-linear models
comparison <- data.frame(
  Combination = paste("Combination", 1:length(predictors_list)),
  Linear_RMSE = rmse_linear,
  Nonlinear_RMSE = rmse_nonlinear,
  Better_Model = ifelse(rmse_linear < rmse_nonlinear, "Linear", "Nonlinear")
)

print(comparison)
##     Combination Linear_RMSE Nonlinear_RMSE Better_Model
## 1 Combination 1    86.54572       86.80157       Linear
## 2 Combination 2    87.23154       84.90962    Nonlinear
## 3 Combination 3    87.60832       88.35960       Linear

YOUR ANSWER for Problem 4C.

Problem 4C. Identifying the Most Important Predictor:

Identify your best model: Based on the comparison of RMSE values, I identify the best model by selecting the model with the lower RMSE. In this case, it appears that the non-linear model (Combination 2) has a lower RMSE compared to the linear models (Combination 1 and Combination 3).

Identify the most important predictor in the model: Hours

Explain how you determine the importance of the predictors: The importance of the predictor “Hours” is determined by considering its coefficient magnitude, statistical significance, effect size (in non-linear models), degree of freedom, ETC.

# Summary of the non-linear model to identify important predictors
summary(model_nonlinear)
## 
## Family: gaussian 
## Link function: identity 
## 
## Formula:
## .outcome ~ GenderMale + s(Hours)
## 
## Parametric coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   1469.2      342.9   4.284 1.93e-05 ***
## GenderMale     807.9      417.0   1.937   0.0529 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Approximate significance of smooth terms:
##            edf Ref.df     F p-value    
## s(Hours) 1.538      9 1.567 7.6e-05 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## R-sq.(adj) =  0.0118   Deviance explained = 1.31%
## GCV = 6.7014e+07  Scale est. = 6.689e+07  n = 1899