1. Exploring the Swiss Dataset

I used the swiss dataset in R, which contains 47 observations and 6 variables. Fertility is the dependent variable, and the other five variables can be used to predict it. First, I checked the structure and summary of the dataset to better understand the variables.

# Load the swiss dataset
data(swiss)

# View the first six rows
head(swiss)
##              Fertility Agriculture Examination Education Catholic
## Courtelary        80.2        17.0          15        12     9.96
## Delemont          83.1        45.1           6         9    84.84
## Franches-Mnt      92.5        39.7           5         5    93.40
## Moutier           85.8        36.5          12         7    33.77
## Neuveville        76.9        43.5          17        15     5.16
## Porrentruy        76.1        35.3           9         7    90.57
##              Infant.Mortality
## Courtelary               22.2
## Delemont                 22.2
## Franches-Mnt             20.2
## Moutier                  20.3
## Neuveville               20.6
## Porrentruy               26.6
# Check the structure of the dataset
str(swiss)
## 'data.frame':    47 obs. of  6 variables:
##  $ Fertility       : num  80.2 83.1 92.5 85.8 76.9 76.1 83.8 92.4 82.4 82.9 ...
##  $ Agriculture     : num  17 45.1 39.7 36.5 43.5 35.3 70.2 67.8 53.3 45.2 ...
##  $ Examination     : int  15 6 5 12 17 9 16 14 12 16 ...
##  $ Education       : int  12 9 5 7 15 7 7 8 7 13 ...
##  $ Catholic        : num  9.96 84.84 93.4 33.77 5.16 ...
##  $ Infant.Mortality: num  22.2 22.2 20.2 20.3 20.6 26.6 23.6 24.9 21 24.4 ...
# Summary statistics
summary(swiss)
##    Fertility      Agriculture     Examination      Education    
##  Min.   :35.00   Min.   : 1.20   Min.   : 3.00   Min.   : 1.00  
##  1st Qu.:64.70   1st Qu.:35.90   1st Qu.:12.00   1st Qu.: 6.00  
##  Median :70.40   Median :54.10   Median :16.00   Median : 8.00  
##  Mean   :70.14   Mean   :50.66   Mean   :16.49   Mean   :10.98  
##  3rd Qu.:78.45   3rd Qu.:67.65   3rd Qu.:22.00   3rd Qu.:12.00  
##  Max.   :92.50   Max.   :89.70   Max.   :37.00   Max.   :53.00  
##     Catholic       Infant.Mortality
##  Min.   :  2.150   Min.   :10.80   
##  1st Qu.:  5.195   1st Qu.:18.15   
##  Median : 15.140   Median :20.00   
##  Mean   : 41.144   Mean   :19.94   
##  3rd Qu.: 93.125   3rd Qu.:21.70   
##  Max.   :100.000   Max.   :26.60

2. Multiple Linear Regression

I first built a multiple linear regression model using all five independent variables to predict Fertility. Then, I checked the results to see which variables were significant.

# Build the full multiple regression model
model_full <- lm(Fertility ~ ., data = swiss)

# View the regression results
summary(model_full)
## 
## Call:
## lm(formula = Fertility ~ ., data = swiss)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -15.2743  -5.2617   0.5032   4.1198  15.3213 
## 
## Coefficients:
##                  Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      66.91518   10.70604   6.250 1.91e-07 ***
## Agriculture      -0.17211    0.07030  -2.448  0.01873 *  
## Examination      -0.25801    0.25388  -1.016  0.31546    
## Education        -0.87094    0.18303  -4.758 2.43e-05 ***
## Catholic          0.10412    0.03526   2.953  0.00519 ** 
## Infant.Mortality  1.07705    0.38172   2.822  0.00734 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 7.165 on 41 degrees of freedom
## Multiple R-squared:  0.7067, Adjusted R-squared:  0.671 
## F-statistic: 19.76 on 5 and 41 DF,  p-value: 5.594e-10

2.1 Model Selection

The full model has an R-squared of 0.7067 and an adjusted R-squared of 0.6710. Most variables are significant, but Examination has a p-value of 0.3155, which is greater than 0.05. I decided to remove Examination and check if the new model performs better.

# Remove Examination from the model
model_reduced <- lm(Fertility ~ Agriculture +
                    Education + Catholic +
                    Infant.Mortality, data = swiss)

# View the reduced model results
summary(model_reduced)
## 
## Call:
## lm(formula = Fertility ~ Agriculture + Education + Catholic + 
##     Infant.Mortality, data = swiss)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -14.6765  -6.0522   0.7514   3.1664  16.1422 
## 
## Coefficients:
##                  Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      62.10131    9.60489   6.466 8.49e-08 ***
## Agriculture      -0.15462    0.06819  -2.267  0.02857 *  
## Education        -0.98026    0.14814  -6.617 5.14e-08 ***
## Catholic          0.12467    0.02889   4.315 9.50e-05 ***
## Infant.Mortality  1.07844    0.38187   2.824  0.00722 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 7.168 on 42 degrees of freedom
## Multiple R-squared:  0.6993, Adjusted R-squared:  0.6707 
## F-statistic: 24.42 on 4 and 42 DF,  p-value: 1.717e-10

2.2 Model Comparison

The reduced model has an adjusted R-squared of 0.6707, which is very close to the full model (0.6710). All four variables in the reduced model are significant. I chose the reduced model because it is simpler and still explains about 70% of the variation in Fertility. In the reduced model, Agriculture and Education have negative coefficients, while Catholic and Infant.Mortality have positive coefficients. This means higher Agriculture and Education values are associated with lower Fertility, while higher Catholic and Infant.Mortality values are associated with higher Fertility, when the other variables are held constant.

3. Checking Model Assumptions

I used diagnostic plots to check the assumptions of the reduced regression model, including linearity, normality of residuals, and constant variance.

# Check regression assumptions
par(mfrow = c(2, 2))

plot(model_reduced)

par(mfrow = c(1, 1))

3.1 Interpretation

The residuals are mostly scattered around zero without a clear pattern. The Q-Q plot shows that most points are close to the line, although there are some small differences at the ends. The Scale-Location plot shows a slight increase in the spread of residuals. The leverage plot does not show any points clearly outside Cook’s distance lines. Overall, the model assumptions seem reasonable, but there may be some small issues with constant variance.

4. Logistic Regression

For the logistic regression model, I created a new binary variable based on Fertility. A value of 1 means Fertility is greater than 70, and a value of 0 means Fertility is 70 or below.

# Create a binary variable for Fertility
swiss$Fertility_High <- ifelse(swiss$Fertility > 70, 1, 0)

# Check the number of observations in each group
table(swiss$Fertility_High)
## 
##  0  1 
## 23 24

4.1 Building the Logistic Regression Model

There are 23 observations with Fertility at or below 70 and 24 observations above 70. I first used all five independent variables to build a logistic regression model and predict whether Fertility is greater than 70.

# Build the full logistic regression model
log_model_full <- glm(
  Fertility_High ~ Agriculture + Examination +
    Education + Catholic + Infant.Mortality,
  data = swiss,
  family = binomial
)

# View the model results
summary(log_model_full)
## 
## Call:
## glm(formula = Fertility_High ~ Agriculture + Examination + Education + 
##     Catholic + Infant.Mortality, family = binomial, data = swiss)
## 
## Coefficients:
##                  Estimate Std. Error z value Pr(>|z|)  
## (Intercept)       4.82826    5.25607   0.919   0.3583  
## Agriculture      -0.09615    0.04011  -2.397   0.0165 *
## Examination      -0.32116    0.13844  -2.320   0.0203 *
## Education        -0.12078    0.08610  -1.403   0.1607  
## Catholic          0.02078    0.01376   1.509   0.1312  
## Infant.Mortality  0.29078    0.21051   1.381   0.1672  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 65.135  on 46  degrees of freedom
## Residual deviance: 32.887  on 41  degrees of freedom
## AIC: 44.887
## 
## Number of Fisher Scoring iterations: 6

4.2 Model Selection

In the full logistic regression model, Agriculture and Examination have p-values below 0.05. The other three variables are not statistically significant. I decided to try a simpler model using Agriculture and Examination, and compare it with the full model. Agriculture and Examination both have negative coefficients in the full model. This means higher values of these variables are associated with a lower probability of Fertility being greater than 70, when the other variables are held constant.

# Build a simpler logistic regression model
log_model_reduced <- glm(
  Fertility_High ~ Agriculture + Examination,
  data = swiss,
  family = binomial
)

# View the reduced model results
summary(log_model_reduced)
## 
## Call:
## glm(formula = Fertility_High ~ Agriculture + Examination, family = binomial, 
##     data = swiss)
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)    
## (Intercept)  9.69552    3.30710   2.932 0.003371 ** 
## Agriculture -0.06706    0.03098  -2.165 0.030420 *  
## Examination -0.37609    0.11286  -3.332 0.000861 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 65.135  on 46  degrees of freedom
## Residual deviance: 40.484  on 44  degrees of freedom
## AIC: 46.484
## 
## Number of Fisher Scoring iterations: 5
# Compare AIC values
AIC(log_model_full, log_model_reduced)
##                   df      AIC
## log_model_full     6 44.88671
## log_model_reduced  3 46.48369

4.3 Model Comparison

The full model has an AIC of 44.89, while the reduced model has an AIC of 46.48. The difference is small, but the full model has a slightly lower AIC. I decided to keep the full model for predicting whether Fertility is greater than 70.

4.4 Prediction and Accuracy

I used the full logistic regression model to predict the probability that Fertility is greater than 70. I used 0.5 as the cutoff to classify the predictions into 0 or 1.

# Predict probabilities
pred_prob <- predict(log_model_full, type = "response")

# Convert probabilities into 0 or 1
pred_class <- ifelse(pred_prob > 0.5, 1, 0)

# Create a confusion matrix
table(Actual = swiss$Fertility_High,
      Predicted = pred_class)
##       Predicted
## Actual  0  1
##      0 18  5
##      1  2 22
# Calculate accuracy
mean(pred_class == swiss$Fertility_High)
## [1] 0.8510638

4.5 Interpretation

The model correctly predicted 40 out of 47 observations, with an accuracy of 85.11%. It correctly classified 18 observations with Fertility at or below 70 and 22 observations with Fertility above 70. There were 7 incorrect predictions. Overall, the model performed well on this dataset. However, the accuracy was calculated using the same data used to build the model, so the results may be different for new data.

5. Conclusion

For multiple linear regression, I chose the reduced model with four independent variables. It explains about 70% of the variation in Fertility, and the diagnostic plots suggest that the model assumptions are mostly reasonable. For logistic regression, I kept the full model because it had a slightly lower AIC. The model achieved 85.11% accuracy on the training data when predicting whether Fertility was greater than 70.