title: “My How-To Guide: Logistic Regression” author: “Sophia Gauhman” output: html_document —

Before you start

Logistic regression is a binomial GLM with a logit link. For binary outcomes such as survived/died or switched/did not switch, it predicts probabilities from 0 to 1, using \(\log(p/(1-p))=\beta_0+\beta_1x\).

1. Simulate binomial distributions

rbinom(n, size, prob) draws n observations; each observation counts the number of successes in size Bernoulli trials with probability prob per trial. Here each simulated outcome ranges from 0 to 10.

b1 <- rbinom(n=50, size=10, prob=.1)
b2 <- rbinom(n=100, size=10, prob=.1)
par(mfrow=c(1,2))
for (i in 1:2) {
  x <- list(b1,b2)[[i]]
  k <- 0:10
  obs <- tabulate(x+1, nbins=11)/length(x)
  plot(k, dbinom(k,size=10,prob=.1), type="h",lwd=3,col="steelblue",
       xlab="Successes out of 10", ylab="Probability / sample proportion",
       ylim=c(0,max(obs,dbinom(k,10,.1))*.12+max(obs,dbinom(k,10,.1))),
       main=paste("n =",length(x),"; size = 10; p = 0.1"))
  points(k,obs,pch=16,col="firebrick")
  legend("topright",c("Theoretical","Simulated"),col=c("steelblue","firebrick"),
         lty=c(1,NA),pch=c(NA,16),bty="n",cex=.75)
}

par(mfrow=c(1,1))
data.frame(n=c(50,100), observed_mean=c(mean(b1),mean(b2)),
           expected_mean=10*.1, observed_variance=c(var(b1),var(b2)),
           expected_variance=10*.1*(1-.1))
##     n observed_mean expected_mean observed_variance expected_variance
## 1  50          0.98             1         0.7138776               0.9
## 2 100          1.06             1         0.7236364               0.9

Interpretation: The expected number of successes is size * p = 1. The theoretical variance is size * p * (1-p) = 0.9. Larger n generally produces a more stable match to the theoretical probabilities, although either individual simulation may vary.

2. Bumpus sparrows — survival versus weight

Question: Does body weight predict the probability of surviving the storm? The supplied course CSV has 136 rows and columns survival (0/1) and weight. There are no missing values. Inspect it yourself before fitting the model. Background: https://search.r-project.org/CRAN/refmans/Sleuth2/html/ex2016.html

bumpus <- read.csv("data/bumpus.csv", stringsAsFactors=FALSE)
names(bumpus)
##  [1] "sex"         "survival"    "length"      "alar_extent" "weight"     
##  [6] "head"        "humer"       "femur"       "tibio"       "skull"      
## [11] "sternum"
str(bumpus)
## 'data.frame':    136 obs. of  11 variables:
##  $ sex        : int  1 1 1 1 1 1 1 1 1 1 ...
##  $ survival   : int  1 1 1 1 1 1 1 1 1 1 ...
##  $ length     : int  154 160 155 154 156 161 157 159 158 158 ...
##  $ alar_extent: int  241 252 243 245 247 253 251 247 247 252 ...
##  $ weight     : num  24.5 26.9 26.9 24.3 24.1 26.5 24.6 24.2 23.6 26.2 ...
##  $ head       : num  31.2 30.8 30.6 31.7 31.5 31.8 31.1 31.4 29.8 32 ...
##  $ humer      : num  0.687 0.736 0.733 0.741 0.715 0.78 0.741 0.728 0.703 0.749 ...
##  $ femur      : num  0.668 0.709 0.704 0.688 0.706 0.743 0.736 0.718 0.673 0.739 ...
##  $ tibio      : num  1.02 1.18 1.15 1.15 1.13 ...
##  $ skull      : num  0.587 0.602 0.602 0.584 0.575 0.607 0.61 0.609 0.602 0.614 ...
##  $ sternum    : num  0.83 0.841 0.846 0.839 0.821 0.893 0.862 0.793 0.82 0.857 ...
head(bumpus)
##   sex survival length alar_extent weight head humer femur tibio skull sternum
## 1   1        1    154         241   24.5 31.2 0.687 0.668 1.022 0.587   0.830
## 2   1        1    160         252   26.9 30.8 0.736 0.709 1.180 0.602   0.841
## 3   1        1    155         243   26.9 30.6 0.733 0.704 1.151 0.602   0.846
## 4   1        1    154         245   24.3 31.7 0.741 0.688 1.146 0.584   0.839
## 5   1        1    156         247   24.1 31.5 0.715 0.706 1.129 0.575   0.821
## 6   1        1    161         253   26.5 31.8 0.780 0.743 1.144 0.607   0.893
summary(bumpus)
##       sex            survival          length       alar_extent   
##  Min.   :0.0000   Min.   :0.0000   Min.   :152.0   Min.   :230.0  
##  1st Qu.:0.0000   1st Qu.:0.0000   1st Qu.:157.0   1st Qu.:242.0  
##  Median :1.0000   Median :1.0000   Median :160.0   Median :246.0  
##  Mean   :0.6397   Mean   :0.5294   Mean   :159.5   Mean   :245.3  
##  3rd Qu.:1.0000   3rd Qu.:1.0000   3rd Qu.:162.0   3rd Qu.:249.0  
##  Max.   :1.0000   Max.   :1.0000   Max.   :167.0   Max.   :256.0  
##      weight           head           humer            femur       
##  Min.   :22.60   Min.   :29.80   Min.   :0.6590   Min.   :0.6530  
##  1st Qu.:24.57   1st Qu.:31.10   1st Qu.:0.7177   1st Qu.:0.7007  
##  Median :25.55   Median :31.60   Median :0.7330   Median :0.7130  
##  Mean   :25.52   Mean   :31.57   Mean   :0.7322   Mean   :0.7129  
##  3rd Qu.:26.50   3rd Qu.:32.02   3rd Qu.:0.7495   3rd Qu.:0.7312  
##  Max.   :31.00   Max.   :33.40   Max.   :0.7800   Max.   :0.7670  
##      tibio           skull           sternum      
##  Min.   :1.011   Min.   :0.5510   Min.   :0.7340  
##  1st Qu.:1.112   1st Qu.:0.5920   1st Qu.:0.8090  
##  Median :1.133   Median :0.6020   Median :0.8410  
##  Mean   :1.134   Mean   :0.6024   Mean   :0.8399  
##  3rd Qu.:1.162   3rd Qu.:0.6110   3rd Qu.:0.8652  
##  Max.   :1.230   Max.   :0.6400   Max.   :0.9270

The actual course dataset uses lowercase survival and weight. We retain the original codes, where 1 is the event being modelled (survival). Confirm this interpretation against your course’s dataset description.

stopifnot(all(c("weight", "survival") %in% names(bumpus)))
table(bumpus$survival, useNA="ifany")
## 
##  0  1 
## 64 72
bumpus_clean <- na.omit(bumpus[, c("weight", "survival")])
bumpus_clean$survived <- bumpus_clean$survival
stopifnot(all(bumpus_clean$survived %in% c(0,1)))

Explore the data

head(bumpus_clean)
##   weight survival survived
## 1   24.5        1        1
## 2   26.9        1        1
## 3   26.9        1        1
## 4   24.3        1        1
## 5   24.1        1        1
## 6   26.5        1        1
table(bumpus_clean$survived)
## 
##  0  1 
## 64 72
aggregate(weight ~ survived, data=bumpus_clean, FUN=mean)
##   survived   weight
## 1        0 25.86094
## 2        1 25.22639
boxplot(weight ~ survived, data=bumpus_clean,
        names=c("Died","Survived"), xlab="Outcome", ylab="Weight")

plot(jitter(bumpus_clean$survived, amount=.05) ~ bumpus_clean$weight,
     xlab="Weight", ylab="Survival (0 = died, 1 = survived)",pch=16,
     col=rgb(0,0,0,.3))

Fit a logistic regression and test weight

bumpus_null <- glm(survived ~ 1, family=binomial(link="logit"), data=bumpus_clean)
bumpus_fit <- glm(survived ~ weight, family=binomial(link="logit"), data=bumpus_clean)
summary(bumpus_fit)
## 
## Call:
## glm(formula = survived ~ weight, family = binomial(link = "logit"), 
##     data = bumpus_clean)
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)  
## (Intercept)   8.0456     3.2516   2.474   0.0133 *
## weight       -0.3105     0.1272  -2.441   0.0146 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 188.07  on 135  degrees of freedom
## Residual deviance: 181.58  on 134  degrees of freedom
## AIC: 185.58
## 
## Number of Fisher Scoring iterations: 4
anova(bumpus_null,bumpus_fit,test="Chisq")
## Analysis of Deviance Table
## 
## Model 1: survived ~ 1
## Model 2: survived ~ weight
##   Resid. Df Resid. Dev Df Deviance Pr(>Chi)  
## 1       135     188.06                       
## 2       134     181.58  1   6.4847  0.01088 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
exp(coef(bumpus_fit))  # odds ratios; exp(beta_weight) is per 1 weight unit
##  (Intercept)       weight 
## 3119.9070408    0.7330572

Assumptions and model fit

  • The outcome is binary and correctly coded (1 = survived).
  • Each bird’s observation should be independent; consider sampling/design limitations.
  • The log odds of survival are approximately linear in weight (NOT necessarily probability itself).
  • Check influential observations and model separation (e.g., fitted probabilities extremely close to 0 or 1, warnings about fitted probabilities).
  • Standard binary logistic regression does not require normally distributed residuals or constant-variance errors.
plot(bumpus_clean$weight, residuals(bumpus_fit,type="deviance"),
     pch=16, xlab="Weight",ylab="Deviance residual")
abline(h=0,lty=2)

plot(bumpus_fit, which=4) # Cook's distances; check influential birds

range(fitted(bumpus_fit))
## [1] 0.1706656 0.7364401
# Check possible departure from a linear weight effect in log odds
bumpus_quad <- glm(survived ~ weight + I(weight^2), family=binomial,
                   data=bumpus_clean)
anova(bumpus_fit,bumpus_quad,test="Chisq")
## Analysis of Deviance Table
## 
## Model 1: survived ~ weight
## Model 2: survived ~ weight + I(weight^2)
##   Resid. Df Resid. Dev Df Deviance Pr(>Chi)
## 1       134     181.58                     
## 2       133     180.41  1   1.1703   0.2794

Plot observed outcomes and fitted survival probability

new_birds <- data.frame(weight=seq(min(bumpus_clean$weight),
                                    max(bumpus_clean$weight),length.out=200))
pred <- predict(bumpus_fit,newdata=new_birds,type="link",se.fit=TRUE)
plot(jitter(survived,amount=.05) ~ weight,data=bumpus_clean,
     xlab="Weight",ylab="Probability of survival",ylim=c(0,1),
     pch=16,col=rgb(0,0,0,.3))
lines(new_birds$weight,plogis(pred$fit),col="blue",lwd=2)
lines(new_birds$weight,plogis(pred$fit-1.96*pred$se.fit),col="blue",lty=2)
lines(new_birds$weight,plogis(pred$fit+1.96*pred$se.fit),col="blue",lty=2)

My interpretation: The CSV uses survival coded 0/1 (verify labels from course materials). Sample size after missing values is . Weight coefficient is , with odds ratio . Comparing null and weight models gives deviance difference , df , p-value . Graph suggests ___. Check assumptions and limitations before drawing conclusions.

3. Wells data — Hector (2021), Chapter 18

Question: Does the effect of arsenic on probability of switching wells depend on the distance to the nearest safe well? This is an interaction question. The carData::Wells public dataset has switch, arsenic, distance, education, and association. See https://rdrr.io/cran/carData/man/Wells.html . Hector’s chapter is https://doi.org/10.1093/oso/9780198798170.003.0018 ; the exact text/code of the full chapter was not available to verify here, so compare this independent workflow with your assigned chapter.

if (!requireNamespace("carData", quietly=TRUE)) {
  stop("Install once from RStudio Console: install.packages('carData'), then Knit again.")
}
data("Wells", package="carData")
wells <- carData::Wells
names(wells)
## [1] "switch"      "arsenic"     "distance"    "education"   "association"
str(wells)
## 'data.frame':    3020 obs. of  5 variables:
##  $ switch     : Factor w/ 2 levels "no","yes": 2 2 1 2 2 2 2 2 2 2 ...
##  $ arsenic    : num  2.36 0.71 2.07 1.15 1.1 3.9 2.97 3.24 3.28 2.52 ...
##  $ distance   : num  16.8 47.3 21 21.5 40.9 ...
##  $ education  : int  0 0 10 12 14 9 4 10 0 0 ...
##  $ association: Factor w/ 2 levels "no","yes": 1 1 1 1 2 2 2 1 2 2 ...
summary(wells)
##  switch        arsenic         distance         education      association
##  no :1283   Min.   :0.510   Min.   :  0.387   Min.   : 0.000   no :1743   
##  yes:1737   1st Qu.:0.820   1st Qu.: 21.117   1st Qu.: 0.000   yes:1277   
##             Median :1.300   Median : 36.761   Median : 5.000              
##             Mean   :1.657   Mean   : 48.332   Mean   : 4.828              
##             3rd Qu.:2.200   3rd Qu.: 64.041   3rd Qu.: 8.000              
##             Max.   :9.650   Max.   :339.531   Max.   :17.000
table(wells$switch, useNA="ifany")
## 
##   no  yes 
## 1283 1737

Make sure the response is 0/1 (the documented Wells dataset uses numeric switch coding). Scale distance to 100 meters, so the coefficients are easier to interpret.

# Load Wells dataset
library(carData)
data("Wells", package = "carData")

wells <- Wells
# Express distance in units of 100
wells$dist100 <- wells$distance / 100
# Convert switch to numeric:
# no = 0, yes = 1
wells$switch_binary <- ifelse(
  wells$switch == "yes", 1, 0
)

# Check that conversion worked
table(wells$switch, wells$switch_binary)
##      
##          0    1
##   no  1283    0
##   yes    0 1737
# Check for missing values
colSums(is.na(wells))
##        switch       arsenic      distance     education   association 
##             0             0             0             0             0 
##       dist100 switch_binary 
##             0             0

Fit two nested models

arsenic + dist100 means both predictors have separate effects (no interaction). arsenic * dist100 expands to arsenic + dist100 + arsenic:dist100, allowing the effect of arsenic to change with distance.

# Model without interaction
well_add <- glm(
  switch_binary ~ arsenic + dist100,
  family = binomial(link = "logit"),
  data = wells
)

# Model with interaction
well_int <- glm(
  switch_binary ~ arsenic * dist100,
  family = binomial(link = "logit"),
  data = wells
)

# Examine results
summary(well_add)
## 
## Call:
## glm(formula = switch_binary ~ arsenic + dist100, family = binomial(link = "logit"), 
##     data = wells)
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)    
## (Intercept)  0.002749   0.079448   0.035    0.972    
## arsenic      0.460775   0.041385  11.134   <2e-16 ***
## dist100     -0.896644   0.104347  -8.593   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 4118.1  on 3019  degrees of freedom
## Residual deviance: 3930.7  on 3017  degrees of freedom
## AIC: 3936.7
## 
## Number of Fisher Scoring iterations: 4
summary(well_int)
## 
## Call:
## glm(formula = switch_binary ~ arsenic * dist100, family = binomial(link = "logit"), 
##     data = wells)
## 
## Coefficients:
##                 Estimate Std. Error z value Pr(>|z|)    
## (Intercept)     -0.14787    0.11754  -1.258  0.20838    
## arsenic          0.55598    0.06932   8.021 1.05e-15 ***
## dist100         -0.57722    0.20918  -2.759  0.00579 ** 
## arsenic:dist100 -0.17891    0.10233  -1.748  0.08040 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 4118.1  on 3019  degrees of freedom
## Residual deviance: 3927.6  on 3016  degrees of freedom
## AIC: 3935.6
## 
## Number of Fisher Scoring iterations: 4
# Test whether the interaction is significant
anova(well_add, well_int, test = "Chisq")
## Analysis of Deviance Table
## 
## Model 1: switch_binary ~ arsenic + dist100
## Model 2: switch_binary ~ arsenic * dist100
##   Resid. Df Resid. Dev Df Deviance Pr(>Chi)  
## 1      3017     3930.7                       
## 2      3016     3927.6  1   3.0399  0.08124 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
# Convert coefficients to odds ratios
exp(coef(well_int))
##     (Intercept)         arsenic         dist100 arsenic:dist100 
##       0.8625449       1.7436433       0.5614583       0.8361845

Interpretation of Wells logistic regression

A binomial logistic regression was performed to investigate whether the association between arsenic concentration and well switching depended on the distance to a safe well.

The interaction model included arsenic concentration, distance (expressed in units of 100), and their interaction.

The estimated interaction coefficient was negative (β = −0.17891, SE = 0.10233, z = −1.748, p = 0.08040), suggesting that the association between arsenic concentration and the log-odds of switching may become weaker as distance increases.

However, the likelihood-ratio test comparing the additive and interaction models showed that including the interaction did not significantly improve model fit (χ²(1) = 3.0399, p = 0.08124).

Therefore, there was insufficient evidence at the 0.05 significance level to conclude that the effect of arsenic on the log-odds of switching depends on distance.

The simpler additive model was preferred for interpretation, while acknowledging that a nonsignificant result does not establish the absence of an interaction.

In this model, the slope of arsenic on the log-odds scale is beta_arsenic + beta_interaction * dist100. A statistically significant interaction suggests this slope varies with distance. Even without a log-odds interaction, probability-scale changes can differ because the logit transformation is nonlinear.

Plot the interaction at two meaningful distances

# Plot arsenic concentration against well switching
plot(
  jitter(wells$arsenic, amount = 0.05),
  jitter(wells$switch_binary, amount = 0.05),
  xlab = "Arsenic concentration",
  ylab = "Switched wells (0 = No, 1 = Yes)",
  main = "Arsenic Concentration and Well Switching",
  pch = 16,
  col = rgb(0, 0, 0, 0.15),
  ylim = c(-0.1, 1.1)
)

# Create a sequence of arsenic concentrations
new_wells <- data.frame(
  arsenic = seq(
    min(wells$arsenic),
    max(wells$arsenic),
    length.out = 200
  ),
  dist100 = median(wells$dist100)
)

# Predict probabilities using the additive model
new_wells$probability <- predict(
  well_add,
  newdata = new_wells,
  type = "response"
)

# Add fitted logistic regression curve
lines(
  new_wells$arsenic,
  new_wells$probability,
  col = "blue",
  lwd = 3
)

Model checks

range(fitted(well_int))
## [1] 0.1014427 0.9825588
plot(fitted(well_int),residuals(well_int,type="deviance"),pch=16,
     col=rgb(0,0,0,.15),xlab="Predicted switching probability",
     ylab="Deviance residual")
abline(h=0,lty=2)

# Check possible nonlinear arsenic/distance effects using simple quadratic terms.
well_curved <- glm(switch ~ arsenic * dist100 + I(arsenic^2) + I(dist100^2),
                   family=binomial,data=wells)
anova(well_int,well_curved,test="Chisq")
## Analysis of Deviance Table
## 
## Model: binomial, link: logit
## 
## Response: switch_binary
## 
## Terms added sequentially (first to last)
## 
## 
##                 Df Deviance Resid. Df Resid. Dev Pr(>Chi)    
## NULL                             3019     4118.1             
## arsenic          1  109.426      3018     4008.7  < 2e-16 ***
## dist100          1   78.005      3017     3930.7  < 2e-16 ***
## arsenic:dist100  1    3.040      3016     3927.6  0.08124 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

My conclusion (complete after running): The interaction coefficient is ; its sign suggests that the arsenic log-odds slope [increases/decreases] as distance rises. The likelihood-ratio test comparing additive and interaction models gives \(\Delta D\) , df , p-value . At significance level 0.05, this [does/does not] support including the interaction. The probability curves are ___. This is observational association, not proof of causation.

Checklist

# Explore the Bumpus dataset
str(bumpus)
## 'data.frame':    136 obs. of  11 variables:
##  $ sex        : int  1 1 1 1 1 1 1 1 1 1 ...
##  $ survival   : int  1 1 1 1 1 1 1 1 1 1 ...
##  $ length     : int  154 160 155 154 156 161 157 159 158 158 ...
##  $ alar_extent: int  241 252 243 245 247 253 251 247 247 252 ...
##  $ weight     : num  24.5 26.9 26.9 24.3 24.1 26.5 24.6 24.2 23.6 26.2 ...
##  $ head       : num  31.2 30.8 30.6 31.7 31.5 31.8 31.1 31.4 29.8 32 ...
##  $ humer      : num  0.687 0.736 0.733 0.741 0.715 0.78 0.741 0.728 0.703 0.749 ...
##  $ femur      : num  0.668 0.709 0.704 0.688 0.706 0.743 0.736 0.718 0.673 0.739 ...
##  $ tibio      : num  1.02 1.18 1.15 1.15 1.13 ...
##  $ skull      : num  0.587 0.602 0.602 0.584 0.575 0.607 0.61 0.609 0.602 0.614 ...
##  $ sternum    : num  0.83 0.841 0.846 0.839 0.821 0.893 0.862 0.793 0.82 0.857 ...
# Count how many survived vs. did not survive
table(bumpus$survival)
## 
##  0  1 
## 64 72
# Summary statistics for weight
summary(bumpus$weight)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   22.60   24.57   25.55   25.52   26.50   31.00
# Check for missing values
colSums(is.na(bumpus))
##         sex    survival      length alar_extent      weight        head 
##           0           0           0           0           0           0 
##       humer       femur       tibio       skull     sternum 
##           0           0           0           0           0
# Compare weights between survival groups
aggregate(weight ~ survival,
          data = bumpus,
          FUN = mean)
##   survival   weight
## 1        0 25.86094
## 2        1 25.22639
# Create boxplot of weight by survival
boxplot(weight ~ survival,
        data = bumpus,
        xlab = "Survival (0 = No, 1 = Yes)",
        ylab = "Weight",
        main = "Sparrow Weight by Survival Status")

### Exploratory analysis: Sparrow weight and survival

The Bumpus dataset contains 136 sparrows, with survival recorded as a binary response variable (0 = did not survive, 1 = survived). There were no missing values in the dataset.

Descriptive statistics showed that sparrows that did not survive had a higher mean weight (25.86) than those that survived (25.23). The boxplot also suggests that non-surviving birds tended to be heavier, although the weight distributions overlap considerably.

These exploratory results suggest a possible negative association between weight and survival. However, logistic regression is required to determine whether this relationship is statistically significant.

# Fit logistic regression model
bumpus_model <- glm(
  survival ~ weight,
  family = binomial(link = "logit"),
  data = bumpus
)

# Display model results
summary(bumpus_model)
## 
## Call:
## glm(formula = survival ~ weight, family = binomial(link = "logit"), 
##     data = bumpus)
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)  
## (Intercept)   8.0456     3.2516   2.474   0.0133 *
## weight       -0.3105     0.1272  -2.441   0.0146 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 188.07  on 135  degrees of freedom
## Residual deviance: 181.58  on 134  degrees of freedom
## AIC: 185.58
## 
## Number of Fisher Scoring iterations: 4

Logistic regression results

A binomial logistic regression with a logit link was conducted to determine whether sparrow weight was associated with survival.

Weight was a statistically significant negative predictor of survival (β = −0.3105, SE = 0.1272, z = −2.441, p = 0.0146).

The odds ratio was approximately 0.733, indicating that each one-unit increase in weight was associated with a 26.7% decrease in the odds of survival.

The null deviance was 188.07 (135 df), while the residual deviance was 181.58 (134 df). Including weight reduced deviance by 6.49 with one degree of freedom, suggesting that weight improved model fit.

Overall, heavier sparrows had significantly lower estimated odds of survival. However, this analysis demonstrates an association rather than a causal relationship.

# Diagnostic plots for logistic regression
par(mfrow = c(2, 2))

plot(bumpus_model)

par(mfrow = c(1, 1))
# Original logistic regression
model_linear <- glm(
  survival ~ weight,
  family = binomial,
  data = bumpus
)

# Model allowing a curved relationship
model_quadratic <- glm(
  survival ~ weight + I(weight^2),
  family = binomial,
  data = bumpus
)

# Compare models
anova(model_linear, model_quadratic, test = "Chisq")
## Analysis of Deviance Table
## 
## Model 1: survival ~ weight
## Model 2: survival ~ weight + I(weight^2)
##   Resid. Df Resid. Dev Df Deviance Pr(>Chi)
## 1       134     181.58                     
## 2       133     180.41  1   1.1703   0.2794

Checking the linearity assumption

Logistic regression assumes that continuous predictors have a linear relationship with the log-odds of the response.

To assess this assumption, I compared the original logistic regression model containing weight with a quadratic model containing both weight and weight².

Adding the quadratic term did not significantly improve model fit (ΔDeviance = 1.1703, df = 1, p = 0.2794).

Therefore, there was insufficient evidence of a quadratic relationship between weight and the log-odds of survival. The simpler linear-logit model was retained, although this test does not rule out all possible forms of nonlinearity.

# Calculate Cook's distance for each sparrow
cooks <- cooks.distance(bumpus_model)

# Plot Cook's distance
plot(cooks,
     type = "h",
     main = "Cook's Distance for Bumpus Logistic Regression",
     xlab = "Sparrow observation number",
     ylab = "Cook's distance")

# Display the five most influential observations
head(sort(cooks, decreasing = TRUE), 5)
##        132        124        112         21         22 
## 0.04906835 0.04209333 0.03037512 0.02981301 0.02733324
# Reference threshold for further investigation
4 / nrow(bumpus)
## [1] 0.02941176

Checking for influential observations

Cook’s distance was calculated to identify observations that may disproportionately influence the fitted logistic regression model.

Using the screening threshold of 4/n = 0.0294, four observations exceeded this value: observations 132, 124, 112 and 21.

The largest Cook’s distance was approximately 0.0491 (observation 132). These observations may warrant further investigation, but Cook’s distance alone does not establish that they are problematic.

No observations were removed from the analysis. The potential influence of these observations should be considered when interpreting the model results.

# Plot survival data (0 = died, 1 = survived)
plot(jitter(bumpus$weight),
     jitter(bumpus$survival, amount = 0.04),
     xlab = "Sparrow weight",
     ylab = "Survival probability",
     main = "Logistic Regression of Sparrow Survival",
     pch = 16,
     col = rgb(0, 0, 0, 0.3),
     ylim = c(0, 1))

# Create an ordered sequence of weights
newdata <- data.frame(
  weight = seq(min(bumpus$weight),
               max(bumpus$weight),
               length.out = 200)
)

# Predict survival probabilities
newdata$probability <- predict(
  bumpus_model,
  newdata = newdata,
  type = "response"
)

# Add the logistic regression curve
lines(newdata$weight,
      newdata$probability,
      col = "blue",
      lwd = 3)

Visualization of logistic regression

The logistic regression plot illustrates the relationship between sparrow weight and predicted probability of survival. Individual observations are shown as points, with survival coded as 0 (did not survive) or 1 (survived). The fitted logistic regression curve represents the predicted probability of survival across the observed range of weights.

The predicted probability of survival decreases as sparrow weight increases, consistent with the negative regression coefficient (β = −0.3105, p = 0.0146).

Although individual observations show considerable variation, the model indicates a statistically significant negative association between weight and survival.

# Identify observations above the Cook's distance screening threshold
influential <- which(cooks > 4 / nrow(bumpus))

# Display their observation numbers
influential
##  21 112 124 132 
##  21 112 124 132
# Fit sensitivity model excluding flagged observations
model_sensitivity <- glm(
  survival ~ weight,
  family = binomial,
  data = bumpus[-influential, ]
)

# Compare weight estimates and p-values
summary(bumpus_model)$coefficients["weight", ]
##    Estimate  Std. Error     z value    Pr(>|z|) 
## -0.31053148  0.12720813 -2.44112925  0.01464141
summary(model_sensitivity)$coefficients["weight", ]
##     Estimate   Std. Error      z value     Pr(>|z|) 
## -0.452663680  0.143746506 -3.149041275  0.001638071

Sensitivity analysis of influential observations

A sensitivity analysis was conducted to determine whether observations with Cook’s distances exceeding the screening threshold of 4/n substantially affected the logistic regression results.

Four observations (21, 112, 124 and 132) were temporarily excluded, and the logistic regression was refitted.

The weight coefficient changed from β = −0.3105 (p = 0.0146) in the original model to β = −0.4527 (p = 0.00164) in the sensitivity model.

The association between weight and survival remained negative and statistically significant after removing these observations. However, the change in coefficient magnitude suggests that the estimated strength of the association is somewhat sensitive to these observations.

Because there was no established reason to exclude these sparrows, the original model containing all 136 observations was retained for the primary analysis.

Conclusion: Bumpus sparrow survival and weight

A binomial logistic regression was conducted to investigate whether sparrow weight was associated with survival.

The analysis identified a statistically significant negative relationship between weight and survival (β = −0.3105, SE = 0.1272, z = −2.441, p = 0.0146). The corresponding odds ratio was approximately 0.733, indicating a 26.7% decrease in the estimated odds of survival for each one-unit increase in weight.

Model diagnostics were examined. Adding a quadratic weight term did not significantly improve model fit (ΔDeviance = 1.1703, df = 1, p = 0.2794), providing no strong evidence of quadratic nonlinearity on the logit scale.

Four observations exceeded the Cook’s distance screening threshold. A sensitivity analysis excluding these observations produced a stronger negative weight coefficient (β = −0.4527, p = 0.00164), indicating that the overall direction of the association was maintained, although its magnitude changed.

Overall, heavier sparrows had lower estimated odds of survival in this dataset. These findings support an association between weight and survival but do not establish a causal relationship. Independence of observations remains an assumption that must be evaluated using information about the sampling design.

Wells logistic regression: Arsenic and distance interaction

To investigate whether the association between arsenic concentration and well switching depended on distance to a safe well, two binomial logistic regression models were fitted.

The additive model included arsenic concentration and distance as explanatory variables, while the interaction model additionally included an arsenic × distance interaction term.

The models were compared using a likelihood-ratio test. The additive model had a residual deviance of 3930.7, while the interaction model had a residual deviance of 3927.6.

Including the interaction reduced deviance by 3.0399 with one degree of freedom. However, this improvement was not statistically significant (χ²(1) = 3.0399, p = 0.08124).

Therefore, there was insufficient evidence at the 0.05 significance level that the association between arsenic concentration and the log-odds of switching wells differed according to distance.

The simpler additive model was retained for interpretation, although a nonsignificant interaction test does not prove the absence of an interaction.