title: “My How-To Guide: Logistic Regression” author: “Sophia Gauhman” output: html_document —
Logistic regression is a binomial GLM with a logit link. For binary outcomes such as survived/died or switched/did not switch, it predicts probabilities from 0 to 1, using \(\log(p/(1-p))=\beta_0+\beta_1x\).
rbinom(n, size, prob) draws n
observations; each observation counts the number of successes
in size Bernoulli trials with probability prob
per trial. Here each simulated outcome ranges from 0 to 10.
b1 <- rbinom(n=50, size=10, prob=.1)
b2 <- rbinom(n=100, size=10, prob=.1)
par(mfrow=c(1,2))
for (i in 1:2) {
x <- list(b1,b2)[[i]]
k <- 0:10
obs <- tabulate(x+1, nbins=11)/length(x)
plot(k, dbinom(k,size=10,prob=.1), type="h",lwd=3,col="steelblue",
xlab="Successes out of 10", ylab="Probability / sample proportion",
ylim=c(0,max(obs,dbinom(k,10,.1))*.12+max(obs,dbinom(k,10,.1))),
main=paste("n =",length(x),"; size = 10; p = 0.1"))
points(k,obs,pch=16,col="firebrick")
legend("topright",c("Theoretical","Simulated"),col=c("steelblue","firebrick"),
lty=c(1,NA),pch=c(NA,16),bty="n",cex=.75)
}
par(mfrow=c(1,1))
data.frame(n=c(50,100), observed_mean=c(mean(b1),mean(b2)),
expected_mean=10*.1, observed_variance=c(var(b1),var(b2)),
expected_variance=10*.1*(1-.1))
## n observed_mean expected_mean observed_variance expected_variance
## 1 50 0.98 1 0.7138776 0.9
## 2 100 1.06 1 0.7236364 0.9
Interpretation: The expected number of successes is
size * p = 1. The theoretical variance is
size * p * (1-p) = 0.9. Larger n generally produces a more
stable match to the theoretical probabilities, although either
individual simulation may vary.
Question: Does body weight predict the probability
of surviving the storm? The supplied course CSV has 136 rows and columns
survival (0/1) and weight. There are no
missing values. Inspect it yourself before fitting the model.
Background: https://search.r-project.org/CRAN/refmans/Sleuth2/html/ex2016.html
bumpus <- read.csv("data/bumpus.csv", stringsAsFactors=FALSE)
names(bumpus)
## [1] "sex" "survival" "length" "alar_extent" "weight"
## [6] "head" "humer" "femur" "tibio" "skull"
## [11] "sternum"
str(bumpus)
## 'data.frame': 136 obs. of 11 variables:
## $ sex : int 1 1 1 1 1 1 1 1 1 1 ...
## $ survival : int 1 1 1 1 1 1 1 1 1 1 ...
## $ length : int 154 160 155 154 156 161 157 159 158 158 ...
## $ alar_extent: int 241 252 243 245 247 253 251 247 247 252 ...
## $ weight : num 24.5 26.9 26.9 24.3 24.1 26.5 24.6 24.2 23.6 26.2 ...
## $ head : num 31.2 30.8 30.6 31.7 31.5 31.8 31.1 31.4 29.8 32 ...
## $ humer : num 0.687 0.736 0.733 0.741 0.715 0.78 0.741 0.728 0.703 0.749 ...
## $ femur : num 0.668 0.709 0.704 0.688 0.706 0.743 0.736 0.718 0.673 0.739 ...
## $ tibio : num 1.02 1.18 1.15 1.15 1.13 ...
## $ skull : num 0.587 0.602 0.602 0.584 0.575 0.607 0.61 0.609 0.602 0.614 ...
## $ sternum : num 0.83 0.841 0.846 0.839 0.821 0.893 0.862 0.793 0.82 0.857 ...
head(bumpus)
## sex survival length alar_extent weight head humer femur tibio skull sternum
## 1 1 1 154 241 24.5 31.2 0.687 0.668 1.022 0.587 0.830
## 2 1 1 160 252 26.9 30.8 0.736 0.709 1.180 0.602 0.841
## 3 1 1 155 243 26.9 30.6 0.733 0.704 1.151 0.602 0.846
## 4 1 1 154 245 24.3 31.7 0.741 0.688 1.146 0.584 0.839
## 5 1 1 156 247 24.1 31.5 0.715 0.706 1.129 0.575 0.821
## 6 1 1 161 253 26.5 31.8 0.780 0.743 1.144 0.607 0.893
summary(bumpus)
## sex survival length alar_extent
## Min. :0.0000 Min. :0.0000 Min. :152.0 Min. :230.0
## 1st Qu.:0.0000 1st Qu.:0.0000 1st Qu.:157.0 1st Qu.:242.0
## Median :1.0000 Median :1.0000 Median :160.0 Median :246.0
## Mean :0.6397 Mean :0.5294 Mean :159.5 Mean :245.3
## 3rd Qu.:1.0000 3rd Qu.:1.0000 3rd Qu.:162.0 3rd Qu.:249.0
## Max. :1.0000 Max. :1.0000 Max. :167.0 Max. :256.0
## weight head humer femur
## Min. :22.60 Min. :29.80 Min. :0.6590 Min. :0.6530
## 1st Qu.:24.57 1st Qu.:31.10 1st Qu.:0.7177 1st Qu.:0.7007
## Median :25.55 Median :31.60 Median :0.7330 Median :0.7130
## Mean :25.52 Mean :31.57 Mean :0.7322 Mean :0.7129
## 3rd Qu.:26.50 3rd Qu.:32.02 3rd Qu.:0.7495 3rd Qu.:0.7312
## Max. :31.00 Max. :33.40 Max. :0.7800 Max. :0.7670
## tibio skull sternum
## Min. :1.011 Min. :0.5510 Min. :0.7340
## 1st Qu.:1.112 1st Qu.:0.5920 1st Qu.:0.8090
## Median :1.133 Median :0.6020 Median :0.8410
## Mean :1.134 Mean :0.6024 Mean :0.8399
## 3rd Qu.:1.162 3rd Qu.:0.6110 3rd Qu.:0.8652
## Max. :1.230 Max. :0.6400 Max. :0.9270
The actual course dataset uses lowercase survival and
weight. We retain the original codes, where 1
is the event being modelled (survival). Confirm this interpretation
against your course’s dataset description.
stopifnot(all(c("weight", "survival") %in% names(bumpus)))
table(bumpus$survival, useNA="ifany")
##
## 0 1
## 64 72
bumpus_clean <- na.omit(bumpus[, c("weight", "survival")])
bumpus_clean$survived <- bumpus_clean$survival
stopifnot(all(bumpus_clean$survived %in% c(0,1)))
head(bumpus_clean)
## weight survival survived
## 1 24.5 1 1
## 2 26.9 1 1
## 3 26.9 1 1
## 4 24.3 1 1
## 5 24.1 1 1
## 6 26.5 1 1
table(bumpus_clean$survived)
##
## 0 1
## 64 72
aggregate(weight ~ survived, data=bumpus_clean, FUN=mean)
## survived weight
## 1 0 25.86094
## 2 1 25.22639
boxplot(weight ~ survived, data=bumpus_clean,
names=c("Died","Survived"), xlab="Outcome", ylab="Weight")
plot(jitter(bumpus_clean$survived, amount=.05) ~ bumpus_clean$weight,
xlab="Weight", ylab="Survival (0 = died, 1 = survived)",pch=16,
col=rgb(0,0,0,.3))
bumpus_null <- glm(survived ~ 1, family=binomial(link="logit"), data=bumpus_clean)
bumpus_fit <- glm(survived ~ weight, family=binomial(link="logit"), data=bumpus_clean)
summary(bumpus_fit)
##
## Call:
## glm(formula = survived ~ weight, family = binomial(link = "logit"),
## data = bumpus_clean)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 8.0456 3.2516 2.474 0.0133 *
## weight -0.3105 0.1272 -2.441 0.0146 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 188.07 on 135 degrees of freedom
## Residual deviance: 181.58 on 134 degrees of freedom
## AIC: 185.58
##
## Number of Fisher Scoring iterations: 4
anova(bumpus_null,bumpus_fit,test="Chisq")
## Analysis of Deviance Table
##
## Model 1: survived ~ 1
## Model 2: survived ~ weight
## Resid. Df Resid. Dev Df Deviance Pr(>Chi)
## 1 135 188.06
## 2 134 181.58 1 6.4847 0.01088 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
exp(coef(bumpus_fit)) # odds ratios; exp(beta_weight) is per 1 weight unit
## (Intercept) weight
## 3119.9070408 0.7330572
plot(bumpus_clean$weight, residuals(bumpus_fit,type="deviance"),
pch=16, xlab="Weight",ylab="Deviance residual")
abline(h=0,lty=2)
plot(bumpus_fit, which=4) # Cook's distances; check influential birds
range(fitted(bumpus_fit))
## [1] 0.1706656 0.7364401
# Check possible departure from a linear weight effect in log odds
bumpus_quad <- glm(survived ~ weight + I(weight^2), family=binomial,
data=bumpus_clean)
anova(bumpus_fit,bumpus_quad,test="Chisq")
## Analysis of Deviance Table
##
## Model 1: survived ~ weight
## Model 2: survived ~ weight + I(weight^2)
## Resid. Df Resid. Dev Df Deviance Pr(>Chi)
## 1 134 181.58
## 2 133 180.41 1 1.1703 0.2794
new_birds <- data.frame(weight=seq(min(bumpus_clean$weight),
max(bumpus_clean$weight),length.out=200))
pred <- predict(bumpus_fit,newdata=new_birds,type="link",se.fit=TRUE)
plot(jitter(survived,amount=.05) ~ weight,data=bumpus_clean,
xlab="Weight",ylab="Probability of survival",ylim=c(0,1),
pch=16,col=rgb(0,0,0,.3))
lines(new_birds$weight,plogis(pred$fit),col="blue",lwd=2)
lines(new_birds$weight,plogis(pred$fit-1.96*pred$se.fit),col="blue",lty=2)
lines(new_birds$weight,plogis(pred$fit+1.96*pred$se.fit),col="blue",lty=2)
My interpretation: The CSV uses survival coded 0/1 (verify labels from course materials). Sample size after missing values is . Weight coefficient is , with odds ratio . Comparing null and weight models gives deviance difference , df , p-value . Graph suggests ___. Check assumptions and limitations before drawing conclusions.
Question: Does the effect of arsenic on
probability of switching wells depend on the distance to the
nearest safe well? This is an interaction question. The
carData::Wells public dataset has switch,
arsenic, distance, education, and
association. See https://rdrr.io/cran/carData/man/Wells.html . Hector’s
chapter is https://doi.org/10.1093/oso/9780198798170.003.0018 ; the
exact text/code of the full chapter was not available to verify here, so
compare this independent workflow with your assigned chapter.
if (!requireNamespace("carData", quietly=TRUE)) {
stop("Install once from RStudio Console: install.packages('carData'), then Knit again.")
}
data("Wells", package="carData")
wells <- carData::Wells
names(wells)
## [1] "switch" "arsenic" "distance" "education" "association"
str(wells)
## 'data.frame': 3020 obs. of 5 variables:
## $ switch : Factor w/ 2 levels "no","yes": 2 2 1 2 2 2 2 2 2 2 ...
## $ arsenic : num 2.36 0.71 2.07 1.15 1.1 3.9 2.97 3.24 3.28 2.52 ...
## $ distance : num 16.8 47.3 21 21.5 40.9 ...
## $ education : int 0 0 10 12 14 9 4 10 0 0 ...
## $ association: Factor w/ 2 levels "no","yes": 1 1 1 1 2 2 2 1 2 2 ...
summary(wells)
## switch arsenic distance education association
## no :1283 Min. :0.510 Min. : 0.387 Min. : 0.000 no :1743
## yes:1737 1st Qu.:0.820 1st Qu.: 21.117 1st Qu.: 0.000 yes:1277
## Median :1.300 Median : 36.761 Median : 5.000
## Mean :1.657 Mean : 48.332 Mean : 4.828
## 3rd Qu.:2.200 3rd Qu.: 64.041 3rd Qu.: 8.000
## Max. :9.650 Max. :339.531 Max. :17.000
table(wells$switch, useNA="ifany")
##
## no yes
## 1283 1737
Make sure the response is 0/1 (the documented Wells
dataset uses numeric switch coding). Scale distance to 100 meters, so
the coefficients are easier to interpret.
# Load Wells dataset
library(carData)
data("Wells", package = "carData")
wells <- Wells
# Express distance in units of 100
wells$dist100 <- wells$distance / 100
# Convert switch to numeric:
# no = 0, yes = 1
wells$switch_binary <- ifelse(
wells$switch == "yes", 1, 0
)
# Check that conversion worked
table(wells$switch, wells$switch_binary)
##
## 0 1
## no 1283 0
## yes 0 1737
# Check for missing values
colSums(is.na(wells))
## switch arsenic distance education association
## 0 0 0 0 0
## dist100 switch_binary
## 0 0
arsenic + dist100 means both predictors have separate
effects (no interaction). arsenic * dist100 expands to
arsenic + dist100 + arsenic:dist100, allowing the effect of
arsenic to change with distance.
# Model without interaction
well_add <- glm(
switch_binary ~ arsenic + dist100,
family = binomial(link = "logit"),
data = wells
)
# Model with interaction
well_int <- glm(
switch_binary ~ arsenic * dist100,
family = binomial(link = "logit"),
data = wells
)
# Examine results
summary(well_add)
##
## Call:
## glm(formula = switch_binary ~ arsenic + dist100, family = binomial(link = "logit"),
## data = wells)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.002749 0.079448 0.035 0.972
## arsenic 0.460775 0.041385 11.134 <2e-16 ***
## dist100 -0.896644 0.104347 -8.593 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 4118.1 on 3019 degrees of freedom
## Residual deviance: 3930.7 on 3017 degrees of freedom
## AIC: 3936.7
##
## Number of Fisher Scoring iterations: 4
summary(well_int)
##
## Call:
## glm(formula = switch_binary ~ arsenic * dist100, family = binomial(link = "logit"),
## data = wells)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.14787 0.11754 -1.258 0.20838
## arsenic 0.55598 0.06932 8.021 1.05e-15 ***
## dist100 -0.57722 0.20918 -2.759 0.00579 **
## arsenic:dist100 -0.17891 0.10233 -1.748 0.08040 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 4118.1 on 3019 degrees of freedom
## Residual deviance: 3927.6 on 3016 degrees of freedom
## AIC: 3935.6
##
## Number of Fisher Scoring iterations: 4
# Test whether the interaction is significant
anova(well_add, well_int, test = "Chisq")
## Analysis of Deviance Table
##
## Model 1: switch_binary ~ arsenic + dist100
## Model 2: switch_binary ~ arsenic * dist100
## Resid. Df Resid. Dev Df Deviance Pr(>Chi)
## 1 3017 3930.7
## 2 3016 3927.6 1 3.0399 0.08124 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
# Convert coefficients to odds ratios
exp(coef(well_int))
## (Intercept) arsenic dist100 arsenic:dist100
## 0.8625449 1.7436433 0.5614583 0.8361845
A binomial logistic regression was performed to investigate whether the association between arsenic concentration and well switching depended on the distance to a safe well.
The interaction model included arsenic concentration, distance (expressed in units of 100), and their interaction.
The estimated interaction coefficient was negative (β = −0.17891, SE = 0.10233, z = −1.748, p = 0.08040), suggesting that the association between arsenic concentration and the log-odds of switching may become weaker as distance increases.
However, the likelihood-ratio test comparing the additive and interaction models showed that including the interaction did not significantly improve model fit (χ²(1) = 3.0399, p = 0.08124).
Therefore, there was insufficient evidence at the 0.05 significance level to conclude that the effect of arsenic on the log-odds of switching depends on distance.
The simpler additive model was preferred for interpretation, while acknowledging that a nonsignificant result does not establish the absence of an interaction.
In this model, the slope of arsenic on the log-odds
scale is
beta_arsenic + beta_interaction * dist100. A statistically
significant interaction suggests this slope varies with distance.
Even without a log-odds interaction, probability-scale
changes can differ because the logit transformation is nonlinear.
# Plot arsenic concentration against well switching
plot(
jitter(wells$arsenic, amount = 0.05),
jitter(wells$switch_binary, amount = 0.05),
xlab = "Arsenic concentration",
ylab = "Switched wells (0 = No, 1 = Yes)",
main = "Arsenic Concentration and Well Switching",
pch = 16,
col = rgb(0, 0, 0, 0.15),
ylim = c(-0.1, 1.1)
)
# Create a sequence of arsenic concentrations
new_wells <- data.frame(
arsenic = seq(
min(wells$arsenic),
max(wells$arsenic),
length.out = 200
),
dist100 = median(wells$dist100)
)
# Predict probabilities using the additive model
new_wells$probability <- predict(
well_add,
newdata = new_wells,
type = "response"
)
# Add fitted logistic regression curve
lines(
new_wells$arsenic,
new_wells$probability,
col = "blue",
lwd = 3
)
range(fitted(well_int))
## [1] 0.1014427 0.9825588
plot(fitted(well_int),residuals(well_int,type="deviance"),pch=16,
col=rgb(0,0,0,.15),xlab="Predicted switching probability",
ylab="Deviance residual")
abline(h=0,lty=2)
# Check possible nonlinear arsenic/distance effects using simple quadratic terms.
well_curved <- glm(switch ~ arsenic * dist100 + I(arsenic^2) + I(dist100^2),
family=binomial,data=wells)
anova(well_int,well_curved,test="Chisq")
## Analysis of Deviance Table
##
## Model: binomial, link: logit
##
## Response: switch_binary
##
## Terms added sequentially (first to last)
##
##
## Df Deviance Resid. Df Resid. Dev Pr(>Chi)
## NULL 3019 4118.1
## arsenic 1 109.426 3018 4008.7 < 2e-16 ***
## dist100 1 78.005 3017 3930.7 < 2e-16 ***
## arsenic:dist100 1 3.040 3016 3927.6 0.08124 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
My conclusion (complete after running): The interaction coefficient is ; its sign suggests that the arsenic log-odds slope [increases/decreases] as distance rises. The likelihood-ratio test comparing additive and interaction models gives \(\Delta D\) , df , p-value . At significance level 0.05, this [does/does not] support including the interaction. The probability curves are ___. This is observational association, not proof of causation.
# Explore the Bumpus dataset
str(bumpus)
## 'data.frame': 136 obs. of 11 variables:
## $ sex : int 1 1 1 1 1 1 1 1 1 1 ...
## $ survival : int 1 1 1 1 1 1 1 1 1 1 ...
## $ length : int 154 160 155 154 156 161 157 159 158 158 ...
## $ alar_extent: int 241 252 243 245 247 253 251 247 247 252 ...
## $ weight : num 24.5 26.9 26.9 24.3 24.1 26.5 24.6 24.2 23.6 26.2 ...
## $ head : num 31.2 30.8 30.6 31.7 31.5 31.8 31.1 31.4 29.8 32 ...
## $ humer : num 0.687 0.736 0.733 0.741 0.715 0.78 0.741 0.728 0.703 0.749 ...
## $ femur : num 0.668 0.709 0.704 0.688 0.706 0.743 0.736 0.718 0.673 0.739 ...
## $ tibio : num 1.02 1.18 1.15 1.15 1.13 ...
## $ skull : num 0.587 0.602 0.602 0.584 0.575 0.607 0.61 0.609 0.602 0.614 ...
## $ sternum : num 0.83 0.841 0.846 0.839 0.821 0.893 0.862 0.793 0.82 0.857 ...
# Count how many survived vs. did not survive
table(bumpus$survival)
##
## 0 1
## 64 72
# Summary statistics for weight
summary(bumpus$weight)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 22.60 24.57 25.55 25.52 26.50 31.00
# Check for missing values
colSums(is.na(bumpus))
## sex survival length alar_extent weight head
## 0 0 0 0 0 0
## humer femur tibio skull sternum
## 0 0 0 0 0
# Compare weights between survival groups
aggregate(weight ~ survival,
data = bumpus,
FUN = mean)
## survival weight
## 1 0 25.86094
## 2 1 25.22639
# Create boxplot of weight by survival
boxplot(weight ~ survival,
data = bumpus,
xlab = "Survival (0 = No, 1 = Yes)",
ylab = "Weight",
main = "Sparrow Weight by Survival Status")
### Exploratory analysis: Sparrow weight and survival
The Bumpus dataset contains 136 sparrows, with survival recorded as a binary response variable (0 = did not survive, 1 = survived). There were no missing values in the dataset.
Descriptive statistics showed that sparrows that did not survive had a higher mean weight (25.86) than those that survived (25.23). The boxplot also suggests that non-surviving birds tended to be heavier, although the weight distributions overlap considerably.
These exploratory results suggest a possible negative association between weight and survival. However, logistic regression is required to determine whether this relationship is statistically significant.
# Fit logistic regression model
bumpus_model <- glm(
survival ~ weight,
family = binomial(link = "logit"),
data = bumpus
)
# Display model results
summary(bumpus_model)
##
## Call:
## glm(formula = survival ~ weight, family = binomial(link = "logit"),
## data = bumpus)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 8.0456 3.2516 2.474 0.0133 *
## weight -0.3105 0.1272 -2.441 0.0146 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 188.07 on 135 degrees of freedom
## Residual deviance: 181.58 on 134 degrees of freedom
## AIC: 185.58
##
## Number of Fisher Scoring iterations: 4
A binomial logistic regression with a logit link was conducted to determine whether sparrow weight was associated with survival.
Weight was a statistically significant negative predictor of survival (β = −0.3105, SE = 0.1272, z = −2.441, p = 0.0146).
The odds ratio was approximately 0.733, indicating that each one-unit increase in weight was associated with a 26.7% decrease in the odds of survival.
The null deviance was 188.07 (135 df), while the residual deviance was 181.58 (134 df). Including weight reduced deviance by 6.49 with one degree of freedom, suggesting that weight improved model fit.
Overall, heavier sparrows had significantly lower estimated odds of survival. However, this analysis demonstrates an association rather than a causal relationship.
# Diagnostic plots for logistic regression
par(mfrow = c(2, 2))
plot(bumpus_model)
par(mfrow = c(1, 1))
# Original logistic regression
model_linear <- glm(
survival ~ weight,
family = binomial,
data = bumpus
)
# Model allowing a curved relationship
model_quadratic <- glm(
survival ~ weight + I(weight^2),
family = binomial,
data = bumpus
)
# Compare models
anova(model_linear, model_quadratic, test = "Chisq")
## Analysis of Deviance Table
##
## Model 1: survival ~ weight
## Model 2: survival ~ weight + I(weight^2)
## Resid. Df Resid. Dev Df Deviance Pr(>Chi)
## 1 134 181.58
## 2 133 180.41 1 1.1703 0.2794
Logistic regression assumes that continuous predictors have a linear relationship with the log-odds of the response.
To assess this assumption, I compared the original logistic regression model containing weight with a quadratic model containing both weight and weight².
Adding the quadratic term did not significantly improve model fit (ΔDeviance = 1.1703, df = 1, p = 0.2794).
Therefore, there was insufficient evidence of a quadratic relationship between weight and the log-odds of survival. The simpler linear-logit model was retained, although this test does not rule out all possible forms of nonlinearity.
# Calculate Cook's distance for each sparrow
cooks <- cooks.distance(bumpus_model)
# Plot Cook's distance
plot(cooks,
type = "h",
main = "Cook's Distance for Bumpus Logistic Regression",
xlab = "Sparrow observation number",
ylab = "Cook's distance")
# Display the five most influential observations
head(sort(cooks, decreasing = TRUE), 5)
## 132 124 112 21 22
## 0.04906835 0.04209333 0.03037512 0.02981301 0.02733324
# Reference threshold for further investigation
4 / nrow(bumpus)
## [1] 0.02941176
Cook’s distance was calculated to identify observations that may disproportionately influence the fitted logistic regression model.
Using the screening threshold of 4/n = 0.0294, four observations exceeded this value: observations 132, 124, 112 and 21.
The largest Cook’s distance was approximately 0.0491 (observation 132). These observations may warrant further investigation, but Cook’s distance alone does not establish that they are problematic.
No observations were removed from the analysis. The potential influence of these observations should be considered when interpreting the model results.
# Plot survival data (0 = died, 1 = survived)
plot(jitter(bumpus$weight),
jitter(bumpus$survival, amount = 0.04),
xlab = "Sparrow weight",
ylab = "Survival probability",
main = "Logistic Regression of Sparrow Survival",
pch = 16,
col = rgb(0, 0, 0, 0.3),
ylim = c(0, 1))
# Create an ordered sequence of weights
newdata <- data.frame(
weight = seq(min(bumpus$weight),
max(bumpus$weight),
length.out = 200)
)
# Predict survival probabilities
newdata$probability <- predict(
bumpus_model,
newdata = newdata,
type = "response"
)
# Add the logistic regression curve
lines(newdata$weight,
newdata$probability,
col = "blue",
lwd = 3)
The logistic regression plot illustrates the relationship between sparrow weight and predicted probability of survival. Individual observations are shown as points, with survival coded as 0 (did not survive) or 1 (survived). The fitted logistic regression curve represents the predicted probability of survival across the observed range of weights.
The predicted probability of survival decreases as sparrow weight increases, consistent with the negative regression coefficient (β = −0.3105, p = 0.0146).
Although individual observations show considerable variation, the model indicates a statistically significant negative association between weight and survival.
# Identify observations above the Cook's distance screening threshold
influential <- which(cooks > 4 / nrow(bumpus))
# Display their observation numbers
influential
## 21 112 124 132
## 21 112 124 132
# Fit sensitivity model excluding flagged observations
model_sensitivity <- glm(
survival ~ weight,
family = binomial,
data = bumpus[-influential, ]
)
# Compare weight estimates and p-values
summary(bumpus_model)$coefficients["weight", ]
## Estimate Std. Error z value Pr(>|z|)
## -0.31053148 0.12720813 -2.44112925 0.01464141
summary(model_sensitivity)$coefficients["weight", ]
## Estimate Std. Error z value Pr(>|z|)
## -0.452663680 0.143746506 -3.149041275 0.001638071
A sensitivity analysis was conducted to determine whether observations with Cook’s distances exceeding the screening threshold of 4/n substantially affected the logistic regression results.
Four observations (21, 112, 124 and 132) were temporarily excluded, and the logistic regression was refitted.
The weight coefficient changed from β = −0.3105 (p = 0.0146) in the original model to β = −0.4527 (p = 0.00164) in the sensitivity model.
The association between weight and survival remained negative and statistically significant after removing these observations. However, the change in coefficient magnitude suggests that the estimated strength of the association is somewhat sensitive to these observations.
Because there was no established reason to exclude these sparrows, the original model containing all 136 observations was retained for the primary analysis.
A binomial logistic regression was conducted to investigate whether sparrow weight was associated with survival.
The analysis identified a statistically significant negative relationship between weight and survival (β = −0.3105, SE = 0.1272, z = −2.441, p = 0.0146). The corresponding odds ratio was approximately 0.733, indicating a 26.7% decrease in the estimated odds of survival for each one-unit increase in weight.
Model diagnostics were examined. Adding a quadratic weight term did not significantly improve model fit (ΔDeviance = 1.1703, df = 1, p = 0.2794), providing no strong evidence of quadratic nonlinearity on the logit scale.
Four observations exceeded the Cook’s distance screening threshold. A sensitivity analysis excluding these observations produced a stronger negative weight coefficient (β = −0.4527, p = 0.00164), indicating that the overall direction of the association was maintained, although its magnitude changed.
Overall, heavier sparrows had lower estimated odds of survival in this dataset. These findings support an association between weight and survival but do not establish a causal relationship. Independence of observations remains an assumption that must be evaluated using information about the sampling design.
To investigate whether the association between arsenic concentration and well switching depended on distance to a safe well, two binomial logistic regression models were fitted.
The additive model included arsenic concentration and distance as explanatory variables, while the interaction model additionally included an arsenic × distance interaction term.
The models were compared using a likelihood-ratio test. The additive model had a residual deviance of 3930.7, while the interaction model had a residual deviance of 3927.6.
Including the interaction reduced deviance by 3.0399 with one degree of freedom. However, this improvement was not statistically significant (χ²(1) = 3.0399, p = 0.08124).
Therefore, there was insufficient evidence at the 0.05 significance level that the association between arsenic concentration and the log-odds of switching wells differed according to distance.
The simpler additive model was retained for interpretation, although a nonsignificant interaction test does not prove the absence of an interaction.