Linear Regression in Practice

Fitting, assumptions and interpretation with birth weight data

The question

Which maternal and pregnancy factors are associated with infant birth weight, and by how much?

  • Outcome: birth weight in grams (continuous)
  • Data: 500 simulated singleton births
  • Method: linear regression, simple first and then adjusted for other factors

The data

Variable Type Description
birth_weight_g outcome Birth weight (g)
gestational_age_wks numeric Weeks of gestation
maternal_age_yrs numeric Mother’s age
maternal_bmi numeric BMI (kg/m²)
smoker binary Smoked in pregnancy
anc_visits count Antenatal visits
residence binary Urban or rural
infant_sex binary Female or male

The workflow

  1. Explore the data: types, ranges, missingness, plots
  2. Fit a simple model, then a multiple model
  3. Interpret coefficients, intervals and fit
  4. Check the assumptions with residual diagnostics
  5. Fix problems, or report the limitations
  6. Report with estimates and confidence intervals

Look before you model

Figure 1: ?(caption)

Roughly linear, and the smoker line sits below the non-smoker line.

Simple linear regression

The model

\[ \text{BirthWeight}_i = \beta_0 + \beta_1\,\text{GestAge}_i + \varepsilon_i, \qquad \varepsilon_i \sim N(0, \sigma^2) \]

  • \(\beta_0\): intercept, the mean outcome when the predictor is 0
  • \(\beta_1\): slope, the mean change in the outcome per one-unit increase in the predictor
  • \(\varepsilon_i\): random error, assumed independent with constant variance
  • Estimated by ordinary least squares: minimise \(\sum (y_i - \hat y_i)^2\)

Fitting it in R

m_simple <- lm(birth_weight_g ~ gestational_age_wks, data = bw)
Term Estimate SE t p CI low CI high
(Intercept) -2076.9 314.7 -6.6 <0.001 -2695.2 -1458.6
gestational_age_wks 135.7 8.1 16.7 <0.001 119.7 151.6

Reading the simple model

  • Slope: each extra week of gestation is associated with 136 g higher birth weight (95% CI 120 to 152)
  • Intercept: -2077 g at 0 weeks, an extrapolation with no real meaning. Centre the predictor (for example at 39 weeks) to make it interpretable
  • \(R^2\) = 0.36: gestational age alone explains about 36% of the variation
  • Residual SE = 325 g: the typical distance of a baby’s weight from the fitted line

Multiple linear regression

Why adjust?

  • Several factors act on birth weight at once
  • Gestational age is a likely confounder of the other associations
  • A multiple model estimates each effect holding the others constant
  • Coefficients can differ from those of simple models, and that is the point

Fitting the multiple model

m_multi <- lm(
  birth_weight_g ~ gestational_age_wks + maternal_age_yrs + maternal_bmi +
    smoker + anc_visits + residence + infant_sex,
  data = bw
)

Factors use their first level as the reference: no, urban, female.

Coefficients

Term Estimate SE t p CI low CI high
(Intercept) -2193.2 316.6 -6.9 < 0.001 -2815.3 -1571.1
gestational_age_wks 129.7 7.7 16.9 < 0.001 114.6 144.8
maternal_age_yrs 1.8 2.5 0.7 0.47629 -3.1 6.7
maternal_bmi 10.6 3.8 2.8 0.00571 3.1 18.2
smokeryes -256.3 44.1 -5.8 < 0.001 -343.1 -169.6
anc_visits 24.3 6.7 3.6 < 0.001 11.1 37.6
residencerural -100.6 27.9 -3.6 < 0.001 -155.5 -45.7
infant_sexmale 49.3 27.6 1.8 0.07505 -5.0 103.6

Interpreting the coefficients

  • Gestational age: 130 g per extra week (95% CI 115 to 145)
  • Smoking: babies of smokers are 256 g lighter than those of non-smokers (95% CI -343 to -170)
  • Antenatal visits: 24.3 g per additional visit
  • Rural residence: -101 g relative to urban
  • Maternal age: CI (-3.1 to 6.7) includes zero, so no clear association after adjustment
  • Infant sex: CI (-5 to 104) also includes zero

Reading categorical coefficients

Rule of thumb

A binary-factor coefficient is the difference in mean outcome versus the reference group, adjusted for everything else in the model.

For smokeryes: smokers minus non-smokers, at the same gestational age, BMI, age, visits, residence and infant sex.

How well does the model fit?

R² Adjusted R² Residual SE (g) F p (F-test)
0.442 0.434 305.71 55.614 0
  • Adjusted \(R^2\) penalises extra predictors, so use it to compare models of different size
  • Residual SE is in grams, which makes it easy to judge in practice
  • The F-test asks whether any slope differs from zero. It rarely tells you more than the individual terms

Checking the assumptions

The LINE assumptions

Assumption Check
L Linearity Residuals vs fitted; partial residual plots
I Independent errors Study design
N Normal residuals Q-Q plot
E Equal variance Scale-location plot; Breusch-Pagan
No collinearity Variance inflation factors
No unduly influential points Cook’s distance

Note: normality applies to the residuals, not to the outcome itself.

Linearity

Figure 2: ?(caption)

A flat, structureless band around zero supports linearity. A curve suggests a non-linear effect.

Independence

  • Cannot be proven from the residuals; it comes from the design
  • Here, one infant per mother and no clustering, so it is reasonable
  • Violated by repeated measures, twins or siblings, and clustering within clinics or villages
  • Remedy: mixed-effects models or cluster-robust standard errors

Normality of residuals

Figure 3: ?(caption)

Points should follow the line. Mild tail departures are usually harmless, and with a large sample the coefficient CIs are robust to moderate non-normality.

Constant variance

Figure 4: ?(caption)

Flat smoother is good. Breusch-Pagan test: p = 0.93. The null hypothesis is constant variance, so a small p would signal a problem.

Multicollinearity

Predictor VIF
gestational_age_wks 1.02
maternal_age_yrs 1.02
maternal_bmi 1.01
smoker 1.02
anc_visits 1.01
residence 1.03
infant_sex 1.01
  • VIF near 1 is ideal; above 5 deserves attention; above 10 is serious
  • Largest VIF here: 1.03
  • High VIFs inflate standard errors and destabilise individual coefficients

Influential observations

Figure 5: ?(caption)

The dashed line is the 4/n screening rule, not a verdict. Largest Cook’s distance here: 0.037. Investigate flagged points, then run a sensitivity analysis without them.

All four classic plots in one view

Figure 6: ?(caption)

When assumptions fail

Problem Typical remedy
Non-linearity Polynomial or spline terms; transform the predictor
Heteroscedasticity Log-transform the outcome; robust (HC3) SEs; weighted least squares
Skewed or non-normal residuals Transform the outcome; GLM; bootstrap CIs
Correlated errors Mixed-effects model; cluster-robust SEs; GEE
Multicollinearity Drop or combine variables; ridge regression
Influential points Check the data; sensitivity analysis; robust regression
Binary or count outcome Use logistic or Poisson regression instead

Using the model

Prediction

GA (wks) Smoker Fitted Mean CI Individual PI
39 no 3268 3219 to 3317 2666 to 3871
39 yes 3012 2925 to 3099 2405 to 3619
36 no 2879 2816 to 2942 2275 to 3483
  • Confidence interval: uncertainty about the average weight for such mothers
  • Prediction interval: uncertainty about one new baby, and much wider
  • Predict only inside the range of the observed data

Reporting results

We fitted a multiple linear regression of birth weight (g) on gestational age, maternal age, BMI, smoking, antenatal visits, residence and infant sex (n = 500). After adjustment, each additional week of gestation was associated with 130 g higher birth weight (95% CI 115 to 145), and maternal smoking with 256 g lower birth weight (95% CI -343 to -170). The model explained 43% of the variance (adjusted \(R^2\) = 0.43). Residual diagnostics were examined graphically and with formal tests.

Common mistakes

  • Saying “causes” when the design supports only “is associated with”
  • Dropping “holding other variables constant”
  • Treating a non-significant result as proof of no effect, instead of reading the CI
  • Comparing raw coefficients across predictors with different units
  • Relying on normality tests alone and skipping the plots
  • Extrapolating beyond the range of the data

Take-home messages

  1. Explore first, then model
  2. Interpret coefficients as adjusted differences, with confidence intervals
  3. Diagnose the residuals, not the raw outcome
  4. Plots first, formal tests second
  5. When assumptions fail, there is almost always a remedy
  6. Report what you checked and what you found

Thank you

Questions and discussion

Timothy | MozBioMed AI

Linear regression: worked example