Lurking Variables

Harold Nelson

2026-11-09

Setup

library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.2.1     ✔ readr     2.2.0
## ✔ forcats   1.0.1     ✔ stringr   1.6.0
## ✔ ggplot2   4.0.3     ✔ tibble    3.3.1
## ✔ lubridate 1.9.5     ✔ tidyr     1.3.2
## ✔ purrr     1.2.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(moderndive)

Create Two Independent Series

Create x and y as vectors of length 100 by sampling from normal distributions with different means and standard deviations.

Note tha x and y are totally independent. Run a linear model to verify.

Solution

x = rnorm(100,mean = 5, sd = 1)

y = rnorm(100,mean = 15, sd = 1)

xy_mod = lm(y~x)
summary(xy_mod)
## 
## Call:
## lm(formula = y ~ x)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -2.7310 -0.6331 -0.0601  0.9430  2.7985 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  14.6703     0.5753  25.500   <2e-16 ***
## x             0.0727     0.1134   0.641    0.523    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.152 on 98 degrees of freedom
## Multiple R-squared:  0.004178,   Adjusted R-squared:  -0.005983 
## F-statistic: 0.4112 on 1 and 98 DF,  p-value: 0.5229

Partial Sums

Create sumx and sumy as the partial sums of x and y. Alos create time with the integers 1 to 100.

Solution

sumx = rep(0,100)
sumx[1] = x[1]
for(i in 2:100){
  sumx[i] = sumx[i - 1] + x[i]
}

sumy = rep(0,100)
sumy[1] = y[1]
for(i in 2:100){
  sumy[i] = sumy[i - 1] + y[i]
}

time = 1:100

Dependence??

Run a linear model to see if sumx and sumy are correlated.

Solution

sums_mod = lm(sumy~sumx)
summary(sums_mod)
## 
## Call:
## lm(formula = sumy ~ sumx)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -12.9610  -5.8698  -0.8582   4.3742  19.0973 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -9.162638   1.445804  -6.337 7.15e-09 ***
## sumx         3.049587   0.004994 610.624  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 7.087 on 98 degrees of freedom
## Multiple R-squared:  0.9997, Adjusted R-squared:  0.9997 
## F-statistic: 3.729e+05 on 1 and 98 DF,  p-value: < 2.2e-16

Time Dendence

Create models to show that sumx and sumy both depend on time.

Solution

time_x = lm(sumx~time)
summary(time_x)
## 
## Call:
## lm(formula = sumx ~ time)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -6.7381 -1.5319  0.3256  2.0080  3.9506 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 4.094323   0.475288   8.614 1.22e-13 ***
## time        4.915512   0.008171 601.582  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.359 on 98 degrees of freedom
## Multiple R-squared:  0.9997, Adjusted R-squared:  0.9997 
## F-statistic: 3.619e+05 on 1 and 98 DF,  p-value: < 2.2e-16
time_y = lm(sumy~time)
summary(time_y)
## 
## Call:
## lm(formula = sumy ~ time)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.1230 -1.3571  0.1013  0.9179  4.7260 
## 
## Coefficients:
##              Estimate Std. Error  t value Pr(>|t|)    
## (Intercept)  3.127216   0.345965    9.039 1.47e-14 ***
## time        14.994165   0.005948 2520.999  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.717 on 98 degrees of freedom
## Multiple R-squared:      1,  Adjusted R-squared:      1 
## F-statistic: 6.355e+06 on 1 and 98 DF,  p-value: < 2.2e-16

Moral of the Story

When two variables, A and B, are both highly correlated with a third variable C, A and B will be highly correlated with each other statistically,

This is a real problem in economics because many economic time series are highly correlated with time.

Random Controlled Trials (RCT)

This is a type of study designed to eliminate the possibility of a lurking variable.

Here’s how it would be used to determine the efficacy of an experimental drug.

  1. A group of subjects is selected on the basis of random choice from the target poppulation.

  2. The group is divided into two subgroups called the treatment group and the control group using some random selection device, like flipping a coin.

  3. The subjects in the treatment group are given the real drug. The subjects in the control group are given a placebo with no real drug content. No subject is aware of their status as treatment or control.

  4. The difference in outcome between the treatments and controls measures the efficacy of the drug,

Observational Studies

These are studies without the benefit of random selection. The danger of lurking variables is very high.

Example 1

A study based on birth records and the exposure of the mother during pregnancy to air pollution reveals that air pollution is associated with premature birth. The study concludes that air pollution makes premature birth more likely.

Identify a likely lurking variable.

Solution

Low income causes people to live in undesirable areas. It also causes reduced access to medical care, such as prenatal visits.

Example 2

A survey asks mothers 2 questions.

  1. Did you take Tylenol while you were pregnant?

  2. Does your child have autism?

The study concludes that Tylenol causes autism.

Identify a potential lurking variable.

Solution

Why does a person take Tylenol?

This could be the underlying cause of autism.