Simpson’s Paradox - Correlations and Slopes

This is an R Markdown document giving an example of Simpson’s Paradox. Looking first at correlation and second at the estimated slopes in regression lines.

The example considers book prices and the number of pages in books. The confounding variable to consider is type of book, either hardcover or paperback.

And some examples of using ggplot2 are included.

Here are some links to get started:

• wikipedia about correlation
• a post on the Analysis with Programming website about the Theory of Linear Least Squares, which gives an introduction to the linear regression and how it is fitted to data.
  
• wikipedia about Simpson’s Paradox
• Cookbook for R

The data.

### synthetic data

# Consider book price (y) by number of pages (x)

z = c("hardcover","hardcover",
      "hardcover","hardcover",
      "paperback", "paperback","paperback", 
      "paperback")

x1 = c( 150, 225, 342, 185)
y1 = c( 27.43, 48.76, 50.25, 32.01 )

x2 = c( 475, 834, 1020, 790)
y2 = c( 10.00, 15.73, 20.00, 17.89 )

x = c(x1, x2)
y = c(y1, y2)

Plotting a scatterplot.

plot(x,y)

Compute the correlation coefficient \(r\) to see the effect of Simpson’s Paradox. The overall correlation is negative and the correlations, taking into consideration the confounding variable, type of book, the correlations change to positive, for both hardcover and paperback books.

# correlation

cor(y, x)

cor(y1, x1)
cor(y2, x2)

Now compute the slopes in the linear regressions to see the same effect.

Overall, the slope is negative.

# linear regression

lm(y ~ x)

For the hardcover books, the slope is positive.

# linear regression

lm(y1 ~ x1)

For the paperback books, the slope is positive.

# linear regression

lm(y2 ~ x2)

Now let’s visualize Simpon’s Paradox in the scatterplot. First by plotting the names of each point in the scatterplot.

plot(x,y, type="n")
text(x,y,z, cex=0.8)

Now plot the linear regression model ignoring the type of book.

# fitted linear regression

plot(x,y)

model0 <- lm(y ~ x)
abline(model0)

Now plot the linear regression models using the type of book.

# fitted linear regression

plot(x,y)

text(200,40,"hardcover")
model1 <- lm(y1 ~ x1)
abline(model1)

text(600,15,"paperback")
model2 <- lm(y2 ~ x2)
abline(model2)

Now using ggplot2. A very nice tutorial is available from Cookbook R.

library(ggplot2)

# put the data into a data.frame
#   cond = type of book
#   xvar = x
#   yvar = y

dat <- data.frame(cond = z, xvar = x, yvar = y)

Plot the overall data.

ggplot(dat, aes(x=xvar, y=yvar)) + geom_point(shape=1)      # Use hollow circles

ggplot(dat, aes(x=xvar, y=yvar)) +
    geom_point(shape=1) +    # Use hollow circles
    geom_smooth(method=lm)   # Add linear regression line 
                             #  (by default includes 95% confidence region)

ggplot(dat, aes(x=xvar, y=yvar)) +
    geom_point(shape=1) +    # Use hollow circles
    geom_smooth(method=lm,   # Add linear regression line
                se=FALSE)    # Don't add shaded confidence region

Now plot the regression lines for each type. See Simpson’s Paradox.

ggplot(dat, aes(x=xvar, y=yvar, color=cond)) + geom_point(shape=1)

# Same, but with different colors and add regression lines
ggplot(dat, aes(x=xvar, y=yvar, color=cond)) +
    geom_point(shape=1) +
    scale_colour_hue(l=50) + # Use a slightly darker palette than normal
    geom_smooth(method=lm,   # Add linear regression lines
                se=FALSE)    # Don't add shaded confidence region

Plot using different shapes.

# Set shape by cond
ggplot(dat, aes(x=xvar, y=yvar, shape=cond)) + geom_point()

# Same, but with different shapes
ggplot(dat, aes(x=xvar, y=yvar, shape=cond)) + geom_point() +
    scale_shape_manual(values=c(1,2))  # Use a hollow circle and triangle