Objectives

The objectives of this problem set is to orient you to a number of activities in R. And to conduct a thoughtful exercise in appreciating the importance of data visualization. For each question create a code chunk or text response that completes/answers the activity or question requested. Finally, upon completion name your final output .html file as: YourName_ANLY512-Section-Year-Semester.html and upload it to the “Problem Set 2” assignmenet on Moodle.

Questions

  1. Anscombes quartet is a set of 4 \(x,y\) data sets that were published by Francis Anscombe in a 1973 paper Graphs in statistical analysis. For this first question load the anscombe data that is part of the library(datasets) in R. And assign that data to a new object called data.
data <- datasets::anscombe
  1. Summarise the data by calculating the mean, variance, for each column and the correlation between each pair (eg. x1 and y1, x2 and y2, etc) (Hint: use the fBasics() package!)
library(fBasics)
## Loading required package: timeDate
## Loading required package: timeSeries
colMeans(data) #calculate the mean of each column, or colAvgs(data)
##       x1       x2       x3       x4       y1       y2       y3       y4 
## 9.000000 9.000000 9.000000 9.000000 7.500909 7.500909 7.500000 7.500909
colVars(data) #calculate the variance of each column
##        x1        x2        x3        x4        y1        y2        y3 
## 11.000000 11.000000 11.000000 11.000000  4.127269  4.127629  4.122620 
##        y4 
##  4.123249
for (x in 1:4) {
  cat(paste0("Variables:\n x",x,", y",x,"\n"))
  print(correlationTest(data[,x],data[,x+4]))
  cat("-----------------------------------\n")
} #calculate the correlation between each pair
## Variables:
##  x1, y1
## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8164
##   STATISTIC:
##     t: 4.2415
##   P VALUE:
##     Alternative Two-Sided: 0.00217 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001085 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4244, 0.9507
##          Less: -1, 0.9388
##       Greater: 0.5113, 1
## 
## Description:
##  Sat Nov 24 21:59:53 2018
## 
## -----------------------------------
## Variables:
##  x2, y2
## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8162
##   STATISTIC:
##     t: 4.2386
##   P VALUE:
##     Alternative Two-Sided: 0.002179 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001089 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4239, 0.9506
##          Less: -1, 0.9387
##       Greater: 0.5109, 1
## 
## Description:
##  Sat Nov 24 21:59:53 2018
## 
## -----------------------------------
## Variables:
##  x3, y3
## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8163
##   STATISTIC:
##     t: 4.2394
##   P VALUE:
##     Alternative Two-Sided: 0.002176 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001088 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4241, 0.9507
##          Less: -1, 0.9387
##       Greater: 0.511, 1
## 
## Description:
##  Sat Nov 24 21:59:53 2018
## 
## -----------------------------------
## Variables:
##  x4, y4
## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8165
##   STATISTIC:
##     t: 4.243
##   P VALUE:
##     Alternative Two-Sided: 0.002165 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001082 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4246, 0.9507
##          Less: -1, 0.9388
##       Greater: 0.5115, 1
## 
## Description:
##  Sat Nov 24 21:59:53 2018
## 
## -----------------------------------
  1. Create scatter plots for each \(x, y\) pair of data.
# There are 4 scatter plots.
for (x in 1:4) {
  plot(data[,x],data[,x+4],xlab=colnames(data)[x],ylab=colnames(data)[x+4],main=paste0("Scatter Plot of ",colnames(data)[x]," and ",colnames(data)[x+4]))
}

  1. Now change the symbols on the scatter plots to solid circles and plot them together as a 4 panel graphic
par(mfrow=c(2,2)) # 4 panel graph
for (x in 1:4) {
  plot(pch=20,data[,x],data[,x+4],xlab=colnames(data)[x],ylab=colnames(data)[x+4],main=paste0("Scatter Plot of ",colnames(data)[x]," and ",colnames(data)[x+4]))
}

  1. Now fit a linear model to each data set using the lm() function.
for (x in 1:4) {
  cat(paste0("Independent Variable: x",x,"\nDependent Variable: y",x,"\n"))
  print(summary(lm(data[,x+4]~data[,x])))
  cat("--------------------------------------------------------------\n")
}
## Independent Variable: x1
## Dependent Variable: y1
## 
## Call:
## lm(formula = data[, x + 4] ~ data[, x])
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.92127 -0.45577 -0.04136  0.70941  1.83882 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)   3.0001     1.1247   2.667  0.02573 * 
## data[, x]     0.5001     0.1179   4.241  0.00217 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.237 on 9 degrees of freedom
## Multiple R-squared:  0.6665, Adjusted R-squared:  0.6295 
## F-statistic: 17.99 on 1 and 9 DF,  p-value: 0.00217
## 
## --------------------------------------------------------------
## Independent Variable: x2
## Dependent Variable: y2
## 
## Call:
## lm(formula = data[, x + 4] ~ data[, x])
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.9009 -0.7609  0.1291  0.9491  1.2691 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)    3.001      1.125   2.667  0.02576 * 
## data[, x]      0.500      0.118   4.239  0.00218 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.237 on 9 degrees of freedom
## Multiple R-squared:  0.6662, Adjusted R-squared:  0.6292 
## F-statistic: 17.97 on 1 and 9 DF,  p-value: 0.002179
## 
## --------------------------------------------------------------
## Independent Variable: x3
## Dependent Variable: y3
## 
## Call:
## lm(formula = data[, x + 4] ~ data[, x])
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.1586 -0.6146 -0.2303  0.1540  3.2411 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)   3.0025     1.1245   2.670  0.02562 * 
## data[, x]     0.4997     0.1179   4.239  0.00218 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.236 on 9 degrees of freedom
## Multiple R-squared:  0.6663, Adjusted R-squared:  0.6292 
## F-statistic: 17.97 on 1 and 9 DF,  p-value: 0.002176
## 
## --------------------------------------------------------------
## Independent Variable: x4
## Dependent Variable: y4
## 
## Call:
## lm(formula = data[, x + 4] ~ data[, x])
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -1.751 -0.831  0.000  0.809  1.839 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)   3.0017     1.1239   2.671  0.02559 * 
## data[, x]     0.4999     0.1178   4.243  0.00216 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.236 on 9 degrees of freedom
## Multiple R-squared:  0.6667, Adjusted R-squared:  0.6297 
## F-statistic:    18 on 1 and 9 DF,  p-value: 0.002165
## 
## --------------------------------------------------------------
  1. Now combine the last two tasks. Create a four panel scatter plot matrix that has both the data points and the regression lines. (hint: the model objects will carry over chunks!)
par(mfrow=c(2,2)) # 4 panel graph
for (x in 1:4) {
  plot(data[,x],data[,x+4],xlab=colnames(data)[x],ylab=colnames(data)[x+4],main=paste0("Scatter Plot of ",colnames(data)[x]," and ",colnames(data)[x+4]),pch=20)
  abline(lm(data[,x+4]~data[,x]),col="red")
}

  1. Now compare the model fits for each model object.
for (x in 1:4) {
  cat(paste0("\nIndependent Variable: x",x,", Dependent Variable: y",x,", "))
  print(anova(lm(data[,x+4]~data[,x])))
}

Independent Variable: x1, Dependent Variable: y1, Analysis of Variance Table

Response: data[, x + 4] Df Sum Sq Mean Sq F value Pr(>F)
data[, x] 1 27.510 27.5100 17.99 0.00217 ** Residuals 9 13.763 1.5292
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ‘’ 1

Independent Variable: x2, Dependent Variable: y2, Analysis of Variance Table

Response: data[, x + 4] Df Sum Sq Mean Sq F value Pr(>F)
data[, x] 1 27.500 27.5000 17.966 0.002179 ** Residuals 9 13.776 1.5307
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ‘’ 1

Independent Variable: x3, Dependent Variable: y3, Analysis of Variance Table

Response: data[, x + 4] Df Sum Sq Mean Sq F value Pr(>F)
data[, x] 1 27.470 27.4700 17.972 0.002176 ** Residuals 9 13.756 1.5285
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ‘’ 1

Independent Variable: x4, Dependent Variable: y4, Analysis of Variance Table

Response: data[, x + 4] Df Sum Sq Mean Sq F value Pr(>F)
data[, x] 1 27.490 27.4900 18.003 0.002165 ** Residuals 9 13.742 1.5269
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ‘’ 1

  1. In text, summarize the lesson of Anscombe’s Quartet and what it says about the value of data visualization.

From Question 2, we see four groups’ x have the same mean, i.e., 9, and the same variance, i.e., 11.

From Question 2, we see four groups’ y have the similar mean, i.e., 7.50, and the similar variance, i.e., 4.13.

From Question 2, we see four groups’ x and y have the similar correlation, i.e., 0.816.

From Question 5, we see four groups’ linear regression model is similar, i.e., y=3+0.5x with Adjusted R-squared 0.629.

From the perspectives of statistical summary, four groups seem to be identical.

However, from Question 3 & 4, we see four graphs are different. Group 1 may follow a linear relationship with seasonality. Group 2 rather follow a curve relationship. Group 3 follows a good linear relationship except one outlier. Group 4 has constant x values except one outlier.

Without the visualization, wrong conclusions can be obtained in this case. Hence, Anscombe’s Quartet values the importance of data visualization.