ANLY512 Yjh LateSummer17

Questions

Anscombes quartet is a set of 4 \(x,y\) data sets that were published by Francis Anscombe in a 1973 paper Graphs in statistical analysis. For this first question load the anscombe data that is part of the library(datasets) in R. And assign that data to a new object called data.

data <- anscombe

Summarise the data by calculating the mean, variance, for each column and the correlation between each pair (eg. x1 and y1, x2 and y2, etc) (Hint: use the fBasics() package!)

library(fBasics)

## Loading required package: timeDate

## Loading required package: timeSeries

##

## Rmetrics Package fBasics

## Analysing Markets and calculating Basic Statistics

## Copyright (C) 2005-2014 Rmetrics Association Zurich

## Educational Software for Financial Engineering and Computational Science

## Rmetrics is free software and comes with ABSOLUTELY NO WARRANTY.

## https://www.rmetrics.org --- Mail to: info@rmetrics.org

??fBasics

summary.data.frame(data)

##        x1             x2             x3             x4    
##  Min.   : 4.0   Min.   : 4.0   Min.   : 4.0   Min.   : 8  
##  1st Qu.: 6.5   1st Qu.: 6.5   1st Qu.: 6.5   1st Qu.: 8  
##  Median : 9.0   Median : 9.0   Median : 9.0   Median : 8  
##  Mean   : 9.0   Mean   : 9.0   Mean   : 9.0   Mean   : 9  
##  3rd Qu.:11.5   3rd Qu.:11.5   3rd Qu.:11.5   3rd Qu.: 8  
##  Max.   :14.0   Max.   :14.0   Max.   :14.0   Max.   :19  
##        y1               y2              y3              y4        
##  Min.   : 4.260   Min.   :3.100   Min.   : 5.39   Min.   : 5.250  
##  1st Qu.: 6.315   1st Qu.:6.695   1st Qu.: 6.25   1st Qu.: 6.170  
##  Median : 7.580   Median :8.140   Median : 7.11   Median : 7.040  
##  Mean   : 7.501   Mean   :7.501   Mean   : 7.50   Mean   : 7.501  
##  3rd Qu.: 8.570   3rd Qu.:8.950   3rd Qu.: 7.98   3rd Qu.: 8.190  
##  Max.   :10.840   Max.   :9.260   Max.   :12.74   Max.   :12.500

#calculating the mean
colMeans(data)

##       x1       x2       x3       x4       y1       y2       y3       y4 
## 9.000000 9.000000 9.000000 9.000000 7.500909 7.500909 7.500000 7.500909

#calculating the variance
colVars(data)

##        x1        x2        x3        x4        y1        y2        y3 
## 11.000000 11.000000 11.000000 11.000000  4.127269  4.127629  4.122620 
##        y4 
##  4.123249

#calculating the correlation between each pair
correlationTest(data$x1, data$y1, method = c("pearson"),
                title = NULL, description = NULL)

## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8164
##   STATISTIC:
##     t: 4.2415
##   P VALUE:
##     Alternative Two-Sided: 0.00217 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001085 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4244, 0.9507
##          Less: -1, 0.9388
##       Greater: 0.5113, 1
## 
## Description:
##  Thu Jul 27 17:34:16 2017

correlationTest(data$x2, data$y2, method = c("pearson"),
                title = NULL, description = NULL)

## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8162
##   STATISTIC:
##     t: 4.2386
##   P VALUE:
##     Alternative Two-Sided: 0.002179 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001089 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4239, 0.9506
##          Less: -1, 0.9387
##       Greater: 0.5109, 1
## 
## Description:
##  Thu Jul 27 17:34:16 2017

correlationTest(data$x3, data$y3, method = c("pearson"),
                title = NULL, description = NULL)

## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8163
##   STATISTIC:
##     t: 4.2394
##   P VALUE:
##     Alternative Two-Sided: 0.002176 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001088 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4241, 0.9507
##          Less: -1, 0.9387
##       Greater: 0.511, 1
## 
## Description:
##  Thu Jul 27 17:34:16 2017

correlationTest(data$x4, data$y4, method = c("pearson"),
                title = NULL, description = NULL)

## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8165
##   STATISTIC:
##     t: 4.243
##   P VALUE:
##     Alternative Two-Sided: 0.002165 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001082 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4246, 0.9507
##          Less: -1, 0.9388
##       Greater: 0.5115, 1
## 
## Description:
##  Thu Jul 27 17:34:16 2017

Create scatter plots for each \(x, y\) pair of data.

correlationTest(data$x1, data$y1, method = c("pearson"),
                title = NULL, description = NULL)

## 
## Title:
##  Pearson's Correlation Test
## 
## Test Results:
##   PARAMETER:
##     Degrees of Freedom: 9
##   SAMPLE ESTIMATES:
##     Correlation: 0.8164
##   STATISTIC:
##     t: 4.2415
##   P VALUE:
##     Alternative Two-Sided: 0.00217 
##     Alternative      Less: 0.9989 
##     Alternative   Greater: 0.001085 
##   CONFIDENCE INTERVAL:
##     Two-Sided: 0.4244, 0.9507
##          Less: -1, 0.9388
##       Greater: 0.5113, 1
## 
## Description:
##  Thu Jul 27 17:34:16 2017

plot(data$x1, data$y1, main="Scatterplot of X1 Y1 Pair", 
     xlab="X1 Value ", ylab="Y1 Value ",
     abline(lm(data$x1~data$y1), col="red"))

plot(data$x2, data$y2, main="Scatterplot of X2 Y2 Pair", 
     xlab="X2 Value ", ylab="Y2 Value ",
     abline(lm(data$x2~data$y2), col="Blue"))

plot(data$x3, data$y3, main="Scatterplot of X3 Y3 Pair", 
     xlab="X3 Value ", ylab="Y3 Value ",
     abline(lm(data$x3~data$y3), col="Pink"))

plot(data$x4, data$y4, main="Scatterplot of X4 Y4 Pair", 
     xlab="X4 Value ", ylab="Y4 Value ",
     abline(lm(data$x4~data$y4), col="Green"))

Now change the symbols on the scatter plots to solid circles and plot them together as a 4 panel graphic

par(mfrow=c(2,2))
plot(data$x1,data$y1, main="Scatterplot x1 ~ y1",pch=19) 
plot(data$x2,data$y2, main="Scatterplot x2 ~ y2",pch=19) 
plot(data$x3,data$y3, main="Scatterplot x3 ~ y3",pch=19) 
plot(data$x4,data$y4, main="Scatterplot x4 ~ y4",pch=19)

Now fit a linear model to each data set using the lm() function.

#Please check the line in the chart of Question 3 

fit1<-lm(data$y1~data$x1)
fit2<-lm(data$y2~data$x2)
fit3<-lm(data$y3~data$x3)
fit4<-lm(data$y4~data$x4)

Now combine the last two tasks. Create a four panel scatter plot matrix that has both the data points and the regression lines. (hint: the model objects will carry over chunks!)

par(mfrow=c(2,2))
plot(data$x1,data$y1, main="Scatterplot x1 ~ y1",pch=19) 
abline(fit1, col="red")
plot(data$x2,data$y2, main="Scatterplot x2 ~ y2",pch=19) 
abline(fit2, col="blue") 
plot(data$x3,data$y3, main="Scatterplot x3 ~ y3",pch=19) 
abline(fit3, col="green") 
plot(data$x4,data$y4, main="Scatterplot x4 ~ y4",pch=19) 
abline(fit4, col="pink")

Now compare the model fits for each model object.

anova(fit1)

Analysis of Variance Table

Response: data\(y1 Df Sum Sq Mean Sq F value Pr(>F) data\)x1 1 27.510 27.5100 17.99 0.00217 ** Residuals 9 13.763 1.5292
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ‘’ 1

anova(fit2)

Analysis of Variance Table

Response: data\(y2 Df Sum Sq Mean Sq F value Pr(>F) data\)x2 1 27.500 27.5000 17.966 0.002179 ** Residuals 9 13.776 1.5307
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ‘’ 1

anova(fit3)

Analysis of Variance Table

Response: data\(y3 Df Sum Sq Mean Sq F value Pr(>F) data\)x3 1 27.470 27.4700 17.972 0.002176 ** Residuals 9 13.756 1.5285
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ‘’ 1

anova(fit4)

Analysis of Variance Table

Response: data\(y4 Df Sum Sq Mean Sq F value Pr(>F) data\)x4 1 27.490 27.4900 18.003 0.002165 ** Residuals 9 13.742 1.5269
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ‘’ 1

In text, summarize the lesson of Anscombe’s Quartet and what it says about the value of data visualization. #According to the scattor plots of Anscombe’s Quartet dataset, I cannot find clear combination of variable pair. #When we check the 4 panels scatter plot, we find that each of the chats display different patern, skewness, and dispersion. #In the plot of X1-Y1, we can see the x1-y1 naturally(randamly) spread and can find out the linear regression area. #However,in the plot of X2-Y2 and the plot of X3-Y3, the dots are just spread right on the curve and line. The accuracy of the data is doubtful. #In the last plot X4-Y4,the x value remain the same whereas the value of Y varies. We cannot tell any negative of positive relationship between X4 and Y4. #Due to lacking of the reliable data, we cannot know too much from the lesson of Anscombe’ Quartet.

ANLY512 Yjh LateSummer17 - Problem Set 2

Anscombe’s quartet

Yun-Ju, Claudia Huang

2017-07-27

Questions