Datasaurus and the ABC principle - Assignment 2

[A]

# define the standardized score
findZscore <- function(x, mean_x, sd_x) {
  (x - mean_x)/sd_x 
} 

# find the z score of women with HDL of 40mg/dl
findZscore(40, 55, 15.5)

## [1] -0.9677419

findZscore(60, 55, 15.5)

## [1] 0.3225806

# find the area to the right of the z score:
protect <- 1 - findZscore(60, 55, 15.5)
1 - protect

## [1] 0.3225806

findZscore(40, 55, 15.5) 

## [1] -0.9677419

findZscore(60, 55, 15.5) 

## [1] 0.3225806

1.138

Quartiles for Normal distributions. The quartiles of any distribution are the values with cumulative proportions 0.25 and 0.75.

(a) What are the quartiles of the standard Normal distribution?

We can use R to calculate the quantiles of the normal distrubution using R. We are going to create 10000 observations with mean equals zero and standard deviation equals 1 using rnorm function in R:

set.seed(10000)
normal <- rnorm(10000, mean = 0, sd = 1)

# calculate the quantiles for this normal distribution:
quantile(normal)

##            0%           25%           50%           75%          100% 
## -4.0226781179 -0.6733417727 -0.0005222524  0.6951116476  3.3967841641

Quantile 1 is shown here with a standardized score equals -0.67 and Q3 is also shown here with 0.67.

Similarly, given that the first quartile and the third quartile have proportions of 0.25 and 0.75, we can look at the z table and see z scores of these quartiles respectively. The first quartile has a z score of approximately -0.67 and the third quartile has a z score of approximately 0.67.

(b) Using your numerical values from part (a), write an equation that gives the quartiles of the N(μ, σ) distribution in terms of μ and σ.

The equation for each quantile is calculated by multiplying the quantile we are given by the SD, then we add the product of it by the mean:

Q1 = mean - 0.674*SD

Q2 = mean - 0.00052*SD

Q3 = mean + 0.674*SD

1.146

CO2 emissions in vehicles. Natural Resources Canada tests new vehicles each year and reports several variables related to fuel consumption for vehicles in different classes. For 2015, it provides data for 526 vehicles that use regular fuel. Two variables reported are carbon dioxide (CO2) emissions and highway fuel consumption. CO2 is measured in grams per kilometer (g/km), and highway fuel consumption measured in liters per 100 kilometers (L/km). Use graphical and numerical summaries to describe the distribution of CO2 emissions for these vehicles. Be sure to justify your choice of summaries.

Answer

First, we display the dataset to get a feel of the data:

## make the table of emission
knitr::kable(
  emission[1:5, ],
  caption = "CO2 emission table"
)

Given that there are a lot of columns that are not essential for our analysis, such as model year or engine sizes, we are going to select only the colums that related to our graphical summary. This is done by selecting only VehicleClass, CO2 and Highway Fuel Consumptions:

# select only Vehicle Class and their emission
newEmission <- emission %>% 
  select(VehicleClass, CO2, FuelConsHwy) %>% 
  mutate(Vehicle = as.factor(as.character(VehicleClass)), 
         VehicleClass = NULL) 


# make a table
knitr::kable(
  newEmission[1:5 , ],
  caption = "Table of CO2 emmissions based on vehicle classes"
)

The above table shows only the variables that we need to analysize. The next step is to create a graphical summary so that we can better visualise the distribution:

#------------------
#Graphical Summary 
#------------------
newEmission %>% 
  ggplot(aes(x = FuelConsHwy, y = CO2, colour = Vehicle)) +
  geom_jitter( alpha = 1/3) +
  facet_wrap(~Vehicle) +
   theme(legend.position = "none",
        plot.title = element_text(face = "bold"),
        panel.background = element_rect(fill = "#fdf9ff"),
        strip.background = element_rect(fill = "#a6bddb", color = "#400156", size =0.5),
        strip.text = element_text(size = 10, color = "black"),
        panel.border = element_rect(color = "grey30", fill = NA, size = 0.5)) +
  labs(title = "CO2 and High Way Consumptions based on Vehicles",
       x = "Fuel Consumption in Litres per 100km (L/km)",
       y = "CO2 emission (g/km)") 

This graphic shows us the relationship between highway fuel consumptions (axis x) and CO2 emission (axis y) based on each vehicle class. We can see that there is a strong, linear correlation between CO2 emmissions and highway fuel consumptions for each vehicle class. This means that the more fuel consumptions vehicles are used, the more CO2 emissions are being produced. This graphic also illustrates which vehicle class has the most CO2 emissions. VAN - PASSENGER produces the most CO2 emissions with over 400 g/km. While TWO-SEATER produces the least CO2 with only about 150 g/km. Some a lot of other vehicle classes, the distributions of CO2 emissions and highway fuel consumptions are vary.

The advantage of this graphic is that it clearly shows the relationship between highway fuel consumptions and CO2 emissions. Additionally, we can see these distributions for each vehicle class individually.

1.165

Blueberries and anthocyanins. Anthocyanins are compounds that have been associated with health benefits associated with the heart, bones, and the brain. Blueberries are a good source of many different anthocyanins. Researchers at the Piedmont Research Station of North Carolina State University have assembled a database giving the concentrations of 18 different anthocyanins for 267 varieties of blueberries.47 Four of the anthocyanins measured are delphinidin-3-arabinoside, malvidin-3-arabinoside, cyanidin-3-galactoside, and delphinidin-3-glucoside, all measured in units of mg/100g of berries. In the data file, we have simplified the names of these anthocyanins to Antho1, Antho2, Antho3, and Antho4. Figure 1.35 gives graphical and numeric summaries from JMP for Antho1. Use this output to write a summary of the distribution of Antho1 using the methods and ideas that you learned in this chapter.

Answer

We will select only the variable of interest. In this case it is Antho1:

# select Antho1
berry <- berry %>% select(Antho1)

## draw the histogram for Antho1
berry %>% 
  ggplot(aes(x = Antho1)) +
  geom_histogram(binwidth = 0.2, alpha = 0.7,  fill = "orange") +
  labs(title = "Histogram of Antho1",
       subtitle = "Antho1 is measured in mg/100g of berries",
       x = "mg/100g")

The above histogram is approximately symmetrical and bell shaped, with one exception being that there is a slightly higher frequency of Antho1 when it reaches near 2mg/100g of berries.

We can also graph the QQplot and some summary statistics table:

# summary statistics for Antho1
quantile(berry$Antho1)

##        0%       25%       50%       75%      100% 
## 0.2070234 1.2856481 1.6204762 1.9153709 3.2373255

describe(berry$Antho1)

##    vars   n mean   sd median trimmed  mad  min  max range skew kurtosis
## X1    1 267 1.63 0.52   1.62    1.61 0.47 0.21 3.24  3.03 0.28     0.51
##      se
## X1 0.03

qqnorm(berry$Antho1)

This plot shows the relationship between the observed quantiles and the theoretical quantiles. We can see that there is a very linear relationship in the graph, which shows a (almost) straight line. This means that the graph is normal distributed.

Datasaurus

[F] Prof. Tucker’s BIG DATA Challenge (Datasaurus). At Moodle, you’ll find a tabLseparated variable (.tsv) file called “DatasaurusDozen.tsv”. It has 1,847 records; there are 12 distinct data sets inside. For example, row #712 reads: “star 58.2136082599 91.881891513”. The value in the first column, star, refers to the data set — all of the “star” rows will form one of the dozen. The next column has “x” values, while the third and final column as “y” values. For each of the 12 data sets, find the mean of X & Y, the standard deviations of X & Y, and the correlation coefficient between the two. The final step should be to plot the data. For the writeL up, you don’t need to include your analysis of all twelve data sets, but describe in high level terms what is going on with this dozen datasets. Why are we seeing what we’re seeing? Hint: data.frame and factor are R ideas that could help make this easier.

Answer

First, we are going to find the mean, the SD, and the correlation coefficients for each of the dataset:

# group the dataset together and produce a summary table
datasaurusNew <- datasaurus %>% 
  group_by(dataset) %>% 
  summarise(mean_x = mean(x),
            mean_y = mean(y),
            sd_x = sd(x),
            sd_y = sd(y),
            cor = cor(x, y))

# display the table
knitr::kable(datasaurusNew[ , ],
             caption = "Datasaurus Table with summary")

Interestingly, the summary statistics of these datasets are almost identical! This is surprising because each individual observation for each dataset is not equal to each other.

We can also plot the datasets to clearly see the distributions:

datasaurus %>% 
  ggplot(aes(x, y, colour = dataset)) +
  geom_point(alpha = 1/3) +
  labs(title = "Plot for each dataset in Datasaurus",
       x = "Value of X",
       y = "Value of Y") +
  theme(legend.position = "none",
        plot.title = element_text(face = "bold"),
        panel.background = element_rect(fill = "#fdf9ff"),
        strip.background = element_rect(fill = "#a6bddb", color = "#400156", size =0.5),
        strip.text = element_text(size = 10, color = "black"),
        panel.border = element_rect(color = "grey30", fill = NA, size = 0.5)) +
    facet_wrap(~dataset)

This graph shows very interesting and beautiful patterns for each of the dataset, though the summary statistics of these datasets are identical. This gives us an insight into how different it is between the summary statistics and the distributions of the datasets. We can conclude from this graph that it’s very, very important to always be charting (A.B.C). Because summary statistics, though can give us a general feel of the data, we cannot clearly see the whole picture, or the distributions of the dataset itself.