Diamond Workshop

Introduction

• The objective of this workshop is to introduce you the the art of Exploratory Data Analysis (EDA).

• The introduction to section 7.1 in R4DS gives a short and useful overview of what EDA is.

• In this project we will be working with the diamonds data set. In the console type ?diamonds to link to a help file describing the dimond data base and its variables.

About this document.

This is an RMarkdown document. RMarkdown is a package for literate coding. Literate coding is a way of programming in which the program instructions are interwoven with the documentation of the program.

In data science this means that we can produce one document which contains all our analytic steps, in such a way that another reader can read what you have have written, but also process the same data using the same software. This is a key requirement for reproducible research.

Moreover, combined with a version control system, like git hub, an RMarkdown document can be collaborative. We will talk more about that in the coming weeks.

Today’s workshop is focused on giving you a opportunity to use some of the skills you worked at developing since the last class. As you work through this document, you should type in your responses to the questions and run your code in the provided code blocks.

Here is an example:

Compute the mean (arithemetic average) of the numbers from 1 to 100.(enter your answer in the block below and run the block by clicking on the little green triangle in the upper right corner of the block.)

mean(1:100) # <- you type this

## [1] 50.5

Getting started:

1. If necessary install the tidyverse.

install.packages(“tidyverse”)

2. In the console enter View(diamonds)

data("diamonds")

3. In the console type ?diamonds this will open a help page describing diamonds. Read the help page and compare it’s contents with the data you see in the View pane.

?diamonds

## starting httpd help server ... done

The data in the view panel shows the dataset whereas the view panel shows the description of the variables.

4. Create a “data dictionary” in which you list each variable and its definition.

Your answer

5. In the code chunck below, create a summary of diamonds using the summary function.

summary(diamonds)

##      carat               cut        color        clarity          depth      
##  Min.   :0.2000   Fair     : 1610   D: 6775   SI1    :13065   Min.   :43.00  
##  1st Qu.:0.4000   Good     : 4906   E: 9797   VS2    :12258   1st Qu.:61.00  
##  Median :0.7000   Very Good:12082   F: 9542   SI2    : 9194   Median :61.80  
##  Mean   :0.7979   Premium  :13791   G:11292   VS1    : 8171   Mean   :61.75  
##  3rd Qu.:1.0400   Ideal    :21551   H: 8304   VVS2   : 5066   3rd Qu.:62.50  
##  Max.   :5.0100                     I: 5422   VVS1   : 3655   Max.   :79.00  
##                                     J: 2808   (Other): 2531                  
##      table           price             x                y         
##  Min.   :43.00   Min.   :  326   Min.   : 0.000   Min.   : 0.000  
##  1st Qu.:56.00   1st Qu.:  950   1st Qu.: 4.710   1st Qu.: 4.720  
##  Median :57.00   Median : 2401   Median : 5.700   Median : 5.710  
##  Mean   :57.46   Mean   : 3933   Mean   : 5.731   Mean   : 5.735  
##  3rd Qu.:59.00   3rd Qu.: 5324   3rd Qu.: 6.540   3rd Qu.: 6.540  
##  Max.   :95.00   Max.   :18823   Max.   :10.740   Max.   :58.900  
##                                                                   
##        z         
##  Min.   : 0.000  
##  1st Qu.: 2.910  
##  Median : 3.530  
##  Mean   : 3.539  
##  3rd Qu.: 4.040  
##  Max.   :31.800  
## 

6. How does the summary of a categorical variable differ from the summary of a quantitative variable?

your answer:

Quantitative variables gives you the min, max and mean. Categorical variables give you variable categories.

7. In the code chunk below create a bar chart visualization of color

 diamonds %>% 
  ggplot() +
  geom_bar(aes(x = color, fill = color), color = "black")

8. Using the dplyr function count produce a frequency table for color in the below code chunck.

Sort the count of diamonds by color

diamonds %>%
  count(color)

## # A tibble: 7 x 2
##   color     n
##   <ord> <int>
## 1 D      6775
## 2 E      9797
## 3 F      9542
## 4 G     11292
## 5 H      8304
## 6 I      5422
## 7 J      2808

9. For examining the variability of a continuous numerical variable the first choice is frequently the histogram, A historam resembles a barchart,with an important difference.

• Plot a histogram of carat from the diamonds data set.

diamonds %>% 
  ggplot() + 
  geom_histogram(aes(x = carat), binwidth = 0.08)

• In a histogram the range of data values is divided into bins. The number of bins is variable depending on the width of the bin. Plot the above histogram with binwidth of 0.01 0.05, 0.1, 0.25, 0.5. What do you observe about the resulting histogram.

• Using the ggplot2 function cut_width() make a table of carat frequencies with binwidth = 0.75, compare your table with the corresponding histogram.

10. Plot a histogram with a binwidth of 0.1 but only for diamonds with carat < 2.

11. Read about geom_freqpoly() and produce overlaid histograms with binwidth = 0.1' for eachcolor, what happens if in the you set ``x = price, y = ..density.. in the aes for geom_freqpoly()?

diamonds %>% 
  ggplot() +
  geom_freqpoly(aes(x = price, y = ..density.., color = color), binwidth = 250)

12. Explore the distribution of each of the x, y, and z variables in diamonds. What do you learn? Think about a diamond and how you might decide which dimension is the length, width, and depth.

13. Explore the distribution of price. Do you discover anything unusual or surprising? (Hint: Carefully think about the binwidth and make sure you try a wide range of values.)

14. Compare and contrast coord_cartesian() vs xlim() or ylim() when zooming in on a histogram. What happens if you leave binwidth unset? What happens if you try and zoom so only half a bar shows?

15. In geom_histogram what is the difference between binwidth and bins? When might you prefer one to another?