In the following notes, you will load data directly from a URL, directly from pre-build datasets in R, and finally from a file you save in your own folder.
Load Data Method 1: Load Data from a URL
You can load data from a folder or you can load data directly from a URL. The next example loads the dataset, “Test Scores”, directly from the URL where it resides.
library(tidyverse) #you will use the readr package (read_csv, NOT read.csv) in tidyverse to read in
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.4 ✔ readr 2.1.5
✔ forcats 1.0.0 ✔ stringr 1.5.1
✔ ggplot2 3.5.2 ✔ tibble 3.2.1
✔ lubridate 1.9.4 ✔ tidyr 1.3.1
✔ purrr 1.0.4
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
allscores <-read_csv("https://goo.gl/MJyzNs")
Rows: 22 Columns: 4
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
dbl (4): group, pre, post, diff
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Notice R interprets the variable “group” as continuous values (col_double). We will fix this later. The command “dim” provides the dimensions of the data, which are 22 observations (rows) by 4 variables (columns).
Use Side-by-Side Boxplots - Done Incorrectly (at first!)
Here is some easy code to create 3 groups of boxplots with some easy-to-access data, filled by group. Since the groups are discrete, you can get rid of the shading.
boxpl <- allscores |># |> is called a pipe operatorggplot(aes(y = diff, group = group, fill = group)) +geom_boxplot()boxpl
Notice that the legend give a continuous range of values for the scores, even though the scores are only 1, 2, or 3. The code guides(fill = false) will get rid of the legend. Also, the x-axis labels make no sense. We will deal with that later.
Try to Correct for the Misrepresenting Legend
boxpl2 <- boxpl +guides(fill =FALSE)
Warning: The `<scale>` argument of `guides()` cannot be `FALSE`. Use "none" instead as
of ggplot2 3.3.4.
If an error occurs doing the following chunk, prompting you to use ‘none’ instead of ‘FALSE’, try again. It should override the issue.
boxpl2
Add Your Own Color Choices for the 3 Different Boxes
Ensure that the groups are considered as factors, rather than numbers.
Use ‘as.factor’ As Another Way to Ensure Numerical Values Are Read as Categorical
Then manually fill with the 3 colors: white, light gray, and dark gray. Make the boxplotsorient horizontally.
boxpl3 <- allscores |>ggplot() +geom_boxplot(aes(y = diff, group = group, fill = group)) +scale_fill_manual(values =c("white", "lightgray", "darkgray")) +theme(axis.text.y =element_blank()) +# Removes the useless y-axis tick values.labs(title ="Score-Improvements Across Three Groups",y ="Difference in Pre and Post Test Scores") +coord_flip() # flips the places of x and y axisboxpl3
Another Way to Convert to Factor Levels
Use the factor function:
mutate creates a new variable “1” categorical factor not a
boxpl3 <- allscores |>mutate(group =factor(group, levels =c("1", "2", "3"), ordered =TRUE)) |>ggplot() +geom_boxplot(aes(y = diff, group = group, fill = group)) +scale_fill_manual(values =c("white", "lightgreen", "lightpink")) +theme(axis.text.y =element_blank()) +labs(title ="Score Improvements Across Three Groups",y ="Difference in Pre and Post Test Groups") +coord_flip()boxpl3
Load Data Method 2: Use Prebuilt Dataset
We will use the penguins dataset that is pre-build in the “palmerpenguins” package to create scatterplots
Load The Package and Feed Data Into Global Environment
library(palmerpenguins)
data("penguins") # Loads the penguins dataset into your global environment
It is essential to recognize that variables may be: int (integer), num (numeric), or double vs char (character) and factor (for categories)
Typically, chr or factor are used for discrete variables and int,dbl,or num for continuous variables.
Use head() Function to View The Tibble and Variable Types
head(penguins)
# A tibble: 6 × 8
species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
<fct> <fct> <dbl> <dbl> <int> <int>
1 Adelie Torgersen 39.1 18.7 181 3750
2 Adelie Torgersen 39.5 17.4 186 3800
3 Adelie Torgersen 40.3 18 195 3250
4 Adelie Torgersen NA NA NA NA
5 Adelie Torgersen 36.7 19.3 193 3450
6 Adelie Torgersen 39.3 20.6 190 3650
# ℹ 2 more variables: sex <fct>, year <int>
Combine fig.cap For The Figure Label and fig.alt For The Alt Text
fig.cap and fig.alt are YAML code embedded in chunks - these are tags for screen readers to improve accessibility in your document. The colors darkorange, purple, and cyan4 improve visibility of colors for colorblind access.
Example of using fig.cap and fig.alt in the next chunk:
{r fig.cap=“Bigger flippers, bigger bills”, fig.alt=“Scatterplot of flipper length by bill length of 3 penguin species, where we show penguins with bigger flippers have bigger bigger bills.”}
Load Data Method 3: Read In A Saved CSV From Your Computer
This is the most important skill, to be able to read in a csv file that is saved on your computer in order to load it into R Studio to work with.
Find a dataset, save it in a folder on your computer. This folder is called your working directory. You will need to set your working directory.
To do so, go to Session > Set Working Directory > Choose Directory (note that there is a shortcut ctrl/command + shift + h). You can then copy the directory from the console to use later.
Clean Data Headings and Variable Names
Data is often messy and not ready to use right away!
The data might require some cleaning. Here are some important points to check:
Be sure the format is .csv
Be sure there are no spaces between variable names (headers).
Set all variable names to lowercase so you do not have to keep track of capitalizing
Loading Data From a Working Directory (folder)
The easiest way to find out what your current working directory is, use the command getwd()
getwd()
[1] "/home/andrewarsaw/DATA 110"
This command shows you (in your console below) the path to your directory. My current path is: [1] “/home/andrewarsaw/DATA 110”
If you want to change the path, there are several ways to do so. I find the easiest way to change it is to click the “Session” tab at the top of R Studio, Select “Set Working Directory”, and then arrow over to “Choose Directory”. At this point, it will take you to your computer folders, and you need to select where your data is held. I suggest you create a folder called “Datasets” and keep all the data you load for this class in that folder.
Notice that down in the console below, it will show the new path you have chosen: setwd(“/home/andrewarsaw/DATA 110”). At this point, I copy that command and put it directly into a new chunk.
Load The Libraries
Zoo is an abbreviation used in time series analysis, which stands for “Z’s Ordered Observations”
# install.packages("zoo")library(zoo) # this package will help us re-format the period to be a useable date
Attaching package: 'zoo'
The following objects are masked from 'package:base':
as.Date, as.Date.numeric
Load The Data
The following data comes from New York Fed (https://www.newyorkfed.org/microeconomics/hhdc.html) regarding household debt for housing and non-housing expenses. Debt amounts are in $ trillions for all US households.
Download this dataset, Household_debt, frim http://bit.ly/2P3O84E and save it in your dataset folder. Change your working directory to load from YOUR folder. Then run this code.
# Be sure to change this to your own directorysetwd("/home/andrewarsaw/DATA 110")household <-read_csv("household_debt.csv")
Rows: 64 Columns: 8
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (1): Period
dbl (7): Mortgage, HE Revolving, Auto Loan, Credit Card, Student Loan, Other...
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Very soon, you will find data from other sources. The data will require some cleaning. Here are some important points to check:
Be sure the format is .csv
Be sure there are no spaces between variable names (headers)
Set all variable names to lowercase so you do not have to keep track of capitalizing
Here Are Some Useful Cleaning Commands:
Make all headings (column names) lowercase. Remove all spaces between words in headings and replace them with underscores with gsub command. Then look at it with “head”.
names(household) <-tolower(names(household)) # makes all characters lowercasenames(household) <-gsub(" ","_", names(household)) # gsub will remove spaces in between words in the headers and replace them with underscorehead(household)
Mutate is a powerful command in tidyverse. It creates a new variable (column) in your dataset. In our dataset, “period” is not anything useful if we want to plot chronological data. So we will use mutate from “tidyverse” with the package “zoo” to create a useable date format.
summary(household$mortgage)
Min. 1st Qu. Median Mean 3rd Qu. Max.
4.940 7.923 8.265 8.041 8.762 9.290
Create Two New Variables
Use mutate to create a variable “mortgage_ratio” that computes each mortgage in proportion to the total
Use mutate to create a variable “mortgage_highlow” that makes a dichotomous variable that is “low” if lower than the median and “high” if otherwise
You should see that there are 64 observations and 8 variables. All variables are “col_double” (continuous values) except “period”, which is interpreted as characters. We need to use the library “zoo” package to fix the unusual format of the “period”. We will mutate it to create a new variable, date.
household_debt <- house |>mutate(date =as.Date(as.yearqtr(period, format ="%y:Q%q")))head(household_debt)
plot1 <- household_debt |>ggplot(aes(date, mortgage)) +geom_point() +labs(title ="Mortgage Debt(in $ trillions) Between 2003 and 2018",x ="Years (2003-2018)",caption ="Source: New York Fed")plot1
Use The mortgage_ratio Values Instead of Mortgage Debt
plot1a <- household_debt |>ggplot(aes(date, mortgage_ratio)) +geom_point() +labs(title ="Mortgage Debt(in $ trillions) Between 2003 and 2018",x ="Years (2003-2018)",caption ="Source: New York Fed")plot1a
Notice that the mortgage ratio to total debt dramatically drops after about 2010.
Credit Card Debt
plot2 <- household_debt |>ggplot(aes(date, credit_card)) +geom_point() +labs(title ="Credit Card Debt (in $ trillions) Between 2003 and 2018",x ="Years (2003-2018)",caption ="Source: New York Fed")plot2
Pivot The Table from “wide” To “long” Format
This will enable us to use the “facet” function in the next step.
