CSS30500 Computing for Social Science

Lecture Notes

LECTURE 1

LECTURE 1: R Markdown

# Class code
# usethis::use_course("css-materials/r-markdown")

Syntax

1. *Italics*
2. **Bold**
3. ~~Strikethrough~~
   
4. inline code: ` backtick before and after text
5. Unordered Listing: (+,-,*)

# + item 1  
#  + sub  
#  + sub  
# - item 2  
#   - sub  
#   - sub  
# + item 3  
#   - sub  
#   * sub  

6. Ordered Listing (use “1.”)

# 1. item 1  
#     + sub  
#       + sub  
#     + sub  
# 1. item 2  
#     * sub  
#       * sub  
#     * sub  
# 1. item 3  
#     - sub  

7. Link | Text in [], URL in ()

# [RStudio](https://www.rstudio.com/) 

8. Image | Start with !, include optional “title”

#![Logo](images/logo.png "Optional title")

9. Table | Columns: |, Header: —

# | Tool    | Use     |
# |---------|---------|
# | R       | Stats   |

Tool	Use
R	Stats

Code Chunk Settings (Optional)

Option	What it does	Default
eval = FALSE	Code is not run, only shown; results do not appear. Useful for code examples or showing code with errors.	eval = TRUE
include = FALSE	Code is run, but both code and results are hidden. Useful for setup code you don’t want to display.	include = TRUE
echo = FALSE	Code is run, results are shown, but code is hidden. Good when readers only need the output.	echo = TRUE
error = TRUE	Code is run, and errors are displayed instead of stopping rendering. Useful when demonstrating broken code.	error = FALSE
message = FALSE / warning = FALSE	Code is run, but messages and warnings are hidden from the rendered output.	message = TRUE / warning = TRUE

LECTURE 2

LECTURE 2: Workflows, Git & GitHub

2 Approaches

Graphical User Interface (GUI) Workflow - using diff. software Programmatic Workflow - using coding + Advantages + Automation + programs can perform tasks systematically + reduce manual effort and human errors (LT) + fast and consistent repeated analyses + Reproducibility + scientific research should share both data and code used for analysis + allow others to verify and reuse + enable exact replication of results even years later

Git vs Github

• git is local, github is cloud
• Version Control
  • system that records every change by versions stored in a folder
  • git is a specific software to record and save changes in the folder (gut save every change)

How to solve a Git Conflict

1. Go to R (Workbench) and open your terminal.
2. Pull the latest changes from GitHub by typing git pull origin main. Then read the message that shows up on Git: it will tell you what to do.
3. Open the file or files and look for the specific conflict markers such as <<<<<<<, =======, >>>>>>>
4. Decide which version to keep or manually combine them
5. Delete the conflict markers
6. Save the file
7. Run the following commands from your terminal: git add <filename> git commit -m "solved conflict" git push origin main

How to avoid a Git Conflict

• Make changes locally, then stage and commit
• Push regularly to keep your GitHub repo in sync
• Always pull before you push to avoid conflicts

git pull → make changes → git add → git commit → git push

LECTURE 3

LECTURE 3: Grammar of Graphics & ggplot2

Why We Visualize Data

• graphs help us identify patterns
• graphs make data more understandable
• graphs allow us to explore questions quickly (initial viz → data/models → more viz)

GGplot-geom_point (Scatter Plot)

library(ggplot2)
data(mpg)
head(mpg)

## # A tibble: 6 × 11
##   manufacturer model displ  year   cyl trans      drv     cty   hwy fl    class 
##   <chr>        <chr> <dbl> <int> <int> <chr>      <chr> <int> <int> <chr> <chr> 
## 1 audi         a4      1.8  1999     4 auto(l5)   f        18    29 p     compa…
## 2 audi         a4      1.8  1999     4 manual(m5) f        21    29 p     compa…
## 3 audi         a4      2    2008     4 manual(m6) f        20    31 p     compa…
## 4 audi         a4      2    2008     4 auto(av)   f        21    30 p     compa…
## 5 audi         a4      2.8  1999     6 auto(l5)   f        16    26 p     compa…
## 6 audi         a4      2.8  1999     6 manual(m5) f        18    26 p     compa…

str(mpg)

## tibble [234 × 11] (S3: tbl_df/tbl/data.frame)
##  $ manufacturer: chr [1:234] "audi" "audi" "audi" "audi" ...
##  $ model       : chr [1:234] "a4" "a4" "a4" "a4" ...
##  $ displ       : num [1:234] 1.8 1.8 2 2 2.8 2.8 3.1 1.8 1.8 2 ...
##  $ year        : int [1:234] 1999 1999 2008 2008 1999 1999 2008 1999 1999 2008 ...
##  $ cyl         : int [1:234] 4 4 4 4 6 6 6 4 4 4 ...
##  $ trans       : chr [1:234] "auto(l5)" "manual(m5)" "manual(m6)" "auto(av)" ...
##  $ drv         : chr [1:234] "f" "f" "f" "f" ...
##  $ cty         : int [1:234] 18 21 20 21 16 18 18 18 16 20 ...
##  $ hwy         : int [1:234] 29 29 31 30 26 26 27 26 25 28 ...
##  $ fl          : chr [1:234] "p" "p" "p" "p" ...
##  $ class       : chr [1:234] "compact" "compact" "compact" "compact" ...

# Plotting Graph 1
ggplot(data = mpg, aes(x = displ, y = hwy)) +
  geom_point() +
  geom_smooth(method = "lm") #TREND LINE

## `geom_smooth()` using formula = 'y ~ x'

  labs(title = "Car Fuel Efficiency on the Highway vs. Car Engine Size",
       x = "Car Engine Size in Liters",
       y = "Car Highway MPG (Fuel Efficiency)") 

## <ggplot2::labels> List of 3
##  $ x    : chr "Car Engine Size in Liters"
##  $ y    : chr "Car Highway MPG (Fuel Efficiency)"
##  $ title: chr "Car Fuel Efficiency on the Highway vs. Car Engine Size"

# Plotting Graph 2
ggplot(mpg) +
  geom_point(aes(x = displ, y = hwy, color = class)) # USES COLORS TO REPRESENT CLASS

ggplot(mpg) +
  geom_point(aes(x = displ, y = hwy, shape = class)) # USES SHAPES TO REP CLASS

## Warning: The shape palette can deal with a maximum of 6 discrete values because more
## than 6 becomes difficult to discriminate
## ℹ you have requested 7 values. Consider specifying shapes manually if you need
##   that many of them.

## Warning: Removed 62 rows containing missing values or values outside the scale range
## (`geom_point()`).

Difference between Graph 1 and 2: extra variable included in 2 by color/shape

Grammar and “Grammar of Graphics”

A grammar is a set of rules for combining words (syntax, morphology, etc.), that helps us structure a language and communicate clearly.

Applied to R and ggplot2…

A Grammar of Graphics is a set of rules for combining visual elements (data, aesthetics, geometries, etc.) that helps us structure plots and communicate clearly.

Main Components of the Grammar of Graphics

1. DATA: the dataset you are using (e.g. mpg data)
2. GEOM: the type of plot (e.g., points, bars, lines)
3. MAPPING: maps variables to aesthetics like x, y, color, etc.
4. STAT: whether the data should be transformed (e.g., counted) or not (e.g., identity)
5. POSITION: how things are arranged on the plot (e.g., stacked or jittered)

code template

#ggplot(data = <DATA>) +
#  <GEOM>(                           ### TYPE OF GRAPH
#    mapping = aes(<MAPPING>),       ### X&Y VARIABLES
#    stat = <STAT>,                  ### stat = "identity"  Plots original values directly 
                                     ### stat = "count" Counts observations in categories
                                     ### stat = "smooth"    Calculates a fitted relationship, as used by geom_smooth()
#    position = <POSITION>           ### stack/fill(proportional)/dodge(side by side)/jitter/identity
#  ) +
#  <COORDINATE SYSTEM> +             ### how x&y are displayed eg. coord_cartesian(yslim=c(10,40))
#  <FACET>                           ### divide graph into multiple panels based on categorical var.

# code template filled with mpg data

ggplot(data = mpg) +
  geom_point(
    mapping = aes(x = displ, y = hwy, color = class),
    stat = "identity",
    position = "identity"
  ) +
  coord_cartesian() +
  facet_wrap(vars(class), nrow = 1)

Make & Store a plot

fuel_plot <- ggplot(data = mpg) +
  geom_point(
    mapping = aes(x = displ, y = hwy, color = class))
fuel_plot # THIS IS STORING

Lookinh up functions

+Function help: type ?mean or help(“mean”) in your R Console +Topic search: type ??mean if you don’t know the function name +RStudio suggestions: type mean( and wait or press Tab to see argument suggestions (in yellow)

Plots

+geom_point() → scatterplot +by defult: stat="identity" +geom_line() → line chart +geom_bar() → bar chart +by defult: stat="count" +geom_histogram() → histogram +geom_boxplot() → boxplot

# To add text elements to your plot, use labs():

ggplot(data = mpg, aes(x = displ, y = hwy, color = class)) +
  geom_point() +
  labs(
    title = "Car Fuel Efficiency on the Highway vs. Car Engine Size",
    subtitle = "Colored by car type",
    x = "Car Engine Size in Liters",
    y = "Car Highway MPG (Fuel Efficiency)",
    caption = "Data: mpg dataset (n = 234)")

# For more control, combine it with theme():

ggplot(data = mpg, aes(x = displ, y = hwy, color = class)) +
  geom_point() +
  labs(
    title = "Car Fuel Efficiency on the Highway vs. Car Engine Size",
    subtitle = "Colored by car type",
    x = "Car Engine Size in Liters",
    y = "Car Highway MPG (Fuel Efficiency)",
    caption = "Data: mpg dataset (n = 234)") +
  theme(plot.title = element_text(face = "bold", size = 18)) # BOLDED AND ENLARGED TITLE

# LAYERS
ggplot(data = mpg) + 
  geom_point(
    mapping = aes(x = displ, y = hwy, color = class),
    stat = "identity", 
    position = "identity"
  ) + 
  geom_smooth(
    mapping = aes(x = displ, y = hwy), # CREATED SHADED AREA 95% CI + TREND LINE
    method = "lm" 
  )

## `geom_smooth()` using formula = 'y ~ x'

# SHORTER VERSION
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) + 
  geom_point(aes(color = class)) + 
  geom_smooth(method = "lm")

## `geom_smooth()` using formula = 'y ~ x'

REVIEW

Question	Answer	Example
What does geom_point() do?	Creates a scatterplot where each point represents an individual observation.	geom_point()
What does stat = "identity" mean?	Uses the original data values without applying additional statistical transformations. It is the default for geom_point().	geom_point(stat = "identity")
When would you use facet_wrap()?	When you want to divide a graph into multiple panels based on a categorical variable to compare patterns across groups.	facet_wrap(~ class)
What’s wrong with aes(color = "blue")?	aes() maps values to aesthetics rather than setting a fixed color. To make all points blue, specify color = "blue" outside aes().	geom_point(color = "blue")
What does it mean that layers can inherit elements?	Layers automatically inherit aesthetic mappings defined in ggplot(), so you do not need to repeat them in every layer.	ggplot(mpg, aes(x = displ, y = hwy)) + geom_point() + geom_smooth(method = "lm")

LECTURE 4

LECTURE 4: Data Visualization & Plot Types

Plots Option

# Box Plot
ggplot(data = scorecard, mapping = aes(x = type, y = satavg)) +
  geom_boxplot()

## Warning: Removed 691 rows containing non-finite outside the scale range
## (`stat_boxplot()`).

# Histogram
ggplot(data = scorecard, mapping = aes(x = satavg)) +
  geom_histogram() + 
  facet_wrap(facets = vars(type))

## `stat_bin()` using `bins = 30`. Pick better value `binwidth`.

## Warning: Removed 691 rows containing non-finite outside the scale range
## (`stat_bin()`).

Types of Visualizations and Best Graph Types

1. Which kind of variation do I want to represent?
   • Univariate: within one single variable
   • Bivariate: among two variables
   • Multivariate: among three variables
2. What type(s) are my variables?
   • Continuous (e.g., satavg average SAT score, cost published cost)
   • Categorical (e.g., type school type, state U.S. state)
   • Other types (often treated as categorical): ordinal, nominal, binary

Univariate (One Variable)

To show how values vary within a single variable:

• One continuous variable → Histogram
• One categorical variable → Bar Chart

Function	What Gets Counted	When to Use
geom_histogram()	How many values fall into each numeric bin	When x is continuous
geom_bar()	How many observations are in each category	When x is categorical

Bivariate (Two Variables)

To show how two variables co-vary:

• Two continuous variables → Scatterplot
• One categorical + one continuous → Box Plot
• Two categorical variables → Grouped or Stacked Bar Chart

Multivariate (Three Variables)

To compare patterns across subgroups:

• One categorical + two continuous → Faceted Scatterplot
• Two categorical + one continuous → Grouped Box Plot

Table:

Graph Type	ggplot2 Function	Input Variables	Purpose
Histogram	geom_histogram()	One continuous variable (e.g., age, income)	Show the distribution of values
Bar Chart	geom_bar()	One categorical variable (e.g., region, gender)	Show frequencies or counts of categories
Stacked Bar Chart	geom_bar() + fill	Two categorical variables (e.g., region by gender)	Compare parts of a whole across categories
Scatterplot	geom_point()	Two continuous variables (e.g., height vs. weight)	Show relationships or correlations
Box Plot	geom_boxplot()	One continuous + one categorical variable (e.g., income by gender)	Compare distributions and identify outliers
Faceted Scatterplot	facet_wrap() + geom_point()	Two continuous + one categorical variable (e.g., relationships by country)	Compare patterns across groups

From Excercise

1. Bar Chart
   • x is a categorical variable
   • Reproducing
     1. directly geom_bar (at defaultstat = "count")
     2. count(DATA, VAR.)then geom_bar(stat=identity) with mapping = aes(x = type, y = n)
     3. count then geom_col()
     • eg.

scorecard_type_counts <- count(scorecard, type) #COUNTING TO CREATE TABLE

ggplot(data = scorecard_type_counts, 
       mapping = aes(x = type, y = n)) + 
  geom_col()

2. Histogram
   • at default stat = "bin" (bin = 30)
   • x is a continuous variable

scorecard |>
  ggplot(mapping = aes(x = cost)) +
  geom_histogram()

## `stat_bin()` using `bins = 30`. Pick better value `binwidth`.

## Warning: Removed 55 rows containing non-finite outside the scale range
## (`stat_bin()`).

# alt 1
ggplot(data = scorecard, mapping = aes(x = cost)) +
  geom_histogram(stat = "bin", 
                 bins = 30) # try changing the integer 

## Warning: Removed 55 rows containing non-finite outside the scale range
## (`stat_bin()`).

# Adding visuals
ggplot(data = scorecard, mapping = aes(x = cost)) +
  geom_histogram() +
  labs(
    title = "Distribution of Annual College Attendance Costs",
    x = "Published Annual Attendance Cost (USD)",
    y = "Number of Colleges",
    caption = paste("Data: College Scorecard (2018–2019; n =", nrow(scorecard), ")") # ADD A GRAY CAPTION BOTTOM RIGHT CORNER
  ) +
  theme(
    plot.title = element_text(size = 16),
    plot.caption = element_text(size = 8, color = "gray")
  )

## `stat_bin()` using `bins = 30`. Pick better value `binwidth`.

## Warning: Removed 55 rows containing non-finite outside the scale range
## (`stat_bin()`).

3. Scatter Plot
   • x is one continuous variable, y is another continuous variable

ggplot(scorecard,mapping = aes(x = cost, y = netcost)) +
  geom_point()

## Warning: Removed 55 rows containing missing values or values outside the scale range
## (`geom_point()`).

4. Box Plot
   • x is categorical, y is continuous

ggplot(scorecard, mapping = aes(x = type, y = cost)) +
  geom_boxplot()

## Warning: Removed 55 rows containing non-finite outside the scale range
## (`stat_boxplot()`).

• alt: faceted histogram

ggplot(scorecard,
  mapping = aes(x = cost)) +
  geom_histogram() +
  facet_wrap(facets = vars(type), nrow = 3) # try without nrow or with nrow = 1

## `stat_bin()` using `bins = 30`. Pick better value `binwidth`.

## Warning: Removed 55 rows containing non-finite outside the scale range
## (`stat_bin()`).

5. Faceted Scatter Plot
   • can be used when 2 continuous variable + 1 categorical variable (will display cat. var. in different panels)

ggplot(scorecard,
  mapping = aes(x = cost, y = netcost)) +
  geom_point() +
  facet_wrap(facets = vars(type), nrow = 2) # NROW DETERMINES TOTAL ROWS OF GRAPHS, EG. NROW = 2 = 2 ROWS OF GRAPHS

## Warning: Removed 55 rows containing missing values or values outside the scale range
## (`geom_point()`).

• alt: using colors to replace facet to show categories

ggplot(scorecard,
  mapping = aes(x = cost, y = netcost, color = type)) +
  geom_point() # USING COLORS TO REP TYPES

## Warning: Removed 55 rows containing missing values or values outside the scale range
## (`geom_point()`).

• size = type also works but is less informative

Cheat Sheets

• R > Help > Cheat Sheets > Data Visualization with ggplot2
• https://posit.co/resources/cheatsheets/

LECTURE 5

LECTURE 5: Data Transformation with dplyr & Pipes

# load packages
library(tidyverse)
library(palmerpenguins)

## 
## Attaching package: 'palmerpenguins'

## The following objects are masked from 'package:datasets':
## 
##     penguins, penguins_raw

# explore data
bind_rows(head(penguins, 4), tail(penguins, 4))

## # A tibble: 8 × 8
##   species   island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
##   <fct>     <fct>              <dbl>         <dbl>             <int>       <int>
## 1 Adelie    Torgersen           39.1          18.7               181        3750
## 2 Adelie    Torgersen           39.5          17.4               186        3800
## 3 Adelie    Torgersen           40.3          18                 195        3250
## 4 Adelie    Torgersen           NA            NA                  NA          NA
## 5 Chinstrap Dream               43.5          18.1               202        3400
## 6 Chinstrap Dream               49.6          18.2               193        3775
## 7 Chinstrap Dream               50.8          19                 210        4100
## 8 Chinstrap Dream               50.2          18.7               198        3775
## # ℹ 2 more variables: sex <fct>, year <int>

glimpse(penguins_raw)

## Rows: 344
## Columns: 17
## $ studyName             <chr> "PAL0708", "PAL0708", "PAL0708", "PAL0708", "PAL…
## $ `Sample Number`       <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 1…
## $ Species               <chr> "Adelie Penguin (Pygoscelis adeliae)", "Adelie P…
## $ Region                <chr> "Anvers", "Anvers", "Anvers", "Anvers", "Anvers"…
## $ Island                <chr> "Torgersen", "Torgersen", "Torgersen", "Torgerse…
## $ Stage                 <chr> "Adult, 1 Egg Stage", "Adult, 1 Egg Stage", "Adu…
## $ `Individual ID`       <chr> "N1A1", "N1A2", "N2A1", "N2A2", "N3A1", "N3A2", …
## $ `Clutch Completion`   <chr> "Yes", "Yes", "Yes", "Yes", "Yes", "Yes", "No", …
## $ `Date Egg`            <date> 2007-11-11, 2007-11-11, 2007-11-16, 2007-11-16,…
## $ `Culmen Length (mm)`  <dbl> 39.1, 39.5, 40.3, NA, 36.7, 39.3, 38.9, 39.2, 34…
## $ `Culmen Depth (mm)`   <dbl> 18.7, 17.4, 18.0, NA, 19.3, 20.6, 17.8, 19.6, 18…
## $ `Flipper Length (mm)` <dbl> 181, 186, 195, NA, 193, 190, 181, 195, 193, 190,…
## $ `Body Mass (g)`       <dbl> 3750, 3800, 3250, NA, 3450, 3650, 3625, 4675, 34…
## $ Sex                   <chr> "MALE", "FEMALE", "FEMALE", NA, "FEMALE", "MALE"…
## $ `Delta 15 N (o/oo)`   <dbl> NA, 8.94956, 8.36821, NA, 8.76651, 8.66496, 9.18…
## $ `Delta 13 C (o/oo)`   <dbl> NA, -24.69454, -25.33302, NA, -25.32426, -25.298…
## $ Comments              <chr> "Not enough blood for isotopes.", NA, NA, "Adult…

Revision

# Relationship between flipper length and body mass across species
ggplot(data = penguins, aes(x = flipper_length_mm, y = body_mass_g, color = species)) +
  geom_point() +
  # facet_wrap(vars(species)) +
  labs(title = "Penguins: Body Mass vs. Flipper Length by Species")

## Warning: Removed 2 rows containing missing values or values outside the scale range
## (`geom_point()`).

dplyr vs ggplot2

• ggplot2 helps us answer questions with graphs
• dplyr helps us answer questions with numbers

Question → Visual Exploration (ggplot2) → Numerical Exploration (dplyr) → Interpretation → New Questions → Etc.

dplyr functions

Function	What it does	Example	Notes
filter()	Keeps rows based on conditions	filter(penguins, species == "Adelie")	can be used with ==,!=,<,>,&
summarize()	Calculates summary statistics for each group (e.g., mean, sum, count)	summarize(penguins, avg_mass = mean(body_mass_g, na.rm = TRUE))	often with mean(), median(), n()
group_by()	Groups data by one or more variables (temp.)	grouped <- group_by(penguins, species)	often used with cat. var. and before summarize() and filter()
count()	Counts the unique values of one or more variables	count(penguins, species)	both cat. and num. but more useful in cat.; shortcut for group_by() + summarize(n = n())
mutate()	Creates new columns or modifies existing ones	penguins <- mutate(penguins, body_mass_kg = body_mass_g / 1000)	make no changes to original data just adding

1. filter()

Operator	Meaning	Description
==	Equal to	Is x equal to y?
!=	Not equal to	Is x different from y?
<	Less than	Is x less than y?
<=	Less than or equal to	Is x less than or equal to y?
>	Greater than	Is x greater than y?
>=	Greater than or equal to	Is x greater than or equal to y?
\|	OR	Either x or y must be TRUE
&	AND	Both x and y must be TRUE
& !	AND NOT	x must be TRUE and y must be FALSE

filter(penguins, species == "Adelie" | species == "Chinstrap") #A OR B

## # A tibble: 220 × 8
##    species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
##    <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
##  1 Adelie  Torgersen           39.1          18.7               181        3750
##  2 Adelie  Torgersen           39.5          17.4               186        3800
##  3 Adelie  Torgersen           40.3          18                 195        3250
##  4 Adelie  Torgersen           NA            NA                  NA          NA
##  5 Adelie  Torgersen           36.7          19.3               193        3450
##  6 Adelie  Torgersen           39.3          20.6               190        3650
##  7 Adelie  Torgersen           38.9          17.8               181        3625
##  8 Adelie  Torgersen           39.2          19.6               195        4675
##  9 Adelie  Torgersen           34.1          18.1               193        3475
## 10 Adelie  Torgersen           42            20.2               190        4250
## # ℹ 210 more rows
## # ℹ 2 more variables: sex <fct>, year <int>

filter(penguins, species == "Adelie" & species == "Chinstrap") #A AND B

## # A tibble: 0 × 8
## # ℹ 8 variables: species <fct>, island <fct>, bill_length_mm <dbl>,
## #   bill_depth_mm <dbl>, flipper_length_mm <int>, body_mass_g <int>, sex <fct>,
## #   year <int>

OR can also be written as %in%

filter(penguins, species %in% c("Adelie", "Chinstrap"))

## # A tibble: 220 × 8
##    species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
##    <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
##  1 Adelie  Torgersen           39.1          18.7               181        3750
##  2 Adelie  Torgersen           39.5          17.4               186        3800
##  3 Adelie  Torgersen           40.3          18                 195        3250
##  4 Adelie  Torgersen           NA            NA                  NA          NA
##  5 Adelie  Torgersen           36.7          19.3               193        3450
##  6 Adelie  Torgersen           39.3          20.6               190        3650
##  7 Adelie  Torgersen           38.9          17.8               181        3625
##  8 Adelie  Torgersen           39.2          19.6               195        4675
##  9 Adelie  Torgersen           34.1          18.1               193        3475
## 10 Adelie  Torgersen           42            20.2               190        4250
## # ℹ 210 more rows
## # ℹ 2 more variables: sex <fct>, year <int>

2. summarize()

# Example 1: only summarize 
summarize(penguins,
  body_mass_mean = mean(body_mass_g, na.rm = TRUE),
  body_mass_sd = sd(body_mass_g, na.rm = TRUE),
  n = n()  # n = nrow(penguins)
)

## # A tibble: 1 × 3
##   body_mass_mean body_mass_sd     n
##            <dbl>        <dbl> <int>
## 1          4202.         802.   344

# Example 2: combine filter with summarize
penguins |> 
  filter(species == "Gentoo") |> 
  summarize(body_mass_mean = mean(body_mass_g, na.rm = TRUE),
            body_mass_sd = sd(body_mass_g, na.rm = TRUE),
            n = n())

## # A tibble: 1 × 3
##   body_mass_mean body_mass_sd     n
##            <dbl>        <dbl> <int>
## 1          5076.         504.   124

3. group_by()

# group by alone? not that useful
# group by and calculate mean body mass
grouped <- group_by(penguins, island, sex)
summarize(grouped, avg_mass = mean(body_mass_g, na.rm = TRUE))

## `summarise()` has regrouped the output.
## ℹ Summaries were computed grouped by island and sex.
## ℹ Output is grouped by island.
## ℹ Use `summarise(.groups = "drop_last")` to silence this message.
## ℹ Use `summarise(.by = c(island, sex))` for per-operation grouping
##   (`?dplyr::dplyr_by`) instead.

## # A tibble: 9 × 3
## # Groups:   island [3]
##   island    sex    avg_mass
##   <fct>     <fct>     <dbl>
## 1 Biscoe    female    4319.
## 2 Biscoe    male      5105.
## 3 Biscoe    <NA>      4588.
## 4 Dream     female    3446.
## 5 Dream     male      3987.
## 6 Dream     <NA>      2975 
## 7 Torgersen female    3396.
## 8 Torgersen male      4035.
## 9 Torgersen <NA>      3681.

# group by and count
grouped <- group_by(penguins, island, sex)
count(grouped)

## # A tibble: 9 × 3
## # Groups:   island, sex [9]
##   island    sex        n
##   <fct>     <fct>  <int>
## 1 Biscoe    female    80
## 2 Biscoe    male      83
## 3 Biscoe    <NA>       5
## 4 Dream     female    61
## 5 Dream     male      62
## 6 Dream     <NA>       1
## 7 Torgersen female    24
## 8 Torgersen male      23
## 9 Torgersen <NA>       5

4. count()

# total penguins by sex without count
penguins |> 
  group_by(sex) |> 
  summarize(total = n())

## # A tibble: 3 × 2
##   sex    total
##   <fct>  <int>
## 1 female   165
## 2 male     168
## 3 <NA>      11

# total penguins by sex with count
penguins |>
  count(sex)

## # A tibble: 3 × 2
##   sex        n
##   <fct>  <int>
## 1 female   165
## 2 male     168
## 3 <NA>      11

5. mutate()

ncol(penguins) #8 here

## [1] 8

penguins <- mutate(penguins, body_mass_kg = body_mass_g / 1000)
ncol(penguins) #now 9

## [1] 9

Solving Questions

1. Identify variables needed
2. filter() only keep necessary ones
3. group_by()
4. summarize()usually calculates the answer

Pipes |> or %>%

• Pipes allow you to write a sequence of operations by passing the result of one function into the next making your code more readable and logical

# Without Pipes
adelie <- filter(penguins, species == "Adelie")
adelie_island <- group_by(adelie, island)
adelie_avg_mass_island <- summarize(adelie_island, body_mass = mean(body_mass_g, na.rm = TRUE))

# With Pipes
adelie_avg_mass_island <- penguins |>
  filter(species == "Adelie") |>
  group_by(island) |>
  summarize(body_mass = mean(body_mass_g, na.rm = TRUE))

Common Errors with Pipes

library(nycflights13)
data(flights)
glimpse(flights)

## Rows: 336,776
## Columns: 19
## $ year           <int> 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2…
## $ month          <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1…
## $ day            <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1…
## $ dep_time       <int> 517, 533, 542, 544, 554, 554, 555, 557, 557, 558, 558, …
## $ sched_dep_time <int> 515, 529, 540, 545, 600, 558, 600, 600, 600, 600, 600, …
## $ dep_delay      <dbl> 2, 4, 2, -1, -6, -4, -5, -3, -3, -2, -2, -2, -2, -2, -1…
## $ arr_time       <int> 830, 850, 923, 1004, 812, 740, 913, 709, 838, 753, 849,…
## $ sched_arr_time <int> 819, 830, 850, 1022, 837, 728, 854, 723, 846, 745, 851,…
## $ arr_delay      <dbl> 11, 20, 33, -18, -25, 12, 19, -14, -8, 8, -2, -3, 7, -1…
## $ carrier        <chr> "UA", "UA", "AA", "B6", "DL", "UA", "B6", "EV", "B6", "…
## $ flight         <int> 1545, 1714, 1141, 725, 461, 1696, 507, 5708, 79, 301, 4…
## $ tailnum        <chr> "N14228", "N24211", "N619AA", "N804JB", "N668DN", "N394…
## $ origin         <chr> "EWR", "LGA", "JFK", "JFK", "LGA", "EWR", "EWR", "LGA",…
## $ dest           <chr> "IAH", "IAH", "MIA", "BQN", "ATL", "ORD", "FLL", "IAD",…
## $ air_time       <dbl> 227, 227, 160, 183, 116, 150, 158, 53, 140, 138, 149, 1…
## $ distance       <dbl> 1400, 1416, 1089, 1576, 762, 719, 1065, 229, 944, 733, …
## $ hour           <dbl> 5, 5, 5, 5, 6, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5, 6, 6, 6…
## $ minute         <dbl> 15, 29, 40, 45, 0, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0, 59, 0…
## $ time_hour      <dttm> 2013-01-01 05:00:00, 2013-01-01 05:00:00, 2013-01-01 0…

1. Dont use <- within the pipe, only use at the beginning
2. Each function in the pipe must be connected by |> or %>%
3. Only need to reference the data at the beginning of the chain, no need to repeat
4. Don’t use + (this is fr ggplot2, not pipes), only use |> or %>%

LECTURE 6

LECTURE 6: More dplyr Functions & Recoding

Revision: Key Combo in the Pipe

filter() + group_by() + summarize()

penguins |> 
  filter(species != "Gentoo") |> 
  group_by(island) |> 
  summarize(body_mass_mean = mean(body_mass_g, na.rm = TRUE))

## # A tibble: 3 × 2
##   island    body_mass_mean
##   <fct>              <dbl>
## 1 Biscoe             3710.
## 2 Dream              3713.
## 3 Torgersen          3706.

New dplyr verbs

Function	What it does	Example	Notes
select()	Chooses specific columns	DATA_subset <- select(DATA, Col.A, Col.B, Col.C)	column ver. of filter()
relocate()	Reorders columns (move to new positions)	DATA_moved <- relocate(DATA, .before/.after = Col.X)	useful when variables we care are buried at the end of a wide dataset
arrange()	Reorders rows based on values in specified columns	DATA_sorted <- arrange(DATA, var.)	By default: smallest to largest; Add desc() into arrange(DATA, desc(var.)) to do largest to smallest; useful for finding largest or smallest value, higher readability
distinct()	Returns (only keeps) unique rows based on columns	NAME <- distinct(Var.A, Var.B, .keep_all = T)	dont need to write .keep_all = T if no need to keep other but only var. listed in code; Useful for spotting duplicate entries or listing the unique values or combinations in our data
pull()	Extracts a single column as a vector	NAME <- pull(DATA, Var.)	returns a vector; similar to DATA$var.
across()	Applies the same operation to multiple columns at once	usually goes inside summarise() / mutate()	useful when we don’t want to repeat code for each column
rename()	changes column name	DATA |> rename(NEWname = OLDname)
recode()	changes a single column’s value/levels of categorical variables (column values)	NAME <- DATA |> mutate (Var. = recode(Var., "Value" = "New Value", "Value2" = "New Value2"))	only works inside mutate();
case_when()	single column like if, then	DATA |> mutate(Var. = case_when(Var. == "value" ~ "new value", TRUE ~ NA_real_))	only works inside mutate(); TRUE ~ NA_real_ = if no previous condition was met, then return NA as a number

Complex Examples

# ARRANGE - LEFT TO RIGHT
penguins |>
  select(species, contains("bill")) |>
  filter(!species == "Adelie") |> 
  arrange(desc(species), bill_length_mm) # WILL ARRANGE SPECIES FIRST THEN BILL_LENGTH_MM

## # A tibble: 192 × 3
##    species bill_length_mm bill_depth_mm
##    <fct>            <dbl>         <dbl>
##  1 Gentoo            40.9          13.7
##  2 Gentoo            41.7          14.7
##  3 Gentoo            42            13.5
##  4 Gentoo            42.6          13.7
##  5 Gentoo            42.7          13.7
##  6 Gentoo            42.8          14.2
##  7 Gentoo            42.9          13.1
##  8 Gentoo            43.2          14.5
##  9 Gentoo            43.3          13.4
## 10 Gentoo            43.3          14  
## # ℹ 182 more rows

# PULL - BETTER IN PIPES than '$'
avg_mass <- penguins |>
  summarise(avg = mean(body_mass_g, na.rm = TRUE)) |> 
  pull(avg) # THIS WILL ONLY DISPLAY 1 VALUE INSTEAD OF SHOWING A WHOLE TABLE

avg_mass

## [1] 4201.754

penguins |> 
  filter(body_mass_g > avg_mass) # CAN BE USED TO COMPARE < >

## # A tibble: 149 × 9
##    species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
##    <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
##  1 Adelie  Torgersen           39.2          19.6               195        4675
##  2 Adelie  Torgersen           42            20.2               190        4250
##  3 Adelie  Torgersen           34.6          21.1               198        4400
##  4 Adelie  Torgersen           42.5          20.7               197        4500
##  5 Adelie  Dream               39.8          19.1               184        4650
##  6 Adelie  Dream               44.1          19.7               196        4400
##  7 Adelie  Dream               39.6          18.8               190        4600
##  8 Adelie  Biscoe              40.1          18.9               188        4300
##  9 Adelie  Biscoe              41.3          21.1               195        4400
## 10 Adelie  Torgersen           41.8          19.4               198        4450
## # ℹ 139 more rows
## # ℹ 3 more variables: sex <fct>, year <int>, body_mass_kg <dbl>

# ACROSS - WITH NUM. VAR.
penguins |>
  summarize( # WITHOUT ACROSS
    mean_bill_length_mm = mean(bill_length_mm, na.rm = TRUE),
    mean_bill_depth_mm = mean(bill_depth_mm, na.rm = TRUE),
    mean_flipper_length_mm = mean(flipper_length_mm, na.rm = TRUE),
    mean_body_mass_g = mean(body_mass_g, na.rm = TRUE)
  )

## # A tibble: 1 × 4
##   mean_bill_length_mm mean_bill_depth_mm mean_flipper_length_mm mean_body_mass_g
##                 <dbl>              <dbl>                  <dbl>            <dbl>
## 1                43.9               17.2                   201.            4202.

penguins |>
  summarize(
    across( # WITH ACROSS
      c(bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g), 
      ~ mean(.x, na.rm = TRUE)  # function(x) mean(x, na.rm = TRUE)
    ))

## # A tibble: 1 × 4
##   bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
##            <dbl>         <dbl>             <dbl>       <dbl>
## 1           43.9          17.2              201.       4202.

# ACROSS - WITH CAT. VAR.
penguins |>
  summarise(across(c(species, island, year), 
                   n_distinct)) # n_distinct: number of distinct entries for columns with across

## # A tibble: 1 × 3
##   species island  year
##     <int>  <int> <int>
## 1       3      3     3

# OR OPP.: using -c() (exclude c())
penguins |>
  summarise(across(-c(species, island, year), 
                   n_distinct))

## # A tibble: 1 × 6
##   bill_length_mm bill_depth_mm flipper_length_mm body_mass_g   sex body_mass_kg
##            <int>         <int>             <int>       <int> <int>        <int>
## 1            165            81                56          95     3           95

# everything
penguins |>
  summarise(across(everything(), 
                   n_distinct))

## # A tibble: 1 × 9
##   species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
##     <int>  <int>          <int>         <int>             <int>       <int>
## 1       3      3            165            81                56          95
## # ℹ 3 more variables: sex <int>, year <int>, body_mass_kg <int>

across() explained

1. which column to use
2. what function to apply to them

Function	What It Does	Example
across()	Applies the same function to multiple columns	summarise(across(c(species, island), n_distinct))
across(c(...))	Applies a function to specifically named columns	summarise(across(c(species, island, year), n_distinct))
across(-c(...))	Applies a function to all columns except those specified	summarise(across(-c(species, island, year), n_distinct))
across(everything())	Applies a function to all columns	summarise(across(everything(), n_distinct))
across(starts_with())	Selects columns whose names begin with specified text	summarise(across(starts_with("bill"), n_distinct))
across(ends_with())	Selects columns whose names end with specified text	summarise(across(ends_with("mm"), n_distinct))
across(contains())	Selects columns whose names contain specified text	summarise(across(contains("length"), n_distinct))
across(where(is.numeric))	Applies a function to all numeric columns	summarise(across(where(is.numeric), ~ mean(.x, na.rm = TRUE)))
across(where(is.character))	Applies a function to all character columns	summarise(across(where(is.character), n_distinct))
across(where(is.factor))	Applies a function to all factor columns	summarise(across(where(is.factor), n_distinct))
across(where(is.logical))	Applies a function to all logical columns	summarise(across(where(is.logical), n_distinct))

recode() vs case_when()

• both are used inside mutate()
• used when the goal is to change the column’s values
• mutate + case_when() is slightly more popular bc it is more flexible and powerful

Function	What It Changes	Syntax	Example	Tips
rename()	Column names	rename(new_name = old_name)	rename(notes = Comments)	No quotes around ordinary variable names
recode()	Column values	recode(variable, "old" = new)	recode(Sex, "MALE" = 1)	Character values generally require quotes
case_when()	Column values	case_when(condition ~ new)	case_when(Sex == "MALE" ~ 1)	Conditions are evaluated in order

Syntactic (Valid) Variable Names in R

• Valid Names in R:

  • Use letters, numbers, and the symbols . or _
  • But cannot start with a number or symbol

• What Makes a Name Invalid:

  • Contains spaces or symbols
  • Starts with a number or symbol
  • Uses reserved words (e.g., TRUE, NULL, if, function)
  • Type ?Reserved in the Console for the full list

• Examples | Variable Name | Valid? | Explanation | |:—|:—|:—| | 3_religion | No | Variable names cannot begin with a number. | | #3_religion | No | # starts a comment in R. | | q3_religion | Yes | Starts with a letter; underscores are allowed. | | q3.religion | Yes | Starts with a letter; periods are allowed. | | q3-religion | No | - is interpreted as subtraction. | | q3 religion | No | Spaces are not allowed in ordinary variable names. | | TRUE | No | TRUE is a reserved constant in R. |

• We should avoid creating non-syntactic names, if encountered:

  1. Use backticks to refer to them (eg.Flipper Length (mm)) - or else errors will appear
  2. Use rename() to rewrite them to syntactic names

df <- tibble(country = c("Italy", "Germany", "France", "Italy", "United States"),
            `4 ideology` = c("communism", "fascism", "anarchism", "fascism", "capitalism"))

select(df, `4 ideology`)

## # A tibble: 5 × 1
##   `4 ideology`
##   <chr>       
## 1 communism   
## 2 fascism     
## 3 anarchism   
## 4 fascism     
## 5 capitalism

df <- rename(df, ideology = `4 ideology`) # REWRITES 4IDEOLOGY INTO IDEOLOGY

LECTURE 7

LECTURE 7: Missing Data & Factors

Missing Data

1. Explicit missing data
   • the missing value is recorded in the data, typically as NA
2. Implicit missing data
   • the missing value was never recorded in the data

Handling Missing Data

• Working with (explicit) missing data is usually a decision process:
  1. is.na() to find missing values
     • WE cannot use == NA to check for missing values: NA means “unknown,” so R cannot evaluate it. Use is.na() instead

data(penguins_raw)

## Warning in data(penguins_raw): data set 'penguins_raw' not found

# check for missing using base R syntax
is.na(penguins_raw$Sex)

##   [1] FALSE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE
##  [13] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
##  [25] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
##  [37] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE
##  [49] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
##  [61] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
##  [73] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
##  [85] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
##  [97] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [109] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [121] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [133] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [145] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [157] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [169] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE FALSE
## [181] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [193] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [205] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [217] FALSE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [229] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [241] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [253] FALSE FALSE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [265] FALSE FALSE FALSE FALSE  TRUE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE
## [277] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [289] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [301] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [313] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [325] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
## [337] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE

sum(is.na(penguins_raw$Sex))

## [1] 11

table(is.na(penguins_raw$Sex))

## 
## FALSE  TRUE 
##   333    11

# check for missing data using tidyverse syntax
penguins_raw |> summarize(sum(is.na(Sex)))

## # A tibble: 1 × 1
##   `sum(is.na(Sex))`
##               <int>
## 1                11

penguins_raw |> count(is.na(Sex)) 

## # A tibble: 2 × 2
##   `is.na(Sex)`     n
##   <lgl>        <int>
## 1 FALSE          333
## 2 TRUE            11

penguins_raw |> summarize(across(everything(), ~ sum(is.na(.)))) |> print(width = Inf)

## # A tibble: 1 × 17
##   studyName `Sample Number` Species Region Island Stage `Individual ID`
##       <int>           <int>   <int>  <int>  <int> <int>           <int>
## 1         0               0       0      0      0     0               0
##   `Clutch Completion` `Date Egg` `Culmen Length (mm)` `Culmen Depth (mm)`
##                 <int>      <int>                <int>               <int>
## 1                   0          0                    2                   2
##   `Flipper Length (mm)` `Body Mass (g)`   Sex `Delta 15 N (o/oo)`
##                   <int>           <int> <int>               <int>
## 1                     2               2    11                  14
##   `Delta 13 C (o/oo)` Comments
##                 <int>    <int>
## 1                  13      290

# USE WITH FILTER()
# keep rows where sex is missing
# filter(penguins_raw, is.na(Sex)) |> View()   # correct NOT PRINTING BC KNITTING ISSUES  
# filter(penguins_raw, Sex == NA)  |> View()   # incorrect

# keep rows where sex is NOT missing
filter(penguins_raw, !is.na(Sex))

## # A tibble: 333 × 17
##    studyName `Sample Number` Species         Region Island Stage `Individual ID`
##    <chr>               <dbl> <chr>           <chr>  <chr>  <chr> <chr>          
##  1 PAL0708                 1 Adelie Penguin… Anvers Torge… Adul… N1A1           
##  2 PAL0708                 2 Adelie Penguin… Anvers Torge… Adul… N1A2           
##  3 PAL0708                 3 Adelie Penguin… Anvers Torge… Adul… N2A1           
##  4 PAL0708                 5 Adelie Penguin… Anvers Torge… Adul… N3A1           
##  5 PAL0708                 6 Adelie Penguin… Anvers Torge… Adul… N3A2           
##  6 PAL0708                 7 Adelie Penguin… Anvers Torge… Adul… N4A1           
##  7 PAL0708                 8 Adelie Penguin… Anvers Torge… Adul… N4A2           
##  8 PAL0708                13 Adelie Penguin… Anvers Torge… Adul… N7A1           
##  9 PAL0708                14 Adelie Penguin… Anvers Torge… Adul… N7A2           
## 10 PAL0708                15 Adelie Penguin… Anvers Torge… Adul… N8A1           
## # ℹ 323 more rows
## # ℹ 10 more variables: `Clutch Completion` <chr>, `Date Egg` <date>,
## #   `Culmen Length (mm)` <dbl>, `Culmen Depth (mm)` <dbl>,
## #   `Flipper Length (mm)` <dbl>, `Body Mass (g)` <dbl>, Sex <chr>,
## #   `Delta 15 N (o/oo)` <dbl>, `Delta 13 C (o/oo)` <dbl>, Comments <chr>

2. na.rm = TRUE to ignore/exclude missing values in calculations
   • often used with summarize() to calculate mean, sum, SD
   • does not remove missing data - skips missing values for that calculation ONLY

penguins_raw |> summarize(avg_mass = mean(`Body Mass (g)`, na.rm = TRUE))

## # A tibble: 1 × 1
##   avg_mass
##      <dbl>
## 1    4202.

penguins_raw |> summarize(sum_mass = sum(`Body Mass (g)`, na.rm = TRUE))

## # A tibble: 1 × 1
##   sum_mass
##      <dbl>
## 1  1437000

3. drop_na() to remove missing values (also entire row); use carefully

penguins_mass <- penguins_raw |>
  drop_na(`Body Mass (g)`)

penguins_mass |>
  summarize(avg_mass = mean(`Body Mass (g)`)) # NO MISSING MASS NOW

## # A tibble: 1 × 1
##   avg_mass
##      <dbl>
## 1    4202.

# BOTH COLUMNS MUST BE NON-MISSING
penguins_raw |>
  drop_na(Sex, `Body Mass (g)`)

## # A tibble: 333 × 17
##    studyName `Sample Number` Species         Region Island Stage `Individual ID`
##    <chr>               <dbl> <chr>           <chr>  <chr>  <chr> <chr>          
##  1 PAL0708                 1 Adelie Penguin… Anvers Torge… Adul… N1A1           
##  2 PAL0708                 2 Adelie Penguin… Anvers Torge… Adul… N1A2           
##  3 PAL0708                 3 Adelie Penguin… Anvers Torge… Adul… N2A1           
##  4 PAL0708                 5 Adelie Penguin… Anvers Torge… Adul… N3A1           
##  5 PAL0708                 6 Adelie Penguin… Anvers Torge… Adul… N3A2           
##  6 PAL0708                 7 Adelie Penguin… Anvers Torge… Adul… N4A1           
##  7 PAL0708                 8 Adelie Penguin… Anvers Torge… Adul… N4A2           
##  8 PAL0708                13 Adelie Penguin… Anvers Torge… Adul… N7A1           
##  9 PAL0708                14 Adelie Penguin… Anvers Torge… Adul… N7A2           
## 10 PAL0708                15 Adelie Penguin… Anvers Torge… Adul… N8A1           
## # ℹ 323 more rows
## # ℹ 10 more variables: `Clutch Completion` <chr>, `Date Egg` <date>,
## #   `Culmen Length (mm)` <dbl>, `Culmen Depth (mm)` <dbl>,
## #   `Flipper Length (mm)` <dbl>, `Body Mass (g)` <dbl>, Sex <chr>,
## #   `Delta 15 N (o/oo)` <dbl>, `Delta 13 C (o/oo)` <dbl>, Comments <chr>

# CHECKS ALL COLUMNS:
# penguins_raw |> drop_na()
# MISSING COMMENTS COULD REMOVE OTHERWISE USEFUL OBSERVATIONS!

Handling Missing Data Routine Example

1. First, rename: use the penguins_raw dataset and rename Flipper Length (mm) to flipper_length_mm. Save the result as a new dataframe; call it penguins_clean or p

# renaming
penguins_clean <- penguins_raw |>
  rename(flipper_length_mm = `Flipper Length (mm)`) ## NEED TO USE =!!!

2. Second, check missing data: use the new dataframe for the steps below (copy and adapt code from these slides):
   • Detect missing values: use is.na() and sum() to count how many are missing in flipper length

sum(is.na(penguins_clean$flipper_length_mm))

## [1] 2

• Exclude from calculations: use na.rm = TRUE inside mean() to calculate the average flipper length

penguins_clean |> 
  summarize(avg_flipper_length = mean(flipper_length_mm, na.rm = T))

## # A tibble: 1 × 1
##   avg_flipper_length
##                <dbl>
## 1               201.

• Drop missing values: use drop_na() to remove rows with missing values in flipper length

penguins_clean_flipperlength <- penguins_clean |>
  drop_na(flipper_length_mm)

# CHECKING
penguins_clean_flipperlength |> 
  summarize(avg_flipper_length = mean(flipper_length_mm))

## # A tibble: 1 × 1
##   avg_flipper_length
##                <dbl>
## 1               201.

# OTHER WAY TO CHECK
sum(is.na(penguins_clean_flipperlength$flipper_length_mm)) # RETURNING 0

## [1] 0

Fill or Replace Missing Data

Function	What it does	Example
replace_na()	Replaces NA with a specified value	replace_na(x, 0)
fill()	Carries values forward or backward in row order	fill(DATA, group, .direction = "down")
coalesce()	Takes the first non-missing value at each position	coalesce(primary, backup)

• Choose based on what missing means; missing is not automatically zero
• fill() depends on row order

replace_na(c(2, NA, 4), 0) # 2, 0, 4; ONLY IF ZERO IS JUSTIFIED

## [1] 2 0 4

record <- tibble(
  group = c("A", NA, "B", NA),
  value = 1:4
)

record |>
  fill(group, .direction = "down") # A, A, B, B

## # A tibble: 4 × 2
##   group value
##   <chr> <int>
## 1 A         1
## 2 A         2
## 3 B         3
## 4 B         4

# OTHER DIRECTIONS: "up", "downup", "updown"

coalesce(c(NA, 2, NA), c(1, 9, NA))

## [1]  1  2 NA

# 1, 2, NA: FIRST AVAILABLE VALUE AT EACH POSITION

Factors

• A factor is a specific data structure in R for categorical data
• Categories come from a limited set of labels
  • months, Likert scales, education, political parties, species, etc.
• Factors allow us to control category order
  • natural order, e.g., January → December
  • order chosen for analysis, e.g., most → least frequent

Structure	Stores categories as	Sorting	When useful
Character vector	Text labels	Alphabetical by default	Plain text
Numeric vector	Numeric codes	Numeric order	Categories recorded as 1, 2, 3
Factor	Values with defined levels	Level order	Control category order in sorting and plots

• Character vectors sort alphabetically by default
• Setting factor levels does not rearrange observations by itself
• sort() sorts vector values; arrange() sorts dataframe rows

1. Character → Factor

x1 <- c("Dec", "Apr", "Jan", "Mar")
class(x1) # "character"

## [1] "character"

month_levels <- c(
  "Jan", "Feb", "Mar", "Apr", "May", "Jun",
  "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"
)

y1 <- factor(x1, levels = month_levels)

class(y1)  # "factor"

## [1] "factor"

levels(y1) # ALL 12 POSSIBLE LEVELS

##  [1] "Jan" "Feb" "Mar" "Apr" "May" "Jun" "Jul" "Aug" "Sep" "Oct" "Nov" "Dec"

sort(x1) # Apr, Dec, Jan, Mar: ALPHABETICAL

## [1] "Apr" "Dec" "Jan" "Mar"

sort(y1) # Jan, Mar, Apr, Dec: CHRONOLOGICAL

## [1] Jan Mar Apr Dec
## Levels: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

# IF WANT REVERSE ORDER, USE REV()
factor(x1, levels = rev(month_levels))

## [1] Dec Apr Jan Mar
## Levels: Dec Nov Oct Sep Aug Jul Jun May Apr Mar Feb Jan

# y1 ITSELF STILL HAS VALUES IN THE ORIGINAL OBSERVATION ORDER

2. Numeric → Factor

x2 <- c(12, 4, 1, 3)

month_labels <- c(
  "Jan", "Feb", "Mar", "Apr", "May", "Jun",
  "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"
)

y2 <- factor(
  x2,
  levels = 1:12,         # ORIGINAL NUMERIC VALUES
  labels = month_labels # DISPLAY LABELS IN MATCHING ORDER
)

y2         # Dec, Apr, Jan, Mar

## [1] Dec Apr Jan Mar
## Levels: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

class(y2)  # "factor"

## [1] "factor"

levels(y2) # Jan THROUGH Dec

##  [1] "Jan" "Feb" "Mar" "Apr" "May" "Jun" "Jul" "Aug" "Sep" "Oct" "Nov" "Dec"

sort(x2) # 1, 3, 4, 12

## [1]  1  3  4 12

sort(y2) # Jan, Mar, Apr, Dec

## [1] Jan Mar Apr Dec
## Levels: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Levels vs Labels

Argument	What it means	Character input	Numeric input
levels	Expected original values, in the desired category order	Use original text values: c("Low", "Medium", "High")	Use original numeric codes: c(1, 2, 3)
labels	Display names matched to levels by position	Optional; omit to keep original names, or supply new names	Optional; use c("Low", "Medium", "High") to label codes: 1 → Low, 2 → Medium, 3 → High
Typical use	Choose based on whether names need changing	Usually only levels: names already readable	Usually levels + labels: order codes and add readable names
If labels omitted	Original values become the displayed labels	"Low" stays "Low"	1 displays as "1"; still a factor category
Common mistake	levels must match the original values	Numeric levels will not match text like "Low" → NA	Text levels like "Low" will not match codes like 1 → NA

• Values not included in levels become NA
• When providing a vector of labels, use one label per level in matching order
• Without specified levels, factor() generally uses sorted unique values
• Numeric factors can retain numeric-looking labels; add labels for readable names
• levels(f) lists categories, not the observed values

a <- c("Low", "High", "Medium", "Low")
b <- factor(a, levels = c("Low", "Medium", "High"))
sort(b) # Low, Low, Medium, High

## [1] Low    Low    Medium High  
## Levels: Low Medium High

i <- c(1, 3, 2, 1)
j <- factor(
  i,
  levels = c(1, 2, 3),
  labels = c("Low", "Medium", "High") # ASSIGNING NUMBERS MEANING THRU LABELS
)
sort(j) # Low, Low, Medium, High

## [1] Low    Low    Medium High  
## Levels: Low Medium High

sort() vs arrange()

Function	Input → Output	Ascending	Descending
sort()	Vector → vector	sort(x)	sort(x, decreasing = TRUE)
arrange()	Dataframe → dataframe	arrange(DATA, var)	arrange(DATA, desc(var))

• NA: sort() removes NA by default; use na.last = TRUE to keep them at the end. arrange() keeps NA rows at the end for local dataframes.

Common Factor Errors

Error	Why it fails	Fix
Character months + numeric levels 1:12	"Jan" does not match 1	Use character month names as levels
12 levels + 4 labels	Label count does not match level count	Supply matching labels
Numeric months include 12, but levels only 1:4	12 is excluded and becomes NA	Include the actual original values
Levels 1:4, labels Jan/Mar/Apr/Dec	Labels map by position; codes are mislabelled	Match each code to its correct month

• character to factor

# Define a character vector representing months
m <- c("Dec", "Apr", "Jan", "Mar")
class(m)

## [1] "character"

# Attempt
mf <- factor(m, 
            levels = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12),
            labels = c("Jan", "Feb", "Mar", "Apr", "May", "Jun",
                       "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"))

mf # NAs APPEAR BECAUSE 1-12 CANNOT REPRESENT JAN-DEC AS IT IS NOT ASSIGNED YET

## [1] <NA> <NA> <NA> <NA>
## Levels: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

# WE CAN DO
MF <- factor(m, 
             levels = c("Jan", "Feb", "Mar", "Apr", "May", "Jun",
                       "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"))

# Compare sorting
sort(m)  

## [1] "Apr" "Dec" "Jan" "Mar"

sort(mf) # NOT SHOWING ANYTHING

## factor()
## Levels: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

sort(MF) # NOW SORTED!

## [1] Jan Mar Apr Dec
## Levels: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

• numeric to factor

# Define a numeric vector representing months
x <- c(12, 4, 1, 3, 1)
class(x)

## [1] "numeric"

# Attempt 1 INCORRECT
# y1 <- factor(x, levels = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12), 
#                 labels = c("Jan", "Mar", "Apr", "Dec")) # LENGTH NOT ALIGNED
              
# Attempt 2 INCORRECT
y2 <- factor(x, levels = c(1, 2, 3, 4), # ONLY 1:4 NOT UNTIL 12, MAKING 12 NA
                labels = c("Jan", "Mar", "Apr", "Dec"))

Y <- factor(x,
            levels = 1:12,
            labels = c("Jan", "Feb", "Mar", "Apr", "May", "Jun",
                       "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"))

sort(Y)

## [1] Jan Jan Mar Apr Dec
## Levels: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Factors in Dataframes

• Check column type and values first
• Use factor() inside mutate() to change a dataframe column
• Bar order comes from factor levels, not dataframe row order

library(rcis)
data(scorecard)

class(scorecard$type)

## [1] "factor"

distinct(scorecard, type)

## # A tibble: 3 × 1
##   type               
##   <fct>              
## 1 Public             
## 2 Private, nonprofit 
## 3 Private, for-profit

s <- scorecard |>
  mutate(
    type = factor(
      type,
      levels = c("Private, nonprofit", "Public", "Private, for-profit")
    )
  )

ggplot(s, aes(x = type)) +
  geom_bar()

# COUNTS SCHOOLS; BARS FOLLOW THE LEVEL ORDER

Get Levels from Data

• For many categories, use distinct() + arrange() to build a code-label lookup
• Keep codes and labels paired, then pull() the columns

survey <- tibble(
  id = 1:6,
  edu_code = c(3, 1, 5, 2, 4, 3),
  edu_label = c(
    "Bachelor", "Less than HS", "Doctorate",
    "High school", "Master's", "Bachelor"
  )
)

edu_lookup <- survey |>
  distinct(edu_code, edu_label) |>
  arrange(edu_code)

survey <- survey |>
  mutate(
    edu = factor(
      edu_code,
      levels = pull(edu_lookup, edu_code),
      labels = pull(edu_lookup, edu_label)
    )
  )

levels(survey$edu)

## [1] "Less than HS" "High school"  "Bachelor"     "Master's"     "Doctorate"

# ASSUMES EACH CODE HAS ONE CONSISTENT LABEL

• With many categories, get factor levels and labels from the data: use distinct(code, label) to keep unique code–label pairs, arrange(code) to order them together, and pull() to extract each column as a vector for factor(code, levels = ..., labels = ...). Labels match levels by position, so keep their order aligned

forcats Functions

• factor() is base R
• forcats is the tidyverse package for working with factors

Function	What it does	When to use	Notes
factor(x, levels = ...)	Creates factor with specified levels	Define categories and order	Excluded values become NA
fct_relevel()	Manually reorders levels	Custom category order	Unmentioned existing levels remain after named ones
fct_infreq()	Orders by frequency	Most common category first	Most → least frequent
fct_reorder(x, y)	Orders by a summary of another variable	Order categories by numeric values	Default: median of y, ascending

# MANUAL ORDER (fct_relevel)
scorecard |>
  mutate(
    type = fct_relevel(
      type, "Private, nonprofit", "Public", "Private, for-profit"
    )
  ) |>
  ggplot(aes(x = type)) +
  geom_bar()

# FREQUENCY ORDER (fct_infreq)
scorecard |>
  mutate(type = fct_infreq(type)) |>
  ggplot(aes(x = type)) +
  geom_bar()

# fct_infreq() USES HOW OFTEN EACH CATEGORY OCCURS
# IT DOES NOT ORDER BY THE SUM OF ANOTHER VARIABLE

# ORDER BY ANOTHER VARIABLE (fct_reorder)
# ORDER SCHOOL TYPES BY MEDIAN COST: LOWEST -> HIGHEST
scorecard |>
  drop_na(cost) |>
  group_by(type) |>
  summarize(median_cost = median(cost), .groups = "drop") |>
  mutate(type = fct_reorder(type, median_cost)) |>
  ggplot(aes(x = type, y = median_cost)) +
  geom_col()

# type = CATEGORIES TO REORDER
# median_cost = NUMERIC VARIABLE THAT DETERMINES THE ORDER
# geom_col(): BAR HEIGHT SHOWS median_cost, NOT ROW COUNTS
# ADD .desc = TRUE FOR HIGHEST -> LOWEST:
# fct_reorder(type, median_cost, .desc = TRUE)

Practice: Tips by Weekday

• Goal: weekdays on x, total tips on y, chronological weekday order

Function	Bar height	Aesthetics
geom_bar()	Row count; default stat = “count”	x
geom_bar(stat = "identity")	Supplied y values; stacked by default	x and y
geom_col()	Same identity behavior	x and y

tips <- tibble(
  week = c("Mon", "Wed", "Fri", "Wed", "Thu", "Sat", "Sat"),
  tip = c(10, 12, 20, 8, 25, 25, 30)
)

# WRONG FOR TOTAL TIP AMOUNTS
ggplot(tips, aes(x = week)) +
  geom_bar()

# SATURDAY HEIGHT = 2 ROWS, NOT $55

# FIX HEIGHT + ORDER USING factor()
tips |>
  mutate(
    week = factor(
      week,
      levels = c("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun")
    )
  ) |>
  ggplot(aes(x = week, y = tip)) +
  geom_bar(stat = "identity") +
  labs(title = "Tips by Weekday", x = "Weekday", y = "Tip ($)")

# SATURDAY: 25 + 30 STACK TO 55
# WEDNESDAY: 12 + 8 STACK TO 20
# UNUSED Tue/Sun LEVELS ARE DROPPED FROM THE AXIS BY DEFAULT

# SAME USING fct_relevel()
days <- c("Mon", "Wed", "Thu", "Fri", "Sat")

tips |>
  mutate(week = fct_relevel(week, days)) |>
  ggplot(aes(x = week, y = tip)) +
  geom_col() +
  labs(title = "Tips by Weekday", x = "Weekday", y = "Tip ($)")

• No labels needed: weekday values are already readable
• With factor(), include the possible levels you want to define
• With fct_relevel(), reorder existing levels; no need to invent absent categories

# ORDER BY TOTAL TIP, NOT WEEKDAY
tips |>
  group_by(week) |>
  summarize(total_tip = sum(tip), .groups = "drop") |>
  mutate(week = fct_reorder(week, total_tip, .desc = TRUE)) |>
  ggplot(aes(x = week, y = total_tip)) +
  geom_col()

# ONE ROW PER WEEKDAY; LARGEST TOTAL FIRST
# .fun CAN CHANGE THE SUMMARY USED BY fct_reorder()

REVIEW

Question	Answer
Why not x == NA?	Unknown comparison returns NA; use is.na(x)
Does na.rm = TRUE remove rows?	No; only skips NA in that calculation
Why use drop_na() carefully?	It removes entire rows, including useful values
What controls category order?	Factor levels
Do factor levels reorder observations?	No; use sort() or arrange()
Why did conversion create NA?	Original values were not included in levels
levels vs labels?	Original values/order vs displayed names
fct_infreq() vs fct_reorder()?	Frequency vs summary of another variable
Why stat = "identity" for tips?	Need tip amounts rather than row counts

LECTURE 8

LECTURE 8: Tidy Data, Strings & Regular Expressions

Tidy Data

• Tidy Data is a standard way of mapping the meaning of a dataset to its structure
• Three Principles:
  1. each variable forms a column
  2. each observation forms a row
  3. each cell is a single measurement
• most real world data is not stored in a tidy format
  • many datasets are designed to make data entry, not analysis, easier
  • tidy data principles take time and practice to master
• When working with the tidyverse (ggplot2, dplyr, etc.), start by putting data in tidy format. Most tidyverse functions assume data are tidy

Tidying Untidy Data

1. Diagnose
   • identify variables
   • identify observations
   • spot what is untidy
2. Fix with tidyr
   • each variable → column
   • each observation → row
   • each cell → one value

• For longitudinal/panel data, one observation may be a country-year pair
• Long format does not automatically mean tidy

Problem	Function	What it does	Key arguments
Variable values are column names	pivot_longer()	Columns → rows	cols, names_to, values_to
Variable names are stored in rows	pivot_wider()	Rows → columns	names_from, values_from
Multiple variables in one column	separate()	Splits one column into several	col, into, sep, convert
One variable split across columns	unite()	Combines columns into one	col, input columns, sep

1. pivot_longer()

• table4a: 1999 and 2000 are values of year, not separate variables
• Each row contains two country-year observations
• names_to receives old column names
• values_to receives old cell values

table4a # BUILT-IN tidyr DATA: 3 ROWS x 3 COLUMNS

## # A tibble: 3 × 3
##   country     `1999` `2000`
##   <chr>        <dbl>  <dbl>
## 1 Afghanistan    745   2666
## 2 Brazil       37737  80488
## 3 China       212258 213766

cases_long <- table4a |>
  pivot_longer(
    cols = c(`1999`, `2000`), # SELECT COLUMNS TO PIVOT
    names_to = "year",       # NEW COLUMN FOR OLD HEADERS
    values_to = "cases"      # NEW COLUMN FOR THEIR VALUES
  )

cases_long

## # A tibble: 6 × 3
##   country     year   cases
##   <chr>       <chr>  <dbl>
## 1 Afghanistan 1999     745
## 2 Afghanistan 2000    2666
## 3 Brazil      1999   37737
## 4 Brazil      2000   80488
## 5 China       1999  212258
## 6 China       2000  213766

# 6 ROWS x 3 COLUMNS: country, year, cases
# BACKTICKS NEEDED FOR COLUMN NAMES STARTING WITH NUMBERS

# MORE FLEXIBLE: ALL COLUMNS EXCEPT THE IDENTIFIER
cases_long <- table4a |>
  pivot_longer(
    cols = -country,
    names_to = "year",
    values_to = "cases"
  ) |>
  mutate(year = as.integer(year))
# OLD HEADERS START AS CHARACTER VALUES

cases_long |>
  group_by(year) |>
  summarize(total_cases = sum(cases), .groups = "drop")

## # A tibble: 2 × 2
##    year total_cases
##   <int>       <dbl>
## 1  1999      250740
## 2  2000      296920

# NO NEED TO WRITE A SEPARATE SUMMARY FOR EVERY YEAR COLUMN

2. pivot_wider()

• table2: type stores variable names (cases, population)
• Each country-year observation is spread across two rows
• names_from supplies new column names
• values_from supplies values for those columns

table2

## # A tibble: 12 × 4
##    country      year type            count
##    <chr>       <dbl> <chr>           <dbl>
##  1 Afghanistan  1999 cases             745
##  2 Afghanistan  1999 population   19987071
##  3 Afghanistan  2000 cases            2666
##  4 Afghanistan  2000 population   20595360
##  5 Brazil       1999 cases           37737
##  6 Brazil       1999 population  172006362
##  7 Brazil       2000 cases           80488
##  8 Brazil       2000 population  174504898
##  9 China        1999 cases          212258
## 10 China        1999 population 1272915272
## 11 China        2000 cases          213766
## 12 China        2000 population 1280428583

# 12 ROWS x 4 COLUMNS: country, year, type, count

tb_wide <- table2 |>
  pivot_wider(
    names_from = type,
    values_from = count
  )

tb_wide

## # A tibble: 6 × 4
##   country      year  cases population
##   <chr>       <dbl>  <dbl>      <dbl>
## 1 Afghanistan  1999    745   19987071
## 2 Afghanistan  2000   2666   20595360
## 3 Brazil       1999  37737  172006362
## 4 Brazil       2000  80488  174504898
## 5 China        1999 212258 1272915272
## 6 China        2000 213766 1280428583

# 6 ROWS x 4 COLUMNS: country, year, cases, population
# country + year IDENTIFY EACH OUTPUT ROW

tb_wide |>
  mutate(
    rate = cases / population,
    rate_per_100000 = rate * 100000
  )

## # A tibble: 6 × 6
##   country      year  cases population      rate rate_per_100000
##   <chr>       <dbl>  <dbl>      <dbl>     <dbl>           <dbl>
## 1 Afghanistan  1999    745   19987071 0.0000373            3.73
## 2 Afghanistan  2000   2666   20595360 0.000129            12.9 
## 3 Brazil       1999  37737  172006362 0.000219            21.9 
## 4 Brazil       2000  80488  174504898 0.000461            46.1 
## 5 China        1999 212258 1272915272 0.000167            16.7 
## 6 China        2000 213766 1280428583 0.000167            16.7

pivot_longer() vs pivot_wider()

Compare	pivot_longer()	pivot_wider()
Direction	Usually more rows, fewer columns	Usually fewer rows, more columns
Names	Headers become values	Values become headers
Name argument	names_to = "year"	names_from = type
Value argument	values_to = "cases"	values_from = count
Quotes	New names are strings	Existing columns usually use bare names

3. separate()

• table3: rate contains text such as "745/19987071"
• This stores two measurements in one cell, not a calculated numeric rate
• Split into cases and population, then calculate the rate

table3

## # A tibble: 6 × 3
##   country      year rate             
##   <chr>       <dbl> <chr>            
## 1 Afghanistan  1999 745/19987071     
## 2 Afghanistan  2000 2666/20595360    
## 3 Brazil       1999 37737/172006362  
## 4 Brazil       2000 80488/174504898  
## 5 China        1999 212258/1272915272
## 6 China        2000 213766/1280428583

tidy_table3 <- table3 |>
  separate(
    col = rate,
    into = c("cases", "population"), # NEW COLUMN NAMES
    sep = "/",                      # SPLIT AT SLASH
    convert = TRUE                  # NUMERIC TEXT -> NUMBERS
  )

tidy_table3

## # A tibble: 6 × 4
##   country      year  cases population
##   <chr>       <dbl>  <int>      <int>
## 1 Afghanistan  1999    745   19987071
## 2 Afghanistan  2000   2666   20595360
## 3 Brazil       1999  37737  172006362
## 4 Brazil       2000  80488  174504898
## 5 China        1999 212258 1272915272
## 6 China        2000 213766 1280428583

tidy_table3 |>
  mutate(rate = cases / population) # THE REAL RATE CALCULATED

## # A tibble: 6 × 5
##   country      year  cases population      rate
##   <chr>       <dbl>  <int>      <int>     <dbl>
## 1 Afghanistan  1999    745   19987071 0.0000373
## 2 Afghanistan  2000   2666   20595360 0.000129 
## 3 Brazil       1999  37737  172006362 0.000219 
## 4 Brazil       2000  80488  174504898 0.000461 
## 5 China        1999 212258 1272915272 0.000167 
## 6 China        2000 213766 1280428583 0.000167

# DEFAULT convert = FALSE: OUTPUT COLUMNS STAY CHARACTER
# DEFAULT remove = TRUE: ORIGINAL COLUMN IS REMOVED
# WITHOUT sep, separate() USES NON-ALPHANUMERIC SEPARATORS

4. unite()

• table5: year is split into century and year
• "19" and "99" should become "1999"
• Default separator is "_"; use sep = "" to join without a separator

table5 |>
  unite(col = "year", century, year)

## # A tibble: 6 × 3
##   country     year  rate             
##   <chr>       <chr> <chr>            
## 1 Afghanistan 19_99 745/19987071     
## 2 Afghanistan 20_00 2666/20595360    
## 3 Brazil      19_99 37737/172006362  
## 4 Brazil      20_00 80488/174504898  
## 5 China       19_99 212258/1272915272
## 6 China       20_00 213766/1280428583

# DEFAULT: "19_99", "20_00"

table5 |>
  unite(col = "year", century, year, sep = "") |> # sep = "" TO REMOVE _ BTWN NUMBERS
  mutate(year = as.integer(year)) # as.numeric() ALSO WORKS HERE!

## # A tibble: 6 × 3
##   country      year rate             
##   <chr>       <int> <chr>            
## 1 Afghanistan  1999 745/19987071     
## 2 Afghanistan  2000 2666/20595360    
## 3 Brazil       1999 37737/172006362  
## 4 Brazil       2000 80488/174504898  
## 5 China        1999 212258/1272915272
## 6 China        2000 213766/1280428583

# "1999" TEXT -> 1999 INTEGER

# INPUT ORDER MATTERS
# unite() JOINS TEXT; IT DOES NOT ADD NUMBERS
# DEFAULT remove = TRUE: INPUT COLUMNS ARE REPLACED

Chain tidyr Functions

(same example but together)

tb_tidy <- table5 |>
  unite(col = "year", century, year, sep = "") |>
  mutate(year = as.integer(year)) |>
  separate(
    col = rate,
    into = c("cases", "population"),
    sep = "/",
    convert = TRUE
  )

tb_tidy

## # A tibble: 6 × 4
##   country      year  cases population
##   <chr>       <int>  <int>      <int>
## 1 Afghanistan  1999    745   19987071
## 2 Afghanistan  2000   2666   20595360
## 3 Brazil       1999  37737  172006362
## 4 Brazil       2000  80488  174504898
## 5 China        1999 212258 1272915272
## 6 China        2000 213766 1280428583

# 6 ROWS: ONE PER COUNTRY-YEAR
# 4 COLUMNS: country, year, cases, population
# unite() ALONE WOULD LEAVE rate UNTIDY

Code	Function	Meaning	Example
sep = ""	unite()	Joins values with nothing between them	"19" + "99" → "1999"
sep = " "	unite()	Joins values with a space between them	"19" + "99" → "19 99"
sep = "_"	unite()	Joins values with an underscore; default	"19" + "99" → "19_99"
sep = "/"	separate()	Splits a value at the slash	"745/19987071" → "745" and "19987071"

String & Character Vector

• A string is a sequence of characters enclosed in double " or single ' quotes
• Can contain letters, numbers, symbols, and whitespace
• A character vector is a collection of strings
  • each element is one string
  • a single string is also a character vector of length one

string <- "Ciao, my name is Sabrina and I am 99 years old!!"

class(string)  # "character"
length(string) # 1 STRING
nchar(string)  # NUMBER OF CHARACTERS

char_vect <- c("Ciao come stai?", "Ciao", "Hello", "etc.")

length(char_vect) # 4 STRINGS
nchar(char_vect)  # CHARACTER COUNT FOR EACH STRING - N N N N

quote_example <- 'She said "Hello"'
# USE THE OTHER QUOTE TYPE INSIDE A STRING
length(quote_example) # STILL 1

Regular Expressions (REGEX)

• Strings containing normal characters and/or special meta-characters
• Describe a specific pattern to match in text
• A pattern-matching language used in R, Python, etc.
• Uses:
  • extract specific text
  • replace text
  • verify formats, e.g., email or phone number
    • is a number/character exists in the string
• stringr is a package, not a function called stringr()
• Common syntax: function(string, pattern, ...)
  • eg.str_detect("Hello Sabrina", "Sabrina")→ TRUE, bc Sabrina exists in the line

words <- c("apple", "banana", "pear", "Apple", "grape")

str_detect(words, "app")

## [1]  TRUE FALSE FALSE FALSE FALSE

# TRUE, FALSE, FALSE, FALSE, FALSE
# NORMAL CHARACTERS MATCH LITERAL TEXT
# DEFAULT MATCHING IS CASE-SENSITIVE

str_detect(words, regex("app", ignore_case = TRUE)) #REGEX() ADDS SETTING TO SEARCH

## [1]  TRUE FALSE FALSE  TRUE FALSE

# TRUE, FALSE, FALSE, TRUE, FALSE

# SLIDE EXAMPLE: EMAIL-LIKE PATTERN
email_pattern <- "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}"

str_extract("Contact: student@example.com", email_pattern)

## [1] "student@example.com"

# "student@example.com"
# TEACHING EXAMPLE, NOT COMPLETE EMAIL VALIDATION

stringr Functions

Function	What it does / returns	No match	Notes
str_view()	Displays strings and highlights matches	Depends on display settings	Inspect patterns visually
str_detect()	Logical vector: match exists?	FALSE	One result per string
str_count()	Number of matches	0	One count per string
str_extract()	First match in each string	NA	Character vector
str_extract_all()	All matches in each string	Empty character vector	List of character vectors
str_replace()	Replaces first match in each string	Original string	Character vector
str_replace_all()	Replaces all matches	Original string	Character vector
str_split()	Splits each string at a pattern	Original string as one piece	List of character vectors

• Version note: slide 33 describes str_view() as showing the first match. Current stringr highlights all matches; str_view_all() is deprecated.
• Use str_extract() when you need the first matched text.
• Reference: str_view documentation

str_view(c("banana", "pear", "sky"), "a")
# VIEW MATCH LOCATIONS

str_view(c("banana", "pear", "sky"), "a", match = NA)
# SHOW ALL STRINGS, INCLUDING THOSE WITHOUT MATCHES

Regex Meta-characters

• Examples below are R strings ready to use as patterns

Symbol	Meaning	R pattern	Example
.	Any single character except newline by default	"a.c"	abc, a-c, a.c
^	Start of string	"^cat"	cat, cats; not bobcat
$	End of string	"cat$"	cat, bobcat; not cats
^...$	Whole string	"^cat$"	Only cat
\|	OR	"cat\|dog"	Either alternative
[]	One character from a set/range	"[aeiou]", "[a-z]"	One vowel / lowercase ASCII letter
[^...]	One character NOT in the set	"[^0-9]"	One non-digit
()	Groups part of the pattern	"(cat\|dog)s"	cats or dogs

animals <- c("cat", "cats", "bobcat", "dog")

str_detect(animals, "^cat")

## [1]  TRUE  TRUE FALSE FALSE

# TRUE, TRUE, FALSE, FALSE

str_detect(animals, "cat$")

## [1]  TRUE FALSE  TRUE FALSE

# TRUE, FALSE, TRUE, FALSE

str_detect(animals, "^cat$")

## [1]  TRUE FALSE FALSE FALSE

# TRUE, FALSE, FALSE, FALSE

str_detect(animals, "cat|dog")

## [1] TRUE TRUE TRUE TRUE

# ALL TRUE: EITHER PATTERN ANYWHERE

str_extract_all("banana", "[aeiou]")

## [[1]]
## [1] "a" "a" "a"

# a, a, a: [] MATCHES ONE CHARACTER AT A TIME

str_detect(c("a.c", "abc"), "a.c")

## [1] TRUE TRUE

# TRUE, TRUE: DOT IS A WILDCARD

str_detect(c("a.c", "abc"), "a\\.c")

## [1]  TRUE FALSE

# TRUE, FALSE: ESCAPED DOT IS LITERAL

# ^ OUTSIDE [] = START OF STRING
# ^ AT THE START INSIDE [] = NEGATION

Escaping & Character Classes

• In R strings, use two backslashes to send one backslash to regex
• Regex \d is written "\\d" in R
• Escape special symbols when matching them literally, e.g., "\\."

Regex	Meaning	Write in R
\d	Digit	"\\d"
\D	Non-digit	"\\D"
\w	Word character: letters, digits, underscore, etc.	"\\w"
\W	Non-word character	"\\W"
\s	Whitespace: space, tab, newline, etc.	"\\s"
\S	Non-whitespace	"\\S"

str_extract_all("ID_12 A!", "\\d")

## [[1]]
## [1] "1" "2"

# "1", "2"

str_extract_all("ID_12 A!", "\\w")

## [[1]]
## [1] "I" "D" "_" "1" "2" "A"

# I, D, _, 1, 2, A

str_extract_all("ID_12 A!", "\\W")

## [[1]]
## [1] " " "!"

# SPACE AND !

str_count("A B\tC", "\\s")

## [1] 2

# 2 WHITESPACE CHARACTERS

str_replace_all("A  B\tC", "\\s+", " ")

## [1] "A B C"

# "A B C"

# \t IN AN R STRING IS A TAB
# \\s IS THE REGEX WHITESPACE CLASS

Regex Quantifiers

• Quantifiers repeat the preceding character, character class, or group

Quantifier	Meaning	R pattern	Example
*	Zero or more	"ab*c"	ac, abc, abbc
+	One or more	"ab+c"	abc, abbc; not ac
?	Zero or one	"colou?r"	color, colour
{n}	Exactly n	"^\\d{4}$"	Exactly four digits
{n,}	At least n	"^a{2,}$"	aa, aaa, …
{n,m}	Between n and m	"^a{2,4}$"	aa, aaa, aaaa

str_detect(c("ac", "abc", "abbc"), "ab*c")

## [1] TRUE TRUE TRUE

# TRUE, TRUE, TRUE

str_detect(c("ac", "abc", "abbc"), "ab+c")

## [1] FALSE  TRUE  TRUE

# FALSE, TRUE, TRUE

str_detect(c("color", "colour"), "colou?r")

## [1] TRUE TRUE

# TRUE, TRUE

str_extract_all("Room 12, floor 3", "\\d")

## [[1]]
## [1] "1" "2" "3"

# "1", "2", "3"

str_extract_all("Room 12, floor 3", "\\d+")

## [[1]]
## [1] "12" "3"

# "12", "3": CONSECUTIVE DIGITS AS ONE MATCH

str_detect(c("2026", "x2026", "20260"), "^\\d{4}$")

## [1]  TRUE FALSE FALSE

# TRUE, FALSE, FALSE
# ANCHORS REQUIRE THE WHOLE STRING TO MATCH

Detect, Count, Extract

text <- c("cat cat dog", "dog", "bird")

str_detect(text, "cat")

## [1]  TRUE FALSE FALSE

# TRUE, FALSE, FALSE

str_count(text, "cat")

## [1] 2 0 0

# 2, 0, 0

str_extract(text, "cat")

## [1] "cat" NA    NA

# "cat", NA, NA

str_extract_all(text, "cat")

## [[1]]
## [1] "cat" "cat"
## 
## [[2]]
## character(0)
## 
## [[3]]
## character(0)

# LIST:
# [[1]] "cat" "cat"
# [[2]] character(0)
# [[3]] character(0)

• Detect asks whether a match exists
• Count asks how many matches exist
• Extract returns the matching text
• _all() handles all matches within each string

Replace & Split

text <- c("cat cat dog", "dog", "bird")

str_replace(text, "cat", "fox")

## [1] "fox cat dog" "dog"         "bird"

# "fox cat dog", "dog", "bird"

str_replace_all(text, "cat", "fox")

## [1] "fox fox dog" "dog"         "bird"

# "fox fox dog", "dog", "bird"

parts <- str_split(c("A,B,C", "D,E"), ",")

parts

## [[1]]
## [1] "A" "B" "C"
## 
## [[2]]
## [1] "D" "E"

# LIST OF TWO CHARACTER VECTORS

parts[[1]]

## [1] "A" "B" "C"

# "A", "B", "C"
# [[1]] EXTRACTS THE FIRST VECTOR FROM THE LIST

Regex with dplyr

responses <- tibble(
  answer = c("age 21", "unknown", "age 34")
)

# DETECT -> KEEP MATCHING ROWS
responses |>
  filter(str_detect(answer, "\\d+"))

## # A tibble: 2 × 1
##   answer
##   <chr> 
## 1 age 21
## 2 age 34

# EXTRACT -> CREATE A COLUMN
responses |>
  mutate(
    age = as.integer(str_extract(answer, "\\d+"))
  )

## # A tibble: 3 × 2
##   answer    age
##   <chr>   <int>
## 1 age 21     21
## 2 unknown    NA
## 3 age 34     34

# age = 21, NA, 34
# EXTRACTED TEXT NEEDS CONVERSION BEFORE NUMERIC CALCULATIONS

REVIEW

Question	Answer
What should we identify before tidying?	Variables and the unit of observation
Year values are column names: which function?	pivot_longer()
Variable names are values of a column: which function?	pivot_wider()
Does long automatically mean tidy?	No; an observation may still span multiple rows
names_to vs names_from?	Name a new column vs select the column supplying new headers
separate() vs unite()?	Split one column vs combine columns
Why convert = TRUE?	Convert split numeric text into numbers
Why sep = "" in unite()?	Avoid the default underscore
length() vs nchar()?	Number of strings vs characters per string
str_detect() vs str_count()?	TRUE/FALSE vs number of matches
str_extract() vs str_extract_all()?	First match as a vector vs all matches in a list
What does "\\d+" match?	One or more consecutive digits
"." vs "\\."?	Wildcard vs literal dot
"cat" vs "^cat$"?	Match anywhere vs match the entire string
Does str_replace() replace every match?	No; use str_replace_all()

LECTURE 9

LECTURE 9: Data Import, Export & File Paths

Quiz Practice

library(tidyverse)

Question

A colleague sends you data on internet access (percent of households with an internet connection) in 3 countries, in a dataset called internet:

country	2010	2015	2020
Kenya	7	17	29
India	8	20	41
Brazil	42	59	80

internet <- tibble(
  country = c("Kenya", "India", "Brazil"),
  `2010`  = c(7, 8, 42),
  `2015`  = c(17, 20, 59),
  `2020`  = c(29, 41, 80)
)
internet

## # A tibble: 3 × 4
##   country `2010` `2015` `2020`
##   <chr>    <dbl>  <dbl>  <dbl>
## 1 Kenya        7     17     29
## 2 India        8     20     41
## 3 Brazil      42     59     80

You want to know the average internet access across countries in each year. An AI assistant computed it without changing the data:

internet |>
  summarize(mean_2010 = mean(`2010`),
            mean_2015 = mean(`2015`),
            mean_2020 = mean(`2020`))

## # A tibble: 1 × 3
##   mean_2010 mean_2015 mean_2020
##       <dbl>     <dbl>     <dbl>
## 1        19        32        50

The code runs and the output is correct.

Your colleague now sends an updated file with 40 countries and with every year from 2000 to 2024, in the same format: one row per country and one column per year.

(a) Explain the limitations of the AI assistant’s approach, especially for the new data. Which tidy data principle does the original data set violate and why?

A limitation is that

(b) What should you do instead? Write pseudo-code to tidy the data. Call the tidy dataset internet_tidy.

(c) Using your tidy data, write dplyr code that computes the average internet access across countries in each year. Compare it with the original AI code, and explain why your approach is better.

internet_tidy <- internet |>
  pivot_longer(cols = -country, 
               names_to = )

internet_tidy |>
  group_by(year) |>
  summarize(mean_access = mean(access), na.rm = T)

Relational Data with dplyr will not be included in exam 1

Importing files into R

• read_csv("file.csv") helps us to import files into R
  • This comes from readr (tidyverse), faster than read.csv
• Options to load in csv:

library(readr)
library(tidyverse)

# Option 1 (not best): Use the full file path (workbench example)
read_csv(file = "/home/graceyin/Lecture Materials/testdata.csv")

# Option 2 (best!): Use only a relative path
read_csv(file = "testdata.csv")

# Check your current working directory
getwd()

# Option 3 (not best): Choose the file manually
# Not recommended for reproducible code
read_csv(file = file.choose())

• if run into error:
  1. Find current working directory using getwd()
  2. Move the folder to current working directory
  3. rerun the code
• Paths
  1. Relative path means relative to (e.g., it starts from) your R project folder
     • make codes easier to share and re-run
     • RStudio Projects make using relative paths easier
       • when opened, it automatically sets wd to the folder
  2. Absolute path means you specify the entire path to reach the file, independent from specific folders

Other readr functions to import data (not tested)

• Comma separated csv use read_csv() from the readr package
• Semi column separated csv use read_csv2()from the readr package
• Tab separated files use read_tsv()from the readr package
• RDS use readRDS() or read_rds()
• Excel use read_excel() from the readxl package
• SAS/SPSS/Stata use the haven package (several functions)

Exporting data using write_csv()

# import
test <- read_csv("testdata.csv", col_types = ("icic"), na = c("", "NA", "na", "None"))

# then you write your data analysis and visualization code etc.

# once done, export 
write_csv(test, file = "testdata_cleaned.csv")

Relational Data

• typically organised in relational databases set of multiple “tables” that are linked based on data common to them.
• single table = single data frame
• these tables provide meaningful insights only when combined together

Exam 1 Prep

Tables

1. R Markdown Syntax

Purpose	Syntax	Reminder
Bold	**text**	Two asterisks on each side
Italics	*text*	One asterisk on each side
Strikethrough	~~text~~	Two tildes on each side
Inline code	A backtick on each side	Use for function names or short code
Link	[text](URL)	Square brackets for text; parentheses for URL
Image		A link with ! at the beginning
Unordered list	+ item	Symbol followed by a space
Ordered list	1. item	Number, period, and space
Level-3 heading	### Heading	Three hashes followed by a space
Table	Pipes separate columns; a separator row follows the header	See the example below

__text__ also produces bold, not strikethrough.

| Tool | Use |
|---|---|
| R | Statistics |

2. R Markdown Chunk Options

Option	What it does	Default
eval = FALSE	Shows code without running it	eval = TRUE
include = FALSE	Runs code but hides both code and results	include = TRUE
echo = FALSE	Runs code and shows results, but hides code	echo = TRUE
error = TRUE	Displays errors instead of stopping rendering	error = FALSE
message = FALSE	Hides messages	message = TRUE
warning = FALSE	Hides warnings	warning = TRUE

3. Choosing a Graph

Graph type	ggplot2 function	Input variables	Purpose
Histogram	geom_histogram()	One continuous variable	Show its distribution
Bar chart	geom_bar()	One categorical variable	Count observations in each category
Stacked bar chart	geom_bar(aes(fill = group))	Two categorical variables	Compare category composition
Grouped bar chart	geom_bar(aes(fill = group), position = "dodge")	Two categorical variables	Compare counts side by side
Scatterplot	geom_point()	Two continuous variables	Show their relationship
Box plot	geom_boxplot()	One categorical + one continuous variable	Compare distributions across categories
Faceted scatterplot	geom_point() + facet_wrap()	Two continuous + one categorical variable	Compare relationships across panels
Grouped box plot	geom_boxplot(aes(fill = group))	One continuous + two categorical variables	Compare distributions across subgroups
Line chart	geom_line()	Usually two numeric variables, with an ordered x	Connect observations in x order

4. ggplot2: Mapping, Layers, and Statistics

Concept	Meaning	Example / Reminder
geom_point()	Creates a scatterplot	Each point represents an observation
Mapping an aesthetic	Connects a variable to a visual property inside aes()	aes(color = species)
Setting an aesthetic	Gives observations a fixed visual property outside aes()	geom_point(color = "blue")
aes(color = "blue")	Maps the constant text "blue" as a category	Does not directly set the points to blue
Global mapping	Mapping in ggplot() is inherited by layers	ggplot(df, aes(x, y))
Layer-specific mapping	Mapping applies only to that layer	geom_point(aes(color = education))
One overall regression line	Put categorical color mapping only in the point layer	Add geom_smooth(method = "lm")
Separate regression lines	Map categorical color globally so the smooth layer inherits it	ggplot(df, aes(x, y, color = group))
facet_wrap()	Creates panels based on categories	facet_wrap(vars(species))
stat = "identity"	Uses supplied values directly	Default for geom_point() and geom_col()
stat = "count"	Counts observations in each category	Default for geom_bar()
stat = "bin"	Counts numeric observations within bins	Default for geom_histogram()
geom_bar()	Counts rows by default	Usually supply x only
geom_col()	Uses supplied y values	Supply both x and y
geom_bar(stat = "identity")	Uses supplied y values	Alternative to geom_col()
Identity bars with repeated x	Positive y values stack by default	Bar height is the sum within each x category
bins = 10	Requests 10 histogram bins	Changing bins is valid
+	Adds ggplot layers	Different from a data-transformation pipe

5. dplyr Functions

Examples below assume df is a dataframe.

Function	Main purpose	Example	Output / Reminder
filter()	Keep rows matching conditions	filter(df, age > 40)	Selects rows, not columns
select()	Choose columns	select(df, name, age)	Selects columns, not rows
mutate()	Add or modify columns	mutate(df, mass_kg = mass_g / 1000)	A new name adds a column; an existing name replaces it
summarize() / summarise()	Calculate summary statistics	summarize(df, avg = mean(income, na.rm = TRUE))	Returns a table, usually one row per group
group_by()	Define groups for later operations	group_by(df, major)	Does not summarize by itself
count()	Count observations by category or combination	count(df, major)	Returns grouping columns and a count column n
relocate()	Move columns	relocate(df, age, .after = name)	Changes column order
arrange()	Sort whole rows	arrange(df, desc(income), age)	Sorts by income first, then age within ties
distinct()	Keep unique rows or combinations	distinct(df, name, age)	Returns unique combinations of selected columns
distinct(..., .keep_all = TRUE)	Keep other columns while removing duplicates	distinct(df, id, .keep_all = TRUE)	Keeps the first row for each id
pull()	Extract one column	pull(df, income)	Returns a vector
rename()	Change column names	rename(df, new_name = old_name)	Does not change column values
across()	Apply the same function to multiple columns	summarize(df, across(where(is.numeric), ~ mean(.x, na.rm = TRUE)))	Commonly used inside summarize() or mutate()
n()	Count rows in the current group	summarize(df, total = n())	Includes rows containing missing values
n_distinct()	Count distinct values	summarize(df, total = n_distinct(major))	Different from counting all rows

6. Logical and Comparison Operators

Operator	Meaning	Example
==	Equal to	species == "Adelie"
!=	Not equal to	species != "Gentoo"
<	Less than	age < 18
<=	Less than or equal to	age <= 18
>	Greater than	income > 50000
>=	Greater than or equal to	income >= 50000
&	AND: both conditions must be TRUE	age > 18 & income > 50000
\|	OR: at least one condition must be TRUE	species == "Adelie" \| species == "Chinstrap"
!	NOT: reverses a logical condition	!is.na(income)
& !	AND NOT: two operators used together	age > 18 & !(region == "West")
%in%	Matches any value in a vector	species %in% c("Adelie", "Chinstrap")
is.na(x)	Checks whether each value is missing	filter(df, is.na(income))

For a single-valued species column, species == "Adelie" & species == "Chinstrap" cannot be TRUE: one observation cannot have both values simultaneously.

7. across() and Column Selection

Selection	What it selects	Example
c(a, b)	Specifically named columns	summarize(df, across(c(age, income), ~ mean(.x, na.rm = TRUE)))
-c(a, b)	All columns except those named	summarize(df, across(-c(name, id), n_distinct))
everything()	All selectable columns	summarize(df, across(everything(), n_distinct))
starts_with("bill")	Names beginning with specified text	summarize(df, across(starts_with("bill"), n_distinct))
ends_with("mm")	Names ending with specified text	summarize(df, across(ends_with("mm"), n_distinct))
contains("length")	Names containing specified text	summarize(df, across(contains("length"), n_distinct))
where(is.numeric)	Numeric columns	summarize(df, across(where(is.numeric), ~ mean(.x, na.rm = TRUE)))
where(is.character)	Character columns	summarize(df, across(where(is.character), n_distinct))
where(is.factor)	Factor columns	summarize(df, across(where(is.factor), n_distinct))
where(is.logical)	Logical columns	summarize(df, across(where(is.logical), n_distinct))

Part of across()	Meaning
First argument	Which columns to use
Second argument	Which function to apply
.x	The current column passed to the function
~ mean(.x, na.rm = TRUE)	Calculate the mean of each selected column while skipping NA
Grouping columns	Automatically handled by the grouping operation; not selected by across()

8. rename() vs recode() vs case_when()

Function	What it changes	Syntax	Example	Reminder
rename()	Column names	rename(df, new_name = old_name)	rename(df, sex = Sex)	New name on the left; old name on the right
recode()	Values in a vector	recode(x, "old" = "new")	mutate(df, sex = recode(sex, "MALE" = "Male", "FEMALE" = "Female"))	Old value on the left; replacement on the right
case_when()	Values determined by conditions	case_when(condition ~ result)	mutate(df, adult = case_when(age >= 18 ~ "Yes", TRUE ~ "No"))	Conditions are checked in order; first match wins

recode() and case_when() return vectors. Use them inside mutate() when adding or replacing a dataframe column; they can also be used directly on vectors.

9. TRUE ~ Defaults in case_when()

Code placed last	Meaning	中文解释
TRUE ~ NA	Otherwise, return a missing value	前面条件都没匹配到的情况，设为缺失值
TRUE ~ NA_real_	Otherwise, return a numeric missing value	剩下的情况，设为数值型缺失值
TRUE ~ "Other"	Otherwise, return "Other"	剩下的情况，归入 "Other"
TRUE ~ 0	Otherwise, return 0	剩下的情况，赋值为 0

Important point	Explanation
Why TRUE?	It always matches, so it catches cases not already matched
Why put it last?	If placed first, it matches every case before later conditions can be used
Does it test whether sex is TRUE?	No; it is a fallback condition
What happens to NA conditions?	They do not count as TRUE; handle them explicitly with is.na() if needed
Result types	All result branches must have compatible types

For case_when(sex == "MALE" ~ 1, sex == "FEMALE" ~ 0, TRUE ~ NA_real_):

Original sex value	Result	Reason
"MALE"	1	First condition matches
"FEMALE"	0	Second condition matches
"OTHER"	NA	Neither earlier condition matches
Missing (NA)	NA	Earlier conditions are not TRUE; the fallback matches

10. Valid Variable Names in R

Variable name	Valid without backticks?	Explanation
q3_religion	Yes	Starts with a letter; underscores are allowed
q3.religion	Yes	Periods are allowed
.religion	Yes	May start with a period if the next character is not a digit
3_religion	No	Cannot start with a digit
.3_religion	No	Cannot start with a period followed by a digit
_religion	No	Cannot start with an underscore
#3_religion	No	# starts a comment
q3-religion	No	- is interpreted as subtraction
q3 religion	No	Contains a space
TRUE	No	Reserved constant
if	No	Reserved word

Use backticks to reference non-syntactic column names, e.g. select(df,4 ideology).

To rename one: rename(df, ideology =4 ideology).

11. Missing Data

Concept / Function	Meaning	Example / Reminder
Explicit missing data	An observation exists, but a value is recorded as missing	A row has income = NA
Implicit missing data	An expected observation is absent entirely	A country-year row is missing
NA	Unknown or missing value	Does not automatically mean zero
is.na(x)	Detect missing values	Returns TRUE/FALSE for each value
x == NA	Does not correctly detect missing values	Use is.na(x)
sum(is.na(x))	Count missing values	TRUE counts as 1
na.rm = TRUE	Skip NA in a particular calculation	mean(x, na.rm = TRUE)
drop_na(df, x)	Remove entire rows where x is missing	Other columns in those rows are also removed
drop_na(df, x, y)	Keep rows where both x and y are non-missing	Checks the named columns
drop_na(df)	Remove rows with NA in any column	May remove otherwise useful observations
replace_na(x, 0)	Replace NA with a specified value	Use zero only when justified
fill(df, group, .direction = "down")	Carry recorded values forward in row order	Also supports "up", "downup", and "updown"
coalesce(primary, backup)	Take the first non-missing value at each position	coalesce(c(NA, 2), c(1, 9)) gives 1, 2

12. Character, Numeric, and Factor

Structure	Stores values as	Sorting	When useful
Character vector	Text	Alphabetical according to locale	Names, text, category labels
Numeric vector	Numbers	Numeric order	Quantities or numeric category codes
Factor	Categorical values with defined levels	Level order	Control category order in sorting and plots

Concept	Meaning	Reminder
Observed values	Values actually present in the vector	May contain only some possible categories
Factor levels	Possible categories and their order	May include unused categories
levels(f)	Lists the factor’s category names	Does not list every observation
factor()	Defines categories and their order	Does not rearrange observations
sort(f)	Sorts observed values using level order	Does not add observations for unused levels
Printed Levels: line	Metadata about categories	Does not mean every level occurs in the data

13. Factor levels vs labels

Argument / Situation	Meaning	Character input	Numeric input
levels	Original values in the desired category order	c("Low", "Medium", "High")	c(1, 2, 3)
labels	Names assigned to levels by position	Optional replacement names	c("Low", "Medium", "High")
Labels omitted	Original values become category names	"Low" stays "Low"	1 displays as "1"
Value excluded from levels	That observation becomes NA	"Other" excluded → NA	4 excluded → NA
Level order	Controls category sorting and discrete plot order	Low → Medium → High	Codes ordered by supplied levels
Label mapping	Matches by position	First label → first level	1 → Low, 2 → Medium, 3 → High

Example: factor(c(1, 3, 2), levels = c(1, 2, 3), labels = c("Low", "Medium", "High")) produces observed values Low, High, Medium.

14. Common Factor Errors

Error	Why it fails	Fix
Character months with numeric levels 1:12	"Jan" does not match 1	Use original character month names as levels
12 levels with a vector of 4 labels	Label count does not match level count	Supply one label per level
Numeric months include 12, but levels contain only 1:4	12 is excluded and becomes NA	Include all intended original values
Levels 1:4 labelled Jan/Mar/Apr/Dec	Labels map by position and mislabel the codes	Match each code to its correct month
Assuming levels reorder rows	Levels define category order only	Use sort() or arrange()
Assuming unused levels create observations	Levels are metadata	Inspect actual vector values separately

15. Sorting and forcats Functions

Function	Purpose	Example / Default
sort(x)	Sort a vector ascending	Returns a vector
sort(x, decreasing = TRUE)	Sort a vector descending	Returns a vector
arrange(df, x)	Sort whole dataframe rows ascending	Related column values stay together
arrange(df, desc(x))	Sort whole dataframe rows descending	Returns a dataframe
factor(x, levels = ...)	Create a factor with specified categories/order	Excluded values become NA
fct_relevel(x, ...)	Manually reorder existing levels	Unmentioned levels remain afterward
fct_infreq(x)	Order levels by category frequency	Most frequent first
fct_reorder(x, y)	Order levels by a summary of another variable	Default: median of y, ascending

sort() removes NA by default; use na.last = TRUE to keep them at the end. For local dataframes, arrange() puts NA rows at the end, including when using desc().

16. Tidy Data Concepts

Term	Meaning	Example
Variable	A characteristic being recorded	Country, year, internet access
Observation	One unit being described	Kenya in 2010
Value	One measurement or entry	7 percent internet access
Tidy data	One variable per column, one observation per row, one value per cell	One row per country-year
Long format	Values spread down rows rather than across many columns	Not automatically tidy
Unit of observation	What one row should represent	Country-year, student, or school

17. Tidyr Functions

Problem	Function	What it does	Key arguments
Variable values are column names	pivot_longer()	Headers become values in a new column	cols, names_to, values_to
Variable names are stored as values	pivot_wider()	Values become column headers	names_from, values_from
Multiple variables are stored in one column	separate()	Splits one column into several	col, into, sep, convert
One variable is split across columns	unite()	Combines columns into one	New column name, input columns, sep

18. pivot_longer() vs pivot_wider()

Compare	pivot_longer()	pivot_wider()
Typical direction	More rows, fewer columns	Fewer rows, more columns
Names	Headers become values	Values become headers
Name argument	names_to = "year"	names_from = role
Value argument	values_to = "access"	values_from = count
Quotes	New names are strings	Existing columns usually use bare names
Example problem	Years stored as separate columns	Students and teachers stored in separate rows

19. separate() and unite() Arguments

Code	Function	Meaning	Example
sep = ""	unite()	Join with nothing between values	"19" + "99" → "1999"
sep = " "	unite()	Join with a space	"19" + "99" → "19 99"
sep = "_"	unite()	Join with an underscore; default	"19" + "99" → "19_99"
sep = "/"	separate()	Split at a slash	"745/19987071" → two pieces
convert = TRUE	separate()	Convert suitable split text into numeric or other appropriate types	"745" → 745

20. Internet Access Practice Data

country	2010	2015	2020
Kenya	7	17	29
India	8	20	41
Brazil	42	59	80

Question	Answer
What is wrong with the structure?	Years are values of a variable but are stored as column names
What should one observation represent?	One country-year pair
Which function tidies it?	pivot_longer(cols = -country, names_to = "year", values_to = "access")
How do we calculate yearly averages afterward?	group_by(year) followed by summarize(mean_access = mean(access, na.rm = TRUE))
Why is this better than a separate mean for each year column?	The same pipeline handles additional years
Where does na.rm = TRUE belong?	Inside mean()

21. Strings and stringr Functions

Concept / Function	What it does / returns	No match / Reminder
String	A sequence of characters	Example: "Student ID: 1234"
Character vector	A collection of strings	A single string is also a character vector of length one
Pattern	A rule describing what to find	Can be a regular expression
length(x)	Number of vector elements	Number of strings, not characters
nchar(x)	Number of characters in each element	One count per string
str_view()	Displays strings and highlights matches	Used to inspect match locations
str_detect()	Checks whether a match exists	FALSE
str_count()	Counts matches in each string	0
str_extract()	Extracts the first match in each string	NA
str_extract_all()	Extracts all matches in each string	List containing an empty character vector for a non-match
str_replace()	Replaces the first match in each string	Original string
str_replace_all()	Replaces all matches in each string	Original string
str_split()	Splits each string at a pattern	List containing the original string as one piece

22. Regex Meta-characters

Examples in the R pattern column are quoted R strings.

Symbol	Meaning	R pattern	Example
.	Any single character except newline by default	"a.c"	Matches abc, a-c, and a.c
^	Start of string	"^cat"	Matches cat and cats, not bobcat
$	End of string	"cat$"	Matches cat and bobcat, not cats
^...$	Match the whole string	"^cat$"	Matches only cat
\|	OR	"cat\|dog"	Matches either alternative
[]	One character from a set or range	"[aeiou]", "[a-z]"	One vowel or lowercase ASCII letter
[^...]	One character outside the set	"[^0-9]"	One non-digit
()	Group part of a pattern	"(cat\|dog)s"	Matches cats or dogs
Escaped dot	Literal period	"\\."	Matches a period rather than any character

23. Regex Character Classes and Escaping

Regex	Meaning	Write in an R string
\d	Digit	"\\d"
\D	Non-digit	"\\D"
\w	Word character, including letters, digits, and underscore	"\\w"
\W	Non-word character	"\\W"
\s	Whitespace	"\\s"
\S	Non-whitespace	"\\S"
\.	Literal period	"\\."

24. Regex Quantifiers

Quantifiers repeat the preceding character, character class, or group.

Quantifier	Meaning	R pattern	Example
*	Zero or more	"ab*c"	ac, abc, abbc
+	One or more	"ab+c"	abc, abbc; not ac
?	Zero or one	"colou?r"	color, colour
{n}	Exactly n	"^\\d{4}$"	Exactly four digits
{n,}	At least n	"^a{2,}$"	aa, aaa, etc.
{n,m}	Between n and m	"^a{2,4}$"	aa, aaa, aaaa

25. String and Regex Review

Question	Answer
Are matches case-sensitive by default?	Yes
How can we ignore case?	regex("app", ignore_case = TRUE)
"cat" vs "^cat$"?	Matches anywhere vs matches the entire string
"." vs "\\."?	Wildcard vs literal period
What does "\\d+" match?	One or more consecutive digits
Does str_replace() replace every match?	No; use str_replace_all()
Does extracted numeric text become numeric automatically?	No; use as.numeric() or another conversion
str_detect() vs str_count()?	TRUE/FALSE vs number of matches
str_view() vs str_extract_all()?	Displays matches in context vs extracts matching text into a list

26. Paths and File Import/Export

Term / Function	Meaning	Example
Absolute path	Full location starting from the filesystem root or drive	"C:/Users/grace/project/data/file.csv"
Relative path	Location starting from the current working directory	"data/file.csv"
getwd()	Show the current working directory	Helps diagnose file-not-found errors
read_csv()	Import a comma-separated file	read_csv("data/file.csv")
write_csv()	Export a dataframe as CSV	write_csv(df, "data/cleaned.csv")
RStudio Project	Helps establish a consistent project working directory	Makes relative paths easier to use
R Markdown knitting directory	May differ from the Console working directory	Usually the Rmd file’s folder by default

27. Vector vs Dataframe vs Tibble

Term	Definition	Example / Reminder
Atomic vector	One-dimensional collection of values of the same basic type	c(90, 80, 70)
Dataframe	Table of equal-length columns; columns may have different types	Names in one column, scores in another
Tibble	Modern dataframe commonly used by tidyverse	tibble(name = c("Amy", "Ben"), score = c(90, 80))
One-column tibble	Still a table	Not automatically a vector
summarize() output	A summary table	Use pull() afterward if a vector is needed
pull() output	A vector	Extracts one column

28. Git and GitHub

Term / Action	Meaning
Git	Version-control software
GitHub	A service that hosts remote Git repositories
Local repository	Repository in the environment where you work
Remote repository	Repository hosted elsewhere, such as GitHub
Workbench in the class workflow	Counts as the local working environment even though accessed through a browser
git add	Stage changes
git commit	Record a version locally
git push	Send local commits to the remote
git pull	Fetch and integrate remote changes
Class workflow	Pull → edit → stage → commit → push

29. Common Exam Contrasts

Compare	First	Second
filter() vs select()	Select rows	Select columns
select() vs pull()	Returns a dataframe	Returns a vector
mutate() vs summarize()	Adds/modifies columns, generally keeping rows	Produces summary rows
group_by() vs summarize()	Defines groups	Calculates summaries
rename() vs recode()	Changes column names	Changes values
sort() vs arrange()	Sorts a vector	Sorts dataframe rows
na.rm = TRUE vs drop_na()	Skips NA in one calculation	Removes entire rows
factor() vs arrange()	Defines category levels/order	Changes row order
fct_relevel() vs fct_infreq()	Manual category order	Order by frequency
fct_infreq() vs fct_reorder()	Category frequency	Summary of another variable
geom_bar() vs geom_col()	Counts rows by default	Uses supplied y values
pivot_longer() vs pivot_wider()	Headers become values	Values become headers
separate() vs unite()	Splits one column	Combines columns
length() vs nchar()	Number of vector elements	Characters in each string
str_extract() vs str_extract_all()	First match per string	All matches per string
str_replace() vs str_replace_all()	First replacement per string	All replacements per string
\|> vs +	Connects data-transformation steps	Adds ggplot layers

From Quiz

1. Local vs Remote Repository

• Local repository: the Git repository in your working environment, e.g., your computer or Workbench.
• Remote repository: a repository hosted elsewhere, e.g., GitHub.
• Workbench counts as the local working environment in this workflow, even though you access it through a browser.
• Class workflow: edit locally → commit → push to GitHub.
• Avoid editing/uploading directly on GitHub when following this workflow.

2. Global vs Layer-specific aes()

• Mappings inside ggplot(aes(...)) are inherited by layers.
• Mappings inside geom_point(aes(...)) apply only to that layer.
• To color points by education but draw one overall regression line, put color = education only inside geom_point().

ggplot(survey, aes(x = age, y = income)) +
  geom_point(aes(color = education)) +
  geom_smooth(method = "lm")
# POINTS COLORED BY EDUCATION
# ONE OVERALL REGRESSION LINE

ggplot(survey, aes(x = age, y = income, color = education)) +
  geom_point() +
  geom_smooth(method = "lm")
# BOTH LAYERS INHERIT color = education
# FOR CATEGORICAL education: A SEPARATE LINE FOR EACH GROUP

3. Histogram vs Bar Chart

• geom_histogram() requires a continuous numeric x.
• geom_bar() counts observations in categories.
• bins = 10 is valid; changing the number of bins does not cause an error.

# INCORRECT: type IS CATEGORICAL
ggplot(scorecard, aes(x = type)) +
  geom_histogram()

# CORRECT: cost IS NUMERIC
ggplot(scorecard, aes(x = cost)) +
  geom_histogram(bins = 10)

# CORRECT: COUNT EACH SCHOOL TYPE
ggplot(scorecard, aes(x = type)) +
  geom_bar()

4. filter() Before summarize()

• To calculate average income for respondents over 40:
  1. Keep the relevant respondents.
  2. Calculate their average income.
• Summarizing first loses the individual rows and usually the age column.

survey |>
  filter(age > 40) |>
  summarize(avg_income = mean(income, na.rm = TRUE))

5. summarize() Returns a Table

students |>
  group_by(major) |>
  summarize(avg_gpa = mean(gpa, na.rm = TRUE))

• Returns a tibble/dataframe, not a vector.
• Contains one row per major and columns major and avg_gpa.
• na.rm = TRUE skips missing GPA values in the calculation.
• Use pull(avg_gpa) afterward if a vector is needed.

6. Pipes vs Plot Layers

• |> passes the result into the next function.
• + adds layers to a ggplot.

flights |>
  group_by(dest) |>
  summarize(count = n())
# CONNECT DATA TRANSFORMATION STEPS WITH |>
# nycflights13 USES dest AS THE DESTINATION COLUMN

ggplot(mpg, aes(x = displ, y = hwy)) +
  geom_point() +
  geom_smooth(method = "lm")
# CONNECT PLOT LAYERS WITH +

7. TRUE ~ NA in case_when()

• Conditions are checked in order; the first matching condition wins.
• TRUE ~ NA means all remaining unmatched cases become NA.
• It does NOT mean “if the value of sex is TRUE.”

survey |>
  mutate(sex = case_when(
    sex == "MALE" ~ 1,
    sex == "FEMALE" ~ 0,
    TRUE ~ NA_real_
  ))
# MALE -> 1
# FEMALE -> 0
# EVERYTHING ELSE -> NUMERIC NA

8. Factor Values vs Levels

• Values: observations actually present in the vector.
• Levels: possible categories and their order.
• sort() sorts the observed values by level order.
• It does not add observations for unused levels.

x <- c("Independent", "Green", "Democrat", "Republican")

party <- c(
  "Democrat", "Republican", "Independent",
  "Green", "Libertarian", "Other"
)

y <- factor(x, levels = party)

sort(y)

## [1] Democrat    Republican  Independent Green      
## Levels: Democrat Republican Independent Green Libertarian Other

# OBSERVED VALUES:
# Democrat Republican Independent Green

levels(y)

## [1] "Democrat"    "Republican"  "Independent" "Green"       "Libertarian"
## [6] "Other"

# ALL SIX LEVELS, INCLUDING Libertarian AND Other

• The printed Levels: line is metadata; it does not mean those categories occur as observations.

9. Internet Access: pivot_longer()

• Problem: years are stored as column names, but they are values of the variable year.
• One observation should be a country-year pair.
• The original approach requires a separate calculation for each year, not each country.
• With more years, the code becomes repetitive and harder to maintain.

internet <- tibble::tibble(
  country = c("Kenya", "India", "Brazil"),
  `2010` = c(7, 8, 42),
  `2015` = c(17, 20, 59),
  `2020` = c(29, 41, 80)
)

internet_tidy <- internet |>
  tidyr::pivot_longer(
    cols = -country,
    names_to = "year",
    values_to = "access"
  ) |>
  dplyr::mutate(year = as.integer(year))

internet_tidy |>
  dplyr::group_by(year) |>
  dplyr::summarize(
    mean_access = mean(access, na.rm = TRUE),
    .groups = "drop"
  )

## # A tibble: 3 × 2
##    year mean_access
##   <int>       <dbl>
## 1  2010          19
## 2  2015          32
## 3  2020          50

• cols = -country: pivot all columns except the country identifier.
• names_to: old column names go into year.
• values_to: old cell values go into access.
• Put na.rm = TRUE inside mean().
• The same pipeline handles additional year columns.

10. Student–Teacher Ratio: pivot_wider()

• students and teachers are variable names stored as values in role.
• Each county-year observation is spread across two rows.
• Use pivot_wider() to give students and teachers separate columns.
• separate() would split text inside a cell; that is not the problem here.

school_data <- tibble::tibble(
  county = c("Cook", "Cook", "DuPage", "DuPage"),
  year = c(2024, 2024, 2024, 2024),
  role = c("students", "teachers", "students", "teachers"),
  count = c(780000, 48000, 155000, 10500)
)

school_data |>
  tidyr::pivot_wider(
    names_from = role,
    values_from = count
  ) |>
  dplyr::mutate(ratio = students / teachers)

## # A tibble: 2 × 5
##   county  year students teachers ratio
##   <chr>  <dbl>    <dbl>    <dbl> <dbl>
## 1 Cook    2024   780000    48000  16.2
## 2 DuPage  2024   155000    10500  14.8

# ONE ROW PER COUNTY-YEAR
# RATIO = NUMBER OF STUDENTS PER TEACHER

Terms

Absolute Path vs Relative Path

Term	Definition	Example
Absolute path	Full location starting from the filesystem root or drive	"C:/Users/grace/project/data/file.csv"
Relative path	Location starting from the current working directory	"data/file.csv"

• getwd() shows the current working directory.
• If the working directory is the project folder, "data/file.csv" refers to the file inside that project’s data folder.
• Relative paths make projects easier to share when folder structure is preserved.
• In R Markdown, the knitting directory may differ from the Console working directory; by default it is usually the Rmd file’s folder.

Vector vs Dataframe vs Tibble

Term	Definition	Example
Atomic vector	A one-dimensional collection of values of the same basic type	c(90, 80, 70)
Dataframe	A table of equal-length columns; different columns can have different types	Names in one column, scores in another
Tibble	A modern form of dataframe commonly used by tidyverse	tibble(name = c("Amy", "Ben"), score = c(90, 80))

• A one-column or one-row tibble is still a table, not automatically a vector.
• summarize() returns a table.
• pull() extracts one column as a vector.

Variable vs Observation vs Value

Term	Meaning	Country-year example
Variable	A characteristic being recorded	Country, year, cases
Observation	One unit being described	Afghanistan in 1999
Value	One measurement or entry	745 cases

• Tidy data: one variable per column, one observation per row, one value per cell.
• The unit of observation depends on the question: country-year, student, school, etc.

Character vs Numeric vs Factor

Term	Meaning	Example
Character	Text values	"1999", "High school"
Numeric	Numbers used in calculations	1999, 3.8
Factor	Categorical values with defined levels	Education categories in a specified order

• "3.8" is text; 3.8 is a number.
• Factor levels control category sorting and discrete plot order.
• Numeric codes can represent categories; the meaning of the variable matters.

Levels vs Labels

Term	Meaning	Example
Levels argument	Expected original values in the desired order	levels = c(1, 2, 3)
Labels argument	Display names assigned to those levels by position	labels = c("Low", "Medium", "High")

• Mapping: 1 → Low, 2 → Medium, 3 → High.
• Values absent from the specified levels become NA.
• levels(f) shows the resulting factor’s category names.
• Setting levels does not reorder observations or create new observations.

Explicit vs Implicit Missing Data

Term	Meaning	Example
Explicit missing data	Missingness is recorded	A row exists, but its income is NA
Implicit missing data	The expected observation is absent	A country-year row is missing entirely

• NA means unknown; it does not automatically mean zero.
• Use is.na(), not == NA, to detect missing values.

Local vs Remote Repository

Term	Meaning
Local repository	Repository in the environment where you work, e.g., computer or Workbench
Remote repository	Repository hosted elsewhere, e.g., GitHub

• commit: records a version locally.
• push: sends local commits to the remote.
• pull: fetches and integrates remote changes into the local repository.

Mapping vs Setting an Aesthetic

Term	Meaning	Example
Mapping	Connects a variable to a visual property; inside aes()	aes(color = species)
Setting	Gives all observations a fixed visual property; outside aes()	geom_point(color = "blue")

• Global mappings are inherited by layers unless overridden.
• Layer-specific mappings apply only to that layer.

Count vs Identity

Term	Meaning	Example
stat = “count”	Counts observations in each category	Default for geom_bar()
stat = “identity”	Uses supplied values directly	geom_col() or geom_bar(stat = "identity")

• With identity bars, supply both x and y.
• Multiple positive y values for the same category stack by default.

String vs Pattern

Term	Meaning	Example
String	Text being searched	"Student ID: 1234"
Pattern	Rule describing what to find	"\\d{4}"
Regex	A language for describing text patterns	"^\\d{4}$" means exactly four digits

• str_detect(string, pattern) checks whether a match exists.
• Matching is case-sensitive by default.
• regex("app", ignore_case = TRUE) matches app, App, APP, etc.
• For whole-string format checks, use ^ and $.

LECTURE 10

**

**

**

**

**

**

**

**

**

**

**

**

**

**

**