install.packages(c(
"tidyverse", "janitor", "readxl", "haven", "data.table", "skimr",
"survey", "srvyr", "broom", "performance", "lme4", "boot", "rsample",
"epitools", "epiR", "MatchIt", "gtsummary", "tableone", "mice",
"survival", "survminer", "forecast", "poLCA", "pwr", "gt", "here",
"sf", "tigris", "shiny", "patchwork", "scales"
))My R Reference Guide
Data transformation, visualization, statistics, and epidemiology
1 How to Use This Guide
The guide moves in the order of a real project: get data in, clean it, transform it, visualize it, run statistics, apply epidemiology designs, then share results. Every section follows the same pattern:
- What it’s for: when you would reach for it
- Packages: which package does what
- Worked example: commented code using a dataset built into R
- Watch out for: common mistakes
- Learn more: official documentation
1.1 Install everything once
Run this one time in the console (not in this file).
library(tidyverse) # loads dplyr, tidyr, ggplot2, stringr, forcats, lubridate, purrr, readr1.2 Built-in datasets used here
| Dataset | What it contains | Used for |
|---|---|---|
mtcars |
32 cars, engine and mileage data | filtering, regression |
iris |
150 flowers, 3 species | grouping, PCA |
airquality |
New York air readings with missing values | missing data, dates |
ToothGrowth |
guinea pig tooth growth by vitamin C dose | comparing groups |
infert, esoph |
case-control studies | odds ratios, matching |
lung (survival) |
lung cancer survival times | survival analysis |
AirPassengers |
monthly airline passengers 1949 to 1960 | time series |
api (survey) |
California school survey sample | survey weighting |
2 Part 1: Getting Data In and Cleaning It
2.1 Importing data
What it’s for: reading files into R as data frames.
| Package | Function | File type |
|---|---|---|
| readr | read_csv() |
.csv |
| readxl | read_excel() |
.xls, .xlsx |
| haven | read_xpt(), read_sas(), read_dta(), read_sav() |
SAS (NHANES), Stata, SPSS |
| here | here() |
builds file paths that work on any computer |
library(readxl); library(haven); library(here)
df <- read_csv(here("data", "my_file.csv"))
xl <- read_excel(here("data", "my_file.xlsx"), sheet = 1)
demo <- read_xpt(here("data", "DEMO_J.XPT")) # NHANES demographics fileWatch out for: read_csv() (readr, underscore) is faster and tidier than base read.csv(). NHANES files are .XPT, so use read_xpt().
2.2 First look and quick cleaning
What it’s for: understanding a dataset and fixing messy names before doing anything else.
library(janitor); library(skimr)
glimpse(airquality) # column types and first valuesRows: 153
Columns: 6
$ Ozone <int> 41, 36, 12, 18, NA, 28, 23, 19, 8, NA, 7, 16, 11, 14, 18, 14, …
$ Solar.R <int> 190, 118, 149, 313, NA, NA, 299, 99, 19, 194, NA, 256, 290, 27…
$ Wind <dbl> 7.4, 8.0, 12.6, 11.5, 14.3, 14.9, 8.6, 13.8, 20.1, 8.6, 6.9, 9…
$ Temp <int> 67, 72, 74, 62, 56, 66, 65, 59, 61, 69, 74, 69, 66, 68, 58, 64…
$ Month <int> 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,…
$ Day <int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,…
skim(airquality) # summary, missing counts, mini histograms| Name | airquality |
| Number of rows | 153 |
| Number of columns | 6 |
| _______________________ | |
| Column type frequency: | |
| numeric | 6 |
| ________________________ | |
| Group variables | None |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| Ozone | 37 | 0.76 | 42.13 | 32.99 | 1.0 | 18.00 | 31.5 | 63.25 | 168.0 | ▇▃▂▁▁ |
| Solar.R | 7 | 0.95 | 185.93 | 90.06 | 7.0 | 115.75 | 205.0 | 258.75 | 334.0 | ▅▃▅▇▅ |
| Wind | 0 | 1.00 | 9.96 | 3.52 | 1.7 | 7.40 | 9.7 | 11.50 | 20.7 | ▂▇▇▃▁ |
| Temp | 0 | 1.00 | 77.88 | 9.47 | 56.0 | 72.00 | 79.0 | 85.00 | 97.0 | ▂▃▇▇▃ |
| Month | 0 | 1.00 | 6.99 | 1.42 | 5.0 | 6.00 | 7.0 | 8.00 | 9.0 | ▇▇▇▇▇ |
| Day | 0 | 1.00 | 15.80 | 8.86 | 1.0 | 8.00 | 16.0 | 23.00 | 31.0 | ▇▇▇▇▆ |
messy <- tibble(`Patient ID` = 1:3, `Age (yrs)` = c(30, 30, 45))
messy |> clean_names() # becomes patient_id, age_yrs# A tibble: 3 × 2
patient_id age_yrs
<int> <dbl>
1 1 30
2 2 30
3 3 45
messy |> get_dupes(`Age (yrs)`) # find duplicate rows# A tibble: 2 × 3
`Age (yrs)` dupe_count `Patient ID`
<dbl> <int> <int>
1 30 2 1
2 30 2 2
mtcars |> tabyl(cyl, am) # quick cross-tab cyl 0 1
4 3 8
6 4 3
8 12 2
Watch out for: clean names right after importing so every later step uses consistent snake_case.
3 Part 2: Transforming Data
3.1 The pipe
|> means “and then.” df |> filter(x > 1) |> select(y) reads “take df, and then filter, and then select.” The older %>% from magrittr works the same way.
3.2 Selecting columns and filtering rows
| Package | Function | Does |
|---|---|---|
| dplyr | select() |
keep or drop columns |
| dplyr | filter() |
keep rows that meet a condition |
| dplyr | arrange() |
sort rows |
| dplyr | distinct(), slice_max() |
unique rows, top rows |
mtcars |>
select(mpg, cyl, hp) |> # keep three columns
filter(cyl == 4, mpg > 25) |> # comma means AND
arrange(desc(mpg)) # highest mpg first mpg cyl hp
Toyota Corolla 33.9 4 65
Fiat 128 32.4 4 66
Honda Civic 30.4 4 52
Lotus Europa 30.4 4 113
Fiat X1-9 27.3 4 66
Porsche 914-2 26.0 4 91
# Base R equivalent
mtcars[mtcars$cyl == 4 & mtcars$mpg > 25, c("mpg", "cyl", "hp")] mpg cyl hp
Fiat 128 32.4 4 66
Honda Civic 30.4 4 52
Toyota Corolla 33.9 4 65
Fiat X1-9 27.3 4 66
Porsche 914-2 26.0 4 91
Lotus Europa 30.4 4 113
# Helpers
iris |> select(starts_with("Petal")) |> head(3) Petal.Length Petal.Width
1 1.4 0.2
2 1.4 0.2
3 1.3 0.2
mtcars |> filter(cyl %in% c(4, 6)) |> nrow()[1] 18
Watch out for: use == to compare and = only to name arguments. filter() silently drops rows where the condition is NA.
3.3 Creating and changing columns
mtcars |>
mutate(
kpl = mpg * 0.425, # new numeric column
power = case_when( # categories from rules
hp < 100 ~ "low",
hp < 200 ~ "medium",
TRUE ~ "high" # everything else
),
heavy = if_else(wt > 3.5, "yes", "no") # two-way choice
) |>
select(mpg, kpl, hp, power, heavy) |>
head() mpg kpl hp power heavy
Mazda RX4 21.0 8.9250 110 medium no
Mazda RX4 Wag 21.0 8.9250 110 medium no
Datsun 710 22.8 9.6900 93 low no
Hornet 4 Drive 21.4 9.0950 110 medium no
Hornet Sportabout 18.7 7.9475 175 medium no
Valiant 18.1 7.6925 105 medium no
Watch out for: case_when() checks rules top to bottom and stops at the first match, so order matters.
3.4 Grouping and summarizing
iris |>
group_by(Species) |>
summarise(
n = n(),
mean_petal = mean(Petal.Length),
sd_petal = sd(Petal.Length)
)# A tibble: 3 × 4
Species n mean_petal sd_petal
<fct> <int> <dbl> <dbl>
1 setosa 50 1.46 0.174
2 versicolor 50 4.26 0.470
3 virginica 50 5.55 0.552
mtcars |> count(cyl, gear) # shortcut for group_by + n() cyl gear n
1 4 3 1
2 4 4 8
3 4 5 2
4 6 3 2
5 6 4 4
6 6 5 1
7 8 3 12
8 8 5 2
# data.table: very fast for millions of rows
library(data.table)
dt <- as.data.table(iris)
dt[, .(mean_petal = mean(Petal.Length)), by = Species] Species mean_petal
<fctr> <num>
1: setosa 1.462
2: versicolor 4.260
3: virginica 5.552
Watch out for: add .groups = "drop" or ungroup() afterward so later steps are not still grouped.
Learn more: dplyr, data.table
3.5 Factors and categories (forcats)
What it’s for: controlling category order and labels, which matters for tables, plots, and the reference group in regression.
df <- mtcars |> mutate(cyl = factor(cyl))
levels(df$cyl)[1] "4" "6" "8"
df |> mutate(cyl = fct_relevel(cyl, "8")) |> pull(cyl) |> levels() # 8 becomes reference[1] "8" "4" "6"
df |> mutate(cyl = fct_recode(cyl, Four = "4", Six = "6", Eight = "8")) |> count(cyl) cyl n
1 Four 11
2 Six 7
3 Eight 14
df |> mutate(cyl = fct_infreq(cyl)) |> count(cyl) # order by frequency cyl n
1 8 14
2 4 11
3 6 7
df |> mutate(cyl = fct_lump_n(cyl, n = 2)) |> count(cyl) # rare levels become "Other" cyl n
1 4 11
2 8 14
3 Other 7
Learn more: forcats
3.6 Text cleaning (stringr)
x <- c(" Type 2 Diabetes ", "type 1 diabetes", "HTN", "Hypertension")
str_trim(x) # remove extra spaces[1] "Type 2 Diabetes" "type 1 diabetes" "HTN" "Hypertension"
str_to_lower(x) # lowercase[1] " type 2 diabetes " "type 1 diabetes" "htn"
[4] "hypertension"
str_detect(x, regex("diabetes", ignore_case = TRUE)) # TRUE/FALSE[1] TRUE TRUE FALSE FALSE
str_replace(x, "HTN", "Hypertension")[1] " Type 2 Diabetes " "type 1 diabetes" "Hypertension"
[4] "Hypertension"
str_extract("ID-00452", "[0-9]+") # pull numbers out[1] "00452"
str_c("Patient", 1:3, sep = "_") # paste together[1] "Patient_1" "Patient_2" "Patient_3"
Watch out for: stringr patterns are regular expressions, so . means “any character.” Use fixed(".") for a literal dot.
Learn more: stringr
3.7 Dates and times (lubridate)
aq <- airquality |> mutate(date = make_date(1973, Month, Day))
head(aq$date)[1] "1973-05-01" "1973-05-02" "1973-05-03" "1973-05-04" "1973-05-05"
[6] "1973-05-06"
ymd("2026-11-20"); mdy("11/20/2026"); dmy("20-11-2026")[1] "2026-11-20"
[1] "2026-11-20"
[1] "2026-11-20"
aq |> mutate(weekday = wday(date, label = TRUE), mon = month(date, label = TRUE)) |>
select(date, weekday, mon) |> head(3)Error in `mutate()`:
ℹ In argument: `weekday = wday(date, label = TRUE)`.
Caused by error in `wday()`:
! unused argument (label = TRUE)
# Time between dates, useful for follow-up time
interval(ymd("2020-01-15"), ymd("2023-06-01")) / years(1)[1] 3.375342
ymd("2026-01-01") + days(90) # 90-day window[1] "2026-04-01"
Learn more: lubridate
3.8 Missing values and multiple imputation
What it’s for: finding, removing, filling, or imputing NAs.
colSums(is.na(airquality)) # NA count per column Ozone Solar.R Wind Temp Month Day
37 7 0 0 0 0
airquality |> drop_na(Ozone) |> nrow() # drop rows missing Ozone[1] 116
airquality |> replace_na(list(Solar.R = 0)) |> head(5) Ozone Solar.R Wind Temp Month Day
1 41 190 7.4 67 5 1
2 36 118 8.0 72 5 2
3 12 149 12.6 74 5 3
4 18 313 11.5 62 5 4
5 NA 0 14.3 56 5 5
mean(airquality$Ozone) # NA[1] NA
mean(airquality$Ozone, na.rm = TRUE) # ignore NA[1] 42.12931
airquality |> fill(Ozone, .direction = "down") |> head(5) # carry forward Ozone Solar.R Wind Temp Month Day
1 41 190 7.4 67 5 1
2 36 118 8.0 72 5 2
3 12 149 12.6 74 5 3
4 18 313 11.5 62 5 4
5 18 NA 14.3 56 5 5
Dropping rows can bias results when missingness is not random. Multiple imputation (mice) creates several filled-in datasets, fits the model on each, and pools the results.
library(mice)
imp <- mice(airquality, m = 5, seed = 123, printFlag = FALSE)
fit <- with(imp, lm(Ozone ~ Temp + Wind))
summary(pool(fit)) term estimate std.error statistic df p.value
1 (Intercept) -66.004935 22.7838035 -2.897011 41.47674 5.991132e-03
2 Temp 1.769961 0.2392335 7.398466 48.30852 1.741085e-09
3 Wind -2.961797 0.6652761 -4.451982 36.10737 7.867647e-05
Learn more: tidyr, mice, van Buuren, Flexible Imputation of Missing Data (free online)
3.9 Reshaping wide and long (tidyr)
What it’s for: “long” format (one row per measurement) is what ggplot2 and most models want. “Wide” is easier to read.
wide <- iris |> mutate(id = row_number()) |> head(3)
long <- wide |>
pivot_longer(cols = Sepal.Length:Petal.Width,
names_to = "measure", values_to = "cm")
long# A tibble: 12 × 4
Species id measure cm
<fct> <int> <chr> <dbl>
1 setosa 1 Sepal.Length 5.1
2 setosa 1 Sepal.Width 3.5
3 setosa 1 Petal.Length 1.4
4 setosa 1 Petal.Width 0.2
5 setosa 2 Sepal.Length 4.9
6 setosa 2 Sepal.Width 3
7 setosa 2 Petal.Length 1.4
8 setosa 2 Petal.Width 0.2
9 setosa 3 Sepal.Length 4.7
10 setosa 3 Sepal.Width 3.2
11 setosa 3 Petal.Length 1.3
12 setosa 3 Petal.Width 0.2
long |> pivot_wider(names_from = measure, values_from = cm)# A tibble: 3 × 6
Species id Sepal.Length Sepal.Width Petal.Length Petal.Width
<fct> <int> <dbl> <dbl> <dbl> <dbl>
1 setosa 1 5.1 3.5 1.4 0.2
2 setosa 2 4.9 3 1.4 0.2
3 setosa 3 4.7 3.2 1.3 0.2
# Split and combine columns
tibble(bp = c("120/80", "135/90")) |> separate_wider_delim(bp, "/", names = c("sys", "dia"))# A tibble: 2 × 2
sys dia
<chr> <chr>
1 120 80
2 135 90
3.10 Joining tables (dplyr)
patients <- tibble(id = 1:4, age = c(34, 51, 29, 62))
labs <- tibble(id = c(1, 2, 2, 5), a1c = c(5.4, 7.1, 6.8, 6.0))
left_join(patients, labs, by = "id") # keep all patients# A tibble: 5 × 3
id age a1c
<dbl> <dbl> <dbl>
1 1 34 5.4
2 2 51 7.1
3 2 51 6.8
4 3 29 NA
5 4 62 NA
inner_join(patients, labs, by = "id") # only matches# A tibble: 3 × 3
id age a1c
<dbl> <dbl> <dbl>
1 1 34 5.4
2 2 51 7.1
3 2 51 6.8
anti_join(patients, labs, by = "id") # patients with no labs# A tibble: 2 × 2
id age
<int> <dbl>
1 3 29
2 4 62
full_join(patients, labs, by = "id") # everyone# A tibble: 6 × 3
id age a1c
<dbl> <dbl> <dbl>
1 1 34 5.4
2 2 51 7.1
3 2 51 6.8
4 3 29 NA
5 4 62 NA
6 5 NA 6
Watch out for: if a key repeats (patient 2 has two labs), rows multiply. Check nrow() before and after every join.
3.11 Repeating steps (across and purrr)
# Same function on many columns
iris |> group_by(Species) |> summarise(across(where(is.numeric), mean))# A tibble: 3 × 5
Species Sepal.Length Sepal.Width Petal.Length Petal.Width
<fct> <dbl> <dbl> <dbl> <dbl>
1 setosa 5.01 3.43 1.46 0.246
2 versicolor 5.94 2.77 4.26 1.33
3 virginica 6.59 2.97 5.55 2.03
mtcars |> mutate(across(c(mpg, hp), ~ round(.x / max(.x), 2))) |> head(3) mpg cyl disp hp drat wt qsec vs am gear carb
Mazda RX4 0.62 6 160 0.33 3.90 2.620 16.46 0 1 4 4
Mazda RX4 Wag 0.62 6 160 0.33 3.90 2.875 17.02 0 1 4 4
Datsun 710 0.67 4 108 0.28 3.85 2.320 18.61 1 1 4 1
# Same model for each group
mtcars |>
split(~ cyl) |>
map(\(d) lm(mpg ~ wt, data = d)) |>
map_dbl(\(m) coef(m)[["wt"]]) 4 6 8
-5.647025 -2.780106 -2.192438
# Read many files at once
# files <- list.files("data", pattern = ".csv", full.names = TRUE)
# all <- map(files, read_csv) |> list_rbind()Learn more: purrr
4 Part 3: Visualizing with ggplot2
4.1 The grammar
Every plot is built from layers joined with +:
ggplot(data, aes(x, y, color, fill, size, shape)): data and mappingsgeom_*(): the shape (points, bars, lines)labs(): wordsscale_*(): colors and axesfacet_*(): small multiplestheme(): everything visual that is not data
4.2 Common geoms
p <- ggplot(mtcars, aes(wt, mpg))
p + geom_point() # scatter
p + geom_point() + geom_smooth(method = "lm") # with trend line
ggplot(mtcars, aes(factor(cyl))) + geom_bar() # counts
ggplot(iris, aes(Species, Petal.Length)) + geom_boxplot()
ggplot(iris, aes(Petal.Length)) + geom_histogram(bins = 20)
ggplot(airquality, aes(Day, Temp, color = factor(Month))) + geom_line()
ggplot(iris, aes(Petal.Length, fill = Species)) + geom_density(alpha = 0.5)
ggplot(ToothGrowth, aes(factor(dose), len)) + geom_violin() + geom_jitter(width = 0.1)geom_bar() counts rows for you. geom_col() uses a y value you already calculated.
4.3 Titles, subtitles, captions, and labels
base <- ggplot(mtcars, aes(wt, mpg, color = factor(cyl))) + geom_point(size = 3)
base + labs(
title = "Heavier cars get fewer miles per gallon",
subtitle = "Motor Trend 1974 road tests",
caption = "Source: mtcars dataset",
tag = "A", # panel letter for figures
x = "Weight (1,000 lbs)",
y = "Miles per gallon",
color = "Cylinders" # legend title
)4.4 Text style: bold, italic, size, angle, alignment
All text is styled with element_text() inside theme().
| Argument | Options | Meaning |
|---|---|---|
face |
“plain”, “bold”, “italic”, “bold.italic” | font style |
size |
number | points |
color |
“navy”, “#1B4F72” | color |
hjust |
0, 0.5, 1 | left, center, right |
vjust |
0, 0.5, 1 | bottom, middle, top |
angle |
0 to 360 | slant the text |
family |
“serif”, “sans”, “mono” | font |
margin |
margin(t, r, b, l) |
space around text |
lineheight |
number | spacing between lines |
Theme pieces you can target: plot.title, plot.subtitle, plot.caption, axis.title.x, axis.title.y, axis.text.x, axis.text.y, legend.title, legend.text, strip.text (facet labels).
base +
labs(title = "Heavier cars get fewer miles per gallon",
subtitle = "Motor Trend 1974", caption = "Source: mtcars",
x = "Weight", y = "MPG", color = "Cylinders") +
theme(
plot.title = element_text(face = "bold", size = 16, hjust = 0.5),
plot.subtitle = element_text(face = "italic", hjust = 0.5, color = "gray40"),
plot.caption = element_text(face = "italic", hjust = 0, size = 8),
plot.title.position = "plot", # align title with whole plot
axis.title.x = element_text(face = "bold", margin = margin(t = 10)),
axis.text.x = element_text(angle = 45, hjust = 1), # slanted labels
legend.title = element_text(face = "bold")
)Italic inside a label (like species names or p):
ggplot(iris, aes(Sepal.Length, Petal.Length)) + geom_point() +
labs(title = expression(paste("Petal length in ", italic("Iris"), " species")),
y = expression(Petal~length~(cm^2)))4.5 Legend placement
base + theme(legend.position = "bottom") # "top", "left", "right", "none"base + theme(legend.position = "inside", legend.position.inside = c(0.85, 0.8))base + guides(color = guide_legend(nrow = 1, reverse = TRUE))4.6 Annotations
base +
annotate("text", x = 5, y = 30, label = "Light, efficient cars", fontface = "bold") +
annotate("rect", xmin = 1.5, xmax = 2.5, ymin = 25, ymax = 35, alpha = 0.1, fill = "blue") +
annotate("segment", x = 4.5, xend = 5.3, y = 20, yend = 15.5,
arrow = arrow(length = unit(0.2, "cm"))) +
geom_hline(yintercept = mean(mtcars$mpg), linetype = "dashed") +
geom_text(data = mtcars |> rownames_to_column("car") |> filter(mpg > 30),
aes(wt, mpg, label = car), vjust = -1, size = 3, inherit.aes = FALSE)4.7 Colors and scales
library(scales)
base + scale_color_brewer(palette = "Set2") # ColorBrewer palettesbase + scale_color_viridis_d() # colorblind friendlybase + scale_color_manual(values = c("4" = "#1B9E77", "6" = "#D95F02", "8" = "#7570B3"))ggplot(mtcars, aes(wt, mpg, color = hp)) + geom_point() + scale_color_gradient(low = "gold", high = "red")base + scale_x_continuous(limits = c(1, 6), breaks = 1:6) +
scale_y_continuous(labels = label_number(suffix = " mpg"))ggplot(mtcars, aes(factor(cyl))) + geom_bar(aes(y = after_stat(count / sum(count)))) +
scale_y_continuous(labels = label_percent())base + coord_flip() # swap axesbase + scale_y_log10() # log scale4.8 Facets
ggplot(mtcars, aes(wt, mpg)) + geom_point() + facet_wrap(~ cyl, labeller = label_both)ggplot(mtcars, aes(wt, mpg)) + geom_point() + facet_grid(am ~ cyl, scales = "free_y")4.9 Themes
base + theme_minimal()
base + theme_classic()
base + theme_bw()
base + theme_void()Set one theme for the whole document: theme_set(theme_minimal(base_size = 13)).
Other theme elements: panel.grid.minor = element_blank() removes minor grid lines, panel.background = element_rect(fill = "white"), axis.line = element_line(color = "black").
4.10 Combining plots (patchwork)
library(patchwork)
p1 <- ggplot(mtcars, aes(wt, mpg)) + geom_point()
p2 <- ggplot(mtcars, aes(factor(cyl))) + geom_bar()
(p1 | p2) + plot_annotation(title = "Two views of mtcars", tag_levels = "A")Learn more: ggplot2 reference, Wickham, ggplot2: Elegant Graphics for Data Analysis (free online), R Graph Gallery
4.11 Maps (sf and tigris)
What it’s for: county or state maps, such as rates by county for health equity work.
library(sf); library(tigris)
options(tigris_use_cache = TRUE)
wa <- counties(state = "WA", cb = TRUE) # downloads WA county shapes
wa$fake_rate <- runif(nrow(wa), 5, 25) # replace with your data via left_join
ggplot(wa) +
geom_sf(aes(fill = fake_rate), color = "white") +
scale_fill_viridis_c(name = "Rate per 100k") +
labs(title = "Example rate by Washington county") +
theme_void()Join your data to the map with left_join(wa, my_data, by = "GEOID") using the county FIPS code.
Learn more: sf, tigris, Lovelace et al., Geocomputation with R (free online)
5 Part 4: Statistics
5.1 Descriptive statistics
summary(mtcars$mpg) Min. 1st Qu. Median Mean 3rd Qu. Max.
10.40 15.43 19.20 20.09 22.80 33.90
mtcars |> summarise(mean = mean(mpg), median = median(mpg), sd = sd(mpg),
iqr = IQR(mpg), min = min(mpg), max = max(mpg)) mean median sd iqr min max
1 20.09062 19.2 6.026948 7.375 10.4 33.9
quantile(mtcars$mpg, c(0.25, 0.5, 0.75)) 25% 50% 75%
15.425 19.200 22.800
table(mtcars$cyl); prop.table(table(mtcars$cyl))
4 6 8
11 7 14
4 6 8
0.34375 0.21875 0.43750
5.2 Which test do I use?
| Outcome | Groups | Test | R function |
|---|---|---|---|
| Numeric | 2 independent | t-test (or Wilcoxon if skewed) | t.test(), wilcox.test() |
| Numeric | 2 paired | paired t-test | t.test(paired = TRUE) |
| Numeric | 3+ | ANOVA (or Kruskal-Wallis) | aov(), kruskal.test() |
| Categorical | any | chi-square (Fisher if counts < 5) | chisq.test(), fisher.test() |
| Two numerics | none | correlation | cor.test() |
t.test(len ~ supp, data = ToothGrowth) # two groups
Welch Two Sample t-test
data: len by supp
t = 1.9153, df = 55.309, p-value = 0.06063
alternative hypothesis: true difference in means between group OJ and group VC is not equal to 0
95 percent confidence interval:
-0.1710156 7.5710156
sample estimates:
mean in group OJ mean in group VC
20.66333 16.96333
wilcox.test(len ~ supp, data = ToothGrowth)
Wilcoxon rank sum exact test
data: len by supp
W = 575.5, p-value = 0.06366
alternative hypothesis: true location shift is not equal to 0
a <- aov(len ~ factor(dose), data = ToothGrowth); summary(a) Df Sum Sq Mean Sq F value Pr(>F)
factor(dose) 2 2426 1213 67.42 9.53e-16 ***
Residuals 57 1026 18
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
TukeyHSD(a) # which pairs differ Tukey multiple comparisons of means
95% family-wise confidence level
Fit: aov(formula = len ~ factor(dose), data = ToothGrowth)
$`factor(dose)`
diff lwr upr p adj
1-0.5 9.130 5.901805 12.358195 0.00e+00
2-0.5 15.495 12.266805 18.723195 0.00e+00
2-1 6.365 3.136805 9.593195 4.25e-05
chisq.test(table(mtcars$cyl, mtcars$am))
Pearson's Chi-squared test
data: table(mtcars$cyl, mtcars$am)
X-squared = 8.7407, df = 2, p-value = 0.01265
fisher.test(table(mtcars$vs, mtcars$am))
Fisher's Exact Test for Count Data
data: table(mtcars$vs, mtcars$am)
p-value = 0.4727
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
0.3825342 10.5916087
sample estimates:
odds ratio
1.956055
cor.test(mtcars$mpg, mtcars$wt) # Pearson
Pearson's product-moment correlation
data: mtcars$mpg and mtcars$wt
t = -9.559, df = 30, p-value = 1.294e-10
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
-0.9338264 -0.7440872
sample estimates:
cor
-0.8676594
cor.test(mtcars$mpg, mtcars$wt, method = "spearman") # rank based
Spearman's rank correlation rho
data: mtcars$mpg and mtcars$wt
S = 10292, p-value = 1.488e-11
alternative hypothesis: true rho is not equal to 0
sample estimates:
rho
-0.886422
round(cor(mtcars[, 1:5]), 2) # correlation matrix mpg cyl disp hp drat
mpg 1.00 -0.85 -0.85 -0.78 0.68
cyl -0.85 1.00 0.90 0.83 -0.70
disp -0.85 0.90 1.00 0.79 -0.71
hp -0.78 0.83 0.79 1.00 -0.45
drat 0.68 -0.70 -0.71 -0.45 1.00
5.3 Multiple testing
Running many tests inflates false positives. Adjust p-values:
pvals <- c(0.01, 0.02, 0.03, 0.04, 0.20)
p.adjust(pvals, method = "bonferroni") # strict[1] 0.05 0.10 0.15 0.20 1.00
p.adjust(pvals, method = "BH") # false discovery rate, common in genomics[1] 0.05 0.05 0.05 0.05 0.20
5.4 Regression
Linear (numeric outcome) and logistic (yes/no outcome):
library(broom)
lin <- lm(mpg ~ wt + hp + factor(cyl), data = mtcars)
summary(lin)
Call:
lm(formula = mpg ~ wt + hp + factor(cyl), data = mtcars)
Residuals:
Min 1Q Median 3Q Max
-4.2612 -1.0320 -0.3210 0.9281 5.3947
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 35.84600 2.04102 17.563 2.67e-16 ***
wt -3.18140 0.71960 -4.421 0.000144 ***
hp -0.02312 0.01195 -1.934 0.063613 .
factor(cyl)6 -3.35902 1.40167 -2.396 0.023747 *
factor(cyl)8 -3.18588 2.17048 -1.468 0.153705
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 2.44 on 27 degrees of freedom
Multiple R-squared: 0.8572, Adjusted R-squared: 0.8361
F-statistic: 40.53 on 4 and 27 DF, p-value: 4.869e-11
tidy(lin, conf.int = TRUE) # clean table of coefficients# A tibble: 5 × 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 35.8 2.04 17.6 2.67e-16 31.7 40.0
2 wt -3.18 0.720 -4.42 1.44e- 4 -4.66 -1.70
3 hp -0.0231 0.0120 -1.93 6.36e- 2 -0.0476 0.00140
4 factor(cyl)6 -3.36 1.40 -2.40 2.37e- 2 -6.24 -0.483
5 factor(cyl)8 -3.19 2.17 -1.47 1.54e- 1 -7.64 1.27
glance(lin) # R-squared, AIC# A tibble: 1 × 12
r.squared adj.r.squared sigma statistic p.value df logLik AIC BIC
<dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 0.857 0.836 2.44 40.5 4.87e-11 4 -71.2 154. 163.
# ℹ 3 more variables: deviance <dbl>, df.residual <int>, nobs <int>
logit <- glm(am ~ wt + hp, data = mtcars, family = binomial)
tidy(logit, exponentiate = TRUE, conf.int = TRUE) # odds ratios# A tibble: 3 × 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 1.56e+8 7.44 2.53 0.0113 4.34e+3 4.08e+17
2 wt 3.09e-4 3.07 -2.63 0.00843 3.36e-8 2.30e- 2
3 hp 1.04e+0 0.0177 2.04 0.0409 1.01e+0 1.09e+ 0
Other families: family = poisson for counts, MASS::glm.nb() for overdispersed counts.
5.5 Checking model assumptions (performance)
library(performance)
check_model(lin) # linearity, normal residuals, equal variance, outliersError in `loadNamespace()`:
! there is no package called 'see'
check_collinearity(lin) # VIF; above 5 to 10 is a concern# Check for Multicollinearity
Low Correlation
Term VIF VIF 95% CI adj. VIF Tolerance Tolerance 95% CI
wt 2.58 [1.45, 3.60] 1.61 0.39 [0.28, 0.69]
hp 3.50 [1.85, 4.89] 1.87 0.29 [0.20, 0.54]
Moderate Correlation
Term VIF VIF 95% CI adj. VIF Tolerance Tolerance 95% CI
factor(cyl) 5.11 [2.71, 7.35] 1.50 0.20 [0.14, 0.37]
r2(lin)# R2 for Linear Regression
R2: 0.857
adj. R2: 0.836
5.6 Mixed models for repeated measures (lme4)
What it’s for: when the same subject is measured more than once, or people are clustered (patients in clinics). A random intercept lets each subject have their own baseline.
library(lme4)
# sleepstudy: reaction time over 10 days of sleep deprivation, 18 subjects
mm <- lmer(Reaction ~ Days + (1 | Subject), data = sleepstudy)
summary(mm)Linear mixed model fit by REML ['lmerMod']
Formula: Reaction ~ Days + (1 | Subject)
Data: sleepstudy
REML criterion at convergence: 1786.5
Scaled residuals:
Min 1Q Median 3Q Max
-3.2257 -0.5529 0.0109 0.5188 4.2506
Random effects:
Groups Name Variance Std.Dev.
Subject (Intercept) 1378.2 37.12
Residual 960.5 30.99
Number of obs: 180, groups: Subject, 18
Fixed effects:
Estimate Std. Error t value
(Intercept) 251.4051 9.7467 25.79
Days 10.4673 0.8042 13.02
Correlation of Fixed Effects:
(Intr)
Days -0.371
mm2 <- lmer(Reaction ~ Days + (Days | Subject), data = sleepstudy) # random slope too
anova(mm, mm2)Data: sleepstudy
Models:
mm: Reaction ~ Days + (1 | Subject)
mm2: Reaction ~ Days + (Days | Subject)
npar AIC BIC logLik -2*log(L) Chisq Df Pr(>Chisq)
mm 4 1802.1 1814.8 -897.04 1794.1
mm2 6 1763.9 1783.1 -875.97 1751.9 42.139 2 7.072e-10 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Learn more: lme4 vignette
5.7 Bootstrapping
What it’s for: confidence intervals without assuming a normal distribution, by resampling your data with replacement many times.
library(boot)
med_fun <- function(data, i) median(data$mpg[i])
set.seed(42)
b <- boot(mtcars, med_fun, R = 2000)
b
ORDINARY NONPARAMETRIC BOOTSTRAP
Call:
boot(data = mtcars, statistic = med_fun, R = 2000)
Bootstrap Statistics :
original bias std. error
t1* 19.2 0.08595 1.255853
boot.ci(b, type = c("perc", "bca"))BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
Based on 2000 bootstrap replicates
CALL :
boot.ci(boot.out = b, type = c("perc", "bca"))
Intervals :
Level Percentile BCa
95% (16.85, 21.40 ) (15.90, 21.20 )
Calculations and Intervals on Original Scale
# Tidyverse style
library(rsample)
set.seed(42)
bootstraps(mtcars, times = 2000) |>
mutate(med = map_dbl(splits, \(s) median(analysis(s)$mpg))) |>
summarise(lower = quantile(med, 0.025), upper = quantile(med, 0.975))# A tibble: 1 × 2
lower upper
<dbl> <dbl>
1 16.8 21.4
5.8 Power and sample size (pwr)
Give three of four values (effect size, n, alpha, power) and pwr solves for the fourth.
library(pwr)
pwr.t.test(d = 0.5, power = 0.8, sig.level = 0.05) # n per group for a medium effect
Two-sample t test power calculation
n = 63.76561
d = 0.5
sig.level = 0.05
power = 0.8
alternative = two.sided
NOTE: n is number in *each* group
pwr.2p.test(h = ES.h(0.30, 0.20), power = 0.8) # comparing two proportions
Difference of proportion power calculation for binomial distribution (arcsine transformation)
h = 0.2319843
n = 291.6887
sig.level = 0.05
power = 0.8
alternative = two.sided
NOTE: same sample sizes
pwr.chisq.test(w = 0.3, df = 2, power = 0.8)
Chi squared power calculation
w = 0.3
N = 107.0521
df = 2
sig.level = 0.05
power = 0.8
NOTE: N is the number of observations
Cohen’s guide for d: 0.2 small, 0.5 medium, 0.8 large.
Learn more: broom, performance, pwr, Cohen, Statistical Power Analysis for the Behavioral Sciences (1988)
6 Part 5: Epidemiology in R
6.1 Survey weighting (survey and srvyr)
What it’s for: NHANES, BRFSS, and similar surveys oversample some groups on purpose. Weights, strata, and clusters make estimates represent the whole population and give correct standard errors. Unweighted results from these surveys are biased.
library(survey); library(srvyr)
data(api) # apistrat: stratified sample of California schools
des <- apistrat |> as_survey_design(strata = stype, weights = pw, fpc = fpc)
des |> summarise(mean_api = survey_mean(api00, vartype = "ci"))# A tibble: 1 × 3
mean_api mean_api_low mean_api_upp
<dbl> <dbl> <dbl>
1 662. 644. 681.
des |> group_by(stype) |> summarise(pct_meals = survey_mean(meals))# A tibble: 3 × 3
stype pct_meals pct_meals_se
<fct> <dbl> <dbl>
1 E 51.8 2.98
2 H 30.4 3.47
3 M 46.1 3.43
# Regression with the design
svyglm(api00 ~ ell + meals, design = des) |> tidy()# A tibble: 3 × 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 824. 8.76 94.1 2.03e-164
2 ell -0.506 0.388 -1.30 1.94e- 1
3 meals -3.11 0.276 -11.3 4.85e- 23
NHANES template (variable names from the CDC tutorials):
nhanes_des <- nhanes |>
as_survey_design(ids = SDMVPSU, strata = SDMVSTRA, weights = WTMEC2YR, nest = TRUE)
# Subset AFTER building the design, never before
adults <- nhanes_des |> filter(RIDAGEYR >= 20)Watch out for: filtering the data before creating the design gives wrong standard errors. Pick the weight that matches your variables (interview vs exam vs fasting subsample).
Learn more: CDC NHANES tutorials, srvyr, Lumley, Complex Surveys (2010)
6.2 Study designs at a glance
| Design | Direction | Measure | Key R tools |
|---|---|---|---|
| Cohort (prospective) | exposure to outcome, forward | risk, risk ratio, incidence rate | epitools, epiR, survival |
| Retrospective cohort | uses existing records, still exposure to outcome | same as cohort | dplyr cohort building |
| Case-control | outcome to past exposure | odds ratio | glm, clogit, MatchIt |
| Cross-sectional | one point in time | prevalence, prevalence ratio | survey, Poisson regression |
6.3 The 2x2 table
| Outcome yes | Outcome no | |
|---|---|---|
| Exposed | a | b |
| Unexposed | c | d |
Risk ratio = [a / (a + b)] / [c / (c + d)]. Odds ratio = (a × d) / (b × c).
library(epiR)
tab <- matrix(c(40, 60, 20, 80), nrow = 2, byrow = TRUE) # a, b, c, d
epi.2by2(tab, method = "cohort.count") # risk ratio, risk difference, attributable fraction Outcome+ Outcome- Total Inc risk *
Exposure+ 40 60 100 40.00 (30.33 to 50.28)
Exposure- 20 80 100 20.00 (12.67 to 29.18)
Total 60 140 200 30.00 (23.74 to 36.86)
Point estimates and 95% CIs:
-------------------------------------------------------------------
Inc risk ratio 2.00 (1.26, 3.17)
Inc odds ratio 2.67 (1.42, 5.02)
Attrib risk in the exposed * 20.00 (7.60, 32.40)
Attrib fraction in the exposed (%) 50.00 (21.73, 68.56)
Attrib risk in the population * 10.00 (-0.09, 20.09)
Attrib fraction in the population (%) 33.33 (20.83, 46.65)
E-value 3.41 (1.84, NA)
-------------------------------------------------------------------
Uncorrected chi2 test that OR = 1: chi2(1) = 9.524 Pr>chi2 = 0.002
Fisher exact test that OR = 1: Pr>chi2 = 0.003
Wald confidence limits
CI: confidence interval
* Outcomes per 100 population units
epi.2by2(tab, method = "case.control") # odds ratio Outcome+ Outcome- Total Odds
Exposed + 40 60 100 0.67 (0.45 to 1.00)
Exposed - 20 80 100 0.25 (0.14 to 0.39)
Total 60 140 200 0.43 (0.32 to 0.57)
Point estimates and 95% CIs:
-------------------------------------------------------------------
Exposure odds ratio 2.67 (1.42, 5.02)
Attrib fraction (est) in the exposed (%) 62.50 (29.66, 79.99)
Attrib fraction (est) in the population (%) 41.67 (32.34, 56.82)
E-value 2.65 (1.67, NA)
-------------------------------------------------------------------
Uncorrected chi2 test that OR = 1: chi2(1) = 9.524 Pr>chi2 = 0.002
Fisher exact test that OR = 1: Pr>chi2 = 0.003
Wald confidence limits
CI: confidence interval
library(epitools)
riskratio(matrix(c(80, 20, 60, 40), 2, byrow = TRUE)) # epitools wants unexposed in row 1, outcome-no first$data
Outcome
Predictor Disease1 Disease2 Total
Exposed1 80 20 100
Exposed2 60 40 100
Total 140 60 200
$measure
risk ratio with 95% C.I.
Predictor estimate lower upper
Exposed1 1 NA NA
Exposed2 2 1.263006 3.167047
$p.value
two-sided
Predictor midp.exact fisher.exact chi.square
Exposed1 NA NA NA
Exposed2 0.002145336 0.003191817 0.002028231
$correction
[1] FALSE
attr(,"method")
[1] "Unconditional MLE & normal approximation (Wald) CI"
6.4 Building a cohort from records
Steps for a retrospective cohort from EHR-style or All of Us data:
- Define the index date (first diagnosis, first prescription, enrollment)
- Apply inclusion and exclusion criteria
- Require a lookback window (no prior outcome in, say, 365 days before index)
- Define follow-up: from index until outcome, death, end of data, or end of study
- Calculate person-time and the event indicator
set.seed(1)
visits <- tibble(
id = rep(1:6, each = 3),
date = ymd("2019-01-01") + sample(0:1500, 18),
code = sample(c("exposure", "outcome", "other"), 18, replace = TRUE)
)
index <- visits |> filter(code == "exposure") |>
group_by(id) |> summarise(index_date = min(date))
prior <- visits |> inner_join(index, by = "id") |>
filter(code == "outcome", date < index_date, date >= index_date - days(365)) |> distinct(id)
cohort <- index |>
anti_join(prior, by = "id") |> # exclude prior outcome
left_join(visits |> filter(code == "outcome"), by = "id", relationship = "many-to-many") |>
filter(is.na(date) | date > index_date) |>
group_by(id, index_date) |>
summarise(event_date = suppressWarnings(min(date)), .groups = "drop") |>
mutate(
event = as.integer(!is.na(event_date) & is.finite(event_date)),
end_date = if_else(event == 1, event_date, ymd("2023-12-31")),
years = as.numeric(end_date - index_date) / 365.25
)
cohort# A tibble: 3 × 6
id index_date event_date event end_date years
<int> <date> <date> <int> <date> <dbl>
1 1 2020-11-09 2021-10-13 1 2021-10-13 0.925
2 4 2019-11-26 2022-07-24 1 2022-07-24 2.66
3 5 2020-12-29 2022-02-02 1 2022-02-02 1.10
cohort |> summarise(events = sum(event), person_years = sum(years),
rate_per_1000py = 1000 * events / person_years)# A tibble: 1 × 3
events person_years rate_per_1000py
<int> <dbl> <dbl>
1 3 4.68 641.
6.5 Case-control analysis
# infert: matched case-control study of infertility after abortion
glm(case ~ spontaneous + induced + education, data = infert, family = binomial) |>
tidy(exponentiate = TRUE, conf.int = TRUE)# A tibble: 5 × 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 0.172 0.728 -2.42 0.0157 0.0372 0.676
2 spontaneous 3.33 0.212 5.67 0.0000000139 2.22 5.12
3 induced 1.53 0.209 2.04 0.0414 1.02 2.32
4 education6-11yrs 1.12 0.706 0.156 0.876 0.293 4.93
5 education12+ yrs 0.976 0.704 -0.0347 0.972 0.257 4.28
# Conditional logistic regression respects the matched sets
library(survival)
clogit(case ~ spontaneous + induced + strata(stratum), data = infert) |> summary()Call:
coxph(formula = Surv(rep(1, 248L), case) ~ spontaneous + induced +
strata(stratum), data = infert, method = "exact")
n= 248, number of events= 83
coef exp(coef) se(coef) z Pr(>|z|)
spontaneous 1.9859 7.2854 0.3524 5.635 1.75e-08 ***
induced 1.4090 4.0919 0.3607 3.906 9.38e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
exp(coef) exp(-coef) lower .95 upper .95
spontaneous 7.285 0.1373 3.651 14.536
induced 4.092 0.2444 2.018 8.298
Concordance= 0.776 (se = 0.044 )
Likelihood ratio test= 53.15 on 2 df, p=3e-12
Wald test = 31.84 on 2 df, p=1e-07
Score (logrank) test = 48.44 on 2 df, p=3e-11
Matching or propensity scores (MatchIt):
library(MatchIt)
m <- matchit(am ~ wt + hp, data = mtcars, method = "nearest", ratio = 1)
summary(m)$sum.matched[, 1:3] # balance after matching Means Treated Means Control Std. Mean Diff.
distance 0.8879044 0.1114532 4.2186553
wt 2.4110000 3.4011538 -1.6048352
hp 126.8461538 148.8461538 -0.2617106
matched <- match.data(m)6.6 Cross-sectional analysis
Prevalence ratios are often preferred over odds ratios when the outcome is common (over about 10%). Use Poisson regression with robust errors, or svyglm(family = quasipoisson) for survey data.
pr <- glm(am ~ factor(cyl), data = mtcars, family = poisson)
tidy(pr, exponentiate = TRUE, conf.int = TRUE)# A tibble: 3 × 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 0.727 0.354 -0.901 0.368 0.332 1.35
2 factor(cyl)6 0.589 0.677 -0.781 0.435 0.129 2.04
3 factor(cyl)8 0.196 0.791 -2.06 0.0395 0.0296 0.784
6.7 Table 1 and confounding
library(gtsummary)
trial |> # built into gtsummary
select(trt, age, grade, response) |>
tbl_summary(by = trt, missing = "ifany") |>
add_p() |> add_overall() |> bold_labels()| Characteristic | Overall N = 2001 |
Drug A N = 981 |
Drug B N = 1021 |
p-value2 |
|---|---|---|---|---|
| Age | 47 (38, 57) | 46 (37, 60) | 48 (39, 56) | 0.7 |
| Unknown | 11 | 7 | 4 | |
| Grade | 0.9 | |||
| I | 68 (34%) | 35 (36%) | 33 (32%) | |
| II | 68 (34%) | 32 (33%) | 36 (35%) | |
| III | 64 (32%) | 31 (32%) | 33 (32%) | |
| Tumor Response | 61 (32%) | 28 (29%) | 33 (34%) | 0.5 |
| Unknown | 7 | 3 | 4 | |
| 1 Median (Q1, Q3); n (%) | ||||
| 2 Wilcoxon rank sum test; Pearson’s Chi-squared test | ||||
# Crude vs adjusted odds ratios
glm(response ~ trt + age + grade, data = trial, family = binomial) |>
tbl_regression(exponentiate = TRUE)Error in `check_pkg_installed()`:
! The package "broom.helpers" (>= 1.20.0) is required.
A big change between crude and adjusted estimates (often more than 10%) suggests confounding.
Learn more: epiR, MatchIt, gtsummary, Batra et al., The Epidemiologist R Handbook (free online), Rothman, Epidemiology: An Introduction
6.8 Survival analysis (time to event)
What it’s for: how long until an event (death, relapse), handling people who leave the study before the event (censoring).
library(survival); library(survminer)
lung2 <- lung |> mutate(status = status - 1, sex = factor(sex, labels = c("Male", "Female")))
km <- survfit(Surv(time, status) ~ sex, data = lung2)
summary(km, times = c(180, 365))Call: survfit(formula = Surv(time, status) ~ sex, data = lung2)
sex=Male
time n.risk n.event survival std.err lower 95% CI upper 95% CI
180 89 49 0.644 0.0408 0.569 0.730
365 35 36 0.336 0.0434 0.261 0.433
sex=Female
time n.risk n.event survival std.err lower 95% CI upper 95% CI
180 71 14 0.842 0.0387 0.770 0.922
365 30 22 0.526 0.0597 0.421 0.658
ggsurvplot(km, conf.int = TRUE, risk.table = TRUE, pval = TRUE,
xlab = "Days", title = "Kaplan-Meier survival by sex")survdiff(Surv(time, status) ~ sex, data = lung2) # log-rank testCall:
survdiff(formula = Surv(time, status) ~ sex, data = lung2)
N Observed Expected (O-E)^2/E (O-E)^2/V
sex=Male 138 112 91.6 4.55 10.3
sex=Female 90 53 73.4 5.68 10.3
Chisq= 10.3 on 1 degrees of freedom, p= 0.001
cox <- coxph(Surv(time, status) ~ sex + age + ph.ecog, data = lung2)
tidy(cox, exponentiate = TRUE, conf.int = TRUE) # hazard ratios# A tibble: 3 × 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 sexFemale 0.575 0.168 -3.29 0.000986 0.414 0.799
2 age 1.01 0.00927 1.19 0.232 0.993 1.03
3 ph.ecog 1.59 0.114 4.08 0.0000445 1.27 1.99
cox.zph(cox) # proportional hazards check chisq df p
sex 2.305 1 0.13
age 0.188 1 0.66
ph.ecog 2.054 1 0.15
GLOBAL 4.464 3 0.22
Learn more: Zabor, Survival Analysis in R tutorial
6.9 Time series
What it’s for: a value tracked over regular time points (monthly ER visits, daily crisis calls). Watch for trend, seasonality, and autocorrelation.
library(forecast)
autoplot(AirPassengers) + labs(title = "Monthly airline passengers")decompose(AirPassengers, type = "multiplicative") |> autoplot()fit <- auto.arima(AirPassengers)
fitSeries: AirPassengers
ARIMA(2,1,1)(0,1,0)[12]
Coefficients:
ar1 ar2 ma1
0.5960 0.2142 -0.9819
s.e. 0.0888 0.0880 0.0292
sigma^2 = 132.3: log likelihood = -504.92
AIC=1017.85 AICc=1018.17 BIC=1029.35
forecast(fit, h = 24) |> autoplot()ggAcf(AirPassengers) # autocorrelationTurn your own data into a time series: ts(df$count, start = c(2019, 1), frequency = 12).
Learn more: Hyndman and Athanasopoulos, Forecasting: Principles and Practice (free online)
7 Part 6: Finding Patterns
7.1 Principal component analysis (PCA)
What it’s for: squeezing many correlated numeric variables into a few components. Works on continuous variables.
pca <- prcomp(USArrests, scale. = TRUE) # always scale when units differ
summary(pca) # variance explainedImportance of components:
PC1 PC2 PC3 PC4
Standard deviation 1.5749 0.9949 0.59713 0.41645
Proportion of Variance 0.6201 0.2474 0.08914 0.04336
Cumulative Proportion 0.6201 0.8675 0.95664 1.00000
pca$rotation # loadings: which variables drive each PC PC1 PC2 PC3 PC4
Murder -0.5358995 -0.4181809 0.3412327 0.64922780
Assault -0.5831836 -0.1879856 0.2681484 -0.74340748
UrbanPop -0.2781909 0.8728062 0.3780158 0.13387773
Rape -0.5434321 0.1673186 -0.8177779 0.08902432
tibble(state = rownames(USArrests), as_tibble(pca$x)) |>
ggplot(aes(PC1, PC2, label = state)) + geom_text(size = 3) +
labs(title = "US states on the first two components")screeplot(pca, type = "lines")7.2 Latent class analysis (LCA)
What it’s for: finding hidden groups of people from categorical answers (like adversity patterns). PCA groups variables; LCA groups people.
library(poLCA)
data(values) # 4 yes/no survey items, coded 1 and 2
f <- cbind(A, B, C, D) ~ 1
set.seed(10)
lca2 <- poLCA(f, values, nclass = 2, nrep = 5, verbose = FALSE)
lca3 <- poLCA(f, values, nclass = 3, nrep = 5, verbose = FALSE)
c(BIC_2 = lca2$bic, BIC_3 = lca3$bic) # lower BIC is better BIC_2 BIC_3
1057.313 1081.856
lca2$P # class sizes[1] 0.7207525 0.2792475
Watch out for: poLCA needs categories coded 1, 2, 3 (no zeros). Fit several class counts and compare BIC.
Learn more: poLCA, James et al., An Introduction to Statistical Learning (free online)
8 Part 7: Sharing Your Work
8.1 Exporting plots and tables
ggsave("figure1.png", plot = base, width = 7, height = 5, dpi = 300) # poster quality
ggsave("figure1.pdf", plot = base, width = 7, height = 5)
write_csv(cohort, "cohort.csv")
library(gt)
mtcars |> head() |> gt() |> tab_header(title = "First six cars") |> gtsave("table.html")
tbl_summary(trial) |> as_flex_table() |> flextable::save_as_docx(path = "table1.docx")8.2 Making your own Quarto documents
A .qmd file has three parts:
- YAML header between
---lines: title, format, options - Text written in markdown:
**bold**,*italic*,# Heading,[link](url) - Code chunks that start with three backticks and
{r}
Chunk options go at the top of a chunk with #|:
| Option | Does |
|---|---|
#| echo: false |
hide code, show output |
#| eval: false |
show code, do not run |
#| include: false |
run silently (good for setup) |
#| warning: false |
hide warnings |
#| fig-cap: "Caption" |
figure caption |
#| fig-width: 7 |
figure size |
#| label: fig-scatter |
lets you cross-reference with @fig-scatter |
Output formats in the YAML:
format: html # web page
format: docx # Word
format: pdf # needs tinytex: quarto install tinytex
format: revealjs # slidesFor slides, each ## heading is a new slide. Images: {width=60%}. Render with the Render button or quarto render file.qmd in the terminal. Use here() for paths so the project works on any computer, and use RStudio Projects to keep each study in its own folder.
Learn more: Quarto guide, Quarto slides, Wickham et al., R for Data Science, 2e (free online)
8.3 R Shiny apps
What it’s for: interactive web apps where users pick options and plots update. Shiny needs a live R session, so this code is shown, not run. Paste it into a file called app.R and click Run App.
Every app has a UI (what users see), a server (what R calculates), and shinyApp() to join them.
library(shiny); library(ggplot2)
ui <- fluidPage(
titlePanel("Explore mtcars"),
sidebarLayout(
sidebarPanel(
selectInput("xvar", "X variable", choices = names(mtcars), selected = "wt"),
selectInput("yvar", "Y variable", choices = names(mtcars), selected = "mpg"),
sliderInput("size", "Point size", min = 1, max = 6, value = 3),
checkboxInput("trend", "Add trend line", value = FALSE)
),
mainPanel(plotOutput("scatter"), tableOutput("summary"))
)
)
server <- function(input, output) {
output$scatter <- renderPlot({
p <- ggplot(mtcars, aes(.data[[input$xvar]], .data[[input$yvar]])) +
geom_point(size = input$size) + theme_minimal()
if (input$trend) p <- p + geom_smooth(method = "lm")
p
})
output$summary <- renderTable({
data.frame(stat = c("mean", "sd"),
value = c(mean(mtcars[[input$yvar]]), sd(mtcars[[input$yvar]])))
})
}
shinyApp(ui, server)Inputs (input$name) come from the UI. Outputs use matching pairs: plotOutput() with renderPlot(), tableOutput() with renderTable(), textOutput() with renderText(). Share free at shinyapps.io with rsconnect::deployApp().
Learn more: Shiny tutorials, Wickham, Mastering Shiny (free online)
9 Part 8: Common Errors and Fixes
| Error message | Usual cause | Fix |
|---|---|---|
could not find function "x" |
package not loaded | library(pkg) |
there is no package called 'x' |
not installed | install.packages("x") |
object 'x' not found |
typo, or column used outside a dplyr verb | check spelling, use df$x |
non-numeric argument to binary operator |
numbers stored as text | as.numeric() |
arguments imply differing number of rows |
vectors of different lengths | check length() |
unexpected symbol |
missing comma or parenthesis | check the line above |
NAs introduced by coercion |
text that is not a number | inspect with unique() |
Result is all NA |
missing values | na.rm = TRUE |
| Rows multiplied after join | duplicate keys | check count(df, id) |> filter(n > 1) |
Error in plot.new(): figure margins too large |
plot pane too small | enlarge pane |
ggplot + error |
+ placed at start of line |
put + at the end of the line |
General tips: read the last line of the error first, restart R (Session, Restart R) when things act strange, and use ?function_name for help.
10 Sources
- Wickham, Çetinkaya-Rundel, and Grolemund. R for Data Science, 2e. https://r4ds.hadley.nz
- Wickham. ggplot2: Elegant Graphics for Data Analysis, 3e. https://ggplot2-book.org
- Batra et al. The Epidemiologist R Handbook. https://epirhandbook.com
- Lumley. Complex Surveys: A Guide to Analysis Using R. Wiley, 2010.
- Hyndman and Athanasopoulos. Forecasting: Principles and Practice, 3e. https://otexts.com/fpp3/
- van Buuren. Flexible Imputation of Missing Data, 2e. https://stefvanbuuren.name/fimd/
- James et al. An Introduction to Statistical Learning. https://www.statlearning.com
- Wickham. Mastering Shiny. https://mastering-shiny.org
- CDC NHANES Tutorials. https://wwwn.cdc.gov/nchs/nhanes/tutorials/
- Quarto documentation. https://quarto.org
- Package documentation on CRAN and tidyverse.org, linked in each section