Introduction

This practical brings three digital humanities techniques together in R:

  1. Interactive time-series graphs with dygraphs
  2. String matching and regular expressions with stringr, applied to a dataset of Reformation pamphlets
  3. Mapping (GIS) with sf and tmap, plotting earthquake-affected villages in Kazakhstan and Kyrgyzstan against elevation and river data

All the data files (reformation-pamphlets.txt, earthquake_coords.csv, the sas/ shapefile folder and the two .gpkg files) sit in the same project folder as this R Markdown file.

Setup

library(tidyverse)
library(tmap)
library(tmaptools)
library(sf)
library(dygraphs)

# When using tmap and sf to map coordinates, turn off the S2 spherical geometry engine
sf::sf_use_s2(FALSE)

Turning off S2 makes sf treat longitude/latitude as flat (planar) coordinates. This avoids geometry errors when joining the shapefiles later on. The cost is a small loss of accuracy, which doesn’t matter at this scale.

Part 1. Dygraphs

dygraphs produces interactive time-series charts: you can hover over points to read values and zoom in on a period. As sample data I use two datasets built into R, mdeaths and fdeaths. They record monthly deaths from lung disease in the UK from 1974 to 1979, for men and women separately.

lungdeaths <- cbind(mdeaths, fdeaths)

Initial graph

lungdeaths %>% dygraph()

Stacked graph

Here each series gets a readable label, the two series are stacked so the total is visible, and a range selector underneath lets you zoom in on part of the timeline.

lungdeaths %>% dygraph() %>%
  dySeries("mdeaths", label = "Male") %>%
  dySeries("fdeaths", label = "Female") %>%
  dyOptions(stackedGraph = TRUE) %>%
  dyRangeSelector(height = 20)

The graph shows a clear seasonal pattern, with deaths peaking each winter, and male deaths consistently higher than female deaths. Being able to zoom in on a single winter is something a static chart can’t offer.

Part 2. String Matching & Regex

Regular expressions (“regex”) are patterns for searching text. For humanities data this matters because sources are messy: spellings vary, dates come in different formats, and titles are inconsistent. str_view() highlights where a pattern matches in each string, shown between < >.

Basic matching

data("fruit")

# Literal match: any fruit containing "berry"
str_view(fruit, "berry")
##  [6] │ bil<berry>
##  [7] │ black<berry>
## [10] │ blue<berry>
## [11] │ boysen<berry>
## [19] │ cloud<berry>
## [21] │ cran<berry>
## [29] │ elder<berry>
## [32] │ goji <berry>
## [33] │ goose<berry>
## [38] │ huckle<berry>
## [50] │ mul<berry>
## [70] │ rasp<berry>
## [73] │ salal <berry>
## [76] │ straw<berry>

The . matches any single character:

str_view(c("a", "ab", "ae", "bd", "ea", "eab"), "a.")
## [2] │ <ab>
## [3] │ <ae>
## [6] │ e<ab>
# "a", then any three characters, then "e"
str_view(fruit, "a...e")
##  [1] │ <apple>
##  [7] │ bl<ackbe>rry
## [48] │ mand<arine>
## [51] │ nect<arine>
## [62] │ pine<apple>
## [64] │ pomegr<anate>
## [70] │ r<aspbe>rry
## [73] │ sal<al be>rry

Quantifiers

Quantifiers control how many times a pattern can match:

  • ? makes a pattern optional (it matches 0 or 1 times)
  • + lets a pattern repeat (it matches at least once)
  • * lets a pattern be optional or repeat (any number of times, including 0)
str_view(c("a", "ab", "abb"), "ab?")
## [1] │ <a>
## [2] │ <ab>
## [3] │ <ab>b
str_view(c("a", "ab", "abb"), "ab+")
## [2] │ <ab>
## [3] │ <abb>
str_view(c("a", "ab", "abb"), "ab*")
## [1] │ <a>
## [2] │ <ab>
## [3] │ <abb>

With ab+, the plain "a" doesn’t match at all, because + needs at least one b.

Character classes

Square brackets [] match any one character from a set: [abcd] matches “a”, “b”, “c” or “d”. Starting the set with ^ inverts it: [^abcd] matches anything except those letters.

# An "x" surrounded by vowels
str_view(words, "[aeiou]x[aeiou]")
## [284] │ <exa>ct
## [285] │ <exa>mple
## [288] │ <exe>rcise
## [289] │ <exi>st
# A "y" surrounded by consonants
str_view(words, "[^aeiou]y[^aeiou]")
## [836] │ <sys>tem
## [901] │ <typ>e

Detecting matches

str_detect() returns TRUE or FALSE for each element. This is what makes regex useful for filtering a dataset.

str_detect(c("a", "b", "c"), "[aeiou]")
## [1]  TRUE FALSE FALSE

Applying regex to the Reformation pamphlets

The dataset contains 153 Reformation-era pamphlets, with author, title, place and year of printing.

reformation_pamphlets <- read_delim("reformation-pamphlets.txt")

glimpse(reformation_pamphlets)
## Rows: 153
## Columns: 5
## $ Author  <chr> "Amherst", "Luther", "Luther", "Luther", "Luther", "Luther", "L…
## $ title   <chr> " ", " Martin", " Martin", " Martin", " Martin", " Martin", " M…
## $ city    <chr> " [The New Testament / translated into German by Martin Luther]…
## $ printer <chr> " s.l.", " pro declaratione virtvtis indvlgentiarvm..", " Augsb…
## $ year    <chr> " s.n., ", " Basel, Adam Petri, 1517", " Sylvan Otmar, 1518", "…

Using grepl() (base R’s equivalent of str_detect()), I can keep only the pamphlets whose year contains “15”, i.e. those printed in the 1500s:

pamphlets_1500s <- reformation_pamphlets %>%
  filter(grepl("15", year))

nrow(pamphlets_1500s)
## [1] 147
head(pamphlets_1500s, 10)
## # A tibble: 10 × 5
##    Author title     city                                            printer year 
##    <chr>  <chr>     <chr>                                           <chr>   <chr>
##  1 Luther " Martin" " Dispvtatio D. Martini Lvther theologi"        " pro … " Ba…
##  2 Luther " Martin" " Theologia / Teütsch : [hand] Das ist ain ede… " Augs… " Sy…
##  3 Luther " Martin" " Eyn gutte trost- / liche predig vō der wirdi… " Leip… " Va…
##  4 Luther " Martin" " Ausslegung des hundert vnd neündten [110] ps… " Augs… " Sy…
##  5 Luther " Martin" " In Epistolam Pauli ad Galatas ... commentari… " Leip… " Me…
##  6 Luther " Martin" " Resolutio Lvtheriana [in answer to Johann vo… " [of … " Ba…
##  7 Luther " Martin" " Die syben Bůsspsalme[n] : Mit teütscher auss… " Stra… " Jo…
##  8 Luther " Martin" " Ain Sermon von dem Wücher Doctor Martini Lut… " Augs… " s.…
##  9 Luther " Martin" " Eyn S[er]mo[n] von der Bereytung zum Sterben… " Erfu… " Ha…
## 10 Luther " Martin" " Sermon Martini Lutheri de praeparatione ad m… " Leip… " Me…

A tighter pattern would be "^15", where ^ anchors the match to the start of the value. "15" on its own would also match a year like 1615, or any stray “15” in a messy date field.

Part 3. Mapping

Earthquake village coordinates

The CSV gives the latitude and longitude of villages affected by an earthquake. I drop rows with missing values, then convert the table into a simple feature (sf) object, R’s standard spatial format. crs = 4326 sets the coordinate reference system to WGS84, the ordinary GPS latitude/longitude system.

earthquake_villages <- read.csv("earthquake_coords.csv", stringsAsFactors = FALSE)

earthquake_villages <- earthquake_villages %>%
  na.omit() %>%
  as_tibble()

earthquake_villages_coords <- st_as_sf(earthquake_villages,
                                       coords = c("Long", "Lat"),
                                       crs = 4326)

earthquake_villages_coords
## Simple feature collection with 28 features and 3 fields
## Geometry type: POINT
## Dimension:     XY
## Bounding box:  xmin: 73.84503 ymin: 42.51188 xmax: 78.5227 ymax: 43.81588
## Geodetic CRS:  WGS 84
## # A tibble: 28 × 4
##     Year Village                 Contemporary.name.lo…¹            geometry
##  * <int> <chr>                   <chr>                          <POINT [°]>
##  1  1885 Belovodskoe             Belovodskoe (Kyrgyzst…  (74.1108 42.82338)
##  2  1885 Karabaltinskoe [Karaba… Kara-Balta, west of B… (73.84503 42.82662)
##  3  1885 Sukulukskoe             Somewhere near Pishpek (74.57022 42.87648)
##  4  1885 Issygatinskoe           Somewhere near Pishpek (74.57022 42.87648)
##  5  1885 Aleksandrovskoe         Somewhere near Pishpek (74.57022 42.87648)
##  6  1885 Alamedinskoe            Somewhere near Pishpek (74.57022 42.87648)
##  7  1887 Sazanovskoe             Anan'evo, northern sh… (77.66989 42.73478)
##  8  1887 Malovodnoe              Baydibek biy, NE of A…  (77.67393 43.5122)
##  9  1887 Zaitsevskoe             Chilik/Shelek (Kazakh…  (78.2576 43.60177)
## 10  1887 Nadezhdinskii           Esik/Issyk, between T… (77.45266 43.35578)
## # ℹ 18 more rows
## # ℹ abbreviated name: ¹​Contemporary.name.location.details..googlemaps.spellings.

Physical geography and administrative boundaries

Next I import shapefiles of elevation contours and watercourses for the region. I also import the national boundaries of Kyrgyzstan and Kazakhstan from GADM (the ADM_0 layer is the country outline).

asia_elevation <- st_read("sas/elev-contour-l.shp")
asia_water     <- st_read("sas/hydro-water-course-l.shp")

kgyz_unit_0   <- st_read("gadm41_KGZ.gpkg", layer = "ADM_ADM_0")
kazakh_unit_0 <- st_read("gadm41_KAZ.gpkg", layer = "ADM_ADM_0")

Static and interactive maps

tmap builds maps in layers: tm_shape() names the data and the next function says how to draw it. tmap_mode("plot") gives a static map; tmap_mode("view") gives an interactive Leaflet map with a basemap you can zoom and pan.

tmap_mode("plot")

tm_shape(earthquake_villages_coords) + tm_dots(size = 0.75)

tmap_mode("view")

tm_shape(earthquake_villages_coords) + tm_dots(size = 0.75)

Clipping the data to each country

The elevation and river files cover a far bigger area than we need (over 220,000 contour lines). st_join(..., left = FALSE) keeps only the features that intersect each country’s boundary, which works like clipping the data to that country.

kazah_elevation_sf <- st_join(asia_elevation, kazakh_unit_0, left = FALSE)
kgyz_elevation_sf  <- st_join(asia_elevation, kgyz_unit_0,  left = FALSE)

kazah_rivers_sf <- st_join(asia_water, kazakh_unit_0, left = FALSE)
kgyz_rivers_sf  <- st_join(asia_water, kgyz_unit_0,  left = FALSE)

Final map

All the layers drawn together: villages as black dots, elevation contours in red, rivers in blue. I switch back to a static map here, because the interactive version holds thousands of contour and river lines and makes the page too large to publish.

tmap_mode("plot")

tm_shape(earthquake_villages_coords) + tm_dots(size = 0.75) +
  tm_shape(kazah_elevation_sf) + tm_lines(col = "red") +
  tm_shape(kgyz_elevation_sf)  + tm_lines(col = "red") +
  tm_shape(kazah_rivers_sf)    + tm_lines(col = "blue") +
  tm_shape(kgyz_rivers_sf)     + tm_lines(col = "blue")

The affected villages cluster in the zone where the dense contours of the Tien Shan mountains meet the lower ground along the Kazakhstan–Kyrgyzstan border. Many also sit close to river valleys, which is where settlement tends to concentrate in mountainous terrain.