Accessible Visualizations

Learning Goals

  • Learn to chose a color palette that is distingushable by everyone
  • Understand the basics of how a screen reader and keyboard navigation works and how someone who is blind or someone with motor impairments might process and interact with data visualizations
  • Learn to create visualizations accessible with a screen reader or keyboard navigation with the highcharter library
  • Learn how to map a variable to sound to create sonificatons using the highcharter library

Download a template .Rmd of this activity. Put it in a folder Assignment_11 in COMP_STAT_112.

Introduction to Accessibility

At the beginning of the COVID-19 pandemic, the CDC created a flatten the curve graphic, which illustrated the importance of staying socially isolated to preven the spread of COVID. Throughout the pandemic, the New York Times and various government organizations used interactive data visualizations that allowed people to see the number of local cases and death rates. These graphics have played a crucial role in allowing people to make informed decisions about protecting themselves. Unfortunately, these visualizations have been largely inaccessible to people who are blind.

As you are learning in this class, data visualizations are an powerful way to quickly draw insights from data and gain a better understanding of the world. However, people who are blind are consistently locked out from accessing them. In this module, we will learn how to make our designs less disabling to this population.

Example 1 (Accessing a Data Visualizations Non-Visually) Chat with a classmate and brainstorm ways that someone who is blind might access the data within a data visualization. You can think about ways that they might already be doing so and future technology that could help. Get creative!

Solution
  • Reading a description given alt text (we’ve been writing this all semester),
  • Look at the data in table form,
  • Using a tactile graphic,
  • Use a screen reader, software that reads aloud the different data elements as you scan through them. On a touchscreen, if it’s programmed correctly, it can read aloud elements as you touch them,
  • Use sonification to map data to different aspects of sound using pitch, volume or instrument instead of our traditional visual aesthetics such as size, color or position,

Chosing an Accessible Color Palette

Globally over 2.2. billion people have some kind of visual impairment. Most people with visual impairments have some functional vision, this can include color blindness or blurry vision, so it’s important to keep those people in mind when you design your visualizations.

A few tips for designing accessible graphics:

  • Get in right in Black and White! Make sure your colors can be distinguished when shown in black and white as well as in full color. You’ll need to vary the saturation of colors as well as the hue. The viridis color scales work well for this in R.
  • If you have points on a background with varied colors (e.g. data points on a chloropleth map, use white outlines around the points to help make them distinguishable). Typically, high contrast is easier to see for people with low-vision.
  • Think about using fills, patterns, and/or dashed lines to distinguish between your data points, but be careful not to make an overwhelming graph.
  • As always, make sure to write good alt text. People with low vision may refer to your alt text to make sense of what they can perceive about the graphic. Reminder that alt text should concisely articulate (1) what your visualization is (e.g. a bar chart showing which the harvest rate of cucumbers), (2) a one sentence description of the what you think is the most important takeaway your visualization is showing, and (3) a link to your data source if it’s not already in the caption check out this (great resource on writing alt text for data visualizations).

Exercise 1 Copy and paste the data visualization below (showing how flipper and bill length differ across three species of penguins) into this color blindness simulator and look at with the Monochromacy lens. Update the color scale so the points are distinguishable in grayscale.

data(penguins, package = "palmerpenguins") 

penguins %>% 
  ggplot(aes(x = flipper_length_mm, y = bill_length_mm, color = species)) +
  geom_point()

Scatterplot showing how flipper and bill length differ across three species of penguins. Gentoo penguins typically have the longest flipper length with values ranging between 210-230mm. Adelie and Chinstrap have flipper lengths ranging between 180-210mm. Chinstrap and Gentoo have longer bill lengths with an average of around 50mm and Adelie penguins have bill lengths ranging between 35-45mm.

HighCharter Library Basics

We are going to be using the Highcharter library to create interactive and accessible graphics. Similar to Leaflet, the HighCharter library is an R wrapper for a JavaScript library (Highcharts). Because there is JavaScript underneath, that means we can use Highcharter to make interactive instead of static graphics.

To install the Highcharter library, run the following code in the console. NOTE Do not install the Highcharter library from CRAN or the packages menu, because the current version does not support the accessibility module!

remotes::install_github("jbkunst/highcharter@8ff41366c8c411b497b5378d27be48617360f81f")

If the current version of highcharter is already installed, you can uninstall it by navigating to the Packages frame in the lower right corner of the RStudio IDE, scrolling down to highcharter and clicking the x on the right-hand side of the row (see image below).

Highcharter Introduction

First, let’s take a look at what a highcharter plot looks like using the same penguin data we used above. Make sure to load the library before running the code below.

library(highcharter)
hchart(
  penguins,
  "scatter",
  hcaes(x = flipper_length_mm, y = bill_length_mm, group = species)
)

Example 2 Chat with someone next to you. How do the default highcharter and ggplots differ for the penguin scatterplot? If you are feeling brave, try right-clicking the graph and selecting Inspect element to see how the images are represented in code.

Solution
  • The highcharter graphic is interactive (Thanks Javascript!) You can hide/show datasets by selecting the names within the menu. You can also hover over data points and see the location and group it belongs to.
  • The default background is a little cleaner for highcharter.
  • If you hit inspect, you’ll see that the ggplot graph is represented as a single img element in html and the highcharter graphic is represented as a scalable vector graphic (SVG), with different elements representing each of the points, text elements, etc… This is good news, both because you can make the elements interactive and abecause you can surface information about the elements to assistive technology such as a screen reader, so someone who is blind does not have to rely on alt text alone.

As you can see the way to write code for highcharter is a little different than ggplot. You use the form: hchart(<data>, <type_of_chart>, hcaes(<aesthetics>), ...)

where

  • data is the data to chart. Note that you can not pipe in data as you would with ggplot.
  • type_of_chart is a string (uses "") to specify the type of chart. This value can be: line, spline, area, heatmap, treemap, etc.
  • aesthetics is the mapping to use for plot. Note that highcharter will map to color if you use group or value.

You can add more layers to the plot using hc_add_series().

Example 3 Use highcharter to recreate the ggplot line graph shown below showing the most popular female names. You can pipe to hc_title() and hc_subtitle() to add in titles.

PopularFemaleNames <- babynames %>% 
  filter(sex == "F") %>% 
  group_by(year) %>% 
  slice_max(n)
PopularFemaleNames %>% 
  ggplot(aes(x= year, y = n, color = name)) +
  geom_line() +
  #geom_point() +
  labs(title = "Most popular name each year", subtitle = "For babies born in the U.S. and assigned the female sex at birth")

Solution 
PopularFemaleNames %>% 
  hchart("line", hcaes(x= year, y = n, group = name)) %>% 
  hc_title(text = "Most popular name each year") %>% 
  hc_subtitle(text = "For babies born in the U.S. and assigned the female sex at birth")

Using a Screen Reader

Android or Apple phone - navigate to this webpage (typing without a physical keyboard can be quite frustrating ) - set up accessibility shortcut - single touch to read aloud (can also be sent to a Braille display), can swipe through the options on the screen - double tap to select or activate an item - rotor - importance of headings and link text on the screen (some websites links are tagged here)

note on Desktop screen reader- VoiceOver or NVDA, windows built=in?

Adding Accessibility Features to Highcharter Graphs

Screen reader and keyboard navigation. Can also see graph in table form. Can export data as well

Highcharter Scatterplot

show what it looks like with inspect –svg vs img

hchart(
  penguins,
  "scatter",
  hcaes(x = flipper_length_mm, y = bill_length_mm, group = species)) 
hchart(
  penguins,
  "scatter",
  hcaes(x = flipper_length_mm, y = bill_length_mm, group = species)) %>% 
  hc_add_dependency(name = "modules/accessibility.js") %>%
  hc_add_dependency(name = "modules/exporting.js") %>%
  hc_add_dependency(name = "modules/export-data.js")

Highcharter Lineplot

work through an example

Highcharter Barplot

Note about bars vs columns in highcharter

PopularNames <- babynames %>%
  filter(sex == "F") %>%
  group_by(name) %>%
  summarize(total = sum(n)) %>%
  arrange(desc(total)) %>%
  head(10)
  hchart(PopularNames, "column", hcaes(x = name, y = total)) 
  hchart(PopularNames, "column", hcaes(x = name, y = total)) %>% 
  hc_add_dependency(name = "modules/series-label.js") %>%
  hc_add_dependency(name = "modules/accessibility.js") %>%
  hc_add_dependency(name = "modules/exporting.js") %>%
  hc_add_dependency(name = "modules/export-data.js")

Other Highcharter Graphics

link to other ones, can do almost anything that Highcharts library can.

There is a lot more documentation about Highcharts. This is a good guide on how to convert from Highcharts (JavaScript) to Highcharter (R) notation.

Publishing on RStudio RPubs

In order to access your files from the phone/iPad, you will actually be publishing your html file to RStudio RPubs

  • After you knit your files, screenshow of the publish button
  • Prompted to create an RPubs username/password
  • given the URL that your site is at, you can navigate to it on your iPad (recommend turning off the screen reader to do this)

Practice: Accessible Graphics

Exercise 2 (Accessibility Fun) Create an accessible graph! You can pick a graph that you’ve created in a previous class or create something new. First convert it to a highcharter graph, then figure out how to make it accessible!

Sonification with Highcharter

Sonification of Single Points

PopularNames %>% 
  hchart("point", hcaes(x = name, y = total)) %>%
  hc_plotOptions(
    series = list(
      point = list(
        events = list(
        click = JS("function() {
        this.sonify({
          duration: 3000,
          order: 'sequential',
          pointPlayTime: 'x',
          afterSeriesWait: 1000,
          instruments: [{
              instrument: 'sine',
              instrumentMapping: {
                  volume: 0.8,
                  duration: 250,
                  pan: 'x',
                  frequency: 'y'
              },
              // Start at C5 note, end at C6
              instrumentOptions: {
                  minFrequency: 520,
                  maxFrequency: 1050
              }
          }]
        })
       }"))))) %>%
   hc_add_dependency(name = "modules/sonification.js")

You can play around with the numbers for:

  • duration,
  • afterSeriesWait,
  • order,
  • instruments (you can use ‘sine’, ‘sawtooth’, ‘trangle’,‘sineMajor’, ‘sawtoothMajor’, ‘trangleMajor’ )
  • pan (works with headphones that support it)

Sonification of Series of Points

highchart() |>  
  hc_series(
    list(
      name = "Tokyo",
      data = c(7.0, 6.9, 9.5, 14.5, 18.4, 21.5, 25.2, 26.5, 23.3, 18.3, 13.9, 9.6)
    ),
    list(
      name = "London",
      data = c(3.9, 4.2, 5.7, 8.5, 11.9, 15.2, 17.0, 16.6, 14.2, 10.3, 6.6, 4.8)
    )
  ) %>% 
  #hc_tooltip(formatter = JS("function () { return 'hello world'}"))  
  hc_plotOptions(
    series = list(
      point = list(
        events = list(
        click = JS("function() {
        this.sonify({
          duration: 3000,
          order: 'sequential',
          pointPlayTime: 'x',
          afterSeriesWait: 1000,
          instruments: [{
              instrument: 'squareMajor',
              instrumentMapping: {
                  volume: 0.8,
                  duration: 250,
                  pan: 'x',
                  frequency: 'y'
              },
              // Start at C5 note, end at C6
              instrumentOptions: {
                  minFrequency: 520,
                  maxFrequency: 1050
              }
          }]
        })
       }"))))) %>%
  hc_add_dependency(name = "modules/sonification.js")
highchart() |>  
  hc_series(
    list(
      name = "Tokyo",
      data = c(7.0, 6.9, 9.5, 14.5, 18.4, 21.5, 25.2, 26.5, 23.3, 18.3, 13.9, 9.6)
    ),
    list(
      name = "London",
      data = c(3.9, 4.2, 5.7, 8.5, 11.9, 15.2, 17.0, 16.6, 14.2, 10.3, 6.6, 4.8)
    )
  ) %>% 
  #hc_tooltip(formatter = JS("function () { return 'hello world'}"))  
  hc_plotOptions(
    series = list(
        events = list(
        click = JS("function() {
        this.sonify({
          duration: 3000,
          pointPlayTime: 'x',
          instruments: [{
              instrument: 'squareMajor',
              instrumentMapping: {
                  volume: 0.8,
                  duration: 250,
                  pan: 'x',
                  frequency: 'y'
              },
              // Start at C5 note, end at C6
              instrumentOptions: {
                  minFrequency: 120,
                  maxFrequency: 350
              }
          }]
        })
       }")))) %>%
  hc_add_dependency(name = "modules/sonification.js")
PopularFemaleNames %>% 
  hchart("line", hcaes(x= year, y = n, group = name)) %>% 
  hc_title(text = "Most popular name each year") %>% 
  hc_subtitle(text = "For babies born in the U.S. and assigned the female sex at birth") %>% 
    hc_plotOptions(
    series = list(
        events = list(
        click = JS("function() {
        this.sonify({
          duration: 3000,
          pointPlayTime: 'x',
          instruments: [{
              instrument: 'triangleMajor',
              instrumentMapping: {
                  volume: 0.8,
                  duration: 250,
                  pan: 'x',
                  frequency: 'y'
              },
              // Start at C5 note, end at C6
              instrumentOptions: {
                  minFrequency: 120,
                  maxFrequency: 350
              }
          }]
        })
       }")))) %>%
  hc_add_dependency(name = "modules/sonification.js")

Practice: Sonification

Exercise 3 (Sonification Fun) Create a sonified graph! You can pick a scatter or line plot graph that you’ve created in a previous class (e.g. for ) or create something new and add sonification!

Acknowledgements

Thanks to Mara Averick for her detailed blog posts on incorporating accessibility into highcharter, which inspired this module, mrjoh3 for the github issue that helped me determine which older version of highcharter to use for accessibility, and the team behind Highcharts and Highcharter for adding such awesome and heavily researched accessibility features.

Notes

  • update the numbering
  • make sure the images work when I transfer
  • alt text for images
  • talk about alt text
  • switch to absolute links
  • add appendix
#more line graphs
babynames %>% 
  filter(name %in% c("Lauren"), sex == "F") %>% 
  ggplot(aes(x= year, y = n)) +
  geom_line() +
  labs(title = "Popularity of the name 'Lauren' over the years", subtitle = "For babies born in the U.S. and assigned the female sex at birth",  y = "Count", x = "")

babynames %>% 
  filter(name == "Lauren", sex == "F") %>% 
  hchart("line", hcaes(x= year, y = n)) 
#more bar plots
PopularNames <- babynames %>%
  filter(sex == "F") %>%
  group_by(name) %>%
  summarize(total = sum(n)) %>%
  arrange(desc(total)) %>%
  head(10)

PopularNames %>% 
  ggplot(aes(x = name, y = total)) +
  geom_col()