Introduction to Graphics

There are a number of different packages and functions that allow you to make graphs in R. Each of them has their own pros and cons (balancing flexibility and ease of use).

The key takeaway is that pretty much any graph that you want to make, whether it is simply for exploratory reasons or you want to present a well formatted and nice looking graph in your manuscript, will be possible in R.

Data visualization is unfortunately something that is underused in psychology. It is often easier to demonstrate a pattern using a graph than it is through statistics or descriptive language. Further, it can be easier to find errors in your data or outliers in the dataset.

ggplot2 package

ggplot2 is a package within the tidyverse collection, so if you loaded the tidyverse, you don’t need to load ggplot. If not, let’s do this now:

library(tidyverse)
# Let's also load the psych package, we have just a couple of psych functions
library(psych)

ggplot2 is also a system for creating graphics, based on The Grammar of Graphics (gg). You provide the data, tell ggplot2 how to map variables to aesthetics, what graphical primitives to use (geoms), and it takes care of the details. ggplot2 relies on a layer system of

Basic layers:

  1. Data
  2. Aesthetics (mapping the data onto the plot)
  3. Geometrics (geoms: points, lines, bars, …)

And, we can add more:

  1. Facets
  2. Statistics
  3. Coordinate
  4. Themes etc.

ggplot is an incredibly flexible function that allows you to make pretty much any graph you’d ever imagined.

What is different from ggplot vs. other functions though is that the syntax used to generate the plots follows a slightly different logic.

The “gg” before plot stands for Grammar of Graphics so it uses a system where you can add layers to your graph by adding them to a general type of graph.

ggplot2 is also part of the tidyverse set of packages so it works very well with the other data manipulation functions in a pipeline.

Here is how you can think about “layering” a plot with ggplot:

Step 1

Provide the name of the dataset and make it known that you want to create a plot with ggplot

All plots (regardless of what they are) start with:

ggplot(data = name_of_dataset)

For example:

ggplot(data = mtcars)  # this will not give you much, because we are missing the aesthetics and geoms! We should have at least the 3 basics layers!

Step 2

Tell the function which variables will be involved from your dataset.

ggplot(data = name_of_data, aes(x = X_var, y = Y_var))

For example:

ggplot(data = mtcars, aes(x = wt, y = mpg))

Look above…the mapping is there, but we’re missing the geoms! That’s why we can’t see any actual data, because R doesn’t know how we want to visualize it (e.g., lines, dots, triangles, etc…)

The x variable will be presented on the x axis of the graph (horizontal axis) and the y variable will be presented on the y axis (vertical axis)

Step 3

Tell the function about the characteristics for the plot you want to make

Here’s what the code would look like to make a scatterplot (i.e., using points/circles to represent the data):

ggplot(data = name_of_data, aes(x = X_var, y = Y_var)) + geom_point()

For example:

ggplot(data = mtcars, aes(x = wt, y = mpg)) + geom_point()

Step 4 (Optional)

Add additional features to change the appearance of your graph

You can add in features to change colours, point size or shape, change the scale of axes, add titles, etc.

E.g.,

ggplot(data = name_of_data, aes(x = X_var, y = Y_var)) + geom_point(size = , shape = ) +
    ggtitle("My graph") + theme_bw()

For example:

ggplot(data = mtcars, aes(x = wt, y = mpg)) + geom_point(size = 5, shape = 1, colour = "light blue") +
    ggtitle("relationship between Miles per Gallon and Car Weight") + theme_dark()

# Try to play around with the shape, size, and colours!

This code adds a title to the top of the graph and changes the appearance to a black background graph (but you can change the colour of points or lines if you want). There are many different ways that you can customize your graph and change its appearance.

If you’re interested in learning more about plotting in R, I would recommend checking out the free online textbook R Graphics Cookbook at https://r-graphics.org/index.html. There are chapters that show how to make common graphs and a chapter devoted to customizing the appearance of ggplot graphs (Chapter 9).

Chapter 3 in the R for Data Science book (https://r4ds.had.co.nz/data-visualisation.html) is also a good resource for getting started with graphics using the ggplot2 package.

Common Graphs for Psychological Data

To demonstrate the different graphs let’s use a fun dataset that is available in R: msleep

This dataset provides information about the sleep habits of various mammals!

# msleep msleep$name

Histograms

Histograms are useful for examining univariate distributions (i.e., the distribution of a single variable). These can be used descriptively or for diagnostic purposes to examine extreme univariate scores.

Here’s the most basic histogram of the distribution of depression at baseline using a few different functions:

# base R - no package loading required
hist(msleep$sleep_total)

multi.hist(msleep$sleep_total)

# ggplot2 (loaded as part of the tidyverse)
ggplot(data = msleep, aes(x = sleep_total)) + geom_histogram(bins = 10, fill = "purple",
    colour = "black") + theme_bw()

You may have noticed that these plots all looked slightly different despite the same data being plotted. One thing that you may want to change is the number of bars that are shown in a histogram. These are called breaks or bins. Less breaks make chunkier bars and more breaks make narrower bars.

You can also add titles, change the colour of the bars, add names to the x and y axes, etc. You can always examine help files to find out how much more can be customized or check out the resources described above.

# examples with too small a number of breaks

hist(msleep$sleep_total, breaks = 4, main = "Histogram with few breaks")

hist(msleep$sleep_total, breaks = 40, main = "Histogram with many breaks")

Boxplots

Boxplots are another graphical technique to examine a univariate distribution. If no outliers: the thick line in the middle is the median, and the upper and lower parts of the box = the 25th and 75th percentiles with the “whiskers” being the minimum and maximum. With outliers, the box is interpreted the same but the whiskers extend to 1.5 * the interquartile range and anything outside of that is a dot or asterisk, representing an outlier.

# ggplot
ggplot(data = msleep, aes(y = sleep_total)) + geom_boxplot()

Some customization examples:

ggplot(data = msleep, aes(y = sleep_total)) + geom_boxplot(colour = "midnightblue",
    fill = "light yellow") + labs(y = "Total Daily Hours of Sleep") + theme_light()

You can also examine boxplots by a grouping variable to compare distribution shapes

# notice that ggplot provides the NAs as a category too (which we may or may not want)
ggplot(data=msleep, aes(x=vore, y=sleep_total))+
geom_boxplot() 

# to remove NAs we need to filter first. ggplot is part of tidyverse so it works with dplyr functions. This means we can use the pipeline (notice we have to remove the data= argument in ggplot though)

msleep %>%
    filter(is.na(vore)==FALSE) %>% # removing the NA "category" in vore
    ggplot(aes(x=vore, y=sleep_total))+
    geom_boxplot() 

# sideways boxplot
ggplot(data=msleep, aes(x=vore, y=sleep_total))+
geom_boxplot() + coord_flip()

Scatterplots

Scatterplots are probably the most useful graph in psychology because we tend to deal with multivariate data or want to examine the pattern of how two variables vary together. Any correlation or regression based analyses would benefit from examining scatterplots (for effects of interest and model diagnostics). We will be using these a fair amount in Stat II.

In our dataset we can investigate the relationship between the animal’s total sleep time and its REM sleep.

ggplot(data = msleep, aes(x = sleep_rem, y = sleep_total)) + geom_point()

Customization:

# Colours reflect different group
ggplot(data = msleep, aes(x = sleep_rem, y = sleep_total, colour = vore)) + geom_point(size = 3) +
    ggtitle("Total Sleep and REM by Vore")

# Shape = different group
ggplot(data = msleep, aes(x = sleep_rem, y = sleep_total, shape = vore)) + geom_point()

# getting a nonparametric curve for the full dataset
ggplot(data = msleep, aes(x = sleep_rem, y = sleep_total)) + geom_point(aes(colour = vore)) +
    geom_smooth(se = FALSE)  # se=TRUE gives confidence bands

# if you left colour in the first line of the code it would give smoothed lines
# for each group, e.g.,
ggplot(data = msleep, aes(x = sleep_rem, y = sleep_total, colour = vore)) + geom_point() +
    geom_smooth(se = FALSE)

# separate mini plots per group
ggplot(data = msleep, aes(x = sleep_rem, y = sleep_total, colour = vore)) + geom_point() +
    facet_wrap(~vore)

See: https://r-graphics.org/chapter-scatter

Line graphs

Line graphs are commonly used when you want to see how things change over conditions or longitudinally

For line graphs, we will use the ToothGrowth dataset we were working with.

tg <- ToothGrowth

Here is a case where having the data in long format is helpful because the graphs will want a single variable denoting time or condition for use in the plot

tg$dose <- as.factor(tg$dose)
# Combining dplyr data wrangling with ggplot2
tg %>%
    group_by(dose) %>%
    summarise(mean_len = mean(len)) %>%
    ggplot(aes(x = dose, y = mean_len, group = 1)) + geom_line() + geom_point()

# with points added and different colour lines by supp

tg %>%
    group_by(dose, supp) %>%
    summarise(mean_len = mean(len)) %>%
    ggplot(aes(x = dose, y = mean_len, group = supp)) + geom_line(aes(color = supp)) +
    geom_point(aes(color = supp))

# with points added and different colour lines by supp and different plots
tg %>%
    group_by(dose, supp) %>%
    summarise(mean_len = mean(len)) %>%
    ggplot(aes(x = dose, y = mean_len, group = supp)) + geom_line(aes(color = supp)) +
    geom_point(aes(color = supp)) + facet_wrap(~supp)

Next: Interactive data visualizations using plotly

promotations <- read.csv("https://raw.githubusercontent.com/udialter/intro-to-plotly/main/promotation%20data%20clean.csv")

library(plotly)
head(promotations)
  X year promoted     sex
1 1 1998      147 Females
2 2 1999      146 Females
3 3 2000      250 Females
4 4 2001      484 Females
5 5 2002      476 Females
6 6 2003      426 Females
p <- ggplot(data = msleep, aes(x = sleep_rem, y = sleep_total, colour = vore)) +
    geom_point(size = 3) + ggtitle("Total Sleep and REM by Vore")

p

ggplotly(p)
# first, creating a ggplot2 object. If issues arise with the font roboto or and
# error message mentioning polygon (usually in macs), you can # or delete the
# hrbrthemes::theme_ft_rc()+ line below (line 45)

p <- ggplot(promotations, aes(x = year, y = promoted, group = sex)) + geom_line(aes(color = sex),
    size = 1.5) + geom_point(aes(color = sex), size = 3) + theme(axis.text.x = element_text(angle = 45,
    vjust = 1, hjust = 1))

p <- p + scale_color_brewer(palette = "Set2") + ggtitle("Canadian Armed Forces Officer Promotions by Sex") +
    labs(x = "Year", y = "Number of personnel promoted", size = 3)
4
[1] 4
ggsave("CAF.png", plot = p, dpi = 700)

#' # ggplotly
(intplt <- ggplotly(p, tooltip = c("x", "y", "group")))
#' ## saving as html object
saveWidget(intplt, "ggplotlyeg.html", selfcontained = F, libdir = "lib/")
LS0tCnRpdGxlOiAiZ2dwbG90MiIKYXV0aG9yOiAiVWRpIEFsdGVyIgpkYXRlOiAiYHIgU3lzLkRhdGUoKWAiCm91dHB1dDogCiAgcm1kZm9ybWF0czo6bWF0ZXJpYWw6CiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICBoaWdobGlnaHQ6IGthdGUKLS0tCgpgYGB7ciBzZXR1cCwgZWNobz1GQUxTRSwgY2FjaGU9RkFMU0UsIHdhcm5pbmc9RiwgbWVzc2FnZT1GfQpsaWJyYXJ5KGtuaXRyKQpsaWJyYXJ5KHJtZGZvcm1hdHMpCmxpYnJhcnkoYnNsaWIpCmxpYnJhcnkodGlkeXZlcnNlKQpsaWJyYXJ5KHJlYWR4bCkKbGlicmFyeSh2aXJpZGlzKQpsaWJyYXJ5KGhyYnJ0aGVtZXMpCmxpYnJhcnkoaHRtbHdpZGdldHMpCmxpYnJhcnkoUkNvbG9yQnJld2VyKQpsaWJyYXJ5KHJlYWRyKQpsaWJyYXJ5KGV4dHJhZm9udCkKbGlicmFyeShwbG90bHkpCgojIEdsb2JhbCBvcHRpb25zCm9wdGlvbnMobWF4LnByaW50PTEwMCkKb3B0c19jaHVuayRzZXQob3V0LndpZHRoID0gIjcwJSIsCiAgICAgICAgICAgICAgIGVjaG89VFJVRSwKCSAgICAgICAgICAgICBjYWNoZT1UUlVFLAogICAgICAgICAgICAgICBwcm9tcHQ9RkFMU0UsCiAgICAgICAgICAgICAgIHRpZHk9VFJVRSwKICAgICAgICAgICAgICAgY29tbWVudD1OQSwKICAgICAgICAgICAgICAgbWVzc2FnZT1GQUxTRSwKICAgICAgICAgICAgICAgd2FybmluZz1GQUxTRSkKb3B0c19rbml0JHNldCh3aWR0aD03NSkKYGBgCgojIEludHJvZHVjdGlvbiB0byBHcmFwaGljcwoKVGhlcmUgYXJlIGEgbnVtYmVyIG9mIGRpZmZlcmVudCBwYWNrYWdlcyBhbmQgZnVuY3Rpb25zIHRoYXQgYWxsb3cgeW91IHRvIG1ha2UgZ3JhcGhzIGluIFIuIEVhY2ggb2YgdGhlbSBoYXMgdGhlaXIgb3duIHByb3MgYW5kIGNvbnMgKGJhbGFuY2luZyBmbGV4aWJpbGl0eSBhbmQgZWFzZSBvZiB1c2UpLiAKClRoZSBrZXkgdGFrZWF3YXkgaXMgdGhhdCBwcmV0dHkgbXVjaCBhbnkgZ3JhcGggdGhhdCB5b3Ugd2FudCB0byBtYWtlLCB3aGV0aGVyIGl0IGlzIHNpbXBseSBmb3IgZXhwbG9yYXRvcnkgcmVhc29ucyBvciB5b3Ugd2FudCB0byBwcmVzZW50IGEgd2VsbCBmb3JtYXR0ZWQgYW5kIG5pY2UgbG9va2luZyBncmFwaCBpbiB5b3VyIG1hbnVzY3JpcHQsIHdpbGwgYmUgcG9zc2libGUgaW4gUi4gCgpEYXRhIHZpc3VhbGl6YXRpb24gaXMgdW5mb3J0dW5hdGVseSBzb21ldGhpbmcgdGhhdCBpcyB1bmRlcnVzZWQgaW4gcHN5Y2hvbG9neS4gSXQgaXMgb2Z0ZW4gZWFzaWVyIHRvIGRlbW9uc3RyYXRlIGEgcGF0dGVybiB1c2luZyBhIGdyYXBoIHRoYW4gaXQgaXMgdGhyb3VnaCBzdGF0aXN0aWNzIG9yIGRlc2NyaXB0aXZlIGxhbmd1YWdlLiBGdXJ0aGVyLCBpdCBjYW4gYmUgZWFzaWVyIHRvIGZpbmQgZXJyb3JzIGluIHlvdXIgZGF0YSBvciBvdXRsaWVycyBpbiB0aGUgZGF0YXNldC4KCiMgZ2dwbG90MiBwYWNrYWdlCgpgZ2dwbG90MmAgaXMgYSBwYWNrYWdlIHdpdGhpbiB0aGUgYHRpZHl2ZXJzZWAgY29sbGVjdGlvbiwgc28gaWYgeW91IGxvYWRlZCB0aGUgYHRpZHl2ZXJzZWAsIHlvdSBkb24ndCBuZWVkIHRvIGxvYWQgYGdncGxvdGAuIElmIG5vdCwgbGV0J3MgZG8gdGhpcyBub3c6CgpgYGB7ciB0aWR5dmVyc2V9CmxpYnJhcnkodGlkeXZlcnNlKQojIExldCdzIGFsc28gbG9hZCB0aGUgcHN5Y2ggcGFja2FnZSwgd2UgaGF2ZSBqdXN0IGEgY291cGxlIG9mIHBzeWNoIGZ1bmN0aW9ucwpsaWJyYXJ5KHBzeWNoKQpgYGAKCmBnZ3Bsb3QyYCBpcyBhbHNvIGEgc3lzdGVtIGZvciBjcmVhdGluZyBncmFwaGljcywgYmFzZWQgb24gVGhlIEdyYW1tYXIgb2YgR3JhcGhpY3MgKGdnKS4gWW91IHByb3ZpZGUgdGhlIGRhdGEsIHRlbGwgYGdncGxvdDJgIGhvdyB0byBtYXAgdmFyaWFibGVzIHRvIGFlc3RoZXRpY3MsIHdoYXQgZ3JhcGhpY2FsIHByaW1pdGl2ZXMgdG8gdXNlIChnZW9tcyksIGFuZCBpdCB0YWtlcyBjYXJlIG9mIHRoZSBkZXRhaWxzLiBgZ2dwbG90MmAgcmVsaWVzIG9uIGEgbGF5ZXIgc3lzdGVtIG9mIAoKQmFzaWMgbGF5ZXJzOgoKMS4gRGF0YQoyLiBBZXN0aGV0aWNzIChtYXBwaW5nIHRoZSBkYXRhIG9udG8gdGhlIHBsb3QpCjMuIEdlb21ldHJpY3MgKGdlb21zOiBwb2ludHMsIGxpbmVzLCBiYXJzLCDigKYpCgpBbmQsIHdlIGNhbiBhZGQgbW9yZToKCjQuIEZhY2V0cwo1LiBTdGF0aXN0aWNzCjYuIENvb3JkaW5hdGUgCjcuIFRoZW1lcwpldGMuCgoKYGdncGxvdGAgaXMgYW4gaW5jcmVkaWJseSBmbGV4aWJsZSBmdW5jdGlvbiB0aGF0IGFsbG93cyB5b3UgdG8gbWFrZSBwcmV0dHkgbXVjaCBhbnkgZ3JhcGggeW91J2QgZXZlciBpbWFnaW5lZC4gCgpXaGF0IGlzIGRpZmZlcmVudCBmcm9tIGdncGxvdCB2cy4gb3RoZXIgZnVuY3Rpb25zIHRob3VnaCBpcyB0aGF0IHRoZSBzeW50YXggdXNlZCB0byBnZW5lcmF0ZSB0aGUgcGxvdHMgZm9sbG93cyBhIHNsaWdodGx5IGRpZmZlcmVudCBsb2dpYy4gCgpUaGUgImdnIiBiZWZvcmUgcGxvdCBzdGFuZHMgZm9yIEdyYW1tYXIgb2YgR3JhcGhpY3Mgc28gaXQgdXNlcyBhIHN5c3RlbSB3aGVyZSB5b3UgY2FuIGFkZCBsYXllcnMgdG8geW91ciBncmFwaCBieSBhZGRpbmcgdGhlbSB0byBhIGdlbmVyYWwgdHlwZSBvZiBncmFwaC4gCgpnZ3Bsb3QyIGlzIGFsc28gcGFydCBvZiB0aGUgdGlkeXZlcnNlIHNldCBvZiBwYWNrYWdlcyBzbyBpdCB3b3JrcyB2ZXJ5IHdlbGwgd2l0aCB0aGUgb3RoZXIgZGF0YSBtYW5pcHVsYXRpb24gZnVuY3Rpb25zIGluIGEgcGlwZWxpbmUuCgpIZXJlIGlzIGhvdyB5b3UgY2FuIHRoaW5rIGFib3V0ICJsYXllcmluZyIgYSBwbG90IHdpdGggZ2dwbG90OgoKIyMgU3RlcCAxCgoqKlByb3ZpZGUgdGhlIG5hbWUgb2YgdGhlIGRhdGFzZXQgYW5kIG1ha2UgaXQga25vd24gdGhhdCB5b3Ugd2FudCB0byBjcmVhdGUgYSBwbG90IHdpdGggZ2dwbG90KioKCkFsbCBwbG90cyAocmVnYXJkbGVzcyBvZiB3aGF0IHRoZXkgYXJlKSBzdGFydCB3aXRoOgoKYGBge3IsIGV2YWw9Rn0KZ2dwbG90KGRhdGE9bmFtZV9vZl9kYXRhc2V0KQpgYGAKCkZvciBleGFtcGxlOgoKYGBge3IsIGV2YWw9Rn0KZ2dwbG90KGRhdGE9bXRjYXJzKSAjIHRoaXMgd2lsbCBub3QgZ2l2ZSB5b3UgbXVjaCwgYmVjYXVzZSB3ZSBhcmUgbWlzc2luZyB0aGUgYWVzdGhldGljcyBhbmQgZ2VvbXMhIFdlIHNob3VsZCBoYXZlIGF0IGxlYXN0IHRoZSAzIGJhc2ljcyBsYXllcnMhCmBgYAoKIyMgU3RlcCAyCgoqKlRlbGwgdGhlIGZ1bmN0aW9uIHdoaWNoIHZhcmlhYmxlcyB3aWxsIGJlIGludm9sdmVkIGZyb20geW91ciBkYXRhc2V0LioqCgpgYGB7ciwgZXZhbD1GfQpnZ3Bsb3QoZGF0YT1uYW1lX29mX2RhdGEsIGFlcyh4PVhfdmFyLCB5PVlfdmFyKSkKYGBgCgpGb3IgZXhhbXBsZToKCmBgYHtyIH0KZ2dwbG90KGRhdGE9bXRjYXJzLCBhZXMoeD13dCwgeT1tcGcpKSAKYGBgCgpMb29rIGFib3ZlLi4udGhlIG1hcHBpbmcgaXMgdGhlcmUsIGJ1dCB3ZSdyZSBtaXNzaW5nIHRoZSBnZW9tcyEgVGhhdCdzIHdoeSB3ZSBjYW4ndCBzZWUgYW55IGFjdHVhbCBkYXRhLCBiZWNhdXNlIFIgZG9lc24ndCBrbm93IGhvdyB3ZSB3YW50IHRvIHZpc3VhbGl6ZSBpdCAoZS5nLiwgbGluZXMsIGRvdHMsIHRyaWFuZ2xlcywgZXRjLi4uKQoKVGhlIHggdmFyaWFibGUgd2lsbCBiZSBwcmVzZW50ZWQgb24gdGhlIHggYXhpcyBvZiB0aGUgZ3JhcGggKGhvcml6b250YWwgYXhpcykgYW5kIHRoZSB5IHZhcmlhYmxlIHdpbGwgYmUgcHJlc2VudGVkIG9uIHRoZSB5IGF4aXMgKHZlcnRpY2FsIGF4aXMpCgojIyBTdGVwIDMKCioqVGVsbCB0aGUgZnVuY3Rpb24gYWJvdXQgdGhlIGNoYXJhY3RlcmlzdGljcyBmb3IgdGhlIHBsb3QgeW91IHdhbnQgdG8gbWFrZSoqCgpIZXJlJ3Mgd2hhdCB0aGUgY29kZSB3b3VsZCBsb29rIGxpa2UgdG8gbWFrZSBhIHNjYXR0ZXJwbG90IChpLmUuLCB1c2luZyBwb2ludHMvY2lyY2xlcyB0byByZXByZXNlbnQgdGhlIGRhdGEpOgoKYGBge3IgZXZhbD1GfQpnZ3Bsb3QoZGF0YT1uYW1lX29mX2RhdGEsIGFlcyh4PVhfdmFyLCB5PVlfdmFyKSkrCiAgICBnZW9tX3BvaW50KCkKYGBgCgpGb3IgZXhhbXBsZToKCmBgYHtyIH0KZ2dwbG90KGRhdGE9bXRjYXJzLCBhZXMoeD13dCwgeT1tcGcpKSAgKwogIGdlb21fcG9pbnQoKQpgYGAKCiMjIFN0ZXAgNCAoT3B0aW9uYWwpCgoqKkFkZCBhZGRpdGlvbmFsIGZlYXR1cmVzIHRvIGNoYW5nZSB0aGUgYXBwZWFyYW5jZSBvZiB5b3VyIGdyYXBoKioKCllvdSBjYW4gYWRkIGluIGZlYXR1cmVzIHRvIGNoYW5nZSBjb2xvdXJzLCBwb2ludCBzaXplIG9yIHNoYXBlLCBjaGFuZ2UgdGhlIHNjYWxlIG9mIGF4ZXMsIGFkZCB0aXRsZXMsIGV0Yy4gCgpFLmcuLAoKYGBge3IgZXZhbD1GfQpnZ3Bsb3QoZGF0YT1uYW1lX29mX2RhdGEsIGFlcyh4PVhfdmFyLCB5PVlfdmFyKSkrCiAgICBnZW9tX3BvaW50KHNpemUgPSAsIHNoYXBlID0gKSArCiAgICBnZ3RpdGxlKCJNeSBncmFwaCIpKyB0aGVtZV9idygpCmBgYAoKRm9yIGV4YW1wbGU6CgpgYGB7ciB9CmdncGxvdChkYXRhPW10Y2FycywgYWVzKHg9d3QsIHk9bXBnKSkgICsKICBnZW9tX3BvaW50KHNpemUgPSA1LCBzaGFwZSA9IDEsIGNvbG91cj0gImxpZ2h0IGJsdWUiKSArIAogICAgZ2d0aXRsZSgicmVsYXRpb25zaGlwIGJldHdlZW4gTWlsZXMgcGVyIEdhbGxvbiBhbmQgQ2FyIFdlaWdodCIpKyAKICAgIHRoZW1lX2RhcmsoKQogIAojIFRyeSB0byBwbGF5IGFyb3VuZCB3aXRoIHRoZSBzaGFwZSwgc2l6ZSwgYW5kIGNvbG91cnMhCmBgYAoKVGhpcyBjb2RlIGFkZHMgYSB0aXRsZSB0byB0aGUgdG9wIG9mIHRoZSBncmFwaCBhbmQgY2hhbmdlcyB0aGUgYXBwZWFyYW5jZSB0byBhIGJsYWNrIGJhY2tncm91bmQgZ3JhcGggKGJ1dCB5b3UgY2FuIGNoYW5nZSB0aGUgY29sb3VyIG9mIHBvaW50cyBvciBsaW5lcyBpZiB5b3Ugd2FudCkuIApUaGVyZSBhcmUgbWFueSBkaWZmZXJlbnQgd2F5cyB0aGF0IHlvdSBjYW4gY3VzdG9taXplIHlvdXIgZ3JhcGggYW5kIGNoYW5nZSBpdHMgYXBwZWFyYW5jZS4gCgpJZiB5b3UncmUgaW50ZXJlc3RlZCBpbiBsZWFybmluZyBtb3JlIGFib3V0IHBsb3R0aW5nIGluIFIsIEkgd291bGQgcmVjb21tZW5kIGNoZWNraW5nIG91dCB0aGUgZnJlZSBvbmxpbmUgdGV4dGJvb2sgUiBHcmFwaGljcyBDb29rYm9vayBhdCA8aHR0cHM6Ly9yLWdyYXBoaWNzLm9yZy9pbmRleC5odG1sPi4gVGhlcmUgYXJlIGNoYXB0ZXJzIHRoYXQgc2hvdyBob3cgdG8gbWFrZSBjb21tb24gZ3JhcGhzIGFuZCBhIGNoYXB0ZXIgZGV2b3RlZCB0byBjdXN0b21pemluZyB0aGUgYXBwZWFyYW5jZSBvZiBnZ3Bsb3QgZ3JhcGhzIChDaGFwdGVyIDkpLgoKQ2hhcHRlciAzIGluIHRoZSBSIGZvciBEYXRhIFNjaWVuY2UgYm9vayAoPGh0dHBzOi8vcjRkcy5oYWQuY28ubnovZGF0YS12aXN1YWxpc2F0aW9uLmh0bWw+KSBpcyBhbHNvIGEgZ29vZCByZXNvdXJjZSBmb3IgZ2V0dGluZyBzdGFydGVkIHdpdGggZ3JhcGhpY3MgdXNpbmcgdGhlIGdncGxvdDIgcGFja2FnZS4KCiMjIENvbW1vbiBHcmFwaHMgZm9yIFBzeWNob2xvZ2ljYWwgRGF0YQoKVG8gZGVtb25zdHJhdGUgdGhlIGRpZmZlcmVudCBncmFwaHMgbGV0J3MgdXNlIGEgZnVuIGRhdGFzZXQgdGhhdCBpcyBhdmFpbGFibGUgaW4gUjogbXNsZWVwCgpUaGlzIGRhdGFzZXQgcHJvdmlkZXMgaW5mb3JtYXRpb24gYWJvdXQgdGhlIHNsZWVwIGhhYml0cyBvZiB2YXJpb3VzIG1hbW1hbHMhCgpgYGB7cn0KI21zbGVlcAojbXNsZWVwJG5hbWUKYGBgCgojIEhpc3RvZ3JhbXMKCkhpc3RvZ3JhbXMgYXJlIHVzZWZ1bCBmb3IgZXhhbWluaW5nIHVuaXZhcmlhdGUgZGlzdHJpYnV0aW9ucyAoaS5lLiwgdGhlIGRpc3RyaWJ1dGlvbiBvZiBhIHNpbmdsZSB2YXJpYWJsZSkuIFRoZXNlIGNhbiBiZSB1c2VkIGRlc2NyaXB0aXZlbHkgb3IgZm9yIGRpYWdub3N0aWMgcHVycG9zZXMgdG8gZXhhbWluZSBleHRyZW1lIHVuaXZhcmlhdGUgc2NvcmVzLiAKCkhlcmUncyB0aGUgbW9zdCBiYXNpYyBoaXN0b2dyYW0gb2YgdGhlIGRpc3RyaWJ1dGlvbiBvZiBkZXByZXNzaW9uIGF0IGJhc2VsaW5lIHVzaW5nIGEgZmV3IGRpZmZlcmVudCBmdW5jdGlvbnM6CgpgYGB7ciBoaXN0b2dyYW1zfQoKIyBiYXNlIFIgLSBubyBwYWNrYWdlIGxvYWRpbmcgcmVxdWlyZWQKaGlzdChtc2xlZXAkc2xlZXBfdG90YWwpCgptdWx0aS5oaXN0KG1zbGVlcCRzbGVlcF90b3RhbCkKCiMgZ2dwbG90MiAobG9hZGVkIGFzIHBhcnQgb2YgdGhlIHRpZHl2ZXJzZSkKZ2dwbG90KGRhdGE9bXNsZWVwLCBhZXMoeD1zbGVlcF90b3RhbCkpKwpnZW9tX2hpc3RvZ3JhbShiaW5zID0gMTAsIGZpbGw9InB1cnBsZSIsIGNvbG91cj0iYmxhY2siKSsKIHRoZW1lX2J3KCkKYGBgCgpZb3UgbWF5IGhhdmUgbm90aWNlZCB0aGF0IHRoZXNlIHBsb3RzIGFsbCBsb29rZWQgc2xpZ2h0bHkgZGlmZmVyZW50IGRlc3BpdGUgdGhlIHNhbWUgZGF0YSBiZWluZyBwbG90dGVkLiBPbmUgdGhpbmcgdGhhdCB5b3UgbWF5IHdhbnQgdG8gY2hhbmdlIGlzIHRoZSBudW1iZXIgb2YgYmFycyB0aGF0IGFyZSBzaG93biBpbiBhIGhpc3RvZ3JhbS4gVGhlc2UgYXJlIGNhbGxlZCAqYnJlYWtzKiBvciAqYmlucyouIExlc3MgYnJlYWtzIG1ha2UgY2h1bmtpZXIgYmFycyBhbmQgbW9yZSBicmVha3MgbWFrZSBuYXJyb3dlciBiYXJzLiAKCllvdSBjYW4gYWxzbyBhZGQgdGl0bGVzLCBjaGFuZ2UgdGhlIGNvbG91ciBvZiB0aGUgYmFycywgYWRkIG5hbWVzIHRvIHRoZSB4IGFuZCB5IGF4ZXMsIGV0Yy4gWW91IGNhbiBhbHdheXMgZXhhbWluZSBoZWxwIGZpbGVzIHRvIGZpbmQgb3V0IGhvdyBtdWNoIG1vcmUgY2FuIGJlIGN1c3RvbWl6ZWQgb3IgY2hlY2sgb3V0IHRoZSByZXNvdXJjZXMgZGVzY3JpYmVkIGFib3ZlLgoKYGBge3IgaGlzdH0KCiMgZXhhbXBsZXMgd2l0aCB0b28gc21hbGwgYSBudW1iZXIgb2YgYnJlYWtzCgpoaXN0KG1zbGVlcCRzbGVlcF90b3RhbCwgYnJlYWtzPTQsIG1haW49Ikhpc3RvZ3JhbSB3aXRoIGZldyBicmVha3MiKQpoaXN0KG1zbGVlcCRzbGVlcF90b3RhbCwgYnJlYWtzPTQwLCBtYWluPSJIaXN0b2dyYW0gd2l0aCBtYW55IGJyZWFrcyIpCgpgYGAKCiMgQm94cGxvdHMKCkJveHBsb3RzIGFyZSBhbm90aGVyIGdyYXBoaWNhbCB0ZWNobmlxdWUgdG8gZXhhbWluZSBhIHVuaXZhcmlhdGUgZGlzdHJpYnV0aW9uLiBJZiBubyBvdXRsaWVyczogdGhlIHRoaWNrIGxpbmUgaW4gdGhlIG1pZGRsZSBpcyB0aGUgbWVkaWFuLCBhbmQgdGhlIHVwcGVyIGFuZCBsb3dlciBwYXJ0cyBvZiB0aGUgYm94ID0gdGhlIDI1dGggYW5kIDc1dGggcGVyY2VudGlsZXMgd2l0aCB0aGUgIndoaXNrZXJzIiBiZWluZyB0aGUgbWluaW11bSBhbmQgbWF4aW11bS4gV2l0aCBvdXRsaWVycywgdGhlIGJveCBpcyBpbnRlcnByZXRlZCB0aGUgc2FtZSBidXQgdGhlIHdoaXNrZXJzIGV4dGVuZCB0byAxLjUgKiB0aGUgaW50ZXJxdWFydGlsZSByYW5nZSBhbmQgYW55dGhpbmcgb3V0c2lkZSBvZiB0aGF0IGlzIGEgZG90IG9yIGFzdGVyaXNrLCByZXByZXNlbnRpbmcgYW4gb3V0bGllci4gCgpgYGB7ciBib3hwbG90fQoKIyBnZ3Bsb3QKZ2dwbG90KGRhdGE9bXNsZWVwLCBhZXMoeT1zbGVlcF90b3RhbCkpKwpnZW9tX2JveHBsb3QoKSAKCmBgYAoKU29tZSBjdXN0b21pemF0aW9uIGV4YW1wbGVzOgoKYGBge3IgYm94cGxvdDJ9CgpnZ3Bsb3QoZGF0YT1tc2xlZXAsIGFlcyh5PXNsZWVwX3RvdGFsKSkrCmdlb21fYm94cGxvdChjb2xvdXI9Im1pZG5pZ2h0Ymx1ZSIsIGZpbGw9ICJsaWdodCB5ZWxsb3ciKSArIAogIGxhYnMoeT0iVG90YWwgRGFpbHkgSG91cnMgb2YgU2xlZXAiKSArIAogIHRoZW1lX2xpZ2h0KCkKYGBgCgpZb3UgY2FuIGFsc28gZXhhbWluZSBib3hwbG90cyBieSBhIGdyb3VwaW5nIHZhcmlhYmxlIHRvIGNvbXBhcmUgZGlzdHJpYnV0aW9uIHNoYXBlcwoKYGBge3IgZ3JwX2JveHBsb3R9CgoKIyBub3RpY2UgdGhhdCBnZ3Bsb3QgcHJvdmlkZXMgdGhlIE5BcyBhcyBhIGNhdGVnb3J5IHRvbyAod2hpY2ggd2UgbWF5IG9yIG1heSBub3Qgd2FudCkKZ2dwbG90KGRhdGE9bXNsZWVwLCBhZXMoeD12b3JlLCB5PXNsZWVwX3RvdGFsKSkrCmdlb21fYm94cGxvdCgpIAoKIyB0byByZW1vdmUgTkFzIHdlIG5lZWQgdG8gZmlsdGVyIGZpcnN0LiBnZ3Bsb3QgaXMgcGFydCBvZiB0aWR5dmVyc2Ugc28gaXQgd29ya3Mgd2l0aCBkcGx5ciBmdW5jdGlvbnMuIFRoaXMgbWVhbnMgd2UgY2FuIHVzZSB0aGUgcGlwZWxpbmUgKG5vdGljZSB3ZSBoYXZlIHRvIHJlbW92ZSB0aGUgZGF0YT0gYXJndW1lbnQgaW4gZ2dwbG90IHRob3VnaCkKCm1zbGVlcCAlPiUKICAgIGZpbHRlcihpcy5uYSh2b3JlKT09RkFMU0UpICU+JSAjIHJlbW92aW5nIHRoZSBOQSAiY2F0ZWdvcnkiIGluIHZvcmUKICAgIGdncGxvdChhZXMoeD12b3JlLCB5PXNsZWVwX3RvdGFsKSkrCiAgICBnZW9tX2JveHBsb3QoKSAKCiMgc2lkZXdheXMgYm94cGxvdApnZ3Bsb3QoZGF0YT1tc2xlZXAsIGFlcyh4PXZvcmUsIHk9c2xlZXBfdG90YWwpKSsKZ2VvbV9ib3hwbG90KCkgKyBjb29yZF9mbGlwKCkKYGBgCgojIFNjYXR0ZXJwbG90cwoKU2NhdHRlcnBsb3RzIGFyZSBwcm9iYWJseSB0aGUgbW9zdCB1c2VmdWwgZ3JhcGggaW4gcHN5Y2hvbG9neSBiZWNhdXNlIHdlIHRlbmQgdG8gZGVhbCB3aXRoIG11bHRpdmFyaWF0ZSBkYXRhIG9yIHdhbnQgdG8gZXhhbWluZSB0aGUgcGF0dGVybiBvZiBob3cgdHdvIHZhcmlhYmxlcyB2YXJ5IHRvZ2V0aGVyLiBBbnkgY29ycmVsYXRpb24gb3IgcmVncmVzc2lvbiBiYXNlZCBhbmFseXNlcyB3b3VsZCBiZW5lZml0IGZyb20gZXhhbWluaW5nIHNjYXR0ZXJwbG90cyAoZm9yIGVmZmVjdHMgb2YgaW50ZXJlc3QgYW5kIG1vZGVsIGRpYWdub3N0aWNzKS4gV2Ugd2lsbCBiZSB1c2luZyB0aGVzZSBhIGZhaXIgYW1vdW50IGluIFN0YXQgSUkuIAoKSW4gb3VyIGRhdGFzZXQgd2UgY2FuIGludmVzdGlnYXRlIHRoZSByZWxhdGlvbnNoaXAgYmV0d2VlbiB0aGUgYW5pbWFsJ3MgdG90YWwgc2xlZXAgdGltZSBhbmQgaXRzIFJFTSBzbGVlcC4KCmBgYHtyIHNjYXR0ZXJwbG90c30KCmdncGxvdChkYXRhPW1zbGVlcCwgYWVzKHg9c2xlZXBfcmVtLCB5PXNsZWVwX3RvdGFsKSkrCmdlb21fcG9pbnQoKQoKYGBgCgojIEN1c3RvbWl6YXRpb246CgpgYGB7ciBzY2F0dGVycGxvdDIsIHdhcm5pbmc9RkFMU0V9CiMgQ29sb3VycyByZWZsZWN0IGRpZmZlcmVudCBncm91cApnZ3Bsb3QoZGF0YT1tc2xlZXAsIGFlcyh4PXNsZWVwX3JlbSwgeT1zbGVlcF90b3RhbCwgY29sb3VyPXZvcmUpKSsKZ2VvbV9wb2ludChzaXplPTMpICsgZ2d0aXRsZSgiVG90YWwgU2xlZXAgYW5kIFJFTSBieSBWb3JlIikKCiMgU2hhcGUgPSBkaWZmZXJlbnQgZ3JvdXAKZ2dwbG90KGRhdGE9bXNsZWVwLCBhZXMoeD1zbGVlcF9yZW0sIHk9c2xlZXBfdG90YWwsIHNoYXBlPXZvcmUpKSsKZ2VvbV9wb2ludCgpCgojIGdldHRpbmcgYSBub25wYXJhbWV0cmljIGN1cnZlIGZvciB0aGUgZnVsbCBkYXRhc2V0CiBnZ3Bsb3QoZGF0YT1tc2xlZXAsIGFlcyh4PXNsZWVwX3JlbSwgeT1zbGVlcF90b3RhbCkpKwogICAgICBnZW9tX3BvaW50KGFlcyhjb2xvdXI9dm9yZSkpKyAKICAgICAgZ2VvbV9zbW9vdGgoc2UgPSBGQUxTRSkgIyBzZT1UUlVFIGdpdmVzIGNvbmZpZGVuY2UgYmFuZHMKIAojIGlmIHlvdSBsZWZ0IGNvbG91ciBpbiB0aGUgZmlyc3QgbGluZSBvZiB0aGUgY29kZSBpdCB3b3VsZCBnaXZlIHNtb290aGVkIGxpbmVzIGZvciBlYWNoIGdyb3VwLCBlLmcuLCAKIGdncGxvdChkYXRhPW1zbGVlcCwgYWVzKHg9c2xlZXBfcmVtLCB5PXNsZWVwX3RvdGFsLCBjb2xvdXI9dm9yZSkpKwpnZW9tX3BvaW50KCkrIAogIGdlb21fc21vb3RoKHNlID0gRkFMU0UpCiAKICMgc2VwYXJhdGUgbWluaSBwbG90cyBwZXIgZ3JvdXAKICBnZ3Bsb3QoZGF0YT1tc2xlZXAsIGFlcyh4PXNsZWVwX3JlbSwgeT1zbGVlcF90b3RhbCwgY29sb3VyPXZvcmUpKSsKICAgICAgZ2VvbV9wb2ludCgpICsKICAgICAgZmFjZXRfd3JhcCh+dm9yZSkKCmBgYAoKU2VlOiA8aHR0cHM6Ly9yLWdyYXBoaWNzLm9yZy9jaGFwdGVyLXNjYXR0ZXI+CgojIExpbmUgZ3JhcGhzIAoKTGluZSBncmFwaHMgYXJlIGNvbW1vbmx5IHVzZWQgd2hlbiB5b3Ugd2FudCB0byBzZWUgaG93IHRoaW5ncyBjaGFuZ2Ugb3ZlciBjb25kaXRpb25zIG9yIGxvbmdpdHVkaW5hbGx5CgpGb3IgbGluZSBncmFwaHMsIHdlIHdpbGwgdXNlIHRoZSBUb290aEdyb3d0aCBkYXRhc2V0IHdlIHdlcmUgd29ya2luZyB3aXRoLgoKYGBge3J9CnRnIDwtIFRvb3RoR3Jvd3RoCmBgYAoKSGVyZSBpcyBhIGNhc2Ugd2hlcmUgaGF2aW5nIHRoZSBkYXRhIGluIGxvbmcgZm9ybWF0IGlzIGhlbHBmdWwgYmVjYXVzZSB0aGUgZ3JhcGhzIHdpbGwgd2FudCBhIHNpbmdsZSB2YXJpYWJsZSBkZW5vdGluZyB0aW1lIG9yIGNvbmRpdGlvbiBmb3IgdXNlIGluIHRoZSBwbG90CgpgYGB7ciBsaW5lLCB3YXJuaW5nPUZBTFNFfQoKdGckZG9zZSA8LSBhcy5mYWN0b3IodGckZG9zZSkKIyBDb21iaW5pbmcgZHBseXIgZGF0YSB3cmFuZ2xpbmcgd2l0aCBnZ3Bsb3QyCiAgICB0ZyAlPiUKICAgIGdyb3VwX2J5KGRvc2UpICU+JQogICAgc3VtbWFyaXNlKG1lYW5fbGVuPW1lYW4obGVuKSkgJT4lCiAgICBnZ3Bsb3QoYWVzKHggPSBkb3NlLCB5ID0gbWVhbl9sZW4sIGdyb3VwPTEpKSArIAogICAgZ2VvbV9saW5lKCkgKyBnZW9tX3BvaW50KCkKCiMgd2l0aCBwb2ludHMgYWRkZWQgYW5kIGRpZmZlcmVudCBjb2xvdXIgbGluZXMgYnkgc3VwcAoKIHRnICU+JQogICAgZ3JvdXBfYnkoZG9zZSwgc3VwcCkgJT4lCiAgICBzdW1tYXJpc2UobWVhbl9sZW49bWVhbihsZW4pKSAlPiUKICAgIGdncGxvdChhZXMoeCA9IGRvc2UsIHkgPSBtZWFuX2xlbiwgZ3JvdXA9c3VwcCkpICsgCiAgICBnZW9tX2xpbmUoYWVzKGNvbG9yPXN1cHApKSsKICAgIGdlb21fcG9pbnQoYWVzKGNvbG9yPXN1cHApKQogCiAjIHdpdGggcG9pbnRzIGFkZGVkIGFuZCBkaWZmZXJlbnQgY29sb3VyIGxpbmVzIGJ5IHN1cHAgYW5kIGRpZmZlcmVudCBwbG90cwogdGcgJT4lCiAgICBncm91cF9ieShkb3NlLCBzdXBwKSAlPiUKICAgIHN1bW1hcmlzZShtZWFuX2xlbj1tZWFuKGxlbikpICU+JQogICAgZ2dwbG90KGFlcyh4ID0gZG9zZSwgeSA9IG1lYW5fbGVuLCBncm91cD1zdXBwKSkgKyAKICAgIGdlb21fbGluZShhZXMoY29sb3I9c3VwcCkpKwogICAgZ2VvbV9wb2ludChhZXMoY29sb3I9c3VwcCkpKyAKICAgZmFjZXRfd3JhcCh+c3VwcCkKCgpgYGAKCiMgTmV4dDogSW50ZXJhY3RpdmUgZGF0YSB2aXN1YWxpemF0aW9ucyB1c2luZyBgcGxvdGx5YAoKYGBge3J9Cgpwcm9tb3RhdGlvbnMgPC0gcmVhZC5jc3YoImh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS91ZGlhbHRlci9pbnRyby10by1wbG90bHkvbWFpbi9wcm9tb3RhdGlvbiUyMGRhdGElMjBjbGVhbi5jc3YiKQoKbGlicmFyeShwbG90bHkpCmhlYWQocHJvbW90YXRpb25zKQoKcCA8LSBnZ3Bsb3QoZGF0YT1tc2xlZXAsIGFlcyh4PXNsZWVwX3JlbSwgeT1zbGVlcF90b3RhbCwgY29sb3VyPXZvcmUpKSsKICAgIGdlb21fcG9pbnQoc2l6ZT0zKSArIGdndGl0bGUoIlRvdGFsIFNsZWVwIGFuZCBSRU0gYnkgVm9yZSIpCgpwCmdncGxvdGx5KHApCgoKIyBmaXJzdCwgY3JlYXRpbmcgYSBnZ3Bsb3QyIG9iamVjdC4gSWYgaXNzdWVzIGFyaXNlIHdpdGggdGhlIGZvbnQgcm9ib3RvIG9yIGFuZCBlcnJvciBtZXNzYWdlIG1lbnRpb25pbmcgcG9seWdvbiAodXN1YWxseSBpbiBtYWNzKSwgCiMgeW91IGNhbiAjIG9yIGRlbGV0ZSB0aGUgIGhyYnJ0aGVtZXM6OnRoZW1lX2Z0X3JjKCkrIGxpbmUgYmVsb3cgKGxpbmUgNDUpCgpwPC1nZ3Bsb3QocHJvbW90YXRpb25zLGFlcyh4PXllYXIsIHk9cHJvbW90ZWQsIGdyb3VwPXNleCkpICsKICBnZW9tX2xpbmUoYWVzKGNvbG9yPXNleCksIHNpemU9MS41KSsKICBnZW9tX3BvaW50KGFlcyhjb2xvcj1zZXgpLHNpemUgPSAzKSsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZT00NSx2anVzdCA9IDEsIGhqdXN0PTEpKQoKcCA8LSBwICsgc2NhbGVfY29sb3JfYnJld2VyKHBhbGV0dGU9IlNldDIiKSsKICBnZ3RpdGxlKCJDYW5hZGlhbiBBcm1lZCBGb3JjZXMgT2ZmaWNlciBQcm9tb3Rpb25zIGJ5IFNleCIpKwogIGxhYnMoeCA9ICJZZWFyIiwgeT0iTnVtYmVyIG9mIHBlcnNvbm5lbCBwcm9tb3RlZCIsc2l6ZT0zKQo0Cmdnc2F2ZSgiQ0FGLnBuZyIsIHBsb3QgPSBwLCBkcGkgPSA3MDApCgojJyAjIGdncGxvdGx5CihpbnRwbHQ8LSBnZ3Bsb3RseShwLCB0b29sdGlwID0gYygneCcsInkiLCJncm91cCIpKSkKIycgIyMgc2F2aW5nIGFzIGh0bWwgb2JqZWN0CnNhdmVXaWRnZXQoaW50cGx0LCAiZ2dwbG90bHllZy5odG1sIiwgc2VsZmNvbnRhaW5lZCA9IEYsIGxpYmRpciA9ICJsaWIvIikKYGBgCg==