Introduction

In this lesson we will explore the best practice for presenting proportional data. The presentation of proportional data is very common, and thus warrant a lesson of its own. Proportional data are data that range from 0 to 1. More importantly, proportional data should add up to 1 if they are derived from the same entity. For example, let’s say on a corn ear, there are yellow and white kernels (and no other colors). The proportion of yellow and white kernels should add up to 1. Let’s say if 75% of the kernel are yellow, it implies the proportion of white ones is 25%.

Goals might differ when we are presenting proportional data. Here are two common objectives:

  1. Showing different proportions of an entity add up to 1.
  2. Comparing proportions of different entities.

These two objectives may not be mutually exclusive. We will explore what are the best practice to achieve these objectives.

Packages

library(tidyverse)
library(readxl)
library(RColorBrewer)
library(ggplot2)

What really is a pie chart?

The most common visualization for proportional data is a pie chart. You literally see them all the time. Pie chart is a common type of visualization for proportional data, where proportions add up to 100%. This is achieved by dividing a circle into sectors, and the sectors add up to a full circle. This all sounds fine, but how do we construct a pie chart using ggplot? The short answer is we make a stacked bar chart and wrap it into a circle.

Let’s make a pie chart. Let’s say we have a corn ear that has been open pollinated. As a result, there are purple, yellow, and white kernels on the ear. Let’s say the proportions are 15% purple, 70% yellow, and 15% white.

ear_1 <- data.frame(
  colors = c("purple", "yellow", "white"),
  proportions = c(0.15, 0.7, 0.15)
)

head(ear_1)
example_1_stacked <- ear_1 %>% 
  ggplot(aes(x = "A Corn Ear", y = proportions)) +
  geom_bar(stat = "identity", aes(fill = colors),
           color = "black", width = 0.5) +
  scale_fill_identity() +
  labs(x = NULL) +
  theme_classic()

example_1_stacked

ggsave("../Results/05_stack1.png", width = 2, height = 2.5) 

Note that there are two coloring options in ggplot. There is color, which is the color of the outline. In this case, the outline of the stacks is black. There is also fill, which is the color of the interior. In this case, the interior of the stacks is colored by the color of the corn kernel.

So now we have a stacked bar plot. Technically, we are done. The proportions are presented by the heights of the stacks within the bar. And this is a perfectly fine visualization of our proportional data. But what if we want to make a pie chart? Well, we will use the polar coordinate.

example_1_stacked +
  coord_polar(theta = "y") +
  theme_void()

ggsave("../Results/05_pie1.png", width = 2, height = 2.5) 

There are just two extra lines of code to convert a stack bar into a pie chart. First, coord_polar(theta = "y") wraps the y axis into a circle. Second, theme_void() turns off the x and y axis. Axis are not all that informative in for pie charts anyway. So if you want to make a pie chart, this is what you do.

A final note about pie chart: what are the data represented by? In this case, the data are represented by the arc length. So a pie chart is a length based visualization. Due the properties of a circle, the data are also presented by the arc angle and sector area.

What is the best visualization for comparing proportional data?

However, oftentimes we don’t want to just present proportions of one entity. Instead, we want to compare the proportions of multiple entities. For this purpose, pie chart is probably not the best option. Pie charts have been criticized, because we are much worse in reading angles, areas, and lengths of arcs than reading lengths of straight lines.

The best way to visualize proportions from multiple entities is stacked bars. Here is an example. Just make a stacked bar chart

In this example, we have two groups, each contains 4 sub-category. In classic pie charts, the angles (and thus arc lengths & sector area) represent the data. The problem is that it is very difficult to compare between entities. We can visually simplify the pie chart into donut charts, where the data are now represented by arc lengths. However, if we want to use lengths to represent the data, why don’t we just unwrap the donut and make stacked bars? In stacked bar graphs, bars are shown side-by-side and thus easier to compare across entities.

Let’s try an example on our own. Let’s say we have another open pollinated corn ear. This ear has 20% purple kernels, 75% yellow kernels, and 5% white kernels.

ear_2 <- data.frame(
  colors = c("purple", "yellow", "white"),
  proportions = c(0.20, 0.75, 0.05)
)

ears_1_and_2 <- rbind(
  ear_1 %>% 
    mutate(ear = "ear1"), 
  ear_2 %>% 
    mutate(ear = "ear2")
)

head(ears_1_and_2)

Say now we want to make a graph to compare the proportions of these two ears. The code is in fact quite straight forward.

ears_1_and_2 %>% 
  ggplot(aes(x = ear, y = proportions)) +
  geom_bar(stat = "identity", aes(fill = colors), 
           color = "black", width = 0.5) +
  scale_fill_identity() +
  theme_classic()

ggsave("../Results/05_stack2.png", width = 2, height = 2.5)

That’s it! I would say stacked bars are my go-to visualization for comparing proportions of multiple entities. A key advantage is that side-by-side stacked bars are more space efficient. Imagine you are comparing proportions of hundreds of entities. It is unrealistic to present hundreds of pie or donut charts, let alone comparing across them. But stacked bars make the task easy.

Donut charts?

Donut charts are good alternatives of pie charts for presenting proportions of a small number of entities. We won’t cover how to make donut charts here. If you are interested, you can explore on your own. But what you should never do is making concentric donuts. Here is an example. concentric donuts For concentric donuts, you might be tempted to say the data are represented by the arc lengths, which is in fact inaccurate. The arc lengths on the outer rings are much longer than those in the inner rings. Group 2 and Group 3 have the same exact values, but the arc lengths of Group 3 are much longer. In fact the data are represented by the arc angles, which we are bad at reading.

Since outer rings are longer, the ordering of the groups (which group goes to which ring) has a big impact on the impression of the plot. It can lead to the apparent paradox where larger values have shorter arcs. The better (and simpler!) alternative is just unwrap the donuts and make a good old stacked bar plot.

Should you use log scale when preseneting proportional data?

This might be something you have not thought about. Let’s look an example. In this hypothetical example, we quantified biodiversity of a rain forest relative to year 1960. The data are normalized to the value of year 1960.

example_2 <- data.frame(
 year = c(1960, 1970, 1980, 1990, 2000, 2010, 2020),
 relative_biodiversity = c(1, 0.6, 0.3, 0.2, 0.15, 0.1, 0.01)
)

head(example_2)
example_2 %>% 
  ggplot(aes(x = year, y = relative_biodiversity)) +
  geom_point(size = 3) +
  labs(y = "biodiversity\n(relative to 1960)") +
  theme_classic()

ggsave("../Results/05_dot1.png", width = 3, height = 2.5)

Looking at this graph, you would probably say the loss of biodiversity has stabilized in the last 4 decades. But is that so?

A related concept of proportion is odds. Odds = proportion / (1 - proportion). For example, if p = 0.5, then the odds is 1:1. If p = 0.1, then the odds is 1:9. If p = 0.01, then then odds is 1:99.
A property of proportions is that is bound between 0 and 1. Thus, any changes near 0 and 1 will appear small by definition. However, if you think about it, going from 0.1 to 0.01 is 10 fold change in proportion and 11 fold change in odds, We can capture the relative changes using the log scale. We can use the log10 or nature log scale. It doesn’t matter.

example_2 %>% 
  mutate(log_odds = log(relative_biodiversity/(1-relative_biodiversity))) %>% 
  ggplot(aes(x = year, y = log_odds)) +
  geom_point(size = 3) +
  labs(y = "biodiversity\n(relative to 1960\nlog odds scale)") +
  theme_classic()

ggsave("../Results/05_dot2.png", width = 3, height = 2.5)

In this case, presenting proportional data in the log odds scale paints a different picture. Biodiversity has decreased sharply relative to the previous decade. It makes sense: From 2000 to 2010, the change was 0.15 to 0.1, or 0.1/0.15 = 0.67, or 33% decrease from 2000. However, from 2010 to 2020, the change was 0.1 to 0.01, which is a 90% decrease from 2010. In practice, you will need to think very carefully about if you need to present your proportional data in log scale.

Exercise

As an exercise, let’s visualize this example. We have two groups, each contains 4 categories.

group1 <- data.frame(
  "Type" = c("Type I", "Type II", "Type III", "Type IV"),
  "Percentage" = c(15, 35, 30, 20)
)
group2 <- data.frame(
  "Type" = c("Type I", "Type II", "Type III", "Type IV"),
  "Percentage" = c(10, 25, 35, 30)
)

Use ggsave to save your visualization.

# Create a pie chart for group1
pie(group1$Percentage, labels = group1$Type, main = "Pie Chart for Group 1")

NA
NA
# Create a pie chart for group2
pie(group2$Percentage, labels = group2$Type, main = "Pie Chart for Group 2")

pie_group1 <- ggplot(group1, aes(x = "", y = Percentage, fill = Type)) +
  geom_bar(stat = "identity", width = 1) +
  coord_polar("y", start = 0) +
  geom_text(aes(label = paste0(round(Percentage * 100), "%")), position = position_stack(vjust = 0.5)) +
  labs(title = "Group 1 Pie Chart")
pie_group2 <- ggplot(group2, aes(x = "", y = Percentage, fill = Type)) +
  geom_bar(stat = "identity", width = 1) +
  coord_polar("y", start = 0) +
  geom_text(aes(label = paste0(round(Percentage * 100), "%")), position = position_stack(vjust = 0.5)) +
  labs(title = "Group 2 Pie Chart")
print(pie_group1)

print(pie_group2)

ggsave("group1_pie_chart.png", pie_group1)
Saving 7 x 7 in image
ggsave("group2_pie_chart.png", pie_group2)

Hint: look in this lesson to see what I did to combine two entities into one data frame while giving each a unique identifier.

LS0tDQp0aXRsZTogIjA1X0ludHJvX3Byb3BvcnRpb25zLlJtZCINCmF1dGhvcjogIkNoZW54aW4gTGkiDQpkYXRlOiAiMjAyMy0wMS0xNCINCm91dHB1dDogaHRtbF9ub3RlYm9vayANCi0tLQ0KDQpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0NCmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSkNCmBgYA0KDQojIEludHJvZHVjdGlvbiANCkluIHRoaXMgbGVzc29uIHdlIHdpbGwgZXhwbG9yZSB0aGUgYmVzdCBwcmFjdGljZSBmb3IgcHJlc2VudGluZyBwcm9wb3J0aW9uYWwgZGF0YS4gDQpUaGUgcHJlc2VudGF0aW9uIG9mIHByb3BvcnRpb25hbCBkYXRhIGlzIHZlcnkgY29tbW9uLCBhbmQgdGh1cyB3YXJyYW50IGEgbGVzc29uIG9mIGl0cyBvd24uIA0KUHJvcG9ydGlvbmFsIGRhdGEgYXJlIGRhdGEgdGhhdCByYW5nZSBmcm9tIDAgdG8gMS4gDQpNb3JlIGltcG9ydGFudGx5LCBwcm9wb3J0aW9uYWwgZGF0YSBzaG91bGQgYWRkIHVwIHRvIDEgaWYgdGhleSBhcmUgZGVyaXZlZCBmcm9tIHRoZSBzYW1lIGVudGl0eS4gDQpGb3IgZXhhbXBsZSwgbGV0J3Mgc2F5IG9uIGEgY29ybiBlYXIsIHRoZXJlIGFyZSB5ZWxsb3cgYW5kIHdoaXRlIGtlcm5lbHMgKGFuZCBubyBvdGhlciBjb2xvcnMpLiANClRoZSBwcm9wb3J0aW9uIG9mIHllbGxvdyBhbmQgd2hpdGUga2VybmVscyBzaG91bGQgYWRkIHVwIHRvIDEuIA0KTGV0J3Mgc2F5IGlmIDc1JSBvZiB0aGUga2VybmVsIGFyZSB5ZWxsb3csIGl0IGltcGxpZXMgdGhlIHByb3BvcnRpb24gb2Ygd2hpdGUgb25lcyBpcyAyNSUuIA0KDQpHb2FscyBtaWdodCBkaWZmZXIgd2hlbiB3ZSBhcmUgcHJlc2VudGluZyBwcm9wb3J0aW9uYWwgZGF0YS4gSGVyZSBhcmUgdHdvIGNvbW1vbiBvYmplY3RpdmVzOg0KDQoxLiBTaG93aW5nIGRpZmZlcmVudCBwcm9wb3J0aW9ucyBvZiBhbiBlbnRpdHkgYWRkIHVwIHRvIDEuIA0KMi4gQ29tcGFyaW5nIHByb3BvcnRpb25zIG9mIGRpZmZlcmVudCBlbnRpdGllcy4gDQoNClRoZXNlIHR3byBvYmplY3RpdmVzIG1heSBub3QgYmUgbXV0dWFsbHkgZXhjbHVzaXZlLiANCldlIHdpbGwgZXhwbG9yZSB3aGF0IGFyZSB0aGUgYmVzdCBwcmFjdGljZSB0byBhY2hpZXZlIHRoZXNlIG9iamVjdGl2ZXMuIA0KDQojIFBhY2thZ2VzIA0KYGBge3J9DQpsaWJyYXJ5KHRpZHl2ZXJzZSkNCmxpYnJhcnkocmVhZHhsKQ0KbGlicmFyeShSQ29sb3JCcmV3ZXIpDQpsaWJyYXJ5KGdncGxvdDIpDQpgYGANCiANCiMgV2hhdCByZWFsbHkgaXMgYSBwaWUgY2hhcnQ/IA0KVGhlIG1vc3QgY29tbW9uIHZpc3VhbGl6YXRpb24gZm9yIHByb3BvcnRpb25hbCBkYXRhIGlzIGEgcGllIGNoYXJ0LiANCllvdSBsaXRlcmFsbHkgc2VlIHRoZW0gYWxsIHRoZSB0aW1lLiANClBpZSBjaGFydCBpcyBhIGNvbW1vbiB0eXBlIG9mIHZpc3VhbGl6YXRpb24gZm9yIHByb3BvcnRpb25hbCBkYXRhLCB3aGVyZSBwcm9wb3J0aW9ucyBhZGQgdXAgdG8gMTAwJS4NClRoaXMgaXMgYWNoaWV2ZWQgYnkgZGl2aWRpbmcgYSBjaXJjbGUgaW50byBzZWN0b3JzLCBhbmQgdGhlIHNlY3RvcnMgYWRkIHVwIHRvIGEgZnVsbCBjaXJjbGUuDQpUaGlzIGFsbCBzb3VuZHMgZmluZSwgYnV0IGhvdyBkbyB3ZSBjb25zdHJ1Y3QgYSBwaWUgY2hhcnQgdXNpbmcgZ2dwbG90PyANClRoZSBzaG9ydCBhbnN3ZXIgaXMgd2UgbWFrZSBhIHN0YWNrZWQgYmFyIGNoYXJ0IGFuZCB3cmFwIGl0IGludG8gYSBjaXJjbGUuIA0KDQpMZXQncyBtYWtlIGEgcGllIGNoYXJ0LiANCkxldCdzIHNheSB3ZSBoYXZlIGEgY29ybiBlYXIgdGhhdCBoYXMgYmVlbiBvcGVuIHBvbGxpbmF0ZWQuIA0KQXMgYSByZXN1bHQsIHRoZXJlIGFyZSBwdXJwbGUsIHllbGxvdywgYW5kIHdoaXRlIGtlcm5lbHMgb24gdGhlIGVhci4gDQpMZXQncyBzYXkgdGhlIHByb3BvcnRpb25zIGFyZSAxNSUgcHVycGxlLCA3MCUgeWVsbG93LCBhbmQgMTUlIHdoaXRlLiANCg0KYGBge3J9DQplYXJfMSA8LSBkYXRhLmZyYW1lKA0KICBjb2xvcnMgPSBjKCJwdXJwbGUiLCAieWVsbG93IiwgIndoaXRlIiksDQogIHByb3BvcnRpb25zID0gYygwLjE1LCAwLjcsIDAuMTUpDQopDQoNCmhlYWQoZWFyXzEpDQpgYGANCg0KYGBge3J9DQpleGFtcGxlXzFfc3RhY2tlZCA8LSBlYXJfMSAlPiUgDQogIGdncGxvdChhZXMoeCA9ICJBIENvcm4gRWFyIiwgeSA9IHByb3BvcnRpb25zKSkgKw0KICBnZW9tX2JhcihzdGF0ID0gImlkZW50aXR5IiwgYWVzKGZpbGwgPSBjb2xvcnMpLA0KICAgICAgICAgICBjb2xvciA9ICJibGFjayIsIHdpZHRoID0gMC41KSArDQogIHNjYWxlX2ZpbGxfaWRlbnRpdHkoKSArDQogIGxhYnMoeCA9IE5VTEwpICsNCiAgdGhlbWVfY2xhc3NpYygpDQoNCmV4YW1wbGVfMV9zdGFja2VkDQoNCmdnc2F2ZSgiLi4vUmVzdWx0cy8wNV9zdGFjazEucG5nIiwgd2lkdGggPSAyLCBoZWlnaHQgPSAyLjUpIA0KYGBgDQoNCk5vdGUgdGhhdCB0aGVyZSBhcmUgdHdvIGNvbG9yaW5nIG9wdGlvbnMgaW4gYGdncGxvdGAuDQpUaGVyZSBpcyBgY29sb3JgLCB3aGljaCBpcyB0aGUgY29sb3Igb2YgdGhlIG91dGxpbmUuIA0KSW4gdGhpcyBjYXNlLCB0aGUgb3V0bGluZSBvZiB0aGUgc3RhY2tzIGlzIGJsYWNrLiANClRoZXJlIGlzIGFsc28gYGZpbGxgLCB3aGljaCBpcyB0aGUgY29sb3Igb2YgdGhlIGludGVyaW9yLiANCkluIHRoaXMgY2FzZSwgdGhlIGludGVyaW9yIG9mIHRoZSBzdGFja3MgaXMgY29sb3JlZCBieSB0aGUgY29sb3Igb2YgdGhlIGNvcm4ga2VybmVsLiANCg0KU28gbm93IHdlIGhhdmUgYSBzdGFja2VkIGJhciBwbG90LiANClRlY2huaWNhbGx5LCB3ZSBhcmUgZG9uZS4gVGhlIHByb3BvcnRpb25zIGFyZSBwcmVzZW50ZWQgYnkgdGhlIGhlaWdodHMgb2YgdGhlIHN0YWNrcyB3aXRoaW4gdGhlIGJhci4NCkFuZCB0aGlzIGlzIGEgcGVyZmVjdGx5IGZpbmUgdmlzdWFsaXphdGlvbiBvZiBvdXIgcHJvcG9ydGlvbmFsIGRhdGEuIA0KQnV0IHdoYXQgaWYgd2Ugd2FudCB0byBtYWtlIGEgcGllIGNoYXJ0PyANCldlbGwsIHdlIHdpbGwgdXNlIHRoZSBwb2xhciBjb29yZGluYXRlLg0KDQpgYGB7cn0NCmV4YW1wbGVfMV9zdGFja2VkICsNCiAgY29vcmRfcG9sYXIodGhldGEgPSAieSIpICsNCiAgdGhlbWVfdm9pZCgpDQoNCmdnc2F2ZSgiLi4vUmVzdWx0cy8wNV9waWUxLnBuZyIsIHdpZHRoID0gMiwgaGVpZ2h0ID0gMi41KSANCmBgYA0KVGhlcmUgYXJlIGp1c3QgdHdvIGV4dHJhIGxpbmVzIG9mIGNvZGUgdG8gY29udmVydCBhIHN0YWNrIGJhciBpbnRvIGEgcGllIGNoYXJ0Lg0KRmlyc3QsIGBjb29yZF9wb2xhcih0aGV0YSA9ICJ5IilgIHdyYXBzIHRoZSB5IGF4aXMgaW50byBhIGNpcmNsZS4gDQpTZWNvbmQsIGB0aGVtZV92b2lkKClgIHR1cm5zIG9mZiB0aGUgeCBhbmQgeSBheGlzLiANCkF4aXMgYXJlIG5vdCBhbGwgdGhhdCBpbmZvcm1hdGl2ZSBpbiBmb3IgcGllIGNoYXJ0cyBhbnl3YXkuIA0KU28gaWYgeW91IHdhbnQgdG8gbWFrZSBhIHBpZSBjaGFydCwgdGhpcyBpcyB3aGF0IHlvdSBkby4gDQoNCkEgZmluYWwgbm90ZSBhYm91dCBwaWUgY2hhcnQ6IHdoYXQgYXJlIHRoZSBkYXRhIHJlcHJlc2VudGVkIGJ5PyANCkluIHRoaXMgY2FzZSwgdGhlIGRhdGEgYXJlIHJlcHJlc2VudGVkIGJ5IHRoZSBhcmMgbGVuZ3RoLiANClNvIGEgcGllIGNoYXJ0IGlzIGEgbGVuZ3RoIGJhc2VkIHZpc3VhbGl6YXRpb24uIA0KRHVlIHRoZSBwcm9wZXJ0aWVzIG9mIGEgY2lyY2xlLCB0aGUgZGF0YSBhcmUgYWxzbyBwcmVzZW50ZWQgYnkgdGhlIGFyYyBhbmdsZSBhbmQgc2VjdG9yIGFyZWEuICAgDQoNCiMgV2hhdCBpcyB0aGUgYmVzdCB2aXN1YWxpemF0aW9uIGZvciBjb21wYXJpbmcgcHJvcG9ydGlvbmFsIGRhdGE/IA0KSG93ZXZlciwgb2Z0ZW50aW1lcyB3ZSBkb24ndCB3YW50IHRvIGp1c3QgcHJlc2VudCBwcm9wb3J0aW9ucyBvZiBvbmUgZW50aXR5LiANCkluc3RlYWQsIHdlIHdhbnQgdG8gY29tcGFyZSB0aGUgcHJvcG9ydGlvbnMgb2YgbXVsdGlwbGUgZW50aXRpZXMuDQpGb3IgdGhpcyBwdXJwb3NlLCBwaWUgY2hhcnQgaXMgcHJvYmFibHkgbm90IHRoZSBiZXN0IG9wdGlvbi4gDQpQaWUgY2hhcnRzIGhhdmUgYmVlbiBjcml0aWNpemVkLCBiZWNhdXNlIHdlIGFyZSBtdWNoIHdvcnNlIGluIHJlYWRpbmcgYW5nbGVzLCBhcmVhcywgYW5kIGxlbmd0aHMgb2YgYXJjcyB0aGFuIHJlYWRpbmcgbGVuZ3RocyBvZiBzdHJhaWdodCBsaW5lcy4gDQoNClRoZSBiZXN0IHdheSB0byB2aXN1YWxpemUgcHJvcG9ydGlvbnMgZnJvbSBtdWx0aXBsZSBlbnRpdGllcyBpcyBzdGFja2VkIGJhcnMuIA0KSGVyZSBpcyBhbiBleGFtcGxlLiANCiFbSnVzdCBtYWtlIGEgc3RhY2tlZCBiYXIgY2hhcnRdKGh0dHBzOi8vZ2l0aHViLmNvbS9jeGxpMjMzL0ZyaWVuZHNEb250TGV0RnJpZW5kcy9ibG9iL21haW4vUmVzdWx0cy9kb250X3BpZV9jaGFydC5zdmcpICAgDQoNCkluIHRoaXMgZXhhbXBsZSwgd2UgaGF2ZSB0d28gZ3JvdXBzLCBlYWNoIGNvbnRhaW5zIDQgc3ViLWNhdGVnb3J5LiANCkluIGNsYXNzaWMgcGllIGNoYXJ0cywgdGhlIGFuZ2xlcyAoYW5kIHRodXMgYXJjIGxlbmd0aHMgJiBzZWN0b3IgYXJlYSkgcmVwcmVzZW50IHRoZSBkYXRhLiANClRoZSBwcm9ibGVtIGlzIHRoYXQgaXQgaXMgdmVyeSBkaWZmaWN1bHQgdG8gY29tcGFyZSBiZXR3ZWVuIGVudGl0aWVzLiANCldlIGNhbiB2aXN1YWxseSBzaW1wbGlmeSB0aGUgcGllIGNoYXJ0IGludG8gZG9udXQgY2hhcnRzLCB3aGVyZSB0aGUgZGF0YSBhcmUgbm93IHJlcHJlc2VudGVkIGJ5IGFyYyBsZW5ndGhzLiANCkhvd2V2ZXIsIGlmIHdlIHdhbnQgdG8gdXNlIGxlbmd0aHMgdG8gcmVwcmVzZW50IHRoZSBkYXRhLCB3aHkgZG9uJ3Qgd2UganVzdCB1bndyYXAgdGhlIGRvbnV0IGFuZCBtYWtlIHN0YWNrZWQgYmFycz8gDQpJbiBzdGFja2VkIGJhciBncmFwaHMsIGJhcnMgYXJlIHNob3duIHNpZGUtYnktc2lkZSBhbmQgdGh1cyBlYXNpZXIgdG8gY29tcGFyZSBhY3Jvc3MgZW50aXRpZXMuIA0KDQpMZXQncyB0cnkgYW4gZXhhbXBsZSBvbiBvdXIgb3duLiANCkxldCdzIHNheSB3ZSBoYXZlIGFub3RoZXIgb3BlbiBwb2xsaW5hdGVkIGNvcm4gZWFyLiANClRoaXMgZWFyIGhhcyAyMCUgcHVycGxlIGtlcm5lbHMsIDc1JSB5ZWxsb3cga2VybmVscywgYW5kIDUlIHdoaXRlIGtlcm5lbHMuDQpgYGB7cn0NCmVhcl8yIDwtIGRhdGEuZnJhbWUoDQogIGNvbG9ycyA9IGMoInB1cnBsZSIsICJ5ZWxsb3ciLCAid2hpdGUiKSwNCiAgcHJvcG9ydGlvbnMgPSBjKDAuMjAsIDAuNzUsIDAuMDUpDQopDQoNCmVhcnNfMV9hbmRfMiA8LSByYmluZCgNCiAgZWFyXzEgJT4lIA0KICAgIG11dGF0ZShlYXIgPSAiZWFyMSIpLCANCiAgZWFyXzIgJT4lIA0KICAgIG11dGF0ZShlYXIgPSAiZWFyMiIpDQopDQoNCmhlYWQoZWFyc18xX2FuZF8yKQ0KYGBgDQpTYXkgbm93IHdlIHdhbnQgdG8gbWFrZSBhIGdyYXBoIHRvIGNvbXBhcmUgdGhlIHByb3BvcnRpb25zIG9mIHRoZXNlIHR3byBlYXJzLiANClRoZSBjb2RlIGlzIGluIGZhY3QgcXVpdGUgc3RyYWlnaHQgZm9yd2FyZC4gDQoNCmBgYHtyfQ0KZWFyc18xX2FuZF8yICU+JSANCiAgZ2dwbG90KGFlcyh4ID0gZWFyLCB5ID0gcHJvcG9ydGlvbnMpKSArDQogIGdlb21fYmFyKHN0YXQgPSAiaWRlbnRpdHkiLCBhZXMoZmlsbCA9IGNvbG9ycyksIA0KICAgICAgICAgICBjb2xvciA9ICJibGFjayIsIHdpZHRoID0gMC41KSArDQogIHNjYWxlX2ZpbGxfaWRlbnRpdHkoKSArDQogIHRoZW1lX2NsYXNzaWMoKQ0KDQpnZ3NhdmUoIi4uL1Jlc3VsdHMvMDVfc3RhY2syLnBuZyIsIHdpZHRoID0gMiwgaGVpZ2h0ID0gMi41KQ0KYGBgDQpUaGF0J3MgaXQhIA0KSSB3b3VsZCBzYXkgc3RhY2tlZCBiYXJzIGFyZSBteSBnby10byB2aXN1YWxpemF0aW9uIGZvciBjb21wYXJpbmcgcHJvcG9ydGlvbnMgb2YgbXVsdGlwbGUgZW50aXRpZXMuIA0KQSBrZXkgYWR2YW50YWdlIGlzIHRoYXQgc2lkZS1ieS1zaWRlIHN0YWNrZWQgYmFycyBhcmUgbW9yZSBzcGFjZSBlZmZpY2llbnQuIA0KSW1hZ2luZSB5b3UgYXJlIGNvbXBhcmluZyBwcm9wb3J0aW9ucyBvZiBodW5kcmVkcyBvZiBlbnRpdGllcy4gDQpJdCBpcyB1bnJlYWxpc3RpYyB0byBwcmVzZW50IGh1bmRyZWRzIG9mIHBpZSBvciBkb251dCBjaGFydHMsIGxldCBhbG9uZSBjb21wYXJpbmcgYWNyb3NzIHRoZW0uIA0KQnV0IHN0YWNrZWQgYmFycyBtYWtlIHRoZSB0YXNrIGVhc3kuIA0KDQojIERvbnV0IGNoYXJ0cz8gDQpEb251dCBjaGFydHMgYXJlIGdvb2QgYWx0ZXJuYXRpdmVzIG9mIHBpZSBjaGFydHMgZm9yIHByZXNlbnRpbmcgcHJvcG9ydGlvbnMgb2YgYSBzbWFsbCBudW1iZXIgb2YgZW50aXRpZXMuIA0KV2Ugd29uJ3QgY292ZXIgaG93IHRvIG1ha2UgZG9udXQgY2hhcnRzIGhlcmUuIA0KSWYgeW91IGFyZSBpbnRlcmVzdGVkLCB5b3UgY2FuIGV4cGxvcmUgb24geW91ciBvd24uIA0KQnV0IHdoYXQgeW91IHNob3VsZCBfbmV2ZXJfIGRvIGlzIG1ha2luZyBjb25jZW50cmljIGRvbnV0cy4gDQpIZXJlIGlzIGFuIGV4YW1wbGUuIA0KIVtjb25jZW50cmljIGRvbnV0c10oaHR0cHM6Ly9naXRodWIuY29tL2N4bGkyMzMvRnJpZW5kc0RvbnRMZXRGcmllbmRzL2Jsb2IvbWFpbi9SZXN1bHRzL2RvbnRfY29uY2VudHJpY19kb251dHMuc3ZnKSANCkZvciBjb25jZW50cmljIGRvbnV0cywgeW91IG1pZ2h0IGJlIHRlbXB0ZWQgdG8gc2F5IHRoZSBkYXRhIGFyZSByZXByZXNlbnRlZCBieSB0aGUgYXJjIGxlbmd0aHMsIA0Kd2hpY2ggaXMgaW4gZmFjdCBpbmFjY3VyYXRlLiANClRoZSBhcmMgbGVuZ3RocyBvbiB0aGUgb3V0ZXIgcmluZ3MgYXJlIG11Y2ggbG9uZ2VyIHRoYW4gdGhvc2UgaW4gdGhlIGlubmVyIHJpbmdzLiANCkdyb3VwIDIgYW5kIEdyb3VwIDMgaGF2ZSB0aGUgc2FtZSBleGFjdCB2YWx1ZXMsIGJ1dCB0aGUgYXJjIGxlbmd0aHMgb2YgR3JvdXAgMyBhcmUgbXVjaCBsb25nZXIuIA0KSW4gZmFjdCB0aGUgZGF0YSBhcmUgcmVwcmVzZW50ZWQgYnkgdGhlIGFyYyBhbmdsZXMsIHdoaWNoIHdlIGFyZSBiYWQgYXQgcmVhZGluZy4NCg0KU2luY2Ugb3V0ZXIgcmluZ3MgYXJlIGxvbmdlciwgdGhlIG9yZGVyaW5nIG9mIHRoZSBncm91cHMgKHdoaWNoIGdyb3VwIGdvZXMgdG8gd2hpY2ggcmluZykgaGFzIGEgYmlnIGltcGFjdCBvbiB0aGUgaW1wcmVzc2lvbiBvZiB0aGUgcGxvdC4gDQpJdCBjYW4gbGVhZCB0byB0aGUgYXBwYXJlbnQgcGFyYWRveCB3aGVyZSBsYXJnZXIgdmFsdWVzIGhhdmUgc2hvcnRlciBhcmNzLiANClRoZSBiZXR0ZXIgKGFuZCBzaW1wbGVyISkgYWx0ZXJuYXRpdmUgaXMganVzdCB1bndyYXAgdGhlIGRvbnV0cyBhbmQgbWFrZSBhIGdvb2Qgb2xkIHN0YWNrZWQgYmFyIHBsb3QuIA0KDQojIFNob3VsZCB5b3UgdXNlIGxvZyBzY2FsZSB3aGVuIHByZXNlbmV0aW5nIHByb3BvcnRpb25hbCBkYXRhPyANClRoaXMgbWlnaHQgYmUgc29tZXRoaW5nIHlvdSBoYXZlIG5vdCB0aG91Z2h0IGFib3V0LiANCkxldCdzIGxvb2sgYW4gZXhhbXBsZS4gDQpJbiB0aGlzIGh5cG90aGV0aWNhbCBleGFtcGxlLCB3ZSBxdWFudGlmaWVkIGJpb2RpdmVyc2l0eSBvZiBhIHJhaW4gZm9yZXN0IHJlbGF0aXZlIHRvIHllYXIgMTk2MC4gDQpUaGUgZGF0YSBhcmUgbm9ybWFsaXplZCB0byB0aGUgdmFsdWUgb2YgeWVhciAxOTYwLiANCg0KYGBge3J9DQpleGFtcGxlXzIgPC0gZGF0YS5mcmFtZSgNCiB5ZWFyID0gYygxOTYwLCAxOTcwLCAxOTgwLCAxOTkwLCAyMDAwLCAyMDEwLCAyMDIwKSwNCiByZWxhdGl2ZV9iaW9kaXZlcnNpdHkgPSBjKDEsIDAuNiwgMC4zLCAwLjIsIDAuMTUsIDAuMSwgMC4wMSkNCikNCg0KaGVhZChleGFtcGxlXzIpDQpgYGANCg0KYGBge3J9DQpleGFtcGxlXzIgJT4lIA0KICBnZ3Bsb3QoYWVzKHggPSB5ZWFyLCB5ID0gcmVsYXRpdmVfYmlvZGl2ZXJzaXR5KSkgKw0KICBnZW9tX3BvaW50KHNpemUgPSAzKSArDQogIGxhYnMoeSA9ICJiaW9kaXZlcnNpdHlcbihyZWxhdGl2ZSB0byAxOTYwKSIpICsNCiAgdGhlbWVfY2xhc3NpYygpDQoNCmdnc2F2ZSgiLi4vUmVzdWx0cy8wNV9kb3QxLnBuZyIsIHdpZHRoID0gMywgaGVpZ2h0ID0gMi41KQ0KYGBgDQpMb29raW5nIGF0IHRoaXMgZ3JhcGgsIHlvdSB3b3VsZCBwcm9iYWJseSBzYXkgdGhlIGxvc3Mgb2YgYmlvZGl2ZXJzaXR5IGhhcyBzdGFiaWxpemVkIGluIHRoZSBsYXN0IDQgZGVjYWRlcy4NCkJ1dCBpcyB0aGF0IHNvPyANCg0KQSByZWxhdGVkIGNvbmNlcHQgb2YgcHJvcG9ydGlvbiBpcyBvZGRzLiANCk9kZHMgPSBwcm9wb3J0aW9uIC8gICgxIC0gcHJvcG9ydGlvbikuIA0KRm9yIGV4YW1wbGUsIGlmIHAgPSAwLjUsIHRoZW4gdGhlIG9kZHMgaXMgMToxLiANCklmIHAgPSAwLjEsIHRoZW4gdGhlIG9kZHMgaXMgMTo5LiANCklmIHAgPSAwLjAxLCB0aGVuIHRoZW4gb2RkcyBpcyAxOjk5LiAgDQpBIHByb3BlcnR5IG9mIHByb3BvcnRpb25zIGlzIHRoYXQgaXMgYm91bmQgYmV0d2VlbiAwIGFuZCAxLiANClRodXMsIGFueSBjaGFuZ2VzIG5lYXIgMCBhbmQgMSB3aWxsIGFwcGVhciBzbWFsbCBieSBkZWZpbml0aW9uLiANCkhvd2V2ZXIsIGlmIHlvdSB0aGluayBhYm91dCBpdCwgZ29pbmcgZnJvbSAwLjEgdG8gMC4wMSBpcyAxMCBmb2xkIGNoYW5nZSBpbiBwcm9wb3J0aW9uIGFuZCAxMSBmb2xkIGNoYW5nZSBpbiBvZGRzLCANCldlIGNhbiBjYXB0dXJlIHRoZSByZWxhdGl2ZSBjaGFuZ2VzIHVzaW5nIHRoZSBsb2cgc2NhbGUuIA0KV2UgY2FuIHVzZSB0aGUgbG9nMTAgb3IgbmF0dXJlIGxvZyBzY2FsZS4gSXQgZG9lc24ndCBtYXR0ZXIuIA0KDQpgYGB7cn0NCmV4YW1wbGVfMiAlPiUgDQogIG11dGF0ZShsb2dfb2RkcyA9IGxvZyhyZWxhdGl2ZV9iaW9kaXZlcnNpdHkvKDEtcmVsYXRpdmVfYmlvZGl2ZXJzaXR5KSkpICU+JSANCiAgZ2dwbG90KGFlcyh4ID0geWVhciwgeSA9IGxvZ19vZGRzKSkgKw0KICBnZW9tX3BvaW50KHNpemUgPSAzKSArDQogIGxhYnMoeSA9ICJiaW9kaXZlcnNpdHlcbihyZWxhdGl2ZSB0byAxOTYwXG5sb2cgb2RkcyBzY2FsZSkiKSArDQogIHRoZW1lX2NsYXNzaWMoKQ0KDQpnZ3NhdmUoIi4uL1Jlc3VsdHMvMDVfZG90Mi5wbmciLCB3aWR0aCA9IDMsIGhlaWdodCA9IDIuNSkNCmBgYA0KSW4gdGhpcyBjYXNlLCBwcmVzZW50aW5nIHByb3BvcnRpb25hbCBkYXRhIGluIHRoZSBsb2cgb2RkcyBzY2FsZSBwYWludHMgYSBkaWZmZXJlbnQgcGljdHVyZS4gDQpCaW9kaXZlcnNpdHkgaGFzIGRlY3JlYXNlZCBzaGFycGx5IHJlbGF0aXZlIHRvIHRoZSBwcmV2aW91cyBkZWNhZGUuDQpJdCBtYWtlcyBzZW5zZTogDQpGcm9tIDIwMDAgdG8gMjAxMCwgdGhlIGNoYW5nZSB3YXMgMC4xNSB0byAwLjEsIG9yIDAuMS8wLjE1ID0gMC42Nywgb3IgMzMlIGRlY3JlYXNlIGZyb20gMjAwMC4gDQpIb3dldmVyLCBmcm9tIDIwMTAgdG8gMjAyMCwgdGhlIGNoYW5nZSB3YXMgMC4xIHRvIDAuMDEsIHdoaWNoIGlzIGEgOTAlIGRlY3JlYXNlIGZyb20gMjAxMC4gDQpJbiBwcmFjdGljZSwgeW91IHdpbGwgbmVlZCB0byB0aGluayB2ZXJ5IGNhcmVmdWxseSBhYm91dCBpZiB5b3UgbmVlZCB0byBwcmVzZW50IHlvdXIgcHJvcG9ydGlvbmFsIGRhdGEgaW4gbG9nIHNjYWxlLiANCg0KIyBFeGVyY2lzZSANCkFzIGFuIGV4ZXJjaXNlLCBsZXQncyB2aXN1YWxpemUgdGhpcyBleGFtcGxlLiANCldlIGhhdmUgdHdvIGdyb3VwcywgZWFjaCBjb250YWlucyA0IGNhdGVnb3JpZXMuIA0KYGBge3J9DQpncm91cDEgPC0gZGF0YS5mcmFtZSgNCiAgIlR5cGUiID0gYygiVHlwZSBJIiwgIlR5cGUgSUkiLCAiVHlwZSBJSUkiLCAiVHlwZSBJViIpLA0KICAiUGVyY2VudGFnZSIgPSBjKDE1LCAzNSwgMzAsIDIwKQ0KKQ0KZ3JvdXAyIDwtIGRhdGEuZnJhbWUoDQogICJUeXBlIiA9IGMoIlR5cGUgSSIsICJUeXBlIElJIiwgIlR5cGUgSUlJIiwgIlR5cGUgSVYiKSwNCiAgIlBlcmNlbnRhZ2UiID0gYygxMCwgMjUsIDM1LCAzMCkNCikNCmBgYA0KDQpVc2UgYGdnc2F2ZWAgdG8gc2F2ZSB5b3VyIHZpc3VhbGl6YXRpb24uIA0KDQoNCmBgYHtyfQ0KIyBDcmVhdGUgYSBwaWUgY2hhcnQgZm9yIGdyb3VwMQ0KcGllKGdyb3VwMSRQZXJjZW50YWdlLCBsYWJlbHMgPSBncm91cDEkVHlwZSwgbWFpbiA9ICJQaWUgQ2hhcnQgZm9yIEdyb3VwIDEiKQ0KDQoNCmBgYA0KYGBge3J9DQojIENyZWF0ZSBhIHBpZSBjaGFydCBmb3IgZ3JvdXAyDQpwaWUoZ3JvdXAyJFBlcmNlbnRhZ2UsIGxhYmVscyA9IGdyb3VwMiRUeXBlLCBtYWluID0gIlBpZSBDaGFydCBmb3IgR3JvdXAgMiIpDQoNCmBgYA0KDQpgYGB7cn0NCnBpZV9ncm91cDEgPC0gZ2dwbG90KGdyb3VwMSwgYWVzKHggPSAiIiwgeSA9IFBlcmNlbnRhZ2UsIGZpbGwgPSBUeXBlKSkgKw0KICBnZW9tX2JhcihzdGF0ID0gImlkZW50aXR5Iiwgd2lkdGggPSAxKSArDQogIGNvb3JkX3BvbGFyKCJ5Iiwgc3RhcnQgPSAwKSArDQogIGdlb21fdGV4dChhZXMobGFiZWwgPSBwYXN0ZTAocm91bmQoUGVyY2VudGFnZSAqIDEwMCksICIlIikpLCBwb3NpdGlvbiA9IHBvc2l0aW9uX3N0YWNrKHZqdXN0ID0gMC41KSkgKw0KICBsYWJzKHRpdGxlID0gIkdyb3VwIDEgUGllIENoYXJ0IikNCg0KYGBgDQpgYGB7cn0NCnBpZV9ncm91cDIgPC0gZ2dwbG90KGdyb3VwMiwgYWVzKHggPSAiIiwgeSA9IFBlcmNlbnRhZ2UsIGZpbGwgPSBUeXBlKSkgKw0KICBnZW9tX2JhcihzdGF0ID0gImlkZW50aXR5Iiwgd2lkdGggPSAxKSArDQogIGNvb3JkX3BvbGFyKCJ5Iiwgc3RhcnQgPSAwKSArDQogIGdlb21fdGV4dChhZXMobGFiZWwgPSBwYXN0ZTAocm91bmQoUGVyY2VudGFnZSAqIDEwMCksICIlIikpLCBwb3NpdGlvbiA9IHBvc2l0aW9uX3N0YWNrKHZqdXN0ID0gMC41KSkgKw0KICBsYWJzKHRpdGxlID0gIkdyb3VwIDIgUGllIENoYXJ0IikNCg0KYGBgDQpgYGB7cn0NCnByaW50KHBpZV9ncm91cDEpDQpwcmludChwaWVfZ3JvdXAyKQ0KDQpgYGANCmBgYHtyfQ0KZ2dzYXZlKCJncm91cDFfcGllX2NoYXJ0LnBuZyIsIHBpZV9ncm91cDEpDQpnZ3NhdmUoImdyb3VwMl9waWVfY2hhcnQucG5nIiwgcGllX2dyb3VwMikNCg0KYGBgDQoNCg0KSGludDogbG9vayBpbiB0aGlzIGxlc3NvbiB0byBzZWUgd2hhdCBJIGRpZCB0byBjb21iaW5lIHR3byBlbnRpdGllcyBpbnRvIG9uZSBkYXRhIGZyYW1lIHdoaWxlIGdpdmluZyBlYWNoIGEgdW5pcXVlIGlkZW50aWZpZXIuIA==