Processing math: 100%
  • Getting started in R
    • Install and load the packages
  • Step 1: Load the data into R
  • Step 2: Perform the ANOVA test
    • One-way ANOVA
    • Two-way ANOVA
    • Adding interactions between variables
    • Adding a blocking variable
  • Step 3: Find the best-fit model
  • Step 4: Check for homoscedasticity
  • Step 5: Do a post-hoc test
  • Step 6: Plot the results in a graph
    • Find the groupwise differences
    • Make a data frame with the group labels
    • Plot the raw data
    • Add the means and standard errors to the graph
    • Split up the data
    • Make the graph ready for publication
  • Step 7: Report the results

ANOVA is a statistical test for estimating how a quantitative dependent variable changes according to the levels of one or more categorical independent variables. ANOVA tests whether there is a difference in means of the groups at each level of the independent variable.

The null hypothesis (H0) of the ANOVA is no difference in means, and the alternative hypothesis (Ha) is that the means are different from one another.

In this guide, we will walk you through the process of a one-way ANOVA (one independent variable) and a two-way ANOVA (two independent variables).

Our sample dataset contains observations from an imaginary study of the effects of fertilizer type and planting density on crop yield.

One-way ANOVA example
In the one-way ANOVA, we test the effects of 3 types of fertilizer on crop yield.

Two-way ANOVA example
In the two-way ANOVA, we add an additional independent variable: planting density. We test the effects of 3 types of fertilizer and 2 different planting densities on crop yield.

We will also include examples of how to perform and interpret a two-way ANOVA with an interaction term, and an ANOVA with a blocking variable.

Getting started in R

If you haven’t used R before, start by downloading R and R Studio. Once you have both of these programs downloaded, open R Studio and click on File > New File > R Script.

Now you can copy and paste the code from the rest of this example into your script. To run the code, highlight the lines you want to run and click on the Run button on the top right of the text editor (or press ctrl + enter on the keyboard).

Install and load the packages

First, install the packages you will need for the analysis (this only needs to be done once):

install.packages(c("ggplot2","ggpubr","tidyverse","broom", "AICcmodavg"))
ABCDEFGHIJ0123456789
 
 
binary
<chr>
source
<chr>
needs_compilation
<lgl>
doBy4.6.204.6.22FALSE
SparseM1.811.84-2TRUE
pbkrtest0.5.20.5.3FALSE
quantreg5.975.98TRUE
minqa1.2.61.2.8TRUE
nloptr2.0.32.1.1TRUE
corrplot0.920.94FALSE
lme41.1-35.31.1-35.5TRUE
TMB1.9.111.9.14TRUE
RcppArmadillo0.12.8.2.114.0.0-1TRUE
Next
12
Previous
1-10 of 15 rows

Then load these packages into your R environment (do this every time you restart the R program):

library(ggplot2)
library(ggpubr)
library(tidyverse)
library(broom)
library(AICcmodavg)

Step 1: Load the data into R

Note that this data was generated for this example, it’s not from a real experiment.

We will use the same dataset for all of our examples in this walkthrough. The only difference between the different analyses is how many independent variables we include and in what combination we include them.

It is common for factors to be read as quantitative variables when importing a dataset into R. To avoid this, you can use the read.csv() command to read in the data, specifying within the command whether each of the variables should be quantitative (“numeric”) or categorical (“factor”).

Use the following code, replacing the path/to/your/file text with the actual path to your file:

crop.data <- read.csv("C:/Users/email/Downloads/cropdata.csv", header = TRUE, colClasses = c("factor", "factor", "factor", "numeric"))

Before continuing, you can check that the data has read in correctly:

summary(crop.data)
 density block  fertilizer     yield      
 1:48    1:24   1:32       Min.   :175.4  
 2:48    2:24   2:32       1st Qu.:176.5  
         3:24   3:32       Median :177.1  
         4:24              Mean   :177.0  
                           3rd Qu.:177.4  
                           Max.   :179.1

You should see ‘density’, ‘block’, and ‘fertilizer’ listed as categorical variables with the number of observations at each level (i.e. 48 observations at density 1 and 48 observations at density 2).

‘Yield’ should be a quantitative variable with a numeric summary (minimum, median, mean, maximum).

Step 2: Perform the ANOVA test

ANOVA tests whether any of the group means are different from the overall mean of the data by checking the variance of each individual group against the overall variance of the data. If one or more groups falls outside the range of variation predicted by the null hypothesis (all group means are equal), then the test is statistically significant.

We can perform an ANOVA in R using the aov() function. This will calculate the test statistic for ANOVA and determine whether there is significant variation among the groups formed by the levels of the independent variable.

One-way ANOVA

In the one-way ANOVA example, we are modeling crop yield as a function of the type of fertilizer used. First we will use aov() to run the model, then we will use summary() to print the summary of the model.

one.way <- aov(yield ~ fertilizer, data = crop.data)
summary(one.way)
            Df Sum Sq Mean Sq F value Pr(>F)    
fertilizer   2   6.07  3.0340   7.863  7e-04 ***
Residuals   93  35.89  0.3859                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

The model summary first lists the independent variables being tested in the model (in this case we have only one, ‘fertilizer’) and the model residuals (‘Residual’). All of the variation that is not explained by the independent variables is called residual variance.

The rest of the values in the output table describe the independent variable and the residuals:

  • The Df column displays the degrees of freedom for the independent variable (the number of levels in the variable minus 1), and the degrees of freedom for the residuals (the total number of observations minus one and minus the number of levels in the independent variables).
  • The Sum Sq column displays the sum of squares (a.k.a. the total variation between the group means and the overall mean).
  • The Mean Sq column is the mean of the sum of squares, calculated by dividing the sum of squares by the degrees of freedom for each parameter.
  • The F value column is the test statistic from the F test. This is the mean square of each independent variable divided by the mean square of the residuals. The larger the F value, the more likely it is that the variation caused by the independent variable is real and not due to chance.
  • The Pr(>F) column is the p value of the F statistic. This shows how likely it is that the F value calculated from the test would have occurred if the null hypothesis of no difference among group means were true. The p-value of the fertilizer variable is low (p < 0.001), so it appears that the type of fertilizer used has a real impact on the final crop yield.

Two-way ANOVA

In the two-way ANOVA example, we are modeling crop yield as a function of type of fertilizer and planting density. First we use aov() to run the model, then we use summary() to print the summary of the model.

two.way <- aov(yield ~ fertilizer + density, data = crop.data)
summary(two.way)
            Df Sum Sq Mean Sq F value   Pr(>F)    
fertilizer   2  6.068   3.034   9.073 0.000253 ***
density      1  5.122   5.122  15.316 0.000174 ***
Residuals   92 30.765   0.334                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Adding planting density to the model seems to have made the model better: it reduced the residual variance (the residual sum of squares went from 35.89 to 30.765), and both planting density and fertilizer are statistically significant (p-values < 0.001).

Adding interactions between variables

Sometimes you have reason to think that two of your independent variables have an interaction effect rather than an additive effect.

For example, in our crop yield experiment, it is possible that planting density affects the plants’ ability to take up fertilizer. This might influence the effect of fertilizer type in a way that isn’t accounted for in the two-way model.

To test whether two variables have an interaction effect in ANOVA, simply use an asterisk instead of a plus-sign in the model:

interaction <- aov(yield ~ fertilizer*density, data = crop.data)
summary(interaction)
                   Df Sum Sq Mean Sq F value   Pr(>F)    
fertilizer          2  6.068   3.034   9.001 0.000273 ***
density             1  5.122   5.122  15.195 0.000186 ***
fertilizer:density  2  0.428   0.214   0.635 0.532500    
Residuals          90 30.337   0.337                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

In the output table, the ‘fertilizer:density’ variable has a low sum-of-squares value and a high p value, which means there is not much variation that can be explained by the interaction between fertilizer and planting density.

Adding a blocking variable

If you have grouped your experimental treatments in some way, or if you have a confounding variable that might affect the relationship you are interested in testing, you should include that element in the model as a blocking variable. The simplest way to do this is just to add the variable into the model with a ‘+’.

For example, in many crop yield studies, treatments are applied within ‘blocks’ in the field that may differ in soil texture, moisture, sunlight, etc. To control for the effect of differences among planting blocks we add a third term, ‘block’, to our ANOVA.

blocking <- aov(yield ~ fertilizer + density + block, data = crop.data)
summary(blocking)
            Df Sum Sq Mean Sq F value   Pr(>F)    
fertilizer   2  6.068   3.034   9.018 0.000269 ***
density      1  5.122   5.122  15.224 0.000184 ***
block        2  0.486   0.243   0.723 0.488329    
Residuals   90 30.278   0.336                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

The ‘block’ variable has a low sum-of-squares value (0.486) and a high p value (p = 0.48), so it’s probably not adding much information to the model. It also doesn’t change the sum of squares for the two independent variables, which means that it’s not affecting how much variation in the dependent variable they explain.

Step 3: Find the best-fit model

There are now four different ANOVA models to explain the data. How do you decide which one to use? Usually you’ll want to use the ‘best-fit’ model – the model that best explains the variation in the dependent variable.

The Akaike information criterion (AIC) is a good test for model fit. AIC calculates the information value of each model by balancing the variation explained against the number of parameters used.

In AIC model selection, we compare the information value of each model and choose the one with the lowest AIC value (a lower number means more information explained!)

library(AICcmodavg)
model.set <- list(one.way, two.way, interaction, blocking)
model.names <- c("one.way", "two.way", "interaction", "blocking")
aictab(model.set, modnames = model.names)

Model selection based on AICc:

            K   AICc Delta_AICc AICcWt Cum.Wt     LL
two.way     5 173.86       0.00   0.71   0.71 -81.59
blocking    7 176.93       3.08   0.15   0.86 -80.83
interaction 7 177.12       3.26   0.14   1.00 -80.92
one.way     4 186.41      12.56   0.00   1.00 -88.99

The model with the lowest AIC score (listed first in the table) is the best fit for the data.

From these results, it appears that the two.way model is the best fit. The two-way model has the lowest AIC value, and 71% of the AIC weight, which means that it explains 71% of the total variation in the dependent variable that can be explained by the full set of models.

The model with blocking term contains an additional 15% of the AIC weight, but because it is more than 2 delta-AIC worse than the best model, it probably isn’t good enough to include in your results.

Step 4: Check for homoscedasticity

To check whether the model fits the assumption of homoscedasticity, look at the model diagnostic plots in R using the plot() function:

par(mfrow=c(2,2))
plot(two.way)
par(mfrow=c(1,1))

The diagnostic plots show the unexplained variance (residuals) across the range of the observed data.

Each plot gives a specific piece of information about the model fit, but it’s enough to know that the red line representing the mean of the residuals should be horizontal and centered on zero (or on one, in the scale-location plot), meaning that there are no large outliers that would cause research bias in the model.

The normal Q-Q plot plots a regression between the theoretical residuals of a perfectly-homoscedastic model and the actual residuals of your model, so the closer to a slope of 1 this is the better. This Q-Q plot is very close, with only a bit of deviation.

From these diagnostic plots we can say that the model fits the assumption of homoscedasticity.

If your model doesn’t fit the assumption of homoscedasticity, you can try the Kruskall-Wallis test instead.

Step 5: Do a post-hoc test

ANOVA tells us if there are differences among group means, but not what the differences are. To find out which groups are statistically different from one another, you can perform a Tukey’s Honestly Significant Difference (Tukey’s HSD) post-hoc test for pairwise comparisons:

tukey.two.way<-TukeyHSD(two.way)
tukey.two.way
  Tukey multiple comparisons of means
    95% family-wise confidence level

Fit: aov(formula = yield ~ fertilizer + density, data = crop.data)

$fertilizer
         diff         lwr       upr     p adj
2-1 0.1761687 -0.16822506 0.5205625 0.4452958
3-1 0.5991256  0.25473179 0.9435194 0.0002219
3-2 0.4229569  0.07856306 0.7673506 0.0119381

$density
        diff       lwr       upr     p adj
2-1 0.461956 0.2275204 0.6963916 0.0001741

From the post-hoc test results, we see that there are statistically significant differences (p < 0.05) between fertilizer groups 3 and 1 and between fertilizer types 3 and 2, but the difference between fertilizer groups 2 and 1 is not statistically significant. There is also a significant difference between the two different levels of planting density.

Step 6: Plot the results in a graph

When plotting the results of a model, it is important to display:

  • the raw data
  • summary information, usually the mean and standard error of each group being compared
  • letters or symbols above each group being compared to indicate the groupwise differences.

Find the groupwise differences

From the ANOVA test we know that both planting density and fertilizer type are significant variables. To display this information on a graph, we need to show which of the combinations of fertilizer type + planting density are statistically different from one another.

To do this, we can run another ANOVA + TukeyHSD test, this time using the interaction of fertilizer and planting density. We aren’t doing this to find out if the interaction term is significant (we already know it’s not), but rather to find out which group means are statistically different from one another so we can add this information to the graph.

tukey.plot.aov<-aov(yield ~ fertilizer:density, data=crop.data)

Instead of printing the TukeyHSD results in a table, we’ll do it in a graph.

tukey.plot.test<-TukeyHSD(tukey.plot.aov)
plot(tukey.plot.test, las = 1)

The significant groupwise differences are any where the 95% confidence interval doesn’t include zero. This is another way of saying that the p value for these pairwise differences is < 0.05.

From this graph, we can see that the fertilizer + planting density combinations which are significantly different from one another are 3:1-1:1 (read as “fertilizer type three + planting density 1 contrasted with fertilizer type 1 + planting density type 1”), 1:2-1:1, 2:2-1:1, 3:2-1:1, and 3:2-2:1.

We can make three labels for our graph: A (representing 1:1), B (representing all the intermediate combinations), and C (representing 3:2).

Make a data frame with the group labels

Now we need to make an additional data frame so we can add these groupwise differences to our graph.

First, summarize the original data using fertilizer type and planting density as grouping variables.

mean.yield.data <- crop.data %>%
  group_by(fertilizer, density) %>%
  summarise(
      yield = mean(yield)
  )
`summarise()` has grouped output by 'fertilizer'. You can override using the `.groups` argument.

Next, add the group labels as a new variable in the data frame.

mean.yield.data$group <- c("a","b","b","b","b","c")
mean.yield.data
ABCDEFGHIJ0123456789
fertilizer
<fctr>
density
<fctr>
yield
<dbl>
group
<chr>
11176.4396a
12177.0745b
21176.7783b
22177.0881b
31177.1356b
32177.5767c
6 rows

Now we are ready to start making the plot for our report.

Plot the raw data

two.way.plot <- ggplot(crop.data, aes(x = density, y = yield, group=fertilizer)) +
  geom_point(cex = 1.5, pch = 1.0,position = position_jitter(w = 0.1, h = 0))
two.way.plot

The output looks like this:

Add the means and standard errors to the graph

two.way.plot <- two.way.plot +
  stat_summary(fun.data = 'mean_se', geom = 'errorbar', width = 0.2) +
  stat_summary(fun.data = 'mean_se', geom = 'pointrange') +
  geom_point(data=mean.yield.data, aes(x=density, y=yield))

two.way.plot

This is very hard to read, since all of the different groupings for fertilizer type are stacked on top of one another. We will solve this in the next step.

Split up the data

To show which groups are different from one another, use facet_wrap() to split the data up over the three types of fertilizer. To add labels, use geom_text(), and add the group letters from the mean.yield.data dataframe you made earlier.

two.way.plot <- two.way.plot +
  geom_text(data=mean.yield.data, label=mean.yield.data$group, vjust = -8, size = 5) +
  facet_wrap(~ fertilizer)

two.way.plot

Make the graph ready for publication

In this step we will remove the grey background and add axis labels.

two.way.plot <- two.way.plot +
  theme_classic2() +
  labs(title = "Crop yield in response to fertilizer mix and planting density",
      x = "Planting density (1=low density, 2=high density)",
      y = "Yield (bushels per acre)")

two.way.plot

The final version of your graph looks like this:

Step 7: Report the results

In addition to a graph, it’s important to state the results of the ANOVA test. Include:

  • A brief description of the variables you tested
  • The F value, degrees of freedom, and p values for each independent variable
  • What the results mean.

Example: Reporting the results of ANOVA We found a statistically-significant difference in average crop yield by both fertilizer type (F(2)=9.018, p < 0.001) and by planting density (F(1)=15.316, p < 0.001). A Tukey post-hoc test revealed that fertilizer mix 3 resulted in a higher yield on average than fertilizer mix 1 (0.59 bushels/acre), and a higher yield on average than fertilizer mix 2 (0.42 bushels/acre). Planting density was also significant, with planting density 2 resulting in an higher yield on average of 0.46 bushels/acre over planting density 1. A subsequent groupwise comparison showed the strongest yield gains at planting density 2, fertilizer mix 3, suggesting that this mix of treatments was most advantageous for crop growth under our experimental conditions.

LS0tDQp0aXRsZTogIkFOT1ZBIGluIFIiDQpvdXRwdXQ6IA0KICBodG1sX25vdGVib29rOg0KICAgIHRvYzogdHJ1ZQ0KICAgIHRvY19mbG9hdDoNCiAgICAgIGNvbGxhcHNlZDogdHJ1ZQ0KICAgIHRoZW1lOiB1bml0ZWQNCi0tLQ0KQU5PVkEgaXMgYSBzdGF0aXN0aWNhbCB0ZXN0IGZvciBlc3RpbWF0aW5nIGhvdyBhIHF1YW50aXRhdGl2ZSBkZXBlbmRlbnQgdmFyaWFibGUgY2hhbmdlcyBhY2NvcmRpbmcgdG8gdGhlIGxldmVscyBvZiBvbmUgb3IgbW9yZSBjYXRlZ29yaWNhbCBpbmRlcGVuZGVudCB2YXJpYWJsZXMuIEFOT1ZBIHRlc3RzIHdoZXRoZXIgdGhlcmUgaXMgYSBkaWZmZXJlbmNlIGluIG1lYW5zIG9mIHRoZSBncm91cHMgYXQgZWFjaCBsZXZlbCBvZiB0aGUgaW5kZXBlbmRlbnQgdmFyaWFibGUuDQoNClRoZSBudWxsIGh5cG90aGVzaXMgKCRIXzAkKSBvZiB0aGUgQU5PVkEgaXMgbm8gZGlmZmVyZW5jZSBpbiBtZWFucywgYW5kIHRoZSBhbHRlcm5hdGl2ZSBoeXBvdGhlc2lzICgkSF9hJCkgaXMgdGhhdCB0aGUgbWVhbnMgYXJlIGRpZmZlcmVudCBmcm9tIG9uZSBhbm90aGVyLg0KDQpJbiB0aGlzIGd1aWRlLCB3ZSB3aWxsIHdhbGsgeW91IHRocm91Z2ggdGhlIHByb2Nlc3Mgb2YgYSBvbmUtd2F5IEFOT1ZBIChvbmUgaW5kZXBlbmRlbnQgdmFyaWFibGUpIGFuZCBhIHR3by13YXkgQU5PVkEgKHR3byBpbmRlcGVuZGVudCB2YXJpYWJsZXMpLg0KDQpPdXIgc2FtcGxlIGRhdGFzZXQgY29udGFpbnMgb2JzZXJ2YXRpb25zIGZyb20gYW4gaW1hZ2luYXJ5IHN0dWR5IG9mIHRoZSBlZmZlY3RzIG9mIGZlcnRpbGl6ZXIgdHlwZSBhbmQgcGxhbnRpbmcgZGVuc2l0eSBvbiBjcm9wIHlpZWxkLg0KDQo+ICoqT25lLXdheSBBTk9WQSBleGFtcGxlKiogIA0KSW4gdGhlIG9uZS13YXkgQU5PVkEsIHdlIHRlc3QgdGhlIGVmZmVjdHMgb2YgMyB0eXBlcyBvZiBmZXJ0aWxpemVyIG9uIGNyb3AgeWllbGQuDQoNCj4gKipUd28td2F5IEFOT1ZBIGV4YW1wbGUqKiAgDQpJbiB0aGUgdHdvLXdheSBBTk9WQSwgd2UgYWRkIGFuIGFkZGl0aW9uYWwgaW5kZXBlbmRlbnQgdmFyaWFibGU6IHBsYW50aW5nIGRlbnNpdHkuIFdlIHRlc3QgdGhlIGVmZmVjdHMgb2YgMyB0eXBlcyBvZiBmZXJ0aWxpemVyIGFuZCAyIGRpZmZlcmVudCBwbGFudGluZyBkZW5zaXRpZXMgb24gY3JvcCB5aWVsZC4NCg0KV2Ugd2lsbCBhbHNvIGluY2x1ZGUgZXhhbXBsZXMgb2YgaG93IHRvIHBlcmZvcm0gYW5kIGludGVycHJldCBhIHR3by13YXkgQU5PVkEgd2l0aCBhbiBpbnRlcmFjdGlvbiB0ZXJtLCBhbmQgYW4gQU5PVkEgd2l0aCBhIGJsb2NraW5nIHZhcmlhYmxlLg0KDQojIEdldHRpbmcgc3RhcnRlZCBpbiBSDQpJZiB5b3UgaGF2ZW7igJl0IHVzZWQgUiBiZWZvcmUsIHN0YXJ0IGJ5IGRvd25sb2FkaW5nIFtSXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy9iaW4vd2luZG93cy9iYXNlLykgYW5kIFtSIFN0dWRpb10oaHR0cHM6Ly9yc3R1ZGlvLmNvbS9wcm9kdWN0cy9yc3R1ZGlvL2Rvd25sb2FkLykuIE9uY2UgeW91IGhhdmUgYm90aCBvZiB0aGVzZSBwcm9ncmFtcyBkb3dubG9hZGVkLCBvcGVuIFIgU3R1ZGlvIGFuZCBjbGljayBvbiBgRmlsZSA+IE5ldyBGaWxlID4gUiBTY3JpcHRgLg0KDQpOb3cgeW91IGNhbiBjb3B5IGFuZCBwYXN0ZSB0aGUgY29kZSBmcm9tIHRoZSByZXN0IG9mIHRoaXMgZXhhbXBsZSBpbnRvIHlvdXIgc2NyaXB0LiBUbyBydW4gdGhlIGNvZGUsIGhpZ2hsaWdodCB0aGUgbGluZXMgeW91IHdhbnQgdG8gcnVuIGFuZCBjbGljayBvbiB0aGUgUnVuIGJ1dHRvbiBvbiB0aGUgdG9wIHJpZ2h0IG9mIHRoZSB0ZXh0IGVkaXRvciAob3IgcHJlc3MgY3RybCArIGVudGVyIG9uIHRoZSBrZXlib2FyZCkuDQoNCiMjIEluc3RhbGwgYW5kIGxvYWQgdGhlIHBhY2thZ2VzDQpGaXJzdCwgaW5zdGFsbCB0aGUgcGFja2FnZXMgeW91IHdpbGwgbmVlZCBmb3IgdGhlIGFuYWx5c2lzICh0aGlzIG9ubHkgbmVlZHMgdG8gYmUgZG9uZSBvbmNlKToNCg0KYGBge3IgZWNobz1ULCByZXN1bHRzPSdoaWRlJ30NCmluc3RhbGwucGFja2FnZXMoYygiZ2dwbG90MiIsImdncHViciIsInRpZHl2ZXJzZSIsImJyb29tIiwgIkFJQ2Ntb2RhdmciKSkNCmBgYA0KDQpUaGVuIGxvYWQgdGhlc2UgcGFja2FnZXMgaW50byB5b3VyIFIgZW52aXJvbm1lbnQgKGRvIHRoaXMgZXZlcnkgdGltZSB5b3UgcmVzdGFydCB0aGUgUiBwcm9ncmFtKToNCg0KYGBge3IgZWNobz1ULCByZXN1bHRzPSdoaWRlJ30NCmxpYnJhcnkoZ2dwbG90MikNCmxpYnJhcnkoZ2dwdWJyKQ0KbGlicmFyeSh0aWR5dmVyc2UpDQpsaWJyYXJ5KGJyb29tKQ0KbGlicmFyeShBSUNjbW9kYXZnKQ0KYGBgDQoNCiMgU3RlcCAxOiBMb2FkIHRoZSBkYXRhIGludG8gUg0KTm90ZSB0aGF0IHRoaXMgZGF0YSB3YXMgZ2VuZXJhdGVkIGZvciB0aGlzIGV4YW1wbGUsIGl04oCZcyBub3QgZnJvbSBhIHJlYWwgZXhwZXJpbWVudC4NCg0KV2Ugd2lsbCB1c2UgdGhlIHNhbWUgZGF0YXNldCBmb3IgYWxsIG9mIG91ciBleGFtcGxlcyBpbiB0aGlzIHdhbGt0aHJvdWdoLiBUaGUgb25seSBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIGRpZmZlcmVudCBhbmFseXNlcyBpcyBob3cgbWFueSBpbmRlcGVuZGVudCB2YXJpYWJsZXMgd2UgaW5jbHVkZSBhbmQgaW4gd2hhdCBjb21iaW5hdGlvbiB3ZSBpbmNsdWRlIHRoZW0uDQoNCkl0IGlzIGNvbW1vbiBmb3IgZmFjdG9ycyB0byBiZSByZWFkIGFzIHF1YW50aXRhdGl2ZSB2YXJpYWJsZXMgd2hlbiBpbXBvcnRpbmcgYSBkYXRhc2V0IGludG8gUi4gVG8gYXZvaWQgdGhpcywgeW91IGNhbiB1c2UgdGhlIHJlYWQuY3N2KCkgY29tbWFuZCB0byByZWFkIGluIHRoZSBkYXRhLCBzcGVjaWZ5aW5nIHdpdGhpbiB0aGUgY29tbWFuZCB3aGV0aGVyIGVhY2ggb2YgdGhlIHZhcmlhYmxlcyBzaG91bGQgYmUgcXVhbnRpdGF0aXZlICjigJxudW1lcmlj4oCdKSBvciBjYXRlZ29yaWNhbCAo4oCcZmFjdG9y4oCdKS4NCg0KVXNlIHRoZSBmb2xsb3dpbmcgY29kZSwgcmVwbGFjaW5nIHRoZSBwYXRoL3RvL3lvdXIvZmlsZSB0ZXh0IHdpdGggdGhlIGFjdHVhbCBwYXRoIHRvIHlvdXIgZmlsZToNCg0KYGBge3J9DQpjcm9wLmRhdGEgPC0gcmVhZC5jc3YoIkM6L1VzZXJzL2VtYWlsL0Rvd25sb2Fkcy9jcm9wZGF0YS5jc3YiLCBoZWFkZXIgPSBUUlVFLCBjb2xDbGFzc2VzID0gYygiZmFjdG9yIiwgImZhY3RvciIsICJmYWN0b3IiLCAibnVtZXJpYyIpKQ0KYGBgDQoNCkJlZm9yZSBjb250aW51aW5nLCB5b3UgY2FuIGNoZWNrIHRoYXQgdGhlIGRhdGEgaGFzIHJlYWQgaW4gY29ycmVjdGx5Og0KDQpgYGB7cn0NCnN1bW1hcnkoY3JvcC5kYXRhKQ0KYGBgDQoNCllvdSBzaG91bGQgc2VlIOKAmGRlbnNpdHnigJksIOKAmGJsb2Nr4oCZLCBhbmQg4oCYZmVydGlsaXplcuKAmSBsaXN0ZWQgYXMgY2F0ZWdvcmljYWwgdmFyaWFibGVzIHdpdGggdGhlIG51bWJlciBvZiBvYnNlcnZhdGlvbnMgYXQgZWFjaCBsZXZlbCAoaS5lLiA0OCBvYnNlcnZhdGlvbnMgYXQgZGVuc2l0eSAxIGFuZCA0OCBvYnNlcnZhdGlvbnMgYXQgZGVuc2l0eSAyKS4NCg0K4oCYWWllbGTigJkgc2hvdWxkIGJlIGEgcXVhbnRpdGF0aXZlIHZhcmlhYmxlIHdpdGggYSBudW1lcmljIHN1bW1hcnkgKG1pbmltdW0sIG1lZGlhbiwgbWVhbiwgbWF4aW11bSkuDQoNCiMgU3RlcCAyOiBQZXJmb3JtIHRoZSBBTk9WQSB0ZXN0DQpBTk9WQSB0ZXN0cyB3aGV0aGVyIGFueSBvZiB0aGUgZ3JvdXAgbWVhbnMgYXJlIGRpZmZlcmVudCBmcm9tIHRoZSBvdmVyYWxsIG1lYW4gb2YgdGhlIGRhdGEgYnkgY2hlY2tpbmcgdGhlIHZhcmlhbmNlIG9mIGVhY2ggaW5kaXZpZHVhbCBncm91cCBhZ2FpbnN0IHRoZSBvdmVyYWxsIHZhcmlhbmNlIG9mIHRoZSBkYXRhLiBJZiBvbmUgb3IgbW9yZSBncm91cHMgZmFsbHMgb3V0c2lkZSB0aGUgcmFuZ2Ugb2YgdmFyaWF0aW9uIHByZWRpY3RlZCBieSB0aGUgbnVsbCBoeXBvdGhlc2lzIChhbGwgZ3JvdXAgbWVhbnMgYXJlIGVxdWFsKSwgdGhlbiB0aGUgdGVzdCBpcyBzdGF0aXN0aWNhbGx5IHNpZ25pZmljYW50Lg0KDQpXZSBjYW4gcGVyZm9ybSBhbiBBTk9WQSBpbiBSIHVzaW5nIHRoZSBhb3YoKSBmdW5jdGlvbi4gVGhpcyB3aWxsIGNhbGN1bGF0ZSB0aGUgdGVzdCBzdGF0aXN0aWMgZm9yIEFOT1ZBIGFuZCBkZXRlcm1pbmUgd2hldGhlciB0aGVyZSBpcyBzaWduaWZpY2FudCB2YXJpYXRpb24gYW1vbmcgdGhlIGdyb3VwcyBmb3JtZWQgYnkgdGhlIGxldmVscyBvZiB0aGUgaW5kZXBlbmRlbnQgdmFyaWFibGUuDQoNCiMjIE9uZS13YXkgQU5PVkENCkluIHRoZSBvbmUtd2F5IEFOT1ZBIGV4YW1wbGUsIHdlIGFyZSBtb2RlbGluZyBjcm9wIHlpZWxkIGFzIGEgZnVuY3Rpb24gb2YgdGhlIHR5cGUgb2YgZmVydGlsaXplciB1c2VkLiBGaXJzdCB3ZSB3aWxsIHVzZSBgYW92KClgIHRvIHJ1biB0aGUgbW9kZWwsIHRoZW4gd2Ugd2lsbCB1c2UgYHN1bW1hcnkoKWAgdG8gcHJpbnQgdGhlIHN1bW1hcnkgb2YgdGhlIG1vZGVsLg0KDQpgYGB7cn0NCm9uZS53YXkgPC0gYW92KHlpZWxkIH4gZmVydGlsaXplciwgZGF0YSA9IGNyb3AuZGF0YSkNCnN1bW1hcnkob25lLndheSkNCmBgYA0KDQpUaGUgbW9kZWwgc3VtbWFyeSBmaXJzdCBsaXN0cyB0aGUgaW5kZXBlbmRlbnQgdmFyaWFibGVzIGJlaW5nIHRlc3RlZCBpbiB0aGUgbW9kZWwgKGluIHRoaXMgY2FzZSB3ZSBoYXZlIG9ubHkgb25lLCDigJhmZXJ0aWxpemVy4oCZKSBhbmQgdGhlIG1vZGVsIHJlc2lkdWFscyAo4oCYUmVzaWR1YWzigJkpLiBBbGwgb2YgdGhlIHZhcmlhdGlvbiB0aGF0IGlzIG5vdCBleHBsYWluZWQgYnkgdGhlIGluZGVwZW5kZW50IHZhcmlhYmxlcyBpcyBjYWxsZWQgcmVzaWR1YWwgdmFyaWFuY2UuDQoNClRoZSByZXN0IG9mIHRoZSB2YWx1ZXMgaW4gdGhlIG91dHB1dCB0YWJsZSBkZXNjcmliZSB0aGUgaW5kZXBlbmRlbnQgdmFyaWFibGUgYW5kIHRoZSByZXNpZHVhbHM6DQoNCiAtIFRoZSAqKkRmKiogY29sdW1uIGRpc3BsYXlzIHRoZSBkZWdyZWVzIG9mIGZyZWVkb20gZm9yIHRoZSBpbmRlcGVuZGVudCB2YXJpYWJsZSAodGhlIG51bWJlciBvZiBsZXZlbHMgaW4gdGhlIHZhcmlhYmxlIG1pbnVzIDEpLCBhbmQgdGhlIGRlZ3JlZXMgb2YgZnJlZWRvbSBmb3IgdGhlIHJlc2lkdWFscyAodGhlIHRvdGFsIG51bWJlciBvZiBvYnNlcnZhdGlvbnMgbWludXMgb25lIGFuZCBtaW51cyB0aGUgbnVtYmVyIG9mIGxldmVscyBpbiB0aGUgaW5kZXBlbmRlbnQgdmFyaWFibGVzKS4NCiAtIFRoZSAqKlN1bSBTcSoqIGNvbHVtbiBkaXNwbGF5cyB0aGUgc3VtIG9mIHNxdWFyZXMgKGEuay5hLiB0aGUgdG90YWwgdmFyaWF0aW9uIGJldHdlZW4gdGhlIGdyb3VwIG1lYW5zIGFuZCB0aGUgb3ZlcmFsbCBtZWFuKS4NCiAtIFRoZSAqKk1lYW4gU3EqKiBjb2x1bW4gaXMgdGhlIG1lYW4gb2YgdGhlIHN1bSBvZiBzcXVhcmVzLCBjYWxjdWxhdGVkIGJ5IGRpdmlkaW5nIHRoZSBzdW0gb2Ygc3F1YXJlcyBieSB0aGUgZGVncmVlcyBvZiBmcmVlZG9tIGZvciBlYWNoIHBhcmFtZXRlci4NCiAtIFRoZSAqKkYgdmFsdWUqKiBjb2x1bW4gaXMgdGhlIHRlc3Qgc3RhdGlzdGljIGZyb20gdGhlIEYgdGVzdC4gVGhpcyBpcyB0aGUgbWVhbiBzcXVhcmUgb2YgZWFjaCBpbmRlcGVuZGVudCB2YXJpYWJsZSBkaXZpZGVkIGJ5IHRoZSBtZWFuIHNxdWFyZSBvZiB0aGUgcmVzaWR1YWxzLiBUaGUgbGFyZ2VyIHRoZSBGIHZhbHVlLCB0aGUgbW9yZSBsaWtlbHkgaXQgaXMgdGhhdCB0aGUgdmFyaWF0aW9uIGNhdXNlZCBieSB0aGUgaW5kZXBlbmRlbnQgdmFyaWFibGUgaXMgcmVhbCBhbmQgbm90IGR1ZSB0byBjaGFuY2UuDQogLSBUaGUgKipQcig+RikqKiBjb2x1bW4gaXMgdGhlIHAgdmFsdWUgb2YgdGhlIEYgc3RhdGlzdGljLiBUaGlzIHNob3dzIGhvdyBsaWtlbHkgaXQgaXMgdGhhdCB0aGUgRiB2YWx1ZSBjYWxjdWxhdGVkIGZyb20gdGhlIHRlc3Qgd291bGQgaGF2ZSBvY2N1cnJlZCBpZiB0aGUgbnVsbCBoeXBvdGhlc2lzIG9mIG5vIGRpZmZlcmVuY2UgYW1vbmcgZ3JvdXAgbWVhbnMgd2VyZSB0cnVlLg0KVGhlICpwKi12YWx1ZSBvZiB0aGUgZmVydGlsaXplciB2YXJpYWJsZSBpcyBsb3cgKCpwKiA8IDAuMDAxKSwgc28gaXQgYXBwZWFycyB0aGF0IHRoZSB0eXBlIG9mIGZlcnRpbGl6ZXIgdXNlZCBoYXMgYSByZWFsIGltcGFjdCBvbiB0aGUgZmluYWwgY3JvcCB5aWVsZC4NCg0KIyMgVHdvLXdheSBBTk9WQQ0KSW4gdGhlIHR3by13YXkgQU5PVkEgZXhhbXBsZSwgd2UgYXJlIG1vZGVsaW5nIGNyb3AgeWllbGQgYXMgYSBmdW5jdGlvbiBvZiB0eXBlIG9mIGZlcnRpbGl6ZXIgYW5kIHBsYW50aW5nIGRlbnNpdHkuIEZpcnN0IHdlIHVzZSBgYW92KClgIHRvIHJ1biB0aGUgbW9kZWwsIHRoZW4gd2UgdXNlIGBzdW1tYXJ5KClgIHRvIHByaW50IHRoZSBzdW1tYXJ5IG9mIHRoZSBtb2RlbC4NCg0KYGBge3J9DQp0d28ud2F5IDwtIGFvdih5aWVsZCB+IGZlcnRpbGl6ZXIgKyBkZW5zaXR5LCBkYXRhID0gY3JvcC5kYXRhKQ0Kc3VtbWFyeSh0d28ud2F5KQ0KYGBgDQoNCkFkZGluZyBwbGFudGluZyBkZW5zaXR5IHRvIHRoZSBtb2RlbCBzZWVtcyB0byBoYXZlIG1hZGUgdGhlIG1vZGVsIGJldHRlcjogaXQgcmVkdWNlZCB0aGUgcmVzaWR1YWwgdmFyaWFuY2UgKHRoZSByZXNpZHVhbCBzdW0gb2Ygc3F1YXJlcyB3ZW50IGZyb20gMzUuODkgdG8gMzAuNzY1KSwgYW5kIGJvdGggcGxhbnRpbmcgZGVuc2l0eSBhbmQgZmVydGlsaXplciBhcmUgc3RhdGlzdGljYWxseSBzaWduaWZpY2FudCAoKnAqLXZhbHVlcyA8IDAuMDAxKS4NCg0KIyMgQWRkaW5nIGludGVyYWN0aW9ucyBiZXR3ZWVuIHZhcmlhYmxlcw0KU29tZXRpbWVzIHlvdSBoYXZlIHJlYXNvbiB0byB0aGluayB0aGF0IHR3byBvZiB5b3VyIGluZGVwZW5kZW50IHZhcmlhYmxlcyBoYXZlIGFuIGludGVyYWN0aW9uIGVmZmVjdCByYXRoZXIgdGhhbiBhbiBhZGRpdGl2ZSBlZmZlY3QuDQoNCkZvciBleGFtcGxlLCBpbiBvdXIgY3JvcCB5aWVsZCBleHBlcmltZW50LCBpdCBpcyBwb3NzaWJsZSB0aGF0IHBsYW50aW5nIGRlbnNpdHkgYWZmZWN0cyB0aGUgcGxhbnRz4oCZIGFiaWxpdHkgdG8gdGFrZSB1cCBmZXJ0aWxpemVyLiBUaGlzIG1pZ2h0IGluZmx1ZW5jZSB0aGUgZWZmZWN0IG9mIGZlcnRpbGl6ZXIgdHlwZSBpbiBhIHdheSB0aGF0IGlzbuKAmXQgYWNjb3VudGVkIGZvciBpbiB0aGUgdHdvLXdheSBtb2RlbC4NCg0KVG8gdGVzdCB3aGV0aGVyIHR3byB2YXJpYWJsZXMgaGF2ZSBhbiBpbnRlcmFjdGlvbiBlZmZlY3QgaW4gQU5PVkEsIHNpbXBseSB1c2UgYW4gYXN0ZXJpc2sgaW5zdGVhZCBvZiBhIHBsdXMtc2lnbiBpbiB0aGUgbW9kZWw6DQoNCmBgYHtyfQ0KaW50ZXJhY3Rpb24gPC0gYW92KHlpZWxkIH4gZmVydGlsaXplcipkZW5zaXR5LCBkYXRhID0gY3JvcC5kYXRhKQ0Kc3VtbWFyeShpbnRlcmFjdGlvbikNCmBgYA0KDQpJbiB0aGUgb3V0cHV0IHRhYmxlLCB0aGUg4oCYZmVydGlsaXplcjpkZW5zaXR54oCZIHZhcmlhYmxlIGhhcyBhIGxvdyBzdW0tb2Ytc3F1YXJlcyB2YWx1ZSBhbmQgYSBoaWdoICpwKiB2YWx1ZSwgd2hpY2ggbWVhbnMgdGhlcmUgaXMgbm90IG11Y2ggdmFyaWF0aW9uIHRoYXQgY2FuIGJlIGV4cGxhaW5lZCBieSB0aGUgaW50ZXJhY3Rpb24gYmV0d2VlbiBmZXJ0aWxpemVyIGFuZCBwbGFudGluZyBkZW5zaXR5Lg0KDQojIyBBZGRpbmcgYSBibG9ja2luZyB2YXJpYWJsZQ0KSWYgeW91IGhhdmUgZ3JvdXBlZCB5b3VyIGV4cGVyaW1lbnRhbCB0cmVhdG1lbnRzIGluIHNvbWUgd2F5LCBvciBpZiB5b3UgaGF2ZSBhIGNvbmZvdW5kaW5nIHZhcmlhYmxlIHRoYXQgbWlnaHQgYWZmZWN0IHRoZSByZWxhdGlvbnNoaXAgeW91IGFyZSBpbnRlcmVzdGVkIGluIHRlc3RpbmcsIHlvdSBzaG91bGQgaW5jbHVkZSB0aGF0IGVsZW1lbnQgaW4gdGhlIG1vZGVsIGFzIGEgYmxvY2tpbmcgdmFyaWFibGUuIFRoZSBzaW1wbGVzdCB3YXkgdG8gZG8gdGhpcyBpcyBqdXN0IHRvIGFkZCB0aGUgdmFyaWFibGUgaW50byB0aGUgbW9kZWwgd2l0aCBhIOKAmCvigJkuDQoNCkZvciBleGFtcGxlLCBpbiBtYW55IGNyb3AgeWllbGQgc3R1ZGllcywgdHJlYXRtZW50cyBhcmUgYXBwbGllZCB3aXRoaW4g4oCYYmxvY2tz4oCZIGluIHRoZSBmaWVsZCB0aGF0IG1heSBkaWZmZXIgaW4gc29pbCB0ZXh0dXJlLCBtb2lzdHVyZSwgc3VubGlnaHQsIGV0Yy4gVG8gY29udHJvbCBmb3IgdGhlIGVmZmVjdCBvZiBkaWZmZXJlbmNlcyBhbW9uZyBwbGFudGluZyBibG9ja3Mgd2UgYWRkIGEgdGhpcmQgdGVybSwg4oCYYmxvY2vigJksIHRvIG91ciBBTk9WQS4NCg0KYGBge3J9DQpibG9ja2luZyA8LSBhb3YoeWllbGQgfiBmZXJ0aWxpemVyICsgZGVuc2l0eSArIGJsb2NrLCBkYXRhID0gY3JvcC5kYXRhKQ0Kc3VtbWFyeShibG9ja2luZykNCmBgYA0KDQpUaGUg4oCYYmxvY2vigJkgdmFyaWFibGUgaGFzIGEgbG93IHN1bS1vZi1zcXVhcmVzIHZhbHVlICgwLjQ4NikgYW5kIGEgaGlnaCBwIHZhbHVlICgqcCogPSAwLjQ4KSwgc28gaXTigJlzIHByb2JhYmx5IG5vdCBhZGRpbmcgbXVjaCBpbmZvcm1hdGlvbiB0byB0aGUgbW9kZWwuIEl0IGFsc28gZG9lc27igJl0IGNoYW5nZSB0aGUgc3VtIG9mIHNxdWFyZXMgZm9yIHRoZSB0d28gaW5kZXBlbmRlbnQgdmFyaWFibGVzLCB3aGljaCBtZWFucyB0aGF0IGl04oCZcyBub3QgYWZmZWN0aW5nIGhvdyBtdWNoIHZhcmlhdGlvbiBpbiB0aGUgZGVwZW5kZW50IHZhcmlhYmxlIHRoZXkgZXhwbGFpbi4NCg0KIyBTdGVwIDM6IEZpbmQgdGhlIGJlc3QtZml0IG1vZGVsDQpUaGVyZSBhcmUgbm93IGZvdXIgZGlmZmVyZW50IEFOT1ZBIG1vZGVscyB0byBleHBsYWluIHRoZSBkYXRhLiBIb3cgZG8geW91IGRlY2lkZSB3aGljaCBvbmUgdG8gdXNlPyBVc3VhbGx5IHlvdeKAmWxsIHdhbnQgdG8gdXNlIHRoZSDigJhiZXN0LWZpdOKAmSBtb2RlbCDigJMgdGhlIG1vZGVsIHRoYXQgYmVzdCBleHBsYWlucyB0aGUgdmFyaWF0aW9uIGluIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUuDQoNClRoZSBBa2Fpa2UgaW5mb3JtYXRpb24gY3JpdGVyaW9uIChBSUMpIGlzIGEgZ29vZCB0ZXN0IGZvciBtb2RlbCBmaXQuIEFJQyBjYWxjdWxhdGVzIHRoZSBpbmZvcm1hdGlvbiB2YWx1ZSBvZiBlYWNoIG1vZGVsIGJ5IGJhbGFuY2luZyB0aGUgdmFyaWF0aW9uIGV4cGxhaW5lZCBhZ2FpbnN0IHRoZSBudW1iZXIgb2YgcGFyYW1ldGVycyB1c2VkLg0KDQpJbiBBSUMgbW9kZWwgc2VsZWN0aW9uLCB3ZSBjb21wYXJlIHRoZSBpbmZvcm1hdGlvbiB2YWx1ZSBvZiBlYWNoIG1vZGVsIGFuZCBjaG9vc2UgdGhlIG9uZSB3aXRoIHRoZSBsb3dlc3QgQUlDIHZhbHVlIChhIGxvd2VyIG51bWJlciBtZWFucyBtb3JlIGluZm9ybWF0aW9uIGV4cGxhaW5lZCEpDQoNCmBgYHtyfQ0KbGlicmFyeShBSUNjbW9kYXZnKQ0KbW9kZWwuc2V0IDwtIGxpc3Qob25lLndheSwgdHdvLndheSwgaW50ZXJhY3Rpb24sIGJsb2NraW5nKQ0KbW9kZWwubmFtZXMgPC0gYygib25lLndheSIsICJ0d28ud2F5IiwgImludGVyYWN0aW9uIiwgImJsb2NraW5nIikNCmFpY3RhYihtb2RlbC5zZXQsIG1vZG5hbWVzID0gbW9kZWwubmFtZXMpDQpgYGANCg0KVGhlIG1vZGVsIHdpdGggdGhlIGxvd2VzdCBBSUMgc2NvcmUgKGxpc3RlZCBmaXJzdCBpbiB0aGUgdGFibGUpIGlzIHRoZSBiZXN0IGZpdCBmb3IgdGhlIGRhdGEuDQoNCkZyb20gdGhlc2UgcmVzdWx0cywgaXQgYXBwZWFycyB0aGF0IHRoZSB0d28ud2F5IG1vZGVsIGlzIHRoZSBiZXN0IGZpdC4gVGhlIHR3by13YXkgbW9kZWwgaGFzIHRoZSBsb3dlc3QgQUlDIHZhbHVlLCBhbmQgNzElIG9mIHRoZSBBSUMgd2VpZ2h0LCB3aGljaCBtZWFucyB0aGF0IGl0IGV4cGxhaW5zIDcxJSBvZiB0aGUgdG90YWwgdmFyaWF0aW9uIGluIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUgdGhhdCBjYW4gYmUgZXhwbGFpbmVkIGJ5IHRoZSBmdWxsIHNldCBvZiBtb2RlbHMuDQoNClRoZSBtb2RlbCB3aXRoIGJsb2NraW5nIHRlcm0gY29udGFpbnMgYW4gYWRkaXRpb25hbCAxNSUgb2YgdGhlIEFJQyB3ZWlnaHQsIGJ1dCBiZWNhdXNlIGl0IGlzIG1vcmUgdGhhbiAyIGRlbHRhLUFJQyB3b3JzZSB0aGFuIHRoZSBiZXN0IG1vZGVsLCBpdCBwcm9iYWJseSBpc27igJl0IGdvb2QgZW5vdWdoIHRvIGluY2x1ZGUgaW4geW91ciByZXN1bHRzLg0KDQojIFN0ZXAgNDogQ2hlY2sgZm9yIGhvbW9zY2VkYXN0aWNpdHkNClRvIGNoZWNrIHdoZXRoZXIgdGhlIG1vZGVsIGZpdHMgdGhlIGFzc3VtcHRpb24gb2YgaG9tb3NjZWRhc3RpY2l0eSwgbG9vayBhdCB0aGUgbW9kZWwgZGlhZ25vc3RpYyBwbG90cyBpbiBSIHVzaW5nIHRoZSBwbG90KCkgZnVuY3Rpb246DQoNCmBgYHtyfQ0KcGFyKG1mcm93PWMoMiwyKSkNCnBsb3QodHdvLndheSkNCnBhcihtZnJvdz1jKDEsMSkpDQpgYGANCg0KVGhlIGRpYWdub3N0aWMgcGxvdHMgc2hvdyB0aGUgdW5leHBsYWluZWQgdmFyaWFuY2UgKHJlc2lkdWFscykgYWNyb3NzIHRoZSByYW5nZSBvZiB0aGUgb2JzZXJ2ZWQgZGF0YS4NCg0KRWFjaCBwbG90IGdpdmVzIGEgc3BlY2lmaWMgcGllY2Ugb2YgaW5mb3JtYXRpb24gYWJvdXQgdGhlIG1vZGVsIGZpdCwgYnV0IGl04oCZcyBlbm91Z2ggdG8ga25vdyB0aGF0IHRoZSByZWQgbGluZSByZXByZXNlbnRpbmcgdGhlIG1lYW4gb2YgdGhlIHJlc2lkdWFscyBzaG91bGQgYmUgaG9yaXpvbnRhbCBhbmQgY2VudGVyZWQgb24gemVybyAob3Igb24gb25lLCBpbiB0aGUgc2NhbGUtbG9jYXRpb24gcGxvdCksIG1lYW5pbmcgdGhhdCB0aGVyZSBhcmUgbm8gbGFyZ2Ugb3V0bGllcnMgdGhhdCB3b3VsZCBjYXVzZSByZXNlYXJjaCBiaWFzIGluIHRoZSBtb2RlbC4NCg0KVGhlIG5vcm1hbCBRLVEgcGxvdCBwbG90cyBhIHJlZ3Jlc3Npb24gYmV0d2VlbiB0aGUgdGhlb3JldGljYWwgcmVzaWR1YWxzIG9mIGEgcGVyZmVjdGx5LWhvbW9zY2VkYXN0aWMgbW9kZWwgYW5kIHRoZSBhY3R1YWwgcmVzaWR1YWxzIG9mIHlvdXIgbW9kZWwsIHNvIHRoZSBjbG9zZXIgdG8gYSBzbG9wZSBvZiAxIHRoaXMgaXMgdGhlIGJldHRlci4gVGhpcyBRLVEgcGxvdCBpcyB2ZXJ5IGNsb3NlLCB3aXRoIG9ubHkgYSBiaXQgb2YgZGV2aWF0aW9uLg0KDQpGcm9tIHRoZXNlIGRpYWdub3N0aWMgcGxvdHMgd2UgY2FuIHNheSB0aGF0IHRoZSBtb2RlbCBmaXRzIHRoZSBhc3N1bXB0aW9uIG9mIGhvbW9zY2VkYXN0aWNpdHkuDQoNCklmIHlvdXIgbW9kZWwgZG9lc27igJl0IGZpdCB0aGUgYXNzdW1wdGlvbiBvZiBob21vc2NlZGFzdGljaXR5LCB5b3UgY2FuIHRyeSB0aGUgS3J1c2thbGwtV2FsbGlzIHRlc3QgaW5zdGVhZC4NCg0KIyBTdGVwIDU6IERvIGEgcG9zdC1ob2MgdGVzdA0KQU5PVkEgdGVsbHMgdXMgaWYgdGhlcmUgYXJlIGRpZmZlcmVuY2VzIGFtb25nIGdyb3VwIG1lYW5zLCBidXQgbm90IHdoYXQgdGhlIGRpZmZlcmVuY2VzIGFyZS4gVG8gZmluZCBvdXQgd2hpY2ggZ3JvdXBzIGFyZSBzdGF0aXN0aWNhbGx5IGRpZmZlcmVudCBmcm9tIG9uZSBhbm90aGVyLCB5b3UgY2FuIHBlcmZvcm0gYSBUdWtleeKAmXMgSG9uZXN0bHkgU2lnbmlmaWNhbnQgRGlmZmVyZW5jZSAoVHVrZXnigJlzIEhTRCkgcG9zdC1ob2MgdGVzdCBmb3IgcGFpcndpc2UgY29tcGFyaXNvbnM6DQoNCmBgYHtyfQ0KdHVrZXkudHdvLndheTwtVHVrZXlIU0QodHdvLndheSkNCnR1a2V5LnR3by53YXkNCmBgYA0KDQpGcm9tIHRoZSBwb3N0LWhvYyB0ZXN0IHJlc3VsdHMsIHdlIHNlZSB0aGF0IHRoZXJlIGFyZSBzdGF0aXN0aWNhbGx5IHNpZ25pZmljYW50IGRpZmZlcmVuY2VzICgqcCogPCAwLjA1KSBiZXR3ZWVuIGZlcnRpbGl6ZXIgZ3JvdXBzIDMgYW5kIDEgYW5kIGJldHdlZW4gZmVydGlsaXplciB0eXBlcyAzIGFuZCAyLCBidXQgdGhlIGRpZmZlcmVuY2UgYmV0d2VlbiBmZXJ0aWxpemVyIGdyb3VwcyAyIGFuZCAxIGlzIG5vdCBzdGF0aXN0aWNhbGx5IHNpZ25pZmljYW50LiBUaGVyZSBpcyBhbHNvIGEgc2lnbmlmaWNhbnQgZGlmZmVyZW5jZSBiZXR3ZWVuIHRoZSB0d28gZGlmZmVyZW50IGxldmVscyBvZiBwbGFudGluZyBkZW5zaXR5Lg0KDQojIFN0ZXAgNjogUGxvdCB0aGUgcmVzdWx0cyBpbiBhIGdyYXBoDQpXaGVuIHBsb3R0aW5nIHRoZSByZXN1bHRzIG9mIGEgbW9kZWwsIGl0IGlzIGltcG9ydGFudCB0byBkaXNwbGF5Og0KDQogLSB0aGUgcmF3IGRhdGENCiAtIHN1bW1hcnkgaW5mb3JtYXRpb24sIHVzdWFsbHkgdGhlIG1lYW4gYW5kIHN0YW5kYXJkIGVycm9yIG9mIGVhY2ggZ3JvdXAgYmVpbmcgY29tcGFyZWQNCiAtIGxldHRlcnMgb3Igc3ltYm9scyBhYm92ZSBlYWNoIGdyb3VwIGJlaW5nIGNvbXBhcmVkIHRvIGluZGljYXRlIHRoZSBncm91cHdpc2UgZGlmZmVyZW5jZXMuDQoNCiMjIEZpbmQgdGhlIGdyb3Vwd2lzZSBkaWZmZXJlbmNlcw0KRnJvbSB0aGUgQU5PVkEgdGVzdCB3ZSBrbm93IHRoYXQgYm90aCBwbGFudGluZyBkZW5zaXR5IGFuZCBmZXJ0aWxpemVyIHR5cGUgYXJlIHNpZ25pZmljYW50IHZhcmlhYmxlcy4gVG8gZGlzcGxheSB0aGlzIGluZm9ybWF0aW9uIG9uIGEgZ3JhcGgsIHdlIG5lZWQgdG8gc2hvdyB3aGljaCBvZiB0aGUgY29tYmluYXRpb25zIG9mIGZlcnRpbGl6ZXIgdHlwZSArIHBsYW50aW5nIGRlbnNpdHkgYXJlIHN0YXRpc3RpY2FsbHkgZGlmZmVyZW50IGZyb20gb25lIGFub3RoZXIuDQoNClRvIGRvIHRoaXMsIHdlIGNhbiBydW4gYW5vdGhlciBBTk9WQSArIFR1a2V5SFNEIHRlc3QsIHRoaXMgdGltZSB1c2luZyB0aGUgaW50ZXJhY3Rpb24gb2YgZmVydGlsaXplciBhbmQgcGxhbnRpbmcgZGVuc2l0eS4gV2UgYXJlbuKAmXQgZG9pbmcgdGhpcyB0byBmaW5kIG91dCBpZiB0aGUgaW50ZXJhY3Rpb24gdGVybSBpcyBzaWduaWZpY2FudCAod2UgYWxyZWFkeSBrbm93IGl04oCZcyBub3QpLCBidXQgcmF0aGVyIHRvIGZpbmQgb3V0IHdoaWNoIGdyb3VwIG1lYW5zIGFyZSBzdGF0aXN0aWNhbGx5IGRpZmZlcmVudCBmcm9tIG9uZSBhbm90aGVyIHNvIHdlIGNhbiBhZGQgdGhpcyBpbmZvcm1hdGlvbiB0byB0aGUgZ3JhcGguDQoNCmBgYHtyfQ0KdHVrZXkucGxvdC5hb3Y8LWFvdih5aWVsZCB+IGZlcnRpbGl6ZXI6ZGVuc2l0eSwgZGF0YT1jcm9wLmRhdGEpDQpgYGANCg0KSW5zdGVhZCBvZiBwcmludGluZyB0aGUgVHVrZXlIU0QgcmVzdWx0cyBpbiBhIHRhYmxlLCB3ZeKAmWxsIGRvIGl0IGluIGEgZ3JhcGguDQoNCmBgYHtyfQ0KdHVrZXkucGxvdC50ZXN0PC1UdWtleUhTRCh0dWtleS5wbG90LmFvdikNCnBsb3QodHVrZXkucGxvdC50ZXN0LCBsYXMgPSAxKQ0KYGBgDQoNClRoZSBzaWduaWZpY2FudCBncm91cHdpc2UgZGlmZmVyZW5jZXMgYXJlIGFueSB3aGVyZSB0aGUgOTUlIGNvbmZpZGVuY2UgaW50ZXJ2YWwgZG9lc27igJl0IGluY2x1ZGUgemVyby4gVGhpcyBpcyBhbm90aGVyIHdheSBvZiBzYXlpbmcgdGhhdCB0aGUgcCB2YWx1ZSBmb3IgdGhlc2UgcGFpcndpc2UgZGlmZmVyZW5jZXMgaXMgPCAwLjA1Lg0KDQpGcm9tIHRoaXMgZ3JhcGgsIHdlIGNhbiBzZWUgdGhhdCB0aGUgZmVydGlsaXplciArIHBsYW50aW5nIGRlbnNpdHkgY29tYmluYXRpb25zIHdoaWNoIGFyZSBzaWduaWZpY2FudGx5IGRpZmZlcmVudCBmcm9tIG9uZSBhbm90aGVyIGFyZSAzOjEtMToxIChyZWFkIGFzIOKAnGZlcnRpbGl6ZXIgdHlwZSB0aHJlZSArIHBsYW50aW5nIGRlbnNpdHkgMSBjb250cmFzdGVkIHdpdGggZmVydGlsaXplciB0eXBlIDEgKyBwbGFudGluZyBkZW5zaXR5IHR5cGUgMeKAnSksIDE6Mi0xOjEsIDI6Mi0xOjEsIDM6Mi0xOjEsIGFuZCAzOjItMjoxLg0KDQpXZSBjYW4gbWFrZSB0aHJlZSBsYWJlbHMgZm9yIG91ciBncmFwaDogQSAocmVwcmVzZW50aW5nIDE6MSksIEIgKHJlcHJlc2VudGluZyBhbGwgdGhlIGludGVybWVkaWF0ZSBjb21iaW5hdGlvbnMpLCBhbmQgQyAocmVwcmVzZW50aW5nIDM6MikuDQoNCiMjIE1ha2UgYSBkYXRhIGZyYW1lIHdpdGggdGhlIGdyb3VwIGxhYmVscw0KTm93IHdlIG5lZWQgdG8gbWFrZSBhbiBhZGRpdGlvbmFsIGRhdGEgZnJhbWUgc28gd2UgY2FuIGFkZCB0aGVzZSBncm91cHdpc2UgZGlmZmVyZW5jZXMgdG8gb3VyIGdyYXBoLg0KDQpGaXJzdCwgc3VtbWFyaXplIHRoZSBvcmlnaW5hbCBkYXRhIHVzaW5nIGZlcnRpbGl6ZXIgdHlwZSBhbmQgcGxhbnRpbmcgZGVuc2l0eSBhcyBncm91cGluZyB2YXJpYWJsZXMuDQoNCmBgYHtyfQ0KbWVhbi55aWVsZC5kYXRhIDwtIGNyb3AuZGF0YSAlPiUNCiAgZ3JvdXBfYnkoZmVydGlsaXplciwgZGVuc2l0eSkgJT4lDQogIHN1bW1hcmlzZSgNCiAgICAgIHlpZWxkID0gbWVhbih5aWVsZCkNCiAgKQ0KYGBgDQoNCk5leHQsIGFkZCB0aGUgZ3JvdXAgbGFiZWxzIGFzIGEgbmV3IHZhcmlhYmxlIGluIHRoZSBkYXRhIGZyYW1lLg0KDQpgYGB7cn0NCm1lYW4ueWllbGQuZGF0YSRncm91cCA8LSBjKCJhIiwiYiIsImIiLCJiIiwiYiIsImMiKQ0KbWVhbi55aWVsZC5kYXRhDQpgYGANCg0KTm93IHdlIGFyZSByZWFkeSB0byBzdGFydCBtYWtpbmcgdGhlIHBsb3QgZm9yIG91ciByZXBvcnQuDQoNCiMjIFBsb3QgdGhlIHJhdyBkYXRhDQpgYGB7cn0NCnR3by53YXkucGxvdCA8LSBnZ3Bsb3QoY3JvcC5kYXRhLCBhZXMoeCA9IGRlbnNpdHksIHkgPSB5aWVsZCwgZ3JvdXA9ZmVydGlsaXplcikpICsNCiAgZ2VvbV9wb2ludChjZXggPSAxLjUsIHBjaCA9IDEuMCxwb3NpdGlvbiA9IHBvc2l0aW9uX2ppdHRlcih3ID0gMC4xLCBoID0gMCkpDQp0d28ud2F5LnBsb3QNCmBgYA0KDQpUaGUgb3V0cHV0IGxvb2tzIGxpa2UgdGhpczoNCg0KIyMgQWRkIHRoZSBtZWFucyBhbmQgc3RhbmRhcmQgZXJyb3JzIHRvIHRoZSBncmFwaA0KYGBge3J9DQp0d28ud2F5LnBsb3QgPC0gdHdvLndheS5wbG90ICsNCiAgc3RhdF9zdW1tYXJ5KGZ1bi5kYXRhID0gJ21lYW5fc2UnLCBnZW9tID0gJ2Vycm9yYmFyJywgd2lkdGggPSAwLjIpICsNCiAgc3RhdF9zdW1tYXJ5KGZ1bi5kYXRhID0gJ21lYW5fc2UnLCBnZW9tID0gJ3BvaW50cmFuZ2UnKSArDQogIGdlb21fcG9pbnQoZGF0YT1tZWFuLnlpZWxkLmRhdGEsIGFlcyh4PWRlbnNpdHksIHk9eWllbGQpKQ0KDQp0d28ud2F5LnBsb3QNCmBgYA0KDQpUaGlzIGlzIHZlcnkgaGFyZCB0byByZWFkLCBzaW5jZSBhbGwgb2YgdGhlIGRpZmZlcmVudCBncm91cGluZ3MgZm9yIGZlcnRpbGl6ZXIgdHlwZSBhcmUgc3RhY2tlZCBvbiB0b3Agb2Ygb25lIGFub3RoZXIuIFdlIHdpbGwgc29sdmUgdGhpcyBpbiB0aGUgbmV4dCBzdGVwLg0KDQojIyBTcGxpdCB1cCB0aGUgZGF0YQ0KVG8gc2hvdyB3aGljaCBncm91cHMgYXJlIGRpZmZlcmVudCBmcm9tIG9uZSBhbm90aGVyLCB1c2UgYGZhY2V0X3dyYXAoKWAgdG8gc3BsaXQgdGhlIGRhdGEgdXAgb3ZlciB0aGUgdGhyZWUgdHlwZXMgb2YgZmVydGlsaXplci4gVG8gYWRkIGxhYmVscywgdXNlIGBnZW9tX3RleHQoKWAsIGFuZCBhZGQgdGhlIGdyb3VwIGxldHRlcnMgZnJvbSB0aGUgYG1lYW4ueWllbGQuZGF0YWAgZGF0YWZyYW1lIHlvdSBtYWRlIGVhcmxpZXIuDQoNCmBgYHtyfQ0KdHdvLndheS5wbG90IDwtIHR3by53YXkucGxvdCArDQogIGdlb21fdGV4dChkYXRhPW1lYW4ueWllbGQuZGF0YSwgbGFiZWw9bWVhbi55aWVsZC5kYXRhJGdyb3VwLCB2anVzdCA9IC04LCBzaXplID0gNSkgKw0KICBmYWNldF93cmFwKH4gZmVydGlsaXplcikNCg0KdHdvLndheS5wbG90DQpgYGANCg0KIyMgTWFrZSB0aGUgZ3JhcGggcmVhZHkgZm9yIHB1YmxpY2F0aW9uDQpJbiB0aGlzIHN0ZXAgd2Ugd2lsbCByZW1vdmUgdGhlIGdyZXkgYmFja2dyb3VuZCBhbmQgYWRkIGF4aXMgbGFiZWxzLg0KDQpgYGB7cn0NCnR3by53YXkucGxvdCA8LSB0d28ud2F5LnBsb3QgKw0KICB0aGVtZV9jbGFzc2ljMigpICsNCiAgbGFicyh0aXRsZSA9ICJDcm9wIHlpZWxkIGluIHJlc3BvbnNlIHRvIGZlcnRpbGl6ZXIgbWl4IGFuZCBwbGFudGluZyBkZW5zaXR5IiwNCiAgICAgIHggPSAiUGxhbnRpbmcgZGVuc2l0eSAoMT1sb3cgZGVuc2l0eSwgMj1oaWdoIGRlbnNpdHkpIiwNCiAgICAgIHkgPSAiWWllbGQgKGJ1c2hlbHMgcGVyIGFjcmUpIikNCg0KdHdvLndheS5wbG90DQpgYGANCg0KVGhlIGZpbmFsIHZlcnNpb24gb2YgeW91ciBncmFwaCBsb29rcyBsaWtlIHRoaXM6DQoNCiMgU3RlcCA3OiBSZXBvcnQgdGhlIHJlc3VsdHMNCkluIGFkZGl0aW9uIHRvIGEgZ3JhcGgsIGl04oCZcyBpbXBvcnRhbnQgdG8gc3RhdGUgdGhlIHJlc3VsdHMgb2YgdGhlIEFOT1ZBIHRlc3QuIEluY2x1ZGU6DQoNCiAtIEEgYnJpZWYgZGVzY3JpcHRpb24gb2YgdGhlIHZhcmlhYmxlcyB5b3UgdGVzdGVkDQogLSBUaGUgRiB2YWx1ZSwgZGVncmVlcyBvZiBmcmVlZG9tLCBhbmQgcCB2YWx1ZXMgZm9yIGVhY2ggaW5kZXBlbmRlbnQgdmFyaWFibGUNCiAtIFdoYXQgdGhlIHJlc3VsdHMgbWVhbi4NCg0KPiBFeGFtcGxlOiBSZXBvcnRpbmcgdGhlIHJlc3VsdHMgb2YgQU5PVkENCldlIGZvdW5kIGEgc3RhdGlzdGljYWxseS1zaWduaWZpY2FudCBkaWZmZXJlbmNlIGluIGF2ZXJhZ2UgY3JvcCB5aWVsZCBieSBib3RoIGZlcnRpbGl6ZXIgdHlwZSAoRigyKT05LjAxOCwgcCA8IDAuMDAxKSBhbmQgYnkgcGxhbnRpbmcgZGVuc2l0eSAoRigxKT0xNS4zMTYsIHAgPCAwLjAwMSkuDQpBIFR1a2V5IHBvc3QtaG9jIHRlc3QgcmV2ZWFsZWQgdGhhdCBmZXJ0aWxpemVyIG1peCAzIHJlc3VsdGVkIGluIGEgaGlnaGVyIHlpZWxkIG9uIGF2ZXJhZ2UgdGhhbiBmZXJ0aWxpemVyIG1peCAxICgwLjU5IGJ1c2hlbHMvYWNyZSksIGFuZCBhIGhpZ2hlciB5aWVsZCBvbiBhdmVyYWdlIHRoYW4gZmVydGlsaXplciBtaXggMiAoMC40MiBidXNoZWxzL2FjcmUpLiBQbGFudGluZyBkZW5zaXR5IHdhcyBhbHNvIHNpZ25pZmljYW50LCB3aXRoIHBsYW50aW5nIGRlbnNpdHkgMiByZXN1bHRpbmcgaW4gYW4gaGlnaGVyIHlpZWxkIG9uIGF2ZXJhZ2Ugb2YgMC40NiBidXNoZWxzL2FjcmUgb3ZlciBwbGFudGluZyBkZW5zaXR5IDEuDQpBIHN1YnNlcXVlbnQgZ3JvdXB3aXNlIGNvbXBhcmlzb24gc2hvd2VkIHRoZSBzdHJvbmdlc3QgeWllbGQgZ2FpbnMgYXQgcGxhbnRpbmcgZGVuc2l0eSAyLCBmZXJ0aWxpemVyIG1peCAzLCBzdWdnZXN0aW5nIHRoYXQgdGhpcyBtaXggb2YgdHJlYXRtZW50cyB3YXMgbW9zdCBhZHZhbnRhZ2VvdXMgZm9yIGNyb3AgZ3Jvd3RoIHVuZGVyIG91ciBleHBlcmltZW50YWwgY29uZGl0aW9ucy4NCg0KDQo=