Data Science Stream
Topic 6B: Writing R Functions
Welcome to the sixth computer lab for the Data Science stream of STM1001.
By now, you will have gained some experience using a variety of functions in R, ranging from inbuilt ‘base R’ functions, to package-specific functions, such as those contained in the plotly
package.
In this computer lab we will extend our R coding skills and focus on writing our own functions in R.
By the end of this lab, you should understand the R code syntax required to create functions, and feel comfortable writing simple and intermediate functions. Let’s get started!
R Functions Overview
Usually, if you need to carry out a specific statistical process in R, there will already be a relevant function for you to use!
However, when you are conducting your own coding or research, sometimes it can be extremely beneficial to write your own custom functions. These can range from simple functions of a few lines of code, to extremely detailed, multi-layered functions that make use of other functions and packages. The creative possibilities are endless.
R Function Composition
Let’s take a look at the basic composition of an R function, using the example function below:
my_simple_function <- function(argument1,
argument2){
output <- argument1 + argument2
}
The code above consists of four main parts:
- The function has a name,
my_simple_function
.
- We use the
function
function to (unsurprisingly) create a function.
- Inside the open brackets
()
following function
, we specify argument1
and argument2
to be the arguments, i.e. inputs, of our function.
- To begin the ‘main body’ of our function, we use a left curly brace,
{
, directly after the right open bracket )
.
- In the ‘main body’ of the function, we specify the calculations the function should perform - for
my_simple_function
, we are simply adding the argument1
and argument2
values provided. This sum is stored in the object output
, and when the function is computed, the output
value is provided as output.
- To wrap up our function, we conclude with a right curly brace,
}
.
You might be wondering what argument1
and argument2
are, and from where they came. In fact, these are arbitrary names, and don’t correspond to any specific numbers. We can provide whatever numbers or string of numbers we would like to our function, and within the my_simple_function
environment these values will be used in place of argument1
and argument2
. For example, suppose we would like to add 39 and 3. We can do this using our function as follows:
res <- my_simple_function(39, 3)
res
## [1] 42
Copy the code in the two code chunks above, paste it into an R script, and run it. You should obtain the result 42
shown above.
Of course, we didn’t really need to use a function for this - we could have simply computed 39 + 3
in R. In general, functions will be a little more complicated than my_simple_function
.
Important Notes
Strictly speaking, we did not need to include the output <-
code within my_simple_function
.
R will output the last evaluated expression in the function, so we could simply have used argument1 + argument2
in the main body of the function. If however we wanted to output more than one result, then specifying our desired output can be informative and help avoid errors.
This additional output <-
code was primarily included here to demonstrate a key feature of R functions - any objects created within a function are part of that function’s local environment
, meaning that they exist only inside that function, and can’t be called from the global environment
outside the function.
To clarify the distinction, take a look at the code below:
test <- 39 + 3
test
## [1] 42
Here, we assign the sum 39 + 3
to the object test
. This is not done within a function, so the object test
is stored in the global environment
. This means we can then call the test
object at a later date, and see our result of 42
.
Similarly, we could compute this result using my_simple_function
, as shown above in 1.1.
However, try to now call the object output
(which is defined within this function). You will receive an error - something along the lines of Error in eval(expr, envir, enclos) : object 'output' not found
.
This is something worth keeping in mind. If you intend to use an object outside of a function, it might be worth defining it separately. It is also a good idea to use different names for all your objects (inside and outside functions) to avoid confusion.
Writing Simple Functions in R
Now that we have covered the basics of R function composition, let’s try to write some simple functions ourselves. These functions actually already exist in base R, but this will be good practice for subsequent work.
Mean Function
Recall that the mean \(\bar{x}\) of a set of \(n\) values \(x_1, x_2, \cdots, x_n\) is computed using the formula \[\bar{x} = \displaystyle \frac{\sum_{i=1}^n x_i}{n} .\]
Suppose that you would like to write an R function that computes the mean of a set of provided values.
There are several ways that we could write this function. We have made a head-start with the partially completed code below. See if you can complete it by filling in the missing ...
parts.
mean_func <- function(values){
# Argument:
# values: This is our list of values
n <- length(values)
sum(...) / ...
}
Note that, based on this format, our function can compute the mean of a string of numbers of any non-zero length.
Verify that your mean_func
function works by computing the mean of the set of values 2, 3, 4, 5, 6, 7, 8
. If your function is correct, you should receive an answer of 5
.
Sample standard deviation Function
Now that you have had some practice writing R functions, let’s tackle a harder problem.
Recall the equation for the sample standard deviation \(s\) (introduced in Topic 2):
\[s = \displaystyle \sqrt{ \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2} .\]
Using your mean_func
function, and your code from 2.1 as a guide, write a function called sd_func
that computes the sample standard deviation of a set of \(n\) values \(x_1, x_2, \cdots, x_n\).
Hints:
- You can use the same input as for your
mean_func
function.
- When you are multiplying parts of a function together, you need to use the \(\ast\) symbol.
- You can use the built-in R function
sqrt()
to compute the square root.
- You can use your
mean_func
function within your new function, when computing the \(\bar{x}\) value.
- Be careful to include brackets to separate different parts of your calculations.
- Don’t forget the square term in the equation.
Verify that your sample standard deviation function works by computing the sample standard deviation of the set of values 2, 3, 4, 5, 6, 7, 8
. If your function is correct, you should receive an answer of 2.160247
.
Note: Your answer may be slightly different due to rounding.
Writing a \(t\)-test Function in R
Now that we have had some practice writing R functions, let’s move on to a slightly harder scenario, and try to write an R function for computing a \(t\)-test.
Recall that in Topic 5 we introduced the \(t\)-test as a common type of hypothesis test.
The test statistic \(t\) used for the one-sample \(t\)-test has the following formula:
\[t = \displaystyle \frac{\bar{x} - \mu_0}{s / \sqrt{n}},\]
where \(\bar{x}\), \(s\) and \(n\) are the sample mean, sample standard deviation, and sample size respectively, as previously defined, and \(\mu_0\) denotes the mean under the null hypothesis.
Note: \(s / \sqrt{n}\) is the standard error of \(t\), and is often denoted by \(SE\).
\(t\)-test test statistic
Using your functions mean_func
and sd_func
, write a new R function called t_test_func
that computes the test statistic \(t\) for a one-sample \(t\)-test with an arbitrary null hypothesis mean \(\mu_0\).
Hints:
- Most of the hard work is already done, since you can use your two functions
mean_func
and sd_func
.
- You will need to provide one new input argument, the \(\mu_0\) value, for your
t_test_func
function.
To verify your test statistic function’s accuracy, evaluate it for the set of values 2, 3, 4, 5, 6, 7, 8
, with the null hypothesis mean \(\mu_0 = 4\).
Compare your result to the result of the inbuilt R function t.test
, by running the code t.test(c(2:8), mu = 4)
. If you have written your function correctly, the t.test
\(t=\) output should be identical to the value of \(t\) computed by your t_test_func
function.
Hint: Don’t worry if your t_test_func
function gives a slightly different value - as long as it is correct to 3 or 4 decimal places that is fine. If it is very different, just go back over your function, and check for errors such as insufficient or misplaced brackets. It’s very easy to make mistakes like this.
Adding Details to our \(t\)-test Function
To conclude this computer lab, let’s try adding some additional details and features to our \(t\)-test function, to make it more informative.
Degrees of freedom
Recall that the degrees of freedom for a one-sample \(t\)-test is equal to \(n-1\). It would be good to include this information in your t_test_func
output.
As noted above in 1.1.1, if you want multiple results to be provided as the output of a function, you will need to combine these results into one output object.
There are several ways in which we can do this.
The simplest option is to use the combine function, c()
. For example, if your test statistic \(t\) result is stored in the object t.val
, and your degrees of freedom result is stored in the object df
, you could use c(t.val, df)
to output your results.
Try this now for your t_test_func
function, and then test your updated function, using the inputs discussed in 3.2.
Note that the above approach won’t provide any extra information about your output - that might be fine if you are the only person using your function, but if you intend your function to be used by others, then adding more detail to the output might be desirable.
As a simple extra step, continuing the example above you could use c("test statistic value" = t.val, "degrees of freedom" = df)
to include some helpful additional details about your function output.
Using this information, update your t_test_func
function so that it outputs the test statistic \(t\) and the degrees of freedom, with clear labels.
Test your updated function, using the inputs discussed in 3.2.
Informative Output
As a further improvement, rather than using e.g. c("test statistic value" = t.val, "degrees of freedom" = df)
, we could instead use the concatenate function cat
to provide more detailed output that is easier to read and interpret.
An example application of the cat
function is provided below:
cat("The test statistic is", round(t.val, 4))
This code will output a single character vector, e.g. The test statistic is 1.2247, which is nicer to read than the c
function method we used in 4.1.
Here, anything that is not an R code result is surrounded by quotation marks "..."
. A comma is used to separate sections of R code results and text (or text related code).
If we would like to include several statements in our output, we can use "\n"
to specify a new line. For example, the layout for two lines could be:
cat("Some text here", some R code here, "\n",
"some more text", some more R code)
Using the information above in 4.2, incorporate the cat
function into your t_test_func
so that the output is in a clear, easy to read format.
Test your updated function, using the inputs discussed in 3.2.
\(p\)-value Function
Suppose that we would also like to compute the \(p\)-value that corresponds to the \(t\)-test test statistic. Recall from Topic 5 that for a two-sided hypothesis test, the \(p\)-value is equal to the probability that we observe a test statistic value as extreme, or more extreme, than the observed test statistic we have computed using our sample.
In other words, we have that \[\text{$p$-value} = 1 - P(-t \leq T \leq t) = P(T \leq -t) + P(T \geq t) .\]
Recall that in Computer Lab 4 we discussed the use of functions like dnorm
, pnorm
and qnorm
for computing the density, probability distribution function and quantile value respectively of normally distributed random variables.
Similarly, we can use the inbuilt R function pt
to compute the probability that the test statistic \(T\) (which follows a Student’s \(t\) distribution) takes a value less than or equal to an arbitrary value. We will however also need to include the degrees of freedom as an argument.
For example, if we would like to compute \(P(T \leq 5)\), and our degrees of freedom is 4, we can use the code pt(5, 4)
.
If we would like to compute \(P(T \geq 5)\), we can be clever and use the code 1 - pt(5, 4)
.
Based on this information, add extra code to your t_test_func
function so that it also provides (as a third output) the corresponding \(p\)-value for the test statistic.
Hint: Try to use the symmetry property of the Student’s \(t\) distribution when including code for the \(p\)-value calculations in your function.
Run both your updated t_test_func
function and the inbuilt t.test
R function, using the inputs discussed in 3.2. Verify that your t_test_func
function \(p\)-value is correct, by comparing it to the output of the t.test
R function.
We now have written functions that, combined, can be used to conduct a one-sample \(t\)-test for a two-sided hypothesis. Well done! That concludes our computer lab on R function writing.
Hopefully this lab has demystified R function creation, and shown you that writing functions in R is perhaps easier than you might have thought.
These notes have been prepared by Rupert Kuveke and Amanda Shaker. The copyright for the material in these notes resides with the authors named above, with the Department of Mathematical and Physical Sciences and with La Trobe University. Copyright in this work is vested in La Trobe University including all La Trobe University branding and naming. Unless otherwise stated, material within this work is licensed under a Creative Commons Attribution-Non Commercial-Non Derivatives License
BY-NC-ND.
LS0tDQp0aXRsZTogIlNUTTEwMDE6IENvbXB1dGVyIExhYiA2QiINCm91dHB1dDoNCiAgYm9va2Rvd246Omh0bWxfZG9jdW1lbnQyOiANCiAgICB0b2M6IHRydWUNCiAgICB0b2NfZmxvYXQ6IHRydWUNCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlDQogICAgdGhlbWU6IHJlYWRhYmxlDQogICAgY29kZV9mb2xkaW5nOiBzaG93DQpiaWJsaW9ncmFwaHk6IFNUTTEwMDFfRFNfQ0xfcmVmZXJlbmNlcy5iaWIgDQpsaW5rLWNpdGF0aW9uczogeWVzDQotLS0NCg0KPHN0eWxlPg0KI1RPQyB7DQogIGJhY2tncm91bmQ6IHVybCgiaHR0cHM6Ly93d3cubGF0cm9iZS5lZHUuYXUvX21lZGlhL2xhLXRyb2JlLWFwaS92NS9pbWcvbG9nby5zdmciKTsNCiAgYmFja2dyb3VuZC1zaXplOiBjb250YWluOw0KICBwYWRkaW5nLXRvcDogODBweCAhaW1wb3J0YW50Ow0KICBiYWNrZ3JvdW5kLXJlcGVhdDogbm8tcmVwZWF0Ow0KfQ0KPC9zdHlsZT4NCg0KIyMjIERhdGEgU2NpZW5jZSBTdHJlYW0gey19DQoNCiMjIyBUb3BpYyA2QjogV3JpdGluZyBSIEZ1bmN0aW9ucyB7LX0NCg0KPGJyPg0KDQpXZWxjb21lIHRvIHRoZSBzaXh0aCBjb21wdXRlciBsYWIgZm9yIHRoZSBEYXRhIFNjaWVuY2Ugc3RyZWFtIG9mIFNUTTEwMDEuDQoNCkJ5IG5vdywgeW91IHdpbGwgaGF2ZSBnYWluZWQgc29tZSBleHBlcmllbmNlIHVzaW5nIGEgdmFyaWV0eSBvZiBmdW5jdGlvbnMgaW4gUiwgcmFuZ2luZyBmcm9tIGluYnVpbHQgJ2Jhc2UgUicgZnVuY3Rpb25zLCB0byBwYWNrYWdlLXNwZWNpZmljIGZ1bmN0aW9ucywgc3VjaCBhcyB0aG9zZSBjb250YWluZWQgaW4gdGhlIGBwbG90bHlgIHBhY2thZ2UuDQoNCkluIHRoaXMgY29tcHV0ZXIgbGFiIHdlIHdpbGwgZXh0ZW5kIG91ciBSIGNvZGluZyBza2lsbHMgYW5kIGZvY3VzIG9uIHdyaXRpbmcgb3VyICoqb3duKiogZnVuY3Rpb25zIGluIFIuDQoNCkJ5IHRoZSBlbmQgb2YgdGhpcyBsYWIsIHlvdSBzaG91bGQgdW5kZXJzdGFuZCB0aGUgUiBjb2RlIHN5bnRheCByZXF1aXJlZCB0byBjcmVhdGUgZnVuY3Rpb25zLCBhbmQgZmVlbCBjb21mb3J0YWJsZSB3cml0aW5nIHNpbXBsZSBhbmQgaW50ZXJtZWRpYXRlIGZ1bmN0aW9ucy4gTGV0J3MgZ2V0IHN0YXJ0ZWQhDQoNCiMgUiBGdW5jdGlvbnMgT3ZlcnZpZXcNCg0KVXN1YWxseSwgaWYgeW91IG5lZWQgdG8gY2Fycnkgb3V0IGEgc3BlY2lmaWMgc3RhdGlzdGljYWwgcHJvY2VzcyBpbiBSLCB0aGVyZSB3aWxsIGFscmVhZHkgYmUgYSByZWxldmFudCBmdW5jdGlvbiBmb3IgeW91IHRvIHVzZSENCkhvd2V2ZXIsIHdoZW4geW91IGFyZSBjb25kdWN0aW5nIHlvdXIgb3duIGNvZGluZyBvciByZXNlYXJjaCwgc29tZXRpbWVzIGl0IGNhbiBiZSBleHRyZW1lbHkgYmVuZWZpY2lhbCB0byB3cml0ZSB5b3VyIG93biBjdXN0b20gZnVuY3Rpb25zLiBUaGVzZSBjYW4gcmFuZ2UgZnJvbSBzaW1wbGUgZnVuY3Rpb25zIG9mIGEgZmV3IGxpbmVzIG9mIGNvZGUsIHRvIGV4dHJlbWVseSBkZXRhaWxlZCwgbXVsdGktbGF5ZXJlZCBmdW5jdGlvbnMgdGhhdCBtYWtlIHVzZSBvZiBvdGhlciBmdW5jdGlvbnMgYW5kIHBhY2thZ2VzLiBUaGUgY3JlYXRpdmUgcG9zc2liaWxpdGllcyBhcmUgZW5kbGVzcy4NCg0KIyMgUiBGdW5jdGlvbiBDb21wb3NpdGlvbiB7I2NvbXB9DQoNCkxldCdzIHRha2UgYSBsb29rIGF0IHRoZSBiYXNpYyBjb21wb3NpdGlvbiBvZiBhbiBSIGZ1bmN0aW9uLCB1c2luZyB0aGUgZXhhbXBsZSBmdW5jdGlvbiBiZWxvdzoNCg0KYGBge3IgY2xhc3Muc291cmNlID0gImZvbGQtc2hvdyIsIGV2YWwgPSBULCBlY2hvID0gVCwgd2FybmluZyA9IEYsIG1lc3NhZ2UgPSBGfQ0KbXlfc2ltcGxlX2Z1bmN0aW9uIDwtIGZ1bmN0aW9uKGFyZ3VtZW50MSwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYXJndW1lbnQyKXsNCiAgICANCiAgb3V0cHV0IDwtIGFyZ3VtZW50MSArIGFyZ3VtZW50Mg0KICANCn0NCmBgYA0KDQpUaGUgY29kZSBhYm92ZSBjb25zaXN0cyBvZiBmb3VyIG1haW4gcGFydHM6DQoNCiogVGhlIGZ1bmN0aW9uIGhhcyBhIG5hbWUsIGBteV9zaW1wbGVfZnVuY3Rpb25gLg0KKiBXZSB1c2UgdGhlIGBmdW5jdGlvbmAgZnVuY3Rpb24gdG8gKHVuc3VycHJpc2luZ2x5KSBjcmVhdGUgYSBmdW5jdGlvbi4NCiogSW5zaWRlIHRoZSBvcGVuIGJyYWNrZXRzIGAoKWAgZm9sbG93aW5nIGBmdW5jdGlvbmAsIHdlIHNwZWNpZnkgYGFyZ3VtZW50MWAgYW5kIGBhcmd1bWVudDJgIHRvIGJlIHRoZSBhcmd1bWVudHMsIGkuZS4gaW5wdXRzLCBvZiBvdXIgZnVuY3Rpb24uDQoqIFRvIGJlZ2luIHRoZSAnbWFpbiBib2R5JyBvZiBvdXIgZnVuY3Rpb24sIHdlIHVzZSBhIGxlZnQgY3VybHkgYnJhY2UsIGB7YCwgZGlyZWN0bHkgYWZ0ZXIgdGhlIHJpZ2h0IG9wZW4gYnJhY2tldCBgKWAuIA0KKiBJbiB0aGUgJ21haW4gYm9keScgb2YgdGhlIGZ1bmN0aW9uLCB3ZSBzcGVjaWZ5IHRoZSBjYWxjdWxhdGlvbnMgdGhlIGZ1bmN0aW9uIHNob3VsZCBwZXJmb3JtIC0gZm9yIGBteV9zaW1wbGVfZnVuY3Rpb25gLCB3ZSBhcmUgc2ltcGx5IGFkZGluZyB0aGUgYGFyZ3VtZW50MWAgYW5kIGBhcmd1bWVudDJgIHZhbHVlcyBwcm92aWRlZC4gVGhpcyBzdW0gaXMgc3RvcmVkIGluIHRoZSBvYmplY3QgYG91dHB1dGAsIGFuZCB3aGVuIHRoZSBmdW5jdGlvbiBpcyBjb21wdXRlZCwgdGhlIGBvdXRwdXRgIHZhbHVlIGlzIHByb3ZpZGVkIGFzIG91dHB1dC4gDQoqIFRvIHdyYXAgdXAgb3VyIGZ1bmN0aW9uLCB3ZSBjb25jbHVkZSB3aXRoIGEgcmlnaHQgY3VybHkgYnJhY2UsIGB9YC4NCg0KWW91IG1pZ2h0IGJlIHdvbmRlcmluZyB3aGF0IGBhcmd1bWVudDFgIGFuZCBgYXJndW1lbnQyYCBhcmUsIGFuZCBmcm9tIHdoZXJlIHRoZXkgY2FtZS4gSW4gZmFjdCwgdGhlc2UgYXJlIGFyYml0cmFyeSBuYW1lcywgYW5kIGRvbid0IGNvcnJlc3BvbmQgdG8gYW55IHNwZWNpZmljIG51bWJlcnMuIFdlIGNhbiBwcm92aWRlIHdoYXRldmVyIG51bWJlcnMgb3Igc3RyaW5nIG9mIG51bWJlcnMgd2Ugd291bGQgbGlrZSB0byBvdXIgZnVuY3Rpb24sIGFuZCB3aXRoaW4gdGhlIGBteV9zaW1wbGVfZnVuY3Rpb25gIGVudmlyb25tZW50IHRoZXNlIHZhbHVlcyB3aWxsIGJlIHVzZWQgaW4gcGxhY2Ugb2YgYGFyZ3VtZW50MWAgYW5kIGBhcmd1bWVudDJgLiBGb3IgZXhhbXBsZSwgc3VwcG9zZSB3ZSB3b3VsZCBsaWtlIHRvIGFkZCAzOSBhbmQgMy4gV2UgY2FuIGRvIHRoaXMgdXNpbmcgb3VyIGZ1bmN0aW9uIGFzIGZvbGxvd3M6DQoNCmBgYHtyIGV2YWwgPSBULCBlY2hvID0gVH0NCnJlcyA8LSBteV9zaW1wbGVfZnVuY3Rpb24oMzksIDMpDQpyZXMNCmBgYA0KDQpDb3B5IHRoZSBjb2RlIGluIHRoZSB0d28gY29kZSBjaHVua3MgYWJvdmUsIHBhc3RlIGl0IGludG8gYW4gUiBzY3JpcHQsIGFuZCBydW4gaXQuIFlvdSBzaG91bGQgb2J0YWluIHRoZSByZXN1bHQgYDQyYCBzaG93biBhYm92ZS4NCg0KKk9mIGNvdXJzZSwgd2UgZGlkbid0IHJlYWxseSBuZWVkIHRvIHVzZSBhIGZ1bmN0aW9uIGZvciB0aGlzIC0gd2UgY291bGQgaGF2ZSBzaW1wbHkgY29tcHV0ZWQgYDM5ICsgM2AgaW4gUi4gSW4gZ2VuZXJhbCwgZnVuY3Rpb25zIHdpbGwgYmUgYSBsaXR0bGUgbW9yZSBjb21wbGljYXRlZCB0aGFuIGBteV9zaW1wbGVfZnVuY3Rpb25gLioNCg0KIyMjIEltcG9ydGFudCBOb3RlcyB7I2ltcH0NCg0KU3RyaWN0bHkgc3BlYWtpbmcsIHdlIGRpZCBub3QgbmVlZCB0byBpbmNsdWRlIHRoZSBgb3V0cHV0IDwtYCBjb2RlIHdpdGhpbiBgbXlfc2ltcGxlX2Z1bmN0aW9uYC4gDQoNClIgd2lsbCBvdXRwdXQgdGhlIGxhc3QgZXZhbHVhdGVkIGV4cHJlc3Npb24gaW4gdGhlIGZ1bmN0aW9uLCBzbyB3ZSBjb3VsZCBzaW1wbHkgaGF2ZSB1c2VkIGBhcmd1bWVudDEgKyBhcmd1bWVudDJgIGluIHRoZSBtYWluIGJvZHkgb2YgdGhlIGZ1bmN0aW9uLiAqKklmIGhvd2V2ZXIgd2Ugd2FudGVkIHRvIG91dHB1dCBtb3JlIHRoYW4gb25lIHJlc3VsdCwgdGhlbiBzcGVjaWZ5aW5nIG91ciBkZXNpcmVkIG91dHB1dCBjYW4gYmUgaW5mb3JtYXRpdmUgYW5kIGhlbHAgYXZvaWQgZXJyb3JzKiouDQoNClRoaXMgYWRkaXRpb25hbCBgb3V0cHV0IDwtYCBjb2RlIHdhcyBwcmltYXJpbHkgaW5jbHVkZWQgaGVyZSB0byBkZW1vbnN0cmF0ZSBhIGtleSBmZWF0dXJlIG9mIFIgZnVuY3Rpb25zIC0gYW55IG9iamVjdHMgY3JlYXRlZCB3aXRoaW4gYSBmdW5jdGlvbiBhcmUgcGFydCBvZiB0aGF0IGZ1bmN0aW9uJ3MgYGxvY2FsIGVudmlyb25tZW50YCwgbWVhbmluZyB0aGF0ICoqdGhleSBleGlzdCBvbmx5IGluc2lkZSB0aGF0IGZ1bmN0aW9uKiosIGFuZCBjYW4ndCBiZSBjYWxsZWQgZnJvbSB0aGUgYGdsb2JhbCBlbnZpcm9ubWVudGAgb3V0c2lkZSB0aGUgZnVuY3Rpb24uDQoNClRvIGNsYXJpZnkgdGhlIGRpc3RpbmN0aW9uLCB0YWtlIGEgbG9vayBhdCB0aGUgY29kZSBiZWxvdzoNCg0KYGBge3IgY2xhc3Muc291cmNlID0gImZvbGQtc2hvdyIsIGV2YWwgPSBULCBlY2hvID0gVCwgd2FybmluZyA9IEYsIG1lc3NhZ2UgPSBGfQ0KdGVzdCA8LSAzOSArIDMNCnRlc3QNCmBgYA0KDQpIZXJlLCB3ZSBhc3NpZ24gdGhlIHN1bSBgMzkgKyAzYCB0byB0aGUgb2JqZWN0IGB0ZXN0YC4gVGhpcyBpcyBub3QgZG9uZSB3aXRoaW4gYSBmdW5jdGlvbiwgc28gdGhlIG9iamVjdCBgdGVzdGAgaXMgc3RvcmVkIGluIHRoZSBgZ2xvYmFsIGVudmlyb25tZW50YC4gVGhpcyBtZWFucyB3ZSBjYW4gdGhlbiBjYWxsIHRoZSBgdGVzdGAgb2JqZWN0IGF0IGEgbGF0ZXIgZGF0ZSwgYW5kIHNlZSBvdXIgcmVzdWx0IG9mIGA0MmAuDQoNClNpbWlsYXJseSwgd2UgY291bGQgY29tcHV0ZSB0aGlzIHJlc3VsdCB1c2luZyBgbXlfc2ltcGxlX2Z1bmN0aW9uYCwgYXMgc2hvd24gYWJvdmUgaW4gXEByZWYoY29tcCkuIA0KDQpIb3dldmVyLCB0cnkgdG8gbm93IGNhbGwgdGhlIG9iamVjdCBgb3V0cHV0YCAod2hpY2ggaXMgZGVmaW5lZCB3aXRoaW4gdGhpcyBmdW5jdGlvbikuIFlvdSB3aWxsIHJlY2VpdmUgYW4gZXJyb3IgLSBzb21ldGhpbmcgYWxvbmcgdGhlIGxpbmVzIG9mIGBFcnJvciBpbiBldmFsKGV4cHIsIGVudmlyLCBlbmNsb3MpIDogb2JqZWN0ICdvdXRwdXQnIG5vdCBmb3VuZGAuDQoNClRoaXMgaXMgc29tZXRoaW5nIHdvcnRoIGtlZXBpbmcgaW4gbWluZC4gSWYgeW91IGludGVuZCB0byB1c2UgYW4gb2JqZWN0IG91dHNpZGUgb2YgYSBmdW5jdGlvbiwgaXQgbWlnaHQgYmUgd29ydGggZGVmaW5pbmcgaXQgc2VwYXJhdGVseS4gSXQgaXMgYWxzbyBhIGdvb2QgaWRlYSB0byB1c2UgZGlmZmVyZW50IG5hbWVzIGZvciBhbGwgeW91ciBvYmplY3RzIChpbnNpZGUgYW5kIG91dHNpZGUgZnVuY3Rpb25zKSB0byBhdm9pZCBjb25mdXNpb24uDQoNCiMgV3JpdGluZyBTaW1wbGUgRnVuY3Rpb25zIGluIFIgeyN2YWx1ZXN9DQoNCk5vdyB0aGF0IHdlIGhhdmUgY292ZXJlZCB0aGUgYmFzaWNzIG9mIFIgZnVuY3Rpb24gY29tcG9zaXRpb24sIGxldCdzIHRyeSB0byB3cml0ZSBzb21lIHNpbXBsZSBmdW5jdGlvbnMgb3Vyc2VsdmVzLiBUaGVzZSBmdW5jdGlvbnMgYWN0dWFsbHkgYWxyZWFkeSBleGlzdCBpbiBiYXNlIFIsIGJ1dCB0aGlzIHdpbGwgYmUgZ29vZCBwcmFjdGljZSBmb3Igc3Vic2VxdWVudCB3b3JrLg0KDQojIyBNZWFuIEZ1bmN0aW9uIHsjbWVhbmZ1bmN9DQoNClJlY2FsbCB0aGF0IHRoZSBtZWFuICRcYmFye3h9JCBvZiBhIHNldCBvZiAkbiQgdmFsdWVzICR4XzEsIHhfMiwgXGNkb3RzLCB4X24kIGlzIGNvbXB1dGVkIHVzaW5nIHRoZSBmb3JtdWxhICQkXGJhcnt4fSA9IFxkaXNwbGF5c3R5bGUgXGZyYWN7XHN1bV97aT0xfV5uIHhfaX17bn0gLiQkDQoNClN1cHBvc2UgdGhhdCB5b3Ugd291bGQgbGlrZSB0byB3cml0ZSBhbiBSIGZ1bmN0aW9uIHRoYXQgY29tcHV0ZXMgdGhlICoqbWVhbioqIG9mIGEgc2V0IG9mIHByb3ZpZGVkIHZhbHVlcy4gDQpUaGVyZSBhcmUgc2V2ZXJhbCB3YXlzIHRoYXQgd2UgY291bGQgd3JpdGUgdGhpcyBmdW5jdGlvbi4gV2UgaGF2ZSBtYWRlIGEgaGVhZC1zdGFydCB3aXRoIHRoZSBwYXJ0aWFsbHkgY29tcGxldGVkIGNvZGUgYmVsb3cuIFNlZSBpZiB5b3UgY2FuIGNvbXBsZXRlIGl0IGJ5IGZpbGxpbmcgaW4gdGhlIG1pc3NpbmcgYC4uLmAgcGFydHMuDQoNCmBgYHtyIGNsYXNzLnNvdXJjZSA9ICJmb2xkLXNob3ciLCBldmFsID0gRiwgZWNobyA9IFQsIHdhcm5pbmcgPSBGLCBtZXNzYWdlID0gRn0NCm1lYW5fZnVuYyA8LSBmdW5jdGlvbih2YWx1ZXMpew0KICAjIEFyZ3VtZW50Og0KICAjIHZhbHVlczogVGhpcyBpcyBvdXIgbGlzdCBvZiB2YWx1ZXMNCiAgDQogIG4gPC0gbGVuZ3RoKHZhbHVlcykNCiAgc3VtKC4uLikgLyAuLi4NCg0KfSAgICAgICAgICAgICAgICAgICAgICANCmBgYA0KDQoqTm90ZSB0aGF0LCBiYXNlZCBvbiB0aGlzIGZvcm1hdCwgb3VyIGZ1bmN0aW9uIGNhbiBjb21wdXRlIHRoZSBtZWFuIG9mIGEgc3RyaW5nIG9mIG51bWJlcnMgb2YgYW55IG5vbi16ZXJvIGxlbmd0aC4qDQoNCiMjIw0KDQpWZXJpZnkgdGhhdCB5b3VyIGBtZWFuX2Z1bmNgIGZ1bmN0aW9uIHdvcmtzIGJ5IGNvbXB1dGluZyB0aGUgbWVhbiBvZiB0aGUgc2V0IG9mIHZhbHVlcyBgMiwgMywgNCwgNSwgNiwgNywgOGAuIElmIHlvdXIgZnVuY3Rpb24gaXMgY29ycmVjdCwgeW91IHNob3VsZCByZWNlaXZlIGFuIGFuc3dlciBvZiBgNWAuDQoNCiMjIFNhbXBsZSBzdGFuZGFyZCBkZXZpYXRpb24gRnVuY3Rpb24NCg0KTm93IHRoYXQgeW91IGhhdmUgaGFkIHNvbWUgcHJhY3RpY2Ugd3JpdGluZyBSIGZ1bmN0aW9ucywgbGV0J3MgdGFja2xlIGEgaGFyZGVyIHByb2JsZW0uDQoNClJlY2FsbCB0aGUgZXF1YXRpb24gZm9yIHRoZSBzYW1wbGUgc3RhbmRhcmQgZGV2aWF0aW9uICRzJCAoaW50cm9kdWNlZCBpbiBbVG9waWMgMl0oaHR0cHM6Ly9ib29rZG93bi5vcmcvYV9zaGFrZXIvU1RNMTAwMV9Ub3BpY18yLzItMS12YXJpYW5jZS1hbmQtc3RhbmRhcmQtZGV2aWF0aW9uLmh0bWwpKToNCg0KJCRzID0gXGRpc3BsYXlzdHlsZSBcc3FydHsgXGZyYWN7MX17bi0xfSBcc3VtX3tpPTF9Xm4gKHhfaSAtIFxiYXJ7eH0pXjJ9IC4kJA0KVXNpbmcgeW91ciBgbWVhbl9mdW5jYCBmdW5jdGlvbiwgYW5kIHlvdXIgY29kZSBmcm9tIFxAcmVmKG1lYW5mdW5jKSBhcyBhIGd1aWRlLCB3cml0ZSBhIGZ1bmN0aW9uIGNhbGxlZCBgc2RfZnVuY2AgdGhhdCBjb21wdXRlcyB0aGUgKipzYW1wbGUgc3RhbmRhcmQgZGV2aWF0aW9uKiogb2YgYSBzZXQgb2YgJG4kIHZhbHVlcyAkeF8xLCB4XzIsIFxjZG90cywgeF9uJC4NCg0KKkhpbnRzOiogDQoNCiogKllvdSBjYW4gdXNlIHRoZSBzYW1lIGlucHV0IGFzIGZvciB5b3VyIGBtZWFuX2Z1bmNgIGZ1bmN0aW9uLioNCiogKldoZW4geW91IGFyZSBtdWx0aXBseWluZyBwYXJ0cyBvZiBhIGZ1bmN0aW9uIHRvZ2V0aGVyLCB5b3UgbmVlZCB0byB1c2UgdGhlICRcYXN0JCBzeW1ib2wuKg0KKiAqWW91IGNhbiB1c2UgdGhlIGJ1aWx0LWluIFIgZnVuY3Rpb24gYHNxcnQoKWAgdG8gY29tcHV0ZSB0aGUgc3F1YXJlIHJvb3QuKg0KKiAqWW91IGNhbiB1c2UgeW91ciBgbWVhbl9mdW5jYCBmdW5jdGlvbiB3aXRoaW4geW91ciBuZXcgZnVuY3Rpb24sIHdoZW4gY29tcHV0aW5nIHRoZSAkXGJhcnt4fSQgdmFsdWUuKg0KKiAqQmUgY2FyZWZ1bCB0byBpbmNsdWRlIGJyYWNrZXRzIHRvIHNlcGFyYXRlIGRpZmZlcmVudCBwYXJ0cyBvZiB5b3VyIGNhbGN1bGF0aW9ucy4qDQoqICpEb24ndCBmb3JnZXQgdGhlIHNxdWFyZSB0ZXJtIGluIHRoZSBlcXVhdGlvbi4qDQoNCiMjIw0KDQpWZXJpZnkgdGhhdCB5b3VyIHNhbXBsZSBzdGFuZGFyZCBkZXZpYXRpb24gZnVuY3Rpb24gd29ya3MgYnkgY29tcHV0aW5nIHRoZSBzYW1wbGUgc3RhbmRhcmQgZGV2aWF0aW9uIG9mIHRoZSBzZXQgb2YgdmFsdWVzIGAyLCAzLCA0LCA1LCA2LCA3LCA4YC4gSWYgeW91ciBmdW5jdGlvbiBpcyBjb3JyZWN0LCB5b3Ugc2hvdWxkIHJlY2VpdmUgYW4gYW5zd2VyIG9mIGAyLjE2MDI0N2AuICANCg0KKk5vdGU6IFlvdXIgYW5zd2VyIG1heSBiZSBzbGlnaHRseSBkaWZmZXJlbnQgZHVlIHRvIHJvdW5kaW5nLioNCg0KIyBXcml0aW5nIGEgJHQkLXRlc3QgRnVuY3Rpb24gaW4gUiB7I3R0ZXN0ZnVuY30NCg0KTm93IHRoYXQgd2UgaGF2ZSBoYWQgc29tZSBwcmFjdGljZSB3cml0aW5nIFIgZnVuY3Rpb25zLCBsZXQncyBtb3ZlIG9uIHRvIGEgc2xpZ2h0bHkgaGFyZGVyIHNjZW5hcmlvLCBhbmQgdHJ5IHRvIHdyaXRlIGFuIFIgZnVuY3Rpb24gZm9yIGNvbXB1dGluZyBhICR0JC10ZXN0Lg0KDQpSZWNhbGwgdGhhdCBpbiBbVG9waWMgNV0oaHR0cHM6Ly9ib29rZG93bi5vcmcvY29udGVudC81MGEzMTc4ZC01NDMyLTQ0YTMtYmI1Yy0xNzE4Y2EzZTFmZTIvMS10ZGlzdC5odG1sKSB3ZSBpbnRyb2R1Y2VkIHRoZSAkdCQtdGVzdCBhcyBhIGNvbW1vbiB0eXBlIG9mIGh5cG90aGVzaXMgdGVzdC4NClRoZSB0ZXN0IHN0YXRpc3RpYyAkdCQgdXNlZCBmb3IgdGhlIG9uZS1zYW1wbGUgJHQkLXRlc3QgaGFzIHRoZSBmb2xsb3dpbmcgZm9ybXVsYToNCg0KJCR0ID0gXGRpc3BsYXlzdHlsZSBcZnJhY3tcYmFye3h9IC0gXG11XzB9e3MgLyBcc3FydHtufX0sJCQNCndoZXJlICRcYmFye3h9JCwgJHMkIGFuZCAkbiQgYXJlIHRoZSBzYW1wbGUgbWVhbiwgc2FtcGxlIHN0YW5kYXJkIGRldmlhdGlvbiwgYW5kIHNhbXBsZSBzaXplIHJlc3BlY3RpdmVseSwgYXMgcHJldmlvdXNseSBkZWZpbmVkLCBhbmQgJFxtdV8wJCBkZW5vdGVzIHRoZSBtZWFuIHVuZGVyIHRoZSBudWxsIGh5cG90aGVzaXMuDQoNCipOb3RlOiAkcyAvIFxzcXJ0e259JCBpcyB0aGUgc3RhbmRhcmQgZXJyb3Igb2YgJHQkLCBhbmQgaXMgb2Z0ZW4gZGVub3RlZCBieSAkU0UkLioNCg0KIyMgJHQkLXRlc3QgdGVzdCBzdGF0aXN0aWMNCg0KVXNpbmcgeW91ciBmdW5jdGlvbnMgYG1lYW5fZnVuY2AgYW5kIGBzZF9mdW5jYCwgd3JpdGUgYSBuZXcgUiBmdW5jdGlvbiBjYWxsZWQgYHRfdGVzdF9mdW5jYCB0aGF0IGNvbXB1dGVzIHRoZSB0ZXN0IHN0YXRpc3RpYyAkdCQgZm9yIGEgb25lLXNhbXBsZSAkdCQtdGVzdCB3aXRoIGFuIGFyYml0cmFyeSBudWxsIGh5cG90aGVzaXMgbWVhbiAkXG11XzAkLg0KDQoqSGludHM6Kg0KDQoqICpNb3N0IG9mIHRoZSBoYXJkIHdvcmsgaXMgYWxyZWFkeSBkb25lLCBzaW5jZSB5b3UgY2FuIHVzZSB5b3VyIHR3byBmdW5jdGlvbnMgYG1lYW5fZnVuY2AgYW5kIGBzZF9mdW5jYC4qDQoqICpZb3Ugd2lsbCBuZWVkIHRvIHByb3ZpZGUgb25lIG5ldyBpbnB1dCBhcmd1bWVudCwgdGhlICRcbXVfMCQgdmFsdWUsIGZvciB5b3VyIGB0X3Rlc3RfZnVuY2AgZnVuY3Rpb24uKg0KDQojIyB7I3R0ZXN0fQ0KDQpUbyB2ZXJpZnkgeW91ciB0ZXN0IHN0YXRpc3RpYyBmdW5jdGlvbidzIGFjY3VyYWN5LCBldmFsdWF0ZSBpdCBmb3IgdGhlIHNldCBvZiB2YWx1ZXMgYDIsIDMsIDQsIDUsIDYsIDcsIDhgLCB3aXRoIHRoZSBudWxsIGh5cG90aGVzaXMgbWVhbiAkXG11XzAgPSA0JC4NCg0KQ29tcGFyZSB5b3VyIHJlc3VsdCB0byB0aGUgcmVzdWx0IG9mIHRoZSBpbmJ1aWx0IFIgZnVuY3Rpb24gYHQudGVzdGAsIGJ5IHJ1bm5pbmcgdGhlIGNvZGUgYHQudGVzdChjKDI6OCksIG11ID0gNClgLiAgSWYgeW91IGhhdmUgd3JpdHRlbiB5b3VyIGZ1bmN0aW9uIGNvcnJlY3RseSwgdGhlIGB0LnRlc3RgICR0PSQgb3V0cHV0IHNob3VsZCBiZSBpZGVudGljYWwgdG8gdGhlIHZhbHVlIG9mICR0JCBjb21wdXRlZCBieSB5b3VyIGB0X3Rlc3RfZnVuY2AgZnVuY3Rpb24uDQoNCipIaW50OiBEb24ndCB3b3JyeSBpZiB5b3VyIGB0X3Rlc3RfZnVuY2AgZnVuY3Rpb24gZ2l2ZXMgYSBzbGlnaHRseSBkaWZmZXJlbnQgdmFsdWUgLSBhcyBsb25nIGFzIGl0IGlzIGNvcnJlY3QgdG8gMyBvciA0IGRlY2ltYWwgcGxhY2VzIHRoYXQgaXMgZmluZS4gSWYgaXQgaXMgdmVyeSBkaWZmZXJlbnQsIGp1c3QgZ28gYmFjayBvdmVyIHlvdXIgZnVuY3Rpb24sIGFuZCBjaGVjayBmb3IgZXJyb3JzIHN1Y2ggYXMgaW5zdWZmaWNpZW50IG9yIG1pc3BsYWNlZCBicmFja2V0cy4gSXQncyB2ZXJ5IGVhc3kgdG8gbWFrZSBtaXN0YWtlcyBsaWtlIHRoaXMuKg0KDQojIEFkZGluZyBEZXRhaWxzIHRvIG91ciAkdCQtdGVzdCBGdW5jdGlvbg0KDQpUbyBjb25jbHVkZSB0aGlzIGNvbXB1dGVyIGxhYiwgbGV0J3MgdHJ5IGFkZGluZyBzb21lIGFkZGl0aW9uYWwgZGV0YWlscyBhbmQgZmVhdHVyZXMgdG8gb3VyICR0JC10ZXN0IGZ1bmN0aW9uLCB0byBtYWtlIGl0IG1vcmUgaW5mb3JtYXRpdmUuDQoNCiMjIERlZ3JlZXMgb2YgZnJlZWRvbSB7I2RmfQ0KDQpSZWNhbGwgdGhhdCB0aGUgZGVncmVlcyBvZiBmcmVlZG9tIGZvciBhIG9uZS1zYW1wbGUgJHQkLXRlc3QgaXMgZXF1YWwgdG8gJG4tMSQuIEl0IHdvdWxkIGJlIGdvb2QgdG8gaW5jbHVkZSB0aGlzIGluZm9ybWF0aW9uIGluIHlvdXIgYHRfdGVzdF9mdW5jYCBvdXRwdXQuDQoNCkFzIG5vdGVkIGFib3ZlIGluIFxAcmVmKGltcCksIGlmIHlvdSB3YW50IG11bHRpcGxlIHJlc3VsdHMgdG8gYmUgcHJvdmlkZWQgYXMgdGhlIG91dHB1dCBvZiBhIGZ1bmN0aW9uLCB5b3Ugd2lsbCBuZWVkIHRvIGNvbWJpbmUgdGhlc2UgcmVzdWx0cyBpbnRvIG9uZSBvdXRwdXQgb2JqZWN0LiANCg0KVGhlcmUgYXJlIHNldmVyYWwgd2F5cyBpbiB3aGljaCB3ZSBjYW4gZG8gdGhpcy4NCg0KVGhlIHNpbXBsZXN0IG9wdGlvbiBpcyB0byB1c2UgdGhlIGNvbWJpbmUgZnVuY3Rpb24sIGBjKClgLiBGb3IgZXhhbXBsZSwgaWYgeW91ciB0ZXN0IHN0YXRpc3RpYyAkdCQgcmVzdWx0IGlzIHN0b3JlZCBpbiB0aGUgb2JqZWN0IGB0LnZhbGAsIGFuZCB5b3VyIGRlZ3JlZXMgb2YgZnJlZWRvbSByZXN1bHQgaXMgc3RvcmVkIGluIHRoZSBvYmplY3QgYGRmYCwgeW91IGNvdWxkIHVzZSBgYyh0LnZhbCwgZGYpYCB0byBvdXRwdXQgeW91ciByZXN1bHRzLiANCg0KVHJ5IHRoaXMgbm93IGZvciB5b3VyIGB0X3Rlc3RfZnVuY2AgZnVuY3Rpb24sIGFuZCB0aGVuIHRlc3QgeW91ciB1cGRhdGVkIGZ1bmN0aW9uLCB1c2luZyB0aGUgaW5wdXRzIGRpc2N1c3NlZCBpbiBcQHJlZih0dGVzdCkuDQoNCiMjIw0KDQpOb3RlIHRoYXQgdGhlIGFib3ZlIGFwcHJvYWNoIHdvbid0IHByb3ZpZGUgYW55IGV4dHJhIGluZm9ybWF0aW9uIGFib3V0IHlvdXIgb3V0cHV0IC0gdGhhdCBtaWdodCBiZSBmaW5lIGlmIHlvdSBhcmUgdGhlIG9ubHkgcGVyc29uIHVzaW5nIHlvdXIgZnVuY3Rpb24sIGJ1dCBpZiB5b3UgaW50ZW5kIHlvdXIgZnVuY3Rpb24gdG8gYmUgdXNlZCBieSBvdGhlcnMsIHRoZW4gYWRkaW5nIG1vcmUgZGV0YWlsIHRvIHRoZSBvdXRwdXQgbWlnaHQgYmUgZGVzaXJhYmxlLg0KDQpBcyBhIHNpbXBsZSBleHRyYSBzdGVwLCBjb250aW51aW5nIHRoZSBleGFtcGxlIGFib3ZlIHlvdSBjb3VsZCB1c2UgYGMoInRlc3Qgc3RhdGlzdGljIHZhbHVlIiA9IHQudmFsLCAiZGVncmVlcyBvZiBmcmVlZG9tIiA9IGRmKWAgdG8gaW5jbHVkZSBzb21lIGhlbHBmdWwgYWRkaXRpb25hbCBkZXRhaWxzIGFib3V0IHlvdXIgZnVuY3Rpb24gb3V0cHV0Lg0KDQpVc2luZyB0aGlzIGluZm9ybWF0aW9uLCB1cGRhdGUgeW91ciBgdF90ZXN0X2Z1bmNgIGZ1bmN0aW9uIHNvIHRoYXQgaXQgb3V0cHV0cyB0aGUgdGVzdCBzdGF0aXN0aWMgJHQkIGFuZCB0aGUgZGVncmVlcyBvZiBmcmVlZG9tLCB3aXRoIGNsZWFyIGxhYmVscy4NCg0KVGVzdCB5b3VyIHVwZGF0ZWQgZnVuY3Rpb24sIHVzaW5nIHRoZSBpbnB1dHMgZGlzY3Vzc2VkIGluIFxAcmVmKHR0ZXN0KS4NCg0KIyMgSW5mb3JtYXRpdmUgT3V0cHV0IHsjY2F0fQ0KDQpBcyBhIGZ1cnRoZXIgaW1wcm92ZW1lbnQsIHJhdGhlciB0aGFuIHVzaW5nIGUuZy4gYGMoInRlc3Qgc3RhdGlzdGljIHZhbHVlIiA9IHQudmFsLCAiZGVncmVlcyBvZiBmcmVlZG9tIiA9IGRmKWAsIHdlIGNvdWxkIGluc3RlYWQgdXNlIHRoZSBjb25jYXRlbmF0ZSBmdW5jdGlvbiBgY2F0YCB0byBwcm92aWRlIG1vcmUgZGV0YWlsZWQgb3V0cHV0IHRoYXQgaXMgZWFzaWVyIHRvIHJlYWQgYW5kIGludGVycHJldC4NCg0KQW4gZXhhbXBsZSBhcHBsaWNhdGlvbiBvZiB0aGUgYGNhdGAgZnVuY3Rpb24gaXMgcHJvdmlkZWQgYmVsb3c6XlsqTm90ZTogVGhpcyBicmllZiBpbnRyb2R1Y3Rpb24gZG9lcyBub3QgY292ZXIgYWxsIGFzcGVjdHMgb2YgdGhlIGNvbmNhdGVuYXRlIGZ1bmN0aW9uLipdDQoNCmBgYHtyIGNsYXNzLnNvdXJjZSA9ICJmb2xkLXNob3ciLCBldmFsID0gRiwgZWNobyA9IFQsIHdhcm5pbmcgPSBGLCBtZXNzYWdlID0gRn0NCmNhdCgiVGhlIHRlc3Qgc3RhdGlzdGljIGlzIiwgcm91bmQodC52YWwsIDQpKQ0KYGBgDQoNClRoaXMgY29kZSB3aWxsIG91dHB1dCBhIHNpbmdsZSBjaGFyYWN0ZXIgdmVjdG9yLCBlLmcuICpUaGUgdGVzdCBzdGF0aXN0aWMgaXMgMS4yMjQ3Kiwgd2hpY2ggaXMgbmljZXIgdG8gcmVhZCB0aGFuIHRoZSBgY2AgZnVuY3Rpb24gbWV0aG9kIHdlIHVzZWQgaW4gXEByZWYoZGYpLg0KDQpIZXJlLCBhbnl0aGluZyB0aGF0IGlzIG5vdCBhbiBSIGNvZGUgcmVzdWx0IGlzIHN1cnJvdW5kZWQgYnkgcXVvdGF0aW9uIG1hcmtzIGAiLi4uImAuIEEgY29tbWEgaXMgdXNlZCB0byBzZXBhcmF0ZSBzZWN0aW9ucyBvZiBSIGNvZGUgcmVzdWx0cyBhbmQgdGV4dCAob3IgdGV4dCByZWxhdGVkIGNvZGUpLg0KDQpJZiB3ZSB3b3VsZCBsaWtlIHRvIGluY2x1ZGUgc2V2ZXJhbCBzdGF0ZW1lbnRzIGluIG91ciBvdXRwdXQsIHdlIGNhbiB1c2UgYCJcbiJgIHRvIHNwZWNpZnkgYSBuZXcgbGluZS4gRm9yIGV4YW1wbGUsIHRoZSBsYXlvdXQgZm9yIHR3byBsaW5lcyBjb3VsZCBiZToNCg0KYGBge3IgY2xhc3Muc291cmNlID0gImZvbGQtc2hvdyIsIGV2YWwgPSBGLCBlY2hvID0gVCwgd2FybmluZyA9IEYsIG1lc3NhZ2UgPSBGfQ0KY2F0KCJTb21lIHRleHQgaGVyZSIsIHNvbWUgUiBjb2RlIGhlcmUsICJcbiIsIA0KICAgICJzb21lIG1vcmUgdGV4dCIsIHNvbWUgbW9yZSBSIGNvZGUpDQpgYGANCg0KIyMjIA0KDQpVc2luZyB0aGUgaW5mb3JtYXRpb24gYWJvdmUgaW4gXEByZWYoY2F0KSwgaW5jb3Jwb3JhdGUgdGhlIGBjYXRgIGZ1bmN0aW9uIGludG8geW91ciBgdF90ZXN0X2Z1bmNgIHNvIHRoYXQgdGhlIG91dHB1dCBpcyBpbiBhIGNsZWFyLCBlYXN5IHRvIHJlYWQgZm9ybWF0Lg0KDQpUZXN0IHlvdXIgdXBkYXRlZCBmdW5jdGlvbiwgdXNpbmcgdGhlIGlucHV0cyBkaXNjdXNzZWQgaW4gXEByZWYodHRlc3QpLg0KDQojIyAkcCQtdmFsdWUgRnVuY3Rpb24NCg0KU3VwcG9zZSB0aGF0IHdlIHdvdWxkIGFsc28gbGlrZSB0byBjb21wdXRlIHRoZSAkcCQtdmFsdWUgdGhhdCBjb3JyZXNwb25kcyB0byB0aGUgJHQkLXRlc3QgdGVzdCBzdGF0aXN0aWMuIFJlY2FsbCBmcm9tIFtUb3BpYyA1XShodHRwczovL2Jvb2tkb3duLm9yZy9jb250ZW50LzUwYTMxNzhkLTU0MzItNDRhMy1iYjVjLTE3MThjYTNlMWZlMi8zLTEtZXhwbGFpbmluZy10aGUtb25lLXNhbXBsZS10LXRlc3QtcmVzdWx0cy5odG1sKSB0aGF0IGZvciBhIHR3by1zaWRlZCBoeXBvdGhlc2lzIHRlc3QsIHRoZSAkcCQtdmFsdWUgaXMgZXF1YWwgdG8gdGhlIHByb2JhYmlsaXR5IHRoYXQgd2Ugb2JzZXJ2ZSBhIHRlc3Qgc3RhdGlzdGljIHZhbHVlIGFzIGV4dHJlbWUsIG9yIG1vcmUgZXh0cmVtZSwgdGhhbiB0aGUgb2JzZXJ2ZWQgdGVzdCBzdGF0aXN0aWMgd2UgaGF2ZSBjb21wdXRlZCB1c2luZyBvdXIgc2FtcGxlLiANCg0KSW4gb3RoZXIgd29yZHMsIHdlIGhhdmUgdGhhdCAkJFx0ZXh0eyRwJC12YWx1ZX0gPSAxIC0gUCgtdCBcbGVxIFQgXGxlcSB0KSA9IFAoVCBcbGVxIC10KSArIFAoVCBcZ2VxIHQpIC4kJA0KDQpSZWNhbGwgdGhhdCBpbiBbQ29tcHV0ZXIgTGFiIDRdKGh0dHBzOi8vcnB1YnMuY29tL0xUVV9TVE0xMDAxL0NMNCkgd2UgZGlzY3Vzc2VkIHRoZSB1c2Ugb2YgZnVuY3Rpb25zIGxpa2UgYGRub3JtYCwgYHBub3JtYCBhbmQgYHFub3JtYCBmb3IgY29tcHV0aW5nIHRoZSBkZW5zaXR5LCBwcm9iYWJpbGl0eSBkaXN0cmlidXRpb24gZnVuY3Rpb24gYW5kIHF1YW50aWxlIHZhbHVlIHJlc3BlY3RpdmVseSBvZiBub3JtYWxseSBkaXN0cmlidXRlZCByYW5kb20gdmFyaWFibGVzLg0KDQpTaW1pbGFybHksIHdlIGNhbiB1c2UgdGhlIGluYnVpbHQgUiBmdW5jdGlvbiBgcHRgIHRvIGNvbXB1dGUgdGhlIHByb2JhYmlsaXR5IHRoYXQgdGhlIHRlc3Qgc3RhdGlzdGljICRUJCAod2hpY2ggZm9sbG93cyBhIFN0dWRlbnQncyAkdCQgZGlzdHJpYnV0aW9uKSB0YWtlcyBhIHZhbHVlIGxlc3MgdGhhbiBvciBlcXVhbCB0byBhbiBhcmJpdHJhcnkgdmFsdWUuIFdlIHdpbGwgaG93ZXZlciBhbHNvIG5lZWQgdG8gaW5jbHVkZSB0aGUgZGVncmVlcyBvZiBmcmVlZG9tIGFzIGFuIGFyZ3VtZW50LiANCg0KRm9yIGV4YW1wbGUsIGlmIHdlIHdvdWxkIGxpa2UgdG8gY29tcHV0ZSAkUChUIFxsZXEgNSkkLCBhbmQgb3VyIGRlZ3JlZXMgb2YgZnJlZWRvbSBpcyA0LCB3ZSBjYW4gdXNlIHRoZSBjb2RlIGBwdCg1LCA0KWAuDQpJZiB3ZSB3b3VsZCBsaWtlIHRvIGNvbXB1dGUgJFAoVCBcZ2VxIDUpJCwgd2UgY2FuIGJlIGNsZXZlciBhbmQgdXNlIHRoZSBjb2RlIGAxIC0gcHQoNSwgNClgXltzaW5jZSB3ZSBhcmUgZGVhbGluZyB3aXRoIGEgY29udGludW91cyByYW5kb20gdmFyaWFibGVdLg0KDQpCYXNlZCBvbiB0aGlzIGluZm9ybWF0aW9uLCBhZGQgZXh0cmEgY29kZSB0byB5b3VyICBgdF90ZXN0X2Z1bmNgIGZ1bmN0aW9uIHNvIHRoYXQgaXQgYWxzbyBwcm92aWRlcyAoYXMgYSB0aGlyZCBvdXRwdXQpIHRoZSBjb3JyZXNwb25kaW5nICRwJC12YWx1ZSBmb3IgdGhlIHRlc3Qgc3RhdGlzdGljLg0KDQoqSGludDogVHJ5IHRvIHVzZSB0aGUgc3ltbWV0cnkgcHJvcGVydHkgb2YgdGhlIFN0dWRlbnQncyAkdCQgZGlzdHJpYnV0aW9uIHdoZW4gaW5jbHVkaW5nIGNvZGUgZm9yIHRoZSAkcCQtdmFsdWUgY2FsY3VsYXRpb25zIGluIHlvdXIgZnVuY3Rpb24uKg0KDQojIyMgDQoNClJ1biBib3RoIHlvdXIgdXBkYXRlZCBgdF90ZXN0X2Z1bmNgIGZ1bmN0aW9uIGFuZCB0aGUgaW5idWlsdCBgdC50ZXN0YCBSIGZ1bmN0aW9uLCB1c2luZyB0aGUgaW5wdXRzIGRpc2N1c3NlZCBpbiBcQHJlZih0dGVzdCkuIFZlcmlmeSB0aGF0IHlvdXIgYHRfdGVzdF9mdW5jYCBmdW5jdGlvbiAkcCQtdmFsdWUgaXMgY29ycmVjdCwgYnkgIGNvbXBhcmluZyBpdCB0byB0aGUgb3V0cHV0IG9mIHRoZSBgdC50ZXN0YCBSIGZ1bmN0aW9uLg0KDQo8YnI+DQoNCiMjIyMgV2Ugbm93IGhhdmUgd3JpdHRlbiBmdW5jdGlvbnMgdGhhdCwgY29tYmluZWQsIGNhbiBiZSB1c2VkIHRvIGNvbmR1Y3QgYSBvbmUtc2FtcGxlICR0JC10ZXN0IGZvciBhIHR3by1zaWRlZCBoeXBvdGhlc2lzLiBXZWxsIGRvbmUhIFRoYXQgY29uY2x1ZGVzIG91ciBjb21wdXRlciBsYWIgb24gUiBmdW5jdGlvbiB3cml0aW5nLiAjIyMjIHstfQ0KDQpIb3BlZnVsbHkgdGhpcyBsYWIgaGFzIGRlbXlzdGlmaWVkIFIgZnVuY3Rpb24gY3JlYXRpb24sIGFuZCBzaG93biB5b3UgdGhhdCB3cml0aW5nIGZ1bmN0aW9ucyBpbiBSIGlzIHBlcmhhcHMgZWFzaWVyIHRoYW4geW91IG1pZ2h0IGhhdmUgdGhvdWdodC4NCg0KPGJyPg0KDQo8Zm9udCBjb2xvciA9ICJncmV5Ij4NClRoZXNlIG5vdGVzIGhhdmUgYmVlbiBwcmVwYXJlZCBieSBSdXBlcnQgS3V2ZWtlIGFuZCBBbWFuZGEgU2hha2VyLiBUaGUgY29weXJpZ2h0IGZvciB0aGUgbWF0ZXJpYWwgaW4gdGhlc2Ugbm90ZXMgcmVzaWRlcyB3aXRoIHRoZSBhdXRob3JzIG5hbWVkIGFib3ZlLCB3aXRoIHRoZSBEZXBhcnRtZW50IG9mIE1hdGhlbWF0aWNhbCBhbmQgUGh5c2ljYWwgU2NpZW5jZXMgYW5kIHdpdGggTGEgVHJvYmUgVW5pdmVyc2l0eS4gQ29weXJpZ2h0IGluIHRoaXMgd29yayBpcyB2ZXN0ZWQgaW4gTGEgVHJvYmUgVW5pdmVyc2l0eSBpbmNsdWRpbmcgYWxsIExhIFRyb2JlIFVuaXZlcnNpdHkgYnJhbmRpbmcgYW5kIG5hbWluZy4gVW5sZXNzIG90aGVyd2lzZSBzdGF0ZWQsIG1hdGVyaWFsIHdpdGhpbiB0aGlzIHdvcmsgaXMgbGljZW5zZWQgdW5kZXIgYSBDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uLU5vbiBDb21tZXJjaWFsLU5vbiBEZXJpdmF0aXZlcyBMaWNlbnNlIA0KPGEgaHJlZiA9ICJodHRwczovL2NyZWF0aXZlY29tbW9ucy5vcmcvbGljZW5zZXMvYnktbmMtbmQvNC4wL0NDIiB0YXJnZXQ9Il9ibGFuayI+IEJZLU5DLU5ELiA8L2E+DQo8L2ZvbnQ+