OBJECTIVE: In our lectures this unit, we spent a good chunk of time
reviewing various hypothesis tests which allow us to compare means to
expected null values and determine if they fit our hypothesized
expectations. For this R lab, you will be tasked with performing various
types of hypothesis tests. For the first few, I will tell you WHAT kind
of test you should run, for the last few, YOU will have to decide based
on the type of data you’re working with and the question I am
asking.
For reference, I have created a markdown file that contains example
code for running all types of hypothesis tests, feel free to use those
or to use internet resources.
Example 1
From the 3rd R lab…Please bring in the “rice.csv” file, you may call
it whatever you want. The folks that collected the data would like you
to analyze it to see what patterns may/may not exist. The first thing
they want you to look at is whether or not the means of RootDryMass
differs between wild-type and gmo rice. To run a comparison of means
between these two groups, you will need to run a two-sample t-test. Use
the code chunk below to bring in the data and to run the test.
rice <- read.csv("~/Desktop/BIN510-files/rice.csv")
t.test(RootDryMass ~ variety, data = rice)
Welch Two Sample t-test
data: RootDryMass by variety
t = -5.2857, df = 40.429, p-value = 4.617e-06
alternative hypothesis: true difference in means between group gmo and group wt is not equal to 0
95 percent confidence interval:
-23.22928 -10.38183
sample estimates:
mean in group gmo mean in group wt
9.666667 26.472222
What are your null and alternative hypotheses for this
test?
What is the p-value for the two sample t test? What does this
mean? #### Answer:
Null hypothesis: the mean of RootDryMass is the same for GMO and
WT. Alt hypothesis: the mean of RootDryMass is different between MO and
WT.
The P-vaule for the two sample t test is 4.617e-06, this is a
very small p-value which can work against the null hypothesis meaning
that the means are most likely different.
Let’s create a boxplot comparing the RootDryMass for each variety
boxplot(RootDryMass ~ variety,
data = rice,
main = "Root Dry Mass (rice variety)",
xlab = "Rice variety",
ylab = "Root Dry Mass",
col=c("red","orange"))

Example 2: rice continued…
Next, the researchers want to check if other factors are influencing
growth of rice. Given that they used a different fertilizer, they are
curious if that impacted the RootDryMass Since there are more than 2
levels of fertilizer, the appropriate test would be an ANOVA. Use the
code chunk below to run an ANOVA of how means of RootDryMass differ
across fertilizer treatments (‘fert’)
anvoa <- aov(RootDryMass ~ fert, data = rice)
summary(aov(RootDryMass ~ fert, data = rice))
Df Sum Sq Mean Sq F value Pr(>F)
fert 2 3640 1819.8 8.855 0.000378 ***
Residuals 69 14181 205.5
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
- What are you null and alternative hypotheses for this test?
- What is the P-value of this test? What does this mean?
- Do we need to run a post-hoc test? Why or why not? a. if you DO need
to run a post-hoc test, run it in the code chunk below. b. What does
your post-hoc analysis tell you?
Answers:
- Null hypothesis: The mean of RootDryMass is the same for both GMO
and WT. Alt hypothesis: The means of RootDryMass is different between
GMO and WT.
- The p-value is 0.000378, this is smaller than 0.05 which means that
the masses were not the same for all of the outcomes.
- Yes we should run a post-hoc test. The post-hoc test shows us that
both NH4Cl-F10 and NH4NO3-F10 are significantly different while
NH4NO3-NH4Cl has no significant difference.
TukeyHSD(aov(RootDryMass ~ fert, data = rice))
Tukey multiple comparisons of means
95% family-wise confidence level
Fit: aov(formula = RootDryMass ~ fert, data = rice)
$fert
diff lwr upr p adj
NH4Cl-F10 -16.875000 -26.787877 -6.962123 0.0003499
NH4NO3-F10 -12.166667 -22.079543 -2.253790 0.0122604
NH4NO3-NH4Cl 4.708333 -5.204543 14.621210 0.4943812
Let’s create a boxplot comparing the RootDryMass for each fert
boxplot(RootDryMass ~ fert,
data = rice,
main = "Root Dry Mass (fert)",
xlab = "Fertilizer",
ylab = "Root Dry Mass",
col=c("pink","yellow","turquoise"))

Example 2
Preventative care is incredibly important for finding and treating
many disease before they become a problem. Researchers have been
collecting data on one of the most common types of cancer, breast
cancer, and how it associates with having preventative screening, in
this case, a mammogram. Researchers have collected data on over 80,000
women over the last few decades and have been tracking whether women
died from breast cancer and whether they had a mammogram. These
researchers have asked you to look into possible links between the two.
Here, since both variables are categorical, you will need to test for
associations using a chi-squared contingency test. In the code chunk
below, please read in the “mammogram.csv” dataset and run the
chi-squared analysis.
Answer:
chisq.test(table(mammogram$treatment, mammogram$breast_cancer_death))
Pearson's Chi-squared test with Yates' continuity correction
data: table(mammogram$treatment, mammogram$breast_cancer_death)
X-squared = 0.01748, df = 1, p-value = 0.8948
- What are your null and alternative hypotheses for this test?
- How do you interpret the p-value for this relationship?
Answers:
- Null hypothesis: There is no association between getting a mammogram
and death by breast cancer. Alt hypothesis: There is a correlation
between getting a mammogram and death by breast cancer.
- Since we were give 0.8948, we are unable to reject the null
hypothesis, meaning there is not enough evidence to support that the two
are correlated.
Let’s create a mosaic plot to show the relationship between cancer
survival and whether or not a patient had a mammogram.
mosaicplot(table(mammogram$treatment, mammogram$breast_cancer_death),
main = "Mammogram and Breast Cancer Death",
xlab = "Mammogram",
ylab = "Breast Cancer Death",
col= c("black","blue"))

****
Example 3: YOU CHOOSE THE RIGHT TEST!
NOTE: for whatever test you end up choosing, you will need to make
sure you address the following: 1. tell me WHY you are choosing the test
you’re choosing 2. Tell me your null and alternative hypotheses 3.
Interpret the p-value and what it means for the potential relationship
4. Create an appropriate graph for the relationship. 5. If you choose an
ANOVA, you need to determine whether or not you need to run a post-hoc
test! Run it if you believe it is necessary.
Background: in the code chunk below, please bring in “poison.csv”,
you may call the dataframe whatever you want. The data comes from a
study looking at how different types of poison and various treatments
(levels) of the poison impact how long an animal can survive for (days).
Given the data from the experiment, the researchs want you to look at
how the mean time until death differs by ‘treat’
HINT: you can run some of the initial code we looked at in the first
few R labs to determine the types of data you have as well as the levels
within the variables.
Answers
- I chose a one-way ANOVA because time is numerical while treat is a
categorical variable. With ANOVA we can compare the two variables.
- Null hypothesis: The mean survival time is the same for all of the
treatment groups. Alt hypothesis: The mean survival differs between the
treatment groups.
- The ANOVA gave a p-value of 0.000992, this is a lot less than 0.05
which means that we reject the null hypothesis. Since the ANOVA proved
to be signigicant I ran a post-hoc test to determine the treatment B was
significantly more different from A and C.
poison <- read.csv("~/Desktop/BIN510-files/poisons.csv")
names(poison)
str(poison)
table(poison$treat)
summary(aov(time ~ treat, data = poison))
TukeyHSD(aov(time ~ treat, data = poison))
boxplot(time ~ treat,
data = poison,
main = "Survival Time by Treatment",
xlab = "Treatment",
ylab = "Time Until Death (days)",
col= c("yellow","lightgreen","lightpink","violet"))
LS0tCnRpdGxlOiAiQklOIDUxMCBMYWIgNCIKc3VidGl0bGU6ICJIeXBvdGhlc2lzIHRlc3RpbmcgZm9yIGRpZmZlcmVuY2VzIGluIG1lYW5zIgphdXRob3I6IEthdGVyaW5hIEdlb3JnaW9wb3Vsb3MgCm91dHB1dDoKICBodG1sX25vdGVib29rOiBkZWZhdWx0CiAgaHRtbF9kb2N1bWVudDoKICAgIGRmX3ByaW50OiBwYWdlZAogIHBkZl9kb2N1bWVudDogZGVmYXVsdAotLS0KCk9CSkVDVElWRTogSW4gb3VyIGxlY3R1cmVzIHRoaXMgdW5pdCwgd2Ugc3BlbnQgYSBnb29kIGNodW5rIG9mIHRpbWUgcmV2aWV3aW5nIHZhcmlvdXMgaHlwb3RoZXNpcyB0ZXN0cyB3aGljaCBhbGxvdyB1cyB0byBjb21wYXJlIG1lYW5zIHRvIGV4cGVjdGVkIG51bGwgdmFsdWVzIGFuZCBkZXRlcm1pbmUgaWYgdGhleSBmaXQgb3VyIGh5cG90aGVzaXplZCBleHBlY3RhdGlvbnMuIEZvciB0aGlzIFIgbGFiLCB5b3Ugd2lsbCBiZSB0YXNrZWQgd2l0aCBwZXJmb3JtaW5nIHZhcmlvdXMgdHlwZXMgb2YgaHlwb3RoZXNpcyB0ZXN0cy4gRm9yIHRoZSBmaXJzdCBmZXcsIEkgd2lsbCB0ZWxsIHlvdSBXSEFUIGtpbmQgb2YgdGVzdCB5b3Ugc2hvdWxkIHJ1biwgZm9yIHRoZSBsYXN0IGZldywgWU9VIHdpbGwgaGF2ZSB0byBkZWNpZGUgYmFzZWQgb24gdGhlIHR5cGUgb2YgZGF0YSB5b3UncmUgd29ya2luZyB3aXRoIGFuZCB0aGUgcXVlc3Rpb24gSSBhbSBhc2tpbmcuCgpGb3IgcmVmZXJlbmNlLCBJIGhhdmUgY3JlYXRlZCBhIG1hcmtkb3duIGZpbGUgdGhhdCBjb250YWlucyBleGFtcGxlIGNvZGUgZm9yIHJ1bm5pbmcgYWxsIHR5cGVzIG9mIGh5cG90aGVzaXMgdGVzdHMsIGZlZWwgZnJlZSB0byB1c2UgdGhvc2Ugb3IgdG8gdXNlIGludGVybmV0IHJlc291cmNlcy4gCgojIyBFeGFtcGxlIDEKRnJvbSB0aGUgM3JkIFIgbGFiLi4uUGxlYXNlIGJyaW5nIGluIHRoZSAicmljZS5jc3YiIGZpbGUsIHlvdSBtYXkgY2FsbCBpdCB3aGF0ZXZlciB5b3Ugd2FudC4gVGhlIGZvbGtzIHRoYXQgY29sbGVjdGVkIHRoZSBkYXRhIHdvdWxkIGxpa2UgeW91IHRvIGFuYWx5emUgaXQgdG8gc2VlIHdoYXQgcGF0dGVybnMgbWF5L21heSBub3QgZXhpc3QuIFRoZSBmaXJzdCB0aGluZyB0aGV5IHdhbnQgeW91IHRvIGxvb2sgYXQgaXMgd2hldGhlciBvciBub3QgdGhlIG1lYW5zIG9mIFJvb3REcnlNYXNzIGRpZmZlcnMgYmV0d2VlbiB3aWxkLXR5cGUgYW5kIGdtbyByaWNlLiBUbyBydW4gYSBjb21wYXJpc29uIG9mIG1lYW5zIGJldHdlZW4gdGhlc2UgdHdvIGdyb3VwcywgeW91IHdpbGwgbmVlZCB0byBydW4gYSB0d28tc2FtcGxlIHQtdGVzdC4gVXNlIHRoZSBjb2RlIGNodW5rIGJlbG93IHRvIGJyaW5nIGluIHRoZSBkYXRhIGFuZCB0byBydW4gdGhlIHRlc3QuCgpgYGB7cn0KcmljZSA8LSByZWFkLmNzdigifi9EZXNrdG9wL0JJTjUxMC1maWxlcy9yaWNlLmNzdiIpCnQudGVzdChSb290RHJ5TWFzcyB+IHZhcmlldHksIGRhdGEgPSByaWNlKQpgYGAKCjEuIFdoYXQgYXJlIHlvdXIgbnVsbCBhbmQgYWx0ZXJuYXRpdmUgaHlwb3RoZXNlcyBmb3IgdGhpcyB0ZXN0PwoyLiBXaGF0IGlzIHRoZSBwLXZhbHVlIGZvciB0aGUgdHdvIHNhbXBsZSB0IHRlc3Q/IFdoYXQgZG9lcyB0aGlzIG1lYW4/CiMjIyMgQW5zd2VyOgoKMS4gTnVsbCBoeXBvdGhlc2lzOiB0aGUgbWVhbiBvZiBSb290RHJ5TWFzcyBpcyB0aGUgc2FtZSBmb3IgR01PIGFuZCBXVC4KICAgQWx0IGh5cG90aGVzaXM6IHRoZSBtZWFuIG9mIFJvb3REcnlNYXNzIGlzIGRpZmZlcmVudCBiZXR3ZWVuIE1PIGFuZCBXVC4KMi4gVGhlIFAtdmF1bGUgZm9yIHRoZSB0d28gc2FtcGxlIHQgdGVzdCBpcyA0LjYxN2UtMDYsIHRoaXMgaXMgYSB2ZXJ5IHNtYWxsIHAtdmFsdWUgd2hpY2ggY2FuIHdvcmsgYWdhaW5zdCB0aGUgbnVsbCBoeXBvdGhlc2lzIG1lYW5pbmcgdGhhdCB0aGUgbWVhbnMgYXJlIG1vc3QgbGlrZWx5IGRpZmZlcmVudC4gIAoKKioqKgoKTGV0J3MgY3JlYXRlIGEgYm94cGxvdCBjb21wYXJpbmcgdGhlIFJvb3REcnlNYXNzIGZvciBlYWNoIHZhcmlldHkKYGBge3J9CmJveHBsb3QoUm9vdERyeU1hc3MgfiB2YXJpZXR5LAogICAgICAgIGRhdGEgPSByaWNlLAogICAgICAgIG1haW4gPSAiUm9vdCBEcnkgTWFzcyAocmljZSB2YXJpZXR5KSIsCiAgICAgICAgeGxhYiA9ICJSaWNlIHZhcmlldHkiLAogICAgICAgIHlsYWIgPSAiUm9vdCBEcnkgTWFzcyIsCiAgICAgICAgY29sPWMoInJlZCIsIm9yYW5nZSIpKQoKYGBgCgojIyBFeGFtcGxlIDI6IHJpY2UgY29udGludWVkLi4uCk5leHQsIHRoZSByZXNlYXJjaGVycyB3YW50IHRvIGNoZWNrIGlmIG90aGVyIGZhY3RvcnMgYXJlIGluZmx1ZW5jaW5nIGdyb3d0aCBvZiByaWNlLiBHaXZlbiB0aGF0IHRoZXkgdXNlZCBhIGRpZmZlcmVudCBmZXJ0aWxpemVyLCB0aGV5IGFyZSBjdXJpb3VzIGlmIHRoYXQgaW1wYWN0ZWQgdGhlIFJvb3REcnlNYXNzIFNpbmNlIHRoZXJlIGFyZSBtb3JlIHRoYW4gMiBsZXZlbHMgb2YgZmVydGlsaXplciwgdGhlIGFwcHJvcHJpYXRlIHRlc3Qgd291bGQgYmUgYW4gQU5PVkEuIFVzZSB0aGUgY29kZSBjaHVuayBiZWxvdyB0byBydW4gYW4gQU5PVkEgb2YgaG93IG1lYW5zIG9mIFJvb3REcnlNYXNzIGRpZmZlciBhY3Jvc3MgZmVydGlsaXplciB0cmVhdG1lbnRzICgnZmVydCcpCgpgYGB7cn0KYW52b2EgPC0gYW92KFJvb3REcnlNYXNzIH4gZmVydCwgZGF0YSA9IHJpY2UpCnN1bW1hcnkoYW92KFJvb3REcnlNYXNzIH4gZmVydCwgZGF0YSA9IHJpY2UpKQpgYGAKCiAgMS4gV2hhdCBhcmUgeW91IG51bGwgYW5kIGFsdGVybmF0aXZlIGh5cG90aGVzZXMgZm9yIHRoaXMgdGVzdD8KICAyLiBXaGF0IGlzIHRoZSBQLXZhbHVlIG9mIHRoaXMgdGVzdD8gV2hhdCBkb2VzIHRoaXMgbWVhbj8KICAzLiBEbyB3ZSBuZWVkIHRvIHJ1biBhIHBvc3QtaG9jIHRlc3Q/IFdoeSBvciB3aHkgbm90PwogICAgYS4gaWYgeW91IERPIG5lZWQgdG8gcnVuIGEgcG9zdC1ob2MgdGVzdCwgcnVuIGl0IGluIHRoZSBjb2RlIGNodW5rIGJlbG93LiAKICAgIGIuIFdoYXQgZG9lcyB5b3VyIHBvc3QtaG9jIGFuYWx5c2lzIHRlbGwgeW91PwoKIyMjIyBBbnN3ZXJzOgoKICAxLiBOdWxsIGh5cG90aGVzaXM6IFRoZSBtZWFuIG9mIFJvb3REcnlNYXNzIGlzIHRoZSBzYW1lIGZvciBib3RoIEdNTyBhbmQgV1QuCiAgICAgQWx0IGh5cG90aGVzaXM6IFRoZSBtZWFucyBvZiBSb290RHJ5TWFzcyBpcyBkaWZmZXJlbnQgYmV0d2VlbiBHTU8gYW5kIFdULiAKICAyLiBUaGUgcC12YWx1ZSBpcyAwLjAwMDM3OCwgdGhpcyBpcyBzbWFsbGVyIHRoYW4gMC4wNSB3aGljaCBtZWFucyB0aGF0IHRoZSBtYXNzZXMgd2VyZSBub3QgdGhlIHNhbWUgZm9yIGFsbCBvZiB0aGUgb3V0Y29tZXMuIAogIDMuIFllcyB3ZSBzaG91bGQgcnVuIGEgcG9zdC1ob2MgdGVzdC4gVGhlIHBvc3QtaG9jIHRlc3Qgc2hvd3MgdXMgdGhhdCBib3RoIE5INENsLUYxMCBhbmQgTkg0Tk8zLUYxMCBhcmUgc2lnbmlmaWNhbnRseSBkaWZmZXJlbnQgd2hpbGUgTkg0Tk8zLU5INENsIGhhcyBubyBzaWduaWZpY2FudCBkaWZmZXJlbmNlLiAKCiAgCioqKiAKCmBgYHtyfQpUdWtleUhTRChhb3YoUm9vdERyeU1hc3MgfiBmZXJ0LCBkYXRhID0gcmljZSkpCmBgYAoKKioqKgoKTGV0J3MgY3JlYXRlIGEgYm94cGxvdCBjb21wYXJpbmcgdGhlIFJvb3REcnlNYXNzIGZvciBlYWNoIGZlcnQKCmBgYHtyfQpib3hwbG90KFJvb3REcnlNYXNzIH4gZmVydCwKICAgICAgICBkYXRhID0gcmljZSwKICAgICAgICBtYWluID0gIlJvb3QgRHJ5IE1hc3MgKGZlcnQpIiwKICAgICAgICB4bGFiID0gIkZlcnRpbGl6ZXIiLAogICAgICAgIHlsYWIgPSAiUm9vdCBEcnkgTWFzcyIsCiAgICAgICAgY29sPWMoInBpbmsiLCJ5ZWxsb3ciLCJ0dXJxdW9pc2UiKSkKYGBgCgoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKIyMgRXhhbXBsZSAyClByZXZlbnRhdGl2ZSBjYXJlIGlzIGluY3JlZGlibHkgaW1wb3J0YW50IGZvciBmaW5kaW5nIGFuZCB0cmVhdGluZyBtYW55IGRpc2Vhc2UgYmVmb3JlIHRoZXkgYmVjb21lIGEgcHJvYmxlbS4gUmVzZWFyY2hlcnMgaGF2ZSBiZWVuIGNvbGxlY3RpbmcgZGF0YSBvbiBvbmUgb2YgdGhlIG1vc3QgY29tbW9uIHR5cGVzIG9mIGNhbmNlciwgYnJlYXN0IGNhbmNlciwgYW5kIGhvdyBpdCBhc3NvY2lhdGVzIHdpdGggaGF2aW5nIHByZXZlbnRhdGl2ZSBzY3JlZW5pbmcsIGluIHRoaXMgY2FzZSwgYSBtYW1tb2dyYW0uIFJlc2VhcmNoZXJzIGhhdmUgY29sbGVjdGVkIGRhdGEgb24gb3ZlciA4MCwwMDAgd29tZW4gb3ZlciB0aGUgbGFzdCBmZXcgZGVjYWRlcyBhbmQgaGF2ZSBiZWVuIHRyYWNraW5nIHdoZXRoZXIgd29tZW4gZGllZCBmcm9tIGJyZWFzdCBjYW5jZXIgYW5kIHdoZXRoZXIgdGhleSBoYWQgYSBtYW1tb2dyYW0uIFRoZXNlIHJlc2VhcmNoZXJzIGhhdmUgYXNrZWQgeW91IHRvIGxvb2sgaW50byBwb3NzaWJsZSBsaW5rcyBiZXR3ZWVuIHRoZSB0d28uIEhlcmUsIHNpbmNlIGJvdGggdmFyaWFibGVzIGFyZSBjYXRlZ29yaWNhbCwgeW91IHdpbGwgbmVlZCB0byB0ZXN0IGZvciBhc3NvY2lhdGlvbnMgdXNpbmcgYSBjaGktc3F1YXJlZCBjb250aW5nZW5jeSB0ZXN0LiBJbiB0aGUgY29kZSBjaHVuayBiZWxvdywgcGxlYXNlIHJlYWQgaW4gdGhlICJtYW1tb2dyYW0uY3N2IiBkYXRhc2V0IGFuZCBydW4gdGhlIGNoaS1zcXVhcmVkIGFuYWx5c2lzLgoKIyMjIyBBbnN3ZXI6CgpgYGB7cn0KbWFtbW9ncmFtIDwtIHJlYWQuY3N2KCJ+L0Rlc2t0b3AvQklONTEwLWZpbGVzL21hbW1vZ3JhbS5jc3YiKQpuYW1lcyhtYW1tb2dyYW0pCnRhYmxlKG1hbW1vZ3JhbSR0cmVhdG1lbnQsIG1hbW1vZ3JhbSRicmVhc3RfY2FuY2VyX2RlYXRoKQpjaGlzcS50ZXN0KHRhYmxlKG1hbW1vZ3JhbSR0cmVhdG1lbnQsIG1hbW1vZ3JhbSRicmVhc3RfY2FuY2VyX2RlYXRoKSkKYGBgCgoqKioqCjEuIFdoYXQgYXJlIHlvdXIgbnVsbCBhbmQgYWx0ZXJuYXRpdmUgaHlwb3RoZXNlcyBmb3IgdGhpcyB0ZXN0PwoyLiBIb3cgZG8geW91IGludGVycHJldCB0aGUgcC12YWx1ZSBmb3IgdGhpcyByZWxhdGlvbnNoaXA/CgojIyMjIEFuc3dlcnM6CgogIDEuIE51bGwgaHlwb3RoZXNpczogVGhlcmUgaXMgbm8gYXNzb2NpYXRpb24gYmV0d2VlbiBnZXR0aW5nIGEgbWFtbW9ncmFtIGFuZCBkZWF0aCBieSBicmVhc3QgY2FuY2VyLgogICAgIEFsdCBoeXBvdGhlc2lzOiBUaGVyZSBpcyBhIGNvcnJlbGF0aW9uIGJldHdlZW4gZ2V0dGluZyBhIG1hbW1vZ3JhbSBhbmQgZGVhdGggYnkgYnJlYXN0IGNhbmNlci4gCiAgMi4gU2luY2Ugd2Ugd2VyZSBnaXZlIDAuODk0OCwgd2UgYXJlIHVuYWJsZSB0byByZWplY3QgdGhlIG51bGwgaHlwb3RoZXNpcywgbWVhbmluZyB0aGVyZSBpcyBub3QgZW5vdWdoIGV2aWRlbmNlIHRvIHN1cHBvcnQgdGhhdCB0aGUgdHdvIGFyZSBjb3JyZWxhdGVkLiAKICAKTGV0J3MgY3JlYXRlIGEgbW9zYWljIHBsb3QgdG8gc2hvdyB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gY2FuY2VyIHN1cnZpdmFsIGFuZCB3aGV0aGVyIG9yIG5vdCBhIHBhdGllbnQgaGFkIGEgbWFtbW9ncmFtLgoKYGBge3J9Cm1vc2FpY3Bsb3QodGFibGUobWFtbW9ncmFtJHRyZWF0bWVudCwgbWFtbW9ncmFtJGJyZWFzdF9jYW5jZXJfZGVhdGgpLAogICAgICAgICAgIG1haW4gPSAiTWFtbW9ncmFtIGFuZCBCcmVhc3QgQ2FuY2VyIERlYXRoIiwKICAgICAgICAgICB4bGFiID0gIk1hbW1vZ3JhbSIsCiAgICAgICAgICAgeWxhYiA9ICJCcmVhc3QgQ2FuY2VyIERlYXRoIiwKICAgICAgICAgICBjb2w9IGMoImJsYWNrIiwiYmx1ZSIpKQpgYGAKCioqKioKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKIyMgRXhhbXBsZSAzOiBZT1UgQ0hPT1NFIFRIRSBSSUdIVCBURVNUIQoKTk9URTogZm9yIHdoYXRldmVyIHRlc3QgeW91IGVuZCB1cCBjaG9vc2luZywgeW91IHdpbGwgbmVlZCB0byBtYWtlIHN1cmUgeW91IGFkZHJlc3MgdGhlIGZvbGxvd2luZzoKMS4gdGVsbCBtZSBXSFkgeW91IGFyZSBjaG9vc2luZyB0aGUgdGVzdCB5b3UncmUgY2hvb3NpbmcKMi4gVGVsbCBtZSB5b3VyIG51bGwgYW5kIGFsdGVybmF0aXZlIGh5cG90aGVzZXMKMy4gSW50ZXJwcmV0IHRoZSBwLXZhbHVlIGFuZCB3aGF0IGl0IG1lYW5zIGZvciB0aGUgcG90ZW50aWFsIHJlbGF0aW9uc2hpcAo0LiBDcmVhdGUgYW4gYXBwcm9wcmlhdGUgZ3JhcGggZm9yIHRoZSByZWxhdGlvbnNoaXAuCjUuIElmIHlvdSBjaG9vc2UgYW4gQU5PVkEsIHlvdSBuZWVkIHRvIGRldGVybWluZSB3aGV0aGVyIG9yIG5vdCB5b3UgbmVlZCB0byBydW4gYSBwb3N0LWhvYyB0ZXN0ISBSdW4gaXQgaWYgeW91IGJlbGlldmUgaXQgaXMgbmVjZXNzYXJ5LgoKQmFja2dyb3VuZDogaW4gdGhlIGNvZGUgY2h1bmsgYmVsb3csIHBsZWFzZSBicmluZyBpbiAicG9pc29uLmNzdiIsIHlvdSBtYXkgY2FsbCB0aGUgZGF0YWZyYW1lIHdoYXRldmVyIHlvdSB3YW50LiBUaGUgZGF0YSBjb21lcyBmcm9tIGEgc3R1ZHkgbG9va2luZyBhdCBob3cgZGlmZmVyZW50IHR5cGVzIG9mIHBvaXNvbiBhbmQgdmFyaW91cyB0cmVhdG1lbnRzIChsZXZlbHMpIG9mIHRoZSBwb2lzb24gaW1wYWN0IGhvdyBsb25nIGFuIGFuaW1hbCBjYW4gc3Vydml2ZSBmb3IgKGRheXMpLiBHaXZlbiB0aGUgZGF0YSBmcm9tIHRoZSBleHBlcmltZW50LCB0aGUgcmVzZWFyY2hzIHdhbnQgeW91IHRvIGxvb2sgYXQgaG93IHRoZSBtZWFuIHRpbWUgdW50aWwgZGVhdGggZGlmZmVycyBieSAndHJlYXQnCgpISU5UOiB5b3UgY2FuIHJ1biBzb21lIG9mIHRoZSBpbml0aWFsIGNvZGUgd2UgbG9va2VkIGF0IGluIHRoZSBmaXJzdCBmZXcgUiBsYWJzIHRvIGRldGVybWluZSB0aGUgdHlwZXMgb2YgZGF0YSB5b3UgaGF2ZSBhcyB3ZWxsIGFzIHRoZSBsZXZlbHMgd2l0aGluIHRoZSB2YXJpYWJsZXMuCgojIyMjIEFuc3dlcnMKIDEuIEkgY2hvc2UgYSBvbmUtd2F5IEFOT1ZBIGJlY2F1c2UgdGltZSBpcyBudW1lcmljYWwgd2hpbGUgdHJlYXQgaXMgYSBjYXRlZ29yaWNhbCB2YXJpYWJsZS4gV2l0aCBBTk9WQSB3ZSBjYW4gY29tcGFyZSB0aGUgdHdvIHZhcmlhYmxlcy4KIDIuIE51bGwgaHlwb3RoZXNpczogVGhlIG1lYW4gc3Vydml2YWwgdGltZSBpcyB0aGUgc2FtZSBmb3IgYWxsIG9mIHRoZSB0cmVhdG1lbnQgZ3JvdXBzLiAKICAgIEFsdCBoeXBvdGhlc2lzOiBUaGUgbWVhbiBzdXJ2aXZhbCBkaWZmZXJzIGJldHdlZW4gdGhlIHRyZWF0bWVudCBncm91cHMuIAogMy4gVGhlIEFOT1ZBIGdhdmUgYSBwLXZhbHVlIG9mIDAuMDAwOTkyLCB0aGlzIGlzIGEgbG90IGxlc3MgdGhhbiAwLjA1IHdoaWNoIG1lYW5zIHRoYXQgd2UgcmVqZWN0IHRoZSBudWxsIGh5cG90aGVzaXMuIFNpbmNlIHRoZSBBTk9WQSBwcm92ZWQgdG8gYmUgc2lnbmlnaWNhbnQgSSByYW4gYSBwb3N0LWhvYyB0ZXN0IHRvIGRldGVybWluZSB0aGUgdHJlYXRtZW50IEIgd2FzIHNpZ25pZmljYW50bHkgbW9yZSBkaWZmZXJlbnQgZnJvbSBBIGFuZCBDLiAKCmBgYHtyfQpwb2lzb24gPC0gcmVhZC5jc3YoIn4vRGVza3RvcC9CSU41MTAtZmlsZXMvcG9pc29ucy5jc3YiKQpuYW1lcyhwb2lzb24pCnN0cihwb2lzb24pCnRhYmxlKHBvaXNvbiR0cmVhdCkKc3VtbWFyeShhb3YodGltZSB+IHRyZWF0LCBkYXRhID0gcG9pc29uKSkKVHVrZXlIU0QoYW92KHRpbWUgfiB0cmVhdCwgZGF0YSA9IHBvaXNvbikpCmJveHBsb3QodGltZSB+IHRyZWF0LAogICAgICAgIGRhdGEgPSBwb2lzb24sCiAgICAgICAgbWFpbiA9ICJTdXJ2aXZhbCBUaW1lIGJ5IFRyZWF0bWVudCIsCiAgICAgICAgeGxhYiA9ICJUcmVhdG1lbnQiLAogICAgICAgIHlsYWIgPSAiVGltZSBVbnRpbCBEZWF0aCAoZGF5cykiLAogICAgICAgIGNvbD0gYygieWVsbG93IiwibGlnaHRncmVlbiIsImxpZ2h0cGluayIsInZpb2xldCIpKQpgYGAKCg==