OBJECTIVE: In our lectures this unit, we spent a good chunk of time reviewing various hypothesis tests which allow us to compare means to expected null values and determine if they fit our hypothesized expectations. For this R lab, you will be tasked with performing various types of hypothesis tests. For the first few, I will tell you WHAT kind of test you should run, for the last few, YOU will have to decide based on the type of data you’re working with and the question I am asking.

For reference, I have created a markdown file that contains example code for running all types of hypothesis tests, feel free to use those or to use internet resources.

Example 1

From the 3rd R lab…Please bring in the “rice.csv” file, you may call it whatever you want. The folks that collected the data would like you to analyze it to see what patterns may/may not exist. The first thing they want you to look at is whether or not the means of RootDryMass differs between wild-type and gmo rice. To run a comparison of means between these two groups, you will need to run a two-sample t-test. Use the code chunk below to bring in the data and to run the test.

rice <- read.csv("~/Desktop/BIN510-files/rice.csv")
t.test(RootDryMass ~ variety, data = rice)

    Welch Two Sample t-test

data:  RootDryMass by variety
t = -5.2857, df = 40.429, p-value = 4.617e-06
alternative hypothesis: true difference in means between group gmo and group wt is not equal to 0
95 percent confidence interval:
 -23.22928 -10.38183
sample estimates:
mean in group gmo  mean in group wt 
         9.666667         26.472222 
  1. What are your null and alternative hypotheses for this test?

  2. What is the p-value for the two sample t test? What does this mean? #### Answer:

  3. Null hypothesis: the mean of RootDryMass is the same for GMO and WT. Alt hypothesis: the mean of RootDryMass is different between MO and WT.

  4. The P-vaule for the two sample t test is 4.617e-06, this is a very small p-value which can work against the null hypothesis meaning that the means are most likely different.


Let’s create a boxplot comparing the RootDryMass for each variety

boxplot(RootDryMass ~ variety,
        data = rice,
        main = "Root Dry Mass (rice variety)",
        xlab = "Rice variety",
        ylab = "Root Dry Mass",
        col=c("red","orange"))

Example 2: rice continued…

Next, the researchers want to check if other factors are influencing growth of rice. Given that they used a different fertilizer, they are curious if that impacted the RootDryMass Since there are more than 2 levels of fertilizer, the appropriate test would be an ANOVA. Use the code chunk below to run an ANOVA of how means of RootDryMass differ across fertilizer treatments (‘fert’)

anvoa <- aov(RootDryMass ~ fert, data = rice)
summary(aov(RootDryMass ~ fert, data = rice))
            Df Sum Sq Mean Sq F value   Pr(>F)    
fert         2   3640  1819.8   8.855 0.000378 ***
Residuals   69  14181   205.5                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
  1. What are you null and alternative hypotheses for this test?
  2. What is the P-value of this test? What does this mean?
  3. Do we need to run a post-hoc test? Why or why not? a. if you DO need to run a post-hoc test, run it in the code chunk below. b. What does your post-hoc analysis tell you?

Answers:

  1. Null hypothesis: The mean of RootDryMass is the same for both GMO and WT. Alt hypothesis: The means of RootDryMass is different between GMO and WT.
  2. The p-value is 0.000378, this is smaller than 0.05 which means that the masses were not the same for all of the outcomes.
  3. Yes we should run a post-hoc test. The post-hoc test shows us that both NH4Cl-F10 and NH4NO3-F10 are significantly different while NH4NO3-NH4Cl has no significant difference.

TukeyHSD(aov(RootDryMass ~ fert, data = rice))
  Tukey multiple comparisons of means
    95% family-wise confidence level

Fit: aov(formula = RootDryMass ~ fert, data = rice)

$fert
                   diff        lwr       upr     p adj
NH4Cl-F10    -16.875000 -26.787877 -6.962123 0.0003499
NH4NO3-F10   -12.166667 -22.079543 -2.253790 0.0122604
NH4NO3-NH4Cl   4.708333  -5.204543 14.621210 0.4943812

Let’s create a boxplot comparing the RootDryMass for each fert

boxplot(RootDryMass ~ fert,
        data = rice,
        main = "Root Dry Mass (fert)",
        xlab = "Fertilizer",
        ylab = "Root Dry Mass",
        col=c("pink","yellow","turquoise"))


Example 2

Preventative care is incredibly important for finding and treating many disease before they become a problem. Researchers have been collecting data on one of the most common types of cancer, breast cancer, and how it associates with having preventative screening, in this case, a mammogram. Researchers have collected data on over 80,000 women over the last few decades and have been tracking whether women died from breast cancer and whether they had a mammogram. These researchers have asked you to look into possible links between the two. Here, since both variables are categorical, you will need to test for associations using a chi-squared contingency test. In the code chunk below, please read in the “mammogram.csv” dataset and run the chi-squared analysis.

Answer:

chisq.test(table(mammogram$treatment, mammogram$breast_cancer_death))

    Pearson's Chi-squared test with Yates' continuity correction

data:  table(mammogram$treatment, mammogram$breast_cancer_death)
X-squared = 0.01748, df = 1, p-value = 0.8948

  1. What are your null and alternative hypotheses for this test?
  2. How do you interpret the p-value for this relationship?

Answers:

  1. Null hypothesis: There is no association between getting a mammogram and death by breast cancer. Alt hypothesis: There is a correlation between getting a mammogram and death by breast cancer.
  2. Since we were give 0.8948, we are unable to reject the null hypothesis, meaning there is not enough evidence to support that the two are correlated.

Let’s create a mosaic plot to show the relationship between cancer survival and whether or not a patient had a mammogram.

mosaicplot(table(mammogram$treatment, mammogram$breast_cancer_death),
           main = "Mammogram and Breast Cancer Death",
           xlab = "Mammogram",
           ylab = "Breast Cancer Death",
           col= c("black","blue"))

****

Example 3: YOU CHOOSE THE RIGHT TEST!

NOTE: for whatever test you end up choosing, you will need to make sure you address the following: 1. tell me WHY you are choosing the test you’re choosing 2. Tell me your null and alternative hypotheses 3. Interpret the p-value and what it means for the potential relationship 4. Create an appropriate graph for the relationship. 5. If you choose an ANOVA, you need to determine whether or not you need to run a post-hoc test! Run it if you believe it is necessary.

Background: in the code chunk below, please bring in “poison.csv”, you may call the dataframe whatever you want. The data comes from a study looking at how different types of poison and various treatments (levels) of the poison impact how long an animal can survive for (days). Given the data from the experiment, the researchs want you to look at how the mean time until death differs by ‘treat’

HINT: you can run some of the initial code we looked at in the first few R labs to determine the types of data you have as well as the levels within the variables.

Answers

  1. I chose a one-way ANOVA because time is numerical while treat is a categorical variable. With ANOVA we can compare the two variables.
  2. Null hypothesis: The mean survival time is the same for all of the treatment groups. Alt hypothesis: The mean survival differs between the treatment groups.
  3. The ANOVA gave a p-value of 0.000992, this is a lot less than 0.05 which means that we reject the null hypothesis. Since the ANOVA proved to be signigicant I ran a post-hoc test to determine the treatment B was significantly more different from A and C.
poison <- read.csv("~/Desktop/BIN510-files/poisons.csv")
names(poison)
str(poison)
table(poison$treat)
summary(aov(time ~ treat, data = poison))
TukeyHSD(aov(time ~ treat, data = poison))
boxplot(time ~ treat,
        data = poison,
        main = "Survival Time by Treatment",
        xlab = "Treatment",
        ylab = "Time Until Death (days)",
        col= c("yellow","lightgreen","lightpink","violet"))
LS0tCnRpdGxlOiAiQklOIDUxMCBMYWIgNCIKc3VidGl0bGU6ICJIeXBvdGhlc2lzIHRlc3RpbmcgZm9yIGRpZmZlcmVuY2VzIGluIG1lYW5zIgphdXRob3I6IEthdGVyaW5hIEdlb3JnaW9wb3Vsb3MgCm91dHB1dDoKICBodG1sX25vdGVib29rOiBkZWZhdWx0CiAgaHRtbF9kb2N1bWVudDoKICAgIGRmX3ByaW50OiBwYWdlZAogIHBkZl9kb2N1bWVudDogZGVmYXVsdAotLS0KCk9CSkVDVElWRTogSW4gb3VyIGxlY3R1cmVzIHRoaXMgdW5pdCwgd2Ugc3BlbnQgYSBnb29kIGNodW5rIG9mIHRpbWUgcmV2aWV3aW5nIHZhcmlvdXMgaHlwb3RoZXNpcyB0ZXN0cyB3aGljaCBhbGxvdyB1cyB0byBjb21wYXJlIG1lYW5zIHRvIGV4cGVjdGVkIG51bGwgdmFsdWVzIGFuZCBkZXRlcm1pbmUgaWYgdGhleSBmaXQgb3VyIGh5cG90aGVzaXplZCBleHBlY3RhdGlvbnMuIEZvciB0aGlzIFIgbGFiLCB5b3Ugd2lsbCBiZSB0YXNrZWQgd2l0aCBwZXJmb3JtaW5nIHZhcmlvdXMgdHlwZXMgb2YgaHlwb3RoZXNpcyB0ZXN0cy4gRm9yIHRoZSBmaXJzdCBmZXcsIEkgd2lsbCB0ZWxsIHlvdSBXSEFUIGtpbmQgb2YgdGVzdCB5b3Ugc2hvdWxkIHJ1biwgZm9yIHRoZSBsYXN0IGZldywgWU9VIHdpbGwgaGF2ZSB0byBkZWNpZGUgYmFzZWQgb24gdGhlIHR5cGUgb2YgZGF0YSB5b3UncmUgd29ya2luZyB3aXRoIGFuZCB0aGUgcXVlc3Rpb24gSSBhbSBhc2tpbmcuCgpGb3IgcmVmZXJlbmNlLCBJIGhhdmUgY3JlYXRlZCBhIG1hcmtkb3duIGZpbGUgdGhhdCBjb250YWlucyBleGFtcGxlIGNvZGUgZm9yIHJ1bm5pbmcgYWxsIHR5cGVzIG9mIGh5cG90aGVzaXMgdGVzdHMsIGZlZWwgZnJlZSB0byB1c2UgdGhvc2Ugb3IgdG8gdXNlIGludGVybmV0IHJlc291cmNlcy4gCgojIyBFeGFtcGxlIDEKRnJvbSB0aGUgM3JkIFIgbGFiLi4uUGxlYXNlIGJyaW5nIGluIHRoZSAicmljZS5jc3YiIGZpbGUsIHlvdSBtYXkgY2FsbCBpdCB3aGF0ZXZlciB5b3Ugd2FudC4gVGhlIGZvbGtzIHRoYXQgY29sbGVjdGVkIHRoZSBkYXRhIHdvdWxkIGxpa2UgeW91IHRvIGFuYWx5emUgaXQgdG8gc2VlIHdoYXQgcGF0dGVybnMgbWF5L21heSBub3QgZXhpc3QuIFRoZSBmaXJzdCB0aGluZyB0aGV5IHdhbnQgeW91IHRvIGxvb2sgYXQgaXMgd2hldGhlciBvciBub3QgdGhlIG1lYW5zIG9mIFJvb3REcnlNYXNzIGRpZmZlcnMgYmV0d2VlbiB3aWxkLXR5cGUgYW5kIGdtbyByaWNlLiBUbyBydW4gYSBjb21wYXJpc29uIG9mIG1lYW5zIGJldHdlZW4gdGhlc2UgdHdvIGdyb3VwcywgeW91IHdpbGwgbmVlZCB0byBydW4gYSB0d28tc2FtcGxlIHQtdGVzdC4gVXNlIHRoZSBjb2RlIGNodW5rIGJlbG93IHRvIGJyaW5nIGluIHRoZSBkYXRhIGFuZCB0byBydW4gdGhlIHRlc3QuCgpgYGB7cn0KcmljZSA8LSByZWFkLmNzdigifi9EZXNrdG9wL0JJTjUxMC1maWxlcy9yaWNlLmNzdiIpCnQudGVzdChSb290RHJ5TWFzcyB+IHZhcmlldHksIGRhdGEgPSByaWNlKQpgYGAKCjEuIFdoYXQgYXJlIHlvdXIgbnVsbCBhbmQgYWx0ZXJuYXRpdmUgaHlwb3RoZXNlcyBmb3IgdGhpcyB0ZXN0PwoyLiBXaGF0IGlzIHRoZSBwLXZhbHVlIGZvciB0aGUgdHdvIHNhbXBsZSB0IHRlc3Q/IFdoYXQgZG9lcyB0aGlzIG1lYW4/CiMjIyMgQW5zd2VyOgoKMS4gTnVsbCBoeXBvdGhlc2lzOiB0aGUgbWVhbiBvZiBSb290RHJ5TWFzcyBpcyB0aGUgc2FtZSBmb3IgR01PIGFuZCBXVC4KICAgQWx0IGh5cG90aGVzaXM6IHRoZSBtZWFuIG9mIFJvb3REcnlNYXNzIGlzIGRpZmZlcmVudCBiZXR3ZWVuIE1PIGFuZCBXVC4KMi4gVGhlIFAtdmF1bGUgZm9yIHRoZSB0d28gc2FtcGxlIHQgdGVzdCBpcyA0LjYxN2UtMDYsIHRoaXMgaXMgYSB2ZXJ5IHNtYWxsIHAtdmFsdWUgd2hpY2ggY2FuIHdvcmsgYWdhaW5zdCB0aGUgbnVsbCBoeXBvdGhlc2lzIG1lYW5pbmcgdGhhdCB0aGUgbWVhbnMgYXJlIG1vc3QgbGlrZWx5IGRpZmZlcmVudC4gIAoKKioqKgoKTGV0J3MgY3JlYXRlIGEgYm94cGxvdCBjb21wYXJpbmcgdGhlIFJvb3REcnlNYXNzIGZvciBlYWNoIHZhcmlldHkKYGBge3J9CmJveHBsb3QoUm9vdERyeU1hc3MgfiB2YXJpZXR5LAogICAgICAgIGRhdGEgPSByaWNlLAogICAgICAgIG1haW4gPSAiUm9vdCBEcnkgTWFzcyAocmljZSB2YXJpZXR5KSIsCiAgICAgICAgeGxhYiA9ICJSaWNlIHZhcmlldHkiLAogICAgICAgIHlsYWIgPSAiUm9vdCBEcnkgTWFzcyIsCiAgICAgICAgY29sPWMoInJlZCIsIm9yYW5nZSIpKQoKYGBgCgojIyBFeGFtcGxlIDI6IHJpY2UgY29udGludWVkLi4uCk5leHQsIHRoZSByZXNlYXJjaGVycyB3YW50IHRvIGNoZWNrIGlmIG90aGVyIGZhY3RvcnMgYXJlIGluZmx1ZW5jaW5nIGdyb3d0aCBvZiByaWNlLiBHaXZlbiB0aGF0IHRoZXkgdXNlZCBhIGRpZmZlcmVudCBmZXJ0aWxpemVyLCB0aGV5IGFyZSBjdXJpb3VzIGlmIHRoYXQgaW1wYWN0ZWQgdGhlIFJvb3REcnlNYXNzIFNpbmNlIHRoZXJlIGFyZSBtb3JlIHRoYW4gMiBsZXZlbHMgb2YgZmVydGlsaXplciwgdGhlIGFwcHJvcHJpYXRlIHRlc3Qgd291bGQgYmUgYW4gQU5PVkEuIFVzZSB0aGUgY29kZSBjaHVuayBiZWxvdyB0byBydW4gYW4gQU5PVkEgb2YgaG93IG1lYW5zIG9mIFJvb3REcnlNYXNzIGRpZmZlciBhY3Jvc3MgZmVydGlsaXplciB0cmVhdG1lbnRzICgnZmVydCcpCgpgYGB7cn0KYW52b2EgPC0gYW92KFJvb3REcnlNYXNzIH4gZmVydCwgZGF0YSA9IHJpY2UpCnN1bW1hcnkoYW92KFJvb3REcnlNYXNzIH4gZmVydCwgZGF0YSA9IHJpY2UpKQpgYGAKCiAgMS4gV2hhdCBhcmUgeW91IG51bGwgYW5kIGFsdGVybmF0aXZlIGh5cG90aGVzZXMgZm9yIHRoaXMgdGVzdD8KICAyLiBXaGF0IGlzIHRoZSBQLXZhbHVlIG9mIHRoaXMgdGVzdD8gV2hhdCBkb2VzIHRoaXMgbWVhbj8KICAzLiBEbyB3ZSBuZWVkIHRvIHJ1biBhIHBvc3QtaG9jIHRlc3Q/IFdoeSBvciB3aHkgbm90PwogICAgYS4gaWYgeW91IERPIG5lZWQgdG8gcnVuIGEgcG9zdC1ob2MgdGVzdCwgcnVuIGl0IGluIHRoZSBjb2RlIGNodW5rIGJlbG93LiAKICAgIGIuIFdoYXQgZG9lcyB5b3VyIHBvc3QtaG9jIGFuYWx5c2lzIHRlbGwgeW91PwoKIyMjIyBBbnN3ZXJzOgoKICAxLiBOdWxsIGh5cG90aGVzaXM6IFRoZSBtZWFuIG9mIFJvb3REcnlNYXNzIGlzIHRoZSBzYW1lIGZvciBib3RoIEdNTyBhbmQgV1QuCiAgICAgQWx0IGh5cG90aGVzaXM6IFRoZSBtZWFucyBvZiBSb290RHJ5TWFzcyBpcyBkaWZmZXJlbnQgYmV0d2VlbiBHTU8gYW5kIFdULiAKICAyLiBUaGUgcC12YWx1ZSBpcyAwLjAwMDM3OCwgdGhpcyBpcyBzbWFsbGVyIHRoYW4gMC4wNSB3aGljaCBtZWFucyB0aGF0IHRoZSBtYXNzZXMgd2VyZSBub3QgdGhlIHNhbWUgZm9yIGFsbCBvZiB0aGUgb3V0Y29tZXMuIAogIDMuIFllcyB3ZSBzaG91bGQgcnVuIGEgcG9zdC1ob2MgdGVzdC4gVGhlIHBvc3QtaG9jIHRlc3Qgc2hvd3MgdXMgdGhhdCBib3RoIE5INENsLUYxMCBhbmQgTkg0Tk8zLUYxMCBhcmUgc2lnbmlmaWNhbnRseSBkaWZmZXJlbnQgd2hpbGUgTkg0Tk8zLU5INENsIGhhcyBubyBzaWduaWZpY2FudCBkaWZmZXJlbmNlLiAKCiAgCioqKiAKCmBgYHtyfQpUdWtleUhTRChhb3YoUm9vdERyeU1hc3MgfiBmZXJ0LCBkYXRhID0gcmljZSkpCmBgYAoKKioqKgoKTGV0J3MgY3JlYXRlIGEgYm94cGxvdCBjb21wYXJpbmcgdGhlIFJvb3REcnlNYXNzIGZvciBlYWNoIGZlcnQKCmBgYHtyfQpib3hwbG90KFJvb3REcnlNYXNzIH4gZmVydCwKICAgICAgICBkYXRhID0gcmljZSwKICAgICAgICBtYWluID0gIlJvb3QgRHJ5IE1hc3MgKGZlcnQpIiwKICAgICAgICB4bGFiID0gIkZlcnRpbGl6ZXIiLAogICAgICAgIHlsYWIgPSAiUm9vdCBEcnkgTWFzcyIsCiAgICAgICAgY29sPWMoInBpbmsiLCJ5ZWxsb3ciLCJ0dXJxdW9pc2UiKSkKYGBgCgoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKIyMgRXhhbXBsZSAyClByZXZlbnRhdGl2ZSBjYXJlIGlzIGluY3JlZGlibHkgaW1wb3J0YW50IGZvciBmaW5kaW5nIGFuZCB0cmVhdGluZyBtYW55IGRpc2Vhc2UgYmVmb3JlIHRoZXkgYmVjb21lIGEgcHJvYmxlbS4gUmVzZWFyY2hlcnMgaGF2ZSBiZWVuIGNvbGxlY3RpbmcgZGF0YSBvbiBvbmUgb2YgdGhlIG1vc3QgY29tbW9uIHR5cGVzIG9mIGNhbmNlciwgYnJlYXN0IGNhbmNlciwgYW5kIGhvdyBpdCBhc3NvY2lhdGVzIHdpdGggaGF2aW5nIHByZXZlbnRhdGl2ZSBzY3JlZW5pbmcsIGluIHRoaXMgY2FzZSwgYSBtYW1tb2dyYW0uIFJlc2VhcmNoZXJzIGhhdmUgY29sbGVjdGVkIGRhdGEgb24gb3ZlciA4MCwwMDAgd29tZW4gb3ZlciB0aGUgbGFzdCBmZXcgZGVjYWRlcyBhbmQgaGF2ZSBiZWVuIHRyYWNraW5nIHdoZXRoZXIgd29tZW4gZGllZCBmcm9tIGJyZWFzdCBjYW5jZXIgYW5kIHdoZXRoZXIgdGhleSBoYWQgYSBtYW1tb2dyYW0uIFRoZXNlIHJlc2VhcmNoZXJzIGhhdmUgYXNrZWQgeW91IHRvIGxvb2sgaW50byBwb3NzaWJsZSBsaW5rcyBiZXR3ZWVuIHRoZSB0d28uIEhlcmUsIHNpbmNlIGJvdGggdmFyaWFibGVzIGFyZSBjYXRlZ29yaWNhbCwgeW91IHdpbGwgbmVlZCB0byB0ZXN0IGZvciBhc3NvY2lhdGlvbnMgdXNpbmcgYSBjaGktc3F1YXJlZCBjb250aW5nZW5jeSB0ZXN0LiBJbiB0aGUgY29kZSBjaHVuayBiZWxvdywgcGxlYXNlIHJlYWQgaW4gdGhlICJtYW1tb2dyYW0uY3N2IiBkYXRhc2V0IGFuZCBydW4gdGhlIGNoaS1zcXVhcmVkIGFuYWx5c2lzLgoKIyMjIyBBbnN3ZXI6CgpgYGB7cn0KbWFtbW9ncmFtIDwtIHJlYWQuY3N2KCJ+L0Rlc2t0b3AvQklONTEwLWZpbGVzL21hbW1vZ3JhbS5jc3YiKQpuYW1lcyhtYW1tb2dyYW0pCnRhYmxlKG1hbW1vZ3JhbSR0cmVhdG1lbnQsIG1hbW1vZ3JhbSRicmVhc3RfY2FuY2VyX2RlYXRoKQpjaGlzcS50ZXN0KHRhYmxlKG1hbW1vZ3JhbSR0cmVhdG1lbnQsIG1hbW1vZ3JhbSRicmVhc3RfY2FuY2VyX2RlYXRoKSkKYGBgCgoqKioqCjEuIFdoYXQgYXJlIHlvdXIgbnVsbCBhbmQgYWx0ZXJuYXRpdmUgaHlwb3RoZXNlcyBmb3IgdGhpcyB0ZXN0PwoyLiBIb3cgZG8geW91IGludGVycHJldCB0aGUgcC12YWx1ZSBmb3IgdGhpcyByZWxhdGlvbnNoaXA/CgojIyMjIEFuc3dlcnM6CgogIDEuIE51bGwgaHlwb3RoZXNpczogVGhlcmUgaXMgbm8gYXNzb2NpYXRpb24gYmV0d2VlbiBnZXR0aW5nIGEgbWFtbW9ncmFtIGFuZCBkZWF0aCBieSBicmVhc3QgY2FuY2VyLgogICAgIEFsdCBoeXBvdGhlc2lzOiBUaGVyZSBpcyBhIGNvcnJlbGF0aW9uIGJldHdlZW4gZ2V0dGluZyBhIG1hbW1vZ3JhbSBhbmQgZGVhdGggYnkgYnJlYXN0IGNhbmNlci4gCiAgMi4gU2luY2Ugd2Ugd2VyZSBnaXZlIDAuODk0OCwgd2UgYXJlIHVuYWJsZSB0byByZWplY3QgdGhlIG51bGwgaHlwb3RoZXNpcywgbWVhbmluZyB0aGVyZSBpcyBub3QgZW5vdWdoIGV2aWRlbmNlIHRvIHN1cHBvcnQgdGhhdCB0aGUgdHdvIGFyZSBjb3JyZWxhdGVkLiAKICAKTGV0J3MgY3JlYXRlIGEgbW9zYWljIHBsb3QgdG8gc2hvdyB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gY2FuY2VyIHN1cnZpdmFsIGFuZCB3aGV0aGVyIG9yIG5vdCBhIHBhdGllbnQgaGFkIGEgbWFtbW9ncmFtLgoKYGBge3J9Cm1vc2FpY3Bsb3QodGFibGUobWFtbW9ncmFtJHRyZWF0bWVudCwgbWFtbW9ncmFtJGJyZWFzdF9jYW5jZXJfZGVhdGgpLAogICAgICAgICAgIG1haW4gPSAiTWFtbW9ncmFtIGFuZCBCcmVhc3QgQ2FuY2VyIERlYXRoIiwKICAgICAgICAgICB4bGFiID0gIk1hbW1vZ3JhbSIsCiAgICAgICAgICAgeWxhYiA9ICJCcmVhc3QgQ2FuY2VyIERlYXRoIiwKICAgICAgICAgICBjb2w9IGMoImJsYWNrIiwiYmx1ZSIpKQpgYGAKCioqKioKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKIyMgRXhhbXBsZSAzOiBZT1UgQ0hPT1NFIFRIRSBSSUdIVCBURVNUIQoKTk9URTogZm9yIHdoYXRldmVyIHRlc3QgeW91IGVuZCB1cCBjaG9vc2luZywgeW91IHdpbGwgbmVlZCB0byBtYWtlIHN1cmUgeW91IGFkZHJlc3MgdGhlIGZvbGxvd2luZzoKMS4gdGVsbCBtZSBXSFkgeW91IGFyZSBjaG9vc2luZyB0aGUgdGVzdCB5b3UncmUgY2hvb3NpbmcKMi4gVGVsbCBtZSB5b3VyIG51bGwgYW5kIGFsdGVybmF0aXZlIGh5cG90aGVzZXMKMy4gSW50ZXJwcmV0IHRoZSBwLXZhbHVlIGFuZCB3aGF0IGl0IG1lYW5zIGZvciB0aGUgcG90ZW50aWFsIHJlbGF0aW9uc2hpcAo0LiBDcmVhdGUgYW4gYXBwcm9wcmlhdGUgZ3JhcGggZm9yIHRoZSByZWxhdGlvbnNoaXAuCjUuIElmIHlvdSBjaG9vc2UgYW4gQU5PVkEsIHlvdSBuZWVkIHRvIGRldGVybWluZSB3aGV0aGVyIG9yIG5vdCB5b3UgbmVlZCB0byBydW4gYSBwb3N0LWhvYyB0ZXN0ISBSdW4gaXQgaWYgeW91IGJlbGlldmUgaXQgaXMgbmVjZXNzYXJ5LgoKQmFja2dyb3VuZDogaW4gdGhlIGNvZGUgY2h1bmsgYmVsb3csIHBsZWFzZSBicmluZyBpbiAicG9pc29uLmNzdiIsIHlvdSBtYXkgY2FsbCB0aGUgZGF0YWZyYW1lIHdoYXRldmVyIHlvdSB3YW50LiBUaGUgZGF0YSBjb21lcyBmcm9tIGEgc3R1ZHkgbG9va2luZyBhdCBob3cgZGlmZmVyZW50IHR5cGVzIG9mIHBvaXNvbiBhbmQgdmFyaW91cyB0cmVhdG1lbnRzIChsZXZlbHMpIG9mIHRoZSBwb2lzb24gaW1wYWN0IGhvdyBsb25nIGFuIGFuaW1hbCBjYW4gc3Vydml2ZSBmb3IgKGRheXMpLiBHaXZlbiB0aGUgZGF0YSBmcm9tIHRoZSBleHBlcmltZW50LCB0aGUgcmVzZWFyY2hzIHdhbnQgeW91IHRvIGxvb2sgYXQgaG93IHRoZSBtZWFuIHRpbWUgdW50aWwgZGVhdGggZGlmZmVycyBieSAndHJlYXQnCgpISU5UOiB5b3UgY2FuIHJ1biBzb21lIG9mIHRoZSBpbml0aWFsIGNvZGUgd2UgbG9va2VkIGF0IGluIHRoZSBmaXJzdCBmZXcgUiBsYWJzIHRvIGRldGVybWluZSB0aGUgdHlwZXMgb2YgZGF0YSB5b3UgaGF2ZSBhcyB3ZWxsIGFzIHRoZSBsZXZlbHMgd2l0aGluIHRoZSB2YXJpYWJsZXMuCgojIyMjIEFuc3dlcnMKIDEuIEkgY2hvc2UgYSBvbmUtd2F5IEFOT1ZBIGJlY2F1c2UgdGltZSBpcyBudW1lcmljYWwgd2hpbGUgdHJlYXQgaXMgYSBjYXRlZ29yaWNhbCB2YXJpYWJsZS4gV2l0aCBBTk9WQSB3ZSBjYW4gY29tcGFyZSB0aGUgdHdvIHZhcmlhYmxlcy4KIDIuIE51bGwgaHlwb3RoZXNpczogVGhlIG1lYW4gc3Vydml2YWwgdGltZSBpcyB0aGUgc2FtZSBmb3IgYWxsIG9mIHRoZSB0cmVhdG1lbnQgZ3JvdXBzLiAKICAgIEFsdCBoeXBvdGhlc2lzOiBUaGUgbWVhbiBzdXJ2aXZhbCBkaWZmZXJzIGJldHdlZW4gdGhlIHRyZWF0bWVudCBncm91cHMuIAogMy4gVGhlIEFOT1ZBIGdhdmUgYSBwLXZhbHVlIG9mIDAuMDAwOTkyLCB0aGlzIGlzIGEgbG90IGxlc3MgdGhhbiAwLjA1IHdoaWNoIG1lYW5zIHRoYXQgd2UgcmVqZWN0IHRoZSBudWxsIGh5cG90aGVzaXMuIFNpbmNlIHRoZSBBTk9WQSBwcm92ZWQgdG8gYmUgc2lnbmlnaWNhbnQgSSByYW4gYSBwb3N0LWhvYyB0ZXN0IHRvIGRldGVybWluZSB0aGUgdHJlYXRtZW50IEIgd2FzIHNpZ25pZmljYW50bHkgbW9yZSBkaWZmZXJlbnQgZnJvbSBBIGFuZCBDLiAKCmBgYHtyfQpwb2lzb24gPC0gcmVhZC5jc3YoIn4vRGVza3RvcC9CSU41MTAtZmlsZXMvcG9pc29ucy5jc3YiKQpuYW1lcyhwb2lzb24pCnN0cihwb2lzb24pCnRhYmxlKHBvaXNvbiR0cmVhdCkKc3VtbWFyeShhb3YodGltZSB+IHRyZWF0LCBkYXRhID0gcG9pc29uKSkKVHVrZXlIU0QoYW92KHRpbWUgfiB0cmVhdCwgZGF0YSA9IHBvaXNvbikpCmJveHBsb3QodGltZSB+IHRyZWF0LAogICAgICAgIGRhdGEgPSBwb2lzb24sCiAgICAgICAgbWFpbiA9ICJTdXJ2aXZhbCBUaW1lIGJ5IFRyZWF0bWVudCIsCiAgICAgICAgeGxhYiA9ICJUcmVhdG1lbnQiLAogICAgICAgIHlsYWIgPSAiVGltZSBVbnRpbCBEZWF0aCAoZGF5cykiLAogICAgICAgIGNvbD0gYygieWVsbG93IiwibGlnaHRncmVlbiIsImxpZ2h0cGluayIsInZpb2xldCIpKQpgYGAKCg==