Propensity Scores and the Right Heart Catheterization Data
Thomas E. Love, Ph.D.
Last Update 2022-02-24 16:28:37
1 Preliminaries
1.1 Study Background
This example is based on the Right Heart Catheterization data set available at Vanderbilt University and described here. The key reference is Connors AF et al. 1996 The effectiveness of RHC in the initial care of critically ill patients. JAMA 276: 889-897. Connors et al. used a logistic regression model to develop a propensity score then: [a] matched RHC to non-RHC patients and [b] adjusted for propensity score in models for outcomes, followed by a sensitivity analysis. The key conclusions were that RHC patients had decreased survival time, and any unmeasured confounder would have to be somewhat strong to explain away the results.
1.2 Loading Packages
library(here); library(janitor); library(naniar)
library(tableone); library(broom); library(survival)
library(Matching); library(cobalt); library(rgenoud)
library(lme4); library(rbounds); library(conflicted)
library(twang); library(survey)
library(tidyverse)
conflict_prefer("select", "dplyr")
conflict_prefer("filter", "dplyr")
theme_set(theme_bw())1.3 Importing the Data
The observations in the rhc data set describe 5,735 subjects who are critically ill, and most variables were obtained during day 1 of a hospitalization. We’ll begin by importing the rhc data into a tibble (a lazy and surly data frame - see R’s help file on tibbles for more details) in R that contains 5,735 observations (rows) on 63 variables (columns).
In the import process, we’re going to do a few things to simplify the process of getting a functional data set.
- The automated column specification in R’s readr package uses the first 1000 rows of the data set - with the
rhcdata, this creates parsing failures for several measures which are assumed to be integers but actually should be parsed as doubles (continuous variables). So we specify the column types for those variables here then allow R to identify the type for the other columns. - By default, readr’s
read_csvcommand treats all string variables as characters, and not as factors. For several multicategorical variables, we want a factor representation in a specific order, so we’ll specify that, too, also using thecolsfunction.
column_types_rhc <-
cols(urin1 = "d", meanbp1 = "d", resp1 = "d",
swang1 = col_factor(c("RHC", "No RHC")),
death = col_factor(c("No", "Yes")),
sex = col_factor(c("Male", "Female")),
cat1 = col_factor(c("ARF", "CHF", "Cirrhosis", "Colon Cancer", "Coma", "COPD",
"Lung Cancer", "MOSF w/Malignancy", "MOSF w/Sepsis")),
dnr1 = col_factor(c("No", "Yes")),
card = col_factor(c("No", "Yes")),
gastr = col_factor(c("No", "Yes")),
hema = col_factor(c("No", "Yes")),
meta = col_factor(c("No", "Yes")),
neuro = col_factor(c("No", "Yes")),
ortho = col_factor(c("No", "Yes")),
renal = col_factor(c("No", "Yes")),
resp = col_factor(c("No", "Yes")),
seps = col_factor(c("No", "Yes")),
trauma = col_factor(c("No", "Yes")),
income = col_factor(c("Under $11k", "$11-$25k", "$25-$50k", "> $50k")),
ninsclas = col_factor(c("Private", "Private & Medicare", "Medicare",
"Medicare & Medicaid", "Medicaid", "No insurance")),
race = col_factor(c("white", "black", "other")),
ca = col_factor(c("No", "Yes", "Metastatic"))
)The data are available at the link below.
rhc_raw <- read_csv("https://biostat.app.vumc.org/wiki/pub/Main/DataSets/rhc.csv",
col_types = column_types_rhc)New names:
* `` -> ...1- After we import the data, we re-order the columns to present the SUPPORT patient identification code first, then the exposure (
swang1), then information on the outcomes, then information on the covariates of interest for our work, dropping the other variables.
rhc_cleaning <- rhc_raw %>%
select(ptid, swang1,
death, sadmdte, dschdte, dthdte, lstctdte,
age, sex, edu, income, ninsclas, race,
cat1, dnr1, wtkilo1, hrt1, meanbp1, resp1, temp1,
card, gastr, hema, meta, neuro, ortho, renal,
resp, seps, trauma,
amihx, ca, cardiohx, chfhx, chrpulhx, dementhx,
gibledhx, immunhx, liverhx, malighx, psychhx,
renalhx, transhx, aps1, das2d3pc, scoma1,
surv2md1, alb1, bili1, crea1, hema1, paco21,
pafi1, ph1, pot1, sod1, wblc1)1.4 The Data Set after Initial Import
The rhc_cleaning data at this stage is a tibble (a lazy and surly data frame) containing 5735 observations (rows) on 57 variables (columns). The observations describe subjects who are critically ill, and most variables were obtained during day 1 of a hospitalization.
rhc_cleaning# A tibble: 5,735 x 57
ptid swang1 death sadmdte dschdte dthdte lstctdte age sex edu income
<chr> <fct> <fct> <dbl> <dbl> <dbl> <dbl> <dbl> <fct> <dbl> <fct>
1 00005 No RHC No 11142 11151 NA 11382 70.3 Male 12 Under ~
2 00007 RHC Yes 11799 11844 11844 11844 78.2 Female 12 Under ~
3 00009 RHC No 12083 12143 NA 12400 46.1 Female 14.1 $25-$5~
4 00010 No RHC Yes 11146 11183 11183 11182 75.3 Female 9 $11-$2~
5 00011 RHC Yes 12035 12037 12037 12036 67.9 Male 9.95 Under ~
6 00012 No RHC No 12389 12396 NA 12590 86.1 Female 8 Under ~
7 00013 No RHC No 12381 12423 NA 12616 55.0 Male 14 $25-$5~
8 00014 No RHC Yes 11453 11487 11491 11490 43.6 Male 12 $25-$5~
9 00016 No RHC No 12426 12437 NA 12560 18.0 Female 12.8 Under ~
10 00017 RHC No 11381 11400 NA 11590 48.4 Female 11.0 Under ~
# ... with 5,725 more rows, and 46 more variables: ninsclas <fct>, race <fct>,
# cat1 <fct>, dnr1 <fct>, wtkilo1 <dbl>, hrt1 <dbl>, meanbp1 <dbl>,
# resp1 <dbl>, temp1 <dbl>, card <fct>, gastr <fct>, hema <fct>, meta <fct>,
# neuro <fct>, ortho <fct>, renal <fct>, resp <fct>, seps <fct>,
# trauma <fct>, amihx <dbl>, ca <fct>, cardiohx <dbl>, chfhx <dbl>,
# chrpulhx <dbl>, dementhx <dbl>, gibledhx <dbl>, immunhx <dbl>,
# liverhx <dbl>, malighx <dbl>, psychhx <dbl>, renalhx <dbl>, ...1.5 Any missingness?
rhc_cleaning %>%
miss_var_summary()# A tibble: 57 x 3
variable n_miss pct_miss
<chr> <int> <dbl>
1 dthdte 2013 35.1
2 dschdte 1 0.0174
3 ptid 0 0
4 swang1 0 0
5 death 0 0
6 sadmdte 0 0
7 lstctdte 0 0
8 age 0 0
9 sex 0 0
10 edu 0 0
# ... with 47 more rowsWe could also graph this with gg_miss_var(rhc_cleaning). As it turns out, there were three variables in the raw rhc data that have missing values, but we’ve omitted one (about day 1 urine output) in creating this rhc_cleaning file. The remaining two are dates, and we’ll address those in the material on creating our outcomes.
2 The Treatment / Exposure We’ll Study
The treatment/exposure we are studying is the presence of a Swan-Ganz (right heart) catheter on day 1, captured in the variable swang1. The Medline Plus description of right heart catheterization is here. Basically, the procedure passes a thin tube into the right side of the heart and the arteries leading to the lungs, to measure the heart’s function and blood flow. It’s done in very ill patients for several reasons, and is also referred to as right heart catheterization and pulmonary artery catheterization.
rhc_cleaning %>% tabyl(swang1) swang1 n percent
RHC 2184 0.3808195
No RHC 3551 0.6191805Of course, the most important issue to us is that each subject was not randomly allocated to either the “RHC” or “No RHC” group. Instead, this is an observational study, where each subject received their treatment (RHC or no) on the basis of what their providers thought would be best for them.
3 The 3 Outcomes We’ll Study
We will define three outcomes here: one binary, one quantitative, and one time-to-event.
- Our first outcome is in-study mortality, captured as a binary variable.
- We will then define hospital length of stay, a quantitative variable, and then
- Time to death, a time-to-event variable that will be right-censored for those who did not die during the study.
3.1 Mortality, a binary outcome: death
One outcome is death during the study, as captured in the death variable. This is a binary outcome.
rhc_cleaning %>% tabyl(death) death n percent
No 2013 0.3510026
Yes 3722 0.6489974Those people without a value for dthdte turn out to be the people with death values of “No” and this makes sense, because dthdte is a code for date of death.
3.3 Length of initial hospital stay, a quantitative outcome: hospdays
Our third outcome is length of initial hospital stay, which is captured by subtracting the study admission date (sadmdte) from the hospital discharge date (dschdte). This is a quantitative outcomes, and we will create it soon, and name it hospdays in our data set.
rhc_cleaning <- rhc_cleaning %>%
mutate(hospdays = dschdte - sadmdte)
mosaic::favstats(hospdays ~ swang1, data = rhc_cleaning) swang1 min Q1 median Q3 max mean sd n missing
1 RHC 2 8 16 31 342 24.69139 27.79865 2184 0
2 No RHC 2 7 12 22 338 19.52915 23.58672 3551 0It looks like all of these values are reasonable (in that they are all positive) and we have no missing values.
3.4 Survival Time in Days, a time-to-event outcome: survdays
Another outcome is survival time in days (which is censored for all patients who did not die during the study), and which is captured by subtracting the study admission date (sadmdte) from the death date (dthdte) for the patients who died in study, and the study admission date (sadmdte) from the date of last contact (lstctdte) for the patients who did not die in study. This is a time-to-event outcome, and we will create it soon, and name it survdays in our data set.
- For the subjects who died, this is straightforward, in that we take the death date and subtract the study admission date.
- For those who survived through the study, we take the date of last contact and subtract the study admission date, in order to get their (right censored) survival times.
rhc_cleaning <- rhc_cleaning %>%
mutate(survdays = ifelse(death == "Yes",
dschdte - sadmdte,
lstctdte - sadmdte))
mosaic::favstats(survdays ~ death, data = rhc_cleaning) death min Q1 median Q3 max mean sd n missing
1 No 2 187 210 236 1351 231.18082 112.24989 2013 0
2 Yes 2 6 12 23 338 19.99893 24.59443 3722 0Again, this looks reasonable. All of these values are strictly positive, for instance, and none seem wildly out of line, and we have no missing values. We also see that those who died had much shorter survival times (typically) than those who survived, which makes sense.
We might also look at the survdays variable broken down by RHC status.
mosaic::favstats(survdays ~ swang1, data = rhc_cleaning) swang1 min Q1 median Q3 max mean sd n missing
1 RHC 2 9 25.5 187 1351 91.75962 129.5763 2184 0
2 No RHC 2 9 25.0 191 1243 95.57871 117.7174 3551 0These seem quite comparable, so far, and again, they make sense (all are positive, and there are no missing values.)
4 The 50 Covariates We’ll Study
Our propensity model will eventually include 50 covariates, described below.
4.1 Group 1: Socio-demographics (6 covariates)
| Variable | Definition |
|---|---|
age |
Age in years |
sex |
Sex |
edu |
Years of Education |
income |
Income Category (4 levels) |
ninsclas |
Insurance Category (6 levels) |
race |
Self-Reported Race (3 levels) |
Here’s a Table 1 for these covariates.
vars01 <- c("age", "sex", "edu", "income", "ninsclas", "race")
table1_01 <-
CreateTableOne(vars = vars01, strata = "swang1",
data = rhc_cleaning, test = FALSE)
print(table1_01, smd = TRUE) Stratified by swang1
RHC No RHC SMD
n 2184 3551
age (mean (SD)) 60.75 (15.63) 61.76 (17.29) 0.061
sex = Female (%) 906 (41.5) 1637 (46.1) 0.093
edu (mean (SD)) 11.86 (3.16) 11.57 (3.13) 0.091
income (%) 0.142
Under $11k 1145 (52.4) 2081 (58.6)
$11-$25k 452 (20.7) 713 (20.1)
$25-$50k 393 (18.0) 500 (14.1)
> $50k 194 ( 8.9) 257 ( 7.2)
ninsclas (%) 0.194
Private 731 (33.5) 967 (27.2)
Private & Medicare 490 (22.4) 746 (21.0)
Medicare 511 (23.4) 947 (26.7)
Medicare & Medicaid 123 ( 5.6) 251 ( 7.1)
Medicaid 193 ( 8.8) 454 (12.8)
No insurance 136 ( 6.2) 186 ( 5.2)
race (%) 0.036
white 1707 (78.2) 2753 (77.5)
black 335 (15.3) 585 (16.5)
other 142 ( 6.5) 213 ( 6.0) - Two of these covariates show standardized mean differences larger than 0.10.
4.2 Group 2: Presentation at Admission (7 covariates)
| Variable | Definition |
|---|---|
cat1 |
Primary disease category (9 levels) |
dnr1 |
Do Not Resuscitate status, day 1 |
wtkilo1 |
Weight in kg, day 1 |
hrt1 |
Heart rate, day 1 |
meanbp1 |
Mean Blood Pressure, day 1 |
resp1 |
Respiratory rate, day 1 |
temp1 |
Temperature, C, day 1 |
vars02 <- c("cat1", "dnr1", "wtkilo1", "hrt1", "meanbp1",
"resp1", "temp1")
table1_02 <-
CreateTableOne(vars = vars02, strata = "swang1",
data = rhc_cleaning, test = FALSE)
print(table1_02, smd = TRUE) Stratified by swang1
RHC No RHC SMD
n 2184 3551
cat1 (%) 0.583
ARF 909 (41.6) 1581 (44.5)
CHF 209 ( 9.6) 247 ( 7.0)
Cirrhosis 49 ( 2.2) 175 ( 4.9)
Colon Cancer 1 ( 0.0) 6 ( 0.2)
Coma 95 ( 4.3) 341 ( 9.6)
COPD 58 ( 2.7) 399 (11.2)
Lung Cancer 5 ( 0.2) 34 ( 1.0)
MOSF w/Malignancy 158 ( 7.2) 241 ( 6.8)
MOSF w/Sepsis 700 (32.1) 527 (14.8)
dnr1 = Yes (%) 155 ( 7.1) 499 (14.1) 0.228
wtkilo1 (mean (SD)) 72.36 (27.73) 65.04 (29.50) 0.256
hrt1 (mean (SD)) 118.93 (41.47) 112.87 (40.94) 0.147
meanbp1 (mean (SD)) 68.20 (34.24) 84.87 (38.87) 0.455
resp1 (mean (SD)) 26.65 (14.17) 28.98 (13.95) 0.165
temp1 (mean (SD)) 37.59 (1.83) 37.63 (1.74) 0.021- Six of these covariates show standardized mean differences larger than 0.10.
4.3 Group 3: Admission Diagnosis Categories (10 covariates)
| Variable | Definition |
|---|---|
card |
Cardiovascular diagnosis |
gastr |
Gastrointestinal diagnosis |
hema |
Hematologic diagnosis |
meta |
Metabolic diagnosis |
neuro |
Neurological diagnosis |
ortho |
Orthopedic diagnosis |
renal |
Renal diagnosis |
resp |
Respiratory diagnosis |
seps |
Sepsis diagnosis |
trauma |
Trauma diagnosis |
vars03 <- c("card", "gastr", "hema", "meta", "neuro", "ortho",
"renal", "resp", "seps", "trauma")
table1_03 <-
CreateTableOne(vars = vars03, strata = "swang1",
data = rhc_cleaning, test = FALSE)
print(table1_03, smd = TRUE) Stratified by swang1
RHC No RHC SMD
n 2184 3551
card = Yes (%) 924 (42.3) 1007 (28.4) 0.295
gastr = Yes (%) 420 (19.2) 522 (14.7) 0.121
hema = Yes (%) 115 ( 5.3) 239 ( 6.7) 0.062
meta = Yes (%) 93 ( 4.3) 172 ( 4.8) 0.028
neuro = Yes (%) 118 ( 5.4) 575 (16.2) 0.353
ortho = Yes (%) 4 ( 0.2) 3 ( 0.1) 0.027
renal = Yes (%) 148 ( 6.8) 147 ( 4.1) 0.116
resp = Yes (%) 632 (28.9) 1481 (41.7) 0.270
seps = Yes (%) 516 (23.6) 515 (14.5) 0.234
trauma = Yes (%) 34 ( 1.6) 18 ( 0.5) 0.104- Seven of these covariates show standardized mean differences larger than 0.10.
4.4 Group 4: Comorbid Illness and Transfer Status (13 covariates)
| Variable | Definition |
|---|---|
amihx |
Definite Myocardial Infarction |
ca |
Cancer (3 levels) |
cardiohx |
Acute MI, Peripheral Vascular Disease, Severe Cardiovascular Symptoms (NYHA-Class III), Very Severe Cardiovascular Symptoms (NYHA- IV) |
chfhx |
Congestive Heart Failure |
chrpulhx |
Chronic Pulmonary Disease, Severe or Very Severe Pulmonary Disease |
dementhx |
Dementia, Stroke or Cerebral Infarct, Parkinson’s Disease |
gibledhx |
Upper GI Bleeding |
immunhx |
Immunosupperssion, Organ Transplant, HIV Positivity, Diabetes Mellitus Without End Organ Damage, Diabetes Mellitus With End Organ Damage, Connective Tissue Disease |
liverhx |
Cirrhosis, Hepatic Failure |
malighx |
Solid Tumor, Metastatic Disease, Chronic Leukemia/Myeloma, Acute Leukemia, Lymphoma |
psychhx |
Psychiatric History, Active Psychosis or Severe Depression |
renalhx |
Chronic Renal Disease, Chronic Hemodialysis or Peritoneal Dialysis |
transhx |
Transfer (> 24 Hours) from Another Hospital |
vars04 <- c("amihx", "ca", "cardiohx", "chfhx", "chrpulhx", "dementhx",
"gibledhx", "immunhx", "liverhx", "malighx", "psychhx", "renalhx", "transhx")
table1_04 <-
CreateTableOne(vars = vars04, strata = "swang1",
data = rhc_cleaning, test = FALSE)
print(table1_04, smd = TRUE) Stratified by swang1
RHC No RHC SMD
n 2184 3551
amihx (mean (SD)) 0.04 (0.20) 0.03 (0.17) 0.074
ca (%) 0.107
No 1727 (79.1) 2652 (74.7)
Yes 334 (15.3) 638 (18.0)
Metastatic 123 ( 5.6) 261 ( 7.4)
cardiohx (mean (SD)) 0.20 (0.40) 0.16 (0.37) 0.116
chfhx (mean (SD)) 0.19 (0.40) 0.17 (0.37) 0.069
chrpulhx (mean (SD)) 0.14 (0.35) 0.22 (0.41) 0.192
dementhx (mean (SD)) 0.07 (0.25) 0.12 (0.32) 0.163
gibledhx (mean (SD)) 0.02 (0.16) 0.04 (0.19) 0.070
immunhx (mean (SD)) 0.29 (0.45) 0.26 (0.44) 0.080
liverhx (mean (SD)) 0.06 (0.24) 0.07 (0.26) 0.049
malighx (mean (SD)) 0.20 (0.40) 0.25 (0.43) 0.101
psychhx (mean (SD)) 0.05 (0.21) 0.08 (0.27) 0.143
renalhx (mean (SD)) 0.05 (0.21) 0.04 (0.20) 0.032
transhx (mean (SD)) 0.15 (0.36) 0.09 (0.29) 0.170- Seven of these covariates show standardized mean differences larger than 0.10.
4.5 Group 5: Day 1 Summary Measures of Presentation / Severity of Illness (4 covariates)
| Variable | Definition |
|---|---|
aps1 |
APACHE III score ignoring Coma, day 1 |
das2d3pc |
DASI (Duke Activity Status Index prior to admission) |
scoma1 |
Support Coma score based on Glasgow, day 1 |
surv2md1 |
SUPPORT model estimate of Prob(surviving 2 months) |
vars05 <- c("aps1", "das2d3pc", "scoma1", "surv2md1")
table1_05 <-
CreateTableOne(vars = vars05, strata = "swang1",
data = rhc_cleaning, test = FALSE)
print(table1_05, smd = TRUE) Stratified by swang1
RHC No RHC SMD
n 2184 3551
aps1 (mean (SD)) 60.74 (20.27) 50.93 (18.81) 0.501
das2d3pc (mean (SD)) 20.70 (5.03) 20.37 (5.48) 0.063
scoma1 (mean (SD)) 18.97 (28.26) 22.25 (31.37) 0.110
surv2md1 (mean (SD)) 0.57 (0.20) 0.61 (0.19) 0.198- Three of these covariates show standardized mean differences larger than 0.10.
4.6 Group 6: Day 1 Lab Results (10 covariates)
| Variable | Definition |
|---|---|
alb1 |
Albumin |
bili1 |
Bilirubin |
crea1 |
Serum Creatinine |
hema1 |
Hematocrit |
paco21 |
PaCo2 |
pafi1 |
PaO2/(.01 FIO2) |
ph1 |
Serum PH (arterial) |
pot1 |
Serum Potassium |
sod1 |
Serum Sodium |
wblc1 |
White blood cell count (4 subjects with value 0) |
vars06 <- c("alb1", "bili1", "crea1", "hema1", "paco21",
"pafi1", "pot1", "sod1", "wblc1")
table1_06 <-
CreateTableOne(vars = vars06, strata = "swang1",
data = rhc_cleaning, test = FALSE)
print(table1_06, smd = TRUE) Stratified by swang1
RHC No RHC SMD
n 2184 3551
alb1 (mean (SD)) 2.98 (0.93) 3.16 (0.67) 0.230
bili1 (mean (SD)) 2.71 (5.33) 2.00 (4.43) 0.145
crea1 (mean (SD)) 2.47 (2.05) 1.92 (2.03) 0.270
hema1 (mean (SD)) 30.51 (7.42) 32.70 (8.79) 0.269
paco21 (mean (SD)) 36.79 (10.97) 39.95 (14.24) 0.249
pafi1 (mean (SD)) 192.43 (105.54) 240.63 (116.66) 0.433
pot1 (mean (SD)) 4.05 (1.01) 4.08 (1.04) 0.027
sod1 (mean (SD)) 136.33 (7.60) 137.04 (7.68) 0.092
wblc1 (mean (SD)) 16.27 (12.55) 15.26 (11.41) 0.084- Six of these covariates show standardized mean differences larger than 0.10.
So, in all, 31 of the 50 covariates show standardized mean differences that exceed 0.10 before any propensity score adjustment.
5 A Little Refactoring
For our outcome, and our treatment, it will be useful to have 1/0 versions of the variables, and it will also be helpful to have the factor versions releveled to put the outcome of interest (death) first and the treatment of interest (RHC) first.
rhc <- rhc_cleaning %>%
mutate(treat_rhc = as.numeric(swang1 == "RHC"),
swang1 = fct_relevel(swang1, "RHC"),
death = fct_relevel(death, "Yes"),
died = as.numeric(death == "Yes"))5.1 Saving the Data File
saveRDS(rhc, here("data", "rhc.Rds"))6 Setting a Seed
I’m going to set a seed for random numbers before we start, which I can change later if I like.
set.seed(50012345)7 Unadjusted Outcome Assessments
Before we estimate a propensity score, we’ll perform unadjusted assessments to describe the impact of RHC (or not) on our three outcomes, without adjustment for any covariates at all.
7.1 Outcome A: In-Study Mortality (binary)
rhc %>% tabyl(swang1, death) %>%
adorn_totals() %>%
adorn_percentages() %>%
adorn_pct_formatting() %>%
adorn_ns(position = "front") %>%
adorn_title death
swang1 Yes No
RHC 1486 (68.0%) 698 (32.0%)
No RHC 2236 (63.0%) 1315 (37.0%)
Total 3722 (64.9%) 2013 (35.1%)The odds ratio of death associated with RHC as opposed to No RHC should be larger than 1, specifically, it should be
\[ \frac{1486 \times 1315}{2236 \times 698} = 1.25 \]
Let’s see if that’s what we get.
7.1.1 Fitting the Model with the 1/0 treatment and outcome
death_unadj <- glm(died ~ treat_rhc, data = rhc, family = binomial())
death_unadj
Call: glm(formula = died ~ treat_rhc, family = binomial(), data = rhc)
Coefficients:
(Intercept) treat_rhc
0.5309 0.2248
Degrees of Freedom: 5734 Total (i.e. Null); 5733 Residual
Null Deviance: 7433
Residual Deviance: 7418 AIC: 7422tidy(death_unadj, conf.int = TRUE, exponentiate = TRUE)# A tibble: 2 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 1.70 0.0348 15.3 1.11e-52 1.59 1.82
2 treat_rhc 1.25 0.0576 3.90 9.43e- 5 1.12 1.407.1.2 Fitting the Model with the treatment and outcome as factors requires care
Alternatively, we could just work with the factors, but then, it’s important to be sure what is actually being modeled. Do this by making the outcome and treatment logical results, as follows.
modelA_unadj1 <- glm((death == "Yes") ~ (swang1 == "RHC"), data = rhc, family = binomial())
tidy(modelA_unadj1, conf.int = TRUE, exponentiate = TRUE)# A tibble: 2 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 "(Intercept)" 1.70 0.0348 15.3 1.11e-52 1.59 1.82
2 "swang1 == \"RHC\"TR~ 1.25 0.0576 3.90 9.43e- 5 1.12 1.40Failing to do this can cause trouble:
modelA_unadj_bad <- glm(death ~ swang1, data = rhc, family = binomial())
tidy(modelA_unadj_bad, conf.int = TRUE, exponentiate = TRUE)# A tibble: 2 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 0.470 0.0459 -16.5 6.32e-61 0.429 0.514
2 swang1No RHC 1.25 0.0576 3.90 9.43e- 5 1.12 1.40 Note that this last version is actually predicting “Death = No” on the basis of “swang1 = No RHC”, which can be extremely confusing.
And if you did this:
modelA_unadj_bad2 <- glm(death ~ swang1 == "RHC", data = rhc, family = binomial())
tidy(modelA_unadj_bad2, conf.int = TRUE, exponentiate = TRUE)# A tibble: 2 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 "(Intercept)" 0.588 0.0348 -15.3 1.11e-52 0.549 0.629
2 "swang1 == \"RHC\"TR~ 0.799 0.0576 -3.90 9.43e- 5 0.713 0.894Now, you’ve inverted the odds ratio! Not good. So, either use 0 and 1 (with 1 being the result you want to study), or use the factors but specify the direction for both treatment and outcome as a logical statement.
7.2 Outcome B: Length of Stay (quantitative)
hospdays_unadj <- lm(hospdays ~ treat_rhc, data = rhc)
hospdays_unadj
Call:
lm(formula = hospdays ~ treat_rhc, data = rhc)
Coefficients:
(Intercept) treat_rhc
19.529 5.162 tidy(hospdays_unadj, conf.int = TRUE)# A tibble: 2 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 19.5 0.424 46.0 0 18.7 20.4
2 treat_rhc 5.16 0.687 7.51 6.76e-14 3.81 6.517.3 Outcome C: Time to death (time-to-event)
Here’s the data on survdays and death for our first six patients:
rhc %>% select(survdays, death) %>% head()# A tibble: 6 x 2
survdays death
<dbl> <fct>
1 240 No
2 45 Yes
3 317 No
4 37 Yes
5 2 Yes
6 201 No Here, we have a (right-censored) time to event outcome, called survdays, and an indicator of censoring (death which is “Yes” if the patient’s time is real and not censored). So the first subject should have a time to death of 240 or more days, because they were still alive when the study ended, while the second should have a time that is exactly 45 days, because that person died during the study. The survival object we need, then, is:
Surv(rhc$survdays, rhc$death == "Yes") %>% head()[1] 240+ 45 317+ 37 2 201+survdays_unadj <- coxph(Surv(survdays, death == "Yes") ~ treat_rhc, data = rhc)
survdays_unadjCall:
coxph(formula = Surv(survdays, death == "Yes") ~ treat_rhc, data = rhc)
coef exp(coef) se(coef) z p
treat_rhc 0.07841 1.08156 0.03347 2.342 0.0192
Likelihood ratio test=5.46 on 1 df, p=0.0195
n= 5735, number of events= 3722 tidy(survdays_unadj, conf.int = TRUE, exponentiate = TRUE)# A tibble: 1 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.08 0.0335 2.34 0.0192 1.01 1.158 Estimate the Propensity Score with Logistic Regression
propensity_model <-
glm(swang1 == "RHC" ~
age + sex + edu + income + ninsclas + race +
cat1 + dnr1 + wtkilo1 + hrt1 + meanbp1 +
resp1 + temp1 +
card + gastr + hema + meta + neuro + ortho +
renal + resp + seps + trauma +
amihx + ca + cardiohx + chfhx + chrpulhx +
dementhx + gibledhx + immunhx + liverhx +
malighx + psychhx + renalhx + transhx +
aps1 + das2d3pc + scoma1 + surv2md1 +
alb1 + bili1 + crea1 + hema1 + paco21 + pafi1 +
ph1 + pot1 + sod1 + wblc1,
family = binomial(link = "logit"), data = rhc)8.1 Store the propensity scores and linear propensity scores
rhc <- rhc %>%
mutate(ps = fitted(propensity_model),
linps = propensity_model$linear.predictors)
rhc %>% select(ptid, ps, linps) %>% head()# A tibble: 6 x 3
ptid ps linps
<chr> <dbl> <dbl>
1 00005 0.351 -0.615
2 00007 0.669 0.704
3 00009 0.634 0.547
4 00010 0.369 -0.537
5 00011 0.446 -0.218
6 00012 0.0553 -2.84 8.2 Plot the Propensity Scores to Verify that they Match Your Expectations
ggplot(rhc, aes(x = ps, fill = swang1)) +
geom_density(alpha = 0.5) +
scale_fill_viridis_d(option = "plasma") +
theme_bw()
ggplot(rhc, aes(x = swang1, y = ps)) +
geom_violin(aes(fill = swang1)) +
geom_boxplot(width = 0.2) +
scale_fill_viridis_d(option = "plasma") +
guides(fill = "none") +
coord_flip() +
theme_bw()
OK. We’re trying to use ps to estimate the probability of having a RHC, and it looks the people who actually had RHC have higher PS, on average, so that’s promising.
9 Checking Rubin’s Rules Prior to Propensity Adjustment
Rubin Rule 1 result should ideally be near 0, and certainly between -50 and +50.
rubin1.unadj <- with(rhc,
abs(100*(mean(linps[swang1=="RHC"]) -
mean(linps[swang1=="No RHC"]))/
sd(linps)))
rubin1.unadj[1] 101.583Rubin Rule 2 result should ideally be near 1, and certainly between 1/2 and 2.
rubin2.unadj <- with(rhc,
var(linps[swang1 == "RHC"]) /
var(linps[swang1 == "No RHC"]))
rubin2.unadj[1] 0.7359463So, we’ve got some work to do. Let’s try matching. In several different ways.
10 Match 1: 1:1 Greedy Matching on the linear PS, without Replacement
X <- rhc$linps
Tr <- as.logical(rhc$swang1 == "RHC")
match1 <- Match(Tr = Tr, X = X, M = 1,
estimand = "ATT", replace = FALSE,
ties = FALSE)
summary(match1)
Estimate... 0
SE......... 0
T-stat..... NaN
p.val...... NA
Original number of observations.............. 5735
Original number of treated obs............... 2184
Matched number of observations............... 2184
Matched number of observations (unweighted). 2184 10.1 Love Plot for Match 1
b1 <- bal.tab(match1,
swang1 == "RHC" ~
age + sex + edu + income + ninsclas + race +
cat1 + dnr1 + wtkilo1 + hrt1 + meanbp1 +
resp1 + temp1 +
card + gastr + hema + meta + neuro + ortho +
renal + resp + seps + trauma +
amihx + ca + cardiohx + chfhx + chrpulhx +
dementhx + gibledhx + immunhx + liverhx +
malighx + psychhx + renalhx + transhx +
aps1 + das2d3pc + scoma1 + surv2md1 +
alb1 + bili1 + crea1 + hema1 + paco21 + pafi1 +
ph1 + pot1 + sod1 + wblc1 + ps + linps,
data=rhc, un = TRUE)
love.plot(b1, threshold = .1, size = 1.5, stars = "raw",
var.order = "unadjusted",
title = "Standardized Differences and Match 1") +
theme_bw()
10.2 Rubin’s Rules 1 and 2 for Match 1
Create a new data frame, containing only the matched sample.
matches_1 <- factor(rep(match1$index.treated, 2))
rhc.matches_1 <- cbind(matches_1,
rhc[c(match1$index.control, match1$index.treated),])Rubin Rule 1 result after matching should ideally be near 0, and certainly between -50 and +50.
rubin1.m1 <- with(rhc.matches_1,
abs(100*(mean(linps[swang1=="RHC"]) -
mean(linps[swang1=="No RHC"]))/
sd(linps)))
rubin1.m1[1] 61.88315Rubin Rule 2 result after matching should ideally be near 1, and certainly between 1/2 and 2.
rubin2.m1 <- with(rhc.matches_1,
var(linps[swang1 == "RHC"]) /
var(linps[swang1 == "No RHC"]))
rubin2.m1[1] 1.68728811 Match 2: 1:2 Greedy Matching on the linear PS, without Replacement
X <- rhc$linps
Tr <- as.logical(rhc$swang1 == "RHC")
match2 <- Match(Tr = Tr, X = X, M = 2,
estimand = "ATT", replace = FALSE,
ties = FALSE)
summary(match2)
Estimate... 0
SE......... 0
T-stat..... NaN
p.val...... NA
Original number of observations.............. 5735
Original number of treated obs............... 2184
Matched number of observations............... 1775
Matched number of observations (unweighted). 3550 11.1 Love Plot for Match 2
b2 <- bal.tab(match2,
swang1 == "RHC" ~
age + sex + edu + income + ninsclas + race +
cat1 + dnr1 + wtkilo1 + hrt1 + meanbp1 +
resp1 + temp1 +
card + gastr + hema + meta + neuro + ortho +
renal + resp + seps + trauma +
amihx + ca + cardiohx + chfhx + chrpulhx +
dementhx + gibledhx + immunhx + liverhx +
malighx + psychhx + renalhx + transhx +
aps1 + das2d3pc + scoma1 + surv2md1 +
alb1 + bili1 + crea1 + hema1 + paco21 + pafi1 +
ph1 + pot1 + sod1 + wblc1 + ps + linps,
data=rhc, un = TRUE)
love.plot(b2, threshold = .1, size = 1.5, stars = "raw",
var.order = "unadjusted",
title = "Standardized Differences and Match 2") +
theme_bw()
11.2 Rubin’s Rules 1 and 2 for Match 2
Create a new data frame, containing only the matched sample.
matches_2 <- factor(rep(match2$index.treated, 2))
rhc.matches_2 <- cbind(matches_2,
rhc[c(match2$index.control, match2$index.treated),])Rubin Rule 1 result after matching should ideally be near 0, and certainly between -50 and +50.
rubin1.m2 <- with(rhc.matches_2,
abs(100*(mean(linps[swang1=="RHC"]) -
mean(linps[swang1=="No RHC"]))/
sd(linps)))
rubin1.m2[1] 101.2148Rubin Rule 2 result after matching should ideally be near 1, and certainly between 1/2 and 2.
rubin2.m2 <- with(rhc.matches_2,
var(linps[swang1 == "RHC"]) /
var(linps[swang1 == "No RHC"]))
rubin2.m2[1] 0.756918712 Match 3: Genetic Search Matching to do 1:1 Matching without replacement
The GenMatch function finds optimal balance using multivariate matching where a genetic search algorithm determines the weight that each covariate is given.
Please note that I am using way too small values of pop.size (especially) and probably max.generations, too, so that things run quickly. I am also only matching on the propensity score and linear propensity score, rather than on the individual covariates. For a course project, I recommend you try matching on the individual covariates if you can, but this reduction in the paramater values is fine.
For actual publications, follow the recommendations of the GenMatch algorithm, as described in the Matching and genoud package help files. The default values (which may be too small themselves) are 100 for both pop.size and max.generations.
X <- cbind(rhc$linps, rhc$ps)
Tr <- as.logical(rhc$swang1 == "RHC")
genout3 <- GenMatch(Tr = Tr, X = X,
estimand = "ATT", M = 1,
pop.size = 10, max.generations = 10,
wait.generations = 4, verbose = FALSE)
Thu Feb 24 16:28:58 2022
Domains:
0.000000e+00 <= X1 <= 1.000000e+03
0.000000e+00 <= X2 <= 1.000000e+03
Data Type: Floating Point
Operators (code number, name, population)
(1) Cloning........................... 0
(2) Uniform Mutation.................. 1
(3) Boundary Mutation................. 1
(4) Non-Uniform Mutation.............. 1
(5) Polytope Crossover................ 1
(6) Simple Crossover.................. 2
(7) Whole Non-Uniform Mutation........ 1
(8) Heuristic Crossover............... 2
(9) Local-Minimum Crossover........... 0
SOFT Maximum Number of Generations: 10
Maximum Nonchanging Generations: 4
Population size : 10
Convergence Tolerance: 1.000000e-03
Not Using the BFGS Derivative Based Optimizer on the Best Individual Each Generation.
Not Checking Gradients before Stopping.
Using Out of Bounds Individuals.
Maximization Problem.
GENERATION: 0 (initializing the population)
Lexical Fit..... 9.956314e-02 1.374329e-01 1.000000e+00 1.000000e+00
#unique......... 10, #Total UniqueCount: 10
var 1:
best............ 2.557446e+02
mean............ 4.477418e+02
variance........ 9.086363e+04
var 2:
best............ 1.546012e+02
mean............ 2.039613e+02
variance........ 1.939734e+04
GENERATION: 1
Lexical Fit..... 1.311405e-01 2.250274e-01 1.000000e+00 1.000000e+00
#unique......... 5, #Total UniqueCount: 15
var 1:
best............ 1.182693e+02
mean............ 2.121052e+02
variance........ 6.281992e+03
var 2:
best............ 8.855767e+02
mean............ 2.551477e+02
variance........ 4.781941e+04
GENERATION: 2
Lexical Fit..... 1.311405e-01 2.250274e-01 1.000000e+00 1.000000e+00
#unique......... 8, #Total UniqueCount: 23
var 1:
best............ 1.182693e+02
mean............ 1.605890e+02
variance........ 6.899419e+03
var 2:
best............ 8.855767e+02
mean............ 7.395392e+02
variance........ 7.794576e+04
GENERATION: 3
Lexical Fit..... 1.311405e-01 2.250274e-01 1.000000e+00 1.000000e+00
#unique......... 8, #Total UniqueCount: 31
var 1:
best............ 1.182693e+02
mean............ 1.654700e+02
variance........ 2.088027e+04
var 2:
best............ 8.855767e+02
mean............ 8.976217e+02
variance........ 5.297551e+02
GENERATION: 4
Lexical Fit..... 1.311405e-01 2.250274e-01 1.000000e+00 1.000000e+00
#unique......... 7, #Total UniqueCount: 38
var 1:
best............ 1.182693e+02
mean............ 1.682741e+02
variance........ 2.316481e+04
var 2:
best............ 8.855767e+02
mean............ 8.898565e+02
variance........ 2.827973e+01
GENERATION: 5
Lexical Fit..... 1.311405e-01 2.250274e-01 1.000000e+00 1.000000e+00
#unique......... 7, #Total UniqueCount: 45
var 1:
best............ 1.182693e+02
mean............ 1.108716e+02
variance........ 3.545226e+02
var 2:
best............ 8.855767e+02
mean............ 8.903722e+02
variance........ 4.330907e+01
GENERATION: 6
Lexical Fit..... 1.311405e-01 2.250274e-01 1.000000e+00 1.000000e+00
#unique......... 9, #Total UniqueCount: 54
var 1:
best............ 1.182693e+02
mean............ 2.017702e+02
variance........ 6.331150e+04
var 2:
best............ 8.855767e+02
mean............ 8.819291e+02
variance........ 2.067346e+02
'wait.generations' limit reached.
No significant improvement in 4 generations.
Solution Lexical Fitness Value:
1.311405e-01 2.250274e-01 1.000000e+00 1.000000e+00
Parameters at the Solution:
X[ 1] : 1.182693e+02
X[ 2] : 8.855767e+02
Solution Found Generation 1
Number of Generations Run 6
Thu Feb 24 16:29:21 2022
Total run time : 0 hours 0 minutes and 23 secondsmatch3 <- Match(Tr = Tr, X = X, estimand = "ATT",
Weight.matrix = genout3)
summary(match3)
Estimate... 0
SE......... 0
T-stat..... NaN
p.val...... NA
Original number of observations.............. 5735
Original number of treated obs............... 2184
Matched number of observations............... 2184
Matched number of observations (unweighted). 2259 12.1 Love Plot for Match 3
b3 <- bal.tab(match3,
swang1 == "RHC" ~
age + sex + edu + income + ninsclas + race +
cat1 + dnr1 + wtkilo1 + hrt1 + meanbp1 +
resp1 + temp1 +
card + gastr + hema + meta + neuro + ortho +
renal + resp + seps + trauma +
amihx + ca + cardiohx + chfhx + chrpulhx +
dementhx + gibledhx + immunhx + liverhx +
malighx + psychhx + renalhx + transhx +
aps1 + das2d3pc + scoma1 + surv2md1 +
alb1 + bili1 + crea1 + hema1 + paco21 + pafi1 +
ph1 + pot1 + sod1 + wblc1 + ps + linps,
data=rhc, un = TRUE)
love.plot(b3, threshold = .1, size = 1.5, stars = "raw",
var.order = "unadjusted",
title = "Standardized Differences and Match 3") +
theme_bw()
12.2 Rubin’s Rules 1 and 2 for Match 3
Create a new data frame, containing only the matched sample.
matches_3 <- factor(rep(match3$index.treated, 2))
rhc.matches_3 <- cbind(matches_3,
rhc[c(match3$index.control, match3$index.treated),])Rubin Rule 1 result after matching should ideally be near 0, and certainly between -50 and +50.
rubin1.m3 <- with(rhc.matches_3,
abs(100*(mean(linps[swang1=="RHC"]) -
mean(linps[swang1=="No RHC"]))/
sd(linps)))
rubin1.m3[1] 0.04940705Rubin Rule 2 result after matching should ideally be near 1, and certainly between 1/2 and 2.
rubin2.m3 <- with(rhc.matches_3,
var(linps[swang1 == "RHC"]) /
var(linps[swang1 == "No RHC"]))
rubin2.m3[1] 1.00256113 Match 4: 1:1 Greedy Matching on the linear PS, WITH Replacement
X <- rhc$linps
Tr <- as.logical(rhc$swang1 == "RHC")
match4 <- Match(Tr = Tr, X = X, M = 1,
estimand = "ATT", replace = TRUE,
ties = FALSE)
summary(match4)
Estimate... 0
SE......... 0
T-stat..... NaN
p.val...... NA
Original number of observations.............. 5735
Original number of treated obs............... 2184
Matched number of observations............... 2184
Matched number of observations (unweighted). 2184 13.1 Love Plot for Match 4
b4 <- bal.tab(match4,
swang1 == "RHC" ~
age + sex + edu + income + ninsclas + race +
cat1 + dnr1 + wtkilo1 + hrt1 + meanbp1 +
resp1 + temp1 +
card + gastr + hema + meta + neuro + ortho +
renal + resp + seps + trauma +
amihx + ca + cardiohx + chfhx + chrpulhx +
dementhx + gibledhx + immunhx + liverhx +
malighx + psychhx + renalhx + transhx +
aps1 + das2d3pc + scoma1 + surv2md1 +
alb1 + bili1 + crea1 + hema1 + paco21 + pafi1 +
ph1 + pot1 + sod1 + wblc1 + ps + linps,
data=rhc, un = TRUE)
love.plot(b4, threshold = .1, size = 1.5, stars = "raw",
var.order = "unadjusted",
title = "Standardized Differences and Match 4") +
theme_bw()
13.2 Rubin’s Rules 1 and 2 for Match 4
Create a new data frame, containing only the matched sample.
matches_4 <- factor(rep(match4$index.treated, 2))
rhc.matches_4 <- cbind(matches_4,
rhc[c(match4$index.control, match4$index.treated),])Rubin Rule 1 result after matching should ideally be near 0, and certainly between -50 and +50.
rubin1.m4 <- with(rhc.matches_4,
abs(100*(mean(linps[swang1=="RHC"]) -
mean(linps[swang1=="No RHC"]))/
sd(linps)))
rubin1.m4[1] 0.05521448Rubin Rule 2 result after matching should ideally be near 1, and certainly between 1/2 and 2.
rubin2.m4 <- with(rhc.matches_4,
var(linps[swang1 == "RHC"]) /
var(linps[swang1 == "No RHC"]))
rubin2.m4[1] 1.00286814 Match 5: 1:1 Caliper Matching on the linear PS, WITHOUT Replacement
X <- rhc$linps
Tr <- as.logical(rhc$swang1 == "RHC")
match5 <- Match(Tr = Tr, X = X, M = 1,
caliper = 0.2,
estimand = "ATT", replace = FALSE,
ties = FALSE)
summary(match5)
Estimate... 0
SE......... 0
T-stat..... NaN
p.val...... NA
Original number of observations.............. 5735
Original number of treated obs............... 2184
Matched number of observations............... 1563
Matched number of observations (unweighted). 1563
Caliper (SDs)........................................ 0.2
Number of obs dropped by 'exact' or 'caliper' 621 14.1 Love Plot for Match 5
b5 <- bal.tab(match5,
swang1 == "RHC" ~
age + sex + edu + income + ninsclas + race +
cat1 + dnr1 + wtkilo1 + hrt1 + meanbp1 +
resp1 + temp1 +
card + gastr + hema + meta + neuro + ortho +
renal + resp + seps + trauma +
amihx + ca + cardiohx + chfhx + chrpulhx +
dementhx + gibledhx + immunhx + liverhx +
malighx + psychhx + renalhx + transhx +
aps1 + das2d3pc + scoma1 + surv2md1 +
alb1 + bili1 + crea1 + hema1 + paco21 + pafi1 +
ph1 + pot1 + sod1 + wblc1 + ps + linps,
data=rhc, un = TRUE)
love.plot(b5, threshold = .1, size = 1.5, stars = "raw",
var.order = "unadjusted",
title = "Standardized Differences and Match 5") +
theme_bw()
14.2 Rubin’s Rules 1 and 2 for Match 5
Create a new data frame, containing only the matched sample.
matches_5 <- factor(rep(match5$index.treated, 2))
rhc.matches_5 <- cbind(matches_5,
rhc[c(match5$index.control, match5$index.treated),])Rubin Rule 1 result after matching should ideally be near 0, and certainly between -50 and +50.
rubin1.m5 <- with(rhc.matches_5,
abs(100*(mean(linps[swang1=="RHC"]) -
mean(linps[swang1=="No RHC"]))/
sd(linps)))
rubin1.m5[1] 1.697439Rubin Rule 2 result after matching should ideally be near 1, and certainly between 1/2 and 2.
rubin2.m5 <- with(rhc.matches_5,
var(linps[swang1 == "RHC"]) /
var(linps[swang1 == "No RHC"]))
rubin2.m5[1] 1.02752715 Match 6: 1:2 Greedy Matching on the linear PS, WITH Replacement
X <- rhc$linps
Tr <- as.logical(rhc$swang1 == "RHC")
match6 <- Match(Tr = Tr, X = X, M = 2,
estimand = "ATT", replace = TRUE,
ties = FALSE)
summary(match6)
Estimate... 0
SE......... 0
T-stat..... NaN
p.val...... NA
Original number of observations.............. 5735
Original number of treated obs............... 2184
Matched number of observations............... 2184
Matched number of observations (unweighted). 4368 15.1 Love Plot for Match 6
b6 <- bal.tab(match6,
swang1 == "RHC" ~
age + sex + edu + income + ninsclas + race +
cat1 + dnr1 + wtkilo1 + hrt1 + meanbp1 +
resp1 + temp1 +
card + gastr + hema + meta + neuro + ortho +
renal + resp + seps + trauma +
amihx + ca + cardiohx + chfhx + chrpulhx +
dementhx + gibledhx + immunhx + liverhx +
malighx + psychhx + renalhx + transhx +
aps1 + das2d3pc + scoma1 + surv2md1 +
alb1 + bili1 + crea1 + hema1 + paco21 + pafi1 +
ph1 + pot1 + sod1 + wblc1 + ps + linps,
data=rhc, un = TRUE)
love.plot(b6, threshold = .1, size = 1.5, stars = "raw",
var.order = "unadjusted",
title = "Standardized Differences and Match 6") +
theme_bw()
15.2 Rubin’s Rules 1 and 2 for Match 6
Create a new data frame, containing only the matched sample.
matches_6 <- factor(rep(match6$index.treated, 2))
rhc.matches_6 <- cbind(matches_6,
rhc[c(match6$index.control, match6$index.treated),])Rubin Rule 1 result after matching should ideally be near 0, and certainly between -50 and +50.
rubin1.m6 <- with(rhc.matches_6,
abs(100*(mean(linps[swang1=="RHC"]) -
mean(linps[swang1=="No RHC"]))/
sd(linps)))
rubin1.m6[1] 0.1186399Rubin Rule 2 result after matching should ideally be near 1, and certainly between 1/2 and 2.
rubin2.m6 <- with(rhc.matches_6,
var(linps[swang1 == "RHC"]) /
var(linps[swang1 == "No RHC"]))
rubin2.m6[1] 1.00543216 Ranking the Matches
| Match | Rubin 1 | Rubin 2 | Love Plot | Matched Sets | Description |
|---|---|---|---|---|---|
| None | 101.6 | 0.74 | – | – | No Matching |
| 1 | 61.9 | 1.69 | Pretty Bad | 2,184 | 1:1 greedy w/o repl |
| 2 | 101.2 | 0.76 | Dreadful | 1,775 | 1:2 greedy w/o repl |
| 3 | 0 | 1 | Very Good | 2,184 | Genetic Search 1:1 w/o repl |
| 4 | 0.1 | 1 | Very Good | 2,184 | 1:1 greedy with repl |
| 5 | 1.7 | 1.03 | Very Good | 1,562 | 1:1 caliper without repl |
| 6 | 0.1 | 1.01 | Very Good | 2,184 | 1:2 greedy with repl |
17 ATT Matching Estimates for the Binary Outcome, Death
17.1 With Match 1, in detail
Note that the variable which identifies the matches (called matches_1) is set up as a factor, as it needs to be, and that I’m using the 1/0 versions of both the outcome (so died rather than death) and treatment (so treat_rhc rather than swang1).
death_adj_m1 <- clogit(died ~ treat_rhc + strata(matches_1), data = rhc.matches_1)
summary(death_adj_m1)Call:
coxph(formula = Surv(rep(1, 4368L), died) ~ treat_rhc + strata(matches_1),
data = rhc.matches_1, method = "exact")
n= 4368, number of events= 2861
coef exp(coef) se(coef) z Pr(>|z|)
treat_rhc 0.23156 1.26056 0.06488 3.569 0.000358 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
exp(coef) exp(-coef) lower .95 upper .95
treat_rhc 1.261 0.7933 1.11 1.432
Concordance= 0.558 (se = 0.023 )
Likelihood ratio test= 12.82 on 1 df, p=3e-04
Wald test = 12.74 on 1 df, p=4e-04
Score (logrank) test = 12.79 on 1 df, p=3e-04The odds ratio in the exp(coef) section above is the average causal effect estimate. It describes the odds of having an event (died) occur associated with being a RHC subject (as identified by treat_rhc), as compared to the odds of that event when a non-RHC subject. We can tidy this with:
tidy(death_adj_m1, exponentiate = TRUE, conf.level = 0.95)# A tibble: 1 x 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.26 0.0649 3.57 0.00035817.2 With Matches 2-6, in much less detail
death_adj_m2 <- clogit(died ~ treat_rhc + strata(matches_2), data = rhc.matches_2)
tidy(death_adj_m2, exponentiate = TRUE, conf.level = 0.95)# A tibble: 1 x 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.33 0.0548 5.27 0.000000137death_adj_m3 <- clogit(died ~ treat_rhc + strata(matches_3), data = rhc.matches_3)
tidy(death_adj_m3, exponentiate = TRUE, conf.level = 0.95)# A tibble: 1 x 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.32 0.0634 4.35 0.0000137death_adj_m4 <- clogit(died ~ treat_rhc + strata(matches_4), data = rhc.matches_4)
tidy(death_adj_m4, exponentiate = TRUE, conf.level = 0.95)# A tibble: 1 x 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.23 0.0636 3.22 0.00127death_adj_m5 <- clogit(died ~ treat_rhc + strata(matches_5), data = rhc.matches_5)
tidy(death_adj_m5, exponentiate = TRUE, conf.level = 0.95)# A tibble: 1 x 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.33 0.0754 3.76 0.000170death_adj_m6 <- clogit(died ~ treat_rhc + strata(matches_6), data = rhc.matches_6)
tidy(death_adj_m6, exponentiate = TRUE, conf.level = 0.95)# A tibble: 1 x 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.36 0.0491 6.25 4.18e-1017.3 Graphing the Results Across Multiple Matching Strategies
temp0 <- tidy(death_unadj, conf.int = TRUE, exponentiate = TRUE) %>%
filter(term == "treat_rhc")
temp1 <- tidy(death_adj_m1, exponentiate = TRUE, conf.int = T, conf.level = 0.95)
temp2 <- tidy(death_adj_m2, exponentiate = TRUE, conf.int = T, conf.level = 0.95)
temp3 <- tidy(death_adj_m3, exponentiate = TRUE, conf.int = T, conf.level = 0.95)
temp4 <- tidy(death_adj_m4, exponentiate = TRUE, conf.int = T, conf.level = 0.95)
temp5 <- tidy(death_adj_m5, exponentiate = TRUE, conf.int = T, conf.level = 0.95)
temp6 <- tidy(death_adj_m6, exponentiate = TRUE, conf.int = T, conf.level = 0.95)
death_match_results <- rbind(temp0, temp1, temp2, temp3, temp4, temp5, temp6)
death_match_results <- death_match_results %>%
mutate(approach = c("Unmatched", "Match 1", "Match 2", "Match 3", "Match 4", "Match 5", "Match 6")) %>%
mutate(description = c("No Matching", "1:1 greedy matching without replacement",
"1:2 greedy matching without replacement",
"1:1 genetic search matching without replacement",
"1:1 greedy matching with replacement",
"1:1 caliper matching on the linear PS without replacement",
"1:2 greedy matching with replacement"))
death_match_results# A tibble: 7 x 9
term estimate std.error statistic p.value conf.low conf.high approach
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
1 treat_rhc 1.25 0.0576 3.90 9.43e- 5 1.12 1.40 Unmatched
2 treat_rhc 1.26 0.0649 3.57 3.58e- 4 1.11 1.43 Match 1
3 treat_rhc 1.33 0.0548 5.27 1.37e- 7 1.20 1.49 Match 2
4 treat_rhc 1.32 0.0634 4.35 1.37e- 5 1.16 1.49 Match 3
5 treat_rhc 1.23 0.0636 3.22 1.27e- 3 1.08 1.39 Match 4
6 treat_rhc 1.33 0.0754 3.76 1.70e- 4 1.15 1.54 Match 5
7 treat_rhc 1.36 0.0491 6.25 4.18e-10 1.23 1.50 Match 6
# ... with 1 more variable: description <chr>ggplot(death_match_results, aes(x = approach, y = estimate, ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 1, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "Comparing ATT Estimates for Death using RHC Propensity Matching",
x = "Propensity Adjustment Approach",
y = "Estimated Odds Ratio for Death associated with RHC")
Let’s look at just the four strategies that achieved stronger balance.
death_match_results %>%
filter(approach %in% c("Match 3", "Match 4", "Match 5", "Match 6")) %>%
ggplot(., aes(x = description, y = estimate, ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 1, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "ATT Matching Estimates in RHC (Death)",
x = "Propensity Adjustment Approach",
y = "Estimated Odds Ratio for Death associated with RHC")
18 ATT Matching Estimates for the Quantitative Outcome, Hospital Length of Stay
18.1 With Match 1, in detail
Again, the variable which identifies the matches (called matches_1) is set up as a factor, as it needs to be, and that I’m using the 1/0 version of treatment (so treat_rhc rather than swang1).
Our result for this quantitative outcome (hospdays) comes from a mixed model where the matches are treated as a random factor, but the treatment group is treated as a fixed factor, using the lme4 package.
hospdays_adj_m1 <- lmer(hospdays ~ treat_rhc + (1 | matches_1),
data = rhc.matches_1)
summary(hospdays_adj_m1); confint(hospdays_adj_m1)Linear mixed model fit by REML ['lmerMod']
Formula: hospdays ~ treat_rhc + (1 | matches_1)
Data: rhc.matches_1
REML criterion at convergence: 40934.5
Scaled residuals:
Min 1Q Median 3Q Max
-0.9292 -0.5540 -0.3137 0.1747 11.9768
Random effects:
Groups Name Variance Std.Dev.
matches_1 (Intercept) 14.5 3.809
Residual 674.0 25.962
Number of obs: 4368, groups: matches_1, 2184
Fixed effects:
Estimate Std. Error t value
(Intercept) 20.8255 0.5615 37.090
treat_rhc 3.8658 0.7857 4.921
Correlation of Fixed Effects:
(Intr)
treat_rhc -0.700Computing profile confidence intervals ... 2.5 % 97.5 %
.sig01 0.000000 6.594183
.sigma 25.205111 26.717184
(Intercept) 19.725069 21.926030
treat_rhc 2.325669 5.406016The estimated effect of treat_rhc in the fixed effects section, combined with the 95% confidence intervals material, provides the average causal effect estimate. It describes the change in the outcome hospdays associated with being a RHC subject (as identified by treat_rhc), as compared to being a non-RHC subject. We can tidy this with:
broom.mixed::tidy(hospdays_adj_m1, conf.int = TRUE, conf.level = 0.95)# A tibble: 4 x 8
effect group term estimate std.error statistic conf.low conf.high
<chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 fixed <NA> (Intercept) 20.8 0.561 37.1 19.7 21.9
2 fixed <NA> treat_rhc 3.87 0.786 4.92 2.33 5.41
3 ran_pars matches_1 sd__(Inter~ 3.81 NA NA NA NA
4 ran_pars Residual sd__Observ~ 26.0 NA NA NA NA or more specifically, we can look at just the treatment effect with:
broom.mixed::tidy(hospdays_adj_m1, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc")# A tibble: 1 x 8
effect group term estimate std.error statistic conf.low conf.high
<chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 fixed <NA> treat_rhc 3.87 0.786 4.92 2.33 5.4118.2 With Matches 2-6, in much less detail
hospdays_adj_m2 <- lmer(hospdays ~ treat_rhc + (1 | matches_2),
data = rhc.matches_2)
broom.mixed::tidy(hospdays_adj_m2, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc")# A tibble: 1 x 8
effect group term estimate std.error statistic conf.low conf.high
<chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 fixed <NA> treat_rhc 4.98 0.545 9.14 3.91 6.05hospdays_adj_m3 <- lmer(hospdays ~ treat_rhc + (1 | matches_3),
data = rhc.matches_3)
broom.mixed::tidy(hospdays_adj_m3, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc")# A tibble: 1 x 8
effect group term estimate std.error statistic conf.low conf.high
<chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 fixed <NA> treat_rhc 1.41 0.842 1.68 -0.240 3.06hospdays_adj_m4 <- lmer(hospdays ~ treat_rhc + (1 | matches_4),
data = rhc.matches_4)boundary (singular) fit: see help('isSingular')broom.mixed::tidy(hospdays_adj_m4, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc")# A tibble: 1 x 8
effect group term estimate std.error statistic conf.low conf.high
<chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 fixed <NA> treat_rhc 1.27 0.859 1.48 -0.411 2.96hospdays_adj_m5 <- lmer(hospdays ~ treat_rhc + (1 | matches_5),
data = rhc.matches_5)boundary (singular) fit: see help('isSingular')broom.mixed::tidy(hospdays_adj_m5, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc")# A tibble: 1 x 8
effect group term estimate std.error statistic conf.low conf.high
<chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 fixed <NA> treat_rhc 2.30 0.919 2.50 0.496 4.10hospdays_adj_m6 <- lmer(hospdays ~ treat_rhc + (1 | matches_6),
data = rhc.matches_6)
broom.mixed::tidy(hospdays_adj_m6, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc")# A tibble: 1 x 8
effect group term estimate std.error statistic conf.low conf.high
<chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 fixed <NA> treat_rhc 1.22 0.556 2.18 0.125 2.3118.3 Graphing the Results Across Multiple Matching Strategies
temp0 <- tidy(hospdays_unadj, conf.int = TRUE) %>%
filter(term == "treat_rhc") %>%
select(term, estimate, std.error, conf.low, conf.high)
temp1 <- tidy(hospdays_adj_m1, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(term, estimate, std.error, conf.low, conf.high)
temp2 <- tidy(hospdays_adj_m2, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(term, estimate, std.error, conf.low, conf.high)
temp3 <- tidy(hospdays_adj_m3, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(term, estimate, std.error, conf.low, conf.high)
temp4 <- tidy(hospdays_adj_m4, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(term, estimate, std.error, conf.low, conf.high)
temp5 <- tidy(hospdays_adj_m5, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(term, estimate, std.error, conf.low, conf.high)
temp6 <- tidy(hospdays_adj_m6, conf.int = TRUE, conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(term, estimate, std.error, conf.low, conf.high)
hospdays_match_results <- rbind(temp0, temp1, temp2, temp3, temp4, temp5, temp6)
hospdays_match_results <- hospdays_match_results %>%
mutate(approach = c("Unmatched", "Match 1", "Match 2", "Match 3", "Match 4", "Match 5", "Match 6")) %>%
mutate(description = c("No Matching", "1:1 greedy matching without replacement",
"1:2 greedy matching without replacement",
"1:1 genetic search matching without replacement",
"1:1 greedy matching with replacement",
"1:1 caliper matching on the linear PS without replacement",
"1:2 greedy matching with replacement"))
hospdays_match_results# A tibble: 7 x 7
term estimate std.error conf.low conf.high approach description
<chr> <dbl> <dbl> <dbl> <dbl> <chr> <chr>
1 treat_rhc 5.16 0.687 3.81 6.51 Unmatched No Matching
2 treat_rhc 3.87 0.786 2.33 5.41 Match 1 1:1 greedy matching~
3 treat_rhc 4.98 0.545 3.91 6.05 Match 2 1:2 greedy matching~
4 treat_rhc 1.41 0.842 -0.240 3.06 Match 3 1:1 genetic search ~
5 treat_rhc 1.27 0.859 -0.411 2.96 Match 4 1:1 greedy matching~
6 treat_rhc 2.30 0.919 0.496 4.10 Match 5 1:1 caliper matchin~
7 treat_rhc 1.22 0.556 0.125 2.31 Match 6 1:2 greedy matching~ggplot(hospdays_match_results, aes(x = approach, y = estimate, ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 0, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "Comparing ATT Estimates for LOS using RHC Propensity Matching",
x = "Propensity Adjustment Approach",
y = "Estimated Change in Hospital Length of Stay associated with RHC")
Let’s look at just the four strategies that achieved stronger balance.
hospdays_match_results %>%
filter(approach %in% c("Match 3", "Match 4", "Match 5", "Match 6")) %>%
ggplot(., aes(x = description, y = estimate, ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 0, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "ATT Matching Estimates in RHC (LOS)",
x = "Propensity Adjustment Approach",
y = "Est. Change in Hospital LOS associated with RHC")
19 ATT Matching Estimates for the Time-to-Event Outcome, In-Study Survival
We’ll use a stratified Cox proportional hazards model to compare the RHC/No RHC groups on our time-to-event outcome survdays, while accounting for the matched pairs. The main result will be a relative hazard rate estimate, with 95% CI. I will use the 0/1 numeric versions of the event indicator (died), and of the treatment indicator (treat_rhc).
19.1 With Match 1, in detail
survdays_adj_m1 <- coxph(Surv(survdays, died) ~ treat_rhc + strata(matches_1), data=rhc.matches_1)
summary(survdays_adj_m1)Call:
coxph(formula = Surv(survdays, died) ~ treat_rhc + strata(matches_1),
data = rhc.matches_1)
n= 4368, number of events= 2861
coef exp(coef) se(coef) z Pr(>|z|)
treat_rhc 0.04768 1.04883 0.04554 1.047 0.295
exp(coef) exp(-coef) lower .95 upper .95
treat_rhc 1.049 0.9534 0.9593 1.147
Concordance= 0.512 (se = 0.016 )
Likelihood ratio test= 1.1 on 1 df, p=0.3
Wald test = 1.1 on 1 df, p=0.3
Score (logrank) test = 1.1 on 1 df, p=0.3The hazard ratio in the exp(coef) section is the average causal effect estimate. It describes the hazard for having an event (died) occur associated with being a RHC subject (as identified by treat_rhc), as compared to the hazard of that event when a non-RHC subject. We can tidy this with:
tidy(survdays_adj_m1, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)# A tibble: 1 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.05 0.0455 1.05 0.295 0.959 1.1519.2 With Matches 2-6, in much less detail
survdays_adj_m2 <- coxph(Surv(survdays, died) ~ treat_rhc + strata(matches_2), data=rhc.matches_2)
tidy(survdays_adj_m2, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)# A tibble: 1 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.27 0.0343 7.02 2.17e-12 1.19 1.36survdays_adj_m3 <- coxph(Surv(survdays, died) ~ treat_rhc + strata(matches_3), data=rhc.matches_3)
tidy(survdays_adj_m3, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)# A tibble: 1 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.09 0.0445 2.03 0.0426 1.00 1.19survdays_adj_m4 <- coxph(Surv(survdays, died) ~ treat_rhc + strata(matches_4), data=rhc.matches_4)
tidy(survdays_adj_m4, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)# A tibble: 1 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.06 0.0452 1.20 0.231 0.966 1.15survdays_adj_m5 <- coxph(Surv(survdays, died) ~ treat_rhc + strata(matches_5), data=rhc.matches_5)
tidy(survdays_adj_m5, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)# A tibble: 1 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.12 0.0535 2.05 0.0399 1.01 1.24survdays_adj_m6 <- coxph(Surv(survdays, died) ~ treat_rhc + strata(matches_6), data=rhc.matches_6)
tidy(survdays_adj_m6, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)# A tibble: 1 x 7
term estimate std.error statistic p.value conf.low conf.high
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.36 0.0313 9.89 4.68e-23 1.28 1.4519.3 Graphing the Results Across Multiple Matching Strategies
temp0 <- tidy(survdays_unadj, conf.int = TRUE, exponentiate = TRUE)
temp1 <- tidy(survdays_adj_m1, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)
temp2 <- tidy(survdays_adj_m2, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)
temp3 <- tidy(survdays_adj_m3, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)
temp4 <- tidy(survdays_adj_m4, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)
temp5 <- tidy(survdays_adj_m5, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)
temp6 <- tidy(survdays_adj_m6, exponentiate = TRUE, conf.int = TRUE, conf.level = 0.95)
survdays_match_results <- rbind(temp0, temp1, temp2, temp3, temp4, temp5, temp6)
survdays_match_results <- survdays_match_results %>%
mutate(approach = c("Unmatched", "Match 1", "Match 2", "Match 3", "Match 4", "Match 5", "Match 6")) %>%
mutate(description = c("No Matching", "1:1 greedy matching without replacement",
"1:2 greedy matching without replacement",
"1:1 genetic search matching without replacement",
"1:1 greedy matching with replacement",
"1:1 caliper matching on the linear PS without replacement",
"1:2 greedy matching with replacement"))
survdays_match_results# A tibble: 7 x 9
term estimate std.error statistic p.value conf.low conf.high approach
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
1 treat_rhc 1.08 0.0335 2.34 1.92e- 2 1.01 1.15 Unmatched
2 treat_rhc 1.05 0.0455 1.05 2.95e- 1 0.959 1.15 Match 1
3 treat_rhc 1.27 0.0343 7.02 2.17e-12 1.19 1.36 Match 2
4 treat_rhc 1.09 0.0445 2.03 4.26e- 2 1.00 1.19 Match 3
5 treat_rhc 1.06 0.0452 1.20 2.31e- 1 0.966 1.15 Match 4
6 treat_rhc 1.12 0.0535 2.05 3.99e- 2 1.01 1.24 Match 5
7 treat_rhc 1.36 0.0313 9.89 4.68e-23 1.28 1.45 Match 6
# ... with 1 more variable: description <chr>ggplot(survdays_match_results, aes(x = approach, y = estimate, ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 1, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "Comparing ATT Estimates for Survival using RHC Propensity Matching",
x = "Propensity Adjustment Approach",
y = "Estimated Hazard Ratio associated with RHC")
Let’s look at just the four strategies that achieved stronger balance.
survdays_match_results %>%
filter(approach %in% c("Match 3", "Match 4", "Match 5", "Match 6")) %>%
ggplot(., aes(x = description, y = estimate, ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 1, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "ATT Matching Estimates in RHC (Survival)",
x = "Propensity Adjustment Approach",
y = "Estimated Hazard Ratio associated with RHC")
20 Propensity Score Weighting (ATT approach)
20.1 Estimate the Propensity Score using Generalized Boosted Regression, and then perfom ATT Weighting
To begin, we’ll re-estimate the propensity score using the twang function ps. This uses a generalized boosted regression approach to estimate the propensity score and produce material for checking balance.
# Recall that twang does not play well with tibbles,
# so we have to use the data frame version of rhc object
# and I'm using the 1/0 version of the treatment
rhc_df <- base::as.data.frame(rhc)
ps_rhc <- ps(treat_rhc ~
age + sex + edu + income + ninsclas +
race + cat1 + dnr1 + wtkilo1 + hrt1 +
meanbp1 + resp1 + temp1 + card + gastr +
hema + meta + neuro + ortho + renal +
resp + seps + trauma + amihx + ca +
cardiohx + chfhx + chrpulhx + dementhx +
gibledhx + immunhx + liverhx + malighx +
psychhx + renalhx + transhx + aps1 +
das2d3pc + scoma1 + surv2md1 + alb1 +
bili1 + crea1 + hema1 + paco21 + pafi1 +
ph1 + pot1 + sod1 + wblc1,
data = rhc_df,
n.trees = 4000,
interaction.depth = 2,
stop.method = c("es.mean"),
estimand = "ATT",
verbose = FALSE)20.1.1 Did we let the simulations run long enough to stabilize estimates?
plot(ps_rhc)
20.1.2 What is the effective sample size of our weighted results?
summary(ps_rhc) n.treat n.ctrl ess.treat ess.ctrl max.es mean.es max.ks
unw 2184 3551 2184 3551.000 0.53367192 0.14968341 0.21274111
es.mean.ATT 2184 3551 2184 1250.501 0.09123789 0.03520429 0.04966016
max.ks.p mean.ks iter
unw NA 0.05831639 NA
es.mean.ATT NA 0.01540652 399920.1.3 How is the balance?
plot(ps_rhc, plots = 2)
plot(ps_rhc, plots = 3)
20.1.4 Assessing Balance with cobalt
b2 <- bal.tab(ps_rhc, full.stop.method = "es.mean.att",
stats = c("m", "v"), un = TRUE)
b2Call
ps.fast(formula = formula, data = data, params = params, n.trees = nrounds,
interaction.depth = max_depth, shrinkage = eta, bag.fraction = subsample,
n.minobsinnode = min_child_weight, perm.test.iters = perm.test.iters,
print.level = print.level, verbose = verbose, estimand = estimand,
stop.method = stop.method, sampw = sampw, multinom = multinom,
ks.exact = ks.exact, version = version, tree_method = tree_method,
n.keep = n.keep, n.grid = n.grid, keep.data = keep.data)
Balance Measures
Type Diff.Un V.Ratio.Un Diff.Adj V.Ratio.Adj
prop.score Distance 1.5554 1.2083 0.5108 0.9058
age Contin. -0.0647 0.8175 0.0027 0.9662
sex_Female Binary -0.0462 . 0.0015 .
edu Contin. 0.0910 1.0147 0.0149 1.0020
income_Under $11k Binary -0.0618 . -0.0144 .
income_$11-$25k Binary 0.0062 . -0.0128 .
income_$25-$50k Binary 0.0391 . 0.0245 .
income_> $50k Binary 0.0165 . 0.0027 .
ninsclas_Private Binary 0.0624 . 0.0195 .
ninsclas_Private & Medicare Binary 0.0143 . -0.0057 .
ninsclas_Medicare Binary -0.0327 . -0.0011 .
ninsclas_Medicare & Medicaid Binary -0.0144 . 0.0023 .
ninsclas_Medicaid Binary -0.0395 . -0.0231 .
ninsclas_No insurance Binary 0.0099 . 0.0082 .
race_white Binary 0.0063 . -0.0088 .
race_black Binary -0.0114 . 0.0062 .
race_other Binary 0.0050 . 0.0026 .
cat1_ARF Binary -0.0290 . -0.0098 .
cat1_CHF Binary 0.0261 . -0.0019 .
cat1_Cirrhosis Binary -0.0268 . -0.0063 .
cat1_Colon Cancer Binary -0.0012 . 0.0000 .
cat1_Coma Binary -0.0525 . -0.0053 .
cat1_COPD Binary -0.0858 . -0.0057 .
cat1_Lung Cancer Binary -0.0073 . -0.0002 .
cat1_MOSF w/Malignancy Binary 0.0045 . 0.0002 .
cat1_MOSF w/Sepsis Binary 0.1721 . 0.0289 .
dnr1_Yes Binary -0.0696 . -0.0129 .
wtkilo1 Contin. 0.2640 0.8835 0.0075 1.0408
hrt1 Contin. 0.1460 1.0260 0.0506 0.9705
meanbp1 Contin. -0.4869 0.7759 -0.0476 0.9655
resp1 Contin. -0.1641 1.0330 -0.0912 1.0078
temp1 Contin. -0.0209 1.1024 -0.0378 1.0288
card_Yes Binary 0.1395 . 0.0136 .
gastr_Yes Binary 0.0453 . 0.0250 .
hema_Yes Binary -0.0146 . -0.0099 .
meta_Yes Binary -0.0059 . 0.0030 .
neuro_Yes Binary -0.1079 . -0.0134 .
ortho_Yes Binary 0.0010 . 0.0016 .
renal_Yes Binary 0.0264 . 0.0083 .
resp_Yes Binary -0.1277 . -0.0374 .
seps_Yes Binary 0.0912 . 0.0178 .
trauma_Yes Binary 0.0105 . 0.0092 .
amihx Binary 0.0139 . 0.0168 .
ca_No Binary 0.0439 . 0.0216 .
ca_Yes Binary -0.0267 . -0.0093 .
ca_Metastatic Binary -0.0172 . -0.0122 .
cardiohx Binary 0.0445 . -0.0094 .
chfhx Binary 0.0268 . -0.0100 .
chrpulhx Binary -0.0737 . -0.0021 .
dementhx Binary -0.0472 . -0.0055 .
gibledhx Binary -0.0122 . -0.0065 .
immunhx Binary 0.0358 . -0.0126 .
liverhx Binary -0.0124 . -0.0075 .
malighx Binary -0.0423 . -0.0211 .
psychhx Binary -0.0348 . -0.0052 .
renalhx Binary 0.0066 . -0.0102 .
transhx Binary 0.0554 . 0.0178 .
aps1 Contin. 0.4837 1.1609 0.0664 1.0236
das2d3pc Contin. 0.0654 0.8429 0.0195 0.9675
scoma1 Contin. -0.1160 0.8116 -0.0114 0.9701
surv2md1 Contin. -0.1954 1.0663 -0.0226 1.0063
alb1 Contin. -0.2010 1.8947 -0.0217 1.6852
bili1 Contin. 0.1329 1.4504 -0.0102 0.9447
crea1 Contin. 0.2678 1.0276 0.0335 0.9294
hema1 Contin. -0.2954 0.7109 -0.0098 0.9668
paco21 Contin. -0.2880 0.5935 -0.0559 1.0124
pafi1 Contin. -0.4566 0.8184 -0.0635 0.9492
ph1 Contin. -0.1163 1.1322 -0.0754 1.1463
pot1 Contin. -0.0274 0.9501 -0.0149 1.0131
sod1 Contin. -0.0927 0.9799 -0.0272 1.0189
wblc1 Contin. 0.0799 1.2088 -0.0110 1.0154
Effective sample sizes
Control Treated
Unadjusted 3551. 2184
Adjusted 1250.5 218420.2 Rubin’s Rule 1 After Weighting
We could simply compare the propensity score balance as reported in the table above, looking at the standardized biases (the standardized difference on a proportion scale, rather than a percentage.) I’ll multiply each by 100 to place things on the usual scale.
100*b2[1]$Balance$Diff.Un[1] # imbalance prior to weighting[1] 155.5377100*b2[1]$Balance$Diff.Adj[1] # imbalance after weighting[1] 51.07571So this is still indicative of some difficulty reaching a completely reasonable level of balance. We’d like to see this much closer to 0.
20.3 Rubin’s Rule 2 After Weighting
b2[1]$Balance$V.Ratio.Un[1] # imbalance prior to weighting[1] 1.208331b2[1]$Balance$V.Ratio.Adj[1] # imbalance after weighting[1] 0.9057622So there’s no particular problem with Rule 2 here.
20.4 Semi-Automated Love plot of Standardized Differences
p <- love.plot(b2,
threshold = .1, size = 3, stars = "raw",
title = "Standardized Diffs and ATT weights")
p + theme_bw()
20.5 Semi-Automated Love plot of Variance Ratios
p <- love.plot(b2, stat = "v",
threshold = 1.25, size = 3,
title = "Variance Ratios: TWANG ATT weighting")
p + theme_bw()
21 ATT Weighting Estimates
We’re not too excited about weighted estimates without (at least) an additional adjustment for differences in the propensity score in this case, but I’ll run them anyway just to show how that would be done.
21.1 Weighted ATT for Outcome A: Death
The relevant regression approach uses the svydesign and svyglm functions from the survey package. For a binary outcome, we build the outcome model using the quasibinomial, rather than the usual binomial family. Note the use of the 1/0 version of both the outcome and the treatment variables here.
rhcwts_design <- svydesign(
ids=~1, weights=~get.weights(ps_rhc,
stop.method = "es.mean"),
data=rhc_df) # using twang ATT weights
adj_death_wtd <- svyglm(died ~ treat_rhc,
design=rhcwts_design,
family=quasibinomial())
wt_twangatt_res_death <-
tidy(adj_death_wtd, exponentiate = TRUE,
conf.int = TRUE,
conf.level = 0.95) %>%
select(term, estimate, std.error,
lo95 = conf.low, hi95 = conf.high)
wt_twangatt_res_death# A tibble: 2 x 5
term estimate std.error lo95 hi95
<chr> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 1.73 0.0602 1.53 1.94
2 treat_rhc 1.23 0.0757 1.06 1.43The estimates we want come from the treat_rhc row.
21.2 Double Robust Weighted Estimate for Death
Now let’s add in a regression adjustment for the linear propensity score, which should alleviate some of our concerns about residual imbalance after weighting.
dr_adj_death <- svyglm(died ~ treat_rhc + linps,
design=rhcwts_design,
family=quasibinomial())
dr_death <-
tidy(dr_adj_death, exponentiate = TRUE,
conf.int = TRUE,
conf.level = 0.95) %>%
select(term, estimate, std.error,
lo95 = conf.low, hi95 = conf.high)
dr_death# A tibble: 3 x 5
term estimate std.error lo95 hi95
<chr> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 1.72 0.0612 1.52 1.94
2 treat_rhc 1.26 0.0812 1.07 1.47
3 linps 0.938 0.0415 0.864 1.02Again, the estimates we want come from the treat_rhc row.
21.3 Graphing the Weighted and Matched Results for Outcome A
temp7 <-
tidy(adj_death_wtd, exponentiate = TRUE,
conf.int = T, conf.level = 0.95) %>%
filter(term == "treat_rhc")
temp8 <-
tidy(dr_adj_death, exponentiate = TRUE,
conf.int = T, conf.level = 0.95) %>%
filter(term == "treat_rhc")
death_new_results <- rbind(temp7, temp8) %>%
mutate(approach = c("ATT weighted", "DR")) %>%
mutate(description = c("ATT weights", "DR: ATT wts + adj"))
death_results <- rbind(death_match_results,
death_new_results)
death_results# A tibble: 9 x 9
term estimate std.error statistic p.value conf.low conf.high approach
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
1 treat_rhc 1.25 0.0576 3.90 9.43e- 5 1.12 1.40 Unmatched
2 treat_rhc 1.26 0.0649 3.57 3.58e- 4 1.11 1.43 Match 1
3 treat_rhc 1.33 0.0548 5.27 1.37e- 7 1.20 1.49 Match 2
4 treat_rhc 1.32 0.0634 4.35 1.37e- 5 1.16 1.49 Match 3
5 treat_rhc 1.23 0.0636 3.22 1.27e- 3 1.08 1.39 Match 4
6 treat_rhc 1.33 0.0754 3.76 1.70e- 4 1.15 1.54 Match 5
7 treat_rhc 1.36 0.0491 6.25 4.18e-10 1.23 1.50 Match 6
8 treat_rhc 1.23 0.0757 2.78 5.50e- 3 1.06 1.43 ATT weight~
9 treat_rhc 1.26 0.0812 2.81 4.94e- 3 1.07 1.47 DR
# ... with 1 more variable: description <chr>So there’s our table. It’s worth remembering that the only analyses that we’ve
death_results %>%
ggplot(., aes(x = description, y = estimate,
ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 1, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "ATT Estimates in RHC (Death)",
x = "Propensity Adjustment Approach",
y = "Estimated Odds Ratio for Death associated with RHC")
And here are the results for only the five methods (adding our double robust approach to the analyses we liked in the matching) which showed reasonable balance after propensity score adjustment…
death_results %>%
filter(approach %in% c("Match 3", "Match 4", "Match 5",
"Match 6", "DR")) %>%
ggplot(., aes(x = description, y = estimate,
ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 1, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "ATT Estimates in RHC (Death)",
x = "Propensity Adjustment Approach",
y = "Estimated Odds Ratio for Death associated with RHC")
21.4 Weighted ATT for Outcome B: Hospital Length of Stay
The relevant regression approach uses the svydesign and svyglm functions from the survey package.
rhcwts_design <- svydesign(
ids=~1, weights=~get.weights(ps_rhc,
stop.method = "es.mean"),
data=rhc_df) # using twang ATT weights
adj_hospdays_wtd <-
svyglm(hospdays ~ treat_rhc,
design= rhcwts_design)
wt_twangatt_res_hospdays <-
tidy(adj_hospdays_wtd, conf.int = TRUE,
conf.level = 0.95) %>%
select(term, estimate, std.error,
lo95 = conf.low, hi95 = conf.high)
wt_twangatt_res_hospdays# A tibble: 2 x 5
term estimate std.error lo95 hi95
<chr> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 22.6 0.844 20.9 24.2
2 treat_rhc 2.10 1.03 0.0750 4.12The estimates we want come from the treat_rhc row.
21.5 Double Robust Weighted Estimate for LOS
Now let’s add in a regression adjustment for the linear propensity score. Again, we should have some additional faith in these results over those with just weighting alone, because of the failure of the weighting approach to pass Rubin’s Rule 1.
dr_adj_days <- svyglm(hospdays ~ treat_rhc + linps,
design=rhcwts_design)
dr_los <-
tidy(dr_adj_days, conf.int = TRUE,
conf.level = 0.95) %>%
select(term, estimate, std.error,
lo95 = conf.low, hi95 = conf.high)
dr_los# A tibble: 3 x 5
term estimate std.error lo95 hi95
<chr> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 22.8 0.873 21.1 24.5
2 treat_rhc 1.56 1.10 -0.605 3.72
3 linps 1.93 0.507 0.934 2.92Once again, the estimates we want come from the treat_rhc row.
21.6 Graphing the Weighted and Matched Results for Outcome B
temp9 <-
tidy(adj_hospdays_wtd, conf.int = T,
conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(-statistic, -p.value)
temp10 <-
tidy(dr_adj_days, conf.int = T,
conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(-statistic, -p.value)
hospdays_new_results <- rbind(temp9, temp10) %>%
mutate(approach = c("ATT weighted", "DR")) %>%
mutate(description = c("ATT weights", "DR: ATT wts + adj"))
hospdays_results <- rbind(hospdays_match_results,
hospdays_new_results)
hospdays_results# A tibble: 9 x 7
term estimate std.error conf.low conf.high approach description
<chr> <dbl> <dbl> <dbl> <dbl> <chr> <chr>
1 treat_rhc 5.16 0.687 3.81 6.51 Unmatched No Matching
2 treat_rhc 3.87 0.786 2.33 5.41 Match 1 1:1 greedy match~
3 treat_rhc 4.98 0.545 3.91 6.05 Match 2 1:2 greedy match~
4 treat_rhc 1.41 0.842 -0.240 3.06 Match 3 1:1 genetic sear~
5 treat_rhc 1.27 0.859 -0.411 2.96 Match 4 1:1 greedy match~
6 treat_rhc 2.30 0.919 0.496 4.10 Match 5 1:1 caliper matc~
7 treat_rhc 1.22 0.556 0.125 2.31 Match 6 1:2 greedy match~
8 treat_rhc 2.10 1.03 0.0750 4.12 ATT weighted ATT weights
9 treat_rhc 1.56 1.10 -0.605 3.72 DR DR: ATT wts + adjSo there’s our table.
hospdays_results %>%
ggplot(., aes(x = description, y = estimate,
ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 0, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "Comparing ATT Estimates for LOS",
x = "Propensity Adjustment Approach",
y = "Estimated Change in Hospital LOS associated with RHC")
And here are the results for only the five methods (adding our double robust approach to the analyses we liked in the matching) which showed reasonable balance after propensity score adjustment…
hospdays_results %>%
filter(approach %in% c("Match 3", "Match 4", "Match 5",
"Match 6", "DR")) %>%
ggplot(., aes(x = description, y = estimate,
ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 0, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "Comparing ATT Estimates for LOS",
x = "Propensity Adjustment Approach",
y = "Estimated Change in Hospital LOS associated with RHC")
21.7 Weighted ATT for Outcome C: In-Study Time to Death
rhc_wts <- get.weights(ps_rhc, stop.method = "es.mean")
adj_survdays_wtd <- coxph(Surv(survdays, died) ~ treat_rhc,
data = rhc_df, weights = rhc_wts)
wt_twangatt_res_surv <-
tidy(adj_survdays_wtd, exponentiate = TRUE,
conf.int = TRUE,
conf.level = 0.95) %>%
select(term, estimate, std.error,
lo95 = conf.low, hi95 = conf.high)
wt_twangatt_res_surv# A tibble: 1 x 5
term estimate std.error lo95 hi95
<chr> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.11 0.0396 1.02 1.22Again, the estimates we want come from the treat_rhc row.
21.8 Double Robust Weighted Estimate for Days to Death
Now let’s add in a regression adjustment for the linear propensity score. Again, in this case, we prefer this double robust approach to weighting alone because of Rubin’s Rule 1 and the Love plot.
dr_survdays <-
coxph(Surv(survdays, died) ~ treat_rhc + linps,
data = rhc_df, weights = rhc_wts)
dr_surv <-
tidy(dr_survdays, exponentiate = TRUE,
conf.int = TRUE,
conf.level = 0.95) %>%
select(term, estimate, std.error,
lo95 = conf.low, hi95 = conf.high)
dr_surv# A tibble: 2 x 5
term estimate std.error lo95 hi95
<chr> <dbl> <dbl> <dbl> <dbl>
1 treat_rhc 1.13 0.0399 1.03 1.23
2 linps 0.955 0.0173 0.913 0.999Once again, the estimates we want come from the treat_rhc row.
21.9 Graphing the Weighted and Matched Results for Outcome C
temp11 <-
tidy(adj_survdays_wtd, exponentiate = T, conf.int = T,
conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(-robust.se)
temp12 <-
tidy(dr_survdays, exponentiate = T, conf.int = T,
conf.level = 0.95) %>%
filter(term == "treat_rhc") %>%
select(-robust.se)
survdays_new_results <- rbind(temp11, temp12) %>%
mutate(approach = c("ATT weighted", "DR")) %>%
mutate(description = c("ATT weights", "DR: ATT wts + adj"))
survdays_results <- rbind(survdays_new_results,
survdays_match_results)
survdays_results# A tibble: 9 x 9
term estimate std.error statistic p.value conf.low conf.high approach
<chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
1 treat_rhc 1.11 0.0396 2.39 1.67e- 2 1.02 1.22 ATT weight~
2 treat_rhc 1.13 0.0399 2.55 1.09e- 2 1.03 1.23 DR
3 treat_rhc 1.08 0.0335 2.34 1.92e- 2 1.01 1.15 Unmatched
4 treat_rhc 1.05 0.0455 1.05 2.95e- 1 0.959 1.15 Match 1
5 treat_rhc 1.27 0.0343 7.02 2.17e-12 1.19 1.36 Match 2
6 treat_rhc 1.09 0.0445 2.03 4.26e- 2 1.00 1.19 Match 3
7 treat_rhc 1.06 0.0452 1.20 2.31e- 1 0.966 1.15 Match 4
8 treat_rhc 1.12 0.0535 2.05 3.99e- 2 1.01 1.24 Match 5
9 treat_rhc 1.36 0.0313 9.89 4.68e-23 1.28 1.45 Match 6
# ... with 1 more variable: description <chr>So there’s our table.
survdays_results %>%
ggplot(., aes(x = description, y = estimate,
ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 1, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "Comparing ATT Estimates for Survival Time",
x = "Propensity Adjustment Approach",
y = "Estimated Hazard Ratio associated with RHC")
And here are the results for only the five methods (adding our double robust approach to the analyses we liked in the matching) which showed reasonable balance after propensity score adjustment…
survdays_results %>%
filter(approach %in% c("Match 3", "Match 4", "Match 5",
"Match 6", "DR")) %>%
ggplot(., aes(x = description, y = estimate,
ymin = conf.low, ymax = conf.high)) +
geom_text(aes(label = round_half_up(estimate,2)), vjust = -1.25) +
geom_pointrange() +
geom_hline(yintercept = 1, col = "red", linetype = "dashed") +
theme_bw() +
coord_flip() +
labs(title = "Comparing ATT Estimates for Survival Time",
x = "Propensity Adjustment Approach",
y = "Estimated Hazard Ratio associated with RHC")
22 Sensitivity Analyses After Matching
22.1 For the Binary Outcome, Death
The rbounds package can evaluate binary outcomes using the binarysens and Fishersens functions. The binarysens function works directly on a matched sample as developed using the Matching package. For example, in our third match, based on the genetic search algorithm with 1:1 matching without replacement, this is match3. Note that in order to do this, we needed to run match3 including the appropriate (binary) outcome (Y).
Y <- rhc$died
X <- cbind(rhc$linps, rhc$ps)
Tr <- as.logical(rhc$swang1 == "RHC")
match3 <- Match(Y = Y, Tr = Tr, X = X, estimand = "ATT",
Weight.matrix = genout3)
binarysens(match3, Gamma = 1.5, GammaInc = 0.05)
Rosenbaum Sensitivity Test
Unconfounded estimate .... 0
Gamma Lower bound Upper bound
1.00 1e-05 0.00001
1.05 0e+00 0.00020
1.10 0e+00 0.00260
1.15 0e+00 0.01854
1.20 0e+00 0.07975
1.25 0e+00 0.22444
1.30 0e+00 0.44660
1.35 0e+00 0.67877
1.40 0e+00 0.85082
1.45 0e+00 0.94458
1.50 0e+00 0.98337
Note: Gamma is Odds of Differential Assignment To
Treatment Due to Unobserved Factors
At a 5% significance level, we retain significance up to a hidden bias of \(\Gamma\) = 1.15, but not at \(\Gamma\) = 1.2. See the toy_2019 example discussed in Class 4, as well as Chapter 9 of Rosenbaum’s Observation and Experiment for more details on interpretation of this result.
We can also use this approach with other Matching results, including Match 6, which is a 1:2 match with replacement. Again, we have to run the match with the appropriate outcome included.
Y <- rhc$died
X <- rhc$linps
Tr <- as.logical(rhc$swang1 == "RHC")
match6 <- Match(Y = Y, Tr = Tr, X = X, M = 2,
estimand = "ATT", replace = TRUE,
ties = FALSE)
binarysens(match6, Gamma = 1.5, GammaInc = 0.05)
Rosenbaum Sensitivity Test
Unconfounded estimate .... 0
Gamma Lower bound Upper bound
1.00 0 0.00000
1.05 0 0.00005
1.10 0 0.00222
1.15 0 0.03116
1.20 0 0.17730
1.25 0 0.48926
1.30 0 0.79823
1.35 0 0.95191
1.40 0 0.99306
1.45 0 0.99938
1.50 0 0.99996
Note: Gamma is Odds of Differential Assignment To
Treatment Due to Unobserved Factors
Here, at a 5% significance level, we retain significance up to a hidden bias of \(\Gamma\) = 1.15, but not at \(\Gamma\) = 1.20.
22.2 For the Quantitative Outcome, In-Hospital Length of Stay
We have already used the Match function from the Matching package to develop our matched samples. For our 1:1 matches, we need only run the psens function from the rbounds package to obtain sensitivity results. For example, in Match 3,
Y <- rhc$hospdays
X <- cbind(rhc$linps, rhc$ps)
Tr <- as.logical(rhc$swang1 == "RHC")
match3 <- Match(Y = Y, Tr = Tr, X = X, estimand = "ATT",
Weight.matrix = genout3)
psens(match3, Gamma = 1.5, GammaInc = 0.1)
Rosenbaum Sensitivity Test for Wilcoxon Signed Rank P-Value
Unconfounded estimate .... 1e-04
Gamma Lower bound Upper bound
1.0 1e-04 0.0001
1.1 0e+00 0.0410
1.2 0e+00 0.5095
1.3 0e+00 0.9501
1.4 0e+00 0.9992
1.5 0e+00 1.0000
Note: Gamma is Odds of Differential Assignment To
Treatment Due to Unobserved Factors
At a 5% significance level, we retain significance up to a hidden bias of \(\Gamma\) = 1.1, but not at \(\Gamma\) = 1.2. See the toy_2019 example discussed in Class 4, as well as Chapter 9 of Rosenbaum’s Observation and Experiment for more details on interpretation of this result.
For a matching with more than 1 control, we need to apply the mcontrol function, which requires the use of the data.prep function from rbounds. So, for example, in Match 6, which is a 1:2 greedy match (so that each matched set has 3 patients in it, making the group.size = 3), we would have:
Y <- rhc$hospdays
X <- rhc$linps
Tr <- as.logical(rhc$swang1 == "RHC")
match6 <- Match(Y = Y, Tr = Tr, X = X, M = 2,
estimand = "ATT", replace = TRUE,
ties = FALSE)
tmp <- data.prep(match6, group.size = 3)
mcontrol(tmp$Y, tmp$id, tmp$treat, group.size=3, Gamma = 1.5, GammaInc = 0.1)
Rosenbaum Sensitivity Test for Wilcoxon Strat. Rank P-Value
Unconfounded estimate .... 0
Gamma Lower bound Upper bound
1.0 0 0.0000
1.1 0 0.0177
1.2 0 0.3501
1.3 0 0.8834
1.4 0 0.9960
1.5 0 1.0000
Note: Gamma is Odds of Differential Assignment To
Treatment Due to Unobserved Factors
Again, at a 5% significance level, we retain significance up to a hidden bias of \(\Gamma\) = 1.1, but not at \(\Gamma\) = 1.2.
22.3 For the Survival Outcome, In-Study Time to Death
In this setting, we would need to use the spreadsheet software provided by Dr. Love. That software is designed only for 1:1 greedy matching without replacement or weighting, which is the approach we took in Match 1 (we could also have looked at matching using a caliper, but without replacement.) We need to identify, for each matched pair of subjects, whether that pair’s set of results was:
- that the RHC patient definitely survived longer than the non-RHC patient
- that the non-RHC patient definitely survived longer than the RHC patient
- that it is unclear which patient survived longer, due to censoring
Here is some very fussy code that accomplishes this counting. This could probably be accomplished just as well with some sort of case_when statement, but it’s a bit tricky.
a1 <- rhc.matches_1 %>%
select(matches_1, swang1, death, survdays) %>%
arrange(matches_1) a2 <- unite(a1, situation, swang1, death, sep = "_", remove = TRUE)
a3 <- spread(a2, situation, survdays)
a4 <- a3 %>%
filter(complete.cases(`No RHC_Yes`, `RHC_Yes`)) %>%
mutate(result = ifelse(`RHC_Yes` > `No RHC_Yes`, "RHC lived longer", "No RHC lived longer"))
a5 <- a3 %>%
filter(complete.cases(`No RHC_No`, `RHC_No`)) %>%
mutate(result = "No clear winner")
a6 <- a3 %>%
filter(complete.cases(`No RHC_No`, `RHC_Yes`)) %>%
mutate(result = ifelse(`RHC_Yes` > `No RHC_No`, "No clear winner", "No RHC lived longer"))
a7 <- a3 %>%
filter(complete.cases(`No RHC_Yes`, `RHC_No`)) %>%
mutate(result = ifelse(`RHC_No` > `No RHC_Yes`, "RHC lived longer", "No clear winner"))
a8 <- bind_rows(a4, a5, a6, a7)
a8 %>% tabyl(result) %>% adorn_totals() result n percent
No clear winner 278 0.1272894
No RHC lived longer 988 0.4523810
RHC lived longer 918 0.4203297
Total 2184 1.0000000So we have counts for the pairs where a clear winner is identified, and we can enter the spreadsheet. We need the number of pairs with a clear winner, and the number of pairs where the No RHC patient lived longer than the RHC patient.
23 Session Information
xfun::session_info()R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19043)
Locale:
LC_COLLATE=English_United States.1252
LC_CTYPE=English_United States.1252
LC_MONETARY=English_United States.1252
LC_NUMERIC=C
LC_TIME=English_United States.1252
Package version:
askpass_1.1 assertthat_0.2.1 backports_1.4.1
base64enc_0.1.3 bit_4.0.4 bit64_4.0.5
blob_1.2.2 boot_1.3-28 brio_1.1.3
broom_0.7.12 broom.mixed_0.2.7 bslib_0.3.1
cachem_1.0.6 callr_3.7.0 cellranger_1.1.0
class_7.3-19 cli_3.1.1 clipr_0.7.1
cobalt_4.3.2 coda_0.19.4 colorspace_2.0-2
compiler_4.1.2 conflicted_1.1.0 cpp11_0.4.2
crayon_1.4.2 crosstalk_1.2.0 curl_4.3.2
data.table_1.14.2 DBI_1.1.2 dbplyr_2.1.1
desc_1.4.0 diffobj_0.3.5 digest_0.6.29
dplyr_1.0.8 dtplyr_1.2.1 e1071_1.7-9
ellipsis_0.3.2 evaluate_0.14 fansi_1.0.2
farver_2.1.0 fastmap_1.1.0 forcats_0.5.1
fs_1.5.2 gargle_1.2.0 gbm_2.1.8
gdata_2.18.0 generics_0.1.2 ggdendro_0.1.22
ggforce_0.3.3 ggformula_0.10.1 ggplot2_3.3.5
ggrepel_0.9.1 ggridges_0.5.3 ggstance_0.3.5
glue_1.6.1 gmodels_2.18.1 googledrive_2.0.0
googlesheets4_1.0.0 graphics_4.1.2 grDevices_4.1.2
grid_4.1.2 gridExtra_2.3 gtable_0.3.0
gtools_3.9.2 haven_2.4.3 here_1.0.1
highr_0.9 hms_1.1.1 htmltools_0.5.2
htmlwidgets_1.5.4 httr_1.4.2 ids_1.0.1
isoband_0.2.5 janitor_2.1.0 jpeg_0.1-9
jquerylib_0.1.4 jsonlite_1.7.3 knitr_1.37
labeling_0.4.2 labelled_2.9.0 lattice_0.20-45
latticeExtra_0.6-29 lazyeval_0.2.2 leaflet_2.0.4.1
leaflet.providers_1.9.0 lifecycle_1.0.1 lme4_1.1-28
lubridate_1.8.0 magrittr_2.0.2 markdown_1.1
MASS_7.3-55 Matching_4.9-11 Matrix_1.3-4
MatrixModels_0.5-0 memoise_2.0.1 methods_4.1.2
mgcv_1.8.38 mime_0.12 minqa_1.2.4
mitools_2.4 modelr_0.1.8 mosaic_1.8.3
mosaicCore_0.9.0 mosaicData_0.20.2 munsell_0.5.0
naniar_0.6.1 nlme_3.1-153 nloptr_2.0.0
norm_1.0.9.5 numDeriv_2016.8.1.1 openssl_1.4.6
parallel_4.1.2 pillar_1.7.0 pkgconfig_2.0.3
pkgload_1.2.4 plyr_1.8.6 png_0.1-7
polyclip_1.10-0 praise_1.0.0 prettyunits_1.1.1
processx_3.5.2 progress_1.2.2 proxy_0.4-26
ps_1.6.0 purrr_0.3.4 R6_2.5.1
rappdirs_0.3.3 raster_3.5.15 rbounds_2.1
RColorBrewer_1.1-2 Rcpp_1.0.8 RcppEigen_0.3.3.9.1
readr_2.1.2 readxl_1.3.1 rematch_1.0.1
rematch2_2.1.2 reprex_2.0.1 rgenoud_5.8-3.0
rlang_1.0.1 rmarkdown_2.11 rprojroot_2.0.2
rstudioapi_0.13 rvest_1.0.2 sass_0.4.0
scales_1.1.1 selectr_0.4.2 snakecase_0.11.0
sp_1.4.6 splines_4.1.2 stats_4.1.2
stringi_1.7.6 stringr_1.4.0 survey_4.1-1
survival_3.2-13 sys_3.4 tableone_0.13.0
terra_1.5.17 testthat_3.1.2 tibble_3.1.6
tidyr_1.2.0 tidyselect_1.1.1 tidyverse_1.3.1
tinytex_0.36 tools_4.1.2 twang_2.5
tweenr_1.0.2 tzdb_0.2.0 UpSetR_1.4.0
utf8_1.2.2 utils_4.1.2 uuid_1.0.3
vctrs_0.3.8 viridis_0.6.2 viridisLite_0.4.0
visdat_0.5.3 vroom_1.5.7 waldo_0.3.1
withr_2.4.3 xfun_0.29 xgboost_1.5.0.2
xml2_1.3.3 xtable_1.8-4 yaml_2.2.2
zoo_1.8-9 LS0tDQp0aXRsZTogIlByb3BlbnNpdHkgU2NvcmVzIGFuZCB0aGUgUmlnaHQgSGVhcnQgQ2F0aGV0ZXJpemF0aW9uIERhdGEiDQphdXRob3I6ICJUaG9tYXMgRS4gTG92ZSwgUGguRC4iDQpkYXRlOiAiTGFzdCBVcGRhdGUgYHIgU3lzLnRpbWUoKWAiDQpvdXRwdXQ6IA0KICAgIGh0bWxfZG9jdW1lbnQ6DQogICAgICAgIHRvYzogVFJVRQ0KICAgICAgICB0b2NfZmxvYXQ6IFRSVUUNCiAgICAgICAgbnVtYmVyX3NlY3Rpb25zOiBUUlVFDQogICAgICAgIGNvZGVfZm9sZGluZzogc2hvdw0KICAgICAgICBjb2RlX2Rvd25sb2FkOiBUUlVFDQotLS0NCg0KYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9DQprbml0cjo6b3B0c19jaHVuayRzZXQoY29tbWVudCA9IE5BKQ0KYGBgDQoNCiMgUHJlbGltaW5hcmllcw0KDQojIyBTdHVkeSBCYWNrZ3JvdW5kDQoNClRoaXMgZXhhbXBsZSBpcyBiYXNlZCBvbiB0aGUgUmlnaHQgSGVhcnQgQ2F0aGV0ZXJpemF0aW9uIGRhdGEgc2V0IGF2YWlsYWJsZSBhdCBbVmFuZGVyYmlsdCBVbml2ZXJzaXR5XShodHRwczovL2Jpb3N0YXQuYXBwLnZ1bWMub3JnL3dpa2kvTWFpbi9EYXRhU2V0cykgYW5kIGRlc2NyaWJlZCBbaGVyZV0oaHR0cHM6Ly9iaW9zdGF0LmFwcC52dW1jLm9yZy93aWtpL3B1Yi9NYWluL0RhdGFTZXRzL3JoYy5odG1sKS4gVGhlIGtleSByZWZlcmVuY2UgaXMgQ29ubm9ycyBBRiBldCBhbC4gMTk5NiBUaGUgZWZmZWN0aXZlbmVzcyBvZiBSSEMgaW4gdGhlIGluaXRpYWwgY2FyZSBvZiBjcml0aWNhbGx5IGlsbCBwYXRpZW50cy4gWypKQU1BKiAyNzY6IDg4OS04OTddKGh0dHA6Ly9qYW1hbmV0d29yay5jb20vam91cm5hbHMvamFtYS9mdWxsYXJ0aWNsZS80MDc5OTApLiBDb25ub3JzIGV0IGFsLiB1c2VkIGEgbG9naXN0aWMgcmVncmVzc2lvbiBtb2RlbCB0byBkZXZlbG9wIGEgcHJvcGVuc2l0eSBzY29yZSB0aGVuOiBbYV0gbWF0Y2hlZCBSSEMgdG8gbm9uLVJIQyBwYXRpZW50cyBhbmQgW2JdIGFkanVzdGVkIGZvciBwcm9wZW5zaXR5IHNjb3JlIGluIG1vZGVscyBmb3Igb3V0Y29tZXMsIGZvbGxvd2VkIGJ5IGEgc2Vuc2l0aXZpdHkgYW5hbHlzaXMuIFRoZSBrZXkgY29uY2x1c2lvbnMgd2VyZSB0aGF0IFJIQyBwYXRpZW50cyBoYWQgZGVjcmVhc2VkIHN1cnZpdmFsIHRpbWUsIGFuZCBhbnkgdW5tZWFzdXJlZCBjb25mb3VuZGVyIHdvdWxkIGhhdmUgdG8gYmUgc29tZXdoYXQgc3Ryb25nIHRvIGV4cGxhaW4gYXdheSB0aGUgcmVzdWx0cy4NCg0KIyMgTG9hZGluZyBQYWNrYWdlcw0KDQpgYGB7ciBsb2FkX3BhY2thZ2VzLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nID0gRkFMU0V9DQpsaWJyYXJ5KGhlcmUpOyBsaWJyYXJ5KGphbml0b3IpOyBsaWJyYXJ5KG5hbmlhcikNCmxpYnJhcnkodGFibGVvbmUpOyBsaWJyYXJ5KGJyb29tKTsgbGlicmFyeShzdXJ2aXZhbCkNCmxpYnJhcnkoTWF0Y2hpbmcpOyBsaWJyYXJ5KGNvYmFsdCk7IGxpYnJhcnkocmdlbm91ZCkNCmxpYnJhcnkobG1lNCk7IGxpYnJhcnkocmJvdW5kcyk7IGxpYnJhcnkoY29uZmxpY3RlZCkNCmxpYnJhcnkodHdhbmcpOyBsaWJyYXJ5KHN1cnZleSkNCmxpYnJhcnkodGlkeXZlcnNlKQ0KDQpjb25mbGljdF9wcmVmZXIoInNlbGVjdCIsICJkcGx5ciIpDQpjb25mbGljdF9wcmVmZXIoImZpbHRlciIsICJkcGx5ciIpDQoNCnRoZW1lX3NldCh0aGVtZV9idygpKQ0KYGBgDQoNCiMjIEltcG9ydGluZyB0aGUgRGF0YQ0KDQpUaGUgb2JzZXJ2YXRpb25zIGluIHRoZSBgcmhjYCBkYXRhIHNldCBkZXNjcmliZSA1LDczNSBzdWJqZWN0cyB3aG8gYXJlIGNyaXRpY2FsbHkgaWxsLCBhbmQgbW9zdCB2YXJpYWJsZXMgd2VyZSBvYnRhaW5lZCBkdXJpbmcgZGF5IDEgb2YgYSBob3NwaXRhbGl6YXRpb24uIFdlJ2xsIGJlZ2luIGJ5IGltcG9ydGluZyB0aGUgYHJoY2AgZGF0YSBpbnRvIGEgKip0aWJibGUqKiAoYSBsYXp5IGFuZCBzdXJseSBkYXRhIGZyYW1lIC0gc2VlIFIncyBoZWxwIGZpbGUgb24gdGliYmxlcyBmb3IgbW9yZSBkZXRhaWxzKSBpbiBSIHRoYXQgY29udGFpbnMgNSw3MzUgb2JzZXJ2YXRpb25zIChyb3dzKSBvbiA2MyB2YXJpYWJsZXMgKGNvbHVtbnMpLiANCg0KSW4gdGhlIGltcG9ydCBwcm9jZXNzLCB3ZSdyZSBnb2luZyB0byBkbyBhIGZldyB0aGluZ3MgdG8gc2ltcGxpZnkgdGhlIHByb2Nlc3Mgb2YgZ2V0dGluZyBhIGZ1bmN0aW9uYWwgZGF0YSBzZXQuDQoNCi0gVGhlIGF1dG9tYXRlZCBjb2x1bW4gc3BlY2lmaWNhdGlvbiBpbiBSJ3MgKipyZWFkcioqIHBhY2thZ2UgdXNlcyB0aGUgZmlyc3QgMTAwMCByb3dzIG9mIHRoZSBkYXRhIHNldCAtIHdpdGggdGhlIGByaGNgIGRhdGEsIHRoaXMgY3JlYXRlcyBwYXJzaW5nIGZhaWx1cmVzIGZvciBzZXZlcmFsIG1lYXN1cmVzIHdoaWNoIGFyZSBhc3N1bWVkIHRvIGJlIGludGVnZXJzIGJ1dCBhY3R1YWxseSBzaG91bGQgYmUgcGFyc2VkIGFzIGRvdWJsZXMgKGNvbnRpbnVvdXMgdmFyaWFibGVzKS4gU28gd2Ugc3BlY2lmeSB0aGUgY29sdW1uIHR5cGVzIGZvciB0aG9zZSB2YXJpYWJsZXMgaGVyZSB0aGVuIGFsbG93IFIgdG8gaWRlbnRpZnkgdGhlIHR5cGUgZm9yIHRoZSBvdGhlciBjb2x1bW5zLg0KLSBCeSBkZWZhdWx0LCAqKnJlYWRyKioncyBgcmVhZF9jc3ZgIGNvbW1hbmQgdHJlYXRzIGFsbCBzdHJpbmcgdmFyaWFibGVzIGFzIGNoYXJhY3RlcnMsIGFuZCBub3QgYXMgZmFjdG9ycy4gRm9yIHNldmVyYWwgbXVsdGljYXRlZ29yaWNhbCB2YXJpYWJsZXMsIHdlIHdhbnQgYSBmYWN0b3IgcmVwcmVzZW50YXRpb24gaW4gYSBzcGVjaWZpYyBvcmRlciwgc28gd2UnbGwgc3BlY2lmeSB0aGF0LCB0b28sIGFsc28gdXNpbmcgdGhlIGBjb2xzYCBmdW5jdGlvbi4NCg0KYGBge3IgcmVhZF93aXRoX2NvbHVtbl9zcGVjc30NCmNvbHVtbl90eXBlc19yaGMgPC0gDQogICAgY29scyh1cmluMSA9ICJkIiwgbWVhbmJwMSA9ICJkIiwgcmVzcDEgPSAiZCIsDQogICAgc3dhbmcxID0gY29sX2ZhY3RvcihjKCJSSEMiLCAiTm8gUkhDIikpLA0KICAgIGRlYXRoID0gY29sX2ZhY3RvcihjKCJObyIsICJZZXMiKSksDQogICAgc2V4ID0gY29sX2ZhY3RvcihjKCJNYWxlIiwgIkZlbWFsZSIpKSwNCiAgICBjYXQxID0gY29sX2ZhY3RvcihjKCJBUkYiLCAiQ0hGIiwgIkNpcnJob3NpcyIsICJDb2xvbiBDYW5jZXIiLCAiQ29tYSIsICJDT1BEIiwNCiAgICAgICAgICAgICAgICAgICAgICAgICJMdW5nIENhbmNlciIsICJNT1NGIHcvTWFsaWduYW5jeSIsICJNT1NGIHcvU2Vwc2lzIikpLA0KICAgIGRucjEgPSBjb2xfZmFjdG9yKGMoIk5vIiwgIlllcyIpKSwNCiAgICBjYXJkID0gY29sX2ZhY3RvcihjKCJObyIsICJZZXMiKSksDQogICAgZ2FzdHIgPSBjb2xfZmFjdG9yKGMoIk5vIiwgIlllcyIpKSwNCiAgICBoZW1hID0gY29sX2ZhY3RvcihjKCJObyIsICJZZXMiKSksDQogICAgbWV0YSA9IGNvbF9mYWN0b3IoYygiTm8iLCAiWWVzIikpLA0KICAgIG5ldXJvID0gY29sX2ZhY3RvcihjKCJObyIsICJZZXMiKSksDQogICAgb3J0aG8gPSBjb2xfZmFjdG9yKGMoIk5vIiwgIlllcyIpKSwNCiAgICByZW5hbCA9IGNvbF9mYWN0b3IoYygiTm8iLCAiWWVzIikpLA0KICAgIHJlc3AgPSBjb2xfZmFjdG9yKGMoIk5vIiwgIlllcyIpKSwNCiAgICBzZXBzID0gY29sX2ZhY3RvcihjKCJObyIsICJZZXMiKSksDQogICAgdHJhdW1hID0gY29sX2ZhY3RvcihjKCJObyIsICJZZXMiKSksDQogICAgaW5jb21lID0gY29sX2ZhY3RvcihjKCJVbmRlciAkMTFrIiwgIiQxMS0kMjVrIiwgIiQyNS0kNTBrIiwgIj4gJDUwayIpKSwNCiAgICBuaW5zY2xhcyA9IGNvbF9mYWN0b3IoYygiUHJpdmF0ZSIsICJQcml2YXRlICYgTWVkaWNhcmUiLCAiTWVkaWNhcmUiLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIk1lZGljYXJlICYgTWVkaWNhaWQiLCAiTWVkaWNhaWQiLCAiTm8gaW5zdXJhbmNlIikpLA0KICAgIHJhY2UgPSBjb2xfZmFjdG9yKGMoIndoaXRlIiwgImJsYWNrIiwgIm90aGVyIikpLA0KICAgIGNhID0gY29sX2ZhY3RvcihjKCJObyIsICJZZXMiLCAiTWV0YXN0YXRpYyIpKQ0KKQ0KDQpgYGANCg0KVGhlIGRhdGEgYXJlIGF2YWlsYWJsZSBhdCB0aGUgbGluayBiZWxvdy4NCg0KYGBge3J9DQpyaGNfcmF3IDwtIHJlYWRfY3N2KCJodHRwczovL2Jpb3N0YXQuYXBwLnZ1bWMub3JnL3dpa2kvcHViL01haW4vRGF0YVNldHMvcmhjLmNzdiIsIA0KICAgICAgICAgICAgICAgICAgICBjb2xfdHlwZXMgPSBjb2x1bW5fdHlwZXNfcmhjKQ0KYGBgDQoNCi0gQWZ0ZXIgd2UgaW1wb3J0IHRoZSBkYXRhLCB3ZSByZS1vcmRlciB0aGUgY29sdW1ucyB0byBwcmVzZW50IHRoZSBTVVBQT1JUIHBhdGllbnQgaWRlbnRpZmljYXRpb24gY29kZSBmaXJzdCwgdGhlbiB0aGUgZXhwb3N1cmUgKGBzd2FuZzFgKSwgdGhlbiBpbmZvcm1hdGlvbiBvbiB0aGUgb3V0Y29tZXMsIHRoZW4gaW5mb3JtYXRpb24gb24gdGhlIGNvdmFyaWF0ZXMgb2YgaW50ZXJlc3QgZm9yIG91ciB3b3JrLCBkcm9wcGluZyB0aGUgb3RoZXIgdmFyaWFibGVzLg0KDQpgYGB7ciBjb21wbGV0ZV9pbXBvcnRfcHJvY2Vzc30NCnJoY19jbGVhbmluZyA8LSByaGNfcmF3ICU+JSANCiAgICBzZWxlY3QocHRpZCwgc3dhbmcxLCANCiAgICAgICAgICAgZGVhdGgsIHNhZG1kdGUsIGRzY2hkdGUsIGR0aGR0ZSwgbHN0Y3RkdGUsDQogICAgICAgICAgIGFnZSwgc2V4LCBlZHUsIGluY29tZSwgbmluc2NsYXMsIHJhY2UsDQogICAgICAgICAgIGNhdDEsIGRucjEsIHd0a2lsbzEsIGhydDEsIG1lYW5icDEsIHJlc3AxLCB0ZW1wMSwNCiAgICAgICAgICAgY2FyZCwgZ2FzdHIsIGhlbWEsIG1ldGEsIG5ldXJvLCBvcnRobywgcmVuYWwsIA0KICAgICAgICAgICByZXNwLCBzZXBzLCB0cmF1bWEsDQogICAgICAgICAgIGFtaWh4LCBjYSwgY2FyZGlvaHgsIGNoZmh4LCBjaHJwdWxoeCwgZGVtZW50aHgsIA0KICAgICAgICAgICBnaWJsZWRoeCwgaW1tdW5oeCwgbGl2ZXJoeCwgbWFsaWdoeCwgcHN5Y2hoeCwgDQogICAgICAgICAgIHJlbmFsaHgsIHRyYW5zaHgsIGFwczEsIGRhczJkM3BjLCBzY29tYTEsIA0KICAgICAgICAgICBzdXJ2Mm1kMSwgYWxiMSwgYmlsaTEsIGNyZWExLCBoZW1hMSwgcGFjbzIxLCANCiAgICAgICAgICAgcGFmaTEsIHBoMSwgcG90MSwgc29kMSwgd2JsYzEpDQpgYGANCg0KIyMgVGhlIERhdGEgU2V0IGFmdGVyIEluaXRpYWwgSW1wb3J0DQoNClRoZSBgcmhjX2NsZWFuaW5nYCBkYXRhIGF0IHRoaXMgc3RhZ2UgaXMgYSB0aWJibGUgKGEgbGF6eSBhbmQgc3VybHkgZGF0YSBmcmFtZSkgY29udGFpbmluZyBgciBucm93KHJoY19jbGVhbmluZylgIG9ic2VydmF0aW9ucyAocm93cykgb24gYHIgbmNvbChyaGNfY2xlYW5pbmcpYCB2YXJpYWJsZXMgKGNvbHVtbnMpLiBUaGUgb2JzZXJ2YXRpb25zIGRlc2NyaWJlIHN1YmplY3RzIHdobyBhcmUgY3JpdGljYWxseSBpbGwsIGFuZCBtb3N0IHZhcmlhYmxlcyB3ZXJlIG9idGFpbmVkIGR1cmluZyAgZGF5IDEgb2YgYSBob3NwaXRhbGl6YXRpb24uDQoNCmBgYHtyfQ0KcmhjX2NsZWFuaW5nDQpgYGANCg0KIyMgQW55IG1pc3NpbmduZXNzPw0KDQpgYGB7ciBpbXB1dGVfZGlzY2hhcmdlX2RhdGV9DQpyaGNfY2xlYW5pbmcgJT4lDQogICAgbWlzc192YXJfc3VtbWFyeSgpDQpgYGANCg0KV2UgY291bGQgYWxzbyBncmFwaCB0aGlzIHdpdGggYGdnX21pc3NfdmFyKHJoY19jbGVhbmluZylgLiBBcyBpdCB0dXJucyBvdXQsIHRoZXJlIHdlcmUgdGhyZWUgdmFyaWFibGVzIGluIHRoZSByYXcgYHJoY2AgZGF0YSB0aGF0IGhhdmUgbWlzc2luZyB2YWx1ZXMsIGJ1dCB3ZSd2ZSBvbWl0dGVkIG9uZSAoYWJvdXQgZGF5IDEgdXJpbmUgb3V0cHV0KSBpbiBjcmVhdGluZyB0aGlzIGByaGNfY2xlYW5pbmdgIGZpbGUuIFRoZSByZW1haW5pbmcgdHdvIGFyZSBkYXRlcywgYW5kIHdlJ2xsIGFkZHJlc3MgdGhvc2UgaW4gdGhlIG1hdGVyaWFsIG9uIGNyZWF0aW5nIG91ciBvdXRjb21lcy4NCg0KIyBUaGUgVHJlYXRtZW50IC8gRXhwb3N1cmUgV2UnbGwgU3R1ZHkNCg0KVGhlIHRyZWF0bWVudC9leHBvc3VyZSB3ZSBhcmUgc3R1ZHlpbmcgaXMgdGhlIHByZXNlbmNlIG9mIGEgU3dhbi1HYW56IChyaWdodCBoZWFydCkgY2F0aGV0ZXIgb24gZGF5IDEsIGNhcHR1cmVkIGluIHRoZSB2YXJpYWJsZSBgc3dhbmcxYC4gVGhlIE1lZGxpbmUgUGx1cyBkZXNjcmlwdGlvbiBvZiByaWdodCBoZWFydCBjYXRoZXRlcml6YXRpb24gW2lzIGhlcmVdKGh0dHBzOi8vbWVkbGluZXBsdXMuZ292L2VuY3kvYXJ0aWNsZS8wMDM4NzAuaHRtKS4gQmFzaWNhbGx5LCB0aGUgcHJvY2VkdXJlIHBhc3NlcyBhIHRoaW4gdHViZSBpbnRvIHRoZSByaWdodCBzaWRlIG9mIHRoZSBoZWFydCBhbmQgdGhlIGFydGVyaWVzIGxlYWRpbmcgdG8gdGhlIGx1bmdzLCB0byBtZWFzdXJlIHRoZSBoZWFydCdzIGZ1bmN0aW9uIGFuZCBibG9vZCBmbG93LiBJdCdzIGRvbmUgaW4gdmVyeSBpbGwgcGF0aWVudHMgZm9yIHNldmVyYWwgcmVhc29ucywgYW5kIGlzIGFsc28gcmVmZXJyZWQgdG8gYXMgcmlnaHQgaGVhcnQgY2F0aGV0ZXJpemF0aW9uIGFuZCBwdWxtb25hcnkgYXJ0ZXJ5IGNhdGhldGVyaXphdGlvbi4gDQoNCmBgYHtyfQ0KcmhjX2NsZWFuaW5nICU+JSB0YWJ5bChzd2FuZzEpDQpgYGANCg0KT2YgY291cnNlLCB0aGUgbW9zdCBpbXBvcnRhbnQgaXNzdWUgdG8gdXMgaXMgdGhhdCBlYWNoIHN1YmplY3Qgd2FzIG5vdCByYW5kb21seSBhbGxvY2F0ZWQgdG8gZWl0aGVyIHRoZSAiUkhDIiBvciAiTm8gUkhDIiBncm91cC4gSW5zdGVhZCwgdGhpcyBpcyBhbiAqKm9ic2VydmF0aW9uYWwqKiBzdHVkeSwgd2hlcmUgZWFjaCBzdWJqZWN0IHJlY2VpdmVkIHRoZWlyIHRyZWF0bWVudCAoUkhDIG9yIG5vKSBvbiB0aGUgYmFzaXMgb2Ygd2hhdCB0aGVpciBwcm92aWRlcnMgdGhvdWdodCB3b3VsZCBiZSBiZXN0IGZvciB0aGVtLg0KDQojIFRoZSAzIE91dGNvbWVzIFdlJ2xsIFN0dWR5DQoNCldlIHdpbGwgZGVmaW5lIHRocmVlIG91dGNvbWVzIGhlcmU6IG9uZSBiaW5hcnksIG9uZSBxdWFudGl0YXRpdmUsIGFuZCBvbmUgdGltZS10by1ldmVudC4gDQoNCi0gT3VyIGZpcnN0IG91dGNvbWUgaXMgaW4tc3R1ZHkgbW9ydGFsaXR5LCBjYXB0dXJlZCBhcyBhIGJpbmFyeSB2YXJpYWJsZS4NCi0gV2Ugd2lsbCB0aGVuIGRlZmluZSBob3NwaXRhbCBsZW5ndGggb2Ygc3RheSwgYSBxdWFudGl0YXRpdmUgdmFyaWFibGUsIGFuZCB0aGVuDQotIFRpbWUgdG8gZGVhdGgsIGEgdGltZS10by1ldmVudCB2YXJpYWJsZSB0aGF0IHdpbGwgYmUgcmlnaHQtY2Vuc29yZWQgZm9yIHRob3NlIHdobyBkaWQgbm90IGRpZSBkdXJpbmcgdGhlIHN0dWR5Lg0KDQojIyBNb3J0YWxpdHksIGEgYmluYXJ5IG91dGNvbWU6IGBkZWF0aGANCg0KT25lIG91dGNvbWUgaXMgZGVhdGggZHVyaW5nIHRoZSBzdHVkeSwgYXMgY2FwdHVyZWQgaW4gdGhlIGBkZWF0aGAgdmFyaWFibGUuIFRoaXMgaXMgYSAqKmJpbmFyeSoqIG91dGNvbWUuDQoNCmBgYHtyfQ0KcmhjX2NsZWFuaW5nICU+JSB0YWJ5bChkZWF0aCkNCmBgYA0KDQpUaG9zZSBwZW9wbGUgd2l0aG91dCBhIHZhbHVlIGZvciBgZHRoZHRlYCB0dXJuIG91dCB0byBiZSB0aGUgcGVvcGxlIHdpdGggYGRlYXRoYCB2YWx1ZXMgb2YgIk5vIiBhbmQgdGhpcyBtYWtlcyBzZW5zZSwgYmVjYXVzZSBgZHRoZHRlYCBpcyBhIGNvZGUgZm9yIGRhdGUgb2YgZGVhdGguDQoNCiMjIFdvcmtpbmcgd2l0aCBEYXRlcyB0byBEZWZpbmUgdGhlIFRpbWUtUmVsYXRlZCBPdXRjb21lcw0KDQojIyMgVGhlIFJhdyBEYXRhIG9uIERhdGVzDQoNClZhcmlhYmxlIHwgRGVmaW5pdGlvbiAgICAgICAgICAgICAgIHwgVmFsdWVzDQotLS0tLS0tLSB8IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSB8IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCmBzYWRtZHRlYAl8IFN0dWR5IEFkbWlzc2lvbiBEYXRlICAgIHwgSW50ZWdlciByYW5naW5nIGZyb20gMTA3NTQgdG8gMTI0NDEsIGluZGljYXRpbmcgdGhlIFwjIG9mIGRheXMgYWZ0ZXIgMTk2MC0wMS0wMSwgc28gc3R1ZHkgYWRtaXNzaW9uIGRhdGUgcmFuZ2UgaXMgYWN0dWFsbHkgMTk4OS0wNi0xMSB0byAxOTk0LTAxLTIzLg0KYGRzY2hkdGVgIHwgSG9zcGl0YWwgRGlzY2hhcmdlIERhdGUgfCBJbnRlZ2VyIHJhbmdpbmcgZnJvbSAxMDc1NyB0byAxMjU2MCwgaW5kaWNhdGluZyB0aGUgXCMgb2YgZGF5cyBhZnRlciAxOTYwLTAxLTAxLCBzbyBob3NwaXRhbCBkaXNjaGFyZ2UgZGF0ZSByYW5nZSBpcyBhY3R1YWxseSAxOTg5LTA2LTE0IHRvIDE5OTQtMDUtMjIuICgqKk5vdGU6IDEgbWlzc2luZyB2YWx1ZSoqKQ0KYGR0aGR0ZWAgfCBEZWF0aCBEYXRlICAgICAgICAgICAgICAgfCBJbnRlZ2VyIHJhbmdpbmcgZnJvbSAxMDc1NyB0byAxMjc4MywgaW5kaWNhdGluZyB0aGUgXCMgb2YgZGF5cyBhZnRlciAxOTYwLTAxLTAxLCBzbyBkZWF0aCBkYXRlIHJhbmdlIGlzIGFjdHVhbGx5IDE5ODktMDYtMTQgdG8gMTk5NC0xMi0zMS4gVGhlICoqMjAxMyBwZW9wbGUgd2l0aCBtaXNzaW5nIHZhbHVlcyoqIGFyZSB0aG9zZSB3aXRoIE5vIGZvciBgZGVhdGhgLg0KYGxzdGN0ZHRlYCB8IERhdGUgb2YgTGFzdCBDb250YWN0ICAgfCBJbnRlZ2VyIHJhbmdpbmcgZnJvbSAxMDc1NiB0byAxMjY0NCwgaW5kaWNhdGluZyB0aGUgXCMgb2YgZGF5cyBhZnRlciAxOTYwLTAxLTAxLCBzbyBkZWF0aCBkYXRlIHJhbmdlIGlzIGFjdHVhbGx5IDE5ODktMDYtMTMgdG8gMTk5NC0wOC0xNC4NCg0KTmV4dCwgd2Ugd2lsbCBkZXZlbG9wIHRoZSB0d28gb3RoZXIgcHJpbWFyeSBvdXRjb21lczogc3Vydml2YWwgdGltZSAod2hpY2ggd2UnbGwgc3RvcmUgaW4gYHN1cnZkYXlzYCksIGFuZCBob3NwaXRhbCBsZW5ndGggb2Ygc3RheSAod2hpY2ggd2lsbCBnbyBpbiBgaG9zcGRheXNgKS4gDQoNCi0gSXQgdHVybnMgb3V0IHRoYXQgZm9yIHRocmVlIHN1YmplY3RzLCB0aGVpciBkYXRlcyBvZiBsYXN0IGNvbnRhY3QgYXBwZWFyIHRvIG9jY3VyIGFmdGVyIHRoZWlyIGRlYXRoIGRhdGVzLCB3aGljaCBpcyBpcnJpdGF0aW5nLCBidXQgZG9lc24ndCBhZmZlY3QgdXMgbXVjaCBzaW5jZSB0aGUgb25seSB3YXkgaW4gd2hpY2ggd2UnbGwgdXNlIHRoZSBsYXN0IGNvbnRhY3QgZGF0ZSBpcyBmb3IgY2FsY3VsYXRpbmcgc3Vydml2YWwgdGltZSBmb3IgcGF0aWVudHMgKndpdGhvdXQqIGEgZGVhdGggZGF0ZS4gDQoNCiMjIyBJbXB1dGUgaG9zcGl0YWwgZGlzY2hhcmdlIGRhdGUgZm9yIHRoZSBvbmUgbWlzc2luZyB2YWx1ZQ0KDQpXZSBjb3VsZCBqdXN0IGltcHV0ZSBhIHJhbmRvbSB2YWx1ZSwgYnV0IHRoYXQncyBub3QgYSBncmVhdCBpZGVhLCBzaW5jZSBob3NwaXRhbCBkaXNjaGFyZ2UgZGF0ZSBpcyB0aWVkIHRvIG90aGVyIGluZm9ybWF0aW9uIHRoYXQgaXMgYXZhaWxhYmxlIHRvIHVzIGZvciB0aGlzIHBhdGllbnQuIFRoZSBzdWJqZWN0IHdpdGggYSBtaXNzaW5nIGhvc3BpdGFsIGRpc2NoYXJnZSBkYXRlIHR1cm5zIG91dCB0byBiZSBgcHRpZGAgMDgzODIuDQoNCmBgYHtyfQ0KcmhjX2NsZWFuaW5nICU+JSBmaWx0ZXIoaXMubmEoZHNjaGR0ZSkpICU+JSANCiAgICBzZWxlY3QocHRpZCwgZGVhdGgsIHNhZG1kdGUsIGRzY2hkdGUsIGR0aGR0ZSwgbHN0Y3RkdGUpDQpgYGANCg0KU28gdGhlIHBhdGllbnQgd2FzIGFkbWl0dGVkIHRvIHRoZSBzdHVkeSBvbiBkYXkgMTIzODYsIGhhZCB0aGVpciBsYXN0IGNvbnRhY3Qgd2l0aCB0aGUgc3R1ZHkgb24gZGF5IDEyNTgyIChpLmUuIDE5NiBkYXlzIGFmdGVyIHN0dWR5IGVudHJ5KSwgYW5kIGRpZWQgb24gZGF5IDEyNzgwIChpLmUuIDM5NCBkYXlzIGFmdGVyIHN0dWR5IGVudHJ5LikgSXQgc2VlbXMgcmF0aW9uYWwgdG8gaW1wdXRlIGEgdmFsdWUgb2YgaG9zcGl0YWwgZGlzY2hhcmdlIGRhdGUgdGhhdCBmYWxscyBpbiBiZXR3ZWVuIHRoZWlyIGRhdGUgb2Ygc3R1ZHkgYWRtaXNzaW9uIGFuZCB0aGVpciBkYXRlIG9mIGxhc3QgY29udGFjdC4gDQoNCi0gQW1vbmcgdGhlIFJIQyBwYXRpZW50cywgd2hhdCBpcyBhIHJlYXNvbmFibGUgZ3Vlc3MgYXMgdG8gdGhlIHRpbWUgZnJvbSBzdHVkeSBhZG1pc3Npb24gdG8gaG9zcGl0YWwgZGlzY2hhcmdlIHRoYXQgd2UgbWlnaHQgYXBwbHkgdG8gdGhpcyBwYXRpZW50PyBXZSBjb3VsZCBzZWUgd2hhdCB0aGUgZGlzdHJpYnV0aW9uIG9mIHRoZXNlIGRpZmZlcmVuY2VzIGFyZSBpbiB0aGUgYXZhaWxhYmxlIGRhdGEsIGxpa2Ugc28uLi4NCg0KYGBge3J9DQptb3NhaWM6OmZhdnN0YXRzKGRzY2hkdGUgLSBzYWRtZHRlIH4gc3dhbmcxLCBkYXRhID0gcmhjX2NsZWFuaW5nKQ0KYGBgDQoNCkknbGwgYXJiaXRyYXJpbHkgY2hvb3NlIHRvIGltcHV0ZSBhIGRpc2NoYXJnZSBkYXRlIGZvciBgcHRpZGAgMDgzODIgc28gYXMgdG8geWllbGQgYSAyNSBkYXkgaG9zcGl0YWxpemF0aW9uLCBzbyBhcyB0byBhZmZlY3QgdGhlIG1lYW4gc3RheSBsZW5ndGggd2l0aGluIHRoZSBSSEMgZ3JvdXAgYXMgbGl0dGxlIGFzIHBvc3NpYmxlLiANCg0KVGhpcyBpbXBsaWVzIHRoYXQgd2UnbGwgaW1wdXRlIGEgdmFsdWUgb2YgMTIzODYgKyAyNSA9IDEyNDExIGZvciB0aGlzIHBhdGllbnQuDQoNCmBgYHtyfQ0KcmhjX2NsZWFuaW5nIDwtIHJoY19jbGVhbmluZyAlPiUNCiAgICBtdXRhdGUoZHNjaGR0ZSA9IHJlcGxhY2VfbmEoZHNjaGR0ZSwgMTI0MTEpKQ0KDQpyaGNfY2xlYW5pbmcgJT4lIHNlbGVjdChkc2NoZHRlKSAlPiUgbWlzc192YXJfc3VtbWFyeSgpDQpgYGANCg0KIyMgTGVuZ3RoIG9mIGluaXRpYWwgaG9zcGl0YWwgc3RheSwgYSBxdWFudGl0YXRpdmUgb3V0Y29tZTogYGhvc3BkYXlzYA0KDQpPdXIgdGhpcmQgb3V0Y29tZSBpcyBsZW5ndGggb2YgaW5pdGlhbCBob3NwaXRhbCBzdGF5LCB3aGljaCBpcyBjYXB0dXJlZCBieSBzdWJ0cmFjdGluZyB0aGUgc3R1ZHkgYWRtaXNzaW9uIGRhdGUgKGBzYWRtZHRlYCkgZnJvbSB0aGUgaG9zcGl0YWwgZGlzY2hhcmdlIGRhdGUgKGBkc2NoZHRlYCkuIFRoaXMgaXMgYSAqKnF1YW50aXRhdGl2ZSoqIG91dGNvbWVzLCBhbmQgd2Ugd2lsbCBjcmVhdGUgaXQgc29vbiwgYW5kIG5hbWUgaXQgYGhvc3BkYXlzYCBpbiBvdXIgZGF0YSBzZXQuDQoNCg0KYGBge3J9DQpyaGNfY2xlYW5pbmcgPC0gcmhjX2NsZWFuaW5nICU+JQ0KICAgIG11dGF0ZShob3NwZGF5cyA9IGRzY2hkdGUgLSBzYWRtZHRlKQ0KDQptb3NhaWM6OmZhdnN0YXRzKGhvc3BkYXlzIH4gc3dhbmcxLCBkYXRhID0gcmhjX2NsZWFuaW5nKQ0KYGBgDQoNCkl0IGxvb2tzIGxpa2UgYWxsIG9mIHRoZXNlIHZhbHVlcyBhcmUgcmVhc29uYWJsZSAoaW4gdGhhdCB0aGV5IGFyZSBhbGwgcG9zaXRpdmUpIGFuZCB3ZSBoYXZlIG5vIG1pc3NpbmcgdmFsdWVzLg0KDQojIyBTdXJ2aXZhbCBUaW1lIGluIERheXMsIGEgdGltZS10by1ldmVudCBvdXRjb21lOiBgc3VydmRheXNgDQoNCkFub3RoZXIgb3V0Y29tZSBpcyBzdXJ2aXZhbCB0aW1lIGluIGRheXMgKHdoaWNoIGlzIGNlbnNvcmVkIGZvciBhbGwgcGF0aWVudHMgd2hvIGRpZCBub3QgZGllIGR1cmluZyB0aGUgc3R1ZHkpLCBhbmQgd2hpY2ggaXMgY2FwdHVyZWQgYnkgc3VidHJhY3RpbmcgdGhlIHN0dWR5IGFkbWlzc2lvbiBkYXRlIChgc2FkbWR0ZWApIGZyb20gdGhlIGRlYXRoIGRhdGUgKGBkdGhkdGVgKSBmb3IgdGhlIHBhdGllbnRzIHdobyBkaWVkIGluIHN0dWR5LCBhbmQgdGhlIHN0dWR5IGFkbWlzc2lvbiBkYXRlIChgc2FkbWR0ZWApIGZyb20gdGhlIGRhdGUgb2YgbGFzdCBjb250YWN0IChgbHN0Y3RkdGVgKSBmb3IgdGhlIHBhdGllbnRzIHdobyBkaWQgbm90IGRpZSBpbiBzdHVkeS4gVGhpcyBpcyBhICoqdGltZS10by1ldmVudCoqIG91dGNvbWUsIGFuZCB3ZSB3aWxsIGNyZWF0ZSBpdCBzb29uLCBhbmQgbmFtZSBpdCBgc3VydmRheXNgIGluIG91ciBkYXRhIHNldC4NCg0KLSBGb3IgdGhlIHN1YmplY3RzIHdobyBkaWVkLCB0aGlzIGlzIHN0cmFpZ2h0Zm9yd2FyZCwgaW4gdGhhdCB3ZSB0YWtlIHRoZSBkZWF0aCBkYXRlIGFuZCBzdWJ0cmFjdCB0aGUgc3R1ZHkgYWRtaXNzaW9uIGRhdGUuIA0KLSBGb3IgdGhvc2Ugd2hvIHN1cnZpdmVkIHRocm91Z2ggdGhlIHN0dWR5LCB3ZSB0YWtlIHRoZSBkYXRlIG9mIGxhc3QgY29udGFjdCBhbmQgc3VidHJhY3QgdGhlIHN0dWR5IGFkbWlzc2lvbiBkYXRlLCBpbiBvcmRlciB0byBnZXQgdGhlaXIgKHJpZ2h0IGNlbnNvcmVkKSBzdXJ2aXZhbCB0aW1lcy4NCg0KYGBge3J9DQpyaGNfY2xlYW5pbmcgPC0gcmhjX2NsZWFuaW5nICU+JQ0KICAgIG11dGF0ZShzdXJ2ZGF5cyA9IGlmZWxzZShkZWF0aCA9PSAiWWVzIiwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRzY2hkdGUgLSBzYWRtZHRlLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsc3RjdGR0ZSAtIHNhZG1kdGUpKQ0KDQptb3NhaWM6OmZhdnN0YXRzKHN1cnZkYXlzIH4gZGVhdGgsIGRhdGEgPSByaGNfY2xlYW5pbmcpDQpgYGANCg0KQWdhaW4sIHRoaXMgbG9va3MgcmVhc29uYWJsZS4gQWxsIG9mIHRoZXNlIHZhbHVlcyBhcmUgc3RyaWN0bHkgcG9zaXRpdmUsIGZvciBpbnN0YW5jZSwgYW5kIG5vbmUgc2VlbSB3aWxkbHkgb3V0IG9mIGxpbmUsIGFuZCB3ZSBoYXZlIG5vIG1pc3NpbmcgdmFsdWVzLiBXZSBhbHNvIHNlZSB0aGF0IHRob3NlIHdobyBkaWVkIGhhZCBtdWNoIHNob3J0ZXIgc3Vydml2YWwgdGltZXMgKHR5cGljYWxseSkgdGhhbiB0aG9zZSB3aG8gc3Vydml2ZWQsIHdoaWNoIG1ha2VzIHNlbnNlLg0KDQpXZSBtaWdodCBhbHNvIGxvb2sgYXQgdGhlIGBzdXJ2ZGF5c2AgdmFyaWFibGUgYnJva2VuIGRvd24gYnkgUkhDIHN0YXR1cy4NCg0KYGBge3J9DQptb3NhaWM6OmZhdnN0YXRzKHN1cnZkYXlzIH4gc3dhbmcxLCBkYXRhID0gcmhjX2NsZWFuaW5nKQ0KYGBgDQoNClRoZXNlIHNlZW0gcXVpdGUgY29tcGFyYWJsZSwgc28gZmFyLCBhbmQgYWdhaW4sIHRoZXkgbWFrZSBzZW5zZSAoYWxsIGFyZSBwb3NpdGl2ZSwgYW5kIHRoZXJlIGFyZSBubyBtaXNzaW5nIHZhbHVlcy4pDQoNCiMgVGhlIDUwIENvdmFyaWF0ZXMgV2UnbGwgU3R1ZHkNCg0KT3VyIHByb3BlbnNpdHkgbW9kZWwgd2lsbCBldmVudHVhbGx5IGluY2x1ZGUgNTAgY292YXJpYXRlcywgZGVzY3JpYmVkIGJlbG93Lg0KDQojIyBHcm91cCAxOiBTb2Npby1kZW1vZ3JhcGhpY3MgKDYgY292YXJpYXRlcykNCg0KVmFyaWFibGUgfCBEZWZpbml0aW9uICAgICAgICAgICAgICAgDQotLS0tLS0tLSB8IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KYGFnZWAgfCBBZ2UgaW4geWVhcnMgDQpgc2V4YAl8IFNleAkNCmBlZHVgCXwgWWVhcnMgb2YgRWR1Y2F0aW9uIA0KYGluY29tZWAgfCBJbmNvbWUgQ2F0ZWdvcnkgKDQgbGV2ZWxzKQ0KYG5pbnNjbGFzYCB8IEluc3VyYW5jZSBDYXRlZ29yeSAoNiBsZXZlbHMpIA0KYHJhY2VgIHwgU2VsZi1SZXBvcnRlZCBSYWNlCSgzIGxldmVscykNCg0KSGVyZSdzIGEgVGFibGUgMSBmb3IgdGhlc2UgY292YXJpYXRlcy4NCg0KYGBge3IgY292YXJpYXRlc19kZW1vZ3JhcGhpY3N9DQp2YXJzMDEgPC0gYygiYWdlIiwgInNleCIsICJlZHUiLCAiaW5jb21lIiwgIm5pbnNjbGFzIiwgInJhY2UiKQ0KDQp0YWJsZTFfMDEgPC0gDQogICAgQ3JlYXRlVGFibGVPbmUodmFycyA9IHZhcnMwMSwgc3RyYXRhID0gInN3YW5nMSIsIA0KICAgICAgICAgICAgICAgICAgIGRhdGEgPSByaGNfY2xlYW5pbmcsIHRlc3QgPSBGQUxTRSkNCg0KcHJpbnQodGFibGUxXzAxLCBzbWQgPSBUUlVFKQ0KYGBgDQoNCi0gVHdvIG9mIHRoZXNlIGNvdmFyaWF0ZXMgc2hvdyBzdGFuZGFyZGl6ZWQgbWVhbiBkaWZmZXJlbmNlcyBsYXJnZXIgdGhhbiAwLjEwLg0KDQojIyBHcm91cCAyOiBQcmVzZW50YXRpb24gYXQgQWRtaXNzaW9uICg3IGNvdmFyaWF0ZXMpDQoNClZhcmlhYmxlIHwgRGVmaW5pdGlvbiAgICAgICAgICAgICAgIA0KLS0tLS0tLS0gfCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCmBjYXQxYAl8IFByaW1hcnkgZGlzZWFzZSBjYXRlZ29yeSAoOSBsZXZlbHMpDQpgZG5yMWAJfCBEbyBOb3QgUmVzdXNjaXRhdGUgc3RhdHVzLCBkYXkgMSANCmB3dGtpbG8xYAl8IFdlaWdodCBpbiBrZywgZGF5IDEJDQpgaHJ0MWAJfCBIZWFydCByYXRlLCBkYXkgMQkNCmBtZWFuYnAxYAl8IE1lYW4gQmxvb2QgUHJlc3N1cmUsIGRheSAxIA0KYHJlc3AxYAl8IFJlc3BpcmF0b3J5IHJhdGUsIGRheSAxDQpgdGVtcDFgCXwgVGVtcGVyYXR1cmUsIEMsIGRheSAxDQoNCmBgYHtyIGNvdmFyaWF0ZXNfYWRtaXNzaW9uX3ByZXNlbnRhdGlvbn0NCnZhcnMwMiA8LSBjKCJjYXQxIiwgImRucjEiLCAid3RraWxvMSIsICJocnQxIiwgIm1lYW5icDEiLCANCiAgICAgICAgICAicmVzcDEiLCAidGVtcDEiKQ0KDQp0YWJsZTFfMDIgPC0gDQogICAgQ3JlYXRlVGFibGVPbmUodmFycyA9IHZhcnMwMiwgc3RyYXRhID0gInN3YW5nMSIsIA0KICAgICAgICAgICAgICAgICAgIGRhdGEgPSByaGNfY2xlYW5pbmcsIHRlc3QgPSBGQUxTRSkNCg0KcHJpbnQodGFibGUxXzAyLCBzbWQgPSBUUlVFKQ0KYGBgDQoNCi0gU2l4IG9mIHRoZXNlIGNvdmFyaWF0ZXMgc2hvdyBzdGFuZGFyZGl6ZWQgbWVhbiBkaWZmZXJlbmNlcyBsYXJnZXIgdGhhbiAwLjEwLg0KDQojIyBHcm91cCAzOiBBZG1pc3Npb24gRGlhZ25vc2lzIENhdGVnb3JpZXMgKDEwIGNvdmFyaWF0ZXMpDQoNClZhcmlhYmxlIHwgRGVmaW5pdGlvbiAgICAgICAgICAgICAgIA0KLS0tLS0tLS0gfCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCmBjYXJkYAl8IENhcmRpb3Zhc2N1bGFyIGRpYWdub3NpcwkNCmBnYXN0cmAgfCBHYXN0cm9pbnRlc3RpbmFsIGRpYWdub3Npcw0KYGhlbWFgCXwgSGVtYXRvbG9naWMgZGlhZ25vc2lzDQpgbWV0YWAJfCBNZXRhYm9saWMgZGlhZ25vc2lzDQpgbmV1cm9gCXwgTmV1cm9sb2dpY2FsIGRpYWdub3Npcw0KYG9ydGhvYAl8IE9ydGhvcGVkaWMgZGlhZ25vc2lzDQpgcmVuYWxgCXwgUmVuYWwgZGlhZ25vc2lzDQpgcmVzcGAJfCBSZXNwaXJhdG9yeSBkaWFnbm9zaXMJDQpgc2Vwc2AJfCBTZXBzaXMgZGlhZ25vc2lzCQ0KYHRyYXVtYWAgfCBUcmF1bWEgZGlhZ25vc2lzCQ0KDQpgYGB7ciBjb3ZhcmlhdGVzX2FkbWlzc2lvbl9kaWFnbm9zZXN9DQp2YXJzMDMgPC0gYygiY2FyZCIsICJnYXN0ciIsICJoZW1hIiwgIm1ldGEiLCAibmV1cm8iLCAib3J0aG8iLA0KICAgICAgICAgICJyZW5hbCIsICJyZXNwIiwgInNlcHMiLCAidHJhdW1hIikNCg0KdGFibGUxXzAzIDwtIA0KICAgIENyZWF0ZVRhYmxlT25lKHZhcnMgPSB2YXJzMDMsIHN0cmF0YSA9ICJzd2FuZzEiLCANCiAgICAgICAgICAgICAgICAgICBkYXRhID0gcmhjX2NsZWFuaW5nLCB0ZXN0ID0gRkFMU0UpDQoNCnByaW50KHRhYmxlMV8wMywgc21kID0gVFJVRSkNCmBgYA0KDQotIFNldmVuIG9mIHRoZXNlIGNvdmFyaWF0ZXMgc2hvdyBzdGFuZGFyZGl6ZWQgbWVhbiBkaWZmZXJlbmNlcyBsYXJnZXIgdGhhbiAwLjEwLg0KDQojIyBHcm91cCA0OiBDb21vcmJpZCBJbGxuZXNzIGFuZCBUcmFuc2ZlciBTdGF0dXMgKDEzIGNvdmFyaWF0ZXMpDQoNClZhcmlhYmxlIHwgRGVmaW5pdGlvbiAgICAgICAgICAgICAgIA0KLS0tLS0tLS0gfCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCmBhbWloeGAJfCBEZWZpbml0ZSBNeW9jYXJkaWFsIEluZmFyY3Rpb24JDQpgY2FgIHwgQ2FuY2VyICgzIGxldmVscykNCmBjYXJkaW9oeGAgfCBBY3V0ZSBNSSwgUGVyaXBoZXJhbCBWYXNjdWxhciBEaXNlYXNlLCBTZXZlcmUgQ2FyZGlvdmFzY3VsYXIgU3ltcHRvbXMgKE5ZSEEtQ2xhc3MgSUlJKSwgVmVyeSBTZXZlcmUgQ2FyZGlvdmFzY3VsYXIgU3ltcHRvbXMgKE5ZSEEtIElWKQ0KYGNoZmh4YCB8CUNvbmdlc3RpdmUgSGVhcnQgRmFpbHVyZQkNCmBjaHJwdWxoeGAgfAlDaHJvbmljIFB1bG1vbmFyeSBEaXNlYXNlLCBTZXZlcmUgb3IgVmVyeSBTZXZlcmUgUHVsbW9uYXJ5IERpc2Vhc2UJDQpgZGVtZW50aHhgIHwJRGVtZW50aWEsIFN0cm9rZSBvciBDZXJlYnJhbCBJbmZhcmN0LCBQYXJraW5zb24ncyBEaXNlYXNlCQ0KYGdpYmxlZGh4YCB8CVVwcGVyIEdJIEJsZWVkaW5nCQ0KYGltbXVuaHhgIHwJSW1tdW5vc3VwcGVyc3Npb24sIE9yZ2FuIFRyYW5zcGxhbnQsIEhJViBQb3NpdGl2aXR5LCBEaWFiZXRlcyBNZWxsaXR1cyBXaXRob3V0IEVuZCBPcmdhbiBEYW1hZ2UsIERpYWJldGVzIE1lbGxpdHVzIFdpdGggRW5kIE9yZ2FuIERhbWFnZSwgQ29ubmVjdGl2ZSBUaXNzdWUgRGlzZWFzZQkNCmBsaXZlcmh4YCB8CUNpcnJob3NpcywgSGVwYXRpYyBGYWlsdXJlIA0KYG1hbGlnaHhgIHwJU29saWQgVHVtb3IsIE1ldGFzdGF0aWMgRGlzZWFzZSwgQ2hyb25pYyBMZXVrZW1pYS9NeWVsb21hLCBBY3V0ZSBMZXVrZW1pYSwgTHltcGhvbWEJDQpgcHN5Y2hoeGAgfAlQc3ljaGlhdHJpYyBIaXN0b3J5LCBBY3RpdmUgUHN5Y2hvc2lzIG9yIFNldmVyZSBEZXByZXNzaW9uCQ0KYHJlbmFsaHhgIHwJQ2hyb25pYyBSZW5hbCBEaXNlYXNlLCBDaHJvbmljIEhlbW9kaWFseXNpcyBvciBQZXJpdG9uZWFsIERpYWx5c2lzCQ0KYHRyYW5zaHhgIHwJVHJhbnNmZXIgKFw+IDI0IEhvdXJzKSBmcm9tIEFub3RoZXIgSG9zcGl0YWwJDQoNCmBgYHtyIGNvdmFyaWF0ZXNfY29tb3JiaWRpdGllc19hbmRfdHJhbnNmZXJ9DQp2YXJzMDQgPC0gYygiYW1paHgiLCAiY2EiLCAiY2FyZGlvaHgiLCAiY2hmaHgiLCAiY2hycHVsaHgiLCAiZGVtZW50aHgiLA0KICAgICAgICAgICJnaWJsZWRoeCIsICJpbW11bmh4IiwgImxpdmVyaHgiLCAibWFsaWdoeCIsICJwc3ljaGh4IiwgInJlbmFsaHgiLCAidHJhbnNoeCIpDQoNCnRhYmxlMV8wNCA8LSANCiAgICBDcmVhdGVUYWJsZU9uZSh2YXJzID0gdmFyczA0LCBzdHJhdGEgPSAic3dhbmcxIiwgDQogICAgICAgICAgICAgICAgICAgZGF0YSA9IHJoY19jbGVhbmluZywgdGVzdCA9IEZBTFNFKQ0KDQpwcmludCh0YWJsZTFfMDQsIHNtZCA9IFRSVUUpDQpgYGANCg0KLSBTZXZlbiBvZiB0aGVzZSBjb3ZhcmlhdGVzIHNob3cgc3RhbmRhcmRpemVkIG1lYW4gZGlmZmVyZW5jZXMgbGFyZ2VyIHRoYW4gMC4xMC4NCg0KIyMgR3JvdXAgNTogRGF5IDEgU3VtbWFyeSBNZWFzdXJlcyBvZiBQcmVzZW50YXRpb24gLyBTZXZlcml0eSBvZiBJbGxuZXNzICg0IGNvdmFyaWF0ZXMpDQoNClZhcmlhYmxlIHwgRGVmaW5pdGlvbiAgICAgICAgICAgICAgIA0KLS0tLS0tLS0gfCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCmBhcHMxYCAgIHwgCUFQQUNIRSBJSUkgc2NvcmUgaWdub3JpbmcgQ29tYSwgZGF5IDEJDQpgZGFzMmQzcGNgCXwgREFTSSAoRHVrZSBBY3Rpdml0eSBTdGF0dXMgSW5kZXggcHJpb3IgdG8gYWRtaXNzaW9uKSANCmBzY29tYTFgCXwgU3VwcG9ydCBDb21hIHNjb3JlIGJhc2VkIG9uIEdsYXNnb3csIGRheSAxIA0KYHN1cnYybWQxYAl8IFNVUFBPUlQgbW9kZWwgZXN0aW1hdGUgb2YgUHJvYihzdXJ2aXZpbmcgMiBtb250aHMpDQoNCmBgYHtyIGNvdmFyaWF0ZXNfcHJlc2VudGF0aW9uX3NldmVyaXR5fQ0KdmFyczA1IDwtIGMoImFwczEiLCAiZGFzMmQzcGMiLCAic2NvbWExIiwgInN1cnYybWQxIikNCg0KdGFibGUxXzA1IDwtIA0KICAgIENyZWF0ZVRhYmxlT25lKHZhcnMgPSB2YXJzMDUsIHN0cmF0YSA9ICJzd2FuZzEiLCANCiAgICAgICAgICAgICAgICAgICBkYXRhID0gcmhjX2NsZWFuaW5nLCB0ZXN0ID0gRkFMU0UpDQoNCnByaW50KHRhYmxlMV8wNSwgc21kID0gVFJVRSkNCmBgYA0KDQotIFRocmVlIG9mIHRoZXNlIGNvdmFyaWF0ZXMgc2hvdyBzdGFuZGFyZGl6ZWQgbWVhbiBkaWZmZXJlbmNlcyBsYXJnZXIgdGhhbiAwLjEwLg0KDQojIyBHcm91cCA2OiBEYXkgMSBMYWIgUmVzdWx0cyAoMTAgY292YXJpYXRlcykNCg0KVmFyaWFibGUgfCBEZWZpbml0aW9uICAgICAgICAgICAgICAgDQotLS0tLS0tLSB8IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KYGFsYjFgICAgfCBBbGJ1bWluICAgICAgICAgICAgICAgICAgDQpgYmlsaTFgCSB8IEJpbGlydWJpbiAgICAgICAgICAgICAgICANCmBjcmVhMWAJIHwgU2VydW0gQ3JlYXRpbmluZSAgICAgICAgIA0KYGhlbWExYAkgfCBIZW1hdG9jcml0ICAgICAgICAgICAgICAgDQpgcGFjbzIxYCB8IFBhQ28yICAgICAgICAgICAgICAgICAgICANCmBwYWZpMWAJIHwgUGFPMlwvKC4wMSBGSU8yKQkgICAgICAgIA0KYHBoMWAJICB8IFNlcnVtIFBIIChhcnRlcmlhbCkJDQpgcG90MWAJIHwgU2VydW0gUG90YXNzaXVtCSAgICAgICAgDQpgc29kMWAJIHwgU2VydW0gU29kaXVtCSAgICAgICAgICAgDQpgd2JsYzFgCSB8IFdoaXRlIGJsb29kIGNlbGwgY291bnQgKDQgc3ViamVjdHMgd2l0aCB2YWx1ZSAwKQ0KDQpgYGB7ciBjb3ZhcmlhdGVzX2RheV8xX2xhYl9yZXN1bHRzfQ0KdmFyczA2IDwtIGMoImFsYjEiLCAiYmlsaTEiLCAiY3JlYTEiLCAiaGVtYTEiLCAicGFjbzIxIiwNCiAgICAgICAgICAgICJwYWZpMSIsICJwb3QxIiwgInNvZDEiLCAid2JsYzEiKQ0KDQp0YWJsZTFfMDYgPC0gDQogICAgQ3JlYXRlVGFibGVPbmUodmFycyA9IHZhcnMwNiwgc3RyYXRhID0gInN3YW5nMSIsIA0KICAgICAgICAgICAgICAgICAgIGRhdGEgPSByaGNfY2xlYW5pbmcsIHRlc3QgPSBGQUxTRSkNCg0KcHJpbnQodGFibGUxXzA2LCBzbWQgPSBUUlVFKQ0KYGBgDQoNCi0gU2l4IG9mIHRoZXNlIGNvdmFyaWF0ZXMgc2hvdyBzdGFuZGFyZGl6ZWQgbWVhbiBkaWZmZXJlbmNlcyBsYXJnZXIgdGhhbiAwLjEwLg0KDQpTbywgaW4gYWxsLCAzMSBvZiB0aGUgNTAgY292YXJpYXRlcyBzaG93IHN0YW5kYXJkaXplZCBtZWFuIGRpZmZlcmVuY2VzIHRoYXQgZXhjZWVkIDAuMTAgYmVmb3JlIGFueSBwcm9wZW5zaXR5IHNjb3JlIGFkanVzdG1lbnQuDQoNCiMgQSBMaXR0bGUgUmVmYWN0b3JpbmcNCg0KRm9yIG91ciBvdXRjb21lLCBhbmQgb3VyIHRyZWF0bWVudCwgaXQgd2lsbCBiZSB1c2VmdWwgdG8gaGF2ZSAxLzAgdmVyc2lvbnMgb2YgdGhlIHZhcmlhYmxlcywgYW5kIGl0IHdpbGwgYWxzbyBiZSBoZWxwZnVsIHRvIGhhdmUgdGhlIGZhY3RvciB2ZXJzaW9ucyByZWxldmVsZWQgdG8gcHV0IHRoZSBvdXRjb21lIG9mIGludGVyZXN0IChkZWF0aCkgZmlyc3QgYW5kIHRoZSB0cmVhdG1lbnQgb2YgaW50ZXJlc3QgKFJIQykgZmlyc3QuDQoNCmBgYHtyfQ0KcmhjIDwtIHJoY19jbGVhbmluZyAlPiUNCiAgICBtdXRhdGUodHJlYXRfcmhjID0gYXMubnVtZXJpYyhzd2FuZzEgPT0gIlJIQyIpLA0KICAgICAgICAgICBzd2FuZzEgPSBmY3RfcmVsZXZlbChzd2FuZzEsICJSSEMiKSwNCiAgICAgICAgICAgZGVhdGggPSBmY3RfcmVsZXZlbChkZWF0aCwgIlllcyIpLA0KICAgICAgICAgICBkaWVkID0gYXMubnVtZXJpYyhkZWF0aCA9PSAiWWVzIikpDQpgYGANCg0KIyMgU2F2aW5nIHRoZSBEYXRhIEZpbGUNCg0KYGBge3J9DQpzYXZlUkRTKHJoYywgaGVyZSgiZGF0YSIsICJyaGMuUmRzIikpDQpgYGANCg0KIyBTZXR0aW5nIGEgU2VlZA0KDQpJJ20gZ29pbmcgdG8gc2V0IGEgc2VlZCBmb3IgcmFuZG9tIG51bWJlcnMgYmVmb3JlIHdlIHN0YXJ0LCB3aGljaCBJIGNhbiBjaGFuZ2UgbGF0ZXIgaWYgSSBsaWtlLg0KDQpgYGB7cn0NCnNldC5zZWVkKDUwMDEyMzQ1KQ0KYGBgDQoNCiMgVW5hZGp1c3RlZCBPdXRjb21lIEFzc2Vzc21lbnRzDQoNCkJlZm9yZSB3ZSBlc3RpbWF0ZSBhIHByb3BlbnNpdHkgc2NvcmUsIHdlJ2xsIHBlcmZvcm0gdW5hZGp1c3RlZCBhc3Nlc3NtZW50cyB0byBkZXNjcmliZSB0aGUgaW1wYWN0IG9mIFJIQyAob3Igbm90KSBvbiBvdXIgdGhyZWUgb3V0Y29tZXMsIHdpdGhvdXQgYWRqdXN0bWVudCBmb3IgYW55IGNvdmFyaWF0ZXMgYXQgYWxsLg0KDQojIyBPdXRjb21lIEE6IEluLVN0dWR5IE1vcnRhbGl0eSAoYmluYXJ5KQ0KDQpgYGB7cn0NCnJoYyAlPiUgdGFieWwoc3dhbmcxLCBkZWF0aCkgJT4lIA0KICAgIGFkb3JuX3RvdGFscygpICU+JQ0KICAgIGFkb3JuX3BlcmNlbnRhZ2VzKCkgJT4lDQogICAgYWRvcm5fcGN0X2Zvcm1hdHRpbmcoKSAlPiUNCiAgICBhZG9ybl9ucyhwb3NpdGlvbiA9ICJmcm9udCIpICU+JQ0KICAgIGFkb3JuX3RpdGxlDQpgYGANCg0KVGhlIG9kZHMgcmF0aW8gb2YgZGVhdGggYXNzb2NpYXRlZCB3aXRoIFJIQyBhcyBvcHBvc2VkIHRvIE5vIFJIQyBzaG91bGQgYmUgbGFyZ2VyIHRoYW4gMSwgc3BlY2lmaWNhbGx5LCBpdCBzaG91bGQgYmUgDQoNClxbDQpcZnJhY3sxNDg2IFx0aW1lcyAxMzE1fXsyMjM2IFx0aW1lcyA2OTh9ID0gMS4yNQ0KXF0NCg0KTGV0J3Mgc2VlIGlmIHRoYXQncyB3aGF0IHdlIGdldC4NCg0KIyMjIEZpdHRpbmcgdGhlIE1vZGVsIHdpdGggdGhlIDEvMCB0cmVhdG1lbnQgYW5kIG91dGNvbWUNCg0KYGBge3J9DQpkZWF0aF91bmFkaiA8LSBnbG0oZGllZCB+IHRyZWF0X3JoYywgZGF0YSA9IHJoYywgZmFtaWx5ID0gYmlub21pYWwoKSkNCg0KZGVhdGhfdW5hZGoNCg0KdGlkeShkZWF0aF91bmFkaiwgY29uZi5pbnQgPSBUUlVFLCBleHBvbmVudGlhdGUgPSBUUlVFKQ0KYGBgDQoNCiMjIyBGaXR0aW5nIHRoZSBNb2RlbCB3aXRoIHRoZSB0cmVhdG1lbnQgYW5kIG91dGNvbWUgYXMgZmFjdG9ycyByZXF1aXJlcyBjYXJlDQoNCkFsdGVybmF0aXZlbHksIHdlIGNvdWxkIGp1c3Qgd29yayB3aXRoIHRoZSBmYWN0b3JzLCBidXQgdGhlbiwgaXQncyBpbXBvcnRhbnQgdG8gYmUgc3VyZSB3aGF0IGlzIGFjdHVhbGx5IGJlaW5nIG1vZGVsZWQuIERvIHRoaXMgYnkgbWFraW5nIHRoZSBvdXRjb21lIGFuZCB0cmVhdG1lbnQgbG9naWNhbCByZXN1bHRzLCBhcyBmb2xsb3dzLg0KDQpgYGB7cn0NCm1vZGVsQV91bmFkajEgPC0gZ2xtKChkZWF0aCA9PSAiWWVzIikgfiAoc3dhbmcxID09ICJSSEMiKSwgZGF0YSA9IHJoYywgZmFtaWx5ID0gYmlub21pYWwoKSkNCg0KdGlkeShtb2RlbEFfdW5hZGoxLCBjb25mLmludCA9IFRSVUUsIGV4cG9uZW50aWF0ZSA9IFRSVUUpDQpgYGANCg0KRmFpbGluZyB0byBkbyB0aGlzIGNhbiBjYXVzZSB0cm91YmxlOg0KDQpgYGB7cn0NCm1vZGVsQV91bmFkal9iYWQgPC0gZ2xtKGRlYXRoIH4gc3dhbmcxLCBkYXRhID0gcmhjLCBmYW1pbHkgPSBiaW5vbWlhbCgpKQ0KDQp0aWR5KG1vZGVsQV91bmFkal9iYWQsIGNvbmYuaW50ID0gVFJVRSwgZXhwb25lbnRpYXRlID0gVFJVRSkNCmBgYA0KDQpOb3RlIHRoYXQgdGhpcyBsYXN0IHZlcnNpb24gaXMgYWN0dWFsbHkgcHJlZGljdGluZyAiRGVhdGggPSBObyIgb24gdGhlIGJhc2lzIG9mICJzd2FuZzEgPSBObyBSSEMiLCB3aGljaCBjYW4gYmUgZXh0cmVtZWx5IGNvbmZ1c2luZy4gDQoNCkFuZCBpZiB5b3UgZGlkIHRoaXM6DQoNCmBgYHtyfQ0KbW9kZWxBX3VuYWRqX2JhZDIgPC0gZ2xtKGRlYXRoIH4gc3dhbmcxID09ICJSSEMiLCBkYXRhID0gcmhjLCBmYW1pbHkgPSBiaW5vbWlhbCgpKQ0KDQp0aWR5KG1vZGVsQV91bmFkal9iYWQyLCBjb25mLmludCA9IFRSVUUsIGV4cG9uZW50aWF0ZSA9IFRSVUUpDQpgYGANCg0KTm93LCB5b3UndmUgaW52ZXJ0ZWQgdGhlIG9kZHMgcmF0aW8hIE5vdCBnb29kLiBTbywgZWl0aGVyIHVzZSAwIGFuZCAxICh3aXRoIDEgYmVpbmcgdGhlIHJlc3VsdCB5b3Ugd2FudCB0byBzdHVkeSksIG9yIHVzZSB0aGUgZmFjdG9ycyBidXQgc3BlY2lmeSB0aGUgZGlyZWN0aW9uIGZvciBib3RoIHRyZWF0bWVudCBhbmQgb3V0Y29tZSBhcyBhIGxvZ2ljYWwgc3RhdGVtZW50Lg0KDQojIyBPdXRjb21lIEI6IExlbmd0aCBvZiBTdGF5IChxdWFudGl0YXRpdmUpDQoNCmBgYHtyfQ0KaG9zcGRheXNfdW5hZGogPC0gbG0oaG9zcGRheXMgfiB0cmVhdF9yaGMsIGRhdGEgPSByaGMpDQoNCmhvc3BkYXlzX3VuYWRqDQoNCnRpZHkoaG9zcGRheXNfdW5hZGosIGNvbmYuaW50ID0gVFJVRSkNCmBgYA0KDQoNCiMjIE91dGNvbWUgQzogVGltZSB0byBkZWF0aCAodGltZS10by1ldmVudCkNCg0KSGVyZSdzIHRoZSBkYXRhIG9uIGBzdXJ2ZGF5c2AgYW5kIGBkZWF0aGAgZm9yIG91ciBmaXJzdCBzaXggcGF0aWVudHM6DQoNCmBgYHtyfQ0KcmhjICU+JSBzZWxlY3Qoc3VydmRheXMsIGRlYXRoKSAlPiUgaGVhZCgpDQpgYGANCg0KSGVyZSwgd2UgaGF2ZSBhIChyaWdodC1jZW5zb3JlZCkgdGltZSB0byBldmVudCBvdXRjb21lLCBjYWxsZWQgYHN1cnZkYXlzYCwgYW5kIGFuIGluZGljYXRvciBvZiBjZW5zb3JpbmcgKGBkZWF0aGAgd2hpY2ggaXMgIlllcyIgaWYgdGhlIHBhdGllbnQncyB0aW1lIGlzIHJlYWwgYW5kIG5vdCBjZW5zb3JlZCkuIFNvIHRoZSBmaXJzdCBzdWJqZWN0IHNob3VsZCBoYXZlIGEgdGltZSB0byBkZWF0aCBvZiAyNDAgb3IgbW9yZSBkYXlzLCBiZWNhdXNlIHRoZXkgd2VyZSBzdGlsbCBhbGl2ZSB3aGVuIHRoZSBzdHVkeSBlbmRlZCwgd2hpbGUgdGhlIHNlY29uZCBzaG91bGQgaGF2ZSBhIHRpbWUgdGhhdCBpcyBleGFjdGx5IDQ1IGRheXMsIGJlY2F1c2UgdGhhdCBwZXJzb24gZGllZCBkdXJpbmcgdGhlIHN0dWR5LiBUaGUgc3Vydml2YWwgb2JqZWN0IHdlIG5lZWQsIHRoZW4sIGlzOg0KDQpgYGB7cn0NClN1cnYocmhjJHN1cnZkYXlzLCByaGMkZGVhdGggPT0gIlllcyIpICU+JSBoZWFkKCkNCmBgYA0KDQoNCmBgYHtyfQ0Kc3VydmRheXNfdW5hZGogPC0gY294cGgoU3VydihzdXJ2ZGF5cywgZGVhdGggPT0gIlllcyIpIH4gdHJlYXRfcmhjLCBkYXRhID0gcmhjKQ0KDQpzdXJ2ZGF5c191bmFkag0KDQp0aWR5KHN1cnZkYXlzX3VuYWRqLCBjb25mLmludCA9IFRSVUUsIGV4cG9uZW50aWF0ZSA9IFRSVUUpDQpgYGANCg0KIyBFc3RpbWF0ZSB0aGUgUHJvcGVuc2l0eSBTY29yZSB3aXRoIExvZ2lzdGljIFJlZ3Jlc3Npb24NCg0KYGBge3J9DQpwcm9wZW5zaXR5X21vZGVsIDwtIA0KICAgIGdsbShzd2FuZzEgPT0gIlJIQyIgIH4gDQogICAgICAgICAgICBhZ2UgKyBzZXggKyBlZHUgKyBpbmNvbWUgKyBuaW5zY2xhcyArIHJhY2UgKw0KICAgICAgICAgICAgY2F0MSArIGRucjEgKyB3dGtpbG8xICsgaHJ0MSArIG1lYW5icDEgKyANCiAgICAgICAgICAgIHJlc3AxICsgdGVtcDEgKw0KICAgICAgICAgICAgY2FyZCArIGdhc3RyICsgaGVtYSArIG1ldGEgKyBuZXVybyArIG9ydGhvICsgDQogICAgICAgICAgICByZW5hbCArIHJlc3AgKyBzZXBzICsgdHJhdW1hICsNCiAgICAgICAgICAgIGFtaWh4ICsgY2EgKyBjYXJkaW9oeCArIGNoZmh4ICsgY2hycHVsaHggKw0KICAgICAgICAgICAgZGVtZW50aHggKyBnaWJsZWRoeCArIGltbXVuaHggKyBsaXZlcmh4ICsNCiAgICAgICAgICAgIG1hbGlnaHggKyBwc3ljaGh4ICsgcmVuYWxoeCArIHRyYW5zaHggKw0KICAgICAgICAgICAgYXBzMSArIGRhczJkM3BjICsgc2NvbWExICsgc3VydjJtZDEgKw0KICAgICAgICAgICAgYWxiMSArIGJpbGkxICsgY3JlYTEgKyBoZW1hMSArIHBhY28yMSArIHBhZmkxICsNCiAgICAgICAgICAgIHBoMSArIHBvdDEgKyBzb2QxICsgd2JsYzEsDQogICAgICAgIGZhbWlseSA9IGJpbm9taWFsKGxpbmsgPSAibG9naXQiKSwgZGF0YSA9IHJoYykNCmBgYA0KDQojIyBTdG9yZSB0aGUgcHJvcGVuc2l0eSBzY29yZXMgYW5kIGxpbmVhciBwcm9wZW5zaXR5IHNjb3Jlcw0KDQpgYGB7cn0NCnJoYyA8LSByaGMgJT4lDQogICAgbXV0YXRlKHBzID0gZml0dGVkKHByb3BlbnNpdHlfbW9kZWwpLA0KICAgICAgICAgICBsaW5wcyA9IHByb3BlbnNpdHlfbW9kZWwkbGluZWFyLnByZWRpY3RvcnMpDQoNCnJoYyAlPiUgc2VsZWN0KHB0aWQsIHBzLCBsaW5wcykgJT4lIGhlYWQoKQ0KYGBgDQoNCiMjIFBsb3QgdGhlIFByb3BlbnNpdHkgU2NvcmVzIHRvIFZlcmlmeSB0aGF0IHRoZXkgTWF0Y2ggWW91ciBFeHBlY3RhdGlvbnMNCg0KYGBge3J9DQpnZ3Bsb3QocmhjLCBhZXMoeCA9IHBzLCBmaWxsID0gc3dhbmcxKSkgKw0KICAgIGdlb21fZGVuc2l0eShhbHBoYSA9IDAuNSkgKw0KICAgIHNjYWxlX2ZpbGxfdmlyaWRpc19kKG9wdGlvbiA9ICJwbGFzbWEiKSArDQogICAgdGhlbWVfYncoKQ0KYGBgDQoNCmBgYHtyfQ0KZ2dwbG90KHJoYywgYWVzKHggPSBzd2FuZzEsIHkgPSBwcykpICsNCiAgICBnZW9tX3Zpb2xpbihhZXMoZmlsbCA9IHN3YW5nMSkpICsgDQogICAgZ2VvbV9ib3hwbG90KHdpZHRoID0gMC4yKSArDQogICAgc2NhbGVfZmlsbF92aXJpZGlzX2Qob3B0aW9uID0gInBsYXNtYSIpICsgDQogICAgZ3VpZGVzKGZpbGwgPSAibm9uZSIpICsgDQogICAgY29vcmRfZmxpcCgpICsNCiAgICB0aGVtZV9idygpDQpgYGANCg0KT0suIFdlJ3JlIHRyeWluZyB0byB1c2UgYHBzYCB0byBlc3RpbWF0ZSB0aGUgcHJvYmFiaWxpdHkgb2YgaGF2aW5nIGEgUkhDLCBhbmQgaXQgbG9va3MgdGhlIHBlb3BsZSB3aG8gYWN0dWFsbHkgaGFkIFJIQyBoYXZlIGhpZ2hlciBQUywgb24gYXZlcmFnZSwgc28gdGhhdCdzIHByb21pc2luZy4NCg0KIyBDaGVja2luZyBSdWJpbidzIFJ1bGVzIFByaW9yIHRvIFByb3BlbnNpdHkgQWRqdXN0bWVudA0KDQpSdWJpbiBSdWxlIDEgcmVzdWx0IHNob3VsZCBpZGVhbGx5IGJlIG5lYXIgMCwgYW5kIGNlcnRhaW5seSBiZXR3ZWVuIC01MCBhbmQgKzUwLg0KDQpgYGB7cn0NCnJ1YmluMS51bmFkaiA8LSB3aXRoKHJoYywgDQogICAgICAgICAgICAgICAgICBhYnMoMTAwKihtZWFuKGxpbnBzW3N3YW5nMT09IlJIQyJdKSAtIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1lYW4obGlucHNbc3dhbmcxPT0iTm8gUkhDIl0pKS8NCiAgICAgICAgICAgICAgICAgICAgICAgICAgc2QobGlucHMpKSkNCnJ1YmluMS51bmFkag0KYGBgDQoNClJ1YmluIFJ1bGUgMiByZXN1bHQgc2hvdWxkIGlkZWFsbHkgYmUgbmVhciAxLCBhbmQgY2VydGFpbmx5IGJldHdlZW4gMS8yIGFuZCAyLg0KDQpgYGB7cn0NCnJ1YmluMi51bmFkaiA8LSB3aXRoKHJoYywgDQogICAgICAgICAgICAgICAgICB2YXIobGlucHNbc3dhbmcxID09ICJSSEMiXSkgLyANCiAgICAgICAgICAgICAgICAgICAgICB2YXIobGlucHNbc3dhbmcxID09ICJObyBSSEMiXSkpDQpydWJpbjIudW5hZGoNCmBgYA0KDQpTbywgd2UndmUgZ290IHNvbWUgd29yayB0byBkby4gTGV0J3MgdHJ5IG1hdGNoaW5nLiBJbiBzZXZlcmFsIGRpZmZlcmVudCB3YXlzLg0KDQojIE1hdGNoIDE6IDE6MSBHcmVlZHkgTWF0Y2hpbmcgb24gdGhlIGxpbmVhciBQUywgd2l0aG91dCBSZXBsYWNlbWVudA0KDQpgYGB7cn0NClggPC0gcmhjJGxpbnBzDQpUciA8LSBhcy5sb2dpY2FsKHJoYyRzd2FuZzEgPT0gIlJIQyIpDQptYXRjaDEgPC0gTWF0Y2goVHIgPSBUciwgWCA9IFgsIE0gPSAxLCANCiAgICAgICAgICAgICAgICBlc3RpbWFuZCA9ICJBVFQiLCByZXBsYWNlID0gRkFMU0UsIA0KICAgICAgICAgICAgICAgIHRpZXMgPSBGQUxTRSkNCnN1bW1hcnkobWF0Y2gxKQ0KYGBgDQoNCg0KDQojIyBMb3ZlIFBsb3QgZm9yIE1hdGNoIDENCg0KYGBge3IsIGZpZy5oZWlnaHQgPSA4fQ0KYjEgPC0gYmFsLnRhYihtYXRjaDEsIA0KICAgICAgICAgICAgICBzd2FuZzEgPT0gIlJIQyIgIH4gDQogICAgICAgICAgICAgICAgICBhZ2UgKyBzZXggKyBlZHUgKyBpbmNvbWUgKyBuaW5zY2xhcyArIHJhY2UgKw0KICAgICAgICAgICAgICAgICAgY2F0MSArIGRucjEgKyB3dGtpbG8xICsgaHJ0MSArIG1lYW5icDEgKyANCiAgICAgICAgICAgICAgICAgIHJlc3AxICsgdGVtcDEgKw0KICAgICAgICAgICAgICAgICAgY2FyZCArIGdhc3RyICsgaGVtYSArIG1ldGEgKyBuZXVybyArIG9ydGhvICsgDQogICAgICAgICAgICAgICAgICByZW5hbCArIHJlc3AgKyBzZXBzICsgdHJhdW1hICsNCiAgICAgICAgICAgICAgICAgIGFtaWh4ICsgY2EgKyBjYXJkaW9oeCArIGNoZmh4ICsgY2hycHVsaHggKw0KICAgICAgICAgICAgICAgICAgZGVtZW50aHggKyBnaWJsZWRoeCArIGltbXVuaHggKyBsaXZlcmh4ICsNCiAgICAgICAgICAgICAgICAgIG1hbGlnaHggKyBwc3ljaGh4ICsgcmVuYWxoeCArIHRyYW5zaHggKw0KICAgICAgICAgICAgICAgICAgYXBzMSArIGRhczJkM3BjICsgc2NvbWExICsgc3VydjJtZDEgKw0KICAgICAgICAgICAgICAgICAgYWxiMSArIGJpbGkxICsgY3JlYTEgKyBoZW1hMSArIHBhY28yMSArIHBhZmkxICsNCiAgICAgICAgICAgICAgICAgIHBoMSArIHBvdDEgKyBzb2QxICsgd2JsYzEgKyBwcyArIGxpbnBzLA0KICAgICAgICAgICAgICBkYXRhPXJoYywgdW4gPSBUUlVFKQ0KDQpsb3ZlLnBsb3QoYjEsIHRocmVzaG9sZCA9IC4xLCBzaXplID0gMS41LCBzdGFycyA9ICJyYXciLA0KICAgICAgICAgICAgICAgdmFyLm9yZGVyID0gInVuYWRqdXN0ZWQiLA0KICAgICAgICAgICAgICAgdGl0bGUgPSAiU3RhbmRhcmRpemVkIERpZmZlcmVuY2VzIGFuZCBNYXRjaCAxIikgKw0KICAgIHRoZW1lX2J3KCkNCmBgYA0KDQojIyBSdWJpbidzIFJ1bGVzIDEgYW5kIDIgZm9yIE1hdGNoIDENCg0KQ3JlYXRlIGEgbmV3IGRhdGEgZnJhbWUsIGNvbnRhaW5pbmcgb25seSB0aGUgbWF0Y2hlZCBzYW1wbGUuDQoNCmBgYHtyfQ0KbWF0Y2hlc18xIDwtIGZhY3RvcihyZXAobWF0Y2gxJGluZGV4LnRyZWF0ZWQsIDIpKQ0KcmhjLm1hdGNoZXNfMSA8LSBjYmluZChtYXRjaGVzXzEsIA0KICAgICAgICAgIHJoY1tjKG1hdGNoMSRpbmRleC5jb250cm9sLCBtYXRjaDEkaW5kZXgudHJlYXRlZCksXSkNCmBgYA0KDQpSdWJpbiBSdWxlIDEgcmVzdWx0IGFmdGVyIG1hdGNoaW5nIHNob3VsZCBpZGVhbGx5IGJlIG5lYXIgMCwgYW5kIGNlcnRhaW5seSBiZXR3ZWVuIC01MCBhbmQgKzUwLg0KDQpgYGB7cn0NCnJ1YmluMS5tMSA8LSB3aXRoKHJoYy5tYXRjaGVzXzEsIA0KICAgICAgICAgICAgICAgICAgYWJzKDEwMCoobWVhbihsaW5wc1tzd2FuZzE9PSJSSEMiXSkgLSANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtZWFuKGxpbnBzW3N3YW5nMT09Ik5vIFJIQyJdKSkvDQogICAgICAgICAgICAgICAgICAgICAgICAgIHNkKGxpbnBzKSkpDQpydWJpbjEubTENCmBgYA0KDQpSdWJpbiBSdWxlIDIgcmVzdWx0IGFmdGVyIG1hdGNoaW5nIHNob3VsZCBpZGVhbGx5IGJlIG5lYXIgMSwgYW5kIGNlcnRhaW5seSBiZXR3ZWVuIDEvMiBhbmQgMi4NCg0KYGBge3J9DQpydWJpbjIubTEgPC0gd2l0aChyaGMubWF0Y2hlc18xLCANCiAgICAgICAgICAgICAgICAgIHZhcihsaW5wc1tzd2FuZzEgPT0gIlJIQyJdKSAvIA0KICAgICAgICAgICAgICAgICAgICAgIHZhcihsaW5wc1tzd2FuZzEgPT0gIk5vIFJIQyJdKSkNCnJ1YmluMi5tMQ0KYGBgDQoNCiMgTWF0Y2ggMjogMToyIEdyZWVkeSBNYXRjaGluZyBvbiB0aGUgbGluZWFyIFBTLCB3aXRob3V0IFJlcGxhY2VtZW50DQoNCmBgYHtyfQ0KWCA8LSByaGMkbGlucHMNClRyIDwtIGFzLmxvZ2ljYWwocmhjJHN3YW5nMSA9PSAiUkhDIikNCm1hdGNoMiA8LSBNYXRjaChUciA9IFRyLCBYID0gWCwgTSA9IDIsIA0KICAgICAgICAgICAgICAgIGVzdGltYW5kID0gIkFUVCIsIHJlcGxhY2UgPSBGQUxTRSwgDQogICAgICAgICAgICAgICAgdGllcyA9IEZBTFNFKQ0Kc3VtbWFyeShtYXRjaDIpDQpgYGANCg0KIyMgTG92ZSBQbG90IGZvciBNYXRjaCAyDQoNCmBgYHtyLCBmaWcuaGVpZ2h0ID0gOH0NCmIyIDwtIGJhbC50YWIobWF0Y2gyLCANCiAgICAgICAgICAgICAgc3dhbmcxID09ICJSSEMiICB+IA0KICAgICAgICAgICAgICAgICAgYWdlICsgc2V4ICsgZWR1ICsgaW5jb21lICsgbmluc2NsYXMgKyByYWNlICsNCiAgICAgICAgICAgICAgICAgIGNhdDEgKyBkbnIxICsgd3RraWxvMSArIGhydDEgKyBtZWFuYnAxICsgDQogICAgICAgICAgICAgICAgICByZXNwMSArIHRlbXAxICsNCiAgICAgICAgICAgICAgICAgIGNhcmQgKyBnYXN0ciArIGhlbWEgKyBtZXRhICsgbmV1cm8gKyBvcnRobyArIA0KICAgICAgICAgICAgICAgICAgcmVuYWwgKyByZXNwICsgc2VwcyArIHRyYXVtYSArDQogICAgICAgICAgICAgICAgICBhbWloeCArIGNhICsgY2FyZGlvaHggKyBjaGZoeCArIGNocnB1bGh4ICsNCiAgICAgICAgICAgICAgICAgIGRlbWVudGh4ICsgZ2libGVkaHggKyBpbW11bmh4ICsgbGl2ZXJoeCArDQogICAgICAgICAgICAgICAgICBtYWxpZ2h4ICsgcHN5Y2hoeCArIHJlbmFsaHggKyB0cmFuc2h4ICsNCiAgICAgICAgICAgICAgICAgIGFwczEgKyBkYXMyZDNwYyArIHNjb21hMSArIHN1cnYybWQxICsNCiAgICAgICAgICAgICAgICAgIGFsYjEgKyBiaWxpMSArIGNyZWExICsgaGVtYTEgKyBwYWNvMjEgKyBwYWZpMSArDQogICAgICAgICAgICAgICAgICBwaDEgKyBwb3QxICsgc29kMSArIHdibGMxICsgcHMgKyBsaW5wcywNCiAgICAgICAgICAgICAgZGF0YT1yaGMsIHVuID0gVFJVRSkNCg0KbG92ZS5wbG90KGIyLCB0aHJlc2hvbGQgPSAuMSwgc2l6ZSA9IDEuNSwgc3RhcnMgPSAicmF3IiwNCiAgICAgICAgICAgICAgIHZhci5vcmRlciA9ICJ1bmFkanVzdGVkIiwNCiAgICAgICAgICAgICAgIHRpdGxlID0gIlN0YW5kYXJkaXplZCBEaWZmZXJlbmNlcyBhbmQgTWF0Y2ggMiIpICsNCiAgICB0aGVtZV9idygpDQpgYGANCg0KIyMgUnViaW4ncyBSdWxlcyAxIGFuZCAyIGZvciBNYXRjaCAyDQoNCkNyZWF0ZSBhIG5ldyBkYXRhIGZyYW1lLCBjb250YWluaW5nIG9ubHkgdGhlIG1hdGNoZWQgc2FtcGxlLg0KDQpgYGB7cn0NCm1hdGNoZXNfMiA8LSBmYWN0b3IocmVwKG1hdGNoMiRpbmRleC50cmVhdGVkLCAyKSkNCnJoYy5tYXRjaGVzXzIgPC0gY2JpbmQobWF0Y2hlc18yLCANCiAgICAgICAgICByaGNbYyhtYXRjaDIkaW5kZXguY29udHJvbCwgbWF0Y2gyJGluZGV4LnRyZWF0ZWQpLF0pDQpgYGANCg0KUnViaW4gUnVsZSAxIHJlc3VsdCBhZnRlciBtYXRjaGluZyBzaG91bGQgaWRlYWxseSBiZSBuZWFyIDAsIGFuZCBjZXJ0YWlubHkgYmV0d2VlbiAtNTAgYW5kICs1MC4NCg0KYGBge3J9DQpydWJpbjEubTIgPC0gd2l0aChyaGMubWF0Y2hlc18yLCANCiAgICAgICAgICAgICAgICAgIGFicygxMDAqKG1lYW4obGlucHNbc3dhbmcxPT0iUkhDIl0pIC0gDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbWVhbihsaW5wc1tzd2FuZzE9PSJObyBSSEMiXSkpLw0KICAgICAgICAgICAgICAgICAgICAgICAgICBzZChsaW5wcykpKQ0KcnViaW4xLm0yDQpgYGANCg0KUnViaW4gUnVsZSAyIHJlc3VsdCBhZnRlciBtYXRjaGluZyBzaG91bGQgaWRlYWxseSBiZSBuZWFyIDEsIGFuZCBjZXJ0YWlubHkgYmV0d2VlbiAxLzIgYW5kIDIuDQoNCmBgYHtyfQ0KcnViaW4yLm0yIDwtIHdpdGgocmhjLm1hdGNoZXNfMiwgDQogICAgICAgICAgICAgICAgICB2YXIobGlucHNbc3dhbmcxID09ICJSSEMiXSkgLyANCiAgICAgICAgICAgICAgICAgICAgICB2YXIobGlucHNbc3dhbmcxID09ICJObyBSSEMiXSkpDQpydWJpbjIubTINCmBgYA0KDQoNCiMgTWF0Y2ggMzogR2VuZXRpYyBTZWFyY2ggTWF0Y2hpbmcgdG8gZG8gMToxIE1hdGNoaW5nIHdpdGhvdXQgcmVwbGFjZW1lbnQNCg0KVGhlIGBHZW5NYXRjaGAgZnVuY3Rpb24gZmluZHMgb3B0aW1hbCBiYWxhbmNlIHVzaW5nIG11bHRpdmFyaWF0ZSBtYXRjaGluZyB3aGVyZSBhIGdlbmV0aWMgc2VhcmNoIGFsZ29yaXRobSBkZXRlcm1pbmVzIHRoZSB3ZWlnaHQgdGhhdCBlYWNoIGNvdmFyaWF0ZSBpcyBnaXZlbi4NCg0KUGxlYXNlIG5vdGUgdGhhdCBJIGFtIHVzaW5nIHdheSB0b28gc21hbGwgdmFsdWVzIG9mIGBwb3Auc2l6ZWAgKGVzcGVjaWFsbHkpIGFuZCBwcm9iYWJseSBgbWF4LmdlbmVyYXRpb25zYCwgdG9vLCBzbyB0aGF0IHRoaW5ncyBydW4gcXVpY2tseS4gSSBhbSBhbHNvIG9ubHkgbWF0Y2hpbmcgb24gdGhlIHByb3BlbnNpdHkgc2NvcmUgYW5kIGxpbmVhciBwcm9wZW5zaXR5IHNjb3JlLCByYXRoZXIgdGhhbiBvbiB0aGUgaW5kaXZpZHVhbCBjb3ZhcmlhdGVzLiBGb3IgYSBjb3Vyc2UgcHJvamVjdCwgSSByZWNvbW1lbmQgeW91IHRyeSBtYXRjaGluZyBvbiB0aGUgaW5kaXZpZHVhbCBjb3ZhcmlhdGVzIGlmIHlvdSBjYW4sIGJ1dCB0aGlzIHJlZHVjdGlvbiBpbiB0aGUgcGFyYW1hdGVyIHZhbHVlcyBpcyBmaW5lLiANCg0KRm9yIGFjdHVhbCBwdWJsaWNhdGlvbnMsIGZvbGxvdyB0aGUgcmVjb21tZW5kYXRpb25zIG9mIHRoZSBgR2VuTWF0Y2hgIGFsZ29yaXRobSwgYXMgZGVzY3JpYmVkIGluIHRoZSBgTWF0Y2hpbmdgIGFuZCBgZ2Vub3VkYCBwYWNrYWdlIGhlbHAgZmlsZXMuIFRoZSBkZWZhdWx0IHZhbHVlcyAod2hpY2ggbWF5IGJlIHRvbyBzbWFsbCB0aGVtc2VsdmVzKSBhcmUgMTAwIGZvciBib3RoIGBwb3Auc2l6ZWAgYW5kIGBtYXguZ2VuZXJhdGlvbnNgLg0KDQpgYGB7cn0NClggPC0gY2JpbmQocmhjJGxpbnBzLCByaGMkcHMpDQpUciA8LSBhcy5sb2dpY2FsKHJoYyRzd2FuZzEgPT0gIlJIQyIpDQpnZW5vdXQzIDwtIEdlbk1hdGNoKFRyID0gVHIsIFggPSBYLA0KICAgICAgICAgICAgICAgICAgICBlc3RpbWFuZCA9ICJBVFQiLCBNID0gMSwNCiAgICAgICAgICAgICAgICAgICAgcG9wLnNpemUgPSAxMCwgbWF4LmdlbmVyYXRpb25zID0gMTAsDQogICAgICAgICAgICAgICAgICAgIHdhaXQuZ2VuZXJhdGlvbnMgPSA0LCB2ZXJib3NlID0gRkFMU0UpDQptYXRjaDMgPC0gTWF0Y2goVHIgPSBUciwgWCA9IFgsIGVzdGltYW5kID0gIkFUVCIsIA0KICAgICAgICAgICAgICAgICBXZWlnaHQubWF0cml4ID0gZ2Vub3V0MykNCnN1bW1hcnkobWF0Y2gzKQ0KYGBgDQoNCiMjIExvdmUgUGxvdCBmb3IgTWF0Y2ggMw0KDQpgYGB7ciwgZmlnLmhlaWdodCA9IDh9DQpiMyA8LSBiYWwudGFiKG1hdGNoMywgDQogICAgICAgICAgICAgIHN3YW5nMSA9PSAiUkhDIiAgfiANCiAgICAgICAgICAgICAgICAgIGFnZSArIHNleCArIGVkdSArIGluY29tZSArIG5pbnNjbGFzICsgcmFjZSArDQogICAgICAgICAgICAgICAgICBjYXQxICsgZG5yMSArIHd0a2lsbzEgKyBocnQxICsgbWVhbmJwMSArIA0KICAgICAgICAgICAgICAgICAgcmVzcDEgKyB0ZW1wMSArDQogICAgICAgICAgICAgICAgICBjYXJkICsgZ2FzdHIgKyBoZW1hICsgbWV0YSArIG5ldXJvICsgb3J0aG8gKyANCiAgICAgICAgICAgICAgICAgIHJlbmFsICsgcmVzcCArIHNlcHMgKyB0cmF1bWEgKw0KICAgICAgICAgICAgICAgICAgYW1paHggKyBjYSArIGNhcmRpb2h4ICsgY2hmaHggKyBjaHJwdWxoeCArDQogICAgICAgICAgICAgICAgICBkZW1lbnRoeCArIGdpYmxlZGh4ICsgaW1tdW5oeCArIGxpdmVyaHggKw0KICAgICAgICAgICAgICAgICAgbWFsaWdoeCArIHBzeWNoaHggKyByZW5hbGh4ICsgdHJhbnNoeCArDQogICAgICAgICAgICAgICAgICBhcHMxICsgZGFzMmQzcGMgKyBzY29tYTEgKyBzdXJ2Mm1kMSArDQogICAgICAgICAgICAgICAgICBhbGIxICsgYmlsaTEgKyBjcmVhMSArIGhlbWExICsgcGFjbzIxICsgcGFmaTEgKw0KICAgICAgICAgICAgICAgICAgcGgxICsgcG90MSArIHNvZDEgKyB3YmxjMSArIHBzICsgbGlucHMsDQogICAgICAgICAgICAgIGRhdGE9cmhjLCB1biA9IFRSVUUpDQoNCmxvdmUucGxvdChiMywgdGhyZXNob2xkID0gLjEsIHNpemUgPSAxLjUsIHN0YXJzID0gInJhdyIsDQogICAgICAgICAgICAgICB2YXIub3JkZXIgPSAidW5hZGp1c3RlZCIsDQogICAgICAgICAgICAgICB0aXRsZSA9ICJTdGFuZGFyZGl6ZWQgRGlmZmVyZW5jZXMgYW5kIE1hdGNoIDMiKSArDQogICAgdGhlbWVfYncoKQ0KYGBgDQoNCiMjIFJ1YmluJ3MgUnVsZXMgMSBhbmQgMiBmb3IgTWF0Y2ggMw0KDQpDcmVhdGUgYSBuZXcgZGF0YSBmcmFtZSwgY29udGFpbmluZyBvbmx5IHRoZSBtYXRjaGVkIHNhbXBsZS4NCg0KYGBge3J9DQptYXRjaGVzXzMgPC0gZmFjdG9yKHJlcChtYXRjaDMkaW5kZXgudHJlYXRlZCwgMikpDQpyaGMubWF0Y2hlc18zIDwtIGNiaW5kKG1hdGNoZXNfMywgDQogICAgICAgICAgcmhjW2MobWF0Y2gzJGluZGV4LmNvbnRyb2wsIG1hdGNoMyRpbmRleC50cmVhdGVkKSxdKQ0KYGBgDQoNClJ1YmluIFJ1bGUgMSByZXN1bHQgYWZ0ZXIgbWF0Y2hpbmcgc2hvdWxkIGlkZWFsbHkgYmUgbmVhciAwLCBhbmQgY2VydGFpbmx5IGJldHdlZW4gLTUwIGFuZCArNTAuDQoNCmBgYHtyfQ0KcnViaW4xLm0zIDwtIHdpdGgocmhjLm1hdGNoZXNfMywgDQogICAgICAgICAgICAgICAgICBhYnMoMTAwKihtZWFuKGxpbnBzW3N3YW5nMT09IlJIQyJdKSAtIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1lYW4obGlucHNbc3dhbmcxPT0iTm8gUkhDIl0pKS8NCiAgICAgICAgICAgICAgICAgICAgICAgICAgc2QobGlucHMpKSkNCnJ1YmluMS5tMw0KYGBgDQoNClJ1YmluIFJ1bGUgMiByZXN1bHQgYWZ0ZXIgbWF0Y2hpbmcgc2hvdWxkIGlkZWFsbHkgYmUgbmVhciAxLCBhbmQgY2VydGFpbmx5IGJldHdlZW4gMS8yIGFuZCAyLg0KDQpgYGB7cn0NCnJ1YmluMi5tMyA8LSB3aXRoKHJoYy5tYXRjaGVzXzMsIA0KICAgICAgICAgICAgICAgICAgdmFyKGxpbnBzW3N3YW5nMSA9PSAiUkhDIl0pIC8gDQogICAgICAgICAgICAgICAgICAgICAgdmFyKGxpbnBzW3N3YW5nMSA9PSAiTm8gUkhDIl0pKQ0KcnViaW4yLm0zDQpgYGANCg0KDQoNCiMgTWF0Y2ggNDogMToxIEdyZWVkeSBNYXRjaGluZyBvbiB0aGUgbGluZWFyIFBTLCBXSVRIIFJlcGxhY2VtZW50DQoNCmBgYHtyfQ0KWCA8LSByaGMkbGlucHMNClRyIDwtIGFzLmxvZ2ljYWwocmhjJHN3YW5nMSA9PSAiUkhDIikNCm1hdGNoNCA8LSBNYXRjaChUciA9IFRyLCBYID0gWCwgTSA9IDEsIA0KICAgICAgICAgICAgICAgIGVzdGltYW5kID0gIkFUVCIsIHJlcGxhY2UgPSBUUlVFLCANCiAgICAgICAgICAgICAgICB0aWVzID0gRkFMU0UpDQpzdW1tYXJ5KG1hdGNoNCkNCmBgYA0KDQojIyBMb3ZlIFBsb3QgZm9yIE1hdGNoIDQNCg0KYGBge3IsIGZpZy5oZWlnaHQgPSA4fQ0KYjQgPC0gYmFsLnRhYihtYXRjaDQsIA0KICAgICAgICAgICAgICBzd2FuZzEgPT0gIlJIQyIgIH4gDQogICAgICAgICAgICAgICAgICBhZ2UgKyBzZXggKyBlZHUgKyBpbmNvbWUgKyBuaW5zY2xhcyArIHJhY2UgKw0KICAgICAgICAgICAgICAgICAgY2F0MSArIGRucjEgKyB3dGtpbG8xICsgaHJ0MSArIG1lYW5icDEgKyANCiAgICAgICAgICAgICAgICAgIHJlc3AxICsgdGVtcDEgKw0KICAgICAgICAgICAgICAgICAgY2FyZCArIGdhc3RyICsgaGVtYSArIG1ldGEgKyBuZXVybyArIG9ydGhvICsgDQogICAgICAgICAgICAgICAgICByZW5hbCArIHJlc3AgKyBzZXBzICsgdHJhdW1hICsNCiAgICAgICAgICAgICAgICAgIGFtaWh4ICsgY2EgKyBjYXJkaW9oeCArIGNoZmh4ICsgY2hycHVsaHggKw0KICAgICAgICAgICAgICAgICAgZGVtZW50aHggKyBnaWJsZWRoeCArIGltbXVuaHggKyBsaXZlcmh4ICsNCiAgICAgICAgICAgICAgICAgIG1hbGlnaHggKyBwc3ljaGh4ICsgcmVuYWxoeCArIHRyYW5zaHggKw0KICAgICAgICAgICAgICAgICAgYXBzMSArIGRhczJkM3BjICsgc2NvbWExICsgc3VydjJtZDEgKw0KICAgICAgICAgICAgICAgICAgYWxiMSArIGJpbGkxICsgY3JlYTEgKyBoZW1hMSArIHBhY28yMSArIHBhZmkxICsNCiAgICAgICAgICAgICAgICAgIHBoMSArIHBvdDEgKyBzb2QxICsgd2JsYzEgKyBwcyArIGxpbnBzLA0KICAgICAgICAgICAgICBkYXRhPXJoYywgdW4gPSBUUlVFKQ0KDQpsb3ZlLnBsb3QoYjQsIHRocmVzaG9sZCA9IC4xLCBzaXplID0gMS41LCBzdGFycyA9ICJyYXciLA0KICAgICAgICAgICAgICAgdmFyLm9yZGVyID0gInVuYWRqdXN0ZWQiLA0KICAgICAgICAgICAgICAgdGl0bGUgPSAiU3RhbmRhcmRpemVkIERpZmZlcmVuY2VzIGFuZCBNYXRjaCA0IikgKw0KICAgIHRoZW1lX2J3KCkNCmBgYA0KDQojIyBSdWJpbidzIFJ1bGVzIDEgYW5kIDIgZm9yIE1hdGNoIDQNCg0KQ3JlYXRlIGEgbmV3IGRhdGEgZnJhbWUsIGNvbnRhaW5pbmcgb25seSB0aGUgbWF0Y2hlZCBzYW1wbGUuDQoNCmBgYHtyfQ0KbWF0Y2hlc180IDwtIGZhY3RvcihyZXAobWF0Y2g0JGluZGV4LnRyZWF0ZWQsIDIpKQ0KcmhjLm1hdGNoZXNfNCA8LSBjYmluZChtYXRjaGVzXzQsIA0KICAgICAgICAgIHJoY1tjKG1hdGNoNCRpbmRleC5jb250cm9sLCBtYXRjaDQkaW5kZXgudHJlYXRlZCksXSkNCmBgYA0KDQpSdWJpbiBSdWxlIDEgcmVzdWx0IGFmdGVyIG1hdGNoaW5nIHNob3VsZCBpZGVhbGx5IGJlIG5lYXIgMCwgYW5kIGNlcnRhaW5seSBiZXR3ZWVuIC01MCBhbmQgKzUwLg0KDQpgYGB7cn0NCnJ1YmluMS5tNCA8LSB3aXRoKHJoYy5tYXRjaGVzXzQsIA0KICAgICAgICAgICAgICAgICAgYWJzKDEwMCoobWVhbihsaW5wc1tzd2FuZzE9PSJSSEMiXSkgLSANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtZWFuKGxpbnBzW3N3YW5nMT09Ik5vIFJIQyJdKSkvDQogICAgICAgICAgICAgICAgICAgICAgICAgIHNkKGxpbnBzKSkpDQpydWJpbjEubTQNCmBgYA0KDQpSdWJpbiBSdWxlIDIgcmVzdWx0IGFmdGVyIG1hdGNoaW5nIHNob3VsZCBpZGVhbGx5IGJlIG5lYXIgMSwgYW5kIGNlcnRhaW5seSBiZXR3ZWVuIDEvMiBhbmQgMi4NCg0KYGBge3J9DQpydWJpbjIubTQgPC0gd2l0aChyaGMubWF0Y2hlc180LCANCiAgICAgICAgICAgICAgICAgIHZhcihsaW5wc1tzd2FuZzEgPT0gIlJIQyJdKSAvIA0KICAgICAgICAgICAgICAgICAgICAgIHZhcihsaW5wc1tzd2FuZzEgPT0gIk5vIFJIQyJdKSkNCnJ1YmluMi5tNA0KYGBgDQoNCg0KDQojIE1hdGNoIDU6IDE6MSBDYWxpcGVyIE1hdGNoaW5nIG9uIHRoZSBsaW5lYXIgUFMsIFdJVEhPVVQgUmVwbGFjZW1lbnQNCg0KYGBge3J9DQpYIDwtIHJoYyRsaW5wcw0KVHIgPC0gYXMubG9naWNhbChyaGMkc3dhbmcxID09ICJSSEMiKQ0KbWF0Y2g1IDwtIE1hdGNoKFRyID0gVHIsIFggPSBYLCBNID0gMSwgDQogICAgICAgICAgICAgICAgY2FsaXBlciA9IDAuMiwNCiAgICAgICAgICAgICAgICBlc3RpbWFuZCA9ICJBVFQiLCByZXBsYWNlID0gRkFMU0UsIA0KICAgICAgICAgICAgICAgIHRpZXMgPSBGQUxTRSkNCnN1bW1hcnkobWF0Y2g1KQ0KYGBgDQoNCiMjIExvdmUgUGxvdCBmb3IgTWF0Y2ggNQ0KDQpgYGB7ciwgZmlnLmhlaWdodCA9IDh9DQpiNSA8LSBiYWwudGFiKG1hdGNoNSwgDQogICAgICAgICAgICAgIHN3YW5nMSA9PSAiUkhDIiAgfiANCiAgICAgICAgICAgICAgICAgIGFnZSArIHNleCArIGVkdSArIGluY29tZSArIG5pbnNjbGFzICsgcmFjZSArDQogICAgICAgICAgICAgICAgICBjYXQxICsgZG5yMSArIHd0a2lsbzEgKyBocnQxICsgbWVhbmJwMSArIA0KICAgICAgICAgICAgICAgICAgcmVzcDEgKyB0ZW1wMSArDQogICAgICAgICAgICAgICAgICBjYXJkICsgZ2FzdHIgKyBoZW1hICsgbWV0YSArIG5ldXJvICsgb3J0aG8gKyANCiAgICAgICAgICAgICAgICAgIHJlbmFsICsgcmVzcCArIHNlcHMgKyB0cmF1bWEgKw0KICAgICAgICAgICAgICAgICAgYW1paHggKyBjYSArIGNhcmRpb2h4ICsgY2hmaHggKyBjaHJwdWxoeCArDQogICAgICAgICAgICAgICAgICBkZW1lbnRoeCArIGdpYmxlZGh4ICsgaW1tdW5oeCArIGxpdmVyaHggKw0KICAgICAgICAgICAgICAgICAgbWFsaWdoeCArIHBzeWNoaHggKyByZW5hbGh4ICsgdHJhbnNoeCArDQogICAgICAgICAgICAgICAgICBhcHMxICsgZGFzMmQzcGMgKyBzY29tYTEgKyBzdXJ2Mm1kMSArDQogICAgICAgICAgICAgICAgICBhbGIxICsgYmlsaTEgKyBjcmVhMSArIGhlbWExICsgcGFjbzIxICsgcGFmaTEgKw0KICAgICAgICAgICAgICAgICAgcGgxICsgcG90MSArIHNvZDEgKyB3YmxjMSArIHBzICsgbGlucHMsDQogICAgICAgICAgICAgIGRhdGE9cmhjLCB1biA9IFRSVUUpDQoNCmxvdmUucGxvdChiNSwgdGhyZXNob2xkID0gLjEsIHNpemUgPSAxLjUsIHN0YXJzID0gInJhdyIsDQogICAgICAgICAgICAgICB2YXIub3JkZXIgPSAidW5hZGp1c3RlZCIsDQogICAgICAgICAgICAgICB0aXRsZSA9ICJTdGFuZGFyZGl6ZWQgRGlmZmVyZW5jZXMgYW5kIE1hdGNoIDUiKSArDQogICAgdGhlbWVfYncoKQ0KYGBgDQoNCiMjIFJ1YmluJ3MgUnVsZXMgMSBhbmQgMiBmb3IgTWF0Y2ggNQ0KDQpDcmVhdGUgYSBuZXcgZGF0YSBmcmFtZSwgY29udGFpbmluZyBvbmx5IHRoZSBtYXRjaGVkIHNhbXBsZS4NCg0KYGBge3J9DQptYXRjaGVzXzUgPC0gZmFjdG9yKHJlcChtYXRjaDUkaW5kZXgudHJlYXRlZCwgMikpDQpyaGMubWF0Y2hlc181IDwtIGNiaW5kKG1hdGNoZXNfNSwgDQogICAgICAgICAgcmhjW2MobWF0Y2g1JGluZGV4LmNvbnRyb2wsIG1hdGNoNSRpbmRleC50cmVhdGVkKSxdKQ0KYGBgDQoNClJ1YmluIFJ1bGUgMSByZXN1bHQgYWZ0ZXIgbWF0Y2hpbmcgc2hvdWxkIGlkZWFsbHkgYmUgbmVhciAwLCBhbmQgY2VydGFpbmx5IGJldHdlZW4gLTUwIGFuZCArNTAuDQoNCmBgYHtyfQ0KcnViaW4xLm01IDwtIHdpdGgocmhjLm1hdGNoZXNfNSwgDQogICAgICAgICAgICAgICAgICBhYnMoMTAwKihtZWFuKGxpbnBzW3N3YW5nMT09IlJIQyJdKSAtIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1lYW4obGlucHNbc3dhbmcxPT0iTm8gUkhDIl0pKS8NCiAgICAgICAgICAgICAgICAgICAgICAgICAgc2QobGlucHMpKSkNCnJ1YmluMS5tNQ0KYGBgDQoNClJ1YmluIFJ1bGUgMiByZXN1bHQgYWZ0ZXIgbWF0Y2hpbmcgc2hvdWxkIGlkZWFsbHkgYmUgbmVhciAxLCBhbmQgY2VydGFpbmx5IGJldHdlZW4gMS8yIGFuZCAyLg0KDQpgYGB7cn0NCnJ1YmluMi5tNSA8LSB3aXRoKHJoYy5tYXRjaGVzXzUsIA0KICAgICAgICAgICAgICAgICAgdmFyKGxpbnBzW3N3YW5nMSA9PSAiUkhDIl0pIC8gDQogICAgICAgICAgICAgICAgICAgICAgdmFyKGxpbnBzW3N3YW5nMSA9PSAiTm8gUkhDIl0pKQ0KcnViaW4yLm01DQpgYGANCg0KIyBNYXRjaCA2OiAxOjIgR3JlZWR5IE1hdGNoaW5nIG9uIHRoZSBsaW5lYXIgUFMsIFdJVEggUmVwbGFjZW1lbnQNCg0KYGBge3J9DQpYIDwtIHJoYyRsaW5wcw0KVHIgPC0gYXMubG9naWNhbChyaGMkc3dhbmcxID09ICJSSEMiKQ0KbWF0Y2g2IDwtIE1hdGNoKFRyID0gVHIsIFggPSBYLCBNID0gMiwgDQogICAgICAgICAgICAgICAgZXN0aW1hbmQgPSAiQVRUIiwgcmVwbGFjZSA9IFRSVUUsIA0KICAgICAgICAgICAgICAgIHRpZXMgPSBGQUxTRSkNCnN1bW1hcnkobWF0Y2g2KQ0KYGBgDQoNCiMjIExvdmUgUGxvdCBmb3IgTWF0Y2ggNg0KDQpgYGB7ciwgZmlnLmhlaWdodCA9IDh9DQpiNiA8LSBiYWwudGFiKG1hdGNoNiwgDQogICAgICAgICAgICAgIHN3YW5nMSA9PSAiUkhDIiAgfiANCiAgICAgICAgICAgICAgICAgIGFnZSArIHNleCArIGVkdSArIGluY29tZSArIG5pbnNjbGFzICsgcmFjZSArDQogICAgICAgICAgICAgICAgICBjYXQxICsgZG5yMSArIHd0a2lsbzEgKyBocnQxICsgbWVhbmJwMSArIA0KICAgICAgICAgICAgICAgICAgcmVzcDEgKyB0ZW1wMSArDQogICAgICAgICAgICAgICAgICBjYXJkICsgZ2FzdHIgKyBoZW1hICsgbWV0YSArIG5ldXJvICsgb3J0aG8gKyANCiAgICAgICAgICAgICAgICAgIHJlbmFsICsgcmVzcCArIHNlcHMgKyB0cmF1bWEgKw0KICAgICAgICAgICAgICAgICAgYW1paHggKyBjYSArIGNhcmRpb2h4ICsgY2hmaHggKyBjaHJwdWxoeCArDQogICAgICAgICAgICAgICAgICBkZW1lbnRoeCArIGdpYmxlZGh4ICsgaW1tdW5oeCArIGxpdmVyaHggKw0KICAgICAgICAgICAgICAgICAgbWFsaWdoeCArIHBzeWNoaHggKyByZW5hbGh4ICsgdHJhbnNoeCArDQogICAgICAgICAgICAgICAgICBhcHMxICsgZGFzMmQzcGMgKyBzY29tYTEgKyBzdXJ2Mm1kMSArDQogICAgICAgICAgICAgICAgICBhbGIxICsgYmlsaTEgKyBjcmVhMSArIGhlbWExICsgcGFjbzIxICsgcGFmaTEgKw0KICAgICAgICAgICAgICAgICAgcGgxICsgcG90MSArIHNvZDEgKyB3YmxjMSArIHBzICsgbGlucHMsDQogICAgICAgICAgICAgIGRhdGE9cmhjLCB1biA9IFRSVUUpDQoNCmxvdmUucGxvdChiNiwgdGhyZXNob2xkID0gLjEsIHNpemUgPSAxLjUsIHN0YXJzID0gInJhdyIsDQogICAgICAgICAgICAgICB2YXIub3JkZXIgPSAidW5hZGp1c3RlZCIsDQogICAgICAgICAgICAgICB0aXRsZSA9ICJTdGFuZGFyZGl6ZWQgRGlmZmVyZW5jZXMgYW5kIE1hdGNoIDYiKSArDQogICAgdGhlbWVfYncoKQ0KYGBgDQoNCiMjIFJ1YmluJ3MgUnVsZXMgMSBhbmQgMiBmb3IgTWF0Y2ggNg0KDQpDcmVhdGUgYSBuZXcgZGF0YSBmcmFtZSwgY29udGFpbmluZyBvbmx5IHRoZSBtYXRjaGVkIHNhbXBsZS4NCg0KYGBge3J9DQptYXRjaGVzXzYgPC0gZmFjdG9yKHJlcChtYXRjaDYkaW5kZXgudHJlYXRlZCwgMikpDQpyaGMubWF0Y2hlc182IDwtIGNiaW5kKG1hdGNoZXNfNiwgDQogICAgICAgICAgcmhjW2MobWF0Y2g2JGluZGV4LmNvbnRyb2wsIG1hdGNoNiRpbmRleC50cmVhdGVkKSxdKQ0KYGBgDQoNClJ1YmluIFJ1bGUgMSByZXN1bHQgYWZ0ZXIgbWF0Y2hpbmcgc2hvdWxkIGlkZWFsbHkgYmUgbmVhciAwLCBhbmQgY2VydGFpbmx5IGJldHdlZW4gLTUwIGFuZCArNTAuDQoNCmBgYHtyfQ0KcnViaW4xLm02IDwtIHdpdGgocmhjLm1hdGNoZXNfNiwgDQogICAgICAgICAgICAgICAgICBhYnMoMTAwKihtZWFuKGxpbnBzW3N3YW5nMT09IlJIQyJdKSAtIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1lYW4obGlucHNbc3dhbmcxPT0iTm8gUkhDIl0pKS8NCiAgICAgICAgICAgICAgICAgICAgICAgICAgc2QobGlucHMpKSkNCnJ1YmluMS5tNg0KYGBgDQoNClJ1YmluIFJ1bGUgMiByZXN1bHQgYWZ0ZXIgbWF0Y2hpbmcgc2hvdWxkIGlkZWFsbHkgYmUgbmVhciAxLCBhbmQgY2VydGFpbmx5IGJldHdlZW4gMS8yIGFuZCAyLg0KDQpgYGB7cn0NCnJ1YmluMi5tNiA8LSB3aXRoKHJoYy5tYXRjaGVzXzYsIA0KICAgICAgICAgICAgICAgICAgdmFyKGxpbnBzW3N3YW5nMSA9PSAiUkhDIl0pIC8gDQogICAgICAgICAgICAgICAgICAgICAgdmFyKGxpbnBzW3N3YW5nMSA9PSAiTm8gUkhDIl0pKQ0KcnViaW4yLm02DQpgYGANCg0KIyBSYW5raW5nIHRoZSBNYXRjaGVzDQoNCk1hdGNoIHwgUnViaW4gMSB8IFJ1YmluIDIgfCBMb3ZlIFBsb3QgfCBNYXRjaGVkIFNldHMgfCBEZXNjcmlwdGlvbg0KLS0tLS06IHwgLS0tLS0tOiB8IC0tLS0tOiB8IC0tLS0tLS0tLSB8IC0tLS0tLS06IHwgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KTm9uZSB8IGByIHJvdW5kX2hhbGZfdXAocnViaW4xLnVuYWRqLDEpYCB8IGByIHJvdW5kX2hhbGZfdXAocnViaW4yLnVuYWRqLDIpYCB8IC0tIHwgLS0gfCAgTm8gTWF0Y2hpbmcNCjEgfCBgciByb3VuZF9oYWxmX3VwKHJ1YmluMS5tMSwgMSlgIHwgYHIgcm91bmRfaGFsZl91cChydWJpbjIubTEsIDIpYCB8IFByZXR0eSBCYWQgfCAyLDE4NCB8IDE6MSBncmVlZHkgdy9vIHJlcGwNCjIgfCBgciByb3VuZF9oYWxmX3VwKHJ1YmluMS5tMiwgMSlgIHwgYHIgcm91bmRfaGFsZl91cChydWJpbjIubTIsIDIpYCB8IERyZWFkZnVsIHwgMSw3NzUgfCAxOjIgZ3JlZWR5IHcvbyByZXBsDQozIHwgYHIgcm91bmRfaGFsZl91cChydWJpbjEubTMsIDEpYCB8IGByIHJvdW5kX2hhbGZfdXAocnViaW4yLm0zLCAyKWAgfCBWZXJ5IEdvb2QgfCAyLDE4NCB8IEdlbmV0aWMgU2VhcmNoIDE6MSB3L28gcmVwbA0KNCB8IGByIHJvdW5kX2hhbGZfdXAocnViaW4xLm00LCAxKWAgfCBgciByb3VuZF9oYWxmX3VwKHJ1YmluMi5tNCwgMilgIHwgVmVyeSBHb29kIHwgMiwxODQgfCAxOjEgZ3JlZWR5IHdpdGggcmVwbA0KNSB8IGByIHJvdW5kX2hhbGZfdXAocnViaW4xLm01LCAxKWAgfCBgciByb3VuZF9oYWxmX3VwKHJ1YmluMi5tNSwgMilgIHwgVmVyeSBHb29kIHwgMSw1NjIgfCAxOjEgY2FsaXBlciB3aXRob3V0IHJlcGwNCjYgfCBgciByb3VuZF9oYWxmX3VwKHJ1YmluMS5tNiwgMSlgIHwgYHIgcm91bmRfaGFsZl91cChydWJpbjIubTYsIDIpYCB8IFZlcnkgR29vZCB8IDIsMTg0IHwgMToyIGdyZWVkeSB3aXRoIHJlcGwNCg0KDQojIEFUVCBNYXRjaGluZyBFc3RpbWF0ZXMgZm9yIHRoZSBCaW5hcnkgT3V0Y29tZSwgRGVhdGgNCg0KIyMgV2l0aCBNYXRjaCAxLCBpbiBkZXRhaWwNCg0KTm90ZSB0aGF0IHRoZSB2YXJpYWJsZSB3aGljaCBpZGVudGlmaWVzIHRoZSBtYXRjaGVzIChjYWxsZWQgYG1hdGNoZXNfMWApIGlzIHNldCB1cCBhcyBhIGZhY3RvciwgYXMgaXQgbmVlZHMgdG8gYmUsIGFuZCB0aGF0IEknbSB1c2luZyB0aGUgMS8wIHZlcnNpb25zIG9mIGJvdGggdGhlIG91dGNvbWUgKHNvIGBkaWVkYCByYXRoZXIgdGhhbiBgZGVhdGhgKSBhbmQgdHJlYXRtZW50IChzbyBgdHJlYXRfcmhjYCByYXRoZXIgdGhhbiBgc3dhbmcxYCkuDQoNCmBgYHtyfQ0KZGVhdGhfYWRqX20xIDwtIGNsb2dpdChkaWVkIH4gdHJlYXRfcmhjICsgc3RyYXRhKG1hdGNoZXNfMSksIGRhdGEgPSByaGMubWF0Y2hlc18xKQ0KDQpzdW1tYXJ5KGRlYXRoX2Fkal9tMSkNCmBgYA0KDQpUaGUgb2RkcyByYXRpbyBpbiB0aGUgYGV4cChjb2VmKWAgc2VjdGlvbiBhYm92ZSBpcyB0aGUgYXZlcmFnZSBjYXVzYWwgZWZmZWN0IGVzdGltYXRlLiBJdCBkZXNjcmliZXMgdGhlIG9kZHMgb2YgaGF2aW5nIGFuIGV2ZW50IChgZGllZGApIG9jY3VyIGFzc29jaWF0ZWQgd2l0aCBiZWluZyBhIFJIQyBzdWJqZWN0IChhcyBpZGVudGlmaWVkIGJ5IGB0cmVhdF9yaGNgKSwgYXMgY29tcGFyZWQgdG8gdGhlIG9kZHMgb2YgdGhhdCBldmVudCB3aGVuIGEgbm9uLVJIQyBzdWJqZWN0LiBXZSBjYW4gdGlkeSB0aGlzIHdpdGg6DQoNCmBgYHtyfQ0KdGlkeShkZWF0aF9hZGpfbTEsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIGNvbmYubGV2ZWwgPSAwLjk1KQ0KYGBgDQoNCiMjIFdpdGggTWF0Y2hlcyAyLTYsIGluIG11Y2ggbGVzcyBkZXRhaWwNCg0KYGBge3J9DQpkZWF0aF9hZGpfbTIgPC0gY2xvZ2l0KGRpZWQgfiB0cmVhdF9yaGMgKyBzdHJhdGEobWF0Y2hlc18yKSwgZGF0YSA9IHJoYy5tYXRjaGVzXzIpDQp0aWR5KGRlYXRoX2Fkal9tMiwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpDQpgYGANCg0KYGBge3J9DQpkZWF0aF9hZGpfbTMgPC0gY2xvZ2l0KGRpZWQgfiB0cmVhdF9yaGMgKyBzdHJhdGEobWF0Y2hlc18zKSwgZGF0YSA9IHJoYy5tYXRjaGVzXzMpDQp0aWR5KGRlYXRoX2Fkal9tMywgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpDQpgYGANCg0KYGBge3J9DQpkZWF0aF9hZGpfbTQgPC0gY2xvZ2l0KGRpZWQgfiB0cmVhdF9yaGMgKyBzdHJhdGEobWF0Y2hlc180KSwgZGF0YSA9IHJoYy5tYXRjaGVzXzQpDQp0aWR5KGRlYXRoX2Fkal9tNCwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpDQpgYGANCg0KYGBge3J9DQpkZWF0aF9hZGpfbTUgPC0gY2xvZ2l0KGRpZWQgfiB0cmVhdF9yaGMgKyBzdHJhdGEobWF0Y2hlc181KSwgZGF0YSA9IHJoYy5tYXRjaGVzXzUpDQp0aWR5KGRlYXRoX2Fkal9tNSwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpDQpgYGANCg0KYGBge3J9DQpkZWF0aF9hZGpfbTYgPC0gY2xvZ2l0KGRpZWQgfiB0cmVhdF9yaGMgKyBzdHJhdGEobWF0Y2hlc182KSwgZGF0YSA9IHJoYy5tYXRjaGVzXzYpDQp0aWR5KGRlYXRoX2Fkal9tNiwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpDQpgYGANCg0KIyMgR3JhcGhpbmcgdGhlIFJlc3VsdHMgQWNyb3NzIE11bHRpcGxlIE1hdGNoaW5nIFN0cmF0ZWdpZXMNCg0KYGBge3J9DQp0ZW1wMCA8LSB0aWR5KGRlYXRoX3VuYWRqLCBjb25mLmludCA9IFRSVUUsIGV4cG9uZW50aWF0ZSA9IFRSVUUpICU+JSANCiAgICBmaWx0ZXIodGVybSA9PSAidHJlYXRfcmhjIikNCnRlbXAxIDwtIHRpZHkoZGVhdGhfYWRqX20xLCBleHBvbmVudGlhdGUgPSBUUlVFLCBjb25mLmludCA9IFQsIGNvbmYubGV2ZWwgPSAwLjk1KQ0KdGVtcDIgPC0gdGlkeShkZWF0aF9hZGpfbTIsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIGNvbmYuaW50ID0gVCwgY29uZi5sZXZlbCA9IDAuOTUpDQp0ZW1wMyA8LSB0aWR5KGRlYXRoX2Fkal9tMywgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBULCBjb25mLmxldmVsID0gMC45NSkNCnRlbXA0IDwtIHRpZHkoZGVhdGhfYWRqX200LCBleHBvbmVudGlhdGUgPSBUUlVFLCBjb25mLmludCA9IFQsIGNvbmYubGV2ZWwgPSAwLjk1KQ0KdGVtcDUgPC0gdGlkeShkZWF0aF9hZGpfbTUsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIGNvbmYuaW50ID0gVCwgY29uZi5sZXZlbCA9IDAuOTUpDQp0ZW1wNiA8LSB0aWR5KGRlYXRoX2Fkal9tNiwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBULCBjb25mLmxldmVsID0gMC45NSkNCg0KZGVhdGhfbWF0Y2hfcmVzdWx0cyA8LSByYmluZCh0ZW1wMCwgdGVtcDEsIHRlbXAyLCB0ZW1wMywgdGVtcDQsIHRlbXA1LCB0ZW1wNikNCg0KZGVhdGhfbWF0Y2hfcmVzdWx0cyA8LSBkZWF0aF9tYXRjaF9yZXN1bHRzICU+JQ0KICAgIG11dGF0ZShhcHByb2FjaCA9IGMoIlVubWF0Y2hlZCIsICJNYXRjaCAxIiwgIk1hdGNoIDIiLCAiTWF0Y2ggMyIsICJNYXRjaCA0IiwgIk1hdGNoIDUiLCAiTWF0Y2ggNiIpKSAlPiUNCiAgICBtdXRhdGUoZGVzY3JpcHRpb24gPSBjKCJObyBNYXRjaGluZyIsICIxOjEgZ3JlZWR5IG1hdGNoaW5nIHdpdGhvdXQgcmVwbGFjZW1lbnQiLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICIxOjIgZ3JlZWR5IG1hdGNoaW5nIHdpdGhvdXQgcmVwbGFjZW1lbnQiLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgIjE6MSBnZW5ldGljIHNlYXJjaCBtYXRjaGluZyB3aXRob3V0IHJlcGxhY2VtZW50IiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICIxOjEgZ3JlZWR5IG1hdGNoaW5nIHdpdGggcmVwbGFjZW1lbnQiLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgIjE6MSBjYWxpcGVyIG1hdGNoaW5nIG9uIHRoZSBsaW5lYXIgUFMgd2l0aG91dCByZXBsYWNlbWVudCIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAiMToyIGdyZWVkeSBtYXRjaGluZyB3aXRoIHJlcGxhY2VtZW50IikpDQoNCmRlYXRoX21hdGNoX3Jlc3VsdHMNCmBgYA0KDQpgYGB7cn0NCmdncGxvdChkZWF0aF9tYXRjaF9yZXN1bHRzLCBhZXMoeCA9IGFwcHJvYWNoLCB5ID0gZXN0aW1hdGUsIHltaW4gPSBjb25mLmxvdywgeW1heCA9IGNvbmYuaGlnaCkpICsNCiAgICBnZW9tX3RleHQoYWVzKGxhYmVsID0gcm91bmRfaGFsZl91cChlc3RpbWF0ZSwyKSksIHZqdXN0ID0gLTEuMjUpICsNCiAgICBnZW9tX3BvaW50cmFuZ2UoKSArDQogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMSwgY29sID0gInJlZCIsIGxpbmV0eXBlID0gImRhc2hlZCIpICsNCiAgICB0aGVtZV9idygpICsNCiAgICBjb29yZF9mbGlwKCkgKw0KICAgIGxhYnModGl0bGUgPSAiQ29tcGFyaW5nIEFUVCBFc3RpbWF0ZXMgZm9yIERlYXRoIHVzaW5nIFJIQyBQcm9wZW5zaXR5IE1hdGNoaW5nIiwNCiAgICAgICAgIHggPSAiUHJvcGVuc2l0eSBBZGp1c3RtZW50IEFwcHJvYWNoIiwgDQogICAgICAgICB5ID0gIkVzdGltYXRlZCBPZGRzIFJhdGlvIGZvciBEZWF0aCBhc3NvY2lhdGVkIHdpdGggUkhDIikNCmBgYA0KDQpMZXQncyBsb29rIGF0IGp1c3QgdGhlIGZvdXIgc3RyYXRlZ2llcyB0aGF0IGFjaGlldmVkIHN0cm9uZ2VyIGJhbGFuY2UuDQoNCmBgYHtyfQ0KZGVhdGhfbWF0Y2hfcmVzdWx0cyAlPiUNCiAgICBmaWx0ZXIoYXBwcm9hY2ggJWluJSBjKCJNYXRjaCAzIiwgIk1hdGNoIDQiLCAiTWF0Y2ggNSIsICJNYXRjaCA2IikpICU+JQ0KZ2dwbG90KC4sIGFlcyh4ID0gZGVzY3JpcHRpb24sIHkgPSBlc3RpbWF0ZSwgeW1pbiA9IGNvbmYubG93LCB5bWF4ID0gY29uZi5oaWdoKSkgKw0KICAgIGdlb21fdGV4dChhZXMobGFiZWwgPSByb3VuZF9oYWxmX3VwKGVzdGltYXRlLDIpKSwgdmp1c3QgPSAtMS4yNSkgKw0KICAgIGdlb21fcG9pbnRyYW5nZSgpICsNCiAgICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSAxLCBjb2wgPSAicmVkIiwgbGluZXR5cGUgPSAiZGFzaGVkIikgKw0KICAgIHRoZW1lX2J3KCkgKw0KICAgIGNvb3JkX2ZsaXAoKSArDQogICAgbGFicyh0aXRsZSA9ICJBVFQgTWF0Y2hpbmcgRXN0aW1hdGVzIGluIFJIQyAoRGVhdGgpIiwNCiAgICAgICAgIHggPSAiUHJvcGVuc2l0eSBBZGp1c3RtZW50IEFwcHJvYWNoIiwgDQogICAgICAgICB5ID0gIkVzdGltYXRlZCBPZGRzIFJhdGlvIGZvciBEZWF0aCBhc3NvY2lhdGVkIHdpdGggUkhDIikNCmBgYA0KDQojIEFUVCBNYXRjaGluZyBFc3RpbWF0ZXMgZm9yIHRoZSBRdWFudGl0YXRpdmUgT3V0Y29tZSwgSG9zcGl0YWwgTGVuZ3RoIG9mIFN0YXkNCg0KIyMgV2l0aCBNYXRjaCAxLCBpbiBkZXRhaWwNCg0KQWdhaW4sIHRoZSB2YXJpYWJsZSB3aGljaCBpZGVudGlmaWVzIHRoZSBtYXRjaGVzIChjYWxsZWQgYG1hdGNoZXNfMWApIGlzIHNldCB1cCBhcyBhIGZhY3RvciwgYXMgaXQgbmVlZHMgdG8gYmUsIGFuZCB0aGF0IEknbSB1c2luZyB0aGUgMS8wIHZlcnNpb24gb2YgdHJlYXRtZW50IChzbyBgdHJlYXRfcmhjYCByYXRoZXIgdGhhbiBgc3dhbmcxYCkuIA0KDQpPdXIgcmVzdWx0IGZvciB0aGlzIHF1YW50aXRhdGl2ZSBvdXRjb21lIChgaG9zcGRheXNgKSBjb21lcyBmcm9tIGEgbWl4ZWQgbW9kZWwgd2hlcmUgdGhlIG1hdGNoZXMgYXJlIHRyZWF0ZWQgYXMgYSByYW5kb20gZmFjdG9yLCBidXQgdGhlIHRyZWF0bWVudCBncm91cCBpcyB0cmVhdGVkIGFzIGEgZml4ZWQgZmFjdG9yLCB1c2luZyB0aGUgYGxtZTRgIHBhY2thZ2UuDQoNCmBgYHtyfQ0KaG9zcGRheXNfYWRqX20xIDwtIGxtZXIoaG9zcGRheXMgfiB0cmVhdF9yaGMgKyAoMSB8IG1hdGNoZXNfMSksDQogICAgICAgICAgICAgICAgICAgICAgICBkYXRhID0gcmhjLm1hdGNoZXNfMSkNCg0Kc3VtbWFyeShob3NwZGF5c19hZGpfbTEpOyBjb25maW50KGhvc3BkYXlzX2Fkal9tMSkNCmBgYA0KDQpUaGUgZXN0aW1hdGVkIGVmZmVjdCBvZiBgdHJlYXRfcmhjYCBpbiB0aGUgZml4ZWQgZWZmZWN0cyBzZWN0aW9uLCBjb21iaW5lZCB3aXRoIHRoZSA5NSUgY29uZmlkZW5jZSBpbnRlcnZhbHMgbWF0ZXJpYWwsIHByb3ZpZGVzIHRoZSBhdmVyYWdlIGNhdXNhbCBlZmZlY3QgZXN0aW1hdGUuIEl0IGRlc2NyaWJlcyB0aGUgY2hhbmdlIGluIHRoZSBvdXRjb21lIGBob3NwZGF5c2AgYXNzb2NpYXRlZCB3aXRoIGJlaW5nIGEgUkhDIHN1YmplY3QgKGFzIGlkZW50aWZpZWQgYnkgYHRyZWF0X3JoY2ApLCBhcyBjb21wYXJlZCB0byBiZWluZyBhIG5vbi1SSEMgc3ViamVjdC4gV2UgY2FuIHRpZHkgdGhpcyB3aXRoOg0KDQpgYGB7ciwgbWVzc2FnZSA9IEZBTFNFfQ0KYnJvb20ubWl4ZWQ6OnRpZHkoaG9zcGRheXNfYWRqX20xLCBjb25mLmludCA9IFRSVUUsIGNvbmYubGV2ZWwgPSAwLjk1KQ0KYGBgDQoNCm9yIG1vcmUgc3BlY2lmaWNhbGx5LCB3ZSBjYW4gbG9vayBhdCBqdXN0IHRoZSB0cmVhdG1lbnQgZWZmZWN0IHdpdGg6DQoNCmBgYHtyLCBtZXNzYWdlID0gRkFMU0V9DQpicm9vbS5taXhlZDo6dGlkeShob3NwZGF5c19hZGpfbTEsIGNvbmYuaW50ID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpICU+JSANCiAgICBmaWx0ZXIodGVybSA9PSAidHJlYXRfcmhjIikNCmBgYA0KDQojIyBXaXRoIE1hdGNoZXMgMi02LCBpbiBtdWNoIGxlc3MgZGV0YWlsDQoNCmBgYHtyfQ0KaG9zcGRheXNfYWRqX20yIDwtIGxtZXIoaG9zcGRheXMgfiB0cmVhdF9yaGMgKyAoMSB8IG1hdGNoZXNfMiksDQogICAgICAgICAgICAgICAgICAgICAgICBkYXRhID0gcmhjLm1hdGNoZXNfMikNCmJyb29tLm1peGVkOjp0aWR5KGhvc3BkYXlzX2Fkal9tMiwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkgJT4lIA0KICAgIGZpbHRlcih0ZXJtID09ICJ0cmVhdF9yaGMiKQ0KYGBgDQoNCmBgYHtyfQ0KaG9zcGRheXNfYWRqX20zIDwtIGxtZXIoaG9zcGRheXMgfiB0cmVhdF9yaGMgKyAoMSB8IG1hdGNoZXNfMyksDQogICAgICAgICAgICAgICAgICAgICAgICBkYXRhID0gcmhjLm1hdGNoZXNfMykNCmJyb29tLm1peGVkOjp0aWR5KGhvc3BkYXlzX2Fkal9tMywgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkgJT4lIA0KICAgIGZpbHRlcih0ZXJtID09ICJ0cmVhdF9yaGMiKQ0KYGBgDQoNCmBgYHtyfQ0KaG9zcGRheXNfYWRqX200IDwtIGxtZXIoaG9zcGRheXMgfiB0cmVhdF9yaGMgKyAoMSB8IG1hdGNoZXNfNCksDQogICAgICAgICAgICAgICAgICAgICAgICBkYXRhID0gcmhjLm1hdGNoZXNfNCkNCmJyb29tLm1peGVkOjp0aWR5KGhvc3BkYXlzX2Fkal9tNCwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkgJT4lIA0KICAgIGZpbHRlcih0ZXJtID09ICJ0cmVhdF9yaGMiKQ0KYGBgDQoNCmBgYHtyfQ0KaG9zcGRheXNfYWRqX201IDwtIGxtZXIoaG9zcGRheXMgfiB0cmVhdF9yaGMgKyAoMSB8IG1hdGNoZXNfNSksDQogICAgICAgICAgICAgICAgICAgICAgICBkYXRhID0gcmhjLm1hdGNoZXNfNSkNCmJyb29tLm1peGVkOjp0aWR5KGhvc3BkYXlzX2Fkal9tNSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkgJT4lIA0KICAgIGZpbHRlcih0ZXJtID09ICJ0cmVhdF9yaGMiKQ0KYGBgDQoNCmBgYHtyfQ0KaG9zcGRheXNfYWRqX202IDwtIGxtZXIoaG9zcGRheXMgfiB0cmVhdF9yaGMgKyAoMSB8IG1hdGNoZXNfNiksDQogICAgICAgICAgICAgICAgICAgICAgICBkYXRhID0gcmhjLm1hdGNoZXNfNikNCmJyb29tLm1peGVkOjp0aWR5KGhvc3BkYXlzX2Fkal9tNiwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkgJT4lIA0KICAgIGZpbHRlcih0ZXJtID09ICJ0cmVhdF9yaGMiKQ0KYGBgDQoNCiMjIEdyYXBoaW5nIHRoZSBSZXN1bHRzIEFjcm9zcyBNdWx0aXBsZSBNYXRjaGluZyBTdHJhdGVnaWVzDQoNCmBgYHtyfQ0KdGVtcDAgPC0gdGlkeShob3NwZGF5c191bmFkaiwgY29uZi5pbnQgPSBUUlVFKSAlPiUgDQogICAgZmlsdGVyKHRlcm0gPT0gInRyZWF0X3JoYyIpICU+JSANCiAgICBzZWxlY3QodGVybSwgZXN0aW1hdGUsIHN0ZC5lcnJvciwgY29uZi5sb3csIGNvbmYuaGlnaCkNCg0KdGVtcDEgPC0gdGlkeShob3NwZGF5c19hZGpfbTEsIGNvbmYuaW50ID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpICU+JSANCiAgICBmaWx0ZXIodGVybSA9PSAidHJlYXRfcmhjIikgJT4lIA0KICAgIHNlbGVjdCh0ZXJtLCBlc3RpbWF0ZSwgc3RkLmVycm9yLCBjb25mLmxvdywgY29uZi5oaWdoKQ0KDQp0ZW1wMiA8LSB0aWR5KGhvc3BkYXlzX2Fkal9tMiwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkgJT4lIA0KICAgIGZpbHRlcih0ZXJtID09ICJ0cmVhdF9yaGMiKSAlPiUgDQogICAgc2VsZWN0KHRlcm0sIGVzdGltYXRlLCBzdGQuZXJyb3IsIGNvbmYubG93LCBjb25mLmhpZ2gpDQoNCnRlbXAzIDwtIHRpZHkoaG9zcGRheXNfYWRqX20zLCBjb25mLmludCA9IFRSVUUsIGNvbmYubGV2ZWwgPSAwLjk1KSAlPiUgDQogICAgZmlsdGVyKHRlcm0gPT0gInRyZWF0X3JoYyIpICU+JSANCiAgICBzZWxlY3QodGVybSwgZXN0aW1hdGUsIHN0ZC5lcnJvciwgY29uZi5sb3csIGNvbmYuaGlnaCkNCg0KdGVtcDQgPC0gdGlkeShob3NwZGF5c19hZGpfbTQsIGNvbmYuaW50ID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpICU+JSANCiAgICBmaWx0ZXIodGVybSA9PSAidHJlYXRfcmhjIikgJT4lIA0KICAgIHNlbGVjdCh0ZXJtLCBlc3RpbWF0ZSwgc3RkLmVycm9yLCBjb25mLmxvdywgY29uZi5oaWdoKQ0KDQp0ZW1wNSA8LSB0aWR5KGhvc3BkYXlzX2Fkal9tNSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkgJT4lIA0KICAgIGZpbHRlcih0ZXJtID09ICJ0cmVhdF9yaGMiKSAlPiUgDQogICAgc2VsZWN0KHRlcm0sIGVzdGltYXRlLCBzdGQuZXJyb3IsIGNvbmYubG93LCBjb25mLmhpZ2gpDQoNCnRlbXA2IDwtIHRpZHkoaG9zcGRheXNfYWRqX202LCBjb25mLmludCA9IFRSVUUsIGNvbmYubGV2ZWwgPSAwLjk1KSAlPiUgDQogICAgZmlsdGVyKHRlcm0gPT0gInRyZWF0X3JoYyIpICU+JSANCiAgICBzZWxlY3QodGVybSwgZXN0aW1hdGUsIHN0ZC5lcnJvciwgY29uZi5sb3csIGNvbmYuaGlnaCkNCg0KaG9zcGRheXNfbWF0Y2hfcmVzdWx0cyA8LSByYmluZCh0ZW1wMCwgdGVtcDEsIHRlbXAyLCB0ZW1wMywgdGVtcDQsIHRlbXA1LCB0ZW1wNikNCg0KaG9zcGRheXNfbWF0Y2hfcmVzdWx0cyA8LSBob3NwZGF5c19tYXRjaF9yZXN1bHRzICU+JQ0KICAgIG11dGF0ZShhcHByb2FjaCA9IGMoIlVubWF0Y2hlZCIsICJNYXRjaCAxIiwgIk1hdGNoIDIiLCAiTWF0Y2ggMyIsICJNYXRjaCA0IiwgIk1hdGNoIDUiLCAiTWF0Y2ggNiIpKSAlPiUNCiAgICBtdXRhdGUoZGVzY3JpcHRpb24gPSBjKCJObyBNYXRjaGluZyIsICIxOjEgZ3JlZWR5IG1hdGNoaW5nIHdpdGhvdXQgcmVwbGFjZW1lbnQiLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICIxOjIgZ3JlZWR5IG1hdGNoaW5nIHdpdGhvdXQgcmVwbGFjZW1lbnQiLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgIjE6MSBnZW5ldGljIHNlYXJjaCBtYXRjaGluZyB3aXRob3V0IHJlcGxhY2VtZW50IiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICIxOjEgZ3JlZWR5IG1hdGNoaW5nIHdpdGggcmVwbGFjZW1lbnQiLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgIjE6MSBjYWxpcGVyIG1hdGNoaW5nIG9uIHRoZSBsaW5lYXIgUFMgd2l0aG91dCByZXBsYWNlbWVudCIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAiMToyIGdyZWVkeSBtYXRjaGluZyB3aXRoIHJlcGxhY2VtZW50IikpDQoNCmhvc3BkYXlzX21hdGNoX3Jlc3VsdHMNCmBgYA0KDQoNCmBgYHtyfQ0KZ2dwbG90KGhvc3BkYXlzX21hdGNoX3Jlc3VsdHMsIGFlcyh4ID0gYXBwcm9hY2gsIHkgPSBlc3RpbWF0ZSwgeW1pbiA9IGNvbmYubG93LCB5bWF4ID0gY29uZi5oaWdoKSkgKw0KICAgIGdlb21fdGV4dChhZXMobGFiZWwgPSByb3VuZF9oYWxmX3VwKGVzdGltYXRlLDIpKSwgdmp1c3QgPSAtMS4yNSkgKw0KICAgIGdlb21fcG9pbnRyYW5nZSgpICsNCiAgICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSAwLCBjb2wgPSAicmVkIiwgbGluZXR5cGUgPSAiZGFzaGVkIikgKw0KICAgIHRoZW1lX2J3KCkgKw0KICAgIGNvb3JkX2ZsaXAoKSArDQogICAgbGFicyh0aXRsZSA9ICJDb21wYXJpbmcgQVRUIEVzdGltYXRlcyBmb3IgTE9TIHVzaW5nIFJIQyBQcm9wZW5zaXR5IE1hdGNoaW5nIiwNCiAgICAgICAgIHggPSAiUHJvcGVuc2l0eSBBZGp1c3RtZW50IEFwcHJvYWNoIiwgDQogICAgICAgICB5ID0gIkVzdGltYXRlZCBDaGFuZ2UgaW4gSG9zcGl0YWwgTGVuZ3RoIG9mIFN0YXkgYXNzb2NpYXRlZCB3aXRoIFJIQyIpDQpgYGANCg0KTGV0J3MgbG9vayBhdCBqdXN0IHRoZSBmb3VyIHN0cmF0ZWdpZXMgdGhhdCBhY2hpZXZlZCBzdHJvbmdlciBiYWxhbmNlLg0KDQpgYGB7cn0NCmhvc3BkYXlzX21hdGNoX3Jlc3VsdHMgJT4lDQogICAgZmlsdGVyKGFwcHJvYWNoICVpbiUgYygiTWF0Y2ggMyIsICJNYXRjaCA0IiwgIk1hdGNoIDUiLCAiTWF0Y2ggNiIpKSAlPiUNCmdncGxvdCguLCBhZXMoeCA9IGRlc2NyaXB0aW9uLCB5ID0gZXN0aW1hdGUsIHltaW4gPSBjb25mLmxvdywgeW1heCA9IGNvbmYuaGlnaCkpICsNCiAgICBnZW9tX3RleHQoYWVzKGxhYmVsID0gcm91bmRfaGFsZl91cChlc3RpbWF0ZSwyKSksIHZqdXN0ID0gLTEuMjUpICsNCiAgICBnZW9tX3BvaW50cmFuZ2UoKSArDQogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMCwgY29sID0gInJlZCIsIGxpbmV0eXBlID0gImRhc2hlZCIpICsNCiAgICB0aGVtZV9idygpICsNCiAgICBjb29yZF9mbGlwKCkgKw0KICAgIGxhYnModGl0bGUgPSAiQVRUIE1hdGNoaW5nIEVzdGltYXRlcyBpbiBSSEMgKExPUykiLA0KICAgICAgICAgeCA9ICJQcm9wZW5zaXR5IEFkanVzdG1lbnQgQXBwcm9hY2giLCANCiAgICAgICAgIHkgPSAiRXN0LiBDaGFuZ2UgaW4gSG9zcGl0YWwgTE9TIGFzc29jaWF0ZWQgd2l0aCBSSEMiKQ0KYGBgDQoNCg0KIyBBVFQgTWF0Y2hpbmcgRXN0aW1hdGVzIGZvciB0aGUgVGltZS10by1FdmVudCBPdXRjb21lLCBJbi1TdHVkeSBTdXJ2aXZhbA0KDQpXZSdsbCB1c2UgYSBzdHJhdGlmaWVkIENveCBwcm9wb3J0aW9uYWwgaGF6YXJkcyBtb2RlbCB0byBjb21wYXJlIHRoZSBSSEMvTm8gUkhDIGdyb3VwcyBvbiBvdXIgdGltZS10by1ldmVudCBvdXRjb21lIGBzdXJ2ZGF5c2AsIHdoaWxlIGFjY291bnRpbmcgZm9yIHRoZSBtYXRjaGVkIHBhaXJzLiBUaGUgbWFpbiByZXN1bHQgd2lsbCBiZSBhIHJlbGF0aXZlIGhhemFyZCByYXRlIGVzdGltYXRlLCB3aXRoIDk1JSBDSS4gSSB3aWxsIHVzZSB0aGUgMC8xIG51bWVyaWMgdmVyc2lvbnMgb2YgdGhlIGV2ZW50IGluZGljYXRvciAoYGRpZWRgKSwgYW5kIG9mIHRoZSB0cmVhdG1lbnQgaW5kaWNhdG9yIChgdHJlYXRfcmhjYCkuDQoNCiMjIFdpdGggTWF0Y2ggMSwgaW4gZGV0YWlsDQoNCmBgYHtyfQ0Kc3VydmRheXNfYWRqX20xIDwtIGNveHBoKFN1cnYoc3VydmRheXMsIGRpZWQpIH4gdHJlYXRfcmhjICsgc3RyYXRhKG1hdGNoZXNfMSksIGRhdGE9cmhjLm1hdGNoZXNfMSkNCnN1bW1hcnkoc3VydmRheXNfYWRqX20xKQ0KYGBgDQoNClRoZSBoYXphcmQgcmF0aW8gaW4gdGhlIGBleHAoY29lZilgIHNlY3Rpb24gaXMgdGhlIGF2ZXJhZ2UgY2F1c2FsIGVmZmVjdCBlc3RpbWF0ZS4gSXQgZGVzY3JpYmVzIHRoZSBoYXphcmQgZm9yIGhhdmluZyBhbiBldmVudCAoYGRpZWRgKSBvY2N1ciBhc3NvY2lhdGVkIHdpdGggYmVpbmcgYSBSSEMgc3ViamVjdCAoYXMgaWRlbnRpZmllZCBieSBgdHJlYXRfcmhjYCksIGFzIGNvbXBhcmVkIHRvIHRoZSBoYXphcmQgb2YgdGhhdCBldmVudCB3aGVuIGEgbm9uLVJIQyBzdWJqZWN0LiBXZSBjYW4gdGlkeSB0aGlzIHdpdGg6DQoNCmBgYHtyfQ0KdGlkeShzdXJ2ZGF5c19hZGpfbTEsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIGNvbmYuaW50ID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpDQpgYGANCg0KIyMgV2l0aCBNYXRjaGVzIDItNiwgaW4gbXVjaCBsZXNzIGRldGFpbA0KDQpgYGB7cn0NCnN1cnZkYXlzX2Fkal9tMiA8LSBjb3hwaChTdXJ2KHN1cnZkYXlzLCBkaWVkKSB+IHRyZWF0X3JoYyArIHN0cmF0YShtYXRjaGVzXzIpLCBkYXRhPXJoYy5tYXRjaGVzXzIpDQp0aWR5KHN1cnZkYXlzX2Fkal9tMiwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkNCmBgYA0KDQpgYGB7cn0NCnN1cnZkYXlzX2Fkal9tMyA8LSBjb3hwaChTdXJ2KHN1cnZkYXlzLCBkaWVkKSB+IHRyZWF0X3JoYyArIHN0cmF0YShtYXRjaGVzXzMpLCBkYXRhPXJoYy5tYXRjaGVzXzMpDQp0aWR5KHN1cnZkYXlzX2Fkal9tMywgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkNCmBgYA0KDQpgYGB7cn0NCnN1cnZkYXlzX2Fkal9tNCA8LSBjb3hwaChTdXJ2KHN1cnZkYXlzLCBkaWVkKSB+IHRyZWF0X3JoYyArIHN0cmF0YShtYXRjaGVzXzQpLCBkYXRhPXJoYy5tYXRjaGVzXzQpDQp0aWR5KHN1cnZkYXlzX2Fkal9tNCwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkNCmBgYA0KDQpgYGB7cn0NCnN1cnZkYXlzX2Fkal9tNSA8LSBjb3hwaChTdXJ2KHN1cnZkYXlzLCBkaWVkKSB+IHRyZWF0X3JoYyArIHN0cmF0YShtYXRjaGVzXzUpLCBkYXRhPXJoYy5tYXRjaGVzXzUpDQp0aWR5KHN1cnZkYXlzX2Fkal9tNSwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkNCmBgYA0KDQpgYGB7cn0NCnN1cnZkYXlzX2Fkal9tNiA8LSBjb3hwaChTdXJ2KHN1cnZkYXlzLCBkaWVkKSB+IHRyZWF0X3JoYyArIHN0cmF0YShtYXRjaGVzXzYpLCBkYXRhPXJoYy5tYXRjaGVzXzYpDQp0aWR5KHN1cnZkYXlzX2Fkal9tNiwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkNCmBgYA0KDQoNCiMjIEdyYXBoaW5nIHRoZSBSZXN1bHRzIEFjcm9zcyBNdWx0aXBsZSBNYXRjaGluZyBTdHJhdGVnaWVzDQoNCmBgYHtyfQ0KdGVtcDAgPC0gdGlkeShzdXJ2ZGF5c191bmFkaiwgY29uZi5pbnQgPSBUUlVFLCBleHBvbmVudGlhdGUgPSBUUlVFKQ0KDQp0ZW1wMSA8LSB0aWR5KHN1cnZkYXlzX2Fkal9tMSwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkNCg0KdGVtcDIgPC0gdGlkeShzdXJ2ZGF5c19hZGpfbTIsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIGNvbmYuaW50ID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpDQoNCnRlbXAzIDwtIHRpZHkoc3VydmRheXNfYWRqX20zLCBleHBvbmVudGlhdGUgPSBUUlVFLCBjb25mLmludCA9IFRSVUUsIGNvbmYubGV2ZWwgPSAwLjk1KQ0KDQp0ZW1wNCA8LSB0aWR5KHN1cnZkYXlzX2Fkal9tNCwgZXhwb25lbnRpYXRlID0gVFJVRSwgY29uZi5pbnQgPSBUUlVFLCBjb25mLmxldmVsID0gMC45NSkNCg0KdGVtcDUgPC0gdGlkeShzdXJ2ZGF5c19hZGpfbTUsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIGNvbmYuaW50ID0gVFJVRSwgY29uZi5sZXZlbCA9IDAuOTUpDQoNCnRlbXA2IDwtIHRpZHkoc3VydmRheXNfYWRqX202LCBleHBvbmVudGlhdGUgPSBUUlVFLCBjb25mLmludCA9IFRSVUUsIGNvbmYubGV2ZWwgPSAwLjk1KQ0KDQoNCnN1cnZkYXlzX21hdGNoX3Jlc3VsdHMgPC0gcmJpbmQodGVtcDAsIHRlbXAxLCB0ZW1wMiwgdGVtcDMsIHRlbXA0LCB0ZW1wNSwgdGVtcDYpDQoNCnN1cnZkYXlzX21hdGNoX3Jlc3VsdHMgPC0gc3VydmRheXNfbWF0Y2hfcmVzdWx0cyAlPiUNCiAgICBtdXRhdGUoYXBwcm9hY2ggPSBjKCJVbm1hdGNoZWQiLCAiTWF0Y2ggMSIsICJNYXRjaCAyIiwgIk1hdGNoIDMiLCAiTWF0Y2ggNCIsICJNYXRjaCA1IiwgIk1hdGNoIDYiKSkgJT4lDQogICAgbXV0YXRlKGRlc2NyaXB0aW9uID0gYygiTm8gTWF0Y2hpbmciLCAiMToxIGdyZWVkeSBtYXRjaGluZyB3aXRob3V0IHJlcGxhY2VtZW50IiwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAiMToyIGdyZWVkeSBtYXRjaGluZyB3aXRob3V0IHJlcGxhY2VtZW50IiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICIxOjEgZ2VuZXRpYyBzZWFyY2ggbWF0Y2hpbmcgd2l0aG91dCByZXBsYWNlbWVudCIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAiMToxIGdyZWVkeSBtYXRjaGluZyB3aXRoIHJlcGxhY2VtZW50IiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICIxOjEgY2FsaXBlciBtYXRjaGluZyBvbiB0aGUgbGluZWFyIFBTIHdpdGhvdXQgcmVwbGFjZW1lbnQiLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgIjE6MiBncmVlZHkgbWF0Y2hpbmcgd2l0aCByZXBsYWNlbWVudCIpKQ0KDQpzdXJ2ZGF5c19tYXRjaF9yZXN1bHRzDQpgYGANCg0KDQpgYGB7cn0NCmdncGxvdChzdXJ2ZGF5c19tYXRjaF9yZXN1bHRzLCBhZXMoeCA9IGFwcHJvYWNoLCB5ID0gZXN0aW1hdGUsIHltaW4gPSBjb25mLmxvdywgeW1heCA9IGNvbmYuaGlnaCkpICsNCiAgICBnZW9tX3RleHQoYWVzKGxhYmVsID0gcm91bmRfaGFsZl91cChlc3RpbWF0ZSwyKSksIHZqdXN0ID0gLTEuMjUpICsNCiAgICBnZW9tX3BvaW50cmFuZ2UoKSArDQogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMSwgY29sID0gInJlZCIsIGxpbmV0eXBlID0gImRhc2hlZCIpICsNCiAgICB0aGVtZV9idygpICsNCiAgICBjb29yZF9mbGlwKCkgKw0KICAgIGxhYnModGl0bGUgPSAiQ29tcGFyaW5nIEFUVCBFc3RpbWF0ZXMgZm9yIFN1cnZpdmFsIHVzaW5nIFJIQyBQcm9wZW5zaXR5IE1hdGNoaW5nIiwNCiAgICAgICAgIHggPSAiUHJvcGVuc2l0eSBBZGp1c3RtZW50IEFwcHJvYWNoIiwgDQogICAgICAgICB5ID0gIkVzdGltYXRlZCBIYXphcmQgUmF0aW8gYXNzb2NpYXRlZCB3aXRoIFJIQyIpDQpgYGANCg0KTGV0J3MgbG9vayBhdCBqdXN0IHRoZSBmb3VyIHN0cmF0ZWdpZXMgdGhhdCBhY2hpZXZlZCBzdHJvbmdlciBiYWxhbmNlLg0KDQpgYGB7cn0NCnN1cnZkYXlzX21hdGNoX3Jlc3VsdHMgJT4lDQogICAgZmlsdGVyKGFwcHJvYWNoICVpbiUgYygiTWF0Y2ggMyIsICJNYXRjaCA0IiwgIk1hdGNoIDUiLCAiTWF0Y2ggNiIpKSAlPiUNCmdncGxvdCguLCBhZXMoeCA9IGRlc2NyaXB0aW9uLCB5ID0gZXN0aW1hdGUsIHltaW4gPSBjb25mLmxvdywgeW1heCA9IGNvbmYuaGlnaCkpICsNCiAgICBnZW9tX3RleHQoYWVzKGxhYmVsID0gcm91bmRfaGFsZl91cChlc3RpbWF0ZSwyKSksIHZqdXN0ID0gLTEuMjUpICsNCiAgICBnZW9tX3BvaW50cmFuZ2UoKSArDQogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMSwgY29sID0gInJlZCIsIGxpbmV0eXBlID0gImRhc2hlZCIpICsNCiAgICB0aGVtZV9idygpICsNCiAgICBjb29yZF9mbGlwKCkgKw0KICAgIGxhYnModGl0bGUgPSAiQVRUIE1hdGNoaW5nIEVzdGltYXRlcyBpbiBSSEMgKFN1cnZpdmFsKSIsDQogICAgICAgICB4ID0gIlByb3BlbnNpdHkgQWRqdXN0bWVudCBBcHByb2FjaCIsIA0KICAgICAgICAgeSA9ICJFc3RpbWF0ZWQgSGF6YXJkIFJhdGlvIGFzc29jaWF0ZWQgd2l0aCBSSEMiKQ0KYGBgDQoNCg0KDQojIFByb3BlbnNpdHkgU2NvcmUgV2VpZ2h0aW5nIChBVFQgYXBwcm9hY2gpDQoNCiMjIEVzdGltYXRlIHRoZSBQcm9wZW5zaXR5IFNjb3JlIHVzaW5nIEdlbmVyYWxpemVkIEJvb3N0ZWQgUmVncmVzc2lvbiwgYW5kIHRoZW4gcGVyZm9tIEFUVCBXZWlnaHRpbmcNCg0KVG8gYmVnaW4sIHdlJ2xsIHJlLWVzdGltYXRlIHRoZSBwcm9wZW5zaXR5IHNjb3JlIHVzaW5nIHRoZSBgdHdhbmdgIGZ1bmN0aW9uIGBwc2AuIFRoaXMgdXNlcyBhICpnZW5lcmFsaXplZCBib29zdGVkIHJlZ3Jlc3Npb24qIGFwcHJvYWNoIHRvIGVzdGltYXRlIHRoZSBwcm9wZW5zaXR5IHNjb3JlIGFuZCBwcm9kdWNlIG1hdGVyaWFsIGZvciBjaGVja2luZyBiYWxhbmNlLg0KDQpgYGB7ciwgd2FybmluZyA9IEZBTFNFfQ0KIyBSZWNhbGwgdGhhdCB0d2FuZyBkb2VzIG5vdCBwbGF5IHdlbGwgd2l0aCB0aWJibGVzLA0KIyBzbyB3ZSBoYXZlIHRvIHVzZSB0aGUgZGF0YSBmcmFtZSB2ZXJzaW9uIG9mIHJoYyBvYmplY3QNCiMgYW5kIEknbSB1c2luZyB0aGUgMS8wIHZlcnNpb24gb2YgdGhlIHRyZWF0bWVudA0KDQpyaGNfZGYgPC0gYmFzZTo6YXMuZGF0YS5mcmFtZShyaGMpDQoNCnBzX3JoYyA8LSBwcyh0cmVhdF9yaGMgfiANCiAgICAgICAgICAgICAgICAgYWdlICsgc2V4ICsgZWR1ICsgaW5jb21lICsgbmluc2NsYXMgKyANCiAgICAgICAgICAgICAgICAgcmFjZSArIGNhdDEgKyBkbnIxICsgd3RraWxvMSArIGhydDEgKyANCiAgICAgICAgICAgICAgICAgbWVhbmJwMSArIHJlc3AxICsgdGVtcDEgKyBjYXJkICsgZ2FzdHIgKw0KICAgICAgICAgICAgICAgICBoZW1hICsgbWV0YSArIG5ldXJvICsgb3J0aG8gKyByZW5hbCArIA0KICAgICAgICAgICAgICAgICByZXNwICsgc2VwcyArIHRyYXVtYSArIGFtaWh4ICsgY2EgKyANCiAgICAgICAgICAgICAgICAgY2FyZGlvaHggKyBjaGZoeCArIGNocnB1bGh4ICsgZGVtZW50aHggKyANCiAgICAgICAgICAgICAgICAgZ2libGVkaHggKyBpbW11bmh4ICsgbGl2ZXJoeCArIG1hbGlnaHggKyANCiAgICAgICAgICAgICAgICAgcHN5Y2hoeCArIHJlbmFsaHggKyB0cmFuc2h4ICsgYXBzMSArIA0KICAgICAgICAgICAgICAgICBkYXMyZDNwYyArIHNjb21hMSArIHN1cnYybWQxICsgYWxiMSArIA0KICAgICAgICAgICAgICAgICBiaWxpMSArIGNyZWExICsgaGVtYTEgKyBwYWNvMjEgKyBwYWZpMSArDQogICAgICAgICAgICAgICAgIHBoMSArIHBvdDEgKyBzb2QxICsgd2JsYzEsDQogICAgICAgICAgICAgZGF0YSA9IHJoY19kZiwNCiAgICAgICAgICAgICBuLnRyZWVzID0gNDAwMCwNCiAgICAgICAgICAgICBpbnRlcmFjdGlvbi5kZXB0aCA9IDIsDQogICAgICAgICAgICAgc3RvcC5tZXRob2QgPSBjKCJlcy5tZWFuIiksDQogICAgICAgICAgICAgZXN0aW1hbmQgPSAiQVRUIiwNCiAgICAgICAgICAgICB2ZXJib3NlID0gRkFMU0UpDQpgYGANCg0KIyMjIERpZCB3ZSBsZXQgdGhlIHNpbXVsYXRpb25zIHJ1biBsb25nIGVub3VnaCB0byBzdGFiaWxpemUgZXN0aW1hdGVzPw0KDQpgYGB7cn0NCnBsb3QocHNfcmhjKQ0KYGBgDQoNCiMjIyBXaGF0IGlzIHRoZSBlZmZlY3RpdmUgc2FtcGxlIHNpemUgb2Ygb3VyIHdlaWdodGVkIHJlc3VsdHM/DQoNCmBgYHtyfQ0Kc3VtbWFyeShwc19yaGMpDQpgYGANCg0KIyMjIEhvdyBpcyB0aGUgYmFsYW5jZT8NCg0KYGBge3J9DQpwbG90KHBzX3JoYywgcGxvdHMgPSAyKQ0KYGBgDQoNCmBgYHtyfQ0KcGxvdChwc19yaGMsIHBsb3RzID0gMykNCmBgYA0KDQojIyMgQXNzZXNzaW5nIEJhbGFuY2Ugd2l0aCBgY29iYWx0YA0KDQpgYGB7cn0NCmIyIDwtIGJhbC50YWIocHNfcmhjLCBmdWxsLnN0b3AubWV0aG9kID0gImVzLm1lYW4uYXR0IiwgDQogICAgICAgIHN0YXRzID0gYygibSIsICJ2IiksIHVuID0gVFJVRSkNCg0KYjINCmBgYA0KDQojIyBSdWJpbidzIFJ1bGUgMSBBZnRlciBXZWlnaHRpbmcNCg0KV2UgY291bGQgc2ltcGx5IGNvbXBhcmUgdGhlIHByb3BlbnNpdHkgc2NvcmUgYmFsYW5jZSBhcyByZXBvcnRlZCBpbiB0aGUgdGFibGUgYWJvdmUsIGxvb2tpbmcgYXQgdGhlIHN0YW5kYXJkaXplZCBiaWFzZXMgKHRoZSBzdGFuZGFyZGl6ZWQgZGlmZmVyZW5jZSBvbiBhIHByb3BvcnRpb24gc2NhbGUsIHJhdGhlciB0aGFuIGEgcGVyY2VudGFnZS4pIEknbGwgbXVsdGlwbHkgZWFjaCBieSAxMDAgdG8gcGxhY2UgdGhpbmdzIG9uIHRoZSB1c3VhbCBzY2FsZS4NCg0KYGBge3J9DQoxMDAqYjJbMV0kQmFsYW5jZSREaWZmLlVuWzFdICMgaW1iYWxhbmNlIHByaW9yIHRvIHdlaWdodGluZw0KMTAwKmIyWzFdJEJhbGFuY2UkRGlmZi5BZGpbMV0gIyBpbWJhbGFuY2UgYWZ0ZXIgd2VpZ2h0aW5nDQpgYGANCg0KU28gdGhpcyBpcyBzdGlsbCBpbmRpY2F0aXZlIG9mIHNvbWUgZGlmZmljdWx0eSByZWFjaGluZyBhIGNvbXBsZXRlbHkgcmVhc29uYWJsZSBsZXZlbCBvZiBiYWxhbmNlLiBXZSdkIGxpa2UgdG8gc2VlIHRoaXMgbXVjaCBjbG9zZXIgdG8gMC4NCg0KIyMgUnViaW4ncyBSdWxlIDIgQWZ0ZXIgV2VpZ2h0aW5nDQoNCmBgYHtyfQ0KYjJbMV0kQmFsYW5jZSRWLlJhdGlvLlVuWzFdICMgaW1iYWxhbmNlIHByaW9yIHRvIHdlaWdodGluZw0KYjJbMV0kQmFsYW5jZSRWLlJhdGlvLkFkalsxXSAjIGltYmFsYW5jZSBhZnRlciB3ZWlnaHRpbmcNCmBgYA0KDQpTbyB0aGVyZSdzIG5vIHBhcnRpY3VsYXIgcHJvYmxlbSB3aXRoIFJ1bGUgMiBoZXJlLg0KDQojIyBTZW1pLUF1dG9tYXRlZCBMb3ZlIHBsb3Qgb2YgU3RhbmRhcmRpemVkIERpZmZlcmVuY2VzDQoNCmBgYHtyLCBmaWcuaGVpZ2h0ID0gOH0NCnAgPC0gbG92ZS5wbG90KGIyLCANCiAgICAgICAgICAgICAgIHRocmVzaG9sZCA9IC4xLCBzaXplID0gMywgc3RhcnMgPSAicmF3IiwNCiAgICAgICAgICAgICAgIHRpdGxlID0gIlN0YW5kYXJkaXplZCBEaWZmcyBhbmQgQVRUIHdlaWdodHMiKQ0KcCArIHRoZW1lX2J3KCkNCmBgYA0KDQojIyBTZW1pLUF1dG9tYXRlZCBMb3ZlIHBsb3Qgb2YgVmFyaWFuY2UgUmF0aW9zDQoNCmBgYHtyLCBmaWcuaGVpZ2h0ID0gOH0NCnAgPC0gbG92ZS5wbG90KGIyLCBzdGF0ID0gInYiLA0KICAgICAgICAgICAgICAgdGhyZXNob2xkID0gMS4yNSwgc2l6ZSA9IDMsIA0KICAgICAgICAgICAgICAgdGl0bGUgPSAiVmFyaWFuY2UgUmF0aW9zOiBUV0FORyBBVFQgd2VpZ2h0aW5nIikNCnAgKyB0aGVtZV9idygpDQpgYGANCg0KIyBBVFQgV2VpZ2h0aW5nIEVzdGltYXRlcyANCg0KV2UncmUgbm90IHRvbyBleGNpdGVkIGFib3V0IHdlaWdodGVkIGVzdGltYXRlcyB3aXRob3V0IChhdCBsZWFzdCkgYW4gYWRkaXRpb25hbCBhZGp1c3RtZW50IGZvciBkaWZmZXJlbmNlcyBpbiB0aGUgcHJvcGVuc2l0eSBzY29yZSBpbiB0aGlzIGNhc2UsIGJ1dCBJJ2xsIHJ1biB0aGVtIGFueXdheSBqdXN0IHRvIHNob3cgaG93IHRoYXQgd291bGQgYmUgZG9uZS4NCg0KIyMgV2VpZ2h0ZWQgQVRUIGZvciBPdXRjb21lIEE6IERlYXRoDQoNClRoZSByZWxldmFudCByZWdyZXNzaW9uIGFwcHJvYWNoIHVzZXMgdGhlIGBzdnlkZXNpZ25gIGFuZCBgc3Z5Z2xtYCBmdW5jdGlvbnMgZnJvbSB0aGUgYHN1cnZleWAgcGFja2FnZS4gRm9yIGEgYmluYXJ5IG91dGNvbWUsIHdlIGJ1aWxkIHRoZSBvdXRjb21lIG1vZGVsIHVzaW5nIHRoZSBxdWFzaWJpbm9taWFsLCByYXRoZXIgdGhhbiB0aGUgdXN1YWwgYmlub21pYWwgZmFtaWx5LiBOb3RlIHRoZSB1c2Ugb2YgdGhlIDEvMCB2ZXJzaW9uIG9mIGJvdGggdGhlIG91dGNvbWUgYW5kIHRoZSB0cmVhdG1lbnQgdmFyaWFibGVzIGhlcmUuDQoNCmBgYHtyfQ0Kcmhjd3RzX2Rlc2lnbiA8LSBzdnlkZXNpZ24oDQogICAgaWRzPX4xLCB3ZWlnaHRzPX5nZXQud2VpZ2h0cyhwc19yaGMsIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RvcC5tZXRob2QgPSAiZXMubWVhbiIpLA0KICAgIGRhdGE9cmhjX2RmKSAjIHVzaW5nIHR3YW5nIEFUVCB3ZWlnaHRzDQoNCg0KYWRqX2RlYXRoX3d0ZCA8LSBzdnlnbG0oZGllZCB+IHRyZWF0X3JoYywgDQogICAgICAgICAgICAgICAgICAgICAgICBkZXNpZ249cmhjd3RzX2Rlc2lnbiwNCiAgICAgICAgICAgICAgICAgICAgICBmYW1pbHk9cXVhc2liaW5vbWlhbCgpKQ0KDQp3dF90d2FuZ2F0dF9yZXNfZGVhdGggPC0gDQogICAgdGlkeShhZGpfZGVhdGhfd3RkLCBleHBvbmVudGlhdGUgPSBUUlVFLA0KICAgICAgICAgY29uZi5pbnQgPSBUUlVFLA0KICAgICAgICAgY29uZi5sZXZlbCA9IDAuOTUpICU+JSANCiAgICBzZWxlY3QodGVybSwgZXN0aW1hdGUsIHN0ZC5lcnJvciwNCiAgICAgICAgICAgbG85NSA9IGNvbmYubG93LCBoaTk1ID0gY29uZi5oaWdoKQ0KDQp3dF90d2FuZ2F0dF9yZXNfZGVhdGgNCmBgYA0KDQpUaGUgZXN0aW1hdGVzIHdlIHdhbnQgY29tZSBmcm9tIHRoZSBgdHJlYXRfcmhjYCByb3cuDQoNCiMjIERvdWJsZSBSb2J1c3QgV2VpZ2h0ZWQgRXN0aW1hdGUgZm9yIERlYXRoDQoNCk5vdyBsZXQncyBhZGQgaW4gYSByZWdyZXNzaW9uIGFkanVzdG1lbnQgZm9yIHRoZSBsaW5lYXIgcHJvcGVuc2l0eSBzY29yZSwgd2hpY2ggc2hvdWxkIGFsbGV2aWF0ZSBzb21lIG9mIG91ciBjb25jZXJucyBhYm91dCByZXNpZHVhbCBpbWJhbGFuY2UgYWZ0ZXIgd2VpZ2h0aW5nLg0KDQpgYGB7cn0NCmRyX2Fkal9kZWF0aCA8LSBzdnlnbG0oZGllZCB+IHRyZWF0X3JoYyArIGxpbnBzLCANCiAgICAgICAgICAgICAgICAgICAgICAgIGRlc2lnbj1yaGN3dHNfZGVzaWduLA0KICAgICAgICAgICAgICAgICAgICAgIGZhbWlseT1xdWFzaWJpbm9taWFsKCkpDQoNCmRyX2RlYXRoIDwtIA0KICAgIHRpZHkoZHJfYWRqX2RlYXRoLCBleHBvbmVudGlhdGUgPSBUUlVFLA0KICAgICAgICAgY29uZi5pbnQgPSBUUlVFLA0KICAgICAgICAgY29uZi5sZXZlbCA9IDAuOTUpICU+JSANCiAgICBzZWxlY3QodGVybSwgZXN0aW1hdGUsIHN0ZC5lcnJvciwNCiAgICAgICAgICAgbG85NSA9IGNvbmYubG93LCBoaTk1ID0gY29uZi5oaWdoKQ0KDQpkcl9kZWF0aA0KYGBgDQoNCkFnYWluLCB0aGUgZXN0aW1hdGVzIHdlIHdhbnQgY29tZSBmcm9tIHRoZSBgdHJlYXRfcmhjYCByb3cuDQoNCiMjIEdyYXBoaW5nIHRoZSBXZWlnaHRlZCBhbmQgTWF0Y2hlZCBSZXN1bHRzIGZvciBPdXRjb21lIEENCg0KYGBge3J9DQp0ZW1wNyA8LSANCiAgICB0aWR5KGFkal9kZWF0aF93dGQsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIA0KICAgICAgICAgICAgICBjb25mLmludCA9IFQsIGNvbmYubGV2ZWwgPSAwLjk1KSAlPiUNCiAgICBmaWx0ZXIodGVybSA9PSAidHJlYXRfcmhjIikNCg0KdGVtcDggPC0gDQogICAgdGlkeShkcl9hZGpfZGVhdGgsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIA0KICAgICAgICAgICAgICBjb25mLmludCA9IFQsIGNvbmYubGV2ZWwgPSAwLjk1KSAlPiUNCiAgICBmaWx0ZXIodGVybSA9PSAidHJlYXRfcmhjIikNCg0KZGVhdGhfbmV3X3Jlc3VsdHMgPC0gcmJpbmQodGVtcDcsIHRlbXA4KSAlPiUNCiAgICBtdXRhdGUoYXBwcm9hY2ggPSBjKCJBVFQgd2VpZ2h0ZWQiLCAiRFIiKSkgJT4lDQogICAgbXV0YXRlKGRlc2NyaXB0aW9uID0gYygiQVRUIHdlaWdodHMiLCAiRFI6IEFUVCB3dHMgKyBhZGoiKSkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICANCmRlYXRoX3Jlc3VsdHMgPC0gcmJpbmQoZGVhdGhfbWF0Y2hfcmVzdWx0cywgDQogICAgICAgICAgICAgICAgICAgICAgIGRlYXRoX25ld19yZXN1bHRzKQ0KDQpkZWF0aF9yZXN1bHRzDQpgYGANCg0KU28gdGhlcmUncyBvdXIgdGFibGUuIEl0J3Mgd29ydGggcmVtZW1iZXJpbmcgdGhhdCB0aGUgb25seSBhbmFseXNlcyB0aGF0IHdlJ3ZlIA0KDQpgYGB7cn0NCmRlYXRoX3Jlc3VsdHMgJT4lDQogICAgZ2dwbG90KC4sIGFlcyh4ID0gZGVzY3JpcHRpb24sIHkgPSBlc3RpbWF0ZSwgDQogICAgICAgICAgICAgICAgICB5bWluID0gY29uZi5sb3csIHltYXggPSBjb25mLmhpZ2gpKSArDQogICAgZ2VvbV90ZXh0KGFlcyhsYWJlbCA9IHJvdW5kX2hhbGZfdXAoZXN0aW1hdGUsMikpLCB2anVzdCA9IC0xLjI1KSArDQogICAgZ2VvbV9wb2ludHJhbmdlKCkgKw0KICAgIGdlb21faGxpbmUoeWludGVyY2VwdCA9IDEsIGNvbCA9ICJyZWQiLCBsaW5ldHlwZSA9ICJkYXNoZWQiKSArDQogICAgdGhlbWVfYncoKSArDQogICAgY29vcmRfZmxpcCgpICsNCiAgICBsYWJzKHRpdGxlID0gIkFUVCBFc3RpbWF0ZXMgaW4gUkhDIChEZWF0aCkiLA0KICAgICAgICAgeCA9ICJQcm9wZW5zaXR5IEFkanVzdG1lbnQgQXBwcm9hY2giLCANCiAgICAgICAgIHkgPSAiRXN0aW1hdGVkIE9kZHMgUmF0aW8gZm9yIERlYXRoIGFzc29jaWF0ZWQgd2l0aCBSSEMiKQ0KYGBgDQoNCkFuZCBoZXJlIGFyZSB0aGUgcmVzdWx0cyBmb3Igb25seSB0aGUgZml2ZSBtZXRob2RzIChhZGRpbmcgb3VyIGRvdWJsZSByb2J1c3QgYXBwcm9hY2ggdG8gdGhlIGFuYWx5c2VzIHdlIGxpa2VkIGluIHRoZSBtYXRjaGluZykgd2hpY2ggc2hvd2VkIHJlYXNvbmFibGUgYmFsYW5jZSBhZnRlciBwcm9wZW5zaXR5IHNjb3JlIGFkanVzdG1lbnQuLi4NCg0KYGBge3J9DQpkZWF0aF9yZXN1bHRzICU+JQ0KICAgIGZpbHRlcihhcHByb2FjaCAlaW4lIGMoIk1hdGNoIDMiLCAiTWF0Y2ggNCIsICJNYXRjaCA1IiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAiTWF0Y2ggNiIsICJEUiIpKSAlPiUNCiAgICBnZ3Bsb3QoLiwgYWVzKHggPSBkZXNjcmlwdGlvbiwgeSA9IGVzdGltYXRlLCANCiAgICAgICAgICAgICAgICAgIHltaW4gPSBjb25mLmxvdywgeW1heCA9IGNvbmYuaGlnaCkpICsNCiAgICBnZW9tX3RleHQoYWVzKGxhYmVsID0gcm91bmRfaGFsZl91cChlc3RpbWF0ZSwyKSksIHZqdXN0ID0gLTEuMjUpICsNCiAgICBnZW9tX3BvaW50cmFuZ2UoKSArDQogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMSwgY29sID0gInJlZCIsIGxpbmV0eXBlID0gImRhc2hlZCIpICsNCiAgICB0aGVtZV9idygpICsNCiAgICBjb29yZF9mbGlwKCkgKw0KICAgIGxhYnModGl0bGUgPSAiQVRUIEVzdGltYXRlcyBpbiBSSEMgKERlYXRoKSIsDQogICAgICAgICB4ID0gIlByb3BlbnNpdHkgQWRqdXN0bWVudCBBcHByb2FjaCIsIA0KICAgICAgICAgeSA9ICJFc3RpbWF0ZWQgT2RkcyBSYXRpbyBmb3IgRGVhdGggYXNzb2NpYXRlZCB3aXRoIFJIQyIpDQpgYGANCg0KDQojIyBXZWlnaHRlZCBBVFQgZm9yIE91dGNvbWUgQjogSG9zcGl0YWwgTGVuZ3RoIG9mIFN0YXkNCg0KVGhlIHJlbGV2YW50IHJlZ3Jlc3Npb24gYXBwcm9hY2ggdXNlcyB0aGUgYHN2eWRlc2lnbmAgYW5kIGBzdnlnbG1gIGZ1bmN0aW9ucyBmcm9tIHRoZSBgc3VydmV5YCBwYWNrYWdlLg0KDQpgYGB7cn0NCnJoY3d0c19kZXNpZ24gPC0gc3Z5ZGVzaWduKA0KICAgIGlkcz1+MSwgd2VpZ2h0cz1+Z2V0LndlaWdodHMocHNfcmhjLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0b3AubWV0aG9kID0gImVzLm1lYW4iKSwNCiAgICBkYXRhPXJoY19kZikgIyB1c2luZyB0d2FuZyBBVFQgd2VpZ2h0cw0KDQphZGpfaG9zcGRheXNfd3RkIDwtIA0KICAgIHN2eWdsbShob3NwZGF5cyB+IHRyZWF0X3JoYywgDQogICAgICAgICAgIGRlc2lnbj0gcmhjd3RzX2Rlc2lnbikNCg0Kd3RfdHdhbmdhdHRfcmVzX2hvc3BkYXlzIDwtIA0KICAgIHRpZHkoYWRqX2hvc3BkYXlzX3d0ZCwgY29uZi5pbnQgPSBUUlVFLA0KICAgICAgICAgY29uZi5sZXZlbCA9IDAuOTUpICU+JSANCiAgICBzZWxlY3QodGVybSwgZXN0aW1hdGUsIHN0ZC5lcnJvciwNCiAgICAgICAgICAgbG85NSA9IGNvbmYubG93LCBoaTk1ID0gY29uZi5oaWdoKQ0KDQp3dF90d2FuZ2F0dF9yZXNfaG9zcGRheXMNCmBgYA0KDQpUaGUgZXN0aW1hdGVzIHdlIHdhbnQgY29tZSBmcm9tIHRoZSBgdHJlYXRfcmhjYCByb3cuDQoNCiMjIERvdWJsZSBSb2J1c3QgV2VpZ2h0ZWQgRXN0aW1hdGUgZm9yIExPUw0KDQpOb3cgbGV0J3MgYWRkIGluIGEgcmVncmVzc2lvbiBhZGp1c3RtZW50IGZvciB0aGUgbGluZWFyIHByb3BlbnNpdHkgc2NvcmUuIEFnYWluLCB3ZSBzaG91bGQgaGF2ZSBzb21lIGFkZGl0aW9uYWwgZmFpdGggaW4gdGhlc2UgcmVzdWx0cyBvdmVyIHRob3NlIHdpdGgganVzdCB3ZWlnaHRpbmcgYWxvbmUsIGJlY2F1c2Ugb2YgdGhlIGZhaWx1cmUgb2YgdGhlIHdlaWdodGluZyBhcHByb2FjaCB0byBwYXNzIFJ1YmluJ3MgUnVsZSAxLg0KDQpgYGB7cn0NCmRyX2Fkal9kYXlzIDwtIHN2eWdsbShob3NwZGF5cyB+IHRyZWF0X3JoYyArIGxpbnBzLCANCiAgICAgICAgICAgICAgICAgICAgICAgIGRlc2lnbj1yaGN3dHNfZGVzaWduKQ0KDQpkcl9sb3MgPC0gDQogICAgdGlkeShkcl9hZGpfZGF5cywgY29uZi5pbnQgPSBUUlVFLA0KICAgICAgICAgY29uZi5sZXZlbCA9IDAuOTUpICU+JSANCiAgICBzZWxlY3QodGVybSwgZXN0aW1hdGUsIHN0ZC5lcnJvciwNCiAgICAgICAgICAgbG85NSA9IGNvbmYubG93LCBoaTk1ID0gY29uZi5oaWdoKQ0KDQpkcl9sb3MNCmBgYA0KDQpPbmNlIGFnYWluLCB0aGUgZXN0aW1hdGVzIHdlIHdhbnQgY29tZSBmcm9tIHRoZSBgdHJlYXRfcmhjYCByb3cuDQoNCiMjIEdyYXBoaW5nIHRoZSBXZWlnaHRlZCBhbmQgTWF0Y2hlZCBSZXN1bHRzIGZvciBPdXRjb21lIEINCg0KYGBge3J9DQp0ZW1wOSA8LSANCiAgICB0aWR5KGFkal9ob3NwZGF5c193dGQsIGNvbmYuaW50ID0gVCwgDQogICAgICAgICBjb25mLmxldmVsID0gMC45NSkgJT4lDQogICAgZmlsdGVyKHRlcm0gPT0gInRyZWF0X3JoYyIpICU+JQ0KICAgIHNlbGVjdCgtc3RhdGlzdGljLCAtcC52YWx1ZSkNCg0KdGVtcDEwIDwtIA0KICAgIHRpZHkoZHJfYWRqX2RheXMsIGNvbmYuaW50ID0gVCwgDQogICAgICAgICBjb25mLmxldmVsID0gMC45NSkgJT4lDQogICAgZmlsdGVyKHRlcm0gPT0gInRyZWF0X3JoYyIpICU+JQ0KICAgIHNlbGVjdCgtc3RhdGlzdGljLCAtcC52YWx1ZSkNCg0KaG9zcGRheXNfbmV3X3Jlc3VsdHMgPC0gcmJpbmQodGVtcDksIHRlbXAxMCkgJT4lDQogICAgbXV0YXRlKGFwcHJvYWNoID0gYygiQVRUIHdlaWdodGVkIiwgIkRSIikpICU+JQ0KICAgIG11dGF0ZShkZXNjcmlwdGlvbiA9IGMoIkFUVCB3ZWlnaHRzIiwgIkRSOiBBVFQgd3RzICsgYWRqIikpDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgDQpob3NwZGF5c19yZXN1bHRzIDwtIHJiaW5kKGhvc3BkYXlzX21hdGNoX3Jlc3VsdHMsIA0KICAgICAgICAgICAgICAgICAgICAgICBob3NwZGF5c19uZXdfcmVzdWx0cykNCg0KaG9zcGRheXNfcmVzdWx0cw0KYGBgDQoNClNvIHRoZXJlJ3Mgb3VyIHRhYmxlLiANCg0KYGBge3J9DQpob3NwZGF5c19yZXN1bHRzICU+JQ0KICAgIGdncGxvdCguLCBhZXMoeCA9IGRlc2NyaXB0aW9uLCB5ID0gZXN0aW1hdGUsIA0KICAgICAgICAgICAgICAgICAgeW1pbiA9IGNvbmYubG93LCB5bWF4ID0gY29uZi5oaWdoKSkgKw0KICAgIGdlb21fdGV4dChhZXMobGFiZWwgPSByb3VuZF9oYWxmX3VwKGVzdGltYXRlLDIpKSwgdmp1c3QgPSAtMS4yNSkgKw0KICAgIGdlb21fcG9pbnRyYW5nZSgpICsNCiAgICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSAwLCBjb2wgPSAicmVkIiwgbGluZXR5cGUgPSAiZGFzaGVkIikgKw0KICAgIHRoZW1lX2J3KCkgKw0KICAgIGNvb3JkX2ZsaXAoKSArDQogICAgbGFicyh0aXRsZSA9ICJDb21wYXJpbmcgQVRUIEVzdGltYXRlcyBmb3IgTE9TIiwNCiAgICAgICAgIHggPSAiUHJvcGVuc2l0eSBBZGp1c3RtZW50IEFwcHJvYWNoIiwgDQogICAgICAgICB5ID0gIkVzdGltYXRlZCBDaGFuZ2UgaW4gSG9zcGl0YWwgTE9TIGFzc29jaWF0ZWQgd2l0aCBSSEMiKQ0KYGBgDQoNCkFuZCBoZXJlIGFyZSB0aGUgcmVzdWx0cyBmb3Igb25seSB0aGUgZml2ZSBtZXRob2RzIChhZGRpbmcgb3VyIGRvdWJsZSByb2J1c3QgYXBwcm9hY2ggdG8gdGhlIGFuYWx5c2VzIHdlIGxpa2VkIGluIHRoZSBtYXRjaGluZykgd2hpY2ggc2hvd2VkIHJlYXNvbmFibGUgYmFsYW5jZSBhZnRlciBwcm9wZW5zaXR5IHNjb3JlIGFkanVzdG1lbnQuLi4NCg0KYGBge3J9DQpob3NwZGF5c19yZXN1bHRzICU+JQ0KICAgIGZpbHRlcihhcHByb2FjaCAlaW4lIGMoIk1hdGNoIDMiLCAiTWF0Y2ggNCIsICJNYXRjaCA1IiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAiTWF0Y2ggNiIsICJEUiIpKSAlPiUNCiAgICBnZ3Bsb3QoLiwgYWVzKHggPSBkZXNjcmlwdGlvbiwgeSA9IGVzdGltYXRlLCANCiAgICAgICAgICAgICAgICAgIHltaW4gPSBjb25mLmxvdywgeW1heCA9IGNvbmYuaGlnaCkpICsNCiAgICBnZW9tX3RleHQoYWVzKGxhYmVsID0gcm91bmRfaGFsZl91cChlc3RpbWF0ZSwyKSksIHZqdXN0ID0gLTEuMjUpICsNCiAgICBnZW9tX3BvaW50cmFuZ2UoKSArDQogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMCwgY29sID0gInJlZCIsIGxpbmV0eXBlID0gImRhc2hlZCIpICsNCiAgICB0aGVtZV9idygpICsNCiAgICBjb29yZF9mbGlwKCkgKw0KICAgIGxhYnModGl0bGUgPSAiQ29tcGFyaW5nIEFUVCBFc3RpbWF0ZXMgZm9yIExPUyIsDQogICAgICAgICB4ID0gIlByb3BlbnNpdHkgQWRqdXN0bWVudCBBcHByb2FjaCIsIA0KICAgICAgICAgeSA9ICJFc3RpbWF0ZWQgQ2hhbmdlIGluIEhvc3BpdGFsIExPUyBhc3NvY2lhdGVkIHdpdGggUkhDIikNCmBgYA0KDQojIyBXZWlnaHRlZCBBVFQgZm9yIE91dGNvbWUgQzogSW4tU3R1ZHkgVGltZSB0byBEZWF0aA0KDQpgYGB7cn0NCnJoY193dHMgPC0gZ2V0LndlaWdodHMocHNfcmhjLCBzdG9wLm1ldGhvZCA9ICJlcy5tZWFuIikNCg0KYWRqX3N1cnZkYXlzX3d0ZCA8LSBjb3hwaChTdXJ2KHN1cnZkYXlzLCBkaWVkKSB+IHRyZWF0X3JoYywNCiAgICAgICAgICAgICAgICAgICAgICAgICAgZGF0YSA9IHJoY19kZiwgd2VpZ2h0cyA9IHJoY193dHMpDQoNCnd0X3R3YW5nYXR0X3Jlc19zdXJ2IDwtIA0KICAgIHRpZHkoYWRqX3N1cnZkYXlzX3d0ZCwgZXhwb25lbnRpYXRlID0gVFJVRSwNCiAgICAgICAgIGNvbmYuaW50ID0gVFJVRSwNCiAgICAgICAgIGNvbmYubGV2ZWwgPSAwLjk1KSAlPiUgDQogICAgc2VsZWN0KHRlcm0sIGVzdGltYXRlLCBzdGQuZXJyb3IsDQogICAgICAgICAgIGxvOTUgPSBjb25mLmxvdywgaGk5NSA9IGNvbmYuaGlnaCkNCg0Kd3RfdHdhbmdhdHRfcmVzX3N1cnYNCmBgYA0KDQpBZ2FpbiwgdGhlIGVzdGltYXRlcyB3ZSB3YW50IGNvbWUgZnJvbSB0aGUgYHRyZWF0X3JoY2Agcm93Lg0KDQojIyBEb3VibGUgUm9idXN0IFdlaWdodGVkIEVzdGltYXRlIGZvciBEYXlzIHRvIERlYXRoDQoNCk5vdyBsZXQncyBhZGQgaW4gYSByZWdyZXNzaW9uIGFkanVzdG1lbnQgZm9yIHRoZSBsaW5lYXIgcHJvcGVuc2l0eSBzY29yZS4gQWdhaW4sIGluIHRoaXMgY2FzZSwgd2UgcHJlZmVyIHRoaXMgZG91YmxlIHJvYnVzdCBhcHByb2FjaCB0byB3ZWlnaHRpbmcgYWxvbmUgYmVjYXVzZSBvZiBSdWJpbidzIFJ1bGUgMSBhbmQgdGhlIExvdmUgcGxvdC4NCg0KYGBge3J9DQpkcl9zdXJ2ZGF5cyA8LSANCiAgICBjb3hwaChTdXJ2KHN1cnZkYXlzLCBkaWVkKSB+IHRyZWF0X3JoYyArIGxpbnBzLA0KICAgICAgICAgIGRhdGEgPSByaGNfZGYsIHdlaWdodHMgPSByaGNfd3RzKQ0KDQpkcl9zdXJ2IDwtIA0KICAgIHRpZHkoZHJfc3VydmRheXMsIGV4cG9uZW50aWF0ZSA9IFRSVUUsIA0KICAgICAgICAgY29uZi5pbnQgPSBUUlVFLA0KICAgICAgICAgY29uZi5sZXZlbCA9IDAuOTUpICU+JSANCiAgICBzZWxlY3QodGVybSwgZXN0aW1hdGUsIHN0ZC5lcnJvciwNCiAgICAgICAgICAgbG85NSA9IGNvbmYubG93LCBoaTk1ID0gY29uZi5oaWdoKQ0KDQpkcl9zdXJ2DQpgYGANCg0KT25jZSBhZ2FpbiwgdGhlIGVzdGltYXRlcyB3ZSB3YW50IGNvbWUgZnJvbSB0aGUgYHRyZWF0X3JoY2Agcm93Lg0KDQojIyBHcmFwaGluZyB0aGUgV2VpZ2h0ZWQgYW5kIE1hdGNoZWQgUmVzdWx0cyBmb3IgT3V0Y29tZSBDDQoNCmBgYHtyfQ0KdGVtcDExIDwtIA0KICAgIHRpZHkoYWRqX3N1cnZkYXlzX3d0ZCwgZXhwb25lbnRpYXRlID0gVCwgY29uZi5pbnQgPSBULCANCiAgICAgICAgIGNvbmYubGV2ZWwgPSAwLjk1KSAlPiUNCiAgICBmaWx0ZXIodGVybSA9PSAidHJlYXRfcmhjIikgJT4lDQogICAgc2VsZWN0KC1yb2J1c3Quc2UpDQoNCnRlbXAxMiA8LSANCiAgICB0aWR5KGRyX3N1cnZkYXlzLCBleHBvbmVudGlhdGUgPSBULCBjb25mLmludCA9IFQsIA0KICAgICAgICAgY29uZi5sZXZlbCA9IDAuOTUpICU+JQ0KICAgIGZpbHRlcih0ZXJtID09ICJ0cmVhdF9yaGMiKSAlPiUNCiAgICBzZWxlY3QoLXJvYnVzdC5zZSkNCg0Kc3VydmRheXNfbmV3X3Jlc3VsdHMgPC0gcmJpbmQodGVtcDExLCB0ZW1wMTIpICU+JQ0KICAgIG11dGF0ZShhcHByb2FjaCA9IGMoIkFUVCB3ZWlnaHRlZCIsICJEUiIpKSAlPiUNCiAgICBtdXRhdGUoZGVzY3JpcHRpb24gPSBjKCJBVFQgd2VpZ2h0cyIsICJEUjogQVRUIHd0cyArIGFkaiIpKQ0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIA0Kc3VydmRheXNfcmVzdWx0cyA8LSByYmluZChzdXJ2ZGF5c19uZXdfcmVzdWx0cywgDQogICAgICAgICAgICAgICAgICAgICAgIHN1cnZkYXlzX21hdGNoX3Jlc3VsdHMpDQoNCnN1cnZkYXlzX3Jlc3VsdHMNCmBgYA0KDQpTbyB0aGVyZSdzIG91ciB0YWJsZS4gDQoNCmBgYHtyfQ0Kc3VydmRheXNfcmVzdWx0cyAlPiUNCiAgICBnZ3Bsb3QoLiwgYWVzKHggPSBkZXNjcmlwdGlvbiwgeSA9IGVzdGltYXRlLCANCiAgICAgICAgICAgICAgICAgIHltaW4gPSBjb25mLmxvdywgeW1heCA9IGNvbmYuaGlnaCkpICsNCiAgICBnZW9tX3RleHQoYWVzKGxhYmVsID0gcm91bmRfaGFsZl91cChlc3RpbWF0ZSwyKSksIHZqdXN0ID0gLTEuMjUpICsNCiAgICBnZW9tX3BvaW50cmFuZ2UoKSArDQogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMSwgY29sID0gInJlZCIsIGxpbmV0eXBlID0gImRhc2hlZCIpICsNCiAgICB0aGVtZV9idygpICsNCiAgICBjb29yZF9mbGlwKCkgKw0KICAgIGxhYnModGl0bGUgPSAiQ29tcGFyaW5nIEFUVCBFc3RpbWF0ZXMgZm9yIFN1cnZpdmFsIFRpbWUiLA0KICAgICAgICAgeCA9ICJQcm9wZW5zaXR5IEFkanVzdG1lbnQgQXBwcm9hY2giLCANCiAgICAgICAgIHkgPSAiRXN0aW1hdGVkIEhhemFyZCBSYXRpbyBhc3NvY2lhdGVkIHdpdGggUkhDIikNCmBgYA0KDQpBbmQgaGVyZSBhcmUgdGhlIHJlc3VsdHMgZm9yIG9ubHkgdGhlIGZpdmUgbWV0aG9kcyAoYWRkaW5nIG91ciBkb3VibGUgcm9idXN0IGFwcHJvYWNoIHRvIHRoZSBhbmFseXNlcyB3ZSBsaWtlZCBpbiB0aGUgbWF0Y2hpbmcpIHdoaWNoIHNob3dlZCByZWFzb25hYmxlIGJhbGFuY2UgYWZ0ZXIgcHJvcGVuc2l0eSBzY29yZSBhZGp1c3RtZW50Li4uDQoNCmBgYHtyfQ0Kc3VydmRheXNfcmVzdWx0cyAlPiUNCiAgICBmaWx0ZXIoYXBwcm9hY2ggJWluJSBjKCJNYXRjaCAzIiwgIk1hdGNoIDQiLCAiTWF0Y2ggNSIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgIk1hdGNoIDYiLCAiRFIiKSkgJT4lDQogICAgZ2dwbG90KC4sIGFlcyh4ID0gZGVzY3JpcHRpb24sIHkgPSBlc3RpbWF0ZSwgDQogICAgICAgICAgICAgICAgICB5bWluID0gY29uZi5sb3csIHltYXggPSBjb25mLmhpZ2gpKSArDQogICAgZ2VvbV90ZXh0KGFlcyhsYWJlbCA9IHJvdW5kX2hhbGZfdXAoZXN0aW1hdGUsMikpLCB2anVzdCA9IC0xLjI1KSArDQogICAgZ2VvbV9wb2ludHJhbmdlKCkgKw0KICAgIGdlb21faGxpbmUoeWludGVyY2VwdCA9IDEsIGNvbCA9ICJyZWQiLCBsaW5ldHlwZSA9ICJkYXNoZWQiKSArDQogICAgdGhlbWVfYncoKSArDQogICAgY29vcmRfZmxpcCgpICsNCiAgICBsYWJzKHRpdGxlID0gIkNvbXBhcmluZyBBVFQgRXN0aW1hdGVzIGZvciBTdXJ2aXZhbCBUaW1lIiwNCiAgICAgICAgIHggPSAiUHJvcGVuc2l0eSBBZGp1c3RtZW50IEFwcHJvYWNoIiwgDQogICAgICAgICB5ID0gIkVzdGltYXRlZCBIYXphcmQgUmF0aW8gYXNzb2NpYXRlZCB3aXRoIFJIQyIpDQpgYGANCg0KIyBTZW5zaXRpdml0eSBBbmFseXNlcyBBZnRlciBNYXRjaGluZw0KDQojIyBGb3IgdGhlIEJpbmFyeSBPdXRjb21lLCBEZWF0aA0KDQpUaGUgYHJib3VuZHNgIHBhY2thZ2UgY2FuIGV2YWx1YXRlIGJpbmFyeSBvdXRjb21lcyB1c2luZyB0aGUgYGJpbmFyeXNlbnNgIGFuZCBgRmlzaGVyc2Vuc2AgZnVuY3Rpb25zLiBUaGUgYGJpbmFyeXNlbnNgIGZ1bmN0aW9uIHdvcmtzIGRpcmVjdGx5IG9uIGEgbWF0Y2hlZCBzYW1wbGUgYXMgZGV2ZWxvcGVkIHVzaW5nIHRoZSBNYXRjaGluZyBwYWNrYWdlLiBGb3IgZXhhbXBsZSwgaW4gb3VyIHRoaXJkIG1hdGNoLCBiYXNlZCBvbiB0aGUgZ2VuZXRpYyBzZWFyY2ggYWxnb3JpdGhtIHdpdGggMToxIG1hdGNoaW5nIHdpdGhvdXQgcmVwbGFjZW1lbnQsIHRoaXMgaXMgYG1hdGNoM2AuIE5vdGUgdGhhdCBpbiBvcmRlciB0byBkbyB0aGlzLCB3ZSBuZWVkZWQgdG8gcnVuIGBtYXRjaDNgIGluY2x1ZGluZyB0aGUgYXBwcm9wcmlhdGUgKGJpbmFyeSkgb3V0Y29tZSAoWSkuDQoNCmBgYHtyfQ0KWSA8LSByaGMkZGllZA0KWCA8LSBjYmluZChyaGMkbGlucHMsIHJoYyRwcykNClRyIDwtIGFzLmxvZ2ljYWwocmhjJHN3YW5nMSA9PSAiUkhDIikNCm1hdGNoMyA8LSBNYXRjaChZID0gWSwgVHIgPSBUciwgWCA9IFgsIGVzdGltYW5kID0gIkFUVCIsIA0KICAgICAgICAgICAgICAgICBXZWlnaHQubWF0cml4ID0gZ2Vub3V0MykNCmJpbmFyeXNlbnMobWF0Y2gzLCBHYW1tYSA9IDEuNSwgR2FtbWFJbmMgPSAwLjA1KQ0KYGBgDQoNCkF0IGEgNSUgc2lnbmlmaWNhbmNlIGxldmVsLCB3ZSByZXRhaW4gc2lnbmlmaWNhbmNlIHVwIHRvIGEgaGlkZGVuIGJpYXMgb2YgJFxHYW1tYSQgPSAxLjE1LCBidXQgbm90IGF0ICRcR2FtbWEkID0gMS4yLiBTZWUgdGhlIGB0b3lfMjAxOWAgZXhhbXBsZSBkaXNjdXNzZWQgaW4gQ2xhc3MgNCwgYXMgd2VsbCBhcyBDaGFwdGVyIDkgb2YgUm9zZW5iYXVtJ3MgKk9ic2VydmF0aW9uIGFuZCBFeHBlcmltZW50KiBmb3IgbW9yZSBkZXRhaWxzIG9uIGludGVycHJldGF0aW9uIG9mIHRoaXMgcmVzdWx0Lg0KDQpXZSBjYW4gYWxzbyB1c2UgdGhpcyBhcHByb2FjaCB3aXRoIG90aGVyIE1hdGNoaW5nIHJlc3VsdHMsIGluY2x1ZGluZyBNYXRjaCA2LCB3aGljaCBpcyBhIDE6MiBtYXRjaCB3aXRoIHJlcGxhY2VtZW50LiBBZ2Fpbiwgd2UgaGF2ZSB0byBydW4gdGhlIG1hdGNoIHdpdGggdGhlIGFwcHJvcHJpYXRlIG91dGNvbWUgaW5jbHVkZWQuDQoNCmBgYHtyfQ0KWSA8LSByaGMkZGllZA0KWCA8LSByaGMkbGlucHMNClRyIDwtIGFzLmxvZ2ljYWwocmhjJHN3YW5nMSA9PSAiUkhDIikNCm1hdGNoNiA8LSBNYXRjaChZID0gWSwgVHIgPSBUciwgWCA9IFgsIE0gPSAyLCANCiAgICAgICAgICAgICAgICBlc3RpbWFuZCA9ICJBVFQiLCByZXBsYWNlID0gVFJVRSwgDQogICAgICAgICAgICAgICAgdGllcyA9IEZBTFNFKQ0KDQpiaW5hcnlzZW5zKG1hdGNoNiwgR2FtbWEgPSAxLjUsIEdhbW1hSW5jID0gMC4wNSkNCmBgYA0KDQpIZXJlLCBhdCBhIDUlIHNpZ25pZmljYW5jZSBsZXZlbCwgd2UgcmV0YWluIHNpZ25pZmljYW5jZSB1cCB0byBhIGhpZGRlbiBiaWFzIG9mICRcR2FtbWEkID0gMS4xNSwgYnV0IG5vdCBhdCAkXEdhbW1hJCA9IDEuMjAuDQoNCiMjIEZvciB0aGUgUXVhbnRpdGF0aXZlIE91dGNvbWUsIEluLUhvc3BpdGFsIExlbmd0aCBvZiBTdGF5DQoNCldlIGhhdmUgYWxyZWFkeSB1c2VkIHRoZSBgTWF0Y2hgIGZ1bmN0aW9uIGZyb20gdGhlIE1hdGNoaW5nIHBhY2thZ2UgdG8gZGV2ZWxvcCBvdXIgbWF0Y2hlZCBzYW1wbGVzLiBGb3Igb3VyIDE6MSBtYXRjaGVzLCB3ZSBuZWVkIG9ubHkgcnVuIHRoZSBgcHNlbnNgIGZ1bmN0aW9uIGZyb20gdGhlIGByYm91bmRzYCBwYWNrYWdlIHRvIG9idGFpbiBzZW5zaXRpdml0eSByZXN1bHRzLiBGb3IgZXhhbXBsZSwgaW4gTWF0Y2ggMywNCg0KYGBge3J9DQpZIDwtIHJoYyRob3NwZGF5cw0KWCA8LSBjYmluZChyaGMkbGlucHMsIHJoYyRwcykNClRyIDwtIGFzLmxvZ2ljYWwocmhjJHN3YW5nMSA9PSAiUkhDIikNCm1hdGNoMyA8LSBNYXRjaChZID0gWSwgVHIgPSBUciwgWCA9IFgsIGVzdGltYW5kID0gIkFUVCIsIA0KICAgICAgICAgICAgICAgICBXZWlnaHQubWF0cml4ID0gZ2Vub3V0MykNCnBzZW5zKG1hdGNoMywgR2FtbWEgPSAxLjUsIEdhbW1hSW5jID0gMC4xKQ0KYGBgDQoNCkF0IGEgNSUgc2lnbmlmaWNhbmNlIGxldmVsLCB3ZSByZXRhaW4gc2lnbmlmaWNhbmNlIHVwIHRvIGEgaGlkZGVuIGJpYXMgb2YgJFxHYW1tYSQgPSAxLjEsIGJ1dCBub3QgYXQgJFxHYW1tYSQgPSAxLjIuIFNlZSB0aGUgYHRveV8yMDE5YCBleGFtcGxlIGRpc2N1c3NlZCBpbiBDbGFzcyA0LCBhcyB3ZWxsIGFzIENoYXB0ZXIgOSBvZiBSb3NlbmJhdW0ncyAqT2JzZXJ2YXRpb24gYW5kIEV4cGVyaW1lbnQqIGZvciBtb3JlIGRldGFpbHMgb24gaW50ZXJwcmV0YXRpb24gb2YgdGhpcyByZXN1bHQuDQoNCkZvciBhIG1hdGNoaW5nIHdpdGggbW9yZSB0aGFuIDEgY29udHJvbCwgd2UgbmVlZCB0byBhcHBseSB0aGUgYG1jb250cm9sYCBmdW5jdGlvbiwgd2hpY2ggcmVxdWlyZXMgdGhlIHVzZSBvZiB0aGUgYGRhdGEucHJlcGAgZnVuY3Rpb24gZnJvbSBgcmJvdW5kc2AuIFNvLCBmb3IgZXhhbXBsZSwgaW4gTWF0Y2ggNiwgd2hpY2ggaXMgYSAxOjIgZ3JlZWR5IG1hdGNoIChzbyB0aGF0IGVhY2ggbWF0Y2hlZCBzZXQgaGFzIDMgcGF0aWVudHMgaW4gaXQsIG1ha2luZyB0aGUgYGdyb3VwLnNpemVgID0gMyksIHdlIHdvdWxkIGhhdmU6DQoNCmBgYHtyfQ0KWSA8LSByaGMkaG9zcGRheXMNClggPC0gcmhjJGxpbnBzDQpUciA8LSBhcy5sb2dpY2FsKHJoYyRzd2FuZzEgPT0gIlJIQyIpDQptYXRjaDYgPC0gTWF0Y2goWSA9IFksIFRyID0gVHIsIFggPSBYLCBNID0gMiwgDQogICAgICAgICAgICAgICAgZXN0aW1hbmQgPSAiQVRUIiwgcmVwbGFjZSA9IFRSVUUsIA0KICAgICAgICAgICAgICAgIHRpZXMgPSBGQUxTRSkNCg0KdG1wIDwtIGRhdGEucHJlcChtYXRjaDYsIGdyb3VwLnNpemUgPSAzKQ0KDQptY29udHJvbCh0bXAkWSwgdG1wJGlkLCB0bXAkdHJlYXQsIGdyb3VwLnNpemU9MywgR2FtbWEgPSAxLjUsIEdhbW1hSW5jID0gMC4xKQ0KYGBgDQoNCkFnYWluLCBhdCBhIDUlIHNpZ25pZmljYW5jZSBsZXZlbCwgd2UgcmV0YWluIHNpZ25pZmljYW5jZSB1cCB0byBhIGhpZGRlbiBiaWFzIG9mICRcR2FtbWEkID0gMS4xLCBidXQgbm90IGF0ICRcR2FtbWEkID0gMS4yLg0KDQojIyBGb3IgdGhlIFN1cnZpdmFsIE91dGNvbWUsIEluLVN0dWR5IFRpbWUgdG8gRGVhdGgNCg0KSW4gdGhpcyBzZXR0aW5nLCB3ZSB3b3VsZCBuZWVkIHRvIHVzZSB0aGUgc3ByZWFkc2hlZXQgc29mdHdhcmUgcHJvdmlkZWQgYnkgRHIuIExvdmUuIFRoYXQgc29mdHdhcmUgaXMgZGVzaWduZWQgb25seSBmb3IgMToxIGdyZWVkeSBtYXRjaGluZyB3aXRob3V0IHJlcGxhY2VtZW50IG9yIHdlaWdodGluZywgd2hpY2ggaXMgdGhlIGFwcHJvYWNoIHdlIHRvb2sgaW4gTWF0Y2ggMSAod2UgY291bGQgYWxzbyBoYXZlIGxvb2tlZCBhdCBtYXRjaGluZyB1c2luZyBhIGNhbGlwZXIsIGJ1dCB3aXRob3V0IHJlcGxhY2VtZW50LikgV2UgbmVlZCB0byBpZGVudGlmeSwgZm9yIGVhY2ggbWF0Y2hlZCBwYWlyIG9mIHN1YmplY3RzLCB3aGV0aGVyIHRoYXQgcGFpcidzIHNldCBvZiByZXN1bHRzIHdhczoNCg0KLSB0aGF0IHRoZSBSSEMgcGF0aWVudCBkZWZpbml0ZWx5IHN1cnZpdmVkIGxvbmdlciB0aGFuIHRoZSBub24tUkhDIHBhdGllbnQNCi0gdGhhdCB0aGUgbm9uLVJIQyBwYXRpZW50IGRlZmluaXRlbHkgc3Vydml2ZWQgbG9uZ2VyIHRoYW4gdGhlIFJIQyBwYXRpZW50DQotIHRoYXQgaXQgaXMgdW5jbGVhciB3aGljaCBwYXRpZW50IHN1cnZpdmVkIGxvbmdlciwgZHVlIHRvIGNlbnNvcmluZw0KDQpIZXJlIGlzIHNvbWUgdmVyeSBmdXNzeSBjb2RlIHRoYXQgYWNjb21wbGlzaGVzIHRoaXMgY291bnRpbmcuIFRoaXMgY291bGQgcHJvYmFibHkgYmUgYWNjb21wbGlzaGVkIGp1c3QgYXMgd2VsbCB3aXRoIHNvbWUgc29ydCBvZiBgY2FzZV93aGVuYCBzdGF0ZW1lbnQsIGJ1dCBpdCdzIGEgYml0IHRyaWNreS4NCg0KYGBge3J9DQphMSA8LSByaGMubWF0Y2hlc18xICU+JSANCiAgICBzZWxlY3QobWF0Y2hlc18xLCBzd2FuZzEsIGRlYXRoLCBzdXJ2ZGF5cykgJT4lDQogICAgYXJyYW5nZShtYXRjaGVzXzEpIA0KYGBgDQoNCmBgYHtyfQ0KYTIgPC0gdW5pdGUoYTEsIHNpdHVhdGlvbiwgc3dhbmcxLCBkZWF0aCwgc2VwID0gIl8iLCByZW1vdmUgPSBUUlVFKQ0KDQphMyA8LSBzcHJlYWQoYTIsIHNpdHVhdGlvbiwgc3VydmRheXMpDQoNCmE0IDwtIGEzICU+JQ0KICAgIGZpbHRlcihjb21wbGV0ZS5jYXNlcyhgTm8gUkhDX1llc2AsIGBSSENfWWVzYCkpICU+JQ0KICAgIG11dGF0ZShyZXN1bHQgPSBpZmVsc2UoYFJIQ19ZZXNgID4gYE5vIFJIQ19ZZXNgLCAiUkhDIGxpdmVkIGxvbmdlciIsICJObyBSSEMgbGl2ZWQgbG9uZ2VyIikpDQoNCmE1IDwtIGEzICU+JQ0KICAgIGZpbHRlcihjb21wbGV0ZS5jYXNlcyhgTm8gUkhDX05vYCwgYFJIQ19Ob2ApKSAlPiUNCiAgICBtdXRhdGUocmVzdWx0ID0gIk5vIGNsZWFyIHdpbm5lciIpDQoNCmE2IDwtIGEzICU+JQ0KICAgIGZpbHRlcihjb21wbGV0ZS5jYXNlcyhgTm8gUkhDX05vYCwgYFJIQ19ZZXNgKSkgJT4lDQogICAgbXV0YXRlKHJlc3VsdCA9IGlmZWxzZShgUkhDX1llc2AgPiBgTm8gUkhDX05vYCwgIk5vIGNsZWFyIHdpbm5lciIsICJObyBSSEMgbGl2ZWQgbG9uZ2VyIikpDQoNCmE3IDwtIGEzICU+JQ0KICAgIGZpbHRlcihjb21wbGV0ZS5jYXNlcyhgTm8gUkhDX1llc2AsIGBSSENfTm9gKSkgJT4lDQogICAgbXV0YXRlKHJlc3VsdCA9IGlmZWxzZShgUkhDX05vYCA+IGBObyBSSENfWWVzYCwgIlJIQyBsaXZlZCBsb25nZXIiLCAiTm8gY2xlYXIgd2lubmVyIikpDQoNCmE4IDwtIGJpbmRfcm93cyhhNCwgYTUsIGE2LCBhNykNCg0KYTggJT4lIHRhYnlsKHJlc3VsdCkgJT4lIGFkb3JuX3RvdGFscygpDQpgYGANCg0KU28gd2UgaGF2ZSBjb3VudHMgZm9yIHRoZSBwYWlycyB3aGVyZSBhIGNsZWFyIHdpbm5lciBpcyBpZGVudGlmaWVkLCBhbmQgd2UgY2FuIGVudGVyIHRoZSBzcHJlYWRzaGVldC4gV2UgbmVlZCB0aGUgbnVtYmVyIG9mIHBhaXJzIHdpdGggYSBjbGVhciB3aW5uZXIsIGFuZCB0aGUgbnVtYmVyIG9mIHBhaXJzIHdoZXJlIHRoZSBObyBSSEMgcGF0aWVudCBsaXZlZCBsb25nZXIgdGhhbiB0aGUgUkhDIHBhdGllbnQuDQoNCg0KIyBTZXNzaW9uIEluZm9ybWF0aW9uDQoNCmBgYHtyfQ0KeGZ1bjo6c2Vzc2lvbl9pbmZvKCkNCmBgYA0K