Introduction
We just finished another presidential election year, full of
surprises and upsets, or maybe upsets and surprises. Either way, the
voting part is over, the counting part is ongoing, and unlike 2020, the
vanquished had a sense of dignity and sanity. All sorts of experts have
crawled out from under their rocks to either claim credit or assign
blame. I’m not interested in that. I’m interested in the story data can
tell.
I asked myself what Presidential elections looked like from a “pure”
numbers perspective, so I collected some data. The voting-eligible
population (VEP) came from the University of Florida
Election Lab. The vote counts came from either the Wikipedia pages
for each election, 1980,
1984,
1988,
1992,
1996,
2000,
2004,
2008,
2012,
2016,
and 2020
or the Election Atlas
state pages for 2024. The 2024 numbers are all preliminary and
should be interpreted with caution. I also collected the number of
Electoral College votes each state had in each election to use as
weights for modeling. This was looked up from several sources, including
Wikipedia.
The data consisted of election year, state names, and counts of VEP
and votes for the Democrat, Republican, and Other candidates. I
subtracted the sum of the votes from the VEP to get Nonvoting counts.
This was all done in Excel, where I also formatted the data into “long”
format. I loaded the clean data. You can get it, yourself in
this Google sheet.
Before I go any further, I will be using R for this work, so I need
to load a bunch of R packages.
library(googlesheets4)
Warning: package ‘googlesheets4’ was built under R version 4.4.2
library(glmmTMB)
library(performance)
library(car)
Loading required package: carData
library(performance)
library(MuMIn)
library(emmeans)
Welcome to emmeans.
Caution: You lose important information if you filter this package's results.
See '? untidy'
library(effectsize)
library(tidyverse)
── Attaching core tidyverse packages ──────────────────────────────────────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.4 ✔ readr 2.1.5
✔ forcats 1.0.0 ✔ stringr 1.5.1
✔ ggplot2 3.5.1 ✔ tibble 3.2.1
✔ lubridate 1.9.3 ✔ tidyr 1.3.1
✔ purrr 1.0.2
── Conflicts ────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
✖ dplyr::recode() masks car::recode()
✖ purrr::some() masks car::some()
ℹ Use the ]8;;http://conflicted.r-lib.org/conflicted package]8;; to force all conflicts to become errors
library(reshape2)
Warning: package ‘reshape2’ was built under R version 4.4.2
Attaching package: ‘reshape2’
The following object is masked from ‘package:tidyr’:
smiths
Vote.dat <- read_sheet("https://docs.google.com/spreadsheets/d/1oySzPkmaVMBHHwAib4kLuca_7XQrXg5x67hQmLKUpTA/edit")
The googlesheets4 package is requesting access to your Google account.
Enter '1' to start a new auth process or select a pre-authorized account.
1: Send me to the browser for a new auth process.
2: maloneywriting18@gmail.com
1
Waiting for authentication in browser...
Press Esc/Ctrl + C to abort
Authentication complete.
✔ Reading from Vote.sheet.
✔ Range Sheet1.
Vote.dat$State <- factor(Vote.dat$State)
Vote.dat$Selection <- factor(Vote.dat$Selection)
Feel free to use the above link to do a manual import if you’d prefer
that.
So, let’s take a look at the raw data.
messy.plot <- ggplot(Vote.dat, aes(x = jitter(Year), y = Votes/VEP, group = Selection, color = Selection)) +
scale_color_manual(values = c("#4477AA", "#228833", "#CCBB44", "#EE6677"),
limits = c("Democrat", "Other", "Nonvoting", "Republican")) +
geom_point(size = 3) +
theme(panel.background = element_rect(fill = "white"),
axis.line = element_line(size = 1, color = "black")) +
scale_y_continuous(labels = scales::percent) +
scale_x_continuous(breaks = c(1980, 1984, 1988, 1992, 1996, 2000, 2004, 2008, 2012, 2016, 2020, 2024)) +
ylab("Frequency (per VEP)") + xlab("Year")
messy.plot

Summarizing
That’s a real mess. Let’s try generating a summary.
# Some background functions
weighted.average <- function(x, w){
## Sum of the weights
sum.w <- sum(w, na.rm = T)
## Sum of the weighted $x_i$
xw <- sum(w*x, na.rm = T)
## Return the weighted average
return(xw/sum.w)
}
weighted.se.mean <- function(x, w, na.rm = T){
## Remove NAs
if (na.rm) {
i <- !is.na(x)
w <- w[i]
x <- x[i]
}
## Calculate effective N and correction factor
n_eff <- (sum(w))^2/(sum(w^2))
correction = n_eff/(n_eff-1)
## Get weighted variance
numerator = sum(w*(x-weighted.average(x,w))^2)
denominator = sum(w)
## get weighted standard error of the mean
se_x = sqrt((correction * (numerator/denominator))/n_eff)
return(se_x)
}
# Making the weighted summary
Vote.sum <- Vote.dat %>%
mutate(pasteup = paste(Year, Selection)) %>%
mutate(ratio = Votes/VEP) %>%
group_by(pasteup) %>%
summarize(weighted_means = weighted.average(ratio, Electoral.Votes)) %>%
cbind(Vote.dat %>%
mutate(pasteup = paste(Year, Selection)) %>%
mutate(ratio = Votes/VEP) %>%
group_by(pasteup) %>%
summarize(weighted_CL = weighted.se.mean(ratio, Electoral.Votes) * qt(0.975, 50)))
Vote.sum <- Vote.sum[, c(1, 2, 4)]
Vote.sum <- cbind(colsplit(Vote.sum$pasteup, " ", c("Year", "Selection")), Vote.sum[,2:3])
Vote.sum$Selection <- factor(Vote.sum$Selection)
Vote.sum
What I did was take weighted means by year of all the states for
votes cast for the Democratic or Republican candidate or any other
candidate, along with the remainder (non-voting VEP), divided by the
VEP. The means were weighted by the Electoral College votes that state
had in that election year. I then calculated the weighted standard
errors of the mean, using a function written by Alex
Stephenson. I converted these to 95% confidence limits by
multiplying by the appropriate quantile of t at 0.975 and 50 degrees of
freedom. Why did I weight the popular votes? It’s a concession to
reality. People might hand-wring and whine all they like about how
horrid the Electoral College might be, but it’s not going away any time
soon. Understanding US presidential politics requires taking it into
account.
So, I have a table, I don’t find tables useful. What are useful are
charts.
summary.plot <- ggplot(Vote.sum, aes(x = Year, y = weighted_means, group = Selection, color = Selection)) +
geom_line(linewidth = 1) +
scale_color_manual(values = c("#4477AA", "#228833", "#CCBB44", "#EE6677"),
limits = c("Democrat", "Other", "Nonvoting", "Republican")) +
geom_point(size = 3) +
geom_errorbar((aes(ymin=weighted_means - weighted_CL, ymax = weighted_means + weighted_CL))) +
theme(panel.background = element_rect(fill = "white"),
axis.line = element_line(size = 1, color = "black"),
legend.position = c(0.8, 0.3)) +
scale_y_continuous(labels = scales::percent) +
scale_x_continuous(breaks = c(1980, 1984, 1988, 1992, 1996, 2000, 2004, 2008, 2012, 2016, 2020, 2024)) +
ylab("Frequency (per VEP)")
summary.plot

I didn’t just pull the colors out of my cute patootie. They’re from
Paul Tol’s site that describes many wonderful (and color-blind safe) color schemes, arranged with
the now-traditional coloring of blue for Democrats and red for
Republicans.
A little non-statistical analysis
We can analyze the above chart without even considering statistics.
I’m going to be mostly just descriptive. The first thing to note is that
the most popular “candidate”, by far, for the last 44 years has been
nobody. Since at least 1980, there has never been a president elected by
the majority of Americans eligible to vote. Let us wring hands. Hand
wringing time is over. While it’s true that “get out of my face” has
usually been the most popular selection, it’s equally true that this
option has been chosen less and less as time has gone on. Whatever is
wrong with America, more Americans have been getting out to vote. True
enough, 2024 was another election of apathy, but it’s hard to call the
future based on a single-point deflection. The overall trend has been
for non-voting to drop by a little less than 1 percentage point every
election. The Democrats and Republicans are basically in a dead heat in
dividing this gain up between themselves. The overall Democrat gain
since 1980 has been greater than the overall Republican gain, but
currently they are indistinguishable. On the other hand, the slow
increase in voting doesn’t seem to have been distributed to “Other”
options.
Another thing to note is that the Republicans and Democrats seem to
have switched performances from 1980 to the present. From 1980 to 2004,
Republican performance fluctuated wildly from year to year. The
Democrats, on the other hand, were experiencing very slow but very
regular gains until 2000. From these inflection points to the present
day, this has switched. The Democrats now have wild swings while the
Republicans’ oscillations have calmed down (although with more variation
than the early-period Democrats showed). How to explain this or what it
means for the future I leave to more knowledgeable people. What I can
say is that it looks like both parties experienced a significant sea
change roughly 2000-2004, and it is having long-lasting effects on each
of them. What these changes were, I can’t even speculate. Feel free.
Calling the Horse Races
Let’s dive deeper and look at each election in turn. Reagan’s tenure
was marked by moving from strong to stronger. Unusually for incumbents,
he gained percentage in his re-election bid. However, this was not at
the expense of Mondale, who also got a (very tiny) gain vs. Carter’s
performance. Instead, Reagan may have attracted some non-voters and a
large portion of people who had voted for Anderson (who?). Poppy Bush’s
election was an exercise in coat-tail riding. The nonvoters bounced back
up, and Dukakis got another (very tiny) gain over the Democrats’ share
of the previous election,
Indeed, from 1980 to 2000 “a very tiny gain” could accurately
describe the Democratic party’s entire presidential performance, even in
years they took the White House. In every election, it was a very small
improvement in attracting a proportion of the voting-eligible
population, while the Republicans see-sawed up and down in popularity.
For example, Clinton took office in a tidal wave of spoilage. Ross
Perot’s leap seems to have been fueled by some Republican-leaning voters
and bigger chunk of people who otherwise wouldn’t have bothered to vote
at all. But Perot was a gadfly candidate. Bush was the anointed leader
of the Republican party, so there was a squabble on that side of the
political spectrum. Clinton rode in on this disunity.
Perot’s huge-for-a-third-dude showing cannot be explained only by
attracting nonvoters. In 1998, there was a giant leap in apathy, which
probably hurt Dole more than it hurt Clinton. However, Clinton didn’t
experience a spike in popularity. He managed to do a bit better than
last time. This set the stage for 2000.
This was an extremely close election. What is interesting is that
there was very little apparent motivation of the nonvoters. It ended up
in a mess. Bush’s re-election was more definite and it was accompanied
by the Democratic party starting to improve more quickly its slug-like
crawl upwards since 1980. However, Bush was able to attract more of the
nonvoters (and possibly Other voters) than did Kerry. What is
interesting is that this election saw a big drop in nonvoters, which
goes against the general trend of an increase in apathy or at very best
a small decrease during a re-election attempt. However, these voters
didn’t turn out for Kerry.
In 2008, Obama ended up continuing a straight-line trend started by
the prior two failed Democrat attempts. There was a small tend downwards
in nonvoters, but the Republicans lost share as well, indicating some
possible aisle-crossing by voters. Obama’s second turn was a typical
successful re-election, with more people choosing to sit it out, but not
enough to force a loss. The Republicans were still experiencing the Bush
2-inspired decline.
2016 was another close election. It’s well-documented that Trump did
not achieve the total popular majority of votes cast. The weighted means
still reflect this. This is not anomalous. It is because weighted means
can “distribute” proportional Electoral outcomes within each state as
well as fractions. What we do see is that Gary Johnson had a bit of a
spike, which may have been acquired from a mixture of nonvoters and both
Democrat and Republican leaning voters.
The Biden election is illustrative of what might be an important
trend. Once again, the number of nonvoters fell (to its lowest
level), and the primary beneficiary seems to have been Biden. It
was the first time in the period I looked at that an actual candidate
managed to get more popular votes than “this election isn’t worth voting
in”!
Finally, Iit should be noted that the 2024 results are based in part
on projections and estimates and ought not be over-interpreted. However,
the biggest winner is “Nobody is worth my vote.” Apathy and
disengagement won again, with one of its biggest spikes since 1996.
However, this time, the Republicans weren’t riding a massive bottom-out
trend. If the projections are to be believed, Trump also lost share from
his previous attempt. He merely lost less than did Harris vs. Biden’s
performance. Trump managed to get less of the voting-eligible public
than he did in 2020. Unfortunately, Harris managed to do even worse. I
can’t explain why. I won’t try to explain why. I can only show the what.
The why requires more in-depth knowledge than I have.
Modeling
We could stop right there, but I looked at the chart and wondered if
there were any possibly verifiable trends in what I think I saw. That’s
when I pulled out the formal modeling tools.
The data was repeated measurements (by state and District of
Columbia) of 12 presidential elections. When I see repeated measures, I
like to use a mixed-level model. If you want to know more, you
can start here. The data is count data. Yes, I presented it as
frequencies, but a frequency is a count divided by a reference count.
The preferred way to analyze count data is with a Poisson distribtion.
However, the Poisson distribution presumes that the mean and the
variance of the data are the same, or dispersion = 1, to use jargon. A
dispersion over 1 is overdispersed, and under 1 is underdispersed. Since
I did not yet know if the data were over or under-dispersed, I chose R’s
glmmTMB package, since it has options that can handle either
eventuality. I started with a first pass model that used the Poisson
family.
options(contrasts = rep("contr.Sum",2))
start.mod <- glmmTMB(Votes ~ Selection * Year + (1 | State) + offset(log(VEP)), weights = Electoral.Votes, family = poisson, data = Vote.dat)
Warning in (function (start, objective, gradient = NULL, hessian = NULL, :
NA/NaN function evaluation
I got a warning. It’s often due to differences in variable scale. The
offending variable is probably Year, which can be fixed.
Vote.dat$Year.s <- scale(Vote.dat$Year)
start.mod <- glmmTMB(Votes ~ Selection * Year.s + (1 | State) + offset(log(VEP)), weights = Electoral.Votes, family = poisson, data = Vote.dat)
And I had to check the dispersion.
check_overdispersion(start.mod)
# Overdispersion test
dispersion ratio = 1112634.483
Pearson's Chi-Squared = 2713715504.977
p-value = < 0.001
Overdispersion detected.
Yes, definitely overdispersed. There are fancy ways to handle this,
like the Conway-Maxwell-Poisson family, but the current implementation
of this family can be slower than a geologic age to run. Fortunately,
the reliable old negative binomial family will handle overdispersion.
Since the sample had over 2000 entries, I felt comfortable exploring
different random effects vis the states. I compared models that
included random intercepts and random slopes for both Selection and
Year, with and without correlated intercepts.
nb.mod1 <- glmmTMB(Votes ~ Selection * Year.s + (1 | State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
nb.mod2 <- glmmTMB(Votes ~ Selection * Year.s + (Selection | State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
Warning in finalizeTMB(TMBStruc, obj, fit, h, data.tmb.old) :
Model convergence problem; singular convergence (7). See vignette('troubleshooting'), help('diagnose')
nb.mod3 <- glmmTMB(Votes ~ Selection * Year.s + (Year.s | State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
nb.mod4 <- glmmTMB(Votes ~ Selection * Year.s + (Selection + Year.s | State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
Warning in finalizeTMB(TMBStruc, obj, fit, h, data.tmb.old) :
Model convergence problem; non-positive-definite Hessian matrix. See vignette('troubleshooting')
Warning in finalizeTMB(TMBStruc, obj, fit, h, data.tmb.old) :
Model convergence problem; singular convergence (7). See vignette('troubleshooting'), help('diagnose')
nb.mod5 <- glmmTMB(Votes ~ Selection * Year.s + (Selection * Year.s | State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
Warning in finalizeTMB(TMBStruc, obj, fit, h, data.tmb.old) :
Model convergence problem; non-positive-definite Hessian matrix. See vignette('troubleshooting')
Warning in finalizeTMB(TMBStruc, obj, fit, h, data.tmb.old) :
Model convergence problem; singular convergence (7). See vignette('troubleshooting'), help('diagnose')
nb.mod6 <- glmmTMB(Votes ~ Selection * Year.s + (Selection || State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
nb.mod7 <- glmmTMB(Votes ~ Selection * Year.s + (Year.s || State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
nb.mod8 <- glmmTMB(Votes ~ Selection * Year.s + (Selection + Year.s || State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
Warning in finalizeTMB(TMBStruc, obj, fit, h, data.tmb.old) :
Model convergence problem; false convergence (8). See vignette('troubleshooting'), help('diagnose')
nb.mod9 <- glmmTMB(Votes ~ Selection * Year.s + (Selection * Year.s || State) + offset(log(VEP)), weights = Electoral.Votes, family = nbinom2, data = Vote.dat)
I compared models that included random intercepts and random slopes
for both Selection and Year, with and without correlated intercepts.
Different models were compared by second-order Akaike information
criterion . I admit, doing this is not rigorous. There’s controversy
over whether AICc can be calculated at all for a mixed model, but I’m
not being paid for this, so I wasn’t going to take the time to do more
in-depth model comparisons.
Alist <- c(AICc(nb.mod1),
AICc(nb.mod2),
AICc(nb.mod3),
AICc(nb.mod4),
AICc(nb.mod5),
AICc(nb.mod6),
AICc(nb.mod7),
AICc(nb.mod8),
AICc(nb.mod9))
Alist
[1] 742668.5 739833.8 742668.2 NA NA 740009.8 742666.3 739999.4 739731.6
The model with the lowest AICc is the 9th model, so that’s what I
adopted. You may have noticed a few lines back that I set the contrasts
to “contr.Sum”. These are simple sum-to-zero contrasts, aka effect
coding. R’s default is dummy coding or treatment contrasts, which are
fine if you’re not worrying about analyzing interactions. We have an
interaction, so I’m using effect coding.
Vote.model <- nb.mod9
summary(Vote.model)
Family: nbinom2 ( log )
Formula: Votes ~ Selection * Year.s + (Selection * Year.s || State) + offset(log(VEP))
Data: Vote.dat
Weights: Electoral.Votes
AIC BIC logLik deviance df.resid
739731.4 739830.0 -369848.7 739697.4 2431
Random effects:
Conditional model:
Groups Name Variance Std.Dev. Corr
State (Intercept) 0.0095931 0.09794
Selection[S.Democrat] 0.0457551 0.21390 0.00
Selection[S.Nonvoting] 0.0474525 0.21784 0.00 0.00
Selection[S.Other] 0.0721619 0.26863 0.00 0.00 0.00
Year.s 0.0005149 0.02269 0.00 0.00 0.00 0.00
Selection[S.Democrat]:Year.s 0.0059620 0.07721 0.00 0.00 0.00 0.00 0.00
Selection[S.Nonvoting]:Year.s 0.0036982 0.06081 0.00 0.00 0.00 0.00 0.00 0.00
Selection[S.Other]:Year.s 0.0063977 0.07999 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Number of obs: 2448, groups: State, 51
Dispersion parameter for nbinom2 family (): 3.74
Conditional model:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.782538 0.014308 -124.58 <2e-16 ***
Selection[S.Democrat] 0.450019 0.030783 14.62 <2e-16 ***
Selection[S.Nonvoting] 0.872299 0.031316 27.85 <2e-16 ***
Selection[S.Other] -1.838467 0.038281 -48.03 <2e-16 ***
Year.s -0.079030 0.004922 -16.06 <2e-16 ***
Selection[S.Democrat]:Year.s 0.184270 0.012791 14.41 <2e-16 ***
Selection[S.Nonvoting]:Year.s -0.001572 0.010808 -0.15 0.884
Selection[S.Other]:Year.s -0.288923 0.013224 -21.85 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
r2(Vote.model)
# R2 for Mixed Models
Conditional R2: 0.453
Marginal R2: 0.417
Anova(Vote.model, type = "III")
Analysis of Deviance Table (Type III Wald chisquare tests)
Response: Votes
Chisq Df Pr(>Chisq)
(Intercept) 15520.80 1 < 2.2e-16 ***
Selection 3253.50 3 < 2.2e-16 ***
Year.s 257.86 1 < 2.2e-16 ***
Selection:Year.s 630.28 3 < 2.2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Don’t spend much time trying to interpret coefficients. I’m going to
estimate marginal means and trends, which are much easier to interpret.
Marginal means (trends for slopes) estimate based on the model and take
into account the effects of all parts of the model on a response. They
summares what the model is saying in a way that is more intuitive than
attempting to puzzle out multiple coefficients. If you want to really go
down a rabbit hole, there’s a very
complete description you can read elsewhere.
If you look at the R2 (Nakagawa’s
R2), this model isn’t too bad a fit. Marginal
R2 estimates the total contribution of the fixed
effects, conditional includes the random effects (individual state
effects).
I’m not interested in comparing outcomes for a single given election.
That’s not very informative from this high up in the clouds. Comparing
the trends, on the other hand, interests me.
Vote.emt <- emtrends(Vote.model, ~Selection:Year.s, "Year.s")
Vote.emt
Selection Year.s Year.s.trend SE df asymp.LCL asymp.UCL
Democrat 0 0.1052 0.0137 Inf 0.0784 0.1321
Nonvoting 0 -0.0806 0.0119 Inf -0.1038 -0.0574
Other 0 -0.3680 0.0142 Inf -0.3958 -0.3401
Republican 0 0.0272 0.0196 Inf -0.0113 0.0656
Confidence level used: 0.95
Visualizing
What does this tell us? We can see that this trend is increasing over
the 1980-2024 period for Democrats, decreasing for non-voting, and votes
for “Other” candidates, and weakly increasing over time for Republicans.
What is interesting is that the confidence limits for the Republican
trend are negative and positive, suggesting that the overall trend may
not significantly differ from zero for this period. But words say so
much less than a picture. First, we have to generate some points from
the model.
Vote.emm <- emmeans(Vote.model, ~Selection:Year.s, type = "response", offset = log(1), at =
list(Year.s = as.numeric(levels(factor(Vote.dat$Year.s)))))
Vote.preds <- data.frame(Vote.emm) %>%
mutate(Year = (Year.s * attr(Vote.dat$Year.s, "scaled:scale") + attr(Vote.dat$Year.s, "scaled:center")))
Vote.preds
model.plot <- ggplot(Vote.preds, aes(x = Year, y = response, group = Selection, color = Selection)) +
geom_line(linewidth = 1) +
scale_color_manual(values = c("#4477AA", "#228833", "#CCBB44", "#EE6677"),
limits = c("Democrat", "Other", "Nonvoting", "Republican")) +
geom_ribbon(aes(ymin = asymp.LCL, ymax = asymp.UCL), alpha = 0.05) +
theme(panel.background = element_rect(fill = "white"),
axis.line = element_line(size = 1, color = "black"),
legend.position = c(0.8, 0.3)) +
scale_y_continuous(labels = scales::percent) +
scale_x_continuous(breaks = c(1980, 1984, 1988, 1992, 1996, 2000, 2004, 2008, 2012, 2016, 2020, 2024)) +
ylab("Frequency (per VEP)")
model.plot

The ribbons around each line are the 95% confidence intervals. The
trends look like they differ from each other, but is that real?
pairs(Vote.emt)
contrast estimate SE df z.ratio p.value
Democrat Year.s0 - Nonvoting Year.s0 0.186 0.0176 Inf 10.573 <.0001
Democrat Year.s0 - Other Year.s0 0.473 0.0193 Inf 24.541 <.0001
Democrat Year.s0 - Republican Year.s0 0.078 0.0281 Inf 2.774 0.0283
Nonvoting Year.s0 - Other Year.s0 0.287 0.0180 Inf 15.997 <.0001
Nonvoting Year.s0 - Republican Year.s0 -0.108 0.0256 Inf -4.217 0.0001
Other Year.s0 - Republican Year.s0 -0.395 0.0287 Inf -13.775 <.0001
P value adjustment: tukey method for comparing a family of 4 estimates
If we look at the adjusted p values for every pairwise comparison of
trends, yes. Even adjusting for all multiple comparisons, every trend
differs from every other trend with a p of < 0.05. How can this be
interpreted? Typically, people say “significant” or “not significant”
and call it a day. There’s another interpretation of the p value, and
that’s the probability of the observed difference occuring by random
chance if the difference did not actually exist. For example,
if there really were no difference between Republican and Democrat
performance, according to this model, we would expect to still see the
difference we estimated about 2.8% of the time.
So, we can generally infer that the differences we think we see are
probably real. However, are they meaningful? The p value can’t tell us
how large a difference is. It can only tell us how
likely it is. Fortunately, standardized effect sizes exist to
explore this question.
Vote.eff <- eff_size(Vote.emt, sigma = sigma(Vote.model), edf = df.residual(Vote.model))
Vote.eff
contrast effect.size SE df asymp.LCL asymp.UCL
Democrat Year.s0 - Nonvoting Year.s0 0.0497 0.00470 Inf 0.04049 0.0589
Democrat Year.s0 - Other Year.s0 0.1266 0.00525 Inf 0.11625 0.1368
Democrat Year.s0 - Republican Year.s0 0.0209 0.00753 Inf 0.00611 0.0356
Nonvoting Year.s0 - Other Year.s0 0.0768 0.00493 Inf 0.06719 0.0865
Nonvoting Year.s0 - Republican Year.s0 -0.0288 0.00684 Inf -0.04223 -0.0154
Other Year.s0 - Republican Year.s0 -0.1057 0.00778 Inf -0.12093 -0.0904
sigma used for effect sizes: 3.739
Confidence level used: 0.95
These effect sizes are Cohen’s d, which boils down to how
far away the trends are from each other, adjusted by how uncertain the
estimates of the trends are. The numbers aren’t very informative by
themselves. Fortunately has been extensive work to define meaningful
categories of effect sizes for political science. I think that the
categories proposed by Erik
Ghneer Larsen are reasonable. However, he presented them in relation
to Pearson’s r. Fortunately, it is possible to convert between the two.
Unfortunately, the standard conversion of d to r is biased to produce
low outcomes, because it converts to the point-biserial r and presumes
that d was estimated with a binomial predictor. Year.s is a continuous
predictor. Therefore, I converted d to the probability of superiority
common language effect size and then converted this to r using the
formula found in Dunlap, 1994.
Vote.d_to_r <- data.frame(summary(Vote.eff)$contrast, summary(Vote.eff)$effect.size,
d_to_p_superiority(summary(Vote.eff)$effect.size),
sin(((d_to_p_superiority(summary(Vote.eff)$effect.size)) -.5) * pi))
colnames(Vote.d_to_r) <- c("Contrast", "d", "CLES", "r")
Vote.d_to_r <- Vote.d_to_r %>% mutate(Larsen = case_when(
abs(r) < 0.05 ~ "Less than Small Effect",
abs(r) >= 0.05 & abs(r) < 0.15 ~ "Small Effect",
abs(r) >= 0.15 & abs(r) < 0.30 ~ "Medium Effect",
abs(r) >= 0.30 ~ "Large Effect"
))
Vote.d_to_r
CLES is the “Common Language Effect Size” or “probability of
superiority”. This is an estimate of the probability (between 0 and 1)
that, for a given trait, a member of one group will be larger than a
member of the other group. A CLES of 0.5 means the groups are identical.
As you can see, none of the differences are large.
LS0tDQp0aXRsZTogIkFtZXJpY2FuIFByZXNpZGVudGlhbCBWb3RpbmcgSGFiaXRzLCBTcG90cyBhbmQgVHJlbmRzIg0Kb3V0cHV0OiBodG1sX25vdGVib29rDQphdXRob3I6ICJCcnlhbiBKLiBNYWxvbmV5Ig0KZGF0ZTogIjEzIE5vdmVtYmVyIDIwMjQiDQotLS0NCg0KIyMgSW50cm9kdWN0aW9uDQpXZSBqdXN0IGZpbmlzaGVkIGFub3RoZXIgcHJlc2lkZW50aWFsIGVsZWN0aW9uIHllYXIsIGZ1bGwgb2Ygc3VycHJpc2VzIGFuZCB1cHNldHMsIG9yIG1heWJlIHVwc2V0cyBhbmQgc3VycHJpc2VzLiBFaXRoZXIgd2F5LCB0aGUgdm90aW5nIHBhcnQgaXMgb3ZlciwgdGhlIGNvdW50aW5nIHBhcnQgaXMgb25nb2luZywgYW5kIHVubGlrZSAyMDIwLCB0aGUgdmFucXVpc2hlZCBoYWQgYSBzZW5zZSBvZiBkaWduaXR5IGFuZCBzYW5pdHkuIEFsbCBzb3J0cyBvZiBleHBlcnRzIGhhdmUgY3Jhd2xlZCBvdXQgZnJvbSB1bmRlciB0aGVpciByb2NrcyB0byBlaXRoZXIgY2xhaW0gY3JlZGl0IG9yIGFzc2lnbiBibGFtZS4gSSdtIG5vdCBpbnRlcmVzdGVkIGluIHRoYXQuIEknbSBpbnRlcmVzdGVkIGluIHRoZSBzdG9yeSBkYXRhIGNhbiB0ZWxsLg0KDQpJIGFza2VkIG15c2VsZiB3aGF0IFByZXNpZGVudGlhbCBlbGVjdGlvbnMgbG9va2VkIGxpa2UgZnJvbSBhICJwdXJlIiBudW1iZXJzIHBlcnNwZWN0aXZlLCBzbyBJIGNvbGxlY3RlZCBzb21lIGRhdGEuIFRoZSB2b3RpbmctZWxpZ2libGUgcG9wdWxhdGlvbiAoVkVQKSBjYW1lIGZyb20gdGhlIFtVbml2ZXJzaXR5IG9mIEZsb3JpZGEgRWxlY3Rpb24gTGFiXShodHRwczovL2VsZWN0aW9uLmxhYi51ZmwuZWR1L3ZvdGVyLXR1cm5vdXQvKS4gVGhlIHZvdGUgY291bnRzIGNhbWUgZnJvbSBlaXRoZXIgdGhlIFdpa2lwZWRpYSBwYWdlcyBmb3IgZWFjaCBlbGVjdGlvbiwgWzE5ODBdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpLzE5ODBfVW5pdGVkX1N0YXRlc19wcmVzaWRlbnRpYWxfZWxlY3Rpb24jUmVzdWx0c19ieV9zdGF0ZSksIFsxOTg0XShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS8xOTg0X1VuaXRlZF9TdGF0ZXNfcHJlc2lkZW50aWFsX2VsZWN0aW9uI1Jlc3VsdHNfYnlfc3RhdGUpLCBbMTk4OF0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvMTk4OF9Vbml0ZWRfU3RhdGVzX3ByZXNpZGVudGlhbF9lbGVjdGlvbiNSZXN1bHRzX2J5X3N0YXRlKSwgWzE5OTJdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpLzE5OTJfVW5pdGVkX1N0YXRlc19wcmVzaWRlbnRpYWxfZWxlY3Rpb24jUmVzdWx0c19ieV9zdGF0ZSksIFsxOTk2XShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS8xOTk2X1VuaXRlZF9TdGF0ZXNfcHJlc2lkZW50aWFsX2VsZWN0aW9uI1Jlc3VsdHNfYnlfc3RhdGUpLCBbMjAwMF0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvMjAwMF9Vbml0ZWRfU3RhdGVzX3ByZXNpZGVudGlhbF9lbGVjdGlvbiNSZXN1bHRzX2J5X3N0YXRlKSwgWzIwMDRdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpLzIwMDRfVW5pdGVkX1N0YXRlc19wcmVzaWRlbnRpYWxfZWxlY3Rpb24jUmVzdWx0c19ieV9zdGF0ZSksIFsyMDA4XShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS8yMDA4X1VuaXRlZF9TdGF0ZXNfcHJlc2lkZW50aWFsX2VsZWN0aW9uI1Jlc3VsdHNfYnlfc3RhdGUpLCBbMjAxMl0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvMjAxMl9Vbml0ZWRfU3RhdGVzX3ByZXNpZGVudGlhbF9lbGVjdGlvbiNSZXN1bHRzX2J5X3N0YXRlKSwgWzIwMTZdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpLzIwMTZfVW5pdGVkX1N0YXRlc19wcmVzaWRlbnRpYWxfZWxlY3Rpb24jUmVzdWx0c19ieV9zdGF0ZSksIGFuZCBbMjAyMF0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvMjAyMF9Vbml0ZWRfU3RhdGVzX3ByZXNpZGVudGlhbF9lbGVjdGlvbiNSZXN1bHRzX2J5X3N0YXRlKSBvciB0aGUgW0VsZWN0aW9uIEF0bGFzIHN0YXRlIHBhZ2VzIGZvciAyMDI0XShodHRwczovL3VzZWxlY3Rpb25hdGxhcy5vcmcvUkVTVUxUUy8pLiBUaGUgMjAyNCBudW1iZXJzIGFyZSBhbGwgcHJlbGltaW5hcnkgYW5kIHNob3VsZCBiZSBpbnRlcnByZXRlZCB3aXRoIGNhdXRpb24uIEkgYWxzbyBjb2xsZWN0ZWQgdGhlIG51bWJlciBvZiBFbGVjdG9yYWwgQ29sbGVnZSB2b3RlcyBlYWNoIHN0YXRlIGhhZCBpbiBlYWNoIGVsZWN0aW9uIHRvIHVzZSBhcyB3ZWlnaHRzIGZvciBtb2RlbGluZy4gVGhpcyB3YXMgbG9va2VkIHVwIGZyb20gc2V2ZXJhbCBzb3VyY2VzLCBbaW5jbHVkaW5nIFdpa2lwZWRpYV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvVW5pdGVkX1N0YXRlc19FbGVjdG9yYWxfQ29sbGVnZSNDaHJvbm9sb2dpY2FsX3RhYmxlKS4NCg0KVGhlIGRhdGEgY29uc2lzdGVkIG9mIGVsZWN0aW9uIHllYXIsIHN0YXRlIG5hbWVzLCBhbmQgY291bnRzIG9mIFZFUCBhbmQgdm90ZXMgZm9yIHRoZSBEZW1vY3JhdCwgUmVwdWJsaWNhbiwgYW5kIE90aGVyIGNhbmRpZGF0ZXMuIEkgc3VidHJhY3RlZCB0aGUgc3VtIG9mIHRoZSB2b3RlcyBmcm9tIHRoZSBWRVAgdG8gZ2V0IE5vbnZvdGluZyBjb3VudHMuIFRoaXMgd2FzIGFsbCBkb25lIGluIEV4Y2VsLCB3aGVyZSBJIGFsc28gZm9ybWF0dGVkIHRoZSBkYXRhIGludG8g4oCcbG9uZ+KAnSBmb3JtYXQuIEkgbG9hZGVkIHRoZSBjbGVhbiBkYXRhLiBZb3UgY2FuIGdldCBpdCwgeW91cnNlbGYgW2luIHRoaXMgR29vZ2xlIHNoZWV0XShodHRwczovL2RvY3MuZ29vZ2xlLmNvbS9zcHJlYWRzaGVldHMvZC8xb3lTelBrbWFWTUJISHdBaWI0a0x1Y2FfN1hRclhnNXg2N2hRbUxLVXBUQS9lZGl0P3VzcD1zaGFyaW5nIikuDQoNCkJlZm9yZSBJIGdvIGFueSBmdXJ0aGVyLCBJIHdpbGwgYmUgdXNpbmcgUiBmb3IgdGhpcyB3b3JrLCBzbyBJIG5lZWQgdG8gbG9hZCBhIGJ1bmNoIG9mIFIgcGFja2FnZXMuDQoNCmBgYHtyfQ0KbGlicmFyeShnb29nbGVzaGVldHM0LCBxdWlldGx5ID0gVFJVRSkNCmxpYnJhcnkoZ2xtbVRNQiwgcXVpZXRseSA9IFRSVUUpDQpsaWJyYXJ5KHBlcmZvcm1hbmNlLCBxdWlldGx5ID0gVFJVRSkNCmxpYnJhcnkoY2FyLCBxdWlldGx5ID0gVFJVRSkNCmxpYnJhcnkocGVyZm9ybWFuY2UsIHF1aWV0bHkgPSBUUlVFKQ0KbGlicmFyeShNdU1JbiwgcXVpZXRseSA9IFRSVUUpDQpsaWJyYXJ5KGVtbWVhbnMsIHF1aWV0bHkgPSBUUlVFKQ0KbGlicmFyeShlZmZlY3RzaXplLCBxdWlldGx5ID0gVFJVRSkNCmxpYnJhcnkodGlkeXZlcnNlLCBxdWlldGx5ID0gVFJVRSkNCmxpYnJhcnkocmVzaGFwZTIsIHF1aWV0bHkgPSBUUlVFKQ0KYGBgDQpgYGB7cn0NClZvdGUuZGF0IDwtIHJlYWRfc2hlZXQoImh0dHBzOi8vZG9jcy5nb29nbGUuY29tL3NwcmVhZHNoZWV0cy9kLzFveVN6UGttYVZNQkhId0FpYjRrTHVjYV83WFFyWGc1eDY3aFFtTEtVcFRBL2VkaXQiKQ0KVm90ZS5kYXQkU3RhdGUgPC0gZmFjdG9yKFZvdGUuZGF0JFN0YXRlKQ0KVm90ZS5kYXQkU2VsZWN0aW9uIDwtIGZhY3RvcihWb3RlLmRhdCRTZWxlY3Rpb24pDQpgYGANCg0KRmVlbCBmcmVlIHRvIHVzZSB0aGUgYWJvdmUgbGluayB0byBkbyBhIG1hbnVhbCBpbXBvcnQgaWYgeW91J2QgcHJlZmVyIHRoYXQuDQoNClNvLCBsZXQncyB0YWtlIGEgbG9vayBhdCB0aGUgcmF3IGRhdGEuDQoNCmBgYHtyfQ0KbWVzc3kucGxvdCA8LSBnZ3Bsb3QoVm90ZS5kYXQsIGFlcyh4ID0gaml0dGVyKFllYXIpLCB5ID0gVm90ZXMvVkVQLCBncm91cCA9IFNlbGVjdGlvbiwgY29sb3IgPSBTZWxlY3Rpb24pKSArIA0KICBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYygiIzQ0NzdBQSIsICIjMjI4ODMzIiwgIiNDQ0JCNDQiLCAiI0VFNjY3NyIpLA0KICAgICAgICAgICAgICAgICAgICAgbGltaXRzID0gYygiRGVtb2NyYXQiLCAiT3RoZXIiLCAiTm9udm90aW5nIiwgIlJlcHVibGljYW4iKSkgKw0KICBnZW9tX3BvaW50KHNpemUgPSAzKSArDQogIHRoZW1lKHBhbmVsLmJhY2tncm91bmQgPSBlbGVtZW50X3JlY3QoZmlsbCA9ICJ3aGl0ZSIpLA0KICAgICAgICBheGlzLmxpbmUgPSBlbGVtZW50X2xpbmUoc2l6ZSA9IDEsIGNvbG9yID0gImJsYWNrIikpICsNCiAgc2NhbGVfeV9jb250aW51b3VzKGxhYmVscyA9IHNjYWxlczo6cGVyY2VudCkgKw0KICBzY2FsZV94X2NvbnRpbnVvdXMoYnJlYWtzID0gYygxOTgwLCAxOTg0LCAxOTg4LCAxOTkyLCAxOTk2LCAyMDAwLCAyMDA0LCAyMDA4LCAyMDEyLCAyMDE2LCAyMDIwLCAyMDI0KSkgKw0KICB5bGFiKCJGcmVxdWVuY3kgKHBlciBWRVApIikgKyB4bGFiKCJZZWFyIikNCg0KbWVzc3kucGxvdA0KYGBgDQoNCiMjIFN1bW1hcml6aW5nDQoNClRoYXQncyBhIHJlYWwgbWVzcy4gTGV0J3MgdHJ5IGdlbmVyYXRpbmcgYSBzdW1tYXJ5Lg0KYGBge3J9DQojIFNvbWUgYmFja2dyb3VuZCBmdW5jdGlvbnMNCndlaWdodGVkLmF2ZXJhZ2UgPC0gZnVuY3Rpb24oeCwgdyl7DQojIyBTdW0gb2YgdGhlIHdlaWdodHMNCnN1bS53IDwtIHN1bSh3LCBuYS5ybSA9IFQpDQojIyBTdW0gb2YgdGhlIHdlaWdodGVkICR4X2kkDQp4dyA8LSBzdW0odyp4LCBuYS5ybSA9IFQpDQojIyBSZXR1cm4gdGhlIHdlaWdodGVkIGF2ZXJhZ2UNCnJldHVybih4dy9zdW0udykNCn0NCg0Kd2VpZ2h0ZWQuc2UubWVhbiA8LSBmdW5jdGlvbih4LCB3LCBuYS5ybSA9IFQpew0KIyMgUmVtb3ZlIE5Bcw0KaWYgKG5hLnJtKSB7DQppIDwtICFpcy5uYSh4KQ0KdyA8LSB3W2ldDQp4IDwtIHhbaV0NCn0NCiMjIENhbGN1bGF0ZSBlZmZlY3RpdmUgTiBhbmQgY29ycmVjdGlvbiBmYWN0b3INCm5fZWZmIDwtIChzdW0odykpXjIvKHN1bSh3XjIpKQ0KY29ycmVjdGlvbiA9IG5fZWZmLyhuX2VmZi0xKQ0KIyMgR2V0IHdlaWdodGVkIHZhcmlhbmNlDQpudW1lcmF0b3IgPSBzdW0odyooeC13ZWlnaHRlZC5hdmVyYWdlKHgsdykpXjIpDQpkZW5vbWluYXRvciA9IHN1bSh3KQ0KIyMgZ2V0IHdlaWdodGVkIHN0YW5kYXJkIGVycm9yIG9mIHRoZSBtZWFuDQpzZV94ID0gc3FydCgoY29ycmVjdGlvbiAqIChudW1lcmF0b3IvZGVub21pbmF0b3IpKS9uX2VmZikNCnJldHVybihzZV94KQ0KfQ0KDQojIE1ha2luZyB0aGUgd2VpZ2h0ZWQgc3VtbWFyeQ0KVm90ZS5zdW0gPC0gVm90ZS5kYXQgJT4lDQogIG11dGF0ZShwYXN0ZXVwID0gcGFzdGUoWWVhciwgU2VsZWN0aW9uKSkgJT4lDQogIG11dGF0ZShyYXRpbyA9IFZvdGVzL1ZFUCkgJT4lDQogIGdyb3VwX2J5KHBhc3RldXApICU+JQ0KICBzdW1tYXJpemUod2VpZ2h0ZWRfbWVhbnMgPSB3ZWlnaHRlZC5hdmVyYWdlKHJhdGlvLCBFbGVjdG9yYWwuVm90ZXMpKSAlPiUNCiAgY2JpbmQoVm90ZS5kYXQgJT4lDQogICAgICAgICAgbXV0YXRlKHBhc3RldXAgPSBwYXN0ZShZZWFyLCBTZWxlY3Rpb24pKSAlPiUNCiAgICAgICAgICBtdXRhdGUocmF0aW8gPSBWb3Rlcy9WRVApICU+JQ0KICAgICAgICAgIGdyb3VwX2J5KHBhc3RldXApICU+JQ0KICAgICAgICAgIHN1bW1hcml6ZSh3ZWlnaHRlZF9DTCA9IHdlaWdodGVkLnNlLm1lYW4ocmF0aW8sIEVsZWN0b3JhbC5Wb3RlcykgKiBxdCgwLjk3NSwgNTApKSkNClZvdGUuc3VtIDwtIFZvdGUuc3VtWywgYygxLCAyLCA0KV0NClZvdGUuc3VtIDwtIGNiaW5kKGNvbHNwbGl0KFZvdGUuc3VtJHBhc3RldXAsICIgIiwgYygiWWVhciIsICJTZWxlY3Rpb24iKSksIFZvdGUuc3VtWywyOjNdKQ0KVm90ZS5zdW0kU2VsZWN0aW9uIDwtIGZhY3RvcihWb3RlLnN1bSRTZWxlY3Rpb24pDQpWb3RlLnN1bQ0KYGBgDQoNCldoYXQgSSBkaWQgd2FzIHRha2Ugd2VpZ2h0ZWQgbWVhbnMgYnkgeWVhciBvZiBhbGwgdGhlIHN0YXRlcyBmb3Igdm90ZXMgY2FzdCBmb3IgdGhlIERlbW9jcmF0aWMgb3IgUmVwdWJsaWNhbiBjYW5kaWRhdGUgb3IgYW55IG90aGVyIGNhbmRpZGF0ZSwgYWxvbmcgd2l0aCB0aGUgcmVtYWluZGVyIChub24tdm90aW5nIFZFUCksIGRpdmlkZWQgYnkgdGhlIFZFUC4gVGhlIG1lYW5zIHdlcmUgd2VpZ2h0ZWQgYnkgdGhlIEVsZWN0b3JhbCBDb2xsZWdlIHZvdGVzIHRoYXQgc3RhdGUgaGFkIGluIHRoYXQgZWxlY3Rpb24geWVhci4gSSB0aGVuIGNhbGN1bGF0ZWQgdGhlIHdlaWdodGVkIHN0YW5kYXJkIGVycm9ycyBvZiB0aGUgbWVhbiwgdXNpbmcgYSBmdW5jdGlvbiB3cml0dGVuIGJ5IFtBbGV4IFN0ZXBoZW5zb25dKGh0dHBzOi8vd3d3LmFsZXhzdGVwaGVuc29uLm1lL3Bvc3QvMjAyMi0wNC0wMi13ZWlnaHRlZC12YXJpYW5jZS1pbi1yLykuIEkgY29udmVydGVkIHRoZXNlIHRvIDk1JSBjb25maWRlbmNlIGxpbWl0cyBieSBtdWx0aXBseWluZyBieSB0aGUgYXBwcm9wcmlhdGUgcXVhbnRpbGUgb2YgdCBhdCAwLjk3NSBhbmQgNTAgZGVncmVlcyBvZiBmcmVlZG9tLiBXaHkgZGlkIEkgd2VpZ2h0IHRoZSBwb3B1bGFyIHZvdGVzPyBJdCdzIGEgY29uY2Vzc2lvbiB0byByZWFsaXR5LiBQZW9wbGUgbWlnaHQgaGFuZC13cmluZyBhbmQgd2hpbmUgYWxsIHRoZXkgbGlrZSBhYm91dCBob3cgaG9ycmlkIHRoZSBFbGVjdG9yYWwgQ29sbGVnZSBtaWdodCBiZSwgYnV0IGl0J3Mgbm90IGdvaW5nIGF3YXkgYW55IHRpbWUgc29vbi4gVW5kZXJzdGFuZGluZyBVUyBwcmVzaWRlbnRpYWwgcG9saXRpY3MgcmVxdWlyZXMgdGFraW5nIGl0IGludG8gYWNjb3VudC4NCg0KU28sIEkgaGF2ZSBhIHRhYmxlLCBJIGRvbid0IGZpbmQgdGFibGVzIHVzZWZ1bC4gV2hhdCBhcmUgdXNlZnVsIGFyZSBjaGFydHMuDQpgYGB7cn0NCnN1bW1hcnkucGxvdCA8LSBnZ3Bsb3QoVm90ZS5zdW0sIGFlcyh4ID0gWWVhciwgeSA9IHdlaWdodGVkX21lYW5zLCBncm91cCA9IFNlbGVjdGlvbiwgY29sb3IgPSBTZWxlY3Rpb24pKSArIA0KICBnZW9tX2xpbmUobGluZXdpZHRoID0gMSkgKw0KICBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYygiIzQ0NzdBQSIsICIjMjI4ODMzIiwgIiNDQ0JCNDQiLCAiI0VFNjY3NyIpLA0KICAgICAgICAgICAgICAgICAgICAgbGltaXRzID0gYygiRGVtb2NyYXQiLCAiT3RoZXIiLCAiTm9udm90aW5nIiwgIlJlcHVibGljYW4iKSkgKw0KICBnZW9tX3BvaW50KHNpemUgPSAzKSArDQogIGdlb21fZXJyb3JiYXIoKGFlcyh5bWluPXdlaWdodGVkX21lYW5zIC0gd2VpZ2h0ZWRfQ0wsIHltYXggPSB3ZWlnaHRlZF9tZWFucyArIHdlaWdodGVkX0NMKSkpICsNCiAgdGhlbWUocGFuZWwuYmFja2dyb3VuZCA9IGVsZW1lbnRfcmVjdChmaWxsID0gIndoaXRlIiksDQogICAgICAgIGF4aXMubGluZSA9IGVsZW1lbnRfbGluZShzaXplID0gMSwgY29sb3IgPSAiYmxhY2siKSwNCiAgICAgICAgbGVnZW5kLnBvc2l0aW9uID0gYygwLjgsIDAuMykpICsNCiAgc2NhbGVfeV9jb250aW51b3VzKGxhYmVscyA9IHNjYWxlczo6cGVyY2VudCkgKw0KICBzY2FsZV94X2NvbnRpbnVvdXMoYnJlYWtzID0gYygxOTgwLCAxOTg0LCAxOTg4LCAxOTkyLCAxOTk2LCAyMDAwLCAyMDA0LCAyMDA4LCAyMDEyLCAyMDE2LCAyMDIwLCAyMDI0KSkgKw0KICB5bGFiKCJGcmVxdWVuY3kgKHBlciBWRVApIikNCg0Kc3VtbWFyeS5wbG90DQpgYGANCg0KSSBkaWRuJ3QganVzdCBwdWxsIHRoZSBjb2xvcnMgb3V0IG9mIG15IGN1dGUgcGF0b290aWUuIFRoZXkncmUgZnJvbSBQYXVsIFRvbCdzIHNpdGUgdGhhdCBkZXNjcmliZXMgbWFueSB3b25kZXJmdWwgKGFuZCBjb2xvci1ibGluZCBzYWZlKSBbY29sb3Igc2NoZW1lc10oaHR0cHM6Ly9wZXJzb25hbC5zcm9uLm5sL35wYXVsdC8pLCBhcnJhbmdlZCB3aXRoIHRoZSBub3ctdHJhZGl0aW9uYWwgY29sb3Jpbmcgb2YgYmx1ZSBmb3IgRGVtb2NyYXRzIGFuZCByZWQgZm9yIFJlcHVibGljYW5zLg0KDQojIyBBIGxpdHRsZSBub24tc3RhdGlzdGljYWwgYW5hbHlzaXMNCldlIGNhbiBhbmFseXplIHRoZSBhYm92ZSBjaGFydCB3aXRob3V0IGV2ZW4gY29uc2lkZXJpbmcgc3RhdGlzdGljcy4gSSdtIGdvaW5nIHRvIGJlIG1vc3RseSBqdXN0IGRlc2NyaXB0aXZlLiBUaGUgZmlyc3QgdGhpbmcgdG8gbm90ZSBpcyB0aGF0IHRoZSBtb3N0IHBvcHVsYXIg4oCcY2FuZGlkYXRl4oCdLCBieSBmYXIsIGZvciB0aGUgbGFzdCA0NCB5ZWFycyBoYXMgYmVlbiBub2JvZHkuIFNpbmNlIGF0IGxlYXN0IDE5ODAsIHRoZXJlIGhhcyBuZXZlciBiZWVuIGEgcHJlc2lkZW50IGVsZWN0ZWQgYnkgdGhlIG1ham9yaXR5IG9mIEFtZXJpY2FucyBlbGlnaWJsZSB0byB2b3RlLiBMZXQgdXMgd3JpbmcgaGFuZHMuIEhhbmQgd3JpbmdpbmcgdGltZSBpcyBvdmVyLiBXaGlsZSBpdOKAmXMgdHJ1ZSB0aGF0IOKAnGdldCBvdXQgb2YgbXkgZmFjZeKAnSBoYXMgdXN1YWxseSBiZWVuIHRoZSBtb3N0IHBvcHVsYXIgc2VsZWN0aW9uLCBpdOKAmXMgZXF1YWxseSB0cnVlIHRoYXQgdGhpcyBvcHRpb24gaGFzIGJlZW4gY2hvc2VuIGxlc3MgYW5kIGxlc3MgYXMgdGltZSBoYXMgZ29uZSBvbi4gV2hhdGV2ZXIgaXMgd3Jvbmcgd2l0aCBBbWVyaWNhLCBtb3JlIEFtZXJpY2FucyBoYXZlIGJlZW4gZ2V0dGluZyBvdXQgdG8gdm90ZS4gVHJ1ZSBlbm91Z2gsIDIwMjQgd2FzIGFub3RoZXIgZWxlY3Rpb24gb2YgYXBhdGh5LCBidXQgaXTigJlzIGhhcmQgdG8gY2FsbCB0aGUgZnV0dXJlIGJhc2VkIG9uIGEgc2luZ2xlLXBvaW50IGRlZmxlY3Rpb24uIFRoZSBvdmVyYWxsIHRyZW5kIGhhcyBiZWVuIGZvciBub24tdm90aW5nIHRvIGRyb3AgYnkgYSBsaXR0bGUgbGVzcyB0aGFuIDEgcGVyY2VudGFnZSBwb2ludCBldmVyeSBlbGVjdGlvbi4gVGhlIERlbW9jcmF0cyBhbmQgUmVwdWJsaWNhbnMgYXJlIGJhc2ljYWxseSBpbiBhIGRlYWQgaGVhdCBpbiBkaXZpZGluZyB0aGlzIGdhaW4gdXAgYmV0d2VlbiB0aGVtc2VsdmVzLiBUaGUgb3ZlcmFsbCBEZW1vY3JhdCBnYWluIHNpbmNlIDE5ODAgaGFzIGJlZW4gZ3JlYXRlciB0aGFuIHRoZSBvdmVyYWxsIFJlcHVibGljYW4gZ2FpbiwgYnV0IGN1cnJlbnRseSB0aGV5IGFyZSBpbmRpc3Rpbmd1aXNoYWJsZS4gT24gdGhlIG90aGVyIGhhbmQsIHRoZSBzbG93IGluY3JlYXNlIGluIHZvdGluZyBkb2VzbuKAmXQgc2VlbSB0byBoYXZlIGJlZW4gZGlzdHJpYnV0ZWQgdG8g4oCcT3RoZXLigJ0gb3B0aW9ucy4NCg0KQW5vdGhlciB0aGluZyB0byBub3RlIGlzIHRoYXQgdGhlIFJlcHVibGljYW5zIGFuZCBEZW1vY3JhdHMgc2VlbSB0byBoYXZlIHN3aXRjaGVkIHBlcmZvcm1hbmNlcyBmcm9tIDE5ODAgdG8gdGhlIHByZXNlbnQuIEZyb20gMTk4MCB0byAyMDA0LCBSZXB1YmxpY2FuIHBlcmZvcm1hbmNlIGZsdWN0dWF0ZWQgd2lsZGx5IGZyb20geWVhciB0byB5ZWFyLiBUaGUgRGVtb2NyYXRzLCBvbiB0aGUgb3RoZXIgaGFuZCwgd2VyZSBleHBlcmllbmNpbmcgdmVyeSBzbG93IGJ1dCB2ZXJ5IHJlZ3VsYXIgZ2FpbnMgdW50aWwgMjAwMC4gRnJvbSB0aGVzZSBpbmZsZWN0aW9uIHBvaW50cyB0byB0aGUgcHJlc2VudCBkYXksIHRoaXMgaGFzIHN3aXRjaGVkLiBUaGUgRGVtb2NyYXRzIG5vdyBoYXZlIHdpbGQgc3dpbmdzIHdoaWxlIHRoZSBSZXB1YmxpY2Fucycgb3NjaWxsYXRpb25zIGhhdmUgY2FsbWVkIGRvd24gKGFsdGhvdWdoIHdpdGggbW9yZSB2YXJpYXRpb24gdGhhbiB0aGUgZWFybHktcGVyaW9kIERlbW9jcmF0cyBzaG93ZWQpLiBIb3cgdG8gZXhwbGFpbiB0aGlzIG9yIHdoYXQgaXQgbWVhbnMgZm9yIHRoZSBmdXR1cmUgSSBsZWF2ZSB0byBtb3JlIGtub3dsZWRnZWFibGUgcGVvcGxlLiBXaGF0IEkgY2FuIHNheSBpcyB0aGF0IGl0IGxvb2tzIGxpa2UgYm90aCBwYXJ0aWVzIGV4cGVyaWVuY2VkIGEgc2lnbmlmaWNhbnQgc2VhIGNoYW5nZSByb3VnaGx5IDIwMDAtMjAwNCwgYW5kIGl0IGlzIGhhdmluZyBsb25nLWxhc3RpbmcgZWZmZWN0cyBvbiBlYWNoIG9mIHRoZW0uIFdoYXQgdGhlc2UgY2hhbmdlcyB3ZXJlLCBJIGNhbuKAmXQgZXZlbiBzcGVjdWxhdGUuIEZlZWwgZnJlZS4NCg0KIyMjIENhbGxpbmcgdGhlIEhvcnNlIFJhY2VzDQpMZXQncyBkaXZlIGRlZXBlciBhbmQgbG9vayBhdCBlYWNoIGVsZWN0aW9uIGluIHR1cm4uIFJlYWdhbuKAmXMgdGVudXJlIHdhcyBtYXJrZWQgYnkgbW92aW5nIGZyb20gc3Ryb25nIHRvIHN0cm9uZ2VyLiBVbnVzdWFsbHkgZm9yICBpbmN1bWJlbnRzLCBoZSBnYWluZWQgcGVyY2VudGFnZSBpbiBoaXMgcmUtZWxlY3Rpb24gYmlkLiBIb3dldmVyLCB0aGlzIHdhcyBub3QgYXQgdGhlIGV4cGVuc2Ugb2YgTW9uZGFsZSwgd2hvIGFsc28gZ290IGEgKHZlcnkgdGlueSkgZ2FpbiB2cy4gQ2FydGVyJ3MgcGVyZm9ybWFuY2UuIEluc3RlYWQsIFJlYWdhbiBtYXkgaGF2ZSBhdHRyYWN0ZWQgc29tZSBub24tdm90ZXJzIGFuZCBhIGxhcmdlIHBvcnRpb24gb2YgcGVvcGxlIHdobyBoYWQgdm90ZWQgZm9yIEFuZGVyc29uICh3aG8/KS4gUG9wcHkgQnVzaOKAmXMgZWxlY3Rpb24gd2FzIGFuIGV4ZXJjaXNlIGluIGNvYXQtdGFpbCByaWRpbmcuIFRoZSBub252b3RlcnMgYm91bmNlZCBiYWNrIHVwLCBhbmQgRHVrYWtpcyBnb3QgYW5vdGhlciAodmVyeSB0aW55KSBnYWluIG92ZXIgdGhlIERlbW9jcmF0c+KAmSBzaGFyZSBvZiB0aGUgcHJldmlvdXMgZWxlY3Rpb24sDQoNCkluZGVlZCwgZnJvbSAxOTgwIHRvIDIwMDAg4oCcYSB2ZXJ5IHRpbnkgZ2FpbuKAnSBjb3VsZCBhY2N1cmF0ZWx5IGRlc2NyaWJlIHRoZSBEZW1vY3JhdGljIHBhcnR54oCZcyBlbnRpcmUgcHJlc2lkZW50aWFsIHBlcmZvcm1hbmNlLCBldmVuIGluIHllYXJzIHRoZXkgdG9vayB0aGUgV2hpdGUgSG91c2UuIEluIGV2ZXJ5IGVsZWN0aW9uLCBpdCB3YXMgYSB2ZXJ5IHNtYWxsIGltcHJvdmVtZW50IGluIGF0dHJhY3RpbmcgYSBwcm9wb3J0aW9uIG9mIHRoZSB2b3RpbmctZWxpZ2libGUgcG9wdWxhdGlvbiwgd2hpbGUgdGhlIFJlcHVibGljYW5zIHNlZS1zYXdlZCB1cCBhbmQgZG93biBpbiBwb3B1bGFyaXR5LiBGb3IgZXhhbXBsZSwgQ2xpbnRvbiB0b29rIG9mZmljZSBpbiBhIHRpZGFsIHdhdmUgb2Ygc3BvaWxhZ2UuIFJvc3MgUGVyb3TigJlzIGxlYXAgc2VlbXMgdG8gaGF2ZSBiZWVuIGZ1ZWxlZCBieSBzb21lIFJlcHVibGljYW4tbGVhbmluZyB2b3RlcnMgYW5kIGJpZ2dlciBjaHVuayBvZiBwZW9wbGUgd2hvIG90aGVyd2lzZSB3b3VsZG7igJl0IGhhdmUgYm90aGVyZWQgdG8gdm90ZSBhdCBhbGwuIEJ1dCBQZXJvdCB3YXMgYSBnYWRmbHkgY2FuZGlkYXRlLiBCdXNoIHdhcyB0aGUgYW5vaW50ZWQgbGVhZGVyIG9mIHRoZSBSZXB1YmxpY2FuIHBhcnR5LCBzbyB0aGVyZSB3YXMgYSBzcXVhYmJsZSBvbiB0aGF0IHNpZGUgb2YgdGhlIHBvbGl0aWNhbCBzcGVjdHJ1bS4gQ2xpbnRvbiByb2RlIGluIG9uIHRoaXMgZGlzdW5pdHkuDQoNClBlcm904oCZcyBodWdlLWZvci1hLXRoaXJkLWR1ZGUgc2hvd2luZyBjYW5ub3QgYmUgZXhwbGFpbmVkIG9ubHkgYnkgYXR0cmFjdGluZyBub252b3RlcnMuIEluIDE5OTgsIHRoZXJlIHdhcyBhIGdpYW50IGxlYXAgaW4gYXBhdGh5LCB3aGljaCBwcm9iYWJseSBodXJ0IERvbGUgbW9yZSB0aGFuIGl0IGh1cnQgQ2xpbnRvbi4gSG93ZXZlciwgQ2xpbnRvbiBkaWRu4oCZdCBleHBlcmllbmNlIGEgc3Bpa2UgaW4gcG9wdWxhcml0eS4gSGUgbWFuYWdlZCB0byBkbyBhIGJpdCBiZXR0ZXIgdGhhbiBsYXN0IHRpbWUuIFRoaXMgc2V0IHRoZSBzdGFnZSBmb3IgMjAwMC4NCg0KVGhpcyB3YXMgYW4gZXh0cmVtZWx5IGNsb3NlIGVsZWN0aW9uLiBXaGF0IGlzIGludGVyZXN0aW5nIGlzIHRoYXQgdGhlcmUgd2FzIHZlcnkgbGl0dGxlIGFwcGFyZW50IG1vdGl2YXRpb24gb2YgdGhlIG5vbnZvdGVycy4gSXQgZW5kZWQgdXAgaW4gYSBtZXNzLiBCdXNo4oCZcyByZS1lbGVjdGlvbiB3YXMgbW9yZSBkZWZpbml0ZSBhbmQgaXQgd2FzIGFjY29tcGFuaWVkIGJ5IHRoZSBEZW1vY3JhdGljIHBhcnR5IHN0YXJ0aW5nIHRvIGltcHJvdmUgbW9yZSBxdWlja2x5IGl0cyBzbHVnLWxpa2UgY3Jhd2wgdXB3YXJkcyBzaW5jZSAxOTgwLiBIb3dldmVyLCBCdXNoIHdhcyBhYmxlIHRvIGF0dHJhY3QgbW9yZSBvZiB0aGUgbm9udm90ZXJzIChhbmQgcG9zc2libHkgT3RoZXIgdm90ZXJzKSB0aGFuIGRpZCBLZXJyeS4gV2hhdCBpcyBpbnRlcmVzdGluZyBpcyB0aGF0IHRoaXMgZWxlY3Rpb24gc2F3IGEgYmlnIGRyb3AgaW4gbm9udm90ZXJzLCB3aGljaCBnb2VzIGFnYWluc3QgdGhlIGdlbmVyYWwgdHJlbmQgb2YgYW4gaW5jcmVhc2UgaW4gYXBhdGh5IG9yIGF0IHZlcnkgYmVzdCBhIHNtYWxsIGRlY3JlYXNlIGR1cmluZyBhIHJlLWVsZWN0aW9uIGF0dGVtcHQuIEhvd2V2ZXIsIHRoZXNlIHZvdGVycyBkaWRu4oCZdCB0dXJuIG91dCBmb3IgS2VycnkuDQoNCkluIDIwMDgsIE9iYW1hIGVuZGVkIHVwIGNvbnRpbnVpbmcgYSBzdHJhaWdodC1saW5lIHRyZW5kIHN0YXJ0ZWQgYnkgdGhlIHByaW9yIHR3byBmYWlsZWQgRGVtb2NyYXQgYXR0ZW1wdHMuIFRoZXJlIHdhcyBhIHNtYWxsIHRlbmQgZG93bndhcmRzIGluIG5vbnZvdGVycywgYnV0IHRoZSBSZXB1YmxpY2FucyBsb3N0IHNoYXJlIGFzIHdlbGwsIGluZGljYXRpbmcgc29tZSBwb3NzaWJsZSBhaXNsZS1jcm9zc2luZyBieSB2b3RlcnMuIE9iYW1h4oCZcyBzZWNvbmQgdHVybiB3YXMgYSB0eXBpY2FsIHN1Y2Nlc3NmdWwgcmUtZWxlY3Rpb24sIHdpdGggbW9yZSBwZW9wbGUgY2hvb3NpbmcgdG8gc2l0IGl0IG91dCwgYnV0IG5vdCBlbm91Z2ggdG8gZm9yY2UgYSBsb3NzLiBUaGUgUmVwdWJsaWNhbnMgd2VyZSBzdGlsbCBleHBlcmllbmNpbmcgdGhlIEJ1c2ggMi1pbnNwaXJlZCBkZWNsaW5lLg0KDQoyMDE2IHdhcyBhbm90aGVyIGNsb3NlIGVsZWN0aW9uLiBJdOKAmXMgd2VsbC1kb2N1bWVudGVkIHRoYXQgVHJ1bXAgZGlkIG5vdCBhY2hpZXZlIHRoZSB0b3RhbCBwb3B1bGFyIG1ham9yaXR5IG9mIHZvdGVzIGNhc3QuIFRoZSB3ZWlnaHRlZCBtZWFucyBzdGlsbCByZWZsZWN0IHRoaXMuIFRoaXMgaXMgbm90IGFub21hbG91cy4gSXQgaXMgYmVjYXVzZSB3ZWlnaHRlZCBtZWFucyBjYW4gImRpc3RyaWJ1dGUiIHByb3BvcnRpb25hbCBFbGVjdG9yYWwgb3V0Y29tZXMgd2l0aGluIGVhY2ggc3RhdGUgYXMgd2VsbCBhcyBmcmFjdGlvbnMuIFdoYXQgd2UgZG8gc2VlIGlzIHRoYXQgR2FyeSBKb2huc29uIGhhZCBhIGJpdCBvZiBhIHNwaWtlLCB3aGljaCBtYXkgaGF2ZSBiZWVuIGFjcXVpcmVkIGZyb20gYSBtaXh0dXJlIG9mIG5vbnZvdGVycyBhbmQgYm90aCBEZW1vY3JhdCBhbmQgUmVwdWJsaWNhbiBsZWFuaW5nIHZvdGVycy4NCg0KVGhlIEJpZGVuIGVsZWN0aW9uIGlzIGlsbHVzdHJhdGl2ZSBvZiB3aGF0IG1pZ2h0IGJlIGFuIGltcG9ydGFudCB0cmVuZC4gT25jZSBhZ2FpbiwgdGhlIG51bWJlciBvZiBub252b3RlcnMgZmVsbCAoKnRvIGl0cyBsb3dlc3QgbGV2ZWwqKSwgYW5kIHRoZSBwcmltYXJ5IGJlbmVmaWNpYXJ5IHNlZW1zIHRvIGhhdmUgYmVlbiBCaWRlbi4gSXQgd2FzIHRoZSBmaXJzdCB0aW1lIGluIHRoZSBwZXJpb2QgSSBsb29rZWQgYXQgdGhhdCBhbiBhY3R1YWwgY2FuZGlkYXRlIG1hbmFnZWQgdG8gZ2V0IG1vcmUgcG9wdWxhciB2b3RlcyB0aGFuIOKAnHRoaXMgZWxlY3Rpb24gaXNu4oCZdCB3b3J0aCB2b3RpbmcgaW7igJ0hDQoNCkZpbmFsbHksIElpdCBzaG91bGQgYmUgbm90ZWQgdGhhdCB0aGUgMjAyNCByZXN1bHRzIGFyZSBiYXNlZCBpbiBwYXJ0IG9uIHByb2plY3Rpb25zIGFuZCBlc3RpbWF0ZXMgYW5kIG91Z2h0IG5vdCBiZSBvdmVyLWludGVycHJldGVkLiBIb3dldmVyLCB0aGUgYmlnZ2VzdCB3aW5uZXIgaXMg4oCcTm9ib2R5IGlzIHdvcnRoIG15IHZvdGUu4oCdIEFwYXRoeSBhbmQgZGlzZW5nYWdlbWVudCB3b24gYWdhaW4sIHdpdGggb25lIG9mIGl0cyBiaWdnZXN0IHNwaWtlcyBzaW5jZSAxOTk2LiBIb3dldmVyLCB0aGlzIHRpbWUsIHRoZSBSZXB1YmxpY2FucyB3ZXJlbuKAmXQgcmlkaW5nIGEgbWFzc2l2ZSBib3R0b20tb3V0IHRyZW5kLiBJZiB0aGUgcHJvamVjdGlvbnMgYXJlIHRvIGJlIGJlbGlldmVkLCBUcnVtcCBhbHNvIGxvc3Qgc2hhcmUgZnJvbSBoaXMgcHJldmlvdXMgYXR0ZW1wdC4gSGUgbWVyZWx5IGxvc3QgbGVzcyB0aGFuIGRpZCBIYXJyaXMgdnMuIEJpZGVuJ3MgcGVyZm9ybWFuY2UuIFRydW1wIG1hbmFnZWQgdG8gZ2V0IGxlc3Mgb2YgdGhlIHZvdGluZy1lbGlnaWJsZSBwdWJsaWMgdGhhbiBoZSBkaWQgaW4gMjAyMC4gVW5mb3J0dW5hdGVseSwgSGFycmlzIG1hbmFnZWQgdG8gZG8gZXZlbiB3b3JzZS4gSSBjYW7igJl0IGV4cGxhaW4gd2h5LiBJIHdvbuKAmXQgdHJ5IHRvIGV4cGxhaW4gd2h5LiBJIGNhbiBvbmx5IHNob3cgdGhlIHdoYXQuIFRoZSB3aHkgcmVxdWlyZXMgbW9yZSBpbi1kZXB0aCBrbm93bGVkZ2UgdGhhbiBJIGhhdmUuDQoNCg0KIyMgTW9kZWxpbmcNCg0KV2UgY291bGQgc3RvcCByaWdodCB0aGVyZSwgYnV0IEkgbG9va2VkIGF0IHRoZSBjaGFydCBhbmQgd29uZGVyZWQgaWYgdGhlcmUgd2VyZSBhbnkgcG9zc2libHkgdmVyaWZpYWJsZSB0cmVuZHMgaW4gd2hhdCBJIHRoaW5rIEkgc2F3LiBUaGF0J3Mgd2hlbiBJIHB1bGxlZCBvdXQgdGhlIGZvcm1hbCBtb2RlbGluZyB0b29scy4NCg0KVGhlIGRhdGEgd2FzIHJlcGVhdGVkIG1lYXN1cmVtZW50cyAoYnkgc3RhdGUgYW5kIERpc3RyaWN0IG9mIENvbHVtYmlhKSBvZiAxMiBwcmVzaWRlbnRpYWwgZWxlY3Rpb25zLiBXaGVuIEkgc2VlIHJlcGVhdGVkIG1lYXN1cmVzLCBJIGxpa2UgdG8gdXNlIGEgbWl4ZWQtbGV2ZWwgbW9kZWwuIElmIHlvdSB3YW50IHRvIGtub3cgbW9yZSwgW3lvdSBjYW4gc3RhcnQgaGVyZV0oaHR0cHM6Ly92c25pLmNvLnVrL2Jsb2dzL21peGVkLW1vZGVscy1mb3ItcmVwZWF0ZWQtbWVhc3VyZXMtYW5kLWxvbmdpdHVkaW5hbC1kYXRhLykuIFRoZSBkYXRhIGlzIGNvdW50IGRhdGEuIFllcywgSSBwcmVzZW50ZWQgaXQgYXMgZnJlcXVlbmNpZXMsIGJ1dCBhIGZyZXF1ZW5jeSBpcyBhIGNvdW50IGRpdmlkZWQgYnkgYSByZWZlcmVuY2UgY291bnQuIFRoZSBwcmVmZXJyZWQgd2F5IHRvIGFuYWx5emUgY291bnQgZGF0YSBpcyB3aXRoIGEgUG9pc3NvbiBkaXN0cmlidGlvbi4gSG93ZXZlciwgdGhlIFBvaXNzb24gZGlzdHJpYnV0aW9uIHByZXN1bWVzIHRoYXQgdGhlIG1lYW4gYW5kIHRoZSB2YXJpYW5jZSBvZiB0aGUgZGF0YSBhcmUgdGhlIHNhbWUsIG9yIGRpc3BlcnNpb24gPSAxLCB0byB1c2UgamFyZ29uLiBBIGRpc3BlcnNpb24gb3ZlciAxIGlzIG92ZXJkaXNwZXJzZWQsIGFuZCB1bmRlciAxIGlzIHVuZGVyZGlzcGVyc2VkLiBTaW5jZSBJIGRpZCBub3QgeWV0IGtub3cgaWYgdGhlIGRhdGEgd2VyZSBvdmVyIG9yIHVuZGVyLWRpc3BlcnNlZCwgSSBjaG9zZSBSJ3MgZ2xtbVRNQiBwYWNrYWdlLCBzaW5jZSBpdCBoYXMgb3B0aW9ucyB0aGF0IGNhbiBoYW5kbGUgZWl0aGVyIGV2ZW50dWFsaXR5LiBJIHN0YXJ0ZWQgd2l0aCBhIGZpcnN0IHBhc3MgbW9kZWwgdGhhdCB1c2VkIHRoZSBQb2lzc29uIGZhbWlseS4NCg0KYGBge3J9DQpvcHRpb25zKGNvbnRyYXN0cyA9IHJlcCgiY29udHIuU3VtIiwyKSkNCnN0YXJ0Lm1vZCA8LSBnbG1tVE1CKFZvdGVzIH4gU2VsZWN0aW9uICogWWVhciArICgxIHwgU3RhdGUpICsgb2Zmc2V0KGxvZyhWRVApKSwgd2VpZ2h0cyA9IEVsZWN0b3JhbC5Wb3RlcywgZmFtaWx5ID0gcG9pc3NvbiwgZGF0YSA9IFZvdGUuZGF0KQ0KYGBgDQpJIGdvdCBhIHdhcm5pbmcuIEl0J3Mgb2Z0ZW4gZHVlIHRvIGRpZmZlcmVuY2VzIGluIHZhcmlhYmxlIHNjYWxlLiBUaGUgb2ZmZW5kaW5nIHZhcmlhYmxlIGlzIHByb2JhYmx5IFllYXIsIHdoaWNoIGNhbiBiZSBmaXhlZC4NCmBgYHtyfQ0KVm90ZS5kYXQkWWVhci5zIDwtIHNjYWxlKFZvdGUuZGF0JFllYXIpDQpzdGFydC5tb2QgPC0gZ2xtbVRNQihWb3RlcyB+IFNlbGVjdGlvbiAqIFllYXIucyArICgxIHwgU3RhdGUpICsgb2Zmc2V0KGxvZyhWRVApKSwgd2VpZ2h0cyA9IEVsZWN0b3JhbC5Wb3RlcywgZmFtaWx5ID0gcG9pc3NvbiwgZGF0YSA9IFZvdGUuZGF0KQ0KYGBgDQoNCkFuZCBJIGhhZCB0byBjaGVjayB0aGUgZGlzcGVyc2lvbi4NCg0KYGBge3J9DQpjaGVja19vdmVyZGlzcGVyc2lvbihzdGFydC5tb2QpDQpgYGANCg0KWWVzLCBkZWZpbml0ZWx5IG92ZXJkaXNwZXJzZWQuIFRoZXJlIGFyZSBmYW5jeSB3YXlzIHRvIGhhbmRsZSB0aGlzLCBsaWtlIHRoZSBDb253YXktTWF4d2VsbC1Qb2lzc29uIGZhbWlseSwgYnV0IHRoZSBjdXJyZW50IGltcGxlbWVudGF0aW9uIG9mIHRoaXMgZmFtaWx5IGNhbiBiZSBzbG93ZXIgdGhhbiBhIGdlb2xvZ2ljIGFnZSB0byBydW4uIEZvcnR1bmF0ZWx5LCB0aGUgcmVsaWFibGUgb2xkIG5lZ2F0aXZlIGJpbm9taWFsIGZhbWlseSB3aWxsIGhhbmRsZSBvdmVyZGlzcGVyc2lvbi4gU2luY2UgdGhlIHNhbXBsZSBoYWQgb3ZlciAyMDAwIGVudHJpZXMsIEkgZmVsdCBjb21mb3J0YWJsZSBleHBsb3JpbmcgZGlmZmVyZW50IHJhbmRvbSBlZmZlY3RzICp2aXMqIHRoZSBzdGF0ZXMuIEkgY29tcGFyZWQgbW9kZWxzIHRoYXQgaW5jbHVkZWQgcmFuZG9tIGludGVyY2VwdHMgYW5kIHJhbmRvbSBzbG9wZXMgZm9yIGJvdGggU2VsZWN0aW9uIGFuZCBZZWFyLCB3aXRoIGFuZCB3aXRob3V0IGNvcnJlbGF0ZWQgaW50ZXJjZXB0cy4NCg0KYGBge3J9DQpuYi5tb2QxIDwtIGdsbW1UTUIoVm90ZXMgfiBTZWxlY3Rpb24gKiBZZWFyLnMgKyAoMSB8IFN0YXRlKSArIG9mZnNldChsb2coVkVQKSksIHdlaWdodHMgPSBFbGVjdG9yYWwuVm90ZXMsIGZhbWlseSA9IG5iaW5vbTIsIGRhdGEgPSBWb3RlLmRhdCkNCm5iLm1vZDIgPC0gZ2xtbVRNQihWb3RlcyB+IFNlbGVjdGlvbiAqIFllYXIucyArIChTZWxlY3Rpb24gfCBTdGF0ZSkgKyBvZmZzZXQobG9nKFZFUCkpLCB3ZWlnaHRzID0gRWxlY3RvcmFsLlZvdGVzLCBmYW1pbHkgPSBuYmlub20yLCBkYXRhID0gVm90ZS5kYXQpDQpuYi5tb2QzIDwtIGdsbW1UTUIoVm90ZXMgfiBTZWxlY3Rpb24gKiBZZWFyLnMgKyAoWWVhci5zIHwgU3RhdGUpICsgb2Zmc2V0KGxvZyhWRVApKSwgd2VpZ2h0cyA9IEVsZWN0b3JhbC5Wb3RlcywgZmFtaWx5ID0gbmJpbm9tMiwgZGF0YSA9IFZvdGUuZGF0KQ0KbmIubW9kNCA8LSBnbG1tVE1CKFZvdGVzIH4gU2VsZWN0aW9uICogWWVhci5zICsgKFNlbGVjdGlvbiArIFllYXIucyB8IFN0YXRlKSArIG9mZnNldChsb2coVkVQKSksIHdlaWdodHMgPSBFbGVjdG9yYWwuVm90ZXMsIGZhbWlseSA9IG5iaW5vbTIsIGRhdGEgPSBWb3RlLmRhdCkNCm5iLm1vZDUgPC0gZ2xtbVRNQihWb3RlcyB+IFNlbGVjdGlvbiAqIFllYXIucyArIChTZWxlY3Rpb24gKiBZZWFyLnMgfCBTdGF0ZSkgKyBvZmZzZXQobG9nKFZFUCkpLCB3ZWlnaHRzID0gRWxlY3RvcmFsLlZvdGVzLCBmYW1pbHkgPSBuYmlub20yLCBkYXRhID0gVm90ZS5kYXQpDQpuYi5tb2Q2IDwtIGdsbW1UTUIoVm90ZXMgfiBTZWxlY3Rpb24gKiBZZWFyLnMgKyAoU2VsZWN0aW9uIHx8IFN0YXRlKSArIG9mZnNldChsb2coVkVQKSksIHdlaWdodHMgPSBFbGVjdG9yYWwuVm90ZXMsIGZhbWlseSA9IG5iaW5vbTIsIGRhdGEgPSBWb3RlLmRhdCkNCm5iLm1vZDcgPC0gZ2xtbVRNQihWb3RlcyB+IFNlbGVjdGlvbiAqIFllYXIucyArIChZZWFyLnMgfHwgU3RhdGUpICsgb2Zmc2V0KGxvZyhWRVApKSwgd2VpZ2h0cyA9IEVsZWN0b3JhbC5Wb3RlcywgZmFtaWx5ID0gbmJpbm9tMiwgZGF0YSA9IFZvdGUuZGF0KQ0KbmIubW9kOCA8LSBnbG1tVE1CKFZvdGVzIH4gU2VsZWN0aW9uICogWWVhci5zICsgKFNlbGVjdGlvbiArIFllYXIucyB8fCBTdGF0ZSkgKyBvZmZzZXQobG9nKFZFUCkpLCB3ZWlnaHRzID0gRWxlY3RvcmFsLlZvdGVzLCBmYW1pbHkgPSBuYmlub20yLCBkYXRhID0gVm90ZS5kYXQpDQpuYi5tb2Q5IDwtIGdsbW1UTUIoVm90ZXMgfiBTZWxlY3Rpb24gKiBZZWFyLnMgKyAoU2VsZWN0aW9uICogWWVhci5zIHx8IFN0YXRlKSArIG9mZnNldChsb2coVkVQKSksIHdlaWdodHMgPSBFbGVjdG9yYWwuVm90ZXMsIGZhbWlseSA9IG5iaW5vbTIsIGRhdGEgPSBWb3RlLmRhdCkNCmBgYA0KDQpJIGNvbXBhcmVkIG1vZGVscyB0aGF0IGluY2x1ZGVkIHJhbmRvbSBpbnRlcmNlcHRzIGFuZCByYW5kb20gc2xvcGVzIGZvciBib3RoIFNlbGVjdGlvbiBhbmQgWWVhciwgd2l0aCBhbmQgd2l0aG91dCBjb3JyZWxhdGVkIGludGVyY2VwdHMuIERpZmZlcmVudCBtb2RlbHMgd2VyZSBjb21wYXJlZCBieSBzZWNvbmQtb3JkZXIgQWthaWtlIGluZm9ybWF0aW9uIGNyaXRlcmlvbiAuIEkgYWRtaXQsIGRvaW5nIHRoaXMgaXMgbm90IHJpZ29yb3VzLiBUaGVyZSdzIGNvbnRyb3ZlcnN5IG92ZXIgd2hldGhlciBBSUNjIGNhbiBiZSBjYWxjdWxhdGVkIGF0IGFsbCBmb3IgYSBtaXhlZCBtb2RlbCwgYnV0IEnigJltIG5vdCBiZWluZyBwYWlkIGZvciB0aGlzLCBzbyBJIHdhc27igJl0IGdvaW5nIHRvIHRha2UgdGhlIHRpbWUgdG8gZG8gbW9yZSBpbi1kZXB0aCBtb2RlbCBjb21wYXJpc29ucy4NCg0KYGBge3J9DQpBbGlzdCA8LSBjKEFJQ2MobmIubW9kMSksDQogICAgICAgICAgIEFJQ2MobmIubW9kMiksDQogICAgICAgICAgIEFJQ2MobmIubW9kMyksDQogICAgICAgICAgIEFJQ2MobmIubW9kNCksDQogICAgICAgICAgIEFJQ2MobmIubW9kNSksDQogICAgICAgICAgIEFJQ2MobmIubW9kNiksDQogICAgICAgICAgIEFJQ2MobmIubW9kNyksDQogICAgICAgICAgIEFJQ2MobmIubW9kOCksDQogICAgICAgICAgIEFJQ2MobmIubW9kOSkpDQpBbGlzdA0KYGBgDQoNClRoZSBtb2RlbCB3aXRoIHRoZSBsb3dlc3QgQUlDYyBpcyB0aGUgOXRoIG1vZGVsLCBzbyB0aGF0J3Mgd2hhdCBJIGFkb3B0ZWQuIFlvdSBtYXkgaGF2ZSBub3RpY2VkIGEgZmV3IGxpbmVzIGJhY2sgdGhhdCBJIHNldCB0aGUgY29udHJhc3RzIHRvICJjb250ci5TdW0iLiBUaGVzZSBhcmUgc2ltcGxlIHN1bS10by16ZXJvIGNvbnRyYXN0cywgYWthIGVmZmVjdCBjb2RpbmcuIFIncyBkZWZhdWx0IGlzIGR1bW15IGNvZGluZyBvciB0cmVhdG1lbnQgY29udHJhc3RzLCB3aGljaCBhcmUgZmluZSBpZiB5b3UncmUgbm90IHdvcnJ5aW5nIGFib3V0IGFuYWx5emluZyBpbnRlcmFjdGlvbnMuIFdlIGhhdmUgYW4gaW50ZXJhY3Rpb24sIHNvIEknbSB1c2luZyBlZmZlY3QgY29kaW5nLg0KDQpgYGB7cn0NClZvdGUubW9kZWwgPC0gbmIubW9kOQ0Kc3VtbWFyeShWb3RlLm1vZGVsKQ0KcjIoVm90ZS5tb2RlbCkNCkFub3ZhKFZvdGUubW9kZWwsIHR5cGUgPSAiSUlJIikNCmBgYA0KDQpEb24ndCBzcGVuZCBtdWNoIHRpbWUgdHJ5aW5nIHRvIGludGVycHJldCBjb2VmZmljaWVudHMuIEknbSBnb2luZyB0byAgZXN0aW1hdGUgbWFyZ2luYWwgbWVhbnMgYW5kIHRyZW5kcywgd2hpY2ggYXJlIG11Y2ggZWFzaWVyIHRvIGludGVycHJldC4gTWFyZ2luYWwgbWVhbnMgKHRyZW5kcyBmb3Igc2xvcGVzKSBlc3RpbWF0ZSBiYXNlZCBvbiB0aGUgbW9kZWwgYW5kICB0YWtlIGludG8gYWNjb3VudCB0aGUgIGVmZmVjdHMgb2YgYWxsIHBhcnRzIG9mIHRoZSBtb2RlbCBvbiBhIHJlc3BvbnNlLiBUaGV5IHN1bW1hcmVzIHdoYXQgdGhlIG1vZGVsIGlzIHNheWluZyBpbiBhIHdheSB0aGF0IGlzIG1vcmUgaW50dWl0aXZlIHRoYW4gYXR0ZW1wdGluZyB0byBwdXp6bGUgb3V0IG11bHRpcGxlIGNvZWZmaWNpZW50cy4gSWYgeW91IHdhbnQgdG8gcmVhbGx5IGdvIGRvd24gYSByYWJiaXQgaG9sZSwgdGhlcmUncyBhIFt2ZXJ5IGNvbXBsZXRlIGRlc2NyaXB0aW9uXShodHRwczovL3d3dy5hbmRyZXdoZWlzcy5jb20vYmxvZy8yMDIyLzA1LzIwL21hcmdpbmFsaWEvKSB5b3UgY2FuIHJlYWQgZWxzZXdoZXJlLg0KDQpJZiB5b3UgbG9vayBhdCB0aGUgKlIqXjJeIChOYWthZ2F3YSdzICpSKl4yXiksIHRoaXMgbW9kZWwgaXNuJ3QgdG9vIGJhZCBhIGZpdC4gTWFyZ2luYWwgKlIqXjJeIGVzdGltYXRlcyB0aGUgdG90YWwgY29udHJpYnV0aW9uIG9mIHRoZSBmaXhlZCBlZmZlY3RzLCBjb25kaXRpb25hbCBpbmNsdWRlcyB0aGUgcmFuZG9tIGVmZmVjdHMgKGluZGl2aWR1YWwgc3RhdGUgZWZmZWN0cykuDQoNCkknbSBub3QgaW50ZXJlc3RlZCBpbiBjb21wYXJpbmcgb3V0Y29tZXMgZm9yIGEgc2luZ2xlIGdpdmVuIGVsZWN0aW9uLiBUaGF0J3Mgbm90IHZlcnkgaW5mb3JtYXRpdmUgZnJvbSB0aGlzIGhpZ2ggdXAgaW4gdGhlIGNsb3Vkcy4gQ29tcGFyaW5nIHRoZSAqdHJlbmRzKiwgb24gdGhlIG90aGVyIGhhbmQsIGludGVyZXN0cyBtZS4NCg0KYGBge3J9DQpWb3RlLmVtdCA8LSBlbXRyZW5kcyhWb3RlLm1vZGVsLCB+U2VsZWN0aW9uOlllYXIucywgIlllYXIucyIpDQpWb3RlLmVtdA0KYGBgDQojIyMgVmlzdWFsaXppbmcNCldoYXQgZG9lcyB0aGlzIHRlbGwgdXM/IFdlIGNhbiBzZWUgdGhhdCB0aGlzIHRyZW5kIGlzIGluY3JlYXNpbmcgb3ZlciB0aGUgMTk4MC0yMDI0IHBlcmlvZCBmb3IgRGVtb2NyYXRzLCBkZWNyZWFzaW5nIGZvciBub24tdm90aW5nLCBhbmQgdm90ZXMgZm9yICJPdGhlciIgY2FuZGlkYXRlcywgYW5kIHdlYWtseSBpbmNyZWFzaW5nIG92ZXIgdGltZSBmb3IgUmVwdWJsaWNhbnMuIFdoYXQgaXMgaW50ZXJlc3RpbmcgaXMgdGhhdCB0aGUgY29uZmlkZW5jZSBsaW1pdHMgZm9yIHRoZSBSZXB1YmxpY2FuIHRyZW5kIGFyZSBuZWdhdGl2ZSBhbmQgcG9zaXRpdmUsIHN1Z2dlc3RpbmcgdGhhdCB0aGUgb3ZlcmFsbCB0cmVuZCBtYXkgbm90IHNpZ25pZmljYW50bHkgZGlmZmVyIGZyb20gemVybyBmb3IgdGhpcyBwZXJpb2QuIEJ1dCB3b3JkcyBzYXkgc28gbXVjaCBsZXNzIHRoYW4gYSBwaWN0dXJlLiBGaXJzdCwgd2UgaGF2ZSB0byBnZW5lcmF0ZSBzb21lIHBvaW50cyBmcm9tIHRoZSBtb2RlbC4NCg0KYGBge3J9DQpWb3RlLmVtbSA8LSBlbW1lYW5zKFZvdGUubW9kZWwsIH5TZWxlY3Rpb246WWVhci5zLCB0eXBlID0gInJlc3BvbnNlIiwgb2Zmc2V0ID0gbG9nKDEpLCBhdCA9IA0KICAgICAgICAgICAgICAgICAgICAgIGxpc3QoWWVhci5zID0gYXMubnVtZXJpYyhsZXZlbHMoZmFjdG9yKFZvdGUuZGF0JFllYXIucykpKSkpDQpWb3RlLnByZWRzIDwtIGRhdGEuZnJhbWUoVm90ZS5lbW0pICU+JQ0KICBtdXRhdGUoWWVhciA9IChZZWFyLnMgKiBhdHRyKFZvdGUuZGF0JFllYXIucywgInNjYWxlZDpzY2FsZSIpICsgYXR0cihWb3RlLmRhdCRZZWFyLnMsICJzY2FsZWQ6Y2VudGVyIikpKQ0KVm90ZS5wcmVkcw0KYGBgDQpgYGB7cn0NCm1vZGVsLnBsb3QgPC0gZ2dwbG90KFZvdGUucHJlZHMsIGFlcyh4ID0gWWVhciwgeSA9IHJlc3BvbnNlLCBncm91cCA9IFNlbGVjdGlvbiwgY29sb3IgPSBTZWxlY3Rpb24pKSArIA0KICBnZW9tX2xpbmUobGluZXdpZHRoID0gMSkgKw0KICBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYygiIzQ0NzdBQSIsICIjMjI4ODMzIiwgIiNDQ0JCNDQiLCAiI0VFNjY3NyIpLA0KICAgICAgICAgICAgICAgICAgICAgbGltaXRzID0gYygiRGVtb2NyYXQiLCAiT3RoZXIiLCAiTm9udm90aW5nIiwgIlJlcHVibGljYW4iKSkgKw0KICBnZW9tX3JpYmJvbihhZXMoeW1pbiA9IGFzeW1wLkxDTCwgeW1heCA9IGFzeW1wLlVDTCksIGFscGhhID0gMC4wNSkgKw0KICB0aGVtZShwYW5lbC5iYWNrZ3JvdW5kID0gZWxlbWVudF9yZWN0KGZpbGwgPSAid2hpdGUiKSwNCiAgICAgICAgYXhpcy5saW5lID0gZWxlbWVudF9saW5lKHNpemUgPSAxLCBjb2xvciA9ICJibGFjayIpLA0KICAgICAgICBsZWdlbmQucG9zaXRpb24gPSBjKDAuOCwgMC4zKSkgKw0KICBzY2FsZV95X2NvbnRpbnVvdXMobGFiZWxzID0gc2NhbGVzOjpwZXJjZW50KSArDQogIHNjYWxlX3hfY29udGludW91cyhicmVha3MgPSBjKDE5ODAsIDE5ODQsIDE5ODgsIDE5OTIsIDE5OTYsIDIwMDAsIDIwMDQsIDIwMDgsIDIwMTIsIDIwMTYsIDIwMjAsIDIwMjQpKSArDQogIHlsYWIoIkZyZXF1ZW5jeSAocGVyIFZFUCkiKQ0KDQptb2RlbC5wbG90DQpgYGANCg0KVGhlIHJpYmJvbnMgYXJvdW5kIGVhY2ggbGluZSBhcmUgdGhlIDk1JSBjb25maWRlbmNlIGludGVydmFscy4gVGhlIHRyZW5kcyBsb29rIGxpa2UgdGhleSBkaWZmZXIgZnJvbSBlYWNoIG90aGVyLCBidXQgaXMgdGhhdCByZWFsPw0KDQpgYGB7cn0NCnBhaXJzKFZvdGUuZW10KQ0KYGBgDQpJZiB3ZSBsb29rIGF0IHRoZSBhZGp1c3RlZCBwIHZhbHVlcyBmb3IgZXZlcnkgcGFpcndpc2UgY29tcGFyaXNvbiBvZiB0cmVuZHMsIHllcy4gRXZlbiBhZGp1c3RpbmcgZm9yIGFsbCBtdWx0aXBsZSBjb21wYXJpc29ucywgZXZlcnkgdHJlbmQgZGlmZmVycyBmcm9tIGV2ZXJ5IG90aGVyIHRyZW5kIHdpdGggYSBwIG9mIDwgMC4wNS4gSG93IGNhbiB0aGlzIGJlIGludGVycHJldGVkPyBUeXBpY2FsbHksIHBlb3BsZSBzYXkgInNpZ25pZmljYW50IiBvciAibm90IHNpZ25pZmljYW50IiBhbmQgY2FsbCBpdCBhIGRheS4gVGhlcmUncyBhbm90aGVyIGludGVycHJldGF0aW9uIG9mIHRoZSBwIHZhbHVlLCBhbmQgdGhhdCdzIHRoZSBwcm9iYWJpbGl0eSBvZiB0aGUgb2JzZXJ2ZWQgZGlmZmVyZW5jZSBvY2N1cmluZyBieSByYW5kb20gY2hhbmNlICppZiB0aGUgZGlmZmVyZW5jZSBkaWQgbm90IGFjdHVhbGx5IGV4aXN0Ki4gRm9yIGV4YW1wbGUsIGlmIHRoZXJlIHJlYWxseSB3ZXJlIG5vIGRpZmZlcmVuY2UgYmV0d2VlbiBSZXB1YmxpY2FuIGFuZCBEZW1vY3JhdCBwZXJmb3JtYW5jZSwgYWNjb3JkaW5nIHRvIHRoaXMgbW9kZWwsIHdlIHdvdWxkIGV4cGVjdCB0byBzdGlsbCBzZWUgdGhlIGRpZmZlcmVuY2Ugd2UgZXN0aW1hdGVkIGFib3V0IDIuOCUgb2YgdGhlIHRpbWUuDQoNClNvLCB3ZSBjYW4gZ2VuZXJhbGx5IGluZmVyIHRoYXQgdGhlIGRpZmZlcmVuY2VzIHdlIHRoaW5rIHdlIHNlZSBhcmUgcHJvYmFibHkgcmVhbC4gSG93ZXZlciwgYXJlIHRoZXkgbWVhbmluZ2Z1bD8gVGhlIHAgdmFsdWUgY2FuJ3QgdGVsbCB1cyBob3cgKmxhcmdlKiBhIGRpZmZlcmVuY2UgaXMuIEl0IGNhbiBvbmx5IHRlbGwgdXMgaG93ICpsaWtlbHkqIGl0IGlzLiBGb3J0dW5hdGVseSwgc3RhbmRhcmRpemVkIGVmZmVjdCBzaXplcyBleGlzdCB0byBleHBsb3JlIHRoaXMgcXVlc3Rpb24uDQoNCmBgYHtyfQ0KVm90ZS5lZmYgPC0gZWZmX3NpemUoVm90ZS5lbXQsIHNpZ21hID0gc2lnbWEoVm90ZS5tb2RlbCksIGVkZiA9IGRmLnJlc2lkdWFsKFZvdGUubW9kZWwpKQ0KVm90ZS5lZmYNCmBgYA0KDQpUaGVzZSBlZmZlY3Qgc2l6ZXMgYXJlIENvaGVuJ3MgKmQqLCB3aGljaCBib2lscyBkb3duIHRvIGhvdyBmYXIgYXdheSB0aGUgdHJlbmRzIGFyZSBmcm9tIGVhY2ggb3RoZXIsIGFkanVzdGVkIGJ5IGhvdyB1bmNlcnRhaW4gdGhlIGVzdGltYXRlcyBvZiB0aGUgdHJlbmRzIGFyZS4gVGhlIG51bWJlcnMgYXJlbid0IHZlcnkgaW5mb3JtYXRpdmUgYnkgdGhlbXNlbHZlcy4gRm9ydHVuYXRlbHkgaGFzIGJlZW4gZXh0ZW5zaXZlIHdvcmsgdG8gZGVmaW5lIG1lYW5pbmdmdWwgY2F0ZWdvcmllcyBvZiBlZmZlY3Qgc2l6ZXMgZm9yIHBvbGl0aWNhbCBzY2llbmNlLiBJIHRoaW5rIHRoYXQgdGhlIGNhdGVnb3JpZXMgcHJvcG9zZWQgYnkgW0VyaWsgR2huZWVyIExhcnNlbl0oaHR0cHM6Ly9lcmlrZ2FobmVyLmRrLzIwMjIvZWZmZWN0LXNpemVzLWluLXBvbGl0aWNhbC1zY2llbmNlLykgYXJlIHJlYXNvbmFibGUuIEhvd2V2ZXIsIGhlIHByZXNlbnRlZCB0aGVtIGluIHJlbGF0aW9uIHRvIFBlYXJzb24ncyByLiBGb3J0dW5hdGVseSwgaXQgaXMgcG9zc2libGUgdG8gY29udmVydCBiZXR3ZWVuIHRoZSB0d28uIFVuZm9ydHVuYXRlbHksIHRoZSBzdGFuZGFyZCBjb252ZXJzaW9uIG9mIGQgdG8gciBpcyBiaWFzZWQgdG8gcHJvZHVjZSBsb3cgb3V0Y29tZXMsIGJlY2F1c2UgaXQgY29udmVydHMgdG8gdGhlIHBvaW50LWJpc2VyaWFsIHIgYW5kIHByZXN1bWVzIHRoYXQgZCB3YXMgZXN0aW1hdGVkIHdpdGggYSBiaW5vbWlhbCBwcmVkaWN0b3IuIFllYXIucyBpcyBhIGNvbnRpbnVvdXMgcHJlZGljdG9yLiBUaGVyZWZvcmUsIEkgY29udmVydGVkIGQgdG8gdGhlIHByb2JhYmlsaXR5IG9mIHN1cGVyaW9yaXR5IGNvbW1vbiBsYW5ndWFnZSBlZmZlY3Qgc2l6ZSBhbmQgdGhlbiBjb252ZXJ0ZWQgdGhpcyB0byByIHVzaW5nIHRoZSBmb3JtdWxhIGZvdW5kIGluIER1bmxhcCwgMTk5NC4NCg0KYGBge3J9DQpWb3RlLmRfdG9fciA8LSBkYXRhLmZyYW1lKHN1bW1hcnkoVm90ZS5lZmYpJGNvbnRyYXN0LCBzdW1tYXJ5KFZvdGUuZWZmKSRlZmZlY3Quc2l6ZSwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgZF90b19wX3N1cGVyaW9yaXR5KHN1bW1hcnkoVm90ZS5lZmYpJGVmZmVjdC5zaXplKSwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgc2luKCgoZF90b19wX3N1cGVyaW9yaXR5KHN1bW1hcnkoVm90ZS5lZmYpJGVmZmVjdC5zaXplKSkgLS41KSAqIHBpKSkNCmNvbG5hbWVzKFZvdGUuZF90b19yKSA8LSBjKCJDb250cmFzdCIsICJkIiwgIkNMRVMiLCAiciIpDQoNCg0KVm90ZS5kX3RvX3IgPC0gVm90ZS5kX3RvX3IgJT4lIG11dGF0ZShMYXJzZW4gPSBjYXNlX3doZW4oDQogIGFicyhyKSA8IDAuMDUgfiAiTGVzcyB0aGFuIFNtYWxsIEVmZmVjdCIsDQogIGFicyhyKSA+PSAwLjA1ICYgYWJzKHIpIDwgMC4xNSB+ICJTbWFsbCBFZmZlY3QiLA0KICBhYnMocikgPj0gMC4xNSAmIGFicyhyKSA8IDAuMzAgfiAiTWVkaXVtIEVmZmVjdCIsDQogIGFicyhyKSA+PSAwLjMwIH4gIkxhcmdlIEVmZmVjdCINCikpDQoNClZvdGUuZF90b19yDQpgYGANCkNMRVMgaXMgdGhlICJDb21tb24gTGFuZ3VhZ2UgRWZmZWN0IFNpemUiIG9yICJwcm9iYWJpbGl0eSBvZiBzdXBlcmlvcml0eSIuIFRoaXMgaXMgYW4gZXN0aW1hdGUgb2YgdGhlIHByb2JhYmlsaXR5IChiZXR3ZWVuIDAgYW5kIDEpIHRoYXQsIGZvciBhIGdpdmVuIHRyYWl0LCBhIG1lbWJlciBvZiBvbmUgZ3JvdXAgd2lsbCBiZSBsYXJnZXIgdGhhbiBhIG1lbWJlciBvZiB0aGUgb3RoZXIgZ3JvdXAuIEEgQ0xFUyBvZiAwLjUgbWVhbnMgdGhlIGdyb3VwcyBhcmUgaWRlbnRpY2FsLiBBcyB5b3UgY2FuIHNlZSwgbm9uZSBvZiB0aGUgZGlmZmVyZW5jZXMgYXJlIGxhcmdlLg0KDQojIyBXaGF0IGRvIHdlIHRha2UgaG9tZT8NCklmIHlvdSd2ZSBtYWRlIGl0IHRoaXMgZmFyLCB5b3UgcmVhbGx5IGxpa2UgY2hvcHBpbmcgdGhyb3VnaCB0aGUgd2VlZHMsIGJ1dCB3aGF0IGhhdmUgd2UgbGVhcm5lZC4gSW4gYWRkaXRpb24gdG8gdGhlIHJhY2UtYnktcmFjZSBub24tc3RhdGlzdGljYWwgYW5hbHlzaXMsIHdlIGNhbiBzYXkgd2l0aCBzb21lIGNlcnRhaW50eSB0aGF0LCBzaW5jZSAxOTgwLCB0aGUgRGVtb2NyYXRpYyBwYXJ0eSBoYXMgYmVlbiBzdGVhZGlseSBnYWluaW5nIGFtb25nIHRoZSB2b3RpbmctZWxpZ2libGUgcG9wdWxhdGlvbiwgd2hpbGUgUmVwdWJsaWNhbiBnYWlucyBoYXZlIGJlZW4gc2xvd2VyLiBUaGlzIGRpZmZlcmVuY2UgaXMgbGlrZWx5IHRvIGJlIHJlYWwsIGJ1dCBpdCdzIG5vdCBhdCBhbGwgbGFyZ2UuIFdoYXQgaXMgYWxzbyBpbnRlcmVzdGluZyBpcyB0aGF0IHRoZSBwcm9wb3J0aW9uIG9mIHBlb3BsZSBqdXN0IHNpdHRpbmcgaXQgb3V0IGhhcyBiZWVuIHNsb3dseSBkZWNyZWFzaW5nIHNpbmNlIDE5ODAuIEFtZXJpY2FucyBoYXZlIGJlZW4gYmVjb21pbmcgbW9yZSBlbmdhZ2VkLg0KDQpXaGF0IGFib3V0IHRoZSAiT3RoZXIiPyBJJ2xsIGZyZWVseSBhZG1pdCB0aGF0IHRoaXMgY2F0ZWdvcnkgZmx1Y3R1YXRlZCB0b28gd2lsZGx5IHRvIG1ha2UgYW55IGRlZXAgY29uY2x1c2lvbnMuIFJvc3MgUGVyb3Qgd2FzIGEgbWFqb3IgaW5mbHVlbmNlLCBidXQgdGhlIFBlcm90IHNwaWtlIGhhcyBuZXZlciBiZWVuIHJlcGVhdGVkLg0KDQojIyBUaGUgRnV0dXJlDQpTaG9ydC10ZXJtLCB0aGVyZSBpcyBubyB3YXkgdG8gcHJlZGljdCAyMDI4LCBjZXJ0YWlubHkgbm90IHJpZ2h0IG5vdy4gRmFydGhlciBpbnRvIHRoZSBmdXR1cmUsIGlmIHRoZSB0cmVuZHMgSSBtYXBwZWQgY29udGludWUsIGFsdGhvdWdoIHRoZXJlIGlzIG5vIHdheSB0byBzYXkgdGhleSBtdXN0LCB0aGUgUmVwdWJsaWNhbiBwYXJ0eSdzIGdhaW5zIHdvdWxkIGJlIG91dHN0cmlwcGVkIGJ5IHRoZSBEZW1vY3JhdHMgYXMgbW9yZSBBbWVyaWNhbnMgY2hvb3NlIHRvIHBhcnRpY2lwYXRlIGluIHByZXNpZGVudGlhbCBlbGVjdGlvbnMuIEknbSBnb2ludCB0byBzcGVjdWxhdGUgdGhhdCBJJ20gbm90IHRoZSBmaXJzdCBwZXJzb24gdG8gZG8gYW4gYW5hbHlzaXMgbGlrZSB0aGlzLiBJJ3ZlIG5vIGRvdWJ0IHRoYXQgcHJvZmVzc2lvbmFsIHN0cmF0ZWdpc3RzIGhhdmUgYWxyZWFkeSBiZWVuIGxvb2tpbmcgYXQgdGhlc2UgdHJlbmRzLiBUaGlzIGNvdWxkIGV4cGxhaW4gdGhlIHdpbGxpbmduZXNzIHRvIGFkb3B0IHNldmVyZSBtZWFzdXJlcyB0aGF0IHNlZW1zIHRvIGhhdmUgZHJpdmVuIHRoZSBSZXB1YmxpY2FuIHBhcnR5IGluIHJlY2VudCB5ZWFycy4gSWYgdGhlIGN1cnJlbnQgcGFyYWRpZ20gaXMgbm90IGRlc3Ryb3llZCwgdGhlIFJlcHVibGljYW5zIG1heSBiZWNvbWUgaXJyZWxldmFudC4NCg==