1 Introduction

1.0.1 Description

In this data set we have data from the 2008 NFL season. More specifically we have factors that go into NFL fielgoals. Some variables include the kicking team, Name, Distance, timerem, defscore, and GOOD.

Kickteam - Name of the kicking team Name - Name of the kicker Distance - How far the ball is from the goal Timerem - How much time is on the game clock remaining in the game Kickdiff - the level of difficulty of the kick Defscore- The score of the opposing team GOOD - If the field goal is made or not, a 1 for a make and 0 for a miss

1.0.2 Question

The objective of this analysis is to build a logistic regression model to predict a made field goal using diffrent factors tat go into a make.

1.0.3 Data Cleaning

fieldgoals <- read.csv("https://raw.githubusercontent.com/TylerBattaglini/STA-321/refs/heads/main/nfl2008_fga.csv", header = TRUE)
clean_fieldgoals <- na.omit(fieldgoals)
clean_fieldgoals <- clean_fieldgoals %>% select(-GameDate, -AwayTeam, -HomeTeam, -qtr, -min, -sec, -def, -down, -togo, -kicker, -ydline, -homekick, -offscore, -season, -Missed, -Blocked, -kickteam, -name)

head(clean_fieldgoals)
  distance kickdiff timerem defscore GOOD
1       30       -3    2822        3    1
2       46        0    3287        0    1
3       28        7    2720        0    1
4       37       14    2742        0    1
5       39        0    3056        0    1
6       40       -3    3043        3    1

We take out any observations with a missing value. We also take out many variables due to there being a high likeleyhood for multicollineairty. We already have a variable for time so we eliminated many variables related to time. We also already have a variable for a make so we do not need any for a miss or blocked, that would just be a repeat our data. The others are just categorical variables that are to identify the kicker or kicking team which again we already have variables that describe that.

library(psych)
pairs.panels(clean_fieldgoals[,-9], 
             method = "pearson",
             hist.col = "#00AFBB",
             density = TRUE,
             ellipses = TRUE
             )

All of our predictor values are unimodal except for defscore.

par(mfrow=c(1,2))
hist(clean_fieldgoals$defscore, xlab="defscore", main = "")

Based on the histogram above we discretize defscore

defscore = clean_fieldgoals$defscore
grp.defscore = defscore
grp.defscore[defscore %in% c(0:10)] = "1-10"
grp.defscore[defscore %in% c(11:18)] = "11-18"
grp.defscore[defscore %in% c(19:26)] = "19-26"
grp.defscore[defscore %in% c(27:99)] = "27+"
clean_fieldgoals$grp.defscore = grp.defscore
head(grp.defscore)
[1] "1-10" "1-10" "1-10" "1-10" "1-10" "1-10"

There is some correlation between the variables kickdiff vs defscore and kickdiff vs timerem.

In our smallest and final model we want to have kick difficulty, distance, time remaining because we know distance is a big indicator on whether or not a field goal is good or not and the same goes for kick difficulty and time remaining.

2 Introduction

2.0.1 Standardizing Numerical Variables

We first standardize our numerical values because we do not care about the interpretation of the coefficients only to identify the best model.

clean_fieldgoals$sd.distance = (clean_fieldgoals$distance-mean(clean_fieldgoals$distance))/sd(clean_fieldgoals$distance)
clean_fieldgoals$sd.kickdiff = (clean_fieldgoals$kickdiff-mean(clean_fieldgoals$kickdiff))/sd(clean_fieldgoals$kickdiff)
clean_fieldgoals$sd.timerem = (clean_fieldgoals$timerem-mean(clean_fieldgoals$timerem))/sd(clean_fieldgoals$timerem)
clean_fieldgoals$sd.defscore = (clean_fieldgoals$defscore-mean(clean_fieldgoals$defscore))/sd(clean_fieldgoals$defscore)

sd.GOOD = clean_fieldgoals[, -(1:4)]

2.0.2 Data Split

We randomly split the data into two sets one with 80% of the data which is used as a training data. We then use the other 20% to assess the performance of the model

n <- dim(sd.GOOD)[1]
train.n <- round(0.8*n)
train.id <- sample(1:n, train.n, replace = FALSE)
train <- sd.GOOD[train.id, ]
test <- sd.GOOD[-train.id, ]

We will use a 5-fold cross validation to make sure we have enough ield goal data.

k=5
fold.size = floor(dim(train)[1]/k)
PE1 = rep(0,5)
PE2 = rep(0,5)
PE3 = rep(0,5)

for(i in 1:k) {
  valid.id = (fold.size*(i-1)+1):(fold.size*i)
  valid = train[valid.id, ]
  train.dat = train[-valid.id,]
  
  candidate01 = glm(GOOD ~ sd.distance + sd.kickdiff + sd.timerem + 
                    grp.defscore, family = binomial(link = "logit"),  
                    data = train.dat)  

  candidate03 = glm(GOOD ~ sd.distance + grp.defscore + sd.timerem, 
                    family = binomial(link = "logit"),  
                    data = train.dat) 

  candidate02 = stepAIC(candidate01, 
                        scope = list(lower=formula(candidate03),upper=formula(candidate01)),
                        direction = "forward",   
                        trace = 0)
  
  pred01 = predict(candidate01, newdata = valid, type="response")
  pred02 = predict(candidate02, newdata = valid, type="response")
  pred03 = predict(candidate03, newdata = valid, type="response")
  
  pre.outcome01 = ifelse(as.vector(pred01) > 0.5, 1, 0)
  pre.outcome02 = ifelse(as.vector(pred02) > 0.5, 1, 0)
  pre.outcome03 = ifelse(as.vector(pred03) > 0.5, 1, 0)

  PE1[i] = sum(pre.outcome01 == valid$GOOD)/length(pred01)
  PE2[i] = sum(pre.outcome02 == valid$GOOD)/length(pred02)
  PE3[i] = sum(pre.outcome03 == valid$GOOD)/length(pred03)
}

avg.pe = cbind(PE1 = mean(PE1), PE2 = mean(PE2), PE3 = mean(PE3))
kable(avg.pe, caption = "Average of prediction errors of candidate models")
Average of prediction errors of candidate models
PE1 PE2 PE3
0.8518072 0.8518072 0.8578313

From our output above we see that model 3 has the highest prediction error of .865. We have a cutoff of .5 so our predictive error of .865 is well above our cutoff.

2.1 Final Model

pred03 = predict(candidate03, newdata = test, type="response")
pred03.outcome = ifelse(as.vector(pred02)>0.5, "1", "0")

accuracy = sum(pred03.outcome == test$GOOD)/length(pred03)
Warning in pred03.outcome == test$GOOD: longer object length is not a multiple
of shorter object length
kable(accuracy, caption="The actual accuracy of the final model")
The actual accuracy of the final model
x
0.8985507

From the output above we observe the withheld test data. We get a accuracy rate of .850.

3 Conclusion

This analysis was on predicting a made field goal. We used three models as candidates for our final model and we used cross validation. Our final result for the accuracy of our final model was approximately 85.02, which shows that the model correctly showed 85% of the test data based on the predicted probabilities and our chosen threshold of .5. An accuracy of 85% shows that the model is performing relatively well, it successfully identified whether an observation falls into the classes made (“1”) or miss (“0”) for 85% of the cases.

LS0tDQp0aXRsZTogIkZhY3RvcnMgSW5mbHVlbmNlIE5GTCBGaWVsZCBHb2FscyINCmF1dGhvcjogJ1R5bGVyIEJhdHRhZ2xpbmknDQpkYXRlOiAiMjAyNC0xMC0xMSINCm91dHB1dDoNCiAgaHRtbF9kb2N1bWVudDogDQogICAgdG9jOiB5ZXMNCiAgICB0b2NfZGVwdGg6IDQNCiAgICB0b2NfZmxvYXQ6IHllcw0KICAgIGZpZ193aWR0aDogNA0KICAgIGZpZ19jYXB0aW9uOiB5ZXMNCiAgICBudW1iZXJfc2VjdGlvbnM6IHllcw0KICAgIHRvY19jb2xsYXBzZWQ6IHllcw0KICAgIGNvZGVfZm9sZGluZzogaGlkZQ0KICAgIGNvZGVfZG93bmxvYWQ6IHllcw0KICAgIHNtb290aF9zY3JvbGw6IHllcw0KICAgIHRoZW1lOiBsdW1lbg0KICB3b3JkX2RvY3VtZW50OiANCiAgICB0b2M6IHllcw0KICAgIHRvY19kZXB0aDogNA0KICAgIGZpZ19jYXB0aW9uOiB5ZXMNCiAgICBrZWVwX21kOiB5ZXMNCiAgcGRmX2RvY3VtZW50OiANCiAgICB0b2M6IHllcw0KICAgIHRvY19kZXB0aDogNA0KICAgIGZpZ19jYXB0aW9uOiB5ZXMNCiAgICBudW1iZXJfc2VjdGlvbnM6IHllcw0KICAgIGZpZ193aWR0aDogMw0KICAgIGZpZ19oZWlnaHQ6IDMNCmVkaXRvcl9vcHRpb25zOiANCiAgY2h1bmtfb3V0cHV0X3R5cGU6IGlubGluZQ0KYWx3YXlzX2FsbG93X2h0bWw6IHRydWUNCi0tLQ0KDQpgYGB7PWh0bWx9DQoNCjxzdHlsZSB0eXBlPSJ0ZXh0L2NzcyI+DQoNCi8qIENhc2NhZGluZyBTdHlsZSBTaGVldHMgKENTUykgaXMgYSBzdHlsZXNoZWV0IGxhbmd1YWdlIHVzZWQgdG8gZGVzY3JpYmUgdGhlIHByZXNlbnRhdGlvbiBvZiBhIGRvY3VtZW50IHdyaXR0ZW4gaW4gSFRNTCBvciBYTUwuIGl0IGlzIGEgc2ltcGxlIG1lY2hhbmlzbSBmb3IgYWRkaW5nIHN0eWxlIChlLmcuLCBmb250cywgY29sb3JzLCBzcGFjaW5nKSB0byBXZWIgZG9jdW1lbnRzLiAqLw0KDQpoMS50aXRsZSB7ICAvKiBUaXRsZSAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgb2YgdGhlIHJlcG9ydCB0aXRsZSAqLw0KICBmb250LXNpemU6IDI0cHg7DQogIGZvbnQtd2VpZ2h0OiBib2xkOw0KICBjb2xvcjogRGFya1JlZDsNCiAgdGV4dC1hbGlnbjogY2VudGVyOw0KICBmb250LWZhbWlseTogIkdpbGwgU2FucyIsIHNhbnMtc2VyaWY7DQp9DQpoNC5hdXRob3IgeyAvKiBIZWFkZXIgNCAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIGF1dGhvcnMgICovDQogIGZvbnQtc2l6ZTogMjBweDsNCiAgZm9udC13ZWlnaHQ6IGJvbGQ7DQogIGZvbnQtZmFtaWx5OiBzeXN0ZW0tdWk7DQogIGNvbG9yOiBEYXJrUmVkOw0KICB0ZXh0LWFsaWduOiBjZW50ZXI7DQp9DQpoNC5kYXRlIHsgLyogSGVhZGVyIDQgLSBmb250IHNwZWNpZmljYXRpb25zIGZvciB0aGUgZGF0ZSAgKi8NCiAgZm9udC1zaXplOiAxOHB4Ow0KICBmb250LXdlaWdodDogYm9sZDsNCiAgZm9udC1mYW1pbHk6IHN5c3RlbS11aTsNCiAgY29sb3I6IERhcmtCbHVlOw0KICB0ZXh0LWFsaWduOiBjZW50ZXI7DQp9DQpoMSB7IC8qIEhlYWRlciAxIC0gZm9udCBzcGVjaWZpY2F0aW9ucyBmb3IgbGV2ZWwgMSBzZWN0aW9uIHRpdGxlICAqLw0KICAgIGZvbnQtc2l6ZTogMjJweDsNCiAgICBmb250LXdlaWdodDogYm9sZDsNCiAgICBmb250LWZhbWlseTogc3lzdGVtLXVpOw0KICAgIGNvbG9yOiBuYXZ5Ow0KICAgIHRleHQtYWxpZ246IGxlZnQ7DQp9DQpoMiB7IC8qIEhlYWRlciAyIC0gZm9udCBzcGVjaWZpY2F0aW9ucyBmb3IgbGV2ZWwgMiBzZWN0aW9uIHRpdGxlICovDQogICAgZm9udC1zaXplOiAyMHB4Ow0KICAgIGZvbnQtd2VpZ2h0OiBib2xkOw0KICAgIGZvbnQtZmFtaWx5OiAiVGltZXMgTmV3IFJvbWFuIiwgVGltZXMsIHNlcmlmOw0KICAgIGNvbG9yOiBuYXZ5Ow0KICAgIHRleHQtYWxpZ246IGxlZnQ7DQp9DQoNCmgzIHsgLyogSGVhZGVyIDMgLSBmb250IHNwZWNpZmljYXRpb25zIG9mIGxldmVsIDMgc2VjdGlvbiB0aXRsZSAgKi8NCiAgICBmb250LXNpemU6IDE4cHg7DQogICAgZm9udC13ZWlnaHQ6IGJvbGQ7DQogICAgZm9udC1mYW1pbHk6ICJUaW1lcyBOZXcgUm9tYW4iLCBUaW1lcywgc2VyaWY7DQogICAgY29sb3I6IG5hdnk7DQogICAgdGV4dC1hbGlnbjogbGVmdDsNCn0NCg0KaDQgeyAvKiBIZWFkZXIgNCAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgb2YgbGV2ZWwgNCBzZWN0aW9uIHRpdGxlICAqLw0KICAgIGZvbnQtc2l6ZTogMThweDsNCiAgICBmb250LXdlaWdodDogYm9sZDsNCiAgICBmb250LWZhbWlseTogIlRpbWVzIE5ldyBSb21hbiIsIFRpbWVzLCBzZXJpZjsNCiAgICBjb2xvcjogZGFya3JlZDsNCiAgICB0ZXh0LWFsaWduOiBsZWZ0Ow0KfQ0KDQpib2R5IHsgYmFja2dyb3VuZC1jb2xvcjp3aGl0ZTsgfQ0KDQouaGlnaGxpZ2h0bWUgeyBiYWNrZ3JvdW5kLWNvbG9yOnllbGxvdzsgfQ0KDQpwIHsgYmFja2dyb3VuZC1jb2xvcjp3aGl0ZTsgfQ0KDQo8L3N0eWxlPg0KYGBgDQoNCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQ0KIyBEZXRlY3QsIGluc3RhbGwgYW5kIGxvYWQgcGFja2FnZXMgaWYgbmVlZGVkLg0KaWYgKCFyZXF1aXJlKCJrbml0ciIpKSB7DQogICBpbnN0YWxsLnBhY2thZ2VzKCJrbml0ciIpDQogICBsaWJyYXJ5KGtuaXRyKQ0KfQ0KaWYgKCFyZXF1aXJlKCJNQVNTIikpIHsNCiAgIGluc3RhbGwucGFja2FnZXMoIk1BU1MiKQ0KICAgbGlicmFyeShNQVNTKQ0KfQ0KaWYgKCFyZXF1aXJlKCJubGVxc2x2IikpIHsNCiAgIGluc3RhbGwucGFja2FnZXMoIm5sZXFzbHYiKQ0KICAgbGlicmFyeShubGVxc2x2KQ0KfQ0KIw0KaWYgKCFyZXF1aXJlKCJwYW5kZXIiKSkgew0KICAgaW5zdGFsbC5wYWNrYWdlcygicGFuZGVyIikNCiAgIGxpYnJhcnkocGFuZGVyKQ0KfQ0KDQppZiAoIXJlcXVpcmUoInBzeWNoIikpIHsgICANCiAgaW5zdGFsbC5wYWNrYWdlcygicHN5Y2giKQ0KICAgbGlicmFyeShwc3ljaCkNCn0NCmlmICghcmVxdWlyZSgiTUFTUyIpKSB7ICAgDQogIGluc3RhbGwucGFja2FnZXMoIk1BU1MiKQ0KICAgbGlicmFyeShNQVNTKQ0KfQ0KaWYgKCFyZXF1aXJlKCJnZ3Bsb3QyIikpIHsgICANCiAgaW5zdGFsbC5wYWNrYWdlcygiZ2dwbG90MiIpDQogICBsaWJyYXJ5KGdncGxvdDIpDQp9DQppZiAoIXJlcXVpcmUoIkdHYWxseSIpKSB7ICAgDQogIGluc3RhbGwucGFja2FnZXMoIkdHYWxseSIpDQogICBsaWJyYXJ5KEdHYWxseSkNCn0NCmlmICghcmVxdWlyZSgiY2FyIikpIHsgICANCiAgaW5zdGFsbC5wYWNrYWdlcygiY2FyIikNCiAgIGxpYnJhcnkoY2FyKQ0KfQ0KaWYgKCFyZXF1aXJlKCJkcGx5ciIpKSB7ICAgDQogIGluc3RhbGwucGFja2FnZXMoImRwbHlyIikNCiAgIGxpYnJhcnkoZHBseXIpDQp9DQppZiAoIXJlcXVpcmUoImNhcmV0IikpIHsgICANCiAgaW5zdGFsbC5wYWNrYWdlcygiY2FyZXQiKQ0KICAgbGlicmFyeShjYXJldCkNCn0NCg0KIyBzcGVjaWZpY2F0aW9ucyBvZiBvdXRwdXRzIG9mIGNvZGUgaW4gY29kZSBjaHVua3MNCmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSwgICAgICAjIGluY2x1ZGUgY29kZSBjaHVuayBpbiB0aGUgb3V0cHV0IGZpbGUNCiAgICAgICAgICAgICAgICAgICAgICB3YXJuaW5ncyA9IEZBTFNFLCAgIyBzb21ldGltZXMsIHlvdSBjb2RlIG1heSBwcm9kdWNlIHdhcm5pbmcgbWVzc2FnZXMsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICMgeW91IGNhbiBjaG9vc2UgdG8gaW5jbHVkZSB0aGUgd2FybmluZyBtZXNzYWdlcyBpbg0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjIHRoZSBvdXRwdXQgZmlsZS4gDQogICAgICAgICAgICAgICAgICAgICAgbWVzc2FnZXMgPSBGQUxTRSwgICMNCiAgICAgICAgICAgICAgICAgICAgICByZXN1bHRzID0gVFJVRSwNCiAgICAgICAgICAgICAgICAgICAgICANCiAgICAgICAgICAgICAgICAgICAgICBjb21tZW50ID0gTkEgICAgICAgIyB5b3UgY2FuIGFsc28gZGVjaWRlIHdoZXRoZXIgdG8gaW5jbHVkZSB0aGUgb3V0cHV0DQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICMgaW4gdGhlIG91dHB1dCBmaWxlLg0KICAgICAgICAgICAgICAgICAgICAgICkgICANCmBgYA0KDQojIEludHJvZHVjdGlvbg0KDQojIyMgRGVzY3JpcHRpb24NCkluIHRoaXMgZGF0YSBzZXQgd2UgaGF2ZSBkYXRhIGZyb20gdGhlIDIwMDggTkZMIHNlYXNvbi4gTW9yZSBzcGVjaWZpY2FsbHkgd2UgaGF2ZSBmYWN0b3JzIHRoYXQgZ28gaW50byBORkwgZmllbGdvYWxzLiBTb21lIHZhcmlhYmxlcyBpbmNsdWRlIHRoZSBraWNraW5nIHRlYW0sIE5hbWUsIERpc3RhbmNlLCB0aW1lcmVtLCBkZWZzY29yZSwgYW5kIEdPT0QuDQoNCktpY2t0ZWFtIC0gTmFtZSBvZiB0aGUga2lja2luZyB0ZWFtIA0KTmFtZSAtIE5hbWUgb2YgdGhlIGtpY2tlcg0KRGlzdGFuY2UgLSBIb3cgZmFyIHRoZSBiYWxsIGlzIGZyb20gdGhlIGdvYWwNClRpbWVyZW0gLSBIb3cgbXVjaCB0aW1lIGlzIG9uIHRoZSBnYW1lIGNsb2NrIHJlbWFpbmluZyBpbiB0aGUgZ2FtZQ0KS2lja2RpZmYgLSB0aGUgbGV2ZWwgb2YgZGlmZmljdWx0eSBvZiB0aGUga2ljaw0KRGVmc2NvcmUtIFRoZSBzY29yZSBvZiB0aGUgb3Bwb3NpbmcgdGVhbQ0KR09PRCAtIElmIHRoZSBmaWVsZCBnb2FsIGlzIG1hZGUgb3Igbm90LCBhIDEgZm9yIGEgbWFrZSBhbmQgMCBmb3IgYSBtaXNzDQoNCiMjIyBRdWVzdGlvbg0KDQpUaGUgb2JqZWN0aXZlIG9mIHRoaXMgYW5hbHlzaXMgaXMgdG8gYnVpbGQgYSBsb2dpc3RpYyByZWdyZXNzaW9uIG1vZGVsIHRvIHByZWRpY3QgYSBtYWRlIGZpZWxkIGdvYWwgdXNpbmcgZGlmZnJlbnQgZmFjdG9ycyB0YXQgZ28gaW50byBhIG1ha2UuDQoNCiMjIyBEYXRhIENsZWFuaW5nDQoNCmBgYHtyfQ0KDQpmaWVsZGdvYWxzIDwtIHJlYWQuY3N2KCJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vVHlsZXJCYXR0YWdsaW5pL1NUQS0zMjEvcmVmcy9oZWFkcy9tYWluL25mbDIwMDhfZmdhLmNzdiIsIGhlYWRlciA9IFRSVUUpDQoNCmBgYA0KDQpgYGB7cn0NCmNsZWFuX2ZpZWxkZ29hbHMgPC0gbmEub21pdChmaWVsZGdvYWxzKQ0KY2xlYW5fZmllbGRnb2FscyA8LSBjbGVhbl9maWVsZGdvYWxzICU+JSBzZWxlY3QoLUdhbWVEYXRlLCAtQXdheVRlYW0sIC1Ib21lVGVhbSwgLXF0ciwgLW1pbiwgLXNlYywgLWRlZiwgLWRvd24sIC10b2dvLCAta2lja2VyLCAteWRsaW5lLCAtaG9tZWtpY2ssIC1vZmZzY29yZSwgLXNlYXNvbiwgLU1pc3NlZCwgLUJsb2NrZWQsIC1raWNrdGVhbSwgLW5hbWUpDQoNCmhlYWQoY2xlYW5fZmllbGRnb2FscykNCmBgYA0KDQpXZSB0YWtlIG91dCBhbnkgb2JzZXJ2YXRpb25zIHdpdGggYSBtaXNzaW5nIHZhbHVlLiBXZSBhbHNvIHRha2Ugb3V0IG1hbnkgdmFyaWFibGVzIGR1ZSB0byB0aGVyZSBiZWluZyBhIGhpZ2ggbGlrZWxleWhvb2QgZm9yIG11bHRpY29sbGluZWFpcnR5LiBXZSBhbHJlYWR5IGhhdmUgYSB2YXJpYWJsZSBmb3IgdGltZSBzbyB3ZSBlbGltaW5hdGVkIG1hbnkgdmFyaWFibGVzIHJlbGF0ZWQgdG8gdGltZS4gV2UgYWxzbyBhbHJlYWR5IGhhdmUgYSB2YXJpYWJsZSBmb3IgYSBtYWtlIHNvIHdlIGRvIG5vdCBuZWVkIGFueSBmb3IgYSBtaXNzIG9yIGJsb2NrZWQsIHRoYXQgd291bGQganVzdCBiZSBhIHJlcGVhdCBvdXIgZGF0YS4gVGhlIG90aGVycyBhcmUganVzdCBjYXRlZ29yaWNhbCB2YXJpYWJsZXMgdGhhdCBhcmUgdG8gaWRlbnRpZnkgdGhlIGtpY2tlciBvciBraWNraW5nIHRlYW0gd2hpY2ggYWdhaW4gd2UgYWxyZWFkeSBoYXZlIHZhcmlhYmxlcyB0aGF0IGRlc2NyaWJlIHRoYXQuIA0KDQpgYGB7cn0NCmxpYnJhcnkocHN5Y2gpDQpwYWlycy5wYW5lbHMoY2xlYW5fZmllbGRnb2Fsc1ssLTldLCANCiAgICAgICAgICAgICBtZXRob2QgPSAicGVhcnNvbiIsDQogICAgICAgICAgICAgaGlzdC5jb2wgPSAiIzAwQUZCQiIsDQogICAgICAgICAgICAgZGVuc2l0eSA9IFRSVUUsDQogICAgICAgICAgICAgZWxsaXBzZXMgPSBUUlVFDQogICAgICAgICAgICAgKQ0KYGBgDQpBbGwgb2Ygb3VyIHByZWRpY3RvciB2YWx1ZXMgYXJlIHVuaW1vZGFsIGV4Y2VwdCBmb3IgZGVmc2NvcmUuIA0KDQpgYGB7cn0NCnBhcihtZnJvdz1jKDEsMikpDQpoaXN0KGNsZWFuX2ZpZWxkZ29hbHMkZGVmc2NvcmUsIHhsYWI9ImRlZnNjb3JlIiwgbWFpbiA9ICIiKQ0KDQpgYGANCg0KQmFzZWQgb24gdGhlIGhpc3RvZ3JhbSBhYm92ZSB3ZSBkaXNjcmV0aXplIGRlZnNjb3JlDQoNCmBgYHtyfQ0KZGVmc2NvcmUgPSBjbGVhbl9maWVsZGdvYWxzJGRlZnNjb3JlDQpncnAuZGVmc2NvcmUgPSBkZWZzY29yZQ0KZ3JwLmRlZnNjb3JlW2RlZnNjb3JlICVpbiUgYygwOjEwKV0gPSAiMS0xMCINCmdycC5kZWZzY29yZVtkZWZzY29yZSAlaW4lIGMoMTE6MTgpXSA9ICIxMS0xOCINCmdycC5kZWZzY29yZVtkZWZzY29yZSAlaW4lIGMoMTk6MjYpXSA9ICIxOS0yNiINCmdycC5kZWZzY29yZVtkZWZzY29yZSAlaW4lIGMoMjc6OTkpXSA9ICIyNysiDQpjbGVhbl9maWVsZGdvYWxzJGdycC5kZWZzY29yZSA9IGdycC5kZWZzY29yZQ0KaGVhZChncnAuZGVmc2NvcmUpDQpgYGANCg0KVGhlcmUgaXMgc29tZSBjb3JyZWxhdGlvbiBiZXR3ZWVuIHRoZSB2YXJpYWJsZXMga2lja2RpZmYgdnMgZGVmc2NvcmUgYW5kIGtpY2tkaWZmIHZzIHRpbWVyZW0uIA0KDQpJbiBvdXIgc21hbGxlc3QgYW5kIGZpbmFsIG1vZGVsIHdlIHdhbnQgdG8gaGF2ZSBraWNrIGRpZmZpY3VsdHksIGRpc3RhbmNlLCB0aW1lIHJlbWFpbmluZyBiZWNhdXNlIHdlIGtub3cgZGlzdGFuY2UgaXMgYSBiaWcgaW5kaWNhdG9yIG9uIHdoZXRoZXIgb3Igbm90IGEgZmllbGQgZ29hbCBpcyBnb29kIG9yIG5vdCBhbmQgdGhlIHNhbWUgZ29lcyBmb3Iga2ljayBkaWZmaWN1bHR5IGFuZCB0aW1lIHJlbWFpbmluZy4NCg0KIyBJbnRyb2R1Y3Rpb24NCg0KIyMjIFN0YW5kYXJkaXppbmcgTnVtZXJpY2FsIFZhcmlhYmxlcw0KDQpXZSBmaXJzdCBzdGFuZGFyZGl6ZSBvdXIgbnVtZXJpY2FsIHZhbHVlcyBiZWNhdXNlIHdlIGRvIG5vdCBjYXJlIGFib3V0IHRoZSBpbnRlcnByZXRhdGlvbiBvZiB0aGUgY29lZmZpY2llbnRzIG9ubHkgdG8gaWRlbnRpZnkgdGhlIGJlc3QgbW9kZWwuDQoNCmBgYHtyfQ0KY2xlYW5fZmllbGRnb2FscyRzZC5kaXN0YW5jZSA9IChjbGVhbl9maWVsZGdvYWxzJGRpc3RhbmNlLW1lYW4oY2xlYW5fZmllbGRnb2FscyRkaXN0YW5jZSkpL3NkKGNsZWFuX2ZpZWxkZ29hbHMkZGlzdGFuY2UpDQpjbGVhbl9maWVsZGdvYWxzJHNkLmtpY2tkaWZmID0gKGNsZWFuX2ZpZWxkZ29hbHMka2lja2RpZmYtbWVhbihjbGVhbl9maWVsZGdvYWxzJGtpY2tkaWZmKSkvc2QoY2xlYW5fZmllbGRnb2FscyRraWNrZGlmZikNCmNsZWFuX2ZpZWxkZ29hbHMkc2QudGltZXJlbSA9IChjbGVhbl9maWVsZGdvYWxzJHRpbWVyZW0tbWVhbihjbGVhbl9maWVsZGdvYWxzJHRpbWVyZW0pKS9zZChjbGVhbl9maWVsZGdvYWxzJHRpbWVyZW0pDQpjbGVhbl9maWVsZGdvYWxzJHNkLmRlZnNjb3JlID0gKGNsZWFuX2ZpZWxkZ29hbHMkZGVmc2NvcmUtbWVhbihjbGVhbl9maWVsZGdvYWxzJGRlZnNjb3JlKSkvc2QoY2xlYW5fZmllbGRnb2FscyRkZWZzY29yZSkNCg0Kc2QuR09PRCA9IGNsZWFuX2ZpZWxkZ29hbHNbLCAtKDE6NCldDQoNCmBgYA0KDQojIyMgRGF0YSBTcGxpdA0KDQpXZSByYW5kb21seSBzcGxpdCB0aGUgZGF0YSBpbnRvIHR3byBzZXRzIG9uZSB3aXRoIDgwJSBvZiB0aGUgZGF0YSB3aGljaCBpcyB1c2VkIGFzIGEgdHJhaW5pbmcgZGF0YS4gV2UgdGhlbiB1c2UgdGhlIG90aGVyIDIwJSB0byBhc3Nlc3MgdGhlIHBlcmZvcm1hbmNlIG9mIHRoZSBtb2RlbA0KDQpgYGB7cn0NCm4gPC0gZGltKHNkLkdPT0QpWzFdDQp0cmFpbi5uIDwtIHJvdW5kKDAuOCpuKQ0KdHJhaW4uaWQgPC0gc2FtcGxlKDE6biwgdHJhaW4ubiwgcmVwbGFjZSA9IEZBTFNFKQ0KdHJhaW4gPC0gc2QuR09PRFt0cmFpbi5pZCwgXQ0KdGVzdCA8LSBzZC5HT09EWy10cmFpbi5pZCwgXQ0KDQpgYGANCg0KV2Ugd2lsbCB1c2UgYSA1LWZvbGQgY3Jvc3MgdmFsaWRhdGlvbiB0byBtYWtlIHN1cmUgd2UgaGF2ZSBlbm91Z2ggaWVsZCBnb2FsIGRhdGEuDQoNCmBgYHtyfQ0Kaz01DQpmb2xkLnNpemUgPSBmbG9vcihkaW0odHJhaW4pWzFdL2spDQpQRTEgPSByZXAoMCw1KQ0KUEUyID0gcmVwKDAsNSkNClBFMyA9IHJlcCgwLDUpDQoNCmZvcihpIGluIDE6aykgew0KICB2YWxpZC5pZCA9IChmb2xkLnNpemUqKGktMSkrMSk6KGZvbGQuc2l6ZSppKQ0KICB2YWxpZCA9IHRyYWluW3ZhbGlkLmlkLCBdDQogIHRyYWluLmRhdCA9IHRyYWluWy12YWxpZC5pZCxdDQogIA0KICBjYW5kaWRhdGUwMSA9IGdsbShHT09EIH4gc2QuZGlzdGFuY2UgKyBzZC5raWNrZGlmZiArIHNkLnRpbWVyZW0gKyANCiAgICAgICAgICAgICAgICAgICAgZ3JwLmRlZnNjb3JlLCBmYW1pbHkgPSBiaW5vbWlhbChsaW5rID0gImxvZ2l0IiksICANCiAgICAgICAgICAgICAgICAgICAgZGF0YSA9IHRyYWluLmRhdCkgIA0KDQogIGNhbmRpZGF0ZTAzID0gZ2xtKEdPT0QgfiBzZC5kaXN0YW5jZSArIGdycC5kZWZzY29yZSArIHNkLnRpbWVyZW0sIA0KICAgICAgICAgICAgICAgICAgICBmYW1pbHkgPSBiaW5vbWlhbChsaW5rID0gImxvZ2l0IiksICANCiAgICAgICAgICAgICAgICAgICAgZGF0YSA9IHRyYWluLmRhdCkgDQoNCiAgY2FuZGlkYXRlMDIgPSBzdGVwQUlDKGNhbmRpZGF0ZTAxLCANCiAgICAgICAgICAgICAgICAgICAgICAgIHNjb3BlID0gbGlzdChsb3dlcj1mb3JtdWxhKGNhbmRpZGF0ZTAzKSx1cHBlcj1mb3JtdWxhKGNhbmRpZGF0ZTAxKSksDQogICAgICAgICAgICAgICAgICAgICAgICBkaXJlY3Rpb24gPSAiZm9yd2FyZCIsICAgDQogICAgICAgICAgICAgICAgICAgICAgICB0cmFjZSA9IDApDQogIA0KICBwcmVkMDEgPSBwcmVkaWN0KGNhbmRpZGF0ZTAxLCBuZXdkYXRhID0gdmFsaWQsIHR5cGU9InJlc3BvbnNlIikNCiAgcHJlZDAyID0gcHJlZGljdChjYW5kaWRhdGUwMiwgbmV3ZGF0YSA9IHZhbGlkLCB0eXBlPSJyZXNwb25zZSIpDQogIHByZWQwMyA9IHByZWRpY3QoY2FuZGlkYXRlMDMsIG5ld2RhdGEgPSB2YWxpZCwgdHlwZT0icmVzcG9uc2UiKQ0KICANCiAgcHJlLm91dGNvbWUwMSA9IGlmZWxzZShhcy52ZWN0b3IocHJlZDAxKSA+IDAuNSwgMSwgMCkNCiAgcHJlLm91dGNvbWUwMiA9IGlmZWxzZShhcy52ZWN0b3IocHJlZDAyKSA+IDAuNSwgMSwgMCkNCiAgcHJlLm91dGNvbWUwMyA9IGlmZWxzZShhcy52ZWN0b3IocHJlZDAzKSA+IDAuNSwgMSwgMCkNCg0KICBQRTFbaV0gPSBzdW0ocHJlLm91dGNvbWUwMSA9PSB2YWxpZCRHT09EKS9sZW5ndGgocHJlZDAxKQ0KICBQRTJbaV0gPSBzdW0ocHJlLm91dGNvbWUwMiA9PSB2YWxpZCRHT09EKS9sZW5ndGgocHJlZDAyKQ0KICBQRTNbaV0gPSBzdW0ocHJlLm91dGNvbWUwMyA9PSB2YWxpZCRHT09EKS9sZW5ndGgocHJlZDAzKQ0KfQ0KDQphdmcucGUgPSBjYmluZChQRTEgPSBtZWFuKFBFMSksIFBFMiA9IG1lYW4oUEUyKSwgUEUzID0gbWVhbihQRTMpKQ0Ka2FibGUoYXZnLnBlLCBjYXB0aW9uID0gIkF2ZXJhZ2Ugb2YgcHJlZGljdGlvbiBlcnJvcnMgb2YgY2FuZGlkYXRlIG1vZGVscyIpDQoNCmBgYA0KDQpGcm9tIG91ciBvdXRwdXQgYWJvdmUgd2Ugc2VlIHRoYXQgbW9kZWwgMyBoYXMgdGhlIGhpZ2hlc3QgcHJlZGljdGlvbiBlcnJvciBvZiAuODY1LiBXZSBoYXZlIGEgY3V0b2ZmIG9mIC41IHNvIG91ciBwcmVkaWN0aXZlIGVycm9yIG9mIC44NjUgaXMgd2VsbCBhYm92ZSBvdXIgY3V0b2ZmLg0KDQojIyBGaW5hbCBNb2RlbA0KDQpgYGB7cn0NCnByZWQwMyA9IHByZWRpY3QoY2FuZGlkYXRlMDMsIG5ld2RhdGEgPSB0ZXN0LCB0eXBlPSJyZXNwb25zZSIpDQpwcmVkMDMub3V0Y29tZSA9IGlmZWxzZShhcy52ZWN0b3IocHJlZDAyKT4wLjUsICIxIiwgIjAiKQ0KDQphY2N1cmFjeSA9IHN1bShwcmVkMDMub3V0Y29tZSA9PSB0ZXN0JEdPT0QpL2xlbmd0aChwcmVkMDMpDQprYWJsZShhY2N1cmFjeSwgY2FwdGlvbj0iVGhlIGFjdHVhbCBhY2N1cmFjeSBvZiB0aGUgZmluYWwgbW9kZWwiKQ0KDQpgYGANCg0KRnJvbSB0aGUgb3V0cHV0IGFib3ZlIHdlIG9ic2VydmUgdGhlIHdpdGhoZWxkIHRlc3QgZGF0YS4gV2UgZ2V0IGEgYWNjdXJhY3kgcmF0ZSBvZiAuODUwLg0KDQojIENvbmNsdXNpb24NCg0KVGhpcyBhbmFseXNpcyB3YXMgb24gcHJlZGljdGluZyBhIG1hZGUgZmllbGQgZ29hbC4gV2UgdXNlZCB0aHJlZSBtb2RlbHMgYXMgY2FuZGlkYXRlcyBmb3Igb3VyIGZpbmFsIG1vZGVsIGFuZCB3ZSB1c2VkIGNyb3NzIHZhbGlkYXRpb24uIE91ciBmaW5hbCByZXN1bHQgZm9yIHRoZSBhY2N1cmFjeSBvZiBvdXIgZmluYWwgbW9kZWwgd2FzIGFwcHJveGltYXRlbHkgODUuMDIsIHdoaWNoIHNob3dzIHRoYXQgdGhlIG1vZGVsIGNvcnJlY3RseSBzaG93ZWQgODUlIG9mIHRoZSB0ZXN0IGRhdGEgYmFzZWQgb24gdGhlIHByZWRpY3RlZCBwcm9iYWJpbGl0aWVzIGFuZCBvdXIgY2hvc2VuIHRocmVzaG9sZCBvZiAuNS4gQW4gYWNjdXJhY3kgb2YgODUlIHNob3dzIHRoYXQgdGhlIG1vZGVsIGlzIHBlcmZvcm1pbmcgcmVsYXRpdmVseSB3ZWxsLCBpdCBzdWNjZXNzZnVsbHkgaWRlbnRpZmllZCB3aGV0aGVyIGFuIG9ic2VydmF0aW9uIGZhbGxzIGludG8gdGhlIGNsYXNzZXMgbWFkZSAoIjEiKSBvciBtaXNzICgiMCIpIGZvciA4NSUgb2YgdGhlIGNhc2VzLg==