For this report, i will again be analyzing Bank Loan Default Data.
The goal of this analysis is to demonstrate 4 types of sampling:
Simplified Random, Stratified Random, Systematic Random, and cluster
sampling.
Again, the data is still split into 9 seperate files. I will combine
all of them into 1 data set.
loan01 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational01.csv", header = TRUE)[, -1]
loan02 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational02.csv", header = TRUE)[, -1]
loan03 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational03.csv", header = TRUE)[, -1]
loan04 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational04.csv", header = TRUE)[, -1]
loan05 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational05.csv", header = TRUE)[, -1]
loan06 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational06.csv", header = TRUE)[, -1]
loan07 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational07.csv", header = TRUE)[, -1]
loan08 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational08.csv", header = TRUE)[, -1]
loan09 <- read.csv("https://pengdsci.github.io/datasets/SBAloan/w06-SBAnational09.csv", header = TRUE)[, -1]
loan = rbind(loan01, loan02, loan03, loan04, loan05, loan06, loan07, loan08, loan09)
Definitions
Before we continue, we must define some things and lay some ground
rules. 1) This data set will be treated as a population. This means that
we have a source population that is finite and contains 899,164
subjects. 2) Target Population: The loans within all states and the
regions they will turned into. 3) Our Study Population: The 899,154
values within the BankLoan data set once it is cleaned up for analysis.
4) Sampling Frame: The sampling frame is the data set “loan,” which
contains historical data from the SBA with 27 variables and 899,164
observations. 5) Sampling Unit: Each individual loan represented in the
data set.
Data Preparation
Deleting Missing
values
To start, we will remove missing values for both MIS_Status and
State, as i will use those variables to both our response and
stratification, respectively.
miss = loan[which(loan$MIS_Status == ""),]
newloan <- loan[-which(loan$MIS_Status == ""),]
newloan1 <- newloan[-which(newloan$State == ""),]
Sampling Methods
Simple Random
Sampling
The first of our methods is Simple Random Sampling. For this, I will
just take a random sample from the population. I will randomly choose to
sample 4,000 times.
RandomSample <- newloan1[sample(nrow(newloan1), 4000), ]
RandomSample_dim <- dim(RandomSample)
size_var_count <- data.frame(Size = RandomSample_dim[1], Var.count = RandomSample_dim[2])
kable(t(size_var_count), col.names = c("Size", "Var.count"))
Systematic Random
Sampling Process:
The second sampling method Systematic. For this method, we choose a
starting point, and then every nth data point after is chosen to be in
the sample. For this, I will let R choose the starting point and then
use an interval size of 4,000.
jump.size = dim(newloan1)[1] %/% 4000
rand.starting.pt = sample(1:jump.size, 1)
sampling.id = seq(rand.starting.pt, dim(newloan1)[1], jump.size)
sys.sample = newloan1[sampling.id, ]
sys.Sample.dim = dim(sys.sample)
names(sys.Sample.dim) = c("Size", "Var.count")
kable(t(sys.Sample.dim))
This resulted in a sample size of 4006. The sample size can vary
depending on start value and jump size.
Stratified Random
Sampling Process:
The third sampling method is stratified. sampling. For this method,
observations are randomly chosen from each strata, aka groups defined
based on a common characteristic. In our case, we will be organizing the
observations by geographical region using state, and then we will sample
from each of the 5 regions that will be created.
Grouping by
Region
Grouping all States by geographical regions:
newloan1 <- newloan1 %>%
mutate(State = case_when(
State %in% c("AL", "AR", "GA", "KY", "LA", "MS", "SC", "TN", "WV", "OK") ~ "SouthWest",
State %in% c("AK", "AZ", "CA", "CO", "HI", "ID", "MT", "NV", "NM", "OR", "UT", "WA", "WY") ~ "West",
State %in% c("CT", "DE", "MA", "MD", "ME", "NH", "NJ", "NY", "PA", "RI", "VT") ~ "Northeast",
State %in% c("IL", "IN", "IA", "KS", "MI", "MN", "MO", "NE", "ND", "OH", "SD", "WI") ~ "Midwest",
State %in% c("FL", "NC", "TX", "VA", "DC") ~ "Southeast",
TRUE ~ as.character(State)
))
Next, we can look at a frequency table for each stratum.
freq.table = table(newloan1$State) # frequency table of strNAICS
rel.freq = freq.table/sum(freq.table) # relative frequency
strata.size = round(rel.freq*4000) # strata size allocation
strata.names=names(strata.size)
Next, we will take a random sample of about 4
strata.sample = newloan1[1,] # create a reference data frame
strata.sample$add.id = 1 # add a temporary ID to because in the loop
# i =2 testing a single iteration
for (i in 1:length(strata.names)){
ith.strata.names = strata.names[i] # extract data frame names
ith.strata.size = strata.size[i] # allocated stratum size
# The following code identifies observations to be selected
ith.sampling.id = which(newloan1$State==ith.strata.names)
ith.strata = newloan1[ith.sampling.id,] # i-th stratified population
ith.strata$add.id = 1:dim(ith.strata)[1] # add sampling list/frame
# The following code generates a subset of random ID
ith.sampling.id = sample(1:dim(ith.strata)[1], ith.strata.size)
## Create a selection status -- pay attention to the operator: %in%
ith.sample =ith.strata[ith.strata$add.id %in%ith.sampling.id,]
## dim(ith.sample) $ check the sample
strata.sample = rbind(strata.sample, ith.sample) # stack all data frame!
}
# dim(strata.sample)
strat.sample.final = strata.sample[-1,] # drop the temporary stratum ID
#kable(head(strat.sample.final)) # accuracy check!
strat.sample.dim = dim(strat.sample.final)
names(strat.sample.dim) = c("Size", "Var.count")
kable(t(strat.sample.dim))
This method yielded a sample size of 3999.
Cluster Random Sampling
Process:
The last of our sampling methods is clustered. With this method,
observations are again organized into groups via a common
characteristic. However, unlike with Stratified sampling, these groups
are typically broken down further and entire groups are chosen to
compose the sample. For this report, i will further group the
observations by unique zip codes, and then groups of observations will
be chosen at random via a loop.
Defining
Clusters:
Clusters to be defined based on zip codes:
Now, we sample based on zip code:
# Take a cluster sample using zip codes
selected_clusters <- sample(clusters, 20) # Adjust the number of clusters as needed
ClusterSample <- newloan1[newloan1$Zip %in% selected_clusters, ]
# Print the size and variable count of the cluster sample
ClusterSample_dim <- dim(ClusterSample)
size_var_count_cluster <- data.frame(Size = ClusterSample_dim[1], Var.count = ClusterSample_dim[2])
kable(t(size_var_count_cluster), col.names = c("Size", "Var.count"))
This method yielded 664 total clusters.
Summary and
Conclusion
I analyzed bank loan default data. The goal of this report was to
demonstrate how simple, systemic, stratified, and cluster sampling
worked. For simple random sampling, i sampled 4000 units from the
population. For Systemic random sampling, i let R choose the random
starting point and used a jump size of 4 thousand, and that yielded a
sample size of 4006. For stratified random sampling, observations were
grouped based on geographical regions and then each group was sampled,
and that yielded a sample size of 3999. For cluster sampling, clusters
were formed via zip codes and then clusters were chosen at random. This
yielded 664 clusters.
LS0tDQp0aXRsZTogIkltcGxlbWVudGluZyBSYW1kb20gU2FtcGxpbmcgUGxhbnMgIg0KYXV0aG9yOiAiSWFuIFZhbldyaWdodCINCmRhdGU6ICIwNS8wOS8yMDI0Ig0Kb3V0cHV0Og0KICBodG1sX2RvY3VtZW50OiANCiAgICB0b2M6IHllcw0KICAgIHRvY19kZXB0aDogNA0KICAgIHRvY19mbG9hdDogeWVzDQogICAgZmlnX3dpZHRoOiA2DQogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMNCiAgICB0b2NfY29sbGFwc2VkOiB5ZXMNCiAgICBjb2RlX2ZvbGRpbmc6IGhpZGUNCiAgICBjb2RlX2Rvd25sb2FkOiB5ZXMNCiAgICBzbW9vdGhfc2Nyb2xsOiB0cnVlDQogICAgdGhlbWU6IHJlYWRhYmxlDQogICAgZmlnX2hlaWdodDogNA0KICB3b3JkX2RvY3VtZW50OiANCiAgICB0b2M6IHllcw0KICAgIHRvY19kZXB0aDogNA0KICAgIGZpZ19jYXB0aW9uOiB5ZXMNCiAgICBrZWVwX21kOiB5ZXMNCiAgcGRmX2RvY3VtZW50Og0KICAgIHRvYzogeWVzDQogICAgdG9jX2RlcHRoOiA0DQogICAgZmlnX2NhcHRpb246IHllcw0KICAgIG51bWJlcl9zZWN0aW9uczogeWVzDQogICAgZmlnX3dpZHRoOiA1DQogICAgZmlnX2hlaWdodDogNA0KLS0tDQoNCmBgYHs9aHRtbH0NCjxzdHlsZSB0eXBlPSJ0ZXh0L2NzcyI+DQpoMS50aXRsZSB7DQogIGZvbnQtc2l6ZTogMjBweDsNCiAgdGV4dC1hbGlnbjogY2VudGVyOw0KfQ0KaDQuYXV0aG9yIHsgDQogICAgZm9udC1zaXplOiAxOHB4Ow0KICAgIHRleHQtYWxpZ246IGNlbnRlcjsNCn0NCmg0LmRhdGUgeyANCiAgZm9udC1zaXplOiAxOHB4Ow0KICB0ZXh0LWFsaWduOiBjZW50ZXI7DQp9DQpoMSB7DQogICAgZm9udC1zaXplOiAyMnB4Ow0KICAgIHRleHQtYWxpZ246IGNlbnRlcjsNCn0NCmgyIHsNCiAgICBmb250LXNpemU6IDE4cHg7DQogICAgdGV4dC1hbGlnbjogbGVmdDsNCn0NCg0KZGl2I1RPQyBsaSB7DQogICAgbGlzdC1zdHlsZTpub25lOw0KfQ0KPC9zdHlsZT4NCmBgYA0KYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9DQojIGNvZGUgY2h1bmsgc3BlY2lmaWVzIHdoZXRoZXIgdGhlIFIgY29kZSwgd2FybmluZ3MsIGFuZCBvdXRwdXQgDQojIHdpbGwgYmUgaW5jbHVkZWQgaW4gdGhlIG91dHB1dCBmaWxlcy4NCmlmICghcmVxdWlyZSgia25pdHIiKSkgew0KICAgaW5zdGFsbC5wYWNrYWdlcygia25pdHIiKQ0KICAgbGlicmFyeShrbml0cikNCn0NCmlmICghcmVxdWlyZSgibGVzc1IiKSkgew0KICAgaW5zdGFsbC5wYWNrYWdlcygibGVzc1IiKQ0KICAgbGlicmFyeShsZXNzUikNCn0NCmxpYnJhcnkoZHBseXIpDQpsaWJyYXJ5KGthYmxlRXh0cmEpDQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUsICAgICAgIA0KICAgICAgICAgICAgICAgICAgICAgIHdhcm5pbmdzID0gRkFMU0UsICAgDQogICAgICAgICAgICAgICAgICAgICAgcmVzdWx0cyA9IFRSVUUsICAgDQogICAgICAgICAgICAgICAgICAgICAgbWVzc2FnZSA9IEZBTFNFLA0KICAgICAgICAgICAgICAgICAgICAgIGNvbW1lbnQgPSBOQSkNCmBgYA0KDQoNCkZvciB0aGlzIHJlcG9ydCwgaSB3aWxsIGFnYWluIGJlIGFuYWx5emluZyBCYW5rIExvYW4gRGVmYXVsdCBEYXRhLiBUaGUgZ29hbCBvZiB0aGlzIGFuYWx5c2lzIGlzIHRvIGRlbW9uc3RyYXRlIDQgdHlwZXMgb2Ygc2FtcGxpbmc6IFNpbXBsaWZpZWQgUmFuZG9tLCBTdHJhdGlmaWVkIFJhbmRvbSwgU3lzdGVtYXRpYyBSYW5kb20sIGFuZCBjbHVzdGVyIHNhbXBsaW5nLg0KIA0KQWdhaW4sIHRoZSBkYXRhIGlzIHN0aWxsIHNwbGl0IGludG8gOSBzZXBlcmF0ZSBmaWxlcy4gSSB3aWxsIGNvbWJpbmUgYWxsIG9mIHRoZW0gaW50byAxIGRhdGEgc2V0Lg0KYGBge3J9DQpsb2FuMDEgPC0gcmVhZC5jc3YoImh0dHBzOi8vcGVuZ2RzY2kuZ2l0aHViLmlvL2RhdGFzZXRzL1NCQWxvYW4vdzA2LVNCQW5hdGlvbmFsMDEuY3N2IiwgaGVhZGVyID0gVFJVRSlbLCAtMV0NCmxvYW4wMiA8LSByZWFkLmNzdigiaHR0cHM6Ly9wZW5nZHNjaS5naXRodWIuaW8vZGF0YXNldHMvU0JBbG9hbi93MDYtU0JBbmF0aW9uYWwwMi5jc3YiLCBoZWFkZXIgPSBUUlVFKVssIC0xXQ0KbG9hbjAzIDwtIHJlYWQuY3N2KCJodHRwczovL3Blbmdkc2NpLmdpdGh1Yi5pby9kYXRhc2V0cy9TQkFsb2FuL3cwNi1TQkFuYXRpb25hbDAzLmNzdiIsIGhlYWRlciA9IFRSVUUpWywgLTFdDQpsb2FuMDQgPC0gcmVhZC5jc3YoImh0dHBzOi8vcGVuZ2RzY2kuZ2l0aHViLmlvL2RhdGFzZXRzL1NCQWxvYW4vdzA2LVNCQW5hdGlvbmFsMDQuY3N2IiwgaGVhZGVyID0gVFJVRSlbLCAtMV0NCmxvYW4wNSA8LSByZWFkLmNzdigiaHR0cHM6Ly9wZW5nZHNjaS5naXRodWIuaW8vZGF0YXNldHMvU0JBbG9hbi93MDYtU0JBbmF0aW9uYWwwNS5jc3YiLCBoZWFkZXIgPSBUUlVFKVssIC0xXQ0KbG9hbjA2IDwtIHJlYWQuY3N2KCJodHRwczovL3Blbmdkc2NpLmdpdGh1Yi5pby9kYXRhc2V0cy9TQkFsb2FuL3cwNi1TQkFuYXRpb25hbDA2LmNzdiIsIGhlYWRlciA9IFRSVUUpWywgLTFdDQpsb2FuMDcgPC0gcmVhZC5jc3YoImh0dHBzOi8vcGVuZ2RzY2kuZ2l0aHViLmlvL2RhdGFzZXRzL1NCQWxvYW4vdzA2LVNCQW5hdGlvbmFsMDcuY3N2IiwgaGVhZGVyID0gVFJVRSlbLCAtMV0NCmxvYW4wOCA8LSByZWFkLmNzdigiaHR0cHM6Ly9wZW5nZHNjaS5naXRodWIuaW8vZGF0YXNldHMvU0JBbG9hbi93MDYtU0JBbmF0aW9uYWwwOC5jc3YiLCBoZWFkZXIgPSBUUlVFKVssIC0xXQ0KbG9hbjA5IDwtIHJlYWQuY3N2KCJodHRwczovL3Blbmdkc2NpLmdpdGh1Yi5pby9kYXRhc2V0cy9TQkFsb2FuL3cwNi1TQkFuYXRpb25hbDA5LmNzdiIsIGhlYWRlciA9IFRSVUUpWywgLTFdDQpsb2FuID0gcmJpbmQobG9hbjAxLCBsb2FuMDIsIGxvYW4wMywgbG9hbjA0LCBsb2FuMDUsIGxvYW4wNiwgbG9hbjA3LCBsb2FuMDgsIGxvYW4wOSkNCmBgYA0KDQoNCiMgRGVmaW5pdGlvbnMNCkJlZm9yZSB3ZSBjb250aW51ZSwgd2UgbXVzdCBkZWZpbmUgc29tZSB0aGluZ3MgYW5kIGxheSBzb21lIGdyb3VuZCBydWxlcy4gDQoxKSBUaGlzIGRhdGEgc2V0IHdpbGwgYmUgdHJlYXRlZCBhcyBhIHBvcHVsYXRpb24uIFRoaXMgbWVhbnMgdGhhdCB3ZSBoYXZlIGEgc291cmNlIHBvcHVsYXRpb24gdGhhdCBpcyBmaW5pdGUgYW5kIGNvbnRhaW5zIDg5OSwxNjQgc3ViamVjdHMuIA0KMikgVGFyZ2V0IFBvcHVsYXRpb246IFRoZSBsb2FucyB3aXRoaW4gYWxsIHN0YXRlcyBhbmQgdGhlIHJlZ2lvbnMgdGhleSB3aWxsIHR1cm5lZCBpbnRvLiANCjMpIE91ciBTdHVkeSBQb3B1bGF0aW9uOiBUaGUgODk5LDE1NCB2YWx1ZXMgd2l0aGluIHRoZSBCYW5rTG9hbiBkYXRhIHNldCBvbmNlIGl0IGlzIGNsZWFuZWQgdXAgZm9yIGFuYWx5c2lzLg0KNCkgU2FtcGxpbmcgRnJhbWU6IFRoZSBzYW1wbGluZyBmcmFtZSBpcyB0aGUgZGF0YSBzZXQgImxvYW4sIiB3aGljaCBjb250YWlucyBoaXN0b3JpY2FsIGRhdGEgZnJvbSB0aGUgU0JBIHdpdGggMjcgdmFyaWFibGVzIGFuZCA4OTksMTY0IG9ic2VydmF0aW9ucy4NCjUpIFNhbXBsaW5nIFVuaXQ6IEVhY2ggaW5kaXZpZHVhbCBsb2FuIHJlcHJlc2VudGVkIGluIHRoZSBkYXRhIHNldC4gDQoNCg0KIyBEYXRhIFByZXBhcmF0aW9uDQoNCiMjIERlbGV0aW5nIE1pc3NpbmcgdmFsdWVzDQoNClRvIHN0YXJ0LCB3ZSB3aWxsIHJlbW92ZSBtaXNzaW5nIHZhbHVlcyBmb3IgYm90aCBNSVNfU3RhdHVzIGFuZCBTdGF0ZSwgYXMgaSB3aWxsIHVzZSB0aG9zZSB2YXJpYWJsZXMgdG8gYm90aCBvdXIgcmVzcG9uc2UgYW5kIHN0cmF0aWZpY2F0aW9uLCByZXNwZWN0aXZlbHkuDQpgYGB7cn0NCm1pc3MgPSBsb2FuW3doaWNoKGxvYW4kTUlTX1N0YXR1cyA9PSAiIiksXQ0KbmV3bG9hbiA8LSBsb2FuWy13aGljaChsb2FuJE1JU19TdGF0dXMgPT0gIiIpLF0NCm5ld2xvYW4xIDwtIG5ld2xvYW5bLXdoaWNoKG5ld2xvYW4kU3RhdGUgPT0gIiIpLF0NCmBgYA0KDQojIFNhbXBsaW5nIE1ldGhvZHMNCiMjIFNpbXBsZSBSYW5kb20gU2FtcGxpbmcNClRoZSBmaXJzdCBvZiBvdXIgbWV0aG9kcyBpcyBTaW1wbGUgUmFuZG9tIFNhbXBsaW5nLiBGb3IgdGhpcywgSSB3aWxsIGp1c3QgdGFrZSBhIHJhbmRvbSBzYW1wbGUgZnJvbSB0aGUgcG9wdWxhdGlvbi4gSSB3aWxsIHJhbmRvbWx5IGNob29zZSB0byBzYW1wbGUgNCwwMDAgdGltZXMuIA0KYGBge3J9DQpSYW5kb21TYW1wbGUgPC0gbmV3bG9hbjFbc2FtcGxlKG5yb3cobmV3bG9hbjEpLCA0MDAwKSwgXQ0KUmFuZG9tU2FtcGxlX2RpbSA8LSBkaW0oUmFuZG9tU2FtcGxlKQ0Kc2l6ZV92YXJfY291bnQgPC0gZGF0YS5mcmFtZShTaXplID0gUmFuZG9tU2FtcGxlX2RpbVsxXSwgVmFyLmNvdW50ID0gUmFuZG9tU2FtcGxlX2RpbVsyXSkNCg0Ka2FibGUodChzaXplX3Zhcl9jb3VudCksIGNvbC5uYW1lcyA9IGMoIlNpemUiLCAiVmFyLmNvdW50IikpDQpgYGANCg0KIyMgU3lzdGVtYXRpYyBSYW5kb20gU2FtcGxpbmcgUHJvY2VzczoNClRoZSBzZWNvbmQgc2FtcGxpbmcgbWV0aG9kIFN5c3RlbWF0aWMuIEZvciB0aGlzIG1ldGhvZCwgd2UgY2hvb3NlIGEgc3RhcnRpbmcgcG9pbnQsIGFuZCB0aGVuIGV2ZXJ5IG50aCBkYXRhIHBvaW50IGFmdGVyIGlzIGNob3NlbiB0byBiZSBpbiB0aGUgc2FtcGxlLiBGb3IgdGhpcywgSSB3aWxsIGxldCBSIGNob29zZSB0aGUgc3RhcnRpbmcgcG9pbnQgYW5kIHRoZW4gdXNlIGFuIGludGVydmFsIHNpemUgb2YgNCwwMDAuDQpgYGB7cn0NCmp1bXAuc2l6ZSA9IGRpbShuZXdsb2FuMSlbMV0gJS8lIDQwMDANCnJhbmQuc3RhcnRpbmcucHQgPSBzYW1wbGUoMTpqdW1wLnNpemUsIDEpDQpzYW1wbGluZy5pZCA9IHNlcShyYW5kLnN0YXJ0aW5nLnB0LCBkaW0obmV3bG9hbjEpWzFdLCBqdW1wLnNpemUpDQpzeXMuc2FtcGxlID0gbmV3bG9hbjFbc2FtcGxpbmcuaWQsIF0NCg0Kc3lzLlNhbXBsZS5kaW0gPSBkaW0oc3lzLnNhbXBsZSkNCm5hbWVzKHN5cy5TYW1wbGUuZGltKSA9IGMoIlNpemUiLCAiVmFyLmNvdW50IikNCmthYmxlKHQoc3lzLlNhbXBsZS5kaW0pKQ0KYGBgDQoNClRoaXMgcmVzdWx0ZWQgaW4gYSBzYW1wbGUgc2l6ZSBvZiA0MDA2LiBUaGUgc2FtcGxlIHNpemUgY2FuIHZhcnkgZGVwZW5kaW5nIG9uIHN0YXJ0IHZhbHVlIGFuZCBqdW1wIHNpemUuDQoNCiMgU3RyYXRpZmllZCBSYW5kb20gU2FtcGxpbmcgUHJvY2VzczoNClRoZSB0aGlyZCBzYW1wbGluZyBtZXRob2QgaXMgc3RyYXRpZmllZC4gc2FtcGxpbmcuIEZvciB0aGlzIG1ldGhvZCwgb2JzZXJ2YXRpb25zIGFyZSByYW5kb21seSBjaG9zZW4gZnJvbSBlYWNoIHN0cmF0YSwgYWthIGdyb3VwcyBkZWZpbmVkIGJhc2VkIG9uIGEgY29tbW9uIGNoYXJhY3RlcmlzdGljLiBJbiBvdXIgY2FzZSwgd2Ugd2lsbCBiZSBvcmdhbml6aW5nIHRoZSBvYnNlcnZhdGlvbnMgYnkgZ2VvZ3JhcGhpY2FsIHJlZ2lvbiB1c2luZyBzdGF0ZSwgYW5kIHRoZW4gd2Ugd2lsbCBzYW1wbGUgZnJvbSBlYWNoIG9mIHRoZSA1IHJlZ2lvbnMgdGhhdCB3aWxsIGJlIGNyZWF0ZWQuDQoNCg0KIyMgR3JvdXBpbmcgYnkgUmVnaW9uDQpHcm91cGluZyBhbGwgU3RhdGVzIGJ5IGdlb2dyYXBoaWNhbCByZWdpb25zOg0KYGBge3J9DQpuZXdsb2FuMSA8LSBuZXdsb2FuMSAlPiUNCiAgbXV0YXRlKFN0YXRlID0gY2FzZV93aGVuKA0KICAgIFN0YXRlICVpbiUgYygiQUwiLCAiQVIiLCAiR0EiLCAiS1kiLCAiTEEiLCAiTVMiLCAiU0MiLCAiVE4iLCAiV1YiLCAiT0siKSB+ICJTb3V0aFdlc3QiLA0KICAgIFN0YXRlICVpbiUgYygiQUsiLCAiQVoiLCAiQ0EiLCAiQ08iLCAiSEkiLCAiSUQiLCAiTVQiLCAiTlYiLCAiTk0iLCAiT1IiLCAiVVQiLCAiV0EiLCAiV1kiKSB+ICJXZXN0IiwNCiAgICBTdGF0ZSAlaW4lIGMoIkNUIiwgIkRFIiwgIk1BIiwgIk1EIiwgIk1FIiwgIk5IIiwgIk5KIiwgIk5ZIiwgIlBBIiwgIlJJIiwgIlZUIikgfiAiTm9ydGhlYXN0IiwNCiAgICBTdGF0ZSAlaW4lIGMoIklMIiwgIklOIiwgIklBIiwgIktTIiwgIk1JIiwgIk1OIiwgIk1PIiwgIk5FIiwgIk5EIiwgIk9IIiwgIlNEIiwgIldJIikgfiAiTWlkd2VzdCIsDQogICAgU3RhdGUgJWluJSBjKCJGTCIsICJOQyIsICJUWCIsICJWQSIsICJEQyIpIH4gIlNvdXRoZWFzdCIsDQogICAgVFJVRSB+IGFzLmNoYXJhY3RlcihTdGF0ZSkNCiAgKSkNCmBgYA0KDQpOZXh0LCB3ZSBjYW4gbG9vayBhdCBhIGZyZXF1ZW5jeSB0YWJsZSBmb3IgZWFjaCBzdHJhdHVtLg0KYGBge3J9DQpmcmVxLnRhYmxlID0gdGFibGUobmV3bG9hbjEkU3RhdGUpICAjIGZyZXF1ZW5jeSB0YWJsZSBvZiBzdHJOQUlDUw0KcmVsLmZyZXEgPSBmcmVxLnRhYmxlL3N1bShmcmVxLnRhYmxlKSAgICMgcmVsYXRpdmUgZnJlcXVlbmN5IA0Kc3RyYXRhLnNpemUgPSByb3VuZChyZWwuZnJlcSo0MDAwKSAgICAgICMgc3RyYXRhIHNpemUgYWxsb2NhdGlvbg0Kc3RyYXRhLm5hbWVzPW5hbWVzKHN0cmF0YS5zaXplKSAgDQpgYGANCg0KTmV4dCwgd2Ugd2lsbCB0YWtlIGEgcmFuZG9tIHNhbXBsZSBvZiBhYm91dCA0DQpgYGB7cn0NCnN0cmF0YS5zYW1wbGUgPSBuZXdsb2FuMVsxLF0gICAgIyBjcmVhdGUgYSByZWZlcmVuY2UgZGF0YSBmcmFtZQ0Kc3RyYXRhLnNhbXBsZSRhZGQuaWQgPSAxICAgIyBhZGQgYSB0ZW1wb3JhcnkgSUQgdG8gYmVjYXVzZSBpbiB0aGUgbG9vcA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgIyBpID0yIHRlc3RpbmcgYSBzaW5nbGUgaXRlcmF0aW9uDQpmb3IgKGkgaW4gMTpsZW5ndGgoc3RyYXRhLm5hbWVzKSl7DQogICBpdGguc3RyYXRhLm5hbWVzID0gc3RyYXRhLm5hbWVzW2ldICAgIyBleHRyYWN0IGRhdGEgZnJhbWUgbmFtZXMNCiAgIGl0aC5zdHJhdGEuc2l6ZSA9IHN0cmF0YS5zaXplW2ldICAgICAjIGFsbG9jYXRlZCBzdHJhdHVtIHNpemUNCiAgICMgVGhlIGZvbGxvd2luZyBjb2RlIGlkZW50aWZpZXMgb2JzZXJ2YXRpb25zIHRvIGJlIHNlbGVjdGVkDQogICBpdGguc2FtcGxpbmcuaWQgPSB3aGljaChuZXdsb2FuMSRTdGF0ZT09aXRoLnN0cmF0YS5uYW1lcykgDQogICBpdGguc3RyYXRhID0gbmV3bG9hbjFbaXRoLnNhbXBsaW5nLmlkLF0gICMgaS10aCBzdHJhdGlmaWVkIHBvcHVsYXRpb24NCiAgIGl0aC5zdHJhdGEkYWRkLmlkID0gMTpkaW0oaXRoLnN0cmF0YSlbMV0gICMgYWRkIHNhbXBsaW5nIGxpc3QvZnJhbWUNCiAgICMgVGhlIGZvbGxvd2luZyBjb2RlIGdlbmVyYXRlcyBhIHN1YnNldCBvZiByYW5kb20gSUQNCiAgIGl0aC5zYW1wbGluZy5pZCA9IHNhbXBsZSgxOmRpbShpdGguc3RyYXRhKVsxXSwgaXRoLnN0cmF0YS5zaXplKSANCiAgICMjIENyZWF0ZSBhIHNlbGVjdGlvbiBzdGF0dXMgLS0gcGF5IGF0dGVudGlvbiB0byB0aGUgb3BlcmF0b3I6ICVpbiUgDQogICBpdGguc2FtcGxlID1pdGguc3RyYXRhW2l0aC5zdHJhdGEkYWRkLmlkICVpbiVpdGguc2FtcGxpbmcuaWQsXQ0KICAgIyMgZGltKGl0aC5zYW1wbGUpICAgICAgICAgJCBjaGVjayB0aGUgc2FtcGxlDQogICBzdHJhdGEuc2FtcGxlID0gcmJpbmQoc3RyYXRhLnNhbXBsZSwgaXRoLnNhbXBsZSkgICMgc3RhY2sgYWxsIGRhdGEgZnJhbWUhDQogfQ0KICMgZGltKHN0cmF0YS5zYW1wbGUpDQogc3RyYXQuc2FtcGxlLmZpbmFsID0gc3RyYXRhLnNhbXBsZVstMSxdICAjIGRyb3AgdGhlIHRlbXBvcmFyeSBzdHJhdHVtIElEDQogI2thYmxlKGhlYWQoc3RyYXQuc2FtcGxlLmZpbmFsKSkgICAgICAgICAjIGFjY3VyYWN5IGNoZWNrIQ0KIA0Kc3RyYXQuc2FtcGxlLmRpbSA9IGRpbShzdHJhdC5zYW1wbGUuZmluYWwpDQpuYW1lcyhzdHJhdC5zYW1wbGUuZGltKSA9IGMoIlNpemUiLCAiVmFyLmNvdW50IikNCmthYmxlKHQoc3RyYXQuc2FtcGxlLmRpbSkpDQogDQpgYGANCg0KVGhpcyBtZXRob2QgeWllbGRlZCBhIHNhbXBsZSBzaXplIG9mIDM5OTkuDQoNCiMgQ2x1c3RlciBSYW5kb20gU2FtcGxpbmcgUHJvY2VzczoNClRoZSBsYXN0IG9mIG91ciBzYW1wbGluZyBtZXRob2RzIGlzIGNsdXN0ZXJlZC4gV2l0aCB0aGlzIG1ldGhvZCwgb2JzZXJ2YXRpb25zIGFyZSBhZ2FpbiBvcmdhbml6ZWQgaW50byBncm91cHMgdmlhIGEgY29tbW9uIGNoYXJhY3RlcmlzdGljLiBIb3dldmVyLCB1bmxpa2Ugd2l0aCBTdHJhdGlmaWVkIHNhbXBsaW5nLCB0aGVzZSBncm91cHMgYXJlIHR5cGljYWxseSBicm9rZW4gZG93biBmdXJ0aGVyIGFuZCBlbnRpcmUgZ3JvdXBzIGFyZSBjaG9zZW4gdG8gY29tcG9zZSB0aGUgc2FtcGxlLiBGb3IgdGhpcyByZXBvcnQsIGkgd2lsbCBmdXJ0aGVyIGdyb3VwIHRoZSBvYnNlcnZhdGlvbnMgYnkgdW5pcXVlIHppcCBjb2RlcywgYW5kIHRoZW4gZ3JvdXBzIG9mIG9ic2VydmF0aW9ucyB3aWxsIGJlIGNob3NlbiBhdCByYW5kb20gdmlhIGEgbG9vcC4NCg0KIyMgRGVmaW5pbmcgQ2x1c3RlcnM6IA0KDQpDbHVzdGVycyB0byBiZSBkZWZpbmVkIGJhc2VkIG9uIHppcCBjb2RlczoNCmBgYHtyLCBlY2hvPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQ0KY2x1c3RlcnMgPC0gdW5pcXVlKG5ld2xvYW4xJFppcCkNCg0KYGBgDQoNCk5vdywgd2Ugc2FtcGxlIGJhc2VkIG9uIHppcCBjb2RlOg0KYGBge3J9DQoNCiMgVGFrZSBhIGNsdXN0ZXIgc2FtcGxlIHVzaW5nIHppcCBjb2Rlcw0Kc2VsZWN0ZWRfY2x1c3RlcnMgPC0gc2FtcGxlKGNsdXN0ZXJzLCAyMCkgICMgQWRqdXN0IHRoZSBudW1iZXIgb2YgY2x1c3RlcnMgYXMgbmVlZGVkDQpDbHVzdGVyU2FtcGxlIDwtIG5ld2xvYW4xW25ld2xvYW4xJFppcCAlaW4lIHNlbGVjdGVkX2NsdXN0ZXJzLCBdDQoNCg0KIyBQcmludCB0aGUgc2l6ZSBhbmQgdmFyaWFibGUgY291bnQgb2YgdGhlIGNsdXN0ZXIgc2FtcGxlDQpDbHVzdGVyU2FtcGxlX2RpbSA8LSBkaW0oQ2x1c3RlclNhbXBsZSkNCnNpemVfdmFyX2NvdW50X2NsdXN0ZXIgPC0gZGF0YS5mcmFtZShTaXplID0gQ2x1c3RlclNhbXBsZV9kaW1bMV0sIFZhci5jb3VudCA9IENsdXN0ZXJTYW1wbGVfZGltWzJdKQ0Ka2FibGUodChzaXplX3Zhcl9jb3VudF9jbHVzdGVyKSwgY29sLm5hbWVzID0gYygiU2l6ZSIsICJWYXIuY291bnQiKSkNCmBgYA0KVGhpcyBtZXRob2QgeWllbGRlZCA2NjQgdG90YWwgY2x1c3RlcnMuDQoNCiMjIFN1bW1hcnkgYW5kIENvbmNsdXNpb24NCkkgYW5hbHl6ZWQgYmFuayBsb2FuIGRlZmF1bHQgZGF0YS4gVGhlIGdvYWwgb2YgdGhpcyByZXBvcnQgd2FzIHRvIGRlbW9uc3RyYXRlIGhvdyBzaW1wbGUsIHN5c3RlbWljLCBzdHJhdGlmaWVkLCBhbmQgY2x1c3RlciBzYW1wbGluZyB3b3JrZWQuIEZvciBzaW1wbGUgcmFuZG9tIHNhbXBsaW5nLCBpIHNhbXBsZWQgNDAwMCB1bml0cyBmcm9tIHRoZSBwb3B1bGF0aW9uLiBGb3IgU3lzdGVtaWMgcmFuZG9tIHNhbXBsaW5nLCBpIGxldCBSIGNob29zZSB0aGUgcmFuZG9tIHN0YXJ0aW5nIHBvaW50IGFuZCB1c2VkIGEganVtcCBzaXplIG9mIDQgdGhvdXNhbmQsIGFuZCB0aGF0IHlpZWxkZWQgYSBzYW1wbGUgc2l6ZSBvZiA0MDA2LiBGb3Igc3RyYXRpZmllZCByYW5kb20gc2FtcGxpbmcsIG9ic2VydmF0aW9ucyB3ZXJlIGdyb3VwZWQgYmFzZWQgb24gZ2VvZ3JhcGhpY2FsIHJlZ2lvbnMgYW5kIHRoZW4gZWFjaCBncm91cCB3YXMgc2FtcGxlZCwgYW5kIHRoYXQgeWllbGRlZCBhIHNhbXBsZSBzaXplIG9mIDM5OTkuIEZvciBjbHVzdGVyIHNhbXBsaW5nLCBjbHVzdGVycyB3ZXJlIGZvcm1lZCB2aWEgemlwIGNvZGVzIGFuZCB0aGVuIGNsdXN0ZXJzIHdlcmUgY2hvc2VuIGF0IHJhbmRvbS4gVGhpcyB5aWVsZGVkIDY2NCBjbHVzdGVycy4NCg0KDQo=