Predicting Employee Churn: A Short Comparision between Xgboost and Keras Deep Learning
R Machine Learning Series
Author: Nguyen Chi Dung
Xgboost vs. Keras Deep Learning
Tò mò tìm hiểu coi những thuật toán nào thường được sử dụng bởi các đội thuộc Top 5 trên sân chơi Kaggle về khoa học dữ liệu thì vấp phải cụm từ Primary ML software used by top-5 teams on Kaggle với cái hình như trên. Đại ý là Keras Deep Learning xếp đầu bảng, LightGBM và Xgboost lần lượt xếp vị trị thứ hai và ba. Tuy vậy quan điểm của mình là Using the Right Tool for the Right Job: không có một mô hình nào chiếm ưu thế, với mỗi bộ dữ liệu, một vấn đề và hoàn cảnh khác nhau thì một mô hình được cho là “yếu” lại có thể hiệu quả hơn.
Cụ thể hơn chúng ta xét một bài toán rất cụ thể: bài toán Binary Classification với bộ dữ liệu IBM Watson Telco Customer Churn Data Set. Mục tiêu của bài toán là xây dựng mô hình dự báo nhân viên sẽ rời bỏ công ti và tiêu chuẩn lựa chọn mô hình tốt, giả sử, là ROC-AUC. Chúng ta sẽ so sánh khả năng dự báo của Artificial Neural Network và Xgboost căn cứ vào tiêu chí này.
Data Preprocessing
Trước hết chúng ta thực hiện việc tiền xử lí số liệu - chuẩn bị số liệu như đã trình bày trong post trước:
# Load data:
library(tidyverse)
churn_data_raw <- read_csv("WA_Fn-UseC_-Telco-Customer-Churn.csv")
# Remove unnecessary data:
churn_data_tbl <- churn_data_raw %>%
select(-customerID) %>%
drop_na()
# Conduct one-hot encoding process (https://rdrr.io/rforge/caret/man/dummyVars.html):
library(caret)
dummies <- dummyVars("~ .", data = churn_data_tbl %>% select(-Churn))
predict(dummies, churn_data_tbl %>% select(-Churn)) %>% as.data.frame() -> df_features
# Normalize 0-1 for data:
df_features %>%
mutate_all(function(x) {(x - min(x)) / (max(x) - min(x))}) %>%
mutate(Churn = case_when(churn_data_tbl$Churn == "Yes" ~ 1, TRUE ~ 0)) -> df_final
# Split data (train data for testing and test data for evaluation/comparing):
set.seed(1)
id <- createDataPartition(df_final$Churn, p = 0.7, list = FALSE)
train <- df_final[id, ]
test <- df_final[-id, ]
x_train_tbl <- train %>%
select(-Churn) %>%
as.matrix()
n <- ncol(x_train_tbl)
y_train_vec <- train %>% pull(Churn)
x_test_tbl <- test %>%
select(-Churn) %>%
as.matrix()
y_test_vec <- test %>% pull(Churn)Artificial Neural Network
Trước hết chúng ta xây dựng một Artificial Neural Network (ANN) hai layers trong đó tỉ lệ drop-out lần lượt là 20%, 20% với learning rate = 0.1:
#--------------------------------------
# Train Artificial Neural Network
# with some parameters selected
#--------------------------------------
# Building our Artificial Neural Network:
library(keras)
model_keras <- keras_model_sequential()
model_keras %>%
# First hidden layer:
layer_dense(units = 16,
kernel_initializer = "uniform",
activation = "relu",
input_shape = n) %>%
# Dropout rate to prevent overfitting:
layer_dropout(rate = 0.2, seed = 29) %>%
# Second hidden layer:
layer_dense(units = 16,
kernel_initializer = "uniform",
activation = "relu") %>%
# Dropout rate to prevent overfitting again:
layer_dropout(rate = 0.2, seed = 29) %>%
# Output layer:
layer_dense(units = 1,
kernel_initializer = "uniform",
activation = "sigmoid") %>%
# Compile ANN:
compile(optimizer = optimizer_adamax(lr = 0.1),
loss = "binary_crossentropy",
metrics = "accuracy")Rồi thực hiện huấn luyện ANN trên train data:
# Train the ANN keras model to the training data:
fit_keras <- fit(object = model_keras,
x = x_train_tbl,
y = y_train_vec,
batch_size = 2^6,
epochs = 30,
validation_split = 0.30)Chúng ta có thể xem quá trình huấn luyện ANN bằng công cụ hình ảnh như sau:
# Training process by plot:
plot(fit_keras) +
labs(title = "Figure 1: Deep Learning Training Results",
subtitle = "Data Source: IBM Watson Telco Customer Churn Data Set")
OK. Với ANN đã có chúng ta có thể đánh giá sơ bộ chất lượng dự báo của nó trên test data. Trước hết là ma trận nhầm lẫn với ngưỡng mặc định 0.5:
# Predicted Probability:
prob_keras <- predict_proba(model_keras, x_test_tbl) %>% as.vector()
# Function calculates confusion matrix:
my_cm <- function(prediction, actual, cutoff) {
pred <- case_when(prediction >= cutoff ~ "Yes", TRUE ~ "No") %>% as.factor()
thuc_te <- case_when(actual == 1 ~ "Yes", TRUE ~ "No") %>% as.factor()
confusionMatrix(pred, thuc_te, positive = "Yes")
}
# Confusion matrix with default cutoff = 0.5:
my_cm(prob_keras, y_test_vec, cutoff = 0.5)## Confusion Matrix and Statistics
##
## Reference
## Prediction No Yes
## No 1333 228
## Yes 200 348
##
## Accuracy : 0.7971
## 95% CI : (0.7793, 0.814)
## No Information Rate : 0.7269
## P-Value [Acc > NIR] : 5.496e-14
##
## Kappa : 0.481
##
## Mcnemar's Test P-Value : 0.1919
##
## Sensitivity : 0.6042
## Specificity : 0.8695
## Pos Pred Value : 0.6350
## Neg Pred Value : 0.8539
## Prevalence : 0.2731
## Detection Rate : 0.1650
## Detection Prevalence : 0.2598
## Balanced Accuracy : 0.7369
##
## 'Positive' Class : Yes
## Và ROC-AUC - tiêu chuẩn được chọn để đánh giá chất lượng dự báo của mô hình là:
# Function calculates ROC-AUC:
library(pROC)
getAUC <- function(prediction, actual) {
auc <- roc(actual, prediction)$auc %>% as.numeric()
return(auc)
}
# AUC on Test data by Artificial Neural Network:
auc_ann <- getAUC(prob_keras, y_test_vec)
auc_ann ## [1] 0.8454012AUC = 0.8454012 khá cao theo một số tiêu chuẩn thường thấy để đánh giá chất lượng của mô hình phân loại.
Xgboost
Chúng ta huấn luyện Xgboost mặc định không tinh chỉnh bất cứ tham số nào như sau:
#------------------------------------------
# Train XGBoost with default parameters
#------------------------------------------
library(xgboost)
dtrain <- xgb.DMatrix(data = x_train_tbl, label = y_train_vec)
# Train a default XGBoost:
xgb_default <- xgboost(data = dtrain,
eval_metric = "auc",
objective = "binary:logistic",
verbose = 0,
nround = 30)Rồi đánh giá khả năng phân loại của Xgboost trên test data qua ma trận nhầm lẫn và AUC như đã làm với Keras ANN:
# Use Xgboost for predicting:
pd_xgb <- predict(xgb_default, x_test_tbl)
# COnfusion matrix by Xgboost:
my_cm(pd_xgb, y_test_vec, cutoff = 0.5)## Confusion Matrix and Statistics
##
## Reference
## Prediction No Yes
## No 1388 267
## Yes 145 309
##
## Accuracy : 0.8046
## 95% CI : (0.7871, 0.8214)
## No Information Rate : 0.7269
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.4732
##
## Mcnemar's Test P-Value : 2.503e-09
##
## Sensitivity : 0.5365
## Specificity : 0.9054
## Pos Pred Value : 0.6806
## Neg Pred Value : 0.8387
## Prevalence : 0.2731
## Detection Rate : 0.1465
## Detection Prevalence : 0.2153
## Balanced Accuracy : 0.7209
##
## 'Positive' Class : Yes
## ## [1] 0.8534515Như vậy Xgboost mặc định là tốt hơn Keras KNN vì AUC = 0.8534515 cao hơn 0.8454012. Không những thế các metrics đánh giá model performance trên ma trận nhầm lẫn của Xgboost cũng ngon hơn Keras ANN.
Bayesian Optimization for Turning Keras ANN Hyperparamaters
Phe “yêu thích” Keras chưa chịu thua. Trong một nỗ lực đánh bại Xgboost họ sử dụng Bayesian Optimization để tinh chỉnh và tìm kiếm tham số tối ưu cho Keras KNN. Giả định răng có ba tham số dropout1, dropout2 và learning_rate (trong số vô vàn tham số của Keras KNN) sẽ được tinh chỉnh. Vậy trước hết là hàm mục tiêu (objective function) trả về thước đo cần tối đa hóa:
# Define objective function:
keras_fit <- function(dropout1, dropout2, learning_rate){
model_keras <- keras_model_sequential()
model_keras %>%
layer_dense(units = 16,
kernel_initializer = "uniform",
activation = "relu",
input_shape = n) %>%
layer_dropout(rate = dropout1) %>%
layer_dense(units = 16,
kernel_initializer = "uniform",
activation = "relu") %>%
layer_dropout(rate = dropout2) %>%
layer_dense(units = 1,
kernel_initializer = "uniform",
activation = "sigmoid") %>%
compile(optimizer = optimizer_adamax(lr = learning_rate),
loss = "binary_crossentropy",
metrics = "accuracy")
fit_keras <- fit(object = model_keras,
x = x_train_tbl,
y = y_train_vec,
verbose = 0,
batch_size = 2^6,
epochs = 30,
validation_split = 0.30)
prob_keras <- predict_proba(model_keras, x_test_tbl) %>% as.vector()
auc <- getAUC(prob_keras, y_test_vec)
result <- list(Score = auc, Pred = NULL)
return(result)
}Rồi thiết lập không gian tìm kiếm và không gian search ban đầu:
# Define domain space:
search_bound_keras <- list(dropout1 = c(0.1, 0.3),
dropout2 = c(0.1, 0.3),
learning_rate = c(0.1, 0.5))
# Define initial search:
set.seed(29)
search_grid_keras <- data.frame(dropout1 = runif(10, 0.1, 0.3),
dropout2 = runif(10, 0.1, 0.3),
learning_rate = runif(10, 0.1, 0.5))Rồi thực hiện tinh chỉnh sử dụng Bayesian Optimization:
# Search optimal hyperparameter by Bayesian Optimization:
library(rBayesianOptimization)
set.seed(29)
system.time(bayes_keras <- BayesianOptimization(keras_fit,
bounds = search_bound_keras,
init_points = 0,
init_grid_dt = search_grid_keras,
n_iter = 10,
kappa = 2.576,
verbose = FALSE,
eps = 0.0,
acq = "ucb"))##
## Best Parameters Found:
## Round = 19 dropout1 = 0.3000 dropout2 = 0.1360 learning_rate = 0.1000 Value = 0.8435## user system elapsed
## 868.931 640.457 220.980Quá trình tinh chỉnh có thể mất nhiều thời gian nên chúng ta sử dụng hàm system.time() để tính luôn thời gian tinh chỉnh. Và kết quả cao nhất của AUC cho Keras ANN là:
## [1] 0.843528Tương ứng với các tham số tối ưu là:
## dropout1 dropout2 learning_rate
## 0.3000000 0.1359839 0.1000000Thật không may là AUC tương ứng với tham số tối ưu tìm được vẫn còn thua AUC của Xgboost mặc định. Và giả định rằng AUC của Keras KNN tinh chỉnh có cao hơn Xgboost mặc định thì “phe Xgboost” họ vẫn có thể tinh chỉnh các tham số của Xgboost.
Cũng cần phải lưu ý rằng huấn luyện Keras ANN mất rất nhiều thời gian, ít nhất là trên CPu. Trên GPU thì có thể đỡ hơn. Một nhược điểm của Keras ANN chính là AUC đặc biệt nhạy cảm với tham số. Thật vậy, AUC có thể nằm đâu đó trong một khoảng rất rộng từ dưới 0.5 đến 0.839 như ta có thể thấy:
# Some statistics about ROC-AUC:
bayes_keras$History %>%
as.data.frame() %>%
pull(Value) %>%
summary()## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.5000 0.7655 0.8250 0.7547 0.8361 0.8435Brief Summary
Không có mô hình nào chiếm ưu thế tuyệt đối. Vay vậy việc xây dựng, đánh giá và lựa chọn mô hình / tool sử dụng hay rộng hơn và cách tiếp cận nó sẽ nên là Right Tool for the Right Job.
About R + Packages and OS
## R version 3.6.2 (2019-12-12)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 18.04.4 LTS
##
## Matrix products: default
## BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
## LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=vi_VN LC_COLLATE=en_US.UTF-8
## [5] LC_MONETARY=vi_VN LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=vi_VN LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=vi_VN LC_IDENTIFICATION=C
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] rBayesianOptimization_1.1.0 xgboost_0.90.0.2
## [3] pROC_1.14.0 keras_2.2.5.0
## [5] caret_6.0-84 lattice_0.20-40
## [7] forcats_0.4.0 stringr_1.4.0
## [9] dplyr_0.8.0.1 purrr_0.3.3
## [11] readr_1.3.1 tidyr_0.8.3
## [13] tibble_2.1.3 ggplot2_3.2.1
## [15] tidyverse_1.2.1
##
## loaded via a namespace (and not attached):
## [1] httr_1.4.0 jsonlite_1.6 splines_3.6.2
## [4] foreach_1.4.4 prodlim_2018.04.18 modelr_0.1.4
## [7] assertthat_0.2.1 stats4_3.6.2 GPfit_1.0-8
## [10] cellranger_1.1.0 yaml_2.2.0 ipred_0.9-9
## [13] pillar_1.4.3 backports_1.1.5 glue_1.3.1
## [16] reticulate_1.14 digest_0.6.23 rvest_0.3.3
## [19] colorspace_1.4-1 recipes_0.1.5 htmltools_0.3.6
## [22] Matrix_1.2-18 plyr_1.8.5 timeDate_3043.102
## [25] pkgconfig_2.0.3 lhs_1.0.1 broom_0.5.2
## [28] haven_2.1.0 scales_1.1.0 whisker_0.3-2
## [31] gower_0.2.0 lava_1.6.5 generics_0.0.2
## [34] farver_2.0.1 withr_2.1.2 nnet_7.3-12
## [37] lazyeval_0.2.2 cli_2.0.0 survival_3.1-8
## [40] magrittr_1.5 crayon_1.3.4 readxl_1.3.1
## [43] evaluate_0.13 fansi_0.4.0 nlme_3.1-144
## [46] MASS_7.3-51.5 xml2_1.2.0 class_7.3-15
## [49] tools_3.6.2 data.table_1.12.2 hms_0.4.2
## [52] lifecycle_0.1.0 munsell_0.5.0 e1071_1.7-1
## [55] compiler_3.6.2 rlang_0.4.2 grid_3.6.2
## [58] iterators_1.0.10 rstudioapi_0.10 rappdirs_0.3.1
## [61] base64enc_0.1-3 labeling_0.3 rmarkdown_1.12
## [64] gtable_0.3.0 ModelMetrics_1.2.2 codetools_0.2-16
## [67] reshape2_1.4.3 R6_2.4.1 tfruns_1.4
## [70] lubridate_1.7.4 knitr_1.22 tensorflow_2.0.0
## [73] zeallot_0.1.0 stringi_1.4.3 Rcpp_1.0.3
## [76] rpart_4.1-15 tidyselect_0.2.5 xfun_0.6LS0tCnRpdGxlOiAnUHJlZGljdGluZyBFbXBsb3llZSBDaHVybjogQSBTaG9ydCBDb21wYXJpc2lvbiBiZXR3ZWVuIFhnYm9vc3QgYW5kIEtlcmFzIERlZXAgTGVhcm5pbmcnCmF1dGhvcjogJ0F1dGhvcjogTmd1eWVuIENoaSBEdW5nJwpzdWJ0aXRsZTogIlIgTWFjaGluZSBMZWFybmluZyBTZXJpZXMiCm91dHB1dDoKICBodG1sX2RvY3VtZW50OiAKICAgIGNvZGVfZG93bmxvYWQ6IHRydWUKICAgICNjb2RlX2ZvbGRpbmc6IGhpZGUKICAgIGhpZ2hsaWdodDogemVuYnVybgogICAgIyBudW1iZXJfc2VjdGlvbnM6IHllcwogICAgdGhlbWU6ICJmbGF0bHkiCiAgICB0b2M6IFRSVUUKICAgIHRvY19mbG9hdDogVFJVRQotLS0KCmBgYHtyIHNldHVwLGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSwgd2FybmluZyA9IEZBTFNFLCBtZXNzYWdlID0gRkFMU0UsIGZpZy53aWR0aCA9IDEwLCBmaWcuaGVpZ2h0ID0gNikKYGBgCgohW10oL2hvbWUvY2hpZHVuZy9Eb2N1bWVudHMvdG9wLmpwZykKCgojIFhnYm9vc3QgdnMuIEtlcmFzIERlZXAgTGVhcm5pbmcKClTDsiBtw7IgdMOsbSBoaeG7g3UgY29pIG5o4buvbmcgdGh14bqtdCB0b8OhbiBuw6BvIHRoxrDhu51uZyDEkcaw4bujYyBz4butIGThu6VuZyBi4bufaSBjw6FjIMSR4buZaSB0aHXhu5ljIFRvcCA1IHRyw6puIHPDom4gY2jGoWkgS2FnZ2xlIHbhu4Ega2hvYSBo4buNYyBk4buvIGxp4buHdSB0aMOsIHbhuqVwIHBo4bqjaSBj4bulbSB04burICoqUHJpbWFyeSBNTCBzb2Z0d2FyZSB1c2VkIGJ5IHRvcC01IHRlYW1zIG9uIEthZ2dsZSoqIHbhu5tpIGPDoWkgaMOsbmggbmjGsCB0csOqbi4gxJDhuqFpIMO9IGzDoCBLZXJhcyBEZWVwIExlYXJuaW5nIHjhur9wIMSR4bqndSBi4bqjbmcsIExpZ2h0R0JNIHbDoCBYZ2Jvb3N0IGzhuqduIGzGsOG7o3QgeOG6v3AgduG7iyB0cuG7iyB0aOG7qSBoYWkgdsOgIGJhLiBUdXkgduG6rXkgcXVhbiDEkWnhu4NtIGPhu6dhIG3DrG5oIGzDoCAqKlVzaW5nIHRoZSBSaWdodCBUb29sIGZvciB0aGUgUmlnaHQgSm9iKio6IGtow7RuZyBjw7MgbeG7mXQgbcO0IGjDrG5oIG7DoG8gY2hp4bq/bSDGsHUgdGjhur8sIHbhu5tpIG3hu5dpIGLhu5kgZOG7ryBsaeG7h3UsIG3hu5l0IHbhuqVuIMSR4buBIHbDoCBob8OgbiBj4bqjbmgga2jDoWMgbmhhdSB0aMOsIG3hu5l0IG3DtCBow6xuaCDEkcaw4bujYyBjaG8gbMOgICJ54bq/dSIgbOG6oWkgY8OzIHRo4buDIGhp4buHdSBxdeG6oyBoxqFuLiAKCgpD4bulIHRo4buDIGjGoW4gY2jDum5nIHRhIHjDqXQgbeG7mXQgYsOgaSB0b8OhbiBy4bqldCBj4bulIHRo4buDOiBiw6BpIHRvw6FuIEJpbmFyeSBDbGFzc2lmaWNhdGlvbiB24bubaSBi4buZIGThu68gbGnhu4d1IFtJQk0gV2F0c29uIFRlbGNvIEN1c3RvbWVyIENodXJuIERhdGEgU2V0XShodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vdHJlc2VsbGUtc3lzdGVtcy9jdXN0b21lcl9jaHVybl9hbmFseXNpcy9tYXN0ZXIvV0FfRm4tVXNlQ18tVGVsY28tQ3VzdG9tZXItQ2h1cm4uY3N2KS4gTeG7pWMgdGnDqnUgY+G7p2EgYsOgaSB0b8OhbiBsw6AgeMOieSBk4buxbmcgbcO0IGjDrG5oIGThu7EgYsOhbyBuaMOibiB2acOqbiBz4bq9IHLhu51pIGLhu48gY8O0bmcgdGkgdsOgIHRpw6p1IGNodeG6qW4gbOG7sWEgY2jhu41uIG3DtCBow6xuaCB04buRdCwgZ2nhuqMgc+G7rSwgbMOgIFJPQy1BVUMuIENow7puZyB0YSAgc+G6vSBzbyBzw6FuaCBraOG6oyBuxINuZyBk4buxIGLDoW8gY+G7p2EgQXJ0aWZpY2lhbCBOZXVyYWwgTmV0d29yayB2w6AgWGdib29zdCBjxINuIGPhu6kgdsOgbyB0acOqdSBjaMOtIG7DoHkuIAoKCiMgRGF0YSBQcmVwcm9jZXNzaW5nIAoKVHLGsOG7m2MgaOG6v3QgY2jDum5nIHRhIHRo4buxYyBoaeG7h24gdmnhu4djIHRp4buBbiB44butIGzDrSBz4buRIGxp4buHdSAtIGNodeG6qW4gYuG7iyBz4buRIGxp4buHdSBuaMawIMSRw6MgdHLDrG5oIGLDoHkgdHJvbmcgW3Bvc3QgdHLGsOG7m2NdKGh0dHBzOi8vcnB1YnMuY29tL2NoaWR1bmdrdC81NzczMzQpOiAKCgpgYGB7cn0KIyBMb2FkIGRhdGE6IApsaWJyYXJ5KHRpZHl2ZXJzZSkKY2h1cm5fZGF0YV9yYXcgPC0gcmVhZF9jc3YoIldBX0ZuLVVzZUNfLVRlbGNvLUN1c3RvbWVyLUNodXJuLmNzdiIpCgojIFJlbW92ZSB1bm5lY2Vzc2FyeSBkYXRhOiAKCmNodXJuX2RhdGFfdGJsIDwtIGNodXJuX2RhdGFfcmF3ICU+JQogIHNlbGVjdCgtY3VzdG9tZXJJRCkgJT4lCiAgZHJvcF9uYSgpCgojIENvbmR1Y3Qgb25lLWhvdCBlbmNvZGluZyBwcm9jZXNzIChodHRwczovL3JkcnIuaW8vcmZvcmdlL2NhcmV0L21hbi9kdW1teVZhcnMuaHRtbCk6IAoKbGlicmFyeShjYXJldCkgIApkdW1taWVzIDwtIGR1bW15VmFycygifiAuIiwgZGF0YSA9IGNodXJuX2RhdGFfdGJsICU+JSBzZWxlY3QoLUNodXJuKSkKcHJlZGljdChkdW1taWVzLCBjaHVybl9kYXRhX3RibCAlPiUgc2VsZWN0KC1DaHVybikpICU+JSBhcy5kYXRhLmZyYW1lKCkgLT4gZGZfZmVhdHVyZXMKCiMgTm9ybWFsaXplIDAtMSBmb3IgZGF0YTogCgpkZl9mZWF0dXJlcyAlPiUgCiAgbXV0YXRlX2FsbChmdW5jdGlvbih4KSB7KHggLSBtaW4oeCkpIC8gKG1heCh4KSAtIG1pbih4KSl9KSAlPiUgCiAgbXV0YXRlKENodXJuID0gY2FzZV93aGVuKGNodXJuX2RhdGFfdGJsJENodXJuID09ICJZZXMiIH4gMSwgVFJVRSB+IDApKSAtPiBkZl9maW5hbAoKCiMgU3BsaXQgZGF0YSAodHJhaW4gZGF0YSBmb3IgdGVzdGluZyBhbmQgdGVzdCBkYXRhIGZvciBldmFsdWF0aW9uL2NvbXBhcmluZyk6IAoKc2V0LnNlZWQoMSkKaWQgPC0gY3JlYXRlRGF0YVBhcnRpdGlvbihkZl9maW5hbCRDaHVybiwgcCA9IDAuNywgbGlzdCA9IEZBTFNFKQp0cmFpbiA8LSBkZl9maW5hbFtpZCwgXQp0ZXN0IDwtIGRmX2ZpbmFsWy1pZCwgXQoKCnhfdHJhaW5fdGJsIDwtIHRyYWluICU+JSAKICBzZWxlY3QoLUNodXJuKSAlPiUgCiAgYXMubWF0cml4KCkKCm4gPC0gbmNvbCh4X3RyYWluX3RibCkKCnlfdHJhaW5fdmVjIDwtIHRyYWluICU+JSBwdWxsKENodXJuKQoKeF90ZXN0X3RibCA8LSB0ZXN0ICU+JSAKICBzZWxlY3QoLUNodXJuKSAlPiUgCiAgYXMubWF0cml4KCkKCnlfdGVzdF92ZWMgPC0gdGVzdCAlPiUgcHVsbChDaHVybikKCmBgYAoKIyBBcnRpZmljaWFsIE5ldXJhbCBOZXR3b3JrCgpUcsaw4bubYyBo4bq/dCBjaMO6bmcgdGEgeMOieSBk4buxbmcgbeG7mXQgQXJ0aWZpY2lhbCBOZXVyYWwgTmV0d29yayAoQU5OKSBoYWkgbGF5ZXJzIHRyb25nIMSRw7MgdOG7iSBs4buHIGRyb3Atb3V0IGzhuqduIGzGsOG7o3QgbMOgIDIwJSwgMjAlIHbhu5tpIGxlYXJuaW5nIHJhdGUgPSAwLjE6ICAKCgpgYGB7cn0KIy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCiMgICBUcmFpbiBBcnRpZmljaWFsIE5ldXJhbCBOZXR3b3JrIAojICAgd2l0aCBzb21lIHBhcmFtZXRlcnMgc2VsZWN0ZWQKIy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgojIEJ1aWxkaW5nIG91ciBBcnRpZmljaWFsIE5ldXJhbCBOZXR3b3JrOiAKCmxpYnJhcnkoa2VyYXMpCgptb2RlbF9rZXJhcyA8LSBrZXJhc19tb2RlbF9zZXF1ZW50aWFsKCkKCm1vZGVsX2tlcmFzICU+JSAKICAjICBGaXJzdCBoaWRkZW4gbGF5ZXI6IAogIGxheWVyX2RlbnNlKHVuaXRzID0gMTYsIAogICAgICAgICAgICAgIGtlcm5lbF9pbml0aWFsaXplciA9ICJ1bmlmb3JtIiwgCiAgICAgICAgICAgICAgYWN0aXZhdGlvbiA9ICJyZWx1IiwgCiAgICAgICAgICAgICAgaW5wdXRfc2hhcGUgPSBuKSAlPiUgCiAgIyBEcm9wb3V0IHJhdGUgdG8gcHJldmVudCBvdmVyZml0dGluZzogCiAgbGF5ZXJfZHJvcG91dChyYXRlID0gMC4yLCBzZWVkID0gMjkpICU+JQogICMgU2Vjb25kIGhpZGRlbiBsYXllcjogCiAgbGF5ZXJfZGVuc2UodW5pdHMgPSAxNiwKICAgICAgICAgICAgICBrZXJuZWxfaW5pdGlhbGl6ZXIgPSAidW5pZm9ybSIsIAogICAgICAgICAgICAgIGFjdGl2YXRpb24gPSAicmVsdSIpICU+JSAKICAjIERyb3BvdXQgcmF0ZSB0byBwcmV2ZW50IG92ZXJmaXR0aW5nIGFnYWluOiAKICBsYXllcl9kcm9wb3V0KHJhdGUgPSAwLjIsIHNlZWQgPSAyOSkgJT4lCiAgIyBPdXRwdXQgbGF5ZXI6IAogIGxheWVyX2RlbnNlKHVuaXRzID0gMSwgCiAgICAgICAgICAgICAga2VybmVsX2luaXRpYWxpemVyID0gInVuaWZvcm0iLCAKICAgICAgICAgICAgICBhY3RpdmF0aW9uID0gInNpZ21vaWQiKSAlPiUgCiAgIyBDb21waWxlIEFOTjogCiAgY29tcGlsZShvcHRpbWl6ZXIgPSBvcHRpbWl6ZXJfYWRhbWF4KGxyID0gMC4xKSwgCiAgICAgICAgICBsb3NzID0gImJpbmFyeV9jcm9zc2VudHJvcHkiLAogICAgICAgICAgbWV0cmljcyA9ICJhY2N1cmFjeSIpCgpgYGAKCgpS4buTaSB0aOG7sWMgaGnhu4duIGh14bqlbiBsdXnhu4duIEFOTiB0csOqbiB0cmFpbiBkYXRhOiAKCgpgYGB7cn0KIyBUcmFpbiB0aGUgQU5OIGtlcmFzIG1vZGVsIHRvIHRoZSB0cmFpbmluZyBkYXRhOiAKZml0X2tlcmFzIDwtIGZpdChvYmplY3QgPSBtb2RlbF9rZXJhcywgCiAgICAgICAgICAgICAgICAgeCA9IHhfdHJhaW5fdGJsLCAKICAgICAgICAgICAgICAgICB5ID0geV90cmFpbl92ZWMsCiAgICAgICAgICAgICAgICAgYmF0Y2hfc2l6ZSA9IDJeNiwgCiAgICAgICAgICAgICAgICAgZXBvY2hzID0gMzAsCiAgICAgICAgICAgICAgICAgdmFsaWRhdGlvbl9zcGxpdCA9IDAuMzApCgpgYGAKCkNow7puZyB0YSBjw7MgdGjhu4MgeGVtIHF1w6EgdHLDrG5oIGh14bqlbiBsdXnhu4duIEFOTiBi4bqxbmcgY8O0bmcgY+G7pSBow6xuaCDhuqNuaCBuaMawIHNhdTogCgpgYGB7cn0KIyBUcmFpbmluZyBwcm9jZXNzIGJ5IHBsb3Q6IAoKcGxvdChmaXRfa2VyYXMpICsKICBsYWJzKHRpdGxlID0gIkZpZ3VyZSAxOiBEZWVwIExlYXJuaW5nIFRyYWluaW5nIFJlc3VsdHMiLCAKICAgICAgIHN1YnRpdGxlID0gIkRhdGEgU291cmNlOiBJQk0gV2F0c29uIFRlbGNvIEN1c3RvbWVyIENodXJuIERhdGEgU2V0IikKCmBgYAoKT0suIFbhu5tpIEFOTiDEkcOjIGPDsyBjaMO6bmcgdGEgY8OzIHRo4buDIMSRw6FuaCBnacOhIHPGoSBi4buZIGNo4bqldCBsxrDhu6NuZyBk4buxIGLDoW8gY+G7p2EgbsOzIHRyw6puIHRlc3QgZGF0YS4gVHLGsOG7m2MgaOG6v3QgbMOgIG1hIHRy4bqtbiBuaOG6p20gbOG6q24gduG7m2kgbmfGsOG7oW5nIG3hurdjIMSR4buLbmggMC41OiAKCgpgYGB7cn0KIyBQcmVkaWN0ZWQgUHJvYmFiaWxpdHk6IApwcm9iX2tlcmFzICA8LSBwcmVkaWN0X3Byb2JhKG1vZGVsX2tlcmFzLCB4X3Rlc3RfdGJsKSAlPiUgYXMudmVjdG9yKCkKCiMgRnVuY3Rpb24gY2FsY3VsYXRlcyBjb25mdXNpb24gbWF0cml4OiAKCm15X2NtIDwtIGZ1bmN0aW9uKHByZWRpY3Rpb24sIGFjdHVhbCwgY3V0b2ZmKSB7CiAgcHJlZCA8LSBjYXNlX3doZW4ocHJlZGljdGlvbiA+PSBjdXRvZmYgfiAiWWVzIiwgVFJVRSB+ICJObyIpICU+JSBhcy5mYWN0b3IoKQogIHRodWNfdGUgPC0gY2FzZV93aGVuKGFjdHVhbCA9PSAxIH4gIlllcyIsIFRSVUUgfiAiTm8iKSAlPiUgYXMuZmFjdG9yKCkKICBjb25mdXNpb25NYXRyaXgocHJlZCwgdGh1Y190ZSwgcG9zaXRpdmUgPSAiWWVzIikKfQoKCiMgQ29uZnVzaW9uIG1hdHJpeCB3aXRoIGRlZmF1bHQgY3V0b2ZmID0gMC41OiAKbXlfY20ocHJvYl9rZXJhcywgeV90ZXN0X3ZlYywgY3V0b2ZmID0gMC41KQoKYGBgCgpWw6AgUk9DLUFVQyAtIHRpw6p1IGNodeG6qW4gxJHGsOG7o2MgY2jhu41uIMSR4buDIMSRw6FuaCBnacOhIGNo4bqldCBsxrDhu6NuZyBk4buxIGLDoW8gY+G7p2EgbcO0IGjDrG5oIGzDoDogIAoKYGBge3J9CgojIEZ1bmN0aW9uIGNhbGN1bGF0ZXMgUk9DLUFVQzogCgpsaWJyYXJ5KHBST0MpCgpnZXRBVUMgPC0gZnVuY3Rpb24ocHJlZGljdGlvbiwgYWN0dWFsKSB7CiAgYXVjIDwtIHJvYyhhY3R1YWwsIHByZWRpY3Rpb24pJGF1YyAlPiUgYXMubnVtZXJpYygpCiAgcmV0dXJuKGF1YykKfQoKIyBBVUMgb24gVGVzdCBkYXRhIGJ5IEFydGlmaWNpYWwgTmV1cmFsIE5ldHdvcms6IAphdWNfYW5uIDwtIGdldEFVQyhwcm9iX2tlcmFzLCB5X3Rlc3RfdmVjKQphdWNfYW5uIAoKYGBgCgpBVUMgPSBgciBhdWNfYW5uYCBraMOhIGNhbyB0aGVvIG3hu5l0IHPhu5EgdGnDqnUgY2h14bqpbiB0aMaw4budbmcgdGjhuqV5IMSR4buDIMSRw6FuaCBnacOhIGNo4bqldCBsxrDhu6NuZyBj4bunYSBtw7QgaMOsbmggcGjDom4gbG/huqFpLiAKCiMgWGdib29zdCAKCkNow7puZyB0YSBodeG6pW4gbHV54buHbiBYZ2Jvb3N0IG3hurdjIMSR4buLbmgga2jDtG5nIHRpbmggY2jhu4luaCBi4bqldCBj4bupIHRoYW0gc+G7kSBuw6BvIG5oxrAgc2F1OiAKCmBgYHtyfQoKIy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQojICAgVHJhaW4gWEdCb29zdCB3aXRoIGRlZmF1bHQgcGFyYW1ldGVycwojLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgpsaWJyYXJ5KHhnYm9vc3QpCmR0cmFpbiA8LSB4Z2IuRE1hdHJpeChkYXRhID0geF90cmFpbl90YmwsIGxhYmVsID0geV90cmFpbl92ZWMpCgojIFRyYWluIGEgZGVmYXVsdCBYR0Jvb3N0OiAKCnhnYl9kZWZhdWx0IDwtIHhnYm9vc3QoZGF0YSA9IGR0cmFpbiwgCiAgICAgICAgICAgICAgICAgICAgICAgZXZhbF9tZXRyaWMgPSAiYXVjIiwgCiAgICAgICAgICAgICAgICAgICAgICAgb2JqZWN0aXZlID0gImJpbmFyeTpsb2dpc3RpYyIsIAogICAgICAgICAgICAgICAgICAgICAgIHZlcmJvc2UgPSAwLCAKICAgICAgICAgICAgICAgICAgICAgICBucm91bmQgPSAzMCkKCmBgYAoKUuG7k2kgxJHDoW5oIGdpw6Ega2jhuqMgbsSDbmcgcGjDom4gbG/huqFpIGPhu6dhIFhnYm9vc3QgdHLDqm4gdGVzdCBkYXRhIHF1YSBtYSB0cuG6rW4gbmjhuqdtIGzhuqtuIHbDoCBBVUMgbmjGsCDEkcOjIGzDoG0gduG7m2kgS2VyYXMgQU5OOiAKCmBgYHtyfQojIFVzZSBYZ2Jvb3N0IGZvciBwcmVkaWN0aW5nOiAKcGRfeGdiIDwtIHByZWRpY3QoeGdiX2RlZmF1bHQsIHhfdGVzdF90YmwpCgojIENPbmZ1c2lvbiBtYXRyaXggYnkgWGdib29zdDogCm15X2NtKHBkX3hnYiwgeV90ZXN0X3ZlYywgY3V0b2ZmID0gMC41KQoKIyBST0MtQVVDIGJ5IFhnYm9vc3Q6IAoKYXVjX3hnYiA8LSBnZXRBVUMocGRfeGdiLCB5X3Rlc3RfdmVjKQphdWNfeGdiIApgYGAKCk5oxrAgduG6rXkgWGdib29zdCBt4bq3YyDEkeG7i25oIGzDoCB04buRdCBoxqFuIEtlcmFzIEtOTiB2w6wgQVVDID0gYHIgYXVjX3hnYmAgY2FvIGjGoW4gYHIgYXVjX2FubmAuIEtow7RuZyBuaOG7r25nIHRo4bq/IGPDoWMgbWV0cmljcyDEkcOhbmggZ2nDoSBtb2RlbCBwZXJmb3JtYW5jZSB0csOqbiBtYSB0cuG6rW4gbmjhuqdtIGzhuqtuIGPhu6dhIFhnYm9vc3QgY8WpbmcgbmdvbiBoxqFuIEtlcmFzIEFOTi4gCgoKIyBCYXllc2lhbiBPcHRpbWl6YXRpb24gZm9yIFR1cm5pbmcgS2VyYXMgQU5OIEh5cGVycGFyYW1hdGVycwoKUGhlICJ5w6p1IHRow61jaCIgS2VyYXMgY2jGsGEgY2jhu4t1IHRodWEuIFRyb25nIG3hu5l0IG7hu5cgbOG7sWMgxJHDoW5oIGLhuqFpIFhnYm9vc3QgaOG7jSBz4butIGThu6VuZyBCYXllc2lhbiBPcHRpbWl6YXRpb24gxJHhu4MgdGluaCBjaOG7iW5oIHbDoCB0w6xtIGtp4bq/bSB0aGFtIHPhu5EgdOG7kWkgxrB1IGNobyBLZXJhcyBLTk4uIEdp4bqjIMSR4buLbmggcsSDbmcgY8OzIGJhIHRoYW0gc+G7kSAqKmRyb3BvdXQxKiosICoqZHJvcG91dDIqKiB2w6AgKipsZWFybmluZ19yYXRlKiogKHRyb25nIHPhu5EgdsO0IHbDoG4gdGhhbSBz4buRIGPhu6dhIEtlcmFzIEtOTikgc+G6vSDEkcaw4bujYyB0aW5oIGNo4buJbmguIFbhuq15IHRyxrDhu5tjIGjhur90IGzDoCBow6BtIG3hu6VjIHRpw6p1IChvYmplY3RpdmUgZnVuY3Rpb24pIHRy4bqjIHbhu4EgdGjGsOG7m2MgxJFvIGPhuqduIHThu5FpIMSRYSBow7NhOiAKCgpgYGB7cn0KIyBEZWZpbmUgb2JqZWN0aXZlIGZ1bmN0aW9uOiAKCmtlcmFzX2ZpdCA8LSBmdW5jdGlvbihkcm9wb3V0MSwgZHJvcG91dDIsIGxlYXJuaW5nX3JhdGUpewogIAogIG1vZGVsX2tlcmFzIDwtIGtlcmFzX21vZGVsX3NlcXVlbnRpYWwoKQogIAogIG1vZGVsX2tlcmFzICU+JSAKICAgIGxheWVyX2RlbnNlKHVuaXRzID0gMTYsIAogICAgICAgICAgICAgICAga2VybmVsX2luaXRpYWxpemVyID0gInVuaWZvcm0iLCAKICAgICAgICAgICAgICAgIGFjdGl2YXRpb24gPSAicmVsdSIsIAogICAgICAgICAgICAgICAgaW5wdXRfc2hhcGUgPSBuKSAlPiUgCiAgICBsYXllcl9kcm9wb3V0KHJhdGUgPSBkcm9wb3V0MSkgJT4lCiAgICBsYXllcl9kZW5zZSh1bml0cyA9IDE2LCAKICAgICAgICAgICAgICAgIGtlcm5lbF9pbml0aWFsaXplciA9ICJ1bmlmb3JtIiwgCiAgICAgICAgICAgICAgICBhY3RpdmF0aW9uID0gInJlbHUiKSAlPiUgCiAgICBsYXllcl9kcm9wb3V0KHJhdGUgPSBkcm9wb3V0MikgJT4lCiAgICBsYXllcl9kZW5zZSh1bml0cyA9IDEsCiAgICAgICAgICAgICAgICBrZXJuZWxfaW5pdGlhbGl6ZXIgPSAidW5pZm9ybSIsIAogICAgICAgICAgICAgICAgYWN0aXZhdGlvbiA9ICJzaWdtb2lkIikgJT4lIAogICAgY29tcGlsZShvcHRpbWl6ZXIgPSBvcHRpbWl6ZXJfYWRhbWF4KGxyID0gbGVhcm5pbmdfcmF0ZSksIAogICAgICAgICAgICBsb3NzID0gImJpbmFyeV9jcm9zc2VudHJvcHkiLCAKICAgICAgICAgICAgbWV0cmljcyA9ICJhY2N1cmFjeSIpCiAgCiAgZml0X2tlcmFzIDwtIGZpdChvYmplY3QgPSBtb2RlbF9rZXJhcywgCiAgICAgICAgICAgICAgICAgICB4ID0geF90cmFpbl90YmwsCiAgICAgICAgICAgICAgICAgICB5ID0geV90cmFpbl92ZWMsCiAgICAgICAgICAgICAgICAgICB2ZXJib3NlID0gMCwgCiAgICAgICAgICAgICAgICAgICBiYXRjaF9zaXplID0gMl42LCAKICAgICAgICAgICAgICAgICAgIGVwb2NocyA9IDMwLAogICAgICAgICAgICAgICAgICAgdmFsaWRhdGlvbl9zcGxpdCA9IDAuMzApCiAgCiAgcHJvYl9rZXJhcyAgPC0gcHJlZGljdF9wcm9iYShtb2RlbF9rZXJhcywgeF90ZXN0X3RibCkgJT4lIGFzLnZlY3RvcigpCiAgCiAgYXVjIDwtIGdldEFVQyhwcm9iX2tlcmFzLCB5X3Rlc3RfdmVjKQogIAogIHJlc3VsdCA8LSBsaXN0KFNjb3JlID0gYXVjLCBQcmVkID0gTlVMTCkKICAKICByZXR1cm4ocmVzdWx0KQp9CgpgYGAKClLhu5NpIHRoaeG6v3QgbOG6rXAga2jDtG5nIGdpYW4gdMOsbSBraeG6v20gdsOgIGtow7RuZyBnaWFuIHNlYXJjaCBiYW4gxJHhuqd1OiAKYGBge3J9CiMgRGVmaW5lIGRvbWFpbiBzcGFjZTogCgpzZWFyY2hfYm91bmRfa2VyYXMgPC0gbGlzdChkcm9wb3V0MSA9IGMoMC4xLCAwLjMpLAogICAgICAgICAgICAgICAgICAgICAgICAgICBkcm9wb3V0MiA9IGMoMC4xLCAwLjMpLAogICAgICAgICAgICAgICAgICAgICAgICAgICBsZWFybmluZ19yYXRlID0gYygwLjEsIDAuNSkpCgojIERlZmluZSBpbml0aWFsIHNlYXJjaDogCgpzZXQuc2VlZCgyOSkKc2VhcmNoX2dyaWRfa2VyYXMgPC0gZGF0YS5mcmFtZShkcm9wb3V0MSA9IHJ1bmlmKDEwLCAwLjEsIDAuMyksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZHJvcG91dDIgPSBydW5pZigxMCwgMC4xLCAwLjMpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxlYXJuaW5nX3JhdGUgPSBydW5pZigxMCwgMC4xLCAwLjUpKQoKCmBgYAoKUuG7k2kgdGjhu7FjIGhp4buHbiB0aW5oIGNo4buJbmggc+G7rSBk4bulbmcgQmF5ZXNpYW4gT3B0aW1pemF0aW9uOiAKCmBgYHtyfQojIFNlYXJjaCBvcHRpbWFsIGh5cGVycGFyYW1ldGVyIGJ5IEJheWVzaWFuIE9wdGltaXphdGlvbjogCmxpYnJhcnkockJheWVzaWFuT3B0aW1pemF0aW9uKQoKc2V0LnNlZWQoMjkpCnN5c3RlbS50aW1lKGJheWVzX2tlcmFzIDwtIEJheWVzaWFuT3B0aW1pemF0aW9uKGtlcmFzX2ZpdCwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJvdW5kcyA9IHNlYXJjaF9ib3VuZF9rZXJhcywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGluaXRfcG9pbnRzID0gMCwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGluaXRfZ3JpZF9kdCA9IHNlYXJjaF9ncmlkX2tlcmFzLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2l0ZXIgPSAxMCwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGthcHBhID0gMi41NzYsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB2ZXJib3NlID0gRkFMU0UsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBlcHMgPSAwLjAsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBhY3EgPSAidWNiIikpCgpgYGAKClF1w6EgdHLDrG5oIHRpbmggY2jhu4luaCBjw7MgdGjhu4MgbeG6pXQgbmhp4buBdSB0aOG7nWkgZ2lhbiBuw6puIGNow7puZyB0YSBz4butIGThu6VuZyBow6BtICoqc3lzdGVtLnRpbWUoKSoqIMSR4buDIHTDrW5oIGx1w7RuIHRo4budaSBnaWFuIHRpbmggY2jhu4luaC4gVsOgIGvhur90IHF14bqjIGNhbyBuaOG6pXQgY+G7p2EgQVVDIGNobyBLZXJhcyBBTk4gbMOgOiAKCmBgYHtyfQojIEJlc3QgUk9DLUFVQyBvbiB0ZXN0IGRhdGE6IApiYXllc19rZXJhcyRCZXN0X1ZhbHVlCgpgYGAKClTGsMahbmcg4bupbmcgduG7m2kgY8OhYyB0aGFtIHPhu5EgdOG7kWkgxrB1IGzDoDogCgpgYGB7cn0KIyBPcHRpbWFsIHBhcmFtZXRlcnM6IApiYXllc19rZXJhcyRCZXN0X1BhcgoKYGBgCgpUaOG6rXQga2jDtG5nIG1heSBsw6AgQVVDIHTGsMahbmcg4bupbmcgduG7m2kgdGhhbSBz4buRIHThu5FpIMawdSB0w6xtIMSRxrDhu6NjIHbhuqtuIGPDsm4gdGh1YSBBVUMgY+G7p2EgWGdib29zdCBt4bq3YyDEkeG7i25oLiBWw6AgZ2nhuqMgxJHhu4tuaCBy4bqxbmcgQVVDIGPhu6dhIEtlcmFzIEtOTiB0aW5oIGNo4buJbmggY8OzIGNhbyBoxqFuIFhnYm9vc3QgbeG6t2MgxJHhu4tuaCB0aMOsICJwaGUgWGdib29zdCIgaOG7jSB24bqrbiBjw7MgdGjhu4MgdGluaCBjaOG7iW5oIGPDoWMgdGhhbSBz4buRIGPhu6dhIFhnYm9vc3QuIAoKQ8WpbmcgY+G6p24gcGjhuqNpIGzGsHUgw70gcuG6sW5nIGh14bqlbiBsdXnhu4duIEtlcmFzIEFOTiBt4bqldCBy4bqldCBuaGnhu4F1IHRo4budaSBnaWFuLCDDrXQgbmjhuqV0IGzDoCB0csOqbiBDUHUuIFRyw6puIEdQVSB0aMOsIGPDsyB0aOG7gyDEkeG7oSBoxqFuLiBN4buZdCBuaMaw4bujYyDEkWnhu4NtIGPhu6dhIEtlcmFzIEFOTiBjaMOtbmggbMOgIEFVQyDEkeG6t2MgYmnhu4d0IG5o4bqheSBj4bqjbSB24bubaSB0aGFtIHPhu5EuIFRo4bqtdCB24bqteSwgQVVDIGPDsyB0aOG7gyBu4bqxbSDEkcOidSDEkcOzIHRyb25nIG3hu5l0IGtob+G6o25nIHLhuqV0IHLhu5luZyB04burIGTGsOG7m2kgMC41IMSR4bq/biAwLjgzOSBuaMawIHRhIGPDsyB0aOG7gyB0aOG6pXk6IAoKCmBgYHtyfQoKIyBTb21lIHN0YXRpc3RpY3MgYWJvdXQgUk9DLUFVQzogCmJheWVzX2tlcmFzJEhpc3RvcnkgJT4lIAogIGFzLmRhdGEuZnJhbWUoKSAlPiUgCiAgcHVsbChWYWx1ZSkgJT4lIAogIHN1bW1hcnkoKQoKYGBgCgojIEJyaWVmIFN1bW1hcnkKCktow7RuZyBjw7MgbcO0IGjDrG5oIG7DoG8gY2hp4bq/bSDGsHUgdGjhur8gdHV54buHdCDEkeG7kWkuIFZheSB24bqteSB2aeG7h2MgeMOieSBk4buxbmcsIMSRw6FuaCBnacOhIHbDoCBs4buxYSBjaOG7jW4gbcO0IGjDrG5oIC8gdG9vbCBz4butIGThu6VuZyBoYXkgcuG7mW5nIGjGoW4gdsOgIGPDoWNoIHRp4bq/cCBj4bqtbiBuw7Mgc+G6vSBuw6puIGzDoCAqKlJpZ2h0IFRvb2wgZm9yIHRoZSBSaWdodCBKb2IqKi4gCgojIEFib3V0IFIgKyBQYWNrYWdlcyBhbmQgT1MKCmBgYHtyfQpzZXNzaW9uSW5mbygpCmBgYAoKCg==