1 Introduction, Research Goal & Objective

3 Dataset Preparation

3.1 R Libraries Used

Here are the R libraries used in this analysis.

library(knitr)      # web widget
library(tidyverse)  # data manipulation
library(data.table) # fast file reading
library(treemap)    # tree visualization
library(caret)      # rocr analysis
library(ROCR)       # rocr analysis
library(kableExtra) # nice table html formating 
library(gridExtra)  # arranging ggplot in grid
#library(corrgram)   # correlation graphics

3.2 Import Datasets

setwd('./datasets')
aisles      = fread('aisles.csv',      stringsAsFactors = TRUE)
departments = fread('departments.csv', stringsAsFactors = TRUE)
products    = fread('products.csv',    stringsAsFactors = TRUE)
orders      = fread('orders.csv',      stringsAsFactors = TRUE)
order_products_train = fread('order_products__train.csv')
order_products_prior = fread('order_products__prior.csv')

3.3 Data Dictionary

The dataset for this competition is a relational set of files describing customers’ orders over time. They are anonymized and contains a sample of over 3 million grocery orders from more than 200,000 Instacart users. For each user, Instacart provided between 4 and 100 of their orders, with the sequence of products purchased in each order, the week and hour of day the order was placed, and a relative measure of time between orders.

Total six datasets were imported. Follwing section will explore each datasets in further detail. These datasets were sourced from an existing Kaggle competiotion (https://www.kaggle.com/c/instacart-market-basket-analysis/data)

orders (3.4m rows, 206k users):

  • order_id: order identifier
  • user_id: customer identifier
  • eval_set: which evaluation set this order belongs in (see SET described below)
  • order_number: the order sequence number for this user (1 = first, n = nth)
  • order_dow: the day of the week the order was placed on
  • order_hour_of_day: the hour of the day the order was placed on
  • days_since_prior: days since the last order, capped at 30 (with NAs for order_number = 1)

products (50k rows):

  • product_id: product identifier
  • product_name: name of the product
  • aisle_id: foreign key
  • department_id: foreign key

aisles (134 rows):

  • aisle_id: aisle identifier
  • aisle: the name of the aisle

deptartments (21 rows):

  • department_id: department identifier
  • department: the name of the department

order_products__SET (30m+ rows):

  • order_id: foreign key
  • product_id: foreign key
  • add_to_cart_order: order in which each product was added to cart
  • reordered: 1 if this product has been ordered by this user in the past, 0 otherwise

where SET is one of the four following evaluation sets (eval_set in orders):

  • "prior": orders prior to that users most recent order (~3.2m orders)
  • "train": training data supplied to participants (~131k orders)
  • "test": test data reserved for machine learning competitions (~75k orders)

3.4 Understanding Datasets

3.4.1 Aisles

There are 134 ailes in this dataset. Here are few sample names of the ailes.

paste(sort(head(aisles$aisle)), collapse=', ')
[1] "energy granola bars, instant foods, marinades meat preparation, other, prepared soups salads, specialty cheeses"

3.4.2 Departments

There are 21 departments in this dataset.Names of all deparments are listed below in aphabetically ordered.

paste(sort(departments$department), collapse = ', ')
[1] "alcohol, babies, bakery, beverages, breakfast, bulk, canned goods, dairy eggs, deli, dry goods pasta, frozen, household, international, meat seafood, missing, other, pantry, personal care, pets, produce, snacks"

3.4.3 Products

There are 49,688 products in the catalogue within 134 aisles and 21 departments. Sample products are as below.

products %>% head %>% kable
product_id product_name aisle_id department_id
1 Chocolate Sandwich Cookies 61 19
2 All-Seasons Salt 104 13
3 Robust Golden Unsweetened Oolong Tea 94 7
4 Smart Ones Classic Favorites Mini Rigatoni With Vodka Cream Sauce 38 1
5 Green Chile Anytime Sauce 5 13
6 Dry Nose Oil 11 11

3.4.4 Departments And Its Relevant Products

Products dataframe is related to Deparments. We shall see below a glimpse of 3 products for each deparment, giving an idea the how the products being departmentized. Only sample five departments are shown.

left_join(departments, products) %>% select(department, product_name ) %>%
  group_by(department) %>%
  sample_n(3) %>%
  summarise(three_examples_product=paste(product_name, collapse=' || ')) %>% sample_n(5) %>% kable
department three_examples_product
pets Small Bone || Grain Free Naturals Dry Dog Food Chicken & Garden Pea Recipe || T-Bonz Porterhouse Flavor Steak-Shaped Dog Treats
deli All Natural Prosciutto Vacuum Pack || Butternut Squash Tamales With Cheese || Mediterranean Orzo Salad
bakery Naan || Gluten Free Blueberry Muffins || Artisan Thin Pizza Crust Rustic White Flatbread
other Natural Calm Anti-Stress Drink || Organic Vitamin D3 400 IU Drops For Babies & Infants Natural Berry Flavor || Yogurt Mint Sauce
breakfast Flavor Variety Instant Oatmeal || Cinnamon Chex || Light and Fluffy Blueberry Pancake Mix

3.4.5 Aisles And Its Relevant Products

Products dataframe is also related to aisles. Each aisle relates to multiple prodcuts. By joining both aisles and products dataframe, we have an idea what type of prodcuts for each ailes. Example below shows sample of aisles and few of its related products.

left_join(aisles, products) %>%
  select(aisle, product_name ) %>%  group_by(aisle) %>%
  sample_n(3) %>% 
  summarise(three_examples_product=paste(product_name, collapse=' || ')) %>% sample_n(5) %>% kable
aisle three_examples_product
poultry counter Whole Chicken Fryer || Uncured Boneless Ham || Fresh Chicken Wings
other creams cheeses X Sharp Cheddar Cheese || Cottage Cheese, Natural Small Curd, 4% Milkfat Min || Cottage Cheese Large Curd
energy granola bars Organic Berry Berry Granola Bars || Pomegranate Blueberry Pistachio Plus Antioxidants 1.4 oz Fruit & Nut Bars || Organic Z Fruit Grape Rope
instant foods Creamy Deluxe Gluten Free Rice Pasta & Extra Cheesy Cheddar Sauce || Slow Cookers Chili Seasoning Mix || Macaroni Shells & White Cheddar Cheese
yogurt YoKids Squeeze Organic Blueberry Blue Yogurt || Greek 100 Calories Mixed Berry Yogurt || Organic Strawberry Coconut Milk Yogurt

3.4.6 Orders

There are over 3 millions observations in orders dataset. Each row represent an unique order. Each variable potentially can be used as predictors. Let’s analyse the construct of one user. For example, user_id 1 had made 10 prior orders (order number from 1 to 10), last order is a train (eval_set).

orders %>% filter (user_id==1) %>% kable
order_id user_id eval_set order_number order_dow order_hour_of_day days_since_prior_order
2539329 1 prior 1 2 8 NA
2398795 1 prior 2 3 7 15
473747 1 prior 3 3 12 21
2254736 1 prior 4 4 7 29
431534 1 prior 5 4 15 28
3367565 1 prior 6 2 7 19
550135 1 prior 7 1 9 20
3108588 1 prior 8 1 14 14
2295261 1 prior 9 1 16 0
2550362 1 prior 10 4 8 30
1187899 1 train 11 4 8 14

Let’s analyse another construct of orders. User_id 3 had made 12 orders before the final order labeled as test (eval_set) order. From the data we know that order_number is being recycled for each user. Instacart did not provide us the basket content for test order. This is in fact the target for prediction.

This also means <user_id, product_id> made up the key for prediction.

orders %>% filter (user_id==3) %>% kable
order_id user_id eval_set order_number order_dow order_hour_of_day days_since_prior_order
1374495 3 prior 1 1 14 NA
444309 3 prior 2 3 19 9
3002854 3 prior 3 3 16 21
2037211 3 prior 4 2 18 20
2710558 3 prior 5 0 17 12
1972919 3 prior 6 0 16 7
1839752 3 prior 7 0 15 7
3225766 3 prior 8 0 17 7
3160850 3 prior 9 0 16 7
676467 3 prior 10 3 16 17
521107 3 prior 11 0 18 11
1402502 3 prior 12 1 15 15
2774568 3 test 13 5 15 11

3.4.7 Order_Product_Train/Prior

order_product_train/prior dataframe tells us which products were purchased in each order, for both train order and prior order. For example, we can know user_id 1 in the last order (order_id 1187899) purchased 10 unique products by quering order_product_train with the relevant order_id.

order_products_train %>% filter (order_id == 1187899) %>% kable
order_id product_id add_to_cart_order reordered
1187899 196 1 1
1187899 25133 2 1
1187899 38928 3 1
1187899 26405 4 1
1187899 39657 5 1
1187899 10258 6 1
1187899 13032 7 1
1187899 26088 8 1
1187899 27845 9 0
1187899 49235 10 1
1187899 46149 11 1

3.4.8 Users

Ther is no dedicated dataframe for users. However, we can derive number of unique users from order dataframe. By grouping the user_id and eval_set column, we found that there are 75,000 test users, 131,209 train users.

We shall use train users for training, and test users for submission to Kaggle.We want to keep this large number of train records in mind for trimming during quick initial model build and verification.

table(orders$eval_set) %>% kable(col.names = c('eval_set','Frequency')) %>%   kable_styling(bootstrap_options = c("striped", "hover"), position = "float_right")
eval_set Frequency
prior 3214874
test 75000
train 131209

4 Exploratory Data Analysis

In this section, we shall try to understand the buying behaviour by asking some interesting quesitons.

4.1 Orders_Products

4.1.4 Products Ordered Day Pattern

We can see that both Day 0 and Day 1 stands out to be the most busy shopping day for instacart. This means that day of order made may influence the basket size.

order_products_prior %>%
  left_join(orders) %>%
  group_by(order_dow) %>%
  summarize(count = n()) %>%
  mutate(percentage=count/sum(count)) %>%
  ggplot (aes(x=as.factor(order_dow), y=percentage)) + 
    geom_col()+ ylab('Percentage of Orders') + ggtitle('Daily Orders')

Top ten products ordered daily contributes between 7% to 8%. It is interesting to see that Limes are part of top ten for Day 0 and Day 6, but not other days. Wherease Organic Whole Milk doesn’t make it to top ten for Day 0. Organic Respberries does not make it to top 10 of Day 6. This means that there is a chance of predictability based on the day order is made.

order_products_prior %>% 
  left_join(orders) %>% left_join(products) %>%
  group_by(order_dow, product_name) %>%
  summarize(n=n()) %>%
  mutate(percentage=n/sum(n)) %>%
  top_n(10, wt=n) %>%
  ggplot (aes(x=as.factor(order_dow), y=percentage, fill=product_name)) + 
    geom_col() + ylab('Proprtion of Orders') + ggtitle('Daily Top 10 Products Ordered') +
    theme(legend.position="bottom",legend.direction="horizontal")

4.1.5 Products Ordered Hour Pattern

Morning to afternoon are the peak shopping hours for instacart customers. The hour order made influences basket size.

order_products_prior %>%
  left_join(orders) %>%
  group_by(order_hour_of_day) %>%
  summarize(count = n()) %>%
  mutate(percentage=count/sum(count)) %>%
  ggplot (aes(x=as.factor(order_hour_of_day), y=percentage)) + 
    geom_col()+ ylab('Percentage of Orders') + ggtitle('Hourly Orders')

In the grocery, there are close to 50,000 products. When we zoom into hourly purchases, we noticed that top 10 products managed to score betwen 6% to 8% of hourly sales. Every hour has slightly diffrent combination of top 10 products (combination out of 12 products). That means certain products are predictable for ordering irregardless of the hour of order.

order_products_prior %>% 
  left_join(orders) %>% left_join(products) %>%
  group_by(order_hour_of_day, product_name) %>%
  summarize(n=n()) %>%
  mutate(percentage=n/sum(n)) %>%
  top_n(10, wt=n) %>%
  ggplot (aes(x=as.factor(order_hour_of_day), y=percentage, fill=product_name)) + 
    geom_col() + ylab('Proprtion of Orders In A Hour') + ggtitle('Hourly Top 10 Products Ordered') +
    theme(legend.position="bottom",legend.direction="horizontal")
Joining, by = "order_id"
Joining, by = "product_id"

4.2 Reordering Analysis

5 Predictive Analysis

5.1 Type of Prediction

Predict what product will the customer purchase in the next basket means estimation of probably if each product being purchased before will be purchased again. This is a classification problem, as well as a regression of probability of repurchases. The output of the prediction is to be evaluated against training eval_set. with below table columns:

  • <user_id product_id>: as key
  • label : 1 or 0 , this is the target label

For submission to kagggle, output from above table should be transformed into the form below:

  • user_id
  • <list of product id seperated by space>

First, let’s create a training label, which is the value or either 1 or 0 to indicate the actual basket content. All prediction will be evaluated against this training label.

## Training Label
train.label = orders %>% 
  filter(eval_set=='train') %>% 
  left_join(order_products_train) %>%
  left_join(products) %>%
  mutate(actual=1) %>%   #this is training label
  select(user_id, order_id, product_id, product_name, actual)

Let’s see an example of training label for user_id 1. As this is training eval_set, the single order_id shows that it is the final order. We shall use this as the label (actual) for training.

train.label %>% filter(user_id==1) %>% kable %>%  kable_styling(bootstrap_options = c("striped"))
user_id order_id product_id product_name actual
1 1187899 196 Soda 1
1 1187899 25133 Organic String Cheese 1
1 1187899 38928 0% Greek Strained Yogurt 1
1 1187899 26405 XL Pick-A-Size Paper Towel Rolls 1
1 1187899 39657 Milk Chocolate Almonds 1
1 1187899 10258 Pistachios 1
1 1187899 13032 Cinnamon Toast Crunch 1
1 1187899 26088 Aged White Cheddar Popcorn 1
1 1187899 27845 Organic Whole Milk 1
1 1187899 49235 Organic Half & Half 1
1 1187899 46149 Zero Calorie Cola 1

5.2 Model Evaluation & Optimization

Instacart has close to 50k products in their catalogue. As the maximum number of items ordered by a user is just a fraction of the 50k available product. This means by simply predicting nothing is purchased in the next basket, we would yeild close to 100% accuracy.

Due to the highly imbalance dataset, Instacart require F1 Score as the competition scoring, instead of accuracy.

To evaluate the performance of the model, we had created a custom function to build a confusion matrix and derive other binary classification metrics.

## Custom Function For Binary Class Performance Evaluation
binclass_eval = function (actual, predict) {
  cm = table(as.integer(actual), as.integer(predict), dnn=c('Actual','Predicted'))
  ac = (cm['1','1']+cm['0','0'])/(cm['0','1'] + cm['1','0'] + cm['1','1'] + cm['0','0'])
  pr = cm['1','1']/(cm['0','1'] + cm['1','1'])
  rc = cm['1','1']/(cm['1','0'] + cm['1','1'])
  fs = 2* pr*rc/(pr+rc)
  list(cm=cm, recall=rc, precision=pr, fscore=fs, accuracy=ac)
}

If the prediction is based on probability, we shall build a function to discover cutoff that optimize various performance metrics.

### Cutoff Threshold Optimization
optimize_cutoff = function (actual, probability) {
  rocr.pred = prediction(predictions = probability, labels = actual)
  rocr.metrics = data.frame(
      cutoff   = rocr.pred@cutoffs[[1]],
      accuracy = (rocr.pred@tp[[1]] + rocr.pred@tn[[1]]) / 
                   (rocr.pred@tp[[1]] + rocr.pred@tn[[1]] + rocr.pred@fp[[1]] + rocr.pred@fn[[1]]),
      tpr = rocr.pred@tp[[1]] / (rocr.pred@tp[[1]] + rocr.pred@fn[[1]]),
      fpr = rocr.pred@fp[[1]] / (rocr.pred@fp[[1]] + rocr.pred@tn[[1]]),
      ppv = rocr.pred@tp[[1]] / (rocr.pred@tp[[1]] + rocr.pred@fp[[1]])
  )
  rocr.metrics$fscore = 2 * (rocr.metrics$tpr * rocr.metrics$ppv) / (rocr.metrics$tpr + rocr.metrics$ppv)
  rocr.metrics$tpr_fpr = rocr.metrics$tpr / rocr.metrics$fpr
  
  ## Discovery the optimal threshold for various metrics
  rocr.best = rbind(
    best.accuracy = c(max = max(rocr.metrics$accuracy, na.rm = TRUE), cutoff=rocr.metrics$cutoff[which.max(rocr.metrics$accuracy)]),
    best.ppv = c(max = max(rocr.metrics$ppv, na.rm = TRUE), cutoff = rocr.metrics$cutoff[which.max(rocr.metrics$ppv)]),
    best.recall = c(max = max(rocr.metrics$tpr, na.rm = TRUE), cutoff = rocr.metrics$cutoff[which.max(rocr.metrics$tpr)]),
    best.fscore = c(max = max(rocr.metrics$fscore, na.rm = TRUE), cutoff = rocr.metrics$cutoff[which.max(rocr.metrics$fscore)]),
    best.tpr_fpr = c(max = max(rocr.metrics$tpr_fpr, na.rm = TRUE), cutoff = rocr.metrics$cutoff[which.max(rocr.metrics$tpr_fpr)])
  )
  
  list(metrics = rocr.metrics, best = rocr.best)
}

5.3 Model 1 : Naive Prediction

5.3.1 Build The Model

We can simply predict the basket based on user last order.

m1.predict = orders %>% 
  filter(eval_set=='prior') %>% 
  group_by(user_id) %>%
  top_n(n=1, wt=order_number) %>%       #last order has the higher order_number
  left_join(order_products_prior) %>%
  mutate (
    predicted=1) %>%              #predict based on last ordered, therefore 1
  select(user_id, product_id, predicted) %>% 
  full_join(train.label) %>%  # join with train.label for items not predicted but in train.label
  select(user_id, product_id, actual, predicted) %>%
  replace(., is.na(.), 0) %>% # treat not predicted as 0
  arrange(user_id, product_id)

5.3.2 Sample Output

m2.predict %>% filter(user_id ==1) %>% kable %>% kable_styling(bootstrap_options = c("striped"))
user_id product_id actual rate
1 196 1 1.0
1 10258 1 0.9
1 10326 0 0.1
1 12427 0 1.0
1 13032 1 0.3
1 13176 0 0.2
1 14084 0 0.1
1 17122 0 0.1
1 25133 1 0.8
1 26088 1 0.2
1 26405 1 0.2
1 27845 1 0.0
1 30450 0 0.1
1 35951 0 0.1
1 38928 1 0.1
1 39657 1 0.1
1 41787 0 0.1
1 46149 1 0.3
1 49235 1 0.2

5.3.3 Confusion Matrix

m1.eval = binclass_eval(m1.predict$actual, m1.predict$predicted)
m1.eval$cm
      Predicted
Actual       0       1
     0       0 1760406
     1 1005235  379382

5.3.4 Model Performance

The result shows only 0.2153 F1 Score.

cat("Accuracy:  ", m1.eval$accuracy,
    "\nPrecision: ", m1.eval$precision,
    "\nRecall:    ", m1.eval$recall,
    "\nFScore:    ", m1.eval$fscore)
Accuracy:   0.1206293 
Precision:  0.1772989 
Recall:     0.2739978 
FScore:     0.2152885

5.4 Model 2 : Smarter Naive Prediction (Baseline)

In this model, we predict products in the basket by estimating their frequency of repurchased. This way we get a ratio to indicate probability of re-purchases. We use ROCR package to predict the best cutoff point (at which above this cutoff we shall predict for re-order) that give us the optimum F1 score.

5.4.1 Build The Model

## Build Model
m2.predict = orders %>% 
  filter(eval_set=='prior') %>% 
  group_by(user_id) %>%
  mutate(total_orders = max(order_number)) %>%  # total number of orders made previously
  ungroup %>% select(user_id, order_id, total_orders) %>%
  left_join(order_products_prior) %>%
  group_by(user_id, product_id) %>%
  summarize(rate=n()/max(total_orders)) %>%
  select(user_id, product_id, rate) %>%
  full_join(train.label) %>%  # join with train.label for items not predicted but in train.label
  select(user_id, product_id, actual, rate) %>%
  replace(., is.na(.), 0) %>% # treat not predicted as 0
  arrange(user_id, product_id)

5.4.2 Sample Output

m2.predict %>% filter(user_id==1) %>% kable %>% kable_styling(bootstrap_options = c("striped"))
user_id product_id actual rate
1 196 1 1.0
1 10258 1 0.9
1 10326 0 0.1
1 12427 0 1.0
1 13032 1 0.3
1 13176 0 0.2
1 14084 0 0.1
1 17122 0 0.1
1 25133 1 0.8
1 26088 1 0.2
1 26405 1 0.2
1 27845 1 0.0
1 30450 0 0.1
1 35951 0 0.1
1 38928 1 0.1
1 39657 1 0.1
1 41787 0 0.1
1 46149 1 0.3
1 49235 1 0.2

5.4.3 Optimize Cutoff

We see that in order to maximize F1 Score, we need to set the cutoff threshold to 0.3368, which is the next step.

max cutoff
best.accuracy 0.9001268 Inf
best.ppv 0.4345635 0.8588235
best.recall 1.0000000 0.0000000
best.fscore 0.2312383 0.3368421
best.tpr_fpr 6.9266631 0.8588235

5.4.4 Confusion Matrix

Let’s set the cutoff to 0.3368 as discovered in previous step.

m2.eval = binclass_eval(m2.predict$actual, m2.predict$rate>0.3368)
m2.eval$cm
      Predicted
Actual        0        1
     0 11548605   930524
     1  1081948   302669

5.4.5 Model Performance

We are getting slightly better F1 Score (0.2312) compare to previous model. We shall use this as the baseline.

cat("Accuracy:  ", m2.eval$accuracy,
    "\nPrecision: ", m2.eval$precision,
    "\nRecall:    ", m2.eval$recall,
    "\nFScore:    ", m2.eval$fscore)
Accuracy:   0.8548392 
Precision:  0.2454352 
Recall:     0.218594 
FScore:     0.2312383

5.5 Model 3 : Machine Learning Classification

We construct all the products that users had purchased in the last 3 orders, then use machine learning classification to predict will each of the product be purchased again. We shall use decision tree and logistic regression for this prediction.

#m3.predict = orders %>% 
#  filter(eval_set=='prior') %>% 
#  left_join(order_products_prior) %>%
#  group_by(user_id, product_id) %>%
#  summarize(rate=n()/max(total_orders))

5.5.1 Feature Engineering

5.5.1.1 User Features

We create three features applied to individual users.

users
- avg_user_order_dow: Average of order_dow
- avg_user_order_hod: Average of order_hour_of_day
- avg_user_items: Average items for each order

users.train.avg_items = orders %>% 
  filter(eval_set=='train') %>%
  select(user_id) %>%
  left_join(orders) %>%
  left_join(order_products_prior) %>%
  count(user_id, order_id) %>%
  group_by(user_id) %>%
  summarize(avg_user_items = mean(n))
users.train.avg_dowhod = orders %>% 
  filter(eval_set=='train') %>%
  select(user_id) %>%
  left_join(orders) %>%
  group_by(user_id) %>%
  summarize( 
    avg_user_order_dow = mean(order_dow),
    avg_user_order_hod = mean(order_hour_of_day))
users.train = users.train.avg_items %>% left_join(users.train.avg_dowhod)
  
users.test.avg_items = orders %>% 
  filter(eval_set=='test') %>%
  select(user_id) %>%
  left_join(orders) %>%
  left_join(order_products_prior) %>%
  count(user_id, order_id) %>%
  group_by(user_id) %>%
  summarize(avg_user_items = mean(n))
users.test.avg_dowhod = orders %>% 
  filter(eval_set=='test') %>%
  select(user_id) %>%
  left_join(orders) %>%
  group_by(user_id) %>%
  summarize( 
    avg_user_order_dow = mean(order_dow),
    avg_user_order_hod = mean(order_hour_of_day))
users.test = users.test.avg_items %>% left_join(users.test.avg_dowhod)
rm(list=c('users.test.avg_items','users.test.avg_dowhod','users.train.avg_items','users.train.avg_dowhod'))

Training Users

user_id avg_user_items avg_user_order_dow avg_user_order_hod
1 5.454546 2.636364 10.09091
2 13.066667 2.066667 10.60000
5 7.600000 1.400000 15.00000
7 9.857143 1.857143 13.47619
8 12.500000 3.250000 5.50000
9 19.250000 3.000000 13.25000

Test Users

head(users.test) %>% kable %>% kable_styling(bootstrap_options = c("striped"))
user_id avg_user_items avg_user_order_dow avg_user_order_hod
3 6.846154 1.384615 16.30769
4 3.166667 4.500000 12.50000
6 3.750000 3.500000 17.00000
11 11.875000 4.500000 10.87500
12 12.500000 2.666667 11.16667
15 3.173913 2.347826 11.26087

5.5.1.2 Product Features

We create two features generally applied across all products.

products

  • avg_product_order_dow: Average of product order_dow
  • avg_product_order_hod: Average of product order_hour_of_day
products_ = orders %>% 
  left_join(order_products_prior) %>%
  group_by(product_id) %>%
  summarize( 
    avg_product_order_dow = mean(order_dow),
    avg_product_order_hod = mean(order_hour_of_day)
  ) %>%
  left_join(products)
head(products_) %>% kable %>% kable_styling(bootstrap_options = c("striped"))
product_id avg_product_order_dow avg_product_order_hod product_name aisle_id department_id
1 2.776458 13.23812 Chocolate Sandwich Cookies 61 19
2 2.922222 13.27778 All-Seasons Salt 104 13
3 2.736462 12.10469 Robust Golden Unsweetened Oolong Tea 94 7
4 2.683891 13.71429 Smart Ones Classic Favorites Mini Rigatoni With Vodka Cream Sauce 38 1
5 2.733333 10.66667 Green Chile Anytime Sauce 5 13
6 4.250000 14.12500 Dry Nose Oil 11 11

5.5.2 Construct Training Data with Label

m3.train.label = train.label %>% 
  left_join(users.train) %>%   #combine with new user features
  left_join(products_)          #combine with new product features
Joining, by = "user_id"
Joining, by = c("product_id", "product_name")
head(m3.train.label) %>% kable %>% kable_styling(bootstrap_options = c("striped"))
user_id order_id product_id product_name actual avg_user_items avg_user_order_dow avg_user_order_hod avg_product_order_dow avg_product_order_hod aisle_id department_id
1 1187899 196 Soda 1 5.454546 2.636364 10.09091 2.898550 12.52396 77 7
1 1187899 25133 Organic String Cheese 1 5.454546 2.636364 10.09091 2.650581 12.97482 21 16
1 1187899 38928 0% Greek Strained Yogurt 1 5.454546 2.636364 10.09091 2.728207 12.28471 120 16
1 1187899 26405 XL Pick-A-Size Paper Towel Rolls 1 5.454546 2.636364 10.09091 2.765239 12.48682 54 17
1 1187899 39657 Milk Chocolate Almonds 1 5.454546 2.636364 10.09091 2.707312 12.27296 45 19
1 1187899 10258 Pistachios 1 5.454546 2.636364 10.09091 2.719425 12.25591 117 19

5.6 Selection of Model

5.7 Data Recoding

5.8 Prediction

6 Analysis & Recommendations

7 Citations

  1. The Instacart Online Grocery Shopping Dataset 2017“, Accessed from https://www.instacart.com/datasets/grocery-shopping-2017
  2. Data Dictionary: https://gist.github.com/jeremystan/c3b39d947d9b88b3ccff3147dbcf6c6b
LS0tDQp0aXRsZTogIkluc3RhY2FydCBNYXJrZXQgQmFza2V0IEFuYWx5c2lzIg0KYXV0aG9yOiAiWW9uZyBLZWggU29vbiINCmRhdGU6ICIxOCBOb3ZlbWJlciAyMDE4Ig0Kb3V0cHV0Og0KICBodG1sX25vdGVib29rOg0KICAgICNjb2RlX2ZvbGRpbmc6IGhpZGUNCiAgICBudW1iZXJfc2VjdGlvbnM6IHllcw0KICAgIHRoZW1lOiBjZXJ1bGVhbg0KICAgIHRvYzogeWVzDQogICAgdG9jX2RlcHRoOiA0DQogIGh0bWxfZG9jdW1lbnQ6DQogICAgZGZfcHJpbnQ6IHBhZ2VkDQogICAgdG9jOiB5ZXMNCiAgICB0b2NfZGVwdGg6ICc0Jw0KLS0tDQoNCg0KPHN0eWxlIHR5cGU9InRleHQvY3NzIj4NCg0KYm9keXsgLyogTm9ybWFsICAqLw0KICAgICAgZm9udC1zaXplOiAxMnB4Ow0KICB9DQp0ZCB7ICAvKiBUYWJsZSAgKi8NCiAgZm9udC1zaXplOiAxMnB4Ow0KfQ0KaDEudGl0bGUgew0KICBmb250LXNpemU6IDM4cHg7DQogIGNvbG9yOiBEYXJrUmVkOw0KfQ0KaDEgeyAvKiBIZWFkZXIgMSAqLw0KICBmb250LXNpemU6IDI0cHg7DQogIGNvbG9yOiBEYXJrQmx1ZTsNCn0NCmgyIHsgLyogSGVhZGVyIDIgKi8NCiAgZm9udC1zaXplOiAyMHB4Ow0KICBjb2xvcjogRGFya0JsdWU7DQp9DQpoMyB7IC8qIEhlYWRlciAzICovDQogIGZvbnQtc2l6ZTogMTZweDsNCiMgIGZvbnQtZmFtaWx5OiAiVGltZXMgTmV3IFJvbWFuIiwgVGltZXMsIHNlcmlmOw0KICBjb2xvcjogRGFya0JsdWU7DQp9DQpoNCB7IC8qIEhlYWRlciA0ICovDQogIGZvbnQtc2l6ZTogMTRweDsNCiAgY29sb3I6IERhcmtCbHVlOw0KfQ0KY29kZS5yeyAvKiBDb2RlIGJsb2NrICovDQogICAgZm9udC1zaXplOiAxMnB4Ow0KfQ0KcHJlIHsgLyogQ29kZSBibG9jayAtIGRldGVybWluZXMgY29kZSBzcGFjaW5nIGJldHdlZW4gbGluZXMgKi8NCiAgICBmb250LXNpemU6IDEycHg7DQp9DQo8L3N0eWxlPg0KDQoNCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQ0KI2tuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSkNCmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvPVRSVUUsIGZpZy5oZWlnaHQ9My41LCBmaWcud2lkdGg9OS4yLCByZXN1bHRzPSdob2xkJywgd2FybmluZz1GQUxTRSwgZmlnLnNob3c9J2hvbGQnLCBtZXNzYWdlPUZBTFNFKSANCmBgYA0KDQojIEludHJvZHVjdGlvbiwgUmVzZWFyY2ggR29hbCAmIE9iamVjdGl2ZQ0KDQoNCiMgUmVsYXRlZCB3b3Jrcw0KDQoNCiMgRGF0YXNldCBQcmVwYXJhdGlvbg0KDQojIyBSIExpYnJhcmllcyBVc2VkDQoNCkhlcmUgYXJlIHRoZSBSIGxpYnJhcmllcyB1c2VkIGluIHRoaXMgYW5hbHlzaXMuIA0KDQpgYGB7ciwgbWVzc2FnZT1GQUxTRX0NCmxpYnJhcnkoa25pdHIpICAgICAgIyB3ZWIgd2lkZ2V0DQpsaWJyYXJ5KHRpZHl2ZXJzZSkgICMgZGF0YSBtYW5pcHVsYXRpb24NCmxpYnJhcnkoZGF0YS50YWJsZSkgIyBmYXN0IGZpbGUgcmVhZGluZw0KbGlicmFyeSh0cmVlbWFwKSAgICAjIHRyZWUgdmlzdWFsaXphdGlvbg0KbGlicmFyeShjYXJldCkgICAgICAjIHJvY3IgYW5hbHlzaXMNCmxpYnJhcnkoUk9DUikgICAgICAgIyByb2NyIGFuYWx5c2lzDQpsaWJyYXJ5KGthYmxlRXh0cmEpICMgbmljZSB0YWJsZSBodG1sIGZvcm1hdGluZyANCmxpYnJhcnkoZ3JpZEV4dHJhKSAgIyBhcnJhbmdpbmcgZ2dwbG90IGluIGdyaWQNCiNsaWJyYXJ5KGNvcnJncmFtKSAgICMgY29ycmVsYXRpb24gZ3JhcGhpY3MNCmBgYA0KDQojIyBJbXBvcnQgRGF0YXNldHMNCg0KYGBge3IsIHJlc3VsdHM9J2hpZGUnLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQ0Kc2V0d2QoJy4vZGF0YXNldHMnKQ0KYWlzbGVzICAgICAgPSBmcmVhZCgnYWlzbGVzLmNzdicsICAgICAgc3RyaW5nc0FzRmFjdG9ycyA9IFRSVUUpDQpkZXBhcnRtZW50cyA9IGZyZWFkKCdkZXBhcnRtZW50cy5jc3YnLCBzdHJpbmdzQXNGYWN0b3JzID0gVFJVRSkNCnByb2R1Y3RzICAgID0gZnJlYWQoJ3Byb2R1Y3RzLmNzdicsICAgIHN0cmluZ3NBc0ZhY3RvcnMgPSBUUlVFKQ0Kb3JkZXJzICAgICAgPSBmcmVhZCgnb3JkZXJzLmNzdicsICAgICAgc3RyaW5nc0FzRmFjdG9ycyA9IFRSVUUpDQpvcmRlcl9wcm9kdWN0c190cmFpbiA9IGZyZWFkKCdvcmRlcl9wcm9kdWN0c19fdHJhaW4uY3N2JykNCm9yZGVyX3Byb2R1Y3RzX3ByaW9yID0gZnJlYWQoJ29yZGVyX3Byb2R1Y3RzX19wcmlvci5jc3YnKQ0KYGBgDQoNCiMjIERhdGEgRGljdGlvbmFyeQ0KDQpUaGUgZGF0YXNldCBmb3IgdGhpcyBjb21wZXRpdGlvbiBpcyBhIHJlbGF0aW9uYWwgc2V0IG9mIGZpbGVzIGRlc2NyaWJpbmcgY3VzdG9tZXJzJyBvcmRlcnMgb3ZlciB0aW1lLiBUaGV5IGFyZSBhbm9ueW1pemVkIGFuZCBjb250YWlucyBhIHNhbXBsZSBvZiBvdmVyICoqMyBtaWxsaW9uIGdyb2Nlcnkgb3JkZXJzKiogZnJvbSBtb3JlIHRoYW4gKioyMDAsMDAwIEluc3RhY2FydCB1c2VycyoqLiBGb3IgZWFjaCB1c2VyLCBJbnN0YWNhcnQgcHJvdmlkZWQgKipiZXR3ZWVuIDQgYW5kIDEwMCoqIG9mIHRoZWlyIG9yZGVycywgd2l0aCB0aGUgc2VxdWVuY2Ugb2YgcHJvZHVjdHMgcHVyY2hhc2VkIGluIGVhY2ggb3JkZXIsIHRoZSAqKndlZWsgYW5kIGhvdXIgb2YgZGF5KiogdGhlIG9yZGVyIHdhcyBwbGFjZWQsIGFuZCBhICoqcmVsYXRpdmUgbWVhc3VyZSBvZiB0aW1lIGJldHdlZW4gb3JkZXJzKiouDQoNClRvdGFsIHNpeCBkYXRhc2V0cyB3ZXJlIGltcG9ydGVkLiBGb2xsd2luZyBzZWN0aW9uIHdpbGwgZXhwbG9yZSBlYWNoIGRhdGFzZXRzIGluIGZ1cnRoZXIgZGV0YWlsLiBUaGVzZSBkYXRhc2V0cyB3ZXJlIHNvdXJjZWQgZnJvbSBhbiBleGlzdGluZyBLYWdnbGUgY29tcGV0aW90aW9uIChodHRwczovL3d3dy5rYWdnbGUuY29tL2MvaW5zdGFjYXJ0LW1hcmtldC1iYXNrZXQtYW5hbHlzaXMvZGF0YSkNCg0KYG9yZGVyc2AgKDMuNG0gcm93cywgMjA2ayB1c2Vycyk6ICANCg0KKiBgb3JkZXJfaWRgOiBvcmRlciBpZGVudGlmaWVyICANCiogYHVzZXJfaWRgOiBjdXN0b21lciBpZGVudGlmaWVyICANCiogYGV2YWxfc2V0YDogd2hpY2ggZXZhbHVhdGlvbiBzZXQgdGhpcyBvcmRlciBiZWxvbmdzIGluIChzZWUgYFNFVGAgZGVzY3JpYmVkIGJlbG93KSAgDQoqIGBvcmRlcl9udW1iZXJgOiB0aGUgb3JkZXIgc2VxdWVuY2UgbnVtYmVyIGZvciB0aGlzIHVzZXIgKDEgPSBmaXJzdCwgbiA9IG50aCkgIA0KKiBgb3JkZXJfZG93YDogdGhlIGRheSBvZiB0aGUgd2VlayB0aGUgb3JkZXIgd2FzIHBsYWNlZCBvbiAgDQoqIGBvcmRlcl9ob3VyX29mX2RheWA6IHRoZSBob3VyIG9mIHRoZSBkYXkgdGhlIG9yZGVyIHdhcyBwbGFjZWQgb24gIA0KKiBgZGF5c19zaW5jZV9wcmlvcmA6IGRheXMgc2luY2UgdGhlIGxhc3Qgb3JkZXIsIGNhcHBlZCBhdCAzMCAod2l0aCBOQXMgZm9yIGBvcmRlcl9udW1iZXJgID0gMSkgIA0KDQpgcHJvZHVjdHNgICg1MGsgcm93cyk6ICANCg0KKiBgcHJvZHVjdF9pZGA6IHByb2R1Y3QgaWRlbnRpZmllciAgDQoqIGBwcm9kdWN0X25hbWVgOiBuYW1lIG9mIHRoZSBwcm9kdWN0ICANCiogYGFpc2xlX2lkYDogZm9yZWlnbiBrZXkgIA0KKiBgZGVwYXJ0bWVudF9pZGA6IGZvcmVpZ24ga2V5ICANCg0KYGFpc2xlc2AgKDEzNCByb3dzKTogIA0KDQoqIGBhaXNsZV9pZGA6IGFpc2xlIGlkZW50aWZpZXIgIA0KKiBgYWlzbGVgOiB0aGUgbmFtZSBvZiB0aGUgYWlzbGUgIA0KDQpgZGVwdGFydG1lbnRzYCAoMjEgcm93cyk6ICANCg0KKiBgZGVwYXJ0bWVudF9pZGA6IGRlcGFydG1lbnQgaWRlbnRpZmllciAgDQoqIGBkZXBhcnRtZW50YDogdGhlIG5hbWUgb2YgdGhlIGRlcGFydG1lbnQgIA0KDQpgb3JkZXJfcHJvZHVjdHNfX1NFVGAgKDMwbSsgcm93cyk6ICANCg0KKiBgb3JkZXJfaWRgOiBmb3JlaWduIGtleSAgDQoqIGBwcm9kdWN0X2lkYDogZm9yZWlnbiBrZXkgIA0KKiBgYWRkX3RvX2NhcnRfb3JkZXJgOiBvcmRlciBpbiB3aGljaCBlYWNoIHByb2R1Y3Qgd2FzIGFkZGVkIHRvIGNhcnQgIA0KKiBgcmVvcmRlcmVkYDogMSBpZiB0aGlzIHByb2R1Y3QgaGFzIGJlZW4gb3JkZXJlZCBieSB0aGlzIHVzZXIgaW4gdGhlIHBhc3QsIDAgb3RoZXJ3aXNlICANCg0Kd2hlcmUgYFNFVGAgaXMgb25lIG9mIHRoZSBmb3VyIGZvbGxvd2luZyBldmFsdWF0aW9uIHNldHMgKGBldmFsX3NldGAgaW4gYG9yZGVyc2ApOiAgDQoNCiogYCJwcmlvciJgOiBvcmRlcnMgcHJpb3IgdG8gdGhhdCB1c2VycyBtb3N0IHJlY2VudCBvcmRlciAofjMuMm0gb3JkZXJzKSAgDQoqIGAidHJhaW4iYDogdHJhaW5pbmcgZGF0YSBzdXBwbGllZCB0byBwYXJ0aWNpcGFudHMgKH4xMzFrIG9yZGVycykgIA0KKiBgInRlc3QiYDogdGVzdCBkYXRhIHJlc2VydmVkIGZvciBtYWNoaW5lIGxlYXJuaW5nIGNvbXBldGl0aW9ucyAofjc1ayBvcmRlcnMpICANCg0KIyMgVW5kZXJzdGFuZGluZyBEYXRhc2V0cw0KDQojIyMgQWlzbGVzDQoNClRoZXJlIGFyZSAqKjEzNCBhaWxlcyoqIGluIHRoaXMgZGF0YXNldC4gSGVyZSBhcmUgZmV3IHNhbXBsZSBuYW1lcyBvZiB0aGUgYWlsZXMuICANCg0KYGBge3IsIHJlc3VsdHM9J2hvbGQnfQ0KcGFzdGUoc29ydChoZWFkKGFpc2xlcyRhaXNsZSkpLCBjb2xsYXBzZT0nLCAnKQ0KYGBgDQoNCiMjIyBEZXBhcnRtZW50cw0KDQpUaGVyZSBhcmUgKioyMSBkZXBhcnRtZW50cyoqIGluIHRoaXMgZGF0YXNldC5OYW1lcyBvZiBhbGwgZGVwYXJtZW50cyBhcmUgbGlzdGVkIGJlbG93IGluIGFwaGFiZXRpY2FsbHkgb3JkZXJlZC4NCg0KYGBge3IsIHJlc3VsdHM9J2hvbGQnfQ0KcGFzdGUoc29ydChkZXBhcnRtZW50cyRkZXBhcnRtZW50KSwgY29sbGFwc2UgPSAnLCAnKQ0KYGBgDQoNCiMjIyBQcm9kdWN0cw0KDQpUaGVyZSBhcmUgKio0OSw2ODggcHJvZHVjdHMqKiBpbiB0aGUgY2F0YWxvZ3VlIHdpdGhpbiAqKjEzNCBhaXNsZXMgYW5kIDIxIGRlcGFydG1lbnRzKiouIFNhbXBsZSBwcm9kdWN0cyBhcmUgYXMgYmVsb3cuDQoNCmBgYHtyLCByZXN1bHRzPSdob2xkJ30NCnByb2R1Y3RzICU+JSBoZWFkICU+JSBrYWJsZQ0KYGBgDQoNCiMjIyBEZXBhcnRtZW50cyBBbmQgSXRzIFJlbGV2YW50IFByb2R1Y3RzDQoNClByb2R1Y3RzIGRhdGFmcmFtZSBpcyByZWxhdGVkIHRvIERlcGFybWVudHMuIFdlIHNoYWxsIHNlZSBiZWxvdyBhIGdsaW1wc2Ugb2YgMyBwcm9kdWN0cyBmb3IgZWFjaCBkZXBhcm1lbnQsIGdpdmluZyBhbiBpZGVhIHRoZSBob3cgdGhlIHByb2R1Y3RzIGJlaW5nIGRlcGFydG1lbnRpemVkLiBPbmx5IHNhbXBsZSBmaXZlIGRlcGFydG1lbnRzIGFyZSBzaG93bi4NCg0KYGBge3IsIHJlc3VsdHM9J2hvbGQnLCBtZXNzYWdlPUZBTFNFfQ0KbGVmdF9qb2luKGRlcGFydG1lbnRzLCBwcm9kdWN0cykgJT4lIHNlbGVjdChkZXBhcnRtZW50LCBwcm9kdWN0X25hbWUgKSAlPiUNCiAgZ3JvdXBfYnkoZGVwYXJ0bWVudCkgJT4lDQogIHNhbXBsZV9uKDMpICU+JQ0KICBzdW1tYXJpc2UodGhyZWVfZXhhbXBsZXNfcHJvZHVjdD1wYXN0ZShwcm9kdWN0X25hbWUsIGNvbGxhcHNlPScgfHwgJykpICU+JSBzYW1wbGVfbig1KSAlPiUga2FibGUNCg0KYGBgDQoNCiMjIyBBaXNsZXMgQW5kIEl0cyBSZWxldmFudCBQcm9kdWN0cw0KDQpQcm9kdWN0cyBkYXRhZnJhbWUgaXMgYWxzbyByZWxhdGVkIHRvIGFpc2xlcy4gRWFjaCBhaXNsZSByZWxhdGVzIHRvIG11bHRpcGxlIHByb2RjdXRzLiBCeSBqb2luaW5nIGJvdGggYWlzbGVzIGFuZCBwcm9kdWN0cyBkYXRhZnJhbWUsIHdlIGhhdmUgYW4gaWRlYSB3aGF0IHR5cGUgb2YgcHJvZGN1dHMgZm9yIGVhY2ggYWlsZXMuIEV4YW1wbGUgYmVsb3cgc2hvd3Mgc2FtcGxlIG9mIGFpc2xlcyBhbmQgZmV3IG9mIGl0cyByZWxhdGVkIHByb2R1Y3RzLg0KDQpgYGB7ciwgcmVzdWx0cz0naG9sZCcsIG1lc3NhZ2U9RkFMU0V9DQpsZWZ0X2pvaW4oYWlzbGVzLCBwcm9kdWN0cykgJT4lDQogIHNlbGVjdChhaXNsZSwgcHJvZHVjdF9uYW1lICkgJT4lICBncm91cF9ieShhaXNsZSkgJT4lDQogIHNhbXBsZV9uKDMpICU+JSANCiAgc3VtbWFyaXNlKHRocmVlX2V4YW1wbGVzX3Byb2R1Y3Q9cGFzdGUocHJvZHVjdF9uYW1lLCBjb2xsYXBzZT0nIHx8ICcpKSAlPiUgc2FtcGxlX24oNSkgJT4lIGthYmxlDQpgYGANCg0KIyMjIE9yZGVycw0KDQpUaGVyZSBhcmUgb3ZlciAqKjMgbWlsbGlvbnMqKiBvYnNlcnZhdGlvbnMgaW4gb3JkZXJzIGRhdGFzZXQuIEVhY2ggcm93IHJlcHJlc2VudCBhbiAqKnVuaXF1ZSBvcmRlcioqLiAgRWFjaCB2YXJpYWJsZSBwb3RlbnRpYWxseSBjYW4gYmUgdXNlZCBhcyBwcmVkaWN0b3JzLiBMZXQncyBhbmFseXNlIHRoZSBjb25zdHJ1Y3Qgb2Ygb25lIHVzZXIuIEZvciBleGFtcGxlLCAqKnVzZXJfaWQgMSoqIGhhZCBtYWRlICoqMTAgcHJpb3Igb3JkZXJzKiogKG9yZGVyIG51bWJlciBmcm9tIDEgdG8gMTApLCBsYXN0IG9yZGVyIGlzIGEgKip0cmFpbioqIChldmFsX3NldCkuDQoNCmBgYHtyfQ0Kb3JkZXJzICU+JSBmaWx0ZXIgKHVzZXJfaWQ9PTEpICU+JSBrYWJsZQ0KYGBgDQoNCkxldCdzIGFuYWx5c2UgYW5vdGhlciBjb25zdHJ1Y3Qgb2Ygb3JkZXJzLiAqKlVzZXJfaWQgMyoqIGhhZCBtYWRlICoqMTIgb3JkZXJzKiogYmVmb3JlIHRoZSBmaW5hbCBvcmRlciBsYWJlbGVkIGFzICoqdGVzdCoqIChldmFsX3NldCkgb3JkZXIuIEZyb20gdGhlIGRhdGEgd2Uga25vdyB0aGF0IG9yZGVyX251bWJlciBpcyBiZWluZyByZWN5Y2xlZCBmb3IgZWFjaCB1c2VyLiAgSW5zdGFjYXJ0IGRpZCBub3QgcHJvdmlkZSB1cyB0aGUgYmFza2V0IGNvbnRlbnQgZm9yIHRlc3Qgb3JkZXIuIFRoaXMgaXMgaW4gZmFjdCB0aGUgKip0YXJnZXQgZm9yIHByZWRpY3Rpb24qKi4gDQoNClRoaXMgYWxzbyBtZWFucyAqKmA8dXNlcl9pZCwgcHJvZHVjdF9pZD5gKiogbWFkZSB1cCB0aGUgKiprZXkqKiBmb3IgcHJlZGljdGlvbi4gIA0KDQpgYGB7cn0NCm9yZGVycyAlPiUgZmlsdGVyICh1c2VyX2lkPT0zKSAlPiUga2FibGUNCmBgYA0KDQojIyMgT3JkZXJfUHJvZHVjdF9UcmFpbi9Qcmlvcg0KDQoqKm9yZGVyX3Byb2R1Y3RfdHJhaW4vcHJpb3IqKiBkYXRhZnJhbWUgdGVsbHMgdXMgd2hpY2ggcHJvZHVjdHMgd2VyZSBwdXJjaGFzZWQgaW4gZWFjaCBvcmRlciwgZm9yIGJvdGggdHJhaW4gb3JkZXIgYW5kIHByaW9yIG9yZGVyLiBGb3IgZXhhbXBsZSwgd2UgY2FuIGtub3cgKip1c2VyX2lkIDEqKiBpbiB0aGUgbGFzdCBvcmRlciAob3JkZXJfaWQgMTE4Nzg5OSkgcHVyY2hhc2VkICoqMTAgdW5pcXVlIHByb2R1Y3RzKiogYnkgKipxdWVyaW5nIG9yZGVyX3Byb2R1Y3RfdHJhaW4qKiB3aXRoIHRoZSByZWxldmFudCBvcmRlcl9pZC4gIA0KDQpgYGB7cn0NCm9yZGVyX3Byb2R1Y3RzX3RyYWluICU+JSBmaWx0ZXIgKG9yZGVyX2lkID09IDExODc4OTkpICU+JSBrYWJsZQ0KYGBgDQoNCiMjIyBVc2Vycw0KDQpUaGVyIGlzIG5vIGRlZGljYXRlZCBkYXRhZnJhbWUgZm9yIHVzZXJzLiBIb3dldmVyLCB3ZSBjYW4gZGVyaXZlIG51bWJlciBvZiB1bmlxdWUgdXNlcnMgZnJvbSAqKm9yZGVyKiogZGF0YWZyYW1lLiBCeSBncm91cGluZyB0aGUgdXNlcl9pZCBhbmQgZXZhbF9zZXQgY29sdW1uLCB3ZSBmb3VuZCB0aGF0IHRoZXJlIGFyZSAqKjc1LDAwMCB0ZXN0KiogdXNlcnMsICoqMTMxLDIwOSB0cmFpbioqIHVzZXJzLiAgDQoNCldlIHNoYWxsIHVzZSAqKnRyYWluIHVzZXJzKiogZm9yIHRyYWluaW5nLCBhbmQgKip0ZXN0IHVzZXJzKiogZm9yIHN1Ym1pc3Npb24gdG8gS2FnZ2xlLldlIHdhbnQgdG8ga2VlcCB0aGlzIGxhcmdlIG51bWJlciBvZiB0cmFpbiByZWNvcmRzIGluIG1pbmQgZm9yICoqdHJpbW1pbmcqKiBkdXJpbmcgcXVpY2sgaW5pdGlhbCBtb2RlbCBidWlsZCBhbmQgdmVyaWZpY2F0aW9uLg0KDQpgYGB7cn0NCnRhYmxlKG9yZGVycyRldmFsX3NldCkgJT4lIGthYmxlKGNvbC5uYW1lcyA9IGMoJ2V2YWxfc2V0JywnRnJlcXVlbmN5JykpICU+JSAgIA0KICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJob3ZlciIpKQ0KYGBgDQoNCiMgRXhwbG9yYXRvcnkgRGF0YSBBbmFseXNpcw0KDQpJbiB0aGlzIHNlY3Rpb24sIHdlIHNoYWxsIHRyeSB0byB1bmRlcnN0YW5kIHRoZSBidXlpbmcgYmVoYXZpb3VyIGJ5IGFza2luZyBzb21lIGludGVyZXN0aW5nIHF1ZXNpdG9ucy4gIA0KDQotIFdoYXQgdXN1YWxseSBkb2VzIHBlb3BsZSBidXksIGFuZCB3aGljaCBvbmUgdGhleSB1c3VhbGx5IHJlb3JkZXINCi0gV2hlbiBkbyB0aGV5IGJ1eSAoZGF5IGFuZCB0aW1lKT8gSXMgdGhlcmUgYSBidXlpbmcgdHJlbmQgYW5kIGRvZXMgaXQgaW5mbHVlbmNlIHdoYXQgdGhleSBidXkgPyAgDQoNCiMjIE9yZGVyc19Qcm9kdWN0cw0KDQojIyMgTW9zdCBQb3B1bGFyIFByb2R1Y3RzIFNvbGQNCg0KV2Uga25vdyB0aGF0ICoqYmFuYW5hKiogYXJlIHRoZSBtb3N0IHBvcHVsYXIgcHJvZHVjdHMuIFRoZSBudW1iZXIgb2Ygb3JkZXJzIHZhcmllcyBncmVhdGx5IGZvciBkaWZmZXJlbnQgcHJvZHVjdHMuIElsbHVzdHJhdGlvbiBiZWxvdyB1c2VzIHNob3dzIHNhbXBsZSBvZiBvbmx5ICoqMzAgdG9wIHByb2R1Y3RzKiouICBOb3RpY2UgaG93ZXZlciB0aGUgdmFyaWVuY2UgaXMgbm90IG9idmlvdXMgYWZ0ZXIgdG9wIDEwIHByb2R1Y3RzLg0KDQpgYGB7ciwgZmlnLndpZHRoPTkuNSwgZmlnLmhlaWdodD0zLjV9DQp0bXAgPSBvcmRlcl9wcm9kdWN0c190cmFpbiAlPiUNCiAgbGVmdF9qb2luKHByb2R1Y3RzKSAlPiUNCiAgZ3JvdXBfYnkocHJvZHVjdF9uYW1lKSAlPiUNCiAgc3VtbWFyaXplKGNvdW50PW4oKSkgJT4lDQogIHRvcF9uKG49MzAsIHd0PWNvdW50KSAlPiUgIG11dGF0ZShwZXJjZW50YWdlPWNvdW50L3N1bShjb3VudCkpDQoNCnAxID0gZ2dwbG90ICh0bXAsIGFlcyh4PXJlb3JkZXIocHJvZHVjdF9uYW1lLGNvdW50KSwgeT1wZXJjZW50YWdlKSkgKyAgDQogIGdlb21fY29sKCkgKyBnZ3RpdGxlKCdQcm9kdWN0cyBUb3AgMzAnKSArIHlsYWIoJ1BlcmNlbnRhZ2Ugb2YgT3JkZXJzJykgKw0KICB0aGVtZSAoDQogICAgYXhpcy50ZXh0Lng9ZWxlbWVudF90ZXh0KGFuZ2xlPTkwLCBoanVzdD0xLCB2anVzdD0wLjUpLA0KICAgIGF4aXMudGl0bGUueCA9IGVsZW1lbnRfYmxhbmsoKSkgDQoNCnAyID0gZ2dwbG90IChkYXRhID0gdG1wLCBhZXMoIHg9ICcnLCB5PXBlcmNlbnRhZ2UgKSkgKyANCiAgZ2d0aXRsZSgnUHJvZHVjdHMgVG9wIDMwJykgKyB5bGFiKCdwZXJjZW50YWdlLm9mLm9yZGVycycpICsgZ2VvbV9ib3hwbG90KCkgKyB4bGFiKCdQcm9kdWN0cycpDQoNCmdyaWQuYXJyYW5nZShwMSwgcDIsIG5jb2wgPSAyKQ0KYGBgDQpgYGB7ciBlY2hvPUZBTFNFfQ0Kcm0obGlzdD1jKCd0bXAnLCdwMScsJ3AyJykpDQpgYGANCg0KIyMjIE1vc3QgUG9wdWxhciBEZXBhcnRtZW50IFNvbGQNCg0KQ2VydGFpbiBkZXBhcnRtZW5zIGFyZSBjbGVhcmx5IG1vcmUgcG9wdWxhciwgbGlrZSAqKnByb2R1Y2UgYW5kIGRhaXJ5IGVnZ3MqKi4gQm90aCBkZXBhcm1lbnRzIGNvbWJpbmVkIGNvbnRyaWJ1dGVkIHRvICoqbW9yZSB0aGFuIDQwJSoqIG9mIHRvdGFsIG9yZGVycy4NCg0KYGBge3IsIGZpZy53aWR0aD05LjUsIGZpZy5oZWlnaHQ9My41fQ0KdG1wID0gb3JkZXJfcHJvZHVjdHNfdHJhaW4gJT4lDQogIGxlZnRfam9pbihwcm9kdWN0cykgJT4lDQogIGxlZnRfam9pbihkZXBhcnRtZW50cykgJT4lDQogIGdyb3VwX2J5KGRlcGFydG1lbnQpICU+JQ0KICBzdW1tYXJpemUoY291bnQ9bigpKSAlPiUNCiAgbXV0YXRlKHBlcmNlbnRhZ2U9Y291bnQvc3VtKGNvdW50KSkNCg0KcDEgPSBnZ3Bsb3QgKHRtcCwgYWVzKHg9cmVvcmRlcihkZXBhcnRtZW50LGNvdW50KSwgeT1wZXJjZW50YWdlKSkgKyAgDQogIGdlb21fY29sKCkgKyBnZ3RpdGxlKCdEZXBhcnRtZW50cycpICsgeWxhYignUGVyY2VudGFnZSBvZiBPcmRlcnMnKSArDQogIHRoZW1lICgNCiAgICBheGlzLnRleHQueD1lbGVtZW50X3RleHQoYW5nbGU9OTAsIGhqdXN0PTEsIHZqdXN0PTAuNSksDQogICAgYXhpcy50aXRsZS54ID0gZWxlbWVudF9ibGFuaygpKSANCg0KcDIgPSBnZ3Bsb3QgKGRhdGEgPSB0bXAsIGFlcyggeD0gJycsIHk9cGVyY2VudGFnZSApKSArIA0KICBnZ3RpdGxlKCdEZXBhcnRtZW50cycpICsgeWxhYigncGVyY2VudGFnZS5vZi5vcmRlcnMnKSArIGdlb21fYm94cGxvdCgpICsgeGxhYignRGVwYXJ0bWVudHMnKQ0KDQpncmlkLmFycmFuZ2UocDEsIHAyLCBuY29sID0gMikNCmBgYA0KDQpgYGB7ciBlY2hvPUZBTFNFfQ0Kcm0obGlzdD1jKCd0bXAnLCdwMScsJ3AyJykpDQpgYGANCg0KIyMjIE1vc3QgUG9wdWxhciBBaXNsZXMgU29sZA0KDQpXZSBsb29rZWQgaW50byB0aGUgYnV5aW5nIHRyZW5kIG9mIHByb2R1Y3QgYnkgYWlsZXMgYW5kIG5vdGljZSB0aGF0IGNlcnRhaW4gYWlzbGUgbGlrZSAqKnZlZ2V0YWJsZXMgYW5kIGZydWl0cyoqIGNvbnRyaWJ1dGVzIHRvICoqYWxtb3N0IDMwJSoqIG9mIHRvdGFsIG9yZGVycy4gQ2hhcnQgYmVsb3cgc2hvd3MgdG9wIDMwIGFpc2xlcy4NCg0KYGBge3IsIGZpZy53aWR0aD05LjUsIGZpZy5oZWlnaHQ9My41fQ0KdG1wID0gb3JkZXJfcHJvZHVjdHNfdHJhaW4gJT4lDQogIGxlZnRfam9pbihwcm9kdWN0cykgJT4lDQogIGxlZnRfam9pbihhaXNsZXMpICU+JQ0KICBncm91cF9ieShhaXNsZSkgJT4lDQogIHN1bW1hcml6ZShjb3VudD1uKCkpICU+JQ0KICB0b3BfbihuPTMwLCB3dD1jb3VudCkgJT4lICBtdXRhdGUocGVyY2VudGFnZT1jb3VudC9zdW0oY291bnQpKQ0KDQpwMSA9IGdncGxvdCAodG1wLCBhZXMoeD1yZW9yZGVyKGFpc2xlLGNvdW50KSwgeT1wZXJjZW50YWdlKSkgKyAgDQogIGdlb21fY29sKCkgKyBnZ3RpdGxlKCdBaXNsZXMgVG9wIDMwJykgKyB5bGFiKCdQZXJjZW50YWdlIG9mIE9yZGVycycpICsNCiAgdGhlbWUgKA0KICAgIGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT05MCwgaGp1c3Q9MSwgdmp1c3Q9MC41KSwNCiAgICBheGlzLnRpdGxlLnggPSBlbGVtZW50X2JsYW5rKCkpICsgIHlsYWIoJ1BlcmNlbnRhZ2Ugb2YgT3JkZXJzJykgKyB4bGFiKCdBaXNsZXMnKQ0KDQpwMiA9IGdncGxvdCAodG1wLCBhZXMoIHg9ICcnLCB5PXBlcmNlbnRhZ2UgKSkgKyANCiAgZ2d0aXRsZSgnQWlzbGVzIFRvcCAzMCcpICsgeWxhYigncGVyY2VudGFnZS5vZi5vcmRlcnMnKSArIGdlb21fYm94cGxvdCgpICsgeGxhYignQWlzbGVzJykNCg0KZ3JpZC5hcnJhbmdlKHAxLCBwMiwgbmNvbCA9IDIpDQpgYGANCg0KYGBge3IgZWNobz1GQUxTRX0NCnJtKGxpc3Q9YygndG1wJywncDEnLCdwMicpKQ0KYGBgDQoNCiMjIyBQcm9kdWN0cyBPcmRlcmVkIERheSBQYXR0ZXJuDQoNCldlIGNhbiBzZWUgdGhhdCBib3RoICoqRGF5IDAgYW5kIERheSAxKiogc3RhbmRzIG91dCB0byBiZSB0aGUgbW9zdCBidXN5IHNob3BwaW5nIGRheSBmb3IgaW5zdGFjYXJ0LiBUaGlzIG1lYW5zIHRoYXQgZGF5IG9mIG9yZGVyIG1hZGUgbWF5IGluZmx1ZW5jZSB0aGUgYmFza2V0IHNpemUuDQoNCmBgYHtyLGZpZy53aWR0aD05LjMsIGZpZy5oZWlnaHQ9My4wfQ0Kb3JkZXJfcHJvZHVjdHNfcHJpb3IgJT4lDQogIGxlZnRfam9pbihvcmRlcnMpICU+JQ0KICBncm91cF9ieShvcmRlcl9kb3cpICU+JQ0KICBzdW1tYXJpemUoY291bnQgPSBuKCkpICU+JQ0KICBtdXRhdGUocGVyY2VudGFnZT1jb3VudC9zdW0oY291bnQpKSAlPiUNCiAgZ2dwbG90IChhZXMoeD1hcy5mYWN0b3Iob3JkZXJfZG93KSwgeT1wZXJjZW50YWdlKSkgKyANCiAgICBnZW9tX2NvbCgpKyB5bGFiKCdQZXJjZW50YWdlIG9mIE9yZGVycycpICsgZ2d0aXRsZSgnRGFpbHkgT3JkZXJzJykNCmBgYA0KDQpUb3AgdGVuIHByb2R1Y3RzIG9yZGVyZWQgZGFpbHkgY29udHJpYnV0ZXMgYmV0d2VlbiA3JSB0byA4JS4gSXQgaXMgaW50ZXJlc3RpbmcgdG8gc2VlIHRoYXQgTGltZXMgYXJlIHBhcnQgb2YgdG9wIHRlbiBmb3IgRGF5IDAgYW5kIERheSA2LCBidXQgbm90IG90aGVyIGRheXMuIFdoZXJlYXNlIE9yZ2FuaWMgV2hvbGUgTWlsayBkb2Vzbid0IG1ha2UgaXQgdG8gdG9wIHRlbiBmb3IgRGF5IDAuIE9yZ2FuaWMgUmVzcGJlcnJpZXMgZG9lcyBub3QgbWFrZSBpdCB0byB0b3AgMTAgb2YgRGF5IDYuIFRoaXMgbWVhbnMgdGhhdCB0aGVyZSBpcyBhIGNoYW5jZSBvZiBwcmVkaWN0YWJpbGl0eSBiYXNlZCBvbiB0aGUgZGF5IG9yZGVyIGlzIG1hZGUuIA0KDQpgYGB7cixmaWcud2lkdGg9OS4zLCBmaWcuaGVpZ2h0PTQuNX0NCm9yZGVyX3Byb2R1Y3RzX3ByaW9yICU+JSANCiAgbGVmdF9qb2luKG9yZGVycykgJT4lIGxlZnRfam9pbihwcm9kdWN0cykgJT4lDQogIGdyb3VwX2J5KG9yZGVyX2RvdywgcHJvZHVjdF9uYW1lKSAlPiUNCiAgc3VtbWFyaXplKG49bigpKSAlPiUNCiAgbXV0YXRlKHBlcmNlbnRhZ2U9bi9zdW0obikpICU+JQ0KICB0b3BfbigxMCwgd3Q9bikgJT4lDQogIGdncGxvdCAoYWVzKHg9YXMuZmFjdG9yKG9yZGVyX2RvdyksIHk9cGVyY2VudGFnZSwgZmlsbD1wcm9kdWN0X25hbWUpKSArIA0KICAgIGdlb21fY29sKCkgKyB5bGFiKCdQcm9wcnRpb24gb2YgT3JkZXJzIEluIEEgRGF5JykgKyBnZ3RpdGxlKCdEYWlseSBUb3AgMTAgUHJvZHVjdHMgT3JkZXJlZCcpICsNCiAgICB0aGVtZShsZWdlbmQucG9zaXRpb249ImJvdHRvbSIsbGVnZW5kLmRpcmVjdGlvbj0iaG9yaXpvbnRhbCIpDQpgYGANCg0KIyMjIFByb2R1Y3RzIE9yZGVyZWQgSG91ciBQYXR0ZXJuDQoNCk1vcm5pbmcgdG8gYWZ0ZXJub29uIGFyZSB0aGUgcGVhayBzaG9wcGluZyBob3VycyBmb3IgaW5zdGFjYXJ0IGN1c3RvbWVycy4gVGhlIGhvdXIgb3JkZXIgbWFkZSBpbmZsdWVuY2VzIGJhc2tldCBzaXplLg0KDQpgYGB7cixmaWcud2lkdGg9OS4zLCBmaWcuaGVpZ2h0PTMuMH0NCm9yZGVyX3Byb2R1Y3RzX3ByaW9yICU+JQ0KICBsZWZ0X2pvaW4ob3JkZXJzKSAlPiUNCiAgZ3JvdXBfYnkob3JkZXJfaG91cl9vZl9kYXkpICU+JQ0KICBzdW1tYXJpemUoY291bnQgPSBuKCkpICU+JQ0KICBtdXRhdGUocGVyY2VudGFnZT1jb3VudC9zdW0oY291bnQpKSAlPiUNCiAgZ2dwbG90IChhZXMoeD1hcy5mYWN0b3Iob3JkZXJfaG91cl9vZl9kYXkpLCB5PXBlcmNlbnRhZ2UpKSArIA0KICAgIGdlb21fY29sKCkrIHlsYWIoJ1BlcmNlbnRhZ2Ugb2YgT3JkZXJzJykgKyBnZ3RpdGxlKCdIb3VybHkgT3JkZXJzJykNCmBgYA0KDQpJbiB0aGUgZ3JvY2VyeSwgdGhlcmUgYXJlIGNsb3NlIHRvIDUwLDAwMCBwcm9kdWN0cy4gV2hlbiB3ZSB6b29tIGludG8gaG91cmx5IHB1cmNoYXNlcywgd2Ugbm90aWNlZCB0aGF0IHRvcCAxMCBwcm9kdWN0cyBtYW5hZ2VkIHRvIHNjb3JlIGJldHdlbiA2JSB0byA4JSBvZiBob3VybHkgc2FsZXMuIEV2ZXJ5IGhvdXIgaGFzIHNsaWdodGx5IGRpZmZyZW50IGNvbWJpbmF0aW9uIG9mIHRvcCAxMCBwcm9kdWN0cyAoY29tYmluYXRpb24gb3V0IG9mIDEyIHByb2R1Y3RzKS4gVGhhdCBtZWFucyBjZXJ0YWluIHByb2R1Y3RzIGFyZSBwcmVkaWN0YWJsZSBmb3Igb3JkZXJpbmcgaXJyZWdhcmRsZXNzIG9mIHRoZSBob3VyIG9mIG9yZGVyLg0KDQpgYGB7cixmaWcud2lkdGg9OS4zLCBmaWcuaGVpZ2h0PTV9DQpvcmRlcl9wcm9kdWN0c19wcmlvciAlPiUgDQogIGxlZnRfam9pbihvcmRlcnMpICU+JSBsZWZ0X2pvaW4ocHJvZHVjdHMpICU+JQ0KICBncm91cF9ieShvcmRlcl9ob3VyX29mX2RheSwgcHJvZHVjdF9uYW1lKSAlPiUNCiAgc3VtbWFyaXplKG49bigpKSAlPiUNCiAgbXV0YXRlKHBlcmNlbnRhZ2U9bi9zdW0obikpICU+JQ0KICB0b3BfbigxMCwgd3Q9bikgJT4lDQogIGdncGxvdCAoYWVzKHg9YXMuZmFjdG9yKG9yZGVyX2hvdXJfb2ZfZGF5KSwgeT1wZXJjZW50YWdlLCBmaWxsPXByb2R1Y3RfbmFtZSkpICsgDQogICAgZ2VvbV9jb2woKSArIHlsYWIoJ1Byb3BydGlvbiBvZiBPcmRlcnMgSW4gQSBIb3VyJykgKyBnZ3RpdGxlKCdIb3VybHkgVG9wIDEwIFByb2R1Y3RzIE9yZGVyZWQnKSArDQogICAgdGhlbWUobGVnZW5kLnBvc2l0aW9uPSJib3R0b20iLGxlZ2VuZC5kaXJlY3Rpb249Imhvcml6b250YWwiKQ0KYGBgDQoNCg0KIyMgUmVvcmRlcmluZyBBbmFseXNpcw0KDQojIFByZWRpY3RpdmUgQW5hbHlzaXMNCg0KIyMgVHlwZSBvZiBQcmVkaWN0aW9uDQoNClByZWRpY3Qgd2hhdCBwcm9kdWN0IHdpbGwgdGhlIGN1c3RvbWVyIHB1cmNoYXNlIGluIHRoZSBuZXh0IGJhc2tldCBtZWFucyBlc3RpbWF0aW9uIG9mIHByb2JhYmx5IGlmIGVhY2ggcHJvZHVjdCBiZWluZyBwdXJjaGFzZWQgYmVmb3JlIHdpbGwgYmUgcHVyY2hhc2VkIGFnYWluLiAqKlRoaXMgaXMgYSBjbGFzc2lmaWNhdGlvbiBwcm9ibGVtKiosIGFzIHdlbGwgYXMgKiphIHJlZ3Jlc3Npb24gb2YgcHJvYmFiaWxpdHkqKiBvZiByZXB1cmNoYXNlcy4gVGhlIG91dHB1dCBvZiB0aGUgcHJlZGljdGlvbiBpcyB0byBiZSBldmFsdWF0ZWQgYWdhaW5zdCB0cmFpbmluZyBldmFsX3NldC4gd2l0aCBiZWxvdyB0YWJsZSBjb2x1bW5zOiAgDQoNCi0gYDx1c2VyX2lkIHByb2R1Y3RfaWQ+YDogYXMga2V5ICANCi0gYGxhYmVsYCA6IDEgb3IgMCAsIHRoaXMgaXMgdGhlIHRhcmdldCBsYWJlbCAgDQoNCkZvciBzdWJtaXNzaW9uIHRvIGthZ2dnbGUsIG91dHB1dCBmcm9tIGFib3ZlIHRhYmxlIHNob3VsZCBiZSB0cmFuc2Zvcm1lZCBpbnRvIHRoZSBmb3JtIGJlbG93Og0KDQotIGB1c2VyX2lkYCAgDQotIGA8bGlzdCBvZiBwcm9kdWN0IGlkIHNlcGVyYXRlZCBieSBzcGFjZT5gICANCg0KRmlyc3QsIGxldCdzIGNyZWF0ZSBhICoqdHJhaW5pbmcgbGFiZWwqKiwgd2hpY2ggaXMgdGhlIHZhbHVlIG9yIGVpdGhlciAxIG9yIDAgdG8gaW5kaWNhdGUgdGhlIGFjdHVhbCBiYXNrZXQgY29udGVudC4gQWxsIHByZWRpY3Rpb24gd2lsbCBiZSBldmFsdWF0ZWQgYWdhaW5zdCB0aGlzIHRyYWluaW5nIGxhYmVsLg0KDQpgYGB7ciBtZXNzYWdlPUZBTFNFfQ0KIyMgVHJhaW5pbmcgTGFiZWwNCnRyYWluLmxhYmVsID0gb3JkZXJzICU+JSANCiAgZmlsdGVyKGV2YWxfc2V0PT0ndHJhaW4nKSAlPiUgDQogIGxlZnRfam9pbihvcmRlcl9wcm9kdWN0c190cmFpbikgJT4lDQogIGxlZnRfam9pbihwcm9kdWN0cykgJT4lDQogIG11dGF0ZShhY3R1YWw9MSkgJT4lICAgI3RoaXMgaXMgdHJhaW5pbmcgbGFiZWwNCiAgc2VsZWN0KHVzZXJfaWQsIG9yZGVyX2lkLCBwcm9kdWN0X2lkLCBwcm9kdWN0X25hbWUsIGFjdHVhbCkNCmBgYA0KDQpMZXQncyBzZWUgYW4gZXhhbXBsZSBvZiAqKnRyYWluaW5nIGxhYmVsKiogZm9yIHVzZXJfaWQgMS4gQXMgdGhpcyBpcyB0cmFpbmluZyBldmFsX3NldCwgdGhlIHNpbmdsZSBvcmRlcl9pZCBzaG93cyB0aGF0IGl0IGlzIHRoZSBmaW5hbCBvcmRlci4gV2Ugc2hhbGwgdXNlIHRoaXMgYXMgdGhlIGxhYmVsIChhY3R1YWwpIGZvciB0cmFpbmluZy4gDQoNCmBgYHtyIG1lc3NhZ2U9RkFMU0V9DQp0cmFpbi5sYWJlbCAlPiUgZmlsdGVyKHVzZXJfaWQ9PTEpICU+JSBrYWJsZSAlPiUgIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIikpDQpgYGANCg0KIyMgTW9kZWwgRXZhbHVhdGlvbiAmIE9wdGltaXphdGlvbg0KDQpJbnN0YWNhcnQgaGFzIGNsb3NlIHRvIDUwayBwcm9kdWN0cyBpbiB0aGVpciBjYXRhbG9ndWUuIEFzIHRoZSBtYXhpbXVtIG51bWJlciBvZiBpdGVtcyBvcmRlcmVkIGJ5IGEgdXNlciBpcyBqdXN0IGEgZnJhY3Rpb24gb2YgdGhlIDUwayBhdmFpbGFibGUgcHJvZHVjdC4gVGhpcyBtZWFucyBieSBzaW1wbHkgcHJlZGljdGluZyBub3RoaW5nIGlzIHB1cmNoYXNlZCBpbiB0aGUgbmV4dCBiYXNrZXQsIHdlIHdvdWxkIHllaWxkICoqY2xvc2UgdG8gMTAwJSBhY2N1cmFjeSoqLiAgDQoNCkR1ZSB0byB0aGUgKipoaWdobHkgaW1iYWxhbmNlKiogZGF0YXNldCwgSW5zdGFjYXJ0IHJlcXVpcmUgKipGMSBTY29yZSoqIGFzIHRoZSBjb21wZXRpdGlvbiBzY29yaW5nLCBpbnN0ZWFkIG9mIGFjY3VyYWN5LiAgDQoNClRvIGV2YWx1YXRlIHRoZSBwZXJmb3JtYW5jZSBvZiB0aGUgbW9kZWwsIHdlIGhhZCBjcmVhdGVkIGEgY3VzdG9tIGZ1bmN0aW9uIHRvIGJ1aWxkIGEgKipjb25mdXNpb24gbWF0cml4IGFuZCBkZXJpdmUgb3RoZXIgYmluYXJ5IGNsYXNzaWZpY2F0aW9uIG1ldHJpY3MqKi4gDQoNCmBgYHtyfQ0KIyMgQ3VzdG9tIEZ1bmN0aW9uIEZvciBCaW5hcnkgQ2xhc3MgUGVyZm9ybWFuY2UgRXZhbHVhdGlvbg0KYmluY2xhc3NfZXZhbCA9IGZ1bmN0aW9uIChhY3R1YWwsIHByZWRpY3QpIHsNCiAgY20gPSB0YWJsZShhcy5pbnRlZ2VyKGFjdHVhbCksIGFzLmludGVnZXIocHJlZGljdCksIGRubj1jKCdBY3R1YWwnLCdQcmVkaWN0ZWQnKSkNCiAgYWMgPSAoY21bJzEnLCcxJ10rY21bJzAnLCcwJ10pLyhjbVsnMCcsJzEnXSArIGNtWycxJywnMCddICsgY21bJzEnLCcxJ10gKyBjbVsnMCcsJzAnXSkNCiAgcHIgPSBjbVsnMScsJzEnXS8oY21bJzAnLCcxJ10gKyBjbVsnMScsJzEnXSkNCiAgcmMgPSBjbVsnMScsJzEnXS8oY21bJzEnLCcwJ10gKyBjbVsnMScsJzEnXSkNCiAgZnMgPSAyKiBwcipyYy8ocHIrcmMpDQogIGxpc3QoY209Y20sIHJlY2FsbD1yYywgcHJlY2lzaW9uPXByLCBmc2NvcmU9ZnMsIGFjY3VyYWN5PWFjKQ0KfQ0KYGBgDQoNCklmIHRoZSBwcmVkaWN0aW9uIGlzIGJhc2VkIG9uIHByb2JhYmlsaXR5LCB3ZSBzaGFsbCBidWlsZCBhIGZ1bmN0aW9uIHRvIGRpc2NvdmVyICoqY3V0b2ZmIHRoYXQgb3B0aW1pemUgdmFyaW91cyBwZXJmb3JtYW5jZSBtZXRyaWNzKiouIA0KDQpgYGB7cn0NCiMjIyBDdXRvZmYgVGhyZXNob2xkIE9wdGltaXphdGlvbg0Kb3B0aW1pemVfY3V0b2ZmID0gZnVuY3Rpb24gKGFjdHVhbCwgcHJvYmFiaWxpdHkpIHsNCiAgcm9jci5wcmVkID0gcHJlZGljdGlvbihwcmVkaWN0aW9ucyA9IHByb2JhYmlsaXR5LCBsYWJlbHMgPSBhY3R1YWwpDQogIHJvY3IubWV0cmljcyA9IGRhdGEuZnJhbWUoDQogICAgICBjdXRvZmYgICA9IHJvY3IucHJlZEBjdXRvZmZzW1sxXV0sDQogICAgICBhY2N1cmFjeSA9IChyb2NyLnByZWRAdHBbWzFdXSArIHJvY3IucHJlZEB0bltbMV1dKSAvIA0KICAgICAgICAgICAgICAgICAgIChyb2NyLnByZWRAdHBbWzFdXSArIHJvY3IucHJlZEB0bltbMV1dICsgcm9jci5wcmVkQGZwW1sxXV0gKyByb2NyLnByZWRAZm5bWzFdXSksDQogICAgICB0cHIgPSByb2NyLnByZWRAdHBbWzFdXSAvIChyb2NyLnByZWRAdHBbWzFdXSArIHJvY3IucHJlZEBmbltbMV1dKSwNCiAgICAgIGZwciA9IHJvY3IucHJlZEBmcFtbMV1dIC8gKHJvY3IucHJlZEBmcFtbMV1dICsgcm9jci5wcmVkQHRuW1sxXV0pLA0KICAgICAgcHB2ID0gcm9jci5wcmVkQHRwW1sxXV0gLyAocm9jci5wcmVkQHRwW1sxXV0gKyByb2NyLnByZWRAZnBbWzFdXSkNCiAgKQ0KICByb2NyLm1ldHJpY3MkZnNjb3JlID0gMiAqIChyb2NyLm1ldHJpY3MkdHByICogcm9jci5tZXRyaWNzJHBwdikgLyAocm9jci5tZXRyaWNzJHRwciArIHJvY3IubWV0cmljcyRwcHYpDQogIHJvY3IubWV0cmljcyR0cHJfZnByID0gcm9jci5tZXRyaWNzJHRwciAvIHJvY3IubWV0cmljcyRmcHINCiAgDQogICMjIERpc2NvdmVyeSB0aGUgb3B0aW1hbCB0aHJlc2hvbGQgZm9yIHZhcmlvdXMgbWV0cmljcw0KICByb2NyLmJlc3QgPSByYmluZCgNCiAgICBiZXN0LmFjY3VyYWN5ID0gYyhtYXggPSBtYXgocm9jci5tZXRyaWNzJGFjY3VyYWN5LCBuYS5ybSA9IFRSVUUpLCBjdXRvZmY9cm9jci5tZXRyaWNzJGN1dG9mZlt3aGljaC5tYXgocm9jci5tZXRyaWNzJGFjY3VyYWN5KV0pLA0KICAgIGJlc3QucHB2ID0gYyhtYXggPSBtYXgocm9jci5tZXRyaWNzJHBwdiwgbmEucm0gPSBUUlVFKSwgY3V0b2ZmID0gcm9jci5tZXRyaWNzJGN1dG9mZlt3aGljaC5tYXgocm9jci5tZXRyaWNzJHBwdildKSwNCiAgICBiZXN0LnJlY2FsbCA9IGMobWF4ID0gbWF4KHJvY3IubWV0cmljcyR0cHIsIG5hLnJtID0gVFJVRSksIGN1dG9mZiA9IHJvY3IubWV0cmljcyRjdXRvZmZbd2hpY2gubWF4KHJvY3IubWV0cmljcyR0cHIpXSksDQogICAgYmVzdC5mc2NvcmUgPSBjKG1heCA9IG1heChyb2NyLm1ldHJpY3MkZnNjb3JlLCBuYS5ybSA9IFRSVUUpLCBjdXRvZmYgPSByb2NyLm1ldHJpY3MkY3V0b2ZmW3doaWNoLm1heChyb2NyLm1ldHJpY3MkZnNjb3JlKV0pLA0KICAgIGJlc3QudHByX2ZwciA9IGMobWF4ID0gbWF4KHJvY3IubWV0cmljcyR0cHJfZnByLCBuYS5ybSA9IFRSVUUpLCBjdXRvZmYgPSByb2NyLm1ldHJpY3MkY3V0b2ZmW3doaWNoLm1heChyb2NyLm1ldHJpY3MkdHByX2ZwcildKQ0KICApDQogIA0KICBsaXN0KG1ldHJpY3MgPSByb2NyLm1ldHJpY3MsIGJlc3QgPSByb2NyLmJlc3QpDQp9DQpgYGANCg0KDQoNCiMjIE1vZGVsIDEgOiBOYWl2ZSBQcmVkaWN0aW9uDQoNCiMjIyBCdWlsZCBUaGUgTW9kZWwNCg0KV2UgY2FuIHNpbXBseSBwcmVkaWN0IHRoZSBiYXNrZXQgYmFzZWQgb24gdXNlciBsYXN0IG9yZGVyLiANCg0KYGBge3IgbWVzc2FnZT1GQUxTRX0NCm0xLnByZWRpY3QgPSBvcmRlcnMgJT4lIA0KICBmaWx0ZXIoZXZhbF9zZXQ9PSdwcmlvcicpICU+JSANCiAgZ3JvdXBfYnkodXNlcl9pZCkgJT4lDQogIHRvcF9uKG49MSwgd3Q9b3JkZXJfbnVtYmVyKSAlPiUgICAgICAgI2xhc3Qgb3JkZXIgaGFzIHRoZSBoaWdoZXIgb3JkZXJfbnVtYmVyDQogIGxlZnRfam9pbihvcmRlcl9wcm9kdWN0c19wcmlvcikgJT4lDQogIG11dGF0ZSAoDQogICAgcHJlZGljdGVkPTEpICU+JSAgICAgICAgICAgICAgI3ByZWRpY3QgYmFzZWQgb24gbGFzdCBvcmRlcmVkLCB0aGVyZWZvcmUgMQ0KICBzZWxlY3QodXNlcl9pZCwgcHJvZHVjdF9pZCwgcHJlZGljdGVkKSAlPiUgDQogIGZ1bGxfam9pbih0cmFpbi5sYWJlbCkgJT4lICAjIGpvaW4gd2l0aCB0cmFpbi5sYWJlbCBmb3IgaXRlbXMgbm90IHByZWRpY3RlZCBidXQgaW4gdHJhaW4ubGFiZWwNCiAgc2VsZWN0KHVzZXJfaWQsIHByb2R1Y3RfaWQsIGFjdHVhbCwgcHJlZGljdGVkKSAlPiUNCiAgcmVwbGFjZSguLCBpcy5uYSguKSwgMCkgJT4lICMgdHJlYXQgbm90IHByZWRpY3RlZCBhcyAwDQogIGFycmFuZ2UodXNlcl9pZCwgcHJvZHVjdF9pZCkNCmBgYA0KIyMjIFNhbXBsZSBPdXRwdXQNCg0KYGBge3J9DQptMi5wcmVkaWN0ICU+JSBmaWx0ZXIodXNlcl9pZCA9PTEpICU+JSBrYWJsZSAlPiUga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiKSkNCmBgYA0KDQoNCiMjIyBDb25mdXNpb24gTWF0cml4DQoNCmBgYHtyIG1vZGVsMS1ldmFsfQ0KbTEuZXZhbCA9IGJpbmNsYXNzX2V2YWwobTEucHJlZGljdCRhY3R1YWwsIG0xLnByZWRpY3QkcHJlZGljdGVkKQ0KbTEuZXZhbCRjbQ0KYGBgDQoNCiMjIyBNb2RlbCBQZXJmb3JtYW5jZQ0KDQpUaGUgcmVzdWx0IHNob3dzIG9ubHkgKiowLjIxNTMgRjEgU2NvcmUqKi4NCg0KYGBge3J9DQpjYXQoIkFjY3VyYWN5OiAgIiwgbTEuZXZhbCRhY2N1cmFjeSwNCiAgICAiXG5QcmVjaXNpb246ICIsIG0xLmV2YWwkcHJlY2lzaW9uLA0KICAgICJcblJlY2FsbDogICAgIiwgbTEuZXZhbCRyZWNhbGwsDQogICAgIlxuRlNjb3JlOiAgICAiLCBtMS5ldmFsJGZzY29yZSkNCmBgYA0KDQpgYGB7ciBlY2hvPUZBTFNFfQ0Kcm0obGlzdD1jKCdtMS5wcmVkaWN0JykpDQpgYGANCg0KIyMgTW9kZWwgMiA6IFNtYXJ0ZXIgTmFpdmUgUHJlZGljdGlvbiAoQmFzZWxpbmUpDQoNCkluIHRoaXMgbW9kZWwsIHdlIHByZWRpY3QgcHJvZHVjdHMgaW4gdGhlIGJhc2tldCBieSBlc3RpbWF0aW5nIHRoZWlyIGZyZXF1ZW5jeSBvZiByZXB1cmNoYXNlZC4gVGhpcyB3YXkgd2UgZ2V0IGEgcmF0aW8gdG8gaW5kaWNhdGUgcHJvYmFiaWxpdHkgb2YgcmUtcHVyY2hhc2VzLiBXZSB1c2UgUk9DUiBwYWNrYWdlIHRvIHByZWRpY3QgdGhlIGJlc3QgY3V0b2ZmIHBvaW50IChhdCB3aGljaCBhYm92ZSB0aGlzIGN1dG9mZiB3ZSBzaGFsbCBwcmVkaWN0IGZvciByZS1vcmRlcikgdGhhdCBnaXZlIHVzIHRoZSAqKm9wdGltdW0gRjEgc2NvcmUqKi4gDQoNCiMjIyBCdWlsZCBUaGUgTW9kZWwNCg0KYGBge3IgbW9kZWwyLWJ1aWxkaW5nLCBtZXNzYWdlPUZBTFNFfQ0KIyMgQnVpbGQgTW9kZWwNCm0yLnByZWRpY3QgPSBvcmRlcnMgJT4lIA0KICBmaWx0ZXIoZXZhbF9zZXQ9PSdwcmlvcicpICU+JSANCiAgZ3JvdXBfYnkodXNlcl9pZCkgJT4lDQogIG11dGF0ZSh0b3RhbF9vcmRlcnMgPSBtYXgob3JkZXJfbnVtYmVyKSkgJT4lICAjIHRvdGFsIG51bWJlciBvZiBvcmRlcnMgbWFkZSBwcmV2aW91c2x5DQogIHVuZ3JvdXAgJT4lIHNlbGVjdCh1c2VyX2lkLCBvcmRlcl9pZCwgdG90YWxfb3JkZXJzKSAlPiUNCiAgbGVmdF9qb2luKG9yZGVyX3Byb2R1Y3RzX3ByaW9yKSAlPiUNCiAgZ3JvdXBfYnkodXNlcl9pZCwgcHJvZHVjdF9pZCkgJT4lDQogIHN1bW1hcml6ZShyYXRlPW4oKS9tYXgodG90YWxfb3JkZXJzKSkgJT4lDQogIHNlbGVjdCh1c2VyX2lkLCBwcm9kdWN0X2lkLCByYXRlKSAlPiUNCiAgZnVsbF9qb2luKHRyYWluLmxhYmVsKSAlPiUgICMgam9pbiB3aXRoIHRyYWluLmxhYmVsIGZvciBpdGVtcyBub3QgcHJlZGljdGVkIGJ1dCBpbiB0cmFpbi5sYWJlbA0KICBzZWxlY3QodXNlcl9pZCwgcHJvZHVjdF9pZCwgYWN0dWFsLCByYXRlKSAlPiUNCiAgcmVwbGFjZSguLCBpcy5uYSguKSwgMCkgJT4lICMgdHJlYXQgbm90IHByZWRpY3RlZCBhcyAwDQogIGFycmFuZ2UodXNlcl9pZCwgcHJvZHVjdF9pZCkNCmBgYA0KDQojIyMgU2FtcGxlIE91dHB1dA0KDQpgYGB7cn0NCm0yLnByZWRpY3QgJT4lIGZpbHRlcih1c2VyX2lkPT0xKSAlPiUga2FibGUgJT4lIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIikpDQpgYGANCiMjIyBPcHRpbWl6ZSBDdXRvZmYNCg0KV2Ugc2VlIHRoYXQgaW4gb3JkZXIgdG8gbWF4aW1pemUgKipGMSBTY29yZSoqLCB3ZSBuZWVkIHRvIHNldCB0aGUgY3V0b2ZmIHRocmVzaG9sZCB0byAwLjMzNjgsIHdoaWNoIGlzIHRoZSBuZXh0IHN0ZXAuDQoNCmBgYHtyIG1vZGVsMi1vcHRpbWl6ZX0NCiMjIyBUaHJlc2hvbGQgT3B0aW1pemF0aW9uDQptMi5yb2NyID0gb3B0aW1pemVfY3V0b2ZmKGFjdHVhbCA9IG0yLnByZWRpY3QkYWN0dWFsLCBwcm9iYWJpbGl0eSA9IG0yLnByZWRpY3QkcmF0ZSkNCmthYmxlKG0yLnJvY3IkYmVzdCkgJT4lIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIikpDQpgYGANCg0KIyMjIENvbmZ1c2lvbiBNYXRyaXgNCg0KTGV0J3Mgc2V0IHRoZSAqKmN1dG9mZiB0byAwLjMzNjgqKiBhcyBkaXNjb3ZlcmVkIGluIHByZXZpb3VzIHN0ZXAuDQoNCmBgYHtyIG1vZGVsMi1ldmFsfQ0KbTIuZXZhbCA9IGJpbmNsYXNzX2V2YWwobTIucHJlZGljdCRhY3R1YWwsIG0yLnByZWRpY3QkcmF0ZT4wLjMzNjgpDQptMi5ldmFsJGNtDQpgYGANCg0KIyMjIE1vZGVsIFBlcmZvcm1hbmNlDQoNCldlIGFyZSBnZXR0aW5nIHNsaWdodGx5ICoqYmV0dGVyIEYxIFNjb3JlICgwLjIzMTIpKiogY29tcGFyZSB0byBwcmV2aW91cyBtb2RlbC4gV2Ugc2hhbGwgdXNlIHRoaXMgYXMgdGhlICoqYmFzZWxpbmUqKi4NCg0KYGBge3J9DQpjYXQoIkFjY3VyYWN5OiAgIiwgbTIuZXZhbCRhY2N1cmFjeSwNCiAgICAiXG5QcmVjaXNpb246ICIsIG0yLmV2YWwkcHJlY2lzaW9uLA0KICAgICJcblJlY2FsbDogICAgIiwgbTIuZXZhbCRyZWNhbGwsDQogICAgIlxuRlNjb3JlOiAgICAiLCBtMi5ldmFsJGZzY29yZSkNCmBgYA0KDQoNCmBgYHtyIGVjaG89RkFMU0V9DQpybShsaXN0PWMoJ20yLnByZWRpY3QnLCdtMi5yb2NyJykpDQpgYGANCg0KIyMgTW9kZWwgMyA6IE1hY2hpbmUgTGVhcm5pbmcgQ2xhc3NpZmljYXRpb24NCg0KV2UgY29uc3RydWN0IGFsbCB0aGUgcHJvZHVjdHMgdGhhdCB1c2VycyBoYWQgcHVyY2hhc2VkIGluIHRoZSBsYXN0IDMgb3JkZXJzLCB0aGVuIHVzZSBtYWNoaW5lIGxlYXJuaW5nIGNsYXNzaWZpY2F0aW9uIHRvIHByZWRpY3Qgd2lsbCBlYWNoIG9mIHRoZSBwcm9kdWN0IGJlIHB1cmNoYXNlZCBhZ2Fpbi4gV2Ugc2hhbGwgdXNlICoqZGVjaXNpb24gdHJlZSBhbmQgbG9naXN0aWMgcmVncmVzc2lvbioqIGZvciB0aGlzIHByZWRpY3Rpb24uDQoNCmBgYHtyfQ0KI20zLnByZWRpY3QgPSBvcmRlcnMgJT4lIA0KIyAgZmlsdGVyKGV2YWxfc2V0PT0ncHJpb3InKSAlPiUgDQojICBsZWZ0X2pvaW4ob3JkZXJfcHJvZHVjdHNfcHJpb3IpICU+JQ0KIyAgZ3JvdXBfYnkodXNlcl9pZCwgcHJvZHVjdF9pZCkgJT4lDQojICBzdW1tYXJpemUocmF0ZT1uKCkvbWF4KHRvdGFsX29yZGVycykpDQpgYGANCg0KDQojIyMgRmVhdHVyZSBFbmdpbmVlcmluZw0KDQojIyMjIFVzZXIgRmVhdHVyZXMNCg0KV2UgY3JlYXRlIHRocmVlIGZlYXR1cmVzIGFwcGxpZWQgdG8gaW5kaXZpZHVhbCB1c2Vycy4NCg0KKipgdXNlcnNgKiogIA0KLSBgYXZnX3VzZXJfb3JkZXJfZG93YDogQXZlcmFnZSBvZiBvcmRlcl9kb3cgIA0KLSBgYXZnX3VzZXJfb3JkZXJfaG9kYDogQXZlcmFnZSBvZiBvcmRlcl9ob3VyX29mX2RheSAgDQotIGBhdmdfdXNlcl9pdGVtc2A6IEF2ZXJhZ2UgaXRlbXMgZm9yIGVhY2ggb3JkZXIgIA0KDQpgYGB7ciBtZXNzYWdlPUZBTFNFfQ0KdXNlcnMudHJhaW4uYXZnX2l0ZW1zID0gb3JkZXJzICU+JSANCiAgZmlsdGVyKGV2YWxfc2V0PT0ndHJhaW4nKSAlPiUNCiAgc2VsZWN0KHVzZXJfaWQpICU+JQ0KICBsZWZ0X2pvaW4ob3JkZXJzKSAlPiUNCiAgbGVmdF9qb2luKG9yZGVyX3Byb2R1Y3RzX3ByaW9yKSAlPiUNCiAgY291bnQodXNlcl9pZCwgb3JkZXJfaWQpICU+JQ0KICBncm91cF9ieSh1c2VyX2lkKSAlPiUNCiAgc3VtbWFyaXplKGF2Z191c2VyX2l0ZW1zID0gbWVhbihuKSkNCg0KdXNlcnMudHJhaW4uYXZnX2Rvd2hvZCA9IG9yZGVycyAlPiUgDQogIGZpbHRlcihldmFsX3NldD09J3RyYWluJykgJT4lDQogIHNlbGVjdCh1c2VyX2lkKSAlPiUNCiAgbGVmdF9qb2luKG9yZGVycykgJT4lDQogIGdyb3VwX2J5KHVzZXJfaWQpICU+JQ0KICBzdW1tYXJpemUoIA0KICAgIGF2Z191c2VyX29yZGVyX2RvdyA9IG1lYW4ob3JkZXJfZG93KSwNCiAgICBhdmdfdXNlcl9vcmRlcl9ob2QgPSBtZWFuKG9yZGVyX2hvdXJfb2ZfZGF5KSkNCg0KdXNlcnMudHJhaW4gPSB1c2Vycy50cmFpbi5hdmdfaXRlbXMgJT4lIGxlZnRfam9pbih1c2Vycy50cmFpbi5hdmdfZG93aG9kKQ0KICANCnVzZXJzLnRlc3QuYXZnX2l0ZW1zID0gb3JkZXJzICU+JSANCiAgZmlsdGVyKGV2YWxfc2V0PT0ndGVzdCcpICU+JQ0KICBzZWxlY3QodXNlcl9pZCkgJT4lDQogIGxlZnRfam9pbihvcmRlcnMpICU+JQ0KICBsZWZ0X2pvaW4ob3JkZXJfcHJvZHVjdHNfcHJpb3IpICU+JQ0KICBjb3VudCh1c2VyX2lkLCBvcmRlcl9pZCkgJT4lDQogIGdyb3VwX2J5KHVzZXJfaWQpICU+JQ0KICBzdW1tYXJpemUoYXZnX3VzZXJfaXRlbXMgPSBtZWFuKG4pKQ0KDQp1c2Vycy50ZXN0LmF2Z19kb3dob2QgPSBvcmRlcnMgJT4lIA0KICBmaWx0ZXIoZXZhbF9zZXQ9PSd0ZXN0JykgJT4lDQogIHNlbGVjdCh1c2VyX2lkKSAlPiUNCiAgbGVmdF9qb2luKG9yZGVycykgJT4lDQogIGdyb3VwX2J5KHVzZXJfaWQpICU+JQ0KICBzdW1tYXJpemUoIA0KICAgIGF2Z191c2VyX29yZGVyX2RvdyA9IG1lYW4ob3JkZXJfZG93KSwNCiAgICBhdmdfdXNlcl9vcmRlcl9ob2QgPSBtZWFuKG9yZGVyX2hvdXJfb2ZfZGF5KSkNCg0KdXNlcnMudGVzdCA9IHVzZXJzLnRlc3QuYXZnX2l0ZW1zICU+JSBsZWZ0X2pvaW4odXNlcnMudGVzdC5hdmdfZG93aG9kKQ0KDQpybShsaXN0PWMoJ3VzZXJzLnRlc3QuYXZnX2l0ZW1zJywndXNlcnMudGVzdC5hdmdfZG93aG9kJywndXNlcnMudHJhaW4uYXZnX2l0ZW1zJywndXNlcnMudHJhaW4uYXZnX2Rvd2hvZCcpKQ0KYGBgDQoNCg0KKipUcmFpbmluZyBVc2VycyoqICANCg0KYGBge3J9DQpoZWFkKHVzZXJzLnRyYWluKSAlPiUga2FibGUgJT4lIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIikpDQpgYGANCg0KKipUZXN0IFVzZXJzKiogICANCg0KYGBge3J9DQpoZWFkKHVzZXJzLnRlc3QpICU+JSBrYWJsZSAlPiUga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiKSkNCmBgYA0KDQojIyMjIFByb2R1Y3QgRmVhdHVyZXMNCg0KV2UgY3JlYXRlIHR3byBmZWF0dXJlcyBnZW5lcmFsbHkgYXBwbGllZCBhY3Jvc3MgYWxsIHByb2R1Y3RzLiAgDQoNCioqYHByb2R1Y3RzYCoqICANCg0KLSBgYXZnX3Byb2R1Y3Rfb3JkZXJfZG93YDogQXZlcmFnZSBvZiBwcm9kdWN0IG9yZGVyX2Rvdw0KLSBgYXZnX3Byb2R1Y3Rfb3JkZXJfaG9kYDogQXZlcmFnZSBvZiBwcm9kdWN0IG9yZGVyX2hvdXJfb2ZfZGF5ICANCg0KYGBge3IsIG1lc3NhZ2U9RkFMU0V9DQpwcm9kdWN0c18gPSBvcmRlcnMgJT4lIA0KICBsZWZ0X2pvaW4ob3JkZXJfcHJvZHVjdHNfcHJpb3IpICU+JQ0KICBncm91cF9ieShwcm9kdWN0X2lkKSAlPiUNCiAgc3VtbWFyaXplKCANCiAgICBhdmdfcHJvZHVjdF9vcmRlcl9kb3cgPSBtZWFuKG9yZGVyX2RvdyksDQogICAgYXZnX3Byb2R1Y3Rfb3JkZXJfaG9kID0gbWVhbihvcmRlcl9ob3VyX29mX2RheSkNCiAgKSAlPiUNCiAgbGVmdF9qb2luKHByb2R1Y3RzKQ0KDQpoZWFkKHByb2R1Y3RzXykgJT4lIGthYmxlICU+JSBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIpKQ0KYGBgDQojIyMgQ29uc3RydWN0IFRyYWluaW5nIERhdGEgd2l0aCBMYWJlbA0KDQpgYGB7cn0NCm0zLnRyYWluLmxhYmVsID0gdHJhaW4ubGFiZWwgJT4lIA0KICBsZWZ0X2pvaW4odXNlcnMudHJhaW4pICU+JSAgICNjb21iaW5lIHdpdGggbmV3IHVzZXIgZmVhdHVyZXMNCiAgbGVmdF9qb2luKHByb2R1Y3RzXykgICAgICAgICAgI2NvbWJpbmUgd2l0aCBuZXcgcHJvZHVjdCBmZWF0dXJlcw0KDQpoZWFkKG0zLnRyYWluLmxhYmVsKSAlPiUga2FibGUgJT4lIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIikpDQpgYGANCg0KDQoNCiMjIFNlbGVjdGlvbiBvZiBNb2RlbA0KDQoNCg0KDQojIyBEYXRhIFJlY29kaW5nDQoNCg0KIyMgUHJlZGljdGlvbg0KDQoNCiMgQW5hbHlzaXMgJiBSZWNvbW1lbmRhdGlvbnMNCg0KIyBDaXRhdGlvbnMNCg0KMS4gVGhlIEluc3RhY2FydCBPbmxpbmUgR3JvY2VyeSBTaG9wcGluZyBEYXRhc2V0IDIwMTciLCBBY2Nlc3NlZCBmcm9tIGh0dHBzOi8vd3d3Lmluc3RhY2FydC5jb20vZGF0YXNldHMvZ3JvY2VyeS1zaG9wcGluZy0yMDE3ICANCjIuIERhdGEgRGljdGlvbmFyeTogaHR0cHM6Ly9naXN0LmdpdGh1Yi5jb20vamVyZW15c3Rhbi9jM2IzOWQ5NDdkOWI4OGIzY2NmZjMxNDdkYmNmNmM2Yg==