Project: R Notebook Group Victor Grau, Jean Rocher, Alba Klipfel et Gautier Tubert
DATASETS : https://www.kaggle.com/datasets/grikomsn/amazon-cell-phones-reviews?resource=download&select=20191226-reviews.csv We have two datasets, one of reviews and the other one items.
We are doing an analysis of reviews when buying phones on Amazon. We are working for Samsung, and would like to know how people reacts to Samsung products. And how Smasung is positioned according to the competition.
# Load datasets
df_items = read.csv2("/Users/pierreemmanuelpichavant/Downloads/archive (1)/20191226-items.csv", sep = ",")
df_reviews = read.csv2("/Users/pierreemmanuelpichavant/Downloads/archive (1)/20191226-reviews.csv", sep = ",")
#structure of the two dataset
summary(df_items) asin brand title url
Length:720 Length:720 Length:720 Length:720
Class :character Class :character Class :character Class :character
Mode :character Mode :character Mode :character Mode :character
image rating reviewUrl totalReviews
Length:720 Length:720 Length:720 Min. : 1.0
Class :character Class :character Class :character 1st Qu.: 7.0
Mode :character Mode :character Mode :character Median : 32.0
Mean :105.7
3rd Qu.:122.2
Max. :983.0
price originalPrice
Length:720 Length:720
Class :character Class :character
Mode :character Mode :character
summary(df_reviews) asin name rating date
Length:67986 Length:67986 Min. :1.000 Length:67986
Class :character Class :character 1st Qu.:3.000 Class :character
Mode :character Mode :character Median :5.000 Mode :character
Mean :3.808
3rd Qu.:5.000
Max. :5.000
verified title body helpfulVotes
Length:67986 Length:67986 Length:67986 Min. : 1.00
Class :character Class :character Class :character 1st Qu.: 1.00
Mode :character Mode :character Mode :character Median : 2.00
Mean : 8.23
3rd Qu.: 5.00
Max. :990.00
NA's :40771 str(df_items)'data.frame': 720 obs. of 10 variables:
$ asin : chr "B0000SX2UC" "B0009N5L7K" "B000SKTZ0S" "B001AO4OUC" ...
$ brand : chr "" "Motorola" "Motorola" "Motorola" ...
$ title : chr "Dual-Band / Tri-Mode Sprint PCS Phone w/ Voice Activated Dialing & Bright White Backlit Screen" "Motorola I265 phone" "MOTOROLA C168i AT&T CINGULAR PREPAID GOPHONE CELL PHONE" "Motorola i335 Cell Phone Boost Mobile" ...
$ url : chr "https://www.amazon.com/Dual-Band-Tri-Mode-Activated-Dialing-Backlit/dp/B0000SX2UC" "https://www.amazon.com/Motorola-i265-I265-phone/dp/B0009N5L7K" "https://www.amazon.com/MOTOROLA-C168i-CINGULAR-PREPAID-GOPHONE/dp/B000SKTZ0S" "https://www.amazon.com/Motorola-i335-Phone-Boost-Mobile/dp/B001AO4OUC" ...
$ image : chr "https://m.media-amazon.com/images/I/2143EBQ210L._AC_UY218_ML3_.jpg" "https://m.media-amazon.com/images/I/419WBAVDARL._AC_UY218_ML3_.jpg" "https://m.media-amazon.com/images/I/71b+q3ydkIS._AC_UY218_ML3_.jpg" "https://m.media-amazon.com/images/I/710UO8gdT+L._AC_UY218_ML3_.jpg" ...
$ rating : chr "3" "3" "2.7" "3.3" ...
$ reviewUrl : chr "https://www.amazon.com/product-reviews/B0000SX2UC" "https://www.amazon.com/product-reviews/B0009N5L7K" "https://www.amazon.com/product-reviews/B000SKTZ0S" "https://www.amazon.com/product-reviews/B001AO4OUC" ...
$ totalReviews : int 14 7 22 21 12 3 29 18 208 352 ...
$ price : chr "0" "49.95" "99.99" "0" ...
$ originalPrice: chr "0" "0" "0" "0" ...Cleaning data
# Join the 2 datasets : items and reviews
# All items have images. It does not seem helpful to keep this variable.
columns_keeped = c("asin", "brand", "title", "price", "originalPrice")
df_whole = merge(df_items[,columns_keeped], df_reviews, by = "asin", suffixes = c("_item", "_review"))
# Clean data
# Remove empty brands
nlevels(unique(df_whole$brand))[1] 0unique(df_whole$brand) [1] "" "Motorola" "Nokia" "Samsung" "HUAWEI" "Sony"
[7] "Apple" "Google" "ASUS" "OnePlus" "Xiaomi" df_whole = df_whole[df_whole$brand != "",]
unique(df_whole$brand) [1] "Motorola" "Nokia" "Samsung" "HUAWEI" "Sony" "Apple"
[7] "Google" "ASUS" "OnePlus" "Xiaomi" # Remove rows with no price (the items in not available anymore ?)
table(df_whole$price[df_whole$price == 0])
0
11741 df_whole = df_whole[df_whole$price != 0,]
summary(df_whole) asin brand title_item price
Length:56045 Length:56045 Length:56045 Length:56045
Class :character Class :character Class :character Class :character
Mode :character Mode :character Mode :character Mode :character
originalPrice name rating date
Length:56045 Length:56045 Min. :1.000 Length:56045
Class :character Class :character 1st Qu.:3.000 Class :character
Mode :character Mode :character Median :5.000 Mode :character
Mean :3.823
3rd Qu.:5.000
Max. :5.000
verified title_review body helpfulVotes
Length:56045 Length:56045 Length:56045 Min. : 1.00
Class :character Class :character Class :character 1st Qu.: 1.00
Mode :character Mode :character Mode :character Median : 2.00
Mean : 8.72
3rd Qu.: 5.00
Max. :990.00
NA's :33766 df_whole$brand = as.factor(df_whole$brand)
df_whole$verified = as.factor(df_whole$verified)
# Run these lines if date conversion failed
lct <- Sys.getlocale("LC_TIME")
Sys.setlocale("LC_TIME", "C")[1] "C"df_whole$date = as.Date(df_whole$date, format = "%B %d, %Y")
df_whole$price = as.integer(df_whole$price)
df_whole$originalPrice = as.integer(df_whole$originalPrice)
df_whole$name = as.factor(df_whole$name)
df_whole$title_item = as.factor(df_whole$title_item)
df_whole$helpfulVotes[is.na(df_whole$helpfulVotes)] = 0
summary(df_whole) asin brand
Length:56045 Samsung :24959
Class :character Motorola: 6967
Mode :character Nokia : 5278
Apple : 5080
Xiaomi : 4346
Google : 3547
(Other) : 5868
title_item
Nokia Lumia 900 Black Factory Unlocked : 925
Samsung Galaxy Note 9 Factory Unlocked Phone with 6.4" Screen and 128GB (U.S. Warranty), Ocean Blue : 791
Samsung Galaxy S7 Edge G935FD 32GB Unlocked GSM 4G LTE : 775
Motorola G6 – 32 GB – Unlocked (AT&T/Sprint/T-Mobile/Verizon) – Deep Indigo - (U.S. Warranty) - PAAE0011US: 773
Samsung Galaxy Note 5 SM-N920V Gold 32GB (Verizon Wireless) : 773
Samsung Galaxy Note 5, Black 64GB (Verizon Wireless) : 773
(Other) :51235
price originalPrice name rating
Min. : 1.0 Min. : 0.0 Amazon Customer: 5260 Min. :1.000
1st Qu.:151.0 1st Qu.: 0.0 Kindle Customer: 286 1st Qu.:3.000
Median :209.0 Median : 0.0 Chris : 77 Median :5.000
Mean :267.2 Mean :100.5 John : 72 Mean :3.823
3rd Qu.:334.0 3rd Qu.: 1.0 Michael : 70 3rd Qu.:5.000
Max. :999.0 Max. :999.0 (Other) :50279 Max. :5.000
NA's : 1
date verified title_review body
Min. :2005-07-21 false: 5356 Length:56045 Length:56045
1st Qu.:2017-09-13 true :50689 Class :character Class :character
Median :2018-12-24 Mode :character Mode :character
Mean :2018-05-08
3rd Qu.:2019-08-06
Max. :2019-12-25
helpfulVotes
Min. : 0.000
1st Qu.: 0.000
Median : 0.000
Mean : 3.467
3rd Qu.: 1.000
Max. :990.000
# Check price distribution. So price are very low (sometimes 0). This is not always coherent. How determine incoherent prices ?
hist(df_whole$price)
boxplot(df_whole$price)
# After transformation, it seems more normally distributed.
hist(log(df_whole$price))
boxplot(log(df_whole$price))
# We can check IQR boudaries
log_price = log(df_whole$price)
price_quartiles = quantile(log_price, probs=c(.25, .75))
IQR_price <- IQR(log_price)
Lower = price_quartiles[1] - 1.5*IQR_price
Upper = price_quartiles[2] + 1.5*IQR_price
Lower 25%
3.84299 Upper 75%
6.992032 # No samples consered as outliers above the upper threshold
length(log_price[log_price > Upper])[1] 0# 437 samples considered as outliers
length(log_price[log_price < Lower])[1] 379# Back to base 10
Lower_b10 = exp(Lower)
# We can check some samples with prices below 46.6$
Lower_b10 25%
46.66479 df_whole_o = df_whole[df_whole$price < Lower_b10,]
df_whole_o$title_item = as.factor(df_whole_o$title_item)# We can see that imte titles are the same many times. We can also see 3 brands. Nokia, Samsung and Huawei. Nokia can have very
# low cost phones. But Samsung is known for premium phones...
summary(df_whole_o) asin brand
Length:379 Nokia :259
Class :character Samsung: 84
Mode :character HUAWEI : 36
Apple : 0
ASUS : 0
Google : 0
(Other): 0
title_item
Nokia 105 [2017] TA-1037 Only 2G Dual-Band (850/1900) Factory Unlocked Mobile Phone Black no warranty (White) :124
Nokia 105 RM-1135 Dual-Band (850/1900 MHz) Factory Unlocked Mobile Phone, Black, 2G Network Only. :123
Samsung Replenish Prepaid Android Phone (Boost Mobile) : 84
Huawei Glory Android Prepaid Phone (Net10) : 36
Nokia 106 Single Sim (2018) TA-1190 Dual-Band (850/1900) Factory GSM Unlocked Feature Phone (International Model): 12
Apple iPad 2 MC774LL/A Tablet (32GB, Wifi + AT&T 3G, Black) 2nd Generation (Renewed) : 0
(Other) : 0
price originalPrice name rating
Min. :14.00 Min. :14.00 Amazon Customer : 35 Min. :1.000
1st Qu.:27.00 1st Qu.:27.00 Daniel : 3 1st Qu.:1.000
Median :34.00 Median :34.00 40 - 40 : 2 Median :3.000
Mean :32.58 Mean :34.52 bargainshopper91: 2 Mean :2.992
3rd Qu.:46.00 3rd Qu.:52.00 G : 2 3rd Qu.:5.000
Max. :46.00 Max. :52.00 Luis A Molina : 2 Max. :5.000
(Other) :333
date verified title_review body
Min. :2012-01-27 false: 53 Length:379 Length:379
1st Qu.:2015-02-16 true :326 Class :character Class :character
Median :2017-07-15 Mode :character Mode :character
Mean :2016-11-06
3rd Qu.:2018-07-02
Max. :2019-12-24
helpfulVotes price.reduction price.log review
Min. : 0.000 Min. :0.00000 Min. :2.639 Length:379
1st Qu.: 0.000 1st Qu.:0.00000 1st Qu.:3.296 Class :character
Median : 0.000 Median :0.00000 Median :3.526 Mode :character
Mean : 1.984 Mean :0.03745 Mean :3.399
3rd Qu.: 1.000 3rd Qu.:0.11538 3rd Qu.:3.829
Max. :54.000 Max. :0.11538 Max. :3.829
review.sentiment review.sentiment.category ratings.category
Min. :-1.1250 Negative: 82 Negative:165
1st Qu.: 0.0000 Neutral : 31 Neutral : 37
Median : 0.2833 Positive:266 Positive:177
Mean : 0.2954
3rd Qu.: 0.5263
Max. : 2.1920
price.category
economical:379
medium : 0
high : 0
# So we can chack samsung phone. Ok, so the first item is a prepaid phone. But the 2 items following are a wireless charge and
# not phone. We can remove it. The keyword is Charging. And the third one is a Samsung Galaxy, so a premium phone. The price is not
# coherent
summary(df_whole_o[df_whole_o$brand == "Samsung", ]) asin brand
Length:84 Samsung :84
Class :character Apple : 0
Mode :character ASUS : 0
Google : 0
HUAWEI : 0
Motorola: 0
(Other) : 0
title_item
Samsung Replenish Prepaid Android Phone (Boost Mobile) :84
Apple iPad 2 MC774LL/A Tablet (32GB, Wifi + AT&T 3G, Black) 2nd Generation (Renewed) : 0
Apple iPad Air MF003LL/A (32GB, Wi-Fi + AT&T, Black with Space Gray) OLD VERSION (Renewed): 0
Apple iPad Air MF529LL/A (32GB, Wi-Fi + at&T, White with Silver) (Renewed) : 0
Apple iPad mini 4 (32GB, Wi-Fi + Cellular, Space Gray) (Renewed) : 0
Apple iPhone 11 Pro, 64GB, Fully Unlocked - Space Gray (Renewed) : 0
(Other) : 0
price originalPrice name rating
Min. :14 Min. :14 bargainshopper91: 2 Min. :1.000
1st Qu.:14 1st Qu.:14 Daniel : 2 1st Qu.:2.000
Median :14 Median :14 N. Stevens : 2 Median :4.000
Mean :14 Mean :14 321andme : 1 Mean :3.536
3rd Qu.:14 3rd Qu.:14 Alana : 1 3rd Qu.:5.000
Max. :14 Max. :14 Alex Moore : 1 Max. :5.000
(Other) :75
date verified title_review body
Min. :2012-01-27 false:20 Length:84 Length:84
1st Qu.:2012-05-07 true :64 Class :character Class :character
Median :2013-01-06 Mode :character Mode :character
Mean :2013-09-21
3rd Qu.:2015-01-04
Max. :2019-12-24
helpfulVotes price.reduction price.log review
Min. : 0.000 Min. :0 Min. :2.639 Length:84
1st Qu.: 0.000 1st Qu.:0 1st Qu.:2.639 Class :character
Median : 0.000 Median :0 Median :2.639 Mode :character
Mean : 1.869 Mean :0 Mean :2.639
3rd Qu.: 2.000 3rd Qu.:0 3rd Qu.:2.639
Max. :28.000 Max. :0 Max. :2.639
review.sentiment review.sentiment.category ratings.category
Min. :-0.58795 Negative:14 Negative:22
1st Qu.: 0.07969 Neutral : 2 Neutral :13
Median : 0.44765 Positive:68 Positive:49
Mean : 0.46580
3rd Qu.: 0.83354
Max. : 1.81990
price.category
economical:84
medium : 0
high : 0
View(df_whole_o[df_whole_o$brand == "Samsung", ])# So we can chack Huawei phone. 2 items. A prepaid phone --> ok. And a Modem... Not a phone. We have to remove it.
summary(df_whole_o[df_whole_o$brand == "HUAWEI", ]) asin brand
Length:36 HUAWEI :36
Class :character Apple : 0
Mode :character ASUS : 0
Google : 0
Motorola: 0
Nokia : 0
(Other) : 0
title_item
Huawei Glory Android Prepaid Phone (Net10) :36
Apple iPad 2 MC774LL/A Tablet (32GB, Wifi + AT&T 3G, Black) 2nd Generation (Renewed) : 0
Apple iPad Air MF003LL/A (32GB, Wi-Fi + AT&T, Black with Space Gray) OLD VERSION (Renewed): 0
Apple iPad Air MF529LL/A (32GB, Wi-Fi + at&T, White with Silver) (Renewed) : 0
Apple iPad mini 4 (32GB, Wi-Fi + Cellular, Space Gray) (Renewed) : 0
Apple iPhone 11 Pro, 64GB, Fully Unlocked - Space Gray (Renewed) : 0
(Other) : 0
price originalPrice name rating
Min. :27 Min. :27 A movie fan : 1 Min. :1.000
1st Qu.:27 1st Qu.:27 Adam R : 1 1st Qu.:1.000
Median :27 Median :27 Agrajag : 1 Median :3.000
Mean :27 Mean :27 Amazon Customer: 1 Mean :2.944
3rd Qu.:27 3rd Qu.:27 Amazon Kid : 1 3rd Qu.:4.250
Max. :27 Max. :27 Ancient DIY'er : 1 Max. :5.000
(Other) :30
date verified title_review body
Min. :2013-09-06 false:24 Length:36 Length:36
1st Qu.:2014-03-09 true :12 Class :character Class :character
Median :2014-11-21 Mode :character Mode :character
Mean :2014-10-25
3rd Qu.:2015-02-26
Max. :2017-09-24
helpfulVotes price.reduction price.log review
Min. : 0.000 Min. :0 Min. :3.296 Length:36
1st Qu.: 0.000 1st Qu.:0 1st Qu.:3.296 Class :character
Median : 1.000 Median :0 Median :3.296 Mode :character
Mean : 5.528 Mean :0 Mean :3.296
3rd Qu.: 6.000 3rd Qu.:0 3rd Qu.:3.296
Max. :39.000 Max. :0 Max. :3.296
review.sentiment review.sentiment.category ratings.category
Min. :-0.5534 Negative:13 Negative:13
1st Qu.:-0.1272 Neutral : 1 Neutral : 9
Median : 0.2147 Positive:22 Positive:14
Mean : 0.2546
3rd Qu.: 0.5433
Max. : 1.2690
price.category
economical:36
medium : 0
high : 0
summary(df_whole_o[df_whole_o$brand == "Nokia", ]) asin brand
Length:259 Nokia :259
Class :character Apple : 0
Mode :character ASUS : 0
Google : 0
HUAWEI : 0
Motorola: 0
(Other) : 0
title_item
Nokia 105 [2017] TA-1037 Only 2G Dual-Band (850/1900) Factory Unlocked Mobile Phone Black no warranty (White) :124
Nokia 105 RM-1135 Dual-Band (850/1900 MHz) Factory Unlocked Mobile Phone, Black, 2G Network Only. :123
Nokia 106 Single Sim (2018) TA-1190 Dual-Band (850/1900) Factory GSM Unlocked Feature Phone (International Model): 12
Apple iPad 2 MC774LL/A Tablet (32GB, Wifi + AT&T 3G, Black) 2nd Generation (Renewed) : 0
Apple iPad Air MF003LL/A (32GB, Wi-Fi + AT&T, Black with Space Gray) OLD VERSION (Renewed) : 0
Apple iPad Air MF529LL/A (32GB, Wi-Fi + at&T, White with Silver) (Renewed) : 0
(Other) : 0
price originalPrice name
Min. :27.00 Min. :27.00 Amazon Customer : 33
1st Qu.:34.00 1st Qu.:34.00 40 - 40 : 2
Median :34.00 Median :34.00 Luis A Molina : 2
Mean :39.37 Mean :42.22 876jgfdtudyu6 : 1
3rd Qu.:46.00 3rd Qu.:52.00 A non-conforming customer: 1
Max. :46.00 Max. :52.00 Adeoye Rashidat Olaide : 1
(Other) :219
rating date verified title_review
Min. :1.000 Min. :2016-01-10 false: 9 Length:259
1st Qu.:1.000 1st Qu.:2017-04-18 true :250 Class :character
Median :2.000 Median :2018-01-15 Mode :character
Mean :2.822 Mean :2018-02-23
3rd Qu.:5.000 3rd Qu.:2019-01-02
Max. :5.000 Max. :2019-12-20
body helpfulVotes price.reduction price.log
Length:259 Min. : 0.000 Min. :0.0000 Min. :3.296
Class :character 1st Qu.: 0.000 1st Qu.:0.0000 1st Qu.:3.526
Mode :character Median : 0.000 Median :0.0000 Median :3.526
Mean : 1.529 Mean :0.0548 Mean :3.659
3rd Qu.: 1.000 3rd Qu.:0.1154 3rd Qu.:3.829
Max. :54.000 Max. :0.1154 Max. :3.829
review review.sentiment review.sentiment.category
Length:259 Min. :-1.1250 Negative: 55
Class :character 1st Qu.: 0.0000 Neutral : 28
Mode :character Median : 0.2449 Positive:176
Mean : 0.2459
3rd Qu.: 0.4497
Max. : 2.1920
ratings.category price.category
Negative:130 economical:259
Neutral : 15 medium : 0
Positive:114 high : 0
To summarize, the outliers concerned often prepaid phones. We can keep it for our analysis. But some items are not phones or are incoherent.
We have to remove: Items with charge as keyworkds –> does not work Items with modem* as keywords –> does not work. We can check the device type through the html, and enrich the data by adding the product category Samsung Galaxy with price lower than the lower log price outliers threshold
summary(df_whole[grepl("charg", df_whole$title_item, ignore.case = TRUE),]) asin brand
Length:153 Samsung:96
Class :character Nokia :57
Mode :character Apple : 0
ASUS : 0
Google : 0
HUAWEI : 0
(Other): 0
title_item
Nokia 9 PureView - Android 9.0 Pie - 128 GB - Single Sim Unlocked Smartphone (at&T/T-Mobile/Metropcs/Cricket/H2O) - 5.99" QHD+ Screen - Qi Wireless Charging - Midnight Blue - U.S. Warranty :57
Samsung Galaxy A70 (128GB, 6GB RAM) 6.7" in-Screen Fingerprint, 25W Super-Fast Charger, US + Global 4G LTE GSM Unlocked International Model A705MN/DS (Black, 128GB + 128GB SD + Case Bundle) :49
Samsung Galaxy A20 (32GB, 3GB RAM) 6.4" Super AMOLED, Infinity-V Display, Fast Charge 4000mAh Battery, US & Global 4G LTE Dual SIM GSM Factory Unlocked A205G/DS - International Model (Black, 32GB):37
Samsung Galaxy A20 (32GB, 3GB RAM) 6.4" Super AMOLED, Fast Charge 4000mAh Battery, US & Global 4G LTE GSM Factory Unlocked A205G - International Model (Black, 32GB + 32GB SD Bundle (Single SIM)) :10
Apple iPad 2 MC774LL/A Tablet (32GB, Wifi + AT&T 3G, Black) 2nd Generation (Renewed) : 0
Apple iPad Air MF003LL/A (32GB, Wi-Fi + AT&T, Black with Space Gray) OLD VERSION (Renewed) : 0
(Other) : 0
price originalPrice name
Min. :195.0 Min. :195 Amazon Customer : 11
1st Qu.:197.0 1st Qu.:197 Nick : 2
Median :369.0 Median :369 A little short for a storm trooper: 1
Mean :364.5 Mean :439 Achint : 1
3rd Qu.:499.0 3rd Qu.:699 Ai Technologies : 1
Max. :499.0 Max. :699 Aiden : 1
(Other) :136
rating date verified title_review
Min. :1.00 Min. :2019-03-01 false: 19 Length:153
1st Qu.:4.00 1st Qu.:2019-06-30 true :134 Class :character
Median :5.00 Median :2019-10-09 Mode :character
Mean :4.15 Mean :2019-09-03
3rd Qu.:5.00 3rd Qu.:2019-11-18
Max. :5.00 Max. :2019-12-23
body helpfulVotes price.reduction price.log
Length:153 Min. : 0.000 Min. :0.0000 Min. :5.273
Class :character 1st Qu.: 0.000 1st Qu.:0.0000 1st Qu.:5.283
Mode :character Median : 1.000 Median :0.0000 Median :5.911
Mean : 4.843 Mean :0.1066 Mean :5.830
3rd Qu.: 3.000 3rd Qu.:0.2861 3rd Qu.:6.213
Max. :280.000 Max. :0.2861 Max. :6.213
review review.sentiment review.sentiment.category
Length:153 Min. :-0.74845 Negative: 24
Class :character 1st Qu.: 0.06666 Neutral : 10
Mode :character Median : 0.55000 Positive:119
Mean : 0.51856
3rd Qu.: 0.87500
Max. : 1.83122
ratings.category price.category
Negative: 21 economical: 47
Neutral : 11 medium :106
Positive:121 high : 0
summary(df_whole[grepl("modem", df_whole$title_item, ignore.case = TRUE),]) asin brand
Length:1 HUAWEI :1
Class :character Apple :0
Mode :character ASUS :0
Google :0
Motorola:0
Nokia :0
(Other) :0
title_item
Huawei E8372h-608 Unlocked 150 Mbps 4G LTE Modem + WiFi USB Wingle (4G LTE in USA (AT&T), (2G Tmobile Metro PCS) Digitel Europe, Asia, Middle East and Africa) (Unit + 2 Antennas):1
Apple iPad 2 MC774LL/A Tablet (32GB, Wifi + AT&T 3G, Black) 2nd Generation (Renewed) :0
Apple iPad Air MF003LL/A (32GB, Wi-Fi + AT&T, Black with Space Gray) OLD VERSION (Renewed) :0
Apple iPad Air MF529LL/A (32GB, Wi-Fi + at&T, White with Silver) (Renewed) :0
Apple iPad mini 4 (32GB, Wi-Fi + Cellular, Space Gray) (Renewed) :0
Apple iPhone 11 Pro, 64GB, Fully Unlocked - Space Gray (Renewed) :0
(Other) :0
price originalPrice name rating
Min. :79 Min. :79 Justen f. :1 Min. :5
1st Qu.:79 1st Qu.:79 _ :0 1st Qu.:5
Median :79 Median :79 __island.kiddo:0 Median :5
Mean :79 Mean :79 _esanna :0 Mean :5
3rd Qu.:79 3rd Qu.:79 _MiMorales_ :0 3rd Qu.:5
Max. :79 Max. :79 -_- :0 Max. :5
(Other) :0
date verified title_review body
Min. :2019-08-22 false:0 Length:1 Length:1
1st Qu.:2019-08-22 true :1 Class :character Class :character
Median :2019-08-22 Mode :character Mode :character
Mean :2019-08-22
3rd Qu.:2019-08-22
Max. :2019-08-22
helpfulVotes price.reduction price.log review
Min. :0 Min. :0 Min. :4.369 Length:1
1st Qu.:0 1st Qu.:0 1st Qu.:4.369 Class :character
Median :0 Median :0 Median :4.369 Mode :character
Mean :0 Mean :0 Mean :4.369
3rd Qu.:0 3rd Qu.:0 3rd Qu.:4.369
Max. :0 Max. :0 Max. :4.369
review.sentiment review.sentiment.category ratings.category
Min. :0.6667 Negative:0 Negative:0
1st Qu.:0.6667 Neutral :0 Neutral :0
Median :0.6667 Positive:1 Positive:1
Mean :0.6667
3rd Qu.:0.6667
Max. :0.6667
price.category
economical:1
medium :0
high :0
# Remove charges items by hand
unwanted_items = c("Samsung DeX Wireless Qi Desktop Charging Dock Station EE-MG950 Galaxy S8 + Note8 (Renewed)",
"Samsung Qi Certified Fast Charge Wireless Charging Pad for Qi Compatible Smartphones with Built-in Cool Fan - Retail Packaging - Black",
"Samsung Galaxy S10+ Plus Verizon + GSM Unlocked 512GB Ceramic Black",
"Modem Huawei USB 3G H+ GSM Unlocked K4605 (E372) USA Latin & Caribbean Europe Bands 850/900/1900 mhz BAM")
df_whole = df_whole[df_whole$title_item %in% unwanted_items == FALSE,]# If original price is not set to 0, we can consider a price reduction. Price column is the real price. The price difference is
# original_price - price. If original price is 0, we don't have any reduction.
df_whole$originalPrice = apply(df_whole[,c("price", "originalPrice")], 1, function(x){if (x["originalPrice"] == 0) return (x["price"]) else (return(x["originalPrice"]))})
View(df_whole)
# Add price reduction column
df_whole$price.reduction = apply(df_whole[,c("price", "originalPrice")], 1, function(x){return((x["originalPrice"] - x["price"])/x["originalPrice"])})
#Some inconsistency
nrow(df_whole[df_whole$price.reduction < 0,])[1] 0View(df_whole[df_whole$price.reduction < 0,])
# Correct these inconsistencies
df_whole[df_whole$price.reduction < 0,]$originalPrice = df_whole[df_whole$price.reduction < 0,]$price
df_whole[df_whole$price.reduction < 0,]$price.reduction = 0Erreur dans `$<-.data.frame`(`*tmp*`, price.reduction, value = 0) :
le tableau de remplacement a 1 lignes, le tableau remplacé en a 0# We can transform with log
hist(df_whole$price)
boxplot(df_whole$price)
# Much better because it's closer to a normal distribution
hist(log(df_whole$price))
boxplot(log(df_whole$price))
df_whole$price.log = log(df_whole$price)# We can transform with log
hist(df_whole$price.reduction)
boxplot(df_whole$price.reduction)
hist(log(df_whole$price.reduction))
boxplot(log(df_whole$price.reduction))
hist(df_whole$rating)
boxplot(df_whole$rating)
hist(log(df_whole$rating))
boxplot(log(df_whole$rating))
hist(df_whole$helpfulVotes)
boxplot(df_whole$helpfulVotes)
hist(log(df_whole$helpfulVotes))
boxplot(log(df_whole$helpfulVotes))
summary(df_whole) asin brand
Length:55987 Samsung :24904
Class :character Motorola: 6967
Mode :character Nokia : 5278
Apple : 5080
Xiaomi : 4346
Google : 3547
(Other) : 5865
title_item
Nokia Lumia 900 Black Factory Unlocked : 925
Samsung Galaxy Note 9 Factory Unlocked Phone with 6.4" Screen and 128GB (U.S. Warranty), Ocean Blue : 791
Samsung Galaxy S7 Edge G935FD 32GB Unlocked GSM 4G LTE : 775
Motorola G6 – 32 GB – Unlocked (AT&T/Sprint/T-Mobile/Verizon) – Deep Indigo - (U.S. Warranty) - PAAE0011US: 773
Samsung Galaxy Note 5 SM-N920V Gold 32GB (Verizon Wireless) : 773
Samsung Galaxy Note 5, Black 64GB (Verizon Wireless) : 773
(Other) :51177
price originalPrice name rating
Min. : 14.0 Min. : 14.0 Amazon Customer: 5256 Min. :1.000
1st Qu.:152.0 1st Qu.:163.0 Kindle Customer: 286 1st Qu.:3.000
Median :209.0 Median :229.0 Chris : 77 Median :5.000
Mean :267.5 Mean :295.3 John : 71 Mean :3.823
3rd Qu.:334.0 3rd Qu.:370.0 Michael : 70 3rd Qu.:5.000
Max. :999.0 Max. :999.0 (Other) :50226 Max. :5.000
NA's : 1
date verified title_review body
Min. :2005-07-21 false: 5355 Length:55987 Length:55987
1st Qu.:2017-09-12 true :50632 Class :character Class :character
Median :2018-12-24 Mode :character Mode :character
Mean :2018-05-07
3rd Qu.:2019-08-05
Max. :2019-12-25
helpfulVotes price.reduction price.log review
Min. : 0.000 Min. :0.00000 Min. :2.639 Length:55987
1st Qu.: 0.000 1st Qu.:0.00000 1st Qu.:5.024 Class :character
Median : 0.000 Median :0.00000 Median :5.342 Mode :character
Mean : 3.469 Mean :0.06558 Mean :5.402
3rd Qu.: 1.000 3rd Qu.:0.00000 3rd Qu.:5.811
Max. :990.000 Max. :0.75721 Max. :6.907
review.sentiment review.sentiment.category ratings.category
Min. :-2.03200 Negative: 9007 Negative:13530
1st Qu.: 0.03491 Neutral : 4394 Neutral : 3888
Median : 0.41542 Positive:42586 Positive:38569
Mean : 0.43166
3rd Qu.: 0.75260
Max. : 3.85155
price.category
economical:42249
medium :11537
high : 2201
barplot(prop.table(table(df_whole$verified)))
barplot(prop.table(table(df_whole$brand)), las = 2)
Textual analysis
# Reviews analysis
install.packages(rlang)Error in install.packages : objet 'rlang' introuvablelibrary(wordcloud)Le chargement a nécessité le package : RColorBrewerlibrary(tidyr)
library(tidyverse)Registered S3 methods overwritten by 'dbplyr':
method from
print.tbl_lazy
print.tbl_sql
── Attaching packages ──────────────────────────────────── tidyverse 1.3.2 ──✔ ggplot2 3.3.6 ✔ dplyr 1.0.10
✔ tibble 3.1.8 ✔ stringr 1.4.1
✔ readr 2.1.2 ✔ forcats 0.5.2
✔ purrr 0.3.4 ── Conflicts ─────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::between() masks maditr::between()
✖ dplyr::coalesce() masks maditr::coalesce()
✖ readr::cols() masks maditr::cols()
✖ dplyr::filter() masks stats::filter()
✖ dplyr::first() masks maditr::first()
✖ dplyr::lag() masks stats::lag()
✖ dplyr::last() masks maditr::last()
✖ purrr::transpose() masks maditr::transpose()library(stopwords)
# Merge title review and body
df_whole$review = paste(df_whole$title_review, df_whole$body)
stopwords_phone = c(stopwords("english"),"phone","Samsung","Nokia","Apple","ASUS","OnePlus","Motorola","HUAWEI","Sony","Google","Xiaomi","great","like","good",
"samsung","nokia","iphone","apple","asus","oneplus","motorola","huawei","sony","google","xiaomi", "phones", "just")
library(dplyr)
library(tidytext)
library(stopwords)
library(ggplot2)
df_count_words = df_whole %>%
select(brand,review) %>%
unnest_tokens(input=review,output=word) %>%
count(brand,word,sort=T) %>%
filter(nchar(word)>3) %>%
filter(!word %in% stopwords_phone) %>%
group_by(brand)
df_count_words = df_count_words %>%
anti_join(stop_words)Joining, by = "word"df_count_words %>%
top_n(n=10,n)%>%
mutate(word = reorder(word, n)) %>%
ggplot(aes(x=word, y=n,fill=brand)) +
geom_col(show.legend=F,col="black")+
facet_wrap(~brand, ncol=2,scales="free")+
xlab(NULL) +
ylab("Count")+
ggtitle("Most Common Words in reviews by Brand") +
coord_flip()
nlevels(unique(df_count_words$brand))[1] 10install.packages(XQ)Error in install.packages : objet 'XQ' introuvablelibrary("sentimentr")
library(tm)Le chargement a nécessité le package : NLP
Attachement du package : ‘NLP’
L'objet suivant est masqué depuis ‘package:ggplot2’:
annotate
Attachement du package : ‘tm’
L'objet suivant est masqué depuis ‘package:stopwords’:
stopwordsdocs = Corpus(VectorSource(df_whole$review))
# Lower case transform
docs = tm_map(docs, content_transformer(tolower))Avis : transformation drops documents# Remove numbers
docs = tm_map(docs, removeNumbers)Avis : transformation drops documentsdocs = tm_map(docs, removeWords, stopwords("english"))Avis : transformation drops documents# Delete punctuation
docs = tm_map(docs, removePunctuation)Avis : transformation drops documents# below 0 --> negative, 0 --> neutral, above 0 --> positive
df_whole$review.sentiment = sentiment_by(docs$content)$ave_sentimentAvis : Each time `sentiment_by` is run it has to do sentence boundary disambiguation when a
raw `character` vector is passed to `text.var`. This may be costly of time and
memory. It is highly recommended that the user first runs the raw `character`
vector through the `get_sentences` function.# We can verify the correlation between the ratings and the predicted sentiments
df_whole$review.sentiment.category = sapply(df_whole$review.sentiment, function(x){if (x<0) return("Negative") else if (x>0) return("Positive") else return("Neutral")})
df_whole$review.sentiment.category = as.factor(df_whole$review.sentiment.category)
barplot(prop.table(table(df_whole$review.sentiment.category)))
##### Positive words
df_count_words_positive = df_whole[df_whole$review.sentiment.category=="Positive",] %>%
select(brand,review) %>%
unnest_tokens(input=review,output=word) %>%
count(brand,word,sort=T) %>%
filter(nchar(word)>3) %>%
filter(!word %in% stopwords_phone) %>%
group_by(brand)
df_count_words_positive = df_count_words_positive %>%
anti_join(stop_words)Joining, by = "word"df_count_words_positive %>%
top_n(n=10,n)%>%
mutate(word = reorder(word, n)) %>%
ggplot(aes(x=word, y=n,fill=brand)) +
geom_col(show.legend=F,col="black")+
facet_wrap(~brand, ncol=2,scales="free")+
xlab(NULL) +
ylab("Count")+
ggtitle("Most Common Words in positive reviews by Brand") +
coord_flip()
##### Negative words
df_count_words_negative = df_whole[df_whole$review.sentiment.category=="Negative",] %>%
select(brand,review) %>%
unnest_tokens(input=review,output=word) %>%
count(brand,word,sort=T) %>%
filter(nchar(word)>3) %>%
filter(!word %in% stopwords_phone) %>%
group_by(brand)
df_count_words_negative = df_count_words_negative %>%
anti_join(stop_words)Joining, by = "word"df_count_words_negative %>%
top_n(n=10,n)%>%
mutate(word = reorder(word, n)) %>%
ggplot(aes(x=word, y=n,fill=brand)) +
geom_col(show.legend=F,col="black")+
facet_wrap(~brand, ncol=2,scales="free")+
xlab(NULL) +
ylab("Count")+
ggtitle("Most Common Words in negative reviews by Brand") +
coord_flip()
Not so much differences between negative and positive sentiments.
# Most common words by sentiments
df_count_words_positive = df_whole %>%
select(review.sentiment.category,review) %>%
unnest_tokens(input=review,output=word) %>%
count(review.sentiment.category,word,sort=T) %>%
filter(nchar(word)>3) %>%
filter(!word %in% stopwords_phone) %>%
group_by(review.sentiment.category)
df_count_words_positive = df_count_words_positive %>%
anti_join(stop_words)Joining, by = "word"df_count_words_positive %>%
top_n(n=10,n)%>%
mutate(word = reorder(word, n)) %>%
ggplot(aes(x=word, y=n,fill=review.sentiment.category)) +
geom_col(show.legend=F,col="black")+
facet_wrap(~review.sentiment.category, ncol=2,scales="free")+
xlab(NULL) +
ylab("Count")+
ggtitle("Most Common Words in positive reviews") +
coord_flip()
df_whole$ratings.category = as.factor(sapply(df_whole$rating, function(x){if (x<3) return("Negative") else if (x>3) return("Positive") else return("Neutral")}))
### Analysis by ratings
df_count_words = df_whole %>%
select(rating,review) %>%
unnest_tokens(input=review,output=word) %>%
count(rating,word,sort=T) %>%
filter(nchar(word)>3) %>%
filter(!word %in% stopwords_phone) %>%
group_by(rating)
df_count_words = df_count_words %>%
anti_join(stop_words)Joining, by = "word"df_count_words %>%
top_n(n=10,n)%>%
mutate(word = reorder(word, n)) %>%
ggplot(aes(x=word, y=n,fill=rating)) +
geom_col(show.legend=F,col="black")+
facet_wrap(~rating, ncol=2,scales="free")+
xlab(NULL) +
ylab("Count")+
ggtitle("Most Common Words in by brands") +
coord_flip()
The word price is not present in the most common words for the 1 and 2 stars rating. So it does not seem to be an issue.
#### Analysis by price
df_whole$price.category = cut(df_whole$price, breaks = 3, labels=c("economical", "medium", "high"))
barplot(prop.table(table(df_whole$price.category)))
df_count_words = df_whole %>%
select(price.category,review) %>%
unnest_tokens(input=review,output=word) %>%
count(price.category,word,sort=T) %>%
filter(nchar(word)>3) %>%
filter(!word %in% stopwords_phone) %>%
group_by(price.category)
df_count_words = df_count_words %>%
anti_join(stop_words)Joining, by = "word"df_count_words %>%
top_n(n=10,n)%>%
mutate(word = reorder(word, n)) %>%
ggplot(aes(x=word, y=n,fill=price.category)) +
geom_col(show.legend=F,col="black")+
facet_wrap(~price.category, ncol=2,scales="free")+
xlab(NULL) +
ylab("Count")+
ggtitle("Most Common Words by prices") +
coord_flip()
Always screen and battery as most common words. But not real differences between prices.
Recommender engine
We’ve built a ollaborative content base filtering. (cf netflix svd algorithm) You have in input: the product_id and the customer_id The Output : how much the product fit with the customer. Base one the previous customer notes.
Here is an explanation of the recosystem package : https://cran.r-project.org/web/packages/recosystem/readme/README.html
library(maditr)
df_whole_reco_engine = na.omit(df_whole[,c("asin", "name", "rating")])
df_whole_reco_engine$name_id = as.numeric(as.factor(df_whole_reco_engine$name))
df_whole_reco_engine$asin_id = as.numeric(as.factor(df_whole_reco_engine$asin))
df_whole_reco_engine$test = sample(c(rep(0, 0.8 * (nrow(df_whole_reco_engine)+1)), rep(1, 0.2 * (nrow(df_whole_reco_engine)+1))))
table(df_whole_reco_engine$test)
0 1
44789 11197 train_set = df_whole_reco_engine[df_whole_reco_engine$test == 0,]
test_set = df_whole_reco_engine[df_whole_reco_engine$test == 1,]
library(recosystem)
r = Reco()
train_set_datasource = data_memory(user_index = train_set$name_id, item_index = train_set$asin_id, rating = train_set$rating)
test_set_datasource = data_memory(user_index = test_set$name_id, item_index = test_set$asin_id, rating = test_set$rating)
r$train(train_set_datasource, opts = c(nthread = 1, niter = 20))iter tr_rmse obj
0 2.5210 3.4991e+05
1 1.2047 1.2365e+05
2 0.9660 9.9280e+04
3 0.8855 9.0443e+04
4 0.8500 8.5949e+04
5 0.8286 8.2660e+04
6 0.8146 8.0566e+04
7 0.8029 7.8577e+04
8 0.7929 7.7001e+04
9 0.7848 7.5311e+04
10 0.7776 7.4218e+04
11 0.7704 7.3075e+04
12 0.7638 7.2020e+04
13 0.7574 7.1185e+04
14 0.7508 7.0142e+04
15 0.7447 6.9298e+04
16 0.7393 6.8660e+04
17 0.7340 6.8031e+04
18 0.7282 6.7343e+04
19 0.7233 6.6818e+04pred_vec = r$predict(test_set_datasource, out_memory())
test_set$prediction = pred_vec## Compute rmse to evaluate the model quality
rmse = sqrt(mean((test_set$rating - test_set$prediction)^2))
print(rmse)[1] 1.552702We have a rmse of 1.5 which seems not so good on a 5 levels rating.
mae = mean(abs(test_set$rating - test_set$prediction))
# In average, the difference between the prediction and the real note is 1.3.
print(mae)[1] 1.282274# Using our model, we can make some predictions# Item; B07YQ58NPF, id: 571
# User: Alison Strittmatter, id: 1557
df_whole_reco_engine[df_whole_reco_engine$name == "Alison Strittmatter",]df_whole_reco_engine[df_whole_reco_engine$asin == "B07YQ58NPF",][1,]
pred_set_datasource = data_memory(user_index = c(1557), item_index = c(571))
pred_vec = r$predict(pred_set_datasource, out_memory())
# 4 stars
pred_vec[1] 4.070508################################################################################################
LS0tCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgZGZfcHJpbnQ6IHBhZ2VkCi0tLQoKUHJvamVjdDogUiBOb3RlYm9vayBHcm91cCBWaWN0b3IgR3JhdSwgSmVhbiBSb2NoZXIsIEFsYmEgS2xpcGZlbCBldCBHYXV0aWVyIFR1YmVydAoKREFUQVNFVFMgOiA8aHR0cHM6Ly93d3cua2FnZ2xlLmNvbS9kYXRhc2V0cy9ncmlrb21zbi9hbWF6b24tY2VsbC1waG9uZXMtcmV2aWV3cz9yZXNvdXJjZT1kb3dubG9hZCZzZWxlY3Q9MjAxOTEyMjYtcmV2aWV3cy5jc3Y+IFdlIGhhdmUgdHdvIGRhdGFzZXRzLCBvbmUgb2YgcmV2aWV3cyBhbmQgdGhlIG90aGVyIG9uZSBpdGVtcy4KCldlIGFyZSBkb2luZyBhbiBhbmFseXNpcyBvZiByZXZpZXdzIHdoZW4gYnV5aW5nIHBob25lcyBvbiBBbWF6b24uIFdlIGFyZSB3b3JraW5nIGZvciBTYW1zdW5nLCBhbmQgd291bGQgbGlrZSB0byBrbm93IGhvdyBwZW9wbGUgcmVhY3RzIHRvIFNhbXN1bmcgcHJvZHVjdHMuIEFuZCBob3cgU21hc3VuZyBpcyBwb3NpdGlvbmVkIGFjY29yZGluZyB0byB0aGUgY29tcGV0aXRpb24uCgpgYGB7cn0KIyBMb2FkIGRhdGFzZXRzCmRmX2l0ZW1zID0gcmVhZC5jc3YyKCIvVXNlcnMvcGllcnJlZW1tYW51ZWxwaWNoYXZhbnQvRG93bmxvYWRzL2FyY2hpdmUgKDEpLzIwMTkxMjI2LWl0ZW1zLmNzdiIsIHNlcCA9ICIsIikKZGZfcmV2aWV3cyA9IHJlYWQuY3N2MigiL1VzZXJzL3BpZXJyZWVtbWFudWVscGljaGF2YW50L0Rvd25sb2Fkcy9hcmNoaXZlICgxKS8yMDE5MTIyNi1yZXZpZXdzLmNzdiIsIHNlcCA9ICIsIikKCiNzdHJ1Y3R1cmUgb2YgdGhlIHR3byBkYXRhc2V0CnN1bW1hcnkoZGZfaXRlbXMpCnN1bW1hcnkoZGZfcmV2aWV3cykKCnN0cihkZl9pdGVtcykKYGBgCgojICoqQ2xlYW5pbmcgZGF0YSoqCgpgYGB7cn0KIyBKb2luIHRoZSAyIGRhdGFzZXRzIDogaXRlbXMgYW5kIHJldmlld3MKIyBBbGwgaXRlbXMgaGF2ZSBpbWFnZXMuIEl0IGRvZXMgbm90IHNlZW0gaGVscGZ1bCB0byBrZWVwIHRoaXMgdmFyaWFibGUuIApjb2x1bW5zX2tlZXBlZCA9IGMoImFzaW4iLCAiYnJhbmQiLCAidGl0bGUiLCAicHJpY2UiLCAib3JpZ2luYWxQcmljZSIpCmRmX3dob2xlID0gbWVyZ2UoZGZfaXRlbXNbLGNvbHVtbnNfa2VlcGVkXSwgZGZfcmV2aWV3cywgYnkgPSAiYXNpbiIsIHN1ZmZpeGVzID0gYygiX2l0ZW0iLCAiX3JldmlldyIpKQoKIyBDbGVhbiBkYXRhCgojIFJlbW92ZSBlbXB0eSBicmFuZHMKbmxldmVscyh1bmlxdWUoZGZfd2hvbGUkYnJhbmQpKQp1bmlxdWUoZGZfd2hvbGUkYnJhbmQpCmRmX3dob2xlID0gZGZfd2hvbGVbZGZfd2hvbGUkYnJhbmQgIT0gIiIsXQp1bmlxdWUoZGZfd2hvbGUkYnJhbmQpCgoKIyBSZW1vdmUgcm93cyB3aXRoIG5vIHByaWNlICh0aGUgaXRlbXMgaW4gbm90IGF2YWlsYWJsZSBhbnltb3JlID8pCnRhYmxlKGRmX3dob2xlJHByaWNlW2RmX3dob2xlJHByaWNlID09IDBdKQpkZl93aG9sZSA9IGRmX3dob2xlW2RmX3dob2xlJHByaWNlICE9IDAsXQpzdW1tYXJ5KGRmX3dob2xlKQoKZGZfd2hvbGUkYnJhbmQgPSBhcy5mYWN0b3IoZGZfd2hvbGUkYnJhbmQpCmRmX3dob2xlJHZlcmlmaWVkID0gYXMuZmFjdG9yKGRmX3dob2xlJHZlcmlmaWVkKQoKIyBSdW4gdGhlc2UgbGluZXMgaWYgZGF0ZSBjb252ZXJzaW9uIGZhaWxlZApsY3QgPC0gU3lzLmdldGxvY2FsZSgiTENfVElNRSIpClN5cy5zZXRsb2NhbGUoIkxDX1RJTUUiLCAiQyIpCmRmX3dob2xlJGRhdGUgPSBhcy5EYXRlKGRmX3dob2xlJGRhdGUsIGZvcm1hdCA9ICIlQiAlZCwgJVkiKQoKZGZfd2hvbGUkcHJpY2UgPSBhcy5pbnRlZ2VyKGRmX3dob2xlJHByaWNlKQpkZl93aG9sZSRvcmlnaW5hbFByaWNlID0gYXMuaW50ZWdlcihkZl93aG9sZSRvcmlnaW5hbFByaWNlKQpkZl93aG9sZSRuYW1lID0gYXMuZmFjdG9yKGRmX3dob2xlJG5hbWUpCmRmX3dob2xlJHRpdGxlX2l0ZW0gPSBhcy5mYWN0b3IoZGZfd2hvbGUkdGl0bGVfaXRlbSkKCmRmX3dob2xlJGhlbHBmdWxWb3Rlc1tpcy5uYShkZl93aG9sZSRoZWxwZnVsVm90ZXMpXSA9IDAKCnN1bW1hcnkoZGZfd2hvbGUpCmBgYAoKYGBge3J9CiMgQ2hlY2sgcHJpY2UgZGlzdHJpYnV0aW9uLiBTbyBwcmljZSBhcmUgdmVyeSBsb3cgKHNvbWV0aW1lcyAwKS4gVGhpcyBpcyBub3QgYWx3YXlzIGNvaGVyZW50LiBIb3cgZGV0ZXJtaW5lIGluY29oZXJlbnQgcHJpY2VzID8gCmhpc3QoZGZfd2hvbGUkcHJpY2UpCmJveHBsb3QoZGZfd2hvbGUkcHJpY2UpCmBgYAoKYGBge3J9CiMgQWZ0ZXIgdHJhbnNmb3JtYXRpb24sIGl0IHNlZW1zIG1vcmUgbm9ybWFsbHkgZGlzdHJpYnV0ZWQuIApoaXN0KGxvZyhkZl93aG9sZSRwcmljZSkpCmJveHBsb3QobG9nKGRmX3dob2xlJHByaWNlKSkKYGBgCgpgYGB7cn0KIyBXZSBjYW4gY2hlY2sgSVFSIGJvdWRhcmllcwpsb2dfcHJpY2UgPSBsb2coZGZfd2hvbGUkcHJpY2UpCnByaWNlX3F1YXJ0aWxlcyA9IHF1YW50aWxlKGxvZ19wcmljZSwgcHJvYnM9YyguMjUsIC43NSkpCklRUl9wcmljZSA8LSBJUVIobG9nX3ByaWNlKQpMb3dlciA9IHByaWNlX3F1YXJ0aWxlc1sxXSAtIDEuNSpJUVJfcHJpY2UKVXBwZXIgPSBwcmljZV9xdWFydGlsZXNbMl0gKyAxLjUqSVFSX3ByaWNlCkxvd2VyClVwcGVyCmBgYAoKYGBge3J9CiMgTm8gc2FtcGxlcyBjb25zZXJlZCBhcyBvdXRsaWVycyBhYm92ZSB0aGUgdXBwZXIgdGhyZXNob2xkCmxlbmd0aChsb2dfcHJpY2VbbG9nX3ByaWNlID4gVXBwZXJdKQoKIyA0Mzcgc2FtcGxlcyBjb25zaWRlcmVkIGFzIG91dGxpZXJzIApsZW5ndGgobG9nX3ByaWNlW2xvZ19wcmljZSA8IExvd2VyXSkKCiMgQmFjayB0byBiYXNlIDEwCkxvd2VyX2IxMCA9IGV4cChMb3dlcikKCiMgV2UgY2FuIGNoZWNrIHNvbWUgc2FtcGxlcyB3aXRoIHByaWNlcyBiZWxvdyA0Ni42JApMb3dlcl9iMTAKCmRmX3dob2xlX28gPSBkZl93aG9sZVtkZl93aG9sZSRwcmljZSA8IExvd2VyX2IxMCxdCmRmX3dob2xlX28kdGl0bGVfaXRlbSA9IGFzLmZhY3RvcihkZl93aG9sZV9vJHRpdGxlX2l0ZW0pCmBgYAoKYGBge3J9CiMgV2UgY2FuIHNlZSB0aGF0IGltdGUgdGl0bGVzIGFyZSB0aGUgc2FtZSBtYW55IHRpbWVzLiBXZSBjYW4gYWxzbyBzZWUgMyBicmFuZHMuIE5va2lhLCBTYW1zdW5nIGFuZCBIdWF3ZWkuIE5va2lhIGNhbiBoYXZlIHZlcnkKIyBsb3cgY29zdCBwaG9uZXMuIEJ1dCBTYW1zdW5nIGlzIGtub3duIGZvciBwcmVtaXVtIHBob25lcy4uLgpzdW1tYXJ5KGRmX3dob2xlX28pCmBgYAoKYGBge3J9CiMgU28gd2UgY2FuIGNoYWNrIHNhbXN1bmcgcGhvbmUuIE9rLCBzbyB0aGUgZmlyc3QgaXRlbSBpcyBhIHByZXBhaWQgcGhvbmUuIEJ1dCB0aGUgMiBpdGVtcyBmb2xsb3dpbmcgYXJlIGEgd2lyZWxlc3MgY2hhcmdlIGFuZAojIG5vdCBwaG9uZS4gV2UgY2FuIHJlbW92ZSBpdC4gVGhlIGtleXdvcmQgaXMgQ2hhcmdpbmcuIEFuZCB0aGUgdGhpcmQgb25lIGlzIGEgU2Ftc3VuZyBHYWxheHksIHNvIGEgcHJlbWl1bSBwaG9uZS4gVGhlIHByaWNlIGlzIG5vdAojIGNvaGVyZW50CnN1bW1hcnkoZGZfd2hvbGVfb1tkZl93aG9sZV9vJGJyYW5kID09ICJTYW1zdW5nIiwgXSkKVmlldyhkZl93aG9sZV9vW2RmX3dob2xlX28kYnJhbmQgPT0gIlNhbXN1bmciLCBdKQpgYGAKCmBgYHtyfQojIFNvIHdlIGNhbiBjaGFjayBIdWF3ZWkgcGhvbmUuIDIgaXRlbXMuIEEgcHJlcGFpZCBwaG9uZSAtLT4gb2suIEFuZCBhIE1vZGVtLi4uIE5vdCBhIHBob25lLiBXZSBoYXZlIHRvIHJlbW92ZSBpdC4gCnN1bW1hcnkoZGZfd2hvbGVfb1tkZl93aG9sZV9vJGJyYW5kID09ICJIVUFXRUkiLCBdKQoKc3VtbWFyeShkZl93aG9sZV9vW2RmX3dob2xlX28kYnJhbmQgPT0gIk5va2lhIiwgXSkKYGBgCgpUbyBzdW1tYXJpemUsIHRoZSBvdXRsaWVycyBjb25jZXJuZWQgb2Z0ZW4gcHJlcGFpZCBwaG9uZXMuIFdlIGNhbiBrZWVwIGl0IGZvciBvdXIgYW5hbHlzaXMuIEJ1dCBzb21lIGl0ZW1zIGFyZSBub3QgcGhvbmVzIG9yIGFyZSBpbmNvaGVyZW50LgoKV2UgaGF2ZSB0byByZW1vdmU6IEl0ZW1zIHdpdGggKmNoYXJnZSogYXMga2V5d29ya2RzIC0tXD4gZG9lcyBub3Qgd29yayBJdGVtcyB3aXRoIG1vZGVtXCogYXMga2V5d29yZHMgLS1cPiBkb2VzIG5vdCB3b3JrLiBXZSBjYW4gY2hlY2sgdGhlIGRldmljZSB0eXBlIHRocm91Z2ggdGhlIGh0bWwsIGFuZCBlbnJpY2ggdGhlIGRhdGEgYnkgYWRkaW5nIHRoZSBwcm9kdWN0IGNhdGVnb3J5IFNhbXN1bmcgR2FsYXh5IHdpdGggcHJpY2UgbG93ZXIgdGhhbiB0aGUgbG93ZXIgbG9nIHByaWNlIG91dGxpZXJzIHRocmVzaG9sZAoKYGBge3J9CnN1bW1hcnkoZGZfd2hvbGVbZ3JlcGwoImNoYXJnIiwgZGZfd2hvbGUkdGl0bGVfaXRlbSwgaWdub3JlLmNhc2UgPSBUUlVFKSxdKQoKc3VtbWFyeShkZl93aG9sZVtncmVwbCgibW9kZW0iLCBkZl93aG9sZSR0aXRsZV9pdGVtLCBpZ25vcmUuY2FzZSA9IFRSVUUpLF0pCgpgYGAKCmBgYHtyfQojIFJlbW92ZSBjaGFyZ2VzIGl0ZW1zIGJ5IGhhbmQKCnVud2FudGVkX2l0ZW1zID0gYygiU2Ftc3VuZyBEZVggV2lyZWxlc3MgUWkgRGVza3RvcCBDaGFyZ2luZyBEb2NrIFN0YXRpb24gRUUtTUc5NTAgR2FsYXh5IFM4ICsgTm90ZTggKFJlbmV3ZWQpIiwgCiAgICAgICAgICAgICAgICAgICAiU2Ftc3VuZyBRaSBDZXJ0aWZpZWQgRmFzdCBDaGFyZ2UgV2lyZWxlc3MgQ2hhcmdpbmcgUGFkIGZvciBRaSBDb21wYXRpYmxlIFNtYXJ0cGhvbmVzIHdpdGggQnVpbHQtaW4gQ29vbCBGYW4gLSBSZXRhaWwgUGFja2FnaW5nIC0gQmxhY2siLCAKICAgICAgICAgICAgICAgICAgICJTYW1zdW5nIEdhbGF4eSBTMTArIFBsdXMgVmVyaXpvbiArIEdTTSBVbmxvY2tlZCA1MTJHQiBDZXJhbWljIEJsYWNrIiwKICAgICAgICAgICAgICAgICAgICJNb2RlbSBIdWF3ZWkgVVNCIDNHIEgrIEdTTSBVbmxvY2tlZCBLNDYwNSAoRTM3MikgVVNBIExhdGluICYgQ2FyaWJiZWFuIEV1cm9wZSBCYW5kcyA4NTAvOTAwLzE5MDAgbWh6IEJBTSIpCgpkZl93aG9sZSA9IGRmX3dob2xlW2RmX3dob2xlJHRpdGxlX2l0ZW0gJWluJSB1bndhbnRlZF9pdGVtcyA9PSBGQUxTRSxdCmBgYAoKYGBge3J9CiMgSWYgb3JpZ2luYWwgcHJpY2UgaXMgbm90IHNldCB0byAwLCB3ZSBjYW4gY29uc2lkZXIgYSBwcmljZSByZWR1Y3Rpb24uIFByaWNlIGNvbHVtbiBpcyB0aGUgcmVhbCBwcmljZS4gVGhlIHByaWNlIGRpZmZlcmVuY2UgaXMKIyBvcmlnaW5hbF9wcmljZSAtIHByaWNlLiBJZiBvcmlnaW5hbCBwcmljZSBpcyAwLCB3ZSBkb24ndCBoYXZlIGFueSByZWR1Y3Rpb24uIAoKZGZfd2hvbGUkb3JpZ2luYWxQcmljZSA9IGFwcGx5KGRmX3dob2xlWyxjKCJwcmljZSIsICJvcmlnaW5hbFByaWNlIildLCAxLCBmdW5jdGlvbih4KXtpZiAoeFsib3JpZ2luYWxQcmljZSJdID09IDApIHJldHVybiAoeFsicHJpY2UiXSkgZWxzZSAocmV0dXJuKHhbIm9yaWdpbmFsUHJpY2UiXSkpfSkKVmlldyhkZl93aG9sZSkKCiMgQWRkIHByaWNlIHJlZHVjdGlvbiBjb2x1bW4KZGZfd2hvbGUkcHJpY2UucmVkdWN0aW9uID0gYXBwbHkoZGZfd2hvbGVbLGMoInByaWNlIiwgIm9yaWdpbmFsUHJpY2UiKV0sIDEsIGZ1bmN0aW9uKHgpe3JldHVybigoeFsib3JpZ2luYWxQcmljZSJdIC0geFsicHJpY2UiXSkveFsib3JpZ2luYWxQcmljZSJdKX0pCgojU29tZSBpbmNvbnNpc3RlbmN5Cm5yb3coZGZfd2hvbGVbZGZfd2hvbGUkcHJpY2UucmVkdWN0aW9uIDwgMCxdKQpWaWV3KGRmX3dob2xlW2RmX3dob2xlJHByaWNlLnJlZHVjdGlvbiA8IDAsXSkKCiMgQ29ycmVjdCB0aGVzZSBpbmNvbnNpc3RlbmNpZXMKZGZfd2hvbGVbZGZfd2hvbGUkcHJpY2UucmVkdWN0aW9uIDwgMCxdJG9yaWdpbmFsUHJpY2UgPSBkZl93aG9sZVtkZl93aG9sZSRwcmljZS5yZWR1Y3Rpb24gPCAwLF0kcHJpY2UKZGZfd2hvbGVbZGZfd2hvbGUkcHJpY2UucmVkdWN0aW9uIDwgMCxdJHByaWNlLnJlZHVjdGlvbiA9IDAKCgpzdW1tYXJ5KGRmX3dob2xlKQpgYGAKCmBgYHtyfQojIFdlIGNhbiB0cmFuc2Zvcm0gd2l0aCBsb2cKaGlzdChkZl93aG9sZSRwcmljZSkKYm94cGxvdChkZl93aG9sZSRwcmljZSkKCiMgTXVjaCBiZXR0ZXIgYmVjYXVzZSBpdCdzIGNsb3NlciB0byBhIG5vcm1hbCBkaXN0cmlidXRpb24KaGlzdChsb2coZGZfd2hvbGUkcHJpY2UpKQpgYGAKCmBgYHtyfQpib3hwbG90KGxvZyhkZl93aG9sZSRwcmljZSkpCgpkZl93aG9sZSRwcmljZS5sb2cgPSBsb2coZGZfd2hvbGUkcHJpY2UpCmBgYAoKYGBge3J9CiMgV2UgY2FuIHRyYW5zZm9ybSB3aXRoIGxvZwpoaXN0KGRmX3dob2xlJHByaWNlLnJlZHVjdGlvbikKYm94cGxvdChkZl93aG9sZSRwcmljZS5yZWR1Y3Rpb24pCgpoaXN0KGxvZyhkZl93aG9sZSRwcmljZS5yZWR1Y3Rpb24pKQpib3hwbG90KGxvZyhkZl93aG9sZSRwcmljZS5yZWR1Y3Rpb24pKQoKaGlzdChkZl93aG9sZSRyYXRpbmcpCmJveHBsb3QoZGZfd2hvbGUkcmF0aW5nKQoKaGlzdChsb2coZGZfd2hvbGUkcmF0aW5nKSkKYm94cGxvdChsb2coZGZfd2hvbGUkcmF0aW5nKSkKCmhpc3QoZGZfd2hvbGUkaGVscGZ1bFZvdGVzKQpib3hwbG90KGRmX3dob2xlJGhlbHBmdWxWb3RlcykKCmhpc3QobG9nKGRmX3dob2xlJGhlbHBmdWxWb3RlcykpCmJveHBsb3QobG9nKGRmX3dob2xlJGhlbHBmdWxWb3RlcykpCgpzdW1tYXJ5KGRmX3dob2xlKQoKYmFycGxvdChwcm9wLnRhYmxlKHRhYmxlKGRmX3dob2xlJHZlcmlmaWVkKSkpCmJhcnBsb3QocHJvcC50YWJsZSh0YWJsZShkZl93aG9sZSRicmFuZCkpLCBsYXMgPSAyKQpgYGAKCiMgKipUZXh0dWFsIGFuYWx5c2lzKioKCmBgYHtyfQojIFJldmlld3MgYW5hbHlzaXMKaW5zdGFsbC5wYWNrYWdlcyhybGFuZykKbGlicmFyeSh3b3JkY2xvdWQpCmxpYnJhcnkodGlkeXIpCmxpYnJhcnkodGlkeXZlcnNlKQpsaWJyYXJ5KHN0b3B3b3JkcykKCiMgTWVyZ2UgdGl0bGUgcmV2aWV3IGFuZCBib2R5CmRmX3dob2xlJHJldmlldyA9IHBhc3RlKGRmX3dob2xlJHRpdGxlX3JldmlldywgZGZfd2hvbGUkYm9keSkKc3RvcHdvcmRzX3Bob25lID0gYyhzdG9wd29yZHMoImVuZ2xpc2giKSwicGhvbmUiLCJTYW1zdW5nIiwiTm9raWEiLCJBcHBsZSIsIkFTVVMiLCJPbmVQbHVzIiwiTW90b3JvbGEiLCJIVUFXRUkiLCJTb255IiwiR29vZ2xlIiwiWGlhb21pIiwiZ3JlYXQiLCJsaWtlIiwiZ29vZCIsCiAgICAgICAgICAgICAgICAgICAgICJzYW1zdW5nIiwibm9raWEiLCJpcGhvbmUiLCJhcHBsZSIsImFzdXMiLCJvbmVwbHVzIiwibW90b3JvbGEiLCJodWF3ZWkiLCJzb255IiwiZ29vZ2xlIiwieGlhb21pIiwgInBob25lcyIsICJqdXN0IikKCmxpYnJhcnkoZHBseXIpCmxpYnJhcnkodGlkeXRleHQpCmxpYnJhcnkoc3RvcHdvcmRzKQpsaWJyYXJ5KGdncGxvdDIpCgpkZl9jb3VudF93b3JkcyA9IGRmX3dob2xlICU+JSAKICBzZWxlY3QoYnJhbmQscmV2aWV3KSAlPiUgCiAgdW5uZXN0X3Rva2VucyhpbnB1dD1yZXZpZXcsb3V0cHV0PXdvcmQpICU+JQogIGNvdW50KGJyYW5kLHdvcmQsc29ydD1UKSAlPiUgCiAgZmlsdGVyKG5jaGFyKHdvcmQpPjMpICU+JQogIGZpbHRlcighd29yZCAlaW4lIHN0b3B3b3Jkc19waG9uZSkgJT4lCiAgZ3JvdXBfYnkoYnJhbmQpCgpkZl9jb3VudF93b3JkcyA9IGRmX2NvdW50X3dvcmRzICU+JSAKICBhbnRpX2pvaW4oc3RvcF93b3JkcykKCmRmX2NvdW50X3dvcmRzICU+JQogIHRvcF9uKG49MTAsbiklPiUKICBtdXRhdGUod29yZCA9IHJlb3JkZXIod29yZCwgbikpICU+JSAKICBnZ3Bsb3QoYWVzKHg9d29yZCwgeT1uLGZpbGw9YnJhbmQpKSArIAogIGdlb21fY29sKHNob3cubGVnZW5kPUYsY29sPSJibGFjayIpKwogIGZhY2V0X3dyYXAofmJyYW5kLCBuY29sPTIsc2NhbGVzPSJmcmVlIikrCiAgeGxhYihOVUxMKSArIAogIHlsYWIoIkNvdW50IikrCiAgZ2d0aXRsZSgiTW9zdCBDb21tb24gV29yZHMgaW4gcmV2aWV3cyBieSBCcmFuZCIpICsKICBjb29yZF9mbGlwKCkKCmBgYAoKYGBge3J9Cm5sZXZlbHModW5pcXVlKGRmX2NvdW50X3dvcmRzJGJyYW5kKSkKCmluc3RhbGwucGFja2FnZXMoWFEpCmxpYnJhcnkoInNlbnRpbWVudHIiKQpsaWJyYXJ5KHRtKQoKZG9jcyA9IENvcnB1cyhWZWN0b3JTb3VyY2UoZGZfd2hvbGUkcmV2aWV3KSkKIyBMb3dlciBjYXNlIHRyYW5zZm9ybQpkb2NzID0gdG1fbWFwKGRvY3MsIGNvbnRlbnRfdHJhbnNmb3JtZXIodG9sb3dlcikpCiMgUmVtb3ZlIG51bWJlcnMKZG9jcyA9IHRtX21hcChkb2NzLCByZW1vdmVOdW1iZXJzKQpkb2NzID0gdG1fbWFwKGRvY3MsIHJlbW92ZVdvcmRzLCBzdG9wd29yZHMoImVuZ2xpc2giKSkKIyBEZWxldGUgcHVuY3R1YXRpb24KZG9jcyA9IHRtX21hcChkb2NzLCByZW1vdmVQdW5jdHVhdGlvbikKCiMgYmVsb3cgMCAtLT4gbmVnYXRpdmUsIDAgLS0+IG5ldXRyYWwsIGFib3ZlIDAgLS0+IHBvc2l0aXZlCmRmX3dob2xlJHJldmlldy5zZW50aW1lbnQgPSBzZW50aW1lbnRfYnkoZG9jcyRjb250ZW50KSRhdmVfc2VudGltZW50CgojIFdlIGNhbiB2ZXJpZnkgdGhlIGNvcnJlbGF0aW9uIGJldHdlZW4gdGhlIHJhdGluZ3MgYW5kIHRoZSBwcmVkaWN0ZWQgc2VudGltZW50cwpkZl93aG9sZSRyZXZpZXcuc2VudGltZW50LmNhdGVnb3J5ID0gc2FwcGx5KGRmX3dob2xlJHJldmlldy5zZW50aW1lbnQsIGZ1bmN0aW9uKHgpe2lmICh4PDApIHJldHVybigiTmVnYXRpdmUiKSBlbHNlIGlmICh4PjApIHJldHVybigiUG9zaXRpdmUiKSBlbHNlIHJldHVybigiTmV1dHJhbCIpfSkKZGZfd2hvbGUkcmV2aWV3LnNlbnRpbWVudC5jYXRlZ29yeSA9IGFzLmZhY3RvcihkZl93aG9sZSRyZXZpZXcuc2VudGltZW50LmNhdGVnb3J5KQoKYmFycGxvdChwcm9wLnRhYmxlKHRhYmxlKGRmX3dob2xlJHJldmlldy5zZW50aW1lbnQuY2F0ZWdvcnkpKSkKCmBgYAoKYGBge3J9CiMjIyMjIFBvc2l0aXZlIHdvcmRzCmRmX2NvdW50X3dvcmRzX3Bvc2l0aXZlID0gZGZfd2hvbGVbZGZfd2hvbGUkcmV2aWV3LnNlbnRpbWVudC5jYXRlZ29yeT09IlBvc2l0aXZlIixdICU+JSAKICBzZWxlY3QoYnJhbmQscmV2aWV3KSAlPiUgCiAgdW5uZXN0X3Rva2VucyhpbnB1dD1yZXZpZXcsb3V0cHV0PXdvcmQpICU+JQogIGNvdW50KGJyYW5kLHdvcmQsc29ydD1UKSAlPiUgCiAgZmlsdGVyKG5jaGFyKHdvcmQpPjMpICU+JQogIGZpbHRlcighd29yZCAlaW4lIHN0b3B3b3Jkc19waG9uZSkgJT4lCiAgZ3JvdXBfYnkoYnJhbmQpCgpkZl9jb3VudF93b3Jkc19wb3NpdGl2ZSA9IGRmX2NvdW50X3dvcmRzX3Bvc2l0aXZlICU+JSAKICBhbnRpX2pvaW4oc3RvcF93b3JkcykKCmRmX2NvdW50X3dvcmRzX3Bvc2l0aXZlICU+JQogIHRvcF9uKG49MTAsbiklPiUKICBtdXRhdGUod29yZCA9IHJlb3JkZXIod29yZCwgbikpICU+JSAKICBnZ3Bsb3QoYWVzKHg9d29yZCwgeT1uLGZpbGw9YnJhbmQpKSArIAogIGdlb21fY29sKHNob3cubGVnZW5kPUYsY29sPSJibGFjayIpKwogIGZhY2V0X3dyYXAofmJyYW5kLCBuY29sPTIsc2NhbGVzPSJmcmVlIikrCiAgeGxhYihOVUxMKSArIAogIHlsYWIoIkNvdW50IikrCiAgZ2d0aXRsZSgiTW9zdCBDb21tb24gV29yZHMgaW4gcG9zaXRpdmUgcmV2aWV3cyBieSBCcmFuZCIpICsKICBjb29yZF9mbGlwKCkKCmBgYAoKYGBge3J9CiMjIyMjIE5lZ2F0aXZlIHdvcmRzCmRmX2NvdW50X3dvcmRzX25lZ2F0aXZlID0gZGZfd2hvbGVbZGZfd2hvbGUkcmV2aWV3LnNlbnRpbWVudC5jYXRlZ29yeT09Ik5lZ2F0aXZlIixdICU+JSAKICBzZWxlY3QoYnJhbmQscmV2aWV3KSAlPiUgCiAgdW5uZXN0X3Rva2VucyhpbnB1dD1yZXZpZXcsb3V0cHV0PXdvcmQpICU+JQogIGNvdW50KGJyYW5kLHdvcmQsc29ydD1UKSAlPiUgCiAgZmlsdGVyKG5jaGFyKHdvcmQpPjMpICU+JQogIGZpbHRlcighd29yZCAlaW4lIHN0b3B3b3Jkc19waG9uZSkgJT4lCiAgZ3JvdXBfYnkoYnJhbmQpCgpkZl9jb3VudF93b3Jkc19uZWdhdGl2ZSA9IGRmX2NvdW50X3dvcmRzX25lZ2F0aXZlICU+JSAKICBhbnRpX2pvaW4oc3RvcF93b3JkcykKCmRmX2NvdW50X3dvcmRzX25lZ2F0aXZlICU+JQogIHRvcF9uKG49MTAsbiklPiUKICBtdXRhdGUod29yZCA9IHJlb3JkZXIod29yZCwgbikpICU+JSAKICBnZ3Bsb3QoYWVzKHg9d29yZCwgeT1uLGZpbGw9YnJhbmQpKSArIAogIGdlb21fY29sKHNob3cubGVnZW5kPUYsY29sPSJibGFjayIpKwogIGZhY2V0X3dyYXAofmJyYW5kLCBuY29sPTIsc2NhbGVzPSJmcmVlIikrCiAgeGxhYihOVUxMKSArIAogIHlsYWIoIkNvdW50IikrCiAgZ2d0aXRsZSgiTW9zdCBDb21tb24gV29yZHMgaW4gbmVnYXRpdmUgcmV2aWV3cyBieSBCcmFuZCIpICsKICBjb29yZF9mbGlwKCkKYGBgCgpOb3Qgc28gbXVjaCBkaWZmZXJlbmNlcyBiZXR3ZWVuIG5lZ2F0aXZlIGFuZCBwb3NpdGl2ZSBzZW50aW1lbnRzLgoKYGBge3J9CiMgTW9zdCBjb21tb24gd29yZHMgYnkgc2VudGltZW50cwpkZl9jb3VudF93b3Jkc19wb3NpdGl2ZSA9IGRmX3dob2xlICU+JSAKICBzZWxlY3QocmV2aWV3LnNlbnRpbWVudC5jYXRlZ29yeSxyZXZpZXcpICU+JSAKICB1bm5lc3RfdG9rZW5zKGlucHV0PXJldmlldyxvdXRwdXQ9d29yZCkgJT4lCiAgY291bnQocmV2aWV3LnNlbnRpbWVudC5jYXRlZ29yeSx3b3JkLHNvcnQ9VCkgJT4lIAogIGZpbHRlcihuY2hhcih3b3JkKT4zKSAlPiUKICBmaWx0ZXIoIXdvcmQgJWluJSBzdG9wd29yZHNfcGhvbmUpICU+JQogIGdyb3VwX2J5KHJldmlldy5zZW50aW1lbnQuY2F0ZWdvcnkpCgpkZl9jb3VudF93b3Jkc19wb3NpdGl2ZSA9IGRmX2NvdW50X3dvcmRzX3Bvc2l0aXZlICU+JSAKICBhbnRpX2pvaW4oc3RvcF93b3JkcykKCmRmX2NvdW50X3dvcmRzX3Bvc2l0aXZlICU+JQogIHRvcF9uKG49MTAsbiklPiUKICBtdXRhdGUod29yZCA9IHJlb3JkZXIod29yZCwgbikpICU+JSAKICBnZ3Bsb3QoYWVzKHg9d29yZCwgeT1uLGZpbGw9cmV2aWV3LnNlbnRpbWVudC5jYXRlZ29yeSkpICsgCiAgZ2VvbV9jb2woc2hvdy5sZWdlbmQ9Rixjb2w9ImJsYWNrIikrCiAgZmFjZXRfd3JhcCh+cmV2aWV3LnNlbnRpbWVudC5jYXRlZ29yeSwgbmNvbD0yLHNjYWxlcz0iZnJlZSIpKwogIHhsYWIoTlVMTCkgKyAKICB5bGFiKCJDb3VudCIpKwogIGdndGl0bGUoIk1vc3QgQ29tbW9uIFdvcmRzIGluIHBvc2l0aXZlIHJldmlld3MiKSArCiAgY29vcmRfZmxpcCgpCgoKZGZfd2hvbGUkcmF0aW5ncy5jYXRlZ29yeSA9IGFzLmZhY3RvcihzYXBwbHkoZGZfd2hvbGUkcmF0aW5nLCBmdW5jdGlvbih4KXtpZiAoeDwzKSByZXR1cm4oIk5lZ2F0aXZlIikgZWxzZSBpZiAoeD4zKSByZXR1cm4oIlBvc2l0aXZlIikgZWxzZSByZXR1cm4oIk5ldXRyYWwiKX0pKQoKYGBgCgpgYGB7cn0KIyMjIEFuYWx5c2lzIGJ5IHJhdGluZ3MKZGZfY291bnRfd29yZHMgPSBkZl93aG9sZSAlPiUgCiAgc2VsZWN0KHJhdGluZyxyZXZpZXcpICU+JSAKICB1bm5lc3RfdG9rZW5zKGlucHV0PXJldmlldyxvdXRwdXQ9d29yZCkgJT4lCiAgY291bnQocmF0aW5nLHdvcmQsc29ydD1UKSAlPiUgCiAgZmlsdGVyKG5jaGFyKHdvcmQpPjMpICU+JQogIGZpbHRlcighd29yZCAlaW4lIHN0b3B3b3Jkc19waG9uZSkgJT4lCiAgZ3JvdXBfYnkocmF0aW5nKQoKZGZfY291bnRfd29yZHMgPSBkZl9jb3VudF93b3JkcyAlPiUgCiAgYW50aV9qb2luKHN0b3Bfd29yZHMpCgpkZl9jb3VudF93b3JkcyAlPiUKICB0b3BfbihuPTEwLG4pJT4lCiAgbXV0YXRlKHdvcmQgPSByZW9yZGVyKHdvcmQsIG4pKSAlPiUgCiAgZ2dwbG90KGFlcyh4PXdvcmQsIHk9bixmaWxsPXJhdGluZykpICsgCiAgZ2VvbV9jb2woc2hvdy5sZWdlbmQ9Rixjb2w9ImJsYWNrIikrCiAgZmFjZXRfd3JhcCh+cmF0aW5nLCBuY29sPTIsc2NhbGVzPSJmcmVlIikrCiAgeGxhYihOVUxMKSArIAogIHlsYWIoIkNvdW50IikrCiAgZ2d0aXRsZSgiTW9zdCBDb21tb24gV29yZHMgaW4gYnkgYnJhbmRzIikgKwogIGNvb3JkX2ZsaXAoKQpgYGAKClRoZSB3b3JkIHByaWNlIGlzIG5vdCBwcmVzZW50IGluIHRoZSBtb3N0IGNvbW1vbiB3b3JkcyBmb3IgdGhlIDEgYW5kIDIgc3RhcnMgcmF0aW5nLiBTbyBpdCBkb2VzIG5vdCBzZWVtIHRvIGJlIGFuIGlzc3VlLgoKYGBge3J9CiMjIyMgQW5hbHlzaXMgYnkgcHJpY2UKCmRmX3dob2xlJHByaWNlLmNhdGVnb3J5ID0gY3V0KGRmX3dob2xlJHByaWNlLCBicmVha3MgPSAzLCBsYWJlbHM9YygiZWNvbm9taWNhbCIsICJtZWRpdW0iLCAiaGlnaCIpKQpiYXJwbG90KHByb3AudGFibGUodGFibGUoZGZfd2hvbGUkcHJpY2UuY2F0ZWdvcnkpKSkKCmRmX2NvdW50X3dvcmRzID0gZGZfd2hvbGUgJT4lIAogIHNlbGVjdChwcmljZS5jYXRlZ29yeSxyZXZpZXcpICU+JSAKICB1bm5lc3RfdG9rZW5zKGlucHV0PXJldmlldyxvdXRwdXQ9d29yZCkgJT4lCiAgY291bnQocHJpY2UuY2F0ZWdvcnksd29yZCxzb3J0PVQpICU+JSAKICBmaWx0ZXIobmNoYXIod29yZCk+MykgJT4lCiAgZmlsdGVyKCF3b3JkICVpbiUgc3RvcHdvcmRzX3Bob25lKSAlPiUKICBncm91cF9ieShwcmljZS5jYXRlZ29yeSkKCmRmX2NvdW50X3dvcmRzID0gZGZfY291bnRfd29yZHMgJT4lIAogIGFudGlfam9pbihzdG9wX3dvcmRzKQoKZGZfY291bnRfd29yZHMgJT4lCiAgdG9wX24obj0xMCxuKSU+JQogIG11dGF0ZSh3b3JkID0gcmVvcmRlcih3b3JkLCBuKSkgJT4lIAogIGdncGxvdChhZXMoeD13b3JkLCB5PW4sZmlsbD1wcmljZS5jYXRlZ29yeSkpICsgCiAgZ2VvbV9jb2woc2hvdy5sZWdlbmQ9Rixjb2w9ImJsYWNrIikrCiAgZmFjZXRfd3JhcCh+cHJpY2UuY2F0ZWdvcnksIG5jb2w9MixzY2FsZXM9ImZyZWUiKSsKICB4bGFiKE5VTEwpICsgCiAgeWxhYigiQ291bnQiKSsKICBnZ3RpdGxlKCJNb3N0IENvbW1vbiBXb3JkcyBieSBwcmljZXMiKSArCiAgY29vcmRfZmxpcCgpCmBgYAoKQWx3YXlzIHNjcmVlbiBhbmQgYmF0dGVyeSBhcyBtb3N0IGNvbW1vbiB3b3Jkcy4gQnV0IG5vdCByZWFsIGRpZmZlcmVuY2VzIGJldHdlZW4gcHJpY2VzLgoKIyAqKlJlY29tbWVuZGVyIGVuZ2luZSoqIAoKV2UndmUgYnVpbHQgYSBvbGxhYm9yYXRpdmUgY29udGVudCBiYXNlIGZpbHRlcmluZy4gKGNmIG5ldGZsaXggc3ZkIGFsZ29yaXRobSkgWW91IGhhdmUgaW4gaW5wdXQ6IHRoZSBwcm9kdWN0X2lkIGFuZCB0aGUgY3VzdG9tZXJfaWQgVGhlIE91dHB1dCA6IGhvdyBtdWNoIHRoZSBwcm9kdWN0IGZpdCB3aXRoIHRoZSBjdXN0b21lci4gQmFzZSBvbmUgdGhlIHByZXZpb3VzIGN1c3RvbWVyIG5vdGVzLgoKSGVyZSBpcyBhbiBleHBsYW5hdGlvbiBvZiB0aGUgcmVjb3N5c3RlbSBwYWNrYWdlIDogPGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9yZWNvc3lzdGVtL3JlYWRtZS9SRUFETUUuaHRtbD4KCmBgYHtyfQpsaWJyYXJ5KG1hZGl0cikKZGZfd2hvbGVfcmVjb19lbmdpbmUgPSBuYS5vbWl0KGRmX3dob2xlWyxjKCJhc2luIiwgIm5hbWUiLCAicmF0aW5nIildKQpkZl93aG9sZV9yZWNvX2VuZ2luZSRuYW1lX2lkID0gYXMubnVtZXJpYyhhcy5mYWN0b3IoZGZfd2hvbGVfcmVjb19lbmdpbmUkbmFtZSkpCmRmX3dob2xlX3JlY29fZW5naW5lJGFzaW5faWQgPSBhcy5udW1lcmljKGFzLmZhY3RvcihkZl93aG9sZV9yZWNvX2VuZ2luZSRhc2luKSkKCmRmX3dob2xlX3JlY29fZW5naW5lJHRlc3QgPSBzYW1wbGUoYyhyZXAoMCwgMC44ICogKG5yb3coZGZfd2hvbGVfcmVjb19lbmdpbmUpKzEpKSwgcmVwKDEsIDAuMiAqIChucm93KGRmX3dob2xlX3JlY29fZW5naW5lKSsxKSkpKQp0YWJsZShkZl93aG9sZV9yZWNvX2VuZ2luZSR0ZXN0KQoKdHJhaW5fc2V0ID0gZGZfd2hvbGVfcmVjb19lbmdpbmVbZGZfd2hvbGVfcmVjb19lbmdpbmUkdGVzdCA9PSAwLF0KdGVzdF9zZXQgPSBkZl93aG9sZV9yZWNvX2VuZ2luZVtkZl93aG9sZV9yZWNvX2VuZ2luZSR0ZXN0ID09IDEsXQoKbGlicmFyeShyZWNvc3lzdGVtKQoKciA9IFJlY28oKQoKdHJhaW5fc2V0X2RhdGFzb3VyY2UgPSBkYXRhX21lbW9yeSh1c2VyX2luZGV4ID0gdHJhaW5fc2V0JG5hbWVfaWQsIGl0ZW1faW5kZXggPSB0cmFpbl9zZXQkYXNpbl9pZCwgcmF0aW5nID0gdHJhaW5fc2V0JHJhdGluZykKdGVzdF9zZXRfZGF0YXNvdXJjZSA9IGRhdGFfbWVtb3J5KHVzZXJfaW5kZXggPSB0ZXN0X3NldCRuYW1lX2lkLCBpdGVtX2luZGV4ID0gdGVzdF9zZXQkYXNpbl9pZCwgcmF0aW5nID0gdGVzdF9zZXQkcmF0aW5nKQoKciR0cmFpbih0cmFpbl9zZXRfZGF0YXNvdXJjZSwgb3B0cyA9IGMobnRocmVhZCA9IDEsIG5pdGVyID0gMjApKQoKcHJlZF92ZWMgPSByJHByZWRpY3QodGVzdF9zZXRfZGF0YXNvdXJjZSwgb3V0X21lbW9yeSgpKQp0ZXN0X3NldCRwcmVkaWN0aW9uID0gcHJlZF92ZWMKYGBgCgpgYGB7cn0KIyMgQ29tcHV0ZSBybXNlIHRvIGV2YWx1YXRlIHRoZSBtb2RlbCBxdWFsaXR5CgpybXNlID0gc3FydChtZWFuKCh0ZXN0X3NldCRyYXRpbmcgLSB0ZXN0X3NldCRwcmVkaWN0aW9uKV4yKSkKcHJpbnQocm1zZSkKCmBgYAoKV2UgaGF2ZSBhIHJtc2Ugb2YgMS41IHdoaWNoIHNlZW1zIG5vdCBzbyBnb29kIG9uIGEgNSBsZXZlbHMgcmF0aW5nLgoKYGBge3J9Cm1hZSA9IG1lYW4oYWJzKHRlc3Rfc2V0JHJhdGluZyAtIHRlc3Rfc2V0JHByZWRpY3Rpb24pKQojIEluIGF2ZXJhZ2UsIHRoZSBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIHByZWRpY3Rpb24gYW5kIHRoZSByZWFsIG5vdGUgaXMgMS4zLiAKcHJpbnQobWFlKQoKIyBVc2luZyBvdXIgbW9kZWwsIHdlIGNhbiBtYWtlIHNvbWUgcHJlZGljdGlvbnMKYGBgCgpgYGB7cn0KIyBJdGVtOyBCMDdZUTU4TlBGLCBpZDogNTcxCiMgVXNlcjogQWxpc29uIFN0cml0dG1hdHRlciwgaWQ6IDE1NTcKZGZfd2hvbGVfcmVjb19lbmdpbmVbZGZfd2hvbGVfcmVjb19lbmdpbmUkbmFtZSA9PSAiQWxpc29uIFN0cml0dG1hdHRlciIsXQpkZl93aG9sZV9yZWNvX2VuZ2luZVtkZl93aG9sZV9yZWNvX2VuZ2luZSRhc2luID09ICJCMDdZUTU4TlBGIixdWzEsXQoKCnByZWRfc2V0X2RhdGFzb3VyY2UgPSBkYXRhX21lbW9yeSh1c2VyX2luZGV4ID0gYygxNTU3KSwgaXRlbV9pbmRleCA9IGMoNTcxKSkKcHJlZF92ZWMgPSByJHByZWRpY3QocHJlZF9zZXRfZGF0YXNvdXJjZSwgb3V0X21lbW9yeSgpKQoKIyA0IHN0YXJzCnByZWRfdmVjCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIwoKYGBgCg==