A story about one of the retail chains (J.C. Penny) releasing the list of stores closing in 2017 crossed paths with my Feedly reading list today and jogged my memory that there were a number of chains closing many of their doors this year and I wanted to see the impact that might have on various states.
library(httr)
library(rvest)
library(knitr)
library(kableExtra)
library(ggalt)
library(statebins)
library(hrbrthemes)
library(epidata)
library(tidyverse)
options(knitr.table.format = "html")
update_geom_font_defaults(font_rc_light, size = 2.75)
“Closing” lists of four major retailers — K-Mart, Sears, Macy’s and J.C. Penny — abound (HTML formatting a list seems to be the “easy way out” story-wise for many blogs and newspapers). We can dig a bit deeper than just a plain set of lists, but first we need the data.
The Boston Globe has a nice, predictable, mostly-uniform pattern to their list-closing “stories”, so we’ll use that data. Site content can change quickly, so it makes sense to try to cache content whenever possible as we scrape it. To that end, we’ll use httr::GET
vs xml2::read_html
since GET
preserves all of the original request and response information and read_html
returns an external pointer that has no current support for serialization without extra work.
closings <- list(
kmart = "https://www.bostonglobe.com/metro/2017/01/05/the-full-list-kmart-stores-closing-around/4kJ0YVofUWHy5QJXuPBAuM/story.html",
sears = "https://www.bostonglobe.com/metro/2017/01/05/the-full-list-sears-stores-closing-around/yHaP6nV2C4gYw7KLhuWuFN/story.html",
macys = "https://www.bostonglobe.com/metro/2017/01/05/the-full-list-macy-stores-closing-around/6TY8a3vy7yneKV1nYcwY7K/story.html",
jcp = "https://www.bostonglobe.com/business/2017/03/17/the-full-list-penney-stores-closing-around/vhoHjI3k75k2pSuQt2mZpO/story.html"
)
saved_pgs <- "saved_store_urls.rds"
if (file.exists(saved_pgs)) {
pgs <- read_rds(saved_pgs)
} else {
pgs <- map(closings, GET)
write_rds(pgs, saved_pgs)
}
This is what we get from that scraping effort:
map(pgs, content) %>%
map(html_table) %>%
walk(~glimpse(.[[1]]))
## Observations: 108
## Variables: 3
## $ X1 <chr> "300 Highway 78 E", "2003 US Hwy 280 Bypass", "3600 Wilson ...
## $ X2 <chr> "Jasper", "Phenix City", "Bakersfield", "Coalinga", "Kingsb...
## $ X3 <chr> "AL", "AL", "CA", "CA", "CA", "CA", "CO", "CO", "CT", "FL",...
## Observations: 42
## Variables: 4
## $ X1 <chr> "301 Cox Creek Pkwy", "1901 S Caraway Road", "90 Elm St; En...
## $ X2 <chr> "Florence", "Jonesboro", "Enfield", "Lake Wales", "Albany",...
## $ X3 <chr> "AL", "AR", "CT", "FL", "GA", "GA", "IN", "KS", "KY", "LA",...
## $ X4 <chr> "Y", "N", "N", "Y", "Y", "N", "N", "N", "Y", "Y", "Y", "Y",...
## Observations: 68
## Variables: 6
## $ X1 <chr> "Mission Valley Apparel", "Paseo Nuevo", "*Laurel Plaza", "...
## $ X2 <chr> "San Diego", "Santa Barbara", "North Hollywood", "Simi Vall...
## $ X3 <chr> "CA", "CA", "CA", "CA", "FL", "FL", "FL", "FL", "FL", "GA",...
## $ X4 <int> 385000, 141000, 475000, 190000, 101000, 195000, 143000, 140...
## $ X5 <int> 1961, 1990, 1995, 2006, 1995, 2000, 1977, 1974, 2000, 1981,...
## $ X6 <int> 140, 77, 105, 105, 68, 83, 86, 73, 72, 69, 9, 57, 54, 87, 5...
## Observations: 138
## Variables: 3
## $ Mall/Shopping Center <chr> "Auburn Mall", "Tannehill Promenade", "Ga...
## $ City <chr> "Auburn", "Bessemer", "Gadsden", "Jasper"...
## $ State <chr> "AL", "AL", "AL", "AL", "AR", "AR", "AZ",...
We now need to normalize the content of the lists.
map(pgs, content) %>%
map(html_table) %>%
map(~.[[1]]) %>%
map_df(select, abb=3, .id = "store") -> closings
We’re ultimately just looking for city/state for this simple exercise, but one could do more precise geolocation (perhaps with rgeocodio
) and combine that with local population data, job loss estimates, current unemployment levels, etc. to make a real story out of the closings vs just do the easy thing and publish a list of stores.
count(closings, abb) %>%
left_join(data_frame(name = state.name, abb = state.abb)) %>%
left_join(usmap::statepop, by = c("abb"="abbr")) %>%
mutate(per_capita = (n/pop_2015) * 1000000) %>%
select(name, n, per_capita) -> closings_by_state
2017 Retail Stores (Combined) Closing
Sears, J.C. Penny, K-Mart and Macy’s are all shutting down retail locations across the U.S.
Sorted by closigs per-capita (1MM)
|
|
Retail Stores Closing (sorted by per-capita)
State
|
Total Stores Closing
|
Per Capita (1MM)
|
North Dakota
|
5
|
6.6056568
|
South Dakota
|
5
|
5.8243221
|
Wyoming
|
2
|
3.4123462
|
West Virginia
|
6
|
3.2535703
|
Maine
|
4
|
3.0090392
|
Kansas
|
8
|
2.7475915
|
Kentucky
|
10
|
2.2598400
|
Iowa
|
7
|
2.2407895
|
Michigan
|
22
|
2.2171662
|
Minnesota
|
12
|
2.1859540
|
Oklahoma
|
8
|
2.0453359
|
Montana
|
2
|
1.9362040
|
Pennsylvania
|
23
|
1.7965237
|
Oregon
|
7
|
1.7374137
|
Louisiana
|
8
|
1.7127966
|
Mississippi
|
5
|
1.6709370
|
Utah
|
5
|
1.6689370
|
Ohio
|
19
|
1.6360379
|
Nebraska
|
3
|
1.5821199
|
Alabama
|
7
|
1.4406319
|
Hawaii
|
2
|
1.3970354
|
Wisconsin
|
8
|
1.3861606
|
Missouri
|
8
|
1.3149953
|
North Carolina
|
13
|
1.2944595
|
South Carolina
|
6
|
1.2254537
|
Indiana
|
8
|
1.2085176
|
Idaho
|
2
|
1.2085103
|
Georgia
|
12
|
1.1747591
|
Colorado
|
6
|
1.0995911
|
Virginia
|
9
|
1.0736022
|
Nevada
|
3
|
1.0377589
|
Arkansas
|
3
|
1.0073185
|
New Mexico
|
2
|
0.9591825
|
Rhode Island
|
1
|
0.9467025
|
Illinois
|
12
|
0.9331263
|
New Jersey
|
8
|
0.8930552
|
Massachusetts
|
6
|
0.8830773
|
Florida
|
17
|
0.8386252
|
Connecticut
|
3
|
0.8354484
|
Texas
|
22
|
0.8008995
|
Tennessee
|
5
|
0.7575414
|
Washington
|
5
|
0.6973159
|
New York
|
12
|
0.6061895
|
Maryland
|
2
|
0.3329781
|
California
|
12
|
0.3065540
|
Arizona
|
1
|
0.1464544
|
Sorted by total stores closing
|
|
Retail Stores Closing (sorted by total stores)
State
|
Total Stores Closing
|
Per Capita (1MM)
|
Pennsylvania
|
23
|
1.7965237
|
Michigan
|
22
|
2.2171662
|
Texas
|
22
|
0.8008995
|
Ohio
|
19
|
1.6360379
|
Florida
|
17
|
0.8386252
|
North Carolina
|
13
|
1.2944595
|
California
|
12
|
0.3065540
|
Georgia
|
12
|
1.1747591
|
Illinois
|
12
|
0.9331263
|
Minnesota
|
12
|
2.1859540
|
New York
|
12
|
0.6061895
|
Kentucky
|
10
|
2.2598400
|
Virginia
|
9
|
1.0736022
|
Indiana
|
8
|
1.2085176
|
Kansas
|
8
|
2.7475915
|
Louisiana
|
8
|
1.7127966
|
Missouri
|
8
|
1.3149953
|
New Jersey
|
8
|
0.8930552
|
Oklahoma
|
8
|
2.0453359
|
Wisconsin
|
8
|
1.3861606
|
Alabama
|
7
|
1.4406319
|
Iowa
|
7
|
2.2407895
|
Oregon
|
7
|
1.7374137
|
Colorado
|
6
|
1.0995911
|
Massachusetts
|
6
|
0.8830773
|
South Carolina
|
6
|
1.2254537
|
West Virginia
|
6
|
3.2535703
|
Mississippi
|
5
|
1.6709370
|
North Dakota
|
5
|
6.6056568
|
South Dakota
|
5
|
5.8243221
|
Tennessee
|
5
|
0.7575414
|
Utah
|
5
|
1.6689370
|
Washington
|
5
|
0.6973159
|
Maine
|
4
|
3.0090392
|
Arkansas
|
3
|
1.0073185
|
Connecticut
|
3
|
0.8354484
|
Nebraska
|
3
|
1.5821199
|
Nevada
|
3
|
1.0377589
|
Hawaii
|
2
|
1.3970354
|
Idaho
|
2
|
1.2085103
|
Maryland
|
2
|
0.3329781
|
Montana
|
2
|
1.9362040
|
New Mexico
|
2
|
0.9591825
|
Wyoming
|
2
|
3.4123462
|
Arizona
|
1
|
0.1464544
|
Rhode Island
|
1
|
0.9467025
|
Sorted by state name
Retail Stores Closing (sorted by state name)
State
|
Total Stores Closing
|
Per Capita (1MM)
|
Alabama
|
7
|
1.4406319
|
Arizona
|
1
|
0.1464544
|
Arkansas
|
3
|
1.0073185
|
California
|
12
|
0.3065540
|
Colorado
|
6
|
1.0995911
|
Connecticut
|
3
|
0.8354484
|
Florida
|
17
|
0.8386252
|
Georgia
|
12
|
1.1747591
|
Hawaii
|
2
|
1.3970354
|
Idaho
|
2
|
1.2085103
|
Illinois
|
12
|
0.9331263
|
Indiana
|
8
|
1.2085176
|
Iowa
|
7
|
2.2407895
|
Kansas
|
8
|
2.7475915
|
Kentucky
|
10
|
2.2598400
|
Louisiana
|
8
|
1.7127966
|
Maine
|
4
|
3.0090392
|
Maryland
|
2
|
0.3329781
|
Massachusetts
|
6
|
0.8830773
|
Michigan
|
22
|
2.2171662
|
Minnesota
|
12
|
2.1859540
|
Mississippi
|
5
|
1.6709370
|
Missouri
|
8
|
1.3149953
|
Montana
|
2
|
1.9362040
|
Nebraska
|
3
|
1.5821199
|
Nevada
|
3
|
1.0377589
|
New Jersey
|
8
|
0.8930552
|
New Mexico
|
2
|
0.9591825
|
New York
|
12
|
0.6061895
|
North Carolina
|
13
|
1.2944595
|
North Dakota
|
5
|
6.6056568
|
Ohio
|
19
|
1.6360379
|
Oklahoma
|
8
|
2.0453359
|
Oregon
|
7
|
1.7374137
|
Pennsylvania
|
23
|
1.7965237
|
Rhode Island
|
1
|
0.9467025
|
South Carolina
|
6
|
1.2254537
|
South Dakota
|
5
|
5.8243221
|
Tennessee
|
5
|
0.7575414
|
Texas
|
22
|
0.8008995
|
Utah
|
5
|
1.6689370
|
Virginia
|
9
|
1.0736022
|
Washington
|
5
|
0.6973159
|
West Virginia
|
6
|
3.2535703
|
Wisconsin
|
8
|
1.3861606
|
Wyoming
|
2
|
3.4123462
|
Compared to unemployment
I’d have used the epidata
to get the current state unemployment data but it’s not current, so we can either use a package to get data from the Bureau of Labor Statistics or just scrape it. We’ll use the U-6 rate since that is an expanded definition including “total unemployed, plus all marginally attached workers, plus total employed part time for economic reasons, as a percent of the civilian labor force plus all marginally attached workers” and is likely to more representative for the populations working at retail chains.
pg <- read_html("https://www.bls.gov/lau/stalt16q4.htm")
html_nodes(pg, "table#alternmeas16\\:IV") %>%
html_table(header = TRUE, fill = TRUE) %>%
.[[1]] %>%
docxtractr::assign_colnames(1) %>%
rename(name=State) %>%
as_data_frame() %>%
slice(2:52) %>%
type_convert() %>%
left_join(closings_by_state, by="name") %>%
filter(!is.na(n)) -> with_unemp
ggplot(with_unemp, aes(per_capita, `U-6`)) +
geom_label(aes(label=name), fill="#8c96c6", color="white", size=3.5, family=font_rc) +
scale_x_continuous(limits=c(-0.125, 6.75)) +
labs(x="Closings per-capita (1MM)",
y="BLS Labor Underutilization (U-6 rate)",
title="Per-capita store closings compared to current BLS U-6 Rate") +
theme_ipsum_rc(grid="XY")
If I were a reporter, I think I’d be digging a bit deeper on the impact of these (and the half-dozen or so other) retailers closing locations in New Mexico, Nevada, West Virginia, Wyoming, (mebbe Maine, though I’m super-b ased :-), North Dakota & South Dakota.
LS0tCnRpdGxlOiAiRXhwbG9yaW5nIDIwMTcgUmV0YWlsIFN0b3JlIENsb3NpbmdzIHdpdGggUiIKb3V0cHV0OgogIGh0bWxfZG9jdW1lbnQ6CiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICBrZWVwX21kOiB0cnVlCiAgICB0aGVtZTogZmxhdGx5CiAgICBoaWdobGlnaHQ6IHRhbmdvCmVkaXRvcl9vcHRpb25zOiAKICBjaHVua19vdXRwdXRfdHlwZTogY29uc29sZQotLS0KYGBge3IgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0UsIGRldj0icG5nIiwgZmlnLnJldGluYT0yKQpgYGAKCmBgYHtjc3N9CmNvZGUgewogIGZvbnQtZmFtaWx5OiBJb3NldmthLCAiRmlyYSBDb2RlIiwgIlNvdXJjZSBDb2RlIFBybyIsIG1vbm9zcGFjZTsKfQpgYGAKCkEgc3RvcnkgYWJvdXQgb25lIG9mIHRoZSByZXRhaWwgY2hhaW5zIChKLkMuIFBlbm55KSByZWxlYXNpbmcgdGhlIGxpc3Qgb2Ygc3RvcmVzIGNsb3NpbmcgaW4gMjAxNyBbY3Jvc3NlZCBwYXRoc10oaHR0cHM6Ly93d3cueWFob28uY29tL25ld3MvamMtcGVubmV5LXJlbGVhc2VzLWZ1bGwtbGlzdC0xMDM0MDMwMDkuaHRtbCkgd2l0aCBteSBGZWVkbHkgcmVhZGluZyBsaXN0IHRvZGF5IGFuZCBqb2dnZWQgbXkgbWVtb3J5IHRoYXQgdGhlcmUgd2VyZSBhIG51bWJlciBvZiBjaGFpbnMgY2xvc2luZyBtYW55IG9mIHRoZWlyIGRvb3JzIHRoaXMgeWVhciBhbmQgSSB3YW50ZWQgdG8gc2VlIHRoZSBpbXBhY3QgdGhhdCBtaWdodCBoYXZlIG9uIHZhcmlvdXMgc3RhdGVzLgoKYGBge3Igc2V0dXB9CmxpYnJhcnkoaHR0cikKbGlicmFyeShydmVzdCkKbGlicmFyeShrbml0cikKbGlicmFyeShrYWJsZUV4dHJhKQpsaWJyYXJ5KGdnYWx0KQpsaWJyYXJ5KHN0YXRlYmlucykKbGlicmFyeShocmJydGhlbWVzKQpsaWJyYXJ5KGVwaWRhdGEpCmxpYnJhcnkodGlkeXZlcnNlKQoKb3B0aW9ucyhrbml0ci50YWJsZS5mb3JtYXQgPSAiaHRtbCIpCnVwZGF0ZV9nZW9tX2ZvbnRfZGVmYXVsdHMoZm9udF9yY19saWdodCwgc2l6ZSA9IDIuNzUpCmBgYAoKIkNsb3NpbmciIGxpc3RzIG9mIGZvdXIgbWFqb3IgcmV0YWlsZXJzIOKAlCBLLU1hcnQsIFNlYXJzLCBNYWN5J3MgYW5kIEouQy4gUGVubnkg4oCUIGFib3VuZCAoSFRNTCBmb3JtYXR0aW5nIGEgbGlzdCBzZWVtcyB0byBiZSB0aGUgImVhc3kgd2F5IG91dCIgc3Rvcnktd2lzZSBmb3IgbWFueSBibG9ncyBhbmQgbmV3c3BhcGVycykuIFdlIGNhbiBkaWcgYSBiaXQgZGVlcGVyIHRoYW4ganVzdCBhIHBsYWluIHNldCBvZiBsaXN0cywgYnV0IGZpcnN0IHdlIG5lZWQgdGhlIGRhdGEuCgpUaGUgQm9zdG9uIEdsb2JlIGhhcyBhIG5pY2UsIHByZWRpY3RhYmxlLCBtb3N0bHktdW5pZm9ybSBwYXR0ZXJuIHRvIHRoZWlyIGxpc3QtY2xvc2luZyAic3RvcmllcyIsIHNvIHdlJ2xsIHVzZSB0aGF0IGRhdGEuIFNpdGUgY29udGVudCBjYW4gY2hhbmdlIHF1aWNrbHksIHNvIGl0IG1ha2VzIHNlbnNlIHRvIHRyeSB0byBjYWNoZSBjb250ZW50IHdoZW5ldmVyIHBvc3NpYmxlIGFzIHdlIHNjcmFwZSBpdC4gVG8gdGhhdCBlbmQsIHdlJ2xsIHVzZSBgaHR0cjo6R0VUYCB2cyBgeG1sMjo6cmVhZF9odG1sYCBzaW5jZSBgR0VUYCBwcmVzZXJ2ZXMgYWxsIG9mIHRoZSBvcmlnaW5hbCByZXF1ZXN0IGFuZCByZXNwb25zZSBpbmZvcm1hdGlvbiBhbmQgYHJlYWRfaHRtbGAgcmV0dXJucyBhbiBleHRlcm5hbCBwb2ludGVyIHRoYXQgaGFzIG5vIGN1cnJlbnQgc3VwcG9ydCBmb3Igc2VyaWFsaXphdGlvbiB3aXRob3V0IGV4dHJhIHdvcmsuCgpgYGB7ciBkYXRhfQpjbG9zaW5ncyA8LSBsaXN0KAogIGttYXJ0ID0gImh0dHBzOi8vd3d3LmJvc3Rvbmdsb2JlLmNvbS9tZXRyby8yMDE3LzAxLzA1L3RoZS1mdWxsLWxpc3Qta21hcnQtc3RvcmVzLWNsb3NpbmctYXJvdW5kLzRrSjBZVm9mVVdIeTVRSlh1UEJBdU0vc3RvcnkuaHRtbCIsCiAgc2VhcnMgPSAiaHR0cHM6Ly93d3cuYm9zdG9uZ2xvYmUuY29tL21ldHJvLzIwMTcvMDEvMDUvdGhlLWZ1bGwtbGlzdC1zZWFycy1zdG9yZXMtY2xvc2luZy1hcm91bmQveUhhUDZuVjJDNGdZdzdLTGh1V3VGTi9zdG9yeS5odG1sIiwKICBtYWN5cyA9ICJodHRwczovL3d3dy5ib3N0b25nbG9iZS5jb20vbWV0cm8vMjAxNy8wMS8wNS90aGUtZnVsbC1saXN0LW1hY3ktc3RvcmVzLWNsb3NpbmctYXJvdW5kLzZUWThhM3Z5N3luZUtWMW5ZY3dZN0svc3RvcnkuaHRtbCIsCiAgICBqY3AgPSAiaHR0cHM6Ly93d3cuYm9zdG9uZ2xvYmUuY29tL2J1c2luZXNzLzIwMTcvMDMvMTcvdGhlLWZ1bGwtbGlzdC1wZW5uZXktc3RvcmVzLWNsb3NpbmctYXJvdW5kL3Zob0hqSTNrNzVrMnBTdVF0Mm1acE8vc3RvcnkuaHRtbCIKKQoKc2F2ZWRfcGdzIDwtICJzYXZlZF9zdG9yZV91cmxzLnJkcyIKCmlmIChmaWxlLmV4aXN0cyhzYXZlZF9wZ3MpKSB7CiAgcGdzIDwtIHJlYWRfcmRzKHNhdmVkX3BncykKfSBlbHNlIHsKICBwZ3MgPC0gbWFwKGNsb3NpbmdzLCBHRVQpCiAgd3JpdGVfcmRzKHBncywgc2F2ZWRfcGdzKQp9CmBgYAoKVGhpcyBpcyB3aGF0IHdlIGdldCBmcm9tIHRoYXQgc2NyYXBpbmcgZWZmb3J0OgoKYGBge3Igc2hvd30KbWFwKHBncywgY29udGVudCkgJT4lCiAgbWFwKGh0bWxfdGFibGUpICU+JQogIHdhbGsofmdsaW1wc2UoLltbMV1dKSkKYGBgCgpXZSBub3cgbmVlZCB0byBub3JtYWxpemUgdGhlIGNvbnRlbnQgb2YgdGhlIGxpc3RzLgoKYGBge3IgbXVuZ2luZ30KbWFwKHBncywgY29udGVudCkgJT4lCiAgbWFwKGh0bWxfdGFibGUpICU+JQogIG1hcCh+LltbMV1dKSAlPiUKICBtYXBfZGYoc2VsZWN0LCBhYmI9MywgLmlkID0gInN0b3JlIikgLT4gY2xvc2luZ3MKYGBgCgpXZSdyZSB1bHRpbWF0ZWx5IGp1c3QgbG9va2luZyBmb3IgY2l0eS9zdGF0ZSBmb3IgdGhpcyBzaW1wbGUgZXhlcmNpc2UsIGJ1dCBvbmUgY291bGQgZG8gbW9yZSBwcmVjaXNlIGdlb2xvY2F0aW9uIChwZXJoYXBzIHdpdGggW2ByZ2VvY29kaW9gXShodHRwczovL2dpdGh1Yi5jb20vaHJicm1zdHIvcmdlb2NvZGlvKSkgYW5kIGNvbWJpbmUgdGhhdCB3aXRoIGxvY2FsIHBvcHVsYXRpb24gZGF0YSwgam9iIGxvc3MgZXN0aW1hdGVzLCBjdXJyZW50IHVuZW1wbG95bWVudCBsZXZlbHMsIGV0Yy4gdG8gbWFrZSBhIF9yZWFsXyBzdG9yeSBvdXQgb2YgdGhlIGNsb3NpbmdzIHZzIGp1c3QgZG8gdGhlIGVhc3kgdGhpbmcgYW5kIHB1Ymxpc2ggYSBsaXN0IG9mIHN0b3Jlcy4KCmBgYHtyIHRhYnVsYXRpb259CmNvdW50KGNsb3NpbmdzLCBhYmIpICU+JQogIGxlZnRfam9pbihkYXRhX2ZyYW1lKG5hbWUgPSBzdGF0ZS5uYW1lLCBhYmIgPSBzdGF0ZS5hYmIpKSAlPiUKICBsZWZ0X2pvaW4odXNtYXA6OnN0YXRlcG9wLCBieSA9IGMoImFiYiI9ImFiYnIiKSkgJT4lCiAgbXV0YXRlKHBlcl9jYXBpdGEgPSAobi9wb3BfMjAxNSkgKiAxMDAwMDAwKSAlPiUKICBzZWxlY3QobmFtZSwgbiwgcGVyX2NhcGl0YSkgLT4gY2xvc2luZ3NfYnlfc3RhdGUKYGBgCgojIyAyMDE3IFJldGFpbCBTdG9yZXMgKENvbWJpbmVkKSBDbG9zaW5nIHsudGFic2V0IC50YWJzZXQtZmFkZSAudGFic2V0LXBpbGxzfQoKU2VhcnMsIEouQy4gUGVubnksIEstTWFydCBhbmQgTWFjeSdzIGFyZSBhbGwgc2h1dHRpbmcgZG93biByZXRhaWwgbG9jYXRpb25zIGFjcm9zcyB0aGUgVS5TLgoKIyMjIFNvcnRlZCBieSBjbG9zaWdzIHBlci1jYXBpdGEgKDFNTSkKCjx0YWJsZSBjbGFzcz0idGFibGUiIHN0eWxlPSJ3aWR0aDogYXV0byAhaW1wb3J0YW50OyBtYXJnaW4tbGVmdDogYXV0bzsgbWFyZ2luLXJpZ2h0OiBhdXRvOyI+PHRyPjx0ZD4KCmBgYHtyIHNiMSwgZWNobz1GQUxTRSwgZmlnLndpZHRoPTcsIGZpZy5oZWlnaHQ9NX0Kc3RhdGViaW5zX2NvbnRpbnVvdXMoY2xvc2luZ3NfYnlfc3RhdGUsIHN0YXRlX2NvbD0ibmFtZSIsIHZhbHVlX2NvbD0icGVyX2NhcGl0YSIsIAogICAgICAgICAgbGVnZW5kX3RpdGxlID0gIkNsb3NpbmdzIHBlci1jYXBpdGEgKDFNTSkiKQpgYGAKCjwvdGQ+PC90cj48L3RhYmxlPgoKPHRhYmxlIGNsYXNzPSJ0YWJsZSIgc3R5bGU9IndpZHRoOiBhdXRvICFpbXBvcnRhbnQ7IG1hcmdpbi1sZWZ0OiBhdXRvOyBtYXJnaW4tcmlnaHQ6IGF1dG87Ij48dHI+PHRkPgoKYGBge3IgY2hhcnQxLCBlY2hvPUZBTFNFLCBmaWcud2lkdGg9NywgZmlnLmhlaWdodD04fQphcnJhbmdlKGNsb3NpbmdzX2J5X3N0YXRlLCBwZXJfY2FwaXRhKSAlPiUKICBtdXRhdGUobmFtZSA9IGZhY3RvcihuYW1lLCBuYW1lKSkgJT4lCiAgZ2dwbG90KGFlcyhwZXJfY2FwaXRhLCBuYW1lKSkgKwogIGdlb21fbG9sbGlwb3AoaG9yaXpvbnRhbCA9IFRSVUUpICsKICBzY2FsZV94X2NvbnRpbnVvdXMobmFtZT0iQ2xvc2luZ3MgcGVyIGNhcGl0YSAoMU1NKSIsIGV4cGFuZD1jKDAsMCksIGxpbWl0cz1jKDAsNy41KSkgKwogIGdlb21fbGFiZWwoYWVzKGxhYmVsPXNwcmludGYoIiUxLjJmIChuPSVkKSIsIHBlcl9jYXBpdGEsIG4pKSwKICAgICAgICAgICAgIGhqdXN0PTAsIG51ZGdlX3g9MC4xLCBsYWJlbC5zaXplPTApICsKICBsYWJzKHk9TlVMTCkgKwogIHRoZW1lX2lwc3VtX3JjKGdyaWQ9IlgiKSArCiAgdGhlbWUoYXhpcy50ZXh0Lng9ZWxlbWVudF9ibGFuaygpKQpgYGAKCjwvdGQ+PC90cj48L3RhYmxlPgoKYGBge3IgdGFibGUxLCBlY2hvPUZBTFNFfQphcnJhbmdlKGNsb3NpbmdzX2J5X3N0YXRlLCBkZXNjKHBlcl9jYXBpdGEpKSAlPiUKICBzZWxlY3QoU3RhdGU9bmFtZSwgYFRvdGFsIFN0b3JlcyBDbG9zaW5nYD1uLCBgUGVyIENhcGl0YSAoMU1NKWA9cGVyX2NhcGl0YSkgJT4lCiAga2FibGUoYWxpZ24gPSAibHJyIiwgY2FwdGlvbj0iUmV0YWlsIFN0b3JlcyBDbG9zaW5nIChzb3J0ZWQgYnkgcGVyLWNhcGl0YSkiKSAlPiUKICBrYWJsZV9zdHlsaW5nKGZ1bGxfd2lkdGggPSBGQUxTRSwgYm9vdHN0cmFwX29wdGlvbnMgPSAiY29uZGVuc2VkIikKYGBgCgojIyMgU29ydGVkIGJ5IHRvdGFsIHN0b3JlcyBjbG9zaW5nCgo8dGFibGUgY2xhc3M9InRhYmxlIiBzdHlsZT0id2lkdGg6IGF1dG8gIWltcG9ydGFudDsgbWFyZ2luLWxlZnQ6IGF1dG87IG1hcmdpbi1yaWdodDogYXV0bzsiPjx0cj48dGQ+CgpgYGB7ciBzYjIsIGVjaG89RkFMU0UsIGZpZy53aWR0aD03LCBmaWcuaGVpZ2h0PTV9CnN0YXRlYmluc19jb250aW51b3VzKGNsb3NpbmdzX2J5X3N0YXRlLCBzdGF0ZV9jb2w9Im5hbWUiLCB2YWx1ZV9jb2w9Im4iLCAKICAgICAgICAgIGxlZ2VuZF90aXRsZSA9ICJUb3RhbCBjbG9zaW5ncyBwZXItc3RhdGUiKQpgYGAKCjwvdGQ+PC90cj48L3RhYmxlPgoKPHRhYmxlIGNsYXNzPSJ0YWJsZSIgc3R5bGU9IndpZHRoOiBhdXRvICFpbXBvcnRhbnQ7IG1hcmdpbi1sZWZ0OiBhdXRvOyBtYXJnaW4tcmlnaHQ6IGF1dG87Ij48dHI+PHRkPgoKYGBge3IgY2hhcnQyLCBlY2hvPUZBTFNFLCBmaWcud2lkdGg9NywgZmlnLmhlaWdodD04fQphcnJhbmdlKGNsb3NpbmdzX2J5X3N0YXRlLCBuKSAlPiUKICBtdXRhdGUobmFtZSA9IGZhY3RvcihuYW1lLCBuYW1lKSkgJT4lCiAgZ2dwbG90KGFlcyhuLCBuYW1lKSkgKwogIGdlb21fbG9sbGlwb3AoaG9yaXpvbnRhbCA9IFRSVUUpICsKICBzY2FsZV94X2NvbnRpbnVvdXMobmFtZT0iVG90YWwgY2xvc2luZ3MiLCBleHBhbmQ9YygwLDApLCBsaW1pdHM9YygwLDMwKSkgKwogIGdlb21fbGFiZWwoYWVzKGxhYmVsPW4pLCBoanVzdD0wLCBudWRnZV94PTAuMjUsIGxhYmVsLnNpemU9MCkgKwogIGxhYnMoeT1OVUxMKSArCiAgdGhlbWVfaXBzdW1fcmMoZ3JpZD0iWCIpICsKICB0aGVtZShheGlzLnRleHQueD1lbGVtZW50X2JsYW5rKCkpCmBgYAoKPC90ZD48L3RyPjwvdGFibGU+CgpgYGB7ciB0YWJsZTIsIGVjaG89RkFMU0V9CmFycmFuZ2UoY2xvc2luZ3NfYnlfc3RhdGUsIGRlc2MobikpICU+JQogIHNlbGVjdChTdGF0ZT1uYW1lLCBgVG90YWwgU3RvcmVzIENsb3NpbmdgPW4sIGBQZXIgQ2FwaXRhICgxTU0pYD1wZXJfY2FwaXRhKSAlPiUKICBrYWJsZShhbGlnbiA9ICJscnIiLCBjYXB0aW9uPSJSZXRhaWwgU3RvcmVzIENsb3NpbmcgKHNvcnRlZCBieSB0b3RhbCBzdG9yZXMpIikgJT4lCiAga2FibGVfc3R5bGluZyhmdWxsX3dpZHRoID0gRkFMU0UsIGJvb3RzdHJhcF9vcHRpb25zID0gImNvbmRlbnNlZCIpCmBgYAoKIyMjIFNvcnRlZCBieSBzdGF0ZSBuYW1lCgpgYGB7ciB0YWJsZTMsIGVjaG89RkFMU0V9CmFycmFuZ2UoY2xvc2luZ3NfYnlfc3RhdGUsIG5hbWUpICU+JQogIHNlbGVjdChTdGF0ZT1uYW1lLCBgVG90YWwgU3RvcmVzIENsb3NpbmdgPW4sIGBQZXIgQ2FwaXRhICgxTU0pYD1wZXJfY2FwaXRhKSAlPiUKICBrYWJsZShhbGlnbiA9ICJscnIiLCBjYXB0aW9uPSJSZXRhaWwgU3RvcmVzIENsb3NpbmcgKHNvcnRlZCBieSBzdGF0ZSBuYW1lKSIpICU+JQogIGthYmxlX3N0eWxpbmcoZnVsbF93aWR0aCA9IEZBTFNFLCBib290c3RyYXBfb3B0aW9ucyA9ICJjb25kZW5zZWQiKQpgYGAKCiMjIENvbXBhcmVkIHRvIHVuZW1wbG95bWVudAoKSSdkIGhhdmUgdXNlZCB0aGUgW2BlcGlkYXRhYF0oaHR0cHM6Ly9jcmFuLnJzdHVkaW8uY29tL3dlYi9wYWNrYWdlcy9lcGlkYXRhLykgdG8gZ2V0IHRoZSBjdXJyZW50IHN0YXRlIHVuZW1wbG95bWVudCBkYXRhIGJ1dCBpdCdzIG5vdCBjdXJyZW50LCBzbyB3ZSBjYW4gZWl0aGVyIHVzZSBhIHBhY2thZ2UgdG8gZ2V0IGRhdGEgZnJvbSB0aGUgQnVyZWF1IG9mIExhYm9yIFN0YXRpc3RpY3Mgb3IganVzdCBzY3JhcGUgaXQuIFdlJ2xsIHVzZSB0aGUgVS02IHJhdGUgc2luY2UgdGhhdCBpcyBhbiBleHBhbmRlZCBkZWZpbml0aW9uIGluY2x1ZGluZyBfInRvdGFsIHVuZW1wbG95ZWQsIHBsdXMgYWxsIG1hcmdpbmFsbHkgYXR0YWNoZWQgd29ya2VycywgcGx1cyB0b3RhbCBlbXBsb3llZCBwYXJ0IHRpbWUgZm9yIGVjb25vbWljIHJlYXNvbnMsIGFzIGEgcGVyY2VudCBvZiB0aGUgY2l2aWxpYW4gbGFib3IgZm9yY2UgcGx1cyBhbGwgbWFyZ2luYWxseSBhdHRhY2hlZCB3b3JrZXJzIl8gYW5kIGlzIGxpa2VseSB0byBtb3JlIHJlcHJlc2VudGF0aXZlIGZvciB0aGUgcG9wdWxhdGlvbnMgd29ya2luZyBhdCByZXRhaWwgY2hhaW5zLgoKYGBge3IgYmxzLCBmaWcud2lkdGg9OCwgZmlnLmhlaWdodD03fQpwZyA8LSByZWFkX2h0bWwoImh0dHBzOi8vd3d3LmJscy5nb3YvbGF1L3N0YWx0MTZxNC5odG0iKQoKaHRtbF9ub2RlcyhwZywgInRhYmxlI2FsdGVybm1lYXMxNlxcOklWIikgJT4lIAogIGh0bWxfdGFibGUoaGVhZGVyID0gVFJVRSwgZmlsbCA9IFRSVUUpICU+JQogIC5bWzFdXSAlPiUgCiAgZG9jeHRyYWN0cjo6YXNzaWduX2NvbG5hbWVzKDEpICU+JSAKICByZW5hbWUobmFtZT1TdGF0ZSkgJT4lIAogIGFzX2RhdGFfZnJhbWUoKSAlPiUgCiAgc2xpY2UoMjo1MikgJT4lIAogIHR5cGVfY29udmVydCgpICU+JSAKICBsZWZ0X2pvaW4oY2xvc2luZ3NfYnlfc3RhdGUsIGJ5PSJuYW1lIikgJT4lIAogIGZpbHRlcighaXMubmEobikpIC0+IHdpdGhfdW5lbXAKCmdncGxvdCh3aXRoX3VuZW1wLCBhZXMocGVyX2NhcGl0YSwgYFUtNmApKSArCiAgZ2VvbV9sYWJlbChhZXMobGFiZWw9bmFtZSksIGZpbGw9IiM4Yzk2YzYiLCBjb2xvcj0id2hpdGUiLCBzaXplPTMuNSwgZmFtaWx5PWZvbnRfcmMpICsKICBzY2FsZV94X2NvbnRpbnVvdXMobGltaXRzPWMoLTAuMTI1LCA2Ljc1KSkgKwogIGxhYnMoeD0iQ2xvc2luZ3MgcGVyLWNhcGl0YSAoMU1NKSIsIAogICAgICAgeT0iQkxTIExhYm9yIFVuZGVydXRpbGl6YXRpb24gKFUtNiByYXRlKSIsCiAgICAgICB0aXRsZT0iUGVyLWNhcGl0YSBzdG9yZSBjbG9zaW5ncyBjb21wYXJlZCB0byBjdXJyZW50IEJMUyBVLTYgUmF0ZSIpICsKICB0aGVtZV9pcHN1bV9yYyhncmlkPSJYWSIpCmBgYAoKSWYgSSB3ZXJlIGEgcmVwb3J0ZXIsIEkgdGhpbmsgSSdkIGJlIGRpZ2dpbmcgYSBiaXQgZGVlcGVyIG9uIHRoZSBpbXBhY3Qgb2YgdGhlc2UgKGFuZCB0aGUgaGFsZi1kb3plbiBvciBzbyBvdGhlcikgcmV0YWlsZXJzIGNsb3NpbmcgbG9jYXRpb25zIGluIE5ldyBNZXhpY28sIE5ldmFkYSwgV2VzdCBWaXJnaW5pYSwgV3lvbWluZywgKG1lYmJlIE1haW5lLCB0aG91Z2ggSSdtIHN1cGVyLWIgYXNlZCA6LSksIE5vcnRoIERha290YSAmIFNvdXRoIERha290YS4K