- 1 National Occupational mean wage
- 1.1 Occupational wage data
- 1.1.1 Initial exploration of the data
- 1.2 Hierarchical clustering
- 1.2.1 Occupation trees
- 1.2.2 Preparing for exploration
- 1.2.3 Plotting occupational clusters
- 1.3 Kmeans
- 1.3.1 Elbow analysis
- 1.3.2 Average Silhouette Widths
- 1.4 The “best” number of clusters
Case Study: National Occupational mean wage
1 National Occupational mean wage
1.1 Occupational wage data
1.1.1 Initial exploration of the data
We are presented with data from the Occupational Employment Statistics (OES) program which produces employment and wage estimates annually. This data contains the yearly average income from 2001 to 2016 for 22 occupation groups. You would like to use this data to identify clusters of occupations that maintained similar income trends.
Before we begin to cluster this data we should determine whether any pre-processing steps (such as scaling and imputation) are necessary.
Leverage the functions head() and summary() to explore the oes data in order to determine which of the pre-processing steps below are necessary: there are no missing values, no categorical and the features are on the same scale.
1.2 Hierarchical clustering
1.2.1 Occupation trees
We will take the necessary steps to build a dendrogram of occupations based on their yearly average salaries and propose clusters using a height of 100,000.
oes <- readRDS("oes.rds")# Calculate Euclidean distance between the occupations
dist_oes <- dist(oes, method = "euclidean")
# Generate an average linkage analysis
hc_oes <- hclust(dist_oes, method = "average")
# Create a dendrogram object from the hclust variable
dend_oes <- as.dendrogram(hc_oes)
# Plot the dendrogram
plot(dend_oes)
# Color branches by cluster formed from the cut at a height of 100000
dend_colored <- color_branches(dend_oes, h = 100000)
# Plot the colored dendrogram
plot(dend_colored)
Based on the dendrogram it may be reasonable to start with the three clusters formed at a height of 100,000. The members of these clusters appear to be tightly grouped but different from one another.
1.2.2 Preparing for exploration
We have now created a potential clustering for the oes data, before we can explore these clusters with ggplot2 we will need to process the oes data matrix into a tidy data frame with each occupation assigned its cluster.
dist_oes <- dist(oes, method = 'euclidean')
hc_oes <- hclust(dist_oes, method = 'average')
library(tibble)
Attaching package: ‘tibble’
The following object is masked from ‘package:wrapr’:
viewlibrary(tidyr)
# Use rownames_to_column to move the rownames into a column of the data frame
df_oes <- rownames_to_column(as.data.frame(oes), var = 'occupation')
# Create a cluster assignment vector at h = 100,000
cut_oes <- cutree(hc_oes, h = 100000)
# Generate the segmented the oes data frame
clust_oes <- mutate(df_oes, cluster = cut_oes)
# Create a tidy data frame by gathering the year and values into two columns
gathered_oes <- gather(data = clust_oes,
key = year,
value = mean_salary,
-occupation, -cluster)head(gathered_oes)occupation <chr> | cluster <int> | year <chr> | mean_salary <dbl> | |
|---|---|---|---|---|
| 1 | Management | 1 | 2001 | 70800 |
| 2 | Business Operations | 2 | 2001 | 50580 |
| 3 | Computer Science | 2 | 2001 | 60350 |
| 4 | Architecture/Engineering | 2 | 2001 | 56330 |
| 5 | Life/Physical/Social Sci. | 2 | 2001 | 49710 |
| 6 | Community Services | 3 | 2001 | 34190 |
We now have the data frames necessary to explore the results of this clustering
1.2.3 Plotting occupational clusters
You have succesfully created all the parts necessary to explore the results of this hierarchical clustering work. Now, we will leverage the named assignment vector cut_oes and the tidy data frame gathered_oes to analyze the resulting clusters.
# View the clustering assignments by sorting the cluster assignment vector
sort(cut_oes) Management Legal
1 1
Business Operations Computer Science
2 2
Architecture/Engineering Life/Physical/Social Sci.
2 2
Healthcare Practitioners Community Services
2 3
Education/Training/Library Arts/Design/Entertainment
3 3
Healthcare Support Protective Service
3 3
Food Preparation Grounds Cleaning & Maint.
3 3
Personal Care Sales
3 3
Office Administrative Farming/Fishing/Forestry
3 3
Construction Installation/Repair/Maint.
3 3
Production Transportation/Moving
3 3 # Plot the relationship between mean_salary and year and color the lines by the assigned cluster
ggplot(gathered_oes, aes(x = year, y = mean_salary, color = factor(cluster))) +
geom_line(aes(group = occupation))
From this work it looks like both Management & Legal professions (cluster 1) experienced the most rapid growth in these 15 years.
1.3 Kmeans
1.3.1 Elbow analysis
We will leverage the k-means elbow plot to propose the “best” number of clusters.
We use map_dbl() to run kmeans() using the oes data for k values ranging from 1 to 10 and extract the total within-cluster sum of squares value from each model: model$tot.withinss
# Use map_dbl to run many models with varying value of k (centers)
tot_withinss <- map_dbl(1:10, function(k){
model <- kmeans(x = oes, centers = k)
model$tot.withinss
})We store the resulting vector as tot_withinss
The new data frame elbow_df containing the values of k and the vector of total within-cluster sum of squares
# Generate a data frame containing both k and tot_withinss
elbow_df <- data.frame(
k = 1:10,
tot_withinss = tot_withinss
)
# Plot the elbow plot
ggplot(elbow_df, aes(x = k, y = tot_withinss)) +
geom_line() +
scale_x_continuous(breaks = 1:10)
So the elbow analysis proposes a different value of k.
1.3.2 Average Silhouette Widths
So hierarchical clustering resulting in 3 clusters and the elbow method suggests 2. We will use average silhouette widths to explore what the “best” value of k should be.
# Use map_dbl to run many models with varying value of k
sil_width <- map_dbl(2:10, function(k){
model <- pam(oes, k = k)
model$silinfo$avg.width
})
# Generate a data frame containing both k and sil_width
sil_df <- data.frame(
k = 2:10,
sil_width = sil_width
)
# Plot the relationship between k and sil_width
ggplot(sil_df, aes(x = k, y = sil_width)) +
geom_line() +
scale_x_continuous(breaks = 2:10)
It seems that this analysis results in another value of k, this time 7 is the top contender (although 2 comes very close).
1.4 The “best” number of clusters
We ran three different methods for finding the optimal number of clusters and their assignments and we arrived with three different answers.
What can you say about the “best” way to cluster this data?
- The clusters generated by the hierarchical clustering all have members with a Euclidean distance amongst one another less than 100,000 and hence is the best clustering method.
- The clusters generated using k-means with a k = 2 was identified using elbow analysis and hence is the best way to cluster this data.
- The clusters generated using k-means with a k = 7 has the largest Average Silhouette Widths among the cluster and hence is the best way to cluster this data. But the best way to cluster is highly dependent on how you would use this data after. There is no quantitative way to determine which of these clustering approaches is the right one without further exploration.
LS0tCnRpdGxlOiAiQ2FzZSBTdHVkeTogTmF0aW9uYWwgT2NjdXBhdGlvbmFsIG1lYW4gd2FnZSIKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6CiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgdG9jX2NvbGxhcHNlZDogZmFsc2UKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgCnRvY19kZXB0aDogMwotLS0KIyBOYXRpb25hbCBPY2N1cGF0aW9uYWwgbWVhbiB3YWdlCgojIyBPY2N1cGF0aW9uYWwgd2FnZSBkYXRhCgojIyMgSW5pdGlhbCBleHBsb3JhdGlvbiBvZiB0aGUgZGF0YQoKV2UgYXJlIHByZXNlbnRlZCB3aXRoIGRhdGEgZnJvbSB0aGUgT2NjdXBhdGlvbmFsIEVtcGxveW1lbnQgU3RhdGlzdGljcyAoT0VTKSBwcm9ncmFtIHdoaWNoIHByb2R1Y2VzIGVtcGxveW1lbnQgYW5kIHdhZ2UgZXN0aW1hdGVzIGFubnVhbGx5LiBUaGlzIGRhdGEgY29udGFpbnMgdGhlIHllYXJseSBhdmVyYWdlIGluY29tZSBmcm9tIDIwMDEgdG8gMjAxNiBmb3IgMjIgb2NjdXBhdGlvbiBncm91cHMuIFlvdSB3b3VsZCBsaWtlIHRvIHVzZSB0aGlzIGRhdGEgdG8gaWRlbnRpZnkgY2x1c3RlcnMgb2Ygb2NjdXBhdGlvbnMgdGhhdCBtYWludGFpbmVkIHNpbWlsYXIgaW5jb21lIHRyZW5kcy4KCkJlZm9yZSB3ZSBiZWdpbiB0byBjbHVzdGVyIHRoaXMgZGF0YSB3ZSBzaG91bGQgZGV0ZXJtaW5lIHdoZXRoZXIgYW55IHByZS1wcm9jZXNzaW5nIHN0ZXBzIChzdWNoIGFzIHNjYWxpbmcgYW5kIGltcHV0YXRpb24pIGFyZSBuZWNlc3NhcnkuCgpMZXZlcmFnZSB0aGUgZnVuY3Rpb25zIGhlYWQoKSBhbmQgc3VtbWFyeSgpIHRvIGV4cGxvcmUgdGhlIG9lcyBkYXRhIGluIG9yZGVyIHRvIGRldGVybWluZSB3aGljaCBvZiB0aGUgcHJlLXByb2Nlc3Npbmcgc3RlcHMgYmVsb3cgYXJlIG5lY2Vzc2FyeTogdGhlcmUgYXJlIG5vIG1pc3NpbmcgdmFsdWVzLCBubyBjYXRlZ29yaWNhbCBhbmQgdGhlIGZlYXR1cmVzIGFyZSBvbiB0aGUgc2FtZSBzY2FsZS4KCiMjIEhpZXJhcmNoaWNhbCBjbHVzdGVyaW5nCgojIyMgT2NjdXBhdGlvbiB0cmVlcwoKV2Ugd2lsbCB0YWtlIHRoZSBuZWNlc3Nhcnkgc3RlcHMgdG8gYnVpbGQgYSBkZW5kcm9ncmFtIG9mIG9jY3VwYXRpb25zIGJhc2VkIG9uIHRoZWlyIHllYXJseSBhdmVyYWdlIHNhbGFyaWVzIGFuZCBwcm9wb3NlIGNsdXN0ZXJzIHVzaW5nIGEgaGVpZ2h0IG9mIDEwMCwwMDAuCmBgYHtyfQpvZXMgPC0gcmVhZFJEUygib2VzLnJkcyIpCmBgYApgYGB7cn0KIyBDYWxjdWxhdGUgRXVjbGlkZWFuIGRpc3RhbmNlIGJldHdlZW4gdGhlIG9jY3VwYXRpb25zCmRpc3Rfb2VzIDwtIGRpc3Qob2VzLCBtZXRob2QgPSAiZXVjbGlkZWFuIikKCiMgR2VuZXJhdGUgYW4gYXZlcmFnZSBsaW5rYWdlIGFuYWx5c2lzIApoY19vZXMgPC0gaGNsdXN0KGRpc3Rfb2VzLCBtZXRob2QgPSAiYXZlcmFnZSIpCgojIENyZWF0ZSBhIGRlbmRyb2dyYW0gb2JqZWN0IGZyb20gdGhlIGhjbHVzdCB2YXJpYWJsZQpkZW5kX29lcyA8LSBhcy5kZW5kcm9ncmFtKGhjX29lcykKCiMgUGxvdCB0aGUgZGVuZHJvZ3JhbQpwbG90KGRlbmRfb2VzKQoKIyBDb2xvciBicmFuY2hlcyBieSBjbHVzdGVyIGZvcm1lZCBmcm9tIHRoZSBjdXQgYXQgYSBoZWlnaHQgb2YgMTAwMDAwCmRlbmRfY29sb3JlZCA8LSBjb2xvcl9icmFuY2hlcyhkZW5kX29lcywgaCA9IDEwMDAwMCkKCiMgUGxvdCB0aGUgY29sb3JlZCBkZW5kcm9ncmFtCnBsb3QoZGVuZF9jb2xvcmVkKQoKYGBgCkJhc2VkIG9uIHRoZSBkZW5kcm9ncmFtIGl0IG1heSBiZSByZWFzb25hYmxlIHRvIHN0YXJ0IHdpdGggdGhlIHRocmVlIGNsdXN0ZXJzIGZvcm1lZCBhdCBhIGhlaWdodCBvZiAxMDAsMDAwLiBUaGUgbWVtYmVycyBvZiB0aGVzZSBjbHVzdGVycyBhcHBlYXIgdG8gYmUgdGlnaHRseSBncm91cGVkIGJ1dCBkaWZmZXJlbnQgZnJvbSBvbmUgYW5vdGhlci4KCiMjIyBQcmVwYXJpbmcgZm9yIGV4cGxvcmF0aW9uCgpXZSBoYXZlIG5vdyBjcmVhdGVkIGEgcG90ZW50aWFsIGNsdXN0ZXJpbmcgZm9yIHRoZSBvZXMgZGF0YSwgYmVmb3JlIHdlIGNhbiBleHBsb3JlIHRoZXNlIGNsdXN0ZXJzIHdpdGggZ2dwbG90MiB3ZSB3aWxsIG5lZWQgdG8gcHJvY2VzcyB0aGUgb2VzIGRhdGEgbWF0cml4IGludG8gYSB0aWR5IGRhdGEgZnJhbWUgd2l0aCBlYWNoIG9jY3VwYXRpb24gYXNzaWduZWQgaXRzIGNsdXN0ZXIuCgpgYGB7cn0KZGlzdF9vZXMgPC0gZGlzdChvZXMsIG1ldGhvZCA9ICdldWNsaWRlYW4nKQpoY19vZXMgPC0gaGNsdXN0KGRpc3Rfb2VzLCBtZXRob2QgPSAnYXZlcmFnZScpCgpsaWJyYXJ5KHRpYmJsZSkKbGlicmFyeSh0aWR5cikKCiMgVXNlIHJvd25hbWVzX3RvX2NvbHVtbiB0byBtb3ZlIHRoZSByb3duYW1lcyBpbnRvIGEgY29sdW1uIG9mIHRoZSBkYXRhIGZyYW1lCmRmX29lcyA8LSByb3duYW1lc190b19jb2x1bW4oYXMuZGF0YS5mcmFtZShvZXMpLCB2YXIgPSAnb2NjdXBhdGlvbicpCgojIENyZWF0ZSBhIGNsdXN0ZXIgYXNzaWdubWVudCB2ZWN0b3IgYXQgaCA9IDEwMCwwMDAKY3V0X29lcyA8LSBjdXRyZWUoaGNfb2VzLCBoID0gMTAwMDAwKQoKIyBHZW5lcmF0ZSB0aGUgc2VnbWVudGVkIHRoZSBvZXMgZGF0YSBmcmFtZQpjbHVzdF9vZXMgPC0gbXV0YXRlKGRmX29lcywgY2x1c3RlciA9IGN1dF9vZXMpCgojIENyZWF0ZSBhIHRpZHkgZGF0YSBmcmFtZSBieSBnYXRoZXJpbmcgdGhlIHllYXIgYW5kIHZhbHVlcyBpbnRvIHR3byBjb2x1bW5zCmdhdGhlcmVkX29lcyA8LSBnYXRoZXIoZGF0YSA9IGNsdXN0X29lcywgCiAgICAgICAgICAgICAgICAgICAgICAga2V5ID0geWVhciwgCiAgICAgICAgICAgICAgICAgICAgICAgdmFsdWUgPSBtZWFuX3NhbGFyeSwgCiAgICAgICAgICAgICAgICAgICAgICAgLW9jY3VwYXRpb24sIC1jbHVzdGVyKQpgYGAKYGBge3J9CmhlYWQoZ2F0aGVyZWRfb2VzKQpgYGAKV2Ugbm93IGhhdmUgdGhlIGRhdGEgZnJhbWVzIG5lY2Vzc2FyeSB0byBleHBsb3JlIHRoZSByZXN1bHRzIG9mIHRoaXMgY2x1c3RlcmluZwoKIyMjIFBsb3R0aW5nIG9jY3VwYXRpb25hbCBjbHVzdGVycwoKWW91IGhhdmUgc3VjY2VzZnVsbHkgY3JlYXRlZCBhbGwgdGhlIHBhcnRzIG5lY2Vzc2FyeSB0byBleHBsb3JlIHRoZSByZXN1bHRzIG9mIHRoaXMgaGllcmFyY2hpY2FsIGNsdXN0ZXJpbmcgd29yay4gTm93LCB3ZSB3aWxsIGxldmVyYWdlIHRoZSBuYW1lZCBhc3NpZ25tZW50IHZlY3RvciBjdXRfb2VzIGFuZCB0aGUgdGlkeSBkYXRhIGZyYW1lIGdhdGhlcmVkX29lcyB0byBhbmFseXplIHRoZSByZXN1bHRpbmcgY2x1c3RlcnMuCmBgYHtyfQojIFZpZXcgdGhlIGNsdXN0ZXJpbmcgYXNzaWdubWVudHMgYnkgc29ydGluZyB0aGUgY2x1c3RlciBhc3NpZ25tZW50IHZlY3Rvcgpzb3J0KGN1dF9vZXMpCgojIFBsb3QgdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIG1lYW5fc2FsYXJ5IGFuZCB5ZWFyIGFuZCBjb2xvciB0aGUgbGluZXMgYnkgdGhlIGFzc2lnbmVkIGNsdXN0ZXIKZ2dwbG90KGdhdGhlcmVkX29lcywgYWVzKHggPSB5ZWFyLCB5ID0gbWVhbl9zYWxhcnksIGNvbG9yID0gZmFjdG9yKGNsdXN0ZXIpKSkgKyAKICAgIGdlb21fbGluZShhZXMoZ3JvdXAgPSBvY2N1cGF0aW9uKSkKYGBgCkZyb20gdGhpcyB3b3JrIGl0IGxvb2tzIGxpa2UgYm90aCBNYW5hZ2VtZW50ICYgTGVnYWwgcHJvZmVzc2lvbnMgKGNsdXN0ZXIgMSkgZXhwZXJpZW5jZWQgdGhlIG1vc3QgcmFwaWQgZ3Jvd3RoIGluIHRoZXNlIDE1IHllYXJzLgoKIyMgS21lYW5zCgojIyMgRWxib3cgYW5hbHlzaXMKCldlIHdpbGwgbGV2ZXJhZ2UgdGhlIGstbWVhbnMgZWxib3cgcGxvdCB0byBwcm9wb3NlIHRoZSAiYmVzdCIgbnVtYmVyIG9mIGNsdXN0ZXJzLgoKV2UgdXNlIG1hcF9kYmwoKSB0byBydW4ga21lYW5zKCkgdXNpbmcgdGhlIG9lcyBkYXRhIGZvciBrIHZhbHVlcyByYW5naW5nIGZyb20gMSB0byAxMCBhbmQgZXh0cmFjdCB0aGUgdG90YWwgd2l0aGluLWNsdXN0ZXIgc3VtIG9mIHNxdWFyZXMgdmFsdWUgZnJvbSBlYWNoIG1vZGVsOiBtb2RlbCR0b3Qud2l0aGluc3MKYGBge3J9CiMgVXNlIG1hcF9kYmwgdG8gcnVuIG1hbnkgbW9kZWxzIHdpdGggdmFyeWluZyB2YWx1ZSBvZiBrIChjZW50ZXJzKQp0b3Rfd2l0aGluc3MgPC0gbWFwX2RibCgxOjEwLCAgZnVuY3Rpb24oayl7CiAgbW9kZWwgPC0ga21lYW5zKHggPSBvZXMsIGNlbnRlcnMgPSBrKQogIG1vZGVsJHRvdC53aXRoaW5zcwp9KQpgYGAKV2Ugc3RvcmUgdGhlIHJlc3VsdGluZyB2ZWN0b3IgYXMgdG90X3dpdGhpbnNzCgpUaGUgbmV3IGRhdGEgZnJhbWUgZWxib3dfZGYgY29udGFpbmluZyB0aGUgdmFsdWVzIG9mIGsgYW5kIHRoZSB2ZWN0b3Igb2YgdG90YWwgd2l0aGluLWNsdXN0ZXIgc3VtIG9mIHNxdWFyZXMKYGBge3J9CiMgR2VuZXJhdGUgYSBkYXRhIGZyYW1lIGNvbnRhaW5pbmcgYm90aCBrIGFuZCB0b3Rfd2l0aGluc3MKZWxib3dfZGYgPC0gZGF0YS5mcmFtZSgKICBrID0gMToxMCwKICB0b3Rfd2l0aGluc3MgPSB0b3Rfd2l0aGluc3MKKQoKIyBQbG90IHRoZSBlbGJvdyBwbG90CmdncGxvdChlbGJvd19kZiwgYWVzKHggPSBrLCB5ID0gdG90X3dpdGhpbnNzKSkgKwogIGdlb21fbGluZSgpICsKICBzY2FsZV94X2NvbnRpbnVvdXMoYnJlYWtzID0gMToxMCkKYGBgClNvIHRoZSBlbGJvdyBhbmFseXNpcyBwcm9wb3NlcyBhIGRpZmZlcmVudCB2YWx1ZSBvZiBrLgoKIyMjIEF2ZXJhZ2UgU2lsaG91ZXR0ZSBXaWR0aHMKClNvIGhpZXJhcmNoaWNhbCBjbHVzdGVyaW5nIHJlc3VsdGluZyBpbiAzIGNsdXN0ZXJzIGFuZCB0aGUgZWxib3cgbWV0aG9kIHN1Z2dlc3RzIDIuIFdlIHdpbGwgdXNlIGF2ZXJhZ2Ugc2lsaG91ZXR0ZSB3aWR0aHMgdG8gZXhwbG9yZSB3aGF0IHRoZSAiYmVzdCIgdmFsdWUgb2YgayBzaG91bGQgYmUuCmBgYHtyfQojIFVzZSBtYXBfZGJsIHRvIHJ1biBtYW55IG1vZGVscyB3aXRoIHZhcnlpbmcgdmFsdWUgb2YgawpzaWxfd2lkdGggPC0gbWFwX2RibCgyOjEwLCAgZnVuY3Rpb24oayl7CiAgbW9kZWwgPC0gcGFtKG9lcywgayA9IGspCiAgbW9kZWwkc2lsaW5mbyRhdmcud2lkdGgKfSkKCiMgR2VuZXJhdGUgYSBkYXRhIGZyYW1lIGNvbnRhaW5pbmcgYm90aCBrIGFuZCBzaWxfd2lkdGgKc2lsX2RmIDwtIGRhdGEuZnJhbWUoCiAgayA9IDI6MTAsCiAgc2lsX3dpZHRoID0gc2lsX3dpZHRoCikKCiMgUGxvdCB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gayBhbmQgc2lsX3dpZHRoCmdncGxvdChzaWxfZGYsIGFlcyh4ID0gaywgeSA9IHNpbF93aWR0aCkpICsKICBnZW9tX2xpbmUoKSArCiAgc2NhbGVfeF9jb250aW51b3VzKGJyZWFrcyA9IDI6MTApCmBgYApJdCBzZWVtcyB0aGF0IHRoaXMgYW5hbHlzaXMgcmVzdWx0cyBpbiBhbm90aGVyIHZhbHVlIG9mIGssIHRoaXMgdGltZSA3IGlzIHRoZSB0b3AgY29udGVuZGVyIChhbHRob3VnaCAyIGNvbWVzIHZlcnkgY2xvc2UpLgoKIyMgVGhlICJiZXN0IiBudW1iZXIgb2YgY2x1c3RlcnMKCldlIHJhbiB0aHJlZSBkaWZmZXJlbnQgbWV0aG9kcyBmb3IgZmluZGluZyB0aGUgb3B0aW1hbCBudW1iZXIgb2YgY2x1c3RlcnMgYW5kIHRoZWlyIGFzc2lnbm1lbnRzIGFuZCB3ZSBhcnJpdmVkIHdpdGggdGhyZWUgZGlmZmVyZW50IGFuc3dlcnMuCgoKV2hhdCBjYW4geW91IHNheSBhYm91dCB0aGUgImJlc3QiIHdheSB0byBjbHVzdGVyIHRoaXMgZGF0YT8KCi0JVGhlIGNsdXN0ZXJzIGdlbmVyYXRlZCBieSB0aGUgaGllcmFyY2hpY2FsIGNsdXN0ZXJpbmcgYWxsIGhhdmUgbWVtYmVycyB3aXRoIGEgRXVjbGlkZWFuIGRpc3RhbmNlIGFtb25nc3Qgb25lIGFub3RoZXIgbGVzcyB0aGFuIDEwMCwwMDAgYW5kIGhlbmNlIGlzIHRoZSBiZXN0IGNsdXN0ZXJpbmcgbWV0aG9kLgotCVRoZSBjbHVzdGVycyBnZW5lcmF0ZWQgdXNpbmcgay1tZWFucyB3aXRoIGEgayA9IDIgd2FzIGlkZW50aWZpZWQgdXNpbmcgZWxib3cgYW5hbHlzaXMgYW5kIGhlbmNlIGlzIHRoZSBiZXN0IHdheSB0byBjbHVzdGVyIHRoaXMgZGF0YS4KLQlUaGUgY2x1c3RlcnMgZ2VuZXJhdGVkIHVzaW5nIGstbWVhbnMgd2l0aCBhIGsgPSA3IGhhcyB0aGUgbGFyZ2VzdCBBdmVyYWdlIFNpbGhvdWV0dGUgV2lkdGhzIGFtb25nIHRoZSBjbHVzdGVyIGFuZCBoZW5jZSBpcyB0aGUgYmVzdCB3YXkgdG8gY2x1c3RlciB0aGlzIGRhdGEuCkJ1dCB0aGUgYmVzdCB3YXkgdG8gY2x1c3RlciBpcyBoaWdobHkgZGVwZW5kZW50IG9uIGhvdyB5b3Ugd291bGQgdXNlIHRoaXMgZGF0YSBhZnRlci4gVGhlcmUgaXMgbm8gcXVhbnRpdGF0aXZlIHdheSB0byBkZXRlcm1pbmUgd2hpY2ggb2YgdGhlc2UgY2x1c3RlcmluZyBhcHByb2FjaGVzIGlzIHRoZSByaWdodCBvbmUgd2l0aG91dCBmdXJ0aGVyIGV4cGxvcmF0aW9uLgo=