Unit 3 Walkthrough: Topic Modeling in MOOC-Eds

0. INTRODUCTION

The Unit 3 walkthrough extends previous research and evaluation work at the Friday Institute for Educational Innovation at North Carolina State University. In addition to many other areas of inquiry, this work was aimed at understanding and improving peer interaction and discussion in the Friday Institute’s Massively Open Online Courses for Educators (MOOC-Ed) and Online Professional Learning programs. To learn more about these courses and programs, visit: https://place.fi.ncsu.edu

Walkthrough Focus

Our focus this week will be on identifying “topics” by examining how words cohere into different latent, or hidden, themes based on patterns of co-occurrence of words within documents. With a bit of tongue-in-cheek, Meeks and Weingart (2012) describe topic modeling as:

…focused on corpora and not individual texts, treating the works themselves as unceremonious ‘buckets of words,’ and providing seductive but obscure results in the forms of easily interpreted (and manipulated) ‘topics’…. To achieve its results, it leverages occult statistical methods like ‘dirichlet priors’ and ‘bayesian models.’ That being said, Weingart also noted that “a topic model is a”clever and exceptionally versatile little algorithm that can be customized to all sorts of applications” and Bail (2020) add that topic modeling can be “a powerful tool for identifying general trends in a corpus that can then be analyzed in a more granular manner using other techniques.”

With respect to the actual R workflow of applying topic models to documents and text of interests, Silge & Robinson and a new bottom row their flowchart consisting new data structures (i.e., a corpus object and document-term matrix) and and the LDA model:

This week will be also be our first introduction to the “Model” process of the data-intensive workflow described in our course text, Learning Analytics Goes to School. As noted by Krumm and Means (2018), this workflow is not always a linear process and there is often a great deal of iteration that occurs within and between wrangling, exploring, modeling. As illustrated by our workflow below, this week we will primarily explore our data after the modeling process in order to gain some additional insight into the topics generated by our model. Specifically, this week covers the following concepts and skills:

Prepare: Prior to analysis, we’ll take a quick look at some of the related MOOC-Ed research and evaluation work to gain some context for our analysis. This should aid in the interpretation of our results and help guide some decisions as we tidy, model, and visualize our data.
Wrangle: In section 2 we again revisit tidying and tokenizing text using the tidytext package but are also introduced to the the stm package. This package makes use of tm text mining package to preprocess text and will also be our first introduction to word stemming.
Model: We take a look at two different approaches to topic modeling: Latent Dirichlet Allocation (LDA) and Structural Topic Modeling (STM), which is very similar to LDA but can use metadata about documents to improve the assignment of words to “topics” in a corpus and examine relationships between topics and covariates.
Explore: To further explore the results of our topic model, we use several handy functions from the topicmodels and stm packages, including the findThoughts function for viewing documents assigned to a given topic and the toLDAvis function for exploring topic and word distributions.

1. PREPARE

To help us better understand the context, questions, and data sources we’ll be using in Unit 3, this section will focus on the following topics:

Context. As context for our analysis this week, we’ll review several related papers by my colleagues relevant to our analysis of MOOC-Ed discussion forums.
Questions. We’ll also examine what insight topic modeling can provide to a question that we asked participants answer in their professional learning teams (PLTs).
Project Setup. This should be very familiar by now, but we’ll set up a new R project and install and load the required packages for the topic modeling walkthrough.

1a. Context

Participating in a MOOC and Professional Learning Team: How a Blended Approach to Professional Development Makes a Difference

Full text: https://www.learntechlib.org/p/195234/

Abstract

Massive Open Online Courses for Educators (MOOC-Eds) provide opportunities for using research-based learning and teaching practices, along with new technological tools and facilitation approaches for delivering quality online professional development. The Teaching Statistics Through Data Investigations MOOC-Ed was built for preparing teachers in pedagogy for teaching statistics, and it has been offered to participants from around the world. During 2016-2017, professional learning teams (PLTs) were formed from a subset of MOOC-Ed participants. These teams met several times to share and discuss their learning and experiences. This study focused on examining the ways that a blended approach to professional development may result in similar or different patterns of engagement to those who only participate in a large-scale online course. Results show the benefits of a blended learning environment for retention, engagement with course materials, and connectedness within the online community of learners in an online professional development on teaching statistics. The findings suggest the use of self-forming autonomous PLTs for supporting a deeper and more comprehensive experience with self-directed online professional developments such as MOOCs. Other online professional development courses, such as MOOCs, may benefit from purposely suggesting and advertising, and perhaps facilitating, the formation of small face-to-face or virtual PLTs who commit to engage in learning together.

Data Source & Analysis

All peer interaction, including peer discussion, take place within discussion forums of MOOC-Eds, which are hosted using the Moodle Learning Management System. To build the dataset you’ll be using for this walkthrough, the research team wrote a query for Moodle’s MySQL database, which records participants’ user-logs of activity in the online forums. This sql query combines separate database tables containing postings and comments including participant IDs, timestamps, discussion text and other attributes or “metadata.”

For further description of the forums and data retrieval process, see also the following papers:

Kellogg, S., & Edelmann, A. (2015). Massively Open Online Course for Educators (MOOC‐Ed) network dataset. British journal of educational technology, 46(5), 977-983.
Ezen-Can, A., Boyer, K. E., Kellogg, S., & Booth, S. (2015, March). Unsupervised modeling for understanding MOOC discussion forums: a learning analytics approach. In Proceedings of the fifth international conference on learning analytics and knowledge (pp. 146-150).
Kellogg, S., Booth, S., & Oliver, K. (2014). A social network perspective on peer supported learning in MOOCs for educators. International Review of Research in Open and Distributed Learning, 15(5), 263-289.

Summary of Key Findings

The following highlight some key findings related to the discussion forums in the papers cited above:

MOOCs designed specifically for K-12 teachers can provide positive self-directed learning experiences and rich engagement in discussion forums that help form online communities for educators.
Analysis of discussion forum data in TSDI provided a very clear picture of how enthusiastic many PLT members and leaders were to talk to others in the online community. They posed their questions and shared ideas with others about teaching statistics throughout the units, even though they were also meeting synchronously several times with their colleagues in small group PLTs.
Findings on knowledge construction demonstrated that over half of the discussions in both courses moved beyond sharing information and statements of agreement and entered a process of dissonance, negotiation and co-construction of knowledge, but seldom moved beyond this phase in which new knowledge was tested or applied. These findings echo similar research on difficulties in promoting knowledge construction in online settings.
Topic modeling provides more interpretable and cohesive models for discussion forums than other popular unsupervised modeling techniques such as k-means and k-medoids clustering algorithms.

1b. Guiding Questions

For the paper, Participating in a MOOC and Professional Learning Team: How a Blended Approach to Professional Development Makes a Difference, the researchers were interested in unpacking how participants who enrolled in the Teaching Statistics through Data Investigations MOOC-Ed might benefit from also being in a smaller group of professionals committed to engaging in the same professional development. The specific research question for this paper was:

What are the similarities and differences between how PLT members and Non-PLT online participants engage and meet course goals in a MOOC-Ed designed for educators in secondary and collegiate settings? Dr. Hollylynne Lee and the TSDI team also developed a facilitation guide designed specifically for PLT teams to help groups synthesize the ideas in the course and make plans for how to implement new strategies in their classroom in order to impact students’ learning of statistics. One question PLT members were asked to address was:

What ideas or issues emerged in the discussion forums this past week? For this walkthrough, we will further examine that question through the use of topic modeling.

And just to reiterate yet again from Unit 1, one overarching question we’ll explore throughout this course, and that Silge and Robinson (2018) identify as a central question to text mining and natural language processing, is:

How do we to quantify what a document or collection of documents is about?

1c. Set Up

As highlighted in Chapter 6 of Data Science in Education Using R (DSIEUR), one of the first steps of every workflow should be to set up a “Project” within RStudio. This will be your “home” for any files and code used or created in Unit 2.

You are welcome to continue using the same project created for Unit 1, or create an entirely new project for Unit 2. However, after you’ve created your project open up a new R script, and load the following packages that we’ll be needing for this walkthrough:

library(tidyverse)
library(tidytext)
library(SnowballC)
library(topicmodels)
library(stm)
library(ldatuning)
library(knitr)
library(LDAvis)

At the end of this week, I encourage you share with me your R script as evidence that you have complete the walkthrough. Although I highly recommend that that you manually type the code shared throughout this walkthrough, for large blocks of text it may be easier to copy and paste.

2. WRANGLE

As noted previously, data wrangling involves some combination of cleaning, reshaping, transforming, and merging data (Wickham & Grolemund, 2017). This week we’ll revisit tidying and tokenizing text using the tidytext package, but are also introduced to the the stm package. This package makes use of tm text mining package to preprocess text (e.g., removing punctuation, stop words, etc.) and will also be our first introduction to word stemming.

Import Data. We’ll be working with .csv files this week and the read_csv() function but will introduce a new argument for changing column types.
Cast a DTM. We revisit the tidytext package to “tidy” and tokenize our forum data and introduce the cast_dtm() function to create the document term matrix (dtm) need for topic modeling.
To Stem or not to STEM? We conclude our data wrangling by also introducing the textProcessor() function for preprocessing and discuss the pros and cons of word stemming.

2a. Import Forum Data

To get started, we need to import, or “read”, our data into R. The function used to import your data will depend on the file format of the data you are trying to import. First, however, you’ll need to do the following:

Download the ts_forum_data.csv file we’ll be using for this Unit from our NCSU Moodle course site.
Create a folder in the directory on your computer where you stored your R Project and name it “data”.
Add the file to your data folder.
Check your Files tab in RStudio to verify that your file is indeed in your data folder.

Now let’s read our data into our Environment using the read_csv() function and assign it to a variable name so we can work with it like any other object in R.

ts_forum_data <- read_csv("data/ts_forum_data.csv", 
     col_types = cols(course_id = col_character(),
                   forum_id = col_character(), 
                   discussion_id = col_character(), 
                   post_id = col_character()
                   )
    )

By default, many of the columns like course_id and forum_id are read in as numeric data. For our purposes, we plan to treat them as unique identifiers or names for out courses, forums, discussions, and posts. The read_csv() function has a handy col_types = argument changing the column types from numeric to characters.

2b. Cast a Document Term Matrix

In this section we’ll revisit some familiar tidytext functions used in Units 1 & 2 for tidying and tokenizing text and introduce some new functions from the stm package for processing text and transforming our data frames into new data structures required for topic modeling.

Functions Used

tidytext functions

unnest_tokens() splits a column into tokens
anti_join() returns all rows from x without a match in y and used to remove stop words from out data.
cast_dtm() takes a tidied data frame take and “casts” it into a document-term matrix (dtm)

dplyr functions

count() lets you quickly count the unique values of one or more variables
group_by() takes a data frame and one or more variables to group by
summarise() creates a summary of data using arguments like sum and mean

stm functions

textProcessor() takes in a vector or column of raw texts and performs text processing like removing punctuation and word stemming.
prepDocuments() performs several corpus manipulations including removing words and renumbering word indices

Tidying Text

Prior to topic modeling, we have a few remaining steps to tidy our text that hopefully should feel familiar by this point. If you recall from Chapter 1 of Text Mining With R, these preprocessing steps include:

Transforming our text into “tokens”
Removing unnecessary characters, punctuation, and whitespace
Converting all text to lowercase
Removing stop words such as “the”, “of”, and “to”

Let’s tokenize our forum text and by using the familiar unnest_tokens() and remove stop words per usual:

forums_tidy <- ts_forum_data %>%
  unnest_tokens(output = word, input = post_content) %>%
  anti_join(stop_words, by = "word")
forums_tidy

## # A tibble: 165,720 × 10
##    course_id course_name      forum_id forum_name  discussion_id discussion_name
##    <chr>     <chr>            <chr>    <chr>       <chr>         <chr>          
##  1 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  2 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  3 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  4 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  5 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  6 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  7 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  8 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  9 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
## 10 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
## # … with 165,710 more rows, and 4 more variables: post_id <chr>,
## #   post_title <chr>, post_date <chr>, word <chr>

Now let’s do a quick word count to see some of the most common words used throughout the forums. This should get a sense of what we’re working with and later we’ll need these word counts for creating our document term matrix for topic modeling:

forums_tidy %>%
  count(word, sort = TRUE)

## # A tibble: 13,666 × 2
##    word            n
##    <chr>       <int>
##  1 students     6837
##  2 data         4338
##  3 statistics   3095
##  4 school       1488
##  5 questions    1470
##  6 time         1252
##  7 class        1208
##  8 agree         999
##  9 teaching      987
## 10 statistical   957
## # … with 13,656 more rows

Terms like “students,” “data,” and “class” are about what we would have expected from a course teaching statistics. The term “agree” and “time” however, are not so intuitive and worth a quick look as well.

✅ Comprehension Check

Use the filter() and grepl() functions introduced in Unit 1. Section 3b to filter for rows in our ts_forum_data data frame that contain the terms “agree” and “time” and another term or terms of your choosing. Select a random sample of 10 posts using the sample_n() function for your terms and answer the following questions:

What, if anything, do these posts have in common? For ‘agree,’ the posts are all responses to someone else. They are amplifying something that was said. ‘Time’ comments focused a lot on planning courses and giving students time to do the work. My third word was ‘math.’ ‘Math’ discussions focused on the difficulty of teaching specific concepts and how the simulation helped students learn faster.
What topics or themes might be apparent, or do you anticipate emerging, from our topic modeling? Planning, simulation, student skill levels, teaching statistics, teaching math.

Your output should look something like this:

## # A tibble: 10 × 1
##    post_content                                                                 
##    <chr>                                                                        
##  1 "Hi George!  Good question- and here is my attempt to give you my opinion.  …
##  2 "Hi Lilian   Thank you for your very productive comment. I give students the…
##  3 "That's a great question Maggie. I think as education students we should ded…
##  4 "You make some excellent point about extensions to this task.  Urban areas w…
##  5 "I agree with you. I think that this helps with the whole ownership part but…
##  6 "I agree the television task requires the students to think more than the ot…
##  7 "This is the first time I have participated in a MOOC offered by the Friday …
##  8 "In the beginning two students said that one lived in Raleigh and one in the…
##  9 "I teach high school and I am surprised that there is less statistics than p…
## 10 "One of my moments that I thought about was students being able to determine…

Creating a Document Term Matrix

As highlighted in Chapter 5 of Text Mining with R, the topicmodels package and the Latent Dirichlet allocation (LDA) algorithm and LDA() function it uses expects document-term matrix as the data input.

Before we create a our document-term matrix, however, we have an important decision to make:

What do we consider to be a “document” in a MOOC-Ed discussion forum? For example, we could consider each individual discussion post, or post_id in our data frame, as a document. It might also make sense to combine texts from all posts within each discussion, or disccussion_id, and consider that as a document since these posts are often interconnected an build off one another.

For now, however, let’s treat each individual post as a unique “document.” As noted above, to create our document term matrix, we’ll need to first count() how many times each word occurs in each document, or post_id in our case, and create a matrix that contains one row per post as our original data frame did, but now contains a column for each word in the entire corpus and a value of n for how many times that word occurs in each post.

To create this document term matrix from our post counts, we’ll use the cast_dtm() function like so and assign it to the variable forums_dtm:

forums_dtm <- forums_tidy %>%
  count(post_id, word) %>%
  cast_dtm(post_id, word, n)

✅ Comprehension Check

Take a look at our forums_dtm object in the console and answer the following question:

What “class” of object is forums_dtm? simple_triplet_matrix
How many unique documents and terms are included our matrix? 5,761 documents and 13,666 terms.
Why might there be fewer documents/posts than were in our original data frame? Blank posts would have nothing to count or possibly figures not recognized as words, like emojis.
What exactly is meant by “sparsity”? The amount of the DTM that has zeros as counts for terms. This is a relative value of how often you see common words throughout a corpus. The higher the value, the easier it is to potentially distinguish individual documents. A lower sparsity implies many of the same words keep appearing in different documents.

## [1] "DocumentTermMatrix"    "simple_triplet_matrix"

## <<DocumentTermMatrix (documents: 5761, terms: 13666)>>
## Non-/sparse entries: 135796/78594030
## Sparsity           : 100%
## Maximal term length: 320
## Weighting          : term frequency (tf)

2c. To Stem or not to Stem?

Next we’ll need to prepare our original data set for structural topic modeling using the textProcessor() function. The stm package has a number of features that extend the functionality of the topicmodels package, including an argument for “stemming” words, which Schofield and Mimno (2016) describe as follows:

Stemming is a popular way to reduce the size of a vocabulary in natural language tasks by conflating words with related meanings. Specifically, stemming aims to convert words with the same “stem” or root (e.g “creative” and “creator”) to a single word type (“create”). Though originally developed in the context of information retrieval (IR) systems, stemmers are now commonly used as a preprocessing step in unsupervised machine learning tasks. The rationale behind stemming is that it can dramatically reduce the number of words or terms to be modeled, which in theory should help simplify and improve the performance of your model. We’ll explore this assumption a little later in this section.

Processing and Stemming for STM

Like unnest_tokens(), the textProcessor() function includes several useful arguments for processing text like converting text to lowercase and removing punctuation and numbers. I’ve included several of these in the script below along with their defaults used if you do not explicitly specify in your function. Most of these are pretty intuitive and you can learn more by viewing the ?textProcessor documentation.

Let’s go ahead and process our discussion forum post_content in preparation for structural topic modeling:

temp <- textProcessor(ts_forum_data$post_content, 
                    metadata = ts_forum_data,  
                    lowercase=TRUE, 
                    removestopwords=TRUE, 
                    removenumbers=TRUE,  
                    removepunctuation=TRUE, 
                    wordLengths=c(3,Inf),
                    stem=TRUE,
                    onlycharacter= FALSE, 
                    striphtml=TRUE, 
                    customstopwords=NULL)

## Building corpus... 
## Converting to Lower Case... 
## Removing punctuation... 
## Removing stopwords... 
## Removing numbers... 
## Stemming... 
## Creating Output...

Note that the first argument the textProcessor function expects is the column in our data frame that contains the text to be processed, the second argument metadata = expects the data frame that contains the text of interest and uses the column names to label the metadata such as course ids and forum names. This meatdata can be used to to improve the assignment of words to topics in a corpus and examine the relationship between topics and various covariates of interest.

Unlike the unnest_tokens() function, the output is not a nice tidy data frame. Topic modeling using the stm package requires a very unique set of inputs that are specific to the package. The following code will pull elements from the temp list that was created that will be required for the stm() function we’ll use in Section 4:

meta <- temp$meta
vocab <- temp$vocab
docs <- temp$documents

Stemming Tidy Text

Notice that the textProcessor stem argument we used above is set to TRUE by default. We haven’t introduced word stemming at this point because there is some debate about the actual value of this process. While words like “students” and “student” might make sense to collapse into their base word and actually make analyses and visualizations more concise and easier to interpret. Hvitfeldt and Silge (2021) note, however, that words like the following have dramatic differences in meaning, semantics, and use and could result in poor models or misinterpreted results:

meaning and mean
likely, like, liking
university and universe

The first word pair is particularly relevant to discussion posts from our Teaching Statistics course data. In addition, collapsing words like “teachers” and “teaching” could dramatically alter the results from a topic model.

Schofield and Mimno (2016) specifically,

Despite their frequent use in topic modeling, we find that stemmers produce no meaningful improvement in likelihood and coherence and in fact can degrade topic stability. For now, we will leave as is the forums_dtm we created earlier with words unstemmed, but what if we wanted to stem words in a “tidy” way?

Since the unnest_tokens() function does not (intentionally I believe) include a stemming function, one approach would be to use the wordStem() function from the snowballC package to either replace our words column with a word stems or create a new variable called stem with our stemmed words. Let’s do the latter and take a look at the original words and the stem that was produced:

stemmed_forums <- ts_forum_data %>%
  unnest_tokens(output = word, input = post_content) %>%
  anti_join(stop_words, by = "word") %>%
  mutate(stem = wordStem(word))
stemmed_forums

## # A tibble: 165,720 × 11
##    course_id course_name      forum_id forum_name  discussion_id discussion_name
##    <chr>     <chr>            <chr>    <chr>       <chr>         <chr>          
##  1 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  2 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  3 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  4 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  5 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  6 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  7 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  8 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
##  9 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
## 10 9         Teaching Statis… 126      Investigat… 6822          Not much compa…
## # … with 165,710 more rows, and 5 more variables: post_id <chr>,
## #   post_title <chr>, post_date <chr>, word <chr>, stem <chr>

You can see that words like “activity” and “activities” that occur frequently in our discussions have been reduced to the word stem “activ”. If you are interested in learning other approaches for word stemming in R, as well as reading a more in depth description of the stemming process, I highly recommend the Chapter 4 Stemming from Hvitfeldt and Silge (2021) book, Supervised Machine Learning for Text Analysis in R.

✅ Comprehension Check

Complete the following code using what we learned in the section on Creating a Document Term Matrix and answer the following questions:

How many fewer terms are in our stemmed document term matrix? 3,606
Did stemming words significantly reduce the sparsity of the network? No

Hint: Make sure your code includes stem counts rather than word counts.

stemmed_dtm <- ts_forum_data %>%
  unnest_tokens(output = word, input = post_content) %>%
  anti_join(stop_words, by = "word") %>%
  mutate(stem = wordStem(word)) %>%
  count(post_id, stem, sort = TRUE) %>%
  cast_dtm(post_id, stem, n)
  
stemmed_dtm

## <<DocumentTermMatrix (documents: 5761, terms: 10060)>>
## Non-/sparse entries: 129593/57826067
## Sparsity           : 100%
## Maximal term length: 320
## Weighting          : term frequency (tf)

## <<DocumentTermMatrix (documents: 5761, terms: 13666)>>
## Non-/sparse entries: 135796/78594030
## Sparsity           : 100%
## Maximal term length: 320
## Weighting          : term frequency (tf)

## # A tibble: 10,060 × 2
##    stem         n
##    <chr>    <int>
##  1 student   7346
##  2 data      4338
##  3 statist   4152
##  4 question  2470
##  5 teach     1841
##  6 school    1606
##  7 class     1520
##  8 time      1424
##  9 learn     1355
## 10 task      1214
## # … with 10,050 more rows

3. MODEL

This unit provides our first opportunity for modeling a text as data. In very simple terms, modeling involves developing a mathematical summary of a dataset. These summaries can help us further explore trends and patterns in our data.

In their book, Learning Analytics Goes to School, Krumm and Means (2018) describe two general types of modeling approaches used in the Data-Intensive Research workflow: unsupervised and supervised machine learning. In distinguishing between the two, they note:

Unsupervised learning algorithms can be used to understand the structure of one’s dataset. Supervised models, on the other hand, help to quantify relationships between features and a known outcome. Known outcomes are also commonly referred to as labels or dependent variables. In Section 3 we focus on Topic Modeling, an unsupervised learning approach to automatically identify topics in a collection of documents. In fact, we’ll explore two different approaches to topic modeling, as well as strategies for identifying the “right” number of topics:

Fitting a Topic Modeling with LDA. In this section we learn to use the topicmodels package and associated LDA() function for unsupervised classification of our forum discussions to find natural groupings of words, or topics.
Fitting a Structural Topic Model. We then explore the use of the stm package and stm() function to fit our model and uses metadata about documents to improve the assignment of words to “topics” in a corpus.
Choosing K. Finally, we wrap up Section 3 by learning about diagnostic properties like exclusivity, semantic coherence, and heldout likelihood for helping to select an appropriate number of topics.

3a. Fitting a Topic Modeling with LDA

Before running our first topic model using the LDA() function, let’s quick recap from our readings some basic principles behind Latent Dirichlet allocation and why LDA is preferred over other automatic classification or clustering approaches.

Unlike simple forms of cluster analysis such as k-means clustering, LDA is a “mixture” model, which in our context means that:

Every document contains a mixture of topics. Unlike algorithms like k-means, LDA treats each document as a mixture of topics, which allows documents to “overlap” each other in terms of content, rather than being separated into discrete groups. So in practice, this means that a discussion forum post could have an estimated topic proportion of 70% for Topic 1 (e.g. be mostly about a Topic 1), but also be partly about Topic 2.
Every topic contains a mixture of words. For example, if we specified in our LDA model just 2 topics for our discussion posts, we might find that one topic seems to be about pedagogy while another is about learning. The most common words in the pedagogy topic might be “teacher”, “strategies”, and “instruction”, while the learning topic may be made up of words like “understanding” and “students”. However, words can be shared between topics and words like “statistics” or “assessment” might appear in both equally.

Similar to k-means and other simple clustering approaches, however, LDA does require us to specify a value of k for the number of topics in our corpus. Selecting k is no trivial matter and can greatly impact your results.

Since we don’t have a have strong rationale about the number of topics that might exist in discussion forums, let’s use the n_distinct() function from the dplyr package to find the number of unique forum names in our course data and run with that:

n_distinct(ts_forum_data$forum_name)

## [1] 20

Since it looks like there are 20 distinct discussion forums, we’ll use that as our value for the k = argument of the LDA(). Be patient while this runs, since the default setting of is to perform a large number of iterations.

n_distinct(ts_forum_data$forum_name)

## [1] 20

forums_lda <- LDA(forums_dtm, 
                  k = 20, 
                  control = list(seed = 588)
                  )
forums_lda

## A LDA_VEM topic model with 20 topics.

Note that we used the control = argument to pass a random number (588) to seed the assignment of topics to each word in our corpus. Since LDA is a stochastic algorithm that could have different results depending on where the algorithm starts, we specified a seed for reproducibility so we’re all seeing the same results every time we specify the same number of topics.

And tying back to our work in Unit 1, Bail (2020) notes that topic assignments for each word are updated in an iterative fashion and that LDA employs the Term Frequency-Inverse Document Frequency (TF-IDF) metric to assign probabilities.

3b. Fitting a Structural Topic Model

Bail notes that LDA, while perhaps the most common approach to topic modeling, is just one of many different types, including Dynamic Topic Models, Correlated Topic Models, Hierarchical Topic Models, and more recently, Structural Topic Modeling (STM). He argues that one reason STM has rising in popularity and use is that it employs meta data about documents to improve the assignment of words to topics in a corpus and that can be used to examine relationships between covariates and documents.

Also, since Julia Silge has indicated that STM is, “my current favorite implementation of topic modeling in R” and has built supports in the tidytext package for building structural topic models, this package definitely is worth discussing in this walkthrough. I also highly recommend her own walkthrough of the stm package: The game is afoot! Topic modeling of Sherlock Holmes stories as well as her follow up post, Training, evaluating, and interpreting topic models.

The `stm` Package

As we’ve seen above, STM produced an unusual temp textProcessor output that is unique to the stm package. And as you’ve probably already guessed, the stm() function for fitting a structural topic model does not take a fairly standard document term matrix like the LDA() function.

Before we fit our model, we’ll have to extract the elements from the temp object created after we processed our text. Specifically, the stm() function expects the following arguments:

documents = the document term matrix to be modeled in the native stm format
data = an optional data frame containing meta data for the prevalence and/or content covariates to include in the model
vocab = a character vector specifying the words in the corpus in the order of the vocab indices in documents.

Let’s go ahead and extract these elements:

docs <- temp$documents 
meta <- temp$meta 
vocab <- temp$vocab

And now use these elements to fit the model using the same number of topics for K that we specified for our LDA topic model. Let’s also take advantage of the fact that we can include the course_id and forum_id covariates in the prevealence = argument to help improve, in theory, our model fit:

forums_stm <- stm(documents=docs, 
         data=meta,
         vocab=vocab, 
         prevalence =~ course_id + forum_id,
         K=20,
         max.em.its=25,
         verbose = FALSE)
forums_stm

## A topic model with 20 topics, 5777 documents and a 9306 word dictionary.

As noted earlier, the stm package has a number of handy features. One of these is the plot.STM() function for viewing the most probable words assigned to each topic.

By default, it only shows the first 3 terms so let’s change that to 5 to help with interpretation:

plot.STM(forums_stm, n = 5)

Note that you can also just use plot() as well:

plot(forums_stm, n = 5)

✅ Comprehension Check

Fit a model for both LDA and STM using different values for K and answer the following questions:

What topics appear to be similar to those using 20 topics for K? For the LDA model, I used a K value of 10 and topics 1, 2, 5, and 7 look similar. For STM, using a K value of 10 yielded similarities only in topics 7 and 8.
Knowing that you don’t have as much context as the researchers of this study do, how might you interpret one of these latent topics or themes using the key terms assigned? For the LDA topics, Topic 5 could be interpreted as data manipulation tasks. For the STM run, Topic 8 could be interpreted as student technology use.
What topic emerged that seem dramatically different and how might you interpret this topic? For the LDA model, topic 10 was very different. I could interpret topic 10 as the blending of two possible areas: Problems understanding statistics and good assessment resources. For the STM runs, many tasks were extremely different.

3c. Finding K

As alluded to earlier, selecting the number of topics for your model is a non-trivial decision and can dramatically impact your results. Bail (2018) notes that

The results of topic models should not be over-interpreted unless the researcher has strong theoretical apriori about the number of topics in a given corpus, or if the researcher has carefully validated the results of a topic model using both the quantitative and qualitative techniques described above. There are several approaches to estimating a value for K and we’ll take a quick look at one from the ldatuning package and one from our stm package.

The FindTopicsNumber Function

The ldatuning package has functions for both calculating and plotting different metrics that can be used to estimate the most preferable number of topics for LDA model. It also conveniently takes the standard document term matrix object that we created from out tidy text and has the added benefit of running fairly quickly, especially compared to the function for finding K from the stm package.

Let’s use the defaults specified in the ?FindTopicNumber documentation and modify slightly get metrics for a sequence of topics from 10-75 counting by 5 and plot the output we saved using the FindTopicsNumber_plot() function:

k_metrics <- FindTopicsNumber(
  forums_dtm,
  topics = seq(10, 75, by = 5),
  metrics = "Griffiths2004",
  method = "Gibbs",
  control = list(),
  mc.cores = NA,
  return_models = FALSE,
  verbose = FALSE,
  libpath = NULL
)
FindTopicsNumber_plot(k_metrics)

Note that the FindTopicNumbers() function contains three additional metrics for calculating metrics that can be used to estimate the most preferable number of topics for LDA model. We used the Griffiths2004 metrics included in the default example and I’ve also found this to produce the most interpretable results as show in the figure below:

As a general rule of thumb and overly simplistic heuristic, we’re looking for an inflection point in our plot which indicates an optimal number of topics to select for a value of K.

The findingK() Function

Finally, Bail (2018) notes that thestm package has a useful function called searchK which allows us to specify a range of values for k and outputs multiple goodness-of-fit measures that are “very useful in identifying a range of values for k that provide the best fit for the data.”

The syntax of this function is very similar to the stm() function we used above, except that we specify a range for k as one of the arguments. In the code below, we search all values of k between 10 and 30.

#I am not expecting you run this code as it will take too long
#findingk <- searchK(docs, 
                    #vocab, 
                    #K = c(5:15),
                    #data = meta, 
                    #verbose=FALSE)
#plot(findingk)

Note that Running searchK() function on this corpus took all night on a pretty powerful MacBook Pro and crashed once as well, so I do not expect you to run this for the walkthrough. I ran a couple iterations and landed on between 5 and 15 with an optimal number of topics somewhere around 14:

Given the somewhat conflicting results, also somewhat selfishly and for the same of simplicity for this walkthrough, I’m just going to stick with the rather arbitrary selection of 20 topics for the remainder of this Unit.

The LDAvis Explorer

One final tool that I want to introduce from the stm package is the toLDAvis() function which provides a great visualizations for exploring topic and word distributions using LDAvis topic browser:

toLDAvis(mod = forums_stm, docs = docs)

## Loading required namespace: servr

As you can see from the browser screen shot below, our current stm model of 20 topics is resulting in a lot of overlap among topics and suggest that 20 may not be an optimal number of topics, as other approaches for finding k also suggests:

4. EXPLORE

Silge and Robinson (2018) note that fitting at topic model is the “easy part.” The hard part is making sense of the model results and that the rest of the analysis involves exploring and interpreting the model using a variety of approaches which we’ll walkthrough in in this section.

Bail (2018) cautions, however, that:

…post-hoc interpretation of topic models is rather dangerous… and can quickly come to resemble the process of “reading tea leaves,” or finding meaning in patterns that are in fact quite arbitrary or even random. ### 4a. Exploring Beta Values

Hidden within this forums_lda topic model object we created are per-topic-per-word probabilities, called β (“beta”). It is the probability of a term (word) belonging to a topic.

Let’s take a look at the 5 most likely terms assigned to each topic, i.e. those with the largest β values using the terms() function from the topicmodels package:

terms(forums_lda, 5)

##      Topic 1    Topic 2      Topic 3       Topic 4    Topic 5    Topic 6   
## [1,] "school"   "statistics" "test"        "grade"    "students" "students"
## [2,] "students" "em"         "students"    "data"     "task"     "level"   
## [3,] "middle"   "unit"       "standard"    "students" "data"     "dice"    
## [4,] "video"    "education"  "results"     "graphs"   "tasks"    "size"    
## [5,] "census"   "teaching"   "statistical" "plots"    "question" "sample"  
##      Topic 7      Topic 8       Topic 9     Topic 10     Topic 11    
## [1,] "students"   "statistics"  "questions" "assessment" "technology"
## [2,] "statistics" "teaching"    "kids"      "resource"   "students"  
## [3,] "feel"       "math"        "love"      "items"      "software"  
## [4,] "teach"      "teachers"    "gapminder" "locus"      "program"   
## [5,] "questions"  "mathematics" "sharing"   "students"   "excel"     
##      Topic 12   Topic 13     Topic 14    Topic 15    Topic 16     Topic 17    
## [1,] "students" "activity"   "question"  "students"  "fuel"       "students"  
## [2,] "project"  "students"   "data"      "questions" "cost"       "makes"     
## [3,] "real"     "agree"      "students"  "class"     "src"        "wondering" 
## [4,] "class"    "lesson"     "questions" "learn"     "codap"      "regression"
## [5,] "life"     "activities" "census"    "answer"    "pluginfile" "model"     
##      Topic 18   Topic 19     Topic 20
## [1,] "data"     "sample"     "stats" 
## [2,] "students" "difference" "ap"    
## [3,] "collect"  "samples"    "class" 
## [4,] "real"     "random"     "stat"  
## [5,] "sets"     "population" "school"

Even though we’ve somewhat arbitrarily selected the number of topics for our corpus, some these topics or themes are fairly intuitive to interpret. For example:

Topic 11 (technology, students, software, program, excel) seems to be about students use of technology including software programs like excel;
Topic 9 (questions, kids, love, gapminder, sharing) seems to be about the gapminder activity from the MOOC-Ed and kids enjoyment of it; and
Topic 18 (data, students, collect, real, sets) seems to be about student collection and use of real world data sets.

Not surprisingly, the tidytext package has a handy function conveniently name tidy() to convert our LDA model to a tidy data frame containing these beta values for each term:

tidy_lda <- tidy(forums_lda)
tidy_lda

## # A tibble: 273,320 × 3
##    topic term       beta
##    <int> <chr>     <dbl>
##  1     1 2015  5.87e- 60
##  2     2 2015  1.55e-  4
##  3     3 2015  1.61e-  4
##  4     4 2015  3.46e-  4
##  5     5 2015  8.10e- 76
##  6     6 2015  1.30e-108
##  7     7 2015  8.78e- 49
##  8     8 2015  1.63e- 38
##  9     9 2015  3.11e- 27
## 10    10 2015  1.10e-  3
## # … with 273,310 more rows

Obviously, it’s not very easy to interpret what the topics are about from a data frame like this so let’s borrow code again from Chapter 8.4.3 Interpreting the topic model in Text Mining with R to examine the top 5 terms for each topic and then look at this information visually:

top_terms <- tidy_lda %>%
  group_by(topic) %>%
  slice_max(beta, n = 5, with_ties = FALSE) %>%
  ungroup() %>%
  arrange(topic, -beta)
top_terms %>%
  mutate(term = reorder_within(term, beta, topic)) %>%
  group_by(topic, term) %>%    
  arrange(desc(beta)) %>%  
  ungroup() %>%
  ggplot(aes(beta, term, fill = as.factor(topic))) +
  geom_col(show.legend = FALSE) +
  scale_y_reordered() +
  labs(title = "Top 5 terms in each LDA topic",
       x = expression(beta), y = NULL) +
  facet_wrap(~ topic, ncol = 4, scales = "free")

4b. Exploring Gamma Values

Now that we have a sense of the most common words associated with each topic, let’s take a look at the topic prevalence in our MOOC-Ed discussion forum corpus, including the words that contribute to each topic we examined above.

Also, hidden within our forums_lda topic model object we created are per-document-per-topic probabilities, called γ (“gamma”). This provides the probabilities that each document is generated from each topic, that gamma matrix. We can combine our beta and gamma values to understand the topic prevalence in our corpus, and which words contribute to each topic.

To do this, we’re going to borrow some code from the Silge (2018) post, Training, evaluating, and interpreting topic models.

First, let’s create two tidy data frames for our beta and gamma values

td_beta <- tidy(forums_lda)
td_gamma <- tidy(forums_lda, matrix = "gamma")
td_beta

## # A tibble: 273,320 × 3
##    topic term       beta
##    <int> <chr>     <dbl>
##  1     1 2015  5.87e- 60
##  2     2 2015  1.55e-  4
##  3     3 2015  1.61e-  4
##  4     4 2015  3.46e-  4
##  5     5 2015  8.10e- 76
##  6     6 2015  1.30e-108
##  7     7 2015  8.78e- 49
##  8     8 2015  1.63e- 38
##  9     9 2015  3.11e- 27
## 10    10 2015  1.10e-  3
## # … with 273,310 more rows

td_gamma

## # A tibble: 115,220 × 3
##    document topic    gamma
##    <chr>    <int>    <dbl>
##  1 11295        1 0.00246 
##  2 12711        1 0.000389
##  3 12725        1 0.0291  
##  4 12733        1 0.141   
##  5 12743        1 0.00944 
##  6 12744        1 0.00485 
##  7 12756        1 0.0291  
##  8 12757        1 0.00360 
##  9 12775        1 0.00360 
## 10 12816        1 0.00360 
## # … with 115,210 more rows

Next, we’ll adopt Julia’s code wholesale to create a filtered data frame of our top_terms, join this to a new data frame for gamma-terms and create a nice clean table using the kabel() function knitr package:

top_terms <- td_beta %>%
  arrange(beta) %>%
  group_by(topic) %>%
  top_n(7, beta) %>%
  arrange(-beta) %>%
  select(topic, term) %>%
  summarise(terms = list(term)) %>%
  mutate(terms = map(terms, paste, collapse = ", ")) %>% 
  unnest()

## Warning: `cols` is now required when using unnest().
## Please use `cols = c(terms)`

gamma_terms <- td_gamma %>%
  group_by(topic) %>%
  summarise(gamma = mean(gamma)) %>%
  arrange(desc(gamma)) %>%
  left_join(top_terms, by = "topic") %>%
  mutate(topic = paste0("Topic ", topic),
         topic = reorder(topic, gamma))

gamma_terms %>%
  select(topic, gamma, terms) %>%
  kable(digits = 3, 
        col.names = c("Topic", "Expected topic proportion", "Top 7 terms"))

Topic	Expected topic proportion	Top 7 terms
Topic 7	0.083	students, statistics, feel, teach, questions, school, level
Topic 13	0.077	activity, students, agree, lesson, activities, ideas, resources
Topic 15	0.073	students, questions, class, learn, answer, struggle, reading
Topic 18	0.067	data, students, collect, real, sets, analyze, set
Topic 5	0.065	students, task, data, tasks, question, statistical, activity
Topic 8	0.064	statistics, teaching, math, teachers, mathematics, teach, science
Topic 9	0.052	questions, kids, love, gapminder, sharing, students, lot
Topic 6	0.050	students, level, dice, size, sample, trials, technology
Topic 11	0.050	technology, students, software, program, excel, core, time
Topic 14	0.047	question, data, students, questions, census, answer, survey
Topic 10	0.047	assessment, resource, items, locus, students, website, resources
Topic 12	0.047	students, project, real, class, life, projects, social
Topic 1	0.046	school, students, middle, video, census, age, watch
Topic 3	0.040	test, students, standard, results, statistical, deviation, understand
Topic 20	0.039	stats, ap, class, stat, school, students, math
Topic 4	0.038	grade, data, students, graphs, plots, 1, box
Topic 17	0.037	students, makes, wondering, regression, model, days, project
Topic 19	0.034	sample, difference, samples, random, population, sampling, minutes
Topic 2	0.030	statistics, em, unit, education, teaching, online, learning
Topic 16	0.013	fuel, cost, src, codap, pluginfile, vehicles, city

And let’s also compare this to the most prevalent topics and terms from our forums_stm model that we created using the plot() function:

plot(forums_stm, n = 7)

4c. Reading the Tea Leaves

Recognizing that topic modeling is best used as a “tool for reading” and provides only an incomplete answer to our overarching, “How do we quantify what a corpus is about?”, the results do suggest some potential topics that have emerges as well as some areas worth following up on.

Specifically, looking at some of the common clusters of words for the more prevalent topics suggest that some key topics or “latent themes” (renamed in bold) might include:

Teaching Statistics: Unsurprising, given the course title, the topics most prevalent in both the forums_stm and forums_lda models contains the terms “teach”, “students”, “statistics”. This could be an “overarching theme” but more likely may simply be just the residue of the course title though being sprinkled throughout the forums and deserves some follow up. Topics 8 from the LDA model may overlap with this topic as well.
Course Utility: The second most prevalent Topics (13 and 2) in the lda and stm models respectively, seem to potentially be about the usefulness of course “resources” like lessons, tools, videos, and activities. I’m wagering this might be a forum dedicated to course feedback. Topic 15 from the STM model also suggest this may be a broader theme.
Using Real-World Data: Topics 18 & 12 from the LDA model particularly intrigue me and I’m wagering this is pretty positive sentiment among participants about the value and benefit of having students collect and analyze real data sets (e.g. Census data in Topic 1) and work on projects relevant to their real life. Will definitely follow up on this one.
Technology Use: Several topics (6 & 11 from LDA and 8 & 19 from STM) appear to be about student use of technology and software like calculators and Excel for teaching statistics and using simulations. Topic 16 from LDA also suggest the use of the Common Online Data Analysis Platform (CODAP).
Student Struggle & Engagement: Topic 15 from LDA and Topic 16 from STM also intrigue me and appear to be two opposite sides of perhaps the same coin. The former includes “struggle” and “reading” which suggests perhaps a barrier to teaching statistics while Topic 16 contains top stems like “engage”, “activ”, and “think” and may suggest participants anticipate activities may engage students.

To serve as a check on my tea leaf reading, I’m going to follow Bail’s recommendation to examine some of these topics qualitatively. The stm package has another useful function though exceptionally fussy function called findThoughts which extracts passages from documents within the corpus associate with topics that you specify.

The first line of code may not be necessary for your independent analysis, but because the textProcessor() function removed several documents during processing, the findthoughts() function can’t properly index the processed docs. This line of code found on stackoverflow removes documents from original ts_forum_data source that were removed during processing so there is a one-to-one correspondence with forums_stm and so you can use the function to find posts associated with a given topic.

Let’s slightly reduce our original data set to match our STM model, pass both to the findThoughts() function, and set our arguments to return n =10 posts from topics = 2 (i.e. Topic 2) that have at least 50% or thresh = 0.5 as a minimum threshold for the estimated topic proportion.

ts_forum_data_reduced <-ts_forum_data$post_content[-temp$docs.removed]
findThoughts(forums_stm,
             texts = ts_forum_data_reduced,
             topics = 2, 
             n = 10,
             thresh = 0.5)

## 
##  Topic 2: 
##       I too have enjoyed playing around with CODAP. It's free (which is essential)  easy to use  and has good statistical capabilities. I look forward to trying it out in the classroom next semester.
##      Good to know-I began watching some of the video tutorials after seeing the Disney Pixar lesson and am excited to implement this tech tool in the classroom.
##      I will definitely use this site!  Such a plus that is free and available!
##      I couldn't agree more.  The resources provided to us through the MOOC have been great and I can't wait to use them in the future.
##      I plan to give the resources provided to the other teachers in our school.  I am hoping the STEM teacher will like the tinkerplots on our laptop lab and utilize the lesson plans provided.
##      I totally agree I can't wait to use it next semester with my AP class.
##      Mike-   Glad to hear that you like it! In the Unit 1 Dive Into Data you will find an activity where you have the opportunity to engage with the GapMinder World tool.   Once you check it out  we would love to hear how you would use it with students.
##      Hi Heidi  Here at The Friday Institute in the Place we have a free portal of materials made specifically for preservice teacher education!!  You should check out the "view.php?28 Preparing to Teach Mathematics with Technology Portal! There are units on statistics (using TinkerPlots and Fathom)  Algebra (using mostly GeoGebra)  and Geometry (using Sketchpad or Geogebra).  In a few months there will also be a whole unit of statistics activities using the free CODAP tool!   The tinkerplots.com and fathom.concord.org websites also have lots of premade activities!!  "
##      Wow!  I totally agree with you.  The Gapminder was great.  Also  I was inspired to complete this unit as well.  I did not know there was a TedTalk about this.  I will have to check it out.
##      Wow!  I totally agree with you.  The Gapminder was great.  Also  I was inspired to complete this unit as well.  I did not know there was a TedTalk about this.  I will have to check it out.

Duplicate posts aside, this Course Utility topic returns posts there were expected based on my interpretation of the key terms for Topic 2. It looks like I may have read those tea leaves correctly.

Now let’s take a look at Topic 16 that we thought might be related to student engagement:

findThoughts(forums_stm,
             texts = ts_forum_data_reduced,
             topics = 16, 
             n = 10,
             thresh = 0.5)

## 
##  Topic 16: 
##       This reminds of the actual Pepsi Challenge and the ramifications of bad statistical design.  The initial experiment had participants drink out of two glasses -- one labeled M and one labeled Q.  Pepsi was always in glass M and coke was always in glass Q.  The results of the experiment found that participants preferred Pepsi to Coke.  Coke countered back that all the study showed was that participants preferred the letter M to the letter Q!    Fun example about the perils of good bad experimental design.
##      I was thinking about the same thing  but not only that. They might also think about the factors that affect the perceive and actual preference such the Coke or Pepsi been flat.
##      I  too  found the Coke-Pepsi activity the most engaging  but would add time into class to have students choose which products we'll test and design the study to minimize bias. I often start my unit on sampling methods and design with my seniors by having them design both a survey and an experiment to choose our school's favorite potato chip. Each year we end up with completely different designs! To add even further to student-design with this activity  I'm interested in having them design and conduct the experiment well  then have them go back and design the experiment more poorly with given motivations. For example  if I work for Coke  how might I conduct the experiment so that we'll always prefer Coke?
##      I agree with this. I think this experience is a good way to get students involved in the Statistical process. One thing that I think would help make this task even more meaningful is if students became involved in pondering about how to answer questions like the Coke Pepsi one without introducing bias to their results. Thus instead of giving the students this pretty-well designed experiment  you can help them get to the point where they recognize why the blinding of study participants is important  etc. That is really my only critique of this task. I think it funnels the students into one way of approaching this statistical question. It is important that they understand why they are doing the things they are doing. One thing that may help would be to open it up to them to decide how best to figure out which soda is more preferred.
##      Great suggestions colleagues! I done the coke pepsi challenge in my class as well. This is a great activity to get students involved which is no small task for a night class of non-traditional working students. I chose this activity because it was very meaningful considering students living in the Southeast.
##      I agree with Shawn here  if there was time in the class it would be great to get ideas form the students of how to design the Coke-Pepsi experiment ... even if you do funnel them into a well-designed one at the end of the day to get the data in! I have done a similar exercise in class in the past with designing an experiment to test one student's claim that thhey can tell the difference between Coke and Pepsi.
##      Hi   I think the Pepsi vs. Coke Activity is good.  It is fun. The students will like it.
##      I agree Eric in that I like this activity the best. It gives guidance and helps students learn how to conduct an experiment. Plus  it gets the entire class involved. And  as you said  it can be expanded a little to hit all 4 phases of the statistical process  and it can be transferred to other activities. I think it has a lot of potential!
##      I could say that we all agree that we want our students to make informed decisions.  In my opinion  this is the reason why we liked the Coke vs Pepsi task.  It helps us in engaging our students and think outside the box about different factors that might affect their results.  I notice that there are disagreements on letting students do these task or ask them to do their own.  I think this task should serve as a guide for students to engage in designing their own experiment and compare their decisions to their classmates decisions and discuss how they would do their classmates experiments different and the factors that might affect the results.
##      I agree this would be a fun activity that students would embrace.  < 1.6 For possible modifications  one could have students identify their favorite prior to running the experiment and then compare class results before and after tasting.¬†< 1.6 If before and after results are tied to each student  the class could also analyze which cola tended to have more tasters whose pre-taste opinion matched their taste-test results. Lots of opportunities to pose additional open-ended questions to the students. "

It looks like my tea reading was a partially correct for Topic 16, though the results seem to be about a specific “Pepsi challenge” activity to conduct with students.

Finally, let’s look at posts from Topic 3 which we though might be an overarching theme about teaching statistics:

ts_forum_data_reduced <-ts_forum_data$post_content[-temp$docs.removed]
findThoughts(forums_stm,
             texts = ts_forum_data_reduced,
             topics = 3, 
             n = 10,
             thresh = 0.5)

## 
##  Topic 3: 
##       I agree that with you. It wasn't the content so much but how to make it relevant for the students to see how statistics are in real life.  The students don't want to learn if it doesn't apply to them and these ideas helped me to see some different ways to incorporate statistics into my classroom.
##      If we think of mathematics as a set of procedures we learn and practice  then I suppose statistics is very different than mathematics.  If we think of mathematics as a mindset of problem solving  creativity  reasoning  and sense making  then there is certainly much of teaching and learning statistics that corresponds nicely with the teaching and learning of mathematics.
##      I have really learned a lot in this course. I have found that a lot of the times I have been teaching and referring to statistics without even knowing! I am a lot more confident in teaching statistics now that I have gone through this course. Although it will be hard to apply some of the things I have learned since I am teaching kindergarten I am excited to try to apply these topics to my grade level when my students are ready for it! I found all the readings in this course super interesting and although some were lengthy I really learned a lot from them!
##      Motivating the students to be interested to learn the material is a big push for me this year.  As others stated  we as teachers have to be motivated in order for our students to be motivated.  I spend a lot of time trying to make my lessons fun and interactive because I know if the students are engaged they will take away more.  Some topics are easier than others.  Just like it's easier to incorporate real world situations with some topics and not others.  I have discovered that once I got the mindset of making them interesting more students was becoming successful on the lesson.
##      Thinking about the differences between math  statistics  and probability has given me a clearer picture in my mind as to why I have not been a lover of probability or statistics the way I have been a lover of mathematics. Recognizing the differences helps understanding and will also make me a much better teacher (should I need to teach statistics or probability in the future).
##      I agree. Connecting the problem to real world will motivate students to get more involved with the math. In return  this will enrich the learning that is taking place inside the classroom.
##      I now understand why mathematics and statistics are different.  In statistics context is important but in math it gets in the way.  Although there is a lot of mathematics involved in statistics it involves a lot of contextual evidence  too.
##      I agree with you. I often feel like my students when learning something that does not interest me finding relevance to content makes me want to learn and know more however if its something I am not interested in or too hard of a concept to grasp I loose interest in it fast. So when teaching we have to think that our students need things they can relate to to make since of it.
##      The difference in probability and statistics is huge. I didn't even really know the difference until this course  but that makes absolute sense.
##      I don't know that it is fair to say that they don't want to think and are just looking for formulas. I think in the past this has been their experience though and so they want to fall back on what they know. Hopefully MOOCs like this as well as the common core curriculum should be driving the thinking end of things a little more than ever before. I know I have already started to see a difference in the thinking skills of my students if not their math skills and this is a exciting for my future as a math educator. The experts in the video talk about how collecting and analyzing real world data is engaging for kids because it isn't a worksheet that is disconnected from their lives. I think this engagement necessarily forces them to think which is way more fun than being a formula robot. Anecdotally  my brother woke me up this week with an interesting geometry problem that made me think about circles in a new way and it was really exciting. Granted  as a math teacher I'm probably to the point where I can be excited about thinking about weird problems but I know that balancing my budget is significantly less exciting that solving weird problems that make me think.  Lon story short- I think if we are engaging students with interesting problems they will be practicing thinking skills  if we drive class discussions with metacognition they will be practicing thinking skills  if we have students pose their own problems that are relevant to their lives they will be practicing thinking skills. If we can make a nation of critical thinkers and problem solvers  there should be no limit to what we can accomplish.   "

Looking at just the 10 posts returned, perhaps a better name for this topic would be Course Reflections on Teaching Statistics.

Unit Takeaway

In addition to some useful R packages and functions for the actual process of topic modeling, hopefully there are two main lessons I’m hoping you take away from this walkthrough:

Topic modeling requires a lot of decisions. Beyond deciding on a value for K, there are a number of key decisions that you have to make that can dramatically affect your results. For example, to stem or not to stem? What qualifies as a document? What flavor or topic modeling is best suited to your data and research questions? How many iterations to run?
Topic modeling is as much art as (data) science. As Bail (2018) noted, the term “topic” is somewhat ambitious, and topic models do not produce highly nuanced classification of texts. Once you’ve fit your model, interpreting your model requires some mental gymnastics and ideally some knowledge of the context from which the data came to help with interpretation of your topics. Moreover, the quantitative approaches for making the decisions highlighted above are imperfect and a good deal of human judgment required.

✅ Comprehension Check

Using the STM model you fit from the Section 3 \[Comprehension Check\] with a different value for K, use the approaches demonstrated in Section 4 to explore and interpret your topics and terms and revisit the following question:

Now that you have a little more context, how might you revise your initial interpretation of some of the latent topics or latent themes from your model? Using a K of 14, based on the Bail STM selection model, the topics appear much more cogent than when I used 10 or 20. As an example, the K=10 run showed a topic focused on video software or capture technology. When using the K=14 data, this topic was clearly more about sharing data resources which may include video files.

forums_stm_14 <- stm(documents=docs, 
                     data=meta,
                     vocab=vocab, 
                     prevalence =~ course_id + forum_id,
                     K=14,
                     max.em.its=25,
                     verbose = FALSE)

findThoughts(forums_stm_14,
             texts = ts_forum_data_reduced,
             topics = 1, 
             n = 10,
             thresh = 0.5)

## 
##  Topic 1: 
##       Are there any differences between the top speeds of older  newer or more recent roller coasters? Explain.      Yes there is differences  the newest coaster tends to be faster than the recent coaster and the recent tends to be faster than the oldest (by substitution the newest tends to be faster than the oldest) back in the populations. I used this rule because the median for one of the samples lies outside the box for the other samples. E.g.  more than half of the newest coasters are above three-quarters of the recent coasters.              Does the type of material make a difference in speed?      The speed of steel coaster tends to be faster than that of a wooden coaster. The distance between medians of MAD<sub>steel< sub>60 minus MAD<sub>wooden< sub>55 = 5mph is greater than about 1 5 of overall visible spread (70-50=20) equals four or .2(20)=4mph.              If you were going to ride a coaster that was built before 1980  what would you expect for the top speed? what about if you were riding a newer roller coaster?      I expect the top speed of a coaster that was built before 1980 equals 65.5mph and for the newer coaster equals 56mph built between 1980 and 1999 and if newest built in 2000 or later equals 50mph.   "
##      Ana  after reading your post  as well as the first post by Pat Engle  it got me thinking about your investigation.  You concluded that ultimately  steel coasters have a higher max height than wood tracks.  An unsurprising result is that steel coasters also have a longer track length than wood tracks.  Pat  however  was interested in whether or not the duration of these rides were different.  In spite of the higher max heights and longer tracks  he was unable to conclude that one type of track had a longer duration.  I wonder how much of this result is confounded by the notion that longer tracks with higher max heights also require a longer climb to get up to the peak.  In my roller coaster riding experience  this seems to be the longest portion of the entire ride.
##      After completing the 8 question assessment  I didn't feel like any of the answer choices in question 2 were an acceptable way to test these shoes. I get why choice B is the best option of these 4 since it randomly assigns the treatments  but it didn't do anything to control other variables. I actually thought combining A and B would be the best way. Input?   <   An advertisement makes the claim  ‚ÄúLighter shoes make you run faster.‚Äù Of the following  which is the best way to investigate this claim?<   Select one <   A. Choose the records of the top twenty runners who are wearing the lighter shoes and compare their times to run 400 meters before and after they began wearing the shoes.<   B. Choose twenty runners and select ten at random to wear lighter shoes and have the other ten wear heavier shoes to run 400 meters and compare their times.  src=place.fi.ncsu.edu theme image.php?theme=fridayinstitute  component=core  rev=1485978893  image=i%2Fgrade_correct Correct<   C. Choose twenty runners at random and have the women wear the lighter shoes and the men wear the heavier shoes to run 400 meters and compare their times.<   D. Choose to observe the results of 400-meter races for the next year and see how many winners are wearing the lighter shoes.  "
##      Do wooden coasters tend to have the same maximum height as coasters made from steel? Does anything surprise you?      The height of steel coaster tends to be higher than the height of wooden coaster back in the populations.      What surprising me       Although the box-plot of height of both types of coasters have outliers  the mean and median in both are almost same. In addition  sample observations looks like follow approximately normal distribution although outliers are exist.      In terms of spread  the observations of wooden type  which is the old version  is more homogeneous than the steel type which is the newest version. In addition  the standard deviation of steel group is almost double compared to the wooden group              Do steel roller coasters tend to have longer drops than wooden roller coasters?      The drop of steel coaster tends to be longer than the drop of wooden coaster back in the populations because the median for the drops of steel coaster lies outside the box of wooden coaster (more than half of the steel coaster are above than three quarters of the wooden group).      I confused about using Age-14 or Age-15 guidelines in the guidelines for analysis file. The sample sizes of 54 and 100 are outside the range 20 and 40 as found in Age-14. So which one we have to use here is it Age-14 or Age-15 guidelines?              Based on what you found  predict what might be reasonable to expect for the height of the new wooden or steel roller coasters opening soon.      I expect the height would be ft100 and ft150 for the wooden and steel coaster respectively   "
##      Yeah I think that something might be up with that data set. I did the same box plot in Codap  and found that the median top speed for steel coasters was 60 mph. Since there are a lot of steel coasters clustered around 60 in your box plot  it seems like some of the steel coaster data did not get converted from mph to kph.
##      I was surprised to see that there was much more spread in speeds of steel coasters. My first guess was that the slower steel coasters were mainly older coasters  but highlighting the older (pre-1980) coasters showed that that wasn't really the case (they are highlighted in blue). "
##      I agree that I didn't think answer B is a very good answer  although it's the best one listed.  It doesn't do enough to address the possibly large differences in runners' ability levels.  If the group of 10 with heavier shoes includes more of the runners who are slower to begin with  then of course they'll be slower  conversely  if that group happens to include the faster runners then it may appear that the shoes make no difference when compared to a group of 10 slower runners in lighter shoes.  I was looking for an answer that included a better control group.  Like maybe  Choose twenty runners and record their individual times running 400 meters.  After a sufficient rest period  randomly assign ten of the runners to wear heavier shoes and the other ten to wear lighter shoes and record their times again.  "
##      This is interesting.  I used the Tuva Labs data and I am not sure I had similar results.  In fact  I was surprised at what I saw.     src=@@PLUGINFILE@@ Screen%20Shot%202017-10-13%20at%207.59.03%20PM.png comparing speeds of different materials 1086 936     See what I mean?  While there certainly faster steel roller coasters (see the outliers and long upper whisker)  75% of wooden roller coasters are faster than Q3 of the steel roller coasters.  I was surprised to see this - I assumed steel coasters would be faster.  I think this would be very appealing to students because of the unexpected results.  There were other unexpected comparisons as well  like age of the coasters.  I think I might give students this tool and ask them to play around and talk about any unexpected results.  "
##      Since I'm more interested in the technology and the metalanguage in English to discuss statistics  I was thinking to do Option C either 2 or 3 suggestion.  If Option C2  then I would make something around these topics    ÎßëÏùÄ Í≥†Îîï10.0pt lang-US<<     1.6 Youth  unemployment in different countries < <<     1.6 Birthrate  declines < <<     1.6 Retirement  ages < <<     1.6 Marriage  ages < <<     1.6 Single  family income vs institutional financial support to orphanages<   ÎßëÏùÄ Í≥†Îîï10.0pt lang-USOr Option C3  would be good too. I could u<     1.6 se  previous student work about birth rate or youth unemployment and animate like one of the previous videos I've watched modeling various student responses.  ÎßëÏùÄ Í≥†Îîï10.0pt lang-US   ÎßëÏùÄ Í≥†Îîï10.0pt lang-US   ÎßëÏùÄ Í≥†Îîï10.0pt lang-US    "
##      Is there a difference in track length between roller coasters that are inverted or not?      The track length of inverted coaster tends to be shorter than that of without inversion. The distance between medians of MAD<sub>steel< sub>2759.5 minus MAD<sub>wooden< sub>3180 = - 420.5ft is greater than about 1 5 of overall visible spread (1659.25 - 4591.5= -2932.25) which equals -586.45 or .2(-2932.25)= -586.45ft. The minus sign here can be translated to the word of shorter.              Has track length of coasters changed much over the years?      Based on the least squares line the track length of coasters was not change much over the years (regression parameter = 13.1ft)  where this change was positive over the years and the coefficient of determination was found very small with 0.03.   "

findThoughts(forums_stm_14,
             texts = ts_forum_data_reduced,
             topics = 9, 
             n = 10,
             thresh = 0.5)

## 
##  Topic 9: 
##       Mario   I would post this in the project forum place.fi.ncsu.edu mod forum view.php?858place.fi.ncsu.edu mod forum view.php?858. This is where people who are interested in a project will be looking for partners  discussing their projects  and providing feedback on them.  "
##      The Statkey home website has a ton of csv files and other formats too.  I haven't tried it  but I bet at least one of the file types would work.   420 CSVExcelASCII  R  MinitabFathom  JMP   www.lock5stat.com datapage.html        "
##      Hi Elaina  This is my second MOOC from the Friday Institute.  I don't believe you will have perpetual access to this course.  I can no longer access the one I took last spring.  Before that class closed   I downloaded all pdfs and created a page in google docs to store links to the featured web resources.
##      Great advice Lee Ann! Thanks for sharing from an experienced teacher's perspective that can help inspire and encourage our newest colleagues (preservice teachers)!
##      Great idea!  In fact I have the entire set for you. Look at the bottom of the list of csv files in drive.google.com drive folders 0B5ZjVvC03tInVi1iTGhQOTAzVlE Google drive folder!  Have fun taking that peek! "
##      Thanks for the link. I have never used that Site before and the nurse Geen example may stimulate some good discussions.
##      Thank you. I will try the CSV files in the Google drive folder.
##      Thanks for the information.  I hadn't run across the ASA link so I'm very glad that you pointed it out.  I will post to the project link as you suggested.
##      When I returned from the preceding link  I accidentally rated this resource.    I would give it zero stars if I could  or one star in a do over  because it takes me to a link to purchase something.
##      Ronald-  We are glad that you have identied resources that you want to return to. If you do bookmark the urls they are available on the web after the course is closed.

LS0tCnRpdGxlOiAnVW5pdCAzIFdhbGt0aHJvdWdoOiBUb3BpYyBNb2RlbGluZyBpbiBNT09DLUVkcycKYXV0aG9yOiAiSmFtZXMgSGFyZGF3YXkiCmRhdGU6ICIyLzE3LzIwMjIiCm91dHB1dDogCiAgaHRtbF9kb2N1bWVudDoKICAgIHRvYzogdHJ1ZQogICAgdG9jX2RlcHRoOiAzCiAgICB0b2NfZmxvYXQ6IHllcwogICAgY29kZV9mb2xkaW5nOiBoaWRlCiAgICBjb2RlX2Rvd25sb2FkOiBUUlVFCmVkaXRvcl9vcHRpb25zOiAKICBtYXJrZG93bjogCiAgICB3cmFwOiA3MgotLS0KCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUpCmBgYAoKIyMgMC4gSU5UUk9EVUNUSU9OCgpUaGUgVW5pdCAzIHdhbGt0aHJvdWdoIGV4dGVuZHMgcHJldmlvdXMgcmVzZWFyY2ggYW5kIGV2YWx1YXRpb24gd29yayBhdAp0aGUgRnJpZGF5IEluc3RpdHV0ZSBmb3IgRWR1Y2F0aW9uYWwgSW5ub3ZhdGlvbiBhdCBOb3J0aCBDYXJvbGluYSBTdGF0ZQpVbml2ZXJzaXR5LiBJbiBhZGRpdGlvbiB0byBtYW55IG90aGVyIGFyZWFzIG9mIGlucXVpcnksIHRoaXMgd29yayB3YXMKYWltZWQgYXQgdW5kZXJzdGFuZGluZyBhbmQgaW1wcm92aW5nIHBlZXIgaW50ZXJhY3Rpb24gYW5kIGRpc2N1c3Npb24gaW4KdGhlIEZyaWRheSBJbnN0aXR1dGUncyBNYXNzaXZlbHkgT3BlbiBPbmxpbmUgQ291cnNlcyBmb3IgRWR1Y2F0b3JzCihNT09DLUVkKSBhbmQgT25saW5lIFByb2Zlc3Npb25hbCBMZWFybmluZyBwcm9ncmFtcy4gVG8gbGVhcm4gbW9yZSBhYm91dAp0aGVzZSBjb3Vyc2VzIGFuZCBwcm9ncmFtcywgdmlzaXQ6IDxodHRwczovL3BsYWNlLmZpLm5jc3UuZWR1PgoKIyMjIFdhbGt0aHJvdWdoIEZvY3VzCgpPdXIgZm9jdXMgdGhpcyB3ZWVrIHdpbGwgYmUgb24gaWRlbnRpZnlpbmcgInRvcGljcyIgYnkgZXhhbWluaW5nIGhvdwp3b3JkcyBjb2hlcmUgaW50byBkaWZmZXJlbnQgbGF0ZW50LCBvciBoaWRkZW4sIHRoZW1lcyBiYXNlZCBvbiBwYXR0ZXJucwpvZiBjby1vY2N1cnJlbmNlIG9mIHdvcmRzIHdpdGhpbiBkb2N1bWVudHMuIFdpdGggYSBiaXQgb2YKdG9uZ3VlLWluLWNoZWVrLCBbTWVla3MgYW5kIFdlaW5nYXJ0CigyMDEyKV0oaHR0cDovL2pvdXJuYWxvZmRpZ2l0YWxodW1hbml0aWVzLm9yZy8yLTEvZGgtY29udHJpYnV0aW9uLXRvLXRvcGljLW1vZGVsaW5nLykKZGVzY3JpYmUgdG9waWMgbW9kZWxpbmcgYXM6Cgo+ICouLi5mb2N1c2VkIG9uIGNvcnBvcmEgYW5kIG5vdCBpbmRpdmlkdWFsIHRleHRzLCB0cmVhdGluZyB0aGUgd29ya3MKPiB0aGVtc2VsdmVzIGFzIHVuY2VyZW1vbmlvdXMgJ2J1Y2tldHMgb2Ygd29yZHMsJyBhbmQgcHJvdmlkaW5nCj4gc2VkdWN0aXZlIGJ1dCBvYnNjdXJlIHJlc3VsdHMgaW4gdGhlIGZvcm1zIG9mIGVhc2lseSBpbnRlcnByZXRlZCAoYW5kCj4gbWFuaXB1bGF0ZWQpICd0b3BpY3MnLi4uLiBUbyBhY2hpZXZlIGl0cyByZXN1bHRzLCBpdCBsZXZlcmFnZXMgb2NjdWx0Cj4gc3RhdGlzdGljYWwgbWV0aG9kcyBsaWtlICdkaXJpY2hsZXQgcHJpb3JzJyBhbmQgJ2JheWVzaWFuIG1vZGVscy4nKgo+IFRoYXQgYmVpbmcgc2FpZCwKPiBbV2VpbmdhcnRdKGh0dHA6Ly9qb3VybmFsb2ZkaWdpdGFsaHVtYW5pdGllcy5vcmcvMi0xL2RoLWNvbnRyaWJ1dGlvbi10by10b3BpYy1tb2RlbGluZy8pCj4gYWxzbyBub3RlZCB0aGF0ICJhIHRvcGljIG1vZGVsIGlzIGEgImNsZXZlciBhbmQgZXhjZXB0aW9uYWxseQo+IHZlcnNhdGlsZSBsaXR0bGUgYWxnb3JpdGhtIHRoYXQgY2FuIGJlIGN1c3RvbWl6ZWQgdG8gYWxsIHNvcnRzIG9mCj4gYXBwbGljYXRpb25zIiBhbmQgW0JhaWwKPiAoMjAyMCldKGh0dHBzOi8vc2ljc3MuaW8vMjAyMC9tYXRlcmlhbHMvZGF5My10ZXh0LWFuYWx5c2lzL3RvcGljLW1vZGVsaW5nL3JtYXJrZG93bi9Ub3BpY19Nb2RlbGluZy5odG1sI3J1bm5pbmcteW91ci1maXJzdC10b3BpYy1tb2RlbCkKPiBhZGQgdGhhdCB0b3BpYyBtb2RlbGluZyBjYW4gYmUgImEgcG93ZXJmdWwgdG9vbCBmb3IgaWRlbnRpZnlpbmcKPiBnZW5lcmFsIHRyZW5kcyBpbiBhIGNvcnB1cyB0aGF0IGNhbiB0aGVuIGJlIGFuYWx5emVkIGluIGEgbW9yZQo+IGdyYW51bGFyIG1hbm5lciB1c2luZyBvdGhlciB0ZWNobmlxdWVzLiIKCldpdGggcmVzcGVjdCB0byB0aGUgYWN0dWFsIFIgd29ya2Zsb3cgb2YgYXBwbHlpbmcgdG9waWMgbW9kZWxzIHRvCmRvY3VtZW50cyBhbmQgdGV4dCBvZiBpbnRlcmVzdHMsIFNpbGdlICYgUm9iaW5zb24gYW5kIGEgbmV3IGJvdHRvbSByb3cKdGhlaXIgZmxvd2NoYXJ0IGNvbnNpc3RpbmcgbmV3IGRhdGEgc3RydWN0dXJlcyAoaS5lLiwgYSBjb3JwdXMgb2JqZWN0CmFuZCBkb2N1bWVudC10ZXJtIG1hdHJpeCkgYW5kIGFuZCB0aGUgTERBIG1vZGVsOgoKWyFbRmlndXJlIHNvdXJjZTogU2lsZ2UsIEouLCAmIFJvYmluc29uLCBELiAoMjAxNykuIFRleHQgbWluaW5nIHdpdGggUjoKQSB0aWR5IGFwcHJvYWNoLiBPJ1JlaWxseSBNZWRpYSwgSW5jLiBSZXRyaWV2ZWQgZnJvbToKaHR0cHM6Ly93d3cudGlkeXRleHRtaW5pbmcuY29tL3RvcGljbW9kZWxpbmcuaHRtbF0oaW1nL3RtX2Zsb3cucG5nICJBIGZsb3djaGFydCBvZiBhIHRleHQgYW5hbHlzaXMgdGhhdCBpbmNvcnBvcmF0ZXMgdG9waWMgbW9kZWxpbmcuIFRoZSB0b3BpY21vZGVscyBwYWNrYWdlIHRha2VzIGEgRG9jdW1lbnQtVGVybSBNYXRyaXggYXMgaW5wdXQgYW5kIHByb2R1Y2VzIGEgbW9kZWwgdGhhdCBjYW4gYmUgdGlkZWQgYnkgdGlkeXRleHQsIHN1Y2ggdGhhdCBpdCBjYW4gYmUgbWFuaXB1bGF0ZWQgYW5kIHZpc3VhbGl6ZWQgd2l0aCBkcGx5ciBhbmQgZ2dwbG90Mi4iKXt3aWR0aD0iOTAlIn1dKGh0dHBzOi8vd3d3LnRpZHl0ZXh0bWluaW5nLmNvbS90b3BpY21vZGVsaW5nLmh0bWwpCgpUaGlzIHdlZWsgd2lsbCBiZSBhbHNvIGJlIG91ciBmaXJzdCBpbnRyb2R1Y3Rpb24gdG8gdGhlICJNb2RlbCIgcHJvY2VzcwpvZiB0aGUgZGF0YS1pbnRlbnNpdmUgd29ya2Zsb3cgZGVzY3JpYmVkIGluIG91ciBjb3Vyc2UgdGV4dCwgWypMZWFybmluZwpBbmFseXRpY3MgR29lcyB0bwpTY2hvb2wqXShodHRwczovL2NhdGFsb2cubGliLm5jc3UuZWR1L2NhdGFsb2cvTkNTVTQ4NjIxMzQpKi4qIEFzIG5vdGVkCmJ5IEtydW1tIGFuZCBNZWFucyAoMjAxOCksIHRoaXMgd29ya2Zsb3cgaXMgbm90IGFsd2F5cyBhIGxpbmVhciBwcm9jZXNzCmFuZCB0aGVyZSBpcyBvZnRlbiBhIGdyZWF0IGRlYWwgb2YgaXRlcmF0aW9uIHRoYXQgb2NjdXJzIHdpdGhpbiBhbmQKYmV0d2VlbiB3cmFuZ2xpbmcsIGV4cGxvcmluZywgbW9kZWxpbmcuIEFzIGlsbHVzdHJhdGVkIGJ5IG91ciB3b3JrZmxvdwpiZWxvdywgdGhpcyB3ZWVrIHdlIHdpbGwgcHJpbWFyaWx5IGV4cGxvcmUgb3VyIGRhdGEgYWZ0ZXIgdGhlIG1vZGVsaW5nCnByb2Nlc3MgaW4gb3JkZXIgdG8gZ2FpbiBzb21lIGFkZGl0aW9uYWwgaW5zaWdodCBpbnRvIHRoZSB0b3BpY3MKZ2VuZXJhdGVkIGJ5IG91ciBtb2RlbC4gU3BlY2lmaWNhbGx5LCB0aGlzIHdlZWsgY292ZXJzIHRoZSBmb2xsb3dpbmcKY29uY2VwdHMgYW5kIHNraWxsczoKCjEuICAqKlByZXBhcmUqKjogUHJpb3IgdG8gYW5hbHlzaXMsIHdlJ2xsIHRha2UgYSBxdWljayBsb29rIGF0IHNvbWUgb2YKICAgIHRoZSByZWxhdGVkIE1PT0MtRWQgcmVzZWFyY2ggYW5kIGV2YWx1YXRpb24gd29yayB0byBnYWluIHNvbWUKICAgIGNvbnRleHQgZm9yIG91ciBhbmFseXNpcy4gVGhpcyBzaG91bGQgYWlkIGluIHRoZSBpbnRlcnByZXRhdGlvbiBvZgogICAgb3VyIHJlc3VsdHMgYW5kIGhlbHAgZ3VpZGUgc29tZSBkZWNpc2lvbnMgYXMgd2UgdGlkeSwgbW9kZWwsIGFuZAogICAgdmlzdWFsaXplIG91ciBkYXRhLgoyLiAgKipXcmFuZ2xlKio6IEluIHNlY3Rpb24gMiB3ZSBhZ2FpbiByZXZpc2l0IHRpZHlpbmcgYW5kIHRva2VuaXppbmcKICAgIHRleHQgdXNpbmcgdGhlIGB0aWR5dGV4dGAgcGFja2FnZSBidXQgYXJlIGFsc28gaW50cm9kdWNlZCB0byB0aGUgdGhlCiAgICBgc3RtYCBwYWNrYWdlLiBUaGlzIHBhY2thZ2UgbWFrZXMgdXNlIG9mIGB0bWAgdGV4dCBtaW5pbmcgcGFja2FnZSB0bwogICAgcHJlcHJvY2VzcyB0ZXh0IGFuZCB3aWxsIGFsc28gYmUgb3VyIGZpcnN0IGludHJvZHVjdGlvbiB0byB3b3JkCiAgICBzdGVtbWluZy4KMy4gICoqTW9kZWwqKjogV2UgdGFrZSBhIGxvb2sgYXQgdHdvIGRpZmZlcmVudCBhcHByb2FjaGVzIHRvIHRvcGljCiAgICBtb2RlbGluZzogTGF0ZW50IERpcmljaGxldCBBbGxvY2F0aW9uIChMREEpIGFuZCBTdHJ1Y3R1cmFsIFRvcGljCiAgICBNb2RlbGluZyAoU1RNKSwgd2hpY2ggaXMgdmVyeSBzaW1pbGFyIHRvIExEQSBidXQgY2FuIHVzZSBtZXRhZGF0YQogICAgYWJvdXQgZG9jdW1lbnRzIHRvIGltcHJvdmUgdGhlIGFzc2lnbm1lbnQgb2Ygd29yZHMgdG8gInRvcGljcyIgaW4gYQogICAgY29ycHVzIGFuZCBleGFtaW5lIHJlbGF0aW9uc2hpcHMgYmV0d2VlbiB0b3BpY3MgYW5kIGNvdmFyaWF0ZXMuCjQuICAqKkV4cGxvcmUqKjogVG8gZnVydGhlciBleHBsb3JlIHRoZSByZXN1bHRzIG9mIG91ciB0b3BpYyBtb2RlbCwgd2UKICAgIHVzZSBzZXZlcmFsIGhhbmR5IGZ1bmN0aW9ucyBmcm9tIHRoZSBgdG9waWNtb2RlbHNgIGFuZCBgc3RtYAogICAgcGFja2FnZXMsIGluY2x1ZGluZyB0aGUgYGZpbmRUaG91Z2h0c2AgZnVuY3Rpb24gZm9yIHZpZXdpbmcKICAgIGRvY3VtZW50cyBhc3NpZ25lZCB0byBhIGdpdmVuIHRvcGljIGFuZCB0aGUgYHRvTERBdmlzYCBmdW5jdGlvbiBmb3IKICAgIGV4cGxvcmluZyB0b3BpYyBhbmQgd29yZCBkaXN0cmlidXRpb25zLgoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgojIyAxLiBQUkVQQVJFCgpUbyBoZWxwIHVzIGJldHRlciB1bmRlcnN0YW5kIHRoZSBjb250ZXh0LCBxdWVzdGlvbnMsIGFuZCBkYXRhIHNvdXJjZXMKd2UnbGwgYmUgdXNpbmcgaW4gVW5pdCAzLCB0aGlzIHNlY3Rpb24gd2lsbCBmb2N1cyBvbiB0aGUgZm9sbG93aW5nCnRvcGljczoKCmEuICAqKkNvbnRleHQqKi4gQXMgY29udGV4dCBmb3Igb3VyIGFuYWx5c2lzIHRoaXMgd2Vlaywgd2UnbGwgcmV2aWV3CiAgICBzZXZlcmFsIHJlbGF0ZWQgcGFwZXJzIGJ5IG15IGNvbGxlYWd1ZXMgcmVsZXZhbnQgdG8gb3VyIGFuYWx5c2lzIG9mCiAgICBNT09DLUVkIGRpc2N1c3Npb24gZm9ydW1zLgpiLiAgKipRdWVzdGlvbnMuKiogV2UnbGwgYWxzbyBleGFtaW5lIHdoYXQgaW5zaWdodCB0b3BpYyBtb2RlbGluZyBjYW4KICAgIHByb3ZpZGUgdG8gYSBxdWVzdGlvbiB0aGF0IHdlIGFza2VkIHBhcnRpY2lwYW50cyBhbnN3ZXIgaW4gdGhlaXIKICAgIHByb2Zlc3Npb25hbCBsZWFybmluZyB0ZWFtcyAoUExUcykuCmMuICAqKlByb2plY3QgU2V0dXAuKiogVGhpcyBzaG91bGQgYmUgdmVyeSBmYW1pbGlhciBieSBub3csIGJ1dCB3ZSdsbAogICAgc2V0IHVwIGEgbmV3IFIgcHJvamVjdCBhbmQgaW5zdGFsbCBhbmQgbG9hZCB0aGUgcmVxdWlyZWQgcGFja2FnZXMKICAgIGZvciB0aGUgdG9waWMgbW9kZWxpbmcgd2Fsa3Rocm91Z2guCgojIyMgMWEuIENvbnRleHQKCiMjIyMgUGFydGljaXBhdGluZyBpbiBhIE1PT0MgYW5kIFByb2Zlc3Npb25hbCBMZWFybmluZyBUZWFtOiBIb3cgYSBCbGVuZGVkIEFwcHJvYWNoIHRvIFByb2Zlc3Npb25hbCBEZXZlbG9wbWVudCBNYWtlcyBhIERpZmZlcmVuY2UKClshW1RlYWNoaW5nIFN0YXRpc3RpY3MgVGhyb3VnaCBEYXRhIEludmVzdGlnYXRpb25zCk1PT0MtRWRdKGltZy90c2RpLnBuZyAiT3VyIHdvcmxkIGlzIHJpY2ggd2l0aCBkYXRhIHNvdXJjZXMsIGFuZCB0ZWNobm9sb2d5IG1ha2VzIGRhdGEgbW9yZSBhY2Nlc3NpYmxlIHRoYW4gZXZlciBiZWZvcmUhIFRvIGhlbHAgZW5zdXJlIHN0dWRlbnRzIGFyZSBmdXR1cmUgcmVhZHkgdG8gdXNlIGRhdGEgZm9yIG1ha2luZyBpbmZvcm1lZCBkZWNpc2lvbnMsIG1hbnkgY291bnRyaWVzIGFyb3VuZCB0aGUgd29ybGQgaGF2ZSBpbmNyZWFzZWQgdGhlIGVtcGhhc2lzIG9uIHN0YXRpc3RpY3MgYW5kIGRhdGEgYW5hbHlzaXMgaW4gc2Nob29sIGN1cnJpY3VsdW3igJNmcm9tIGVsZW1lbnRhcnkvcHJpbWFyeSBncmFkZXMgdGhyb3VnaCBjb2xsZWdlLiBUaGlzIGNvdXJzZSBhbGxvd3MgeW91IHRvIGxlYXJuLCBhbG9uZyB3aXRoIGNvbGxlYWd1ZXMgZnJvbSBvdGhlciBzY2hvb2xzLCBhbiBpbnZlc3RpZ2F0aW9uIGN5Y2xlIHRvIHRlYWNoIHN0YXRpc3RpY3MgYW5kIHRvIGhlbHAgc3R1ZGVudHMgZXhwbG9yZSBkYXRhIHRvIG1ha2UgZXZpZGVuY2UtYmFzZWQgY2xhaW1zLiBUbyBsZWFybiBtb3JlIGFib3V0IGVuZ2FnaW5nIGxlYXJuZXJzIGluIG1ha2luZyBpbmZlcmVuY2VzIGFuZCBjbGFpbXMgc3VwcG9ydGVkIGJ5IGRhdGEgYW5kIGhvdyB0byBlbXBoYXNpemUgaW5mZXJlbnRpYWwgcmVhc29uaW5nIGluIHRlYWNoaW5nIHN0YXRpc3RpY3MgdGhyb3VnaCBwb3NpbmcgZGlmZmVyZW50IHR5cGVzIG9mIGludmVzdGlnYXRpdmUgcXVlc3Rpb25zLCBlbnJvbGwgaW4gb3VyIFRlYWNoaW5nIFN0YXRpc3RpY3MgdGhyb3VnaCBJbmZlcmVudGlhbCBSZWFzb25pbmcgTU9PQy1FZC4iKXt3aWR0aD0iNTAlIn1dKGh0dHBzOi8vcGxhY2UuZmkubmNzdS5lZHUvbG9jYWwvY2F0YWxvZy9jb3Vyc2UucGhwP2lkPTQmcmVmPTEpCgpGdWxsIHRleHQ6IDxodHRwczovL3d3dy5sZWFybnRlY2hsaWIub3JnL3AvMTk1MjM0Lz4KCioqQWJzdHJhY3QqKgoKTWFzc2l2ZSBPcGVuIE9ubGluZSBDb3Vyc2VzIGZvciBFZHVjYXRvcnMgKE1PT0MtRWRzKSBwcm92aWRlCm9wcG9ydHVuaXRpZXMgZm9yIHVzaW5nIHJlc2VhcmNoLWJhc2VkIGxlYXJuaW5nIGFuZCB0ZWFjaGluZyBwcmFjdGljZXMsCmFsb25nIHdpdGggbmV3IHRlY2hub2xvZ2ljYWwgdG9vbHMgYW5kIGZhY2lsaXRhdGlvbiBhcHByb2FjaGVzIGZvcgpkZWxpdmVyaW5nIHF1YWxpdHkgb25saW5lIHByb2Zlc3Npb25hbCBkZXZlbG9wbWVudC4gVGhlIFRlYWNoaW5nClN0YXRpc3RpY3MgVGhyb3VnaCBEYXRhIEludmVzdGlnYXRpb25zIE1PT0MtRWQgd2FzIGJ1aWx0IGZvciBwcmVwYXJpbmcKdGVhY2hlcnMgaW4gcGVkYWdvZ3kgZm9yIHRlYWNoaW5nIHN0YXRpc3RpY3MsIGFuZCBpdCBoYXMgYmVlbiBvZmZlcmVkIHRvCnBhcnRpY2lwYW50cyBmcm9tIGFyb3VuZCB0aGUgd29ybGQuIER1cmluZyAyMDE2LTIwMTcsIHByb2Zlc3Npb25hbApsZWFybmluZyB0ZWFtcyAoUExUcykgd2VyZSBmb3JtZWQgZnJvbSBhIHN1YnNldCBvZiBNT09DLUVkIHBhcnRpY2lwYW50cy4KVGhlc2UgdGVhbXMgbWV0IHNldmVyYWwgdGltZXMgdG8gc2hhcmUgYW5kIGRpc2N1c3MgdGhlaXIgbGVhcm5pbmcgYW5kCmV4cGVyaWVuY2VzLiBUaGlzIHN0dWR5IGZvY3VzZWQgb24gZXhhbWluaW5nIHRoZSB3YXlzIHRoYXQgYSBibGVuZGVkCmFwcHJvYWNoIHRvIHByb2Zlc3Npb25hbCBkZXZlbG9wbWVudCBtYXkgcmVzdWx0IGluIHNpbWlsYXIgb3IgZGlmZmVyZW50CnBhdHRlcm5zIG9mIGVuZ2FnZW1lbnQgdG8gdGhvc2Ugd2hvIG9ubHkgcGFydGljaXBhdGUgaW4gYSBsYXJnZS1zY2FsZQpvbmxpbmUgY291cnNlLiBSZXN1bHRzIHNob3cgdGhlIGJlbmVmaXRzIG9mIGEgYmxlbmRlZCBsZWFybmluZwplbnZpcm9ubWVudCBmb3IgcmV0ZW50aW9uLCBlbmdhZ2VtZW50IHdpdGggY291cnNlIG1hdGVyaWFscywgYW5kCmNvbm5lY3RlZG5lc3Mgd2l0aGluIHRoZSBvbmxpbmUgY29tbXVuaXR5IG9mIGxlYXJuZXJzIGluIGFuIG9ubGluZQpwcm9mZXNzaW9uYWwgZGV2ZWxvcG1lbnQgb24gdGVhY2hpbmcgc3RhdGlzdGljcy4gVGhlIGZpbmRpbmdzIHN1Z2dlc3QKdGhlIHVzZSBvZiBzZWxmLWZvcm1pbmcgYXV0b25vbW91cyBQTFRzIGZvciBzdXBwb3J0aW5nIGEgZGVlcGVyIGFuZCBtb3JlCmNvbXByZWhlbnNpdmUgZXhwZXJpZW5jZSB3aXRoIHNlbGYtZGlyZWN0ZWQgb25saW5lIHByb2Zlc3Npb25hbApkZXZlbG9wbWVudHMgc3VjaCBhcyBNT09Dcy4gT3RoZXIgb25saW5lIHByb2Zlc3Npb25hbCBkZXZlbG9wbWVudApjb3Vyc2VzLCBzdWNoIGFzIE1PT0NzLCBtYXkgYmVuZWZpdCBmcm9tIHB1cnBvc2VseSBzdWdnZXN0aW5nIGFuZAphZHZlcnRpc2luZywgYW5kIHBlcmhhcHMgZmFjaWxpdGF0aW5nLCB0aGUgZm9ybWF0aW9uIG9mIHNtYWxsCmZhY2UtdG8tZmFjZSBvciB2aXJ0dWFsIFBMVHMgd2hvIGNvbW1pdCB0byBlbmdhZ2UgaW4gbGVhcm5pbmcgdG9nZXRoZXIuCgoqKkRhdGEgU291cmNlICYgQW5hbHlzaXMqKgoKQWxsIHBlZXIgaW50ZXJhY3Rpb24sIGluY2x1ZGluZyBwZWVyIGRpc2N1c3Npb24sIHRha2UgcGxhY2Ugd2l0aGluCmRpc2N1c3Npb24gZm9ydW1zIG9mIE1PT0MtRWRzLCB3aGljaCBhcmUgaG9zdGVkIHVzaW5nIHRoZSBNb29kbGUKTGVhcm5pbmcgTWFuYWdlbWVudCBTeXN0ZW0uIFRvIGJ1aWxkIHRoZSBkYXRhc2V0IHlvdSdsbCBiZSB1c2luZyBmb3IKdGhpcyB3YWxrdGhyb3VnaCwgdGhlIHJlc2VhcmNoIHRlYW0gd3JvdGUgYSBxdWVyeSBmb3IgTW9vZGxlJ3MgTXlTUUwKZGF0YWJhc2UsIHdoaWNoIHJlY29yZHMgcGFydGljaXBhbnRzJyB1c2VyLWxvZ3Mgb2YgYWN0aXZpdHkgaW4gdGhlCm9ubGluZSBmb3J1bXMuIFRoaXMgc3FsIHF1ZXJ5IGNvbWJpbmVzIHNlcGFyYXRlIGRhdGFiYXNlIHRhYmxlcwpjb250YWluaW5nIHBvc3RpbmdzIGFuZCBjb21tZW50cyBpbmNsdWRpbmcgcGFydGljaXBhbnQgSURzLCB0aW1lc3RhbXBzLApkaXNjdXNzaW9uIHRleHQgYW5kIG90aGVyIGF0dHJpYnV0ZXMgb3IgIm1ldGFkYXRhLiIKCkZvciBmdXJ0aGVyIGRlc2NyaXB0aW9uIG9mIHRoZSBmb3J1bXMgYW5kIGRhdGEgcmV0cmlldmFsIHByb2Nlc3MsIHNlZQphbHNvIHRoZSBmb2xsb3dpbmcgcGFwZXJzOgoKLSAgIEtlbGxvZ2csIFMuLCAmIEVkZWxtYW5uLCBBLiAoMjAxNSkuIFtNYXNzaXZlbHkgT3BlbiBPbmxpbmUgQ291cnNlCiAgICBmb3IgRWR1Y2F0b3JzIChNT09D4oCQRWQpIG5ldHdvcmsKICAgIGRhdGFzZXRdKGh0dHBzOi8vYmVyYS1qb3VybmFscy5vbmxpbmVsaWJyYXJ5LndpbGV5LmNvbS9kb2kvcGRmZGlyZWN0LzEwLjExMTEvYmpldC4xMjMxMikuCiAgICAqQnJpdGlzaCBqb3VybmFsIG9mIGVkdWNhdGlvbmFsIHRlY2hub2xvZ3kqLCAqNDYqKDUpLCA5NzctOTgzLgoKLSAgIEV6ZW4tQ2FuLCBBLiwgQm95ZXIsIEsuIEUuLCBLZWxsb2dnLCBTLiwgJiBCb290aCwgUy4gKDIwMTUsIE1hcmNoKS4KICAgIFtVbnN1cGVydmlzZWQgbW9kZWxpbmcgZm9yIHVuZGVyc3RhbmRpbmcgTU9PQyBkaXNjdXNzaW9uIGZvcnVtczogYQogICAgbGVhcm5pbmcgYW5hbHl0aWNzCiAgICBhcHByb2FjaF0oaHR0cHM6Ly9kbC5hY20ub3JnL2RvaS9wZGYvMTAuMTE0NS8yNzIzNTc2LjI3MjM1ODkpLiBJbgogICAgKlByb2NlZWRpbmdzIG9mIHRoZSBmaWZ0aCBpbnRlcm5hdGlvbmFsIGNvbmZlcmVuY2Ugb24gbGVhcm5pbmcKICAgIGFuYWx5dGljcyBhbmQga25vd2xlZGdlKiAocHAuIDE0Ni0xNTApLgoKLSAgIEtlbGxvZ2csIFMuLCBCb290aCwgUy4sICYgT2xpdmVyLCBLLiAoMjAxNCkuIFtBIHNvY2lhbCBuZXR3b3JrCiAgICBwZXJzcGVjdGl2ZSBvbiBwZWVyIHN1cHBvcnRlZCBsZWFybmluZyBpbiBNT09DcyBmb3IKICAgIGVkdWNhdG9ycy5dKGh0dHBzOi8vd3d3LmVydWRpdC5vcmcvZW4vam91cm5hbHMvaXJyb2RsLzE5MDAtdjEtbjEtaXJyb2RsMDQ5NDUvMTA2NTU0NWFyLnBkZikKICAgICpJbnRlcm5hdGlvbmFsIFJldmlldyBvZiBSZXNlYXJjaCBpbiBPcGVuIGFuZCBEaXN0cmlidXRlZCBMZWFybmluZyosCiAgICAqMTUqKDUpLCAyNjMtMjg5LgoKKipTdW1tYXJ5IG9mIEtleSBGaW5kaW5ncyoqCgpUaGUgZm9sbG93aW5nIGhpZ2hsaWdodCBzb21lIGtleSBmaW5kaW5ncyByZWxhdGVkIHRvIHRoZSBkaXNjdXNzaW9uCmZvcnVtcyBpbiB0aGUgcGFwZXJzIGNpdGVkIGFib3ZlOgoKMS4gIE1PT0NzIGRlc2lnbmVkIHNwZWNpZmljYWxseSBmb3IgSy0xMiB0ZWFjaGVycyBjYW4gcHJvdmlkZSBwb3NpdGl2ZQogICAgc2VsZi1kaXJlY3RlZCBsZWFybmluZyBleHBlcmllbmNlcyBhbmQgcmljaCBlbmdhZ2VtZW50IGluIGRpc2N1c3Npb24KICAgIGZvcnVtcyB0aGF0IGhlbHAgZm9ybSBvbmxpbmUgY29tbXVuaXRpZXMgZm9yIGVkdWNhdG9ycy4KMi4gIEFuYWx5c2lzIG9mIGRpc2N1c3Npb24gZm9ydW0gZGF0YSBpbiBUU0RJIHByb3ZpZGVkIGEgdmVyeSBjbGVhcgogICAgcGljdHVyZSBvZiBob3cgZW50aHVzaWFzdGljIG1hbnkgUExUIG1lbWJlcnMgYW5kIGxlYWRlcnMgd2VyZSB0bwogICAgdGFsayB0byBvdGhlcnMgaW4gdGhlIG9ubGluZSBjb21tdW5pdHkuIFRoZXkgcG9zZWQgdGhlaXIgcXVlc3Rpb25zCiAgICBhbmQgc2hhcmVkIGlkZWFzIHdpdGggb3RoZXJzIGFib3V0IHRlYWNoaW5nIHN0YXRpc3RpY3MgdGhyb3VnaG91dAogICAgdGhlIHVuaXRzLCBldmVuIHRob3VnaCB0aGV5IHdlcmUgYWxzbyBtZWV0aW5nIHN5bmNocm9ub3VzbHkgc2V2ZXJhbAogICAgdGltZXMgd2l0aCB0aGVpciBjb2xsZWFndWVzIGluIHNtYWxsIGdyb3VwIFBMVHMuCjMuICBGaW5kaW5ncyBvbiBrbm93bGVkZ2UgY29uc3RydWN0aW9uIGRlbW9uc3RyYXRlZCB0aGF0IG92ZXIgaGFsZiBvZgogICAgdGhlIGRpc2N1c3Npb25zIGluIGJvdGggY291cnNlcyBtb3ZlZCBiZXlvbmQgc2hhcmluZyBpbmZvcm1hdGlvbiBhbmQKICAgIHN0YXRlbWVudHMgb2YgYWdyZWVtZW50IGFuZCBlbnRlcmVkIGEgcHJvY2VzcyBvZiBkaXNzb25hbmNlLAogICAgbmVnb3RpYXRpb24gYW5kIGNvLWNvbnN0cnVjdGlvbiBvZiBrbm93bGVkZ2UsIGJ1dCBzZWxkb20gbW92ZWQKICAgIGJleW9uZCB0aGlzIHBoYXNlIGluIHdoaWNoIG5ldyBrbm93bGVkZ2Ugd2FzIHRlc3RlZCBvciBhcHBsaWVkLgogICAgVGhlc2UgZmluZGluZ3MgZWNobyBzaW1pbGFyIHJlc2VhcmNoIG9uIGRpZmZpY3VsdGllcyBpbiBwcm9tb3RpbmcKICAgIGtub3dsZWRnZSBjb25zdHJ1Y3Rpb24gaW4gb25saW5lIHNldHRpbmdzLgo0LiAgVG9waWMgbW9kZWxpbmcgcHJvdmlkZXMgbW9yZSBpbnRlcnByZXRhYmxlIGFuZCBjb2hlc2l2ZSBtb2RlbHMgZm9yCiAgICBkaXNjdXNzaW9uIGZvcnVtcyB0aGFuIG90aGVyIHBvcHVsYXIgdW5zdXBlcnZpc2VkIG1vZGVsaW5nCiAgICB0ZWNobmlxdWVzIHN1Y2ggYXMgay1tZWFucyBhbmQgay1tZWRvaWRzIGNsdXN0ZXJpbmcgYWxnb3JpdGhtcy4KCiMjIyAxYi4gR3VpZGluZyBRdWVzdGlvbnMKCkZvciB0aGUgcGFwZXIsIFsqUGFydGljaXBhdGluZyBpbiBhIE1PT0MgYW5kIFByb2Zlc3Npb25hbCBMZWFybmluZyBUZWFtOgpIb3cgYSBCbGVuZGVkIEFwcHJvYWNoIHRvIFByb2Zlc3Npb25hbCBEZXZlbG9wbWVudCBNYWtlcyBhCkRpZmZlcmVuY2UqXShodHRwczovL3d3dy5sZWFybnRlY2hsaWIub3JnL3AvMTk1MjM0LyksIHRoZSByZXNlYXJjaGVycwp3ZXJlIGludGVyZXN0ZWQgaW4gdW5wYWNraW5nIGhvdyBwYXJ0aWNpcGFudHMgd2hvIGVucm9sbGVkIGluIHRoZQpUZWFjaGluZyBTdGF0aXN0aWNzIHRocm91Z2ggRGF0YSBJbnZlc3RpZ2F0aW9ucyBNT09DLUVkIG1pZ2h0IGJlbmVmaXQKZnJvbSBhbHNvIGJlaW5nIGluIGEgc21hbGxlciBncm91cCBvZiBwcm9mZXNzaW9uYWxzIGNvbW1pdHRlZCB0bwplbmdhZ2luZyBpbiB0aGUgc2FtZSBwcm9mZXNzaW9uYWwgZGV2ZWxvcG1lbnQuIFRoZSBzcGVjaWZpYyByZXNlYXJjaApxdWVzdGlvbiBmb3IgdGhpcyBwYXBlciB3YXM6Cgo+IFdoYXQgYXJlIHRoZSBzaW1pbGFyaXRpZXMgYW5kIGRpZmZlcmVuY2VzIGJldHdlZW4gaG93IFBMVCBtZW1iZXJzIGFuZAo+IE5vbi1QTFQgb25saW5lIHBhcnRpY2lwYW50cyBlbmdhZ2UgYW5kIG1lZXQgY291cnNlIGdvYWxzIGluIGEgTU9PQy1FZAo+IGRlc2lnbmVkIGZvciBlZHVjYXRvcnMgaW4gc2Vjb25kYXJ5IGFuZCBjb2xsZWdpYXRlIHNldHRpbmdzPyBEci4KPiBIb2xseWx5bm5lIExlZSBhbmQgdGhlIFRTREkgdGVhbSBhbHNvIGRldmVsb3BlZCBhIGZhY2lsaXRhdGlvbiBndWlkZQo+IGRlc2lnbmVkIHNwZWNpZmljYWxseSBmb3IgUExUIHRlYW1zIHRvIGhlbHAgZ3JvdXBzIHN5bnRoZXNpemUgdGhlCj4gaWRlYXMgaW4gdGhlIGNvdXJzZSBhbmQgbWFrZSBwbGFucyBmb3IgaG93IHRvIGltcGxlbWVudCBuZXcgc3RyYXRlZ2llcwo+IGluIHRoZWlyIGNsYXNzcm9vbSBpbiBvcmRlciB0byBpbXBhY3Qgc3R1ZGVudHMnIGxlYXJuaW5nIG9mCj4gc3RhdGlzdGljcy4gT25lIHF1ZXN0aW9uIFBMVCBtZW1iZXJzIHdlcmUgYXNrZWQgdG8gYWRkcmVzcyB3YXM6Cgo+IFdoYXQgaWRlYXMgb3IgaXNzdWVzIGVtZXJnZWQgaW4gdGhlIGRpc2N1c3Npb24gZm9ydW1zIHRoaXMgcGFzdCB3ZWVrPwo+IEZvciB0aGlzIHdhbGt0aHJvdWdoLCB3ZSB3aWxsIGZ1cnRoZXIgZXhhbWluZSB0aGF0IHF1ZXN0aW9uIHRocm91Z2gKPiB0aGUgdXNlIG9mIHRvcGljIG1vZGVsaW5nLgoKQW5kIGp1c3QgdG8gcmVpdGVyYXRlIHlldCBhZ2FpbiBmcm9tIFVuaXQgMSwgb25lIG92ZXJhcmNoaW5nIHF1ZXN0aW9uCndlJ2xsIGV4cGxvcmUgdGhyb3VnaG91dCB0aGlzIGNvdXJzZSwgYW5kIHRoYXQgU2lsZ2UgYW5kIFJvYmluc29uICgyMDE4KQppZGVudGlmeSBhcyBhIGNlbnRyYWwgcXVlc3Rpb24gdG8gdGV4dCBtaW5pbmcgYW5kIG5hdHVyYWwgbGFuZ3VhZ2UKcHJvY2Vzc2luZywgaXM6Cgo+IEhvdyBkbyB3ZSB0byAqKnF1YW50aWZ5Kiogd2hhdCBhIGRvY3VtZW50IG9yIGNvbGxlY3Rpb24gb2YgZG9jdW1lbnRzCj4gaXMgYWJvdXQ/CgojIyMgMWMuIFNldCBVcAoKQXMgaGlnaGxpZ2h0ZWQgaW4gW0NoYXB0ZXIgNiBvZiBEYXRhIFNjaWVuY2UgaW4gRWR1Y2F0aW9uIFVzaW5nClJdKGh0dHBzOi8vZGF0YXNjaWVuY2VpbmVkdWNhdGlvbi5jb20vYzA2Lmh0bWwpIChEU0lFVVIpLCBvbmUgb2YgdGhlCmZpcnN0IHN0ZXBzIG9mIGV2ZXJ5IHdvcmtmbG93IHNob3VsZCBiZSB0byBzZXQgdXAgYSAiUHJvamVjdCIgd2l0aGluClJTdHVkaW8uIFRoaXMgd2lsbCBiZSB5b3VyICJob21lIiBmb3IgYW55IGZpbGVzIGFuZCBjb2RlIHVzZWQgb3IgY3JlYXRlZAppbiBVbml0IDIuCgpZb3UgYXJlIHdlbGNvbWUgdG8gY29udGludWUgdXNpbmcgdGhlIHNhbWUgcHJvamVjdCBjcmVhdGVkIGZvciBVbml0IDEsCm9yIGNyZWF0ZSBhbiBlbnRpcmVseSBuZXcgcHJvamVjdCBmb3IgVW5pdCAyLiBIb3dldmVyLCBhZnRlciB5b3UndmUKY3JlYXRlZCB5b3VyIHByb2plY3Qgb3BlbiB1cCBhIG5ldyBSIHNjcmlwdCwgYW5kIGxvYWQgdGhlIGZvbGxvd2luZwpwYWNrYWdlcyB0aGF0IHdlJ2xsIGJlIG5lZWRpbmcgZm9yIHRoaXMgd2Fsa3Rocm91Z2g6CgpgYGB7ciBsb2FkLXBhY2thZ2VzLCBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeSh0aWR5dGV4dCkKbGlicmFyeShTbm93YmFsbEMpCmxpYnJhcnkodG9waWNtb2RlbHMpCmxpYnJhcnkoc3RtKQpsaWJyYXJ5KGxkYXR1bmluZykKbGlicmFyeShrbml0cikKbGlicmFyeShMREF2aXMpCmBgYAoKQXQgdGhlIGVuZCBvZiB0aGlzIHdlZWssIEkgZW5jb3VyYWdlIHlvdSBzaGFyZSB3aXRoIG1lIHlvdXIgUiBzY3JpcHQgYXMKZXZpZGVuY2UgdGhhdCB5b3UgaGF2ZSBjb21wbGV0ZSB0aGUgd2Fsa3Rocm91Z2guIEFsdGhvdWdoIEkgaGlnaGx5CnJlY29tbWVuZCB0aGF0IHRoYXQgeW91IG1hbnVhbGx5IHR5cGUgdGhlIGNvZGUgc2hhcmVkIHRocm91Z2hvdXQgdGhpcwp3YWxrdGhyb3VnaCwgZm9yIGxhcmdlIGJsb2NrcyBvZiB0ZXh0IGl0IG1heSBiZSBlYXNpZXIgdG8gY29weSBhbmQKcGFzdGUuCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KCiMjIDIuIFdSQU5HTEUKCkFzIG5vdGVkIHByZXZpb3VzbHksIGRhdGEgd3JhbmdsaW5nIGludm9sdmVzIHNvbWUgY29tYmluYXRpb24gb2YKY2xlYW5pbmcsIHJlc2hhcGluZywgdHJhbnNmb3JtaW5nLCBhbmQgbWVyZ2luZyBkYXRhIChXaWNraGFtICYKR3JvbGVtdW5kLCAyMDE3KS4gVGhpcyB3ZWVrIHdlJ2xsIHJldmlzaXQgdGlkeWluZyBhbmQgdG9rZW5pemluZyB0ZXh0CnVzaW5nIHRoZSBgdGlkeXRleHRgIHBhY2thZ2UsIGJ1dCBhcmUgYWxzbyBpbnRyb2R1Y2VkIHRvIHRoZSB0aGUgYHN0bWAKcGFja2FnZS4gVGhpcyBwYWNrYWdlIG1ha2VzIHVzZSBvZiBgdG1gIHRleHQgbWluaW5nIHBhY2thZ2UgdG8KcHJlcHJvY2VzcyB0ZXh0IChlLmcuLCByZW1vdmluZyBwdW5jdHVhdGlvbiwgc3RvcCB3b3JkcywgZXRjLikgYW5kIHdpbGwKYWxzbyBiZSBvdXIgZmlyc3QgaW50cm9kdWN0aW9uIHRvIHdvcmQgc3RlbW1pbmcuCgphLiAgKipJbXBvcnQgRGF0YSoqLiBXZSdsbCBiZSB3b3JraW5nIHdpdGggLmNzdiBmaWxlcyB0aGlzIHdlZWsgYW5kIHRoZQogICAgYHJlYWRfY3N2KClgIGZ1bmN0aW9uIGJ1dCB3aWxsIGludHJvZHVjZSBhIG5ldyBhcmd1bWVudCBmb3IgY2hhbmdpbmcKICAgIGNvbHVtbiB0eXBlcy4KYi4gICoqQ2FzdCBhIERUTSoqLiBXZSByZXZpc2l0IHRoZSBgdGlkeXRleHRgIHBhY2thZ2UgdG8gInRpZHkiIGFuZAogICAgdG9rZW5pemUgb3VyIGZvcnVtIGRhdGEgYW5kIGludHJvZHVjZSB0aGUgYGNhc3RfZHRtKClgIGZ1bmN0aW9uIHRvCiAgICBjcmVhdGUgdGhlIGRvY3VtZW50IHRlcm0gbWF0cml4IChkdG0pIG5lZWQgZm9yIHRvcGljIG1vZGVsaW5nLgpjLiAgKipUbyBTdGVtIG9yIG5vdCB0byBTVEVNPyoqIFdlIGNvbmNsdWRlIG91ciBkYXRhIHdyYW5nbGluZyBieSBhbHNvCiAgICBpbnRyb2R1Y2luZyB0aGUgYHRleHRQcm9jZXNzb3IoKWAgZnVuY3Rpb24gZm9yIHByZXByb2Nlc3NpbmcgYW5kCiAgICBkaXNjdXNzIHRoZSBwcm9zIGFuZCBjb25zIG9mIHdvcmQgc3RlbW1pbmcuCgojIyMgMmEuIEltcG9ydCBGb3J1bSBEYXRhCgpUbyBnZXQgc3RhcnRlZCwgd2UgbmVlZCB0byBpbXBvcnQsIG9yICJyZWFkIiwgb3VyIGRhdGEgaW50byBSLiBUaGUKZnVuY3Rpb24gdXNlZCB0byBpbXBvcnQgeW91ciBkYXRhIHdpbGwgZGVwZW5kIG9uIHRoZSBmaWxlIGZvcm1hdCBvZiB0aGUKZGF0YSB5b3UgYXJlIHRyeWluZyB0byBpbXBvcnQuIEZpcnN0LCBob3dldmVyLCB5b3UnbGwgbmVlZCB0byBkbyB0aGUKZm9sbG93aW5nOgoKMS4gIERvd25sb2FkIHRoZSBgdHNfZm9ydW1fZGF0YS5jc3ZgIGZpbGUgd2UnbGwgYmUgdXNpbmcgZm9yIHRoaXMgVW5pdAogICAgZnJvbSBvdXIgTkNTVSBNb29kbGUgY291cnNlIHNpdGUuCjIuICBDcmVhdGUgYSBmb2xkZXIgaW4gdGhlIGRpcmVjdG9yeSBvbiB5b3VyIGNvbXB1dGVyIHdoZXJlIHlvdSBzdG9yZWQKICAgIHlvdXIgUiBQcm9qZWN0IGFuZCBuYW1lIGl0ICJkYXRhIi4KMy4gIEFkZCB0aGUgZmlsZSB0byB5b3VyIGRhdGEgZm9sZGVyLgo0LiAgQ2hlY2sgeW91ciBGaWxlcyB0YWIgaW4gUlN0dWRpbyB0byB2ZXJpZnkgdGhhdCB5b3VyIGZpbGUgaXMgaW5kZWVkCiAgICBpbiB5b3VyIGRhdGEgZm9sZGVyLgoKTm93IGxldCdzIHJlYWQgb3VyIGRhdGEgaW50byBvdXIgRW52aXJvbm1lbnQgdXNpbmcgdGhlIGByZWFkX2NzdigpYApmdW5jdGlvbiBhbmQgYXNzaWduIGl0IHRvIGEgdmFyaWFibGUgbmFtZSBzbyB3ZSBjYW4gd29yayB3aXRoIGl0IGxpa2UKYW55IG90aGVyIG9iamVjdCBpbiBSLgoKYGBge3IgcmVhZC1jc3Z9CnRzX2ZvcnVtX2RhdGEgPC0gcmVhZF9jc3YoImRhdGEvdHNfZm9ydW1fZGF0YS5jc3YiLCAKICAgICBjb2xfdHlwZXMgPSBjb2xzKGNvdXJzZV9pZCA9IGNvbF9jaGFyYWN0ZXIoKSwKICAgICAgICAgICAgICAgICAgIGZvcnVtX2lkID0gY29sX2NoYXJhY3RlcigpLCAKICAgICAgICAgICAgICAgICAgIGRpc2N1c3Npb25faWQgPSBjb2xfY2hhcmFjdGVyKCksIAogICAgICAgICAgICAgICAgICAgcG9zdF9pZCA9IGNvbF9jaGFyYWN0ZXIoKQogICAgICAgICAgICAgICAgICAgKQogICAgKQpgYGAKCkJ5IGRlZmF1bHQsIG1hbnkgb2YgdGhlIGNvbHVtbnMgbGlrZSBgY291cnNlX2lkYCBhbmQgYGZvcnVtX2lkYCBhcmUgcmVhZAppbiBhcyBudW1lcmljIGRhdGEuIEZvciBvdXIgcHVycG9zZXMsIHdlIHBsYW4gdG8gdHJlYXQgdGhlbSBhcyB1bmlxdWUKaWRlbnRpZmllcnMgb3IgbmFtZXMgZm9yIG91dCBjb3Vyc2VzLCBmb3J1bXMsIGRpc2N1c3Npb25zLCBhbmQgcG9zdHMuClRoZSBgcmVhZF9jc3YoKWAgZnVuY3Rpb24gaGFzIGEgaGFuZHkgYGNvbF90eXBlcyA9YCBhcmd1bWVudCBjaGFuZ2luZwp0aGUgY29sdW1uIHR5cGVzIGZyb20gbnVtZXJpYyB0byBjaGFyYWN0ZXJzLgoKIyMjIDJiLiBDYXN0IGEgRG9jdW1lbnQgVGVybSBNYXRyaXgKCkluIHRoaXMgc2VjdGlvbiB3ZSdsbCByZXZpc2l0IHNvbWUgZmFtaWxpYXIgYHRpZHl0ZXh0YCBmdW5jdGlvbnMgdXNlZCBpbgpVbml0cyAxICYgMiBmb3IgdGlkeWluZyBhbmQgdG9rZW5pemluZyB0ZXh0IGFuZCBpbnRyb2R1Y2Ugc29tZSBuZXcKZnVuY3Rpb25zIGZyb20gdGhlIGBzdG1gIHBhY2thZ2UgZm9yIHByb2Nlc3NpbmcgdGV4dCBhbmQgdHJhbnNmb3JtaW5nCm91ciBkYXRhIGZyYW1lcyBpbnRvIG5ldyBkYXRhIHN0cnVjdHVyZXMgcmVxdWlyZWQgZm9yIHRvcGljIG1vZGVsaW5nLgoKIyMjIyBGdW5jdGlvbnMgVXNlZAoKKipgdGlkeXRleHRgIGZ1bmN0aW9ucyoqCgotICAgYHVubmVzdF90b2tlbnMoKWAgc3BsaXRzIGEgY29sdW1uIGludG8gdG9rZW5zCi0gICBgYW50aV9qb2luKClgIHJldHVybnMgYWxsIHJvd3MgZnJvbSB4IHdpdGhvdXQgYSBtYXRjaCBpbiB5IGFuZCB1c2VkCiAgICB0byByZW1vdmUgYHN0b3Agd29yZHNgIGZyb20gb3V0IGRhdGEuCi0gICBgY2FzdF9kdG0oKWAgdGFrZXMgYSB0aWRpZWQgZGF0YSBmcmFtZSB0YWtlIGFuZCAiY2FzdHMiIGl0IGludG8gYQogICAgZG9jdW1lbnQtdGVybSBtYXRyaXggKGR0bSkKCioqYGRwbHlyYCoqICoqZnVuY3Rpb25zKioKCi0gICBgY291bnQoKWAgbGV0cyB5b3UgcXVpY2tseSBjb3VudCB0aGUgdW5pcXVlIHZhbHVlcyBvZiBvbmUgb3IgbW9yZQogICAgdmFyaWFibGVzCi0gICBgZ3JvdXBfYnkoKWAgdGFrZXMgYSBkYXRhIGZyYW1lIGFuZCBvbmUgb3IgbW9yZSB2YXJpYWJsZXMgdG8gZ3JvdXAKICAgIGJ5Ci0gICBgc3VtbWFyaXNlKClgIGNyZWF0ZXMgYSBzdW1tYXJ5IG9mIGRhdGEgdXNpbmcgYXJndW1lbnRzIGxpa2Ugc3VtIGFuZAogICAgbWVhbgoKKipgc3RtYCBmdW5jdGlvbnMqKgoKLSAgIGB0ZXh0UHJvY2Vzc29yKClgIHRha2VzIGluIGEgdmVjdG9yIG9yIGNvbHVtbiBvZiByYXcgdGV4dHMgYW5kCiAgICBwZXJmb3JtcyB0ZXh0IHByb2Nlc3NpbmcgbGlrZSByZW1vdmluZyBwdW5jdHVhdGlvbiBhbmQgd29yZAogICAgc3RlbW1pbmcuCi0gICBgcHJlcERvY3VtZW50cygpYCBwZXJmb3JtcyBzZXZlcmFsIGNvcnB1cyBtYW5pcHVsYXRpb25zIGluY2x1ZGluZwogICAgcmVtb3Zpbmcgd29yZHMgYW5kIHJlbnVtYmVyaW5nIHdvcmQgaW5kaWNlcwoKIyMjIyBUaWR5aW5nIFRleHQKClByaW9yIHRvIHRvcGljIG1vZGVsaW5nLCB3ZSBoYXZlIGEgZmV3IHJlbWFpbmluZyBzdGVwcyB0byB0aWR5IG91ciB0ZXh0CnRoYXQgaG9wZWZ1bGx5IHNob3VsZCBmZWVsIGZhbWlsaWFyIGJ5IHRoaXMgcG9pbnQuIElmIHlvdSByZWNhbGwgZnJvbQpbQ2hhcHRlciAxIG9mIFRleHQgTWluaW5nIFdpdGgKUl0oaHR0cHM6Ly93d3cudGlkeXRleHRtaW5pbmcuY29tL3RpZHl0ZXh0Lmh0bWwpLCB0aGVzZSBwcmVwcm9jZXNzaW5nCnN0ZXBzIGluY2x1ZGU6CgoxLiAgVHJhbnNmb3JtaW5nIG91ciB0ZXh0IGludG8gInRva2VucyIKMi4gIFJlbW92aW5nIHVubmVjZXNzYXJ5IGNoYXJhY3RlcnMsIHB1bmN0dWF0aW9uLCBhbmQgd2hpdGVzcGFjZQozLiAgQ29udmVydGluZyBhbGwgdGV4dCB0byBsb3dlcmNhc2UKNC4gIFJlbW92aW5nIHN0b3Agd29yZHMgc3VjaCBhcyAidGhlIiwgIm9mIiwgYW5kICJ0byIKCkxldCdzIHRva2VuaXplIG91ciBmb3J1bSB0ZXh0IGFuZCBieSB1c2luZyB0aGUgZmFtaWxpYXIKYHVubmVzdF90b2tlbnMoKWAgYW5kIHJlbW92ZSBzdG9wIHdvcmRzIHBlciB1c3VhbDoKCmBgYHtyIHRva2VuaXplLWZvcnVtc30KZm9ydW1zX3RpZHkgPC0gdHNfZm9ydW1fZGF0YSAlPiUKICB1bm5lc3RfdG9rZW5zKG91dHB1dCA9IHdvcmQsIGlucHV0ID0gcG9zdF9jb250ZW50KSAlPiUKICBhbnRpX2pvaW4oc3RvcF93b3JkcywgYnkgPSAid29yZCIpCmZvcnVtc190aWR5CmBgYAoKTm93IGxldCdzIGRvIGEgcXVpY2sgd29yZCBjb3VudCB0byBzZWUgc29tZSBvZiB0aGUgbW9zdCBjb21tb24gd29yZHMKdXNlZCB0aHJvdWdob3V0IHRoZSBmb3J1bXMuIFRoaXMgc2hvdWxkIGdldCBhIHNlbnNlIG9mIHdoYXQgd2UncmUKd29ya2luZyB3aXRoIGFuZCBsYXRlciB3ZSdsbCBuZWVkIHRoZXNlIHdvcmQgY291bnRzIGZvciBjcmVhdGluZyBvdXIKZG9jdW1lbnQgdGVybSBtYXRyaXggZm9yIHRvcGljIG1vZGVsaW5nOgoKYGBge3IgY291bnQtd29yZHN9CmZvcnVtc190aWR5ICU+JQogIGNvdW50KHdvcmQsIHNvcnQgPSBUUlVFKQpgYGAKClRlcm1zIGxpa2UgInN0dWRlbnRzLCIgImRhdGEsIiBhbmQgImNsYXNzIiBhcmUgYWJvdXQgd2hhdCB3ZSB3b3VsZCBoYXZlCmV4cGVjdGVkIGZyb20gYSBjb3Vyc2UgdGVhY2hpbmcgc3RhdGlzdGljcy4gVGhlIHRlcm0gImFncmVlIiBhbmQgInRpbWUiCmhvd2V2ZXIsIGFyZSBub3Qgc28gaW50dWl0aXZlIGFuZCB3b3J0aCBhIHF1aWNrIGxvb2sgYXMgd2VsbC4KCiMjIyMjIOKchSBDb21wcmVoZW5zaW9uIENoZWNrCgpVc2UgdGhlIGBmaWx0ZXIoKWAgYW5kIGBncmVwbCgpYCBmdW5jdGlvbnMgaW50cm9kdWNlZCBpbiBbVW5pdCAxLgpTZWN0aW9uCjNiXShodHRwOi8vc2hpeWFuMjAzMC5jb20vdW5pdC0xLXdhbGt0aHJvdWdoLmh0bWwjM2JfV29yZF9TZWFyY2gpIHRvCmZpbHRlciBmb3Igcm93cyBpbiBvdXIgYHRzX2ZvcnVtX2RhdGFgIGRhdGEgZnJhbWUgdGhhdCBjb250YWluIHRoZSB0ZXJtcwoiYWdyZWUiIGFuZCAidGltZSIgYW5kIGFub3RoZXIgdGVybSBvciB0ZXJtcyBvZiB5b3VyIGNob29zaW5nLiBTZWxlY3QgYQpyYW5kb20gc2FtcGxlIG9mIDEwIHBvc3RzIHVzaW5nIHRoZSBgc2FtcGxlX24oKWAgZnVuY3Rpb24gZm9yIHlvdXIgdGVybXMKYW5kIGFuc3dlciB0aGUgZm9sbG93aW5nIHF1ZXN0aW9uczoKCjEuICBXaGF0LCBpZiBhbnl0aGluZywgZG8gdGhlc2UgcG9zdHMgaGF2ZSBpbiBjb21tb24/ICpGb3IgJ2FncmVlLCcgdGhlCiAgICBwb3N0cyBhcmUgYWxsIHJlc3BvbnNlcyB0byBzb21lb25lIGVsc2UuIFRoZXkgYXJlIGFtcGxpZnlpbmcKICAgIHNvbWV0aGluZyB0aGF0IHdhcyBzYWlkLiAnVGltZScgY29tbWVudHMgZm9jdXNlZCBhIGxvdCBvbiBwbGFubmluZwogICAgY291cnNlcyBhbmQgZ2l2aW5nIHN0dWRlbnRzIHRpbWUgdG8gZG8gdGhlIHdvcmsuIE15IHRoaXJkIHdvcmQgd2FzCiAgICAnbWF0aC4nICdNYXRoJyBkaXNjdXNzaW9ucyBmb2N1c2VkIG9uIHRoZSBkaWZmaWN1bHR5IG9mIHRlYWNoaW5nCiAgICBzcGVjaWZpYyBjb25jZXB0cyBhbmQgaG93IHRoZSBzaW11bGF0aW9uIGhlbHBlZCBzdHVkZW50cyBsZWFybgogICAgZmFzdGVyLioKCjIuICBXaGF0IHRvcGljcyBvciB0aGVtZXMgbWlnaHQgYmUgYXBwYXJlbnQsIG9yIGRvIHlvdSBhbnRpY2lwYXRlCiAgICBlbWVyZ2luZywgZnJvbSBvdXIgdG9waWMgbW9kZWxpbmc/ICpQbGFubmluZywgc2ltdWxhdGlvbiwgc3R1ZGVudAogICAgc2tpbGwgbGV2ZWxzLCB0ZWFjaGluZyBzdGF0aXN0aWNzLCB0ZWFjaGluZyBtYXRoLioKCllvdXIgb3V0cHV0IHNob3VsZCBsb29rIHNvbWV0aGluZyBsaWtlIHRoaXM6CgpgYGB7ciBmaW5kLXF1b3RlcywgZWNobz1GQUxTRX0KZm9ydW1fcXVvdGVzIDwtIHRzX2ZvcnVtX2RhdGEgJT4lCiAgc2VsZWN0KHBvc3RfY29udGVudCkgJT4lIAogIGZpbHRlcihncmVwbCgndGltZScsIHBvc3RfY29udGVudCkpCnNhbXBsZV9uKGZvcnVtX3F1b3RlcywxMCkKYGBgCgojIyMjIENyZWF0aW5nIGEgRG9jdW1lbnQgVGVybSBNYXRyaXgKCkFzIGhpZ2hsaWdodGVkIGluIFtDaGFwdGVyIDUgb2YgVGV4dCBNaW5pbmcgd2l0aApSXShodHRwczovL3d3dy50aWR5dGV4dG1pbmluZy5jb20vZHRtLmh0bWwjY2FzdC1kdG0pLCB0aGUgYHRvcGljbW9kZWxzYApwYWNrYWdlIGFuZCB0aGUgW0xhdGVudCBEaXJpY2hsZXQgYWxsb2NhdGlvbgooTERBKV0oaHR0cHM6Ly93d3cudGlkeXRleHRtaW5pbmcuY29tL3RvcGljbW9kZWxpbmcuaHRtbCNsYXRlbnQtZGlyaWNobGV0LWFsbG9jYXRpb24pCmFsZ29yaXRobSBhbmQgYExEQSgpYCBmdW5jdGlvbiBpdCB1c2VzIGV4cGVjdHMgZG9jdW1lbnQtdGVybSBtYXRyaXggYXMKdGhlIGRhdGEgaW5wdXQuCgpCZWZvcmUgd2UgY3JlYXRlIGEgb3VyIGRvY3VtZW50LXRlcm0gbWF0cml4LCBob3dldmVyLCB3ZSBoYXZlIGFuCmltcG9ydGFudCBkZWNpc2lvbiB0byBtYWtlOgoKPiAqKldoYXQgZG8gd2UgY29uc2lkZXIgdG8gYmUgYSAiZG9jdW1lbnQiIGluIGEgTU9PQy1FZCBkaXNjdXNzaW9uCj4gZm9ydW0/KiogRm9yIGV4YW1wbGUsIHdlIGNvdWxkIGNvbnNpZGVyIGVhY2ggaW5kaXZpZHVhbCBkaXNjdXNzaW9uCj4gcG9zdCwgb3IgYHBvc3RfaWRgIGluIG91ciBkYXRhIGZyYW1lLCBhcyBhIGRvY3VtZW50LiBJdCBtaWdodCBhbHNvCj4gbWFrZSBzZW5zZSB0byBjb21iaW5lIHRleHRzIGZyb20gYWxsIHBvc3RzIHdpdGhpbiBlYWNoIGRpc2N1c3Npb24sIG9yCj4gYGRpc2NjdXNzaW9uX2lkYCwgYW5kIGNvbnNpZGVyIHRoYXQgYXMgYSBkb2N1bWVudCBzaW5jZSB0aGVzZSBwb3N0cwo+IGFyZSBvZnRlbiBpbnRlcmNvbm5lY3RlZCBhbiBidWlsZCBvZmYgb25lIGFub3RoZXIuCgpGb3Igbm93LCBob3dldmVyLCBsZXQncyB0cmVhdCBlYWNoIGluZGl2aWR1YWwgcG9zdCBhcyBhIHVuaXF1ZQoiZG9jdW1lbnQuIiBBcyBub3RlZCBhYm92ZSwgdG8gY3JlYXRlIG91ciBkb2N1bWVudCB0ZXJtIG1hdHJpeCwgd2UnbGwKbmVlZCB0byBmaXJzdCBgY291bnQoKWAgaG93IG1hbnkgdGltZXMgZWFjaCBgd29yZGAgb2NjdXJzIGluIGVhY2gKZG9jdW1lbnQsIG9yIGBwb3N0X2lkYCBpbiBvdXIgY2FzZSwgYW5kIGNyZWF0ZSBhIG1hdHJpeCB0aGF0IGNvbnRhaW5zCm9uZSByb3cgcGVyIHBvc3QgYXMgb3VyIG9yaWdpbmFsIGRhdGEgZnJhbWUgZGlkLCBidXQgbm93IGNvbnRhaW5zIGEKY29sdW1uIGZvciBlYWNoIGB3b3JkYCBpbiB0aGUgZW50aXJlIGNvcnB1cyBhbmQgYSB2YWx1ZSBvZiBgbmAgZm9yIGhvdwptYW55IHRpbWVzIHRoYXQgd29yZCBvY2N1cnMgaW4gZWFjaCBwb3N0LgoKVG8gY3JlYXRlIHRoaXMgZG9jdW1lbnQgdGVybSBtYXRyaXggZnJvbSBvdXIgcG9zdCBjb3VudHMsIHdlJ2xsIHVzZSB0aGUKYGNhc3RfZHRtKClgIGZ1bmN0aW9uIGxpa2Ugc28gYW5kIGFzc2lnbiBpdCB0byB0aGUgdmFyaWFibGUKYGZvcnVtc19kdG1gOgoKYGBge3IgY2FzdC1kdG19CmZvcnVtc19kdG0gPC0gZm9ydW1zX3RpZHkgJT4lCiAgY291bnQocG9zdF9pZCwgd29yZCkgJT4lCiAgY2FzdF9kdG0ocG9zdF9pZCwgd29yZCwgbikKYGBgCgojIyMjIyDinIUgQ29tcHJlaGVuc2lvbiBDaGVjawoKVGFrZSBhIGxvb2sgYXQgb3VyIGBmb3J1bXNfZHRtYCBvYmplY3QgaW4gdGhlIGNvbnNvbGUgYW5kIGFuc3dlciB0aGUKZm9sbG93aW5nIHF1ZXN0aW9uOgoKMS4gIFdoYXQgImNsYXNzIiBvZiBvYmplY3QgaXMgYGZvcnVtc19kdG1gPyAqc2ltcGxlX3RyaXBsZXRfbWF0cml4KgoyLiAgSG93IG1hbnkgdW5pcXVlIGRvY3VtZW50cyBhbmQgdGVybXMgYXJlIGluY2x1ZGVkIG91ciBtYXRyaXg/ICo1LDc2MQogICAgZG9jdW1lbnRzIGFuZCAxMyw2NjYgdGVybXMuKgozLiAgV2h5IG1pZ2h0IHRoZXJlIGJlIGZld2VyIGRvY3VtZW50cy9wb3N0cyB0aGFuIHdlcmUgaW4gb3VyIG9yaWdpbmFsCiAgICBkYXRhIGZyYW1lPyAqQmxhbmsgcG9zdHMgd291bGQgaGF2ZSBub3RoaW5nIHRvIGNvdW50IG9yIHBvc3NpYmx5CiAgICBmaWd1cmVzIG5vdCByZWNvZ25pemVkIGFzIHdvcmRzLCBsaWtlIGVtb2ppcy4qCjQuICBXaGF0IGV4YWN0bHkgaXMgbWVhbnQgYnkgInNwYXJzaXR5Ij8gKlRoZSBhbW91bnQgb2YgdGhlIERUTSB0aGF0IGhhcwogICAgemVyb3MgYXMgY291bnRzIGZvciB0ZXJtcy4gVGhpcyBpcyBhIHJlbGF0aXZlIHZhbHVlIG9mIGhvdyBvZnRlbiB5b3UKICAgIHNlZSBjb21tb24gd29yZHMgdGhyb3VnaG91dCBhIGNvcnB1cy4gVGhlIGhpZ2hlciB0aGUgdmFsdWUsIHRoZQogICAgZWFzaWVyIGl0IGlzIHRvIHBvdGVudGlhbGx5IGRpc3Rpbmd1aXNoIGluZGl2aWR1YWwgZG9jdW1lbnRzLiBBCiAgICBsb3dlciBzcGFyc2l0eSBpbXBsaWVzIG1hbnkgb2YgdGhlIHNhbWUgd29yZHMga2VlcCBhcHBlYXJpbmcgaW4KICAgIGRpZmZlcmVudCBkb2N1bWVudHMuKgoKYGBge3IgY2xhc3MtZHRtLCBlY2hvPUZBTFNFfQpjbGFzcyhmb3J1bXNfZHRtKQpmb3J1bXNfZHRtCmBgYAoKIyMjIDJjLiBUbyBTdGVtIG9yIG5vdCB0byBTdGVtPwoKTmV4dCB3ZSdsbCBuZWVkIHRvIHByZXBhcmUgb3VyIG9yaWdpbmFsIGRhdGEgc2V0IGZvciBzdHJ1Y3R1cmFsIHRvcGljCm1vZGVsaW5nIHVzaW5nIHRoZSBgdGV4dFByb2Nlc3NvcigpYCBmdW5jdGlvbi4gVGhlIGBzdG1gIHBhY2thZ2UgaGFzIGEKbnVtYmVyIG9mIGZlYXR1cmVzIHRoYXQgZXh0ZW5kIHRoZSBmdW5jdGlvbmFsaXR5IG9mIHRoZSBgdG9waWNtb2RlbHNgCnBhY2thZ2UsIGluY2x1ZGluZyBhbiBhcmd1bWVudCBmb3IgInN0ZW1taW5nIiB3b3Jkcywgd2hpY2ggW1NjaG9maWVsZAphbmQgTWltbm8KKDIwMTYpXShodHRwczovL2RpcmVjdC5taXQuZWR1L3RhY2wvYXJ0aWNsZS9kb2kvMTAuMTE2Mi90YWNsX2FfMDAwOTkvNDMzNzAvQ29tcGFyaW5nLUFwcGxlcy10by1BcHBsZS1UaGUtRWZmZWN0cy1vZi1TdGVtbWVycykKZGVzY3JpYmUgYXMgZm9sbG93czoKCj4gU3RlbW1pbmcgaXMgYSBwb3B1bGFyIHdheSB0byByZWR1Y2UgdGhlIHNpemUgb2YgYSB2b2NhYnVsYXJ5IGluCj4gbmF0dXJhbCBsYW5ndWFnZSB0YXNrcyBieSBjb25mbGF0aW5nIHdvcmRzIHdpdGggcmVsYXRlZCBtZWFuaW5ncy4KPiBTcGVjaWZpY2FsbHksIHN0ZW1taW5nIGFpbXMgdG8gY29udmVydCB3b3JkcyB3aXRoIHRoZSBzYW1lICJzdGVtIiBvcgo+IHJvb3QgKGUuZyAiY3JlYXRpdmUiIGFuZCAiY3JlYXRvciIpIHRvIGEgc2luZ2xlIHdvcmQgdHlwZSAoImNyZWF0ZSIpLgo+IFRob3VnaCBvcmlnaW5hbGx5IGRldmVsb3BlZCBpbiB0aGUgY29udGV4dCBvZiBpbmZvcm1hdGlvbiByZXRyaWV2YWwKPiAoSVIpIHN5c3RlbXMsIHN0ZW1tZXJzIGFyZSBub3cgY29tbW9ubHkgdXNlZCBhcyBhIHByZXByb2Nlc3Npbmcgc3RlcAo+IGluIHVuc3VwZXJ2aXNlZCBtYWNoaW5lIGxlYXJuaW5nIHRhc2tzLiBUaGUgcmF0aW9uYWxlIGJlaGluZCBzdGVtbWluZwo+IGlzIHRoYXQgaXQgY2FuIGRyYW1hdGljYWxseSByZWR1Y2UgdGhlIG51bWJlciBvZiB3b3JkcyBvciB0ZXJtcyB0byBiZQo+IG1vZGVsZWQsIHdoaWNoIGluIHRoZW9yeSBzaG91bGQgaGVscCBzaW1wbGlmeSBhbmQgaW1wcm92ZSB0aGUKPiBwZXJmb3JtYW5jZSBvZiB5b3VyIG1vZGVsLiBXZSdsbCBleHBsb3JlIHRoaXMgYXNzdW1wdGlvbiBhIGxpdHRsZQo+IGxhdGVyIGluIHRoaXMgc2VjdGlvbi4KCiMjIyMgUHJvY2Vzc2luZyBhbmQgU3RlbW1pbmcgZm9yIFNUTQoKTGlrZSBgdW5uZXN0X3Rva2VucygpYCwgdGhlIGB0ZXh0UHJvY2Vzc29yKClgIGZ1bmN0aW9uIGluY2x1ZGVzIHNldmVyYWwKdXNlZnVsIGFyZ3VtZW50cyBmb3IgcHJvY2Vzc2luZyB0ZXh0IGxpa2UgY29udmVydGluZyB0ZXh0IHRvIGxvd2VyY2FzZQphbmQgcmVtb3ZpbmcgcHVuY3R1YXRpb24gYW5kIG51bWJlcnMuIEkndmUgaW5jbHVkZWQgc2V2ZXJhbCBvZiB0aGVzZSBpbgp0aGUgc2NyaXB0IGJlbG93IGFsb25nIHdpdGggdGhlaXIgZGVmYXVsdHMgdXNlZCBpZiB5b3UgZG8gbm90IGV4cGxpY2l0bHkKc3BlY2lmeSBpbiB5b3VyIGZ1bmN0aW9uLiBNb3N0IG9mIHRoZXNlIGFyZSBwcmV0dHkgaW50dWl0aXZlIGFuZCB5b3UgY2FuCmxlYXJuIG1vcmUgYnkgdmlld2luZyB0aGUgYD90ZXh0UHJvY2Vzc29yYCBkb2N1bWVudGF0aW9uLgoKTGV0J3MgZ28gYWhlYWQgYW5kIHByb2Nlc3Mgb3VyIGRpc2N1c3Npb24gZm9ydW0gYHBvc3RfY29udGVudGAgaW4KcHJlcGFyYXRpb24gZm9yIHN0cnVjdHVyYWwgdG9waWMgbW9kZWxpbmc6CgpgYGB7ciB0ZXh0UHJvY2Vzc29yfQp0ZW1wIDwtIHRleHRQcm9jZXNzb3IodHNfZm9ydW1fZGF0YSRwb3N0X2NvbnRlbnQsIAogICAgICAgICAgICAgICAgICAgIG1ldGFkYXRhID0gdHNfZm9ydW1fZGF0YSwgIAogICAgICAgICAgICAgICAgICAgIGxvd2VyY2FzZT1UUlVFLCAKICAgICAgICAgICAgICAgICAgICByZW1vdmVzdG9wd29yZHM9VFJVRSwgCiAgICAgICAgICAgICAgICAgICAgcmVtb3ZlbnVtYmVycz1UUlVFLCAgCiAgICAgICAgICAgICAgICAgICAgcmVtb3ZlcHVuY3R1YXRpb249VFJVRSwgCiAgICAgICAgICAgICAgICAgICAgd29yZExlbmd0aHM9YygzLEluZiksCiAgICAgICAgICAgICAgICAgICAgc3RlbT1UUlVFLAogICAgICAgICAgICAgICAgICAgIG9ubHljaGFyYWN0ZXI9IEZBTFNFLCAKICAgICAgICAgICAgICAgICAgICBzdHJpcGh0bWw9VFJVRSwgCiAgICAgICAgICAgICAgICAgICAgY3VzdG9tc3RvcHdvcmRzPU5VTEwpCmBgYAoKTm90ZSB0aGF0IHRoZSBmaXJzdCBhcmd1bWVudCB0aGUgYHRleHRQcm9jZXNzb3JgIGZ1bmN0aW9uIGV4cGVjdHMgaXMgdGhlCmNvbHVtbiBpbiBvdXIgZGF0YSBmcmFtZSB0aGF0IGNvbnRhaW5zIHRoZSB0ZXh0IHRvIGJlIHByb2Nlc3NlZCwgdGhlCnNlY29uZCBhcmd1bWVudCBgbWV0YWRhdGEgPWAgZXhwZWN0cyB0aGUgZGF0YSBmcmFtZSB0aGF0IGNvbnRhaW5zIHRoZQp0ZXh0IG9mIGludGVyZXN0IGFuZCB1c2VzIHRoZSBjb2x1bW4gbmFtZXMgdG8gbGFiZWwgdGhlIG1ldGFkYXRhIHN1Y2ggYXMKY291cnNlIGlkcyBhbmQgZm9ydW0gbmFtZXMuIFRoaXMgbWVhdGRhdGEgY2FuIGJlIHVzZWQgdG8gdG8gaW1wcm92ZSB0aGUKYXNzaWdubWVudCBvZiB3b3JkcyB0byB0b3BpY3MgaW4gYSBjb3JwdXMgYW5kIGV4YW1pbmUgdGhlIHJlbGF0aW9uc2hpcApiZXR3ZWVuIHRvcGljcyBhbmQgdmFyaW91cyBjb3ZhcmlhdGVzIG9mIGludGVyZXN0LgoKVW5saWtlIHRoZSBgdW5uZXN0X3Rva2VucygpYCBmdW5jdGlvbiwgdGhlIG91dHB1dCBpcyBub3QgYSBuaWNlIHRpZHkKZGF0YSBmcmFtZS4gVG9waWMgbW9kZWxpbmcgdXNpbmcgdGhlIGBzdG1gIHBhY2thZ2UgcmVxdWlyZXMgYSB2ZXJ5CnVuaXF1ZSBzZXQgb2YgaW5wdXRzIHRoYXQgYXJlIHNwZWNpZmljIHRvIHRoZSBwYWNrYWdlLiBUaGUgZm9sbG93aW5nCmNvZGUgd2lsbCBwdWxsIGVsZW1lbnRzIGZyb20gdGhlIGB0ZW1wYCBsaXN0IHRoYXQgd2FzIGNyZWF0ZWQgdGhhdCB3aWxsCmJlIHJlcXVpcmVkIGZvciB0aGUgYHN0bSgpYCBmdW5jdGlvbiB3ZSdsbCB1c2UgaW4gU2VjdGlvbiA0OgoKYGBge3Igc3RtLWlucHV0c30KbWV0YSA8LSB0ZW1wJG1ldGEKdm9jYWIgPC0gdGVtcCR2b2NhYgpkb2NzIDwtIHRlbXAkZG9jdW1lbnRzCmBgYAoKIyMjIyBTdGVtbWluZyBUaWR5IFRleHQKCk5vdGljZSB0aGF0IHRoZSBgdGV4dFByb2Nlc3NvcmAgc3RlbSBhcmd1bWVudCB3ZSB1c2VkIGFib3ZlIGlzIHNldCB0bwpgVFJVRWAgYnkgZGVmYXVsdC4gV2UgaGF2ZW4ndCBpbnRyb2R1Y2VkIHdvcmQgc3RlbW1pbmcgYXQgdGhpcyBwb2ludApiZWNhdXNlIHRoZXJlIGlzIHNvbWUgZGViYXRlIGFib3V0IHRoZSBhY3R1YWwgdmFsdWUgb2YgdGhpcyBwcm9jZXNzLgpXaGlsZSB3b3JkcyBsaWtlICJzdHVkZW50cyIgYW5kICJzdHVkZW50IiBtaWdodCBtYWtlIHNlbnNlIHRvIGNvbGxhcHNlCmludG8gdGhlaXIgYmFzZSB3b3JkIGFuZCBhY3R1YWxseSBtYWtlIGFuYWx5c2VzIGFuZCB2aXN1YWxpemF0aW9ucyBtb3JlCmNvbmNpc2UgYW5kIGVhc2llciB0byBpbnRlcnByZXQuIFtIdml0ZmVsZHQgYW5kIFNpbGdlCigyMDIxKV0oaHR0cHM6Ly9zbWx0YXIuY29tL3N0ZW1taW5nLmh0bWwpIG5vdGUsIGhvd2V2ZXIsIHRoYXQgd29yZHMgbGlrZQp0aGUgZm9sbG93aW5nIGhhdmUgZHJhbWF0aWMgZGlmZmVyZW5jZXMgaW4gbWVhbmluZywgc2VtYW50aWNzLCBhbmQgdXNlCmFuZCBjb3VsZCByZXN1bHQgaW4gcG9vciBtb2RlbHMgb3IgbWlzaW50ZXJwcmV0ZWQgcmVzdWx0czoKCi0gICBtZWFuaW5nIGFuZCBtZWFuCi0gICBsaWtlbHksIGxpa2UsIGxpa2luZwotICAgdW5pdmVyc2l0eSBhbmQgdW5pdmVyc2UKClRoZSBmaXJzdCB3b3JkIHBhaXIgaXMgcGFydGljdWxhcmx5IHJlbGV2YW50IHRvIGRpc2N1c3Npb24gcG9zdHMgZnJvbQpvdXIgVGVhY2hpbmcgU3RhdGlzdGljcyBjb3Vyc2UgZGF0YS4gSW4gYWRkaXRpb24sIGNvbGxhcHNpbmcgd29yZHMgbGlrZQoidGVhY2hlcnMiIGFuZCAidGVhY2hpbmciIGNvdWxkIGRyYW1hdGljYWxseSBhbHRlciB0aGUgcmVzdWx0cyBmcm9tIGEKdG9waWMgbW9kZWwuCgpbU2Nob2ZpZWxkIGFuZCBNaW1ubwooMjAxNildKGh0dHBzOi8vZGlyZWN0Lm1pdC5lZHUvdGFjbC9hcnRpY2xlL2RvaS8xMC4xMTYyL3RhY2xfYV8wMDA5OS80MzM3MC9Db21wYXJpbmctQXBwbGVzLXRvLUFwcGxlLVRoZS1FZmZlY3RzLW9mLVN0ZW1tZXJzKQpzcGVjaWZpY2FsbHksCgo+ICpEZXNwaXRlIHRoZWlyIGZyZXF1ZW50IHVzZSBpbiB0b3BpYyBtb2RlbGluZywgd2UgZmluZCB0aGF0IHN0ZW1tZXJzCj4gcHJvZHVjZSBubyBtZWFuaW5nZnVsIGltcHJvdmVtZW50IGluIGxpa2VsaWhvb2QgYW5kIGNvaGVyZW5jZSBhbmQgaW4KPiBmYWN0IGNhbiBkZWdyYWRlIHRvcGljIHN0YWJpbGl0eS4qIEZvciBub3csIHdlIHdpbGwgbGVhdmUgYXMgaXMgdGhlCj4gYGZvcnVtc19kdG1gIHdlIGNyZWF0ZWQgZWFybGllciB3aXRoIHdvcmRzIHVuc3RlbW1lZCwgYnV0IHdoYXQgaWYgd2UKPiB3YW50ZWQgdG8gc3RlbSB3b3JkcyBpbiBhICJ0aWR5IiB3YXk/CgpTaW5jZSB0aGUgYHVubmVzdF90b2tlbnMoKWAgZnVuY3Rpb24gZG9lcyBub3QgKGludGVudGlvbmFsbHkgSSBiZWxpZXZlKQppbmNsdWRlIGEgc3RlbW1pbmcgZnVuY3Rpb24sIG9uZSBhcHByb2FjaCB3b3VsZCBiZSB0byB1c2UgdGhlCmB3b3JkU3RlbSgpYCBmdW5jdGlvbiBmcm9tIHRoZSBgc25vd2JhbGxDYCBwYWNrYWdlIHRvIGVpdGhlciByZXBsYWNlIG91cgpgd29yZHNgIGNvbHVtbiB3aXRoIGEgd29yZCBzdGVtcyBvciBjcmVhdGUgYSBuZXcgdmFyaWFibGUgY2FsbGVkIGBzdGVtYAp3aXRoIG91ciBzdGVtbWVkIHdvcmRzLiBMZXQncyBkbyB0aGUgbGF0dGVyIGFuZCB0YWtlIGEgbG9vayBhdCB0aGUKb3JpZ2luYWwgd29yZHMgYW5kIHRoZSBzdGVtIHRoYXQgd2FzIHByb2R1Y2VkOgoKYGBge3Igd29yZFN0ZW19CnN0ZW1tZWRfZm9ydW1zIDwtIHRzX2ZvcnVtX2RhdGEgJT4lCiAgdW5uZXN0X3Rva2VucyhvdXRwdXQgPSB3b3JkLCBpbnB1dCA9IHBvc3RfY29udGVudCkgJT4lCiAgYW50aV9qb2luKHN0b3Bfd29yZHMsIGJ5ID0gIndvcmQiKSAlPiUKICBtdXRhdGUoc3RlbSA9IHdvcmRTdGVtKHdvcmQpKQpzdGVtbWVkX2ZvcnVtcwpgYGAKCllvdSBjYW4gc2VlIHRoYXQgd29yZHMgbGlrZSAiYWN0aXZpdHkiIGFuZCAiYWN0aXZpdGllcyIgdGhhdCBvY2N1cgpmcmVxdWVudGx5IGluIG91ciBkaXNjdXNzaW9ucyBoYXZlIGJlZW4gcmVkdWNlZCB0byB0aGUgd29yZCBzdGVtCiJhY3RpdiIuIElmIHlvdSBhcmUgaW50ZXJlc3RlZCBpbiBsZWFybmluZyBvdGhlciBhcHByb2FjaGVzIGZvciB3b3JkCnN0ZW1taW5nIGluIFIsIGFzIHdlbGwgYXMgcmVhZGluZyBhIG1vcmUgaW4gZGVwdGggZGVzY3JpcHRpb24gb2YgdGhlCnN0ZW1taW5nIHByb2Nlc3MsIEkgaGlnaGx5IHJlY29tbWVuZCB0aGUgW0NoYXB0ZXIgNApTdGVtbWluZ10oaHR0cHM6Ly9zbWx0YXIuY29tL3N0ZW1taW5nLmh0bWwpIGZyb20gSHZpdGZlbGR0IGFuZCBTaWxnZQooMjAyMSkgYm9vaywgKlN1cGVydmlzZWQgTWFjaGluZSBMZWFybmluZyBmb3IgVGV4dCBBbmFseXNpcyBpbiBSKi4KCiMjIyMjIOKchSBDb21wcmVoZW5zaW9uIENoZWNrCgpDb21wbGV0ZSB0aGUgZm9sbG93aW5nIGNvZGUgdXNpbmcgd2hhdCB3ZSBsZWFybmVkIGluIHRoZSBzZWN0aW9uIG9uCltDcmVhdGluZyBhIERvY3VtZW50IFRlcm0gTWF0cml4XSBhbmQgYW5zd2VyIHRoZSBmb2xsb3dpbmcgcXVlc3Rpb25zOgoKMS4gIEhvdyBtYW55IGZld2VyIHRlcm1zIGFyZSBpbiBvdXIgc3RlbW1lZCBkb2N1bWVudCB0ZXJtIG1hdHJpeD8KICAgICozLDYwNioKMi4gIERpZCBzdGVtbWluZyB3b3JkcyBzaWduaWZpY2FudGx5IHJlZHVjZSB0aGUgc3BhcnNpdHkgb2YgdGhlIG5ldHdvcms/CiAgICAqTm8qCgoqKkhpbnQ6KiogTWFrZSBzdXJlIHlvdXIgY29kZSBpbmNsdWRlcyBzdGVtIGNvdW50cyByYXRoZXIgdGhhbiB3b3JkCmNvdW50cy4KCmBgYHtyIHN0ZW0tcHJhY3RpY2UsIGV2YWw9RkFMU0V9CnN0ZW1tZWRfZHRtIDwtIHRzX2ZvcnVtX2RhdGEgJT4lCiAgdW5uZXN0X3Rva2VucyhvdXRwdXQgPSB3b3JkLCBpbnB1dCA9IHBvc3RfY29udGVudCkgJT4lCiAgYW50aV9qb2luKHN0b3Bfd29yZHMsIGJ5ID0gIndvcmQiKSAlPiUKICBtdXRhdGUoc3RlbSA9IHdvcmRTdGVtKHdvcmQpKSAlPiUKICBjb3VudChwb3N0X2lkLCBzdGVtLCBzb3J0ID0gVFJVRSkgJT4lCiAgY2FzdF9kdG0ocG9zdF9pZCwgc3RlbSwgbikKICAKc3RlbW1lZF9kdG0KYGBgCgpgYGB7ciBzdGVtLWNvdW50cywgZWNobz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0Kc3RlbW1lZF9kdG0gPC0gdHNfZm9ydW1fZGF0YSAlPiUKICB1bm5lc3RfdG9rZW5zKG91dHB1dCA9IHdvcmQsIGlucHV0ID0gcG9zdF9jb250ZW50KSAlPiUKICBhbnRpX2pvaW4oc3RvcF93b3JkcywgYnkgPSAid29yZCIpICU+JQogIG11dGF0ZShzdGVtID0gd29yZFN0ZW0od29yZCkpICU+JQogIGNvdW50KHBvc3RfaWQsIHN0ZW0sIHNvcnQgPSBUUlVFKSAlPiUKICBjYXN0X2R0bShwb3N0X2lkLCBzdGVtLCBuKQpzdGVtbWVkX2R0bQpmb3J1bXNfZHRtCnN0ZW1fY291bnRzIDwtIHRzX2ZvcnVtX2RhdGEgJT4lCiAgdW5uZXN0X3Rva2VucyhvdXRwdXQgPSB3b3JkLCBpbnB1dCA9IHBvc3RfY29udGVudCkgJT4lCiAgYW50aV9qb2luKHN0b3Bfd29yZHMsIGJ5ID0gIndvcmQiKSAlPiUKICBtdXRhdGUoc3RlbSA9IHdvcmRTdGVtKHdvcmQpKSAlPiUKICBjb3VudChzdGVtLCBzb3J0ID0gVFJVRSkKc3RlbV9jb3VudHMKYGBgCgojIyAzLiBNT0RFTAoKVGhpcyB1bml0IHByb3ZpZGVzIG91ciBmaXJzdCBvcHBvcnR1bml0eSBmb3IgbW9kZWxpbmcgYSB0ZXh0IGFzIGRhdGEuIEluCnZlcnkgc2ltcGxlIHRlcm1zLCBtb2RlbGluZyBpbnZvbHZlcyBkZXZlbG9waW5nIGEgbWF0aGVtYXRpY2FsIHN1bW1hcnkKb2YgYSBkYXRhc2V0LiBUaGVzZSBzdW1tYXJpZXMgY2FuIGhlbHAgdXMgZnVydGhlciBleHBsb3JlIHRyZW5kcyBhbmQKcGF0dGVybnMgaW4gb3VyIGRhdGEuCgpJbiB0aGVpciBib29rLCAqTGVhcm5pbmcgQW5hbHl0aWNzIEdvZXMgdG8gU2Nob29sLCogS3J1bW0gYW5kIE1lYW5zCigyMDE4KSBkZXNjcmliZSB0d28gZ2VuZXJhbCB0eXBlcyBvZiBtb2RlbGluZyBhcHByb2FjaGVzIHVzZWQgaW4gdGhlCkRhdGEtSW50ZW5zaXZlIFJlc2VhcmNoIHdvcmtmbG93OiB1bnN1cGVydmlzZWQgYW5kIHN1cGVydmlzZWQgbWFjaGluZQpsZWFybmluZy4gSW4gZGlzdGluZ3Vpc2hpbmcgYmV0d2VlbiB0aGUgdHdvLCB0aGV5IG5vdGU6Cgo+ICpVbnN1cGVydmlzZWQgbGVhcm5pbmcgYWxnb3JpdGhtcyBjYW4gYmUgdXNlZCB0byB1bmRlcnN0YW5kIHRoZQo+IHN0cnVjdHVyZSBvZiBvbmUncyBkYXRhc2V0LiBTdXBlcnZpc2VkIG1vZGVscywgb24gdGhlIG90aGVyIGhhbmQsIGhlbHAKPiB0byBxdWFudGlmeSByZWxhdGlvbnNoaXBzIGJldHdlZW4gZmVhdHVyZXMgYW5kIGEga25vd24gb3V0Y29tZS4gS25vd24KPiBvdXRjb21lcyBhcmUgYWxzbyBjb21tb25seSByZWZlcnJlZCB0byBhcyBsYWJlbHMgb3IgZGVwZW5kZW50Cj4gdmFyaWFibGVzLiogSW4gU2VjdGlvbiAzIHdlIGZvY3VzIG9uIFRvcGljIE1vZGVsaW5nLCBhbiB1bnN1cGVydmlzZWQKPiBsZWFybmluZyBhcHByb2FjaCB0byBhdXRvbWF0aWNhbGx5IGlkZW50aWZ5IHRvcGljcyBpbiBhIGNvbGxlY3Rpb24gb2YKPiBkb2N1bWVudHMuIEluIGZhY3QsIHdlJ2xsIGV4cGxvcmUgdHdvIGRpZmZlcmVudCBhcHByb2FjaGVzIHRvIHRvcGljCj4gbW9kZWxpbmcsIGFzIHdlbGwgYXMgc3RyYXRlZ2llcyBmb3IgaWRlbnRpZnlpbmcgdGhlICJyaWdodCIgbnVtYmVyIG9mCj4gdG9waWNzOgoKYS4gICoqRml0dGluZyBhIFRvcGljIE1vZGVsaW5nIHdpdGggTERBKiouIEluIHRoaXMgc2VjdGlvbiB3ZSBsZWFybiB0bwogICAgdXNlIHRoZSBgdG9waWNtb2RlbHNgIHBhY2thZ2UgYW5kIGFzc29jaWF0ZWQgYExEQSgpYCBmdW5jdGlvbiBmb3IKICAgIHVuc3VwZXJ2aXNlZCBjbGFzc2lmaWNhdGlvbiBvZiBvdXIgZm9ydW0gZGlzY3Vzc2lvbnMgdG8gZmluZCBuYXR1cmFsCiAgICBncm91cGluZ3Mgb2Ygd29yZHMsIG9yIHRvcGljcy4KYi4gICoqRml0dGluZyBhIFN0cnVjdHVyYWwgVG9waWMgTW9kZWwqKi4gV2UgdGhlbiBleHBsb3JlIHRoZSB1c2Ugb2YgdGhlCiAgICBgc3RtYCBwYWNrYWdlIGFuZCBgc3RtKClgIGZ1bmN0aW9uIHRvIGZpdCBvdXIgbW9kZWwgYW5kIHVzZXMKICAgIG1ldGFkYXRhIGFib3V0IGRvY3VtZW50cyB0byBpbXByb3ZlIHRoZSBhc3NpZ25tZW50IG9mIHdvcmRzIHRvCiAgICAidG9waWNzIiBpbiBhIGNvcnB1cy4KYy4gICoqQ2hvb3NpbmcgSy4qKiBGaW5hbGx5LCB3ZSB3cmFwIHVwIFNlY3Rpb24gMyBieSBsZWFybmluZyBhYm91dAogICAgZGlhZ25vc3RpYyBwcm9wZXJ0aWVzIGxpa2UgZXhjbHVzaXZpdHksIHNlbWFudGljIGNvaGVyZW5jZSwgYW5kCiAgICBoZWxkb3V0IGxpa2VsaWhvb2QgZm9yIGhlbHBpbmcgdG8gc2VsZWN0IGFuIGFwcHJvcHJpYXRlIG51bWJlciBvZgogICAgdG9waWNzLgoKIyMjIDNhLiBGaXR0aW5nIGEgVG9waWMgTW9kZWxpbmcgd2l0aCBMREEKCkJlZm9yZSBydW5uaW5nIG91ciBmaXJzdCB0b3BpYyBtb2RlbCB1c2luZyB0aGUgYExEQSgpYCBmdW5jdGlvbiwgbGV0J3MKcXVpY2sgcmVjYXAgZnJvbSBvdXIgcmVhZGluZ3Mgc29tZSBiYXNpYyBwcmluY2lwbGVzIGJlaGluZCBMYXRlbnQKRGlyaWNobGV0IGFsbG9jYXRpb24gYW5kIHdoeSBMREEgaXMgcHJlZmVycmVkIG92ZXIgb3RoZXIgYXV0b21hdGljCmNsYXNzaWZpY2F0aW9uIG9yIGNsdXN0ZXJpbmcgYXBwcm9hY2hlcy4KClVubGlrZSBzaW1wbGUgZm9ybXMgb2YgY2x1c3RlciBhbmFseXNpcyBzdWNoIGFzIGstbWVhbnMgY2x1c3RlcmluZywgTERBCmlzIGEgKioibWl4dHVyZSIgbW9kZWwqKiwgd2hpY2ggaW4gb3VyIGNvbnRleHQgbWVhbnMgdGhhdDoKCjEuICAqKkV2ZXJ5IFtkb2N1bWVudF17LnVsfSBjb250YWlucyBhIG1peHR1cmUgb2YgdG9waWNzLioqIFVubGlrZQogICAgYWxnb3JpdGhtcyBsaWtlIGstbWVhbnMsIExEQSB0cmVhdHMgZWFjaCBkb2N1bWVudCBhcyBhIG1peHR1cmUgb2YKICAgIHRvcGljcywgd2hpY2ggYWxsb3dzIGRvY3VtZW50cyB0byAib3ZlcmxhcCIgZWFjaCBvdGhlciBpbiB0ZXJtcyBvZgogICAgY29udGVudCwgcmF0aGVyIHRoYW4gYmVpbmcgc2VwYXJhdGVkIGludG8gZGlzY3JldGUgZ3JvdXBzLiBTbyBpbgogICAgcHJhY3RpY2UsIHRoaXMgbWVhbnMgdGhhdCBhIGRpc2N1c3Npb24gZm9ydW0gcG9zdCBjb3VsZCBoYXZlIGFuCiAgICBlc3RpbWF0ZWQgdG9waWMgcHJvcG9ydGlvbiBvZiA3MCUgZm9yIFRvcGljIDEgKGUuZy4gYmUgbW9zdGx5IGFib3V0CiAgICBhIFRvcGljIDEpLCBidXQgYWxzbyBiZSBwYXJ0bHkgYWJvdXQgVG9waWMgMi4KMi4gICoqRXZlcnkgW3RvcGljXXsudWx9IGNvbnRhaW5zIGEgbWl4dHVyZSBvZiB3b3Jkcy4qKiBGb3IgZXhhbXBsZSwgaWYKICAgIHdlIHNwZWNpZmllZCBpbiBvdXIgTERBIG1vZGVsIGp1c3QgMiB0b3BpY3MgZm9yIG91ciBkaXNjdXNzaW9uCiAgICBwb3N0cywgd2UgbWlnaHQgZmluZCB0aGF0IG9uZSB0b3BpYyBzZWVtcyB0byBiZSBhYm91dCBwZWRhZ29neSB3aGlsZQogICAgYW5vdGhlciBpcyBhYm91dCBsZWFybmluZy4gVGhlIG1vc3QgY29tbW9uIHdvcmRzIGluIHRoZSBwZWRhZ29neQogICAgdG9waWMgbWlnaHQgYmUgInRlYWNoZXIiLCAic3RyYXRlZ2llcyIsIGFuZCAiaW5zdHJ1Y3Rpb24iLCB3aGlsZSB0aGUKICAgIGxlYXJuaW5nIHRvcGljIG1heSBiZSBtYWRlIHVwIG9mIHdvcmRzIGxpa2UgInVuZGVyc3RhbmRpbmciIGFuZAogICAgInN0dWRlbnRzIi4gSG93ZXZlciwgd29yZHMgY2FuIGJlIHNoYXJlZCBiZXR3ZWVuIHRvcGljcyBhbmQgd29yZHMKICAgIGxpa2UgInN0YXRpc3RpY3MiIG9yICJhc3Nlc3NtZW50IiBtaWdodCBhcHBlYXIgaW4gYm90aCBlcXVhbGx5LgoKU2ltaWxhciB0byBrLW1lYW5zIGFuZCBvdGhlciBzaW1wbGUgY2x1c3RlcmluZyBhcHByb2FjaGVzLCBob3dldmVyLCBMREEKZG9lcyByZXF1aXJlIHVzIHRvIHNwZWNpZnkgYSB2YWx1ZSBvZiAqayogZm9yIHRoZSBudW1iZXIgb2YgdG9waWNzIGluCm91ciBjb3JwdXMuIFNlbGVjdGluZyAqayogaXMgbm8gdHJpdmlhbCBtYXR0ZXIgYW5kIGNhbiBncmVhdGx5IGltcGFjdAp5b3VyIHJlc3VsdHMuCgpTaW5jZSB3ZSBkb24ndCBoYXZlIGEgaGF2ZSBzdHJvbmcgcmF0aW9uYWxlIGFib3V0IHRoZSBudW1iZXIgb2YgdG9waWNzCnRoYXQgbWlnaHQgZXhpc3QgaW4gZGlzY3Vzc2lvbiBmb3J1bXMsIGxldCdzIHVzZSB0aGUgYG5fZGlzdGluY3QoKWAKZnVuY3Rpb24gZnJvbSB0aGUgYGRwbHlyYCBwYWNrYWdlIHRvIGZpbmQgdGhlIG51bWJlciBvZiB1bmlxdWUgZm9ydW0KbmFtZXMgaW4gb3VyIGNvdXJzZSBkYXRhIGFuZCBydW4gd2l0aCB0aGF0OgoKYGBge3Igbi1kaXN0aW5jdH0Kbl9kaXN0aW5jdCh0c19mb3J1bV9kYXRhJGZvcnVtX25hbWUpCmBgYAoKU2luY2UgaXQgbG9va3MgbGlrZSB0aGVyZSBhcmUgMjAgZGlzdGluY3QgZGlzY3Vzc2lvbiBmb3J1bXMsIHdlJ2xsIHVzZQp0aGF0IGFzIG91ciB2YWx1ZSBmb3IgdGhlIGBrID1gIGFyZ3VtZW50IG9mIHRoZSBgTERBKClgLiBCZSBwYXRpZW50CndoaWxlIHRoaXMgcnVucywgc2luY2UgdGhlIGRlZmF1bHQgc2V0dGluZyBvZiBpcyB0byBwZXJmb3JtIGEgbGFyZ2UKbnVtYmVyIG9mIGl0ZXJhdGlvbnMuCgpgYGB7ciBMREF9Cm5fZGlzdGluY3QodHNfZm9ydW1fZGF0YSRmb3J1bV9uYW1lKQpmb3J1bXNfbGRhIDwtIExEQShmb3J1bXNfZHRtLCAKICAgICAgICAgICAgICAgICAgayA9IDIwLCAKICAgICAgICAgICAgICAgICAgY29udHJvbCA9IGxpc3Qoc2VlZCA9IDU4OCkKICAgICAgICAgICAgICAgICAgKQpmb3J1bXNfbGRhCmBgYAoKTm90ZSB0aGF0IHdlIHVzZWQgdGhlIGBjb250cm9sID1gIGFyZ3VtZW50IHRvIHBhc3MgYSByYW5kb20gbnVtYmVyCihgNTg4YCkgdG8gc2VlZCB0aGUgYXNzaWdubWVudCBvZiB0b3BpY3MgdG8gZWFjaCB3b3JkIGluIG91ciBjb3JwdXMuClNpbmNlIExEQSBpcyBhIFtzdG9jaGFzdGljCmFsZ29yaXRobV0oaHR0cHM6Ly9tYWNoaW5lbGVhcm5pbmdtYXN0ZXJ5LmNvbS9zdG9jaGFzdGljLWluLW1hY2hpbmUtbGVhcm5pbmcvKQp0aGF0IGNvdWxkIGhhdmUgZGlmZmVyZW50IHJlc3VsdHMgZGVwZW5kaW5nIG9uIHdoZXJlIHRoZSBhbGdvcml0aG0Kc3RhcnRzLCB3ZSBzcGVjaWZpZWQgYSBgc2VlZGAgZm9yIHJlcHJvZHVjaWJpbGl0eSBzbyB3ZSdyZSBhbGwgc2VlaW5nCnRoZSBzYW1lIHJlc3VsdHMgZXZlcnkgdGltZSB3ZSBzcGVjaWZ5IHRoZSBzYW1lIG51bWJlciBvZiB0b3BpY3MuCgpBbmQgdHlpbmcgYmFjayB0byBvdXIgd29yayBpbiBVbml0IDEsIEJhaWwgKDIwMjApIG5vdGVzIHRoYXQgdG9waWMKYXNzaWdubWVudHMgZm9yIGVhY2ggd29yZCBhcmUgdXBkYXRlZCBpbiBhbiBpdGVyYXRpdmUgZmFzaGlvbiBhbmQgdGhhdApMREEgZW1wbG95cyB0aGUgVGVybSBGcmVxdWVuY3ktSW52ZXJzZSBEb2N1bWVudCBGcmVxdWVuY3kgKFRGLUlERikKbWV0cmljIHRvIGFzc2lnbiBwcm9iYWJpbGl0aWVzLgoKIyMjIDNiLiBGaXR0aW5nIGEgU3RydWN0dXJhbCBUb3BpYyBNb2RlbAoKQmFpbCBub3RlcyB0aGF0IExEQSwgd2hpbGUgcGVyaGFwcyB0aGUgbW9zdCBjb21tb24gYXBwcm9hY2ggdG8gdG9waWMKbW9kZWxpbmcsIGlzIGp1c3Qgb25lIG9mIG1hbnkgZGlmZmVyZW50IHR5cGVzLCBpbmNsdWRpbmcgRHluYW1pYyBUb3BpYwpNb2RlbHMsIENvcnJlbGF0ZWQgVG9waWMgTW9kZWxzLCBIaWVyYXJjaGljYWwgVG9waWMgTW9kZWxzLCBhbmQgbW9yZQpyZWNlbnRseSwgU3RydWN0dXJhbCBUb3BpYyBNb2RlbGluZyAoU1RNKS4gSGUgYXJndWVzIHRoYXQgb25lIHJlYXNvbiBTVE0KaGFzIHJpc2luZyBpbiBwb3B1bGFyaXR5IGFuZCB1c2UgaXMgdGhhdCBpdCBlbXBsb3lzIG1ldGEgZGF0YSBhYm91dApkb2N1bWVudHMgdG8gaW1wcm92ZSB0aGUgYXNzaWdubWVudCBvZiB3b3JkcyB0byB0b3BpY3MgaW4gYSBjb3JwdXMgYW5kCnRoYXQgY2FuIGJlIHVzZWQgdG8gZXhhbWluZSByZWxhdGlvbnNoaXBzIGJldHdlZW4gY292YXJpYXRlcyBhbmQKZG9jdW1lbnRzLgoKQWxzbywgc2luY2UgSnVsaWEgU2lsZ2UgaGFzIGluZGljYXRlZCB0aGF0IFNUTSBpcywgIm15IGN1cnJlbnQgZmF2b3JpdGUKaW1wbGVtZW50YXRpb24gb2YgdG9waWMgbW9kZWxpbmcgaW4gUiIgYW5kIGhhcyBidWlsdCBzdXBwb3J0cyBpbiB0aGUKYHRpZHl0ZXh0YCBwYWNrYWdlIGZvciBidWlsZGluZyBzdHJ1Y3R1cmFsIHRvcGljIG1vZGVscywgdGhpcyBwYWNrYWdlCmRlZmluaXRlbHkgaXMgd29ydGggZGlzY3Vzc2luZyBpbiB0aGlzIHdhbGt0aHJvdWdoLiBJIGFsc28gaGlnaGx5CnJlY29tbWVuZCBoZXIgb3duIHdhbGt0aHJvdWdoIG9mIHRoZSBgc3RtYCBwYWNrYWdlOiBbVGhlIGdhbWUgaXMgYWZvb3QhClRvcGljIG1vZGVsaW5nIG9mIFNoZXJsb2NrIEhvbG1lcwpzdG9yaWVzXShodHRwczovL2p1bGlhc2lsZ2UuY29tL2Jsb2cvc2hlcmxvY2staG9sbWVzLXN0bS8pIGFzIHdlbGwgYXMKaGVyIGZvbGxvdyB1cCBwb3N0LCBbVHJhaW5pbmcsIGV2YWx1YXRpbmcsIGFuZCBpbnRlcnByZXRpbmcgdG9waWMKbW9kZWxzXShodHRwczovL2p1bGlhc2lsZ2UuY29tL2Jsb2cvZXZhbHVhdGluZy1zdG0vKS4KCiMjIyMgVGhlIGBzdG1gIFBhY2thZ2UKCkFzIHdlJ3ZlIHNlZW4gYWJvdmUsIFNUTSBwcm9kdWNlZCBhbiB1bnVzdWFsIGB0ZW1wYCB0ZXh0UHJvY2Vzc29yIG91dHB1dAp0aGF0IGlzIHVuaXF1ZSB0byB0aGUgYHN0bWAgcGFja2FnZS4gQW5kIGFzIHlvdSd2ZSBwcm9iYWJseSBhbHJlYWR5Cmd1ZXNzZWQsIHRoZSBgc3RtKClgIGZ1bmN0aW9uIGZvciBmaXR0aW5nIGEgc3RydWN0dXJhbCB0b3BpYyBtb2RlbCBkb2VzCm5vdCB0YWtlIGEgZmFpcmx5IHN0YW5kYXJkIGRvY3VtZW50IHRlcm0gbWF0cml4IGxpa2UgdGhlIGBMREEoKWAKZnVuY3Rpb24uCgpCZWZvcmUgd2UgZml0IG91ciBtb2RlbCwgd2UnbGwgaGF2ZSB0byBleHRyYWN0IHRoZSBlbGVtZW50cyBmcm9tIHRoZQpgdGVtcGAgb2JqZWN0IGNyZWF0ZWQgYWZ0ZXIgd2UgcHJvY2Vzc2VkIG91ciB0ZXh0LiBTcGVjaWZpY2FsbHksIHRoZQpgc3RtKClgIGZ1bmN0aW9uIGV4cGVjdHMgdGhlIGZvbGxvd2luZyBhcmd1bWVudHM6CgotICAgYGRvY3VtZW50cyA9YCB0aGUgZG9jdW1lbnQgdGVybSBtYXRyaXggdG8gYmUgbW9kZWxlZCBpbiB0aGUgbmF0aXZlCiAgICBzdG0gZm9ybWF0Ci0gICBgZGF0YSA9YCBhbiBvcHRpb25hbCBkYXRhIGZyYW1lIGNvbnRhaW5pbmcgbWV0YSBkYXRhIGZvciB0aGUKICAgIHByZXZhbGVuY2UgYW5kL29yIGNvbnRlbnQgY292YXJpYXRlcyB0byBpbmNsdWRlIGluIHRoZSBtb2RlbAotICAgYHZvY2FiID1gIGEgY2hhcmFjdGVyIHZlY3RvciBzcGVjaWZ5aW5nIHRoZSB3b3JkcyBpbiB0aGUgY29ycHVzIGluCiAgICB0aGUgb3JkZXIgb2YgdGhlIHZvY2FiIGluZGljZXMgaW4gZG9jdW1lbnRzLgoKTGV0J3MgZ28gYWhlYWQgYW5kIGV4dHJhY3QgdGhlc2UgZWxlbWVudHM6CgpgYGB7ciBzdG0tZG9jc30KZG9jcyA8LSB0ZW1wJGRvY3VtZW50cyAKbWV0YSA8LSB0ZW1wJG1ldGEgCnZvY2FiIDwtIHRlbXAkdm9jYWIgCmBgYAoKQW5kIG5vdyB1c2UgdGhlc2UgZWxlbWVudHMgdG8gZml0IHRoZSBtb2RlbCB1c2luZyB0aGUgc2FtZSBudW1iZXIgb2YKdG9waWNzIGZvciAqSyogdGhhdCB3ZSBzcGVjaWZpZWQgZm9yIG91ciBMREEgdG9waWMgbW9kZWwuIExldCdzIGFsc28KdGFrZSBhZHZhbnRhZ2Ugb2YgdGhlIGZhY3QgdGhhdCB3ZSBjYW4gaW5jbHVkZSB0aGUgYGNvdXJzZV9pZGAgYW5kCmBmb3J1bV9pZGAgY292YXJpYXRlcyBpbiB0aGUgYHByZXZlYWxlbmNlID1gIGFyZ3VtZW50IHRvIGhlbHAgaW1wcm92ZSwKaW4gdGhlb3J5LCBvdXIgbW9kZWwgZml0OgoKYGBge3Igc3RtfQpmb3J1bXNfc3RtIDwtIHN0bShkb2N1bWVudHM9ZG9jcywgCiAgICAgICAgIGRhdGE9bWV0YSwKICAgICAgICAgdm9jYWI9dm9jYWIsIAogICAgICAgICBwcmV2YWxlbmNlID1+IGNvdXJzZV9pZCArIGZvcnVtX2lkLAogICAgICAgICBLPTIwLAogICAgICAgICBtYXguZW0uaXRzPTI1LAogICAgICAgICB2ZXJib3NlID0gRkFMU0UpCmZvcnVtc19zdG0KYGBgCgpBcyBub3RlZCBlYXJsaWVyLCB0aGUgYHN0bWAgcGFja2FnZSBoYXMgYSBudW1iZXIgb2YgaGFuZHkgZmVhdHVyZXMuIE9uZQpvZiB0aGVzZSBpcyB0aGUgYHBsb3QuU1RNKClgIGZ1bmN0aW9uIGZvciB2aWV3aW5nIHRoZSBtb3N0IHByb2JhYmxlCndvcmRzIGFzc2lnbmVkIHRvIGVhY2ggdG9waWMuCgpCeSBkZWZhdWx0LCBpdCBvbmx5IHNob3dzIHRoZSBmaXJzdCAzIHRlcm1zIHNvIGxldCdzIGNoYW5nZSB0aGF0IHRvIDUgdG8KaGVscCB3aXRoIGludGVycHJldGF0aW9uOgoKYGBge3IgcGxvdC1zdG19CnBsb3QuU1RNKGZvcnVtc19zdG0sIG4gPSA1KQpgYGAKCk5vdGUgdGhhdCB5b3UgY2FuIGFsc28ganVzdCB1c2UgYHBsb3QoKWAgYXMgd2VsbDoKCmBgYHtyIHBsb3R9CnBsb3QoZm9ydW1zX3N0bSwgbiA9IDUpCmBgYAoKIyMjIyMg4pyFIENvbXByZWhlbnNpb24gQ2hlY2sKCkZpdCBhIG1vZGVsIGZvciBib3RoIExEQSBhbmQgU1RNIHVzaW5nIGRpZmZlcmVudCB2YWx1ZXMgZm9yIEsgYW5kIGFuc3dlcgp0aGUgZm9sbG93aW5nIHF1ZXN0aW9uczoKCjEuICBXaGF0IHRvcGljcyBhcHBlYXIgdG8gYmUgc2ltaWxhciB0byB0aG9zZSB1c2luZyAyMCB0b3BpY3MgZm9yIEs/CiAgICAqRm9yIHRoZSBMREEgbW9kZWwsIEkgdXNlZCBhIEsgdmFsdWUgb2YgMTAgYW5kIHRvcGljcyAxLCAyLCA1LCBhbmQgNwogICAgbG9vayBzaW1pbGFyLiBGb3IgU1RNLCB1c2luZyBhIEsgdmFsdWUgb2YgMTAgeWllbGRlZCBzaW1pbGFyaXRpZXMKICAgIG9ubHkgaW4gdG9waWNzIDcgYW5kIDguKgoyLiAgS25vd2luZyB0aGF0IHlvdSBkb24ndCBoYXZlIGFzIG11Y2ggY29udGV4dCBhcyB0aGUgcmVzZWFyY2hlcnMgb2YKICAgIHRoaXMgc3R1ZHkgZG8sIGhvdyBtaWdodCB5b3UgaW50ZXJwcmV0IG9uZSBvZiB0aGVzZSBsYXRlbnQgdG9waWNzIG9yCiAgICB0aGVtZXMgdXNpbmcgdGhlIGtleSB0ZXJtcyBhc3NpZ25lZD8gKkZvciB0aGUgTERBIHRvcGljcywgVG9waWMgNQogICAgY291bGQgYmUgaW50ZXJwcmV0ZWQgYXMgZGF0YSBtYW5pcHVsYXRpb24gdGFza3MuIEZvciB0aGUgU1RNIHJ1biwKICAgIFRvcGljIDggY291bGQgYmUgaW50ZXJwcmV0ZWQgYXMgc3R1ZGVudCB0ZWNobm9sb2d5IHVzZS4qCjMuICBXaGF0IHRvcGljIGVtZXJnZWQgdGhhdCBzZWVtIGRyYW1hdGljYWxseSBkaWZmZXJlbnQgYW5kIGhvdyBtaWdodAogICAgeW91IGludGVycHJldCB0aGlzIHRvcGljPyAqRm9yIHRoZSBMREEgbW9kZWwsIHRvcGljIDEwIHdhcyB2ZXJ5CiAgICBkaWZmZXJlbnQuIEkgY291bGQgaW50ZXJwcmV0IHRvcGljIDEwIGFzIHRoZSBibGVuZGluZyBvZiB0d28KICAgIHBvc3NpYmxlIGFyZWFzOiBQcm9ibGVtcyB1bmRlcnN0YW5kaW5nIHN0YXRpc3RpY3MgYW5kIGdvb2QKICAgIGFzc2Vzc21lbnQgcmVzb3VyY2VzLiBGb3IgdGhlIFNUTSBydW5zLCBtYW55IHRhc2tzIHdlcmUgZXh0cmVtZWx5CiAgICBkaWZmZXJlbnQuKgoKIyMjIDNjLiBGaW5kaW5nICpLKgoKQXMgYWxsdWRlZCB0byBlYXJsaWVyLCBzZWxlY3RpbmcgdGhlIG51bWJlciBvZiB0b3BpY3MgZm9yIHlvdXIgbW9kZWwgaXMKYSBub24tdHJpdmlhbCBkZWNpc2lvbiBhbmQgY2FuIGRyYW1hdGljYWxseSBpbXBhY3QgeW91ciByZXN1bHRzLiBCYWlsCigyMDE4KSBub3RlcyB0aGF0Cgo+ICpUaGUgcmVzdWx0cyBvZiB0b3BpYyBtb2RlbHMgc2hvdWxkIG5vdCBiZSBvdmVyLWludGVycHJldGVkIHVubGVzcyB0aGUKPiByZXNlYXJjaGVyIGhhcyBzdHJvbmcgdGhlb3JldGljYWwgYXByaW9yaSBhYm91dCB0aGUgbnVtYmVyIG9mIHRvcGljcwo+IGluIGEgZ2l2ZW4gY29ycHVzLCBvciBpZiB0aGUgcmVzZWFyY2hlciBoYXMgY2FyZWZ1bGx5IHZhbGlkYXRlZCB0aGUKPiByZXN1bHRzIG9mIGEgdG9waWMgbW9kZWwgdXNpbmcgYm90aCB0aGUgcXVhbnRpdGF0aXZlIGFuZCBxdWFsaXRhdGl2ZQo+IHRlY2huaXF1ZXMgZGVzY3JpYmVkIGFib3ZlLiogVGhlcmUgYXJlIHNldmVyYWwgYXBwcm9hY2hlcyB0bwo+IGVzdGltYXRpbmcgYSB2YWx1ZSBmb3IgSyBhbmQgd2UnbGwgdGFrZSBhIHF1aWNrIGxvb2sgYXQgb25lIGZyb20gdGhlCj4gYGxkYXR1bmluZ2AgcGFja2FnZSBhbmQgb25lIGZyb20gb3VyIGBzdG1gIHBhY2thZ2UuCgojIyMjIFRoZSBGaW5kVG9waWNzTnVtYmVyIEZ1bmN0aW9uCgpUaGUgYGxkYXR1bmluZ2AgcGFja2FnZSBoYXMgZnVuY3Rpb25zIGZvciBib3RoIGNhbGN1bGF0aW5nIGFuZCBwbG90dGluZwpkaWZmZXJlbnQgbWV0cmljcyB0aGF0IGNhbiBiZSB1c2VkIHRvIGVzdGltYXRlIHRoZSBtb3N0IHByZWZlcmFibGUKbnVtYmVyIG9mIHRvcGljcyBmb3IgTERBIG1vZGVsLiBJdCBhbHNvIGNvbnZlbmllbnRseSB0YWtlcyB0aGUgc3RhbmRhcmQKZG9jdW1lbnQgdGVybSBtYXRyaXggb2JqZWN0IHRoYXQgd2UgY3JlYXRlZCBmcm9tIG91dCB0aWR5IHRleHQgYW5kIGhhcwp0aGUgYWRkZWQgYmVuZWZpdCBvZiBydW5uaW5nIGZhaXJseSBxdWlja2x5LCBlc3BlY2lhbGx5IGNvbXBhcmVkIHRvIHRoZQpmdW5jdGlvbiBmb3IgZmluZGluZyBLIGZyb20gdGhlIGBzdG1gIHBhY2thZ2UuCgpMZXQncyB1c2UgdGhlIGRlZmF1bHRzIHNwZWNpZmllZCBpbiB0aGUgYD9GaW5kVG9waWNOdW1iZXJgIGRvY3VtZW50YXRpb24KYW5kIG1vZGlmeSBzbGlnaHRseSBnZXQgbWV0cmljcyBmb3IgYSBzZXF1ZW5jZSBvZiB0b3BpY3MgZnJvbSAxMC03NQpjb3VudGluZyBieSA1IGFuZCBwbG90IHRoZSBvdXRwdXQgd2Ugc2F2ZWQgdXNpbmcgdGhlCmBGaW5kVG9waWNzTnVtYmVyX3Bsb3QoKWAgZnVuY3Rpb246CgpgYGB7ciBmaW5kLXRvcGljLCBldmFsPUZBTFNFfQprX21ldHJpY3MgPC0gRmluZFRvcGljc051bWJlcigKICBmb3J1bXNfZHRtLAogIHRvcGljcyA9IHNlcSgxMCwgNzUsIGJ5ID0gNSksCiAgbWV0cmljcyA9ICJHcmlmZml0aHMyMDA0IiwKICBtZXRob2QgPSAiR2liYnMiLAogIGNvbnRyb2wgPSBsaXN0KCksCiAgbWMuY29yZXMgPSBOQSwKICByZXR1cm5fbW9kZWxzID0gRkFMU0UsCiAgdmVyYm9zZSA9IEZBTFNFLAogIGxpYnBhdGggPSBOVUxMCikKRmluZFRvcGljc051bWJlcl9wbG90KGtfbWV0cmljcykKYGBgCgpOb3RlIHRoYXQgdGhlIGBGaW5kVG9waWNOdW1iZXJzKClgIGZ1bmN0aW9uIGNvbnRhaW5zIHRocmVlIGFkZGl0aW9uYWwKbWV0cmljcyBmb3IgY2FsY3VsYXRpbmcgbWV0cmljcyB0aGF0IGNhbiBiZSB1c2VkIHRvIGVzdGltYXRlIHRoZSBtb3N0CnByZWZlcmFibGUgbnVtYmVyIG9mIHRvcGljcyBmb3IgTERBIG1vZGVsLiBXZSB1c2VkIHRoZSBHcmlmZml0aHMyMDA0Cm1ldHJpY3MgaW5jbHVkZWQgaW4gdGhlIGRlZmF1bHQgZXhhbXBsZSBhbmQgSSd2ZSBhbHNvIGZvdW5kIHRoaXMgdG8KcHJvZHVjZSB0aGUgbW9zdCBpbnRlcnByZXRhYmxlIHJlc3VsdHMgYXMgc2hvdyBpbiB0aGUgZmlndXJlIGJlbG93OgoKIVtdKGltZy9rX21ldHJpY3MucG5nKXt3aWR0aD0iOTAlIn0KCkFzIGEgZ2VuZXJhbCBydWxlIG9mIHRodW1iIGFuZCBvdmVybHkgc2ltcGxpc3RpYyBoZXVyaXN0aWMsIHdlJ3JlCmxvb2tpbmcgZm9yIGFuIGluZmxlY3Rpb24gcG9pbnQgaW4gb3VyIHBsb3Qgd2hpY2ggaW5kaWNhdGVzIGFuIG9wdGltYWwKbnVtYmVyIG9mIHRvcGljcyB0byBzZWxlY3QgZm9yIGEgdmFsdWUgb2YgSy4KCiMjIyMgVGhlIGZpbmRpbmdLKCkgRnVuY3Rpb24KCkZpbmFsbHksIEJhaWwgKDIwMTgpIG5vdGVzIHRoYXQgdGhlYHN0bWAgcGFja2FnZSBoYXMgYSB1c2VmdWwgZnVuY3Rpb24KY2FsbGVkIGBzZWFyY2hLYCB3aGljaCBhbGxvd3MgdXMgdG8gc3BlY2lmeSBhIHJhbmdlIG9mIHZhbHVlcyBmb3IgYGtgCmFuZCBvdXRwdXRzIG11bHRpcGxlIGdvb2RuZXNzLW9mLWZpdCBtZWFzdXJlcyB0aGF0IGFyZSAidmVyeSB1c2VmdWwgaW4KaWRlbnRpZnlpbmcgYSByYW5nZSBvZiB2YWx1ZXMgZm9yIGBrYCB0aGF0IHByb3ZpZGUgdGhlIGJlc3QgZml0IGZvciB0aGUKZGF0YS4iCgpUaGUgc3ludGF4IG9mIHRoaXMgZnVuY3Rpb24gaXMgdmVyeSBzaW1pbGFyIHRvIHRoZSBgc3RtKClgIGZ1bmN0aW9uIHdlCnVzZWQgYWJvdmUsIGV4Y2VwdCB0aGF0IHdlIHNwZWNpZnkgYSByYW5nZSBmb3IgYGtgIGFzIG9uZSBvZiB0aGUKYXJndW1lbnRzLiBJbiB0aGUgY29kZSBiZWxvdywgd2Ugc2VhcmNoIGFsbCB2YWx1ZXMgb2YgYGtgIGJldHdlZW4gMTAgYW5kCjMwLgoKYGBge3Igc2VhcmNrLWssIGV2YWw9RkFMU0V9CiNJIGFtIG5vdCBleHBlY3RpbmcgeW91IHJ1biB0aGlzIGNvZGUgYXMgaXQgd2lsbCB0YWtlIHRvbyBsb25nCiNmaW5kaW5nayA8LSBzZWFyY2hLKGRvY3MsIAogICAgICAgICAgICAgICAgICAgICN2b2NhYiwgCiAgICAgICAgICAgICAgICAgICAgI0sgPSBjKDU6MTUpLAogICAgICAgICAgICAgICAgICAgICNkYXRhID0gbWV0YSwgCiAgICAgICAgICAgICAgICAgICAgI3ZlcmJvc2U9RkFMU0UpCiNwbG90KGZpbmRpbmdrKQpgYGAKCk5vdGUgdGhhdCBSdW5uaW5nIGBzZWFyY2hLKClgIGZ1bmN0aW9uIG9uIHRoaXMgY29ycHVzIHRvb2sgYWxsIG5pZ2h0IG9uCmEgcHJldHR5IHBvd2VyZnVsIE1hY0Jvb2sgUHJvIGFuZCBjcmFzaGVkIG9uY2UgYXMgd2VsbCwgc28gSSBkbyBub3QKZXhwZWN0IHlvdSB0byBydW4gdGhpcyBmb3IgdGhlIHdhbGt0aHJvdWdoLiBJIHJhbiBhIGNvdXBsZSBpdGVyYXRpb25zCmFuZCBsYW5kZWQgb24gYmV0d2VlbiA1IGFuZCAxNSB3aXRoIGFuIG9wdGltYWwgbnVtYmVyIG9mIHRvcGljcwpzb21ld2hlcmUgYXJvdW5kIDE0OgoKIVtdKGltZy9zZWFyY2hrX3Jlc3VsdHMucG5nKXt3aWR0aD0iOTAlIn0KCkdpdmVuIHRoZSBzb21ld2hhdCBjb25mbGljdGluZyByZXN1bHRzLCBhbHNvIHNvbWV3aGF0IHNlbGZpc2hseSBhbmQgZm9yCnRoZSBzYW1lIG9mIHNpbXBsaWNpdHkgZm9yIHRoaXMgd2Fsa3Rocm91Z2gsIEknbSBqdXN0IGdvaW5nIHRvIHN0aWNrCndpdGggdGhlIHJhdGhlciBhcmJpdHJhcnkgc2VsZWN0aW9uIG9mIDIwIHRvcGljcyBmb3IgdGhlIHJlbWFpbmRlciBvZgp0aGlzIFVuaXQuCgojIyMjIFRoZSBMREF2aXMgRXhwbG9yZXIKCk9uZSBmaW5hbCB0b29sIHRoYXQgSSB3YW50IHRvIGludHJvZHVjZSBmcm9tIHRoZSBgc3RtYCBwYWNrYWdlIGlzIHRoZQpgdG9MREF2aXMoKWAgZnVuY3Rpb24gd2hpY2ggcHJvdmlkZXMgYSBncmVhdCB2aXN1YWxpemF0aW9ucyBmb3IKZXhwbG9yaW5nIHRvcGljIGFuZCB3b3JkIGRpc3RyaWJ1dGlvbnMgdXNpbmcgYExEQXZpc2AgdG9waWMgYnJvd3NlcjoKCmBgYHtyIExEQXZpc30KdG9MREF2aXMobW9kID0gZm9ydW1zX3N0bSwgZG9jcyA9IGRvY3MpCmBgYAoKQXMgeW91IGNhbiBzZWUgZnJvbSB0aGUgYnJvd3NlciBzY3JlZW4gc2hvdCBiZWxvdywgb3VyIGN1cnJlbnQgc3RtIG1vZGVsCm9mIDIwIHRvcGljcyBpcyByZXN1bHRpbmcgaW4gYSBsb3Qgb2Ygb3ZlcmxhcCBhbW9uZyB0b3BpY3MgYW5kIHN1Z2dlc3QKdGhhdCAyMCBtYXkgbm90IGJlIGFuIG9wdGltYWwgbnVtYmVyIG9mIHRvcGljcywgYXMgb3RoZXIgYXBwcm9hY2hlcyBmb3IKZmluZGluZyBrIGFsc28gc3VnZ2VzdHM6CgohW10oaW1nL2xkYXZpcy5wbmcpCgojIyA0LiBFWFBMT1JFCgpTaWxnZSBhbmQgUm9iaW5zb24gKDIwMTgpIG5vdGUgdGhhdCBmaXR0aW5nIGF0IHRvcGljIG1vZGVsIGlzIHRoZSAiZWFzeQpwYXJ0LiIgVGhlIGhhcmQgcGFydCBpcyBtYWtpbmcgc2Vuc2Ugb2YgdGhlIG1vZGVsIHJlc3VsdHMgYW5kIHRoYXQgdGhlCnJlc3Qgb2YgdGhlIGFuYWx5c2lzIGludm9sdmVzIGV4cGxvcmluZyBhbmQgaW50ZXJwcmV0aW5nIHRoZSBtb2RlbCB1c2luZwphIHZhcmlldHkgb2YgYXBwcm9hY2hlcyB3aGljaCB3ZSdsbCB3YWxrdGhyb3VnaCBpbiBpbiB0aGlzIHNlY3Rpb24uCgpCYWlsICgyMDE4KSBjYXV0aW9ucywgaG93ZXZlciwgdGhhdDoKCj4gKi4uLnBvc3QtaG9jIGludGVycHJldGF0aW9uIG9mIHRvcGljIG1vZGVscyBpcyByYXRoZXIgZGFuZ2Vyb3VzLi4uIGFuZAo+IGNhbiBxdWlja2x5IGNvbWUgdG8gcmVzZW1ibGUgdGhlIHByb2Nlc3Mgb2YgInJlYWRpbmcgdGVhIGxlYXZlcywiIG9yCj4gZmluZGluZyBtZWFuaW5nIGluIHBhdHRlcm5zIHRoYXQgYXJlIGluIGZhY3QgcXVpdGUgYXJiaXRyYXJ5IG9yIGV2ZW4KPiByYW5kb20uKiAjI1wjIDRhLiBFeHBsb3JpbmcgQmV0YSBWYWx1ZXMKCkhpZGRlbiB3aXRoaW4gdGhpcyBgZm9ydW1zX2xkYWAgdG9waWMgbW9kZWwgb2JqZWN0IHdlIGNyZWF0ZWQgYXJlCnBlci10b3BpYy1wZXItd29yZCBwcm9iYWJpbGl0aWVzLCBjYWxsZWQgzrIgKCJiZXRhIikuIEl0IGlzIHRoZQpwcm9iYWJpbGl0eSBvZiBhIHRlcm0gKHdvcmQpIGJlbG9uZ2luZyB0byBhIHRvcGljLgoKTGV0J3MgdGFrZSBhIGxvb2sgYXQgdGhlIDUgbW9zdCBsaWtlbHkgdGVybXMgYXNzaWduZWQgdG8gZWFjaCB0b3BpYywKaS5lLiB0aG9zZSB3aXRoIHRoZSBsYXJnZXN0IM6yIHZhbHVlcyB1c2luZyB0aGUgYHRlcm1zKClgIGZ1bmN0aW9uIGZyb20KdGhlIGB0b3BpY21vZGVsc2AgcGFja2FnZToKCmBgYHtyIHRlcm1zfQp0ZXJtcyhmb3J1bXNfbGRhLCA1KQpgYGAKCkV2ZW4gdGhvdWdoIHdlJ3ZlIHNvbWV3aGF0IGFyYml0cmFyaWx5IHNlbGVjdGVkIHRoZSBudW1iZXIgb2YgdG9waWNzIGZvcgpvdXIgY29ycHVzLCBzb21lIHRoZXNlIHRvcGljcyBvciB0aGVtZXMgYXJlIGZhaXJseSBpbnR1aXRpdmUgdG8KaW50ZXJwcmV0LiBGb3IgZXhhbXBsZToKCi0gICBUb3BpYyAxMSAodGVjaG5vbG9neSwgc3R1ZGVudHMsIHNvZnR3YXJlLCBwcm9ncmFtLCBleGNlbCkgc2VlbXMgdG8KICAgIGJlIGFib3V0IHN0dWRlbnRzIHVzZSBvZiB0ZWNobm9sb2d5IGluY2x1ZGluZyBzb2Z0d2FyZSBwcm9ncmFtcyBsaWtlCiAgICBleGNlbDsKCi0gICBUb3BpYyA5IChxdWVzdGlvbnMsIGtpZHMsIGxvdmUsIGdhcG1pbmRlciwgc2hhcmluZykgc2VlbXMgdG8gYmUKICAgIGFib3V0IHRoZSBnYXBtaW5kZXIgYWN0aXZpdHkgZnJvbSB0aGUgTU9PQy1FZCBhbmQga2lkcyBlbmpveW1lbnQgb2YKICAgIGl0OyBhbmQKCi0gICBUb3BpYyAxOCAoZGF0YSwgc3R1ZGVudHMsIGNvbGxlY3QsIHJlYWwsIHNldHMpIHNlZW1zIHRvIGJlIGFib3V0CiAgICBzdHVkZW50IGNvbGxlY3Rpb24gYW5kIHVzZSBvZiByZWFsIHdvcmxkIGRhdGEgc2V0cy4KCk5vdCBzdXJwcmlzaW5nbHksIHRoZSBgdGlkeXRleHRgIHBhY2thZ2UgaGFzIGEgaGFuZHkgZnVuY3Rpb24KY29udmVuaWVudGx5IG5hbWUgYHRpZHkoKWAgdG8gY29udmVydCBvdXIgTERBIG1vZGVsIHRvIGEgdGlkeSBkYXRhIGZyYW1lCmNvbnRhaW5pbmcgdGhlc2UgYmV0YSB2YWx1ZXMgZm9yIGVhY2ggdGVybToKCmBgYHtyIHRpZHlfbGRhfQp0aWR5X2xkYSA8LSB0aWR5KGZvcnVtc19sZGEpCnRpZHlfbGRhCmBgYAoKT2J2aW91c2x5LCBpdCdzIG5vdCB2ZXJ5IGVhc3kgdG8gaW50ZXJwcmV0IHdoYXQgdGhlIHRvcGljcyBhcmUgYWJvdXQKZnJvbSBhIGRhdGEgZnJhbWUgbGlrZSB0aGlzIHNvIGxldCdzIGJvcnJvdyBjb2RlIGFnYWluIGZyb20gW0NoYXB0ZXIKOC40LjMgSW50ZXJwcmV0aW5nIHRoZSB0b3BpYwptb2RlbF0oaHR0cHM6Ly93d3cudGlkeXRleHRtaW5pbmcuY29tL25hc2EuaHRtbD9xPWJldGEjaW50ZXJwcmV0aW5nLXRoZS10b3BpYy1tb2RlbCkKaW4gVGV4dCBNaW5pbmcgd2l0aCBSIHRvIGV4YW1pbmUgdGhlIHRvcCA1IHRlcm1zIGZvciBlYWNoIHRvcGljIGFuZCB0aGVuCmxvb2sgYXQgdGhpcyBpbmZvcm1hdGlvbiB2aXN1YWxseToKCmBgYHtyIHRvcF90ZXJtc30KdG9wX3Rlcm1zIDwtIHRpZHlfbGRhICU+JQogIGdyb3VwX2J5KHRvcGljKSAlPiUKICBzbGljZV9tYXgoYmV0YSwgbiA9IDUsIHdpdGhfdGllcyA9IEZBTFNFKSAlPiUKICB1bmdyb3VwKCkgJT4lCiAgYXJyYW5nZSh0b3BpYywgLWJldGEpCnRvcF90ZXJtcyAlPiUKICBtdXRhdGUodGVybSA9IHJlb3JkZXJfd2l0aGluKHRlcm0sIGJldGEsIHRvcGljKSkgJT4lCiAgZ3JvdXBfYnkodG9waWMsIHRlcm0pICU+JSAgICAKICBhcnJhbmdlKGRlc2MoYmV0YSkpICU+JSAgCiAgdW5ncm91cCgpICU+JQogIGdncGxvdChhZXMoYmV0YSwgdGVybSwgZmlsbCA9IGFzLmZhY3Rvcih0b3BpYykpKSArCiAgZ2VvbV9jb2woc2hvdy5sZWdlbmQgPSBGQUxTRSkgKwogIHNjYWxlX3lfcmVvcmRlcmVkKCkgKwogIGxhYnModGl0bGUgPSAiVG9wIDUgdGVybXMgaW4gZWFjaCBMREEgdG9waWMiLAogICAgICAgeCA9IGV4cHJlc3Npb24oYmV0YSksIHkgPSBOVUxMKSArCiAgZmFjZXRfd3JhcCh+IHRvcGljLCBuY29sID0gNCwgc2NhbGVzID0gImZyZWUiKQpgYGAKCiMjIyA0Yi4gRXhwbG9yaW5nIEdhbW1hIFZhbHVlcwoKTm93IHRoYXQgd2UgaGF2ZSBhIHNlbnNlIG9mIHRoZSBtb3N0IGNvbW1vbiB3b3JkcyBhc3NvY2lhdGVkIHdpdGggZWFjaAp0b3BpYywgbGV0J3MgdGFrZSBhIGxvb2sgYXQgdGhlIHRvcGljIHByZXZhbGVuY2UgaW4gb3VyIE1PT0MtRWQKZGlzY3Vzc2lvbiBmb3J1bSBjb3JwdXMsIGluY2x1ZGluZyB0aGUgd29yZHMgdGhhdCBjb250cmlidXRlIHRvIGVhY2gKdG9waWMgd2UgZXhhbWluZWQgYWJvdmUuCgpBbHNvLCBoaWRkZW4gd2l0aGluIG91ciBgZm9ydW1zX2xkYWAgdG9waWMgbW9kZWwgb2JqZWN0IHdlIGNyZWF0ZWQgYXJlCnBlci1kb2N1bWVudC1wZXItdG9waWMgcHJvYmFiaWxpdGllcywgY2FsbGVkIM6zICgiZ2FtbWEiKS4gVGhpcyBwcm92aWRlcwp0aGUgcHJvYmFiaWxpdGllcyB0aGF0IGVhY2ggZG9jdW1lbnQgaXMgZ2VuZXJhdGVkIGZyb20gZWFjaCB0b3BpYywgdGhhdApnYW1tYSBtYXRyaXguIFdlIGNhbiBjb21iaW5lIG91ciBiZXRhIGFuZCBnYW1tYSB2YWx1ZXMgdG8gdW5kZXJzdGFuZCB0aGUKdG9waWMgcHJldmFsZW5jZSBpbiBvdXIgY29ycHVzLCBhbmQgd2hpY2ggd29yZHMgY29udHJpYnV0ZSB0byBlYWNoCnRvcGljLgoKVG8gZG8gdGhpcywgd2UncmUgZ29pbmcgdG8gYm9ycm93IHNvbWUgY29kZSBmcm9tIHRoZSBTaWxnZSAoMjAxOCkgcG9zdCwKW1RyYWluaW5nLCBldmFsdWF0aW5nLCBhbmQgaW50ZXJwcmV0aW5nIHRvcGljCm1vZGVsc10oaHR0cHM6Ly9qdWxpYXNpbGdlLmNvbS9ibG9nL2V2YWx1YXRpbmctc3RtLykuCgpGaXJzdCwgbGV0J3MgY3JlYXRlIHR3byB0aWR5IGRhdGEgZnJhbWVzIGZvciBvdXIgYmV0YSBhbmQgZ2FtbWEgdmFsdWVzCgpgYGB7ciBiZXRhX2dhbW1hfQp0ZF9iZXRhIDwtIHRpZHkoZm9ydW1zX2xkYSkKdGRfZ2FtbWEgPC0gdGlkeShmb3J1bXNfbGRhLCBtYXRyaXggPSAiZ2FtbWEiKQp0ZF9iZXRhCnRkX2dhbW1hCmBgYAoKTmV4dCwgd2UnbGwgYWRvcHQgSnVsaWEncyBjb2RlIHdob2xlc2FsZSB0byBjcmVhdGUgYSBmaWx0ZXJlZCBkYXRhIGZyYW1lCm9mIG91ciBgdG9wX3Rlcm1zYCwgam9pbiB0aGlzIHRvIGEgbmV3IGRhdGEgZnJhbWUgZm9yIGBnYW1tYS10ZXJtc2AgYW5kCmNyZWF0ZSBhIG5pY2UgY2xlYW4gdGFibGUgdXNpbmcgdGhlIGBrYWJlbCgpYCBmdW5jdGlvbiBga25pdHJgIHBhY2thZ2U6CgpgYGB7ciBwcmV2YWxlbmNlX3RhYmxlfQp0b3BfdGVybXMgPC0gdGRfYmV0YSAlPiUKICBhcnJhbmdlKGJldGEpICU+JQogIGdyb3VwX2J5KHRvcGljKSAlPiUKICB0b3Bfbig3LCBiZXRhKSAlPiUKICBhcnJhbmdlKC1iZXRhKSAlPiUKICBzZWxlY3QodG9waWMsIHRlcm0pICU+JQogIHN1bW1hcmlzZSh0ZXJtcyA9IGxpc3QodGVybSkpICU+JQogIG11dGF0ZSh0ZXJtcyA9IG1hcCh0ZXJtcywgcGFzdGUsIGNvbGxhcHNlID0gIiwgIikpICU+JSAKICB1bm5lc3QoKQoKZ2FtbWFfdGVybXMgPC0gdGRfZ2FtbWEgJT4lCiAgZ3JvdXBfYnkodG9waWMpICU+JQogIHN1bW1hcmlzZShnYW1tYSA9IG1lYW4oZ2FtbWEpKSAlPiUKICBhcnJhbmdlKGRlc2MoZ2FtbWEpKSAlPiUKICBsZWZ0X2pvaW4odG9wX3Rlcm1zLCBieSA9ICJ0b3BpYyIpICU+JQogIG11dGF0ZSh0b3BpYyA9IHBhc3RlMCgiVG9waWMgIiwgdG9waWMpLAogICAgICAgICB0b3BpYyA9IHJlb3JkZXIodG9waWMsIGdhbW1hKSkKCmdhbW1hX3Rlcm1zICU+JQogIHNlbGVjdCh0b3BpYywgZ2FtbWEsIHRlcm1zKSAlPiUKICBrYWJsZShkaWdpdHMgPSAzLCAKICAgICAgICBjb2wubmFtZXMgPSBjKCJUb3BpYyIsICJFeHBlY3RlZCB0b3BpYyBwcm9wb3J0aW9uIiwgIlRvcCA3IHRlcm1zIikpCmBgYAoKQW5kIGxldCdzIGFsc28gY29tcGFyZSB0aGlzIHRvIHRoZSBtb3N0IHByZXZhbGVudCB0b3BpY3MgYW5kIHRlcm1zIGZyb20Kb3VyIGBmb3J1bXNfc3RtYCBtb2RlbCB0aGF0IHdlIGNyZWF0ZWQgdXNpbmcgdGhlIGBwbG90KClgIGZ1bmN0aW9uOgoKYGBge3IgcGxvdF9zdG19CnBsb3QoZm9ydW1zX3N0bSwgbiA9IDcpCmBgYAoKIyMjIDRjLiBSZWFkaW5nIHRoZSBUZWEgTGVhdmVzCgpSZWNvZ25pemluZyB0aGF0IHRvcGljIG1vZGVsaW5nIGlzIGJlc3QgdXNlZCBhcyBhICJ0b29sIGZvciByZWFkaW5nIiBhbmQKcHJvdmlkZXMgb25seSBhbiBpbmNvbXBsZXRlIGFuc3dlciB0byBvdXIgb3ZlcmFyY2hpbmcsICoqIkhvdyBkbyB3ZQpxdWFudGlmeSB3aGF0IGEgY29ycHVzIGlzIGFib3V0PyIqKiwgdGhlIHJlc3VsdHMgZG8gc3VnZ2VzdCBzb21lCnBvdGVudGlhbCB0b3BpY3MgdGhhdCBoYXZlIGVtZXJnZXMgYXMgd2VsbCBhcyBzb21lIGFyZWFzIHdvcnRoIGZvbGxvd2luZwp1cCBvbi4KClNwZWNpZmljYWxseSwgbG9va2luZyBhdCBzb21lIG9mIHRoZSBjb21tb24gY2x1c3RlcnMgb2Ygd29yZHMgZm9yIHRoZQptb3JlIHByZXZhbGVudCB0b3BpY3Mgc3VnZ2VzdCB0aGF0IHNvbWUga2V5IHRvcGljcyBvciAibGF0ZW50IHRoZW1lcyIKKHJlbmFtZWQgaW4gYm9sZCkgbWlnaHQgaW5jbHVkZToKCi0gICAqKlRlYWNoaW5nIFN0YXRpc3RpY3M6KiogVW5zdXJwcmlzaW5nLCBnaXZlbiB0aGUgY291cnNlIHRpdGxlLCB0aGUKICAgIHRvcGljcyBtb3N0IHByZXZhbGVudCBpbiBib3RoIHRoZSBgZm9ydW1zX3N0bWAgYW5kIGBmb3J1bXNfbGRhYAogICAgbW9kZWxzIGNvbnRhaW5zIHRoZSB0ZXJtcyAidGVhY2giLCAic3R1ZGVudHMiLCAic3RhdGlzdGljcyIuIFRoaXMKICAgIGNvdWxkIGJlIGFuICJvdmVyYXJjaGluZyB0aGVtZSIgYnV0IG1vcmUgbGlrZWx5IG1heSBzaW1wbHkgYmUganVzdAogICAgdGhlIHJlc2lkdWUgb2YgdGhlIGNvdXJzZSB0aXRsZSB0aG91Z2ggYmVpbmcgc3ByaW5rbGVkIHRocm91Z2hvdXQKICAgIHRoZSBmb3J1bXMgYW5kIGRlc2VydmVzIHNvbWUgZm9sbG93IHVwLiBUb3BpY3MgOCBmcm9tIHRoZSBMREEgbW9kZWwKICAgIG1heSBvdmVybGFwIHdpdGggdGhpcyB0b3BpYyBhcyB3ZWxsLgotICAgKipDb3Vyc2UgVXRpbGl0eToqKiBUaGUgc2Vjb25kIG1vc3QgcHJldmFsZW50IFRvcGljcyAoMTMgYW5kIDIpIGluCiAgICB0aGUgYGxkYWAgYW5kIGBzdG1gIG1vZGVscyByZXNwZWN0aXZlbHksIHNlZW0gdG8gcG90ZW50aWFsbHkgYmUKICAgIGFib3V0IHRoZSB1c2VmdWxuZXNzIG9mIGNvdXJzZSAicmVzb3VyY2VzIiBsaWtlIGxlc3NvbnMsIHRvb2xzLAogICAgdmlkZW9zLCBhbmQgYWN0aXZpdGllcy4gSSdtIHdhZ2VyaW5nIHRoaXMgbWlnaHQgYmUgYSBmb3J1bSBkZWRpY2F0ZWQKICAgIHRvIGNvdXJzZSBmZWVkYmFjay4gVG9waWMgMTUgZnJvbSB0aGUgU1RNIG1vZGVsIGFsc28gc3VnZ2VzdCB0aGlzCiAgICBtYXkgYmUgYSBicm9hZGVyIHRoZW1lLgotICAgKipVc2luZyBSZWFsLVdvcmxkIERhdGE6KiogVG9waWNzIDE4ICYgMTIgZnJvbSB0aGUgTERBIG1vZGVsCiAgICBwYXJ0aWN1bGFybHkgaW50cmlndWUgbWUgYW5kIEknbSB3YWdlcmluZyB0aGlzIGlzIHByZXR0eSBwb3NpdGl2ZQogICAgc2VudGltZW50IGFtb25nIHBhcnRpY2lwYW50cyBhYm91dCB0aGUgdmFsdWUgYW5kIGJlbmVmaXQgb2YgaGF2aW5nCiAgICBzdHVkZW50cyBjb2xsZWN0IGFuZCBhbmFseXplIHJlYWwgZGF0YSBzZXRzIChlLmcuIENlbnN1cyBkYXRhIGluCiAgICBUb3BpYyAxKSBhbmQgd29yayBvbiBwcm9qZWN0cyByZWxldmFudCB0byB0aGVpciByZWFsIGxpZmUuIFdpbGwKICAgIGRlZmluaXRlbHkgZm9sbG93IHVwIG9uIHRoaXMgb25lLgotICAgKipUZWNobm9sb2d5IFVzZToqKiBTZXZlcmFsIHRvcGljcyAoNiAmIDExIGZyb20gTERBIGFuZCA4ICYgMTkgZnJvbQogICAgU1RNKSBhcHBlYXIgdG8gYmUgYWJvdXQgc3R1ZGVudCB1c2Ugb2YgdGVjaG5vbG9neSBhbmQgc29mdHdhcmUgbGlrZQogICAgY2FsY3VsYXRvcnMgYW5kIEV4Y2VsIGZvciB0ZWFjaGluZyBzdGF0aXN0aWNzIGFuZCB1c2luZyBzaW11bGF0aW9ucy4KICAgIFRvcGljIDE2IGZyb20gTERBIGFsc28gc3VnZ2VzdCB0aGUgdXNlIG9mIHRoZSBDb21tb24gT25saW5lIERhdGEKICAgIEFuYWx5c2lzIFBsYXRmb3JtCiAgICAoW0NPREFQXShDb21tb24lMjBPbmxpbmUlMjBEYXRhJTIwQW5hbHlzaXMlMjBQbGF0Zm9ybSUyMChDT0RBUCklMjBPcGVuLXNvdXJjZSUyMHNvZnR3YXJlJTIwZm9yJTIwZHluYW1pYyUyMGRhdGElMjBleHBsb3JhdGlvbikpLgotICAgKipTdHVkZW50IFN0cnVnZ2xlICYgRW5nYWdlbWVudDoqKiBUb3BpYyAxNSBmcm9tIExEQSBhbmQgVG9waWMgMTYKICAgIGZyb20gU1RNIGFsc28gaW50cmlndWUgbWUgYW5kIGFwcGVhciB0byBiZSB0d28gb3Bwb3NpdGUgc2lkZXMgb2YKICAgIHBlcmhhcHMgdGhlIHNhbWUgY29pbi4gVGhlIGZvcm1lciBpbmNsdWRlcyAic3RydWdnbGUiIGFuZCAicmVhZGluZyIKICAgIHdoaWNoIHN1Z2dlc3RzIHBlcmhhcHMgYSBiYXJyaWVyIHRvIHRlYWNoaW5nIHN0YXRpc3RpY3Mgd2hpbGUgVG9waWMKICAgIDE2IGNvbnRhaW5zIHRvcCBzdGVtcyBsaWtlICJlbmdhZ2UiLCAiYWN0aXYiLCBhbmQgInRoaW5rIiBhbmQgbWF5CiAgICBzdWdnZXN0IHBhcnRpY2lwYW50cyBhbnRpY2lwYXRlIGFjdGl2aXRpZXMgbWF5IGVuZ2FnZSBzdHVkZW50cy4KClRvIHNlcnZlIGFzIGEgY2hlY2sgb24gbXkgdGVhIGxlYWYgcmVhZGluZywgSSdtIGdvaW5nIHRvIGZvbGxvdyBCYWlsJ3MKcmVjb21tZW5kYXRpb24gdG8gZXhhbWluZSBzb21lIG9mIHRoZXNlIHRvcGljcyBxdWFsaXRhdGl2ZWx5LiBUaGUgYHN0bWAKcGFja2FnZSBoYXMgYW5vdGhlciB1c2VmdWwgZnVuY3Rpb24gdGhvdWdoIGV4Y2VwdGlvbmFsbHkgZnVzc3kgZnVuY3Rpb24KY2FsbGVkIGBmaW5kVGhvdWdodHNgIHdoaWNoIGV4dHJhY3RzIHBhc3NhZ2VzIGZyb20gZG9jdW1lbnRzIHdpdGhpbiB0aGUKY29ycHVzIGFzc29jaWF0ZSB3aXRoIHRvcGljcyB0aGF0IHlvdSBzcGVjaWZ5LgoKVGhlIGZpcnN0IGxpbmUgb2YgY29kZSBtYXkgbm90IGJlIG5lY2Vzc2FyeSBmb3IgeW91ciBpbmRlcGVuZGVudAphbmFseXNpcywgYnV0IGJlY2F1c2UgdGhlIGB0ZXh0UHJvY2Vzc29yKClgIGZ1bmN0aW9uIHJlbW92ZWQgc2V2ZXJhbApkb2N1bWVudHMgZHVyaW5nIHByb2Nlc3NpbmcsIHRoZSBgZmluZHRob3VnaHRzKClgIGZ1bmN0aW9uIGNhbid0CnByb3Blcmx5IGluZGV4IHRoZSBwcm9jZXNzZWQgZG9jcy4gVGhpcyBbbGluZSBvZiBjb2RlIGZvdW5kIG9uCnN0YWNrb3ZlcmZsb3ddKGh0dHBzOi8vc3RhY2tvdmVyZmxvdy5jb20vcXVlc3Rpb25zLzQzNDkyNjY3L3Itc3RtLW51bWJlci1vZi1wcm92aWRlZC10ZXh0cy1hbmQtbnVtYmVyLW9mLWRvY3VtZW50cy1tb2RlbGVkLWRvLW5vdC1tYXRjaCkKcmVtb3ZlcyBkb2N1bWVudHMgZnJvbSBvcmlnaW5hbCBgdHNfZm9ydW1fZGF0YWAgc291cmNlIHRoYXQgd2VyZSByZW1vdmVkCmR1cmluZyBwcm9jZXNzaW5nIHNvIHRoZXJlIGlzIGEgb25lLXRvLW9uZSBjb3JyZXNwb25kZW5jZSB3aXRoCmBmb3J1bXNfc3RtYCBhbmQgc28geW91IGNhbiB1c2UgdGhlIGZ1bmN0aW9uIHRvIGZpbmQgcG9zdHMgYXNzb2NpYXRlZAp3aXRoIGEgZ2l2ZW4gdG9waWMuCgpMZXQncyBzbGlnaHRseSByZWR1Y2Ugb3VyIG9yaWdpbmFsIGRhdGEgc2V0IHRvIG1hdGNoIG91ciBTVE0gbW9kZWwsIHBhc3MKYm90aCB0byB0aGUgYGZpbmRUaG91Z2h0cygpYCBmdW5jdGlvbiwgYW5kIHNldCBvdXIgYXJndW1lbnRzIHRvIHJldHVybgpgbiA9MTBgIHBvc3RzIGZyb20gYHRvcGljcyA9IDJgIChpLmUuIFRvcGljIDIpIHRoYXQgaGF2ZSBhdCBsZWFzdCA1MCUgb3IKYHRocmVzaCA9IDAuNWAgYXMgYSBtaW5pbXVtIHRocmVzaG9sZCBmb3IgdGhlIGVzdGltYXRlZCB0b3BpYwpwcm9wb3J0aW9uLgoKYGBge3IgZmluZFRob3VnaHRzXzJ9CnRzX2ZvcnVtX2RhdGFfcmVkdWNlZCA8LXRzX2ZvcnVtX2RhdGEkcG9zdF9jb250ZW50Wy10ZW1wJGRvY3MucmVtb3ZlZF0KZmluZFRob3VnaHRzKGZvcnVtc19zdG0sCiAgICAgICAgICAgICB0ZXh0cyA9IHRzX2ZvcnVtX2RhdGFfcmVkdWNlZCwKICAgICAgICAgICAgIHRvcGljcyA9IDIsIAogICAgICAgICAgICAgbiA9IDEwLAogICAgICAgICAgICAgdGhyZXNoID0gMC41KQpgYGAKCkR1cGxpY2F0ZSBwb3N0cyBhc2lkZSwgdGhpcyAqKkNvdXJzZSBVdGlsaXR5KiogdG9waWMgcmV0dXJucyBwb3N0cyB0aGVyZQp3ZXJlIGV4cGVjdGVkIGJhc2VkIG9uIG15IGludGVycHJldGF0aW9uIG9mIHRoZSBrZXkgdGVybXMgZm9yIFRvcGljIDIuCkl0IGxvb2tzIGxpa2UgSSBtYXkgaGF2ZSByZWFkIHRob3NlIHRlYSBsZWF2ZXMgY29ycmVjdGx5LgoKTm93IGxldCdzIHRha2UgYSBsb29rIGF0IFRvcGljIDE2IHRoYXQgd2UgdGhvdWdodCBtaWdodCBiZSByZWxhdGVkIHRvCnN0dWRlbnQgZW5nYWdlbWVudDoKCmBgYHtyIGZpbmRUaG91Z2h0c18xNn0KZmluZFRob3VnaHRzKGZvcnVtc19zdG0sCiAgICAgICAgICAgICB0ZXh0cyA9IHRzX2ZvcnVtX2RhdGFfcmVkdWNlZCwKICAgICAgICAgICAgIHRvcGljcyA9IDE2LCAKICAgICAgICAgICAgIG4gPSAxMCwKICAgICAgICAgICAgIHRocmVzaCA9IDAuNSkKYGBgCgpJdCBsb29rcyBsaWtlIG15IHRlYSByZWFkaW5nIHdhcyBhIHBhcnRpYWxseSBjb3JyZWN0IGZvciBUb3BpYyAxNiwKdGhvdWdoIHRoZSByZXN1bHRzIHNlZW0gdG8gYmUgYWJvdXQgYSBzcGVjaWZpYyAiUGVwc2kgY2hhbGxlbmdlIgphY3Rpdml0eSB0byBjb25kdWN0IHdpdGggc3R1ZGVudHMuCgpGaW5hbGx5LCBsZXQncyBsb29rIGF0IHBvc3RzIGZyb20gVG9waWMgMyB3aGljaCB3ZSB0aG91Z2ggbWlnaHQgYmUgYW4Kb3ZlcmFyY2hpbmcgdGhlbWUgYWJvdXQgdGVhY2hpbmcgc3RhdGlzdGljczoKCmBgYHtyIGZpbmRUaG91Z2h0c18zfQp0c19mb3J1bV9kYXRhX3JlZHVjZWQgPC10c19mb3J1bV9kYXRhJHBvc3RfY29udGVudFstdGVtcCRkb2NzLnJlbW92ZWRdCmZpbmRUaG91Z2h0cyhmb3J1bXNfc3RtLAogICAgICAgICAgICAgdGV4dHMgPSB0c19mb3J1bV9kYXRhX3JlZHVjZWQsCiAgICAgICAgICAgICB0b3BpY3MgPSAzLCAKICAgICAgICAgICAgIG4gPSAxMCwKICAgICAgICAgICAgIHRocmVzaCA9IDAuNSkKYGBgCgpMb29raW5nIGF0IGp1c3QgdGhlIDEwIHBvc3RzIHJldHVybmVkLCBwZXJoYXBzIGEgYmV0dGVyIG5hbWUgZm9yIHRoaXMKdG9waWMgd291bGQgYmUgKipDb3Vyc2UgUmVmbGVjdGlvbnMgb24gVGVhY2hpbmcgU3RhdGlzdGljcyoqLgoKIyMjIyBVbml0IFRha2Vhd2F5CgpJbiBhZGRpdGlvbiB0byBzb21lIHVzZWZ1bCBSIHBhY2thZ2VzIGFuZCBmdW5jdGlvbnMgZm9yIHRoZSBhY3R1YWwKcHJvY2VzcyBvZiB0b3BpYyBtb2RlbGluZywgaG9wZWZ1bGx5IHRoZXJlIGFyZSB0d28gbWFpbiBsZXNzb25zIEknbQpob3BpbmcgeW91IHRha2UgYXdheSBmcm9tIHRoaXMgd2Fsa3Rocm91Z2g6CgoxLiAgKipUb3BpYyBtb2RlbGluZyByZXF1aXJlcyBhIGxvdCBvZiBkZWNpc2lvbnMuKiogQmV5b25kIGRlY2lkaW5nIG9uIGEKICAgIHZhbHVlIGZvciBLLCB0aGVyZSBhcmUgYSBudW1iZXIgb2Yga2V5IGRlY2lzaW9ucyB0aGF0IHlvdSBoYXZlIHRvCiAgICBtYWtlIHRoYXQgY2FuIGRyYW1hdGljYWxseSBhZmZlY3QgeW91ciByZXN1bHRzLiBGb3IgZXhhbXBsZSwgdG8gc3RlbQogICAgb3Igbm90IHRvIHN0ZW0/IFdoYXQgcXVhbGlmaWVzIGFzIGEgZG9jdW1lbnQ/IFdoYXQgZmxhdm9yIG9yIHRvcGljCiAgICBtb2RlbGluZyBpcyBiZXN0IHN1aXRlZCB0byB5b3VyIGRhdGEgYW5kIHJlc2VhcmNoIHF1ZXN0aW9ucz8gSG93CiAgICBtYW55IGl0ZXJhdGlvbnMgdG8gcnVuPwoyLiAgKipUb3BpYyBtb2RlbGluZyBpcyBhcyBtdWNoIGFydCBhcyAoZGF0YSkgc2NpZW5jZS4qKiBBcyBCYWlsICgyMDE4KQogICAgbm90ZWQsIHRoZSB0ZXJtICJ0b3BpYyIgaXMgc29tZXdoYXQgYW1iaXRpb3VzLCBhbmQgdG9waWMgbW9kZWxzIGRvCiAgICBub3QgcHJvZHVjZSBoaWdobHkgbnVhbmNlZCBjbGFzc2lmaWNhdGlvbiBvZiB0ZXh0cy4gT25jZSB5b3UndmUgZml0CiAgICB5b3VyIG1vZGVsLCBpbnRlcnByZXRpbmcgeW91ciBtb2RlbCByZXF1aXJlcyBzb21lIG1lbnRhbCBneW1uYXN0aWNzCiAgICBhbmQgaWRlYWxseSBzb21lIGtub3dsZWRnZSBvZiB0aGUgY29udGV4dCBmcm9tIHdoaWNoIHRoZSBkYXRhIGNhbWUKICAgIHRvIGhlbHAgd2l0aCBpbnRlcnByZXRhdGlvbiBvZiB5b3VyIHRvcGljcy4gTW9yZW92ZXIsIHRoZQogICAgcXVhbnRpdGF0aXZlIGFwcHJvYWNoZXMgZm9yIG1ha2luZyB0aGUgZGVjaXNpb25zIGhpZ2hsaWdodGVkIGFib3ZlCiAgICBhcmUgaW1wZXJmZWN0IGFuZCBhIGdvb2QgZGVhbCBvZiBodW1hbiBqdWRnbWVudCByZXF1aXJlZC4KCiMjIyMjIOKchSBDb21wcmVoZW5zaW9uIENoZWNrCgpVc2luZyB0aGUgU1RNIG1vZGVsIHlvdSBmaXQgZnJvbSB0aGUgU2VjdGlvbiAzIFxbQ29tcHJlaGVuc2lvbiBDaGVja1xdCndpdGggYSBkaWZmZXJlbnQgdmFsdWUgZm9yIEssIHVzZSB0aGUgYXBwcm9hY2hlcyBkZW1vbnN0cmF0ZWQgaW4gU2VjdGlvbgo0IHRvIGV4cGxvcmUgYW5kIGludGVycHJldCB5b3VyIHRvcGljcyBhbmQgdGVybXMgYW5kIHJldmlzaXQgdGhlCmZvbGxvd2luZyBxdWVzdGlvbjoKCjEuICBOb3cgdGhhdCB5b3UgaGF2ZSBhIGxpdHRsZSBtb3JlIGNvbnRleHQsIGhvdyBtaWdodCB5b3UgcmV2aXNlIHlvdXIKICAgIGluaXRpYWwgaW50ZXJwcmV0YXRpb24gb2Ygc29tZSBvZiB0aGUgbGF0ZW50IHRvcGljcyBvciBsYXRlbnQgdGhlbWVzCiAgICBmcm9tIHlvdXIgbW9kZWw/ICpVc2luZyBhIEsgb2YgMTQsIGJhc2VkIG9uIHRoZSBCYWlsIFNUTSBzZWxlY3Rpb24KICAgIG1vZGVsLCB0aGUgdG9waWNzIGFwcGVhciBtdWNoIG1vcmUgY29nZW50IHRoYW4gd2hlbiBJIHVzZWQgMTAgb3IgMjAuCiAgICBBcyBhbiBleGFtcGxlLCB0aGUgSz0xMCBydW4gc2hvd2VkIGEgdG9waWMgZm9jdXNlZCBvbiB2aWRlbyBzb2Z0d2FyZQogICAgb3IgY2FwdHVyZSB0ZWNobm9sb2d5LiBXaGVuIHVzaW5nIHRoZSBLPTE0IGRhdGEsIHRoaXMgdG9waWMgd2FzCiAgICBjbGVhcmx5IG1vcmUgYWJvdXQgc2hhcmluZyBkYXRhIHJlc291cmNlcyB3aGljaCBtYXkgaW5jbHVkZSB2aWRlbwogICAgZmlsZXMuKgoKYGBge3J9CmZvcnVtc19zdG1fMTQgPC0gc3RtKGRvY3VtZW50cz1kb2NzLCAKICAgICAgICAgICAgICAgICAgICAgZGF0YT1tZXRhLAogICAgICAgICAgICAgICAgICAgICB2b2NhYj12b2NhYiwgCiAgICAgICAgICAgICAgICAgICAgIHByZXZhbGVuY2UgPX4gY291cnNlX2lkICsgZm9ydW1faWQsCiAgICAgICAgICAgICAgICAgICAgIEs9MTQsCiAgICAgICAgICAgICAgICAgICAgIG1heC5lbS5pdHM9MjUsCiAgICAgICAgICAgICAgICAgICAgIHZlcmJvc2UgPSBGQUxTRSkKCmZpbmRUaG91Z2h0cyhmb3J1bXNfc3RtXzE0LAogICAgICAgICAgICAgdGV4dHMgPSB0c19mb3J1bV9kYXRhX3JlZHVjZWQsCiAgICAgICAgICAgICB0b3BpY3MgPSAxLCAKICAgICAgICAgICAgIG4gPSAxMCwKICAgICAgICAgICAgIHRocmVzaCA9IDAuNSkKYGBgCgpgYGB7cn0KZmluZFRob3VnaHRzKGZvcnVtc19zdG1fMTQsCiAgICAgICAgICAgICB0ZXh0cyA9IHRzX2ZvcnVtX2RhdGFfcmVkdWNlZCwKICAgICAgICAgICAgIHRvcGljcyA9IDksIAogICAgICAgICAgICAgbiA9IDEwLAogICAgICAgICAgICAgdGhyZXNoID0gMC41KQpgYGAK

Unit 3 Walkthrough: Topic Modeling in MOOC-Eds

James Hardaway

2/17/2022

0. INTRODUCTION

Walkthrough Focus

1. PREPARE

1a. Context

Participating in a MOOC and Professional Learning Team: How a Blended Approach to Professional Development Makes a Difference

1b. Guiding Questions

1c. Set Up

2. WRANGLE

2a. Import Forum Data

2b. Cast a Document Term Matrix

Functions Used

Tidying Text

✅ Comprehension Check

Creating a Document Term Matrix

✅ Comprehension Check

2c. To Stem or not to Stem?

Processing and Stemming for STM

Stemming Tidy Text

✅ Comprehension Check

3. MODEL

3a. Fitting a Topic Modeling with LDA

3b. Fitting a Structural Topic Model

The `stm` Package

✅ Comprehension Check

3c. Finding K

The FindTopicsNumber Function

The findingK() Function

The LDAvis Explorer

4. EXPLORE

4b. Exploring Gamma Values

4c. Reading the Tea Leaves

Unit Takeaway

✅ Comprehension Check

Unit 3 Walkthrough: Topic Modeling in MOOC-Eds

James Hardaway

2/17/2022

0. INTRODUCTION

Walkthrough Focus

1. PREPARE

1a. Context

Participating in a MOOC and Professional Learning Team: How a Blended Approach to Professional Development Makes a Difference

1b. Guiding Questions

1c. Set Up

2. WRANGLE

2a. Import Forum Data

2b. Cast a Document Term Matrix

Functions Used

Tidying Text

✅ Comprehension Check

Creating a Document Term Matrix

✅ Comprehension Check

2c. To Stem or not to Stem?

Processing and Stemming for STM

Stemming Tidy Text

✅ Comprehension Check

3. MODEL

3a. Fitting a Topic Modeling with LDA

3b. Fitting a Structural Topic Model

The stm Package

✅ Comprehension Check

3c. Finding K

The FindTopicsNumber Function

The findingK() Function

The LDAvis Explorer

4. EXPLORE

4b. Exploring Gamma Values

4c. Reading the Tea Leaves

Unit Takeaway

✅ Comprehension Check

The `stm` Package