BANL 3200: Supervises Machine Learning
Midterm
Jaden Sampson
2025-01-10
Instructions
- Complete the coded in the code chunks.
- Type the answers as text below each question.
- Knit your file.
- Upload resulting HTML file to the Canvas
Questions 1 (12 pts)
The MNIST dataset is one of the most well-known datasets in the field of machine learning and is widely used for training various image processing systems. It contains images of handwritten digits, with each image being a 28x28 pixel grayscale image. Each pixel ranges from 0 (white) to 255 (black). The dataset is structured as follows:
Label: This variable denotes the class of the handwritten digit image. It represents the digit depicted in the image (0 through 9). In some variations of the MNIST dataset (though not standard), the label might represent letters if it’s modified and extended to alphabets.
Pix1, Pix2, …, Pix784: These variables represent the pixel values of the 28x28 pixel images. Each image is “flattened” into a single row with 784 columns (28 multiplied by 28), where each
PixNcorresponds to the grayscale value of a pixel. Each pixel value ranges from 0 to 255, where 0 corresponds to a completely white pixel and 255 corresponds to a completely black pixel.
Structure of the Data:
- Rows: Each row in the dataset corresponds to a
single image (a single handwritten digit) along with its label.
- Columns: The first column is typically the
Labeland the remaining 784 columns are the pixel intensity values from the top-left to the bottom-right of the image.
Goal: The goal with MNIST is to build a
model that can predict the Label from the 784 pixel values,
effectively allowing a computer to recognize handwritten digits.
Instructions
- Load the necessary libraries that will be used for performing
k-Nearest Neighbors (kNN) classification and creating a confusion
matrix.
- Load the MNIST dataset from a file named “MN500.rds” and ensure that
the
Labelvariable is set as a factor, as it represents categorical data.
- Split the dataset into training and testing subsets, using 70% of the data for training and the remaining 30% for testing. Set a random seed for reproducibility.
- Extract the label column from both the training and test datasets
for later use in the kNN model.
- Remove the label column from the training and test datasets so that
only pixel values are used for the kNN algorithm.
- Use the
knnfunction to classify test data based on the training data, usingk = 5as the number of nearest neighbors.
- Use
CrossTablefrom thegmodelslibrary to construct a confusion matrix to compare the actual and predicted labels.
- Calculate the accuracy of the KNN predictions by comparing them to the true test labels, then print the accuracy as a percentage.
**Complete the code below by filling spaces (“___“).**
## Warning: package 'class' was built under R version 4.3.3## Warning: package 'gmodels' was built under R version 4.3.3# Load MNIST dataset
DataMnist <- readRDS("MN500.rds")
DataMnist$Label <- factor(DataMnist$Label)
# Create Training and Testing Data
set.seed(123)
index <- sample(1:nrow(DataMnist), 0.7 * nrow(DataMnist))
DataTrain <- DataMnist[index, ]
DataTest <- DataMnist[-index, ]
# Prepare the data for KNN (excluding the label column for training/testing dataset)
train_labels <- DataTrain$Label
test_labels <- DataTest$Label
# Remove the label column from the datasets for KNN
DataTrain <- DataTrain[, -1]
DataTest <- DataTest[,-1]
# Perform KNN for k = 5
predicted_labels <- knn(train = DataTrain, test = DataTest, cl = train_labels, k = 5)
print(predicted_labels)## [1] 7 1 1 9 4 6 1 9 7 3 4 0 7 1 1 3 1 4 4 0 1 9 1 4 7 4 0 7 1 1 3 1 1 6 9 6 5
## [38] 1 9 4 7 6 9 5 6 6 1 6 7 1 8 0 1 5 6 6 4 1 2 8 1 8 5 2 5 1 2 1 1 3 5 0 5 3
## [75] 5 2 4 1 0 2 1 2 5 3 9 9 7 9 0 8 1 0 6 9 3 0 1 5 3 9 7 6 5 1 0 1 9 5 6 7 4
## [112] 1 4 6 6 5 9 4 0 7 7 9 2 4 7 1 9 8 3 2 9 8 3 0 4 6 6 7 7 5 8 9 7 1 7 1 5 4
## [149] 0 6
## Levels: 0 1 2 3 4 5 6 7 8 9# Create Confusion Matrix using CrossTable
CrossTable(x = test_labels, y = predicted_labels, prop.chisq = FALSE)##
##
## Cell Contents
## |-------------------------|
## | N |
## | N / Row Total |
## | N / Col Total |
## | N / Table Total |
## |-------------------------|
##
##
## Total Observations in Table: 150
##
##
## | predicted_labels
## test_labels | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Row Total |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 0 | 12 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 13 |
## | 0.923 | 0.000 | 0.077 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.087 |
## | 0.923 | 0.000 | 0.125 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | |
## | 0.080 | 0.000 | 0.007 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 1 | 0 | 23 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 23 |
## | 0.000 | 1.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.153 |
## | 0.000 | 0.742 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | |
## | 0.000 | 0.153 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 2 | 0 | 2 | 6 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 10 |
## | 0.000 | 0.200 | 0.600 | 0.000 | 0.100 | 0.000 | 0.000 | 0.100 | 0.000 | 0.000 | 0.067 |
## | 0.000 | 0.065 | 0.750 | 0.000 | 0.067 | 0.000 | 0.000 | 0.059 | 0.000 | 0.000 | |
## | 0.000 | 0.013 | 0.040 | 0.000 | 0.007 | 0.000 | 0.000 | 0.007 | 0.000 | 0.000 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 3 | 0 | 1 | 0 | 9 | 0 | 0 | 0 | 1 | 0 | 0 | 11 |
## | 0.000 | 0.091 | 0.000 | 0.818 | 0.000 | 0.000 | 0.000 | 0.091 | 0.000 | 0.000 | 0.073 |
## | 0.000 | 0.032 | 0.000 | 0.900 | 0.000 | 0.000 | 0.000 | 0.059 | 0.000 | 0.000 | |
## | 0.000 | 0.007 | 0.000 | 0.060 | 0.000 | 0.000 | 0.000 | 0.007 | 0.000 | 0.000 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 4 | 0 | 0 | 1 | 0 | 10 | 0 | 0 | 0 | 0 | 2 | 13 |
## | 0.000 | 0.000 | 0.077 | 0.000 | 0.769 | 0.000 | 0.000 | 0.000 | 0.000 | 0.154 | 0.087 |
## | 0.000 | 0.000 | 0.125 | 0.000 | 0.667 | 0.000 | 0.000 | 0.000 | 0.000 | 0.118 | |
## | 0.000 | 0.000 | 0.007 | 0.000 | 0.067 | 0.000 | 0.000 | 0.000 | 0.000 | 0.013 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 5 | 0 | 2 | 0 | 1 | 2 | 12 | 1 | 0 | 0 | 0 | 18 |
## | 0.000 | 0.111 | 0.000 | 0.056 | 0.111 | 0.667 | 0.056 | 0.000 | 0.000 | 0.000 | 0.120 |
## | 0.000 | 0.065 | 0.000 | 0.100 | 0.133 | 0.800 | 0.059 | 0.000 | 0.000 | 0.000 | |
## | 0.000 | 0.013 | 0.000 | 0.007 | 0.013 | 0.080 | 0.007 | 0.000 | 0.000 | 0.000 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 6 | 1 | 0 | 0 | 0 | 0 | 0 | 16 | 0 | 0 | 0 | 17 |
## | 0.059 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.941 | 0.000 | 0.000 | 0.000 | 0.113 |
## | 0.077 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.941 | 0.000 | 0.000 | 0.000 | |
## | 0.007 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.107 | 0.000 | 0.000 | 0.000 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 7 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 13 | 0 | 0 | 13 |
## | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 1.000 | 0.000 | 0.000 | 0.087 |
## | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.765 | 0.000 | 0.000 | |
## | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.087 | 0.000 | 0.000 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 8 | 0 | 2 | 0 | 0 | 1 | 3 | 0 | 0 | 6 | 1 | 13 |
## | 0.000 | 0.154 | 0.000 | 0.000 | 0.077 | 0.231 | 0.000 | 0.000 | 0.462 | 0.077 | 0.087 |
## | 0.000 | 0.065 | 0.000 | 0.000 | 0.067 | 0.200 | 0.000 | 0.000 | 0.857 | 0.059 | |
## | 0.000 | 0.013 | 0.000 | 0.000 | 0.007 | 0.020 | 0.000 | 0.000 | 0.040 | 0.007 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## 9 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 2 | 1 | 14 | 19 |
## | 0.000 | 0.053 | 0.000 | 0.000 | 0.053 | 0.000 | 0.000 | 0.105 | 0.053 | 0.737 | 0.127 |
## | 0.000 | 0.032 | 0.000 | 0.000 | 0.067 | 0.000 | 0.000 | 0.118 | 0.143 | 0.824 | |
## | 0.000 | 0.007 | 0.000 | 0.000 | 0.007 | 0.000 | 0.000 | 0.013 | 0.007 | 0.093 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
## Column Total | 13 | 31 | 8 | 10 | 15 | 15 | 17 | 17 | 7 | 17 | 150 |
## | 0.087 | 0.207 | 0.053 | 0.067 | 0.100 | 0.100 | 0.113 | 0.113 | 0.047 | 0.113 | |
## -------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
##
## # Calculate accuracy
accuracy <- mean(predicted_labels == test_labels)
print(paste("Accuracy:", round(accuracy * 100, 2), "%"))## [1] "Accuracy: 80.67 %"Questions 2 (18 pts)
The Titanic dataset is a classic dataset in data science and machine learning typically used for demonstrating classification tasks. This dataset is available in the file `Titanic.csv1. Here’s a general description of the Titanic dataset and its variables:
Description of the Titanic Dataset:
The dataset contains data about the passengers who were onboard the ill-fated RMS Titanic. Your goal is to predict the survival of the passengers based on various features. Below are the variables you will use in this dataset:
Survived: Indicates if the passenger Survived (1) or did not survive (0).
Class: Passenger class, a proxy for socio-economic status (1 = 1st class, 2 = 2nd class, 3 = 3rd class).
Name: Full name of the passenger.
Sex: Gender of the passenger (male or female).
Age: Age of the passenger in years. Some entries may have missing ages.
SibSp: Number of siblings and spouses aboard the Titanic.
Parch: Number of parents and children aboard the Titanic.
Fare: Passenger fare.
Instructions
- Load the
rpartlibrary for decision tree modeling and thegmodelslibrary for creating a confusion matrix.
- Filter the dataset to include only the relevant columns: “Survived”,
“Sex”, “Class”, “Age”, and “Fare”.
- Convert the “Survived” column to a factor, as it represents
categorical binary outcomes.
- Set a random seed for reproducibility, and split the dataset into
training and test sets. Use 75% of the data for training and 25% for
testing.
- Train a decision tree model using the
rpartfunction. Predict “Survived” based on all other features, with a maximum depth for tree branches set to 3 for simplicity.
- Visualize the trained decision tree using the
rpart.plotfunction, customizing the yes/no labels on each node.
- Use the trained model to predict outcomes for the test data,
resulting in predicted class labels.
- Use the
CrossTablefunction from thegmodelspackage to generate a confusion matrix comparing actual and predicted outcomes.
- Calculate the accuracy of the predictions by comparing predicted labels against actual labels. Print the accuracy percentage.
**Complete the code below by filling spaces (“___“).**
# Load necessary libraries
library(rpart)
#install.packages('gmodels')
library(gmodels)
# Load the dataset; assuming the Titanic dataset
Titanic <- read.csv("Titanic.csv")
# Select relevant columns
Titanic <- Titanic[,c("Survived", "Sex", "Class", "Age", "Fare")]
# Factorize the Survived column
Titanic$Survived <- as.factor(Titanic$Survived)
# Set seed and split the data
set.seed(777)
train_indices <- sample(1:nrow(Titanic), 0.75 * nrow(Titanic))
DataTrain <- Titanic[train_indices, ]
DataTest <- Titanic[-train_indices, ]
# Train the decision tree model
ModelDesignDecTree <- rpart(Survived ~ ., data = DataTrain, method = "class", control = rpart.control(cp=0, maxdepth = 3))
# Visualize the decision tree
library(rpart.plot)## Warning: package 'rpart.plot' was built under R version 4.3.3#install.packages('rpart.plot')
rpart.plot(ModelDesignDecTree, yes.text = "YES", no.text = "NO", roundint = FALSE)
# Predict on the test set
predicted_labels <- predict(ModelDesignDecTree, DataTest, type = "class")
# Create confusion matrix using CrossTable
CrossTable(x = DataTest$Survived, y = predicted_labels , prop.chisq = FALSE)##
##
## Cell Contents
## |-------------------------|
## | N |
## | N / Row Total |
## | N / Col Total |
## | N / Table Total |
## |-------------------------|
##
##
## Total Observations in Table: 222
##
##
## | predicted_labels
## DataTest$Survived | 0 | 1 | Row Total |
## ------------------|-----------|-----------|-----------|
## 0 | 115 | 12 | 127 |
## | 0.906 | 0.094 | 0.572 |
## | 0.846 | 0.140 | |
## | 0.518 | 0.054 | |
## ------------------|-----------|-----------|-----------|
## 1 | 21 | 74 | 95 |
## | 0.221 | 0.779 | 0.428 |
## | 0.154 | 0.860 | |
## | 0.095 | 0.333 | |
## ------------------|-----------|-----------|-----------|
## Column Total | 136 | 86 | 222 |
## | 0.613 | 0.387 | |
## ------------------|-----------|-----------|-----------|
##
## # Calculate accuracy
accuracy <- mean(predicted_labels == DataTest$Survived)
print(paste(accuracy, round(accuracy * 100, 2), "%"))## [1] "0.851351351351351 85.14 %"Based on the plot of the decision tree, Asnswer the following questions by typing your answer below each question.
```
Based on the plot of the decision tree, Asnswer the following questions by typing your answer below each question.
- What is the survival rate for adult male passengers 13 years or older, regardless of the class they traveled in and the fare they paid?
Answer: 46%
- What is the survival rate for young male passengers (younger than 13 years), regardless of which class they traveled and the fare they paid?
Answer: 2%
- What is the survival rate for young male passengers (younger than 13 years) traveling in Third Class regardless of the fare they paid?
Answer: 2%
- What is the survival rate for female passengers, regardless of age and not considering the class they traveled in or the fare they paid?
Answer: 34%
- When considering the class female passengers traveled in, we can see female passengers, regardless of age, had a survival rate of ___ when they traveled in First or Second Class regardless of the fare they paid.
Answer: 18%
LS0tDQp0aXRsZTogIkJBTkwgMzIwMDogU3VwZXJ2aXNlcyBNYWNoaW5lIExlYXJuaW5nIg0Kc3VidGl0bGU6ICJNaWR0ZXJtIg0KYXV0aG9yOiAiSmFkZW4gU2FtcHNvbiINCmRhdGU6ICJgciBTeXMuRGF0ZSgpYCINCm91dHB1dDogb3BlbmludHJvOjpsYWJfcmVwb3J0DQotLS0NCg0KYGBge3IsIHNldHVwLCBpbmNsdWRlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQ0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KA0KICBldmFsID0gVFJVRQ0KKQ0KYGBgDQoNCmBgYHtyIGxvYWQtcGFja2FnZXMsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9DQpsaWJyYXJ5KHRpZHl2ZXJzZSkNCmxpYnJhcnkob3BlbmludHJvKQ0KYGBgDQoNCiMjIyBJbnN0cnVjdGlvbnMNCg0KLSBDb21wbGV0ZSB0aGUgY29kZWQgaW4gdGhlIGNvZGUgY2h1bmtzLg0KLSBUeXBlIHRoZSBhbnN3ZXJzIGFzIHRleHQgYmVsb3cgZWFjaCBxdWVzdGlvbi4NCi0gS25pdCB5b3VyIGZpbGUuDQotIFVwbG9hZCByZXN1bHRpbmcgSFRNTCBmaWxlIHRvIHRoZSBDYW52YXMNCg0KDQojIyMgUXVlc3Rpb25zIDEgKDEyIHB0cykNCg0KVGhlIE1OSVNUIGRhdGFzZXQgaXMgb25lIG9mIHRoZSBtb3N0IHdlbGwta25vd24gZGF0YXNldHMgaW4gdGhlIGZpZWxkIG9mIG1hY2hpbmUgbGVhcm5pbmcgYW5kIGlzIHdpZGVseSB1c2VkIGZvciB0cmFpbmluZyB2YXJpb3VzIGltYWdlIHByb2Nlc3Npbmcgc3lzdGVtcy4gSXQgY29udGFpbnMgaW1hZ2VzIG9mIGhhbmR3cml0dGVuIGRpZ2l0cywgd2l0aCBlYWNoIGltYWdlIGJlaW5nIGEgMjh4MjggcGl4ZWwgZ3JheXNjYWxlIGltYWdlLiBFYWNoIHBpeGVsIHJhbmdlcyBmcm9tIDAgKHdoaXRlKSB0byAyNTUgKGJsYWNrKS4gVGhlIGRhdGFzZXQgaXMgc3RydWN0dXJlZCBhcyBmb2xsb3dzOg0KDQotICoqTGFiZWwqKjogVGhpcyB2YXJpYWJsZSBkZW5vdGVzIHRoZSBjbGFzcyBvZiB0aGUgaGFuZHdyaXR0ZW4gZGlnaXQgaW1hZ2UuIEl0IHJlcHJlc2VudHMgdGhlIGRpZ2l0IGRlcGljdGVkIGluIHRoZSBpbWFnZSAoMCB0aHJvdWdoIDkpLiBJbiBzb21lIHZhcmlhdGlvbnMgb2YgdGhlIE1OSVNUIGRhdGFzZXQgKHRob3VnaCBub3Qgc3RhbmRhcmQpLCB0aGUgbGFiZWwgbWlnaHQgcmVwcmVzZW50IGxldHRlcnMgaWYgaXTigJlzIG1vZGlmaWVkIGFuZCBleHRlbmRlZCB0byBhbHBoYWJldHMuICANCg0KLSAqKlBpeDEsIFBpeDIsIC4uLiwgUGl4Nzg0Kio6IFRoZXNlIHZhcmlhYmxlcyByZXByZXNlbnQgdGhlIHBpeGVsIHZhbHVlcyBvZiB0aGUgMjh4MjggcGl4ZWwgaW1hZ2VzLiBFYWNoIGltYWdlIGlzICJmbGF0dGVuZWQiIGludG8gYSBzaW5nbGUgcm93IHdpdGggNzg0IGNvbHVtbnMgKDI4IG11bHRpcGxpZWQgYnkgMjgpLCB3aGVyZSBlYWNoIGBQaXhOYCBjb3JyZXNwb25kcyB0byB0aGUgZ3JheXNjYWxlIHZhbHVlIG9mIGEgcGl4ZWwuIEVhY2ggcGl4ZWwgdmFsdWUgcmFuZ2VzIGZyb20gMCB0byAyNTUsIHdoZXJlIDAgY29ycmVzcG9uZHMgdG8gYSBjb21wbGV0ZWx5IHdoaXRlIHBpeGVsIGFuZCAyNTUgY29ycmVzcG9uZHMgdG8gYSBjb21wbGV0ZWx5IGJsYWNrIHBpeGVsLg0KDQoqKipTdHJ1Y3R1cmUgb2YgdGhlIERhdGE6KioqDQoNCi0gKipSb3dzKio6IEVhY2ggcm93IGluIHRoZSBkYXRhc2V0IGNvcnJlc3BvbmRzIHRvIGEgc2luZ2xlIGltYWdlIChhIHNpbmdsZSBoYW5kd3JpdHRlbiBkaWdpdCkgYWxvbmcgd2l0aCBpdHMgbGFiZWwuICANCi0gKipDb2x1bW5zKio6IFRoZSBmaXJzdCBjb2x1bW4gaXMgdHlwaWNhbGx5IHRoZSBgTGFiZWxgIGFuZCB0aGUgcmVtYWluaW5nIDc4NCBjb2x1bW5zIGFyZSB0aGUgcGl4ZWwgaW50ZW5zaXR5IHZhbHVlcyBmcm9tIHRoZSB0b3AtbGVmdCB0byB0aGUgYm90dG9tLXJpZ2h0IG9mIHRoZSBpbWFnZS4NCg0KKioqR29hbDoqKiogDQpUaGUgZ29hbCB3aXRoIE1OSVNUIGlzIHRvIGJ1aWxkIGEgbW9kZWwgdGhhdCBjYW4gcHJlZGljdCB0aGUgYExhYmVsYCBmcm9tIHRoZSA3ODQgcGl4ZWwgdmFsdWVzLCBlZmZlY3RpdmVseSBhbGxvd2luZyBhIGNvbXB1dGVyIHRvIHJlY29nbml6ZSBoYW5kd3JpdHRlbiBkaWdpdHMuDQoNCiMjIyMgSW5zdHJ1Y3Rpb25zDQoNCjEuIExvYWQgdGhlIG5lY2Vzc2FyeSBsaWJyYXJpZXMgdGhhdCB3aWxsIGJlIHVzZWQgZm9yIHBlcmZvcm1pbmcgay1OZWFyZXN0IE5laWdoYm9ycyAoa05OKSBjbGFzc2lmaWNhdGlvbiBhbmQgY3JlYXRpbmcgYSBjb25mdXNpb24gbWF0cml4LiAgDQoyLiBMb2FkIHRoZSBNTklTVCBkYXRhc2V0IGZyb20gYSBmaWxlIG5hbWVkICJNTjUwMC5yZHMiIGFuZCBlbnN1cmUgdGhhdCB0aGUgYExhYmVsYCB2YXJpYWJsZSBpcyBzZXQgYXMgYSBmYWN0b3IsIGFzIGl0IHJlcHJlc2VudHMgY2F0ZWdvcmljYWwgZGF0YS4gIA0KMy4gU3BsaXQgdGhlIGRhdGFzZXQgaW50byB0cmFpbmluZyBhbmQgdGVzdGluZyBzdWJzZXRzLCB1c2luZyA3MCUgb2YgdGhlIGRhdGEgZm9yIHRyYWluaW5nIGFuZCB0aGUgcmVtYWluaW5nIDMwJSBmb3IgdGVzdGluZy4gU2V0IGEgcmFuZG9tIHNlZWQgZm9yIHJlcHJvZHVjaWJpbGl0eS4NCjQuIEV4dHJhY3QgdGhlIGxhYmVsIGNvbHVtbiBmcm9tIGJvdGggdGhlIHRyYWluaW5nIGFuZCB0ZXN0IGRhdGFzZXRzIGZvciBsYXRlciB1c2UgaW4gdGhlIGtOTiBtb2RlbC4gIA0KNS4gUmVtb3ZlIHRoZSBsYWJlbCBjb2x1bW4gZnJvbSB0aGUgdHJhaW5pbmcgYW5kIHRlc3QgZGF0YXNldHMgc28gdGhhdCBvbmx5IHBpeGVsIHZhbHVlcyBhcmUgdXNlZCBmb3IgdGhlIGtOTiBhbGdvcml0aG0uICANCjYuIFVzZSB0aGUgYGtubmAgZnVuY3Rpb24gdG8gY2xhc3NpZnkgdGVzdCBkYXRhIGJhc2VkIG9uIHRoZSB0cmFpbmluZyBkYXRhLCB1c2luZyBgayA9IDVgIGFzIHRoZSBudW1iZXIgb2YgbmVhcmVzdCBuZWlnaGJvcnMuICANCjcuIFVzZSBgQ3Jvc3NUYWJsZWAgZnJvbSB0aGUgYGdtb2RlbHNgIGxpYnJhcnkgdG8gY29uc3RydWN0IGEgY29uZnVzaW9uIG1hdHJpeCB0byBjb21wYXJlIHRoZSBhY3R1YWwgYW5kIHByZWRpY3RlZCBsYWJlbHMuICANCjguIENhbGN1bGF0ZSB0aGUgYWNjdXJhY3kgb2YgdGhlIEtOTiBwcmVkaWN0aW9ucyBieSBjb21wYXJpbmcgdGhlbSB0byB0aGUgdHJ1ZSB0ZXN0IGxhYmVscywgdGhlbiBwcmludCB0aGUgYWNjdXJhY3kgYXMgYSBwZXJjZW50YWdlLiAgDQoNCg0KKipDb21wbGV0ZSB0aGUgY29kZSBiZWxvdyBieSBmaWxsaW5nIHNwYWNlcyAoIl9fXyIpLioqDQoNCg0KYGBge3IgUTF9DQojIExvYWQgbmVjZXNzYXJ5IGxpYnJhcmllcw0KbGlicmFyeShjbGFzcykNCmxpYnJhcnkoZ21vZGVscykNCg0KIyBMb2FkIE1OSVNUIGRhdGFzZXQNCkRhdGFNbmlzdCA8LSByZWFkUkRTKCJNTjUwMC5yZHMiKQ0KRGF0YU1uaXN0JExhYmVsIDwtIGZhY3RvcihEYXRhTW5pc3QkTGFiZWwpDQoNCiMgQ3JlYXRlIFRyYWluaW5nIGFuZCBUZXN0aW5nIERhdGENCnNldC5zZWVkKDEyMykNCmluZGV4IDwtIHNhbXBsZSgxOm5yb3coRGF0YU1uaXN0KSwgMC43ICogbnJvdyhEYXRhTW5pc3QpKQ0KRGF0YVRyYWluIDwtIERhdGFNbmlzdFtpbmRleCwgXQ0KRGF0YVRlc3QgPC0gRGF0YU1uaXN0Wy1pbmRleCwgXQ0KDQojIFByZXBhcmUgdGhlIGRhdGEgZm9yIEtOTiAoZXhjbHVkaW5nIHRoZSBsYWJlbCBjb2x1bW4gZm9yIHRyYWluaW5nL3Rlc3RpbmcgZGF0YXNldCkNCnRyYWluX2xhYmVscyA8LSBEYXRhVHJhaW4kTGFiZWwNCnRlc3RfbGFiZWxzIDwtIERhdGFUZXN0JExhYmVsDQoNCiMgUmVtb3ZlIHRoZSBsYWJlbCBjb2x1bW4gZnJvbSB0aGUgZGF0YXNldHMgZm9yIEtOTg0KRGF0YVRyYWluIDwtIERhdGFUcmFpblssIC0xXQ0KRGF0YVRlc3QgPC0gRGF0YVRlc3RbLC0xXQ0KDQojIFBlcmZvcm0gS05OIGZvciBrID0gNQ0KcHJlZGljdGVkX2xhYmVscyA8LSBrbm4odHJhaW4gPSBEYXRhVHJhaW4sIHRlc3QgPSBEYXRhVGVzdCwgY2wgPSB0cmFpbl9sYWJlbHMsIGsgPSA1KQ0KcHJpbnQocHJlZGljdGVkX2xhYmVscykNCg0KIyBDcmVhdGUgQ29uZnVzaW9uIE1hdHJpeCB1c2luZyBDcm9zc1RhYmxlDQpDcm9zc1RhYmxlKHggPSB0ZXN0X2xhYmVscywgeSA9IHByZWRpY3RlZF9sYWJlbHMsIHByb3AuY2hpc3EgPSBGQUxTRSkNCg0KIyBDYWxjdWxhdGUgYWNjdXJhY3kNCmFjY3VyYWN5IDwtIG1lYW4ocHJlZGljdGVkX2xhYmVscyA9PSB0ZXN0X2xhYmVscykNCnByaW50KHBhc3RlKCJBY2N1cmFjeToiLCByb3VuZChhY2N1cmFjeSAqIDEwMCwgMiksICIlIikpDQpgYGANCg0KDQojIyMgUXVlc3Rpb25zIDIgKDE4IHB0cykNCg0KVGhlIFRpdGFuaWMgZGF0YXNldCBpcyBhIGNsYXNzaWMgZGF0YXNldCBpbiBkYXRhIHNjaWVuY2UgYW5kIG1hY2hpbmUgbGVhcm5pbmcgdHlwaWNhbGx5IHVzZWQgZm9yIGRlbW9uc3RyYXRpbmcgY2xhc3NpZmljYXRpb24gdGFza3MuIFRoaXMgZGF0YXNldCBpcyBhdmFpbGFibGUgaW4gdGhlIGZpbGUgYFRpdGFuaWMuY3N2MS4gSGVyZSdzIGEgZ2VuZXJhbCBkZXNjcmlwdGlvbiBvZiB0aGUgVGl0YW5pYyBkYXRhc2V0IGFuZCBpdHMgdmFyaWFibGVzOg0KDQojIyMgRGVzY3JpcHRpb24gb2YgdGhlIFRpdGFuaWMgRGF0YXNldDoNCg0KVGhlIGRhdGFzZXQgY29udGFpbnMgZGF0YSBhYm91dCB0aGUgcGFzc2VuZ2VycyB3aG8gd2VyZSBvbmJvYXJkIHRoZSBpbGwtZmF0ZWQgUk1TIFRpdGFuaWMuIFlvdXIgZ29hbCBpcyB0byBwcmVkaWN0IHRoZSBzdXJ2aXZhbCBvZiB0aGUgcGFzc2VuZ2VycyBiYXNlZCBvbiB2YXJpb3VzIGZlYXR1cmVzLiBCZWxvdyBhcmUgdGhlIHZhcmlhYmxlcyB5b3Ugd2lsbCB1c2UgaW4gdGhpcyBkYXRhc2V0Og0KDQoqKlN1cnZpdmVkKio6IEluZGljYXRlcyBpZiB0aGUgcGFzc2VuZ2VyIFN1cnZpdmVkICgxKSBvciBkaWQgbm90IHN1cnZpdmUgKDApLg0KDQoqKkNsYXNzKio6IFBhc3NlbmdlciBjbGFzcywgYSBwcm94eSBmb3Igc29jaW8tZWNvbm9taWMgc3RhdHVzICgxID0gMXN0IGNsYXNzLCAyID0gMm5kIGNsYXNzLCAzID0gM3JkIGNsYXNzKS4NCg0KKipOYW1lKio6IEZ1bGwgbmFtZSBvZiB0aGUgcGFzc2VuZ2VyLg0KDQoqKlNleCoqOiBHZW5kZXIgb2YgdGhlIHBhc3NlbmdlciAobWFsZSBvciBmZW1hbGUpLg0KDQoqKkFnZSoqOiBBZ2Ugb2YgdGhlIHBhc3NlbmdlciBpbiB5ZWFycy4gU29tZSBlbnRyaWVzIG1heSBoYXZlIG1pc3NpbmcgYWdlcy4NCg0KKipTaWJTcCoqOiBOdW1iZXIgb2Ygc2libGluZ3MgYW5kIHNwb3VzZXMgYWJvYXJkIHRoZSBUaXRhbmljLg0KDQoqKlBhcmNoKio6IE51bWJlciBvZiBwYXJlbnRzIGFuZCBjaGlsZHJlbiBhYm9hcmQgdGhlIFRpdGFuaWMuDQoNCioqRmFyZSoqOiBQYXNzZW5nZXIgZmFyZS4NCg0KIyMjIyBJbnN0cnVjdGlvbnMNCg0KMS4gTG9hZCB0aGUgYHJwYXJ0YCBsaWJyYXJ5IGZvciBkZWNpc2lvbiB0cmVlIG1vZGVsaW5nIGFuZCB0aGUgYGdtb2RlbHNgIGxpYnJhcnkgZm9yIGNyZWF0aW5nIGEgY29uZnVzaW9uIG1hdHJpeC4gIA0KMi4gRmlsdGVyIHRoZSBkYXRhc2V0IHRvIGluY2x1ZGUgb25seSB0aGUgcmVsZXZhbnQgY29sdW1uczogIlN1cnZpdmVkIiwgIlNleCIsICJDbGFzcyIsICJBZ2UiLCBhbmQgIkZhcmUiLiAgDQozLiBDb252ZXJ0IHRoZSAiU3Vydml2ZWQiIGNvbHVtbiB0byBhIGZhY3RvciwgYXMgaXQgcmVwcmVzZW50cyBjYXRlZ29yaWNhbCBiaW5hcnkgb3V0Y29tZXMuICANCjQuIFNldCBhIHJhbmRvbSBzZWVkIGZvciByZXByb2R1Y2liaWxpdHksIGFuZCBzcGxpdCB0aGUgZGF0YXNldCBpbnRvIHRyYWluaW5nIGFuZCB0ZXN0IHNldHMuIFVzZSA3NSUgb2YgdGhlIGRhdGEgZm9yIHRyYWluaW5nIGFuZCAyNSUgZm9yIHRlc3RpbmcuICANCjUuIFRyYWluIGEgZGVjaXNpb24gdHJlZSBtb2RlbCB1c2luZyB0aGUgYHJwYXJ0YCBmdW5jdGlvbi4gUHJlZGljdCAiU3Vydml2ZWQiIGJhc2VkIG9uIGFsbCBvdGhlciBmZWF0dXJlcywgd2l0aCBhIG1heGltdW0gZGVwdGggZm9yIHRyZWUgYnJhbmNoZXMgc2V0IHRvIDMgZm9yIHNpbXBsaWNpdHkuICANCjYuIFZpc3VhbGl6ZSB0aGUgdHJhaW5lZCBkZWNpc2lvbiB0cmVlIHVzaW5nIHRoZSBgcnBhcnQucGxvdGAgZnVuY3Rpb24sIGN1c3RvbWl6aW5nIHRoZSB5ZXMvbm8gbGFiZWxzIG9uIGVhY2ggbm9kZS4gIA0KNy4gVXNlIHRoZSB0cmFpbmVkIG1vZGVsIHRvIHByZWRpY3Qgb3V0Y29tZXMgZm9yIHRoZSB0ZXN0IGRhdGEsIHJlc3VsdGluZyBpbiBwcmVkaWN0ZWQgY2xhc3MgbGFiZWxzLiAgDQo4LiBVc2UgdGhlIGBDcm9zc1RhYmxlYCBmdW5jdGlvbiBmcm9tIHRoZSBgZ21vZGVsc2AgcGFja2FnZSB0byBnZW5lcmF0ZSBhIGNvbmZ1c2lvbiBtYXRyaXggY29tcGFyaW5nIGFjdHVhbCBhbmQgcHJlZGljdGVkIG91dGNvbWVzLiAgDQo5LiBDYWxjdWxhdGUgdGhlIGFjY3VyYWN5IG9mIHRoZSBwcmVkaWN0aW9ucyBieSBjb21wYXJpbmcgcHJlZGljdGVkIGxhYmVscyBhZ2FpbnN0IGFjdHVhbCBsYWJlbHMuIFByaW50IHRoZSBhY2N1cmFjeSBwZXJjZW50YWdlLiAgDQoNCioqQ29tcGxldGUgdGhlIGNvZGUgYmVsb3cgYnkgZmlsbGluZyBzcGFjZXMgKCJfX18iKS4qKg0KDQoNCmBgYHtyIFEyfQ0KIyBMb2FkIG5lY2Vzc2FyeSBsaWJyYXJpZXMNCmxpYnJhcnkocnBhcnQpDQojaW5zdGFsbC5wYWNrYWdlcygnZ21vZGVscycpDQpsaWJyYXJ5KGdtb2RlbHMpDQoNCiMgTG9hZCB0aGUgZGF0YXNldDsgYXNzdW1pbmcgdGhlIFRpdGFuaWMgZGF0YXNldA0KVGl0YW5pYyA8LSByZWFkLmNzdigiVGl0YW5pYy5jc3YiKQ0KDQojIFNlbGVjdCByZWxldmFudCBjb2x1bW5zDQpUaXRhbmljIDwtIFRpdGFuaWNbLGMoIlN1cnZpdmVkIiwgIlNleCIsICJDbGFzcyIsICJBZ2UiLCAiRmFyZSIpXQ0KDQojIEZhY3Rvcml6ZSB0aGUgU3Vydml2ZWQgY29sdW1uDQpUaXRhbmljJFN1cnZpdmVkIDwtIGFzLmZhY3RvcihUaXRhbmljJFN1cnZpdmVkKQ0KDQojIFNldCBzZWVkIGFuZCBzcGxpdCB0aGUgZGF0YQ0Kc2V0LnNlZWQoNzc3KQ0KdHJhaW5faW5kaWNlcyA8LSBzYW1wbGUoMTpucm93KFRpdGFuaWMpLCAwLjc1ICogbnJvdyhUaXRhbmljKSkNCkRhdGFUcmFpbiA8LSBUaXRhbmljW3RyYWluX2luZGljZXMsIF0NCkRhdGFUZXN0IDwtIFRpdGFuaWNbLXRyYWluX2luZGljZXMsIF0NCg0KIyBUcmFpbiB0aGUgZGVjaXNpb24gdHJlZSBtb2RlbA0KTW9kZWxEZXNpZ25EZWNUcmVlIDwtIHJwYXJ0KFN1cnZpdmVkIH4gLiwgZGF0YSA9IERhdGFUcmFpbiwgbWV0aG9kID0gImNsYXNzIiwgY29udHJvbCA9IHJwYXJ0LmNvbnRyb2woY3A9MCwgbWF4ZGVwdGggPSAzKSkNCg0KIyBWaXN1YWxpemUgdGhlIGRlY2lzaW9uIHRyZWUNCmxpYnJhcnkocnBhcnQucGxvdCkNCiNpbnN0YWxsLnBhY2thZ2VzKCdycGFydC5wbG90JykNCnJwYXJ0LnBsb3QoTW9kZWxEZXNpZ25EZWNUcmVlLCB5ZXMudGV4dCA9ICJZRVMiLCBuby50ZXh0ID0gIk5PIiwgcm91bmRpbnQgPSBGQUxTRSkNCg0KIyBQcmVkaWN0IG9uIHRoZSB0ZXN0IHNldA0KcHJlZGljdGVkX2xhYmVscyA8LSBwcmVkaWN0KE1vZGVsRGVzaWduRGVjVHJlZSwgRGF0YVRlc3QsIHR5cGUgPSAiY2xhc3MiKQ0KDQojIENyZWF0ZSBjb25mdXNpb24gbWF0cml4IHVzaW5nIENyb3NzVGFibGUNCkNyb3NzVGFibGUoeCA9IERhdGFUZXN0JFN1cnZpdmVkLCB5ID0gcHJlZGljdGVkX2xhYmVscyAsIHByb3AuY2hpc3EgPSBGQUxTRSkNCg0KIyBDYWxjdWxhdGUgYWNjdXJhY3kNCmFjY3VyYWN5IDwtIG1lYW4ocHJlZGljdGVkX2xhYmVscyA9PSBEYXRhVGVzdCRTdXJ2aXZlZCkNCnByaW50KHBhc3RlKGFjY3VyYWN5LCByb3VuZChhY2N1cmFjeSAqIDEwMCwgMiksICIlIikpDQpgYGANCg0KKipCYXNlZCBvbiB0aGUgcGxvdCBvZiB0aGUgZGVjaXNpb24gdHJlZSwgQXNuc3dlciB0aGUgZm9sbG93aW5nIHF1ZXN0aW9ucyBieSB0eXBpbmcgeW91ciBhbnN3ZXIgYmVsb3cgZWFjaCBxdWVzdGlvbi4qKg0KDQoNCg0KYGBgDQoNCioqQmFzZWQgb24gdGhlIHBsb3Qgb2YgdGhlIGRlY2lzaW9uIHRyZWUsIEFzbnN3ZXIgdGhlIGZvbGxvd2luZyBxdWVzdGlvbnMgYnkgdHlwaW5nIHlvdXIgYW5zd2VyIGJlbG93IGVhY2ggcXVlc3Rpb24uKioNCg0KMS4gV2hhdCBpcyB0aGUgc3Vydml2YWwgcmF0ZSBmb3IgYWR1bHQgbWFsZSBwYXNzZW5nZXJzIDEzIHllYXJzIG9yIG9sZGVyLCByZWdhcmRsZXNzIG9mIHRoZSBjbGFzcyB0aGV5IHRyYXZlbGVkIGluIGFuZCB0aGUgZmFyZSB0aGV5IHBhaWQ/DQoNCkFuc3dlcjogNDYlDQoNCjIuIFdoYXQgaXMgdGhlIHN1cnZpdmFsIHJhdGUgZm9yIHlvdW5nIG1hbGUgcGFzc2VuZ2VycyAoeW91bmdlciB0aGFuIDEzIHllYXJzKSwgcmVnYXJkbGVzcyBvZiB3aGljaCBjbGFzcyB0aGV5IHRyYXZlbGVkIGFuZCB0aGUgZmFyZSB0aGV5IHBhaWQ/DQoNCkFuc3dlcjogMiUNCg0KMy4gV2hhdCBpcyB0aGUgc3Vydml2YWwgcmF0ZSBmb3IgeW91bmcgbWFsZSBwYXNzZW5nZXJzICh5b3VuZ2VyIHRoYW4gMTMgeWVhcnMpIHRyYXZlbGluZyBpbiBUaGlyZCBDbGFzcyByZWdhcmRsZXNzIG9mIHRoZSBmYXJlIHRoZXkgcGFpZD8NCg0KQW5zd2VyOiAyJQ0KDQo0LiBXaGF0IGlzIHRoZSBzdXJ2aXZhbCByYXRlIGZvciBmZW1hbGUgcGFzc2VuZ2VycywgcmVnYXJkbGVzcyBvZiBhZ2UgYW5kIG5vdCBjb25zaWRlcmluZyB0aGUgY2xhc3MgdGhleSB0cmF2ZWxlZCBpbiBvciB0aGUgZmFyZSB0aGV5IHBhaWQ/DQoNCkFuc3dlcjogMzQlDQoNCjUuIFdoZW4gY29uc2lkZXJpbmcgdGhlIGNsYXNzIGZlbWFsZSBwYXNzZW5nZXJzIHRyYXZlbGVkIGluLCB3ZSBjYW4gc2VlIGZlbWFsZSBwYXNzZW5nZXJzLCByZWdhcmRsZXNzIG9mIGFnZSwgaGFkIGEgc3Vydml2YWwgcmF0ZSBvZiBfX18gd2hlbiB0aGV5IHRyYXZlbGVkIGluIEZpcnN0IG9yIFNlY29uZCBDbGFzcyByZWdhcmRsZXNzIG9mIHRoZSBmYXJlIHRoZXkgcGFpZC4NCg0KQW5zd2VyOiAxOCUNCg0KDQoNCg==