Exercise 1

Use the nnet package to analyze the iris data set. Use 80% of the 150 samples as the training data and the rest for validation. Discuss the results.

data(iris)
set.seed(123)
samples <- sample(nrow(iris), nrow(iris)*0.80)
train <- iris[samples,]
test <- iris[-samples,]


iris_nn <- nnet(Species ~ ., size = 2,data=train)
## # weights:  19
## initial  value 135.090792 
## iter  10 value 58.046705
## iter  20 value 16.784114
## iter  30 value 3.034567
## iter  40 value 0.346292
## iter  50 value 0.000433
## iter  60 value 0.000159
## final  value 0.000100 
## converged
table(test$Species,predict(iris_nn,test,type='class'))
##             
##              setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0         14         1
##   virginica       0          0         5

The true positive rate is 96% and only 1 data point was incorrectly classified.

Exercise 2

As a mini project, install the keras package and learn how to use it. Then, carry out various tasks that may be useful to your project and studies.

install_tensorflow()
## 
## Installation complete.
install_keras()
## 
## Installation complete.

There are many articles online with great explanations of how to use keras using the mnist dataset. I’m going to replicate some of those studies here to learn the package.

The mnist dataset is a set of 60,000 training images and 10,000 testing images. Each image represents a number. The y-label represents the number in the image.

library(keras)
mnist <- dataset_mnist()

images_training  <- mnist$train$x
number_training <- mnist$train$y
images_testing  <- mnist$test$x
number_testing <- mnist$test$y

Here is an example of what the image looks like:

image <- images_training[173,,] 
number <- number_training[173]
print("The label for the below is image is:",str(number))
##  int 9
## [1] "The label for the below is image is:"
plot(as.raster(image, max = 255))

Here we are flattening the data to be 60,000 rows by 784 instead of the original 60,000 x 28 x 28:

dim(images_training) <- c(nrow(images_training),dim(images_training)[2]*dim(images_training)[3])
dim(images_testing) <- c(nrow(images_testing),dim(images_testing)[2]*dim(images_testing)[3])


number_training <- to_categorical(number_training, 10)
number_testing <- to_categorical(number_testing, 10)

Here we are setting up 3 layers using the ReLU activation function for the first 2 followed by softmax. The shape of the network must be specified in the first layers. In our previous step we reshaped our data from 28 by 28 to a single object of length 784. You must pass in the shape of the data into the first layer.

model <- keras_model_sequential() 
model %>% 
  layer_dense(units = 256, activation = "relu", input_shape = c(784)) %>% 
  layer_dense(units = 128, activation = "relu") %>%
  layer_dense(units = 10, activation = "softmax")

summary(model)
## Model: "sequential"
## ________________________________________________________________________________
## Layer (type)                        Output Shape                    Param #     
## ================================================================================
## dense_2 (Dense)                     (None, 256)                     200960      
## ________________________________________________________________________________
## dense_1 (Dense)                     (None, 128)                     32896       
## ________________________________________________________________________________
## dense (Dense)                       (None, 10)                      1290        
## ================================================================================
## Total params: 235,146
## Trainable params: 235,146
## Non-trainable params: 0
## ________________________________________________________________________________
model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_rmsprop(),
  metrics = c("accuracy")
)

Now we will run our training data through our model. The epoch refers to how many times the model sees the entire dataset. Typically a lower number of epochs results in underfitting and too many epochs results in overfitting. We will plot the results to find the optimal epoch size.

The batch refers how many data points will go together to be run through the model. A single epoch will have multiple batches. (e.g. a dataset of 100 data points could have 4 batches of 25 points each). We will use a batch size of 128 data points.

a <- model %>% fit(
  images_training, 
  number_training, 
  epochs = 30, 
  batch_size = 128, 
  validation_split = 0.2
)

The loss goes up as the number of epochs increase. The accuracy goes up slightly as the number of epochs increase and bounces around a bit. The accuracy is very close for all number of epochs. The loss function represents error in the model and how well the model works. Less loss is better.

plot(a)
## `geom_smooth()` using formula 'y ~ x'

Here is the loss and accuracy for the model:

model %>% evaluate(images_testing, number_testing,verbose = 0)
##      loss  accuracy 
## 0.5218769 0.9708000

Here are the number predictions for our test data using our model:

pred_test <- model %>% predict_classes(images_testing)
pred_test
##     [1] 7 2 1 0 4 1 4 9 6 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1 3 1 3 4 7 2
##    [37] 7 1 2 1 1 7 4 2 3 5 1 2 4 4 6 3 5 5 6 0 4 1 9 5 7 8 9 3 7 4 6 4 3 0 7 0
##    [73] 2 9 1 7 3 2 9 7 7 6 2 7 8 4 7 3 6 1 3 6 9 3 1 4 1 7 6 9 6 0 5 4 9 9 2 1
##   [109] 9 4 8 7 3 9 7 4 4 4 9 2 5 4 7 6 7 9 0 5 8 5 6 6 5 7 8 1 0 1 6 4 6 7 3 1
##   [145] 7 1 8 2 0 7 9 9 5 5 1 5 6 0 3 4 4 6 5 4 6 5 4 5 1 4 4 7 2 3 2 7 1 8 1 8
##   [181] 1 8 5 0 8 9 2 5 0 1 1 1 0 9 0 3 1 6 4 2 9 6 1 1 1 3 9 5 2 9 4 5 9 3 8 0
##   [217] 3 6 5 5 7 2 2 7 1 2 8 4 1 7 3 3 8 8 7 9 2 2 4 1 5 5 8 7 2 3 0 2 4 2 4 1
##   [253] 9 5 7 7 2 8 2 6 8 5 7 7 9 1 8 1 8 0 3 0 1 9 9 4 1 8 2 1 2 9 7 5 9 2 6 4
##   [289] 1 5 8 2 9 2 0 4 0 0 2 8 4 7 1 2 4 0 2 7 4 3 3 0 0 3 1 9 6 5 2 5 9 7 9 3
##   [325] 0 4 2 0 7 1 1 2 1 5 3 3 9 7 8 6 5 6 1 3 8 1 0 5 1 3 1 5 5 6 1 8 5 1 4 9
##   [361] 4 6 2 2 5 0 6 5 6 3 7 2 0 8 8 5 4 1 1 4 0 3 3 7 6 1 6 2 1 9 2 8 6 1 9 5
##   [397] 2 5 4 4 2 8 3 8 2 4 5 0 3 1 7 7 5 7 9 7 1 9 2 1 4 0 9 2 0 4 9 1 4 8 1 8
##   [433] 4 5 9 8 8 3 7 6 0 0 3 0 2 0 6 9 9 3 3 3 2 3 9 1 2 6 8 0 5 6 6 6 3 8 8 2
##   [469] 7 5 8 9 6 1 8 4 1 2 5 9 1 9 7 5 4 0 8 9 9 1 0 5 2 3 7 8 9 4 0 6 3 9 5 2
##   [505] 1 3 1 3 6 5 7 4 2 2 6 3 2 6 5 4 8 9 7 1 3 0 3 8 3 1 9 3 4 4 6 4 2 1 8 2
##   [541] 5 4 8 8 4 0 0 2 3 2 7 3 0 8 7 4 4 7 9 6 9 0 9 8 0 4 6 0 6 3 5 4 8 3 3 9
##   [577] 3 3 3 7 8 0 2 8 1 7 0 6 5 4 3 8 0 9 6 3 8 0 9 9 6 8 6 8 5 7 8 6 0 2 4 0
##   [613] 2 2 3 1 9 7 5 1 0 8 4 6 2 6 7 9 3 2 9 8 2 2 9 2 7 3 5 9 1 8 0 2 0 5 6 1
##   [649] 3 7 6 7 1 2 5 8 0 3 7 7 4 0 9 1 8 6 7 7 4 3 4 9 1 9 5 1 7 3 9 7 6 9 1 3
##   [685] 3 8 3 3 6 7 2 4 5 8 5 1 1 4 4 3 1 0 7 7 0 7 9 4 4 8 5 5 4 0 8 2 1 6 8 4
##   [721] 5 0 4 7 6 1 7 3 2 6 7 2 6 9 3 1 4 6 2 5 9 2 0 6 2 1 7 3 4 1 0 5 4 3 1 1
##   [757] 7 4 9 9 4 8 4 0 2 4 5 1 1 6 4 7 1 9 4 2 4 1 5 5 3 8 3 1 4 5 8 8 9 4 1 5
##   [793] 3 8 0 3 2 5 1 2 8 3 4 4 0 8 8 3 3 1 7 3 5 9 6 3 2 6 1 3 6 0 7 2 1 7 1 4
##   [829] 2 4 2 1 7 9 6 1 1 2 4 8 1 7 7 4 8 0 7 3 1 3 1 0 7 7 0 3 5 5 2 7 6 6 9 2
##   [865] 8 3 5 2 2 5 6 0 8 2 9 2 8 8 8 8 7 4 9 3 0 6 6 3 2 1 3 2 2 9 5 0 0 5 7 8
##   [901] 1 4 4 6 0 2 9 1 4 7 4 7 3 9 8 8 4 7 1 2 1 2 2 3 2 3 2 3 9 1 7 4 0 3 5 5
##   [937] 8 6 3 2 6 7 6 6 3 2 7 8 1 1 7 5 6 4 9 5 1 3 3 4 7 8 9 1 1 5 9 1 4 4 5 4
##   [973] 0 6 2 2 3 1 5 1 2 0 3 8 1 2 6 7 1 6 2 3 9 0 1 2 2 0 8 9 9 0 2 5 1 9 7 8
##  [1009] 1 0 4 1 7 9 5 4 2 6 8 1 3 7 5 4 4 1 8 1 3 8 1 2 5 8 0 6 2 1 1 1 1 5 3 4
##  [1045] 8 9 5 0 9 2 2 4 8 2 1 7 2 4 9 4 4 0 3 9 2 2 3 3 8 3 5 7 3 5 8 1 2 4 4 6
##  [1081] 4 9 5 1 0 6 9 5 9 5 9 7 3 8 0 3 7 1 3 6 7 8 5 9 7 9 6 9 6 3 7 4 6 5 3 5
##  [1117] 4 7 8 7 8 0 7 6 8 8 7 3 3 1 9 5 2 7 3 5 1 1 2 1 4 7 4 7 5 4 5 4 0 8 3 6
##  [1153] 9 6 0 2 5 4 4 4 4 6 6 4 7 9 2 4 5 5 8 7 3 9 2 7 0 2 4 1 1 1 8 9 2 8 7 2
##  [1189] 0 1 5 0 9 1 9 0 6 0 8 6 8 1 8 0 3 3 7 2 3 6 2 1 6 1 1 3 7 9 0 8 0 5 4 0
##  [1225] 2 8 2 2 9 8 4 0 1 5 2 5 1 2 1 3 1 7 9 5 7 2 0 5 8 8 6 2 5 6 1 9 2 1 5 8
##  [1261] 7 0 2 4 4 3 6 8 8 2 4 0 5 0 4 4 7 9 3 4 1 5 9 7 3 5 8 8 0 5 5 3 6 6 0 1
##  [1297] 6 0 3 7 4 4 1 2 9 1 4 6 9 9 3 9 8 4 4 3 1 3 1 5 8 7 9 4 8 8 7 9 9 1 4 5
##  [1333] 6 0 5 2 2 2 1 5 5 2 4 9 6 2 7 7 2 2 1 1 2 8 3 7 2 4 1 7 1 7 6 7 8 2 7 3
##  [1369] 1 7 5 8 2 6 2 2 5 6 6 0 9 2 4 3 3 9 7 6 6 8 0 4 1 5 8 8 9 1 8 0 6 7 2 1
##  [1405] 0 5 5 2 0 2 2 0 2 4 5 8 0 9 9 4 6 5 4 9 1 5 3 4 9 9 1 2 2 8 1 9 6 4 0 9
##  [1441] 4 8 3 8 6 0 2 5 1 9 6 2 9 4 0 9 6 0 6 2 5 4 2 3 8 4 5 5 0 3 8 5 3 5 8 6
##  [1477] 5 7 6 3 3 9 6 1 1 2 9 0 4 3 3 6 9 5 9 3 7 7 7 8 7 9 8 3 0 7 2 7 9 4 5 4
##  [1513] 9 3 2 1 4 0 2 3 7 5 9 8 8 5 0 1 1 4 7 3 9 0 0 0 6 6 2 3 7 8 4 9 7 9 2 4
##  [1549] 1 6 5 2 4 9 8 1 8 4 0 9 8 4 8 7 7 0 7 8 5 6 0 4 8 8 2 4 7 6 6 6 4 7 1 8
##  [1585] 8 2 3 6 3 0 0 3 7 6 9 7 9 9 5 4 3 3 6 1 2 3 7 3 3 6 0 9 3 8 4 3 6 3 5 0
##  [1621] 2 0 9 0 7 4 5 9 3 5 1 9 6 1 4 5 4 5 0 5 9 5 2 1 2 9 1 9 9 4 0 8 4 5 2 9
##  [1657] 2 1 2 1 7 3 6 8 8 4 9 1 9 8 5 3 5 1 1 8 6 5 0 4 4 7 2 3 5 6 8 8 6 7 3 1
##  [1693] 0 5 8 9 2 9 6 7 0 4 5 7 1 7 4 1 0 9 7 2 0 0 9 1 7 8 7 8 4 7 2 0 4 6 0 3
##  [1729] 1 1 9 3 9 6 7 4 1 5 3 0 8 7 3 9 6 9 3 5 0 2 7 4 5 1 7 5 8 0 8 8 1 5 0 3
##  [1765] 0 3 1 4 0 3 7 2 7 1 8 0 7 0 4 3 1 9 8 7 7 1 4 9 9 3 8 1 7 9 0 2 0 3 3 7
##  [1801] 6 9 2 3 3 7 7 0 0 9 5 2 9 8 7 4 4 2 6 6 1 9 6 8 2 8 0 8 5 1 1 6 3 5 1 1
##  [1837] 1 3 1 2 3 0 2 0 1 3 5 5 7 4 8 9 6 9 6 8 3 6 6 8 5 1 4 2 4 4 5 1 1 9 0 1
##  [1873] 4 9 5 7 1 8 8 5 6 9 8 7 1 1 6 7 6 3 2 2 0 8 9 2 5 1 0 8 1 4 5 7 9 6 9 0
##  [1909] 6 1 5 5 8 3 8 2 6 5 0 7 4 6 1 3 4 7 3 2 3 4 2 5 2 7 1 7 2 6 4 1 5 2 8 6
##  [1945] 0 1 8 2 5 7 7 6 9 3 5 8 4 2 4 0 8 8 3 4 9 2 7 5 8 6 5 6 0 8 6 7 3 6 4 9
##  [1981] 4 6 6 3 0 4 1 0 1 4 6 2 9 1 1 0 6 3 9 5 6 5 6 5 8 4 6 4 3 9 1 3 4 1 9 1
##  [2017] 7 1 2 9 3 5 4 0 7 3 6 1 7 5 5 3 3 7 1 5 7 5 8 6 5 1 0 4 7 3 4 6 7 9 8 1
##  [2053] 5 4 9 2 8 6 2 7 0 0 6 7 5 8 6 0 9 3 7 1 3 6 4 3 3 5 5 6 3 0 2 3 4 2 3 0
##  [2089] 9 9 4 7 2 8 4 7 0 6 0 8 5 2 8 5 7 3 0 8 2 7 2 8 2 5 5 7 6 4 6 8 4 8 2 7
##  [2125] 4 5 2 0 3 9 9 6 7 2 5 1 1 1 2 3 6 7 8 7 6 4 8 9 4 8 6 3 8 3 1 0 6 2 2 5
##  [2161] 6 9 8 8 1 4 1 7 2 4 6 1 8 4 3 1 2 8 0 8 5 9 2 4 2 5 2 7 0 9 0 2 5 7 6 7
##  [2197] 9 4 2 6 2 4 4 8 0 4 4 5 8 0 6 8 9 8 5 6 9 0 4 8 7 1 3 4 8 8 0 9 1 3 3 6
##  [2233] 9 8 7 1 0 6 7 1 7 5 2 7 9 1 8 5 2 4 9 4 7 2 2 3 4 9 1 9 2 1 7 9 4 4 1 6
##  [2269] 7 2 7 8 8 1 9 7 1 1 7 5 5 3 5 1 3 7 6 1 3 8 7 3 9 0 0 0 2 8 8 2 3 7 1 3
##  [2305] 0 3 4 4 3 8 9 2 3 9 7 1 1 7 0 4 9 6 5 9 1 7 0 2 0 0 4 6 7 0 7 1 4 6 4 5
##  [2341] 4 9 9 1 7 9 5 3 3 8 2 3 6 2 2 1 1 1 1 1 6 9 8 4 3 7 1 6 4 8 0 4 7 4 2 4
##  [2377] 0 7 0 1 9 8 8 6 0 0 4 1 6 8 2 2 3 8 4 8 2 2 1 7 5 4 4 0 4 3 4 7 3 1 0 1
##  [2413] 2 5 9 2 1 0 1 8 9 1 6 8 3 3 9 3 6 2 8 3 2 2 1 0 4 1 9 2 4 3 7 9 1 5 2 4
##  [2449] 9 0 3 8 5 3 6 0 9 4 6 2 5 0 0 7 4 6 6 8 6 6 8 6 9 1 7 2 5 9 9 0 7 2 7 6
##  [2485] 7 0 6 5 4 4 7 2 0 9 9 2 2 9 4 4 2 3 3 2 1 7 0 7 6 4 1 3 8 7 9 5 9 2 5 1
##  [2521] 8 7 3 7 1 5 5 0 9 1 4 0 6 3 3 6 0 4 9 7 5 1 6 8 9 5 5 7 9 3 8 3 8 1 5 3
##  [2557] 5 0 5 5 5 8 6 7 7 7 3 7 0 5 9 0 2 5 5 3 1 7 7 8 6 5 9 3 8 9 5 3 7 0 1 7
##  [2593] 0 0 3 7 2 3 8 1 8 6 2 9 5 7 5 4 8 6 2 5 1 4 8 4 5 8 3 0 6 2 7 3 3 2 1 0
##  [2629] 7 3 4 0 3 9 3 2 8 9 0 3 8 0 7 6 5 4 7 3 5 0 8 6 2 5 1 1 0 0 4 4 0 1 2 3
##  [2665] 2 7 7 8 5 2 5 7 6 9 1 4 1 6 4 2 4 3 5 4 3 9 5 0 1 5 3 8 9 1 9 7 9 5 5 2
##  [2701] 7 4 6 0 1 1 1 0 4 4 7 6 3 0 0 4 3 0 6 1 9 6 1 3 8 1 2 5 6 2 7 3 6 0 1 9
##  [2737] 7 6 6 8 9 2 9 5 8 3 1 0 0 7 6 6 2 1 6 9 3 1 8 6 9 0 6 0 0 0 6 3 5 9 3 4
##  [2773] 5 5 8 5 3 0 4 0 2 9 6 8 2 3 1 2 1 1 5 6 9 8 0 6 6 5 5 3 8 6 2 1 4 5 4 3
##  [2809] 7 8 5 0 9 3 5 1 1 0 4 4 7 0 1 7 0 1 6 1 4 5 6 6 5 7 8 4 4 7 2 5 3 7 0 7
##  [2845] 7 9 6 4 2 8 5 7 8 3 9 5 8 9 9 8 6 2 8 4 2 3 6 1 1 8 9 3 4 0 7 9 6 7 1 4
##  [2881] 1 3 4 9 3 1 4 7 7 4 7 2 9 3 0 8 5 8 4 0 4 4 1 5 2 8 5 4 9 5 2 8 1 5 3 7
##  [2917] 9 4 2 5 6 0 5 9 3 5 9 2 1 9 5 3 0 6 9 8 4 0 4 7 2 9 0 1 0 3 1 6 5 8 1 5
##  [2953] 3 5 0 3 5 5 9 2 8 7 0 4 9 1 9 7 7 5 5 2 0 9 1 8 6 2 3 9 6 2 1 9 1 3 5 5
##  [2989] 0 3 8 3 3 7 6 6 0 1 4 0 6 9 8 1 2 9 9 5 9 7 3 7 8 0 1 3 0 4 6 1 0 2 5 8
##  [3025] 4 4 1 1 5 4 8 6 0 6 9 2 6 2 7 1 7 9 4 0 0 3 8 2 2 3 1 6 0 5 7 7 9 2 6 7
##  [3061] 9 7 3 6 8 8 4 6 8 4 1 2 8 2 3 9 4 0 3 7 3 2 3 3 7 3 4 0 6 2 0 8 1 5 3 5
##  [3097] 4 1 7 1 5 7 5 7 3 2 2 7 3 7 3 7 8 5 4 5 2 5 6 5 3 6 7 4 1 7 1 5 2 3 5 3
##  [3133] 1 4 2 6 7 4 3 8 0 6 2 1 6 5 3 9 1 9 3 2 1 8 4 4 6 5 8 6 9 7 7 8 6 9 7 3
##  [3169] 9 4 0 5 4 6 4 1 2 3 0 0 2 6 6 5 7 0 8 6 4 7 9 0 7 3 4 2 1 8 8 5 9 2 7 1
##  [3205] 8 8 8 2 7 6 0 1 2 7 1 0 8 3 6 0 5 3 6 2 8 7 0 1 4 2 1 1 4 4 4 4 7 1 6 2
##  [3241] 9 9 0 0 1 8 8 4 3 4 2 0 6 1 6 1 2 2 2 1 2 3 7 8 1 0 7 2 1 6 6 0 1 6 2 5
##  [3277] 1 7 4 8 2 1 4 3 8 3 9 9 4 3 3 4 7 2 7 5 7 0 4 3 3 2 6 7 6 0 0 6 7 7 0 5
##  [3313] 5 8 1 0 7 0 2 8 1 5 0 8 8 0 3 2 7 7 2 6 4 7 5 5 5 7 9 2 8 4 6 8 6 5 0 0
##  [3349] 8 7 6 1 7 1 1 2 7 4 0 0 7 7 6 3 8 6 4 2 0 9 4 0 5 7 8 2 7 4 7 1 1 3 6 6
##  [3385] 2 9 1 9 4 8 3 6 9 5 9 6 2 4 6 7 7 0 6 6 9 4 8 3 5 3 4 9 0 0 5 2 5 0 7 1
##  [3421] 1 1 0 7 6 7 9 6 6 4 1 4 3 1 1 2 2 4 1 0 8 8 6 3 4 0 0 6 3 3 0 3 1 7 1 1
##  [3457] 3 1 0 9 9 7 5 4 1 4 8 9 5 3 5 1 9 8 2 7 3 9 9 0 1 0 2 9 3 9 3 3 6 2 4 9
##  [3493] 8 3 7 4 0 4 7 8 4 9 8 1 9 7 5 9 2 8 2 2 0 2 2 3 8 4 6 8 4 8 2 4 6 7 9 3
##  [3529] 3 9 4 3 1 4 4 7 0 5 9 6 0 4 4 4 4 6 1 2 3 3 6 4 5 9 6 8 5 6 0 5 6 4 1 8
##  [3565] 6 5 2 5 4 5 5 4 7 7 0 7 8 2 2 3 7 0 1 8 0 7 1 9 8 7 5 5 9 1 7 5 4 3 1 2
##  [3601] 2 1 6 6 0 1 1 4 0 7 4 2 4 0 6 4 7 6 9 5 3 4 6 5 0 1 8 8 2 8 3 5 7 8 0 8
##  [3637] 5 7 1 1 0 1 3 7 8 5 0 7 1 1 0 1 1 4 5 2 7 6 2 3 0 2 8 5 9 6 9 7 2 1 3 6
##  [3673] 4 1 5 2 4 0 5 1 0 2 2 6 4 4 3 9 6 1 6 5 7 9 2 0 2 6 0 1 4 3 3 2 8 8 0 8
##  [3709] 8 9 0 9 6 7 6 3 9 3 4 7 7 7 4 9 0 6 4 4 4 2 7 2 8 1 0 0 7 5 3 3 3 1 3 7
##  [3745] 6 1 3 1 6 6 5 7 4 7 5 9 5 8 4 9 9 1 6 5 0 1 3 7 0 3 4 8 2 2 0 2 5 1 5 1
##  [3781] 6 8 8 9 1 2 1 3 5 1 0 9 4 4 8 3 8 5 9 7 6 6 2 0 0 0 5 8 8 1 5 3 3 8 5 1
##  [3817] 8 2 0 4 9 9 6 2 3 3 5 6 4 8 0 9 2 8 3 6 7 5 7 2 9 4 9 1 2 8 6 0 7 0 9 1
##  [3853] 1 0 7 5 9 9 1 9 5 9 2 5 0 4 1 0 8 9 0 8 9 8 9 4 2 5 7 9 8 9 8 0 9 9 6 8
##  [3889] 9 9 5 9 8 6 1 0 3 3 5 2 1 6 3 0 2 8 3 5 6 2 3 0 2 2 6 4 3 5 5 1 7 2 1 6
##  [3925] 9 1 9 9 5 5 1 6 2 2 8 6 7 1 4 6 0 4 0 3 3 2 2 3 6 8 9 8 5 3 8 5 4 5 2 0
##  [3961] 5 6 3 2 8 3 9 9 3 7 9 4 6 7 1 3 7 3 6 6 0 9 0 1 9 4 2 8 8 0 1 6 9 7 5 3
##  [3997] 4 7 4 9 9 4 3 6 3 1 1 8 6 9 1 8 4 1 1 9 9 4 3 6 8 1 6 0 4 1 3 1 7 4 9 5
##  [4033] 1 0 0 1 1 6 2 1 9 8 4 0 3 6 4 9 0 7 1 6 5 7 5 2 5 1 8 5 4 7 0 6 7 9 2 5
##  [4069] 8 1 0 4 5 7 1 3 5 1 9 0 0 6 0 7 3 1 8 3 9 7 0 0 8 9 5 9 8 3 2 7 2 9 7 2
##  [4105] 1 1 3 7 5 3 1 9 8 2 2 2 5 8 5 7 3 8 9 3 8 6 8 2 3 9 7 5 6 2 9 2 8 8 1 6
##  [4141] 8 8 7 9 1 8 0 1 7 2 0 7 5 1 4 0 3 0 9 8 6 2 3 5 3 8 0 2 1 1 1 1 4 2 9 7
##  [4177] 7 5 1 1 2 1 9 9 9 1 0 2 0 2 1 1 4 6 4 1 5 4 9 7 7 7 5 6 2 2 2 8 0 6 9 5
##  [4213] 1 9 7 7 1 4 8 5 3 4 3 4 9 7 5 0 7 4 8 8 1 5 3 9 5 9 3 6 9 0 3 6 3 9 8 2
##  [4249] 8 1 2 8 6 8 5 5 3 9 4 9 2 5 1 5 1 5 4 1 4 4 3 5 9 1 2 2 3 3 0 2 9 0 0 9
##  [4285] 9 6 0 9 3 8 8 4 1 9 5 7 2 7 9 9 5 9 5 1 1 8 7 5 1 9 5 3 5 4 9 5 9 3 1 9
##  [4321] 0 9 7 5 4 9 2 0 1 0 5 1 4 9 3 3 6 1 5 2 5 2 2 0 9 2 6 6 0 1 2 0 3 0 2 5
##  [4357] 5 7 9 5 3 0 8 9 5 0 3 2 5 4 0 8 8 4 8 8 8 4 5 4 8 5 9 9 2 2 1 2 6 8 8 7
##  [4393] 0 3 6 6 4 3 8 8 7 2 2 0 0 9 3 9 9 1 9 8 6 6 4 0 6 9 2 8 5 4 5 7 9 9 9 2
##  [4429] 1 8 3 4 0 7 8 3 9 3 4 6 5 6 2 2 9 2 6 0 0 6 1 2 8 7 9 8 2 0 4 7 7 5 0 5
##  [4465] 6 4 6 7 4 3 0 7 5 0 7 4 2 6 8 9 9 4 2 4 6 7 8 8 6 9 4 1 3 7 3 0 8 8 7 6
##  [4501] 8 3 9 2 7 9 2 1 8 3 2 9 6 8 4 0 1 2 8 4 5 2 7 8 1 1 3 0 3 5 7 0 3 1 9 3
##  [4537] 5 3 1 7 7 3 0 8 4 8 2 6 6 2 9 4 3 9 0 9 9 6 4 2 9 7 2 1 1 6 7 4 7 5 9 1
##  [4573] 8 2 1 4 4 5 7 6 1 3 2 5 9 9 3 6 1 1 4 6 9 7 2 1 5 1 4 6 3 8 1 1 0 3 1 6
##  [4609] 8 4 9 0 7 3 0 6 9 0 6 6 6 3 6 7 7 2 8 6 0 8 3 0 2 9 8 5 2 5 3 8 8 0 0 1
##  [4645] 9 5 1 3 9 6 0 1 4 1 7 1 2 3 7 9 7 4 9 9 3 9 2 8 2 7 1 8 0 9 1 0 1 7 7 9
##  [4681] 6 9 9 9 2 1 6 1 3 5 7 1 9 7 6 4 5 7 6 6 9 9 6 3 6 2 9 8 1 2 2 5 5 2 3 7
##  [4717] 2 1 0 1 0 4 5 2 8 2 8 3 5 1 7 9 1 1 2 9 7 8 4 0 5 0 7 8 8 4 7 7 8 5 8 4
##  [4753] 9 8 1 3 8 0 3 1 7 7 5 5 1 6 5 7 4 9 3 5 4 7 1 2 0 8 1 6 0 7 3 4 7 3 9 6
##  [4789] 0 5 6 4 8 7 7 9 3 8 6 9 7 2 3 4 0 2 1 8 5 5 5 7 2 4 4 7 2 8 3 0 8 7 8 4
##  [4825] 0 8 4 4 5 8 5 6 6 3 0 9 3 7 6 8 9 3 4 9 5 8 9 1 2 8 8 6 8 1 3 7 9 0 1 1
##  [4861] 4 3 0 8 1 7 4 5 7 1 2 1 1 3 9 6 2 1 2 6 8 7 6 6 9 3 7 0 5 2 8 0 5 4 3 8
##  [4897] 4 6 6 2 7 9 5 1 3 2 4 3 6 1 9 4 4 7 6 5 4 1 9 9 2 7 8 0 1 3 6 1 3 4 1 1
##  [4933] 1 5 6 0 7 0 7 2 3 2 5 2 2 9 4 9 8 1 2 1 6 1 2 7 4 0 0 0 8 2 2 9 2 2 9 9
##  [4969] 9 2 7 5 1 3 4 9 4 1 8 5 6 2 8 3 1 2 8 4 9 9 5 7 0 7 7 2 3 2 4 0 3 9 9 8
##  [5005] 4 1 0 6 0 9 6 8 6 1 1 9 8 9 2 3 5 5 9 4 2 1 9 4 3 9 6 0 4 0 6 0 1 2 3 4
##  [5041] 7 8 9 0 1 2 3 4 7 8 9 0 1 2 3 4 5 6 7 8 9 8 3 4 7 8 6 3 9 0 9 7 1 9 3 8
##  [5077] 4 7 5 0 9 1 4 5 4 6 9 0 6 2 1 1 1 1 7 2 4 7 5 2 9 4 5 8 4 2 9 7 0 0 7 5
##  [5113] 1 1 7 6 6 6 8 2 2 7 7 4 0 2 4 2 1 8 9 6 1 0 5 9 6 9 5 0 5 0 8 3 9 6 3 0
##  [5149] 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 5 4 4 7 4 7 7 3 9 8
##  [5185] 8 3 1 5 8 2 7 4 2 1 5 4 5 5 8 6 4 4 4 1 8 7 5 5 1 8 9 1 3 6 3 3 2 2 6 9
##  [5221] 9 6 5 5 3 3 8 1 6 5 6 8 1 9 7 6 8 3 7 4 7 0 9 0 0 3 7 9 3 0 2 0 1 0 1 0
##  [5257] 4 0 1 0 4 7 9 6 2 6 2 2 9 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
##  [5293] 2 3 4 5 6 7 8 9 8 0 5 6 6 0 8 0 2 3 7 9 4 7 1 9 1 7 1 4 0 0 4 1 7 5 7 1
##  [5329] 3 3 3 6 6 9 7 4 3 0 2 5 2 6 0 8 9 4 3 5 4 8 1 5 9 0 6 4 3 6 3 3 8 1 4 7
##  [5365] 5 7 2 2 0 0 1 7 7 9 5 9 8 9 6 8 8 2 3 6 1 2 9 8 9 5 2 6 2 4 8 4 6 5 0 1
##  [5401] 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 7 4 2 0 9 0 1 5 8 8 0
##  [5437] 2 7 8 4 4 6 1 0 4 5 3 9 4 2 0 5 0 1 3 2 9 1 6 0 1 1 8 0 4 7 7 6 3 6 0 7
##  [5473] 3 5 4 2 4 1 8 3 5 6 7 0 6 7 1 2 5 8 1 9 3 8 2 8 7 6 7 1 4 6 2 9 3 0 1 2
##  [5509] 3 4 5 6 7 0 1 2 3 4 5 0 1 2 8 9 1 4 0 9 5 0 8 0 7 7 1 1 2 9 3 6 7 2 3 8
##  [5545] 1 2 9 8 8 7 1 7 1 1 0 3 4 2 6 4 7 4 2 7 4 9 1 0 6 0 5 5 5 3 5 9 7 4 8 5
##  [5581] 9 6 9 3 0 3 8 9 1 8 1 6 0 6 1 2 3 4 5 6 9 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
##  [5617] 3 4 5 6 7 8 9 3 5 3 2 9 3 2 1 4 5 5 5 3 2 1 3 9 7 2 5 2 8 9 1 8 8 7 8 1
##  [5653] 0 0 9 7 8 7 5 0 6 1 5 7 4 6 1 2 5 0 7 9 9 0 3 8 2 4 8 1 8 6 5 9 0 0 0 3
##  [5689] 7 1 6 4 2 6 6 0 4 5 4 1 3 8 6 3 9 9 5 9 3 9 8 5 6 4 7 6 2 2 0 9 4 0 1 2
##  [5725] 3 4 5 6 7 8 9 0 1 2 2 5 6 0 1 2 3 4 5 6 8 7 1 3 2 8 0 7 5 9 9 6 0 9 4 1
##  [5761] 3 2 1 2 3 8 3 2 6 5 6 8 2 7 4 8 1 8 0 5 3 9 4 1 9 2 1 9 6 7 9 0 4 6 1 7
##  [5797] 3 8 7 2 9 6 5 8 3 9 0 5 7 1 6 1 0 9 3 3 4 4 0 6 2 5 4 2 3 4 6 0 0 2 0 1
##  [5833] 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 9 8 7 1 3 7 5 2 8 0 7 5
##  [5869] 9 9 0 9 1 1 5 8 8 6 3 2 1 8 3 2 6 5 6 0 4 1 0 5 3 1 9 2 1 9 6 0 4 6 1 7
##  [5905] 3 8 7 2 9 6 5 8 3 5 7 1 6 1 0 9 6 2 5 4 2 3 4 4 6 0 0 2 0 1 2 3 9 5 6 7
##  [5941] 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 8 4 5 6 7 8 9 8 6 5 0 6 8 9 4 1 9 3 8 0 4
##  [5977] 8 9 1 4 0 5 5 2 1 5 4 0 7 6 0 1 7 0 6 8 9 9 1 7 9 8 6 0 8 1 7 7 1 3 2 3
##  [6013] 1 4 2 0 0 7 8 4 6 4 9 3 8 4 7 2 5 6 3 6 9 6 3 2 2 4 6 9 0 2 5 5 1 3 3 9
##  [6049] 7 8 7 2 2 5 7 9 8 2 1 9 1 3 0 1 2 3 4 5 6 7 8 3 0 1 2 3 4 5 6 7 8 3 0 1
##  [6085] 2 3 4 5 6 7 8 3 1 2 6 5 3 0 7 0 4 8 4 3 6 7 2 3 1 2 1 2 9 6 0 1 3 0 2 7
##  [6121] 5 7 6 2 9 1 9 0 6 0 6 0 2 0 6 1 5 8 4 3 0 1 5 4 4 8 5 7 5 7 8 3 4 8 8 5
##  [6157] 2 9 7 1 3 8 1 0 7 5 3 6 3 4 7 7 3 9 3 4 4 3 8 6 2 0 1 2 3 4 5 6 7 8 9 0
##  [6193] 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 8 3 9 5 5 2 6 8 4 9 1 7 1 2 3 5
##  [6229] 9 6 9 1 1 1 2 9 5 6 8 1 2 0 7 7 5 8 2 9 8 9 0 4 6 7 1 3 4 5 6 0 3 6 8 7
##  [6265] 0 4 2 7 4 7 5 4 3 4 2 8 1 5 1 2 0 2 5 6 4 3 0 0 0 3 3 5 7 0 6 4 8 8 6 3
##  [6301] 4 6 9 9 8 2 7 7 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7
##  [6337] 8 2 1 7 2 5 0 8 0 2 7 8 8 3 6 0 2 7 6 6 1 2 8 8 7 7 4 7 7 3 7 4 5 4 3 3
##  [6373] 8 4 1 1 9 7 4 3 7 3 3 0 2 5 5 6 6 3 5 2 5 9 9 8 4 1 0 6 0 9 6 8 8 5 6 1
##  [6409] 1 9 8 9 2 3 5 5 9 4 2 1 9 3 9 2 0 6 0 4 0 0 1 2 3 4 7 8 9 0 1 2 3 7 8 9
##  [6445] 0 1 2 3 4 7 8 9 7 3 0 3 1 8 7 6 4 0 2 6 8 3 2 8 1 2 0 7 1 0 4 4 5 8 0 6
##  [6481] 2 3 1 5 1 8 5 9 4 0 7 5 8 8 3 8 9 2 6 2 5 3 1 7 3 4 1 9 9 6 0 3 9 2 8 1
##  [6517] 4 3 5 2 9 2 5 8 9 5 0 1 2 4 5 6 0 1 2 3 4 5 6 7 1 2 3 4 5 1 0 4 5 6 6 3
##  [6553] 4 4 2 9 1 0 6 4 9 7 2 3 3 9 2 0 9 3 3 9 1 5 6 3 7 7 8 4 0 2 4 0 2 4 7 8
##  [6589] 0 7 0 6 9 3 2 8 6 7 5 7 5 1 0 8 1 6 7 2 9 7 9 5 8 6 2 6 2 8 1 7 5 0 1 1
##  [6625] 3 7 4 9 1 8 6 8 5 0 1 2 3 4 5 6 7 5 9 0 1 2 3 4 7 8 9 5 1 7 8 9 9 8 9 8
##  [6661] 4 1 7 7 3 3 7 6 6 6 1 9 0 1 7 6 3 2 1 7 1 3 9 1 7 6 8 4 1 4 3 6 9 6 1 4
##  [6697] 4 7 2 4 4 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 9 0 1 2 3 4 7 8 1 3 5 1 7 7
##  [6733] 2 1 4 8 3 4 4 3 9 7 4 1 2 3 5 9 1 6 0 1 0 0 2 9 7 1 1 4 0 4 7 3 6 8 0 3
##  [6769] 7 4 0 6 9 2 6 5 8 6 9 0 4 0 6 6 9 2 0 9 5 1 3 7 6 9 3 0 2 2 0 1 2 3 4 5
##  [6805] 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 2 1 7 2 5 0 8 0 2 7 8 8
##  [6841] 3 0 6 0 2 7 6 6 1 2 8 8 7 7 4 7 7 3 7 4 5 4 3 3 8 4 5 4 1 1 9 7 4 3 7 3
##  [6877] 3 0 2 5 5 6 3 1 5 2 5 9 9 8 4 1 0 6 0 9 6 8 8 5 6 1 1 9 8 9 2 3 5 5 9 4
##  [6913] 2 1 9 4 9 1 3 9 2 0 6 0 4 0 6 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0
##  [6949] 1 2 3 4 5 6 7 8 9 3 8 0 7 1 0 7 5 5 6 9 0 1 0 0 8 3 4 3 1 5 0 0 9 5 3 4
##  [6985] 9 3 7 6 9 2 4 5 7 2 6 4 9 4 9 4 1 2 2 5 8 1 3 2 9 4 3 8 2 2 1 2 8 6 5 1
##  [7021] 6 7 2 1 3 9 3 8 7 5 7 0 7 4 8 8 5 0 6 6 3 7 6 9 9 4 8 4 1 0 6 6 0 1 2 3
##  [7057] 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 7 4 0 4 0 1 7 9 5 1
##  [7093] 4 2 8 9 4 3 7 8 2 4 4 3 3 6 9 9 5 8 6 7 0 6 8 2 6 3 9 3 2 8 6 1 7 4 8 8
##  [7129] 9 0 3 3 9 0 5 2 9 4 1 0 3 7 5 8 7 7 8 2 9 7 1 2 6 4 2 5 2 3 6 6 5 0 0 2
##  [7165] 8 1 6 1 0 4 3 1 6 1 9 0 1 4 5 6 7 8 9 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 8 9
##  [7201] 8 4 0 0 7 2 4 3 8 6 6 3 2 6 3 3 6 1 4 7 8 0 3 1 9 0 1 9 1 2 7 0 1 3 8 2
##  [7237] 9 2 7 6 5 5 9 9 8 2 9 1 3 2 3 4 3 1 9 0 9 3 6 8 7 0 1 0 5 8 2 7 7 0 1 2
##  [7273] 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 1 7 4 8 1 5 6 5 7
##  [7309] 2 8 6 3 3 8 6 5 4 0 9 1 7 2 9 1 5 1 3 2 2 3 0 6 4 3 7 6 9 0 4 8 1 4 0 6
##  [7345] 1 2 6 9 2 2 3 5 5 1 0 7 7 9 6 2 9 4 7 0 2 3 4 0 0 8 8 8 5 1 3 7 4 9 8 8
##  [7381] 9 0 9 8 9 0 2 6 5 6 7 4 7 5 4 1 3 5 3 1 2 3 4 5 6 1 2 3 4 6 0 1 2 4 5 6
##  [7417] 7 8 1 7 2 4 1 4 1 4 9 6 8 4 5 3 7 8 8 3 3 5 6 7 0 6 1 6 8 7 0 1 5 0 8 5
##  [7453] 0 1 5 8 4 2 3 9 7 6 9 1 9 0 6 7 1 2 3 9 2 4 5 5 3 7 5 3 1 5 2 2 3 0 2 9
##  [7489] 4 9 7 0 2 7 4 9 9 2 5 9 8 3 8 6 7 0 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7
##  [7525] 8 9 0 1 2 3 4 5 6 7 8 9 0 0 7 2 6 5 5 3 7 8 6 6 6 6 4 3 8 8 3 0 1 9 0 5
##  [7561] 4 1 9 1 2 7 0 1 3 8 2 9 2 7 4 2 6 5 5 9 9 1 1 5 7 6 8 2 9 4 3 1 9 0 9 3
##  [7597] 6 8 7 0 1 0 5 8 2 7 7 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 8 9 0 1 2 3 4 5 6
##  [7633] 7 8 9 2 1 2 1 3 9 9 8 5 3 7 0 7 7 5 7 9 9 4 7 0 3 4 1 5 8 1 4 8 4 1 8 6
##  [7669] 6 4 6 0 5 5 3 3 5 7 2 5 9 6 9 2 6 2 1 2 0 8 3 8 3 0 8 7 4 9 5 0 9 7 0 0
##  [7705] 4 6 0 9 1 6 2 7 6 5 3 5 2 1 8 3 8 6 1 0 2 1 4 0 1 2 3 4 5 6 7 8 9 0 1 2
##  [7741] 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 7 6 4 7 6 2 3 4 8 7 8 6 9 8 3 2 2 8 4
##  [7777] 8 5 6 5 0 2 0 1 1 2 9 6 8 2 1 0 6 5 2 9 7 6 3 9 3 7 1 8 3 8 1 9 5 5 0 1
##  [7813] 3 9 8 2 6 0 4 5 0 3 1 2 6 7 5 9 9 3 0 3 1 4 4 0 4 9 0 1 2 3 5 6 7 8 0 1
##  [7849] 2 5 5 6 7 8 9 0 1 2 5 5 6 7 8 9 9 7 0 9 0 1 5 8 8 0 9 3 2 7 8 4 6 1 0 4
##  [7885] 9 4 4 0 5 0 1 6 9 3 2 9 1 6 0 8 1 8 7 7 6 5 6 0 7 2 4 1 7 0 6 8 1 2 5 8
##  [7921] 1 8 2 8 7 6 8 7 8 6 2 9 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
##  [7957] 3 4 5 6 7 8 9 8 8 5 7 0 3 1 6 8 4 1 5 6 4 2 7 8 1 3 4 3 4 7 2 0 5 0 8 9
##  [7993] 2 3 2 3 5 5 7 8 4 9 9 7 1 1 9 0 7 8 5 4 8 6 3 8 0 9 6 2 1 0 1 0 6 2 3 8
##  [8029] 9 0 7 2 3 4 5 5 2 8 5 4 6 6 6 7 9 1 8 2 1 5 3 4 7 9 4 0 0 0 1 1 3 4 5 6
##  [8065] 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 9 0 1 3 1 5 1 1 4 9 1 4 6 8 0 1
##  [8101] 1 9 1 6 6 8 7 8 2 9 9 0 7 1 0 3 6 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8
##  [8137] 9 0 1 2 3 4 5 6 7 8 9 8 6 5 9 7 0 2 3 4 3 8 5 1 5 2 3 0 1 2 1 3 2 6 5 3
##  [8173] 0 7 2 7 4 6 4 0 5 9 0 5 9 5 3 1 7 4 7 6 5 4 0 0 6 6 2 0 6 3 7 7 4 4 3 9
##  [8209] 2 8 9 6 0 9 5 3 8 8 7 1 4 0 4 8 5 2 3 9 0 1 9 1 5 1 7 4 8 6 2 1 6 8 8 0
##  [8245] 1 2 8 4 7 8 9 0 1 2 3 4 6 7 8 9 0 1 2 3 4 7 8 9 1 4 5 3 3 0 9 5 4 8 0 8
##  [8281] 4 6 7 0 7 7 1 6 9 1 3 6 2 3 5 2 3 8 9 5 8 8 7 1 7 1 1 0 3 4 2 6 4 7 4 2
##  [8317] 7 4 2 9 2 7 9 2 1 0 6 5 3 4 8 5 9 6 9 0 6 3 0 6 1 6 0 0 1 2 3 4 5 6 7 0
##  [8353] 1 2 3 4 7 8 9 0 1 2 3 4 7 2 5 1 6 4 3 9 9 0 9 7 1 6 4 9 6 2 0 9 8 6 5 7
##  [8389] 0 0 1 7 4 3 2 4 1 3 7 6 4 7 7 7 9 8 4 3 5 2 5 3 5 8 0 5 4 7 1 3 1 7 9 6
##  [8425] 2 0 9 1 7 3 3 9 1 6 4 3 9 8 2 1 8 6 4 1 5 5 6 5 0 1 2 3 4 5 6 7 6 9 0 1
##  [8461] 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 6 9 7 0 2 3 4 3 8 5 1 3 0 1 2 1 3 2
##  [8497] 0 7 2 6 4 0 5 9 9 8 9 5 3 1 7 4 7 0 0 6 6 6 3 7 4 2 8 9 8 7 1 8 0 4 8 5
##  [8533] 2 3 9 0 1 9 1 5 1 7 6 1 2 1 6 8 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 0
##  [8569] 1 2 3 5 6 7 8 1 0 4 5 6 6 3 4 4 3 8 1 0 6 4 9 7 2 9 2 0 9 3 3 9 1 5 2 3
##  [8605] 1 6 7 3 7 8 4 0 2 4 0 2 4 7 8 0 7 0 6 9 3 2 4 8 6 0 5 7 5 1 0 8 1 6 7 2
##  [8641] 9 7 9 5 6 5 2 6 2 8 1 7 5 5 7 3 5 0 1 1 3 8 4 9 4 5 1 8 6 8 9 0 1 2 3 4
##  [8677] 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 3 5 3 2 9 3 2 1 4 5 5
##  [8713] 2 3 2 1 3 9 7 2 1 2 8 9 1 8 8 7 8 1 0 0 6 7 7 8 7 5 0 6 1 5 7 4 6 1 2 5
##  [8749] 0 7 9 9 0 3 4 4 8 4 1 8 6 5 9 0 0 0 3 7 1 6 4 6 0 4 5 4 1 3 8 6 3 9 9 5
##  [8785] 9 3 7 8 5 6 4 7 6 2 2 0 9 4 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
##  [8821] 2 3 4 5 6 7 8 9 6 4 2 6 4 7 5 5 4 7 2 9 3 9 3 8 2 0 9 5 6 0 1 0 6 5 3 5
##  [8857] 3 8 0 0 3 4 1 5 3 0 8 3 0 6 2 7 8 1 7 1 3 8 5 4 2 0 9 7 6 7 4 1 6 5 6 7
##  [8893] 1 9 8 0 6 9 4 9 9 6 2 3 7 1 9 2 2 5 3 7 8 0 1 2 3 4 7 8 9 0 1 2 3 4 7 8
##  [8929] 9 0 1 7 8 9 8 9 2 6 1 3 5 4 8 2 6 4 3 4 5 9 2 0 3 9 4 9 7 3 8 7 4 4 9 8
##  [8965] 5 8 2 6 6 2 3 1 3 2 7 3 1 9 0 1 1 3 5 0 7 8 1 5 1 4 6 0 0 4 9 1 6 6 9 0
##  [9001] 7 6 1 1 0 1 2 3 4 2 2 3 4 5 6 2 0 1 2 7 8 6 3 9 2 1 9 3 9 6 1 7 2 4 4 5
##  [9037] 7 0 0 1 6 6 8 2 7 7 2 4 2 1 6 1 0 6 9 8 3 9 6 3 0 1 2 3 4 5 6 7 8 9 0 1
##  [9073] 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 1 6 8 9 9 0 1 2 4 4 3 7 4 4 4 0 3 8
##  [9109] 7 5 8 2 1 7 5 3 8 5 2 5 1 1 6 2 1 3 8 6 4 2 6 2 5 5 0 2 8 0 6 8 1 7 9 1
##  [9145] 9 2 6 7 6 6 8 7 4 9 2 1 3 3 0 5 5 8 0 3 7 9 7 0 2 7 9 1 7 8 0 3 5 3 6 0
##  [9181] 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 7 8 9 6 4 2 6 4 7 8 9 2
##  [9217] 9 3 9 3 0 0 1 0 4 2 6 3 5 3 0 3 4 1 5 3 0 8 3 0 6 1 7 8 0 9 2 6 7 1 9 6
##  [9253] 9 4 9 9 6 7 1 2 5 3 7 8 0 1 2 4 5 6 7 8 9 0 1 3 4 5 6 7 5 0 1 3 4 7 8 9
##  [9289] 7 5 5 1 9 9 7 1 0 0 5 9 7 1 7 2 2 3 6 8 3 2 0 0 6 1 7 5 5 6 2 9 4 8 8 7
##  [9325] 1 0 8 7 7 5 8 5 3 4 6 1 1 5 5 0 7 2 3 6 4 1 2 4 1 5 4 2 0 4 8 6 1 9 0 2
##  [9361] 5 6 9 3 6 3 6 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 5 6 7 8 1
##  [9397] 0 9 5 7 5 1 8 6 9 0 4 1 9 3 8 4 4 7 0 1 9 2 8 7 8 2 5 9 6 0 6 5 5 3 3 3
##  [9433] 9 8 1 1 0 6 1 0 0 6 2 1 1 3 2 7 7 8 8 7 8 4 6 0 2 0 7 0 3 6 8 7 1 5 9 9
##  [9469] 3 7 2 4 9 4 3 6 2 2 5 3 2 5 5 9 4 1 7 2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
##  [9505] 6 7 8 9 0 1 2 3 4 5 6 7 8 9 1 0 1 2 7 5 3 4 4 0 0 6 9 6 6 5 7 1 3 4 4 9
##  [9541] 1 4 0 7 9 5 7 2 3 1 4 4 0 9 9 6 1 8 3 3 7 3 9 8 8 4 7 7 6 2 1 9 8 7 8 8
##  [9577] 7 2 2 3 9 3 3 5 5 0 7 4 5 6 5 1 4 1 1 2 8 2 6 1 5 0 1 2 3 4 5 6 7 8 9 0
##  [9613] 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 6 0 1 2 3 7 9 4 7 1 9 1 7 1 4 0
##  [9649] 0 1 7 5 7 1 3 3 3 1 6 9 7 1 3 0 7 6 0 8 9 5 3 5 4 8 1 5 9 0 6 5 3 8 1 4
##  [9685] 7 5 2 0 0 1 7 8 9 6 8 8 2 3 6 1 2 9 5 2 0 1 2 3 4 5 6 7 5 9 0 1 2 3 4 5
##  [9721] 6 7 8 9 0 1 2 3 4 6 6 7 8 9 7 4 6 1 4 0 9 9 3 7 8 0 7 5 8 5 3 2 2 0 5 5
##  [9757] 6 0 3 8 1 0 3 0 4 7 4 9 0 9 5 7 1 7 1 6 6 5 6 2 8 7 6 4 9 9 5 3 7 4 3 0
##  [9793] 9 6 6 1 1 3 2 1 0 0 1 2 3 4 7 8 9 0 1 2 3 4 5 6 7 8 0 1 2 3 4 7 8 9 0 8
##  [9829] 3 9 5 5 2 6 8 4 1 7 1 7 3 5 6 9 1 1 1 2 1 2 0 7 7 5 8 2 9 8 6 7 3 4 6 8
##  [9865] 7 0 4 2 7 7 5 4 3 4 2 8 1 5 1 0 2 3 3 5 7 0 6 8 6 3 9 9 5 2 7 7 1 0 1 7
##  [9901] 8 9 0 1 0 3 4 5 6 7 8 0 1 2 3 4 7 8 9 7 8 6 4 1 9 5 8 4 4 7 0 1 9 2 8 7
##  [9937] 5 2 6 0 4 5 3 3 5 9 1 4 0 6 1 0 0 6 2 1 1 7 7 8 4 6 0 7 0 3 6 8 7 1 5 2
##  [9973] 4 9 4 3 6 4 1 7 2 6 5 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6

If we look at the first prediction then the actual first data item in our test data and see that they do match. Our prediction was correct.

pred_test[1]
## [1] 7
mnist$test$y[1]
## [1] 7