MNIST Notebook

The MNIST dataset is included with Keras and can be accessed using the dataset_mnist() function. Here we load the dataset then create variables for our test and training data:

library(keras)
mnist <- dataset_mnist()
x_train <- mnist$train$x
y_train <- mnist$train$y
x_test <- mnist$test$x
y_test <- mnist$test$y

The x data is a 3-d array (images,width,height) of grayscale values . To prepare the data for training we convert the 3-d arrays into matrices by reshaping width and height into a single dimension (28x28 images are flattened into length 784 vectors). Then, we convert the grayscale values from integers ranging between 0 to 255 into floating point values ranging between 0 and 1:

# reshape
x_train <- array_reshape(x_train, c(nrow(x_train), 784))
x_test <- array_reshape(x_test, c(nrow(x_test), 784))
# rescale
x_train <- x_train / 255
x_test <- x_test / 255

Note that we use the array_reshape() function rather than the dim<-() function to reshape the array. This is so that the data is re-interpreted using row-major semantics (as opposed to R’s default column-major semantics), which is in turn compatible with the way that the numerical libraries called by Keras interpret array dimensions.

The y data is an integer vector with values ranging from 0 to 9. To prepare this data for training we one-hot encode the vectors into binary class matrices using the Keras to_categorical() function:

y_train <- to_categorical(y_train, 10)
y_test <- to_categorical(y_test, 10)

Defining the Model

The core data structure of Keras is a model, a way to organize layers. The simplest type of model is the Sequential model, a linear stack of layers.

We begin by creating a sequential model and then adding layers using the pipe (%>%) operator:

model <- keras_model_sequential() 
model %>% 
  layer_dense(units = 256, activation = 'relu', input_shape = c(784)) %>% 
  layer_dropout(rate = 0.4) %>% 
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dropout(rate = 0.3) %>%
  layer_dense(units = 10, activation = 'softmax')

The input_shape argument to the first layer specifies the shape of the input data (a length 784 numeric vector representing a grayscale image). The final layer outputs a length 10 numeric vector (probabilities for each digit) using a softmax activation function.

Use the summary() function to print the details of the model:

summary(model)

__________________________________________________________________
Layer (type)                 Output Shape               Param #   
==================================================================
dense_6 (Dense)              (None, 256)                200960    
__________________________________________________________________
dropout_4 (Dropout)          (None, 256)                0         
__________________________________________________________________
dense_7 (Dense)              (None, 128)                32896     
__________________________________________________________________
dropout_5 (Dropout)          (None, 128)                0         
__________________________________________________________________
dense_8 (Dense)              (None, 10)                 1290      
==================================================================
Total params: 235,146
Trainable params: 235,146
Non-trainable params: 0
__________________________________________________________________

Next, compile the model with appropriate loss function, optimizer, and metrics:

model %>% compile(
  loss = 'categorical_crossentropy',
  optimizer = optimizer_rmsprop(),
  metrics = c('accuracy')
)

Training and Evaluation

Use the fit() function to train the model for 30 epochs using batches of 128 images:

history <- model %>% fit(
  x_train, y_train, 
  epochs = 30, batch_size = 128, 
  validation_split = 0.2
)

Train on 48000 samples, validate on 12000 samples
Epoch 1/30

  128/48000 [..............................] - ETA: 1:31 - loss: 2.4827 - acc: 0.0547
 2816/48000 [>.............................] - ETA: 4s - loss: 1.4091 - acc: 0.5586  
 5376/48000 [==>...........................] - ETA: 2s - loss: 1.0883 - acc: 0.6590
 8064/48000 [====>.........................] - ETA: 1s - loss: 0.9215 - acc: 0.7132
10752/48000 [=====>........................] - ETA: 1s - loss: 0.8226 - acc: 0.7479
13312/48000 [=======>......................] - ETA: 1s - loss: 0.7510 - acc: 0.7702
15872/48000 [========>.....................] - ETA: 1s - loss: 0.6973 - acc: 0.7869
18560/48000 [==========>...................] - ETA: 0s - loss: 0.6511 - acc: 0.7999
21248/48000 [============>.................] - ETA: 0s - loss: 0.6165 - acc: 0.8114
23936/48000 [=============>................] - ETA: 0s - loss: 0.5876 - acc: 0.8199
26624/48000 [===============>..............] - ETA: 0s - loss: 0.5625 - acc: 0.8281
29312/48000 [=================>............] - ETA: 0s - loss: 0.5392 - acc: 0.8359
31872/48000 [==================>...........] - ETA: 0s - loss: 0.5177 - acc: 0.8424
34560/48000 [====================>.........] - ETA: 0s - loss: 0.4990 - acc: 0.8482
37120/48000 [======================>.......] - ETA: 0s - loss: 0.4835 - acc: 0.8531
39680/48000 [=======================>......] - ETA: 0s - loss: 0.4714 - acc: 0.8573
42240/48000 [=========================>....] - ETA: 0s - loss: 0.4604 - acc: 0.8610
44800/48000 [===========================>..] - ETA: 0s - loss: 0.4485 - acc: 0.8643
47360/48000 [============================>.] - ETA: 0s - loss: 0.4377 - acc: 0.8676
48000/48000 [==============================] - 1s 29us/step - loss: 0.4350 - acc: 0.8685 - val_loss: 0.1578 - val_acc: 0.9524
Epoch 2/30

  128/48000 [..............................] - ETA: 3s - loss: 0.1923 - acc: 0.9531
 1280/48000 [..............................] - ETA: 2s - loss: 0.2363 - acc: 0.9344
 2944/48000 [>.............................] - ETA: 1s - loss: 0.2052 - acc: 0.9409
 4736/48000 [=>............................] - ETA: 1s - loss: 0.2078 - acc: 0.9398
 6528/48000 [===>..........................] - ETA: 1s - loss: 0.2197 - acc: 0.9370
 8448/48000 [====>.........................] - ETA: 1s - loss: 0.2225 - acc: 0.9353
11008/48000 [=====>........................] - ETA: 1s - loss: 0.2218 - acc: 0.9361
13568/48000 [=======>......................] - ETA: 0s - loss: 0.2168 - acc: 0.9373
16128/48000 [=========>....................] - ETA: 0s - loss: 0.2128 - acc: 0.9378
18688/48000 [==========>...................] - ETA: 0s - loss: 0.2137 - acc: 0.9377
21248/48000 [============>.................] - ETA: 0s - loss: 0.2130 - acc: 0.9380
23552/48000 [=============>................] - ETA: 0s - loss: 0.2140 - acc: 0.9374
26240/48000 [===============>..............] - ETA: 0s - loss: 0.2124 - acc: 0.9379
28928/48000 [=================>............] - ETA: 0s - loss: 0.2119 - acc: 0.9386
31744/48000 [==================>...........] - ETA: 0s - loss: 0.2099 - acc: 0.9389
34432/48000 [====================>.........] - ETA: 0s - loss: 0.2083 - acc: 0.9391
37120/48000 [======================>.......] - ETA: 0s - loss: 0.2088 - acc: 0.9387
39936/48000 [=======================>......] - ETA: 0s - loss: 0.2068 - acc: 0.9393
42624/48000 [=========================>....] - ETA: 0s - loss: 0.2054 - acc: 0.9393
45440/48000 [===========================>..] - ETA: 0s - loss: 0.2047 - acc: 0.9396
48000/48000 [==============================] - 1s 25us/step - loss: 0.2016 - acc: 0.9405 - val_loss: 0.1215 - val_acc: 0.9637
Epoch 3/30

  128/48000 [..............................] - ETA: 1s - loss: 0.1257 - acc: 0.9609
 2432/48000 [>.............................] - ETA: 1s - loss: 0.1551 - acc: 0.9544
 5120/48000 [==>...........................] - ETA: 0s - loss: 0.1590 - acc: 0.9512
 7936/48000 [===>..........................] - ETA: 0s - loss: 0.1617 - acc: 0.9502
10624/48000 [=====>........................] - ETA: 0s - loss: 0.1652 - acc: 0.9509
13312/48000 [=======>......................] - ETA: 0s - loss: 0.1647 - acc: 0.9507
15872/48000 [========>.....................] - ETA: 0s - loss: 0.1603 - acc: 0.9519
18432/48000 [==========>...................] - ETA: 0s - loss: 0.1587 - acc: 0.9526
21120/48000 [============>.................] - ETA: 0s - loss: 0.1603 - acc: 0.9525
23552/48000 [=============>................] - ETA: 0s - loss: 0.1577 - acc: 0.9526
26112/48000 [===============>..............] - ETA: 0s - loss: 0.1595 - acc: 0.9526
28544/48000 [================>.............] - ETA: 0s - loss: 0.1577 - acc: 0.9530
30976/48000 [==================>...........] - ETA: 0s - loss: 0.1570 - acc: 0.9531
33408/48000 [===================>..........] - ETA: 0s - loss: 0.1568 - acc: 0.9532
35840/48000 [=====================>........] - ETA: 0s - loss: 0.1571 - acc: 0.9535
38272/48000 [======================>.......] - ETA: 0s - loss: 0.1560 - acc: 0.9535
40704/48000 [========================>.....] - ETA: 0s - loss: 0.1558 - acc: 0.9537
43136/48000 [=========================>....] - ETA: 0s - loss: 0.1570 - acc: 0.9535
45568/48000 [===========================>..] - ETA: 0s - loss: 0.1554 - acc: 0.9538
47872/48000 [============================>.] - ETA: 0s - loss: 0.1557 - acc: 0.9538
48000/48000 [==============================] - 1s 24us/step - loss: 0.1558 - acc: 0.9537 - val_loss: 0.1047 - val_acc: 0.9682
Epoch 4/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0703 - acc: 0.9844
 1280/48000 [..............................] - ETA: 2s - loss: 0.1255 - acc: 0.9570
 2432/48000 [>.............................] - ETA: 2s - loss: 0.1359 - acc: 0.9576
 3712/48000 [=>............................] - ETA: 1s - loss: 0.1418 - acc: 0.9553
 5632/48000 [==>...........................] - ETA: 1s - loss: 0.1305 - acc: 0.9592
 7552/48000 [===>..........................] - ETA: 1s - loss: 0.1314 - acc: 0.9596
10240/48000 [=====>........................] - ETA: 1s - loss: 0.1324 - acc: 0.9600
12544/48000 [======>.......................] - ETA: 1s - loss: 0.1309 - acc: 0.9607
15232/48000 [========>.....................] - ETA: 0s - loss: 0.1341 - acc: 0.9595
17920/48000 [==========>...................] - ETA: 0s - loss: 0.1342 - acc: 0.9598
20736/48000 [===========>..................] - ETA: 0s - loss: 0.1334 - acc: 0.9597
23552/48000 [=============>................] - ETA: 0s - loss: 0.1342 - acc: 0.9594
26368/48000 [===============>..............] - ETA: 0s - loss: 0.1355 - acc: 0.9591
29056/48000 [=================>............] - ETA: 0s - loss: 0.1357 - acc: 0.9592
31744/48000 [==================>...........] - ETA: 0s - loss: 0.1339 - acc: 0.9595
34432/48000 [====================>.........] - ETA: 0s - loss: 0.1339 - acc: 0.9597
37120/48000 [======================>.......] - ETA: 0s - loss: 0.1342 - acc: 0.9594
39808/48000 [=======================>......] - ETA: 0s - loss: 0.1341 - acc: 0.9596
42368/48000 [=========================>....] - ETA: 0s - loss: 0.1326 - acc: 0.9600
45056/48000 [===========================>..] - ETA: 0s - loss: 0.1320 - acc: 0.9602
47872/48000 [============================>.] - ETA: 0s - loss: 0.1329 - acc: 0.9601
48000/48000 [==============================] - 1s 25us/step - loss: 0.1330 - acc: 0.9601 - val_loss: 0.0945 - val_acc: 0.9738
Epoch 5/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0844 - acc: 0.9688
 1280/48000 [..............................] - ETA: 2s - loss: 0.0913 - acc: 0.9680
 2432/48000 [>.............................] - ETA: 2s - loss: 0.1069 - acc: 0.9671
 4352/48000 [=>............................] - ETA: 1s - loss: 0.1107 - acc: 0.9685
 7040/48000 [===>..........................] - ETA: 1s - loss: 0.1140 - acc: 0.9665
 9600/48000 [=====>........................] - ETA: 1s - loss: 0.1158 - acc: 0.9656
12288/48000 [======>.......................] - ETA: 0s - loss: 0.1139 - acc: 0.9657
14848/48000 [========>.....................] - ETA: 0s - loss: 0.1148 - acc: 0.9661
17536/48000 [=========>....................] - ETA: 0s - loss: 0.1145 - acc: 0.9658
20224/48000 [===========>..................] - ETA: 0s - loss: 0.1144 - acc: 0.9661
22784/48000 [=============>................] - ETA: 0s - loss: 0.1131 - acc: 0.9667
25344/48000 [==============>...............] - ETA: 0s - loss: 0.1148 - acc: 0.9659
28032/48000 [================>.............] - ETA: 0s - loss: 0.1131 - acc: 0.9665
30720/48000 [==================>...........] - ETA: 0s - loss: 0.1138 - acc: 0.9661
33408/48000 [===================>..........] - ETA: 0s - loss: 0.1156 - acc: 0.9656
36096/48000 [=====================>........] - ETA: 0s - loss: 0.1156 - acc: 0.9654
38656/48000 [=======================>......] - ETA: 0s - loss: 0.1145 - acc: 0.9657
41344/48000 [========================>.....] - ETA: 0s - loss: 0.1151 - acc: 0.9655
43904/48000 [==========================>...] - ETA: 0s - loss: 0.1159 - acc: 0.9652
46592/48000 [============================>.] - ETA: 0s - loss: 0.1161 - acc: 0.9651
48000/48000 [==============================] - 1s 25us/step - loss: 0.1161 - acc: 0.9653 - val_loss: 0.0970 - val_acc: 0.9739
Epoch 6/30

  128/48000 [..............................] - ETA: 2s - loss: 0.0646 - acc: 0.9766
 1280/48000 [..............................] - ETA: 2s - loss: 0.1014 - acc: 0.9695
 2560/48000 [>.............................] - ETA: 2s - loss: 0.0945 - acc: 0.9684
 3968/48000 [=>............................] - ETA: 1s - loss: 0.0911 - acc: 0.9708
 5376/48000 [==>...........................] - ETA: 1s - loss: 0.0897 - acc: 0.9727
 6784/48000 [===>..........................] - ETA: 1s - loss: 0.0977 - acc: 0.9708
 8192/48000 [====>.........................] - ETA: 1s - loss: 0.0982 - acc: 0.9705
10240/48000 [=====>........................] - ETA: 1s - loss: 0.0988 - acc: 0.9703
12288/48000 [======>.......................] - ETA: 1s - loss: 0.0976 - acc: 0.9711
14336/48000 [=======>......................] - ETA: 1s - loss: 0.0958 - acc: 0.9713
16256/48000 [=========>....................] - ETA: 1s - loss: 0.0956 - acc: 0.9715
18560/48000 [==========>...................] - ETA: 0s - loss: 0.0964 - acc: 0.9710
20736/48000 [===========>..................] - ETA: 0s - loss: 0.0966 - acc: 0.9710
23040/48000 [=============>................] - ETA: 0s - loss: 0.0988 - acc: 0.9704
25472/48000 [==============>...............] - ETA: 0s - loss: 0.0963 - acc: 0.9711
28160/48000 [================>.............] - ETA: 0s - loss: 0.0984 - acc: 0.9706
30848/48000 [==================>...........] - ETA: 0s - loss: 0.0987 - acc: 0.9709
33536/48000 [===================>..........] - ETA: 0s - loss: 0.1006 - acc: 0.9706
36224/48000 [=====================>........] - ETA: 0s - loss: 0.1019 - acc: 0.9701
39040/48000 [=======================>......] - ETA: 0s - loss: 0.1035 - acc: 0.9697
41728/48000 [=========================>....] - ETA: 0s - loss: 0.1034 - acc: 0.9698
44544/48000 [==========================>...] - ETA: 0s - loss: 0.1037 - acc: 0.9695
47232/48000 [============================>.] - ETA: 0s - loss: 0.1029 - acc: 0.9697
48000/48000 [==============================] - 1s 28us/step - loss: 0.1029 - acc: 0.9696 - val_loss: 0.0875 - val_acc: 0.9763
Epoch 7/30

  128/48000 [..............................] - ETA: 2s - loss: 0.0627 - acc: 0.9766
 1280/48000 [..............................] - ETA: 2s - loss: 0.0834 - acc: 0.9727
 2432/48000 [>.............................] - ETA: 2s - loss: 0.0921 - acc: 0.9733
 4096/48000 [=>............................] - ETA: 1s - loss: 0.0939 - acc: 0.9719
 6400/48000 [===>..........................] - ETA: 1s - loss: 0.0867 - acc: 0.9731
 8704/48000 [====>.........................] - ETA: 1s - loss: 0.0852 - acc: 0.9737
10880/48000 [=====>........................] - ETA: 1s - loss: 0.0871 - acc: 0.9728
13184/48000 [=======>......................] - ETA: 0s - loss: 0.0893 - acc: 0.9720
15360/48000 [========>.....................] - ETA: 0s - loss: 0.0932 - acc: 0.9716
17664/48000 [==========>...................] - ETA: 0s - loss: 0.0938 - acc: 0.9714
19968/48000 [===========>..................] - ETA: 0s - loss: 0.0950 - acc: 0.9716
22272/48000 [============>.................] - ETA: 0s - loss: 0.0982 - acc: 0.9707
24576/48000 [==============>...............] - ETA: 0s - loss: 0.0973 - acc: 0.9710
27008/48000 [===============>..............] - ETA: 0s - loss: 0.0954 - acc: 0.9712
29440/48000 [=================>............] - ETA: 0s - loss: 0.0940 - acc: 0.9711
32256/48000 [===================>..........] - ETA: 0s - loss: 0.0946 - acc: 0.9711
34944/48000 [====================>.........] - ETA: 0s - loss: 0.0957 - acc: 0.9709
37248/48000 [======================>.......] - ETA: 0s - loss: 0.0956 - acc: 0.9712
39808/48000 [=======================>......] - ETA: 0s - loss: 0.0963 - acc: 0.9708
42496/48000 [=========================>....] - ETA: 0s - loss: 0.0962 - acc: 0.9710
45056/48000 [===========================>..] - ETA: 0s - loss: 0.0962 - acc: 0.9712
47232/48000 [============================>.] - ETA: 0s - loss: 0.0960 - acc: 0.9711
48000/48000 [==============================] - 1s 27us/step - loss: 0.0961 - acc: 0.9711 - val_loss: 0.0987 - val_acc: 0.9760
Epoch 8/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0824 - acc: 0.9688
 1280/48000 [..............................] - ETA: 2s - loss: 0.0848 - acc: 0.9695
 2432/48000 [>.............................] - ETA: 2s - loss: 0.0898 - acc: 0.9688
 4736/48000 [=>............................] - ETA: 1s - loss: 0.0909 - acc: 0.9711
 7424/48000 [===>..........................] - ETA: 1s - loss: 0.0883 - acc: 0.9713
10112/48000 [=====>........................] - ETA: 1s - loss: 0.0814 - acc: 0.9738
12928/48000 [=======>......................] - ETA: 0s - loss: 0.0817 - acc: 0.9746
15744/48000 [========>.....................] - ETA: 0s - loss: 0.0845 - acc: 0.9741
18560/48000 [==========>...................] - ETA: 0s - loss: 0.0835 - acc: 0.9739
21376/48000 [============>.................] - ETA: 0s - loss: 0.0871 - acc: 0.9735
24064/48000 [==============>...............] - ETA: 0s - loss: 0.0890 - acc: 0.9734
26880/48000 [===============>..............] - ETA: 0s - loss: 0.0896 - acc: 0.9734
29696/48000 [=================>............] - ETA: 0s - loss: 0.0895 - acc: 0.9736
32384/48000 [===================>..........] - ETA: 0s - loss: 0.0893 - acc: 0.9738
35072/48000 [====================>.........] - ETA: 0s - loss: 0.0895 - acc: 0.9739
37760/48000 [======================>.......] - ETA: 0s - loss: 0.0896 - acc: 0.9736
40448/48000 [========================>.....] - ETA: 0s - loss: 0.0894 - acc: 0.9735
43264/48000 [==========================>...] - ETA: 0s - loss: 0.0891 - acc: 0.9737
45952/48000 [===========================>..] - ETA: 0s - loss: 0.0893 - acc: 0.9738
48000/48000 [==============================] - 1s 24us/step - loss: 0.0903 - acc: 0.9734 - val_loss: 0.0897 - val_acc: 0.9768
Epoch 9/30

  128/48000 [..............................] - ETA: 3s - loss: 0.1020 - acc: 0.9688
 1280/48000 [..............................] - ETA: 2s - loss: 0.0884 - acc: 0.9719
 2432/48000 [>.............................] - ETA: 2s - loss: 0.0721 - acc: 0.9762
 3968/48000 [=>............................] - ETA: 1s - loss: 0.0708 - acc: 0.9783
 5760/48000 [==>...........................] - ETA: 1s - loss: 0.0697 - acc: 0.9783
 7424/48000 [===>..........................] - ETA: 1s - loss: 0.0715 - acc: 0.9776
 9216/48000 [====>.........................] - ETA: 1s - loss: 0.0693 - acc: 0.9792
11008/48000 [=====>........................] - ETA: 1s - loss: 0.0766 - acc: 0.9776
12800/48000 [=======>......................] - ETA: 1s - loss: 0.0787 - acc: 0.9770
15104/48000 [========>.....................] - ETA: 1s - loss: 0.0803 - acc: 0.9767
17408/48000 [=========>....................] - ETA: 0s - loss: 0.0805 - acc: 0.9765
19712/48000 [===========>..................] - ETA: 0s - loss: 0.0823 - acc: 0.9755
22016/48000 [============>.................] - ETA: 0s - loss: 0.0824 - acc: 0.9755
24448/48000 [==============>...............] - ETA: 0s - loss: 0.0823 - acc: 0.9752
26368/48000 [===============>..............] - ETA: 0s - loss: 0.0824 - acc: 0.9752
28800/48000 [=================>............] - ETA: 0s - loss: 0.0820 - acc: 0.9754
31232/48000 [==================>...........] - ETA: 0s - loss: 0.0811 - acc: 0.9759
33536/48000 [===================>..........] - ETA: 0s - loss: 0.0820 - acc: 0.9758
35840/48000 [=====================>........] - ETA: 0s - loss: 0.0820 - acc: 0.9757
38144/48000 [======================>.......] - ETA: 0s - loss: 0.0824 - acc: 0.9756
40448/48000 [========================>.....] - ETA: 0s - loss: 0.0826 - acc: 0.9755
42752/48000 [=========================>....] - ETA: 0s - loss: 0.0828 - acc: 0.9756
45056/48000 [===========================>..] - ETA: 0s - loss: 0.0832 - acc: 0.9755
47360/48000 [============================>.] - ETA: 0s - loss: 0.0834 - acc: 0.9755
48000/48000 [==============================] - 1s 29us/step - loss: 0.0840 - acc: 0.9754 - val_loss: 0.0914 - val_acc: 0.9764
Epoch 10/30

  128/48000 [..............................] - ETA: 2s - loss: 0.0378 - acc: 0.9922
 1280/48000 [..............................] - ETA: 2s - loss: 0.0567 - acc: 0.9828
 2816/48000 [>.............................] - ETA: 1s - loss: 0.0643 - acc: 0.9790
 4608/48000 [=>............................] - ETA: 1s - loss: 0.0649 - acc: 0.9805
 6400/48000 [===>..........................] - ETA: 1s - loss: 0.0720 - acc: 0.9778
 8192/48000 [====>.........................] - ETA: 1s - loss: 0.0744 - acc: 0.9769
10368/48000 [=====>........................] - ETA: 1s - loss: 0.0743 - acc: 0.9768
12544/48000 [======>.......................] - ETA: 1s - loss: 0.0743 - acc: 0.9766
14592/48000 [========>.....................] - ETA: 0s - loss: 0.0726 - acc: 0.9772
16640/48000 [=========>....................] - ETA: 0s - loss: 0.0720 - acc: 0.9773
18944/48000 [==========>...................] - ETA: 0s - loss: 0.0731 - acc: 0.9770
21632/48000 [============>.................] - ETA: 0s - loss: 0.0758 - acc: 0.9766
24192/48000 [==============>...............] - ETA: 0s - loss: 0.0761 - acc: 0.9770
26880/48000 [===============>..............] - ETA: 0s - loss: 0.0784 - acc: 0.9768
29568/48000 [=================>............] - ETA: 0s - loss: 0.0789 - acc: 0.9768
32256/48000 [===================>..........] - ETA: 0s - loss: 0.0786 - acc: 0.9771
34944/48000 [====================>.........] - ETA: 0s - loss: 0.0779 - acc: 0.9774
37632/48000 [======================>.......] - ETA: 0s - loss: 0.0785 - acc: 0.9773
40192/48000 [========================>.....] - ETA: 0s - loss: 0.0779 - acc: 0.9775
42752/48000 [=========================>....] - ETA: 0s - loss: 0.0787 - acc: 0.9774
45440/48000 [===========================>..] - ETA: 0s - loss: 0.0795 - acc: 0.9773
47616/48000 [============================>.] - ETA: 0s - loss: 0.0801 - acc: 0.9772
48000/48000 [==============================] - 1s 27us/step - loss: 0.0802 - acc: 0.9770 - val_loss: 0.0933 - val_acc: 0.9775
Epoch 11/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0801 - acc: 0.9688
 1280/48000 [..............................] - ETA: 2s - loss: 0.1097 - acc: 0.9719
 3200/48000 [=>............................] - ETA: 1s - loss: 0.0858 - acc: 0.9778
 5504/48000 [==>...........................] - ETA: 1s - loss: 0.0797 - acc: 0.9789
 7808/48000 [===>..........................] - ETA: 1s - loss: 0.0772 - acc: 0.9791
10112/48000 [=====>........................] - ETA: 0s - loss: 0.0738 - acc: 0.9795
12544/48000 [======>.......................] - ETA: 0s - loss: 0.0726 - acc: 0.9796
14592/48000 [========>.....................] - ETA: 0s - loss: 0.0771 - acc: 0.9788
17024/48000 [=========>....................] - ETA: 0s - loss: 0.0753 - acc: 0.9792
19456/48000 [===========>..................] - ETA: 0s - loss: 0.0767 - acc: 0.9785
21888/48000 [============>.................] - ETA: 0s - loss: 0.0767 - acc: 0.9783
24576/48000 [==============>...............] - ETA: 0s - loss: 0.0760 - acc: 0.9787
27136/48000 [===============>..............] - ETA: 0s - loss: 0.0748 - acc: 0.9787
29824/48000 [=================>............] - ETA: 0s - loss: 0.0737 - acc: 0.9788
32384/48000 [===================>..........] - ETA: 0s - loss: 0.0734 - acc: 0.9789
35200/48000 [=====================>........] - ETA: 0s - loss: 0.0746 - acc: 0.9788
37888/48000 [======================>.......] - ETA: 0s - loss: 0.0747 - acc: 0.9785
40704/48000 [========================>.....] - ETA: 0s - loss: 0.0746 - acc: 0.9785
43520/48000 [==========================>...] - ETA: 0s - loss: 0.0755 - acc: 0.9783
46336/48000 [===========================>..] - ETA: 0s - loss: 0.0754 - acc: 0.9784
48000/48000 [==============================] - 1s 25us/step - loss: 0.0747 - acc: 0.9784 - val_loss: 0.0951 - val_acc: 0.9779
Epoch 12/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0258 - acc: 0.9922
 1280/48000 [..............................] - ETA: 2s - loss: 0.0493 - acc: 0.9844
 2560/48000 [>.............................] - ETA: 2s - loss: 0.0622 - acc: 0.9816
 4992/48000 [==>...........................] - ETA: 1s - loss: 0.0537 - acc: 0.9846
 7552/48000 [===>..........................] - ETA: 1s - loss: 0.0591 - acc: 0.9828
10240/48000 [=====>........................] - ETA: 0s - loss: 0.0654 - acc: 0.9810
12928/48000 [=======>......................] - ETA: 0s - loss: 0.0663 - acc: 0.9805
15616/48000 [========>.....................] - ETA: 0s - loss: 0.0656 - acc: 0.9804
18304/48000 [==========>...................] - ETA: 0s - loss: 0.0656 - acc: 0.9804
20992/48000 [============>.................] - ETA: 0s - loss: 0.0665 - acc: 0.9798
23680/48000 [=============>................] - ETA: 0s - loss: 0.0673 - acc: 0.9797
26240/48000 [===============>..............] - ETA: 0s - loss: 0.0680 - acc: 0.9798
28800/48000 [=================>............] - ETA: 0s - loss: 0.0694 - acc: 0.9794
31488/48000 [==================>...........] - ETA: 0s - loss: 0.0701 - acc: 0.9793
34176/48000 [====================>.........] - ETA: 0s - loss: 0.0714 - acc: 0.9791
36864/48000 [======================>.......] - ETA: 0s - loss: 0.0716 - acc: 0.9791
39552/48000 [=======================>......] - ETA: 0s - loss: 0.0716 - acc: 0.9789
42368/48000 [=========================>....] - ETA: 0s - loss: 0.0717 - acc: 0.9789
45056/48000 [===========================>..] - ETA: 0s - loss: 0.0716 - acc: 0.9790
47744/48000 [============================>.] - ETA: 0s - loss: 0.0727 - acc: 0.9787
48000/48000 [==============================] - 1s 24us/step - loss: 0.0726 - acc: 0.9787 - val_loss: 0.0899 - val_acc: 0.9787
Epoch 13/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0291 - acc: 0.9922
 1280/48000 [..............................] - ETA: 2s - loss: 0.0750 - acc: 0.9750
 2304/48000 [>.............................] - ETA: 2s - loss: 0.0774 - acc: 0.9766
 4480/48000 [=>............................] - ETA: 1s - loss: 0.0726 - acc: 0.9786
 6528/48000 [===>..........................] - ETA: 1s - loss: 0.0739 - acc: 0.9784
 8448/48000 [====>.........................] - ETA: 1s - loss: 0.0692 - acc: 0.9792
10752/48000 [=====>........................] - ETA: 1s - loss: 0.0689 - acc: 0.9787
13184/48000 [=======>......................] - ETA: 0s - loss: 0.0663 - acc: 0.9798
15488/48000 [========>.....................] - ETA: 0s - loss: 0.0632 - acc: 0.9804
17792/48000 [==========>...................] - ETA: 0s - loss: 0.0662 - acc: 0.9794
19968/48000 [===========>..................] - ETA: 0s - loss: 0.0679 - acc: 0.9792
22144/48000 [============>.................] - ETA: 0s - loss: 0.0690 - acc: 0.9790
24448/48000 [==============>...............] - ETA: 0s - loss: 0.0703 - acc: 0.9784
26880/48000 [===============>..............] - ETA: 0s - loss: 0.0712 - acc: 0.9785
29184/48000 [=================>............] - ETA: 0s - loss: 0.0718 - acc: 0.9786
31360/48000 [==================>...........] - ETA: 0s - loss: 0.0708 - acc: 0.9788
33792/48000 [====================>.........] - ETA: 0s - loss: 0.0720 - acc: 0.9785
36352/48000 [=====================>........] - ETA: 0s - loss: 0.0727 - acc: 0.9786
38912/48000 [=======================>......] - ETA: 0s - loss: 0.0733 - acc: 0.9786
41600/48000 [=========================>....] - ETA: 0s - loss: 0.0741 - acc: 0.9786
44288/48000 [==========================>...] - ETA: 0s - loss: 0.0738 - acc: 0.9788
46976/48000 [============================>.] - ETA: 0s - loss: 0.0740 - acc: 0.9789
48000/48000 [==============================] - 1s 27us/step - loss: 0.0745 - acc: 0.9790 - val_loss: 0.0876 - val_acc: 0.9780
Epoch 14/30

  128/48000 [..............................] - ETA: 2s - loss: 0.0516 - acc: 0.9922
 1152/48000 [..............................] - ETA: 2s - loss: 0.0435 - acc: 0.9870
 2304/48000 [>.............................] - ETA: 2s - loss: 0.0612 - acc: 0.9839
 3968/48000 [=>............................] - ETA: 1s - loss: 0.0576 - acc: 0.9834
 6656/48000 [===>..........................] - ETA: 1s - loss: 0.0598 - acc: 0.9821
 9344/48000 [====>.........................] - ETA: 1s - loss: 0.0641 - acc: 0.9817
11904/48000 [======>.......................] - ETA: 0s - loss: 0.0636 - acc: 0.9819
14080/48000 [=======>......................] - ETA: 0s - loss: 0.0663 - acc: 0.9814
16256/48000 [=========>....................] - ETA: 0s - loss: 0.0685 - acc: 0.9804
18688/48000 [==========>...................] - ETA: 0s - loss: 0.0651 - acc: 0.9813
20864/48000 [============>.................] - ETA: 0s - loss: 0.0639 - acc: 0.9815
23424/48000 [=============>................] - ETA: 0s - loss: 0.0630 - acc: 0.9819
26112/48000 [===============>..............] - ETA: 0s - loss: 0.0622 - acc: 0.9822
28800/48000 [=================>............] - ETA: 0s - loss: 0.0644 - acc: 0.9817
31488/48000 [==================>...........] - ETA: 0s - loss: 0.0635 - acc: 0.9820
34304/48000 [====================>.........] - ETA: 0s - loss: 0.0639 - acc: 0.9820
36992/48000 [======================>.......] - ETA: 0s - loss: 0.0648 - acc: 0.9816
39296/48000 [=======================>......] - ETA: 0s - loss: 0.0644 - acc: 0.9814
41984/48000 [=========================>....] - ETA: 0s - loss: 0.0655 - acc: 0.9813
44672/48000 [==========================>...] - ETA: 0s - loss: 0.0647 - acc: 0.9814
47360/48000 [============================>.] - ETA: 0s - loss: 0.0657 - acc: 0.9810
48000/48000 [==============================] - 1s 26us/step - loss: 0.0657 - acc: 0.9810 - val_loss: 0.0896 - val_acc: 0.9791
Epoch 15/30

  128/48000 [..............................] - ETA: 3s - loss: 0.1185 - acc: 0.9688
 1280/48000 [..............................] - ETA: 2s - loss: 0.0524 - acc: 0.9867
 2432/48000 [>.............................] - ETA: 2s - loss: 0.0551 - acc: 0.9864
 3840/48000 [=>............................] - ETA: 1s - loss: 0.0560 - acc: 0.9841
 5632/48000 [==>...........................] - ETA: 1s - loss: 0.0623 - acc: 0.9828
 7424/48000 [===>..........................] - ETA: 1s - loss: 0.0587 - acc: 0.9824
 9088/48000 [====>.........................] - ETA: 1s - loss: 0.0631 - acc: 0.9811
10752/48000 [=====>........................] - ETA: 1s - loss: 0.0636 - acc: 0.9812
12800/48000 [=======>......................] - ETA: 1s - loss: 0.0641 - acc: 0.9809
15488/48000 [========>.....................] - ETA: 1s - loss: 0.0629 - acc: 0.9811
18176/48000 [==========>...................] - ETA: 0s - loss: 0.0644 - acc: 0.9805
20864/48000 [============>.................] - ETA: 0s - loss: 0.0652 - acc: 0.9805
23552/48000 [=============>................] - ETA: 0s - loss: 0.0644 - acc: 0.9808
25600/48000 [===============>..............] - ETA: 0s - loss: 0.0644 - acc: 0.9806
27648/48000 [================>.............] - ETA: 0s - loss: 0.0634 - acc: 0.9808
29952/48000 [=================>............] - ETA: 0s - loss: 0.0630 - acc: 0.9811
32256/48000 [===================>..........] - ETA: 0s - loss: 0.0629 - acc: 0.9815
34432/48000 [====================>.........] - ETA: 0s - loss: 0.0628 - acc: 0.9816
36480/48000 [=====================>........] - ETA: 0s - loss: 0.0624 - acc: 0.9816
38528/48000 [=======================>......] - ETA: 0s - loss: 0.0624 - acc: 0.9814
40576/48000 [========================>.....] - ETA: 0s - loss: 0.0622 - acc: 0.9815
42624/48000 [=========================>....] - ETA: 0s - loss: 0.0630 - acc: 0.9814
45312/48000 [===========================>..] - ETA: 0s - loss: 0.0637 - acc: 0.9812
48000/48000 [==============================] - 1s 29us/step - loss: 0.0641 - acc: 0.9812 - val_loss: 0.0950 - val_acc: 0.9781
Epoch 16/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0414 - acc: 0.9922
 1152/48000 [..............................] - ETA: 2s - loss: 0.0838 - acc: 0.9766
 2304/48000 [>.............................] - ETA: 2s - loss: 0.0666 - acc: 0.9813
 3456/48000 [=>............................] - ETA: 2s - loss: 0.0653 - acc: 0.9809
 5632/48000 [==>...........................] - ETA: 1s - loss: 0.0705 - acc: 0.9801
 7808/48000 [===>..........................] - ETA: 1s - loss: 0.0680 - acc: 0.9814
10112/48000 [=====>........................] - ETA: 1s - loss: 0.0665 - acc: 0.9810
12544/48000 [======>.......................] - ETA: 1s - loss: 0.0650 - acc: 0.9820
14848/48000 [========>.....................] - ETA: 0s - loss: 0.0655 - acc: 0.9814
17024/48000 [=========>....................] - ETA: 0s - loss: 0.0686 - acc: 0.9811
19456/48000 [===========>..................] - ETA: 0s - loss: 0.0683 - acc: 0.9813
21760/48000 [============>.................] - ETA: 0s - loss: 0.0678 - acc: 0.9815
24064/48000 [==============>...............] - ETA: 0s - loss: 0.0681 - acc: 0.9817
26240/48000 [===============>..............] - ETA: 0s - loss: 0.0679 - acc: 0.9815
28160/48000 [================>.............] - ETA: 0s - loss: 0.0689 - acc: 0.9815
30208/48000 [=================>............] - ETA: 0s - loss: 0.0677 - acc: 0.9817
32640/48000 [===================>..........] - ETA: 0s - loss: 0.0659 - acc: 0.9820
35328/48000 [=====================>........] - ETA: 0s - loss: 0.0662 - acc: 0.9818
37888/48000 [======================>.......] - ETA: 0s - loss: 0.0668 - acc: 0.9817
40448/48000 [========================>.....] - ETA: 0s - loss: 0.0660 - acc: 0.9819
43264/48000 [==========================>...] - ETA: 0s - loss: 0.0679 - acc: 0.9816
45952/48000 [===========================>..] - ETA: 0s - loss: 0.0682 - acc: 0.9815
48000/48000 [==============================] - 1s 27us/step - loss: 0.0689 - acc: 0.9814 - val_loss: 0.0904 - val_acc: 0.9799
Epoch 17/30

  128/48000 [..............................] - ETA: 3s - loss: 0.1761 - acc: 0.9375
 1280/48000 [..............................] - ETA: 2s - loss: 0.0883 - acc: 0.9773
 2432/48000 [>.............................] - ETA: 2s - loss: 0.0744 - acc: 0.9790
 3584/48000 [=>............................] - ETA: 2s - loss: 0.0660 - acc: 0.9805
 5120/48000 [==>...........................] - ETA: 1s - loss: 0.0590 - acc: 0.9812
 7552/48000 [===>..........................] - ETA: 1s - loss: 0.0579 - acc: 0.9827
10112/48000 [=====>........................] - ETA: 1s - loss: 0.0606 - acc: 0.9826
12672/48000 [======>.......................] - ETA: 1s - loss: 0.0639 - acc: 0.9818
15488/48000 [========>.....................] - ETA: 0s - loss: 0.0648 - acc: 0.9817
18176/48000 [==========>...................] - ETA: 0s - loss: 0.0647 - acc: 0.9821
20992/48000 [============>.................] - ETA: 0s - loss: 0.0663 - acc: 0.9819
23680/48000 [=============>................] - ETA: 0s - loss: 0.0674 - acc: 0.9819
26368/48000 [===============>..............] - ETA: 0s - loss: 0.0655 - acc: 0.9824
28928/48000 [=================>............] - ETA: 0s - loss: 0.0654 - acc: 0.9823
31616/48000 [==================>...........] - ETA: 0s - loss: 0.0664 - acc: 0.9823
34304/48000 [====================>.........] - ETA: 0s - loss: 0.0654 - acc: 0.9825
37120/48000 [======================>.......] - ETA: 0s - loss: 0.0650 - acc: 0.9823
39552/48000 [=======================>......] - ETA: 0s - loss: 0.0650 - acc: 0.9823
42240/48000 [=========================>....] - ETA: 0s - loss: 0.0661 - acc: 0.9820
45056/48000 [===========================>..] - ETA: 0s - loss: 0.0668 - acc: 0.9818
47744/48000 [============================>.] - ETA: 0s - loss: 0.0661 - acc: 0.9818
48000/48000 [==============================] - 1s 25us/step - loss: 0.0661 - acc: 0.9819 - val_loss: 0.0910 - val_acc: 0.9805
Epoch 18/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0122 - acc: 1.0000
 1280/48000 [..............................] - ETA: 2s - loss: 0.0375 - acc: 0.9844
 3072/48000 [>.............................] - ETA: 1s - loss: 0.0442 - acc: 0.9860
 4736/48000 [=>............................] - ETA: 1s - loss: 0.0531 - acc: 0.9848
 6528/48000 [===>..........................] - ETA: 1s - loss: 0.0521 - acc: 0.9848
 8320/48000 [====>.........................] - ETA: 1s - loss: 0.0545 - acc: 0.9839
10112/48000 [=====>........................] - ETA: 1s - loss: 0.0523 - acc: 0.9842
12800/48000 [=======>......................] - ETA: 1s - loss: 0.0564 - acc: 0.9834
14848/48000 [========>.....................] - ETA: 0s - loss: 0.0554 - acc: 0.9841
17408/48000 [=========>....................] - ETA: 0s - loss: 0.0570 - acc: 0.9839
20096/48000 [===========>..................] - ETA: 0s - loss: 0.0560 - acc: 0.9839
22784/48000 [=============>................] - ETA: 0s - loss: 0.0554 - acc: 0.9839
25600/48000 [===============>..............] - ETA: 0s - loss: 0.0545 - acc: 0.9843
28416/48000 [================>.............] - ETA: 0s - loss: 0.0557 - acc: 0.9840
31104/48000 [==================>...........] - ETA: 0s - loss: 0.0586 - acc: 0.9836
33920/48000 [====================>.........] - ETA: 0s - loss: 0.0588 - acc: 0.9837
36480/48000 [=====================>........] - ETA: 0s - loss: 0.0580 - acc: 0.9836
39040/48000 [=======================>......] - ETA: 0s - loss: 0.0591 - acc: 0.9836
41728/48000 [=========================>....] - ETA: 0s - loss: 0.0592 - acc: 0.9836
44416/48000 [==========================>...] - ETA: 0s - loss: 0.0597 - acc: 0.9835
46336/48000 [===========================>..] - ETA: 0s - loss: 0.0596 - acc: 0.9835
48000/48000 [==============================] - 1s 27us/step - loss: 0.0598 - acc: 0.9835 - val_loss: 0.0972 - val_acc: 0.9792
Epoch 19/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0397 - acc: 0.9844
 1152/48000 [..............................] - ETA: 2s - loss: 0.0565 - acc: 0.9826
 2304/48000 [>.............................] - ETA: 2s - loss: 0.0570 - acc: 0.9835
 3328/48000 [=>............................] - ETA: 2s - loss: 0.0566 - acc: 0.9844
 4480/48000 [=>............................] - ETA: 2s - loss: 0.0532 - acc: 0.9844
 5504/48000 [==>...........................] - ETA: 2s - loss: 0.0544 - acc: 0.9846
 7168/48000 [===>..........................] - ETA: 1s - loss: 0.0534 - acc: 0.9838
 9216/48000 [====>.........................] - ETA: 1s - loss: 0.0568 - acc: 0.9840
11648/48000 [======>.......................] - ETA: 1s - loss: 0.0556 - acc: 0.9840
13952/48000 [=======>......................] - ETA: 1s - loss: 0.0577 - acc: 0.9834
16384/48000 [=========>....................] - ETA: 1s - loss: 0.0571 - acc: 0.9838
18688/48000 [==========>...................] - ETA: 0s - loss: 0.0559 - acc: 0.9838
21120/48000 [============>.................] - ETA: 0s - loss: 0.0584 - acc: 0.9835
23552/48000 [=============>................] - ETA: 0s - loss: 0.0601 - acc: 0.9828
26240/48000 [===============>..............] - ETA: 0s - loss: 0.0582 - acc: 0.9833
28928/48000 [=================>............] - ETA: 0s - loss: 0.0582 - acc: 0.9836
31616/48000 [==================>...........] - ETA: 0s - loss: 0.0590 - acc: 0.9835
34304/48000 [====================>.........] - ETA: 0s - loss: 0.0596 - acc: 0.9836
37120/48000 [======================>.......] - ETA: 0s - loss: 0.0591 - acc: 0.9836
39808/48000 [=======================>......] - ETA: 0s - loss: 0.0599 - acc: 0.9835
42496/48000 [=========================>....] - ETA: 0s - loss: 0.0597 - acc: 0.9836
45184/48000 [===========================>..] - ETA: 0s - loss: 0.0594 - acc: 0.9836
47744/48000 [============================>.] - ETA: 0s - loss: 0.0594 - acc: 0.9836
48000/48000 [==============================] - 1s 28us/step - loss: 0.0593 - acc: 0.9836 - val_loss: 0.1009 - val_acc: 0.9792
Epoch 20/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0831 - acc: 0.9531
 1536/48000 [..............................] - ETA: 2s - loss: 0.0589 - acc: 0.9798
 3840/48000 [=>............................] - ETA: 1s - loss: 0.0528 - acc: 0.9831
 6272/48000 [==>...........................] - ETA: 1s - loss: 0.0519 - acc: 0.9841
 8832/48000 [====>.........................] - ETA: 0s - loss: 0.0576 - acc: 0.9830
11392/48000 [======>.......................] - ETA: 0s - loss: 0.0595 - acc: 0.9831
14080/48000 [=======>......................] - ETA: 0s - loss: 0.0604 - acc: 0.9824
16768/48000 [=========>....................] - ETA: 0s - loss: 0.0585 - acc: 0.9834
19328/48000 [===========>..................] - ETA: 0s - loss: 0.0568 - acc: 0.9837
22016/48000 [============>.................] - ETA: 0s - loss: 0.0557 - acc: 0.9833
24704/48000 [==============>...............] - ETA: 0s - loss: 0.0550 - acc: 0.9836
27392/48000 [================>.............] - ETA: 0s - loss: 0.0561 - acc: 0.9832
30080/48000 [=================>............] - ETA: 0s - loss: 0.0560 - acc: 0.9831
32768/48000 [===================>..........] - ETA: 0s - loss: 0.0567 - acc: 0.9830
35456/48000 [=====================>........] - ETA: 0s - loss: 0.0570 - acc: 0.9830
38144/48000 [======================>.......] - ETA: 0s - loss: 0.0565 - acc: 0.9831
40832/48000 [========================>.....] - ETA: 0s - loss: 0.0566 - acc: 0.9831
43520/48000 [==========================>...] - ETA: 0s - loss: 0.0569 - acc: 0.9832
46080/48000 [===========================>..] - ETA: 0s - loss: 0.0569 - acc: 0.9831
48000/48000 [==============================] - 1s 24us/step - loss: 0.0587 - acc: 0.9827 - val_loss: 0.0993 - val_acc: 0.9794
Epoch 21/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0249 - acc: 0.9844
 2176/48000 [>.............................] - ETA: 1s - loss: 0.0578 - acc: 0.9858
 4864/48000 [==>...........................] - ETA: 0s - loss: 0.0534 - acc: 0.9856
 7680/48000 [===>..........................] - ETA: 0s - loss: 0.0497 - acc: 0.9857
10368/48000 [=====>........................] - ETA: 0s - loss: 0.0546 - acc: 0.9850
13184/48000 [=======>......................] - ETA: 0s - loss: 0.0570 - acc: 0.9845
15872/48000 [========>.....................] - ETA: 0s - loss: 0.0572 - acc: 0.9846
18432/48000 [==========>...................] - ETA: 0s - loss: 0.0565 - acc: 0.9847
21120/48000 [============>.................] - ETA: 0s - loss: 0.0560 - acc: 0.9845
23936/48000 [=============>................] - ETA: 0s - loss: 0.0551 - acc: 0.9846
26752/48000 [===============>..............] - ETA: 0s - loss: 0.0561 - acc: 0.9843
29568/48000 [=================>............] - ETA: 0s - loss: 0.0549 - acc: 0.9846
32128/48000 [===================>..........] - ETA: 0s - loss: 0.0564 - acc: 0.9841
34688/48000 [====================>.........] - ETA: 0s - loss: 0.0568 - acc: 0.9838
37376/48000 [======================>.......] - ETA: 0s - loss: 0.0564 - acc: 0.9838
39424/48000 [=======================>......] - ETA: 0s - loss: 0.0575 - acc: 0.9837
42112/48000 [=========================>....] - ETA: 0s - loss: 0.0578 - acc: 0.9836
44544/48000 [==========================>...] - ETA: 0s - loss: 0.0576 - acc: 0.9837
47232/48000 [============================>.] - ETA: 0s - loss: 0.0583 - acc: 0.9836
48000/48000 [==============================] - 1s 23us/step - loss: 0.0582 - acc: 0.9836 - val_loss: 0.0967 - val_acc: 0.9811
Epoch 22/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0191 - acc: 1.0000
 1280/48000 [..............................] - ETA: 2s - loss: 0.0320 - acc: 0.9883
 2432/48000 [>.............................] - ETA: 2s - loss: 0.0429 - acc: 0.9852
 3840/48000 [=>............................] - ETA: 1s - loss: 0.0469 - acc: 0.9852
 5504/48000 [==>...........................] - ETA: 1s - loss: 0.0470 - acc: 0.9856
 7168/48000 [===>..........................] - ETA: 1s - loss: 0.0478 - acc: 0.9860
 9344/48000 [====>.........................] - ETA: 1s - loss: 0.0521 - acc: 0.9860
11520/48000 [======>.......................] - ETA: 1s - loss: 0.0522 - acc: 0.9861
14080/48000 [=======>......................] - ETA: 1s - loss: 0.0509 - acc: 0.9861
16384/48000 [=========>....................] - ETA: 0s - loss: 0.0515 - acc: 0.9861
18688/48000 [==========>...................] - ETA: 0s - loss: 0.0526 - acc: 0.9858
21376/48000 [============>.................] - ETA: 0s - loss: 0.0541 - acc: 0.9850
24064/48000 [==============>...............] - ETA: 0s - loss: 0.0541 - acc: 0.9851
26752/48000 [===============>..............] - ETA: 0s - loss: 0.0555 - acc: 0.9846
29440/48000 [=================>............] - ETA: 0s - loss: 0.0566 - acc: 0.9843
32256/48000 [===================>..........] - ETA: 0s - loss: 0.0570 - acc: 0.9845
34944/48000 [====================>.........] - ETA: 0s - loss: 0.0562 - acc: 0.9846
37632/48000 [======================>.......] - ETA: 0s - loss: 0.0558 - acc: 0.9847
40320/48000 [========================>.....] - ETA: 0s - loss: 0.0566 - acc: 0.9845
43136/48000 [=========================>....] - ETA: 0s - loss: 0.0556 - acc: 0.9845
45824/48000 [===========================>..] - ETA: 0s - loss: 0.0560 - acc: 0.9845
48000/48000 [==============================] - 1s 26us/step - loss: 0.0563 - acc: 0.9845 - val_loss: 0.1019 - val_acc: 0.9809
Epoch 23/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0189 - acc: 0.9844
 1280/48000 [..............................] - ETA: 2s - loss: 0.0517 - acc: 0.9836
 2432/48000 [>.............................] - ETA: 2s - loss: 0.0462 - acc: 0.9852
 3712/48000 [=>............................] - ETA: 1s - loss: 0.0535 - acc: 0.9852
 5248/48000 [==>...........................] - ETA: 1s - loss: 0.0510 - acc: 0.9851
 6656/48000 [===>..........................] - ETA: 1s - loss: 0.0511 - acc: 0.9845
 8960/48000 [====>.........................] - ETA: 1s - loss: 0.0489 - acc: 0.9850
11648/48000 [======>.......................] - ETA: 1s - loss: 0.0515 - acc: 0.9854
14336/48000 [=======>......................] - ETA: 1s - loss: 0.0543 - acc: 0.9845
17024/48000 [=========>....................] - ETA: 0s - loss: 0.0516 - acc: 0.9853
19712/48000 [===========>..................] - ETA: 0s - loss: 0.0519 - acc: 0.9852
22400/48000 [=============>................] - ETA: 0s - loss: 0.0520 - acc: 0.9854
25088/48000 [==============>...............] - ETA: 0s - loss: 0.0550 - acc: 0.9851
27776/48000 [================>.............] - ETA: 0s - loss: 0.0554 - acc: 0.9847
30592/48000 [==================>...........] - ETA: 0s - loss: 0.0559 - acc: 0.9847
33280/48000 [===================>..........] - ETA: 0s - loss: 0.0554 - acc: 0.9847
35584/48000 [=====================>........] - ETA: 0s - loss: 0.0554 - acc: 0.9849
38272/48000 [======================>.......] - ETA: 0s - loss: 0.0556 - acc: 0.9849
40704/48000 [========================>.....] - ETA: 0s - loss: 0.0554 - acc: 0.9850
43008/48000 [=========================>....] - ETA: 0s - loss: 0.0565 - acc: 0.9848
44928/48000 [===========================>..] - ETA: 0s - loss: 0.0567 - acc: 0.9847
47232/48000 [============================>.] - ETA: 0s - loss: 0.0562 - acc: 0.9848
48000/48000 [==============================] - 1s 27us/step - loss: 0.0567 - acc: 0.9846 - val_loss: 0.0953 - val_acc: 0.9822
Epoch 24/30

  128/48000 [..............................] - ETA: 1s - loss: 0.0082 - acc: 1.0000
 2304/48000 [>.............................] - ETA: 1s - loss: 0.0359 - acc: 0.9887
 4608/48000 [=>............................] - ETA: 1s - loss: 0.0436 - acc: 0.9874
 7296/48000 [===>..........................] - ETA: 0s - loss: 0.0449 - acc: 0.9868
 9984/48000 [=====>........................] - ETA: 0s - loss: 0.0543 - acc: 0.9859
12672/48000 [======>.......................] - ETA: 0s - loss: 0.0527 - acc: 0.9861
15232/48000 [========>.....................] - ETA: 0s - loss: 0.0507 - acc: 0.9868
17920/48000 [==========>...................] - ETA: 0s - loss: 0.0500 - acc: 0.9868
20608/48000 [===========>..................] - ETA: 0s - loss: 0.0498 - acc: 0.9864
23296/48000 [=============>................] - ETA: 0s - loss: 0.0494 - acc: 0.9866
25984/48000 [===============>..............] - ETA: 0s - loss: 0.0514 - acc: 0.9862
28800/48000 [=================>............] - ETA: 0s - loss: 0.0524 - acc: 0.9858
31488/48000 [==================>...........] - ETA: 0s - loss: 0.0541 - acc: 0.9856
34176/48000 [====================>.........] - ETA: 0s - loss: 0.0555 - acc: 0.9854
36864/48000 [======================>.......] - ETA: 0s - loss: 0.0557 - acc: 0.9853
39552/48000 [=======================>......] - ETA: 0s - loss: 0.0558 - acc: 0.9853
42368/48000 [=========================>....] - ETA: 0s - loss: 0.0548 - acc: 0.9852
45056/48000 [===========================>..] - ETA: 0s - loss: 0.0542 - acc: 0.9854
47744/48000 [============================>.] - ETA: 0s - loss: 0.0547 - acc: 0.9852
48000/48000 [==============================] - 1s 23us/step - loss: 0.0548 - acc: 0.9851 - val_loss: 0.0947 - val_acc: 0.9806
Epoch 25/30

  128/48000 [..............................] - ETA: 2s - loss: 0.1008 - acc: 0.9844
 1280/48000 [..............................] - ETA: 2s - loss: 0.0718 - acc: 0.9812
 3328/48000 [=>............................] - ETA: 1s - loss: 0.0561 - acc: 0.9874
 5504/48000 [==>...........................] - ETA: 1s - loss: 0.0545 - acc: 0.9873
 8192/48000 [====>.........................] - ETA: 1s - loss: 0.0585 - acc: 0.9861
10880/48000 [=====>........................] - ETA: 0s - loss: 0.0527 - acc: 0.9870
13184/48000 [=======>......................] - ETA: 0s - loss: 0.0510 - acc: 0.9873
15872/48000 [========>.....................] - ETA: 0s - loss: 0.0489 - acc: 0.9871
18560/48000 [==========>...................] - ETA: 0s - loss: 0.0493 - acc: 0.9867
21248/48000 [============>.................] - ETA: 0s - loss: 0.0520 - acc: 0.9864
23936/48000 [=============>................] - ETA: 0s - loss: 0.0512 - acc: 0.9865
26496/48000 [===============>..............] - ETA: 0s - loss: 0.0520 - acc: 0.9862
29184/48000 [=================>............] - ETA: 0s - loss: 0.0514 - acc: 0.9862
31872/48000 [==================>...........] - ETA: 0s - loss: 0.0526 - acc: 0.9859
34688/48000 [====================>.........] - ETA: 0s - loss: 0.0528 - acc: 0.9860
37376/48000 [======================>.......] - ETA: 0s - loss: 0.0537 - acc: 0.9857
40064/48000 [========================>.....] - ETA: 0s - loss: 0.0544 - acc: 0.9855
42752/48000 [=========================>....] - ETA: 0s - loss: 0.0536 - acc: 0.9857
45440/48000 [===========================>..] - ETA: 0s - loss: 0.0535 - acc: 0.9857
48000/48000 [==============================] - 1s 24us/step - loss: 0.0531 - acc: 0.9855 - val_loss: 0.1035 - val_acc: 0.9794
Epoch 26/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0767 - acc: 0.9766
 1280/48000 [..............................] - ETA: 2s - loss: 0.0360 - acc: 0.9883
 3456/48000 [=>............................] - ETA: 1s - loss: 0.0518 - acc: 0.9850
 6144/48000 [==>...........................] - ETA: 1s - loss: 0.0510 - acc: 0.9862
 8832/48000 [====>.........................] - ETA: 0s - loss: 0.0545 - acc: 0.9858
11648/48000 [======>.......................] - ETA: 0s - loss: 0.0532 - acc: 0.9858
14336/48000 [=======>......................] - ETA: 0s - loss: 0.0532 - acc: 0.9858
17152/48000 [=========>....................] - ETA: 0s - loss: 0.0527 - acc: 0.9860
19840/48000 [===========>..................] - ETA: 0s - loss: 0.0542 - acc: 0.9857
22528/48000 [=============>................] - ETA: 0s - loss: 0.0539 - acc: 0.9860
25344/48000 [==============>...............] - ETA: 0s - loss: 0.0526 - acc: 0.9862
28032/48000 [================>.............] - ETA: 0s - loss: 0.0522 - acc: 0.9861
30720/48000 [==================>...........] - ETA: 0s - loss: 0.0523 - acc: 0.9861
33280/48000 [===================>..........] - ETA: 0s - loss: 0.0531 - acc: 0.9860
35712/48000 [=====================>........] - ETA: 0s - loss: 0.0540 - acc: 0.9858
38400/48000 [=======================>......] - ETA: 0s - loss: 0.0537 - acc: 0.9856
41088/48000 [========================>.....] - ETA: 0s - loss: 0.0548 - acc: 0.9854
43776/48000 [==========================>...] - ETA: 0s - loss: 0.0543 - acc: 0.9856
46464/48000 [============================>.] - ETA: 0s - loss: 0.0536 - acc: 0.9857
48000/48000 [==============================] - 1s 24us/step - loss: 0.0532 - acc: 0.9858 - val_loss: 0.1114 - val_acc: 0.9802
Epoch 27/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0153 - acc: 0.9922
 1280/48000 [..............................] - ETA: 2s - loss: 0.0323 - acc: 0.9914
 2944/48000 [>.............................] - ETA: 1s - loss: 0.0578 - acc: 0.9861
 4992/48000 [==>...........................] - ETA: 1s - loss: 0.0566 - acc: 0.9860
 6912/48000 [===>..........................] - ETA: 1s - loss: 0.0631 - acc: 0.9844
 8960/48000 [====>.........................] - ETA: 1s - loss: 0.0627 - acc: 0.9845
11520/48000 [======>.......................] - ETA: 0s - loss: 0.0566 - acc: 0.9858
14336/48000 [=======>......................] - ETA: 0s - loss: 0.0536 - acc: 0.9858
17152/48000 [=========>....................] - ETA: 0s - loss: 0.0505 - acc: 0.9864
19840/48000 [===========>..................] - ETA: 0s - loss: 0.0516 - acc: 0.9861
22528/48000 [=============>................] - ETA: 0s - loss: 0.0515 - acc: 0.9862
25088/48000 [==============>...............] - ETA: 0s - loss: 0.0508 - acc: 0.9862
27776/48000 [================>.............] - ETA: 0s - loss: 0.0506 - acc: 0.9862
30464/48000 [==================>...........] - ETA: 0s - loss: 0.0501 - acc: 0.9862
33152/48000 [===================>..........] - ETA: 0s - loss: 0.0498 - acc: 0.9861
35840/48000 [=====================>........] - ETA: 0s - loss: 0.0499 - acc: 0.9859
38528/48000 [=======================>......] - ETA: 0s - loss: 0.0500 - acc: 0.9858
41216/48000 [========================>.....] - ETA: 0s - loss: 0.0509 - acc: 0.9857
43904/48000 [==========================>...] - ETA: 0s - loss: 0.0515 - acc: 0.9856
46592/48000 [============================>.] - ETA: 0s - loss: 0.0517 - acc: 0.9857
48000/48000 [==============================] - 1s 25us/step - loss: 0.0517 - acc: 0.9858 - val_loss: 0.1007 - val_acc: 0.9807
Epoch 28/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0174 - acc: 1.0000
 2048/48000 [>.............................] - ETA: 1s - loss: 0.0510 - acc: 0.9883
 4352/48000 [=>............................] - ETA: 1s - loss: 0.0550 - acc: 0.9881
 6784/48000 [===>..........................] - ETA: 0s - loss: 0.0599 - acc: 0.9873
 9344/48000 [====>.........................] - ETA: 0s - loss: 0.0565 - acc: 0.9871
11904/48000 [======>.......................] - ETA: 0s - loss: 0.0553 - acc: 0.9871
14592/48000 [========>.....................] - ETA: 0s - loss: 0.0557 - acc: 0.9869
17152/48000 [=========>....................] - ETA: 0s - loss: 0.0541 - acc: 0.9868
19712/48000 [===========>..................] - ETA: 0s - loss: 0.0529 - acc: 0.9867
22272/48000 [============>.................] - ETA: 0s - loss: 0.0527 - acc: 0.9866
25088/48000 [==============>...............] - ETA: 0s - loss: 0.0518 - acc: 0.9864
27776/48000 [================>.............] - ETA: 0s - loss: 0.0509 - acc: 0.9863
30336/48000 [=================>............] - ETA: 0s - loss: 0.0516 - acc: 0.9862
32000/48000 [===================>..........] - ETA: 0s - loss: 0.0513 - acc: 0.9863
34560/48000 [====================>.........] - ETA: 0s - loss: 0.0514 - acc: 0.9862
37248/48000 [======================>.......] - ETA: 0s - loss: 0.0511 - acc: 0.9863
39936/48000 [=======================>......] - ETA: 0s - loss: 0.0523 - acc: 0.9861
42752/48000 [=========================>....] - ETA: 0s - loss: 0.0519 - acc: 0.9862
45440/48000 [===========================>..] - ETA: 0s - loss: 0.0515 - acc: 0.9864
48000/48000 [==============================] - 1s 25us/step - loss: 0.0516 - acc: 0.9863 - val_loss: 0.1098 - val_acc: 0.9801
Epoch 29/30

  128/48000 [..............................] - ETA: 1s - loss: 0.0371 - acc: 0.9766
 2304/48000 [>.............................] - ETA: 1s - loss: 0.0383 - acc: 0.9865
 4608/48000 [=>............................] - ETA: 1s - loss: 0.0505 - acc: 0.9857
 6784/48000 [===>..........................] - ETA: 0s - loss: 0.0552 - acc: 0.9847
 9088/48000 [====>.........................] - ETA: 0s - loss: 0.0497 - acc: 0.9856
11264/48000 [======>.......................] - ETA: 0s - loss: 0.0511 - acc: 0.9854
13696/48000 [=======>......................] - ETA: 0s - loss: 0.0500 - acc: 0.9859
16384/48000 [=========>....................] - ETA: 0s - loss: 0.0493 - acc: 0.9866
19072/48000 [==========>...................] - ETA: 0s - loss: 0.0489 - acc: 0.9865
21760/48000 [============>.................] - ETA: 0s - loss: 0.0488 - acc: 0.9864
24576/48000 [==============>...............] - ETA: 0s - loss: 0.0480 - acc: 0.9864
27264/48000 [================>.............] - ETA: 0s - loss: 0.0499 - acc: 0.9864
30080/48000 [=================>............] - ETA: 0s - loss: 0.0494 - acc: 0.9862
32768/48000 [===================>..........] - ETA: 0s - loss: 0.0501 - acc: 0.9860
35456/48000 [=====================>........] - ETA: 0s - loss: 0.0511 - acc: 0.9860
38144/48000 [======================>.......] - ETA: 0s - loss: 0.0508 - acc: 0.9860
40832/48000 [========================>.....] - ETA: 0s - loss: 0.0513 - acc: 0.9860
43520/48000 [==========================>...] - ETA: 0s - loss: 0.0523 - acc: 0.9858
46080/48000 [===========================>..] - ETA: 0s - loss: 0.0520 - acc: 0.9859
48000/48000 [==============================] - 1s 24us/step - loss: 0.0517 - acc: 0.9858 - val_loss: 0.1049 - val_acc: 0.9807
Epoch 30/30

  128/48000 [..............................] - ETA: 3s - loss: 0.0173 - acc: 0.9844
 1152/48000 [..............................] - ETA: 2s - loss: 0.0409 - acc: 0.9852
 2432/48000 [>.............................] - ETA: 2s - loss: 0.0387 - acc: 0.9877
 4480/48000 [=>............................] - ETA: 1s - loss: 0.0442 - acc: 0.9877
 7168/48000 [===>..........................] - ETA: 1s - loss: 0.0525 - acc: 0.9866
 9856/48000 [=====>........................] - ETA: 1s - loss: 0.0550 - acc: 0.9859
12544/48000 [======>.......................] - ETA: 0s - loss: 0.0544 - acc: 0.9850
15232/48000 [========>.....................] - ETA: 0s - loss: 0.0521 - acc: 0.9856
17920/48000 [==========>...................] - ETA: 0s - loss: 0.0571 - acc: 0.9848
20608/48000 [===========>..................] - ETA: 0s - loss: 0.0550 - acc: 0.9851
22912/48000 [=============>................] - ETA: 0s - loss: 0.0520 - acc: 0.9859
25600/48000 [===============>..............] - ETA: 0s - loss: 0.0535 - acc: 0.9859
28288/48000 [================>.............] - ETA: 0s - loss: 0.0531 - acc: 0.9856
30976/48000 [==================>...........] - ETA: 0s - loss: 0.0542 - acc: 0.9855
33664/48000 [====================>.........] - ETA: 0s - loss: 0.0529 - acc: 0.9858
36224/48000 [=====================>........] - ETA: 0s - loss: 0.0529 - acc: 0.9857
38528/48000 [=======================>......] - ETA: 0s - loss: 0.0541 - acc: 0.9855
40704/48000 [========================>.....] - ETA: 0s - loss: 0.0541 - acc: 0.9856
43136/48000 [=========================>....] - ETA: 0s - loss: 0.0537 - acc: 0.9856
45312/48000 [===========================>..] - ETA: 0s - loss: 0.0548 - acc: 0.9855
47744/48000 [============================>.] - ETA: 0s - loss: 0.0559 - acc: 0.9851
48000/48000 [==============================] - 1s 26us/step - loss: 0.0558 - acc: 0.9852 - val_loss: 0.1009 - val_acc: 0.9812

The history object returned by fit() includes loss and accuracy metrics which we can plot:

plot(history)

Evaluate the model’s performance on the test data:

model %>% evaluate(x_test, y_test)


   32/10000 [..............................] - ETA: 0s
 2368/10000 [======>.......................] - ETA: 0s
 4672/10000 [=============>................] - ETA: 0s
 7072/10000 [====================>.........] - ETA: 0s
 9472/10000 [===========================>..] - ETA: 0s
10000/10000 [==============================] - 0s 21us/step
$loss
[1] 0.09632611

$acc
[1] 0.9823

Generate predictions on new data:

model %>% predict_classes(x_test)

   [1] 7 2 1 0 4 1 4 9 5 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1
  [31] 3 1 3 4 7 2 7 1 2 1 1 7 4 2 3 5 1 2 4 4 6 3 5 5 6 0 4 1 9 5
  [61] 7 8 9 3 7 4 6 4 3 0 7 0 2 9 1 7 3 2 9 7 7 6 2 7 8 4 7 3 6 1
  [91] 3 6 9 3 1 4 1 7 6 9 6 0 5 4 9 9 2 1 9 4 8 7 3 9 7 4 4 4 9 2
 [121] 5 4 7 6 7 9 0 5 8 5 6 6 5 7 8 1 0 1 6 4 6 7 3 1 7 1 8 2 0 2
 [151] 9 9 5 5 1 5 6 0 3 4 4 6 5 4 6 5 4 5 1 4 4 7 2 3 2 7 1 8 1 8
 [181] 1 8 5 0 8 9 2 5 0 1 1 1 0 9 0 3 1 6 4 2 3 6 1 1 1 3 9 5 2 9
 [211] 4 5 9 3 9 0 3 6 5 5 7 2 2 7 1 2 8 4 1 7 3 3 8 8 7 9 2 2 4 1
 [241] 5 9 8 7 2 3 0 6 4 2 4 1 9 5 7 7 2 8 2 0 8 5 7 7 9 1 8 1 8 0
 [271] 3 0 1 9 9 4 1 8 2 1 2 9 7 5 9 2 6 4 1 5 8 2 9 2 0 4 0 0 2 8
 [301] 4 7 1 2 4 0 2 7 4 3 3 0 0 3 1 9 6 5 2 5 9 7 9 3 0 4 2 0 7 1
 [331] 1 2 1 5 3 3 9 7 8 6 3 6 1 3 8 1 0 5 1 3 1 5 5 6 1 8 5 1 7 9
 [361] 4 6 2 2 5 0 6 5 6 3 7 2 0 8 8 5 4 1 1 4 0 7 3 7 6 1 6 2 1 9
 [391] 2 8 6 1 9 5 2 5 4 4 2 8 3 8 2 4 5 0 3 1 7 7 5 7 9 7 1 9 2 1
 [421] 4 2 9 2 0 4 9 1 4 8 1 8 4 5 9 8 8 3 7 6 0 0 3 0 2 0 6 9 9 3
 [451] 3 3 2 3 9 1 2 6 8 0 5 6 6 6 3 8 8 2 7 5 8 9 6 1 8 4 1 2 5 9
 [481] 1 9 7 5 4 0 8 9 9 1 0 5 2 3 7 0 9 4 0 6 3 9 5 2 1 3 1 3 6 5
 [511] 7 4 2 2 6 3 2 6 5 4 8 9 7 1 3 0 3 8 3 1 9 3 4 4 6 4 2 1 8 2
 [541] 5 4 8 8 4 0 0 2 3 2 7 7 0 8 7 4 4 7 9 6 9 0 9 8 0 4 6 0 6 3
 [571] 5 4 8 3 3 9 3 3 3 7 8 0 2 2 1 7 0 6 5 4 3 8 0 9 6 3 8 0 9 9
 [601] 6 8 6 8 5 7 8 6 0 2 4 0 2 2 3 1 9 7 5 1 0 8 4 6 2 6 7 9 3 2
 [631] 9 8 2 2 9 2 7 3 5 9 1 8 0 2 0 5 2 1 3 7 6 7 1 2 5 8 0 3 7 7
 [661] 4 0 9 1 8 6 7 7 4 3 4 9 1 9 5 1 7 3 9 7 6 9 1 3 3 8 3 3 6 7
 [691] 2 4 5 8 5 1 1 4 4 3 1 0 7 7 0 7 9 4 4 8 5 5 4 0 8 2 1 0 8 4
 [721] 8 0 4 0 6 1 7 3 2 6 7 2 6 9 3 1 4 6 2 5 9 2 0 6 2 1 7 3 4 1
 [751] 0 5 4 3 1 1 7 4 9 9 4 8 4 0 2 4 5 1 1 6 4 7 1 9 4 2 4 1 5 5
 [781] 3 8 3 1 4 5 6 8 9 4 1 5 3 8 0 3 2 5 1 2 8 3 4 4 0 8 8 3 3 1
 [811] 7 3 5 9 6 3 2 6 1 3 6 0 7 2 1 7 1 4 2 4 2 1 7 9 6 1 1 2 4 8
 [841] 1 7 7 4 8 0 7 3 1 3 1 0 7 7 0 3 5 5 2 7 6 6 9 2 8 3 5 2 2 5
 [871] 6 0 8 2 9 2 8 8 8 8 7 4 7 3 0 6 6 3 2 1 3 2 2 9 3 0 0 5 7 8
 [901] 1 4 4 6 0 2 9 1 4 7 4 7 3 9 8 8 4 7 1 2 1 2 2 3 2 3 2 3 9 1
 [931] 7 4 0 3 5 5 8 6 3 2 6 7 6 6 3 2 7 8 1 1 7 5 6 4 9 5 1 3 3 4
 [961] 7 8 9 1 1 0 9 1 4 4 5 4 0 6 2 2 3 1 5 1 2 0 3 8 1 2 6 7 1 6
 [991] 2 3 9 0 1 2 2 0 8 9
 [ reached getOption("max.print") -- omitted 9000 entries ]

LS0tCnRpdGxlOiAiTU5JU1QgTm90ZWJvb2siCm91dHB1dDogaHRtbF9ub3RlYm9vawotLS0KClRoZSBNTklTVCBkYXRhc2V0IGlzIGluY2x1ZGVkIHdpdGggS2VyYXMgYW5kIGNhbiBiZSBhY2Nlc3NlZCB1c2luZyB0aGUgZGF0YXNldF9tbmlzdCgpIGZ1bmN0aW9uLiBIZXJlIHdlIGxvYWQgdGhlIGRhdGFzZXQgdGhlbiBjcmVhdGUgdmFyaWFibGVzIGZvciBvdXIgdGVzdCBhbmQgdHJhaW5pbmcgZGF0YToKCmBgYHtyfQpsaWJyYXJ5KGtlcmFzKQptbmlzdCA8LSBkYXRhc2V0X21uaXN0KCkKeF90cmFpbiA8LSBtbmlzdCR0cmFpbiR4CnlfdHJhaW4gPC0gbW5pc3QkdHJhaW4keQp4X3Rlc3QgPC0gbW5pc3QkdGVzdCR4CnlfdGVzdCA8LSBtbmlzdCR0ZXN0JHkKYGBgCgpUaGUgeCBkYXRhIGlzIGEgMy1kIGFycmF5IChpbWFnZXMsd2lkdGgsaGVpZ2h0KSBvZiBncmF5c2NhbGUgdmFsdWVzIC4gVG8gcHJlcGFyZSB0aGUgZGF0YSBmb3IgdHJhaW5pbmcgd2UgY29udmVydCB0aGUgMy1kIGFycmF5cyBpbnRvIG1hdHJpY2VzIGJ5IHJlc2hhcGluZyB3aWR0aCBhbmQgaGVpZ2h0IGludG8gYSBzaW5nbGUgZGltZW5zaW9uICgyOHgyOCBpbWFnZXMgYXJlIGZsYXR0ZW5lZCBpbnRvIGxlbmd0aCA3ODQgdmVjdG9ycykuIFRoZW4sIHdlIGNvbnZlcnQgdGhlIGdyYXlzY2FsZSB2YWx1ZXMgZnJvbSBpbnRlZ2VycyByYW5naW5nIGJldHdlZW4gMCB0byAyNTUgaW50byBmbG9hdGluZyBwb2ludCB2YWx1ZXMgcmFuZ2luZyBiZXR3ZWVuIDAgYW5kIDE6CgpgYGB7cn0KIyByZXNoYXBlCnhfdHJhaW4gPC0gYXJyYXlfcmVzaGFwZSh4X3RyYWluLCBjKG5yb3coeF90cmFpbiksIDc4NCkpCnhfdGVzdCA8LSBhcnJheV9yZXNoYXBlKHhfdGVzdCwgYyhucm93KHhfdGVzdCksIDc4NCkpCiMgcmVzY2FsZQp4X3RyYWluIDwtIHhfdHJhaW4gLyAyNTUKeF90ZXN0IDwtIHhfdGVzdCAvIDI1NQpgYGAKCk5vdGUgdGhhdCB3ZSB1c2UgdGhlIGFycmF5X3Jlc2hhcGUoKSBmdW5jdGlvbiByYXRoZXIgdGhhbiB0aGUgZGltPC0oKSBmdW5jdGlvbiB0byByZXNoYXBlIHRoZSBhcnJheS4gVGhpcyBpcyBzbyB0aGF0IHRoZSBkYXRhIGlzIHJlLWludGVycHJldGVkIHVzaW5nIHJvdy1tYWpvciBzZW1hbnRpY3MgKGFzIG9wcG9zZWQgdG8gUuKAmXMgZGVmYXVsdCBjb2x1bW4tbWFqb3Igc2VtYW50aWNzKSwgd2hpY2ggaXMgaW4gdHVybiBjb21wYXRpYmxlIHdpdGggdGhlIHdheSB0aGF0IHRoZSBudW1lcmljYWwgbGlicmFyaWVzIGNhbGxlZCBieSBLZXJhcyBpbnRlcnByZXQgYXJyYXkgZGltZW5zaW9ucy4KClRoZSB5IGRhdGEgaXMgYW4gaW50ZWdlciB2ZWN0b3Igd2l0aCB2YWx1ZXMgcmFuZ2luZyBmcm9tIDAgdG8gOS4gVG8gcHJlcGFyZSB0aGlzIGRhdGEgZm9yIHRyYWluaW5nIHdlIG9uZS1ob3QgZW5jb2RlIHRoZSB2ZWN0b3JzIGludG8gYmluYXJ5IGNsYXNzIG1hdHJpY2VzIHVzaW5nIHRoZSBLZXJhcyB0b19jYXRlZ29yaWNhbCgpIGZ1bmN0aW9uOgoKYGBge3J9CnlfdHJhaW4gPC0gdG9fY2F0ZWdvcmljYWwoeV90cmFpbiwgMTApCnlfdGVzdCA8LSB0b19jYXRlZ29yaWNhbCh5X3Rlc3QsIDEwKQpgYGAKCiMjIERlZmluaW5nIHRoZSBNb2RlbApUaGUgY29yZSBkYXRhIHN0cnVjdHVyZSBvZiBLZXJhcyBpcyBhIG1vZGVsLCBhIHdheSB0byBvcmdhbml6ZSBsYXllcnMuIFRoZSBzaW1wbGVzdCB0eXBlIG9mIG1vZGVsIGlzIHRoZSBTZXF1ZW50aWFsIG1vZGVsLCBhIGxpbmVhciBzdGFjayBvZiBsYXllcnMuCgpXZSBiZWdpbiBieSBjcmVhdGluZyBhIHNlcXVlbnRpYWwgbW9kZWwgYW5kIHRoZW4gYWRkaW5nIGxheWVycyB1c2luZyB0aGUgcGlwZSAoJT4lKSBvcGVyYXRvcjoKCmBgYHtyfQptb2RlbCA8LSBrZXJhc19tb2RlbF9zZXF1ZW50aWFsKCkgCm1vZGVsICU+JSAKICBsYXllcl9kZW5zZSh1bml0cyA9IDI1NiwgYWN0aXZhdGlvbiA9ICdyZWx1JywgaW5wdXRfc2hhcGUgPSBjKDc4NCkpICU+JSAKICBsYXllcl9kcm9wb3V0KHJhdGUgPSAwLjQpICU+JSAKICBsYXllcl9kZW5zZSh1bml0cyA9IDEyOCwgYWN0aXZhdGlvbiA9ICdyZWx1JykgJT4lCiAgbGF5ZXJfZHJvcG91dChyYXRlID0gMC4zKSAlPiUKICBsYXllcl9kZW5zZSh1bml0cyA9IDEwLCBhY3RpdmF0aW9uID0gJ3NvZnRtYXgnKQpgYGAKVGhlIGlucHV0X3NoYXBlIGFyZ3VtZW50IHRvIHRoZSBmaXJzdCBsYXllciBzcGVjaWZpZXMgdGhlIHNoYXBlIG9mIHRoZSBpbnB1dCBkYXRhIChhIGxlbmd0aCA3ODQgbnVtZXJpYyB2ZWN0b3IgcmVwcmVzZW50aW5nIGEgZ3JheXNjYWxlIGltYWdlKS4gVGhlIGZpbmFsIGxheWVyIG91dHB1dHMgYSBsZW5ndGggMTAgbnVtZXJpYyB2ZWN0b3IgKHByb2JhYmlsaXRpZXMgZm9yIGVhY2ggZGlnaXQpIHVzaW5nIGEgc29mdG1heCBhY3RpdmF0aW9uIGZ1bmN0aW9uLgoKVXNlIHRoZSBzdW1tYXJ5KCkgZnVuY3Rpb24gdG8gcHJpbnQgdGhlIGRldGFpbHMgb2YgdGhlIG1vZGVsOgpgYGB7cn0Kc3VtbWFyeShtb2RlbCkKYGBgCgpOZXh0LCBjb21waWxlIHRoZSBtb2RlbCB3aXRoIGFwcHJvcHJpYXRlIGxvc3MgZnVuY3Rpb24sIG9wdGltaXplciwgYW5kIG1ldHJpY3M6CmBgYHtyfQptb2RlbCAlPiUgY29tcGlsZSgKICBsb3NzID0gJ2NhdGVnb3JpY2FsX2Nyb3NzZW50cm9weScsCiAgb3B0aW1pemVyID0gb3B0aW1pemVyX3Jtc3Byb3AoKSwKICBtZXRyaWNzID0gYygnYWNjdXJhY3knKQopCmBgYAoKIyMgVHJhaW5pbmcgYW5kIEV2YWx1YXRpb24KClVzZSB0aGUgZml0KCkgZnVuY3Rpb24gdG8gdHJhaW4gdGhlIG1vZGVsIGZvciAzMCBlcG9jaHMgdXNpbmcgYmF0Y2hlcyBvZiAxMjggaW1hZ2VzOgoKYGBge3J9Cmhpc3RvcnkgPC0gbW9kZWwgJT4lIGZpdCgKICB4X3RyYWluLCB5X3RyYWluLCAKICBlcG9jaHMgPSAzMCwgYmF0Y2hfc2l6ZSA9IDEyOCwgCiAgdmFsaWRhdGlvbl9zcGxpdCA9IDAuMgopCmBgYAoKVGhlIGhpc3Rvcnkgb2JqZWN0IHJldHVybmVkIGJ5IGZpdCgpIGluY2x1ZGVzIGxvc3MgYW5kIGFjY3VyYWN5IG1ldHJpY3Mgd2hpY2ggd2UgY2FuIHBsb3Q6CmBgYHtyfQpwbG90KGhpc3RvcnkpCmBgYApFdmFsdWF0ZSB0aGUgbW9kZWzigJlzIHBlcmZvcm1hbmNlIG9uIHRoZSB0ZXN0IGRhdGE6CgpgYGB7cn0KbW9kZWwgJT4lIGV2YWx1YXRlKHhfdGVzdCwgeV90ZXN0KQpgYGAKR2VuZXJhdGUgcHJlZGljdGlvbnMgb24gbmV3IGRhdGE6CmBgYHtyfQptb2RlbCAlPiUgcHJlZGljdF9jbGFzc2VzKHhfdGVzdCkKYGBgCgo=