1 Objective : CIFAR_10 Image Classification with R + Tensorflow_GPU
- The main task is to build a feed-forward convolutional neural networks(CNNs) for images classification with the benchmark CIFAR_10 dataset, which can be downloaded from https://www.cs.toronto.edu/~kriz/cifar.html .
2 Software and Hardward Configuration :
- R version 3.5.2 Patched (2019-01-14 r75992)
- keras, tensorflow support for CPU
- Platform: x86_64-w64-mingw32/x64 (64-bit)
- Processor Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 1800 Mhz, 4 Core(s), 8 Logical Processor(s)
- Physical Memory : 4GB
- Graphic Display Card : GeFore MX150
- backend: window platform under minconda + tensorflow_gpu envirnoment
- backend: python 3.7 verion + Tensorflow_gpu in RStduio
3 load image file : dataset_cifar10()
- dataset_cifar10 contains 50,000 32x32 color training images, labeled over 10 categories, and 10,000 test images
- Lists of training and test data: train_x, train_y, test_x, test_y.
- The x data is an array of RGB image data with shape (num_samples, 3, 32, 32).
- The y data is an array of category labels (integers in range 0-9) with shape (num_samples).
4 Data Preparation : Normalisation of RGB value
cifar10 = dataset_cifar10()
x_train = cifar10$train$x/255
x_test = cifar10$test$x/255
y_train = keras::to_categorical(cifar10$train$y, num_classes = 10)
y_test = keras::to_categorical(cifar10$test$y, num_classes = 10)RGB Values and its object shapes
library(EBImage)
pictures = c(9802, 5, 7, 10, 4, 28,1, 8, 9, 2)
fig_img = list()
for (i in 1:10 ) {
fig_mat = cifar10$train$x[pictures[i], , , ]
fig_img[[i]] = normalize(Image(transpose(fig_mat), dim=c(32,32,3), colormode='Color'))
}
fig_img_comb = combine(fig_img[1:10])
fig_img_obj = tile(fig_img_comb,5)
plot(fig_img_obj, all=T)
4.1 the order of object_label :
label_name = c("flyer", "car", "bird", "cat", "deer", "dog", "frog ", "horse", "ship", "truck")
label_name## [1] "flyer" "car" "bird" "cat" "deer" "dog" "frog " "horse"
## [9] "ship" "truck"5 Model Defining : keras_model_sequential (Tensorflow 1.0 GPU version in R)
model_runexternal_01 = keras_model_sequential()
model_runexternal_01 %>%
# Start with hidden 2D CNN layer
layer_conv_2d( filters = 32, kernel_size = c(3,3), padding = "same", input_shape = c(32, 32, 3) ) %>%
layer_activation("relu") %>%
# Second hidden layer
layer_conv_2d(filters = 32, kernel_size = c(3,3)) %>%
layer_activation("relu") %>%
# Use max pooling
layer_max_pooling_2d(pool_size = c(2,2)) %>%
layer_dropout(0.25) %>%
# 2 additional hidden 2D CNN layers
layer_conv_2d(filters = 32, kernel_size = c(3,3), padding = "same") %>%
layer_activation("relu") %>%
layer_conv_2d(filters = 32, kernel_size = c(3,3)) %>%
layer_activation("relu") %>%
# Use max pooling
layer_max_pooling_2d(pool_size = c(2,2)) %>%
layer_dropout(0.25) %>%
# Flatten max filtered output into feature vector
layer_flatten() %>%
# and feed into dense layer
layer_dense(512) %>%
layer_activation("relu") %>%
layer_dropout(0.5) %>%
# Outputs from dense layer are projected onto 10 unit output layer
layer_dense(10) %>%
layer_activation("softmax")%>%
compile(
loss = "categorical_crossentropy",
optimizer = optimizer_rmsprop(lr = 0.0001, decay = 1e-6),
metrics = "accuracy" )
summary(model_runexternal_01)## Model: "sequential"
## ___________________________________________________________________________
## Layer (type) Output Shape Param #
## ===========================================================================
## conv2d (Conv2D) (None, 32, 32, 32) 896
## ___________________________________________________________________________
## activation (Activation) (None, 32, 32, 32) 0
## ___________________________________________________________________________
## conv2d_1 (Conv2D) (None, 30, 30, 32) 9248
## ___________________________________________________________________________
## activation_1 (Activation) (None, 30, 30, 32) 0
## ___________________________________________________________________________
## max_pooling2d (MaxPooling2D) (None, 15, 15, 32) 0
## ___________________________________________________________________________
## dropout (Dropout) (None, 15, 15, 32) 0
## ___________________________________________________________________________
## conv2d_2 (Conv2D) (None, 15, 15, 32) 9248
## ___________________________________________________________________________
## activation_2 (Activation) (None, 15, 15, 32) 0
## ___________________________________________________________________________
## conv2d_3 (Conv2D) (None, 13, 13, 32) 9248
## ___________________________________________________________________________
## activation_3 (Activation) (None, 13, 13, 32) 0
## ___________________________________________________________________________
## max_pooling2d_1 (MaxPooling2D) (None, 6, 6, 32) 0
## ___________________________________________________________________________
## dropout_1 (Dropout) (None, 6, 6, 32) 0
## ___________________________________________________________________________
## flatten (Flatten) (None, 1152) 0
## ___________________________________________________________________________
## dense (Dense) (None, 512) 590336
## ___________________________________________________________________________
## activation_4 (Activation) (None, 512) 0
## ___________________________________________________________________________
## dropout_2 (Dropout) (None, 512) 0
## ___________________________________________________________________________
## dense_1 (Dense) (None, 10) 5130
## ___________________________________________________________________________
## activation_5 (Activation) (None, 10) 0
## ===========================================================================
## Total params: 624,106
## Trainable params: 624,106
## Non-trainable params: 0
## ___________________________________________________________________________start.time = Sys.time()
batch_size = 32
epochs = 60
Train_model_runexternal_01 = model_runexternal_01 %>% fit(
x_train, y_train,
batch_size = batch_size,
epochs = epochs,
validation_data = list(x_test, y_test),
shuffle = TRUE )
end.time = Sys.time()## Time difference of 30.82586 mins
CIFAR_10 Image Classification: Accuracy Plot
6 load my model
7 evaluation and prediction for test samples
## $loss
## [1] 0.7124344
##
## $acc
## [1] 0.75998 Loadmymodel: Confusion Matrix
| flyer | car | bird | cat | deer | dog | frog | horse | ship | truck | |
|---|---|---|---|---|---|---|---|---|---|---|
| flyer | 84.1 | 1.4 | 9.6 | 2.2 | 2.6 | 1.5 | 0.8 | 1.5 | 5.3 | 3.3 |
| car | 2.7 | 92.3 | 0.3 | 1.1 | 0.2 | 0.0 | 0.2 | 0.3 | 3.2 | 10.4 |
| bird | 2.3 | 0.1 | 57.0 | 3.9 | 5.6 | 1.8 | 4.5 | 1.8 | 0.3 | 0.3 |
| cat | 2.1 | 0.4 | 9.6 | 66.7 | 10.7 | 18.2 | 9.3 | 5.4 | 2.1 | 2.0 |
| deer | 0.4 | 0.1 | 5.9 | 2.8 | 61.8 | 2.0 | 2.1 | 2.3 | 0.2 | 0.1 |
| dog | 0.2 | 0.0 | 7.6 | 14.7 | 4.3 | 70.8 | 2.0 | 7.0 | 0.1 | 0.1 |
| frog | 0.3 | 0.6 | 6.1 | 4.0 | 4.4 | 1.1 | 79.0 | 0.1 | 0.2 | 0.4 |
| horse | 1.5 | 0.1 | 3.1 | 2.3 | 9.3 | 4.2 | 1.1 | 80.9 | 0.3 | 1.4 |
| ship | 4.5 | 1.6 | 0.6 | 1.2 | 0.9 | 0.3 | 0.8 | 0.0 | 87.6 | 2.3 |
| truck | 1.9 | 3.4 | 0.2 | 1.1 | 0.2 | 0.1 | 0.2 | 0.7 | 0.7 | 79.7 |
9 References:
- Krizhevsky, Hinton, and others (2009)
- Hinton et al. (2012)
Hinton, Geoffrey E., Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. 2012. “Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors,” July. http://arxiv.org/abs/http://arxiv.org/abs/1207.0580v1.
Krizhevsky, Alex, Geoffrey Hinton, and others. 2009. “Learning Multiple Layers of Features from Tiny Images.” Citeseer.