Neural Network
Yuli Ristiana
2023-08-18
INTRODUCTION
Bahasa isyarat adalah bahasa komunikasi sama hal nya seperti bahasa lainnya. Namun bahasa ini lebih banyak digunakan di kalangan tunarungu. Kumpulan data ini adalah kumpulan isyarat yang lengkap yang digunakan dalam bahasa isyarat dan dapat digunakan oleh orang normal lainnya untuk lebih memahami isyarat bahasa kalangan tunarungu. Dalam data ini terdapat panel montase gambar yang dipotong dari berbagai pengguna dan latar belakang untuk surat-surat Bahasa Isyarat Amerika.link : https://www.kaggle.com/datasets/datamunge/sign-language-mnist
IMPORT LIBRARY
READ DATA
Hal pertama yang harus kita lakukan yaitu import data kita terlebih
dahulu read.csv.
DATA INSPECTIONS
Kita gunakan
head()untuk melihat urutan data teratas dan gunakantail()untuk melihat urutan data paling bawah.
*Jadi data pada table diatas menunjukkan bahwa 1 baris mewakili 1 gambar, dan nilai pada kolom-kolom tersebut adalah pixelnya.
Menampilkan visualisasi data.
Sebelum kita lanjut pada proses pembuatan model, kita coba visualisasikan dulu gambar digit yang akan digunakan dengan mengubah nilai pixel menjadi sebuah gambar menggunakan function di bawah.
# fungsi untuk visualisai data gambar dari csv
vizTrain <- function(input){ # data yng ingin ditampilkan
dimmax <- sqrt(ncol(input[,-1]))
dimn <- ceiling(sqrt(nrow(input)))
par(mfrow=c(dimn, dimn), mar=c(.1, .1, .1, .1))
for (i in 1:nrow(input)){
m1 <- as.matrix(input[i,2:785])
dim(m1) <- c(28,28)
m1 <- apply(apply(m1, 1, rev), 1, t)
image(1:28, 1:28,
m1, col=grey.colors(255),
# remove axis text
xaxt = 'n', yaxt = 'n')
text(2, 20, col="white", cex=1.2, input[i, 1])
}
}
Dengan menggunakan fungsi
anyNA()ataucolsums(is.na())yaitu untuk mengecheck apakah terdapat missing value atau tidak.
#> [1] FALSE#> [1] FALSECek dimensi datanya menggunakan fungsi
dim().
#> [1] 27455 785#> [1] 7172 785#> [1] 28Pada setiap gambar berdimensi sign.train -> 27455. Sedangkan sign.test -> 7172. gambar: 28 x 28 = 784 pixel -> ukuran inputnya: 784.
Cek label dari setiap data
sign_traindansign_testmenggunakan fungsitable().
#>
#> 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16
#> 1126 1010 1144 1196 957 1204 1090 1013 1162 1114 1241 1055 1151 1196 1088 1279
#> 17 18 19 20 21 22 23 24
#> 1294 1199 1186 1161 1082 1225 1164 1118#>
#> 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18 19 20
#> 331 432 310 245 498 247 348 436 288 331 209 394 291 246 347 164 144 246 248 266
#> 21 22 23 24
#> 346 206 267 332Dari hasil table diatas kita menemukan bahwa label 9 dan 25 tidak ada.
Karena label 9 dan 25 tidak ada, kita dapat mengurangkan dengan 1 semua label yang bernilai lebih besar dari 9. Dengan cara ini, label kita menjadi semua bilangan bulat dari 0 hingga 23.
sign.train <- sign.train %>%
mutate(label = ifelse(label > 9, label-1, label))
sign.test <- sign.test %>%
mutate(label = ifelse(label > 9, label-1, label))#>
#> 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
#> 1126 1010 1144 1196 957 1204 1090 1013 1162 1114 1241 1055 1151 1196 1088 1279
#> 16 17 18 19 20 21 22 23
#> 1294 1199 1186 1161 1082 1225 1164 1118#>
#> 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
#> 331 432 310 245 498 247 348 436 288 331 209 394 291 246 347 164 144 246 248 266
#> 20 21 22 23
#> 346 206 267 332#Cross-Validation
set.seed(100)
index <- initial_split(data = sign.train,
prop = 0.8,
strata = "label")
data_train <- training(index)
data_test <- testing(index)Selanjutnya kita cek proporsi kelas target pada data train dan data test.
#>
#> 0 1 2 3 4 5 6
#> 0.04101257 0.03678747 0.04166818 0.04356219 0.03485704 0.04385358 0.03970133
#> 7 8 9 10 11 12 13
#> 0.03689674 0.04232380 0.04057549 0.04520124 0.03842652 0.04192315 0.04356219
#> 14 15 16 17 18 19 20
#> 0.03962848 0.04658532 0.04713167 0.04367146 0.04319796 0.04228738 0.03940994
#> 21 22 23
#> 0.04461847 0.04239665 0.04072118#>
#> 0 1 2 3 4 5 6
#> 0.04615170 0.06023424 0.04322365 0.03416062 0.06943670 0.03443949 0.04852203
#> 7 8 9 10 11 12 13
#> 0.06079197 0.04015616 0.04615170 0.02914110 0.05493586 0.04057446 0.03430006
#> 14 15 16 17 18 19 20
#> 0.04838260 0.02286670 0.02007808 0.03430006 0.03457892 0.03708868 0.04824317
#> 21 22 23
#> 0.02872281 0.03722811 0.04629113Lakukan features scaLling dengan cara membaginya dengan menggunakan fungsi
range().
#> [1] 0 255Didapatkan nilai rangenya adalah 255.
Untuk membuat model, pisahkan dulu label prediktor dan variabel pixelnya. Dan Merubah menjadi data Matrix, scaling.
# Predictor variables in sign_train`
train_x <- sign.train %>%
select(-label) %>%
as.matrix()/255
# Target variable in `sign_train`
train_y <- sign.train$label
# Predictor variables in `sign_test`
test_x <- sign.test %>%
select(-label) %>%
as.matrix()/255
# Target variable in `sign_test`
test_y <- sign.test$labelProses mengubah matrix menjadi array keras , agar diterima sebagai input model keras.
Ubah kategorinya dari angka 0-9 menjadi one-hot encoding menggunakan
to_categorical(data)dari librarykeras.
BUILD NEURAL NETWORK ARCHITECTURE
Buat dulu object
input_dimuntuk menyimpan informasi jumlah kolom dari prediktor dan jumlah kategori dari target variabel dalam objectnum_class.
#> [1] 784#> [1] 24# set seed bobot awal
RNGkind(sample.kind = "Rounding")
set.seed(100)
nitializer <- initializer_random_normal(seed = 100)Membuat model pertama
model_signin- Input layer: 784 prediktor (gambar 28x28 pixel) - Input1: 64 neuron dengan activation function = ReLu - Input2: 32 neuron dengan activation function = ReLu - Output: 24 neuron (sesuai jumlah kategori 0-9) dengan activation function = Softmax
model_signin <- keras_model_sequential(name = "signin_model")
model_signin %>%
layer_dense(
input_shape = 784,
units = 64,
activation = "relu",
name = "Input1") %>%
layer_dense(units = 32,
activation = "relu",
name = "Input2") %>%
layer_dense(units = 24,
activation = "softmax",
name = "Output")
summary(model_signin)#> Model: "signin_model"
#> ________________________________________________________________________________
#> Layer (type) Output Shape Param #
#> ================================================================================
#> Input1 (Dense) (None, 64) 50240
#> Input2 (Dense) (None, 32) 2080
#> Output (Dense) (None, 24) 792
#> ================================================================================
#> Total params: 53,112
#> Trainable params: 53,112
#> Non-trainable params: 0
#> ________________________________________________________________________________Membuat model kedua
model_signout. - Input layer: 784 prediktor (gambar 28x28 pixel) - Input1: 256 neuron dengan activation function = ReLu (Total nodes pada Input1) - Input2: 128 neuron dengan activation function = ReLu - Input3: 64 neuron dengan activation function = ReLu - Output: 24 neuron (sesuai jumlah kategori 0-9) dengan activation function = Softmax
model_signout <- keras_model_sequential(name = "signout_model")
model_signout %>%
layer_dense(input_shape = 784,
units = 256,
activation = "relu",
name = "Input1") %>%
layer_dense(units = 128,
activation = "relu",
name = "Input2") %>%
layer_dense(units = 64,
activation = "relu",
name = "Input3") %>%
layer_dense(units = 24,
activation = "softmax",
name = "Output")
summary(model_signout)#> Model: "signout_model"
#> ________________________________________________________________________________
#> Layer (type) Output Shape Param #
#> ================================================================================
#> Input1 (Dense) (None, 256) 200960
#> Input2 (Dense) (None, 128) 32896
#> Input3 (Dense) (None, 64) 8256
#> Output (Dense) (None, 24) 1560
#> ================================================================================
#> Total params: 243,672
#> Trainable params: 243,672
#> Non-trainable params: 0
#> ________________________________________________________________________________Menentukan cost function dan optimizer.
model_signin %>% compile(loss = 'sparse_categorical_crossentropy',
optimizer = optimizer_adam(learning_rate = 0.001),
metrics = 'accuracy')
model_signout %>% compile(loss = 'sparse_categorical_crossentropy',
optimizer = optimizer_adam(learning_rate = 0.001),
metrics = 'accuracy')Mencoba melakukan training data dengan menggunakan fungsi
fitterhadap data train dengan epoch 10, batch_size = 150.
history_signin <- model_signin %>%
fit(x = train_x,
y = train_y,
epoch = 10,
batch_size = 150,
shuffle = F,
validation_data = list(test_x, test_y),
verbose = 1)
plot(history_signin)
history_signout <- model_signout %>%
fit(x = train_x,
y = train_y,
epoch = 10,
batch_size = 150,
shuffle = F,
verbose = 1)
plot(history_signout)
MODEL EVALUATION
Untuk memprediksi hasilnya, kita dapat menggunakan metode
predict().
#predict model_signin
pred.signin <- predict(model_signin, test_x) %>%
k_argmax() %>%
as.array() %>%
as.factor()
head(pred.signin)#> [1] 6 5 9 0 3 3
#> Levels: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23#predict model_signout
pred.signout <- predict(model_signout, test_x) %>%
k_argmax() %>%
as.array() %>%
as.factor()
head(pred.signout)#> [1] 6 5 9 0 3 21
#> Levels: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23Terakhir, kita evaluasi performa model menggunakan Confusion Matrix dengan metode
confusionMatrixdari librarycaret.
#> Confusion Matrix and Statistics
#>
#> Reference
#> Prediction 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
#> 0 302 0 0 0 0 0 0 0 23 0 0 7 79 0 0 0 0
#> 1 0 352 0 0 2 0 0 17 0 0 0 0 0 0 0 0 0
#> 2 0 0 268 0 0 7 0 0 0 0 0 0 0 0 0 0 0
#> 3 0 36 0 202 0 0 20 0 0 0 0 0 2 0 0 0 0
#> 4 0 0 0 0 400 0 0 0 0 0 0 58 0 3 0 0 0
#> 5 0 0 21 0 0 225 0 0 0 19 0 0 0 44 0 2 0
#> 6 0 0 0 0 0 0 166 15 0 0 0 0 0 20 0 0 0
#> 7 0 0 0 0 0 0 46 318 0 0 0 0 0 1 10 0 0
#> 8 2 0 0 0 0 0 0 21 164 0 0 1 0 0 0 0 20
#> 9 0 21 0 0 0 0 0 0 0 203 0 0 0 0 2 0 0
#> 10 0 0 0 5 0 14 20 0 21 2 209 0 0 0 13 0 21
#> 11 0 0 0 0 29 0 0 21 0 0 0 203 49 0 0 21 0
#> 12 0 0 0 0 0 0 0 0 15 0 0 37 82 0 0 0 0
#> 13 27 0 0 0 0 0 3 16 0 0 0 0 22 137 8 0 0
#> 14 0 0 0 0 0 0 0 0 0 1 0 0 0 0 282 0 0
#> 15 0 0 0 0 0 0 35 0 0 0 0 20 16 5 3 141 0
#> 16 0 0 0 2 0 0 0 0 21 65 0 0 0 0 0 0 82
#> 17 0 0 0 0 67 0 0 0 6 0 0 68 18 0 0 0 0
#> 18 0 0 0 0 0 0 52 5 0 0 0 0 20 36 0 0 0
#> 19 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20
#> 20 0 1 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0
#> 21 0 2 0 0 0 1 0 0 0 0 0 0 0 0 19 0 0
#> 22 0 0 21 36 0 0 6 23 0 0 0 0 3 0 6 0 0
#> 23 0 0 0 0 0 0 0 0 38 41 0 0 0 0 0 0 1
#> Reference
#> Prediction 17 18 19 20 21 22 23
#> 0 0 0 0 0 0 0 0
#> 1 0 0 0 0 0 0 0
#> 2 0 0 0 0 0 0 0
#> 3 0 0 20 15 11 0 0
#> 4 22 0 0 0 0 0 0
#> 5 0 0 0 20 0 0 0
#> 6 0 7 0 0 0 0 0
#> 7 0 0 0 0 1 0 0
#> 8 55 21 0 0 0 21 37
#> 9 0 0 50 59 39 3 0
#> 10 0 34 39 42 0 33 41
#> 11 62 0 0 0 0 0 0
#> 12 19 0 0 0 0 0 0
#> 13 0 0 0 0 0 0 0
#> 14 0 0 0 0 0 0 0
#> 15 0 0 0 0 0 0 0
#> 16 0 0 43 21 40 11 12
#> 17 49 0 0 0 0 21 6
#> 18 0 76 0 5 0 0 21
#> 19 0 20 73 14 21 0 0
#> 20 0 2 0 123 14 0 1
#> 21 0 0 0 6 80 33 0
#> 22 0 88 0 20 0 145 0
#> 23 39 0 41 21 0 0 214
#>
#> Overall Statistics
#>
#> Accuracy : 0.6269
#> 95% CI : (0.6156, 0.6381)
#> No Information Rate : 0.0694
#> P-Value [Acc > NIR] : < 0.00000000000000022
#>
#> Kappa : 0.61
#>
#> Mcnemar's Test P-Value : NA
#>
#> Statistics by Class:
#>
#> Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
#> Sensitivity 0.91239 0.81481 0.86452 0.82449 0.80321 0.91093
#> Specificity 0.98407 0.99718 0.99898 0.98499 0.98756 0.98469
#> Pos Pred Value 0.73479 0.94879 0.97455 0.66013 0.82816 0.67976
#> Neg Pred Value 0.99571 0.98824 0.99391 0.99374 0.98535 0.99678
#> Prevalence 0.04615 0.06023 0.04322 0.03416 0.06944 0.03444
#> Detection Rate 0.04211 0.04908 0.03737 0.02817 0.05577 0.03137
#> Detection Prevalence 0.05731 0.05173 0.03834 0.04267 0.06735 0.04615
#> Balanced Accuracy 0.94823 0.90600 0.93175 0.90474 0.89539 0.94781
#> Class: 6 Class: 7 Class: 8 Class: 9 Class: 10 Class: 11
#> Sensitivity 0.47701 0.72936 0.56944 0.61329 1.00000 0.51523
#> Specificity 0.99385 0.99139 0.97414 0.97457 0.95907 0.97315
#> Pos Pred Value 0.79808 0.84574 0.47953 0.53846 0.42308 0.52727
#> Neg Pred Value 0.97387 0.98264 0.98184 0.98116 1.00000 0.97186
#> Prevalence 0.04852 0.06079 0.04016 0.04615 0.02914 0.05494
#> Detection Rate 0.02315 0.04434 0.02287 0.02830 0.02914 0.02830
#> Detection Prevalence 0.02900 0.05243 0.04769 0.05257 0.06888 0.05368
#> Balanced Accuracy 0.73543 0.86037 0.77179 0.79393 0.97953 0.74419
#> Class: 12 Class: 13 Class: 14 Class: 15 Class: 16
#> Sensitivity 0.28179 0.5569 0.81268 0.85976 0.56944
#> Specificity 0.98968 0.9890 0.99985 0.98873 0.96941
#> Pos Pred Value 0.53595 0.6432 0.99647 0.64091 0.27609
#> Neg Pred Value 0.97022 0.9843 0.99056 0.99669 0.99098
#> Prevalence 0.04057 0.0343 0.04838 0.02287 0.02008
#> Detection Rate 0.01143 0.0191 0.03932 0.01966 0.01143
#> Detection Prevalence 0.02133 0.0297 0.03946 0.03067 0.04141
#> Balanced Accuracy 0.63573 0.7730 0.90627 0.92424 0.76943
#> Class: 17 Class: 18 Class: 19 Class: 20 Class: 21
#> Sensitivity 0.199187 0.30645 0.27444 0.35549 0.38835
#> Specificity 0.973145 0.97992 0.98624 0.99678 0.99124
#> Pos Pred Value 0.208511 0.35349 0.43452 0.84828 0.56738
#> Neg Pred Value 0.971602 0.97528 0.97244 0.96827 0.98208
#> Prevalence 0.034300 0.03458 0.03709 0.04824 0.02872
#> Detection Rate 0.006832 0.01060 0.01018 0.01715 0.01115
#> Detection Prevalence 0.032766 0.02998 0.02342 0.02022 0.01966
#> Balanced Accuracy 0.586166 0.64319 0.63034 0.67613 0.68980
#> Class: 22 Class: 23
#> Sensitivity 0.54307 0.64458
#> Specificity 0.97060 0.97354
#> Pos Pred Value 0.41667 0.54177
#> Neg Pred Value 0.98212 0.98259
#> Prevalence 0.03723 0.04629
#> Detection Rate 0.02022 0.02984
#> Detection Prevalence 0.04852 0.05508
#> Balanced Accuracy 0.75684 0.80906#> Confusion Matrix and Statistics
#>
#> Reference
#> Prediction 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
#> 0 325 0 0 0 0 0 0 0 0 0 0 2 36 2 0 0 0
#> 1 0 351 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
#> 2 0 0 289 0 0 21 0 0 0 0 0 0 13 20 16 0 0
#> 3 0 39 0 223 0 0 0 0 0 0 0 0 0 0 0 0 0
#> 4 0 0 0 0 440 0 0 21 0 0 0 0 18 21 0 0 0
#> 5 0 0 21 0 0 204 0 0 0 0 0 0 0 20 0 0 0
#> 6 0 0 0 0 0 0 228 22 0 0 0 0 0 20 0 0 19
#> 7 0 0 0 0 0 0 41 371 0 0 0 0 0 0 0 0 0
#> 8 0 0 0 0 0 0 0 0 194 0 0 0 0 0 0 0 0
#> 9 0 21 0 0 0 21 0 0 0 223 0 0 0 0 0 0 0
#> 10 0 0 0 0 0 1 0 0 0 0 188 0 0 0 0 0 0
#> 11 0 0 0 0 26 0 0 0 0 0 0 331 29 9 0 1 0
#> 12 6 0 0 0 0 0 4 0 0 0 0 40 166 0 0 0 0
#> 13 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0 0 0
#> 14 0 0 0 0 0 0 21 0 0 0 0 0 0 0 323 0 0
#> 15 0 0 0 0 0 0 37 0 4 1 0 0 21 18 8 153 0
#> 16 0 0 0 0 0 0 0 0 42 56 0 0 0 0 0 0 103
#> 17 0 0 0 0 32 0 0 0 2 9 0 21 0 0 0 10 0
#> 18 0 0 0 0 0 0 17 21 0 0 0 0 8 9 0 0 0
#> 19 0 20 0 2 0 0 0 1 0 1 0 0 0 0 0 0 1
#> 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21
#> 21 0 1 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0
#> 22 0 0 0 20 0 0 0 0 4 0 21 0 0 0 0 0 0
#> 23 0 0 0 0 0 0 0 0 42 21 0 0 0 0 0 0 0
#> Reference
#> Prediction 17 18 19 20 21 22 23
#> 0 0 0 0 0 0 0 0
#> 1 0 0 0 0 0 0 16
#> 2 0 0 0 0 0 0 0
#> 3 0 0 13 0 0 0 0
#> 4 21 0 0 0 0 0 0
#> 5 0 0 0 20 0 0 0
#> 6 0 0 0 0 0 0 0
#> 7 14 0 0 0 0 0 0
#> 8 27 0 0 0 0 0 25
#> 9 0 0 102 41 0 0 21
#> 10 0 20 0 0 0 1 40
#> 11 42 0 0 0 0 0 0
#> 12 20 0 0 0 0 0 0
#> 13 0 0 0 0 0 0 0
#> 14 0 0 0 10 0 0 0
#> 15 17 1 0 3 0 0 0
#> 16 0 13 21 0 41 9 23
#> 17 105 0 0 0 0 3 20
#> 18 0 132 0 0 0 0 21
#> 19 0 0 110 32 20 0 0
#> 20 0 17 20 200 20 1 19
#> 21 0 0 0 33 125 56 0
#> 22 0 65 0 7 0 197 0
#> 23 0 0 0 0 0 0 147
#>
#> Overall Statistics
#>
#> Accuracy : 0.7327
#> 95% CI : (0.7223, 0.7429)
#> No Information Rate : 0.0694
#> P-Value [Acc > NIR] : < 0.00000000000000022
#>
#> Kappa : 0.7204
#>
#> Mcnemar's Test P-Value : NA
#>
#> Statistics by Class:
#>
#> Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
#> Sensitivity 0.98187 0.81250 0.93226 0.91020 0.88353 0.82591
#> Specificity 0.99415 0.99763 0.98980 0.99249 0.98786 0.99119
#> Pos Pred Value 0.89041 0.95640 0.80501 0.81091 0.84453 0.76981
#> Neg Pred Value 0.99912 0.98810 0.99692 0.99681 0.99128 0.99377
#> Prevalence 0.04615 0.06023 0.04322 0.03416 0.06944 0.03444
#> Detection Rate 0.04532 0.04894 0.04030 0.03109 0.06135 0.02844
#> Detection Prevalence 0.05089 0.05117 0.05006 0.03834 0.07264 0.03695
#> Balanced Accuracy 0.98801 0.90506 0.96103 0.95135 0.93570 0.90855
#> Class: 6 Class: 7 Class: 8 Class: 9 Class: 10 Class: 11
#> Sensitivity 0.65517 0.85092 0.67361 0.67372 0.89952 0.84010
#> Specificity 0.99106 0.99183 0.99245 0.96989 0.99110 0.98421
#> Pos Pred Value 0.78893 0.87089 0.78862 0.51981 0.75200 0.75571
#> Neg Pred Value 0.98257 0.99036 0.98643 0.98398 0.99697 0.99064
#> Prevalence 0.04852 0.06079 0.04016 0.04615 0.02914 0.05494
#> Detection Rate 0.03179 0.05173 0.02705 0.03109 0.02621 0.04615
#> Detection Prevalence 0.04030 0.05940 0.03430 0.05982 0.03486 0.06107
#> Balanced Accuracy 0.82312 0.92138 0.83303 0.82180 0.94531 0.91216
#> Class: 12 Class: 13 Class: 14 Class: 15 Class: 16
#> Sensitivity 0.57045 0.51626 0.93084 0.93293 0.71528
#> Specificity 0.98983 1.00000 0.99546 0.98430 0.97083
#> Pos Pred Value 0.70339 1.00000 0.91243 0.58175 0.33442
#> Neg Pred Value 0.98198 0.98311 0.99648 0.99841 0.99403
#> Prevalence 0.04057 0.03430 0.04838 0.02287 0.02008
#> Detection Rate 0.02315 0.01771 0.04504 0.02133 0.01436
#> Detection Prevalence 0.03291 0.01771 0.04936 0.03667 0.04294
#> Balanced Accuracy 0.78014 0.75813 0.96315 0.95862 0.84305
#> Class: 17 Class: 18 Class: 19 Class: 20 Class: 21
#> Sensitivity 0.42683 0.53226 0.41353 0.57803 0.60680
#> Specificity 0.98599 0.98902 0.98885 0.98564 0.98421
#> Pos Pred Value 0.51980 0.63462 0.58824 0.67114 0.53191
#> Neg Pred Value 0.97977 0.98334 0.97767 0.97876 0.98832
#> Prevalence 0.03430 0.03458 0.03709 0.04824 0.02872
#> Detection Rate 0.01464 0.01840 0.01534 0.02789 0.01743
#> Detection Prevalence 0.02817 0.02900 0.02607 0.04155 0.03277
#> Balanced Accuracy 0.70641 0.76064 0.70119 0.78184 0.79550
#> Class: 22 Class: 23
#> Sensitivity 0.73783 0.44277
#> Specificity 0.98306 0.99079
#> Pos Pred Value 0.62739 0.70000
#> Neg Pred Value 0.98979 0.97343
#> Prevalence 0.03723 0.04629
#> Detection Rate 0.02747 0.02050
#> Detection Prevalence 0.04378 0.02928
#> Balanced Accuracy 0.86044 0.71678Model Tunning
Dari kedua model diatas model_signin dan
model_signout didapatkan hasil bahwa kedua model belum
memberikan performa cukup baik, dimana masih cenderung overfitting. Maka
perlu di lakukan perbaikan menggunakan model terakhir kita yaitu
model_signout.
Mmebuat model ketiga
signout_tuning- Input layer: 784 prediktor (gambar 28x28 pixel) - Input1: 128 neuron dengan activation function = ReLu - Input2: 64 neuron dengan activation function = ReLu - Output: 24 neuron (sesuai jumlah kategori 0-9) dengan activation function = Softmax
signout_tuning <- keras_model_sequential(name = "signout_tuning")
signout_tuning %>%
layer_dense(
input_shape = 784,
units = 128,
activation = "relu",
name = "Input1") %>%
layer_dense(units = 64,
activation = "relu",
name = "Input2") %>%
layer_dense(units = 24,
activation = "softmax",
name = "Output")
summary(signout_tuning)#> Model: "signout_tuning"
#> ________________________________________________________________________________
#> Layer (type) Output Shape Param #
#> ================================================================================
#> Input1 (Dense) (None, 128) 100480
#> Input2 (Dense) (None, 64) 8256
#> Output (Dense) (None, 24) 1560
#> ================================================================================
#> Total params: 110,296
#> Trainable params: 110,296
#> Non-trainable params: 0
#> ________________________________________________________________________________Menentukan cost function dan optimizer.
signout_tuning %>%
compile(loss = "sparse_categorical_crossentropy",
optimizer = optimizer_adam(learning_rate = 0.001),
metrics = "accuracy")Training data dengan menggunakan fungsi
fitterhadap data train dengan epoch 10, batch_size = 150.
his_sign_tun <- signout_tuning %>%
fit(x = train_x,
y = train_y,
epoch = 10,
batch_size = 150,
shuffle = F,
validation_data = list(test_x, test_y),
verbose = 1)
plot(his_sign_tun)
Kita prediksi hasilnya menggunakan metode
predict().
kita evaluasi performa model menggunakan Confusion Matrix dengan metode
confusionMatrixdari librarycaret.
#> Confusion Matrix and Statistics
#>
#> Reference
#> Prediction 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
#> 0 331 0 0 0 0 0 0 0 0 0 0 0 79 0 0 0 0
#> 1 0 332 0 19 3 0 0 0 0 0 0 0 0 0 0 0 0
#> 2 0 0 268 0 0 20 0 0 0 0 2 0 0 0 0 0 0
#> 3 0 5 0 134 0 0 0 0 0 0 0 0 0 0 0 0 0
#> 4 0 0 0 0 410 0 0 21 0 0 0 0 21 0 0 0 0
#> 5 0 0 21 0 0 220 0 0 0 20 0 0 0 42 0 0 0
#> 6 0 0 0 0 0 0 246 26 0 0 1 0 0 21 0 0 0
#> 7 0 0 0 0 0 0 44 374 21 0 0 0 0 0 0 0 0
#> 8 0 0 0 0 0 0 0 0 199 0 0 0 0 0 1 0 0
#> 9 0 17 0 0 0 0 0 0 0 162 0 0 0 0 0 0 4
#> 10 0 0 2 0 0 0 0 0 0 0 203 0 0 0 0 0 21
#> 11 0 0 0 0 22 0 0 0 0 0 0 191 5 0 0 21 0
#> 12 0 0 0 0 0 0 18 0 0 0 0 21 97 0 0 0 0
#> 13 0 0 0 0 0 0 0 0 0 0 0 21 24 162 0 0 0
#> 14 0 0 0 0 0 0 0 15 0 1 0 0 0 0 346 2 0
#> 15 0 0 0 0 0 0 18 0 0 0 0 16 21 12 0 141 0
#> 16 0 0 0 35 0 0 0 0 0 96 0 0 0 0 0 0 74
#> 17 0 0 0 0 63 0 0 0 4 0 0 145 26 0 0 0 21
#> 18 0 0 0 0 0 1 17 0 0 0 0 0 18 9 0 0 0
#> 19 0 24 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0
#> 20 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 8
#> 21 0 54 0 0 0 6 0 0 21 6 2 0 0 0 0 0 16
#> 22 0 0 19 36 0 0 5 0 1 0 1 0 0 0 0 0 0
#> 23 0 0 0 0 0 0 0 0 42 32 0 0 0 0 0 0 0
#> Reference
#> Prediction 17 18 19 20 21 22 23
#> 0 1 0 0 0 0 0 0
#> 1 0 0 0 0 0 0 0
#> 2 0 0 0 0 0 0 0
#> 3 0 0 0 0 0 0 0
#> 4 21 0 0 0 0 0 0
#> 5 0 0 0 38 0 0 0
#> 6 0 25 0 0 0 0 0
#> 7 8 13 0 0 0 0 0
#> 8 21 0 0 0 0 0 3
#> 9 0 0 109 51 0 0 20
#> 10 0 21 0 0 0 12 41
#> 11 48 0 0 0 0 0 0
#> 12 0 0 0 0 0 0 0
#> 13 0 0 0 0 0 0 0
#> 14 0 0 0 0 0 0 0
#> 15 0 0 0 0 0 0 0
#> 16 0 20 42 1 11 11 16
#> 17 147 0 0 0 0 0 43
#> 18 0 88 0 0 0 0 21
#> 19 0 0 60 0 3 0 0
#> 20 0 0 53 189 17 7 19
#> 21 0 19 0 33 175 63 0
#> 22 0 62 0 20 0 174 0
#> 23 0 0 2 14 0 0 169
#>
#> Overall Statistics
#>
#> Accuracy : 0.6821
#> 95% CI : (0.6712, 0.6929)
#> No Information Rate : 0.0694
#> P-Value [Acc > NIR] : < 0.00000000000000022
#>
#> Kappa : 0.6676
#>
#> Mcnemar's Test P-Value : NA
#>
#> Statistics by Class:
#>
#> Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
#> Sensitivity 1.00000 0.76852 0.86452 0.54694 0.82329 0.89069
#> Specificity 0.98831 0.99674 0.99679 0.99928 0.99056 0.98253
#> Pos Pred Value 0.80535 0.93785 0.92414 0.96403 0.86681 0.64516
#> Neg Pred Value 1.00000 0.98533 0.99390 0.98422 0.98686 0.99605
#> Prevalence 0.04615 0.06023 0.04322 0.03416 0.06944 0.03444
#> Detection Rate 0.04615 0.04629 0.03737 0.01868 0.05717 0.03067
#> Detection Prevalence 0.05731 0.04936 0.04044 0.01938 0.06595 0.04755
#> Balanced Accuracy 0.99415 0.88263 0.93066 0.77311 0.90693 0.93661
#> Class: 6 Class: 7 Class: 8 Class: 9 Class: 10 Class: 11
#> Sensitivity 0.70690 0.85780 0.69097 0.48943 0.97129 0.48477
#> Specificity 0.98930 0.98723 0.99637 0.97062 0.98607 0.98584
#> Pos Pred Value 0.77116 0.81304 0.88839 0.44628 0.67667 0.66551
#> Neg Pred Value 0.98512 0.99076 0.98719 0.97518 0.99913 0.97052
#> Prevalence 0.04852 0.06079 0.04016 0.04615 0.02914 0.05494
#> Detection Rate 0.03430 0.05215 0.02775 0.02259 0.02830 0.02663
#> Detection Prevalence 0.04448 0.06414 0.03123 0.05061 0.04183 0.04002
#> Balanced Accuracy 0.84810 0.92252 0.84367 0.73002 0.97868 0.73530
#> Class: 12 Class: 13 Class: 14 Class: 15 Class: 16
#> Sensitivity 0.33333 0.65854 0.99712 0.85976 0.51389
#> Specificity 0.99433 0.99350 0.99736 0.99044 0.96699
#> Pos Pred Value 0.71324 0.78261 0.95055 0.67788 0.24183
#> Neg Pred Value 0.97243 0.98794 0.99985 0.99670 0.98980
#> Prevalence 0.04057 0.03430 0.04838 0.02287 0.02008
#> Detection Rate 0.01352 0.02259 0.04824 0.01966 0.01032
#> Detection Prevalence 0.01896 0.02886 0.05075 0.02900 0.04267
#> Balanced Accuracy 0.66383 0.82602 0.99724 0.92510 0.74044
#> Class: 17 Class: 18 Class: 19 Class: 20 Class: 21
#> Sensitivity 0.5976 0.35484 0.225564 0.54624 0.84951
#> Specificity 0.9564 0.99047 0.994063 0.98169 0.96842
#> Pos Pred Value 0.3274 0.57143 0.594059 0.60191 0.44304
#> Neg Pred Value 0.9853 0.97720 0.970867 0.97711 0.99543
#> Prevalence 0.0343 0.03458 0.037089 0.04824 0.02872
#> Detection Rate 0.0205 0.01227 0.008366 0.02635 0.02440
#> Detection Prevalence 0.0626 0.02147 0.014083 0.04378 0.05508
#> Balanced Accuracy 0.7770 0.67265 0.609814 0.76397 0.90897
#> Class: 22 Class: 23
#> Sensitivity 0.65169 0.50904
#> Specificity 0.97915 0.98684
#> Pos Pred Value 0.54717 0.65251
#> Neg Pred Value 0.98643 0.97642
#> Prevalence 0.03723 0.04629
#> Detection Rate 0.02426 0.02356
#> Detection Prevalence 0.04434 0.03611
#> Balanced Accuracy 0.81542 0.74794SUMMARY
Kesimpulan yang bisa diambil dari ketiga model kita
(model_signin, model_signout,
signout_tuning). Model yang paling baik adalah model ke
tiga yaitu signout_tuning. Dimana memiliki nilai akurasinya
cukup tinggi dan selisih akurasi antara data train dan data test paling
kecil.