LBB NN
Adrian Djaya Prathama
2024-03-26
1 Introduction
Sign language MNIST is a dataset inspired by the Fashion-MNIST 2 and the machine learning pipeline for gestures by Sreehari 4. A robust visual recognition algorithm could provide not only new benchmarks that challenge modern machine learning methods such as Convolutional Neural Nets but also could pragmatically help the deaf and hard-of-hearing better communicate using computer vision applications.
The National Institute on Deafness and other Communications Disorders (NIDCD) indicates that the 200-year-old American Sign Language is a complete, complex language (of which letter gestures are only part) but is the primary language for many deaf North Americans.
2 Data Preparation
2.1 Import Library
Before we create a neural network model, we will import library that we need.
library(dplyr)
library(keras)
library(caret)
library(tensorflow)2.2 Import Data
Then we will import the data. In this case we have 2 data,
train and test.
sign_train <- read.csv("data_input/sign_mnist_train/sign_mnist_train.csv")
sign_test <- read.csv("data_input/sign_mnist_test/sign_mnist_test.csv")
head(sign_train, 6)The data consist 785 columns, a target variable column and 784 columns as our pixel value which could project a 28x28 with grayscale color.
To illustrate our image, we will create a function that named
vizTrain
vizTrain <- function(input){
dimmax <- sqrt(ncol(input[,-1]))
dimn <- ceiling(sqrt(nrow(input)))
par(mfrow=c(dimn, dimn), mar=c(.1, .1, .1, .1))
for (i in 1:nrow(input)){
m1 <- as.matrix(input[i,2:785])
dim(m1) <- c(28,28)
m1 <- apply(m1, 1, t)
image(1:28, 1:28,
m1, col=grey.colors(255),
xaxt = 'n', yaxt = 'n')
text(2, 20, col="white", cex=1.2, input[i, 1])
}
}Then we apply our dataset into the function
vizTrain(head(sign_train, 30))
3 Data Preprocessing
Before we create a model, we need to process our data first.
Our first step in processing our data is by separating predictor in our data with from our variable.
We will split our variable data and convert it into matrix
train_X <- sign_train %>% select(-label) %>% as.matrix()
test_X <- sign_test %>% select(-label) %>% as.matrix()Then we will take our predictor data and perform
One-hot encoding using to_categorical()
function from keras library
train_y <- to_categorical(sign_train$label)
test_y <- to_categorical(sign_test$label)Next we will convert our variable data into array
train_x_array <- as.array(x = train_X, dim = dim(train_X))
test_x_array <- as.array(x = test_X, dim = dim(test_X))Lastly we will scale the variable data by divide it by 255. By doing so we make our data scale from 0 until 1.
train_x <- train_x_array / 255
test_x <- test_x_array / 2554 Modeling
We will start our model by create a base model. We will add 1 hidden
layer first as our base with 64 neurons with relu
activation. Our output will be 25 neuron with softmax
activation. input_shape = 784 also means our input data
consist of 784 neuron.
set_random_seed(189)
model <- keras_model_sequential(name = "Model") %>%
layer_dense(input_shape = 784,
units = 64,
activation = "relu",
name = "Hidden1") %>%
layer_dense(units = 25,
activation = "softmax",
name = "Output")
model## Model: "Model"
## ________________________________________________________________________________
## Layer (type) Output Shape Param #
## ================================================================================
## Hidden1 (Dense) (None, 64) 50240
## Output (Dense) (None, 25) 1625
## ================================================================================
## Total params: 51,865
## Trainable params: 51,865
## Non-trainable params: 0
## ________________________________________________________________________________Then we will compile it to determine our optimizer, learning rate, error function, and metrics. For our base model we will use:
- Optimizer =
optimizer_adam() - Learning Rate = 0.01
- Error Function =
Categorical Cross-Entropy - Metrics =
accuracy
model %>%
compile(optimizer = optimizer_adam(learning_rate = 0.01),
loss = "categorical_crossentropy",
metrics = "accuracy")Next we will try to fit our model with our data. We will set our
epoch 15 and our batch_size 128
history <- model %>%
fit(x = train_x,
y = train_y,
epochs = 15,
batch_size = 128)## Epoch 1/15
## 215/215 - 1s - loss: 3.2386 - accuracy: 0.0444 - 880ms/epoch - 4ms/step
## Epoch 2/15
## 215/215 - 0s - loss: 3.1824 - accuracy: 0.0445 - 412ms/epoch - 2ms/step
## Epoch 3/15
## 215/215 - 0s - loss: 3.1794 - accuracy: 0.0453 - 370ms/epoch - 2ms/step
## Epoch 4/15
## 215/215 - 0s - loss: 3.1786 - accuracy: 0.0461 - 379ms/epoch - 2ms/step
## Epoch 5/15
## 215/215 - 0s - loss: 3.1780 - accuracy: 0.0458 - 370ms/epoch - 2ms/step
## Epoch 6/15
## 215/215 - 0s - loss: 3.1776 - accuracy: 0.0445 - 358ms/epoch - 2ms/step
## Epoch 7/15
## 215/215 - 0s - loss: 3.1777 - accuracy: 0.0447 - 382ms/epoch - 2ms/step
## Epoch 8/15
## 215/215 - 0s - loss: 3.1773 - accuracy: 0.0448 - 374ms/epoch - 2ms/step
## Epoch 9/15
## 215/215 - 0s - loss: 3.1771 - accuracy: 0.0464 - 383ms/epoch - 2ms/step
## Epoch 10/15
## 215/215 - 0s - loss: 3.1772 - accuracy: 0.0453 - 392ms/epoch - 2ms/step
## Epoch 11/15
## 215/215 - 0s - loss: 3.1773 - accuracy: 0.0460 - 379ms/epoch - 2ms/step
## Epoch 12/15
## 215/215 - 0s - loss: 3.1770 - accuracy: 0.0442 - 397ms/epoch - 2ms/step
## Epoch 13/15
## 215/215 - 0s - loss: 3.1770 - accuracy: 0.0443 - 405ms/epoch - 2ms/step
## Epoch 14/15
## 215/215 - 0s - loss: 3.1771 - accuracy: 0.0460 - 390ms/epoch - 2ms/step
## Epoch 15/15
## 215/215 - 0s - loss: 3.1769 - accuracy: 0.0469 - 388ms/epoch - 2ms/stepplot(history)
From fitting our model with our data, it seems that our model is really bad in predicting the data. But we will then evaluate how it performs.
5 Model Evaluation
To evaluate the model, we need to predict it with our test data using
predict() function.
pred <- predict(model, test_x) %>%
k_argmax() %>%
as.array() %>%
as.factor()## 225/225 - 0s - 217ms/epoch - 965us/stephead(pred) ## [1] 16 16 16 16 16 16
## Levels: 16Then we use confusionMatrix() to evaluate the result of
our model prediction
confusionMatrix(pred,
as.factor(sign_test$label))## Confusion Matrix and Statistics
##
## Reference
## Prediction 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 331 432 310 245 498 247 348 436 288 331 209 394 291 246 347 164 144
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## Reference
## Prediction 18 19 20 21 22 23 24
## 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0
## 16 246 248 266 346 206 267 332
## 17 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0
##
## Overall Statistics
##
## Accuracy : 0.0229
## 95% CI : (0.0195, 0.0266)
## No Information Rate : 0.0694
## P-Value [Acc > NIR] : 1
##
## Kappa : 0
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
## Sensitivity 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000
## Specificity 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
## Pos Pred Value NaN NaN NaN NaN NaN NaN
## Neg Pred Value 0.95385 0.93977 0.95678 0.96584 0.93056 0.96556
## Prevalence 0.04615 0.06023 0.04322 0.03416 0.06944 0.03444
## Detection Rate 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000
## Detection Prevalence 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000
## Balanced Accuracy 0.50000 0.50000 0.50000 0.50000 0.50000 0.50000
## Class: 6 Class: 7 Class: 8 Class: 10 Class: 11 Class: 12
## Sensitivity 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000
## Specificity 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
## Pos Pred Value NaN NaN NaN NaN NaN NaN
## Neg Pred Value 0.95148 0.93921 0.95984 0.95385 0.97086 0.94506
## Prevalence 0.04852 0.06079 0.04016 0.04615 0.02914 0.05494
## Detection Rate 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000
## Detection Prevalence 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000
## Balanced Accuracy 0.50000 0.50000 0.50000 0.50000 0.50000 0.50000
## Class: 13 Class: 14 Class: 15 Class: 16 Class: 17
## Sensitivity 0.00000 0.0000 0.00000 1.00000 0.00000
## Specificity 1.00000 1.0000 1.00000 0.00000 1.00000
## Pos Pred Value NaN NaN NaN 0.02287 NaN
## Neg Pred Value 0.95943 0.9657 0.95162 NaN 0.97992
## Prevalence 0.04057 0.0343 0.04838 0.02287 0.02008
## Detection Rate 0.00000 0.0000 0.00000 0.02287 0.00000
## Detection Prevalence 0.00000 0.0000 0.00000 1.00000 0.00000
## Balanced Accuracy 0.50000 0.5000 0.50000 0.50000 0.50000
## Class: 18 Class: 19 Class: 20 Class: 21 Class: 22
## Sensitivity 0.0000 0.00000 0.00000 0.00000 0.00000
## Specificity 1.0000 1.00000 1.00000 1.00000 1.00000
## Pos Pred Value NaN NaN NaN NaN NaN
## Neg Pred Value 0.9657 0.96542 0.96291 0.95176 0.97128
## Prevalence 0.0343 0.03458 0.03709 0.04824 0.02872
## Detection Rate 0.0000 0.00000 0.00000 0.00000 0.00000
## Detection Prevalence 0.0000 0.00000 0.00000 0.00000 0.00000
## Balanced Accuracy 0.5000 0.50000 0.50000 0.50000 0.50000
## Class: 23 Class: 24
## Sensitivity 0.00000 0.00000
## Specificity 1.00000 1.00000
## Pos Pred Value NaN NaN
## Neg Pred Value 0.96277 0.95371
## Prevalence 0.03723 0.04629
## Detection Rate 0.00000 0.00000
## Detection Prevalence 0.00000 0.00000
## Balanced Accuracy 0.50000 0.50000Our model accuracy is only 2.29% which is really bad. We need to improve our model to further increase the accuracy.
6 Model Improvement
We will add 4 more hidden layers in our tune model with 128, 256,
512, and 512 neuron for each of our new hidden layers with
relu activation
set_random_seed(189)
model1 <- keras_model_sequential(name = "Model_Tun") %>%
layer_dense(input_shape = 784,
units = 64,
activation = "relu",
name = "Hidden1") %>%
layer_dense(units = 128,
activation = "relu",
name = "Hidden2") %>%
layer_dense(units = 256,
activation = "relu",
name = "Hidden3") %>%
layer_dense(units = 512,
activation = "relu",
name = "Hidden4") %>%
layer_dense(units = 512,
activation = "relu",
name = "Hidden5") %>%
layer_dense(units = 25,
activation = "softmax",
name = "Output")
model1## Model: "Model_Tun"
## ________________________________________________________________________________
## Layer (type) Output Shape Param #
## ================================================================================
## Hidden1 (Dense) (None, 64) 50240
## Hidden2 (Dense) (None, 128) 8320
## Hidden3 (Dense) (None, 256) 33024
## Hidden4 (Dense) (None, 512) 131584
## Hidden5 (Dense) (None, 512) 262656
## Output (Dense) (None, 25) 12825
## ================================================================================
## Total params: 498,649
## Trainable params: 498,649
## Non-trainable params: 0
## ________________________________________________________________________________Then we will lower our learning_rate into 0.001
model1 %>%
compile(optimizer = optimizer_adam(learning_rate = 0.001),
loss = "categorical_crossentropy",
metrics = "accuracy")And set our epoch into 15
history <- model1 %>%
fit(x = train_x,
y = train_y,
epochs = 15,
batch_size = 128)## Epoch 1/15
## 215/215 - 2s - loss: 2.0371 - accuracy: 0.3355 - 2s/epoch - 10ms/step
## Epoch 2/15
## 215/215 - 1s - loss: 1.0415 - accuracy: 0.6385 - 1s/epoch - 6ms/step
## Epoch 3/15
## 215/215 - 1s - loss: 0.5511 - accuracy: 0.8079 - 1s/epoch - 6ms/step
## Epoch 4/15
## 215/215 - 1s - loss: 0.3070 - accuracy: 0.8953 - 1s/epoch - 6ms/step
## Epoch 5/15
## 215/215 - 1s - loss: 0.1541 - accuracy: 0.9494 - 1s/epoch - 6ms/step
## Epoch 6/15
## 215/215 - 1s - loss: 0.1348 - accuracy: 0.9569 - 1s/epoch - 6ms/step
## Epoch 7/15
## 215/215 - 1s - loss: 0.1174 - accuracy: 0.9640 - 1s/epoch - 6ms/step
## Epoch 8/15
## 215/215 - 1s - loss: 0.0112 - accuracy: 0.9987 - 1s/epoch - 6ms/step
## Epoch 9/15
## 215/215 - 1s - loss: 0.1121 - accuracy: 0.9656 - 1s/epoch - 6ms/step
## Epoch 10/15
## 215/215 - 1s - loss: 0.0036 - accuracy: 0.9999 - 1s/epoch - 6ms/step
## Epoch 11/15
## 215/215 - 1s - loss: 0.0014 - accuracy: 1.0000 - 1s/epoch - 6ms/step
## Epoch 12/15
## 215/215 - 1s - loss: 9.7756e-04 - accuracy: 1.0000 - 1s/epoch - 6ms/step
## Epoch 13/15
## 215/215 - 1s - loss: 7.0478e-04 - accuracy: 1.0000 - 1s/epoch - 6ms/step
## Epoch 14/15
## 215/215 - 1s - loss: 5.0481e-04 - accuracy: 1.0000 - 1s/epoch - 6ms/step
## Epoch 15/15
## 215/215 - 1s - loss: 4.1451e-04 - accuracy: 1.0000 - 1s/epoch - 6ms/stepLastly we will predict and evaluate our newly tuned model with our test data
pred1 <- predict(model1, test_x) %>%
k_argmax() %>%
as.array() %>%
as.factor()## 225/225 - 0s - 356ms/epoch - 2ms/stepconfusionMatrix(pred1,
as.factor(sign_test$label))## Confusion Matrix and Statistics
##
## Reference
## Prediction 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17
## 0 331 0 0 0 0 0 0 0 0 0 0 7 10 0 0 20 0
## 1 0 391 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 289 0 0 2 0 0 0 0 16 0 17 11 0 0 0
## 3 0 0 0 210 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 478 0 0 0 0 0 0 0 21 0 0 0 0
## 5 0 0 21 0 0 245 0 0 0 0 0 0 0 22 0 0 0
## 6 0 0 0 0 0 0 222 41 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 24 392 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 207 20 0 0 0 0 0 0 0
## 10 0 20 0 0 0 0 0 0 0 232 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 190 0 0 0 0 0 15
## 12 0 0 0 0 0 0 0 0 0 0 0 309 34 0 0 0 0
## 13 0 0 0 0 0 0 0 0 3 0 0 37 149 0 0 1 0
## 14 0 0 0 0 0 0 0 3 0 0 0 0 3 181 0 0 0
## 15 0 0 0 0 0 0 0 0 0 5 0 0 0 0 344 0 0
## 16 0 0 0 0 0 0 76 0 0 0 0 0 29 15 0 143 0
## 17 0 0 0 0 0 0 0 0 20 9 0 0 0 0 0 0 90
## 18 0 0 0 0 20 0 0 0 20 0 0 41 27 0 0 0 0
## 19 0 0 0 0 0 0 26 0 0 0 0 0 1 17 0 0 0
## 20 0 21 0 0 0 0 0 0 0 20 0 0 0 0 0 0 33
## 21 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 6
## 22 0 0 0 0 0 0 0 0 0 21 0 0 0 0 3 0 0
## 23 0 0 0 35 0 0 0 0 0 1 3 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 38 20 0 0 0 0 0 0 0
## Reference
## Prediction 18 19 20 21 22 23 24
## 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0
## 3 0 0 38 0 0 14 0
## 4 27 0 0 0 0 0 0
## 5 0 7 0 0 0 0 0
## 6 0 0 0 26 0 0 0
## 7 0 0 0 0 0 0 0
## 8 6 0 0 0 0 0 41
## 10 0 0 20 16 0 0 0
## 11 0 0 0 0 0 6 0
## 12 15 0 0 0 0 0 0
## 13 31 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0
## 15 0 0 0 0 0 4 0
## 16 0 3 0 9 0 0 0
## 17 0 0 0 0 20 4 42
## 18 166 0 0 0 0 16 0
## 19 0 172 0 0 0 0 21
## 20 0 4 185 19 39 0 0
## 21 0 0 5 231 8 0 19
## 22 0 0 0 21 139 15 0
## 23 0 62 0 5 0 187 0
## 24 1 0 18 19 0 21 209
##
## Overall Statistics
##
## Accuracy : 0.7936
## 95% CI : (0.7841, 0.803)
## No Information Rate : 0.0694
## P-Value [Acc > NIR] : < 0.00000000000000022
##
## Kappa : 0.7841
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
## Sensitivity 1.00000 0.90509 0.93226 0.85714 0.95984 0.99190
## Specificity 0.99459 1.00000 0.99330 0.99249 0.99281 0.99278
## Pos Pred Value 0.89946 1.00000 0.86269 0.80153 0.90875 0.83051
## Neg Pred Value 1.00000 0.99395 0.99693 0.99493 0.99699 0.99971
## Prevalence 0.04615 0.06023 0.04322 0.03416 0.06944 0.03444
## Detection Rate 0.04615 0.05452 0.04030 0.02928 0.06665 0.03416
## Detection Prevalence 0.05131 0.05452 0.04671 0.03653 0.07334 0.04113
## Balanced Accuracy 0.99730 0.95255 0.96278 0.92482 0.97632 0.99234
## Class: 6 Class: 7 Class: 8 Class: 10 Class: 11 Class: 12
## Sensitivity 0.63793 0.89908 0.71875 0.70091 0.90909 0.78426
## Specificity 0.99018 0.99644 0.99027 0.99181 0.99698 0.99277
## Pos Pred Value 0.76817 0.94231 0.75547 0.80556 0.90047 0.86313
## Neg Pred Value 0.98169 0.99349 0.98826 0.98562 0.99727 0.98753
## Prevalence 0.04852 0.06079 0.04016 0.04615 0.02914 0.05494
## Detection Rate 0.03095 0.05466 0.02886 0.03235 0.02649 0.04308
## Detection Prevalence 0.04030 0.05800 0.03820 0.04016 0.02942 0.04992
## Balanced Accuracy 0.81406 0.94776 0.85451 0.84636 0.95304 0.88852
## Class: 13 Class: 14 Class: 15 Class: 16 Class: 17
## Sensitivity 0.51203 0.73577 0.99135 0.87195 0.62500
## Specificity 0.98954 0.99913 0.99868 0.98116 0.98648
## Pos Pred Value 0.67421 0.96791 0.97450 0.52000 0.48649
## Neg Pred Value 0.97957 0.99069 0.99956 0.99696 0.99227
## Prevalence 0.04057 0.03430 0.04838 0.02287 0.02008
## Detection Rate 0.02078 0.02524 0.04796 0.01994 0.01255
## Detection Prevalence 0.03081 0.02607 0.04922 0.03834 0.02579
## Balanced Accuracy 0.75078 0.86745 0.99502 0.92656 0.80574
## Class: 18 Class: 19 Class: 20 Class: 21 Class: 22
## Sensitivity 0.67480 0.69355 0.69549 0.66763 0.67476
## Specificity 0.98210 0.99061 0.98031 0.99399 0.99139
## Pos Pred Value 0.57241 0.72574 0.57632 0.84926 0.69849
## Neg Pred Value 0.98838 0.98904 0.98818 0.98333 0.99039
## Prevalence 0.03430 0.03458 0.03709 0.04824 0.02872
## Detection Rate 0.02315 0.02398 0.02579 0.03221 0.01938
## Detection Prevalence 0.04044 0.03305 0.04476 0.03793 0.02775
## Balanced Accuracy 0.82845 0.84208 0.83790 0.83081 0.83307
## Class: 23 Class: 24
## Sensitivity 0.70037 0.62952
## Specificity 0.98465 0.98289
## Pos Pred Value 0.63823 0.64110
## Neg Pred Value 0.98837 0.98203
## Prevalence 0.03723 0.04629
## Detection Rate 0.02607 0.02914
## Detection Prevalence 0.04085 0.04545
## Balanced Accuracy 0.84251 0.80621Our new model perform much better than our base model with 79.36% accuracy.
7 References
Data was taken from [kaggle] (https://www.kaggle.com/datasets/datamunge/sign-language-mnist/data)