library(tidyverse)
library(ggplot2)
library(class)
library(caTools)
library(caret)
library(nnet)
library(keras)
library(NeuralNetTools)
library(tensorflow)
library(reticulate)1) Using nnet on Iris
80/20 training test
Nnet is a r package that models multi-nominal data through neutral networks. In short, we want to observe the linear logistic regression of one and more variables into one step. Below, we will observe the iris data set and its nodes in the neutral network.
Looking at our plot, we can see the predicted weights between our independent and dependent variables. It appears the species “Virginica” has a strong weight to the H3 Node in the hidden layer. This can mean that the species has a stronger probability of identifying that particular species with its combination of the indepdent values.
set.seed(3)
#Scale dataset
data<-iris
data[,1:4]<-scale(data[,1:4])
#We need 120 items in the train set, 80/20 ratio will work for the items below
split<-sample.split(data,SplitRatio = 0.8)
train_set<-subset(data,split=="TRUE")
test_set<-subset(data,split=="FALSE")
#creating the neutral network model, we want the network to predict the species of the independent variables in Iris (sepal/petal width and length)
#The hidden layer will have four nodes to
#we can run this nets for 600 iterations to train the model prediction accuracy and decay to remove any weights lower than our standard
model<-nnet(Species~.,data=train_set,size=4,decay=5e-4,maxit=600)## # weights: 35
## initial value 162.695787
## iter 10 value 8.399923
## iter 20 value 5.472354
## iter 30 value 3.893571
## iter 40 value 2.753033
## iter 50 value 2.182215
## iter 60 value 2.016873
## iter 70 value 1.906286
## iter 80 value 1.765621
## iter 90 value 1.588294
## iter 100 value 1.448619
## iter 110 value 1.254763
## iter 120 value 1.179623
## iter 130 value 1.109970
## iter 140 value 1.099364
## iter 150 value 1.092702
## iter 160 value 1.089777
## iter 170 value 1.086638
## iter 180 value 1.085667
## iter 190 value 1.085254
## iter 200 value 1.085146
## iter 210 value 1.085008
## iter 220 value 1.084997
## iter 230 value 1.084994
## iter 240 value 1.084992
## iter 250 value 1.084992
## iter 260 value 1.084991
## iter 260 value 1.084991
## iter 260 value 1.084991
## final value 1.084991
## converged#plotting our neutral network model
plotnet(model)
2) Using Keras
Keras is another package in r to create a neutral network models in r. Like nnet, I can use keras to create a neutral network to identify the correlation between multiple variables in a data set and training models with higher accuracy. Higher accurate model can contribute better detection in recommendations in social media or better ways to detect for diseases and cancers earlier. For my mini project, let’s do another neutral network model with the iris data set. As it is my first time in keras, I followed a walkthrough on the keras package found here
Looking at our results, we only see a accuracy from our model of on average 75-79% as the number of iterations exceed 100. In order to improve the accuracy, we could add on another layer to our model. However, we must be careful on adding too many layers, as it will cause our neutral network model to be overfitted.
#unlike nnet, keras needs a "dummy variable" for classification, so do we to separate the set to create our fake variables from the labels
#unlike nnet, we must transform the categorical value of the species into a numeric value to process the model in r's python
data<- iris
data[,5]<-as.numeric(data[,5])-1
data<-as.matrix(data)
#Using the same rnadom split in the previous section
train_set<-subset(data,split=="TRUE")
test_set<-subset(data,split=="FALSE")
#transform the dat frame into a matrix for both train/test
train_set<-as.matrix(train_set)
test_set<-as.matrix(test_set)
x_train<-train_set[,1:4]
x_test<-test_set[,1:4]
y_train<-train_set[,5]
y_test<-test_set[,5]
#create the dummy variable for the categorical data witht to_categorical() in keras
train_set.trainlbs<-to_categorical(y_train)
test_set.testlbs<-to_categorical(y_test)
#create the sequence model by layer, the hidden nodes will make of seven nodes as it takes on 4 independent variables, it suspects three classes of the dat
model <- keras_model_sequential()
model %>% layer_dense(units = 7, activation = 'relu', input_shape = c(4)) %>%layer_dense(units = 3,activation = 'softmax')
model %>% compile(loss = 'categorical_crossentropy',optimizer = 'adam',metrics = 'accuracy')
#fitting our model with of training data and plot the results
m<-model%>%fit(x_train,train_set.trainlbs,epochs=300,batch_size=3,validation_split = 0.2)
plot(m)