Introduction

This demonstration will walk you through a perceptron classification problem. The objective is to learn about some basic neural network terminology and to follow through an example.

Neural Networts are inspired by biological neural networks is a collection of artificial neurons to perform specific tasks like clustering, classification and pattern recognition. Acquires knowledge through learning and retains this knowledge in inter-neuron connections. Requires a set of inputs and produces output signals.

This picture shows the inputs, weights, bias and activation function. For the purpose of this example, the activation function is linearly seperable.

First, we set up a function in R to visualize graphs that will compare the desired output to what the neural net is classifying. The objective of the classification is to create a model to distinguise between circles and squares.

Initial Setup

Now we can set up the input matrix (co-ordinates) and classification for training. 1 indicates a square, and -1 indicates a circle. We will also set up a learning rate for the neural net and visualize the setup.

##setting up initial matrix (x,y,classification)
matrixX <- matrix(c(1,1,1,
                    2,-2,-1,
                    -1,-1.5,-1,
                    -2,-1, -1,
                    -2,1, 1,
                    1.5,-0.5,1), ncol = 3, byrow = T)

##Inputs Outputs and predictions
#m is a matrix of zeros, this is where the predictions will come
m<-mat.or.vec(6, 1)

#combining the initial matrix and predictions
matrixX <- cbind(matrixX,m)
w <- matrix(c(0,0,0), ncol = 1)

##set learning rate
learningRate <- 0.02

GraphIT()

Iterative Plots and all output

Next, we create a random start vector which will estabish a plane for classification. Anything to the right of the plane will potentially be classified as a square and to the left a circle.

##Assign Random starting point
##Weights and Bias
w <- matrix(c(0,1,0.5), ncol = 1)

GraphIT()

This next set of code iterates through the classification, observation by observation until it is solved.

#Observation count, cycle through all observations

for (i in 1:nrow(matrixX)) {
  
  #x are the inputs, for that observation, the bias input is 1
  x <- c(1,matrixX[i,1],matrixX[i,2])
  
  #checks if the initial prediction is correct
  matrixX[i,4] <- ifelse(t(w)%*%x>0,1,-1)
  
  #Goes into repeat loop if not until prediction is correct and then next observation
  repeat{
    print(w)
    if(matrixX[i,4] == matrixX[i,3]) {break}
    w[1,] <- w[1,] +matrixX[i,3]*learningRate*1
    w[2,] <- w[2,] +matrixX[i,3]*learningRate*matrixX[i,1]
    w[3,] <- w[3,] +matrixX[i,3]*learningRate*matrixX[i,2]
    matrixX[i,4] <- ifelse(t(w)%*%x>0,1,-1)
  }
  
  Sys.sleep(2)
  GraphIT()
  
}
##      [,1]
## [1,]  0.0
## [2,]  1.0
## [3,]  0.5

##      [,1]
## [1,]  0.0
## [2,]  1.0
## [3,]  0.5
##       [,1]
## [1,] -0.02
## [2,]  0.96
## [3,]  0.54
##       [,1]
## [1,] -0.04
## [2,]  0.92
## [3,]  0.58
##       [,1]
## [1,] -0.06
## [2,]  0.88
## [3,]  0.62
##       [,1]
## [1,] -0.08
## [2,]  0.84
## [3,]  0.66
##      [,1]
## [1,] -0.1
## [2,]  0.8
## [3,]  0.7
##       [,1]
## [1,] -0.12
## [2,]  0.76
## [3,]  0.74

##       [,1]
## [1,] -0.12
## [2,]  0.76
## [3,]  0.74

##       [,1]
## [1,] -0.12
## [2,]  0.76
## [3,]  0.74

##       [,1]
## [1,] -0.12
## [2,]  0.76
## [3,]  0.74
##       [,1]
## [1,] -0.10
## [2,]  0.72
## [3,]  0.76
##       [,1]
## [1,] -0.08
## [2,]  0.68
## [3,]  0.78
##       [,1]
## [1,] -0.06
## [2,]  0.64
## [3,]  0.80
##       [,1]
## [1,] -0.04
## [2,]  0.60
## [3,]  0.82
##       [,1]
## [1,] -0.02
## [2,]  0.56
## [3,]  0.84
##              [,1]
## [1,] 6.938894e-18
## [2,] 5.200000e-01
## [3,] 8.600000e-01
##      [,1]
## [1,] 0.02
## [2,] 0.48
## [3,] 0.88
##      [,1]
## [1,] 0.04
## [2,] 0.44
## [3,] 0.90

##      [,1]
## [1,] 0.04
## [2,] 0.44
## [3,] 0.90

The above plots show all iterations the neural network went through until all observations were classified. At this point, the last vector is the trained model’s bias and weight that allows for the correct classification of the squares and circles.

Closing Remarks

Neural networks learn more through each iteration and change weights and bias accordingly. A very low learning rate will become more computationally intensive additional training of the model may be required as more data is gathered.

Hope you enjoy the quick lesson on neural nets.