mxnet

Introduction

This is my implementation of using mxnet for the problem of digit recognition.

Installation guide of mxnet for R is available.

Load data

require(mxnet)

## Loading required package: mxnet

## Init Rcpp

train <- read.csv("../data/train.csv", header=TRUE)
test <- read.csv("../data/test.csv", header=TRUE)
train <- data.matrix(train)
test <- data.matrix(test)

train.x <- train[,-1]
train.y <- train[,1]

We can normalize the input data (originally values range from 0 to 255 as grayscale to between 0 and 1).

train.x <- t(train.x/255)
test <- t(test/255)

Build a neural network

Deep Neural Network

Let’s try with a feed-forward neural network.

data <- mx.symbol.Variable("data")
fc1 <- mx.symbol.FullyConnected(data, name="fc1", num_hidden=128)
act1 <- mx.symbol.Activation(fc1, name="relu1", act_type="relu")
fc2 <- mx.symbol.FullyConnected(act1, name="fc2", num_hidden=64)
act2 <- mx.symbol.Activation(fc2, name="relu2", act_type="relu")
fc3 <- mx.symbol.FullyConnected(act2, name="fc3", num_hidden=10)
softmax <- mx.symbol.SoftmaxOutput(fc3, name="sm")

Training

devices <- mx.cpu()
mx.set.seed(0)
model <- mx.model.FeedForward.create(softmax, X=train.x, y=train.y,
                                     ctx=devices, num.round=10, array.batch.size=100,
                                     learning.rate=0.07, momentum=0.9,  eval.metric=mx.metric.accuracy,
                                     initializer=mx.init.uniform(0.07),
                                     batch.end.callback=mx.callback.log.train.metric(100))

## Warning in mx.model.select.layout.train(X, y): Auto detect layout input matrix, use colmajor..

## Start training with 1 devices
## Batch [100] Train-accuracy=0.6563
## Batch [200] Train-accuracy=0.777999999999999
## Batch [300] Train-accuracy=0.827466666666665
## Batch [400] Train-accuracy=0.855499999999999
## [1] Train-accuracy=0.859832935560859
## Batch [100] Train-accuracy=0.9529
## Batch [200] Train-accuracy=0.953049999999999
## Batch [300] Train-accuracy=0.955866666666666
## Batch [400] Train-accuracy=0.957525000000001
## [2] Train-accuracy=0.958309523809525
## Batch [100] Train-accuracy=0.968
## Batch [200] Train-accuracy=0.9677
## Batch [300] Train-accuracy=0.969366666666667
## Batch [400] Train-accuracy=0.970775000000003
## [3] Train-accuracy=0.971000000000004
## Batch [100] Train-accuracy=0.9734
## Batch [200] Train-accuracy=0.974699999999999
## Batch [300] Train-accuracy=0.976500000000001
## Batch [400] Train-accuracy=0.977200000000003
## [4] Train-accuracy=0.977547619047622
## Batch [100] Train-accuracy=0.978399999999999
## Batch [200] Train-accuracy=0.97925
## Batch [300] Train-accuracy=0.980400000000001
## Batch [400] Train-accuracy=0.981650000000003
## [5] Train-accuracy=0.981952380952384
## Batch [100] Train-accuracy=0.9849
## Batch [200] Train-accuracy=0.986
## Batch [300] Train-accuracy=0.986066666666668
## Batch [400] Train-accuracy=0.986225000000003
## [6] Train-accuracy=0.986404761904765
## Batch [100] Train-accuracy=0.987699999999999
## Batch [200] Train-accuracy=0.988349999999999
## Batch [300] Train-accuracy=0.988100000000001
## Batch [400] Train-accuracy=0.988125000000003
## [7] Train-accuracy=0.98816666666667
## Batch [100] Train-accuracy=0.988999999999999
## Batch [200] Train-accuracy=0.98905
## Batch [300] Train-accuracy=0.989466666666668
## Batch [400] Train-accuracy=0.989675000000003
## [8] Train-accuracy=0.989809523809527
## Batch [100] Train-accuracy=0.9914
## Batch [200] Train-accuracy=0.990600000000001
## Batch [300] Train-accuracy=0.990933333333335
## Batch [400] Train-accuracy=0.991100000000003
## [9] Train-accuracy=0.991095238095241
## Batch [100] Train-accuracy=0.9905
## Batch [200] Train-accuracy=0.9905
## Batch [300] Train-accuracy=0.990900000000001
## Batch [400] Train-accuracy=0.991400000000003
## [10] Train-accuracy=0.991500000000003

Prediction

preds <- predict(model, test)

## Warning in mx.model.select.layout.predict(X, model): Auto detect layout input matrix, use colmajor..

dim(preds)

## [1]    10 28000

pred.label <- max.col(t(preds)) - 1
table(pred.label)

## pred.label
##    0    1    2    3    4    5    6    7    8    9 
## 2847 3201 2742 2911 2683 2464 2721 2877 2769 2785

submission <- data.frame(ImageId=1:ncol(test), Label=pred.label)
write.csv(submission, file='submission_dnn.csv', row.names=FALSE, quote=FALSE)

It should not take more than 1 minute on my Macbook Pro Mid-2014, and the accuracy is 97%.