Training Backpropagation Neural Nets on the handwritten digits dataset

1. There are 5000 examples of handwritten digits in the dataset. Some of the examples are shown below. We have 10 class labels here: the digits 0:9. Given an image we want to classify it as one of the 10 classes.

2. The next set of equations show how the feedforward and backpropagation learning happens in a neural net with a single hidden layer.

3. The next figure shows how the cost function decreases with #iterations of the numerical optimization algorithm used (cojugate gradient).

4. The next figures show the hidden features learnt for different number of hidden units in the neural net.

5. The next figure shows the accuracy on the training dataset obtained with different number of hidden units in the neural net.