Implementing Radial Basis Function Classifier in R
Sandipan Dey
25 August 2016
In this article, an RBF (Radial Basis Function) Classifier will be used for binary classification.
- The algorithm consists of two different parts:
- Unsupervised: Find k << N centers (N = # data points) fro the RBF network.
- Supervised: Compute the weights w of the classifier hypothesis h(x) from the data using the labels.
k centroids will be used for the RBF network and those centroids will first be found with Lloyd’s algorithm.
Then parameter vector w will be computed by solving the least square equation which is linear in \(\phi\).
The following figure shows the outline of the algorithm to be used.
Number of centers k and the the \(\gamma\) parameters will be varied to obtain different decision boundaries for classification.
As can be seen from the experiments on a few 2 dimensional (labelled) datasets, \(\gamma\) works as regularization parameter for the classifier, at lower value of the parameter, the classifier in-sample accuracy is higher.
The next two animations show the decision boundary contours learnt by the RBF classifier as a dark blue line on the same dataset for different \(\gamma\) values. The centers of the RBF network is represented by the blue stars surrounded by the black circles. The first animation shows the results of the RBF classifier starting with 3 centers, while the second one starting with 5 centers.
- The next two animations show the decision boundary contours learnt by the RBF classifier as a solid black line on another two different datasets, for different \(\gamma\) values. The centers of the RBF network is represented by the black stars surrounded by the black circles. The first animation shows the results of the RBF classifier starting with 3 centers, while the second one starting with 5 centers.
