k-Nearest Neighbors (kNN)

Machine Learning in Marketing using kNN

---

Let's assume we have several groups of labeled samples. The items present in the groups are homogeneous in nature.

Now, suppose we have an unlabeled example which needs to be classified into one of the several labeled groups.

k nearest neighbors is a simple algorithm that stores all available / known cases (training data) and classifies new cases by a majority vote of its k neighbors.

A large k value reduces the variance due to the noisy data;

But it can create a bias due to which the learner tends to ignore the smaller patterns which may have useful insights.

Let's consider 10 'drinking items' which are rated on two parameters on a scale of 1 to 10.

The two parameters are “sweetness” and “fizziness”.

This is a perception based rating and so may vary between individuals.

Sample Ratings:

Calculate the distance between 'Maaza' and its nearest neighbors ('ACTIV', 'Vodka', 'Pepsi' and 'Monster') using the popular metric: Euclidean distance.

Using the co-ordinates of Maaza (8,2) and Vodka (2,1), the distance between 'Maaza' and 'Vodka' can be calculated as: dist(Maaza,Vodka) = 6.08

Using Euclidean distance, we can calculate the distance of Maaza from each of its nearest neighbors. Distance between Maaza and ACTIV being the least, it may be inferred that Maaza is same as ACTIV in nature which in turn belongs to the group of drinks (Health Drinks).

If k=1, the algorithm considers the nearest neighbor to Maaza i.e, ACTIV;

if k=3, the algorithm considers '3' nearest neighbors to Maaza by comparing the Euclidean distances (ACTIV, Real, Monster)