Introduction

This presentation will briefly explains about the “Data Science Specialization Course” from Coursera’s final project. The project is about building a predictive modeling for word recommendations upon user prompt. The idea itself came from SwiftKey (now becomes Microsoft SwiftKey), a company that develops TouchType for Android and iOS devices.

Dataset & Preprocessing

The original SwiftKey Dataset consists of several languages including Deutch, French, Russian. However, in this project, I only used the English dataset only. The English dataset itself has three separate files from different sources: blogs, news, and Twitter. In each file, the data is separated by lines, this means that the dataset is in a form of collection of sentences.

Before feeding the data into the model, there are several preprocessing steps that need to be done:

• Removing punctuations a.k.a removing non-alphanumeric characters.
• Convert all words to lowercase
• Removing stopwords. Stopwords are words that usually do not give significant meaning in a sentence/paragraph (e.g. “a”, “the”, “is”, etc.)

Predictive Modelling (Back-off Algorithm)

To perform next word predictions, there are several predictive modelling algorithms. In this case, I utilized the Katz’s Back-off model. This is one of the most widely used algorithm in language modelling. Katz’s Back-off or Back-off is a generative n-gram language model which calculates the probability of a word given its history in the n-gram.

In this project I modelled the n-gram up from 2-gram up to 4-gram. The pipeline is as follows:

1. Predict the next word using 4-gram model (we will be starting predicting with the largest n-gram model first).
2. If the 4-gram model does not output a good prediction, move on to the 3-gram model.
3. Repeat the same steps as step (1) and (2)
4. If the 3-gram model does not output a good prediction, move on to the 2-gram model.
5. If it is still does not output a good prediction with the 2-gram model, recommend the word ‘the’.

The Web App

The image below is the display of my web application for this project. You will see a text box on the left side for the user prompt to type a sentence and the prediction result will appear on the right side.

Conclusion & Final Words

In this project we have seen how Back-off (n-gram) model can predict/recommend words from user input. We start off with a raw dataset, clean & preprocess it, and feed the corpus into the model. This is a common approach as it does not require sophisticated and deep understanding about the NLP topic. In addition, it is quite memory and time efficient.

Perhaps this project can be further improved by implementing other models like deep learning models for language modelling. Deep learning models can capture meaning and connections between words which the Back-off algorithm cannot do.