June 14 / 2017

Description of the Algorithm

The algorithm is based on data from a corpus called HC Corpora which has texts from twitter, blogs and news (click here to see data).

All numbers and punctuation marks were removed. Then the databases were tokenized and all text was changed to lowercase.

With the latter bi-grams, tri-grams and quad-grams were created and joined to have a complete set of words from the three databases.

The algorithm recieves an input text which is processed first as described before (numbers and puntuation marks are removed and the text was changed to lowercase).

Then this text is splitted in words that will be used to filter the data according to the last 3 words, depending the n-gram that will be used to find the next word.

First, the last three words will be used to check with the quad-grams which one is the most probable word that will match the text. If no possible words are available for this combination, then the last two words will be used to search with the tri-grams and finally if no words match with the tri-gram, the last word will be used with search on the bi-grams.

Description of the App

In the app you have to write a phrase in the "Text input" box (red box in Figure 1), and you will have the prediction and the top 3 of the possible words according to the algorithm (blue boxes in Figure 1).

Figure 1:

If the algorithm has no enough information the messagge that will be shown is "NO PREDICTION AVAILABLE" by default.

The algorithm predicts the next word for a text with one word to a whole phrase.

As can be seen from Figure 2, for "word", the next word predicted for the algorithm is "of" and the next possible options are "for" and "and".

Figure 2:

(Link to the app)