Pandora: The Text Prediction Application

Overview

• This presentation illustrateds the word prediction Shiny app, Pandora.
• The Shiny application predicts the most probable word following a partial sentence entered by Users
• This application is developed for the purpose of completing the Capstone Project for the Coursera Data Science course.
• The Application is hosted on the shinyapp https://diptamdutta.shinyapps.io/Drozen_Predict/

The Algorithm & Data Used

• The complete text data for buidling text model is downloaded from https://d396qusza40orc.cloudfront.net/dsscapstone/dataset/Coursera-SwiftKey.zip
• The zip file contains four folders, in which the one names en_US is used
• The en_US file folder contains three text input files. For complete analysis of the dataset please refer to https://rpubs.com/diptamdutta/capstoneWeek2
• 1% of the total text data from Corpora was used
• Text was cleaned by converting all letters to lowercase, removing punctuation and numbers, and stripping whitespace.
• A table of the most frequent n-grams were created.
• The next word is predicted using an N-Gram model,smoothing method known as dubbed Stupid Backof, which was coined by Thorsten Brants in his research article Large Language Models in Machine Translation. It can be find in this link http://www.aclweb.org/anthology/D07-1090.pdf
• The model checks for the highest order n-gram to predict the next word, and outputs the resultant

Application Runthrough

• The above screenshot is from the application where I’ve entered one sentence " I’d give anything to see arctic monkeys this " and the application returns the output.
• You can also control the number of prediction by increding and decreasing the input ‘Number of words to predict’

Conclusion

• The application itself is fast in nature as due to the implementation of the model dubbed stupid backof.

• Future Improvement Plans

– Learn from user input to predict better prediction.

– Use more and better dataset to predict better.