Capstone Project

• The goal of this project is to build a text app, which can predict the next word with given sentences

• The proposed app suggests six likely words

• The predictive model is a custom natural language processing (NLP) model, based upon the analysis of a large corpus data from the Capstone Dataset

• Three data corpus were employed (twitts, news and blogs data)

• 50000 lines of each corpus were randomly selected and put together

• The resulting corpus was cleaned and tokenized to create a termdocumentmatrix (tdm) for bigram, trigram, and fourgram

• The tdm was converted to a data frame for each gram type, containing the term names and their counts…

• The dataframes were saved in separate csv files and uploaded to the Shinyapps website: these files are loaded at the beginning of the shiny app inizialization

• The text in the input-Box and the prediction functions for bigram/trigram/fourgram are then employed to get a list of possible terms

• The app employs a Markov chain rule to predict the next word: the Bigram model uses the last word and search through the bigram dataframe to find the most likely term; the Trigram model uses the last two words, while the Four-gram uses the last three words.

Just follow these simple steps:

Step 1: Type a sentence in the box.

Step 2: Press the Submit button.

Step 3: See the predicted words, from the most likely to the least likely.

Step 4: you can choose among three N-gram prediction methods (Bigram/Trigram/Fourgram).

• The prediction app is hosted on shinyapps.io

• Wait for the app to initialize: it will take a couple of minutes

• the project presentation can be find at rpubs

• Future development: extend the app to include the Katz back off model without increasing (significantly) the computational time required.