Data Science Capstone Presentation

This presentation describes the Swiftkey capstone project which is part of John Hopkins/ Coursera Data Science Specialization Program.

• Create a corpus of data from twitter, news and blogs
• Create a prediction algorithm
• Build a Shiny app that takes as input phrase(multiple words) in a text box and outputs a prediction of the next word
• A slide deck describing the app and the algorithm to pitch to investors and future customers

This project was broken into multiple steps to accomplish the final task of predicting the next word. We briefly describe the major tasks

• Understanding the problem
• Acquiring and Exploring data
• Data Preprocessing and Preparation
• Creating Prediction models
• Testing model accurary
• Build a data product
• Create a Slide deck presentation

We took a sample of cleaned data (removing punctuation, lowercase, removing sparse terms) from corpora (twitter, blogs and news) and merge them. We used 10 % of data from each source before building the prediction model. We applied NLP methods such as

• Tokenization (https://en.wikipedia.org/wiki/N-gram)
• Extracting unigrams, bigrams and trigrams using Rweka packages (https://cran.r-project.org/web/packages/RWeka/RWeka.pdf)
• Storing these ngrams into data frames with the frequency counts

Link to Shiny app https://medh.shinyapps.io/CapstoneApp/

How to run the app

• A user enter a word or a phrase in a text box
• App will predicts the next word Limitations
• The app is using on sample of the dataset due to memory and processing time.