Prediction Word

Capstone Data Science Course Johns Hopkins

Luccas Betton

2023-10-10

Objective

The main goal for this project is to create an application for word prediction making easier for the user for typing and building texts, as we have in our smartphones or e-mails.

JHS_Image

Data

The database was made available by SwiftKey, our corporate partner in this capstone, from three different sources - Blogs, News and Twitter.

  • Data Sample - Initial data set with more than 4 Millions Lines / Sampled 30% for prediction model

  • Data Cleaning - Remove bad words, digits, and non characters

  • Data Processing - Build n-grams from two words up to five words / Around 5 Million grams

Predictive Model

The predictive uses Markov chain for evaluation of probabilities for the next word. The most frequent n-grams will be used to determine the next words based in the last words.

First, it will be checked the n-grams with 5 words, than 4 words, up to grams with two words.