1. JHU/SwiftKey Natural Language Processing Project

• The goal of this project is to develop an app for predicting the next word in a series of words
• The database for the project consists of a sample of news articles, blog posts, and tweets
• There are 1,010,242 news articles (34,372,530 words); 899,288 blog posts (37,334,131 words); and 2,360,148 tweets (30,373,543 words) in the database
• Due to computational limitations, this project works with a subset of the data (the training data) that includes 56,654 unique words; 1,045,385 unique bi-grams; 1,307,613 unique tri-grams; 720,301 unique 4-grams; and 292,508 unique 5-grams

2. Procedure for Developing the App

• The app uses the Katz Back-Off (KBO) approach to weight results based on n-grams of different lengths
• The longest considered n-gram is 5 words long
• The discount rate is set to 0.8, so 80% weight is assigned to matching 5-grams (if any); 80% of the remaining weight is assigned to matching 4-grams (if any); and so forth down to matching bigrams
• If there are no matching bigrams, predictions are made based on the frequency of individual words in the training data
• This KBO model has an accuracy of approximately 12% evaluated using a set of validation data from all three sources
• The accuracy is low likely because this modeling approach requires seeing the specific combination of words to predict it correctly
• Using a larger set of training data would improve the model’s predictions

3. Features of the App (1/2)

• The function returns the most likely next word by default, but the user can also obtain a table of all words with weight greater than zero and their respective weights

mostLikely <- TRUE
returnTable <- FALSE
test <- predictWord("go to the",mostLikely,returnTable)
test

## [1] "gym"

• However, changing the options allows the user to obtain a randomly selected word instead of the most likely word

set.seed(12)
mostLikely <- FALSE
returnTable <- FALSE
test <- predictWord("go to the",mostLikely,returnTable)
test

## [1] "gym"

4. Features of the App (2/2)

• Finally, the user can also change the settings to obtain a table of all of the probabilities assigned to various words

returnTable <- TRUE
test <- predictWord("go to the",mostLikely,returnTable)
test

## # A tibble: 3,237 × 3
##    ngram               freq      p
##    <chr>              <int>  <dbl>
##  1 go to the gym         44 0.107 
##  2 go to the bathroom    41 0.0996
##  3 go to the beach       36 0.0874
##  4 go to the movies      36 0.0874
##  5 go to the store       27 0.0656
##  6 go to the hospital    25 0.0607
##  7 go to the mall        23 0.0559
##  8 go to the doctor      18 0.0437
##  9 go to the game        16 0.0389
## 10 go to the polls       13 0.0316
## # … with 3,227 more rows

5. Using the App

• To use the app, users input a string containing one or more words
• Using the default settings, the app returns the most likely next word

test <- predictWord("what the",mostLikely,returnTable)
test

## [1] "heck"

test <- predictWord("what can we",mostLikely,returnTable)
test

## [1] "do"

test <- predictWord("how can we",mostLikely,returnTable)
test

## [1] "be"