2023-03-27

The Project

This project involves Natural Language Processing. The critical task is to take a user’s input phrase (group of words) and to output a predicted next word.

Project Deliverable

  • Next Word Prediction Model, as basis for an app
  • Next Word Prediction App hosted at shinyapps.io
  • This presentation hosted at R pubs

Next Word Prediction Model

The next word prediction model uses the principles of “tidy data” applied to text mining in R. Key model steps:

  1. Input: raw text files for model training
  2. Clean training data; separate into 2 word, 3 word, and 4 word n grams, save as tibbles
  3. Sort n grams tibbles by frequency
  4. N grams function: - user supplies an input phrase - model uses last 3, 2, or 1 words to predict the best 4th, 3rd, or 2nd match
  5. Output: next word prediction

Next Word Prediction App

The next word prediction app provides a simple user interface to the next word prediction model.

Key Features:

  1. Text box for user input
  2. Predicted next word outputs dynamically below user input
  3. Side panel with user instructions

Key Benefits:

  1. Fast response
  2. Method allows for large training sets leading to better next word predictions

Shiny App Link