Next Word Prediction using Natural Language Processing

Author

Jhansi Jhaveri

Overview

This project is the final capstone for the Johns Hopkins Data Science Specialization.
The goal is to build an interactive data product using real-world text data that predicts the next word in a given phrase.
It simulates how predictive keyboards (like those in smartphones) work using NLP techniques.

Data Source

The model is trained using the SwiftKey dataset, which includes: * Blogs * News articles * Twitter posts (English)

Preprocessing:

  • Removed profanity, punctuation, numbers, and symbols
  • Lowercased all text
  • Tokenized into unigrams, bigrams, and trigrams

Methodology

Data Cleaning:
Handled in clean_text.R using regex, tm, and stringr packages.

N-gram Modeling:
Implemented in ngram_model.R using the quanteda package.

Prediction Logic (in predict_next.R): * Trigram match
* Back-off to bigram
* If no match, return most common unigram

The Shiny App

  • Built using app.R in RStudio
  • User inputs a phrase → App predicts the most likely next word
  • Fast and minimal UI
  • Hosted on shinyapps.io

Key Skills Demonstrated

  • Text preprocessing and tokenization
  • Exploratory data analysis
  • Language modeling with n-grams
  • Building interactive Shiny apps
  • Communicating results via R Presentations