July 25, 2026

Overview & Data Architecture

Next Word Predictor — Shiny app that predicts the next word based on user input.

  • Model: N-grams (2 to 4) with Stupid Backoff

  • Corpus: HC Corpora — blogs, news and Twitter (English)

  • Sample: ~2% (~43k lines) due to Posit Cloud Free tier limits

  • Live app: CapstoneDaraScience

Data Architecture

├── en_US.blogs.txt    ─┐
├── en_US.news.txt      ├── sample(2%) ──→ clean ──→ n-grams
└── en_US.twitter.txt   ┘
                        ├── unigram.rds   (unique words)
                        ├── bigram.rds    (2-grams)
                        ├── trigram.rds   (3-grams)
                        └── fourgram.rds  (4-grams)
                        ▼
                    Shiny App
                    Input: any sentence
                    Output: top 5 next words
                    Method: Stupid Backoff (α=0.4)

Algorithm

Input: "I went to the"
                       ┌─────────────────────────────┐
4-gram level:          │ base = "went to the"         │  Score = freq / total
                       │ pred = {store, beach, gym}   │
                       └─────────────────────────────┘
                                    ↓ if not found
                       ┌─────────────────────────────┐
3-gram level:          │ base = "to the"              │  Score × α (0.4)
                       │ pred = {store, beach, ...}   │
                       └─────────────────────────────┘
                                    ↓ if not found
                       ┌─────────────────────────────┐
2-gram level:          │ base = "the"                 │  Score × α²
                       │ pred = {first, same, ...}    │
                       └─────────────────────────────┘
                                    ↓ if not found
                       ┌─────────────────────────────┐
1-gram level:          │ most frequent unigram        │  Score × α³
                       │ pred = {the, to, and, ...}   │
                       └─────────────────────────────┘

Installation & Usage

Upload

run_first.R  |   global.R   |   app.R

Setup

install.packages(c("shiny", "shinythemes", "data.table", "stringi", "quanteda", "dplyr"))

Train model

 source("run_first.R")

Run app

source("global.R")
predict_next_word("I went to the", 5)
shiny::runApp()

Quick tests

"on my" → way   |   "in quite some" → time    |   "at the end of the" → day

Limitations + Future Work & References

Limitations + Future Work

Current limitations:

  • Reduced sample (2%) due to Posit Cloud Free tier
  • Lower accuracy on rare phrase combinations
  • Stupid Backoff (no smoothing)

Future improvements:

  • Kneser-Ney smoothing for rare n-grams
  • Larger sample with cloud resources
  • Bidirectional LSTM / transformer model

References