Introduction

The goal of this project is to build a predictive text application that suggests the next word a user may type.

The project uses a large corpus of English text data from three sources:

  • Blogs
  • News articles
  • Twitter messages

The objective of this report is to describe the exploratory analysis performed on the dataset and summarize the approach used to develop the prediction algorithm and Shiny application.

Data Loading

The three text files were loaded into R:

  • en_US.blogs.txt
  • en_US.news.txt
  • en_US.twitter.txt
library(knitr)
library(stringi)
library(ggplot2)
library(dplyr)

load("data/raw_data.RData")

Dataset Summary

The following table shows the size of each dataset.

data_summary <- data.frame(

  File = c(
    "Blogs",
    "News",
    "Twitter"
  ),

  Lines = c(
    length(blogs),
    length(news),
    length(twitter)
  ),

  Words = c(
    sum(stringi::stri_count_words(blogs)),
    sum(stringi::stri_count_words(news)),
    sum(stringi::stri_count_words(twitter))
  )

)

kable(
  data_summary,
  caption = "Summary of Training Data"
)
Summary of Training Data
File Lines Words
Blogs 899288 37546806
News 1010206 34761151
Twitter 2360148 30096690

The dataset contains more than four million lines of English text.

Blogs provide longer informal writing, news provides formal language, and Twitter provides shorter conversational text.

File Size Analysis

The file sizes were calculated to understand the scale of the dataset.

file_size <- data.frame(

  File = c(
    "Blogs",
    "News",
    "Twitter"
  ),

  Size_MB = c(

    file.info(
      "data/en_US/en_US.blogs.txt"
    )$size / 1024^2,

    file.info(
      "data/en_US/en_US.news.txt"
    )$size / 1024^2,

    file.info(
      "data/en_US/en_US.twitter.txt"
    )$size / 1024^2

  )

)


kable(
  file_size,
  digits = 2,
  caption = "Text File Sizes in MB"
)
Text File Sizes in MB
File Size_MB
Blogs 200.42
News 196.28
Twitter 159.36

Exploratory Analysis

Word Frequency

Word frequency analysis was performed to understand common language patterns.

The most frequent words are usually short functional words such as articles, pronouns, and conjunctions. These words are important for predicting natural language patterns.

Top Word Frequency Plot

The following plot shows the most common words identified during exploratory analysis.

Word Cloud

A word cloud was generated to visualize frequently occurring words.

Text Cleaning and Preparation

Before creating the prediction model, the text data was cleaned.

The following preprocessing steps were performed:

  • Converted text to lowercase
  • Removed URLs
  • Removed email addresses
  • Removed punctuation
  • Removed numbers
  • Removed extra whitespace
  • Converted sentences into individual word tokens

A 5% sample of the complete dataset was used for model development to balance memory usage and processing speed.

The cleaned dataset contained approximately:

  • 4.8 million word tokens
  • 112,901 unique words

Prediction Algorithm

The prediction model uses an n-gram language model.

An n-gram model predicts the next word using previous words as context.

Unigram Model

The unigram model uses individual word frequencies.

Example:

the
and
you

This model is used when no previous word pattern is available.

Bigram Model

The bigram model predicts the next word using one previous word.

Example:

thank you → for

Trigram Model

The trigram model uses two previous words.

Example:

going to → be

Four-gram Model

The four-gram model uses three previous words.

Example:

I want to → be

Handling Unknown Word Combinations

Some word combinations may not exist in the training data.

To handle unseen combinations, a backoff strategy was implemented.

The model searches in this order:

Four-gram
     |
     ↓
Trigram
     |
     ↓
Bigram
     |
     ↓
Unigram

If a prediction is unavailable in a more complex model, the system falls back to a simpler model.

This allows predictions even for new input phrases.

Model Statistics

The final model contained:

model_summary <- data.frame(

Model = c(
"Unique Words",
"Bigrams",
"Trigrams",
"Four-grams"
),

Count = c(
112901,
393523,
333563,
267638
)

)

kable(
model_summary,
caption="N-gram Model Size"
)
N-gram Model Size
Model Count
Unique Words 112901
Bigrams 393523
Trigrams 333563
Four-grams 267638

The model was optimized by storing only useful word combinations and keeping the most frequent predictions.

Shiny Application

The predictive text model was deployed as a Shiny application.

The application allows users to:

  • Enter a partial sentence
  • Generate possible next words
  • Interact with the prediction model in real time

The application can be accessed here:

Predictive Text Application

Conclusion

This project demonstrates the complete workflow for building a predictive text application.

The process included:

  • Loading and exploring large text datasets
  • Performing text cleaning and preparation
  • Building n-gram language models
  • Implementing a backoff prediction strategy
  • Optimizing the model for faster predictions
  • Deploying the model using Shiny

The final application provides a simple demonstration of predictive text generation using statistical language modelling.