Predicting the next word

Predictive keyboards suggest words based on the text a user has already entered.

Objective: Build an interactive next-word prediction system using statistical language modelling.

Project workflow

• Analyse real-world English text.
• Identify recurring word sequences.
• Estimate the next-word probability.
• Deliver predictions through a Shiny application.

Can patterns in everyday language help predict what a user types next?

Learning from real-world language

Dataset: English-language SwiftKey corpus.

Three sources: Blogs (200.4 MB) · News (196.3 MB) · Twitter (159.4 MB)

Exploratory analysis - Examined dataset size and structure. - Analysed word and character distributions. - Cleaned and tokenised text.

N-gram language model

• Unigram: Individual word frequencies.
• Bigram: Two consecutive words.
• Trigram: Three consecutive words.

Backoff: Trigram → Bigram → Unigram. The model falls back to shorter sequences when longer patterns are unavailable.

Evaluating the model

<h2>8.11%</h2>
<p>Top-1 accuracy</p>

<h2>14.01%</h2>
<p>Top-3 accuracy</p>

Held-out test results

• Top-1: First prediction matches the actual next word.
• Top-3: Actual next word appears among three suggestions.

Limitation: Frequency-based models have limited semantic understanding and may struggle with unseen phrases.

An interactive prediction tool

The trained model powers a Shiny application.

User workflow

1. Enter a phrase.
2. Click Predict next word.
3. The model analyses the context.
4. A suggested word is displayed.

Future scope: Larger training datasets, better handling of unseen phrases and more advanced language models.

A practical application of natural language processing and statistical language modelling.