SwiftKey Next Word Prediction: Exploratory Analysis
Srishti Pandey
2026-11-01
1. Introduction
The purpose of this project is to develop a next-word prediction application using the English text data provided in the SwiftKey dataset.
The project uses three text sources: blogs, news and Twitter. These sources contain different writing styles and help the prediction model learn common word patterns.
The final goal is to build an application that accepts a phrase and suggests possible next words.
2. Data Sources
The dataset contains three English text files:
- Blogs: Informal and personal writing.
- News: Formal and informative articles.
- Twitter: Short and conversational messages.
These sources provide a broad range of language patterns for training a prediction model.
3. Exploratory Data Analysis
The following table presents the basic statistics of the three text files.
| Source | Size_MB | Lines | Words |
|---|---|---|---|
| Blogs | 200.42 | 899288 | 37546806 |
| News | 196.28 | 1010206 | 34761151 |
| 159.36 | 2360148 | 30096649 |
3.1 File Size Comparison
3.2 Line Count Comparison
3.3 Word Count Comparison
4. Key Observations
The exploratory analysis provides the following observations:
- The blogs file contains a large amount of written text.
- The news file contains more than one million lines.
- The Twitter file contains the highest number of lines among the three sources.
- All three sources provide useful language patterns.
- The different writing styles help the model learn from both formal and informal text.
5. Prediction Algorithm
The project uses an n-gram-based prediction approach.
An n-gram is a sequence of consecutive words. The model studies word sequences from the training data and estimates which word is most likely to appear next.
The prediction process includes:
- Cleaning and preparing the text.
- Tokenising the text into word sequences.
- Building n-gram frequency information.
- Matching the user’s input phrase with learned patterns.
- Returning the most frequent next-word suggestions.
The model is designed to return up to five suggestions.
6. Shiny Application Plan
The final application is built using Shiny.
The user enters a phrase in a text box and selects the number of suggestions. After clicking the prediction button, the application displays possible next words with their frequencies.
The interface is designed to be simple and easy to understand.
7. Conclusion
The exploratory analysis provides an initial understanding of the SwiftKey English text dataset.
Blogs, news and Twitter contain different language patterns. These patterns form the foundation for building an n-gram-based next-word prediction model.
The final application demonstrates how text data can be processed and used to support predictive text suggestions.