Exploratory Analysis of the SwiftKey Dataset
Shivam Kumar
November 9, 2026
1. Introduction
The objective of this project is to explore the SwiftKey text dataset and understand patterns in written language.
The analysis covers three English-language data sources: blogs, news articles, and Twitter messages. These datasets will be used to develop a predictive text algorithm that can suggest the next word while a user types.
2. Dataset Description
The analysis uses three files from the English SwiftKey dataset:
- Blogs: General written content from blog posts.
- News: Text from news articles.
- Twitter: Short messages posted on Twitter.
3. Basic Data Summary
The following analysis calculates file size, number of lines, total words, and average words per line.
The files are processed in smaller chunks to reduce memory usage.
| Dataset | File_Size_MB | Lines | Words | Average_Words_Per_Line | |
|---|---|---|---|---|---|
| Blogs | Blogs | 200.42 | 899288 | 37334131 | 41.52 |
| News | News | 196.28 | 1010242 | 34372530 | 34.02 |
| 159.36 | 2360148 | 30373543 | 12.87 |
4. Dataset Size Comparison
This chart compares the storage size of the three text files.
5. Exploratory Analysis of Text Length
The number of words in each line helps us understand the different writing styles in the datasets.
6. Key Findings
The largest dataset by file size is Blogs.
The dataset containing the highest number of words is Blogs.
The three sources provide different types of written language. Blogs and news articles generally contain longer text, while Twitter messages provide shorter and more conversational text.
7. Prediction Algorithm Plan
The next stage will focus on developing a predictive text algorithm.
The planned approach includes:
- Cleaning the text by handling punctuation, numbers, and unwanted symbols.
- Converting text into lowercase.
- Tokenizing the text into individual words.
- Creating unigram, bigram, and trigram frequency tables.
- Using n-gram probabilities to predict the next word.
- Testing the algorithm with sample sentences.
The model will aim to provide useful next-word suggestions while maintaining reasonable prediction speed.
8. Shiny Application Plan
A Shiny application will be developed to allow users to enter a phrase and receive predicted next-word suggestions.
The application will include:
- A simple text input field.
- A prediction button.
- A display area for suggested words.
- A clear and user-friendly interface.
The goal is to make the prediction system easy to use for people without technical knowledge.
9. Conclusion
The three English SwiftKey datasets have been loaded for exploratory analysis. Their file sizes, line counts, word counts, and text-length distributions are examined in this report.
This initial analysis provides a foundation for cleaning the text, building a predictive algorithm, and developing an interactive Shiny application.
The next phase will focus on model development, prediction accuracy, and application testing.