Overview

This report presents a analysis of the SwiftKey Dataset provided by Johns Hopkins and Coursera.

1. Data Summary

Dataset Summary
Source Lines Size_MB
Blogs 899288 200.4
News 1010242 196.3
Twitter 2360148 159.4

The dataset is large — over 4 million lines and 556 MB of raw text. To keep analysis efficient, a random sample of 10,000 lines per source was used throughout this report.

2. Average Words Per Line

Blogs contain the longest text averaging around 42 words per line, while Twitter is shortest at around 12 words due to character limits. This difference will influence how the prediction model handles short vs long context.

3. Most Common Words

The most frequent words — time, people, day, love, life — reflect the personal and conversational nature of the data sources.

4. Bigrams (Two-Word Pairs)

5. Trigrams (Three-Word Sequences)

Trigrams like “one of the” and “a lot of” reveal natural English phrasing patterns that the prediction model will learn from.

6. Plan for Prediction Algorithm

The next word prediction model will work like:

  1. N-gram lookup — When a usir types a word or phrase, the system will compare it with bigrams and trigrams and predict the next word.
  2. Backoff strategy — If no trigram found, it will go back to bigram then unigram (most common word)
  3. Training — For better acuracy the model need to be trained on larger dataset.
  4. Cleaning — filtering and punctuation removal will be applied

7. Plan for Shiny App

The Shiny app will:

Conclusion

The analysis confirms the data is rich, diverse, and suitable for building a next word prediction. findings include significant differences in text length across sources, and clear patterns in word and phrase frequency that will directly inform the prediction algorithm.