3/21/2021

Introduction

In this presentation I will describe some of the features of a shiny app that I developed as part of the Coursera data science Capstone

During the course a large data set of texts was provided in order to use it as a training set to develop a predictive text App. The idea was to create a simple app that could predict the next word on a phrase after the first words were provided.

The App was developed in light of some of the issues and the data processing techniques (like text mining) that a company like Swiftly uses when developing responsive keyboards for their users.

Data processing

The text data set was provided by Swifty under the following link: https://d396qusza40orc.cloudfront.net/dsscapstone/dataset/Coursera-SwiftKey.zip. It consisted of a number or text files in different languages from wich only three where used for this exercise: Blogs_file, News_file,Twitter_file

Given the huge size of the data a random sample (0.10) was selected cleaning and tokenization process.

The corpus was cleaned mainly through the tm R package. The cleaning process included removing punctuation, turning all letters into lower case, removing special characters, numbers and stopwords. N-grams work as the tokens in this exercise.

Algorithm

Frequency dictionary was created by classifying the data into n grams. Such n grams range from 2 to 4 words and were stored as tibbles

The algorithm works with this information (which was obtained from a sampe of texts) and runs in a sequential manner. It looks at the input of the user and it considers the first quadram to make the prediction.

If the algorithm cannot come up with any match with the quadrams it then looks at the three grams and finally it considers the bigrams. If the algorithm does not come up with any match an error message is displayed for the user.

App explained

Frequency dictionary was created by classifying the data into n grams. Such n grams range from 2 to 4 words and were stored as tibbles

The algorithm works with this information (which was obtained from a sampe of texts) and runs in a sequential manner. It looks at the input of the user and it considers the first quadram to make the prediction. If the algorithm cannot come up with any match with the quadrams it then looks at the three grams and finally it considers the bigrams. If the algorithm does not come up with any match an error message is displayed for the user.

The following url provides access to the App https://santiagocardenas.shinyapps.io/Predictive_App_Coursera_Capstone/