Swift Prompt

eulerwang
3/14/2017

The Process

To get the Swift Prompt work , There four necessary steps:

clean the coprus ,remove the punct,digit, http url ,stopwords
tokenize the corpus by ngrams with parameter 2
save the bigrams data and load data on shiny app
run the shiny app , filter the bigrams data by the input

the key Code

#after sample the data , tidy the data
tokens <-quanteda::tokenize(toLower(dataset), removeNumbers = TRUE, removePunct = TRUE, removeSeparators = TRUE, removeTwitter = TRUE, removeHyphens = TRUE ,stem=TRUE)

# ngrams with parameter 2 , and then transform the result to dataframe
 bigrams <- data.table(as.data.frame(table(unlist(quanteda::ngrams(tokens, n=2)))))

#for the filtering operation ,separate the words 
 colnames(bigrams) <- c('bigram','n')

 bigrams <- separate(bigrams,bigram, c("word1","word2"),sep="_")

the shiny app input

you can input a phrase , the shiny server will split your phrase to words , then get the last word , the last word will filter the bigrams data , get the next word distribution

the shiny app output

the display will show your word prompt by bar graph and star graph . in addtion , the app will always show the bar graph and star graph on top 30 bigrams data .