The code is extremely simple, it just:
Reads the provided training .txt files
Samples the files up to 1%
Converts the files and preprocess them using the tm package
Saves the function to local to be loaded by the server.R
2023-07-01
The code is extremely simple, it just:
Reads the provided training .txt files
Samples the files up to 1%
Converts the files and preprocess them using the tm package
Saves the function to local to be loaded by the server.R
library(tm)
library(NLP)
library(RWeka)
library(plotly)
library(dplyr)
library(stringr)
library(quanteda)
library(data.table)
initialPrediction <- readRDS("./data/start-word-prediction.RData")
freq2ngram <- readRDS("./data/bigram.RData")
freq3ngram <- readRDS("./data/trigram.RData")
freq4ngram <- readRDS("./data/quadgram.RData")
}
preprocess <- function(text){
toSpace <- content_transformer(function(x, pattern) gsub(pattern, " ", x))
text <- tm_map(text, toSpace, "/|@|//|$|:|:)|*|&|!|?|_|-|#|")
text <- tm_map(text, removeNumbers)
text <- tm_map(text, content_transformer(tolower))
text <- tm_map(text, removePunctuation)
text <- tm_map(text, stemDocument)
text <- tm_map(text, stripWhitespace)
text
}
saveRDS(getCorpus(),'corpus.RData')
The code can be found in my GitHub page: https://github.com/AndresBioMed/DataScience_Coursera_Repo/tree/0b9c08805151a5012efd7ea17fa010917c0c9455/Final_Capstone
The application is in: https://andresgordoortiz.shinyapps.io/DataScience_Coursera_Repo/