2023-07-01

Quick explanation

The code is extremely simple, it just:

  1. Reads the provided training .txt files

  2. Samples the files up to 1%

  3. Converts the files and preprocess them using the tm package

  4. Saves the function to local to be loaded by the server.R

Code

library(tm)
library(NLP)
library(RWeka)
library(plotly)
library(dplyr)
library(stringr)
library(quanteda)
library(data.table)
initialPrediction <- readRDS("./data/start-word-prediction.RData")
freq2ngram <- readRDS("./data/bigram.RData")
freq3ngram <- readRDS("./data/trigram.RData")
freq4ngram <- readRDS("./data/quadgram.RData")
}

preprocess <- function(text){
  toSpace <- content_transformer(function(x, pattern) gsub(pattern, " ", x))
  text <- tm_map(text, toSpace, "/|@|//|$|:|:)|*|&|!|?|_|-|#|")
  text <- tm_map(text, removeNumbers)
  text <- tm_map(text, content_transformer(tolower))
  text <- tm_map(text, removePunctuation)
  text <- tm_map(text, stemDocument)
  text <- tm_map(text, stripWhitespace)
  text
}

saveRDS(getCorpus(),'corpus.RData')

How to use it

  • Use the Slider to select the proper N-gram
  • Write your sentence to be completed
  • You are done!! The next words will be predicted. I hope you enjoyed it

References