Text Classification and Sentiment Analysis

Note: Some of the code has been omitted to enhance readability.

The following terms might be used interchangeably with each other:

         Rows = Observations = Records
         Columns = Variables = Features

Purpose and Objectives:

The purpose of this project is to showcase my knowledge in Data Analytics, focusing on the application of machine learning techniques for text classification. My objective is to attempt predicting the sentiment —positive, neutral, or negative— of individuals based on their Twitter messages in regards to COVID-19, using those words that get repeated 100 times or more. The model that will be used here is the Multinomial Naive Bayes.

Data Collection

• The data was obtained from https://www.kaggle.com/datasets/datatattle/covid-19-nlp-text-classification?resource=download on January/26/2024.
• Two files were collected with the names: “Corona_NLP_train.csv” and “Corona_NLP_test.csv”
• The files are in the comma-separated values (csv) format.

Loading the data:

training <- read.csv(file = "Corona_NLP_train.csv", header = TRUE, sep = ",", stringsAsFactors = FALSE, strip.white = TRUE, encoding = "latin1")

#encoding = "UTF-8"
testing <- read.csv(file = "Corona_NLP_test.csv", header = TRUE, sep = ",", stringsAsFactors = FALSE, strip.white = TRUE, encoding = "latin1")

Data Exploration

Training data set
Displaying the structure of the Training data set:

'data.frame':   41157 obs. of  6 variables:
 $ UserName     : int  3799 3800 3801 3802 3803 3804 3805 3806 3807 3808 ...
 $ ScreenName   : int  48751 48752 48753 48754 48755 48756 48757 48758 48759 48760 ...
 $ Location     : chr  "London" "UK" "Vagabonds" "" ...
 $ TweetAt      : chr  "16-03-2020" "16-03-2020" "16-03-2020" "16-03-2020" ...
 $ OriginalTweet: chr  "@MeNyrbie @Phil_Gahan @Chrisitv https://t.co/iFz9FAn2Pa and https://t.co/xX6ghGFzCC and https://t.co/I2NlzdxNo8" "advice Talk to your neighbours family to exchange phone numbers create contact list with phone numbers of neigh"| __truncated__ "Coronavirus Australia: Woolworths to give elderly, disabled dedicated shopping hours amid COVID-19 outbreak htt"| __truncated__ "My food stock is not the only one which is empty...\n\n\n\n\n\nPLEASE, don't panic, THERE WILL BE ENOUGH FOOD F"| __truncated__ ...
 $ Sentiment    : chr  "Neutral" "Positive" "Positive" "Positive" ...

Identifying missing (NAs) values in the Training data set:

     UserName    ScreenName      Location       TweetAt OriginalTweet     Sentiment 
            0             0             0             0             0             0 

Based on the frequency table, there are no missing values (NAs) in the Training data frame.

Displaying the content of the first two observations of the OriginalTweet variable within the Training data set:

[1] "@MeNyrbie @Phil_Gahan @Chrisitv https://t.co/iFz9FAn2Pa and https://t.co/xX6ghGFzCC and https://t.co/I2NlzdxNo8"

[1] "advice Talk to your neighbours family to exchange phone numbers create contact list with phone numbers of neighbours schools employer chemist GP set up online shopping accounts if poss adequate supplies of regular meds but not over order"

Displaying the frequency and the distribution of the Sentiment variable within the Training data set:

Extremely Negative Extremely Positive           Negative            Neutral           Positive 
              5481               6624               9917               7713              11422 

Testing data set
Displaying the structure of the Testing data set:

'data.frame':   3798 obs. of  6 variables:
 $ UserName     : int  1 2 3 4 5 6 7 8 9 10 ...
 $ ScreenName   : int  44953 44954 44955 44956 44957 44958 44959 44960 44961 44962 ...
 $ Location     : chr  "NYC" "Seattle, WA" "" "Chicagoland" ...
 $ TweetAt      : chr  "02-03-2020" "02-03-2020" "02-03-2020" "02-03-2020" ...
 $ OriginalTweet: chr  "TRENDING: New Yorkers encounter empty supermarket shelves (pictured, Wegmans in Brooklyn), sold-out online groc"| __truncated__ "When I couldn't find hand sanitizer at Fred Meyer, I turned to #Amazon. But $114.97 for a 2 pack of Purell??!!C"| __truncated__ "Find out how you can protect yourself and loved ones from #coronavirus. ?" "#Panic buying hits #NewYork City as anxious shoppers stock up on food&medical supplies after #healthcare wo"| __truncated__ ...
 $ Sentiment    : chr  "Extremely Negative" "Positive" "Extremely Positive" "Negative" ...

Identifying missing (NAs) values in the Testing data set:

     UserName    ScreenName      Location       TweetAt OriginalTweet     Sentiment 
            0             0             0             0             0             0 

Based on the frequency table, there are also no missing values (NAs) in the Testing data frame.

Displaying the content of the first two observations of the OriginalTweet variable within the Testing data set:

[1] "TRENDING: New Yorkers encounter empty supermarket shelves (pictured, Wegmans in Brooklyn), sold-out online grocers (FoodKick, MaxDelivery) as #coronavirus-fearing shoppers stock up https://t.co/Gr76pcrLWh https://t.co/ivMKMsqdT1"

[1] "When I couldn't find hand sanitizer at Fred Meyer, I turned to #Amazon. But $114.97 for a 2 pack of Purell??!!Check out how  #coronavirus concerns are driving up prices. https://t.co/ygbipBflMY"

Displaying the frequency and the distribution of the Sentiment variable within the Testing data set:

Extremely Negative Extremely Positive           Negative            Neutral           Positive 
               592                599               1041                619                947 

Data Preprocessing

Training data set
Replacing the entries labeled as Extremely Positive with Positive and the entries labeled as Extremely Negative with Negative of the Sentiment variable in the Training data set:

training$Sentiment[training$Sentiment == "Extremely Positive"] <- "Positive"
training$Sentiment[training$Sentiment == "Extremely Negative"] <- "Negative"

Displaying the frequency and the distribution of the Sentiment variable within the Training data set:

Negative  Neutral Positive 
   15398     7713    18046 

Factorizing the Sentiment variable, assigning numeric values to the resulting factors, and adding them to the Training data set as a new column named Labels:

training["Labels"] <- as.integer(as.factor(training$Sentiment)) - 1

Displaying the frequency of the Labels variable within the Training data set:

    0     1     2 
15398  7713 18046 

The frequency counts of the Labels variable exactly match those of the frequency of the Sentiment column in the Training data set..

Cleaning and transforming the text of the OriginalTweet variable in the Training data set:

training$OriginalTweet <- replace_non_ascii(training$OriginalTweet)
training$OriginalTweet <- tolower(training$OriginalTweet)
training$OriginalTweet <- replace_date(training$OriginalTweet)
training$OriginalTweet <- replace_time(training$OriginalTweet)
training$OriginalTweet <- replace_money(training$OriginalTweet)
training$OriginalTweet <- replace_number(training$OriginalTweet)
training$OriginalTweet <- replace_contraction(training$OriginalTweet, ignore.case = TRUE)

training$OriginalTweet <- replace_grade(training$OriginalTweet)
training$OriginalTweet <- replace_email(training$OriginalTweet)
training$OriginalTweet <- replace_hash(training$OriginalTweet)
training$OriginalTweet <- replace_tag(training$OriginalTweet)
training$OriginalTweet <- replace_html(training$OriginalTweet)
training$OriginalTweet <- replace_url(training$OriginalTweet)
training$OriginalTweet <- replace_internet_slang(training$OriginalTweet, ignore.case = TRUE)
training$OriginalTweet <- replace_emoji(training$OriginalTweet)
training$OriginalTweet <- replace_emoticon(training$OriginalTweet)

training$OriginalTweet <- removeWords(training$OriginalTweet, stopwords("english"))

training$OriginalTweet <- replace_symbol(training$OriginalTweet)

training$OriginalTweet <- mgsub(training$OriginalTweet, "'", "")

training$OriginalTweet <- strip(training$OriginalTweet)
training$OriginalTweet <- replace_kern(training$OriginalTweet)
training$OriginalTweet <- wordStem(training$OriginalTweet, language = "english")
training$OriginalTweet <- replace_white(training$OriginalTweet)

training <- training[!(training$OriginalTweet== ""), ]

Setting up a Vector Source to treat each element of the OriginalTweet variable in the Training data set as individual documents:

training.vs <- VectorSource(training$OriginalTweet)

Displaying the structure of the Training vector source:

Classes 'VectorSource', 'SimpleSource', 'Source'  hidden list of 5
 $ encoding: chr ""
 $ length  : int 41119
 $ position: num 0
 $ reader  :function (elem, language, id)  
 $ content : chr [1:41119] "advice talk neighbours family exchange phone numbers create contact list phone numbers neighbours schools emplo"| __truncated__ "coronavirus australia woolworths give elderly disabled dedicated shopping hours amid covid outbreak" "food stock one empty please panic will enough food everyone take need stay calm stay saf" "ready go supermarket outbreak i paranoid food stock litteraly empty serious thing please panic causes shortag" ...

Creating a Volatile Corpus from the Training vector source:

training.vc <- VCorpus(training.vs)

Displaying details about the Volatile Corpus:

<<VCorpus>>
Metadata:  corpus specific: 0, document level (indexed): 0
Content:  documents: 41119

Displaying details of the first two documents within the Training Volatile Corpus:

[[1]]
<<PlainTextDocument>>
Metadata:  7
Content:  chars: 194

[[2]]
<<PlainTextDocument>>
Metadata:  7
Content:  chars: 99

Displaying the content of the first two documents within the Training Volatile Corpus:

[1] "advice talk neighbours family exchange phone numbers create contact list phone numbers neighbours schools employer chemist gp set online shopping accounts poss adequate supplies regular meds ord"

[1] "coronavirus australia woolworths give elderly disabled dedicated shopping hours amid covid outbreak"

Displaying a Wordcloud of the top 100 most frequent words:

Displaying a list of the top 10 most frequent words:

Displaying a Wordcloud of the top 100 most frequent words of the positive sentiments:

Displaying a list of the top 10 most frequent words of the positive sentiments:

Generating a Document-Term Matrix from the Training Volatile Corpus to depict the frequency of terms within each document in the collection:

dtm <- DocumentTermMatrix(training.vc)

Inspecting the contents of the Document-Term Matrix and displaying it in a human-readable format. This displays terms and their frequencies within each document in the collection:

tm::inspect(dtm)

<<DocumentTermMatrix (documents: 41119, terms: 37869)>>
Non-/sparse entries: 638798/1556496613
Sparsity           : 100%
Maximal term length: 127
Weighting          : term frequency (tf)
Sample             :
       Terms
Docs    can consumer covid food grocery people prices store supermarket will
  18898   0        0     0    0       0      0      1     0           0    1
  26959   0        0     0    0       0      1      1     0           0    0
  28881   0        0     1    0       0      0      1     0           0    0
  31912   0        0     1    0       0      0      1     0           0    0
  32027   0        0     1    0       0      0      1     0           0    0
  32028   0        0     1    0       0      0      1     0           0    0
  33965   0        0     1    1       0      0      0     0           0    0
  34261   0        0     0    0       0      0      0     0           0    0
  37744   0        0     0    0       0      0      0     0           0    0
  9770    0        0     0    0       0      0      1     0           0    0

Showing the number of terms that are repeated 100 times or more in the Document-Term Matrix from the Training data set:

# Gets a list of those terms repeated 100 times or more from the Document-Term Matrix:
high.freq.terms <- findFreqTerms(dtm, lowfreq = 100)
# Display the quantity of those terms repeated 100 times or more:
length(high.freq.terms)

[1] 1121

Displaying a list of terms repeated 100 times or more in the Document-Term Matrix from the Training data set:

Showing the number of terms that are repeated less than 100 times in the Document-Term Matrix from the Training data set:

[1] 36748

Displaying a list of terms repeated less 100 times (with a low frequency of 90) in the Document-Term Matrix from the Training data set:

Removing those terms repeated less than 100 times from the Document-Term Matrix from the Training data set:

dtm <- dtm[, colnames(dtm) %in% high.freq.terms]

Confirming there are no terms repeated less than 100 times in the Document-Term Matrix from the Training data set:

cat("Terms repeated less than 100 times: ", length(findFreqTerms(dtm, highfreq = 99)))

Terms repeated less than 100 times:  0

Testing data set
Replacing the entries labeled as Extremely Positive with Positive and the entries labeled as Extremely Negative with Negative of the Sentiment variable in the Testing data set:

testing$Sentiment[testing$Sentiment == "Extremely Positive"] <- "Positive"
testing$Sentiment[testing$Sentiment == "Extremely Negative"] <- "Negative"

Displaying the frequency and the distribution of the Sentiment variable within the Testing data set:

Negative  Neutral Positive 
    1633      619     1546 

Factorizing the Sentiment variable, assigning numeric values to the resulting factors, and adding them to the Testing data set as a new column named Labels:

testing["Labels"] <- as.integer(as.factor(testing$Sentiment)) - 1

Displaying the frequency of the Labels variable within the Testing data set:

   0    1    2 
1633  619 1546 

The frequency counts of the Labels variable exactly match those of the frequency of the Sentiment column in the Testing data set.

Cleaning and transforming the text of the OriginalTweet variable in the Testing data set:

testing$OriginalTweet <- replace_non_ascii(testing$OriginalTweet)
testing$OriginalTweet <- tolower(testing$OriginalTweet)
testing$OriginalTweet <- replace_date(testing$OriginalTweet)
testing$OriginalTweet <- replace_time(testing$OriginalTweet)
testing$OriginalTweet <- replace_money(testing$OriginalTweet)
testing$OriginalTweet <- replace_number(testing$OriginalTweet)
testing$OriginalTweet <- replace_contraction(testing$OriginalTweet, ignore.case = TRUE)

testing$OriginalTweet <- replace_grade(testing$OriginalTweet)
testing$OriginalTweet <- replace_email(testing$OriginalTweet)
testing$OriginalTweet <- replace_hash(testing$OriginalTweet)
testing$OriginalTweet <- replace_tag(testing$OriginalTweet)
testing$OriginalTweet <- replace_html(testing$OriginalTweet)
testing$OriginalTweet <- replace_url(testing$OriginalTweet)
testing$OriginalTweet <- replace_internet_slang(testing$OriginalTweet, ignore.case = TRUE)
testing$OriginalTweet <- replace_emoji(testing$OriginalTweet)
testing$OriginalTweet <- replace_emoticon(testing$OriginalTweet)

testing$OriginalTweet <- removeWords(testing$OriginalTweet, stopwords("english"))

testing$OriginalTweet <- replace_symbol(testing$OriginalTweet)

testing$OriginalTweet <- mgsub(testing$OriginalTweet, "'", "")

testing$OriginalTweet <- strip(testing$OriginalTweet)
testing$OriginalTweet <- replace_kern(testing$OriginalTweet)
testing$OriginalTweet <- wordStem(testing$OriginalTweet, language = "english")
testing$OriginalTweet <- replace_white(testing$OriginalTweet)

testing <- testing[!(testing$OriginalTweet== ""), ]

Setting up a Vector Source to treat each element of the OriginalTweet variable in the Testing data set as individual documents:

testing.vs <- VectorSource(testing$OriginalTweet)

Creating a Volatile Corpus from the Testing vector source:

testing.vc <- VCorpus(testing.vs)

Generating a Document-Term Matrix utilizing the Training Document-Term Matrix and the Testing Volatile Corpus. This process ensures that the terms found in the Training Document-Term Matrix are mirrored in the Testing Document-Term Matrix:

test_dtm <- DocumentTermMatrix(testing.vc, control = list(dictionary = Terms(dtm)))

Showing the count of documents and terms in both Document-Term Matrices (number of terms should be the same):

DTM          Documents   Terms 
[Training]   41119       1121 
[Testing]    3797        1121 

 >>> The number of terms are the same!!!!!

Presenting the lists of terms contained within both Document-Term Matrices (the terms should be the same):

>>> The terms are identical!!!!!

Model Training and Evaluation.:

Training a Multinomial Naive Bayes model on the training data:

# Train the Naïve Bayes model on the training data,
multinomial.naive.bayes.model <- multinomial_naive_bayes(as.matrix(dtm), as.factor(training$Labels), laplace = 1)

Getting the predictions of the model on the testing data:

# Getting the predictions of the Naïve model on the validation data.
multinomial.naive.bayes.predictions <- predict(multinomial.naive.bayes.model, newdata = as.matrix(test_dtm))

Displaying the accuracy of the model:

Accuracy: 64.52%

Showing the count of the incorrect predictions labels of the model:

Amount of incorrect predictions: 1347

Displaying a list of the actual labels next to incorrect predictions labels of the model:

Presenting a frequency table of the actual labels of the Testing data set:

Actual labels table: 

   0    1    2 
1633  618 1546 

Showing a frequency table of the predictions labels of the model:

Predictions labels table: 

multinomial.naive.bayes.predictions
   0    1    2 
1613  602 1582 

Displaying a Confusion Matrix of the actual labels of the Testing data set vs the predictions labels of the model:

Confusion Matrix: 

              multinomial.naive.bayes.predictions
               (Pred Neg) 0 (Pred Neu) 1 (Pred Pos) 2
  (True Neg) 0         1106          194          333
  (True Neu) 1          181          266          171
  (True Pos) 2          326          142         1078

Presenting a Proportion Confusion Matrix comparing the total count of the actual labels from the Testing data set against the predicted labels generated by the model.:

Proportion Confusion Matrix: 

              multinomial.naive.bayes.predictions
               (Pred Neg) 0 (Pred Neu) 1 (Pred Pos) 2
  (True Neg) 0   0.29128259   0.05109297   0.08770082
  (True Neu) 1   0.04766921   0.07005531   0.04503555
  (True Pos) 2   0.08585726   0.03739795   0.28390835

Based on the Confusion Matrices, 2450 of the predictions were correctly labeled, accounting for 64.52% of all predictions. These findings corroborate previous data from this study. Subtracting 2450 from the total predictions made (3797) yields 1347 (the number of incorrect predictions calculated previously), and the percentage obtained from the Confusion Matrices precisely matches the accuracy percentage of the model (without rounding).

Of the incorrectly predicted values, it appears that the model mislabeled more Negative values as Positive and vice versa. Specifically, 333 Negative values were labeled as Positive, constituting approximately 8.77% of all predictions. This was closely followed by 326 Positive values labeled as Negative, representing around 8.59% of all predictions.

Note: Use the following table for the Confusion Matrices interpretation.

           True-Negative      False-Neutral     False-Positive
           False-Negative     True-Neutral      False-Positive
           False-Negative     False-Neutral     True-Positive