Load the libraries + functions

Load all the libraries or functions that you will use to for the rest of the assignment. It is helpful to define your libraries and functions at the top of a report, so that others can know what they need for the report to compile correctly.

##r chunk
library(reticulate)
##r chunk

py_install('pandas')
py_install('numpy')
py_install('bs4', pip=T)
py_install('regex', pip = T)
py_install('nltk')
py_install('gensim')
py_install('lxml')

Load the Python libraries or functions that you will use for that section.

##python chunk


import pandas as pd
import numpy as np
from bs4 import BeautifulSoup
import re
import nltk
import gensim
from nltk.corpus import abc
from nltk.corpus import stopwords
import nltk

The Data

The dataset is a set of text messages that have been coded as: - “ham”: normal text messages - “spam”: bad text messages

Import the data using either R or Python. I put a Python chunk here because you will need one to import the data, but if you want to first import into R, that’s fine.

##python chunk

import pandas as pd
from sklearn.metrics import classification_report
from sklearn.metrics import accuracy_score
SOdata = pd.read_csv("spam_text.csv")
SOdata.head()
##   category                                               text
## 0      ham  Go until jurong point, crazy.. Available only ...
## 1      ham                      Ok lar... Joking wif u oni...
## 2     spam  Free entry in 2 a wkly comp to win FA Cup fina...
## 3      ham  U dun say so early hor... U c already then say...
## 4      ham  Nah I don't think he goes to usf, he lives aro...

Clean up the data

Use one of our clean text functions to clean up the text column in the dataset.

##python chunk
import re

from nltk.corpus import stopwords
import nltk
import lxml

REPLACE_BY_SPACE_RE = re.compile('[/(){}\[\]\|@,;]') #remove symbols with space
BAD_SYMBOLS_RE = re.compile('[^0-9a-z #+_]') #take out symbols altogether
STOPWORDS = set(stopwords.words('english')) #stopwrods

def clean_text(text):
    text = BeautifulSoup(text, "html.parser").text # HTML decoding
    text = text.lower() # lowercase text
    text = REPLACE_BY_SPACE_RE.sub(' ', text) # replace REPLACE_BY_SPACE_RE symbols by space in text
    text = BAD_SYMBOLS_RE.sub('', text) # delete symbols which are in BAD_SYMBOLS_RE from text
    text = ' '.join(word for word in text.split() if word not in STOPWORDS) # delete stopwors from text
    return text
    
SOdata['category'] = SOdata['category'].apply(clean_text)
SOdata.head()
##   category                                               text
## 0      ham  Go until jurong point, crazy.. Available only ...
## 1      ham                      Ok lar... Joking wif u oni...
## 2     spam  Free entry in 2 a wkly comp to win FA Cup fina...
## 3      ham  U dun say so early hor... U c already then say...
## 4      ham  Nah I don't think he goes to usf, he lives aro...

Split the data

Split the data into testing and training data.

##python chunk

X = SOdata['category']
y = SOdata['text']

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state = 42)

Process the data

For FastText and word2vec, create the tokenized vectors of the text.

##python chunk
tokenized_train = [nltk.tokenize.word_tokenize(text)
                   for text in X_train.to_list()]
tokenized_test = [nltk.tokenize.word_tokenize(text)
                   for text in X_test.to_list()]

Word2Vec

Build the word2vec model.

##python chunk

w2v_model = gensim.models.Word2Vec(tokenized_train, 
                                   size=100, window=6,
                                   min_count=2, iter=5, workers=4)

Convert the model

Convert the model into a set of features to use in our classifier.

##python chunk

def document_vectorizer(corpus, model, num_features):
    vocabulary = set(model.wv.index2word)
    
    def average_word_vectors(words, model, vocabulary, num_features):
        feature_vector = np.zeros((num_features,), dtype="float64")
        nwords = 0.
        
        for word in words:
            if word in vocabulary: 
                nwords = nwords + 1.
                feature_vector = np.add(feature_vector, model.wv[word])
        if nwords:
            feature_vector = np.divide(feature_vector, nwords)

        return feature_vector

    features = [average_word_vectors(tokenized_sentence, model, vocabulary, num_features)
                    for tokenized_sentence in corpus]
    return np.array(features)
    

avg_wv_train_features = document_vectorizer(corpus=tokenized_train,
                                                    model=w2v_model,
                                                     num_features=100)
avg_wv_test_features = document_vectorizer(corpus=tokenized_test, 
                                                    model=w2v_model,
                                                    num_features=100)

Build a classifier model

In class, we used logistic regression to classify the data. You can use any machine learning algorithm you want here, and build a classification model.

##python chunk

my_category = ["ham", "spam"]

#build a log model
from sklearn.linear_model import LogisticRegression
logreg = LogisticRegression(solver='lbfgs', multi_class='ovr', max_iter=10000)

#fit the data to the log model
logreg = logreg.fit(avg_wv_train_features, y_train)

Examine the results

Print out the accuracy, recall, and precision of your model.

##python chunk

y_pred = logreg.predict(avg_wv_test_features)

#print out results
print('accuracy %s' % accuracy_score(y_pred, y_test))
#print(classification_report(y_test, y_pred,target_names=my_category))
## accuracy 0.008071748878923767

Build a FastText model

Using the same data, build a FastText model.

##python chunk

from gensim.models.fasttext import FastText

#build a fast test model
ft_model = FastText(tokenized_train, size=100, window=6, 
                    min_count=2, iter=5, workers=4)

Extract the features

Convert the FastText model into features for prediction.

##python chunk

avg_ft_train_features = document_vectorizer(corpus=tokenized_train, model=ft_model,
                                                     num_features=100)
avg_ft_test_features = document_vectorizer(corpus=tokenized_test, model=ft_model,
                                                    num_features=100)

Build a classifier model

Using the same machine learning algorithm as above, build a classifier model that uses the FastText data to predict the categories.

##python chunk

logreg = LogisticRegression(solver='lbfgs', multi_class='ovr', max_iter=10000)
logreg = logreg.fit(avg_ft_train_features, y_train)
y_pred = logreg.predict(avg_ft_test_features)

Examine the results

Print out the accuracy, recall, and precision of your model.

##python chunk

print('accuracy %s' % accuracy_score(y_pred, y_test))
#print(classification_report(y_test, y_pred,target_names=my_category))

## accuracy 0.008071748878923767

Interpretation

From accuracy result logistic regression model does not show good predictive power.