Introduction

Natural Language Processing (NLP) has countless applications in commercial settings. The ability to use code and algorithms to analyze, model, and even predict large amounts of user-input text is extremely valuable in many business settings.

In this post I will utilize clothing shopper reviews to predict the Department Name that the clothing item being reviewed came from. Because there are seven department name factors, this will be a multinomial-response NLP classification and prediction exercise.

The Data

The data used is from https://www.kaggle.com/nicapotato/womens-ecommerce-clothing-reviews. I will be using a 90/10 Train/Test split on this data, resulting in the following observation counts:

After formatting, my data consists of three variables:

So ultimately, the data looks like this:

id sentiment Review.Text
0 Intimate Absolutely wonderful - silky and sexy and comfortable
1 Dresses Love this dress! it’s sooo pretty. i happened to find it in a store, and i’m glad i did bc i never would have ordered it online bc it’s petite. i bought a petite and am 5’8“. i love the length on me- hits just a little below the knee. would definitely be a true midi on someone who is truly petite.
2 Dresses I had such high hopes for this dress and really wanted it to work for me. i initially ordered the petite small (my usual size) but i found this to be outrageously small. so small in fact that i could not zip it up! i reordered it in petite medium, which was just ok. overall, the top half was comfortable and fit nicely, but the bottom half had a very tight under layer and several somewhat cheap (net) over layers. imo, a major design flaw was the net over layer sewn directly into the zipper - it c
3 Bottoms I love, love, love this jumpsuit. it’s fun, flirty, and fabulous! every time i wear it, i get nothing but great compliments!
4 Tops This shirt is very flattering to all due to the adjustable front tie. it is the perfect length to wear with leggings and it is sleeveless so it pairs well with any cardigan. love this shirt!!!
5 Dresses I love tracy reese dresses, but this one is not for the very petite. i am just under 5 feet tall and usually wear a 0p in this brand. this dress was very pretty out of the package but its a lot of dress. the skirt is long and very full so it overwhelmed my small frame. not a stranger to alterations, shortening and narrowing the skirt would take away from the embellishment of the garment. i love the color and the idea of the style but it just did not work on me. i returned this dress.
6 Tops I aded this in my basket at hte last mintue to see what it would look like in person. (store pick up). i went with teh darkler color only because i am so pale :-) hte color is really gorgeous, and turns out it mathced everythiing i was trying on with it prefectly. it is a little baggy on me and hte xs is hte msallet size (bummer, no petite). i decided to jkeep it though, because as i said, it matvehd everything. my ejans, pants, and the 3 skirts i waas trying on (of which i ]kept all ) oops.
7 Tops I ordered this in carbon for store pick up, and had a ton of stuff (as always) to try on and used this top to pair (skirts and pants). everything went with it. the color is really nice charcoal with shimmer, and went well with pencil skirts, flare pants, etc. my only compaint is it is a bit big, sleeves are long and it doesn’t go in petite. also a bit loose for me, but no xxs… so i kept it and wil ldecide later since the light color is already sold out in hte smallest size…
8 Dresses I love this dress. i usually get an xs but it runs a little snug in bust so i ordered up a size. very flattering and feminine with the usual retailer flair for style.
9 Dresses I’m 5“5’ and 125 lbs. i ordered the s petite to make sure the length wasn’t too long. i typically wear an xs regular in retailer dresses. if you’re less busty (34b cup or smaller), a s petite will fit you perfectly (snug, but not tight). i love that i could dress it up for a party, or down for work. i love that the tulle is longer then the fabric underneath.

The multinomial ‘sentiment’ response variable is distributed as follows:

And once it is converted to a numeric (for modeling purposes):

Creating DTM Train & Test objects:

Training and Test sets have to be converted to Document-Term Matrix (DTM) format. This is done using a couple of functions I wrote that are powered by the text2vec R package. The function code is not shown here for the sake of brevity, but I am happy to provide it to anyone intersted.

Classification Using ‘xgboost’

The Model:

The data used is the DTM Train object that I created above, and the label (‘response’) object is the ‘sentiment’ (ie Department Name) variable.

I used a n-min and n-max of 1 for my n-gram parameters, as trials of other n-gram values resulted in a less-robust model.

Boost Rounds:

An important takeaway here is the number of boost rounds (‘nrounds’) that I am using; 10,000 rounds for this particular exercise. With each round, the train RMSE value decreases, ie the model is getting better with every round.

However, that improvement is definitely not a linear progression with regards to iteration. It has a floor:

Feature Importance:

Also of interest is the ability to extract feature importance when using ‘xgboost’ (this is analogous to the importance output that can be retrieved using the Random Forest algorithm).

In NLP analysis our features are unique words, therefore by extracting importance from our model, we are actually able to see which words are most “descriptive” for our sentiment response:

Feature Gain Cover Frequency
dress 0.2067478 0.0021940 0.0097856
top 0.0923768 0.0020901 0.0074034
shirt 0.0665656 0.0013475 0.0033047
sweater 0.0549642 0.0013305 0.0032463
jeans 0.0534868 0.0020417 0.0043206
skirt 0.0448284 0.0012525 0.0021837

It is easily understandable why being able to extract the most important words from an NLP classification is quite valuable. After all, these are the words that most-influence the model’s outcome.

Predicting ‘sentiment’

Sentiment prediction was done using the boosted model from above with the 2,641 observation test data set.

Multinomial Response Prediction Accuracy:

To assess accuracy, I first looked at “prediction off actual”; in other words, the literal difference of actual minus predicted for the response factor.

As can be seen, the vast majority of the predictions were of by only 2 factor points, and ~42% predicted with complete accuracy:

When looking at the stats of our predicted responses, we can see that the model has trouble with predicting Department Name factors 1 and 7. However, when looking at Department Name factors 2, 3, 4, 5, and 6, it does quiet well:

An important consideration here is that Department Name factors 1 and 7 had the least amount of observations as compared to Department Name factors 2, 3, 4, 5, and 6 (see distribution graphs above). So, the model had the least amount of training with those particular factors.

Summary:

The performance of this model in its prediction accuracy might be improved by a variety of facets. Using a different stop-word list and/or trying out dynamic n-gram values are just two exaples of these facets.

After all, NLP classification and prediction is an ever-expanding field that is fuled by trial and error.