The Goal

In class, we have been discussing that words can be useful features when attempting to classify text data. We have seen how to use the frequency of a word to choose the top 10 words to use as features. Today, we are going use TF-IDF.

The Data Set

We will work with the same data on \(n=2000\) headlines from our last lab. (Yes, I know we’ve been working with this a lot! This is because it helps to see how different techniques applied to the same data set can yield different results. You will have a different data set to work with for Lab 4!)

headlines <- read.csv("https://www.dropbox.com/scl/fi/r9p76t3v8aluz2jfypy6u/headlines.csv?rlkey=pi5rpu21xkwjw8qm7bofkrrej&st=jhc4e0ad&dl=1")

The columns are:

• title: the title of the article
• clickbait: a human generated indicator for whether or not an article is clickbait.
• ids: a numeric variable assigned to each title; think of this like an article identifier.

Once you have loaded the data, load the packages you will need for this lab:

library(tidytext)
library(tidyr)
library(dplyr)
library(ggplot2)
library(tm)
# New!! 
library(forcats)

TF-IDF

We have already determined which words tend to occur most frequently in clickbait and non-clickbait titles using raw counts. However, that relied on (1) removing stop words and numbers and (2) focusing on words with the highest frequency.

Today, we are going to use TF-IDF as a different tool for deciding which words might distinguish these two article types. This means we are curious to know in general what sorts of words are associated with clickbait titles rather than true news titles and vice versa. This is different that just finding the words that occur the most often; we want to find terms that define these two different title types.

The TF-IDF has two scores: the TF score and IDF score.

clickbait_words <- tidy_clickbait |>
  # Count the number of times each word appears 
  count(word, sort = TRUE)|>
  # Add a column for the total number of words
  mutate(total = sum(n))

# Tokenize Headlines 
tidy_headlines <- headlines |>
  mutate(title = removePunctuation(title)) |>
  unnest_tokens(word, title)

headlines_tfidf <- tidy_headlines |>
  # Count the number of times each word appears 
  count(clickbait, word, sort = TRUE) |>
  # Add on the TF-IDF
  bind_tf_idf(word, document = clickbait, n)

headlines_tfidf <- headlines |>
  # Tokenize Headlines 
  mutate(title = removePunctuation(title)) |>
  unnest_tokens(word, title) |> 
  # Count the number of times each word appears 
  count(clickbait, word, sort = TRUE) |>
  # Add on the TF-IDF
  bind_tf_idf(word, document = clickbait, n)

# Get the top 15 words in terms of TFIDF
# in each document 
top15_tfidf <- headlines_tfidf |>
  # Divide into the two documents 
  group_by(  REPLACE  ) |>
  # Choose the top 15 words in terms of 
  # TF-IDF in each document 
  slice_max(  REPLACE    , n = REPLACE   ) |>
  ungroup() |>
  # Reorder the word counts so the highest
  # TF-IDF is on top 
  mutate(word = reorder(word, REPLACE     ))

# Create the plot 
ggplot(top15_tfidf, aes(REPLACE, word, fill = REPLACE)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~ REPLACE, ncol = 2, scales = "free") +
  labs(x = "TF-IDF", y = NULL) +
  theme_light()

Pronouns

One big thing we should notice from our plots is that the clickbait titles contain pronouns like “you” and “we”, while the non-clickbait titles generally do not. This is something y’all noticed in our first class. So, maybe we can use the number of pronouns as a feature!

Let’s start with only a few pronouns. There are many, and the nice thing about the technique we are going to try is that we can easily add or subtract from this list.

# A list of *some* (not all) pronouns
pronouns <- c("I", "me", "you", "he", "him", "she", "her",
    "it", "we", "us", "they", "them", "one", "your", "my", "yours", "theirs")

Once we have specified a list of words we are interested in looking at (pronouns for now), we can count the number of times those specific words occur in clickbait and non-clickbait.

pronoun_counts <- tidy_headlines |>
    # Group by clickbait 
    group_by(word, clickbait) |>
    # Look at only words that are pronouns
    filter(word %in% pronouns)|>
    # And count how many times they occur
    count(word,sort = TRUE)

As a note…it turns out that this is not actually because this pronoun occurs more often. This pronoun is actually an acronym for a country!!

This is something that is tricky with text analysis. If an acronym actually spells out a word, it can be very difficult to distinguish between that acronym and the word, especially as tidyTex removes capitalization by default.

In this case, it is likely that these are not actually pronouns - it is more likely that these are acronyms, so we could remove this pronoun from our list.

The number of pronouns is an example of a feature we can created which is specific to text data. There are many other such features, such as the number of question words or the number of contractions. We will start exploring these very soon!

Next Steps

Okay, so now we have seen how to use TF-IDF to determine words that distinguish documents. This can be very helpful for things like determining key differences between different texts.

We are now going to transition from the counts of words to the actual meanings of the words. What if instead of looking at counts, we considered what the words actually mean? That is where we are headed next week!

References

Data

Code

The code was adapted from of “Text Mining with R: A Tidy Approach”, written by Julia Silge and David Robinson. The book was last built on 2024-06-20.

Activity