Sentiment Analysis

library(here)

## here() starts at /home/munira.odigie/Sentimental Analysis

library(dplyr)

## 
## Attaching package: 'dplyr'

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

library(tidytext)
library(tidyr)
library(knitr)
library(ggplot2)

Sentiment analysis allows us to use standard lexicons to explore the sentiments expressed in texts.

load("~/Preparingdata/data/Thissideofparadise_trimmed.rda")
load("~/Preparingdata/data/Thissideofparadise_long.rda")

Will will use the long data (one word per line). In this data you should also have a row number variable and a variable that defines which chapter/story/section the word is in. We will be using those variables to help us to explore sentiments in the text.

bing <- get_sentiments("bing")
bing |> inner_join( Thissideofparadise_long, by = "word", multiple = "all" ) -> tsop_bing

## Warning in inner_join(bing, Thissideofparadise_long, by = "word", multiple = "all"): Detected an unexpected many-to-many relationship between `x` and `y`.
## ℹ Row 15 of `x` matches multiple rows in `y`.
## ℹ Row 7888 of `y` matches multiple rows in `x`.
## ℹ If a many-to-many relationship is expected, set `relationship =
##   "many-to-many"` to silence this warning.

table(tsop_bing$sentiment)

## 
## negative positive 
##     3110     1925

table(tsop_bing$sentiment) |> prop.table() |>round(2)

## 
## negative positive 
##     0.62     0.38

Overall the lannguage is more negative than positive.

#table(qs_bing$sentiment, qs_bing$younger) |> prop.table(2) |> round(2)

If we compare the stories deemed suitable for younger children to the others, the children’s are more negative.

#table(qs_bing$sentiment, qs_bing$section_type) |> prop.table(2) |> round(2)

Interestingly the stories are more negative than the other materials.

This does tend to support the idea that the epigraphs are more flowery.

Based on this, I’m going to redo the earlier comparisons of young and old with just stories.

#tsop_bing |> filter(section_type == "Story") -> tsop_stories_bing
##table(qs_stories_bing$sentiment, qs_stories_bing$younger) |> 
  #prop.table(2) |> round(2)

The results are essentially the same.

How much variation is there from story to story?

table( tsop_bing$chapter_titles, tsop_bing$sentiment) |> 
 prop.table(1) |> round(2)

##                                             
##                                              negative positive
##   CHAPTER 1. AMORY, SON OF BEATRICE              0.55     0.45
##   CHAPTER 1. THE DEBUTANTE                       0.54     0.46
##   CHAPTER 2. EXPERIMENTS IN CONVALESCENCE        0.61     0.39
##   CHAPTER 2. SPIRES AND GARGOYLES                0.58     0.42
##   CHAPTER 3. THE EGOTIST CONSIDERS               0.72     0.28
##   CHAPTER 3. YOUNG IRONY                         0.65     0.35
##   CHAPTER 4. NARCISSUS OFF DUTY                  0.62     0.38
##   CHAPTER 4. THE SUPERCILIOUS SACRIFICE          0.73     0.27
##   CHAPTER 5. THE EGOTIST BECOMES A PERSONAGE     0.66     0.34

This is a lot; is there a better way to display this?

 tsop_bing2 <- tsop_bing |> group_by(chapter_titles) |> 
  summarize(Positive = round(sum(sentiment == "positive")/n(), 2)) |>
  arrange(Positive)

Visualizing this would be better. This is one possible way.
For yours make sure to add labels. In this case I might also want to add a line highlighting .5 or to change the tick marks to help visualize that. Do what makes sense for your data.

tsop_bing2 <-tsop_bing2 |> filter(chapter_titles != 'NA')

 tsop_bing2 |> 
  ggplot(aes(x=reorder(chapter_titles, Positive), y =Positive)) +
  geom_point(size = 4) +
   geom_segment(aes(x=chapter_titles, xend=chapter_titles, y=0, yend=Positive)) +
  coord_flip()

Another thing to look at will be how sentiments change during the stories.

How do we have to fix this to make it meaningful?

 tsop_bing |> 
  drop_na(chapter_titles) |>
  ggplot( aes(row_number, sentiment, fill =chapter_titles)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~chapter_titles, ncol = 2, scales = "free_x")

On your own do the same kind of analysis but using the afinn lexicon.
You will have to think about what needs to change when you have numbers rather than just positive/negative.