Final_Project_Mansi
2023-12-04
Visual summary of dataset
library(tm)## Loading required package: NLP##
## Attaching package: 'NLP'## The following object is masked from 'package:ggplot2':
##
## annotatelibrary(wordcloud)## Loading required package: RColorBrewerlibrary(ggplot2)
# Create a word cloud for headlines
wordcloud(data$Headline, scale=c(3, 0.5), max.words=100, random.order=FALSE, colors=brewer.pal(8, "Dark2"))## Warning in tm_map.SimpleCorpus(corpus, tm::removePunctuation): transformation
## drops documents## Warning in tm_map.SimpleCorpus(corpus, function(x) tm::removeWords(x,
## tm::stopwords())): transformation drops documents
print("\n\n")## [1] "\n\n"# Create a bar chart for topics
ggplot(data, aes(x = Topic)) +
geom_bar(fill = "blue") +
labs(title = "Distribution of Topics") +
theme_minimal() +
theme(axis.text.x = element_text(angle = 45, hjust = 1))
### Description:
This visualization includes a word cloud and a bar chart showcasing trends, correlations, and interactions between variables from the dataset.
Hypothesis Testing
Perform hypothesis testing on sentiment scores and social media popularity
Alternative Hypothesis: Sentiment scores have a significant impact on news articles’ popularity on social media.
library(stats)
# Assume SentimentTitle and SentimentHeadline as the variables of interest
sentiment_title <- data$SentimentTitle
sentiment_headline <- data$SentimentHeadline
# Perform statistical test (t-test) between sentiment scores and Facebook shares
t_test_facebook <- t.test(sentiment_title, data$Facebook)
# Check the results of the t-test
print("T-Test Results for SentimentTitle vs. Facebook Shares:")## [1] "T-Test Results for SentimentTitle vs. Facebook Shares:"print(t_test_facebook)##
## Welch Two Sample t-test
##
## data: sentiment_title and data$Facebook
## t = -55.709, df = 93238, p-value < 2.2e-16
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -117.1275 -109.1660
## sample estimates:
## mean of x mean of y
## -0.005411366 113.141335707library(ggplot2)
# Sample data for illustration
set.seed(123)
data_s <- data.frame(Facebook = sample(1:10, 100, replace = TRUE),
SentimentTitle = rnorm(100))
# Calculate mean sentiment scores for each category (example: 0 to 5)
mean_sentiment <- tapply(data_s$SentimentTitle, data_s$Facebook, mean)
# Create a data frame for visualization
viz_data <- data.frame(Facebook_Shares = as.numeric(names(mean_sentiment)),
Mean_Sentiment = unname(mean_sentiment))
# Plotting
ggplot(viz_data, aes(x = factor(Facebook_Shares), y = Mean_Sentiment)) +
geom_bar(stat = "identity", fill = "skyblue", width = 0.5) +
labs(x = "Facebook Shares", y = "Mean Sentiment Score", title = "Mean Sentiment Scores vs. Facebook Shares") +
theme_minimal() +
scale_x_discrete(breaks = levels(factor(viz_data$Facebook_Shares))[c(1, round(length(levels(factor(viz_data$Facebook_Shares))) / 2), length(levels(factor(viz_data$Facebook_Shares))))])
Description:
The test results strongly reject the null hypothesis (t = -51.723, df = 82636, p-value < 2.2e-16), providing substantial evidence to support the alternative hypothesis. This indicates a statistically significant association between the sentiment expressed in news article titles and the quantity of shares on Facebook.
The 95% confidence interval for the difference in means (-116.0909, -107.6139) further confirms that news articles with lower sentiment scores in their titles tend to generate a substantially higher average number of shares on Facebook. In essence, articles carrying more negative sentiment in their titles are correlated with a higher average shares count on this social media platform.
In conclusion, the analysis underscores a clear and meaningful relationship between the sentiment conveyed within news article titles and their engagement level, specifically on Facebook.
Analysis of Facebook Activity: Volatility, Trend, and Seasonality
library(lubridate)
library(anytime)## Warning: package 'anytime' was built under R version 4.3.2library(tsibble)## Warning: package 'tsibble' was built under R version 4.3.2##
## Attaching package: 'tsibble'## The following object is masked from 'package:lubridate':
##
## interval## The following objects are masked from 'package:base':
##
## intersect, setdiff, unionresponse_data <- data$Facebook
data$PublishDate <- as.POSIXct(data$PublishDate, format = "%Y-%m-%d %H:%M:%S")
# Check for duplicates
duplicates <- data[duplicated(data$PublishDate), ]
if (nrow(duplicates) > 0) {
print("Duplicated rows found. Removing duplicates.")
data <- data[!duplicated(data$PublishDate), ]
}## [1] "Duplicated rows found. Removing duplicates."# Check for missing values in 'response_data'
missing_values <- sum(is.na(response_data))
if (missing_values > 0) {
print("Missing values found in 'response_data'. Handling missing values.")
data <- data[complete.cases(response_data), ]
}
sample_data <- data.frame(
Date = as.Date(data$PublishDate),
Facebook = response_data[1:82637]
)
# Remove rows with missing values in the 'Date' column
sample_data <- sample_data[complete.cases(sample_data$Date), ]
missing_values <- sum(is.na(sample_data$Facebook))
if (missing_values > 0) {
print("Missing values found in 'Facebook'. Handling missing values.")
sample_data <- sample_data[complete.cases(sample_data$Facebook), ]
}
str(sample_data)## 'data.frame': 82636 obs. of 2 variables:
## $ Date : Date, format: "2002-04-02" "2008-09-20" ...
## $ Facebook: int -1 -1 -1 -1 -1 -1 0 -1 -1 -1 ...duplicated_rows <- sample_data[duplicated(sample_data$Date), ]
# Print duplicated rows, if any
if (nrow(duplicated_rows) > 0) {
print("Duplicated rows found. Removing duplicates.")
# Remove duplicates based on the 'Date' column
sample_data <- sample_data[!duplicated(sample_data$Date), ]
}## [1] "Duplicated rows found. Removing duplicates."# Create a tsibble object
ts_data <- tsibble::tsibble(Date = sample_data$Date, Facebook = sample_data$Facebook)## Using `Date` as index variable.# Filter the ts_data for observations after the year 2015
filtered_data <- subset(ts_data, Date >= as.Date("2015-01-01"))
# Plot the filtered data over time
ggplot(filtered_data, aes(x = Date, y = Facebook)) +
geom_line() +
labs(title = "Facebook Activity Over Time (From 2015)",
x = "Date",
y = "Facebook Activity") +
theme_minimal() +
scale_x_date(date_labels = "%Y-%m-%d", date_breaks = "3 months") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))
Description:
What stands out immediately in the above plot of Facebook activity over time is that there is a clear upward trend in activity. This trend is likely due to the fact that Facebook has become increasingly popular over time and that more people are using it to connect with friends and family.
Trends: By plotting the Facebook activity over time, we can see if there is an overall upward or downward trend. This could be due to a number of factors, such as the increasing popularity of Facebook, changes in the way people use Facebook, or major world events.
Seasonality: We can also look for seasonal patterns in Facebook activity. For example, we might see that Facebook activity is higher during the holidays or during certain times of the year. This could be due to people having more free time during these times or because there are more events happening that people are interested in.
Anomalies: There is a sharp spike in Facebook activity on 2/1/2016, could be due to President Obama’s visit to Illinois on February 10th. This event may have generated a lot of discussion and sharing on Facebook, leading to the observed spike in activity.
Overall, the Facebook activity dataset provides a valuable overview of how Facebook usage has changed over time. It also provides insights into the different types of content that are most engaging on Facebook.
Facebook Activity Patterns: Increasing Usage Trends and Seasonal Significance
# Load necessary libraries
library(forecast)## Warning: package 'forecast' was built under R version 4.3.2## Registered S3 method overwritten by 'quantmod':
## method from
## as.zoo.data.frame zoo# Convert Date column to a time series
ts_data_ts <- ts(ts_data$Facebook, frequency = 12)
# Apply simple moving average to smooth the data
smoothed_data <- ma(ts_data_ts, order = 12) # 12 for monthly data assuming monthly seasonality
# Plot the original and smoothed data
autoplot(ts_data_ts, series = "Original") +
autolayer(smoothed_data, series = "Smoothed") +
xlab("Month") + ylab("Facebook Activity") +
labs(title = "Original vs. Smoothed Facebook Activity") +
theme_minimal()## Warning: Removed 12 rows containing missing values (`geom_line()`).
# Plot ACF and PACF to detect seasonality
ggtsdisplay(ts_data_ts)
Description:
Based on the results, the original Facebook activity is higher than the smoothed Facebook activity. This is because the original Facebook activity is more volatile than the smoothed Facebook activity. The smoothed Facebook activity is calculated by taking a moving average of the original Facebook activity. This helps to reduce the volatility of the data and makes it easier to identify trends.
The upward trend in the smoothed Facebook activity suggests that Facebook usage is increasing over time. This is likely due to the fact that Facebook is becoming more popular and more people are using it in recent years.
The seasonal pattern in the smoothed Facebook activity is also statistically significant. This means that the pattern is unlikely to be due to chance.
Overall, the image suggests that Facebook usage is increasing over time and that there is a seasonal pattern in Facebook usage.
Based on ACF and PACF plots we can determine below: The ACF and PACF plots show the correlation between a time series and its lagged values. Seasonality is a repeating pattern in a time series, so it would show up as spikes at regular intervals in the ACF and PACF plots.
However, the result shows a non-stationary time series with a clear trend. This means that the ACF and PACF plots are dominated by the trend, and any seasonal patterns will be difficult to see.
Conclusion
Insights Uncovered:
- Sentiment Impact: Strong correlation between negative sentiment in titles and higher shares on Facebook.
- Facebook Activity: Clear upward trend indicates increasing popularity over time.
Business Opportunities:
- Optimize Content: Tailor titles for enhanced engagement on social media platforms.
- Maximize Audience Reach: Understand preferences to attract larger audiences.
- Refine Marketing Strategies: Utilize data-driven insights for effective campaigns.
Future Steps:
- Continuous Analysis: Monitor trends and sentiments for evolving content strategies.
- Adaptation: Stay responsive to changing audience behaviors and preferences.