Week 7 Data Dive

Null Hypothesis #1

Null Hypothesis: There is no relationship a book’s rating and the number of units sold.

Value Reasoning

Significance Level

⍺ = 0.01

 I opted to choose as more stringent significance level so that I could reduce the chance of Type I errors. I felt as though for this hypothesis test, a Type 1 error would have more severe consequences. For example, making a claim of a significant relationship when there is none could cause harm to authors.

Power Level

(1-𝜷) = 0.85

 A power level of 0.85 ensures that I am reducing the risk of overlooking a meaningful relationship.

Effect Size

𝒅 = -0.89

 Calculating Cohen’s D suggests that there is a very large negative association between book ratings and units sold.

Determining Cohen’s D for effect size.

#determining cohen's D for effect size

#calculate cohen's D
cohen_d_result <- cohen.d(book_sales$Book_average_rating, book_sales$`units sold`)

cohen_d_value <- cohen_d_result$estimate

cat("Cohen's d for the relationship between Book Rating and Units Sold:", cohen_d_value, "\n")
## Cohen's d for the relationship between Book Rating and Units Sold: -0.8899897

Now that I have my necessary values, I will need to determine if I have enough data to perform a Neyman-Pearson test.

#cohen's d
observed_d <- -0.89

#signnificance
alpha <- 0.01

#power
desired_power <- 0.85

#determine sample size with power test
sample_size <- pwr.t.test(d = observed_d, sig.level = alpha, power = desired_power, 
                          alternative = "two.sided")$n

cat("Required Sample Size for Neyman-Pearson Test:", sample_size, "\n")
## Required Sample Size for Neyman-Pearson Test: 34.6434

There is a required sample size of 35 to perform a Neyman-Pearson test. I have over 1000 rows of data in my dataset, therefore I am able to perform this test.

alpha <- 0.01

#neyman-pearson
neyman_pearson_result <- cor.test(
  book_sales$Book_average_rating,
  book_sales$`units sold`,
  method = "pearson",
  alternative = "greater"
)

#interpret p-value
if (neyman_pearson_result$p.value < alpha) {
  cat("Reject the null hypothesis.\n")
} else {
  cat("Failure to reject the null hypothesis.")
}
## Failure to reject the null hypothesis.
print(neyman_pearson_result)
## 
##  Pearson's product-moment correlation
## 
## data:  book_sales$Book_average_rating and book_sales$`units sold`
## t = -0.27832, df = 1068, p-value = 0.6096
## alternative hypothesis: true correlation is greater than 0
## 95 percent confidence interval:
##  -0.0588037  1.0000000
## sample estimates:
##          cor 
## -0.008516131

This test determined a p-value of .6096, which is greater than the chosen significance value. From these results, it can be concluded that the null hypothesis cannot be rejected, therefore there is no meaningful correlation between average book rating and units sold.

contingency_table <- table(cut(book_sales$Book_average_rating, breaks = 5), cut(book_sales$`units sold`, breaks = 5))

#fisher's exact test
fisher_test_result <- fisher.test(contingency_table, simulate.p.value = TRUE)

#interpret the p-value
if (fisher_test_result$p.value < alpha) {
  cat("Reject the null hypothesis.\n")
} else {
  cat("Failure to reject the null hypothesis.\n")
}
## Failure to reject the null hypothesis.
print(fisher_test_result)
## 
##  Fisher's Exact Test for Count Data with simulated p-value (based on
##  2000 replicates)
## 
## data:  contingency_table
## p-value = 0.918
## alternative hypothesis: two.sided

The p-value of 0.916 found with this test once again suggest that the null hypothesis shall not be rejected, therefore there not enough evidence to conclude that there is much of a relationship between book ratings and their units sold.

ggplot(book_sales, aes(x = Book_average_rating, y = `units sold`)) +
  geom_point() +
  geom_smooth(method = "lm", se = FALSE, color = "blue") +
  labs(x = "Book Rating", y = "Units Sold", title = "Scatter Plot of Book Rating vs Units Sold")
## `geom_smooth()` using formula = 'y ~ x'

The scatter plot of book rating vs units sold does indicate that there may not be a strong correlation between the two variables.

Null Hypothesis #2

Null Hypothesis: There is no difference in the average book ratings between novice authors and famous authors.

Value Reasoning

Significance Level

⍺ = 0.01

 I opted to choose as more stringent significance level so that I could reduce the chance of Type I errors. I felt as though for this hypothesis test, a Type 1 error would have more severe consequences. For example, making a claim of a significant relationship when there is none could cause a lot of wasted resources.

Power Level

(1-𝜷) = 0.85

 A power level of 0.85 ensures that I am reducing the risk of overlooking a meaningful relationship.

Effect Size

𝒅 = -5.978

 Calculating Cohen’s D suggests that there is a very large negative association between the average book rating of novice authors and famous authors 
#split data between novice and famous authors
novice_ratings <- book_sales$Book_average_rating[book_sales$Author_Rating == "Novice"]
famous_ratings <- book_sales$Book_average_rating[book_sales$Author_Rating == "Famous"]

#calculate Cohen's d
cohen_d_result <- cohen.d(novice_ratings, famous_ratings)
cohen_d_value <- cohen_d_result$estimate

cat("Cohen's d for the difference in average book ratings between novice and famous authors:", cohen_d_value)
## Cohen's d for the difference in average book ratings between novice and famous authors: -5.977738

Now that I have my necessary values, I will need to determine if I have enough data to perform a Neyman-Pearson test.

#cohen's d
observed_d <- -5.978

#signnificance
alpha <- 0.01

#power
desired_power <- 0.85

#determine sample size with power test
sample_size <- pwr.t.test(d = observed_d, sig.level = alpha, power = desired_power, 
                          alternative = "two.sided")$n

cat("Required Sample Size for Neyman-Pearson Test:", sample_size, "\n")
## Required Sample Size for Neyman-Pearson Test: 2.733077

There is a required sample size of 2.7 to perform a Neyman-Pearson test, however, I will be performing a two sample t-test since I will be comparing two groups.

#two-sample t-test
t_test_result <- t_test_result <- t.test(novice_ratings, famous_ratings)

print(t_test_result)
## 
##  Welch Two Sample t-test
## 
## data:  novice_ratings and famous_ratings
## t = -31.12, df = 79.723, p-value < 2.2e-16
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -1.138891 -1.001977
## sample estimates:
## mean of x mean of y 
##  3.369000  4.439434

The two sample t-test provided a lot of interesting insight regarding the relationship of book ratings between novice authors and famous authors. The magnitude of the t value suggests that there is a difference between the book ratings means. Further more, the very small p-value suggests that there is evidence against the null hypothesis.

hist_novice <- ggplot(book_sales[book_sales$Author_Rating == "Novice", ], aes(x = Book_average_rating)) +
  geom_histogram(fill = "pink", color = "black", bins = 20) +
  labs(title = "Distribution of Book Ratings for Novice Authors",
       x = "Book Rating",
       y = "Frequency") +
  theme_minimal()

hist_famous <- ggplot(book_sales[book_sales$Author_Rating == "Famous", ], aes(x = Book_average_rating)) +
  geom_histogram(fill = "aquamarine3", color = "black", bins = 20) +
  labs(title = "Distribution of Book Ratings for Famous Authors",
       x = "Book Rating",
       y = "Frequency") +
  theme_minimal()

print(hist_novice)

print(hist_famous)