Week 7
2024-02-27
Week 7 Data Dive
library(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.4 ✔ readr 2.1.4
## ✔ forcats 1.0.0 ✔ stringr 1.5.1
## ✔ ggplot2 3.4.4 ✔ tibble 3.2.1
## ✔ lubridate 1.9.3 ✔ tidyr 1.3.0
## ✔ purrr 1.0.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorslibrary(ggthemes)
library(ggrepel)
library(effsize)
library(pwrss)##
## Attaching package: 'pwrss'
##
## The following object is masked from 'package:stats':
##
## power.t.testlibrary(ggplot2)library(tidyverse)
library(dplyr)
chocolate <- read.csv("~/Desktop/chocolate.csv")
library(readr)
summary(chocolate)## ref company_manufacturer company_location review_date
## Min. : 5 Length:2530 Length:2530 Min. :2006
## 1st Qu.: 802 Class :character Class :character 1st Qu.:2012
## Median :1454 Mode :character Mode :character Median :2015
## Mean :1430 Mean :2014
## 3rd Qu.:2079 3rd Qu.:2018
## Max. :2712 Max. :2021
## country_of_bean_origin specific_bean_origin_or_bar_name cocoa_percent
## Length:2530 Length:2530 Min. :0.4200
## Class :character Class :character 1st Qu.:0.7000
## Mode :character Mode :character Median :0.7000
## Mean :0.7164
## 3rd Qu.:0.7400
## Max. :1.0000
## ingredients most_memorable_characteristics rating
## Length:2530 Length:2530 Min. :1.000
## Class :character Class :character 1st Qu.:3.000
## Mode :character Mode :character Median :3.250
## Mean :3.196
## 3rd Qu.:3.500
## Max. :4.000Null Hypothesis 1: Cocoa Percentage vs Rating
Null Hypothesis (H0): The cocoa percentage does not affect the rating of chocolate bars.
Alternative Hypothesis (H1): There is a significant difference in the average rating of chocolate bars with different cocoa percentages.
Come up with an alpha level, power level, and minimum effect size, and explain why you chose each value
Significance Level (⍺=0.05) , this is the probability of rejecting the null hypothesis when it is true. Power Level(1-𝞫=0.80), this is the probability of rejecting the null hypothesis when it is false. Minimum effect size of 0.2 in rating between different cocoa percentages.
Determine if you have enough data to perform a Neyman-Pearson hypothesis test. If you do, show your sample size calculation, perform the test, and interpret results. If not, explain why there isn’t enough data.
chocolate$rating <- as.numeric(as.character(chocolate$rating))
standard_deviation <- sd(chocolate$rating, na.rm = TRUE)
print(standard_deviation)## [1] 0.4453213#making sure everything is numeric and finding sd of rating
chocolate$rating <- as.numeric(chocolate$rating)
standard_deviation <- sd(chocolate$rating)
# Using the estimated standard deviation in the sample size calculation
alpha <- 0.05
power <- 0.80
effect_size <- 0.2 # for example
z_alpha <- qnorm(1 - alpha/2)
z_beta <- qnorm(power)
# Calculated required sample size
sample_size <- ((z_alpha + z_beta)^2 * standard_deviation^2) / effect_size^2
print(sample_size)## [1] 38.91299#Made two categories for cocoa percent (high and low)
chocolate$cocoa_category <- ifelse(chocolate$cocoa_percent > 0.75, "High", "Low")
chocolate$cocoa_category <- as.factor(chocolate$cocoa_category)
#Made subset
set.seed(123)
subset_data <- chocolate[sample(nrow(chocolate), size = 38), ]
# Performed the t-test on subset data
t_test_result_subset <- t.test(rating ~ cocoa_category, data = subset_data)
# Printed the results
print(t_test_result_subset)##
## Welch Two Sample t-test
##
## data: rating by cocoa_category
## t = -1.1097, df = 2.2494, p-value = 0.3716
## alternative hypothesis: true difference in means between group High and group Low is not equal to 0
## 95 percent confidence interval:
## -1.711650 0.949745
## sample estimates:
## mean in group High mean in group Low
## 2.833333 3.214286Determine if you have enough data to perform a Neyman-Pearson hypothesis test. If you do, show your sample size calculation, perform the test, and interpret results. If not, explain why there isn’t enough data.
I was able to see that I did have enough data to perform a Neyman-Pearson hypothesis test. The sample size calculation was 38 and the Neyman-Pearson hypothesis test provided us with the following significances:
The t-value( -1.1097) indicates how many standard errors the sample mean of rating of the “high cocoa percent” is away from the sample mean of the “low cocoa percent” category. The negative value suggests that the “higher cocoa percentage” category has a mean rating lower to the “low cocoa percent category”.
The p-value (0.3716) is greater than the typical significance level of 0.05 so it suggests that I do not have enough evidence to reject the null hypothesis.
The confidence interval includes 0 meaning that the difference is not statistically significant.
Given the high p-value and the confidence interval I have to FAIL to reject the null hypothesis. This suggests that based on our random 38 row sample we did not receive enough evidence to conclude that there is a difference in rating between a chocolate that has high cocoa percentage versus low cocoa percentage. This is contrary to what I’d assume, that chocolate bars with higher percentage of cocoa results in lower ratings due to a more bitter taste. but again food palettes vary!
Perform a Fisher’s style test for significance on the same hypothesis, and interpret the p-value.
# I created two subsets based on cocoa percentage again
high_cocoa <- chocolate[chocolate$cocoa_percent > 0.75, ]
low_cocoa <- chocolate[chocolate$cocoa_percent <= 0.75, ]
var_test_result <- var.test(high_cocoa$rating, low_cocoa$rating)
print(var_test_result)##
## F test to compare two variances
##
## data: high_cocoa$rating and low_cocoa$rating
## F = 1.3206, num df = 308, denom df = 2220, p-value = 0.0007527
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
## 1.121610 1.571995
## sample estimates:
## ratio of variances
## 1.320619The resulting p-value is 0.0007527 is lower than the chosen significance level of 0.05 which would lead me to reject the null hypothesis, so there is evidence to suggest that the variances of the high cocoa percentage vs low cocoa percentage are significantly different.
Build two visualizations that best illustrate your results, one for each null hypothesis.
high_cocoa <- chocolate[chocolate$cocoa_percent > 0.75, ]
low_cocoa <- chocolate[chocolate$cocoa_percent <= 0.75, ]
boxplot(list(high_cocoa$rating, low_cocoa$rating), names = c("High Cocoa", "Low Cocoa"),
main = "High vs. Low Cocoa Percentage", xlab = "Cocoa Percentage", ylab = "Rating")
Null Hypothesis 2: Rating vs Company Location (France vs Canada)
Null Hypothesis (H0): There is no difference in the average ratings of chocolate bars company locations in the France vs Canada.
Alternative Hypothesis (H1): There is a significant difference in the average ratings of chocolate bars company locations in the France vs Canada.
Come up with an alpha level, power level, and minimum effect size, and explain why you chose each value
Significance Level (⍺=0.05) , this is the probability of rejecting the null hypothesis when it is true. Power Level(1-𝞫=0.80), this is the probability of rejecting the null hypothesis when it is false. Minimum effect size of 0.2 in rating between different cocoa percentages.
Determine if you have enough data to perform a Neyman-Pearson hypothesis test. If you do, show your sample size calculation, perform the test, and interpret results. If not, explain why there isn’t enough data.
company1 <- "U.S.A."
company2 <- "Canada"
subset_company1 <- chocolate[chocolate$company_location == company1, "rating"]
subset_company2 <- chocolate[chocolate$company_location == company2, "rating"]
# Calculate standard deviations
sd_company1 <- sd(subset_company1)
sd_company2 <- sd(subset_company2)
# Set values for alpha, power, and effect size
alpha <- 0.05
power <- 0.80
effect_size <- 0.2
z_alpha <- qnorm(1 - alpha/2)
z_beta <- qnorm(power)
# Calculated sample size
sample_size <- ((z_alpha + z_beta)^2 * (sd_company1^2 + sd_company2^2)) / effect_size^2
print(sample_size)## [1] 69.31874# Checking if we have enough data to perform the test
company1 <- "U.S.A."
company2 <- "Canada"
subset_company1 <- chocolate[chocolate$company_location == company1, ]
subset_company2 <- chocolate[chocolate$company_location == company2, ]
# Checking the size of the subsets
subset_size_company1 <- nrow(subset_company1)
subset_size_company2 <- nrow(subset_company2)
cat("Subset size for U.S.A. manufacturer:", subset_size_company1, "\n")## Subset size for U.S.A. manufacturer: 1136cat("Subset size for Canada manufacturer:", subset_size_company2, "\n")## Subset size for Canada manufacturer: 177company1 <- "U.S.A."
company2 <- "Canada"
subset_company1 <- chocolate[chocolate$company_location == company1, "rating"]
subset_company2 <- chocolate[chocolate$company_location == company2, "rating"]
# calculated sample size
desired_sample_size <- 69
# Sampled the data to calculated sample size
subset_company1 <- sample(subset_company1, size = desired_sample_size, replace = TRUE)
subset_company2 <- sample(subset_company2, size = desired_sample_size, replace = TRUE)
t_test_result <- t.test(subset_company1, subset_company2)
print(t_test_result)##
## Welch Two Sample t-test
##
## data: subset_company1 and subset_company2
## t = -1.4575, df = 136, p-value = 0.1473
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.26471980 0.04008212
## sample estimates:
## mean of x mean of y
## 3.184783 3.297101I was able to see that I did have enough data to perform a Neyman-Pearson hypothesis test. The sample size calculation was 69 and the Neyman-Pearson hypothesis test provided us with the following significances:
The t-value( -1.4575) indicates how many standard errors the sample mean of rating of company location U.S.A. is away from the sample mean of the Canada chocolates. The negative value suggests that the Canada has a mean rating higher than chocolate companies in the USA.
The p-value (0.1473) is greater than the typical significance level of 0.05 so it suggests that I do not have enough evidence to reject the null hypothesis. So I cannot conclude there is a significant difference in average ratings of chocolate from chocolate companies in USA or Canada.
The confidence interval includes 0 meaning that the difference is not statistically significant.
Given the high p-value and the confidence interval I have to FAIL to reject the null hypothesis. This suggests that based on our random 69 row sample we did not receive enough evidence to conclude that there is a difference in rating between chocolate companies in the US versus in Canada. I guess I can be content with my chocolates from here and can save myself a trip to Canada for just chocolate!
Perform a Fisher’s style test for significance on the same hypothesis, and interpret the p-value.
company1 <- "U.S.A."
company2 <- "Canada"
subset_company1 <- chocolate[chocolate$company_location == company1, "rating"]
subset_company2 <- chocolate[chocolate$company_location == company2, "rating"]
var_test_result <- var.test(subset_company1, subset_company2)
print(var_test_result)##
## F test to compare two variances
##
## data: subset_company1 and subset_company2
## F = 1.0392, num df = 1135, denom df = 176, p-value = 0.7598
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
## 0.8219522 1.2894188
## sample estimates:
## ratio of variances
## 1.039185The p-value of 0.7598 is way higher than the significance level there it suggests there is no significant difference in variances between chocolates from companies in the USA or Canada.
Build two visualizations that best illustrate your results, one for each null hypothesis.
company1 <- "U.S.A."
company2 <- "Canada"
subset_company1 <- chocolate[chocolate$company_location == company1, "rating"]
subset_company2 <- chocolate[chocolate$company_location == company2, "rating"]
boxplot(list(subset_company1, subset_company2), names = c(company1, company2),
main = "Company Location vs. Rating", xlab = "Company Location", ylab = "Rating")