Week 4 Data Dive
Jenna Horrall
2024-02-15
Week 4 Data Dive
Analysis of Data
Creating and Comparing Samples
In order to get a general idea how each sample looks, I chose to summarize them. To do so, I put each sample dataframe in a list, and used a for loop to summarize each one.
#defining sample size (about 500)
sample_size <- 500
#five new dataframes with randomly sampled data
df_1 <- book_sales[sample(nrow(book_sales), size = sample_size, replace = TRUE), , drop = FALSE]
df_2 <- book_sales[sample(nrow(book_sales), size = sample_size, replace = TRUE), , drop = FALSE]
df_3 <- book_sales[sample(nrow(book_sales), size = sample_size, replace = TRUE), , drop = FALSE]
df_4 <- book_sales[sample(nrow(book_sales), size = sample_size, replace = TRUE), , drop = FALSE]
df_5 <- book_sales[sample(nrow(book_sales), size = sample_size, replace = TRUE), , drop = FALSE]
#create a list of sample dataframes to loop through
df_list <- list(df_1, df_2, df_3, df_4, df_5)
#empty list to store results
publisher_sales_summary <- list()
#for each df in the df_list
for (i in seq_along(df_list)) {
current_df <- df_list[[i]]
summary_stats <- current_df |>
#group by publisher
group_by(Publisher) |>
#summarize columns
summarize(
total_units_sold = sum(`units sold`),
total_gross_sales = sum(`gross sales`),
total_publisher_revenue = sum(`publisher revenue`)
) |>
#arrange desc by total gross sales
arrange(desc(total_gross_sales))
publisher_sales_summary[[i]] <- summary_stats
}
print(publisher_sales_summary)## [[1]]
## # A tibble: 7 × 4
## Publisher total_units_sold total_gross_sales total_publisher_reve…¹
## <chr> <dbl> <dbl> <dbl>
## 1 Amazon Digital Serv… 3242256 315321. 64772.
## 2 Penguin Group (USA)… 399595 155938. 93563.
## 3 Random House LLC 625767 119544. 71727.
## 4 HarperCollins Publi… 145269 51024. 30615.
## 5 Hachette Book Group 264371 28783. 17270.
## 6 Simon and Schuster … 183340 22055. 13233.
## 7 Macmillan 127017 13727. 8236.
## # ℹ abbreviated name: ¹​total_publisher_revenue
##
## [[2]]
## # A tibble: 7 × 4
## Publisher total_units_sold total_gross_sales total_publisher_reve…¹
## <chr> <dbl> <dbl> <dbl>
## 1 Amazon Digital Serv… 2419216 393557. 84785.
## 2 Penguin Group (USA)… 492521 145924. 87554.
## 3 Random House LLC 360961 139921. 83953.
## 4 Hachette Book Group 385083 101706. 61024.
## 5 HarperCollins Publi… 305084 62229. 37337.
## 6 Simon and Schuster … 244616 38723. 23234.
## 7 Macmillan 155305 32060. 19236.
## # ℹ abbreviated name: ¹​total_publisher_revenue
##
## [[3]]
## # A tibble: 7 × 4
## Publisher total_units_sold total_gross_sales total_publisher_reve…¹
## <chr> <dbl> <dbl> <dbl>
## 1 Amazon Digital Serv… 2529533 382027. 98357.
## 2 Penguin Group (USA)… 406776 145505. 87303.
## 3 Random House LLC 191713 138577. 83146.
## 4 Hachette Book Group 192933 94112. 56467.
## 5 HarperCollins Publi… 347010 90250. 54150.
## 6 Simon and Schuster … 140895 61723. 37034.
## 7 Macmillan 222154 28650. 17190.
## # ℹ abbreviated name: ¹​total_publisher_revenue
##
## [[4]]
## # A tibble: 7 × 4
## Publisher total_units_sold total_gross_sales total_publisher_reve…¹
## <chr> <dbl> <dbl> <dbl>
## 1 Amazon Digital Serv… 2644790 357531. 56497.
## 2 Penguin Group (USA)… 597968 182869. 109722.
## 3 Hachette Book Group 263536 148082. 88849.
## 4 Random House LLC 735772 128171. 76903.
## 5 HarperCollins Publi… 289785 91561. 54937.
## 6 Simon and Schuster … 320456 38679. 23207.
## 7 Macmillan 285105 20309. 12185.
## # ℹ abbreviated name: ¹​total_publisher_revenue
##
## [[5]]
## # A tibble: 7 × 4
## Publisher total_units_sold total_gross_sales total_publisher_reve…¹
## <chr> <dbl> <dbl> <dbl>
## 1 Amazon Digital Serv… 2875285 345715. 59287.
## 2 Hachette Book Group 224600 178674. 107205.
## 3 Penguin Group (USA)… 366816 167319. 100391.
## 4 Random House LLC 552013 144247. 86548.
## 5 HarperCollins Publi… 317670 73060. 43836.
## 6 Simon and Schuster … 208942 37384. 22430.
## 7 Macmillan 91935 20029. 12018.
## # ℹ abbreviated name: ¹​total_publisher_revenueFrom this, we can see a few minor changes between each sampled dataframe. Amazon Digital Services, Inc. is consistently has the highest gross sales across all dataframes and Macmillan and Simon and Schuster Digital Sales Inc. consistently have the lowest gross sales, but spots two through four vary a bit.
I also wanted to get an idea on how book ratings vary between each dataframe.
author_sales_summary <- list()
#for each df in the df_list
for (i in seq_along(df_list)) {
current_df <- df_list[[i]]
summary_stats <- current_df |>
#group by author
group_by(Author) |>
summarize(
total_units_sold = sum(`units sold`),
total_gross_sales = sum(`gross sales`),
total_revenue = sum(`publisher revenue`),
total_ratings_count = sum(Book_ratings_count),
avg_book_rating = mean(Book_average_rating)
) |>
arrange(desc(total_gross_sales))
author_sales_summary[[i]] <- summary_stats
}
print(author_sales_summary)## [[1]]
## # A tibble: 315 × 6
## Author total_units_sold total_gross_sales total_revenue total_ratings_count
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 Laini T… 8990 38278. 22967. 284029
## 2 Frank M… 12500 24875 14925 290534
## 3 Stephen… 160955 24544. 3520. 1289729
## 4 Frances… 3666 23792. 14275. 199872
## 5 Nick Ho… 2268 21389. 12833. 236848
## 6 Sara Sh… 3200 15968 0 156328
## 7 Kate Mo… 12708 15388. 399. 594051
## 8 John Gr… 49506 14987. 1257. 647231
## 9 Michael… 33106 14862. 7744. 404541
## 10 Jodi Pi… 21585 13140. 6392. 677116
## # ℹ 305 more rows
## # ℹ 1 more variable: avg_book_rating <dbl>
##
## [[2]]
## # A tibble: 308 × 6
## Author total_units_sold total_gross_sales total_revenue total_ratings_count
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 Harper … 5500 47795 28677 138669
## 2 Robert … 6532 32595. 0 278822
## 3 Sarah D… 4385 27376. 5647. 331136
## 4 Charlai… 6460 27148. 14863. 713509
## 5 John Be… 3800 26182 15709. 167997
## 6 Nick Ho… 2268 21389. 12833. 236848
## 7 Victori… 4000 19960 0 83354
## 8 J.R.R. … 3876 19341. 11605. 402318
## 9 James P… 10835 18250. 9780. 913644
## 10 Garth N… 882 17622. 10573. 274094
## # ℹ 298 more rows
## # ℹ 1 more variable: avg_book_rating <dbl>
##
## [[3]]
## # A tibble: 313 × 6
## Author total_units_sold total_gross_sales total_revenue total_ratings_count
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 Harper … 11000 95590 57354 277338
## 2 Margery… 6666 43596. 26157. 346938
## 3 David S… 10873 41845. 24750 233489
## 4 Laini T… 13230 38604. 23163. 369775
## 5 Colleen… 9692 27180. 15656. 453164
## 6 Nichola… 13905 23065. 1207. 811775
## 7 Emily G… 2790 22832. 13699. 498807
## 8 Brandon… 7410 20706. 12424. 374141
## 9 Kenneth… 610 20655. 12393. 249224
## 10 Sarah D… 3600 17964 0 179415
## # ℹ 303 more rows
## # ℹ 1 more variable: avg_book_rating <dbl>
##
## [[4]]
## # A tibble: 329 × 6
## Author total_units_sold total_gross_sales total_revenue total_ratings_count
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 Unknown… 7000 34160 20496 155903
## 2 John Gr… 52258 28158. 1823. 772544
## 3 Jeffrey… 8163 27984. 16791. 249800
## 4 Miguel … 2984 26826. 16096. 269862
## 5 John Be… 3800 26182 15709. 167997
## 6 Douglas… 3325 25701. 15420. 439238
## 7 Diana G… 8199 24515. 0 472185
## 8 Margery… 3333 21798. 13079. 173469
## 9 C.S. Le… 3600 21564 12938. 189671
## 10 Victori… 4000 19960 0 83354
## # ℹ 319 more rows
## # ℹ 1 more variable: avg_book_rating <dbl>
##
## [[5]]
## # A tibble: 316 × 6
## Author total_units_sold total_gross_sales total_revenue total_ratings_count
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 David S… 15320 82978. 49787. 383412
## 2 Laini T… 4750 37952. 22772. 198283
## 3 Unknown… 7000 34160 20496 155903
## 4 Robert … 6532 32595. 0 278822
## 5 Jeffrey… 2466 26904. 16142. 159249
## 6 John Be… 3800 26182 15709. 167997
## 7 John Gr… 7091 25479. 838. 379425
## 8 Stephen… 162309 22117. 6453. 1572567
## 9 Margery… 3333 21798. 13079. 173469
## 10 Brandon… 4696 21641. 11678. 442702
## # ℹ 306 more rows
## # ℹ 1 more variable: avg_book_rating <dbl>There is a lot of variation in authors with the highest grossing sales between each dataframe, which I found to be very interesting and opens the door for more evaluation.
#combine the data for visualization
combined_data <- bind_rows(df_list, .id = "sample_number")
# Plot a boxplot
ggplot(combined_data, aes(x = factor(sample_number), y = `gross sales`)) +
geom_boxplot() +
labs(title = "Boxplot Comparison of Total Gross Sales for 5 Samples",
x = "Sample Number",
y = "Total Gross Sales")
Monte Carlo Simulation
I have previously had a lot of focus on gross sales, so in similar fashion I have decided to do a Monte Carlo simulation for the gross sales by author.
#simulation count
simulation_count <- 10
simulation_data <- book_sales
#empty list to store result
simulated_results <- list()
#begin monte carlo
for (i in 1:simulation_count) {
#duplicated data set with random variations
simulated_data <- mutate(simulation_data,
`gross sales` = rlnorm(n(), meanlog = log(mean(`gross sales`)), sdlog = sd(log(`gross sales`))) #log-normal distribution to prevent negative values because they do not make sense in this situation
)
#summary statistics
summary_stats <- simulated_data |>
group_by(Author) |>
summarize(
total_gross_sales = sum(`gross sales`)
) |>
arrange(desc(total_gross_sales))
simulated_results[[i]] <- summary_stats
}
#combine results into one dataframe to help with visualization
combined_results <- bind_rows(simulated_results)
# Plot distribution of total_gross_sales for one author
ggplot(combined_results, aes(x = total_gross_sales)) +
geom_histogram(bins = 30, fill = "aquamarine3", color = "black", alpha = 0.7) +
labs(title = "Distribution of Total Gross Sales (Monte Carlo Simulation)",
x = "Total Gross Sales",
y = "Frequency")
Identifying Anomalies
I have chosen to examine the book sales prices in each of my five samples to find instances where a price is an anomaly in one sample but not another. In order to do this, I started by creating a boxplot.
#combine the data for visualization
combined_data <- bind_rows(df_list, .id = "sample_number")
# Plot a boxplot
ggplot(combined_data, aes(x = factor(sample_number), y = `sale price`)) +
geom_boxplot() +
labs(title = "Boxplot Comparison of Book Sale Price",
x = "Sample Number",
y = "Sale Price")
Sample 5 looks like it contains a few outliers that are possibly not
outliers for sample 4. I decided to create an additonal box plot
highlighting this.
#combine only samples 4 and 5 for visualization
combined_data_4_5 <- bind_rows(df_list[c(4, 5)], .id = "sample_number")
#plot a boxplot
ggplot(combined_data_4_5, aes(x = factor(sample_number), y = `sale price`)) +
geom_boxplot() +
labs(title = "Boxplot Comparison of Book Sale Price",
x = "Sample Number",
y = "Sale Price")
Although the X labels say 1 and 2, the box plots show samples 4 and 5
respectively. Sample 5 (shown as 2 on the box plot) has a few anomalies
that appear as though they would fall between the upper quartile and
maximum for sample 4 (shown as 1 on the box plot). In order to pinpoint
more specific values, I proceeded to create a summaries for the samples
as seen below.
summary_4 <- summary(df_4$`sale price`)
print("Summary statistics for Sample 4:")## [1] "Summary statistics for Sample 4:"print(summary_4)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.990 1.990 3.990 4.956 7.190 33.860summary_5 <- summary(df_5$`sale price`)
print("Summary statistics for Sample 5:")## [1] "Summary statistics for Sample 5:"print(summary_5)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.990 0.990 3.990 4.691 7.500 16.980df_5_sorted <- df_5 |>
arrange(desc(`sale price`)) |>
head(5)
print("Sample 5 top 5 highest sales prices:")## [1] "Sample 5 top 5 highest sales prices:"print(df_5_sorted$`sale price`)## [1] 16.98 15.37 14.99 14.99 14.99df_4_sorted <- df_4 |>
arrange(desc(`sale price`)) |>
head(5)
print("Sample 4 top 5 highest sales prices:")## [1] "Sample 4 top 5 highest sales prices:"print(df_4_sorted$`sale price`)## [1] 33.86 25.89 17.98 17.98 13.78