Nneka’s Project 3 - Books
Nneka Okigbo
2024-05-05
Introduction + Questions to Explore
For my project I will be working with a data set on books and their characteristics such as category, sub-category, name, material, author, rank, rating, rating count, sale price, list price, number of pages, published date, and ISBN number.
This data was collected by a Singaporean Github user named Hui Xiang Chua. They collected this data by scraping data from Amazon’s bookdepository.com. This is raw data on the best selling books as of January 11, 2018.
It honestly took me a few days to clean this data because I kept getting frustrated (lol). But to clean this data I used a few new functions that I never have such as grepl(), gsub(), and trimws(). I used grepl to find values that contained a certain string and place them in a certain category. I did this for material. I used gsub to get rid of unwanted character strings in numbers so I could turn them numeric values (using as.numeric) and use this for statistical analysis. I did this for columns like prices, pages, rating, and rating count. I used trimws() for the authors because there was a lot of new line and tab characters embedded in the data set. Trimws() got rid of pesky white spaces! I also changed created new columns using the mutate function in dplyr. Also in dplyr, I used select() To be honest, the actual topic of the data set (books), does not have much meaning to me. I chose this data set because I thought it would be great practice cleaning and discovering new functions. I want to become a data scientist and I hear them say that most of data science is cleaning so I thought this data set would be a brilliant start for me. It was not too difficult but also allowed me to challenge myself which I liked.
Some questions I’m looking to explore is what category of books produces the best rated books, do hardback or paperback books cost have a higher sale price, how does number of pages relate to rating, and does the demand for books relate to its rating?
Importing Libraries and Data sets
library(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.3 ✔ readr 2.1.4
## ✔ forcats 1.0.0 ✔ stringr 1.5.0
## ✔ ggplot2 3.4.3 ✔ tibble 3.2.1
## ✔ lubridate 1.9.2 ✔ tidyr 1.3.0
## ✔ purrr 1.0.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorslibrary(plotly)##
## Attaching package: 'plotly'
##
## The following object is masked from 'package:ggplot2':
##
## last_plot
##
## The following object is masked from 'package:stats':
##
## filter
##
## The following object is masked from 'package:graphics':
##
## layoutlibrary(dplyr)
library(highcharter)## Registered S3 method overwritten by 'quantmod':
## method from
## as.zoo.data.frame zoolibrary(stringr)
library(datasets)
library(ggfortify)
library(htmltools)
library(GGally)## Registered S3 method overwritten by 'GGally':
## method from
## +.gg ggplot2library(DataExplorer)
library("RColorBrewer")
setwd("/Users/nnokigbo/Downloads")
booksraw <- read_csv("bookdepo.csv", locale = locale(encoding = "Latin1"))## New names:
## Rows: 1050 Columns: 14
## ── Column specification
## ──────────────────────────────────────────────────────── Delimiter: "," chr
## (10): name, material, author, maincat, subcat, ratingcount, saleprice, l... dbl
## (3): ...1, rating, isbn13 num (1): rank
## ℹ Use `spec()` to retrieve the full column specification for this data. ℹ
## Specify the column types or set `show_col_types = FALSE` to quiet this message.
## • `` -> `...1`head(booksraw)## # A tibble: 6 × 14
## ...1 name material author rank maincat subcat rating ratingcount saleprice
## <dbl> <chr> <chr> <chr> <dbl> <chr> <chr> <dbl> <chr> <chr>
## 1 0 Fire … "Paperb… "By (… 1 Politi… <NA> 4 (1,929 rat… S$19.80
## 2 1 Fire … "Hardba… "By (… 2 Politi… <NA> 4 (1,929 rat… S$25.19
## 3 2 The B… "Paperb… "By (… 3 Invest… Perso… 4.59 (2,625 rat… S$26.33
## 4 3 Fire … "Hardba… "By (… 4 Histor… Polit… 4 (1,929 rat… S$34.19
## 5 4 The S… "Hardba… "By (… 5 Advice… Asser… 4.02 (64,910 ra… S$21.47
## 6 5 5 Ing… "Hardba… "By (… 6 Celebr… Quick… 4.27 (503 ratin… S$39.27
## # ℹ 4 more variables: listprice <chr>, numofpages <chr>, datepub <chr>,
## # isbn13 <dbl>Cleaning Data
This is a raw web scraped data set so it needs some cleaning. First I will start by creating a new modified data set so that my original data set remains dirty and unclean. Also I will be getting rid of removing extra words or characters from variables and turning some variables into numeric
# Creating new dataset to modify
books <- booksraw
## Removing unnecessary characters and turning variables into numeric.
books$ratingcount<-str_replace_all(books$ratingcount, " ratings by Goodreads", "")
books$ratingcount <- gsub("\\(","", books$ratingcount)
books$ratingcount <- gsub("\\)","", books$ratingcount)
books$ratingcount <- gsub(",","", books$ratingcount) |>
as.numeric(books$ratingcount)## Warning: NAs introduced by coercionbooks$saleprice <- gsub("S\\$", "", books$saleprice) |>
as.numeric(books$saleprice)
books$listprice <- gsub("S\\$", "", books$listprice) |>
as.numeric(books$listprice)
books$numofpages <- gsub(" pages", "", books$numofpages) |>
as.numeric(books$numofpages)
# Turning date published into a readable format
books$datepub <- as.Date(books$datepub,format='%d %B %Y')Since I will be exploring the material of the books, it’s important to clean it since it is in multiple different formats. For this, I will separate materials into 3 categories: Paperback, Hardback, and Other. Paperback and Hardback are the most prevalent material types and all other material types will go under the Other category.
books <- books |>
mutate(material = case_when((grepl("Paperback", books$material) == FALSE & (grepl("Hardback", books$material) == FALSE)) ~ "Other",
(grepl("Paperback", books$material) == TRUE) ~ "Paperback",
(grepl("Hardback", books$material) == TRUE) ~ "Hardback"))Although I will not be exploring the individual authors of each book, it’s important to put it in a format that’s easy to read if anyone were to want to look back at my work.
books <- books |>
relocate(author, .before = ...1)
books$author <- gsub("By \\(author\\)", "", books$author)
books$author <- gsub("\r", "", books$author)
books$author <- gsub("\n", "", books$author)
books$author <- gsub(",", "", books$author)
books$author <- gsub("\\h\\v]", "", books$author)
books$author <- trimws(books$author, whitespace = "[\\h\\v]")
books## # A tibble: 1,050 × 14
## author ...1 name material rank maincat subcat rating ratingcount saleprice
## <chr> <dbl> <chr> <chr> <dbl> <chr> <chr> <dbl> <dbl> <dbl>
## 1 Micha… 0 Fire… Paperba… 1 Politi… <NA> 4 1929 19.8
## 2 Micha… 1 Fire… Hardback 2 Politi… <NA> 4 1929 25.2
## 3 Scott… 2 The … Paperba… 3 Invest… Perso… 4.59 2625 26.3
## 4 Micha… 3 Fire… Hardback 4 Histor… Polit… 4 1929 34.2
## 5 Mark … 4 The … Hardback 5 Advice… Asser… 4.02 64910 21.5
## 6 Jamie… 5 5 In… Hardback 6 Celebr… Quick… 4.27 503 39.3
## 7 Rupi … 6 Milk… Paperba… 7 Poetry… Poetr… 4.2 123806 14.2
## 8 Yuval… 7 Sapi… Paperba… 8 Genera… Class… 4.43 89413 14.2
## 9 Elena… 8 Good… Hardback 9 Biogra… Gener… 4.46 4562 29.2
## 10 Dr. P… 9 The … Paperba… 11 Coping… Popul… 3.81 814 8.57
## # ℹ 1,040 more rows
## # ℹ 4 more variables: listprice <dbl>, numofpages <dbl>, datepub <date>,
## # isbn13 <dbl>Now that I have cleaned up some variables and made them easier to read, I will now get rid of columns I don’t need and reorganizing the columns. I will be getting rid of sub-category since a large chunk of that column in Na’s. And I will also be getting rid of ISBN because it is useless unless someone would want to search up the book.
# Getting rid of columns I don't need
books <- books |>
select(-c(subcat, isbn13))Next, I wanted to create a new column called demand. The way this demand is determined is by subtracting the list price from the sale price. The higher the sale price is compared to the list price, the more popular it is. This means the book was in a high demand and therefore more popular. So the higher the demand number is, the higher the demand for the book is.
# Creating new demand column
books <- books |>
mutate(demand = saleprice - listprice) |>
mutate(across(c('demand'), round, 2))## Warning: There was 1 warning in `mutate()`.
## ℹ In argument: `across(c("demand"), round, 2)`.
## Caused by warning:
## ! The `...` argument of `across()` is deprecated as of dplyr 1.1.0.
## Supply arguments directly to `.fns` through an anonymous function instead.
##
## # Previously
## across(a:b, mean, na.rm = TRUE)
##
## # Now
## across(a:b, \(x) mean(x, na.rm = TRUE))# Checking to see if demand is ever positive
books |>
select(demand) |>
filter(demand >= 0)## # A tibble: 0 × 1
## # ℹ 1 variable: demand <dbl>Interesting, we can see that all the values of demand are below zero meaning that they were sold at a lesser price than they were listed.
Linear Regression Comparing Rating and Demand
LM Plot
I would like to see how demand and rating are compared to each other. Let’s look in a scatter plot
books |>
filter(demand > -30) |>
ggplot(aes(x = rating, y = demand)) +
geom_point() +
geom_smooth (method = "lm", col = "red")## `geom_smooth()` using formula = 'y ~ x'## Warning: Removed 40 rows containing non-finite values (`stat_smooth()`).## Warning: Removed 40 rows containing missing values (`geom_point()`).
There seems to be a bit of a relationship. Let’s make a linear regression model to see the exact relationship.
LM Model
lmRating = lm(rating ~ demand, data = books) #Create the linear regression
summary(lmRating) #Review the results##
## Call:
## lm(formula = rating ~ demand, data = books)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.20299 -0.16718 0.01765 0.18615 0.83128
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.164560 0.014960 278.388 <2e-16 ***
## demand -0.003127 0.001903 -1.643 0.101
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2951 on 927 degrees of freedom
## (121 observations deleted due to missingness)
## Multiple R-squared: 0.002903, Adjusted R-squared: 0.001827
## F-statistic: 2.699 on 1 and 927 DF, p-value: 0.1008LM Equation
The equation to the linear regression model is rating = -0.003127(demand) + 4.164.
LM Analysis
According to the linear model, for everyone 1 increase in demand, rating decreases by 0.003127. However, this is not a good model. The p-value for demand is 0.101 and the p-value for the model is 0.1008 which is bad. A p-value of 0.05 or less is more desirable. Another indicator that this is a bad model is the Adjusted R-Squared. The Adjusted R-Squared is 0.001827. This means that 0.18% of the variation in the observations may be explained by the model. This also means that 99.82% of the variation in the data is likely not explained by this model. Demand is not an accurate explanatory variable for rating and would better be placed with other variables.
Diagnostic Plots
Diagnostic Plot
# Checking diagnostic plots
autoplot(lmRating, 1:4, nrow=2, ncol=2)
Diagnostic Analysis
In the residuals plot, you can see that there is a pattern so it indicated the model would not be appropriate
The QQ plot shows that the distribution in relatively even and there are not too many outrageous outliers.
The Scale-Location looks very interesting and I am unsure how to analyze it.
The Cook’s distance plot shows that very few outliers have high leverage in this model.
First plot: Plotting Book Materials
Figuring which book material is the cheapest
Before we plot the materials and how prevalent they are, let’s see how the materials vary in price. This could be useful in examining why there are more of one material than another. According to writer Max Florschutz, prices of books are rising but paperback is still the cheapest option. Let’s see if the data agrees.
# Dataframe for paperback
books_paperback <- books |>
filter(material == "Paperback") |>
select(saleprice) |>
drop_na()
# Dataframe for hardback
books_hardback <- books |>
filter(material == "Hardback") |>
select(saleprice) |>
drop_na()
# Dataframe for other
books_other <- books |>
filter(material == "Other") |>
select(saleprice) |>
drop_na()
# Finding average saleprice of all
summarise(books_paperback, paper_mean = mean(saleprice))## # A tibble: 1 × 1
## paper_mean
## <dbl>
## 1 15.6summarise(books_hardback, hard_mean = mean(saleprice))## # A tibble: 1 × 1
## hard_mean
## <dbl>
## 1 27.3summarise(books_other, other_mean = mean(saleprice))## # A tibble: 1 × 1
## other_mean
## <dbl>
## 1 16.3Interesting, we can see that the paperback books seem to have the lowest sale price at a cost of $15.60 while hardback has an average cost of $27.30 and others have an average of $16.31.
Now that we have a bit of information on the materials. Let’s plot it.
I want to see if I like the idea of using something like a bar graph.
ggplot(books, aes(x = material)) +
geom_bar()
I like it but I want to make an item plot to see the proportions. Let’s first create a separate data set with the information to work through high chart.
# Creating a dataset for plot
booksdf4hc <- books |>
select(material) |>
count(material)
booksdf4hc## # A tibble: 3 × 2
## material n
## <chr> <int>
## 1 Hardback 242
## 2 Other 69
## 3 Paperback 739Final 1st Plot
# Creating colors for chart
colors = c("#9A6EFA", "#E8594F", "#F453FA" )
# Creating plot1
plot1 <- hchart(
booksdf4hc,
"item",
hcaes(
name = material,
y = n,
color = colors
),
name = "Books",
showInLegend = TRUE,
size = "150%",
center = list("50%", "75%"),
startAngle = -100,
endAngle = 100
) |>
hc_title(text = "Counts of Books with Specific Materials") |>
hc_legend(labelFormat = '{name} <span style="opacity: 0.4">{y}</span>') |>
hc_add_theme(hc_theme_ffx()) |>
hc_caption(text = "Source: Github - Hui Xiang Chua Books Dataset") |>
hc_subtitle(text = "Average Sale Price for Hardback: $27.30 Average Sale Price for Paperback: $15.60 Average Sale Price for Other Materials: $16.31")
plot1Plot 1 analysis
This plot shows the counts of books on the best selling list that are described via their material. The option are Hardback, Paperback, and Other. The other option includes materials such as Diary, Spiral, Notebook, Cards, Calendar and etc. This plot shows that of all the books, most books were paperback, with the second being hardback, and other being a small proportion of the material of the best selling books. The fact that paperback is the most popular option could be due to several reasons. Paperback is typically cheaper than hardback and other materials. While other materials have an average sale price of $16.31 and hardback has an average sale price of $27.30, paperback has an average sale price of $15.60. Another reason could be because hardback is typically used for decoration purposes while people prefer paperback for reading and annotating. This plot shows the vast difference in sale price and what type of material is preferred by book readers.
Second Plot: Plotting Rank and Number of Pages
Exploring Rank and Number of Pages
When I was growing up, it was always common that the best books had the most pages such as Percy Jackson, Land of Stories, and Harry Potter. I wonder if other people thought the same. Let’s quickly plot it to see.
ggplot(books, aes(x = rating, y = numofpages)) +
geom_point()## Warning: Removed 53 rows containing missing values (`geom_point()`).
Hm. There seems to be a slight correlation. I would like to use this as my final visualization
Final 2nd plot
plot2 <- books |>
filter(numofpages < 1700) |>
hchart('point', hcaes(x = rating, y = numofpages, group = material)) |>
hc_tooltip(useHTML = TRUE,
headerFormat = "{series.name} -
Rating: {point.x} stars -
{point.y} pages", pointFormat = "") |>
hc_title(text = "Relationship between Number of Pages and Rating") |>
hc_colors(c("#EB37D2", "#E8E32E", "#001BE5")) |>
hc_xAxis(title = list(text = "Rating"),
labels = list(format = "{value} stars")) |>
hc_add_theme(hc_theme_bloom()) |>
hc_caption(text = "Source: Github - Hui Xiang Chua Books Dataset") |>
hc_yAxis(title = list(text = "Pages"),
labels = list(format = "{value}"))
plot2Plot 2 analysis
This plot shows the relationship between number of pages in a book and rating. I wanted to explore this because as I grew up, all the books with a lot of pages were most favored. I wanted to see if that was the same here. After plotting this with high charter, I do not believe that is the case. There is a slight increase in rating as pages increase, however, I do not think it is big enough to say that books with more pages are better. Although there are a few outliers were pages increase significantly as rating increases but I do not believe it is significant. It is interesting to see that books with other materials generally have a small amount of pages around all ratings. Also, paperback and hardback have about the same rating-number of pages proportion.
Last plot: Plotting Main Categories
Exploring the Main Category variable
Lastly, I want to know what category of books produces the best rated books. First I want see which categories we are working with.
unique(books$maincat)## [1] "Politics & Government"
## [2] "Investment & Securities"
## [3] "History Of The Americas"
## [4] "Advice On Careers & Achieving Success"
## [5] "Celebrity Chefs"
## [6] "Poetry Books"
## [7] "General & World History"
## [8] "Biography"
## [9] "Coping With Stress"
## [10] "Children's Literature Studies: General"
## [11] "Poetry By Individual Poets"
## [12] "Memoirs"
## [13] "Contemporary Fiction"
## [14] "Dietetics & Nutrition"
## [15] "21st Century History: From C 2000 -"
## [16] "True Story Books"
## [17] "Social, Group Or Collective Psychology"
## [18] "Cognition & Cognitive Psychology"
## [19] "Biography: Historical, Political & Military"
## [20] "Nonfiction History Books for Kids"
## [21] "Professional Interior Design"
## [22] "Picture Books"
## [23] "Business Innovation"
## [24] "Comparative Religion"
## [25] "Business Strategy"
## [26] "Astronomy, Space & Time"
## [27] "Space Opera"
## [28] "Popular Psychology"
## [29] "Children's Fiction"
## [30] "Personal Development"
## [31] "Diabetes"
## [32] "Baby Books"
## [33] "Autobiography: General"
## [34] "Miscellaneous Items"
## [35] "Biography: General"
## [36] "Activity Books"
## [37] "Physiological & Neuro-psychology, Biopsychology"
## [38] "Popular Science"
## [39] "Psychology"
## [40] "Popular Philosophy"
## [41] "Management: Leadership & Motivation"
## [42] "Endocrinology"
## [43] "Fiction Books"
## [44] "Handicrafts"
## [45] "Social & Cultural History"
## [46] "Storybooks"
## [47] "Advice On Parenting"
## [48] "Encyclopaedias & Reference Works"
## [49] "Finance"
## [50] "Business & Management"
## [51] "Women's Health"
## [52] "Self, Ego, Identity, Personality"
## [53] "Psychology: Emotions"
## [54] "Diets & Dieting"
## [55] "Stationery"
## [56] "Geopolitics"
## [57] "Oriental & Indian Philosophy"
## [58] "Personal Finance"
## [59] "Indigenous Peoples"
## [60] "Exhibition Catalogues & Specific Collections"
## [61] "Holocaust"
## [62] "Children's General Story Books"
## [63] "Classic Books for Children"
## [64] "Health Books"
## [65] "Fitness & Diet"
## [66] "Islamic Story & History Books"
## [67] "Self-Esteem"
## [68] "Christian Aspects Of Sexuality, Gender & Relationships"
## [69] "Literary Studies: General"
## [70] "Literary Essays"
## [71] "Child & Developmental Psychology"
## [72] "Abnormal Psychology"
## [73] "Complementary Therapies, Healing & Health"
## [74] "Adventure Books"
## [75] "Vegetarian Cookery"
## [76] "Autobiography: Business & Industry"
## [77] "Thriller Books for Children"
## [78] "Biography: Arts & Entertainment"
## [79] "Dance & Other Performing Arts"
## [80] "Historical Fiction"
## [81] "Guidebooks"
## [82] "Individual Actors & Performers"
## [83] "Romance Books for Teenagers"
## [84] "British & Irish History"
## [85] "Early Learning"
## [86] "Crime"
## [87] "Thriller Books"
## [88] "Animal Husbandry"
## [89] "Classic Books & Novels"
## [90] "Fantasy Books for Kids"
## [91] "Music"
## [92] "Science Fiction"
## [93] "Espionage"
## [94] "Public Health & Preventive Medicine"
## [95] "Electronic & Video Art"
## [96] "Graphic Novels: Literary & Memoirs"
## [97] "Adventure Books for Kids"
## [98] "Mind, Body & Spirit"
## [99] "Decision Theory: General"
## [100] "Comic Book & Cartoon Art"
## [101] "Child Care & Upbringing"
## [102] "Fashion & Textiles: Design"
## [103] "Buddhist Worship, Rites & Ceremonies"
## [104] "Giving Up Smoking"
## [105] "Social & Political Philosophy"
## [106] "Family & Relationships"
## [107] "Funny Books for Kids"
## [108] "Religious Books"
## [109] "Christian Spirituality & Religious Experience"
## [110] "Individual Artists, Art Monographs"
## [111] "First Experiences"
## [112] "ABC"
## [113] "Coping With Death & Bereavement"
## [114] "Mind, Body, Spirit: Thought & Practice"
## [115] "Health & Wholefood Cookery"
## [116] "Philosophy"
## [117] "Economics"
## [118] "Trees, Wildflowers & Plants"
## [119] "Feminism & Feminist Theory"
## [120] "Fantasy"
## [121] "Management & Management Techniques"
## [122] "Family"
## [123] "Characters"
## [124] "Biography: Royalty"
## [125] "Films, Cinema"
## [126] "Plays, Playscripts"
## [127] "Quick & Easy Cooking"
## [128] "Tibetan Buddhism"
## [129] "Macroeconomics"
## [130] "Literary Studies: From C 1900 -"
## [131] "Art History"
## [132] "Biography: Business & Industry"
## [133] "Track & Field Sports, Athletics"
## [134] "Coping With Illness"
## [135] "Clinical & Internal Medicine"
## [136] "Occupational Therapy"
## [137] "Diaries, Letters & Journals"
## [138] "Poetry Anthologies (various Poets)"
## [139] "Christian Books"
## [140] "Monetary Economics"
## [141] "Classical History / Classical Civilisation"
## [142] "People & Places"
## [143] "Popular Medicine & Health"
## [144] "Epidemiology & Medical Statistics"
## [145] "Other Performing Arts"
## [146] "True Crime Biographies"
## [147] "Home"
## [148] "Buddhist Books"
## [149] "Art"
## [150] "Poetry & Drama"
## [151] "European History"
## [152] "Computer Programming / Software Development"
## [153] "Autobiography: Sport"
## [154] "Yearbooks, Annuals, Almanacs"
## [155] "Reference Works"
## [156] "Sci Fi Books for Kids"
## [157] "Zen Buddhism"
## [158] "Shakespeare Plays"
## [159] "Human Biology"
## [160] "Autobiography: Science, Technology & Engineering"
## [161] "Risk Assessment"
## [162] "Romance"
## [163] "Gynaecology & Obstetrics"
## [164] "Spirituality & Religious Experience"
## [165] "Exercise & Workout Books"
## [166] "Internet Guides & Online Services"
## [167] "General Cookery"
## [168] "Fortune-telling By Cards (cartomancy)"
## [169] "Political Ideologies"
## [170] "National & Regional Cuisine"
## [171] "Communication Studies"
## [172] "Rock & Pop Music"
## [173] "Knitting & Crochet"
## [174] "Horror"
## [175] "Graphic Novels, Anime & Manga"
## [176] "Gender Studies: Women"
## [177] "Political Leaders & Leadership"
## [178] "Language: Reference & General"
## [179] "Social & Cultural Anthropology"
## [180] "Keyboard Instruments"
## [181] "Individual Photographers"
## [182] "Middle Eastern History"
## [183] "Graphic Novels: Manga"
## [184] "Time Management"
## [185] "Religion: General"
## [186] "Assertiveness, Motivation & Self-esteem"
## [187] "Prose: Non-fiction"
## [188] "Organizational Theory & Behaviour"
## [189] "Animals & Nature In Art (still Life, Landscapes & Seascapes, Etc)"
## [190] "Poverty & Unemployment"
## [191] "Language Teaching & Learning Material & Coursework"
## [192] "Watercolours"
## [193] "Language Self-study Texts"
## [194] "Society & Culture: General"
## [195] "Graphic Art"
## [196] "Psychotherapy"
## [197] "Military History"
## [198] "Art Theory"
## [199] "Hand Writing"
## [200] "Religious Life & Practice"
## [201] "Walking, Hiking, Trekking"
## [202] "Drawing"
## [203] "Children's Books"
## [204] "Art History: From c 1960"
## [205] "Food & Drink"
## [206] "Mysteries & The Supernatural (Children's/YA)"
## [207] "Cookery Dishes & Courses"
## [208] "Clinical Psychology"
## [209] "Film Scripts & Screenplays"
## [210] "Mystery Books for Kids"
## [211] "Coping With Anxiety & Phobias"
## [212] "Television & Film"
## [213] "Business Ethics"
## [214] "Sociology"
## [215] "Humour Collections & Anthologies"
## [216] "General Encyclopaedias"
## [217] "Flower Arranging & Floral Crafts"
## [218] "Biography: Science, Technology & Engineering"
## [219] "Graphic Novels"
## [220] "Dogs As Pets"
## [221] "Art Books"
## [222] "Autobiography: Religious & Spiritual"
## [223] "Dictionaries Of Quotations"
## [224] "Novelty Books"
## [225] "Entrepreneurship"
## [226] "Central Government Policies"
## [227] "Sales & Marketing"
## [228] "Science: General Issues"
## [229] "Hobby and Gaming Books"
## [230] "Gender Studies, Gender Groups"
## [231] "Social Work"
## [232] "Children's Music Books"
## [233] "Creative Writing & Creative Writing Guides"
## [234] "Popular Beliefs & Controversial Knowledge"
## [235] "Sex & Sexuality, Sex Manuals"
## [236] "Anthologies (non-poetry)"
## [237] "Translation & Interpretation"
## [238] "Television"
## [239] "Biography: Sport"
## [240] "Occupational & Industrial Psychology"
## [241] "Astrology"
## [242] "Business Studies: General"
## [243] "Development Economics"
## [244] "Regional Studies"
## [245] "Nature & Existence Of God"
## [246] "Ethics & Moral Philosophy"
## [247] "Coping With Personal Problems"
## [248] "Teaching Of A Specific Subject"
## [249] "Society & Social Issues"
## [250] "Hindu Books"
## [251] "World Atlases / World Maps"
## [252] "Horror & Ghost"
## [253] "Decorative Arts"
## [254] "Food & Beverage Technology"
## [255] "Bilingual & Multilingual Dictionaries"
## [256] "Computer Science"
## [257] "Information Theory"
## [258] "Graphical & Digital Media Applications"
## [259] "Personnel & Human Resources Management"
## [260] "Teaching Skills & Techniques"
## [261] "Linguistics"
## [262] "Needlework & Fabric Crafts"
## [263] "Ethical Issues: Scientific & Technological Developments"
## [264] "Crafts & Hobbies"Figuring out which categories to use
Wow, we have a lot of categories - over 200! It would be difficult to separate them all into groups. Some categories may be used only once. Let’s examine the counts of each category to see which ones are most prevalent and important.
books |>
group_by(maincat) |>
count(maincat) |>
arrange(desc(n)) ## # A tibble: 264 × 2
## # Groups: maincat [264]
## maincat n
## <chr> <int>
## 1 Contemporary Fiction 141
## 2 Children's Fiction 41
## 3 Picture Books 30
## 4 Fantasy 29
## 5 Children's General Story Books 24
## 6 Biography: General 21
## 7 Science Fiction 21
## 8 Memoirs 17
## 9 Storybooks 16
## 10 Children's Literature Studies: General 15
## # ℹ 254 more rowsWe can see that a lot of categories have only one or two books associated with them. This could skew the data and make it seem like a category has a high rating when it is really just the book. Because of this, let’s filter the data to only include categories with at least nine books.
# Findind which categories to put in final visualization
category_counts <- books |>
group_by(maincat) |>
count(maincat) |>
arrange(desc(n)) |>
filter(n > 8)Now that we have our categories that we will be looking at, let’s plot them and see which one’s have the best rating
Final 3rd Plot
plot3 <- books |>
group_by(maincat) |>
summarise(avg = mean(rating, na.rm = TRUE)) |>
# Filtering our categories with less than nine books
filter(maincat %in% c("Advice On Careers & Achieving Success", "Diets & Dieting", "Graphic Novels, Anime & Manga", "Health Books", "Management: Leadership & Motivation", "Personal Development", "Crime", "Popular Psychology", "Baby Books", "Miscellaneous Items", "Activity Books", "Adventure Books for Kids", "Children's Literature Studies: General", "Psychology", "Storybooks", "Memoirs", "Biography: General", "Science Fiction", "Children's General Story Books", "Fantasy", "Picture Books", "Children's Fiction", "Contemporary Fiction")) |>
mutate(across(c('avg'), round, 2)) |>
hchart(type = "treemap", hcaes(x = maincat, value = avg, color = avg, colorByPoint = TRUE)) |>
hc_colorAxis(
maxColor = "#F227BD",
minColor = "#FFFFFF") |>
hc_add_theme(hc_theme_bloom()) |>
hc_title(text = "Category Ratings") |>
hc_subtitle(text = "Shaded by Average Rating on a Scale 0-5 stars") |>
hc_caption(text = "Source: Github - Hui Xiang Chua Books Dataset")
plot3This plot is a tree map of the category ratings. The size of the squares are based on the average rating of that category. The bigger the square, the higher the rating. Also, the darker the square, the higher the rating. It is seen that activity books have an average rating of 4.49 stars. This could be because activity books can produce real results like coloring or even fitness. Since the results of the book are more likely to be seen, people may give it a higher rating. Oppositely, personal development has the lowest average rating of 3.88. Although this is not low it could be better. This could be due to people varying greatly. According to Jacob Brogan from the Washington Post, it’s important to pick a self-help book that works for you. So what may work for one person’s personal development may not work for another. So peolpe would think it would not work and rate it poorly.
For this visualization, I used high chart to get the visualization. The tooltip shows the name of the category and average rating.
Essay Questions
#1: Background Research
The two main things I was most excited about with this topic was covering material and category ratings. When I was younger I always loved hardback fiction books. However, I never bought them, I always borrowed from the library. As Max Florschutz from maxonwriting talks about the increase in prices in books, he talks about how paperback books have always remained cheapest. This is because of the cost that comes to produce it and the lack of durability. With more stability comes more expensive prices which is shown in my project. The average hardcover book was more the $10 more than the average paperback book. This is insane but makes sense. Hardcover is more durable.
Also, I covered category ratings. It was interested to figure out how self-help books had the least rating. However, it makes sense. Self-help books are universal but truly cannot help everyone which could contribute to the low ratings according to Jacob Brogan from the Washington Post.
Books have always been an insane staple in society. However recently, a lot has been online and poeple can simply pirate books. This takes away appreciation and money from the authors who have worked so hard to produce their work. This crisis gives less incentive for writers to publish since they would get little commission. Unfortunately this is an ongoing problem in the writing community which I think is interesting.
2. Visualization Surprises + Could’ve done better.
The first visualization represents the proportion of paperback, hardback, and other materials on the best selling list. This was not surprising as paper had the highest count because it is cheaper.
The second visualization represents the relationship between number of pages and rating. This surprised me that there was very little relationship because of what I grew up thinking.
The third visualization represents the average rating among categories. As states above, I was shocked that self-help books had the least rating because they are supposed to be helping people, however they cannot help everyone.
One thing I wish I could’ve done more of in this project is incorporate dates in my visualizations. However it was difficult because the dates were not in a linear fashion. Other than that, I am proud of my projects and my visualizations and am excited to share with the class.