For this assignment we are to select three books on a single subject of our choosing, with at least one authored by multiple authors. For each book include the title, authors, and two or three other attributes like publication year, publisher, ISBN, genre, etc., and store the book’s information as both HTML and JSON file formats.
John Grisham is one of my favorite authors. A former attorney, he has been writing legal thrillers and crime dramas since 1989, beginning with his New York Times bestselling debut novel The Firm. Since then, many of his novels have landed on the New York Times best seller list. Using data from Goodreads.com, I gathered information on three of his works to satisfy the requirements of this assignment. They are:
Pelican Brief
The Widow
Framed: Astonishing True Stories of Wrongful Conditions, co-authored with Jim MCloskey
Using this selection, I created an HTML and JSON file, each with the following fields:
author 1
author 2
publication year
genre
number of pages
type (hardcover or paperback)
ratings
Next, I will write R code to load both files into R separate data frames for analysis and comparison.
#These packages were needed for this assignment,
#install.packages('rvest') - for loading html files
#install.packages('XML') for loading xml files
#install.packages('jsonlite') for loading json files
library(dplyr)
Attaching package: 'dplyr'The following objects are masked from 'package:stats':
filter, lagThe following objects are masked from 'package:base':
intersect, setdiff, setequal, unionlibrary(rvest)
library(XML)
library(jsonlite)
library(magrittr)
library(httr)gitHtml <- "https://raw.githubusercontent.com/carolc57/DATA607/refs/heads/main/booklist.html"
booklisthtml <- read_html(gitHtml) #Read the HTML page
tables <- html_table(booklisthtml, fill = TRUE) # Extract all tables from the page
tables # View the tables found[[1]]
# A tibble: 3 × 8
title author_1 author_2 genre pub_year num_pages type ratings
<chr> <chr> <chr> <chr> <int> <int> <chr> <dbl>
1 Framed: Astonishing … John Gr… "Jim Mc… Nonf… 2024 423 hard… 4.11
2 The Pelican Brief John Gr… "" Fict… 1992 386 hard… 4.05
3 The Widow John Gr… "" Myst… 2025 417 pape… 4.13gitJson <- "https://raw.githubusercontent.com/carolc57/DATA607/refs/heads/main/booklist.json"
booklistjson <- fromJSON(gitJson, flatten = TRUE)
str(booklistjson) #check structure'data.frame': 3 obs. of 8 variables:
$ title : chr "Framed: Astonishing True Stories of Wrongful Convictions" "The Pelican Brief" "The Widow"
$ author_1: chr "John Grisham" "John Grisham" "John Grisham"
$ author_2: chr "Jim McCloskey" NA NA
$ genre : chr "Nonfiction" "Fiction, Legal Thriller, Crime" "Mystery, Fiction, Legal Thriller, Crime"
$ pub_year: chr "2024" "1992" "2025"
$ num_pgs : int 423 386 417
$ type : chr "hardcover,paperback" "hardcover" "paperback"
$ ratings : chr "4.11" "4.05" "4.13"head(booklistjson) title author_1
1 Framed: Astonishing True Stories of Wrongful Convictions John Grisham
2 The Pelican Brief John Grisham
3 The Widow John Grisham
author_2 genre pub_year num_pgs
1 Jim McCloskey Nonfiction 2024 423
2 <NA> Fiction, Legal Thriller, Crime 1992 386
3 <NA> Mystery, Fiction, Legal Thriller, Crime 2025 417
type ratings
1 hardcover,paperback 4.11
2 hardcover 4.05
3 paperback 4.13My html file “tables” came in as type ‘tibble’ whereas the “booklinstjson” file came in as type data.frame therefore I will convert the “tables” into a data frame before making any comparison.
#convert to data frame
booklisthtml_df <- as.data.frame(tables) # Save the first table as a data frame
str(booklisthtml_df) # check structure 'data.frame': 3 obs. of 8 variables:
$ title : chr "Framed: Astonishing True Stories of Wrongful Convictions" "The Pelican Brief" "The Widow"
$ author_1 : chr "John Grisham" "John Grisham" "John Grisham"
$ author_2 : chr "Jim McCloskey" "" ""
$ genre : chr "Nonfiction" "Fiction, Legal Thriller, Crime" "Mystery, Fiction, Legal Thriller, Crime"
$ pub_year : int 2024 1992 2025
$ num_pages: int 423 386 417
$ type : chr "hardcover,paperback" "hardcover" "paperback"
$ ratings : num 4.11 4.05 4.13Use the identical function to evaluate
# Checking if the dataframes are identical
identical(booklistjson, booklisthtml_df)[1] FALSEAs loaded, the files are not identical. I will standardize them and compare them again.
Multiple ratings column from the JSON came in as chr type, whereas they are either integer or numeric from the HTML, so I will change them so they match.
#multiple ratings column came in as chr type; need to change them to numeric
booklistjson2 <- booklistjson %>%
mutate(pub_year = as.integer(pub_year)) %>%
mutate(ratings = as.double(ratings))
str(booklistjson2)'data.frame': 3 obs. of 8 variables:
$ title : chr "Framed: Astonishing True Stories of Wrongful Convictions" "The Pelican Brief" "The Widow"
$ author_1: chr "John Grisham" "John Grisham" "John Grisham"
$ author_2: chr "Jim McCloskey" NA NA
$ genre : chr "Nonfiction" "Fiction, Legal Thriller, Crime" "Mystery, Fiction, Legal Thriller, Crime"
$ pub_year: int 2024 1992 2025
$ num_pgs : int 423 386 417
$ type : chr "hardcover,paperback" "hardcover" "paperback"
$ ratings : num 4.11 4.05 4.13Check JSON data frame for missing data
booklistjson2 %>%
summarise(across(everything(), ~ sum(. == "", na.rm = TRUE))) title author_1 author_2 genre pub_year num_pgs type ratings
1 0 0 0 0 0 0 0 0There is no missing data in the JSON data frame.
Check HTML data frame for missing data
booklisthtml_df %>%
summarise(across(everything(), ~ sum(. == "", na.rm = TRUE))) title author_1 author_2 genre pub_year num_pages type ratings
1 0 0 2 0 0 0 0 0Column 2, “author_2” has 2 empty or blank strings. These need to be replace with NA
# Convert all blank strings into proper NA values across the whole data frame
booklisthtml_df2 <- booklisthtml_df %>%
mutate(across(everything(), ~ ifelse(. == "", NA, .)))
booklisthtml_df2 title author_1
1 Framed: Astonishing True Stories of Wrongful Convictions John Grisham
2 The Pelican Brief John Grisham
3 The Widow John Grisham
author_2 genre pub_year num_pages
1 Jim McCloskey Nonfiction 2024 423
2 <NA> Fiction, Legal Thriller, Crime 1992 386
3 <NA> Mystery, Fiction, Legal Thriller, Crime 2025 417
type ratings
1 hardcover,paperback 4.11
2 hardcover 4.05
3 paperback 4.13The “author_2” column for rows 2 and 3 now have NA values.
Use the identical function to evaluate
```{r file comparison2
identical(booklistjson2, booklisthtml_df2)
::: {.cell}
```{.r .cell-code}
anti_join(booklistjson2, booklisthtml_df2)Joining with `by = join_by(title, author_1, author_2, genre, pub_year, type,
ratings)`[1] title author_1 author_2 genre pub_year num_pgs type ratings
<0 rows> (or 0-length row.names):::
The HTML and JSON files should be identical after tidying, however, such is not the case for my files.I tried the anti_join function, but it was not helpful.