This assignment explores two commonly used data formats, HTML and JSON, and how they can be used to organize and store information. The main objective is to understand the differences between these formats and learn how to import their data into R for further analysis.
For this project, I will select three books related to Data Science and manually create two files containing the same information. The HTML file will present the book details in a table, while the JSON file will organize the information using key-value pairs and arrays. After creating both files, I will use R to read the data into separate data frames. Finally, I will compare the two data frames to verify that they contain the same information, even though the original files use different structures.
Book Selection and Data Attributes
For this assignment, I chose three books related to data science and machine learning, which are closely connected to the topics covered in this course.
The selected books are:
These books provide a useful dataset for exploring HTML and JSON formats. Two of the selected books have multiple authors, meeting the assignment requirement.
For each book, I will include five attributes: title, authors, publication year, publisher, and subject. The same information will be stored in both HTML and JSON files to compare how the two formats organize and represent data.
Creating the HTML Book Dataset
I will manually create an HTML file named books.html to
organize the information about the three selected books in a table
format. The table will have a header row identifying the attributes,
followed by three rows containing the details of each book. Each column
will represent a specific attribute, including the book title, authors,
publication year, publisher, and subject.
Creating this file manually will help me better understand how HTML uses table elements, rows, and columns to display structured information on a web page.
This dataset was exported as a CSV file from Project 1 . The file will be loaded into R using the GitHub raw link.
html_url <- "https://raw.githubusercontent.com/BIKASHBHOWMIK15/Data-607/main/Assignment7/books.html"
books_html <- html_url %>%
read_html() %>%
html_table(fill = TRUE)
books_html <- books_html[[1]]
books_html# A tibble: 3 × 5
title authors publication_year publisher subject
<chr> <chr> <int> <chr> <chr>
1 R for Data Science (2nd Edition) "Hadle… 2023 O'Reilly… Data S…
2 Python for Data Analysis (3rd Edit… "Wes M… 2022 O'Reilly… Data A…
3 An Introduction to Statistical Lea… "Garet… 2021 Springer Statis…
Creating the JSON Book Dataset
I will manually create a JSON file named books.json
containing the same information about the three selected books. The data
will be organized as an array of objects, with each object representing
one book. Each book object will include the same attributes used in the
HTML table, such as title, authors, publication year, publisher, and
subject.
Unlike HTML, which displays information in rows and columns, JSON stores data using key-value pairs and arrays. Creating this file manually will help me understand how JSON organizes structured information and how the same dataset can be stored in different formats.
json_url <- "https://raw.githubusercontent.com/BIKASHBHOWMIK15/Data-607/main/Assignment7/books.json"
books_json <- fromJSON(json_url)
books_json title
1 R for Data Science (2nd Edition)
2 Python for Data Analysis (3rd Edition)
3 An Introduction to Statistical Learning (2nd Edition)
authors
1 Hadley Wickham, Mine Cetinkaya-Rundel, Garrett Grolemund
2 Wes McKinney
3 Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani
publication_year publisher subject
1 2023 O'Reilly Media Data Science
2 2022 O'Reilly Media Data Analysis
3 2021 Springer Statistical Learning
Inspect structure of both data frames
Before comparing the two datasets, I inspect their structure to verify the variable names and data types.
Rows: 3
Columns: 5
$ title <chr> "R for Data Science (2nd Edition)", "Python for Data …
$ authors <chr> "Hadley Wickham, Mine Cetinkaya-Rundel,\r\n …
$ publication_year <int> 2023, 2022, 2021
$ publisher <chr> "O'Reilly Media", "O'Reilly Media", "Springer"
$ subject <chr> "Data Science", "Data Analysis", "Statistical Learnin…
Rows: 3
Columns: 5
$ title <chr> "R for Data Science (2nd Edition)", "Python for Data …
$ authors <chr> "Hadley Wickham, Mine Cetinkaya-Rundel, Garrett Grole…
$ publication_year <int> 2023, 2022, 2021
$ publisher <chr> "O'Reilly Media", "O'Reilly Media", "Springer"
$ subject <chr> "Data Science", "Data Analysis", "Statistical Learnin…
This step helps confirm whether both imported datasets have the same variables and whether any columns require type conversion before comparison.
Because HTML and JSON imports can sometimes differ slightly in structure, I standardize both data frames by converting all columns to character type, matching column names and order, and resetting row names.
# DATA 607 - Assignment 7
# Compare HTML and JSON datasets
# Standardize column names
clean_names <- function(x) {
x <- tolower(trimws(x))
x <- gsub("[^a-z0-9]+", "_", x)
x <- gsub("^_|_$", "", x)
x
}
names(books_html) <- clean_names(names(books_html))
names(books_json) <- clean_names(names(books_json))
# Expected fields
expected_cols <- c(
"title", "authors", "publication_year",
"publisher", "subject"
)
# Display imported column names
cat("HTML columns:\n")HTML columns:
[1] "title" "authors" "publication_year" "publisher"
[5] "subject"
JSON columns:
[1] "title" "authors" "publication_year" "publisher"
[5] "subject"
# Check that both datasets contain the required fields
missing_html <- setdiff(expected_cols, names(books_html))
missing_json <- setdiff(expected_cols, names(books_json))
if (length(missing_html) > 0 || length(missing_json) > 0) {
cat("Missing HTML columns:", paste(missing_html, collapse = ", "), "\n")
cat("Missing JSON columns:", paste(missing_json, collapse = ", "), "\n")
stop("Column mismatch: check the imported datasets.")
}
# Select matching columns
html_compare <- books_html[, expected_cols, drop = FALSE]
json_compare <- books_json[, expected_cols, drop = FALSE]
# Convert author arrays to comma-separated text
normalize_authors <- function(x) {
if (is.list(x)) {
x <- vapply(
x,
function(a) paste(unlist(a), collapse = ", "),
character(1)
)
}
gsub("\\s+", " ", trimws(as.character(x)))
}
html_compare$authors <- normalize_authors(html_compare$authors)
json_compare$authors <- normalize_authors(json_compare$authors)
# Standardize remaining fields
for (col in c("title", "publisher", "subject")) {
html_compare[[col]] <- trimws(as.character(html_compare[[col]]))
json_compare[[col]] <- trimws(as.character(json_compare[[col]]))
}
html_compare$publication_year <- as.numeric(
html_compare$publication_year
)
json_compare$publication_year <- as.numeric(
json_compare$publication_year
)
# Compare the two datasets
comparison_result <- all.equal(
html_compare,
json_compare,
check.attributes = FALSE
)
cat("\nComparison result:\n")
Comparison result:
[1] TRUE
| title | authors | publication_year | publisher | subject |
|---|---|---|---|---|
| R for Data Science (2nd Edition) | Hadley Wickham, Mine Cetinkaya-Rundel, Garrett Grolemund | 2023 | O’Reilly Media | Data Science |
| Python for Data Analysis (3rd Edition) | Wes McKinney | 2022 | O’Reilly Media | Data Analysis |
| An Introduction to Statistical Learning (2nd Edition) | Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani | 2021 | Springer | Statistical Learning |
| title | authors | publication_year | publisher | subject |
|---|---|---|---|---|
| R for Data Science (2nd Edition) | Hadley Wickham, Mine Cetinkaya-Rundel, Garrett Grolemund | 2023 | O’Reilly Media | Data Science |
| Python for Data Analysis (3rd Edition) | Wes McKinney | 2022 | O’Reilly Media | Data Analysis |
| An Introduction to Statistical Learning (2nd Edition) | Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani | 2021 | Springer | Statistical Learning |
After this step, both data frames are aligned for a strict comparison.
Compare the two data frames using identical() The identical() function checks whether the two data frames are exactly the same in both values and structure.
# Convert both objects to standard data frames
html_compare <- as.data.frame(html_compare)
json_compare <- as.data.frame(json_compare)
# Check the structure of both datasets
str(html_compare)'data.frame': 3 obs. of 5 variables:
$ title : chr "R for Data Science (2nd Edition)" "Python for Data Analysis (3rd Edition)" "An Introduction to Statistical Learning (2nd Edition)"
$ authors : chr "Hadley Wickham, Mine Cetinkaya-Rundel, Garrett Grolemund" "Wes McKinney" "Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani"
$ publication_year: num 2023 2022 2021
$ publisher : chr "O'Reilly Media" "O'Reilly Media" "Springer"
$ subject : chr "Data Science" "Data Analysis" "Statistical Learning"
'data.frame': 3 obs. of 5 variables:
$ title : chr "R for Data Science (2nd Edition)" "Python for Data Analysis (3rd Edition)" "An Introduction to Statistical Learning (2nd Edition)"
$ authors : chr "Hadley Wickham, Mine Cetinkaya-Rundel, Garrett Grolemund" "Wes McKinney" "Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani"
$ publication_year: num 2023 2022 2021
$ publisher : chr "O'Reilly Media" "O'Reilly Media" "Springer"
$ subject : chr "Data Science" "Data Analysis" "Statistical Learning"
title authors publication_year publisher
"character" "character" "double" "character"
subject
"character"
title authors publication_year publisher
"character" "character" "double" "character"
subject
"character"
[1] "1" "2" "3"
[1] "1" "2" "3"
identical_result <- identical(html_compare, json_compare)
cat("Are the standardized datasets identical? ",
identical_result, "\n")Are the standardized datasets identical? TRUE
If the result is TRUE, then both the HTML and JSON files produce exactly the same data frame in R.
Compare the two data frames using all.equal() To provide an additional check, I also use all.equal(), which is slightly more flexible and can help identify minor structural differences if they exist.
# Compare standardized datasets using all.equal()
all_equal_result <- all.equal(
html_compare,
json_compare,
check.attributes = FALSE
)
cat("Are the standardized datasets equivalent? ",
isTRUE(all_equal_result), "\n")Are the standardized datasets equivalent? TRUE
[1] TRUE
If this result returns TRUE, it confirms that the two imported datasets are equivalent.
Display final combined comparison view To make the comparison more transparent, I print both data frames again after type alignment.
knitr::kable(
html_compare,
format = "html",
caption = "Book Data Imported from HTML",
row.names = FALSE,
align = c("l", "l", "c", "l", "l")
)| title | authors | publication_year | publisher | subject |
|---|---|---|---|---|
| R for Data Science (2nd Edition) | Hadley Wickham, Mine Cetinkaya-Rundel, Garrett Grolemund | 2023 | O’Reilly Media | Data Science |
| Python for Data Analysis (3rd Edition) | Wes McKinney | 2022 | O’Reilly Media | Data Analysis |
| An Introduction to Statistical Learning (2nd Edition) | Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani | 2021 | Springer | Statistical Learning |
knitr::kable(
json_compare,
format = "html",
caption = "Book Data Imported from JSON",
row.names = FALSE,
align = c("l", "l", "c", "l", "l")
)| title | authors | publication_year | publisher | subject |
|---|---|---|---|---|
| R for Data Science (2nd Edition) | Hadley Wickham, Mine Cetinkaya-Rundel, Garrett Grolemund | 2023 | O’Reilly Media | Data Science |
| Python for Data Analysis (3rd Edition) | Wes McKinney | 2022 | O’Reilly Media | Data Analysis |
| An Introduction to Statistical Learning (2nd Edition) | Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani | 2021 | Springer | Statistical Learning |
These tables show that both source files contain the same book information after being imported into R.
This assignment demonstrated how the same dataset can be represented in both HTML and JSON formats and then imported into R for comparison. The HTML file stored the data in a table, while the JSON file stored the same information in a structured object format.
After loading both files into R and standardizing their structure, both identical() and all.equal() returned TRUE, confirming that the two formats produced the same final data frame. This exercise helped reinforce the structural differences between HTML and JSON while showing that both can serve as valid data sources for downstream analysis in R.