Pick three of your favorite books on one of your favorite subjects. At least one of the books should have more than one author. For each book, include the title, authors, and two or three other attributes that you find interesting. Take the information that you’ve selected about these three books, and separately create three files which store the book’s information in HTML (using an html table), XML, and JSON formats (e.g. “books.html”, “books.xml”, and “books.json”). To help you better understand the different file structures, I’d prefer that you create each of these files “by hand” unless you’re already very comfortable with the file formats. Write R code, using your packages of choice, to load the information from each of the three sources into separate R data frames. Are the three data frames identical? Your deliverable is the three source files and the R code. If you can, package your assignment solution up into an .Rmd file and publish to rpubs.com. [This will also require finding a way to make your three text files accessible from the web].
For this assignment, I picked three fantasy novels: Lord of the Rings (1954) by J.R.R Tolkien, A Game of Thrones (1996) by George R.R. Martin and Windhaven(1981) by George R.R Martin and Lisa Tuttle.
library(httr)
library(XML)
library(jsonlite)
library (dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionThe HTML source file can be found here
html <- "https://raw.githubusercontent.com/AtinaKarim/DATA607/master/Books.html"
html <- GET(html)
html <- rawToChar(html$content)
html <- htmlParse(html)
html <- readHTMLTable(html)
HTML <- data.frame(html)
knitr::kable(HTML, caption="HTML")| NULL.Genre | NULL.Title | NULL.Author | NULL.Publisher | NULL.Year |
|---|---|---|---|---|
| Fantasy | The Lord of the Rings (The Lord of the Rings #1-3) | J.R.R. Tolkien | Allen & Unwin | 1954 |
| Fantasy | A Game of Thrones | George R.R. Martin | Bantam Books | 1996 |
| Fantasy | Windhaven | George R.R. Martin, Lisa Tuttle | Timescape Books | 1981 |
The HTML file headers seem to start with a NULL followed by the column header, such as Null.Author. My hypothesis is that the ‘NULL’ is the table name (because we didn’t specify a table name maybe).
The JSON source file can be found here
json <- "https://raw.githubusercontent.com/AtinaKarim/DATA607/master/books.json"
json <- GET(json)
json <- rawToChar(json$content)
json <- fromJSON(json)
JSON <- data.frame(json)
knitr::kable(JSON, caption="JSON")| genre | title | author | publisher | year |
|---|---|---|---|---|
| Fantasy | The Lord of the Rings (The Lord of the Rings #1-3) | J.R.R. Tolkien | Allen & Unwin | 1954 |
| Fantasy | A Game of Thrones | George R.R. Martin | Bantam Books | 1996 |
| Fantasy | Windhaven | George R.R. Martin | Lisa Tuttle | Timescape Books |
The JSON headers look good and there seems to be an added ID column (first column) as well.
The XML source can be found here
xml <- "https://raw.githubusercontent.com/AtinaKarim/DATA607/master/Books6.XML"
xml <- GET(xml)
xml <- rawToChar(xml$content)
xml <- xmlParse(xml)
xml <- xmlToList(xml)
XML <- data.frame(xml)
knitr::kable(XML, caption="XML")| genre | book.title | book.publisher | book.year | book.author.text | book.author..attrs | book..attrs | book.title.1 | book.publisher.1 | book.year.1 | book.author.text.1 | book.author..attrs.1 | book..attrs.1 | book.title.2 | book.publisher.2 | book.year.2 | book.author.text.2 | book.author..attrs.2 | book.author.text.3 | book.author..attrs.3 | book..attrs.2 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | Fantasy | The Lord of the Rings | Allen and Unwin | 1954 | J.R.R. Tolkien | 1 | 1 | A Game of Thrones | Bantam Books | 1996 | George R.R. Martin | 1 | 2 | Windhaven | Timescape Books | 1981 | George R.R. Martin | 1 | Lisa Tuttle | 2 | 3 |
The XML headers start with the name of the table (book) followed by the column header, i.e. title, publisher, etc. However, there are additional columns ‘attrs’ but I am not sure what values they are storing.
Using Dplyr’s all equal function, we’ll compare the dataframes.
all_equal(
HTML,
JSON,
ignore_col_order = TRUE,
ignore_row_order = TRUE,
)## [1] "not compatible: \n- Cols in y but not x: `genre`, `title`, `author`, `publisher`, `year`.\n- Cols in x but not y: `NULL.Genre`, `NULL.Title`, `NULL.Author`, `NULL.Publisher`, `NULL.Year`.\n"As observed earlier, the column headers are stored differently between HTML and JSON.
all_equal(
HTML,
XML,
ignore_col_order = TRUE,
ignore_row_order = TRUE,
)## - different number of columns: 5 vs 21The number of columns are different between the two files because: 1. The XML file is storing the data for the three books in one row instead of three different rows. 2. The XML file also has additional ‘attrs’ columns - I am not sure what these columns are capturing or why they are appearing.
all_equal(
JSON,
XML,
ignore_col_order = TRUE,
ignore_row_order = TRUE,
)## - different number of columns: 5 vs 21The same issue as above, our xml file is storing all data in one row and adding ‘attrs’ columns.