DATA 607 ASSIGNMENT 7

Overview

I used Dreamwaever to create three different files in HTML, XML, and JSON formats. These files are tables containing exact same information, which are the title, authors, edition, publisher, and publication year of three of my favorite books in learning data science. Note that two of my three books have two authors.

I will load these files into 3 three different data frames and then compare them two by two to check their similarity and difference in R if there is any, and will share the findings.

Libraries

library(XML)
library(jsonlite)
library(RCurl)

library(tidyverse)

## -- Attaching packages --------------------------------------------------------------------------------------- tidyverse 1.3.0 --

## v ggplot2 3.3.2     v purrr   0.3.4
## v tibble  3.0.3     v dplyr   1.0.2
## v tidyr   1.1.2     v stringr 1.4.0
## v readr   1.3.1     v forcats 0.5.0

## -- Conflicts ------------------------------------------------------------------------------------------ tidyverse_conflicts() --
## x tidyr::complete() masks RCurl::complete()
## x dplyr::filter()   masks stats::filter()
## x purrr::flatten()  masks jsonlite::flatten()
## x dplyr::lag()      masks stats::lag()

library(kableExtra)

## 
## Attaching package: 'kableExtra'

## The following object is masked from 'package:dplyr':
## 
##     group_rows

Data & Data frames

Get URL

html_url <- "https://raw.githubusercontent.com/jnataky/DATA-607/master/Web_technologies/books_html.html"

xml_url <- "https://raw.githubusercontent.com/jnataky/DATA-607/master/Web_technologies/books_xml.xml"

json_url <- "https://raw.githubusercontent.com/jnataky/DATA-607/master/Web_technologies/books_json.json"

Load files

HTML

file_html <- getURL(html_url)

df_html <- readHTMLTable(file_html, which =1)

# Kable for tidy table

df_html %>%
  kbl(caption = "My Favorite books") %>%
  kable_material(c("striped", "hover")) %>%
  row_spec(0, color = "indigo")

My Favorite books

Title	Authors	Edition	Publisher	Publication_Year
Understandable Statistics	Charles H. Brase and Corrinne P. Brase	Twelfth	Cengage Learning	2018
Hands-On Machine Learning with Scikit-Learn and TensorFlow	Aurelien Geron	First	O’Reilly	2017
Data Science for Business	Foster Provost and Tom Fawcett	First	O’Reilly	2013

XML

file_xml <- getURL(xml_url)

df_xml <- xmlToDataFrame(file_xml)


# Kable for tidy table

df_xml %>%
  kbl(caption = "My Favorite books") %>%
  kable_material(c("striped", "hover")) %>%
  row_spec(0, color = "indigo")

My Favorite books

Title	Authors	Edition	Publisher	Publication_Year
Understandable Statistics	Charles H. Brase and Corrinne P. Brase	Twelfth	Cengage Learning	2018
Hands-On Machine Learning with Scikit-Learn and TensorFlow	Aurelien Geron	First	O’Reilly	2017
Data Science for Business	Foster Provost and Tom Fawcett	First	O’Reilly	2013

JSON

df_json <- as.data.frame(fromJSON(json_url))

# Change columns names to match others

names(df_json) <- c("Title", "Authors", "Edition", "Publisher", "Publication_Year")

# Kable for tidy table

df_json %>%
  kbl(caption = "My Favorite books") %>%
  kable_material(c("striped", "hover")) %>%
  row_spec(0, color = "indigo")

My Favorite books

Title	Authors	Edition	Publisher	Publication_Year
Understandable Statistics	Charles H. Brase and Corrinne P. Brase	Twelfth	Cengage Learning	2018
Hands-On Machine Learning with Scikit-Learn and TensorFlow	Aurelien Geron	First	O’Reilly	2017
Data Science for Business	Foster Provost and Tom Fawcett	First	O’Reilly	2013

Comparison

# HTML & XML

all.equal(df_html, df_xml)

## [1] TRUE

# HTML & JSON

all.equal(df_html, df_json)

## [1] TRUE

Since the two first comparisons are true, we can say that all three frame have the same contents (after renaming to json data frame columns as the structure looked different from the others). They look the same but the internal structure might be different.

Find more info on those formats in the Findings section below.

Findings

This practice on these different data exchange formats bring to this conclusion:

The native structure of HTML does not naturally map into R objects. We can import HTML files as raw text, but this deprives us of the most useful features of these documents. And XML is a more generic counterpart to HTML and a frequently used format to exchange data on the Web. In the other hand, JSON is more lightweight due to its less verbose syntax and only allows a limited set of data types that are compatible with many programming languages.

Source: Munzert, S. (2015). Automated Data Collection with R: A practical guide to web scraping and text mining. Chichester: Wiley.