Data Visualization

Kable package

In a previous weeks, we saw R Markdown in action, where multiple things can be created in one location: code, commentary, and output.

In this chapter we will explore package which will facilitate the creation of presentation-worthy tables: “kableExtra”.

Let’s work with the cross-sectional data on the credit history for a sample of applicants for a type of credit card.

data(CreditCard)
cardHead <- head(CreditCard)
cardHead

##   card reports      age income       share expenditure owner selfemp dependents
## 1  yes       0 37.66667 4.5200 0.033269910  124.983300   yes      no          3
## 2  yes       0 33.25000 2.4200 0.005216942    9.854167    no      no          3
## 3  yes       0 33.66667 4.5000 0.004155556   15.000000   yes      no          4
## 4  yes       0 30.50000 2.5400 0.065213780  137.869200    no      no          0
## 5  yes       0 32.16667 9.7867 0.067050590  546.503300   yes      no          2
## 6  yes       0 23.25000 2.5000 0.044438400   91.996670    no      no          0
##   months majorcards active
## 1     54          1     12
## 2     34          1     13
## 3     58          1      5
## 4     25          1      7
## 5     64          1      5
## 6     54          1      1

cardHead %>%
  kbl()

card	reports	age	income	share	expenditure	owner	selfemp	dependents	months	majorcards	active
yes	0	37.66667	4.5200	0.0332699	124.983300	yes	no	3	54	1	12
yes	0	33.25000	2.4200	0.0052169	9.854167	no	no	3	34	1	13
yes	0	33.66667	4.5000	0.0041556	15.000000	yes	no	4	58	1	5
yes	0	30.50000	2.5400	0.0652138	137.869200	no	no	0	25	1	7
yes	0	32.16667	9.7867	0.0670506	546.503300	yes	no	2	64	1	5
yes	0	23.25000	2.5000	0.0444384	91.996670	no	no	0	54	1	1

Let’s tweak the appearance of this with the “align” and the “caption” arguments.

The align argument takes a character vector with letters “l”, “c”, or “r” - specifying where you want the columns to be aligned.

The caption argument gives a caption to the table.

base <- cardHead %>%
  kbl(align = c(rep("c", 7), rep("r", 5)), caption = "kable example with card data")
base

kable example with card data

card	reports	age	income	share	expenditure	owner	selfemp	dependents	months	majorcards	active
yes	0	37.66667	4.5200	0.0332699	124.983300	yes	no	3	54	1	12
yes	0	33.25000	2.4200	0.0052169	9.854167	no	no	3	34	1	13
yes	0	33.66667	4.5000	0.0041556	15.000000	yes	no	4	58	1	5
yes	0	30.50000	2.5400	0.0652138	137.869200	no	no	0	25	1	7
yes	0	32.16667	9.7867	0.0670506	546.503300	yes	no	2	64	1	5
yes	0	23.25000	2.5000	0.0444384	91.996670	no	no	0	54	1	1

A key function, where we can enjoy much of the configuration for the table, is via kable_styling().

We have options “bootstrap_options” or “latex_options”, where the latter requires the use of the package “tinytex” and a local installation of LaTeX.

Possible options for “bootstrap_options” include ‘basic’, ‘striped’, ‘bordered’, ‘hover’, ‘condensed’, ‘responsive’, and none.

Possible for “latex_options” include ‘basic’, ‘striped’, ‘hold_position’, ‘HOLD_position’, ‘scale_down’, and ‘repeat_header’.

base %>%
  kable_styling(bootstrap_options = "striped")

kable example with card data

card	reports	age	income	share	expenditure	owner	selfemp	dependents	months	majorcards	active
yes	0	37.66667	4.5200	0.0332699	124.983300	yes	no	3	54	1	12
yes	0	33.25000	2.4200	0.0052169	9.854167	no	no	3	34	1	13
yes	0	33.66667	4.5000	0.0041556	15.000000	yes	no	4	58	1	5
yes	0	30.50000	2.5400	0.0652138	137.869200	no	no	0	25	1	7
yes	0	32.16667	9.7867	0.0670506	546.503300	yes	no	2	64	1	5
yes	0	23.25000	2.5000	0.0444384	91.996670	no	no	0	54	1	1

Next, we can customize the look and feel of particular rows and columns.

Let’s see an example here, where we make the last three rows blue.

base %>%
  kable_styling(bootstrap_options = "bordered") %>%
  column_spec(8:12, bold = T) %>%
  row_spec(4:6, italic = T, color = "gold", background = "blue")

kable example with card data

card	reports	age	income	share	expenditure	owner	selfemp	dependents	months	majorcards	active
yes	0	37.66667	4.5200	0.0332699	124.983300	yes	no	3	54	1	12
yes	0	33.25000	2.4200	0.0052169	9.854167	no	no	3	34	1	13
yes	0	33.66667	4.5000	0.0041556	15.000000	yes	no	4	58	1	5
yes	0	30.50000	2.5400	0.0652138	137.869200	no	no	0	25	1	7
yes	0	32.16667	9.7867	0.0670506	546.503300	yes	no	2	64	1	5
yes	0	23.25000	2.5000	0.0444384	91.996670	no	no	0	54	1	1

We can also create groups for our columns.

base %>%
  kable_styling(bootstrap_options = "bordered") %>%
  add_header_above(c("Group 1" = 4, "Group 2" = 2, "Group 3" = 6))

kable example with card data

Group 1				Group 2		Group 3
card	reports	age	income	share	expenditure	owner	selfemp	dependents	months	majorcards	active
yes	0	37.66667	4.5200	0.0332699	124.983300	yes	no	3	54	1	12
yes	0	33.25000	2.4200	0.0052169	9.854167	no	no	3	34	1	13
yes	0	33.66667	4.5000	0.0041556	15.000000	yes	no	4	58	1	5
yes	0	30.50000	2.5400	0.0652138	137.869200	no	no	0	25	1	7
yes	0	32.16667	9.7867	0.0670506	546.503300	yes	no	2	64	1	5
yes	0	23.25000	2.5000	0.0444384	91.996670	no	no	0	54	1	1

Data Aggregation

In the first stage of our analysis we are going to group our data in the form of the simple frequency table.

First, let’s take a look at the distribution of income in our sample and verify the tabular accuracy using TAI measure:

options(scipen=999)

limits<- cut(CreditCard$income,seq(0,14,by=2))
tabelka <- freq(limits,type="html")

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

tabelka

## $`x:`
##                x label Freq Percent Valid Percent Cumulative Percent
##    Valid   (0,2]        236    17.9          17.9               17.9
##            (2,4]        783    59.4          59.4               77.3
##            (4,6]        205    15.5          15.5               92.8
##            (6,8]         63     4.8           4.8               97.6
##           (8,10]         23     1.7           1.7               99.3
##          (10,12]          7     0.5           0.5               99.8
##          (12,14]          2     0.2           0.2              100.0
##            Total       1319   100.0         100.0                   
##  Missing <blank>          0     0.0                                 
##             <NA>          0     0.0                                 
##            Total       1319   100.0

Without ‘kable’ styling it’s quite ugly right? ;-)

Tabular accuracy

An index of tabular accuracy TAI, described by Jenks and Casspal in 1971 is to optimize the class distribution used in a cartograms/frequency tables etc.

The TAI indicator takes values in the range (0;1). The numerator of the expression is the sum of the absolute deviations of the values classified into classes, and the denominator is the sum of the absolute deviations of the entire classified set.

The better the class division reflects the nature of the data, the larger the indicator will be. As the number of classes increases, the indicator will take on larger values.

Let’s calculate TAI index to check the properties of the tabulated data:

tabelka2 <- classIntervals(CreditCard$income, n=7, style="fixed", fixedBreaks=seq(0,14,by=2))
jenks.tests(tabelka2)

##        # classes  Goodness of fit Tabular accuracy 
##        7.0000000        0.9085328        0.6568085

As we can see - TAI index…

We can use different recipes… (styles):

tabelka3<-classIntervals(CreditCard$income, n=10, style="sd")
plot(tabelka3,pal=c(1:10))

jenks.tests(tabelka3)

##        # classes  Goodness of fit Tabular accuracy 
##        8.0000000        0.9274792        0.6909392

Still, the TAI indicator is not satisfactory. What should we change in the final frequency table design?

hist(CreditCard$income)

Continuous variables

We can calculate the absolute and relative frequencies of a vector x with the function ‘Freq’ from the DescTools packages. Continuous (numeric) variables will be cut using the same logic as used by the function hist. Categorical variables will be aggregated by table. The result will contain single and cumulative frequencies for both, absolute values and percentages.

tabela4<-Freq(CreditCard$income,breaks=seq(0,14,by=2),useNA="ifany")

tabela4 %>%
  kable(col.names = c("Incomes in kUSD","Frequency","Percentage %","Cumulative frequency","Cumulative percentage %")) %>%
  kable_classic(full_width = F, html_font = "Cambria")

Incomes in kUSD	Frequency	Percentage %	Cumulative frequency	Cumulative percentage %
[0,2]	236	0.1789234	236	0.1789234
(2,4]	783	0.5936315	1019	0.7725550
(4,6]	205	0.1554208	1224	0.9279757
(6,8]	63	0.0477635	1287	0.9757392
(8,10]	23	0.0174375	1310	0.9931766
(10,12]	7	0.0053071	1317	0.9984837
(12,14]	2	0.0015163	1319	1.0000000

BTW: what about TAI of that table?…

Categorical variables

Now, let’s take a look at the categorical data and make some tabulations. The xtabs function works like table except it can produce tables from frequencies using the formula interface.

Let’s say we want to see the table with data on how many card applications was accepted or not:

## card
##   no  yes 
##  296 1023

We may easily produce cross-tabs (status vs. Does the individual own their home?) as well:

crosstab<-xtabs(~ card + owner, data=CreditCard)
crosstab

##      owner
## card   no yes
##   no  206  90
##   yes 532 491

and transform it into pretty html table with the kable function:

crosstab %>% 
  kbl() %>%
  kable_styling(full_width = F) %>%
  column_spec(1, bold = T, border_right = T) %>%
  column_spec(2, background = "yellow")

	no	yes
no	206	90
yes	532	491

Data Visualization

We will explore the “ggplot2” package of the tidyverse for data visualization purposes. The “ggplot2” packages involve the the following three mandatory components:

1. Data
2. An aesthetic mapping
3. Geoms (aka objects)

The following components can also optionally be added:

4. Stats (aka transformations)
5. Scales
6. Facets
7. Coordinate systems
8. Position adjustments
9. Themes

Please note that code in this tutorial was adapted from Chapters 3 of the book “R for Data Science” by Hadley Wickham and Garrett Grolemund.

The full book can be found at: https://r4ds.had.co.nz/

A good cheat sheet for ggplot2 functions can be found at: https://rstudio.com/wp-content/uploads/2015/03/ggplot2-cheatsheet.pdf

Scatterplots

Let’s create an extremely simple scatterplot.

We will use the function ggplot() to do this.

The format of any ggplot graph is this function, followed by another function to add objects.

The objects on a graph in the case of a scatterplot are points. The function we add to it is geom_point.

These functions rely on a function on the inside called aes().

The data and aesthetic mapping components can be added to either the ggplot() or geom functions.

ggplot(data = mpg) +
  geom_point(aes(x = displ, y = hwy))

This is one of the most basic graphs that one can make using the ggplot2 framework.

Next, let’s add color.

geom_point() understands the following aesthetics: x, y, alpha, color, fill, group, shape, size, and stroke (see help documentation).

Let’s map the color argument to the variable “class” from mpg.

ggplot(mpg) +
  geom_point(aes(x = displ, y = hwy, color = class))

This is not the only way to color objects.

Including the color argument inside of the aes() function can map colors to a choice of variable.

However, we can specify colors manually, by specifying color outside of the aes() function. We will also illustrate the “size” argument.

ggplot(mpg) +
  geom_point(aes(x = displ, y = hwy, size = class), color = "blue")

Barplots

Lastly, let’s examine other objects that we can plot using ggplot(). We will create a bar chart using the function geom_bar().

ggplot(mpg) +
  geom_bar(aes(x = class))

With the geom_bar() function, we have a great use-case for a stat transformation.

The following code can be used to convert these counts to proportions:

ggplot(mpg) +
  geom_bar(aes(x = class, y = stat(prop), group = 1))

Histograms

Next, let’s create a histogram with the geom_histogram() function.

ggplot(mpg) +
  geom_histogram(aes(x = hwy))

The geom_histogram() function accepts the argument “binwidth”, and has two key arguments for color: fill (this controls the overall color), and color (this controls the border).

Let’s fill all these in:

ggplot(mpg) +
  geom_histogram(aes(x = hwy), binwidth = 5, fill = "navy", color = "gold")

geom_histogram() provides a great example to modify the scale.

Notice in this example that the axis is automatically broken up by units of 10, and does not begin at 0.

We can modify this with the function scale_x_continuous(), as well as the y-axis with the function scale_y_continuous().

There are three key arguments we will feed this function: “breaks”, “limits”, and “expand”.

“breaks” will define the breaks on the axis.

“limits” will define the beginning and end of the axis, and the “expand” argument can be used to start the axes at 0 by using “expand = c(0,0)”.

ggplot(mpg) +
  geom_histogram(aes(x = hwy), binwidth = 5, fill = "navy", color = "gold") +
  scale_x_continuous(breaks = seq(0, 45, 5), limits = c(0, 50), expand = c(0,0)) +
  scale_y_continuous(breaks = seq(0, 90, 10), limits = c(0, 90), expand = c(0,0))

Boxplots

Next, we will create boxplots.

p <- ggplot(mpg) +
  geom_boxplot(aes(x = class, y = cty, fill = class))
p

Facets

Faceting generates small multiples each showing a different subset of the data. Small multiples are a powerful tool for exploratory data analysis: you can rapidly compare patterns in different parts of the data and see whether they are the same or different.

Read more about facets here.

Notice in this document the use of the fig.height and fig.width options.

Key arguments to facet_wrap() are “facets”, “nrow”, and “ncol”.

ggplot(mpg) +
  geom_boxplot(aes(x = class, y = cty, fill = class)) +
  facet_wrap(facets = ~cyl, nrow = 2, ncol = 2)

Coordinates

Other coordinate systems can be applied to graphs created from ggplot2.

One example is coord_polar(), which uses polar coordinates. Most of these are quite rare. Probably the most common one is coord_flip(), which will flip the X and Y axes. Let’s also illustrate the labs() function, which can be used to change labels.

ggplot(mpg) +
  geom_bar(aes(x = class, fill = factor(cyl))) +
  labs(title = "Cylinders by Class", fill = "cylinders") +
  coord_flip()

These bars are stacked on top of each of other, due to the “cyl” variable being mapped to the “fill” argument. There are various position adjustments that can be used. Again, most of these are not very common, but a common one is the argument “position = ‘dodge’”, which will put items side-by-side.

See this example:

ggplot(mpg) +
  geom_bar(aes(x = class, fill = factor(cyl)), position = "dodge") +
  labs(title = "Cylinders by Class", fill = "cylinders") + 
  coord_flip()

Themes

Lastly, we can alter the “theme”, or the overall appearance of our plot.

I recommend using the ggThemeAssist package, because this will make this incredibly easy, with an interface that will automatically generate reproducible code.

This can be used by highlighting a ggplot2 object, and navigating to Addins > ggplot Theme Assistant.

We’ll make the following changes: eliminating the panel grid lines, eliminating axis ticks, adding a title called “Boxplot Example”, making it bigger and putting it in bold, and adjusting it to the center.

# p
p + theme(axis.ticks = element_line(linetype = "blank"),
    panel.grid.major = element_line(linetype = "blank"),
    panel.grid.minor = element_line(linetype = "blank"),
    plot.title = element_text(size = 14,
        face = "bold", hjust = 0.5)) +labs(title = "Boxplot Example")

There are many more examples of things that can be done with ggplot2.

It is an amazingly powerful and flexible package, and it is worth getting acquainted with the cheat sheet.

Exercise 1.

Using data on credit card applications’ status please present the frequency table with the nice, kable format for average monthly credit card expenditures of applicants.

average_expenditure <- mean(CreditCard$expenditure)

freq_table <- table(CreditCard$expenditure)

freq_df <- data.frame(
  expenditure = as.numeric(names(freq_table)),
  Frequency = as.numeric(freq_table)
)

freq_df <- freq_df[order(freq_df$expenditure),]

kable(freq_df, caption = "Frequency for average monthly credit card expenditures")

Frequency for average monthly credit card expenditures

expenditure	Frequency
0.0000000	317
0.3125000	1
0.9166667	1
3.1666670	1
3.7500000	2
4.5833330	9
5.5208330	1
5.5833330	1
5.8333330	1
6.1458330	1
6.4166670	1
6.5000000	1
6.5175000	1
7.0833330	5
7.1666670	1
7.3333330	1
7.4166670	1
8.1250000	1
8.3333330	8
8.7500000	1
8.7783340	1
9.1666670	1
9.5833330	1
9.7416670	1
9.7533330	1
9.8541670	1
10.2908300	1
10.4200000	1
10.5833300	1
11.0041700	1
11.1666700	1
11.6725000	1
11.7125000	1
12.0833300	1
12.2500000	1
12.6508300	1
13.3333300	1
13.4025000	1
13.6858300	1
13.9166700	1
14.0000000	1
14.3316700	1
14.6666700	1
15.0000000	1
15.3325000	1
15.4166700	1
15.5833300	2
15.8333300	1
15.9408300	1
16.2500000	1
16.6341700	1
16.9133300	1
17.0833300	1
17.1716700	1
17.5833300	1
17.6666700	1
17.7850000	1
18.2541700	1
19.5900000	1
20.0833300	1
20.1808300	1
20.2500000	1
20.3333300	1
20.4791700	1
21.3333300	1
21.4650000	1
21.7150000	1
22.3575000	1
22.4625000	1
22.9866700	1
23.2500000	1
23.5833300	1
24.0700000	1
24.5750000	1
25.2916700	1
25.5075000	1
26.0000000	1
26.5000000	1
26.8783300	1
26.9966700	1
27.4166700	1
27.7800000	1
29.5158300	1
29.9566700	1
29.9750000	1
30.0525000	1
30.3691700	1
30.4458300	1
30.5483300	1
31.0000000	1
32.4641600	1
32.6666700	1
32.7750000	1
33.0133300	1
33.2150000	1
33.5075000	1
33.5708300	1
34.0916700	1
34.1250000	1
34.1558300	1
34.5208300	1
34.5316700	1
35.1666700	1
35.1683300	1
35.6750000	1
36.2050000	1
36.2500000	1
36.3333300	1
37.0150000	1
37.0833300	1
37.4291600	1
37.5833300	2
37.7133300	1
37.9741700	1
38.0625000	1
38.3650000	1
38.7083300	1
38.7141600	1
38.8333300	1
38.9975000	1
39.9116700	1
40.1991700	1
40.3275000	1
40.8333300	1
41.1891700	1
41.3333300	1
41.4816700	1
41.8166700	1
42.3108300	1
42.5166700	1
42.6150000	1
42.6308300	1
42.6908300	1
43.0391700	1
43.2066700	1
43.3391700	1
43.4366700	1
43.7441700	1
43.9741700	1
44.0000000	1
44.0208300	1
44.4133300	1
44.4633300	1
44.9425000	1
45.1241700	1
45.3541700	1
45.8758300	1
46.0558400	1
46.1416700	1
46.2916700	1
46.5233300	1
46.6058300	1
47.0816700	1
47.2416600	1
47.7875000	1
48.0833300	1
48.6150000	1
49.2500000	1
49.4166700	1
49.5558300	1
50.0283300	1
50.0833300	1
50.1783300	1
51.6450000	1
51.7500000	1
52.4891700	1
52.5800000	1
53.3016700	1
53.6450000	1
54.6466700	1
54.9691700	1
55.7608300	1
56.1491700	1
56.3333300	1
56.4633300	1
56.6941700	1
57.3066700	1
57.4408300	1
57.9158300	1
57.9808300	1
58.0541600	1
58.0808300	1
58.2458300	1
58.5058300	1
58.6083300	1
58.9308300	1
58.9908300	1
59.0891600	1
59.1191700	1
59.1475000	1
59.1941700	1
59.6191700	1
59.7866700	1
59.8541700	1
60.0666700	1
60.5841700	1
61.1666700	1
61.1991700	1
61.3333300	1
61.7358300	1
61.7425000	1
61.9525000	1
62.5083300	1
63.4566700	1
63.5516700	1
63.9166700	1
63.9916600	1
64.0741700	1
64.5600000	1
64.7558400	1
65.2466700	1
65.8183400	1
66.2641700	1
66.3008300	1
66.7325000	1
67.0975000	1
67.2016700	1
67.3983300	1
67.5358400	1
67.6666600	1
68.3591700	1
68.3841700	1
68.5333300	1
68.5458300	1
68.8708300	1
69.0325000	1
69.2733300	1
69.4275000	1
69.7933300	1
70.2850000	1
71.5300000	1
71.6666600	1
71.9591700	1
73.1725000	1
73.1766700	1
73.7975000	1
74.7266700	1
75.6341700	1
76.2441600	1
77.0466700	1
77.2866700	1
77.3566700	1
77.8066600	1
78.2558400	1
78.4283400	1
78.5416600	1
78.7508300	1
78.8741700	1
78.8850000	1
79.1758300	1
79.3608300	1
79.5108300	1
79.6091700	1
79.6850000	1
80.0550000	1
80.4508400	1
80.8258400	1
80.8683300	1
81.2650000	1
81.4350000	1
82.3425000	1
82.6591600	1
82.9208400	1
83.0833400	1
84.2750000	1
84.3908300	1
84.7316700	1
84.9025000	1
85.2408300	1
85.9783300	1
86.1483300	1
86.2266700	1
86.5466700	1
87.1050000	1
87.1491700	1
87.3066600	1
87.6366700	1
88.2333300	1
88.5925000	1
88.7391700	1
88.8175000	1
88.9466700	1
89.3708300	1
89.8266700	1
89.9550000	1
90.0633300	1
91.1766700	1
91.4125000	1
91.6875000	1
91.7325000	1
91.9641700	1
91.9891700	1
91.9966700	1
92.2633400	1
92.4416700	1
93.1066700	1
93.1975000	1
93.3100000	1
93.6250000	1
93.9191700	1
94.1183300	1
94.1450000	1
94.2683300	1
95.5283400	1
95.7991600	1
95.9975000	1
97.1841700	1
97.4241600	1
97.5125000	1
97.9400000	1
98.3450000	1
98.4641600	1
98.5791600	1
98.6825000	1
99.1033300	1
99.9166600	1
100.1433000	1
100.3367000	1
100.4167000	1
101.2008000	1
101.2625000	1
101.2983000	1
101.4942000	1
101.6833000	1
101.7633000	1
101.8542000	1
102.9767000	1
103.3283000	1
103.9167000	1
104.5358000	1
104.7833000	1
105.0450000	1
105.0950000	1
105.1250000	1
105.2833000	1
105.3250000	1
105.5075000	1
105.9617000	1
106.1692000	1
106.2008000	1
106.7208000	1
107.7642000	1
108.2958000	1
108.3967000	1
108.6075000	1
108.6233000	1
108.9767000	1
108.9775000	1
109.6092000	1
110.3033000	1
111.6175000	1
111.6750000	1
111.6925000	1
111.8183000	1
111.8392000	1
112.8783000	1
112.9167000	1
112.9942000	1
113.0992000	1
113.1808000	1
113.2492000	1
113.2958000	1
113.5892000	1
113.6308000	1
113.7500000	1
113.9475000	1
114.2533000	1
114.5575000	1
114.6017000	1
114.8158000	1
115.4167000	1
116.3150000	1
116.3525000	1
116.4492000	1
118.7133000	1
118.7217000	1
119.0000000	1
119.3017000	1
119.6250000	1
120.4033000	1
120.5592000	1
121.0308000	1
121.0558000	1
121.2208000	1
121.3367000	1
121.5558000	1
121.6333000	1
122.1917000	1
122.5150000	1
122.9292000	1
123.2225000	1
123.8917000	1
124.4758000	1
124.5033000	1
124.6325000	1
124.9833000	1
125.4817000	1
125.8917000	1
126.0225000	1
126.1033000	1
126.3867000	1
126.6508000	1
127.1183000	1
127.3725000	1
127.4725000	1
127.5342000	1
127.9833000	1
128.0725000	1
128.4267000	1
129.3333000	1
129.3733000	1
129.3933000	1
129.4167000	1
130.0267000	1
131.5242000	1
131.6742000	1
131.7658000	1
131.8783000	1
131.9892000	1
132.1292000	1
132.4717000	1
132.5883000	1
132.8508000	1
133.0683000	1
133.9692000	1
134.2008000	1
134.3858000	1
134.9183000	1
135.0450000	1
135.1842000	1
136.0408000	1
136.1242000	1
137.0717000	1
137.8692000	1
138.5467000	1
138.5692000	1
138.7133000	1
140.0150000	1
140.2992000	1
140.3467000	1
141.6883000	1
142.2425000	1
142.5142000	1
142.8775000	1
142.9508000	1
143.0158000	1
143.3467000	1
143.4292000	1
144.3842000	1
144.3975000	1
144.6417000	1
145.3792000	1
145.6958000	1
145.7192000	1
146.2700000	1
146.4275000	1
146.4417000	1
146.8625000	1
147.3333000	1
147.7233000	1
148.0592000	1
148.1900000	1
148.2600000	1
148.3725000	1
149.0967000	1
149.2992000	1
149.5450000	1
149.5508000	1
149.7000000	1
150.1767000	1
150.2917000	1
150.7900000	1
151.0325000	1
151.3367000	1
153.7267000	1
153.7517000	1
154.7400000	1
154.8592000	1
156.0892000	1
156.5725000	1
156.5833000	1
156.9000000	1
157.8442000	1
158.6017000	1
158.6067000	1
159.1958000	1
159.3600000	1
159.5950000	1
159.6550000	1
159.7300000	1
160.0283000	1
160.4517000	1
160.4792000	1
160.8750000	1
161.1617000	1
162.4458000	1
163.0392000	1
163.1800000	1
163.4175000	1
163.8392000	1
164.6433000	1
164.8100000	1
165.8550000	1
165.9300000	1
166.0592000	1
166.2383000	1
166.7267000	1
166.9017000	1
167.0400000	1
167.2125000	1
167.8267000	1
169.0575000	1
169.8767000	1
169.8883000	1
170.1808000	1
170.4625000	1
170.4667000	1
170.6408000	1
171.5125000	1
171.8608000	1
172.5717000	1
173.0233000	1
173.4675000	1
174.9250000	1
175.1367000	1
175.4442000	1
175.6442000	1
175.9975000	1
176.2717000	1
177.6158000	1
178.4558000	1
178.9158000	1
179.1783000	1
179.7167000	1
179.7658000	1
179.7708000	1
180.1775000	1
180.6550000	1
181.0150000	1
181.4142000	1
182.0958000	1
182.1675000	1
182.6092000	1
182.8250000	1
183.5608000	1
183.8983000	1
184.9433000	1
185.7383000	1
186.3533000	1
187.1425000	1
188.1050000	1
188.1858000	1
188.2950000	1
188.6208000	1
188.7275000	1
189.1767000	1
189.6842000	1
190.4308000	1
191.7017000	1
191.8358000	1
192.4508000	1
192.6400000	1
192.7208000	1
192.8183000	1
193.0675000	1
193.2933000	1
193.3225000	1
194.4667000	1
195.0233000	1
195.4508000	1
197.9233000	1
198.0233000	1
198.5483000	1
199.0042000	1
199.3692000	1
199.7333000	1
199.7950000	1
200.2375000	1
200.4933000	1
200.5867000	1
201.4825000	1
201.5825000	1
201.6583000	1
202.4700000	1
203.1333000	1
203.3667000	1
203.8917000	1
204.0875000	1
204.5367000	1
205.2542000	1
205.5133000	1
205.9358000	1
206.3817000	1
207.7133000	1
208.0833000	1
209.0558000	1
209.8442000	1
210.2283000	1
210.6475000	1
212.4783000	1
213.0000000	1
213.1292000	1
213.2925000	1
214.0650000	1
214.0975000	1
214.8600000	1
214.9342000	1
215.0692000	1
215.4392000	1
215.7392000	1
215.8817000	1
216.1650000	1
218.5200000	1
219.2717000	1
220.7925000	1
220.8942000	1
221.8892000	1
222.5392000	1
224.5817000	1
225.2692000	1
225.4592000	1
227.3683000	1
227.6017000	1
228.9658000	1
229.4933000	1
229.7975000	1
232.2758000	1
233.0417000	1
234.0483000	1
235.5717000	1
238.3275000	1
238.3650000	1
238.8558000	1
239.4867000	1
240.2775000	1
240.3800000	1
240.4842000	1
240.8208000	1
241.4233000	1
242.1283000	1
242.1450000	1
242.1950000	1
244.4233000	1
244.6533000	1
245.3308000	1
245.3517000	1
246.5708000	1
247.2992000	1
248.5058000	1
248.7192000	1
248.9058000	1
249.1658000	1
250.5208000	1
251.5208000	1
253.3658000	1
253.6292000	1
256.1867000	1
256.6642000	1
257.3017000	1
258.1667000	1
258.2908000	1
258.5492000	1
258.8783000	1
260.1208000	1
263.3275000	1
264.9658000	1
265.0142000	1
265.6517000	1
266.1908000	1
266.3433000	1
267.3483000	1
268.7242000	1
269.1533000	1
270.3683000	1
271.4675000	1
271.5533000	1
273.4900000	1
274.9467000	1
276.9842000	1
277.9892000	1
281.8067000	1
282.5633000	1
282.7258000	1
282.8758000	1
284.2558000	1
285.2550000	1
285.2825000	1
286.5700000	1
287.7583000	1
288.3633000	1
288.8017000	1
289.0442000	1
292.6633000	1
292.9483000	1
293.0417000	1
293.0658000	1
294.5592000	1
294.5800000	1
297.0317000	1
297.9717000	1
298.0233000	1
298.4783000	1
299.8858000	1
300.4608000	1
300.9942000	1
301.0175000	1
301.2525000	1
302.2467000	1
303.6742000	1
304.3817000	1
304.6225000	1
304.8950000	1
305.8375000	1
306.0317000	1
306.4683000	1
308.0492000	1
308.2500000	1
308.9250000	1
309.9100000	1
310.7358000	1
310.8417000	1
310.9375000	1
311.1425000	1
313.3525000	1
313.4333000	1
314.4908000	1
315.2017000	1
315.9425000	1
317.4642000	1
318.5100000	1
319.4300000	1
319.4900000	1
319.7925000	1
320.4867000	1
320.5167000	1
320.6758000	1
321.2108000	1
323.0633000	1
323.5875000	1
323.9517000	1
324.0275000	1
324.6183000	1
324.7050000	1
325.8500000	1
326.5667000	1
327.1100000	1
327.7367000	1
332.2392000	1
332.7042000	1
333.6117000	1
335.4350000	1
335.5625000	1
337.0583000	1
341.2450000	1
343.6542000	1
344.0067000	1
344.1575000	1
344.1808000	1
344.4708000	1
347.9483000	1
349.2017000	1
351.3633000	1
352.1733000	1
352.8483000	1
355.9683000	1
356.7817000	1
356.8792000	1
357.5908000	1
358.9725000	1
359.1225000	1
360.9950000	1
361.1617000	1
362.6383000	1
362.7542000	1
364.8192000	1
365.9058000	1
366.7450000	1
373.4167000	1
374.0517000	1
376.6342000	1
376.7650000	1
379.2983000	1
379.8658000	1
383.3117000	1
383.5667000	1
384.1725000	1
384.8992000	1
385.7750000	1
386.0525000	1
387.1742000	1
387.2183000	1
390.1667000	1
391.4175000	1
394.5717000	1
395.5850000	1
396.2717000	1
396.5000000	1
397.0333000	1
398.0475000	1
398.8508000	1
400.0708000	1
400.4475000	1
400.9117000	1
401.9300000	1
402.7225000	1
402.7875000	1
404.4900000	1
405.3517000	1
406.9792000	1
408.0825000	1
412.1867000	1
412.9933000	1
413.5975000	1
415.8058000	1
417.8350000	1
420.5808000	1
423.1267000	1
424.1767000	1
424.6292000	1
426.4467000	1
426.8325000	1
429.9050000	1
430.6925000	1
431.1433000	1
432.3592000	1
434.1250000	1
434.8775000	1
435.9908000	1
436.2592000	1
436.2750000	1
437.1342000	1
438.3667000	1
440.6908000	1
442.4458000	1
445.8058000	1
446.4150000	1
448.0233000	1
449.0392000	1
451.2050000	1
457.4875000	1
461.2600000	1
461.9217000	1
462.0208000	1
462.6558000	1
463.1517000	1
465.1950000	1
465.9383000	1
466.3475000	1
469.2475000	1
470.2967000	1
470.9733000	1
471.3450000	1
474.0650000	1
474.1500000	1
474.5000000	1
475.0183000	1
476.3158000	1
478.7125000	1
479.4367000	1
480.3233000	1
482.6575000	1
483.0608000	1
484.4392000	1
487.4817000	1
492.2558000	1
493.7333000	1
494.4875000	1
494.8875000	1
497.7058000	1
498.9267000	1
500.6158000	1
501.8025000	1
502.2017000	1
503.5475000	1
508.2908000	1
509.0542000	1
510.0900000	1
512.0125000	1
517.2117000	1
520.3417000	1
523.1250000	1
527.9417000	1
530.1800000	1
533.6275000	1
536.2500000	1
541.2958000	1
544.5208000	1
546.5033000	1
548.0350000	1
552.3417000	1
552.7242000	1
553.1067000	1
562.3592000	1
565.3292000	1
568.2433000	1
568.7742000	1
569.5458000	1
569.9183000	1
577.9833000	1
580.9725000	1
583.7783000	1
590.9484000	1
602.9208000	1
612.8633000	1
613.6425000	1
615.1208000	1
620.0550000	1
621.6600000	1
625.3675000	1
631.9725000	1
632.0808000	1
633.4133000	1
634.8625000	1
635.0508000	1
640.6533000	1
641.4216000	1
642.4742000	1
643.3233000	1
644.8283000	1
647.2067000	1
650.4375000	1
663.7692000	1
664.1208000	1
664.6025000	1
674.9092000	1
675.7708000	1
683.4283000	1
683.9883000	1
703.4067000	1
719.5059000	1
721.0600000	1
732.9792000	1
737.0808000	1
741.5042000	1
745.0383000	1
747.6700000	1
758.9391000	1
762.3784000	1
777.8217000	1
810.3917000	1
821.8208000	1
830.2767000	1
831.7317000	1
832.2433000	1
839.3008000	1
854.7850000	1
858.3475000	1
860.7550000	1
862.9642000	1
875.8884000	1
881.2642000	1
885.0858000	1
889.5850000	1
896.9384000	1
898.4258000	1
906.3092000	1
917.5400000	1
940.3925000	1
948.3625000	1
950.8641000	1
1010.5020000	1
1011.3330000	1
1028.9610000	1
1088.7390000	1
1096.6570000	1
1292.0030000	1
1352.4880000	1
1482.3680000	1
1510.5340000	1
1532.7730000	1
1569.6770000	1
1637.5270000	1
1774.3560000	1
1786.2770000	1
1836.9760000	1
1898.0330000	1
1902.0000000	1
1949.8620000	1
2001.5470000	1
2291.1740000	1
3099.5050000	1

Exercise 2.

The data comes from https://flixgem.com/ (dataset version as of March 12, 2021). The data contains information on 9425 movies and series available on Netlix.

Answer with the most appropriate data visualization for the following questions:

1. What is the distribution of Imdb scores for Polish movies and movie-series?

polish_data <- mydata %>% 
  filter(str_detect(Tags, "Polish"))

ggplot(polish_data, aes(x = IMDb.Score)) +
  geom_histogram(binwidth = 0.5, fill = "blue", color = "black") +
  labs(title = "Distribution of IMDb Scores for Polish Movies and TV Shows",
       x = "IMDb Score") +
  theme_minimal()

2. What is the density function of Imdb scores for Polish movies and movie-series?

cleaned_data <- na.omit(mydata$IMDb.Score)

den_lmdb <- density(cleaned_data)

ggplot() +
  geom_density(aes(x = cleaned_data), fill = "black") +
  labs(title = "Density of IMDb Scores",
       x = "IMDb Score", y = "Density")

3. What are the most popular languages available on Netflix?

seperated <- separate_rows(mydata, Languages, sep = ", ")

Popular_languages <- sort(table(seperated$Languages), decreasing = TRUE)

head(Popular_languages)

## 
##  English Japanese  Spanish   French   Korean   German 
##     6170     1177      837      801      562      489

For extra credits:

Extra challenge 1.: Create a chart showing actors starring in the most popular productions.

popular_films <- arrange(mydata, desc(IMDb.Votes)) %>%
  head(100) %>%
  separate_rows(Actors, sep = ", ") %>%
  group_by(Actors) %>%
  summarise(count = n()) %>%
  arrange(desc(count)) %>%
  head(10)


ggplot(popular_films, aes(x = Actors, y = count)) +
  geom_point() +
  coord_flip()

Extra challenge 2.: For movies and series, create rating charts from the various portals (Hidden Gem, IMDb, Rotten Tomatoes, Metacritic). Hint: it’s a good idea to reshape the data to long format.

long_form <- mydata %>%
  pivot_longer(cols = c(Hidden.Gem.Score, IMDb.Score, Rotten.Tomatoes.Score, Metacritic.Score), names_to = "Portal", values_to = "Rating")

ggplot(long_form, aes(x = Rating)) +
  geom_density(fill = "blue") +
  facet_wrap(~ Portal, scales = "free") +
  labs(title = "Rating Distribution for every Portal",
       x = "Rating", y = "Density")

Extra challenge 3.: Which film studios produce the most and how has this changed over the years?

mydata$Release.Date <- as.Date(mydata$Release.Date, format = "%m/%d/%Y")

studio_counts <- mydata %>%
  group_by(Production.House, year(Release.Date)) %>%
  summarise(Productions = n()) %>%
  ungroup()

top_studios <- studio_counts %>%
  group_by(Production.House) %>%
  summarise(TotalProductions = sum(Productions)) %>%
  arrange(desc(TotalProductions)) %>%
  top_n(10)

top_studio_counts <- studio_counts %>%
  filter(Production.House %in% top_studios$Production.House)

top_studio_counts %>%
  kbl()

Production.House	year(Release.Date)	Productions
	1930	1
	1931	1
	1941	1
	1942	1
	1949	2
	1951	2
	1953	1
	1954	1
	1955	3
	1956	1
	1957	1
	1959	4
	1960	1
	1962	1
	1963	3
	1964	6
	1965	3
	1966	5
	1967	2
	1968	1
	1969	5
	1970	3
	1971	3
	1972	11
	1973	3
	1974	6
	1975	3
	1976	4
	1977	6
	1978	6
	1979	6
	1980	4
	1981	9
	1982	7
	1983	8
	1984	6
	1985	11
	1986	4
	1987	13
	1988	13
	1989	14
	1990	16
	1991	13
	1992	15
	1993	19
	1994	20
	1995	17
	1996	12
	1997	13
	1998	30
	1999	20
	2000	28
	2001	36
	2002	34
	2003	64
	2004	50
	2005	65
	2006	84
	2007	79
	2008	105
	2009	96
	2010	132
	2011	148
	2012	175
	2013	200
	2014	215
	2015	341
	2016	430
	2017	547
	2018	610
	2019	554
	2020	434
	2021	43
	2029	3
	NA	198
Columbia Pictures Corporation	1955	1
Columbia Pictures Corporation	1969	1
Columbia Pictures Corporation	1979	1
Columbia Pictures Corporation	1980	1
Columbia Pictures Corporation	1981	1
Columbia Pictures Corporation	1983	2
Columbia Pictures Corporation	1984	1
Columbia Pictures Corporation	1985	2
Columbia Pictures Corporation	1986	3
Columbia Pictures Corporation	1987	1
Columbia Pictures Corporation	1991	3
Columbia Pictures Corporation	1993	1
Columbia Pictures Corporation	1994	1
Columbia Pictures Corporation	1995	2
Columbia Pictures Corporation	1996	4
Columbia Pictures Corporation	1997	4
Netflix	2013	2
Netflix	2014	3
Netflix	2015	3
Netflix	2016	10
Netflix	2017	15
Netflix	2018	8
Netflix	2019	12
New Line Cinema	1990	1
New Line Cinema	1991	1
New Line Cinema	1992	1
New Line Cinema	1993	1
New Line Cinema	1995	2
New Line Cinema	1997	1
New Line Cinema	2002	1
New Line Cinema	2003	3
New Line Cinema	2006	1
New Line Cinema	2007	2
New Line Cinema	2009	1
New Line Cinema	2016	1
Paramount	1960	2
Paramount	1979	1
Paramount	1980	1
Paramount	1982	2
Paramount	1986	2
Paramount	1988	1
Paramount	1990	1
Paramount	2000	1
Paramount	2002	1
Paramount	2003	2
Paramount	2004	1
Paramount	2011	1
Paramount Pictures	1930	1
Paramount Pictures	1953	2
Paramount Pictures	1954	1
Paramount Pictures	1968	1
Paramount Pictures	1969	1
Paramount Pictures	1970	1
Paramount Pictures	1972	1
Paramount Pictures	1974	1
Paramount Pictures	1976	1
Paramount Pictures	1977	1
Paramount Pictures	1978	1
Paramount Pictures	1979	1
Paramount Pictures	1980	1
Paramount Pictures	1981	1
Paramount Pictures	1984	1
Paramount Pictures	1986	3
Paramount Pictures	1987	2
Paramount Pictures	1988	2
Paramount Pictures	1989	1
Paramount Pictures	1990	2
Paramount Pictures	1991	2
Paramount Pictures	1992	3
Paramount Pictures	1993	3
Paramount Pictures	1994	4
Paramount Pictures	1995	1
Paramount Pictures	1996	3
Paramount Pictures	1999	1
Paramount Pictures	2004	2
Paramount Pictures	2005	1
Paramount Pictures	2006	3
Paramount Pictures	2020	1
Toho Company Ltd.	1957	1
Toho Company Ltd.	1962	1
Toho Company Ltd.	1977	1
Toho Company Ltd.	1991	1
Toho Company Ltd.	1993	1
Toho Company Ltd.	1994	1
Toho Company Ltd.	1995	1
Toho Company Ltd.	2000	1
Toho Company Ltd.	2002	1
Toho Company Ltd.	2003	1
Toho Company Ltd.	2010	2
Toho Company Ltd.	2012	1
Toho Company Ltd.	2014	1
Toho Company Ltd.	2015	1
Toho Company Ltd.	2016	2
Toho Company Ltd.	2017	1
TriStar Pictures	1984	1
TriStar Pictures	1988	2
TriStar Pictures	1989	1
TriStar Pictures	1990	1
TriStar Pictures	1991	3
TriStar Pictures	1993	4
TriStar Pictures	1994	2
TriStar Pictures	1995	1
TriStar Pictures	1997	1
TriStar Pictures	1998	2
TriStar Pictures	2016	1
Universal Pictures	1978	1
Universal Pictures	1980	1
Universal Pictures	1982	1
Universal Pictures	1983	1
Universal Pictures	1985	3
Universal Pictures	1989	1
Universal Pictures	1992	1
Universal Pictures	1993	1
Universal Pictures	1995	2
Universal Pictures	1996	5
Universal Pictures	1998	1
Universal Pictures	1999	1
Universal Pictures	2001	1
Universal Pictures	2003	2
Universal Pictures	2005	3
Universal Pictures	2006	3
Universal Pictures	2008	1
Universal Pictures	2010	1
Universal Pictures	2015	1
Universal Pictures	2016	1
Warner Brothers	1967	1
Warner Brothers	1979	1
Warner Brothers	1983	1
Warner Brothers	1984	1
Warner Brothers	1987	1
Warner Brothers	1988	1
Warner Brothers	1990	2
Warner Brothers	1992	1
Warner Brothers	1993	3
Warner Brothers	1996	1
Warner Brothers	1998	2
Warner Brothers	1999	1
Warner Brothers	2003	1
Warner Brothers	2005	2
Warner Brothers/Seven Arts	1965	1
Warner Brothers/Seven Arts	1968	1
Warner Brothers/Seven Arts	1972	1
Warner Brothers/Seven Arts	1976	1
Warner Brothers/Seven Arts	1980	1
Warner Brothers/Seven Arts	1986	1
Warner Brothers/Seven Arts	1988	1
Warner Brothers/Seven Arts	1989	1
Warner Brothers/Seven Arts	1990	2
Warner Brothers/Seven Arts	2000	1
Warner Brothers/Seven Arts	2001	1
Warner Brothers/Seven Arts	2003	1
Warner Brothers/Seven Arts	2005	2
Warner Brothers/Seven Arts	2017	1
Working Title Films	2004	2
Working Title Films	2008	4
Working Title Films	2010	2
Working Title Films	2011	2
Working Title Films	2012	1
Working Title Films	2013	3
Working Title Films	2014	1
Working Title Films	2015	3
Working Title Films	2016	1
Working Title Films	2017	1
Working Title Films	2018	1
Working Title Films	2019	1