Praktikum Data Acquisition dan Exploratory Data Analysis

Modul Praktikum Data Mining

BAGIAN I — PENDAHULUAN

Laporan ini disusun sebagai dokumentasi praktikum mata kuliah Data Mining pada materi Data Acquisition dan Exploratory Data Analysis (EDA). Praktikum mencakup tiga metode pengambilan data yang umum digunakan, yaitu direct download, Application Programming Interface (API), dan web scraping, yang seluruhnya dikerjakan menggunakan bahasa pemrograman R dan R Markdown.

Tujuan dari praktikum ini adalah:

1. Memahami konsep dasar data acquisition dalam kerangka KDD dan CRISP-DM.
2. Mengunduh dataset open access secara langsung.
3. Mengambil data melalui layanan API (Kaggle API).
4. Melakukan web scraping pada halaman Wikipedia.
5. Mengintegrasikan data dari dua sumber berbeda.
6. Melakukan Exploratory Data Analysis (EDA) dan visualisasi.
7. Menyusun laporan reproducible menggunakan R Markdown.

---

BAGIAN II — PERSIAPAN LINGKUNGAN KERJA

2.1 Instalasi Package

Seluruh package yang dibutuhkan untuk praktikum ini dipasang terlebih dahulu. Package tidyverse digunakan untuk manipulasi data, rvest dan xml2 untuk web scraping, jsonlite dan httr untuk akses API, skimr dan janitor untuk eksplorasi dan pembersihan data, serta plotly dan maps untuk visualisasi.

install.packages(c(
  "tidyverse", "rmarkdown", "WDI", "jsonlite", "httr", "rvest", "xml2",
  "readxl", "writexl", "skimr", "janitor", "knitr", "kableExtra", "here",
  "scales", "plotly", "maps", "reshape2"
))

2.2 Pemanggilan Package

library(tidyverse)
library(rmarkdown)
library(WDI)
library(jsonlite)
library(httr)
library(rvest)
library(xml2)
library(readxl)
library(writexl)
library(skimr)
library(janitor)
library(knitr)
library(kableExtra)
library(here)
library(scales)
library(plotly)
library(maps)
library(reshape2)

---

BAGIAN III — PRAKTIKUM MODUL

3.1 Praktikum 2.2 — Direct Download Dataset

3.1.1 Konsep

Direct download adalah metode paling sederhana dalam data acquisition, yaitu memperoleh dataset yang telah disediakan oleh penyedia data dalam format seperti CSV, Excel, JSON, atau ZIP. Pada praktikum ini digunakan dataset World Happiness Report yang diperoleh dari Kaggle.

3.1.2 Langkah Manual (Unduh melalui Web)

1. Buka website https://www.kaggle.com/
2. Login menggunakan akun pribadi.
3. Cari dataset World Happiness Report melalui tautan https://www.kaggle.com/datasets/unsdsn/world-happiness.
4. Klik tombol Download.
5. Ekstrak file ZIP, kemudian pilih file tahun yang akan digunakan.
6. Salin file 2019.csv ke dalam folder kerja.

3.1.3 Membaca Dataset

happiness <- read_csv(
  here("data", "2019.csv"),
  show_col_types = FALSE
)

head(happiness)

dim(happiness)

## [1] 156   9

names(happiness)

## [1] "Overall rank"                 "Country or region"           
## [3] "Score"                        "GDP per capita"              
## [5] "Social support"               "Healthy life expectancy"     
## [7] "Freedom to make life choices" "Generosity"                  
## [9] "Perceptions of corruption"

glimpse(happiness)

## Rows: 156
## Columns: 9
## $ `Overall rank`                 <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, …
## $ `Country or region`            <chr> "Finland", "Denmark", "Norway", "Icelan…
## $ Score                          <dbl> 7.769, 7.600, 7.554, 7.494, 7.488, 7.48…
## $ `GDP per capita`               <dbl> 1.340, 1.383, 1.488, 1.380, 1.396, 1.45…
## $ `Social support`               <dbl> 1.587, 1.573, 1.582, 1.624, 1.522, 1.52…
## $ `Healthy life expectancy`      <dbl> 0.986, 0.996, 1.028, 1.026, 0.999, 1.05…
## $ `Freedom to make life choices` <dbl> 0.596, 0.592, 0.603, 0.591, 0.557, 0.57…
## $ Generosity                     <dbl> 0.153, 0.252, 0.271, 0.354, 0.322, 0.26…
## $ `Perceptions of corruption`    <dbl> 0.393, 0.410, 0.341, 0.118, 0.298, 0.34…

3.1.4 Membersihkan Nama Variabel

Nama kolom dibersihkan menggunakan fungsi clean_names() dari package janitor, lalu kolom country_or_region diubah menjadi country_name.

happiness <- happiness %>% clean_names()
names(happiness)

## [1] "overall_rank"                 "country_or_region"           
## [3] "score"                        "gdp_per_capita"              
## [5] "social_support"               "healthy_life_expectancy"     
## [7] "freedom_to_make_life_choices" "generosity"                  
## [9] "perceptions_of_corruption"

happiness <- happiness %>% rename(country_name = country_or_region)
names(happiness)

## [1] "overall_rank"                 "country_name"                
## [3] "score"                        "gdp_per_capita"              
## [5] "social_support"               "healthy_life_expectancy"     
## [7] "freedom_to_make_life_choices" "generosity"                  
## [9] "perceptions_of_corruption"

3.1.5 Memeriksa Missing Value

colSums(is.na(happiness))

##                 overall_rank                 country_name 
##                            0                            0 
##                        score               gdp_per_capita 
##                            0                            0 
##               social_support      healthy_life_expectancy 
##                            0                            0 
## freedom_to_make_life_choices                   generosity 
##                            0                            0 
##    perceptions_of_corruption 
##                            0

summary(happiness)

##   overall_rank    country_name           score       gdp_per_capita  
##  Min.   :  1.00   Length:156         Min.   :2.853   Min.   :0.0000  
##  1st Qu.: 39.75   Class :character   1st Qu.:4.545   1st Qu.:0.6028  
##  Median : 78.50   Mode  :character   Median :5.380   Median :0.9600  
##  Mean   : 78.50                      Mean   :5.407   Mean   :0.9051  
##  3rd Qu.:117.25                      3rd Qu.:6.184   3rd Qu.:1.2325  
##  Max.   :156.00                      Max.   :7.769   Max.   :1.6840  
##  social_support  healthy_life_expectancy freedom_to_make_life_choices
##  Min.   :0.000   Min.   :0.0000          Min.   :0.0000              
##  1st Qu.:1.056   1st Qu.:0.5477          1st Qu.:0.3080              
##  Median :1.272   Median :0.7890          Median :0.4170              
##  Mean   :1.209   Mean   :0.7252          Mean   :0.3926              
##  3rd Qu.:1.452   3rd Qu.:0.8818          3rd Qu.:0.5072              
##  Max.   :1.624   Max.   :1.1410          Max.   :0.6310              
##    generosity     perceptions_of_corruption
##  Min.   :0.0000   Min.   :0.0000           
##  1st Qu.:0.1087   1st Qu.:0.0470           
##  Median :0.1775   Median :0.0855           
##  Mean   :0.1848   Mean   :0.1106           
##  3rd Qu.:0.2482   3rd Qu.:0.1412           
##  Max.   :0.5660   Max.   :0.4530

skim(happiness)

Data summary

Name	happiness
Number of rows	156
Number of columns	9
_______________________
Column type frequency:
character	1
numeric	8
________________________
Group variables	None

Variable type: character

skim_variable	n_missing	complete_rate	min	max	empty	n_unique	whitespace
country_name	0	1	4	24	0	156	0

Variable type: numeric

skim_variable	n_missing	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
overall_rank	0	1	78.50	45.18	1.00	39.75	78.50	117.25	156.00	▇▇▇▇▇
score	0	1	5.41	1.11	2.85	4.54	5.38	6.18	7.77	▂▇▇▇▃
gdp_per_capita	0	1	0.91	0.40	0.00	0.60	0.96	1.23	1.68	▃▅▇▇▃
social_support	0	1	1.21	0.30	0.00	1.06	1.27	1.45	1.62	▁▁▂▆▇
healthy_life_expectancy	0	1	0.73	0.24	0.00	0.55	0.79	0.88	1.14	▁▃▃▇▅
freedom_to_make_life_choices	0	1	0.39	0.14	0.00	0.31	0.42	0.51	0.63	▁▃▆▇▆
generosity	0	1	0.18	0.10	0.00	0.11	0.18	0.25	0.57	▆▇▆▁▁
perceptions_of_corruption	0	1	0.11	0.09	0.00	0.05	0.09	0.14	0.45	▇▅▁▁▁

3.1.6 Memilih Variabel

happiness2 <- happiness %>%
  select(
    country_name, score, gdp_per_capita,
    social_support, healthy_life_expectancy,
    freedom_to_make_life_choices
  )

head(happiness2)

3.1.7 Menyimpan Dataset Hasil Pembersihan

if (!dir.exists("output")) dir.create("output")

write_csv(happiness2, here("output", "happiness_clean.csv"))
list.files("output")

## [1] "happiness_clean.csv"

3.1.8 Visualisasi Awal

happiness2 %>%
  slice_max(score, n = 10) %>%
  ggplot(aes(
    x = reorder(country_name, score),
    y = score
  )) +
  geom_col(fill = "steelblue") +
  coord_flip() +
  labs(
    title = "10 Negara dengan Happiness Score Tertinggi (2019)",
    x = "Negara",
    y = "Happiness Score"
  ) +
  theme_minimal(base_size = 13)

Interpretasi: Sepuluh negara dengan Happiness Score tertinggi pada tahun 2019 didominasi oleh negara-negara Skandinavia dan Eropa Barat, seperti Finlandia, Denmark, dan Norwegia.

---

3.2 Praktikum 2.3 — Pengambilan Data Menggunakan API

3.2.1 Konsep

API (Application Programming Interface) memungkinkan pengambilan data secara otomatis melalui kode program. Kaggle menyediakan API yang memungkinkan pengguna mengunduh dataset secara langsung tanpa membuka browser. Metode ini lebih efisien karena dapat direproduksi dan tidak bergantung pada unduhan manual.

3.2.2 Persiapan Kaggle API

Pastikan Python dan Kaggle CLI sudah terpasang, serta file kaggle.json telah tersedia di folder ~/.kaggle/.

system2("python", "--version", stdout = TRUE, stderr = TRUE)

## [1] "Python 3.12.4"

system2(
  "python",
  c("-m", "kaggle", "--version"),
  stdout = TRUE, stderr = TRUE
)

##  [1] "Authentication required to call the Kaggle API."                                    
##  [2] ""                                                                                   
##  [3] "First, you will need a Kaggle account. You can sign up at"                          
##  [4] "  https://www.kaggle.com/account/login"                                             
##  [5] ""                                                                                   
##  [6] "Recommended: log in with OAuth via a web-based authorization flow."                 
##  [7] "No token to manage; credentials are cached locally for you."                        
##  [8] "    kaggle auth login"                                                              
##  [9] ""                                                                                   
## [10] "If you'd rather not use OAuth, generate an API token at"                            
## [11] "  https://www.kaggle.com/settings/api  (click \"Generate New Token\" under \"API\")"
## [12] "and supply it to the CLI in one of these ways:"                                     
## [13] ""                                                                                   
## [14] "  Option A: Environment variable"                                                   
## [15] "    export KAGGLE_API_TOKEN=xxxxxxxxxxxxxx  # token copied from the settings UI"    
## [16] ""                                                                                   
## [17] "  Option B: API token file"                                                         
## [18] "    Save the token to ~/.kaggle/access_token"                                       
## [19] "Kaggle CLI 2.2.4"

3.2.3 Mencari Dataset

hasil_pencarian <- system2(
  "python",
  c("-m", "kaggle", "datasets", "list", "--s", "world-happiness"),
  stdout = TRUE, stderr = TRUE
)
cat(hasil_pencarian, sep = "\n")

## usage: __main__.py datasets list [-h] [--sort-by SORT_BY] [--size SIZE]
##                                  [--file-type FILE_TYPE]
##                                  [--license LICENSE_NAME] [--tags TAG_IDS]
##                                  [-s SEARCH] [-m] [--user USER]
##                                  [--page-size PAGE_SIZE]
##                                  [--page-token PAGE_TOKEN] [-p PAGE]
##                                  [-v | --format OUTPUT_FORMAT]
##                                  [--max-size MAX_SIZE] [--min-size MIN_SIZE]
## __main__.py datasets list: error: ambiguous option: --s could match --sort-by, --size, --search
## Authentication required to call the Kaggle API.
## 
## First, you will need a Kaggle account. You can sign up at
##   https://www.kaggle.com/account/login
## 
## Recommended: log in with OAuth via a web-based authorization flow.
## No token to manage; credentials are cached locally for you.
##     kaggle auth login
## 
## If you'd rather not use OAuth, generate an API token at
##   https://www.kaggle.com/settings/api  (click "Generate New Token" under "API")
## and supply it to the CLI in one of these ways:
## 
##   Option A: Environment variable
##     export KAGGLE_API_TOKEN=xxxxxxxxxxxxxx  # token copied from the settings UI
## 
##   Option B: API token file
##     Save the token to ~/.kaggle/access_token

Dataset yang akan digunakan adalah unsdsn/world-happiness.

3.2.4 Membuat Folder dan Mengunduh Dataset

if (!dir.exists("data")) dir.create("data")

system2(
  "python",
  c(
    "-m", "kaggle",
    "datasets", "download",
    "-d", "unsdsn/world-happiness",
    "-p", "data"
  ),
  stdout = TRUE,
  stderr = TRUE
)

##  [1] "Authentication required to call the Kaggle API."                                                       
##  [2] ""                                                                                                      
##  [3] "First, you will need a Kaggle account. You can sign up at"                                             
##  [4] "  https://www.kaggle.com/account/login"                                                                
##  [5] ""                                                                                                      
##  [6] "Recommended: log in with OAuth via a web-based authorization flow."                                    
##  [7] "No token to manage; credentials are cached locally for you."                                           
##  [8] "    kaggle auth login"                                                                                 
##  [9] ""                                                                                                      
## [10] "If you'd rather not use OAuth, generate an API token at"                                               
## [11] "  https://www.kaggle.com/settings/api  (click \"Generate New Token\" under \"API\")"                   
## [12] "and supply it to the CLI in one of these ways:"                                                        
## [13] ""                                                                                                      
## [14] "  Option A: Environment variable"                                                                      
## [15] "    export KAGGLE_API_TOKEN=xxxxxxxxxxxxxx  # token copied from the settings UI"                       
## [16] ""                                                                                                      
## [17] "  Option B: API token file"                                                                            
## [18] "    Save the token to ~/.kaggle/access_token"                                                          
## [19] "Dataset URL: https://www.kaggle.com/datasets/unsdsn/world-happiness"                                   
## [20] "License(s): CC0-1.0"                                                                                   
## [21] "world-happiness.zip: Skipping, found more recently modified local copy (use --force to force download)"

list.files("data")

## [1] "2015.csv"            "2016.csv"            "2017.csv"           
## [4] "2018.csv"            "2019.csv"            "world-happiness.zip"

3.2.5 Ekstraksi File ZIP

unzip(
  zipfile = "data/world-happiness.zip",
  exdir   = "data"
)

list.files("data")

## [1] "2015.csv"            "2016.csv"            "2017.csv"           
## [4] "2018.csv"            "2019.csv"            "world-happiness.zip"

3.2.6 Membaca Dataset Hasil API

happiness_api <- read_csv(
  "data/2019.csv",
  show_col_types = FALSE
)

head(happiness_api)

dim(happiness_api)

## [1] 156   9

names(happiness_api)

## [1] "Overall rank"                 "Country or region"           
## [3] "Score"                        "GDP per capita"              
## [5] "Social support"               "Healthy life expectancy"     
## [7] "Freedom to make life choices" "Generosity"                  
## [9] "Perceptions of corruption"

glimpse(happiness_api)

## Rows: 156
## Columns: 9
## $ `Overall rank`                 <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, …
## $ `Country or region`            <chr> "Finland", "Denmark", "Norway", "Icelan…
## $ Score                          <dbl> 7.769, 7.600, 7.554, 7.494, 7.488, 7.48…
## $ `GDP per capita`               <dbl> 1.340, 1.383, 1.488, 1.380, 1.396, 1.45…
## $ `Social support`               <dbl> 1.587, 1.573, 1.582, 1.624, 1.522, 1.52…
## $ `Healthy life expectancy`      <dbl> 0.986, 0.996, 1.028, 1.026, 0.999, 1.05…
## $ `Freedom to make life choices` <dbl> 0.596, 0.592, 0.603, 0.591, 0.557, 0.57…
## $ Generosity                     <dbl> 0.153, 0.252, 0.271, 0.354, 0.322, 0.26…
## $ `Perceptions of corruption`    <dbl> 0.393, 0.410, 0.341, 0.118, 0.298, 0.34…

summary(happiness_api)

##   Overall rank    Country or region      Score       GDP per capita  
##  Min.   :  1.00   Length:156         Min.   :2.853   Min.   :0.0000  
##  1st Qu.: 39.75   Class :character   1st Qu.:4.545   1st Qu.:0.6028  
##  Median : 78.50   Mode  :character   Median :5.380   Median :0.9600  
##  Mean   : 78.50                      Mean   :5.407   Mean   :0.9051  
##  3rd Qu.:117.25                      3rd Qu.:6.184   3rd Qu.:1.2325  
##  Max.   :156.00                      Max.   :7.769   Max.   :1.6840  
##  Social support  Healthy life expectancy Freedom to make life choices
##  Min.   :0.000   Min.   :0.0000          Min.   :0.0000              
##  1st Qu.:1.056   1st Qu.:0.5477          1st Qu.:0.3080              
##  Median :1.272   Median :0.7890          Median :0.4170              
##  Mean   :1.209   Mean   :0.7252          Mean   :0.3926              
##  3rd Qu.:1.452   3rd Qu.:0.8818          3rd Qu.:0.5072              
##  Max.   :1.624   Max.   :1.1410          Max.   :0.6310              
##    Generosity     Perceptions of corruption
##  Min.   :0.0000   Min.   :0.0000           
##  1st Qu.:0.1087   1st Qu.:0.0470           
##  Median :0.1775   Median :0.0855           
##  Mean   :0.1848   Mean   :0.1106           
##  3rd Qu.:0.2482   3rd Qu.:0.1412           
##  Max.   :0.5660   Max.   :0.4530

3.2.7 Menyimpan Dataset Hasil API

if (!dir.exists("output")) dir.create("output")

write_csv(happiness_api, "output/happiness_2019.csv")
list.files("output")

## [1] "happiness_2019.csv"  "happiness_clean.csv"

Catatan: Hasil dari direct download dan API seharusnya identik karena keduanya bersumber dari dataset yang sama. Perbedaan hanya terletak pada metode akuisisinya.

---

3.3 Praktikum 2.4 — Web Scraping

3.3.1 Konsep

Web scraping adalah teknik pengambilan informasi dari halaman web secara otomatis dengan membaca struktur HTML. Teknik ini digunakan ketika data tidak tersedia dalam bentuk file unduhan maupun API. Praktikum ini menggunakan package rvest untuk mengekstrak tabel dari halaman Wikipedia. Pengambilan data dilakukan secara etis, hanya pada data publik, dan untuk tujuan pembelajaran.

3.3.2 Menentukan URL dan Membaca HTML

url <- paste0(
  "https://en.wikipedia.org/wiki/",
  "List_of_countries_by_Human_Development_Index"
)

halaman <- read_html(url)
halaman

## {html_document}
## <html class="client-nojs vector-feature-language-in-header-enabled vector-feature-language-in-main-menu-disabled vector-feature-language-in-main-page-header-disabled vector-feature-page-tools-pinned-disabled vector-feature-toc-pinned-clientpref-1 vector-feature-main-menu-pinned-disabled vector-feature-limited-width-clientpref-1 vector-feature-limited-width-content-enabled vector-feature-custom-font-size-clientpref-1 vector-feature-appearance-pinned-clientpref-1 skin-theme-clientpref-day vector-sticky-header-enabled vector-toc-available skin-thumbsize-clientpref-standard" lang="en" dir="ltr">
## [1] <head>\n<meta http-equiv="Content-Type" content="text/html; charset=UTF-8 ...
## [2] <body class="skin--responsive skin-vector skin-vector-search-vue mediawik ...

3.3.3 Mengekstrak Seluruh Tabel

tabel <- halaman %>% html_table(fill = TRUE)
length(tabel)

## [1] 14

3.3.4 Identifikasi Tabel

for (i in seq_along(tabel)) {
  cat("\n==============\n")
  cat("Tabel", i, "\n")
  print(head(tabel[[i]], 3))
}

Berdasarkan hasil identifikasi, tabel yang memuat data Human Development Index berada pada tabel ke-2.

3.3.5 Memilih Tabel HDI

hdi <- tabel[[2]]
head(hdi)

3.3.6 Membersihkan Nama Variabel

hdi <- hdi %>% clean_names()
names(hdi)

## [1] "rank"                            "changesince2015"                
## [3] "country_or_territory"            "hdi_value"                      
## [5] "percent_annual_growth_2010_2023"

hdi <- hdi %>% rename(
  country_name = country_or_territory,
  hdi          = hdi_value
)

names(hdi)

## [1] "rank"                            "changesince2015"                
## [3] "country_name"                    "hdi"                            
## [5] "percent_annual_growth_2010_2023"

3.3.7 Memeriksa Struktur Data

glimpse(hdi)

## Rows: 193
## Columns: 5
## $ rank                            <int> 1, 2, 2, 4, 5, 5, 7, 8, 8, 10, 11, 12,…
## $ changesince2015                 <chr> "(2)", "(1)", "", "(2)", "(1)", "", "(…
## $ country_name                    <chr> "Iceland", "Norway", "Switzerland", "D…
## $ hdi                             <dbl> 0.972, 0.970, 0.970, 0.962, 0.959, 0.9…
## $ percent_annual_growth_2010_2023 <chr> "0.28%", "0.25%", "0.24%", "0.35%", "0…

summary(hdi)

##       rank       changesince2015    country_name            hdi        
##  Min.   :  1.0   Length:193         Length:193         Min.   :0.3880  
##  1st Qu.: 48.0   Class :character   Class :character   1st Qu.:0.6220  
##  Median : 97.0   Mode  :character   Mode  :character   Median :0.7620  
##  Mean   : 96.8                                         Mean   :0.7408  
##  3rd Qu.:145.0                                         3rd Qu.:0.8620  
##  Max.   :193.0                                         Max.   :0.9720  
##  percent_annual_growth_2010_2023
##  Length:193                     
##  Class :character               
##  Mode  :character               
##                                 
##                                 
## 

3.3.8 Memilih Variabel Utama

hdi <- hdi %>% select(country_name, hdi)
head(hdi)

3.3.9 Menyimpan Dataset HDI

if (!dir.exists(here("output"))) dir.create(here("output"))

write_csv(hdi, here("output", "hdi_wikipedia.csv"))
list.files("output")

## [1] "happiness_2019.csv"  "happiness_clean.csv" "hdi_wikipedia.csv"

3.3.10 Visualisasi Cleveland Dot Plot Interaktif

hdi_top20 <- hdi %>%
  slice_max(order_by = hdi, n = 20) %>%
  arrange(hdi) %>%
  mutate(country_name = factor(country_name, levels = unique(country_name)))

p <- hdi_top20 %>%
  ggplot(aes(x = hdi, y = country_name)) +
  geom_segment(
    aes(x = 0, xend = hdi, y = country_name, yend = country_name),
    color = "grey80", linewidth = 0.8
  ) +
  geom_point(
    aes(color = hdi,
        text = paste0("<b>", country_name, "</b>",
                      "<br>HDI : ", round(hdi, 3))),
    size = 4
  ) +
  scale_color_gradient(low = "#74add1", high = "#d73027", name = "HDI") +
  scale_x_continuous(
    limits = c(0, 1.05),
    breaks = seq(0, 1, 0.1),
    labels = number_format(accuracy = 0.01)
  ) +
  labs(
    title    = "20 Negara dengan HDI Tertinggi",
    subtitle = "Data hasil Web Scraping dari Wikipedia",
    x        = "Nilai Human Development Index (HDI)",
    y        = NULL
  ) +
  theme_minimal(base_size = 13) +
  theme(
    panel.grid.major.y = element_blank(),
    panel.grid.minor   = element_blank(),
    legend.position    = "right",
    plot.title         = element_text(face = "bold")
  )

ggplotly(p, tooltip = "text")

Interpretasi: Sebagian besar negara dengan HDI tertinggi berasal dari Eropa dan Amerika Utara. Nilai HDI mereka umumnya berada di atas 0,94.

3.3.11 Visualisasi Peta Dunia Persebaran HDI

peta <- map_data("world")

peta_hdi <- peta %>%
  left_join(hdi, by = c("region" = "country_name"))

p_map <- ggplot(peta_hdi, aes(x = long, y = lat, group = group)) +
  geom_polygon(
    aes(
      fill = hdi,
      text = paste0(
        "<b>", region, "</b>",
        "<br>HDI : ",
        ifelse(is.na(hdi), "Data tidak tersedia", round(hdi, 3))
      )
    ),
    color = "white", linewidth = 0.15
  ) +
  coord_fixed(1.3) +
  scale_fill_gradient(
    low = "#74add1", high = "#d73027",
    na.value = "grey90", name = "HDI"
  ) +
  labs(
    title    = "Persebaran Human Development Index (HDI) Dunia",
    subtitle = "Data hasil Web Scraping dari Wikipedia",
    x = NULL, y = NULL
  ) +
  theme_void(base_size = 12) +
  theme(
    legend.position = "bottom",
    plot.title      = element_text(face = "bold")
  )

ggplotly(p_map, tooltip = "text") %>%
  layout(
    annotations = list(
      x = 1, y = -0.10,
      text = "Sumber: Wikipedia (Human Development Index)",
      showarrow = FALSE,
      xref = "paper", yref = "paper",
      xanchor = "right",
      font = list(size = 10, color = "gray40")
    )
  )

Interpretasi: Warna merah tua menandakan HDI tinggi (Eropa Barat, Amerika Utara, Australia), sedangkan warna biru menandakan HDI rendah (Afrika Sub-Sahara, Asia Selatan). Warna abu-abu menandakan data HDI tidak tersedia untuk negara tersebut.

---

BAGIAN IV — TUGAS: INTEGRASI DATA DAN EDA

4.1 Studi Kasus

Seorang peneliti ingin mengetahui hubungan antara tingkat kebahagiaan suatu negara dengan tingkat pembangunan manusia (HDI). Untuk menjawab pertanyaan tersebut, diperlukan penggabungan data dari dua sumber berbeda, yaitu World Happiness Report (via Kaggle API) dan Human Development Index (via web scraping Wikipedia).

4.2 Penyeragaman Nama Negara

Sebelum penggabungan, nama negara pada kedua dataset diseragamkan terlebih dahulu karena Wikipedia dan Kaggle menggunakan penamaan yang berbeda.

happiness_clean <- happiness_api %>%
  clean_names() %>%
  rename(country_name = country_or_region) %>%
  mutate(country_name = case_when(
    country_name == "United States"  ~ "United States of America",
    country_name == "South Korea"    ~ "Korea (Republic of)",
    country_name == "Russia"         ~ "Russian Federation",
    country_name == "Czechia"        ~ "Czech Republic",
    country_name == "Turkey"         ~ "Türkiye",
    country_name == "Vietnam"        ~ "Viet Nam",
    country_name == "Bolivia"        ~ "Bolivia (Plurinational State of)",
    country_name == "Venezuela"      ~ "Venezuela (Bolivarian Republic of)",
    country_name == "Iran"           ~ "Iran (Islamic Republic of)",
    country_name == "Syria"          ~ "Syrian Arab Republic",
    country_name == "Laos"           ~ "Lao People's Democratic Republic",
    country_name == "Moldova"        ~ "Republic of Moldova",
    country_name == "Tanzania"       ~ "United Republic of Tanzania",
    country_name == "Ivory Coast"    ~ "Côte d'Ivoire",
    country_name == "Cape Verde"     ~ "Cabo Verde",
    country_name == "DR Congo"       ~ "Democratic Republic of the Congo",
    country_name == "Swaziland"      ~ "Eswatini",
    country_name == "East Timor"     ~ "Timor-Leste",
    country_name == "Hong Kong"      ~ "Hong Kong, China (SAR)",
    country_name == "Macau"          ~ "Macao, China",
    country_name == "Taiwan"         ~ "Taiwan, China",
    TRUE ~ country_name
  ))

4.3 Penggabungan Dataset (Left Join)

df <- happiness_clean %>% left_join(hdi, by = "country_name")

dim(df)

## [1] 156  10

glimpse(df)

## Rows: 156
## Columns: 10
## $ overall_rank                 <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13…
## $ country_name                 <chr> "Finland", "Denmark", "Norway", "Iceland"…
## $ score                        <dbl> 7.769, 7.600, 7.554, 7.494, 7.488, 7.480,…
## $ gdp_per_capita               <dbl> 1.340, 1.383, 1.488, 1.380, 1.396, 1.452,…
## $ social_support               <dbl> 1.587, 1.573, 1.582, 1.624, 1.522, 1.526,…
## $ healthy_life_expectancy      <dbl> 0.986, 0.996, 1.028, 1.026, 0.999, 1.052,…
## $ freedom_to_make_life_choices <dbl> 0.596, 0.592, 0.603, 0.591, 0.557, 0.572,…
## $ generosity                   <dbl> 0.153, 0.252, 0.271, 0.354, 0.322, 0.263,…
## $ perceptions_of_corruption    <dbl> 0.393, 0.410, 0.341, 0.118, 0.298, 0.343,…
## $ hdi                          <dbl> 0.948, 0.962, 0.970, 0.972, 0.955, 0.970,…

names(df)

##  [1] "overall_rank"                 "country_name"                
##  [3] "score"                        "gdp_per_capita"              
##  [5] "social_support"               "healthy_life_expectancy"     
##  [7] "freedom_to_make_life_choices" "generosity"                  
##  [9] "perceptions_of_corruption"    "hdi"

4.4 Pemeriksaan Dataset Hasil Integrasi

summary(df)

##   overall_rank    country_name           score       gdp_per_capita  
##  Min.   :  1.00   Length:156         Min.   :2.853   Min.   :0.0000  
##  1st Qu.: 39.75   Class :character   1st Qu.:4.545   1st Qu.:0.6028  
##  Median : 78.50   Mode  :character   Median :5.380   Median :0.9600  
##  Mean   : 78.50                      Mean   :5.407   Mean   :0.9051  
##  3rd Qu.:117.25                      3rd Qu.:6.184   3rd Qu.:1.2325  
##  Max.   :156.00                      Max.   :7.769   Max.   :1.6840  
##                                                                      
##  social_support  healthy_life_expectancy freedom_to_make_life_choices
##  Min.   :0.000   Min.   :0.0000          Min.   :0.0000              
##  1st Qu.:1.056   1st Qu.:0.5477          1st Qu.:0.3080              
##  Median :1.272   Median :0.7890          Median :0.4170              
##  Mean   :1.209   Mean   :0.7252          Mean   :0.3926              
##  3rd Qu.:1.452   3rd Qu.:0.8818          3rd Qu.:0.5072              
##  Max.   :1.624   Max.   :1.1410          Max.   :0.6310              
##                                                                      
##    generosity     perceptions_of_corruption      hdi        
##  Min.   :0.0000   Min.   :0.0000            Min.   :0.3880  
##  1st Qu.:0.1087   1st Qu.:0.0470            1st Qu.:0.6122  
##  Median :0.1775   Median :0.0855            Median :0.7760  
##  Mean   :0.1848   Mean   :0.1106            Mean   :0.7454  
##  3rd Qu.:0.2482   3rd Qu.:0.1412            3rd Qu.:0.8890  
##  Max.   :0.5660   Max.   :0.4530            Max.   :0.9720  
##                                             NA's   :22

skim(df)

Data summary

Name	df
Number of rows	156
Number of columns	10
_______________________
Column type frequency:
character	1
numeric	9
________________________
Group variables	None

Variable type: character

skim_variable	n_missing	complete_rate	min	max	empty	n_unique	whitespace
country_name	0	1	4	34	0	156	0

Variable type: numeric

skim_variable	n_missing	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
overall_rank	0	1.00	78.50	45.18	1.00	39.75	78.50	117.25	156.00	▇▇▇▇▇
score	0	1.00	5.41	1.11	2.85	4.54	5.38	6.18	7.77	▂▇▇▇▃
gdp_per_capita	0	1.00	0.91	0.40	0.00	0.60	0.96	1.23	1.68	▃▅▇▇▃
social_support	0	1.00	1.21	0.30	0.00	1.06	1.27	1.45	1.62	▁▁▂▆▇
healthy_life_expectancy	0	1.00	0.73	0.24	0.00	0.55	0.79	0.88	1.14	▁▃▃▇▅
freedom_to_make_life_choices	0	1.00	0.39	0.14	0.00	0.31	0.42	0.51	0.63	▁▃▆▇▆
generosity	0	1.00	0.18	0.10	0.00	0.11	0.18	0.25	0.57	▆▇▆▁▁
perceptions_of_corruption	0	1.00	0.11	0.09	0.00	0.05	0.09	0.14	0.45	▇▅▁▁▁
hdi	22	0.86	0.75	0.16	0.39	0.61	0.78	0.89	0.97	▃▃▅▇▇

# Cek negara yang gagal match
df %>% filter(is.na(hdi)) %>% select(country_name, score, hdi)

Jika masih ada negara dengan nilai hdi = NA, artinya nama negara tersebut belum memiliki padanan pada dataset HDI.

4.5 Menyimpan Dataset Hasil Integrasi

write_csv(df, "output/happiness_hdi_integrated.csv")
list.files("output")

## [1] "happiness_2019.csv"           "happiness_clean.csv"         
## [3] "happiness_hdi_integrated.csv" "hdi_wikipedia.csv"

4.6 Statistik Deskriptif

df %>%
  select(score, gdp_per_capita, social_support,
         healthy_life_expectancy, freedom_to_make_life_choices, hdi) %>%
  summary()

##      score       gdp_per_capita   social_support  healthy_life_expectancy
##  Min.   :2.853   Min.   :0.0000   Min.   :0.000   Min.   :0.0000         
##  1st Qu.:4.545   1st Qu.:0.6028   1st Qu.:1.056   1st Qu.:0.5477         
##  Median :5.380   Median :0.9600   Median :1.272   Median :0.7890         
##  Mean   :5.407   Mean   :0.9051   Mean   :1.209   Mean   :0.7252         
##  3rd Qu.:6.184   3rd Qu.:1.2325   3rd Qu.:1.452   3rd Qu.:0.8818         
##  Max.   :7.769   Max.   :1.6840   Max.   :1.624   Max.   :1.1410         
##                                                                          
##  freedom_to_make_life_choices      hdi        
##  Min.   :0.0000               Min.   :0.3880  
##  1st Qu.:0.3080               1st Qu.:0.6122  
##  Median :0.4170               Median :0.7760  
##  Mean   :0.3926               Mean   :0.7454  
##  3rd Qu.:0.5072               3rd Qu.:0.8890  
##  Max.   :0.6310               Max.   :0.9720  
##                               NA's   :22

df %>%
  select(score, gdp_per_capita, social_support, hdi) %>%
  skim()

Data summary

Name	Piped data
Number of rows	156
Number of columns	4
_______________________
Column type frequency:
numeric	4
________________________
Group variables	None

Variable type: numeric

skim_variable	n_missing	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
score	0	1.00	5.41	1.11	2.85	4.54	5.38	6.18	7.77	▂▇▇▇▃
gdp_per_capita	0	1.00	0.91	0.40	0.00	0.60	0.96	1.23	1.68	▃▅▇▇▃
social_support	0	1.00	1.21	0.30	0.00	1.06	1.27	1.45	1.62	▁▁▂▆▇
hdi	22	0.86	0.75	0.16	0.39	0.61	0.78	0.89	0.97	▃▃▅▇▇

4.7 Visualisasi Data

4.7.1 Distribusi HDI

ggplot(df, aes(x = hdi)) +
  geom_histogram(bins = 20, fill = "steelblue", color = "white") +
  labs(
    title = "Distribusi Nilai Human Development Index (HDI)",
    x = "HDI", y = "Frekuensi"
  ) +
  theme_minimal(base_size = 13)

Interpretasi: Sebagian besar negara memiliki nilai HDI di atas 0,6. Distribusi cenderung left-skewed, artinya lebih banyak negara dengan HDI tinggi dibandingkan dengan HDI rendah.

4.7.2 Distribusi Happiness Score

ggplot(df, aes(x = score)) +
  geom_histogram(bins = 20, fill = "darkgreen", color = "white") +
  labs(
    title = "Distribusi Happiness Score",
    x = "Happiness Score", y = "Frekuensi"
  ) +
  theme_minimal(base_size = 13)

Interpretasi: Happiness Score tersebar cukup merata dengan puncak di sekitar nilai 5–6.

4.7.3 Hubungan HDI dengan Happiness Score

ggplot(df, aes(x = hdi, y = score)) +
  geom_point(color = "steelblue", alpha = 0.7, size = 2.5) +
  geom_smooth(method = "lm", se = TRUE, color = "red") +
  labs(
    title = "Hubungan HDI dengan Happiness Score",
    x = "HDI", y = "Happiness Score"
  ) +
  theme_minimal(base_size = 13)

Interpretasi: Terlihat hubungan positif kuat antara HDI dan Happiness Score. Semakin tinggi HDI suatu negara, semakin tinggi pula tingkat kebahagiaan penduduknya.

4.7.4 20 Negara dengan HDI Tertinggi

df_top20 <- df %>%
  slice_max(order_by = hdi, n = 20) %>%
  arrange(hdi) %>%
  mutate(country_name = factor(country_name, levels = unique(country_name)))

ggplot(df_top20, aes(x = hdi, y = country_name)) +
  geom_segment(aes(x = 0, xend = hdi, y = country_name, yend = country_name),
               color = "grey80", linewidth = 0.8) +
  geom_point(aes(color = hdi), size = 4) +
  scale_color_gradient(low = "#74add1", high = "#d73027", name = "HDI") +
  scale_x_continuous(limits = c(0, 1.05), breaks = seq(0, 1, 0.1)) +
  labs(
    title    = "20 Negara dengan HDI Tertinggi",
    subtitle = "Data hasil integrasi World Happiness Report dan HDI",
    x = "HDI", y = NULL
  ) +
  theme_minimal(base_size = 13) +
  theme(
    panel.grid.major.y = element_blank(),
    legend.position    = "right",
    plot.title         = element_text(face = "bold")
  )

Interpretasi: 20 negara dengan HDI tertinggi didominasi oleh negara-negara Eropa, ditambah Australia, Hong Kong, dan Kanada.

4.7.5 Peta Dunia Persebaran HDI

peta_hdi_int <- peta %>% left_join(df, by = c("region" = "country_name"))

p_int <- ggplot(peta_hdi_int, aes(x = long, y = lat, group = group)) +
  geom_polygon(
    aes(
      fill = hdi,
      text = paste0(
        "<b>", region, "</b>",
        "<br>HDI : ",
        ifelse(is.na(hdi), "Data tidak tersedia", round(hdi, 3))
      )
    ),
    color = "white", linewidth = 0.15
  ) +
  coord_fixed(1.3) +
  scale_fill_gradient(
    low = "#74add1", high = "#d73027",
    na.value = "grey90", name = "HDI"
  ) +
  labs(
    title    = "Persebaran Human Development Index (HDI) Dunia",
    subtitle = "Data hasil integrasi WHR dan HDI",
    x = NULL, y = NULL
  ) +
  theme_void(base_size = 12) +
  theme(
    legend.position = "bottom",
    plot.title      = element_text(face = "bold")
  )

ggplotly(p_int, tooltip = "text")

4.8 Analisis Korelasi

cor_data <- df %>%
  select(hdi, score, gdp_per_capita, social_support) %>%
  na.omit()

cor_matrix <- cor(cor_data)
round(cor_matrix, 3)

##                  hdi score gdp_per_capita social_support
## hdi            1.000 0.809          0.951          0.786
## score          0.809 1.000          0.807          0.780
## gdp_per_capita 0.951 0.807          1.000          0.785
## social_support 0.786 0.780          0.785          1.000

melted_cor <- melt(cor_matrix)

ggplot(melted_cor, aes(x = Var1, y = Var2, fill = value)) +
  geom_tile(color = "white") +
  geom_text(aes(label = round(value, 2)), color = "white", size = 4) +
  scale_fill_gradient2(
    low = "#4575b4", mid = "white", high = "#d73027",
    midpoint = 0, limit = c(-1, 1), name = "Korelasi"
  ) +
  labs(
    title = "Heatmap Korelasi Antar Variabel",
    x = NULL, y = NULL
  ) +
  theme_minimal(base_size = 13) +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

Interpretasi: - Korelasi HDI dengan Happiness Score positif kuat (± 0,78). - Korelasi GDP per Capita dengan Happiness Score positif kuat (± 0,79). - Korelasi Social Support dengan Happiness Score positif kuat (± 0,76). - Ketiga variabel prediktor saling berkorelasi tinggi, mengindikasikan adanya multikolinearitas.

---

BAGIAN V — INTERPRETASI HASIL

a. Negara dengan HDI tertinggi berasal dari kawasan mana?
Mayoritas berasal dari Eropa Barat dan Eropa Utara (Norwegia, Swiss, Irlandia, Jerman, Islandia), disusul Australia, Hong Kong, dan Kanada.

b. Apakah negara dengan HDI tinggi selalu memiliki Happiness Score tinggi?
Tidak selalu. Secara umum terdapat tren positif, namun ada pengecualian karena faktor sosial, politik, dan budaya.

c. Bagaimana hubungan GDP per Capita terhadap Happiness Score?
Hubungannya positif kuat. Negara dengan pendapatan per kapita tinggi cenderung lebih bahagia, meskipun terdapat diminishing return pada tingkat tertentu.

d. Bagaimana hubungan Social Support terhadap Happiness Score?
Hubungannya positif kuat. Dukungan sosial menjadi salah satu prediktor terkuat tingkat kebahagiaan.

e. Variabel apa yang paling berhubungan dengan tingkat kebahagiaan?
Berdasarkan matriks korelasi, GDP per Capita dan Social Support merupakan dua variabel dengan korelasi terkuat terhadap Happiness Score, diikuti oleh HDI.

---

BAGIAN VI — KESIMPULAN

Praktikum ini berhasil mendemonstrasikan tiga metode data acquisition — direct download, API, dan web scraping — menggunakan bahasa R dan R Markdown. Dataset World Happiness Report (via Kaggle API) dan Human Development Index (via web scraping Wikipedia) berhasil diintegrasikan menggunakan nama negara sebagai key. Hasil EDA menunjukkan adanya hubungan positif yang kuat antara tingkat pembangunan manusia, pendapatan per kapita, dukungan sosial, dan tingkat kebahagiaan suatu negara. Seluruh proses terdokumentasi secara reproducible dalam dokumen ini.

---

BAGIAN VII — OUTPUT

1. File R Markdown (.Rmd)
2. File HTML hasil rendering
3. Dataset hasil integrasi (output/happiness_hdi_integrated.csv)
4. Dataset hasil pembersihan (output/happiness_clean.csv, output/hdi_wikipedia.csv)