Coba model - Hotel Sentiment Analysis
Rizky Fadilah
september 2023
1 Project Proposal : Hotel Sentiment Analysis
2 Background
Pariwisata telah menjadi aktivitas sosial penting bagi manusia sejak zaman dahulu kala. Industri pariwisata mendapatkan berawal dari motivasi manusia untuk bepergian, pada zaman-zaman sebelumnya orang bepergian untuk bertahan hidup dan kemudian untuk menjalin hubungan perdagangan antara satu sama lain. Seiring manusia berevolusi, manusia mulai bepergian untuk menjelajahi dunia, menjinakkan hewan yang digunakan sebagai alat transportasi hingga penemuan roda membuat perjalanan menjadi lebih mudah dan cepat. Rincian evolusi dan pertumbuhan industri pariwisata dan perhotelan diberikan dalam unit ini untuk memahami bagaimana pariwisata dan perhotelan menanamkan benih-benih industri hotel. 1.
Hotel biasanya diperlukan bagi mereka yang butuh tempat tinggal sementara untuk urusan wisata atau bisnis dan sebagainya. Hotel bukanlah tempat tinggal selamanya atau untuk waktu yang lama seperti kost atau apartemen. Dalam Keputusan Menteri Parpostel no Km 94/HK103/MPPT 1987 disebutkan bahwa hotel merupakan salah satu jenis akomodasi yang menerapkan sebagian atau semua bagian untuk jasa fasilitas penginapan, fasilitator makanan dan minuman serta jasa lainnya bagi masyarakat umum yang dioperasikan secara komersil. 2.
Kunjungan wisatawan mancanegara pada Juni 2023 (Badan Pusat Statistik)
Infografis diatas menunjukkan bahwa memasuki bulan Juni tahun 2023, kunjungan wisatawan mancanegara (wisman) di Indonesia mencapai 1,06 juta kunjungan. Jumlah ini naik 11,44 persen dibandingkan Mei 2023 (month to month) dan naik 119,64 persen dibandingkan bulan yang sama pada tahun lalu (year on year). Wisman yang berkunjung ke Indonesia pada Juni 2023 didominasi oleh wisman yang berasal dari Singapura (16,41 persen), Malaysia (15,88 persen), dan Australia (12,47 persen). Selanjutnya, tingkat Penghunian Kamar (TPK) di hotel bintang pada Juni 2023 mencapai 53,67 persen, naik 3,39 poin secara y-on-y dan naik 4,65 poin secara secara m-to-m. Sejalan dengan TPK hotel bintang, TPK hotel nonbintang pada Juni 2023 mencapai 24,58 persen, naik 0,66 poin secara y-on-y dan naik 0,34 poin secara m-to-m. Sedangkan rata-rata lama tamu menginap di hotel berbintang mengalami kenaikan sebesar 0,05 poin dibandingkan tahun lalu, yaitu mencapai 1,66 hari.
Berdasarkan infografis diatas, jumlah wisatawan di Indonesia terus naik sejak hingga juni tahun lalu hingga tahun ini, dimana kenaikan jumlah wisatawan juga diiringi dengan naiknya kebutuhan akomodasi sementara salah satunya adalah hotel. Namun disamping permintaan kebutuhan akomodasi sementara yang banyak, penyedia layanan akomodasi juga perlu memperhatikan kualitas jasa akomodasinya yang diberikan agar menjadi pilihan pelanggan.
Salah satu cara untuk mempertahankan dan/atau meningkatkan kualitas akomodasi yang diberikan oleh hotel adalah dengan memperhatikan dan melakukan penyesuaian dari umpan balik yang diberikan oleh pengguna berdasarkan pengalamannya selama memakai jasa akomodasi yang diberikan. Projek ini hadir dengan tujuan untuk memberikan sebuah sentiment analysis dari kumpulan umpan balik dari pengguna yang mana informasi yang diperoleh dapat digunakan sebagai saran kepada pihak pengelola dalam mempertahan atau memperbaiki kualitas dari jasa akomodasi yang digunakan.
3 Problem Statement
Untuk dapat mengindentifikasi umpan balik dari pengguna kita dapat melakukan analisa satu per satu umpan balik yang diberikan lalu mengambil kesimpulan apa masukan atau saran yang diberikan oleh pengguna tersebut. Namun hal ini akan menjadi masalah yang dapat memakan banyak waktu ketika umpan balik yang diterima oleh sebuah pengelola berjumlah sangat banyak. Karena itu projek ini hadir untuk membantu melakukan analisa terhadap umpan balik pengguna yang telah menggunakan jasa akomodasi agar pengelola atau pemilik bisnis dapat melakukan penyesesuain dengan cepat.
4 Project Idea
Project ini akan melibatkan pembangunan model machine learning, khususnya dengan memanfaatkan algoritma Naive Bayes dan/atau random forest, atau model machine learning lainnya yang digunakan untuk melakukan analisa umpan balik pelanggan dimana sentiment analysis diterapkan pada umpan balik pengguna untuk mengukur kepuasan pelanggan dan mengidentifikasi area yang perlu ditingkatkan. Projek ini diharapkan menjadi sebuah sistem yang dapat digunakan untuk analisa umpan balik pengguna yang komprehensif dan cepat yang dapat mengumpulkan, memproses, dan menganalisis data umpan balik pengguna. Sistem ini akan memberikan informasi yang dapat ditindaklanjuti untuk membantu bisnis akomodasi dalam meningkatkan penawaran dan kepuasan pelanggan.
5 Problem Scope
Pada projek ini digunakan sebuah data umpan balik pengguna hotel di Indonesia yang telah dikumpulkan dari platform hotel aggregator, AiryRooms HoASA (IndoNLU) Dataset. Kumpulan data mencakup sepuluh aspek berbeda dari kualitas hotel. Terdapat empat kemungkinan kelas sentimen untuk setiap label sentimen: positif, negatif, netral, dan positif-negatif., dalam bahasa Indonesia. Berisi 2.854 dalam format file CSV. Dimana setiap variable dari dataset ini adalah :
review : berisi umpan balik dari pengguna hotel dalam
bentuk kalimat.
ac : tingkat kepuasan pengguna terhadap Air Conditioner
pada hotel.
air_panas : tingkat kepuasan pengguna terhadap keadaan
air panas hotel.
bau : tingkat kepuasan pengguna terhadap kondisi bau
ruangan hotel.
general : tingkat kepuasan pengguna secara umum.
kebersihan : tingkat kepuasan pengguna terhadap
kebersihan hotel.
linen : tingkat kepuasan pengguna terhadap linen
hotel.
service : tingkat kepuasan pengguna terhadap layanan
hotel.
sunrise_meal : tingkat kepuasan pengguna terhadap
sarapan dari hotel.
tv : tingkat kepuasan pengguna terhadap kondisi TV
hotel.
wifi : tingkat kepuasan pengguna terhadap kondisi WIFI
hotel.
Variabel target untuk model pada projek ini adalah
review, ac, air_panas,
general, kebersihan, linen,
service, sunrise_meal, tv, dan
wifi. Terdapat sembilan variable target sehingga akan
dibuat sembilan model machine learning yang berbeda untuk setiap
variable target yang digunakan. Model yang digunakan berupa model
klasifikasi yaitu Naive Bayes dan/atau random forest.
6 Output
Output dari project ini berupa dashboard analysis yang menampilkan Exploratory Data dari hasil sentiment analysis yang dilakukan dari data umpan balik pengguna hotel. User dapat memilih input untuk dilakukan sentiment analysis yaitu : 1. Melakukan input kalimat secara manual pada kolom yang tertera, 2. mengunggah file berisi kumpulan umpan balik pengguna. Selanjutnya, kedua input akan dilakukan analisa umpan balik oleh model.
7 Bussiness Impact / Implementasi serupa
Selain analisa pada umpan balik hotel, projek ini dapat dikembangkan untuk memberikan sentiment analysis pada ranah bisnis bidang lainnya mengingat setiap bisnis tentu akan memilki umpan balik dari pengguna sebagai contoh seperti customer feedback analysis pada retail dan feedback analysis pada sebuah restaurant.
8 Input Data
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(tm)## Warning: package 'tm' was built under R version 4.3.1## Loading required package: NLPlibrary(SnowballC)
library(e1071)
library(caret)## Loading required package: ggplot2##
## Attaching package: 'ggplot2'## The following object is masked from 'package:NLP':
##
## annotate## Loading required package: latticelibrary(purrr)##
## Attaching package: 'purrr'## The following object is masked from 'package:caret':
##
## lifthotel <- read.csv("train_preprocess.csv")
hotelDeskripsi dataset :
review : berisi umpan balik dari pengguna hotel dalam
bentuk kalimat.
ac : tingkat kepuasan pengguna terhadap Air Conditioner
pada hotel.
air_panas : tingkat kepuasan pengguna terhadap keadaan
air panas hotel.
bau : tingkat kepuasan pengguna terhadap kondisi bau
ruangan hotel.
general : tingkat kepuasan pengguna secara umum.
kebersihan : tingkat kepuasan pengguna terhadap
kebersihan hotel.
linen : tingkat kepuasan pengguna terhadap linen
hotel.
service : tingkat kepuasan pengguna terhadap layanan
hotel.
sunrise_meal : tingkat kepuasan pengguna terhadap
sarapan dari hotel.
tv : tingkat kepuasan pengguna terhadap kondisi TV
hotel.
wifi : tingkat kepuasan pengguna terhadap kondisi WIFI
hotel.
Variabel target untuk model pada projek ini adalah
review, ac, air_panas,
general, kebersihan, linen,
service, sunrise_meal, tv, dan
wifi. Terdapat sembilan variable target sehingga akan
dibuat sembilan model machine learning yang berbeda untuk setiap
variable target yang digunakan.
9 Data Wrangling
str(hotel)## 'data.frame': 2612 obs. of 11 variables:
## $ review : chr "kebersihan kurang..." "sangat mengecewakan... hotel bad image, kebersihan kurang, berisik" "Tempat nyaman bersih tapi tv terlalu tinggi tidak bisa di lihat " "semuanya bagus sesuai profile,dan harga promo untuk suite nya jg sangat murah..komplain cuma water heater yg ka"| __truncated__ ...
## $ ac : chr "neut" "neut" "neut" "neut" ...
## $ air_panas : chr "neut" "neut" "neut" "neg" ...
## $ bau : chr "neut" "neut" "neut" "neut" ...
## $ general : chr "neut" "neut" "neut" "pos" ...
## $ kebersihan : chr "neg" "neg" "pos" "neut" ...
## $ linen : chr "neut" "neut" "neut" "neut" ...
## $ service : chr "neut" "neut" "neut" "neut" ...
## $ sunrise_meal: chr "neut" "neut" "neut" "neut" ...
## $ tv : chr "neut" "neut" "neg" "neut" ...
## $ wifi : chr "neut" "neut" "neut" "neut" ...hotel <- hotel %>%
mutate(ac = as.factor(ac),
air_panas = as.factor(air_panas),
bau = as.factor(bau),
general = as.factor(general),
kebersihan = as.factor(kebersihan),
linen = as.factor(linen),
service = as.factor(service),
sunrise_meal = (sunrise_meal),
tv = as.factor(tv),
wifi = as.factor(wifi)
)10 Data Preparation
corpus <- VCorpus(VectorSource(hotel$review))
corpus[[10]]$content## [1] "AC tidak dingin, kunci kamar tidak bisa dipakai karena kehabisan tap card sehingga tiap keluar masuk kamar harus panggil petugas, kamar byk ngengat kecil yg bikin gatal."Menghilangkan Number
corpus_cl <- tm_map(x = corpus, FUN = removeNumbers)
corpus_cl[[10]]$content## [1] "AC tidak dingin, kunci kamar tidak bisa dipakai karena kehabisan tap card sehingga tiap keluar masuk kamar harus panggil petugas, kamar byk ngengat kecil yg bikin gatal."lower case
corpus_cl <- tm_map(x = corpus_cl, FUN = content_transformer(tolower))
corpus_cl[[10]]$content## [1] "ac tidak dingin, kunci kamar tidak bisa dipakai karena kehabisan tap card sehingga tiap keluar masuk kamar harus panggil petugas, kamar byk ngengat kecil yg bikin gatal."Mengubah slang word menjadi kata baku
# Read your CSV file into a data frame
# Read the CSV file containing the slang word mappings
slang_data <- read.csv("new_kamusalay.csv", header = FALSE, stringsAsFactors = FALSE)
# Create a mapping from slang words to their meanings
slang_mapping <- setNames(slang_data$V2, slang_data$V1)
# Create a function to replace slang words with their meanings
replace_slang <- function(text) {
words <- unlist(strsplit(text, " ")) # Split text into words
replaced_words <- sapply(words, function(word) {
if (word %in% names(slang_mapping)) {
return(slang_mapping[word])
} else {
return(word)
}
})
return(paste(replaced_words, collapse = " "))
}
# Create a corpus with your sentences
corpus1 <- Corpus(VectorSource(c(
"I hope aamiin for your success.",
"abis is a common slang word.",
"ado you have any plans?"
)))corpus_cl <- tm_map(x = corpus_cl, FUN = content_transformer(replace_slang))
corpus_cl[[10]]$content## [1] "ac tidak dingin, kunci kamar tidak bisa dipakai karena kehabisan tap card sehingga tiap keluar masuk kamar harus panggil petugas, kamar banyak ngengat kecil yang bikin gatal."Remove Stopword
stopwords_id <- tolower(readLines("stopwords_id.txt"))## Warning in readLines("stopwords_id.txt"): incomplete final line found on
## 'stopwords_id.txt'corpus_cl[[9]]$content## [1] "kamar tidak sesuai dengan gambar...kurang bersih kasurnya tidak empuk..."corpus_cl <- tm_map(x = corpus_cl, FUN = removeWords, stopwords_id)
corpus_cl[[10]]$content## [1] "ac dingin, kunci kamar dipakai kehabisan tap card keluar masuk kamar panggil petugas, kamar ngengat bikin gatal."Remove Punctuation
corpus_cl <- tm_map(x = corpus_cl, FUN = removePunctuation)
corpus_cl[[10]]$content## [1] "ac dingin kunci kamar dipakai kehabisan tap card keluar masuk kamar panggil petugas kamar ngengat bikin gatal"Stemming
library(koRpus)## Loading required package: sylly## For information on available language packages for 'koRpus', run
##
## available.koRpus.lang()
##
## and see ?install.koRpus.lang()##
## Attaching package: 'koRpus'## The following object is masked from 'package:tm':
##
## readTaggedlibrary(tm)
library(katadasaR)library(tokenizers)## Warning: package 'tokenizers' was built under R version 4.3.1library(katadasaR)
stem_katadasaR <- content_transformer(function(x) {
paste(sapply(unlist(tokenizers::tokenize_words(x)), katadasaR::katadasaR), collapse = ' ')
})
corpus_cl <- tm_map(corpus_cl, stem_katadasaR)corpus_cl[[10]]$content## [1] "ac dingin kunci kamar pakai habis tap card keluar masuk kamar panggil tugas kamar ngengat bikin gatal"Remove Whitespaces
corpus_cl <- tm_map(x = corpus_cl, FUN = stripWhitespace)
corpus_cl[[10]]$content## [1] "ac dingin kunci kamar pakai habis tap card keluar masuk kamar panggil tugas kamar ngengat bikin gatal"Dokumen Term Matriks
hotel_dtm <- DocumentTermMatrix(corpus_cl)
# cek singkat struktur dtm
inspect(hotel_dtm)## <<DocumentTermMatrix (documents: 2612, terms: 3493)>>
## Non-/sparse entries: 27955/9095761
## Sparsity : 100%
## Maximal term length: 23
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih dingin kamar kotor kurang mandi panas
## 1148 0 3 1 0 0 6 0 2 1 0
## 1267 1 0 0 1 0 3 0 0 1 0
## 1285 0 0 0 0 0 0 0 1 0 0
## 1351 1 0 1 1 0 1 0 0 1 0
## 2014 3 0 0 2 0 1 0 0 2 1
## 2115 1 0 0 2 0 3 2 0 1 0
## 2188 0 1 1 0 0 2 0 1 1 0
## 2573 1 0 1 0 0 5 6 0 1 0
## 337 0 0 0 1 0 1 0 0 0 0
## 488 0 0 0 0 0 1 0 0 0 011 EDA
library(glue)
library(ggplot2)
library(plotly)##
## Attaching package: 'plotly'## The following object is masked from 'package:ggplot2':
##
## last_plot## The following object is masked from 'package:stats':
##
## filter## The following object is masked from 'package:graphics':
##
## layout# Sorting frekuensi kemunculan secara descending
text_hotel <- as.matrix(hotel_dtm)
hotel_list <- sort(colSums(text_hotel), decreasing = T)
# Membuat data ke bentuk dataframe untuk kebutuhan visualisasi
hotel_df <- data.frame(word = names(hotel_list), freq=hotel_list)
hotel_df <- hotel_df %>%
mutate(label = glue(
"Frekuensi: {freq}"))11.1 Worldcloud
library(wordcloud2)
library(RColorBrewer)
colors.wc <- brewer.pal(8, "Dark2")
wordcloud2(hotel_df, size = 1.5)11.2 Frekuensi Kata Tertinggi
plot <- ggplot(head(hotel_df,7), aes(y = reorder(word,freq), x = freq)) +
geom_col(aes(fill = freq, text = label), show.legend = F) +
labs(x = "Frekuensi",
y = "Terms/Kata",
title = "Frekuensi Kata Tertinggi") +
#scale_x_continuous(labels = "kata") +
scale_fill_gradient(low = "#85c946", high = "#304919") +
theme_minimal() +
theme(axis.text.y = element_text(face = "bold", size = 11))## Warning in geom_col(aes(fill = freq, text = label), show.legend = F): Ignoring
## unknown aesthetics: textggplotly(plot, tooltip = "text")11.3 Distribusi Label
library(ggplot2)
library(tidyr)
df_label <- hotel %>%
select(-review)
# Reshape the dataframe for plotting
df_long <- gather(df_label, key = "column", value = "label")
# Create a grouped bar plot
ggplot(df_long, aes(x = column, fill = label)) +
geom_bar(position = "dodge") +
labs(title = "Distribution of Labels (pos, neut, neg) for Each Column") +
xlab("Columns") +
ylab("Count") +
scale_fill_manual(values = c("pos" = "green", "neut" = "gray", "neg" = "red")) +
theme_minimal() +
theme(axis.text.x = element_text(angle = 45, hjust = 1))
Distribusi tanpa label neutral
# Filter out 'neut' values
df_filtered <- df_long[df_long$label != "neut", ]
# Create a grouped bar plot without 'neut'
ggplot(df_filtered, aes(x = column, fill = label)) +
geom_bar(position = "dodge") +
labs(title = "Distribution of Labels (pos, neg) for Each Column (Excluding 'neut')") +
xlab("Columns") +
ylab("Count") +
scale_fill_manual(values = c("pos" = "green", "neg" = "red")) +
theme_minimal() +
theme(axis.text.x = element_text(angle = 45, hjust = 1))
12 Cross Validation
RNGkind(sample.kind = "Rounding")## Warning in RNGkind(sample.kind = "Rounding"): non-uniform 'Rounding' sampler
## usedset.seed(100)
# train-test splitting
index <- sample(nrow(hotel_dtm), nrow(hotel_dtm)*0.75)
# code here
hotel_train <- hotel_dtm[index,]
hotel_test <- hotel_dtm[-index,]13 Model Fitting
Akan dilakukan pembuatan model terhadap untuk setiap variabel target
13.1 Model Variabel General
Memilih target variable (general)
train_labels_gen <- hotel[index,5]
test_labels_gen <- hotel[-index,5]Cek label
check ham and spam message distribution in train and test set
prop.table(table(train_labels_gen))## train_labels_gen
## neg neut pos
## 0.01480347 0.87289433 0.11230219prop.table(table(test_labels_gen))## test_labels_gen
## neg neut pos
## 0.01684533 0.85145482 0.13169985Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 4)
length(freq)## [1] 680hotel_train_gen_filter <- hotel_train[,freq]
inspect(hotel_train_gen_filter)## <<DocumentTermMatrix (documents: 1959, terms: 680)>>
## Non-/sparse entries: 18063/1314057
## Sparsity : 99%
## Maximal term length: 13
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1351 1 0 1 1 1 0 0 1 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2555 2 1 1 0 3 0 0 0 1 1
## 2573 1 0 1 0 5 6 0 1 0 0
## 275 1 0 1 0 0 3 1 0 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0
## 854 1 3 0 2 3 0 1 0 1 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)## Warning: package 'smotefamily' was built under R version 4.3.1hotel_train_gen_df <- as.data.frame(as.matrix(hotel_train_gen_filter))
train_smote_gen <- SMOTE(X = hotel_train_gen_df,
target = train_labels_gen,
dup_size = 5)
train_smote_gen <- train_smote_gen$data # extract only the balanced dataset
train_smote_gen$class <- as.factor(train_smote_gen$class)
prop.table(table(train_smote_gen$class))##
## neg neut pos
## 0.08269962 0.81273764 0.10456274hotel_test_df <- as.data.frame(as.matrix(hotel_test))
naive_model <- naiveBayes(class ~.,
data = train_smote_gen,
laplace = 1)general_predict_bayes <- predict(naive_model, hotel_test_df)confusionMatrix(general_predict_bayes, test_labels_gen)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 11 556 86
## neut 0 0 0
## pos 0 0 0
##
## Overall Statistics
##
## Accuracy : 0.0168
## 95% CI : (0.0084, 0.0299)
## No Information Rate : 0.8515
## P-Value [Acc > NIR] : 1
##
## Kappa : 0
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 1.00000 0.0000 0.0000
## Specificity 0.00000 1.0000 1.0000
## Pos Pred Value 0.01685 NaN NaN
## Neg Pred Value NaN 0.1485 0.8683
## Prevalence 0.01685 0.8515 0.1317
## Detection Rate 0.01685 0.0000 0.0000
## Detection Prevalence 1.00000 0.0000 0.0000
## Balanced Accuracy 0.50000 0.5000 0.5000library(partykit)## Warning: package 'partykit' was built under R version 4.3.1## Loading required package: grid## Loading required package: libcoin## Warning: package 'libcoin' was built under R version 4.3.1## Loading required package: mvtnorm## Warning: package 'mvtnorm' was built under R version 4.3.1library(randomForest)## Warning: package 'randomForest' was built under R version 4.3.1## randomForest 4.7-1.1## Type rfNews() to see new features/changes/bug fixes.##
## Attaching package: 'randomForest'## The following object is masked from 'package:ggplot2':
##
## margin## The following object is masked from 'package:dplyr':
##
## combine# forest_train_gen <- train_smote_gen %>%
# select(-class)
# forest_label_gen <- unlist(train_smote_gen %>% select(class))
# forest_gen_smote <- randomForest( x = forest_train_gen,
# y = forest_label_gen,
# ntree = 500)# saveRDS(forest_gen_smote, "forest_general.rds")
forest_gen_smote <- readRDS("forest_general.rds")general_predict <- predict(forest_gen_smote, hotel_test_df)confusionMatrix(general_predict, test_labels_gen)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 2 0 0
## neut 7 547 31
## pos 2 9 55
##
## Overall Statistics
##
## Accuracy : 0.925
## 95% CI : (0.902, 0.944)
## No Information Rate : 0.8515
## P-Value [Acc > NIR] : 7.038e-09
##
## Kappa : 0.6648
##
## Mcnemar's Test P-Value : 0.0001004
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.181818 0.9838 0.63953
## Specificity 1.000000 0.6082 0.98060
## Pos Pred Value 1.000000 0.9350 0.83333
## Neg Pred Value 0.986175 0.8676 0.94719
## Prevalence 0.016845 0.8515 0.13170
## Detection Rate 0.003063 0.8377 0.08423
## Detection Prevalence 0.003063 0.8959 0.10107
## Balanced Accuracy 0.590909 0.7960 0.81007Dapat dilihat bahwa hasil performa model random forest lebih baik dari yang diberikan oleh model Naive Bayes, ini dapat terjadi karena pada label terdapat data imbalance. Sehingga model random forest akan digunakan untuk pembuatan model pada variabel selanjutnya
13.2 Model Variabel : AC
Memilih target variable (general)
train_labels_ac <- hotel[index,2]
test_labels_ac <- hotel[-index,2]Cek label
prop.table(table(train_labels_ac))## train_labels_ac
## neg neut pos
## 0.1786626 0.7968351 0.0245023prop.table(table(test_labels_ac))## test_labels_ac
## neg neut pos
## 0.1837672 0.7932619 0.0229709Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 0)
length(freq)## [1] 3493hotel_train_ac_filter <- hotel_train[,freq]
inspect(hotel_train_ac_filter)## <<DocumentTermMatrix (documents: 1959, terms: 3493)>>
## Non-/sparse entries: 20941/6821846
## Sparsity : 100%
## Maximal term length: 23
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1351 1 0 1 1 1 0 0 1 0 0
## 1560 0 1 0 0 3 0 0 0 0 0
## 1914 2 1 2 0 2 1 0 1 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2157 0 0 0 0 2 0 1 0 1 0
## 2573 1 0 1 0 5 6 0 1 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)
hotel_train_ac_df <- as.data.frame(as.matrix(hotel_train_ac_filter))
train_smote_ac <- SMOTE(X = hotel_train_ac_df,
target = train_labels_ac,
dup_size = 5)
train_smote_ac <- train_smote_ac$data # extract only the balanced dataset
train_smote_ac$class <- as.factor(train_smote_ac$class)
prop.table(table(train_smote_ac$class))##
## neg neut pos
## 0.1591633 0.7098681 0.1309686# forest_train_ac <- train_smote_ac %>%
# select(-class)
# forest_label_ac <- unlist(train_smote_ac %>% select(class))
# forest_ac_smote <- randomForest( x = forest_train_ac,
# y = forest_label_ac,
# ntree = 500)#saveRDS(forest_ac_smote, "forest_ac.rds")
forest_ac_smote <- readRDS("forest_ac.rds")ac_predict <- predict(forest_ac_smote, hotel_test_df)confusionMatrix(ac_predict, test_labels_ac)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 77 10 5
## neut 43 508 10
## pos 0 0 0
##
## Overall Statistics
##
## Accuracy : 0.8959
## 95% CI : (0.8699, 0.9182)
## No Information Rate : 0.7933
## P-Value [Acc > NIR] : 2.034e-12
##
## Kappa : 0.6441
##
## Mcnemar's Test P-Value : 9.335e-08
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.6417 0.9807 0.00000
## Specificity 0.9719 0.6074 1.00000
## Pos Pred Value 0.8370 0.9055 NaN
## Neg Pred Value 0.9234 0.8913 0.97703
## Prevalence 0.1838 0.7933 0.02297
## Detection Rate 0.1179 0.7779 0.00000
## Detection Prevalence 0.1409 0.8591 0.00000
## Balanced Accuracy 0.8068 0.7941 0.5000013.3 Model Variabel : Air Panas
Memilih target variable (general)
train_labels_ap <- hotel[index,3]
test_labels_ap <- hotel[-index,3]Cek label
prop.table(table(train_labels_ap))## train_labels_ap
## neg neut pos
## 0.14344053 0.84583971 0.01071975prop.table(table(test_labels_ap))## test_labels_ap
## neg neut pos
## 0.15926493 0.82695253 0.01378254Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 3)
length(freq)## [1] 834hotel_train_ap_filter <- hotel_train[,freq]
inspect(hotel_train_ap_filter)## <<DocumentTermMatrix (documents: 1959, terms: 834)>>
## Non-/sparse entries: 18509/1615297
## Sparsity : 99%
## Maximal term length: 15
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1351 1 0 1 1 1 0 0 1 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2185 0 0 0 1 3 1 0 2 0 0
## 2573 1 0 1 0 5 6 0 1 0 0
## 275 1 0 1 0 0 3 1 0 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0
## 854 1 3 0 2 3 0 1 0 1 0Mengurangi data imbalance dengan SMOTE
hotel_train_ap_df <- as.data.frame(as.matrix(hotel_train_ap_filter))
train_smote_ap <- SMOTE(X = hotel_train_ap_df,
target = train_labels_ap,
dup_size = 30)
train_smote_ap <- train_smote_ap$data # extract only the balanced dataset
train_smote_ap$class <- as.factor(train_smote_ap$class)
prop.table(table(train_smote_ap$class))##
## neg neut pos
## 0.1085361 0.6400154 0.2514484# forest_train_ap <- train_smote_ap %>%
# select(-class)
# forest_label_ap <- unlist(train_smote_ap %>% select(class))
# forest_ap_smote <- randomForest( x = forest_train_ap,
# y = forest_label_ap,
# ntree = 50)#saveRDS(forest_ap_smote, "forest_ap.rds")
forest_ap_smote <- readRDS("forest_ap.rds")ap_predict <- predict(forest_ap_smote, hotel_test_df)confusionMatrix(ap_predict, test_labels_ap)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 92 1 6
## neut 12 539 3
## pos 0 0 0
##
## Overall Statistics
##
## Accuracy : 0.9663
## 95% CI : (0.9494, 0.9788)
## No Information Rate : 0.827
## P-Value [Acc > NIR] : < 2e-16
##
## Kappa : 0.8772
##
## Mcnemar's Test P-Value : 0.00038
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.8846 0.9981 0.00000
## Specificity 0.9872 0.8673 1.00000
## Pos Pred Value 0.9293 0.9729 NaN
## Neg Pred Value 0.9783 0.9899 0.98622
## Prevalence 0.1593 0.8270 0.01378
## Detection Rate 0.1409 0.8254 0.00000
## Detection Prevalence 0.1516 0.8484 0.00000
## Balanced Accuracy 0.9359 0.9327 0.5000013.4 Model Variabel : Bau
Memilih target variable (general)
train_labels_bau <- hotel[index,4]
test_labels_bau <- hotel[-index,4]Cek label
prop.table(table(train_labels_bau))## train_labels_bau
## neg neut pos
## 0.155691679 0.840224604 0.004083716prop.table(table(test_labels_bau))## test_labels_bau
## neg neut pos
## 0.157733538 0.834609495 0.007656968Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 0)
length(freq)## [1] 3493hotel_train_bau_filter <- hotel_train[,freq]
inspect(hotel_train_bau_filter)## <<DocumentTermMatrix (documents: 1959, terms: 3493)>>
## Non-/sparse entries: 20941/6821846
## Sparsity : 100%
## Maximal term length: 23
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1351 1 0 1 1 1 0 0 1 0 0
## 1560 0 1 0 0 3 0 0 0 0 0
## 1914 2 1 2 0 2 1 0 1 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2157 0 0 0 0 2 0 1 0 1 0
## 2573 1 0 1 0 5 6 0 1 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)
hotel_train_bau_df <- as.data.frame(as.matrix(hotel_train_bau_filter))
train_smote_bau <- SMOTE(X = hotel_train_bau_df,
target = train_labels_bau,
dup_size = 40)
train_smote_bau <- train_smote_bau$data # extract only the balanced dataset
train_smote_bau$class <- as.factor(train_smote_bau$class)
prop.table(table(train_smote_bau$class))##
## neg neut pos
## 0.1338306 0.7222466 0.1439228# forest_train_bau <- train_smote_bau %>%
# select(-class)
# forest_label_bau <- unlist(train_smote_bau %>% select(class))
# forest_bau_smote <- randomForest( x = forest_train_bau,
# y = forest_label_bau,
# ntree = 100)# saveRDS(forest_bau_smote, "forest_bau.rds")
forest_bau_smote <- readRDS("forest_bau.rds")bau_predict <- predict(forest_bau_smote, hotel_test_df)confusionMatrix(bau_predict, test_labels_bau)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 84 12 2
## neut 19 533 3
## pos 0 0 0
##
## Overall Statistics
##
## Accuracy : 0.9449
## 95% CI : (0.9245, 0.9611)
## No Information Rate : 0.8346
## P-Value [Acc > NIR] : < 2e-16
##
## Kappa : 0.7935
##
## Mcnemar's Test P-Value : 0.08654
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.8155 0.9780 0.000000
## Specificity 0.9745 0.7963 1.000000
## Pos Pred Value 0.8571 0.9604 NaN
## Neg Pred Value 0.9658 0.8776 0.992343
## Prevalence 0.1577 0.8346 0.007657
## Detection Rate 0.1286 0.8162 0.000000
## Detection Prevalence 0.1501 0.8499 0.000000
## Balanced Accuracy 0.8950 0.8871 0.50000013.5 Model Variabel : kebersihan
Memilih target variable (general)
train_labels_bersih <- hotel[index,6]
test_labels_bersih <- hotel[-index,6]Cek label
prop.table(table(train_labels_bersih))## train_labels_bersih
## neg neut pos
## 0.32159265 0.58448188 0.09392547prop.table(table(test_labels_bersih))## test_labels_bersih
## neg neut pos
## 0.30934150 0.59418070 0.09647779Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 5)
length(freq)## [1] 564hotel_train_bersih_filter <- hotel_train[,freq]
inspect(hotel_train_bersih_filter)## <<DocumentTermMatrix (documents: 1959, terms: 564)>>
## Non-/sparse entries: 17614/1087262
## Sparsity : 98%
## Maximal term length: 13
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1175 0 0 0 0 5 1 0 1 0 0
## 1351 1 0 1 1 1 0 0 1 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2573 1 0 1 0 5 6 0 1 0 0
## 275 1 0 1 0 0 3 1 0 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0
## 854 1 3 0 2 3 0 1 0 1 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)
hotel_train_bersih_df <- as.data.frame(as.matrix(hotel_train_bersih_filter))
train_smote_bersih <- SMOTE(X = hotel_train_bersih_df,
target = train_labels_bersih,
dup_size = 10)
train_smote_bersih <- train_smote_bersih$data # extract only the balanced dataset
train_smote_bersih$class <- as.factor(train_smote_bersih$class)
prop.table(table(train_smote_bersih$class))##
## neg neut pos
## 0.1658331 0.3013951 0.5327718# forest_train_bersih <- train_smote_bersih %>%
# select(-class)
# forest_label_bersih <- unlist(train_smote_bersih %>% select(class))
# forest_bersih_smote <- randomForest( x = forest_train_bersih,
# y = forest_label_bersih,
# ntree = 200)# saveRDS(forest_bersih_smote, "forest_bersih.rds")
forest_bersih_smote <- readRDS("forest_bersih.rds")bersih_predict <- predict(forest_bersih_smote, hotel_test_df)confusionMatrix(bersih_predict, test_labels_bersih)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 159 17 25
## neut 36 367 2
## pos 7 4 36
##
## Overall Statistics
##
## Accuracy : 0.8606
## 95% CI : (0.8317, 0.8863)
## No Information Rate : 0.5942
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.7367
##
## Mcnemar's Test P-Value : 0.0005311
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.7871 0.9459 0.57143
## Specificity 0.9069 0.8566 0.98136
## Pos Pred Value 0.7910 0.9062 0.76596
## Neg Pred Value 0.9049 0.9153 0.95545
## Prevalence 0.3093 0.5942 0.09648
## Detection Rate 0.2435 0.5620 0.05513
## Detection Prevalence 0.3078 0.6202 0.07198
## Balanced Accuracy 0.8470 0.9012 0.7763913.6 Model Variabel : Linen
Memilih target variable (general)
train_labels_linen <- hotel[index,7]
test_labels_linen <- hotel[-index,7]Cek label
prop.table(table(train_labels_linen))## train_labels_linen
## neg neut pos
## 0.26748341 0.70444104 0.02807555prop.table(table(test_labels_linen))## test_labels_linen
## neg neut pos
## 0.26186830 0.70597243 0.03215926Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 5)
length(freq)## [1] 564hotel_train_linen_filter <- hotel_train[,freq]
inspect(hotel_train_linen_filter)## <<DocumentTermMatrix (documents: 1959, terms: 564)>>
## Non-/sparse entries: 17614/1087262
## Sparsity : 98%
## Maximal term length: 13
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1175 0 0 0 0 5 1 0 1 0 0
## 1351 1 0 1 1 1 0 0 1 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2573 1 0 1 0 5 6 0 1 0 0
## 275 1 0 1 0 0 3 1 0 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0
## 854 1 3 0 2 3 0 1 0 1 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)
hotel_train_linen_df <- as.data.frame(as.matrix(hotel_train_linen_filter))
train_smote_linen <- SMOTE(X = hotel_train_linen_df,
target = train_labels_linen,
dup_size = 20)
train_smote_linen <- train_smote_linen$data # extract only the balanced dataset
train_smote_linen$class <- as.factor(train_smote_linen$class)
prop.table(table(train_smote_linen$class))##
## neg neut pos
## 0.1712978 0.4511278 0.3775744# forest_train_linen <- train_smote_linen %>%
# select(-class)
# forest_label_linen <- unlist(train_smote_linen %>% select(class))
# forest_linen_smote <- randomForest( x = forest_train_linen,
# y = forest_label_linen,
# ntree = 200)# saveRDS(forest_linen_smote, "forest_linen.rds")
forest_linen_smote <- readRDS("forest_linen.rds")linen_predict <- predict(forest_linen_smote, hotel_test_df)confusionMatrix(linen_predict, test_labels_linen)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 132 10 14
## neut 39 450 6
## pos 0 1 1
##
## Overall Statistics
##
## Accuracy : 0.8928
## 95% CI : (0.8665, 0.9155)
## No Information Rate : 0.706
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.7335
##
## Mcnemar's Test P-Value : 1.386e-07
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.7719 0.9761 0.047619
## Specificity 0.9502 0.7656 0.998418
## Pos Pred Value 0.8462 0.9091 0.500000
## Neg Pred Value 0.9215 0.9304 0.969278
## Prevalence 0.2619 0.7060 0.032159
## Detection Rate 0.2021 0.6891 0.001531
## Detection Prevalence 0.2389 0.7580 0.003063
## Balanced Accuracy 0.8611 0.8709 0.52301813.7 Model Variabel : Service
Memilih target variable (general)
train_labels_service <- hotel[index,8]
test_labels_service <- hotel[-index,8]Cek label
prop.table(table(train_labels_service))## train_labels_service
## neg neut pos
## 0.1720265 0.7207759 0.1071975prop.table(table(test_labels_service))## test_labels_service
## neg neut pos
## 0.1454824 0.7320061 0.1225115Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 4)
length(freq)## [1] 680hotel_train_service_filter <- hotel_train[,freq]
inspect(hotel_train_service_filter)## <<DocumentTermMatrix (documents: 1959, terms: 680)>>
## Non-/sparse entries: 18063/1314057
## Sparsity : 99%
## Maximal term length: 13
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1351 1 0 1 1 1 0 0 1 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2555 2 1 1 0 3 0 0 0 1 1
## 2573 1 0 1 0 5 6 0 1 0 0
## 275 1 0 1 0 0 3 1 0 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0
## 854 1 3 0 2 3 0 1 0 1 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)
hotel_train_service_df <- as.data.frame(as.matrix(hotel_train_service_filter))
train_smote_service <- SMOTE(X = hotel_train_service_df,
target = train_labels_service,
dup_size = 2)
train_smote_service <- train_smote_service$data # extract only the balanced dataset
train_smote_service$class <- as.factor(train_smote_service$class)
prop.table(table(train_smote_service$class))##
## neg neut pos
## 0.1416562 0.5935267 0.2648172# forest_train_service <- train_smote_service %>%
# select(-class)
# forest_label_service <- unlist(train_smote_service %>% select(class))
# forest_service_smote <- randomForest( x = forest_train_service,
# y = forest_label_service,
# ntree = 1000)# saveRDS(forest_service_smote, "forest_service.rds")
forest_service_smote <- readRDS("forest_service.rds")service_predict <- predict(forest_service_smote, hotel_test_df)confusionMatrix(service_predict, test_labels_service)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 56 6 10
## neut 24 469 11
## pos 15 3 59
##
## Overall Statistics
##
## Accuracy : 0.8943
## 95% CI : (0.8682, 0.9169)
## No Information Rate : 0.732
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.7388
##
## Mcnemar's Test P-Value : 0.0009515
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.58947 0.9812 0.73750
## Specificity 0.97133 0.8000 0.96859
## Pos Pred Value 0.77778 0.9306 0.76623
## Neg Pred Value 0.93287 0.9396 0.96354
## Prevalence 0.14548 0.7320 0.12251
## Detection Rate 0.08576 0.7182 0.09035
## Detection Prevalence 0.11026 0.7718 0.11792
## Balanced Accuracy 0.78040 0.8906 0.8530413.8 Model Variabel : TV
Memilih target variable (general)
train_labels_tv <- hotel[index,10]
test_labels_tv <- hotel[-index,10]Cek label
prop.table(table(train_labels_tv))## train_labels_tv
## neg neut pos
## 0.090352221 0.902501276 0.007146503prop.table(table(test_labels_tv))## test_labels_tv
## neg neut pos
## 0.073506891 0.918836141 0.007656968Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 2)
length(freq)## [1] 1191hotel_train_tv_filter <- hotel_train[,freq]
inspect(hotel_train_tv_filter)## <<DocumentTermMatrix (documents: 1959, terms: 1191)>>
## Non-/sparse entries: 19190/2313979
## Sparsity : 99%
## Maximal term length: 16
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1251 1 0 1 0 5 3 0 1 0 0
## 1351 1 0 1 1 1 0 0 1 0 0
## 1560 0 1 0 0 3 0 0 0 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2573 1 0 1 0 5 6 0 1 0 0
## 275 1 0 1 0 0 3 1 0 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)
hotel_train_tv_df <- as.data.frame(as.matrix(hotel_train_tv_filter))
train_smote_tv <- SMOTE(X = hotel_train_tv_df,
target = train_labels_tv,
dup_size = 20)
train_smote_tv <- train_smote_tv$data # extract only the balanced dataset
train_smote_tv$class <- as.factor(train_smote_tv$class)
prop.table(table(train_smote_linen$class))##
## neg neut pos
## 0.1712978 0.4511278 0.3775744forest_train_tv <- train_smote_tv %>%
select(-class)
forest_label_tv <- unlist(train_smote_tv %>% select(class))
forest_tv_smote <- randomForest( x = forest_train_tv,
y = forest_label_tv,
ntree = 100)saveRDS(forest_tv_smote, "forest_tv.rds")
forest_tv_smote <- readRDS("forest_tv.rds")tv_predict <- predict(forest_tv_smote, hotel_test_df)confusionMatrix(tv_predict, test_labels_tv)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 46 5 4
## neut 2 595 1
## pos 0 0 0
##
## Overall Statistics
##
## Accuracy : 0.9816
## 95% CI : (0.9681, 0.9905)
## No Information Rate : 0.9188
## P-Value [Acc > NIR] : 3.185e-12
##
## Kappa : 0.8794
##
## Mcnemar's Test P-Value : 0.09851
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.95833 0.9917 0.000000
## Specificity 0.98512 0.9434 1.000000
## Pos Pred Value 0.83636 0.9950 NaN
## Neg Pred Value 0.99666 0.9091 0.992343
## Prevalence 0.07351 0.9188 0.007657
## Detection Rate 0.07044 0.9112 0.000000
## Detection Prevalence 0.08423 0.9158 0.000000
## Balanced Accuracy 0.97173 0.9675 0.50000013.9 Model Variabel : Wifi
Memilih target variable (general)
train_labels_wifi <- hotel[index,11]
test_labels_wifi <- hotel[-index,11]Cek label
prop.table(table(train_labels_wifi))## train_labels_wifi
## neg neut pos
## 0.143950995 0.846860643 0.009188361prop.table(table(test_labels_wifi))## test_labels_wifi
## neg neut pos
## 0.13476263 0.85298622 0.01225115Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 2)
length(freq)## [1] 1191hotel_train_wifi_filter <- hotel_train[,freq]
inspect(hotel_train_wifi_filter)## <<DocumentTermMatrix (documents: 1959, terms: 1191)>>
## Non-/sparse entries: 19190/2313979
## Sparsity : 99%
## Maximal term length: 16
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1251 1 0 1 0 5 3 0 1 0 0
## 1351 1 0 1 1 1 0 0 1 0 0
## 1560 0 1 0 0 3 0 0 0 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2573 1 0 1 0 5 6 0 1 0 0
## 275 1 0 1 0 0 3 1 0 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)
hotel_train_wifi_df <- as.data.frame(as.matrix(hotel_train_wifi_filter))
train_smote_wifi <- SMOTE(X = hotel_train_wifi_df,
target = train_labels_wifi,
dup_size = 24)
train_smote_wifi <- train_smote_wifi$data # extract only the balanced dataset
train_smote_wifi$class <- as.factor(train_smote_wifi$class)
prop.table(table(train_smote_wifi$class))##
## neg neut pos
## 0.1179423 0.6938519 0.1882058# forest_train_wifi <- train_smote_wifi %>%
# select(-class)
# forest_label_wifi <- unlist(train_smote_wifi %>% select(class))
# forest_wifi_smote <- randomForest( x = forest_train_wifi,
# y = forest_label_wifi,
# ntree = 100)# saveRDS(forest_wifi_smote, "forest_wifi.rds")
forest_wifi_smote <- readRDS("forest_wifi.rds")wifi_predict <- predict(forest_wifi_smote, hotel_test_df)confusionMatrix(wifi_predict, test_labels_wifi)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 76 0 4
## neut 12 557 4
## pos 0 0 0
##
## Overall Statistics
##
## Accuracy : 0.9694
## 95% CI : (0.9531, 0.9812)
## No Information Rate : 0.853
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.8697
##
## Mcnemar's Test P-Value : 0.0001697
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.8636 1.0000 0.00000
## Specificity 0.9929 0.8333 1.00000
## Pos Pred Value 0.9500 0.9721 NaN
## Neg Pred Value 0.9791 1.0000 0.98775
## Prevalence 0.1348 0.8530 0.01225
## Detection Rate 0.1164 0.8530 0.00000
## Detection Prevalence 0.1225 0.8775 0.00000
## Balanced Accuracy 0.9283 0.9167 0.5000013.10 Model Variabel : sunrise meal (sarapan)
Memilih target variable (general)
train_labels_sm <- hotel[index,9]
test_labels_sm <- as.factor(hotel[-index,9])Cek label
prop.table(table(train_labels_sm))## train_labels_sm
## neg neut pos
## 0.04798367 0.91985707 0.03215926prop.table(table(test_labels_sm))## test_labels_sm
## neg neut pos
## 0.03675345 0.93261868 0.03062787Data Preparation lanjutan
Menghilangkan kata jarang muncul
print(dim(hotel_train))## [1] 1959 3493Prediktor memiliki 3493 kata, dilakukan penghapusan kata yang jarang muncul (dibawah 4)
freq <- findFreqTerms(hotel_train,lowfreq = 2)
length(freq)## [1] 1191hotel_train_sm_filter <- hotel_train[,freq]
inspect(hotel_train_sm_filter)## <<DocumentTermMatrix (documents: 1959, terms: 1191)>>
## Non-/sparse entries: 19190/2313979
## Sparsity : 99%
## Maximal term length: 16
## Weighting : term frequency (tf)
## Sample :
## Terms
## Docs air baik bau bersih kamar kotor kurang mandi nyaman panas
## 1251 1 0 1 0 5 3 0 1 0 0
## 1351 1 0 1 1 1 0 0 1 0 0
## 1560 0 1 0 0 3 0 0 0 0 0
## 2115 1 0 0 2 3 2 0 1 0 0
## 2573 1 0 1 0 5 6 0 1 0 0
## 275 1 0 1 0 0 3 1 0 0 0
## 337 0 0 0 1 1 0 0 0 1 0
## 461 0 0 1 0 3 0 0 1 0 0
## 488 0 0 0 0 1 0 0 0 1 0
## 540 0 0 0 1 1 0 0 0 0 0Mengurangi data imbalance dengan SMOTE
library(smotefamily)
hotel_train_sm_df <- as.data.frame(as.matrix(hotel_train_sm_filter))
train_smote_sm <- SMOTE(X = hotel_train_sm_df,
target = train_labels_sm,
dup_size = 15)
train_smote_sm <- train_smote_sm$data # extract only the balanced dataset
train_smote_sm$class <- as.factor(train_smote_sm$class)
prop.table(table(train_smote_sm$class))##
## neg neut pos
## 0.03236915 0.62052342 0.34710744# forest_train_sm <- train_smote_sm %>%
# select(-class)
# forest_label_sm <- unlist(train_smote_sm %>% select(class))
# forest_sm_smote <- randomForest( x = forest_train_sm,
# y = forest_label_sm,
# ntree = 500)# saveRDS(forest_sm_smote, "forest_sm.rds")
forest_sm_smote <- readRDS("forest_sm.rds")sm_predict <- predict(forest_sm_smote, hotel_test_df)confusionMatrix(sm_predict, test_labels_sm)## Confusion Matrix and Statistics
##
## Reference
## Prediction neg neut pos
## neg 3 0 1
## neut 18 608 12
## pos 3 1 7
##
## Overall Statistics
##
## Accuracy : 0.9464
## 95% CI : (0.9262, 0.9624)
## No Information Rate : 0.9326
## P-Value [Acc > NIR] : 0.08904
##
## Kappa : 0.3914
##
## Mcnemar's Test P-Value : 3.13e-06
##
## Statistics by Class:
##
## Class: neg Class: neut Class: pos
## Sensitivity 0.125000 0.9984 0.35000
## Specificity 0.998410 0.3182 0.99368
## Pos Pred Value 0.750000 0.9530 0.63636
## Neg Pred Value 0.967643 0.9333 0.97975
## Prevalence 0.036753 0.9326 0.03063
## Detection Rate 0.004594 0.9311 0.01072
## Detection Prevalence 0.006126 0.9770 0.01685
## Balanced Accuracy 0.561705 0.6583 0.67184