LBB C2
Adrian Djaya Prathama
2024-02-29
1 Introduction
Breast cancer is a disease in which cells in the breast grow out of control. There are different kinds of breast cancer. The kind of breast cancer depends on which cells in the breast turn into cancer. Breast cancer is a formidable health challenge affecting millions of women globally. Characterized by abnormal cell growth in breast tissue, early detection is paramount for effective treatment and improved outcomes. Breast cancer can spread outside the breast through blood vessels and lymph vessels. When breast cancer spreads to other parts of the body, it is said to have metastasized.
In this LBB we will make a model to predict wheter it’s Benign (non-cancerous) or Maligant (cancerous)
2 Data Preparation
2.1 Import Library
library(caret)
library(dplyr)
library(e1071)
library(inspectdf)
library(ROCR)
library(partykit)
library(rsample)
library(randomForest)2.2 Import Data
cancer <- read.csv("data_input/data.csv")
glimpse(cancer)## Rows: 569
## Columns: 33
## $ id <int> 842302, 842517, 84300903, 84348301, 84358402, …
## $ diagnosis <chr> "M", "M", "M", "M", "M", "M", "M", "M", "M", "…
## $ radius_mean <dbl> 17.990, 20.570, 19.690, 11.420, 20.290, 12.450…
## $ texture_mean <dbl> 10.38, 17.77, 21.25, 20.38, 14.34, 15.70, 19.9…
## $ perimeter_mean <dbl> 122.80, 132.90, 130.00, 77.58, 135.10, 82.57, …
## $ area_mean <dbl> 1001.0, 1326.0, 1203.0, 386.1, 1297.0, 477.1, …
## $ smoothness_mean <dbl> 0.11840, 0.08474, 0.10960, 0.14250, 0.10030, 0…
## $ compactness_mean <dbl> 0.27760, 0.07864, 0.15990, 0.28390, 0.13280, 0…
## $ concavity_mean <dbl> 0.30010, 0.08690, 0.19740, 0.24140, 0.19800, 0…
## $ concave.points_mean <dbl> 0.14710, 0.07017, 0.12790, 0.10520, 0.10430, 0…
## $ symmetry_mean <dbl> 0.2419, 0.1812, 0.2069, 0.2597, 0.1809, 0.2087…
## $ fractal_dimension_mean <dbl> 0.07871, 0.05667, 0.05999, 0.09744, 0.05883, 0…
## $ radius_se <dbl> 1.0950, 0.5435, 0.7456, 0.4956, 0.7572, 0.3345…
## $ texture_se <dbl> 0.9053, 0.7339, 0.7869, 1.1560, 0.7813, 0.8902…
## $ perimeter_se <dbl> 8.589, 3.398, 4.585, 3.445, 5.438, 2.217, 3.18…
## $ area_se <dbl> 153.40, 74.08, 94.03, 27.23, 94.44, 27.19, 53.…
## $ smoothness_se <dbl> 0.006399, 0.005225, 0.006150, 0.009110, 0.0114…
## $ compactness_se <dbl> 0.049040, 0.013080, 0.040060, 0.074580, 0.0246…
## $ concavity_se <dbl> 0.05373, 0.01860, 0.03832, 0.05661, 0.05688, 0…
## $ concave.points_se <dbl> 0.015870, 0.013400, 0.020580, 0.018670, 0.0188…
## $ symmetry_se <dbl> 0.03003, 0.01389, 0.02250, 0.05963, 0.01756, 0…
## $ fractal_dimension_se <dbl> 0.006193, 0.003532, 0.004571, 0.009208, 0.0051…
## $ radius_worst <dbl> 25.38, 24.99, 23.57, 14.91, 22.54, 15.47, 22.8…
## $ texture_worst <dbl> 17.33, 23.41, 25.53, 26.50, 16.67, 23.75, 27.6…
## $ perimeter_worst <dbl> 184.60, 158.80, 152.50, 98.87, 152.20, 103.40,…
## $ area_worst <dbl> 2019.0, 1956.0, 1709.0, 567.7, 1575.0, 741.6, …
## $ smoothness_worst <dbl> 0.1622, 0.1238, 0.1444, 0.2098, 0.1374, 0.1791…
## $ compactness_worst <dbl> 0.6656, 0.1866, 0.4245, 0.8663, 0.2050, 0.5249…
## $ concavity_worst <dbl> 0.71190, 0.24160, 0.45040, 0.68690, 0.40000, 0…
## $ concave.points_worst <dbl> 0.26540, 0.18600, 0.24300, 0.25750, 0.16250, 0…
## $ symmetry_worst <dbl> 0.4601, 0.2750, 0.3613, 0.6638, 0.2364, 0.3985…
## $ fractal_dimension_worst <dbl> 0.11890, 0.08902, 0.08758, 0.17300, 0.07678, 0…
## $ X <lgl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA…In this case we will use only the mean data. Therefore we will remove
SE and Worst data. Also we want to change diagnosis type
into factor.
cancer_clean <- cancer %>%
select(-c(1, 13:33)) %>%
mutate(diagnosis = as.factor(diagnosis))
glimpse(cancer_clean)## Rows: 569
## Columns: 11
## $ diagnosis <fct> M, M, M, M, M, M, M, M, M, M, M, M, M, M, M, M,…
## $ radius_mean <dbl> 17.990, 20.570, 19.690, 11.420, 20.290, 12.450,…
## $ texture_mean <dbl> 10.38, 17.77, 21.25, 20.38, 14.34, 15.70, 19.98…
## $ perimeter_mean <dbl> 122.80, 132.90, 130.00, 77.58, 135.10, 82.57, 1…
## $ area_mean <dbl> 1001.0, 1326.0, 1203.0, 386.1, 1297.0, 477.1, 1…
## $ smoothness_mean <dbl> 0.11840, 0.08474, 0.10960, 0.14250, 0.10030, 0.…
## $ compactness_mean <dbl> 0.27760, 0.07864, 0.15990, 0.28390, 0.13280, 0.…
## $ concavity_mean <dbl> 0.30010, 0.08690, 0.19740, 0.24140, 0.19800, 0.…
## $ concave.points_mean <dbl> 0.14710, 0.07017, 0.12790, 0.10520, 0.10430, 0.…
## $ symmetry_mean <dbl> 0.2419, 0.1812, 0.2069, 0.2597, 0.1809, 0.2087,…
## $ fractal_dimension_mean <dbl> 0.07871, 0.05667, 0.05999, 0.09744, 0.05883, 0.…Data is cleaned. From glimpse function above we can see
there are 569 rows with 11 columns.
All the columns are:
diagnosis: The diagnosis of breast tissue samples, where ‘M’ indicates malignant (cancerous) and ‘B’ indicates benign (non-cancerous)radius_mean: Radius of the nucleus of the cellstexture_mean: Texture of the cellsperimeter_mean: perimeter of the cellsarea_mean: area of the cellssmoothness_mean: Smoothness (local variation in radius lengths) of the cellscompactness_mean: Compactnessconcavity_mean: Severity of concave portions of the contourconcave.points_mean: Number of concave portions of the contoursymmetry_mean: Symmetry of the cellsfractal_dimension_mean: Fractal dimension
Next we will check data proportion
cancer_clean$diagnosis %>%
table() %>%
prop.table()## .
## B M
## 0.6274165 0.3725835Data proportion is 62.7 : 37.2. The proportion is balanced enough to be process further.
Next we will check is there any missing value in the data before we start modeling.
colSums(is.na(cancer_clean))## diagnosis radius_mean texture_mean
## 0 0 0
## perimeter_mean area_mean smoothness_mean
## 0 0 0
## compactness_mean concavity_mean concave.points_mean
## 0 0 0
## symmetry_mean fractal_dimension_mean
## 0 0There is no missing value in the data.
3 Modeling
3.1 Naive Bayes
3.1.1 Splitting Train-Test
First step before creating a model is we will seperate our data into train and test data randomly from our cleaned data. We will use 80% of our cleaned data as train data and the rest for test data
set.seed(189)
index <- sample(nrow(cancer_clean), nrow(cancer_clean)*0.80)
cancer_train <- cancer_clean[index, ]
cancer_test <- cancer_clean[-index, ]cancer_train$diagnosis %>%
table() %>%
prop.table()## .
## B M
## 0.632967 0.367033cancer_test$diagnosis %>%
table() %>%
prop.table()## .
## B M
## 0.6052632 0.3947368Both train and test data proportion is still balance.
3.1.2 Creating the Model
We will use diagnosis column as our target variable and
use the rest of the columns as our predictor variables in our model. We
will also use laplace = 1 just in case that there are some 0 values
inside the row.
model_bayes <- naiveBayes(formula = diagnosis ~ .,
data = cancer_train,
laplace = 1)3.1.3 Naive Bayes Model Evaluation
Then we will predict our new model with the test data. We use
"class" for our type because we want our output to be
cancer_predict <- predict(model_bayes,
newdata = cancer_test,
type = "class")Then we will interpret it with confusionMatrix
function
confusionMatrix(data = cancer_predict,
reference = cancer_test$diagnosis)## Confusion Matrix and Statistics
##
## Reference
## Prediction B M
## B 66 2
## M 3 43
##
## Accuracy : 0.9561
## 95% CI : (0.9006, 0.9856)
## No Information Rate : 0.6053
## P-Value [Acc > NIR] : <2e-16
##
## Kappa : 0.9086
##
## Mcnemar's Test P-Value : 1
##
## Sensitivity : 0.9565
## Specificity : 0.9556
## Pos Pred Value : 0.9706
## Neg Pred Value : 0.9348
## Prevalence : 0.6053
## Detection Rate : 0.5789
## Detection Prevalence : 0.5965
## Balanced Accuracy : 0.9560
##
## 'Positive' Class : B
## Based from our test, we knew that:
- Our model could predict correctly with 95.61% accuracy.
- Our model could predict recall with 95.65%.
- Our model could predict Precission with 97.06%.
- Our model could predict Specificity with 95.56%.
3.1.4 ROC & AUC
To evaluate our model further we will use ROC and AUC method.
3.1.4.1 ROC
First we will predict in odd form
cancer_pred_test <- predict(model_bayes,
cancer_test,
type = "raw")Then we will take only the positive class
cancer_pred_prob <- cancer_pred_test[, 2]After that we will create a prediction using the positive class
cancer_roc <- prediction(predictions = cancer_pred_prob,
labels = cancer_test$diagnosis)Then we will plot our new prediction
model_roc <- performance(cancer_roc,
"tpr",
"fpr")
plot(model_roc)
abline(0,1, lty = 2)
From plot above we can see that our model is good enough. Then we will
use AUC to validate our argument
3.1.4.2 AUC
model_auc <- performance(cancer_roc, "auc")
model_auc@y.values## [[1]]
## [1] 0.9847021y.values shows 0.9847 which means value is closer to 1,
model is good enough to be separate between Benign class and malignant
class.
3.2 Decision Tree
Next we will create decision tree model with confidence interval by 50%
dtree_model <- ctree(formula = diagnosis ~ .,
data = cancer_clean,
control = ctree_control(mincriterion = 0.5,
minsplit = 10,
minbucket = 7))
plot(dtree_model, type = "simple")
3.2.1 Decision Tree Model Evaluation
To evaluate our new model, we need to predict it first using our test data.
dtree_model_pred <- predict(dtree_model,
cancer_test,
type = "response")Next we will use confusionMatrix to evaluate the
model
confusionMatrix(dtree_model_pred,
cancer_test$diagnosis,
positive = "M")## Confusion Matrix and Statistics
##
## Reference
## Prediction B M
## B 68 2
## M 1 43
##
## Accuracy : 0.9737
## 95% CI : (0.925, 0.9945)
## No Information Rate : 0.6053
## P-Value [Acc > NIR] : <2e-16
##
## Kappa : 0.9447
##
## Mcnemar's Test P-Value : 1
##
## Sensitivity : 0.9556
## Specificity : 0.9855
## Pos Pred Value : 0.9773
## Neg Pred Value : 0.9714
## Prevalence : 0.3947
## Detection Rate : 0.3772
## Detection Prevalence : 0.3860
## Balanced Accuracy : 0.9705
##
## 'Positive' Class : M
## Based on our model evaluation, we can conclude:
- Our Decision Tree model could predict correctly with 97.37% accuracy.
- Our Decision Tree model could predict recall with 95.56%.
- Our Decision Tree model could predict Precission with 97.73%.
- Our Decision Tree model could predict Specificity with 98.55%.
3.2.2 Pruning Tree
We will try improve our Decision Tree model by improving our confidence interval to 98%
dtree_model_prun <- ctree(formula = diagnosis ~ .,
data = cancer_clean,
control = ctree_control(mincriterion = 0.98,
minsplit = 10,
minbucket = 7))
plot(dtree_model, type = "simple")
dtree_model_prun_pred <- predict(dtree_model_prun,
cancer_test,
type = "response")confusionMatrix(dtree_model_prun_pred,
cancer_test$diagnosis,
positive = "M")## Confusion Matrix and Statistics
##
## Reference
## Prediction B M
## B 65 1
## M 4 44
##
## Accuracy : 0.9561
## 95% CI : (0.9006, 0.9856)
## No Information Rate : 0.6053
## P-Value [Acc > NIR] : <2e-16
##
## Kappa : 0.9093
##
## Mcnemar's Test P-Value : 0.3711
##
## Sensitivity : 0.9778
## Specificity : 0.9420
## Pos Pred Value : 0.9167
## Neg Pred Value : 0.9848
## Prevalence : 0.3947
## Detection Rate : 0.3860
## Detection Prevalence : 0.4211
## Balanced Accuracy : 0.9599
##
## 'Positive' Class : M
## Based on our newly tuned model evaluation, we can conclude:
- Our new Decision Tree model could predict correctly with 95.61% accuracy.
- Our new Decision Tree model could predict recall with 97.78%.
- Our new Decision Tree model could predict Precission with 91.67%.
- Our new Decision Tree model could predict Specificity with 94.2%.
3.3 Random Forest
Next we will try to create a new model using Random Forest method.
We will use ourtest data from Splitting Train-Test for our data.
To make our model simplified, we will use 3 k-fold with 3 repetition
set.seed(189)
control <- trainControl(method = "repeatedcv",
number = 3,
repeats = 3)
cancer_forest <- train(diagnosis ~ .,
data = cancer_test,
method = "rf",
trControl = control)
cancer_forest## Random Forest
##
## 114 samples
## 10 predictor
## 2 classes: 'B', 'M'
##
## No pre-processing
## Resampling: Cross-Validated (3 fold, repeated 3 times)
## Summary of sample sizes: 76, 76, 76, 76, 76, 76, ...
## Resampling results across tuning parameters:
##
## mtry Accuracy Kappa
## 2 0.9473684 0.8892145
## 6 0.9415205 0.8779883
## 10 0.9415205 0.8788648
##
## Accuracy was used to select the optimal model using the largest value.
## The final value used for the model was mtry = 2.From our Random Forest model, we knew that:
- There are 114 samples
- There are 10 predictors
- There are 2 classes in the data. Benign and Malignant
3.3.1 Out-of-Bag Error
OOB error test is used to calculate error percentage that could happen in the model.
cancer_forest$finalModel##
## Call:
## randomForest(x = x, y = y, mtry = param$mtry)
## Type of random forest: classification
## Number of trees: 500
## No. of variables tried at each split: 2
##
## OOB estimate of error rate: 4.39%
## Confusion matrix:
## B M class.error
## B 67 2 0.02898551
## M 3 42 0.06666667From result above, we knew that OOB error rate is 4.39% which means our model accuracy is 95.61%.
4 Conclusion
Our best model to be used is our first Decision Tree
model with 97.37% accuracy. But in our case the best possible scenario
is we doesn’t want our patient to be diagnosed with Benign but actually
our patient got Malignant type. So the best matrix that compatible with
our case are Recall matrix. Therefore the best model that
we could use are our Tuned Decision Tree model because our
tuned model could predict with 97.78% accuracy