Learner, Task, Cross Validation in mlr3

Why the need for an upgrade?

mlr3 was created to better support parallelization and larger datasets (out-of-memory data-backends). This upgrade works with R6 objects which allow for simpler syntax. Tasks, Learners, and evaluation measures are all stored in dictionaries.

Installation of mlr3 It is recommended to install the package “mlr3verse” instead of simply “mlr3” so that the most important packages are installed. Mlr3 differs from mlr in that learners and other extensions need to be installed separately according to need. Mlr3verse avoids having to do some of these additional installations.

install.packages("mlr3verse", dependencies = TRUE)

Loading the dataset and formatting it as a tibble

Loading the data is the same as with mlr, using the mclust package.

library(mlr3verse)
library(tidyverse)

data(diabetes, package = "mclust")
diabetesTib <- as_tibble(diabetes)
summary(diabetesTib)

      class       glucose       insulin            sspg      
 Chemical:36   Min.   : 70   Min.   :  45.0   Min.   : 10.0  
 Normal  :76   1st Qu.: 90   1st Qu.: 352.0   1st Qu.:118.0  
 Overt   :33   Median : 97   Median : 403.0   Median :156.0  
               Mean   :122   Mean   : 540.8   Mean   :186.1  
               3rd Qu.:112   3rd Qu.: 558.0   3rd Qu.:221.0  
               Max.   :353   Max.   :1568.0   Max.   :748.0

Taking a look at the new tibble

diabetesTib

Using mlr3 to train a model

How does building a machine learning model with the mlr3 package differ from mlr?

Just as a reminder training a model with mlr has three main stages:

1. Define the task. The task consists of the data and what we want to do with it.

2. Define the learner. The learner is the algorithm we plan to use.

3. Train the model: you pass the task to the learner, this generates a model that you can use to make future predictions.

These 3 main stages are still necessary in mlr3, however the way in which these stages are executed differs slightly.

Let’s take a look and see just exactly how they differ.

1. Define the task. Let’s begin by creating our own task in mlr3 with as_classif().

diabetesTask<- as_task_classif(diabetesTib, target = "class")
diabetesTask

<TaskClassif:diabetesTib> (145 x 4)
* Target: class
* Properties: multiclass
* Features (3):
  - dbl (3): glucose, insulin, sspg

How does this step look in mlr:

diabetesTask <- makeClassifTask(data = diabetesTib, target = “class”)

We no do not need to specify data = diabetesTib, we can simply input diabetesTib and it is passed down.

Defining the learner. In mlr3 we define the learner using lrn(). To check which learners are available run: as.data.table(mlr_learners)

knn <- lrn("classif.kknn", k = 2)
#if this does not work add this before running lrn(): install.packages("kknn") 
#default k = 7

How does this step look in mlr:
knn <- makeLearner(“classif.knn”, par.vals = list(k = 2)).

MakeLearner no longer used, it is replaced with lrn()
par.vals = list() -> sets hyperparameter in mlr. No longer used in mlr3, hyperparameter can be set directly

Training the model. In mlr3 certain methods such as train(), model(), and predict() are called using the $ operator. Pass the learner to the model via $train():

knn_model <- knn$train(diabetesTask)
knn_model

<LearnerClassifKKNN:classif.kknn>: k-Nearest-Neighbor
* Model: list
* Parameters: k=2
* Packages: mlr3, mlr3learners, kknn
* Predict Types:  [response], prob
* Feature Types: logical, integer, numeric, factor, ordered
* Properties: multiclass, twoclass

How does this look in mlr:
knnModel <- train(knn, diabetesTask)

After training, the fitted model is stored in the $model field for a more detailed inspection

knn_model$model

$formula
class ~ .
NULL

$data

$pv
$pv$k
[1] 2


$kknn
NULL

Prediction Predict from the trained model using $predict():

knn_predict <- knn_model$predict(diabetesTask)
knn_predict

<PredictionClassif> for 145 observations:

How does this look in mlr:
knnPred <- predict(knnModel, newdata = diabetesTib)

model is inside parenthesis
data needed to be set as a parameter

Performance Measures Stored in mlr_measures dictionary, accessed with msr() for a single measurement and msrs() for multiple measurements.

knn_measure <- msrs(c("classif.acc", "classif.ce")) #must be set up as a vector c() in order to treat it as a single argument. 
knn_predict$score(knn_measure)

classif.acc  classif.ce 
          1           0

How does this look in mlr:
performance(knnPred, measures = list(mmce, acc))

Cross Validation in mlr3

K-fold cross-validation We will use repeated k-fold cross-validation, since we already know this is the best option if the dataset is not too large.

Resampling Description: We first need to make our resampling description which is simply the set of instructions for how the data will be split into test and training sets.

knn_cv <- rsmp("repeated_cv", repeats = 50, folds = 10) #stratify automatically done in mlr3

Using the resample() function as in mlr, we can run the cross-validation on our learner.

kfoldCv <- resample(diabetesTask, knn, knn_cv)

INFO  [21:58:08.922] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 1/500)
INFO  [21:58:10.022] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 2/500)
INFO  [21:58:10.076] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 3/500)
INFO  [21:58:10.114] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 4/500)
INFO  [21:58:10.158] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 5/500)
INFO  [21:58:10.204] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 6/500)
INFO  [21:58:10.259] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 7/500)
INFO  [21:58:10.301] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 8/500)
INFO  [21:58:10.344] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 9/500)
INFO  [21:58:10.380] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 10/500)
INFO  [21:58:10.420] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 11/500)
INFO  [21:58:10.460] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 12/500)
INFO  [21:58:10.498] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 13/500)
INFO  [21:58:10.523] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 14/500)
INFO  [21:58:10.548] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 15/500)
INFO  [21:58:10.571] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 16/500)
INFO  [21:58:10.595] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 17/500)
INFO  [21:58:10.618] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 18/500)
INFO  [21:58:10.643] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 19/500)
INFO  [21:58:10.671] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 20/500)
INFO  [21:58:10.700] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 21/500)
INFO  [21:58:10.731] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 22/500)
INFO  [21:58:10.793] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 23/500)
INFO  [21:58:10.818] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 24/500)
INFO  [21:58:10.843] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 25/500)
INFO  [21:58:10.867] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 26/500)
INFO  [21:58:10.891] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 27/500)
INFO  [21:58:10.915] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 28/500)
INFO  [21:58:10.939] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 29/500)
INFO  [21:58:10.963] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 30/500)
INFO  [21:58:10.988] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 31/500)
INFO  [21:58:11.017] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 32/500)
INFO  [21:58:11.038] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 33/500)
INFO  [21:58:11.059] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 34/500)
INFO  [21:58:11.080] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 35/500)
INFO  [21:58:11.102] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 36/500)
INFO  [21:58:11.123] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 37/500)
INFO  [21:58:11.144] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 38/500)
INFO  [21:58:11.165] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 39/500)
INFO  [21:58:11.185] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 40/500)
INFO  [21:58:11.212] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 41/500)
INFO  [21:58:11.233] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 42/500)
INFO  [21:58:11.254] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 43/500)
INFO  [21:58:11.274] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 44/500)
INFO  [21:58:11.294] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 45/500)
INFO  [21:58:11.315] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 46/500)
INFO  [21:58:11.335] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 47/500)
INFO  [21:58:11.356] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 48/500)
INFO  [21:58:11.378] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 49/500)
INFO  [21:58:11.399] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 50/500)
INFO  [21:58:11.420] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 51/500)
INFO  [21:58:11.446] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 52/500)
INFO  [21:58:11.478] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 53/500)
INFO  [21:58:11.498] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 54/500)
INFO  [21:58:11.517] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 55/500)
INFO  [21:58:11.537] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 56/500)
INFO  [21:58:11.558] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 57/500)
INFO  [21:58:11.580] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 58/500)
INFO  [21:58:11.601] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 59/500)
INFO  [21:58:11.622] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 60/500)
INFO  [21:58:11.642] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 61/500)
INFO  [21:58:11.663] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 62/500)
INFO  [21:58:11.688] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 63/500)
INFO  [21:58:11.709] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 64/500)
INFO  [21:58:11.730] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 65/500)
INFO  [21:58:11.751] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 66/500)
INFO  [21:58:11.773] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 67/500)
INFO  [21:58:11.794] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 68/500)
INFO  [21:58:11.817] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 69/500)
INFO  [21:58:11.839] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 70/500)
INFO  [21:58:11.859] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 71/500)
INFO  [21:58:11.881] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 72/500)
INFO  [21:58:11.900] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 73/500)
INFO  [21:58:11.922] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 74/500)
INFO  [21:58:11.945] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 75/500)
INFO  [21:58:11.975] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 76/500)
INFO  [21:58:11.995] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 77/500)
INFO  [21:58:12.014] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 78/500)
INFO  [21:58:12.037] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 79/500)
INFO  [21:58:12.058] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 80/500)
INFO  [21:58:12.079] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 81/500)
INFO  [21:58:12.102] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 82/500)
INFO  [21:58:12.123] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 83/500)
INFO  [21:58:12.144] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 84/500)
INFO  [21:58:12.167] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 85/500)
INFO  [21:58:12.189] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 86/500)
INFO  [21:58:12.211] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 87/500)
INFO  [21:58:12.232] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 88/500)
INFO  [21:58:12.253] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 89/500)
INFO  [21:58:12.279] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 90/500)
INFO  [21:58:12.302] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 91/500)
INFO  [21:58:12.322] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 92/500)
INFO  [21:58:12.343] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 93/500)
INFO  [21:58:12.364] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 94/500)
INFO  [21:58:12.385] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 95/500)
INFO  [21:58:12.405] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 96/500)
INFO  [21:58:12.425] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 97/500)
INFO  [21:58:12.455] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 98/500)
INFO  [21:58:12.475] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 99/500)
INFO  [21:58:12.495] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 100/500)
INFO  [21:58:12.515] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 101/500)
INFO  [21:58:12.536] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 102/500)
INFO  [21:58:12.557] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 103/500)
INFO  [21:58:12.577] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 104/500)
INFO  [21:58:12.606] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 105/500)
INFO  [21:58:12.626] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 106/500)
INFO  [21:58:12.645] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 107/500)
INFO  [21:58:12.665] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 108/500)
INFO  [21:58:12.685] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 109/500)
INFO  [21:58:12.706] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 110/500)
INFO  [21:58:12.727] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 111/500)
INFO  [21:58:12.748] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 112/500)
INFO  [21:58:12.767] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 113/500)
INFO  [21:58:12.788] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 114/500)
INFO  [21:58:12.809] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 115/500)
INFO  [21:58:12.831] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 116/500)
INFO  [21:58:12.851] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 117/500)
INFO  [21:58:12.873] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 118/500)
INFO  [21:58:12.893] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 119/500)
INFO  [21:58:12.933] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 120/500)
INFO  [21:58:12.953] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 121/500)
INFO  [21:58:12.974] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 122/500)
INFO  [21:58:12.999] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 123/500)
INFO  [21:58:13.019] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 124/500)
INFO  [21:58:13.039] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 125/500)
INFO  [21:58:13.060] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 126/500)
INFO  [21:58:13.082] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 127/500)
INFO  [21:58:13.101] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 128/500)
INFO  [21:58:13.123] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 129/500)
INFO  [21:58:13.144] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 130/500)
INFO  [21:58:13.165] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 131/500)
INFO  [21:58:13.186] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 132/500)
INFO  [21:58:13.207] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 133/500)
INFO  [21:58:13.231] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 134/500)
INFO  [21:58:13.251] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 135/500)
INFO  [21:58:13.272] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 136/500)
INFO  [21:58:13.293] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 137/500)
INFO  [21:58:13.314] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 138/500)
INFO  [21:58:13.335] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 139/500)
INFO  [21:58:13.356] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 140/500)
INFO  [21:58:13.386] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 141/500)
INFO  [21:58:13.404] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 142/500)
INFO  [21:58:13.424] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 143/500)
INFO  [21:58:13.445] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 144/500)
INFO  [21:58:13.466] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 145/500)
INFO  [21:58:13.485] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 146/500)
INFO  [21:58:13.505] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 147/500)
INFO  [21:58:13.525] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 148/500)
INFO  [21:58:13.546] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 149/500)
INFO  [21:58:13.568] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 150/500)
INFO  [21:58:13.593] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 151/500)
INFO  [21:58:13.625] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 152/500)
INFO  [21:58:13.646] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 153/500)
INFO  [21:58:13.666] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 154/500)
INFO  [21:58:13.687] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 155/500)
INFO  [21:58:13.707] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 156/500)
INFO  [21:58:13.727] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 157/500)
INFO  [21:58:13.747] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 158/500)
INFO  [21:58:13.768] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 159/500)
INFO  [21:58:13.789] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 160/500)
INFO  [21:58:13.810] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 161/500)
INFO  [21:58:13.829] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 162/500)
INFO  [21:58:13.857] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 163/500)
INFO  [21:58:13.877] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 164/500)
INFO  [21:58:13.896] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 165/500)
INFO  [21:58:13.917] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 166/500)
INFO  [21:58:13.935] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 167/500)
INFO  [21:58:13.955] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 168/500)
INFO  [21:58:13.975] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 169/500)
INFO  [21:58:13.995] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 170/500)
INFO  [21:58:14.015] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 171/500)
INFO  [21:58:14.034] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 172/500)
INFO  [21:58:14.055] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 173/500)
INFO  [21:58:14.075] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 174/500)
INFO  [21:58:14.094] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 175/500)
INFO  [21:58:14.116] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 176/500)
INFO  [21:58:14.139] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 177/500)
INFO  [21:58:14.160] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 178/500)
INFO  [21:58:14.179] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 179/500)
INFO  [21:58:14.200] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 180/500)
INFO  [21:58:14.222] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 181/500)
INFO  [21:58:14.242] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 182/500)
INFO  [21:58:14.270] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 183/500)
INFO  [21:58:14.289] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 184/500)
INFO  [21:58:14.310] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 185/500)
INFO  [21:58:14.328] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 186/500)
INFO  [21:58:14.348] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 187/500)
INFO  [21:58:14.368] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 188/500)
INFO  [21:58:14.388] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 189/500)
INFO  [21:58:14.407] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 190/500)
INFO  [21:58:14.427] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 191/500)
INFO  [21:58:14.448] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 192/500)
INFO  [21:58:14.467] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 193/500)
INFO  [21:58:14.487] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 194/500)
INFO  [21:58:14.507] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 195/500)
INFO  [21:58:14.526] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 196/500)
INFO  [21:58:14.547] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 197/500)
INFO  [21:58:14.567] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 198/500)
INFO  [21:58:14.587] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 199/500)
INFO  [21:58:14.612] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 200/500)
INFO  [21:58:14.633] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 201/500)
INFO  [21:58:14.654] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 202/500)
INFO  [21:58:14.683] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 203/500)
INFO  [21:58:14.702] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 204/500)
INFO  [21:58:14.722] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 205/500)
INFO  [21:58:14.741] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 206/500)
INFO  [21:58:14.761] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 207/500)
INFO  [21:58:14.781] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 208/500)
INFO  [21:58:14.800] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 209/500)
INFO  [21:58:14.821] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 210/500)
INFO  [21:58:14.842] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 211/500)
INFO  [21:58:14.863] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 212/500)
INFO  [21:58:14.882] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 213/500)
INFO  [21:58:14.902] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 214/500)
INFO  [21:58:14.922] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 215/500)
INFO  [21:58:14.944] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 216/500)
INFO  [21:58:14.965] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 217/500)
INFO  [21:58:14.985] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 218/500)
INFO  [21:58:15.004] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 219/500)
INFO  [21:58:15.027] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 220/500)
INFO  [21:58:15.049] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 221/500)
INFO  [21:58:15.069] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 222/500)
INFO  [21:58:15.098] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 223/500)
INFO  [21:58:15.118] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 224/500)
INFO  [21:58:15.138] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 225/500)
INFO  [21:58:15.157] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 226/500)
INFO  [21:58:15.177] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 227/500)
INFO  [21:58:15.196] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 228/500)
INFO  [21:58:15.218] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 229/500)
INFO  [21:58:15.238] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 230/500)
INFO  [21:58:15.258] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 231/500)
INFO  [21:58:15.278] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 232/500)
INFO  [21:58:15.298] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 233/500)
INFO  [21:58:15.317] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 234/500)
INFO  [21:58:15.338] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 235/500)
INFO  [21:58:15.359] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 236/500)
INFO  [21:58:15.379] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 237/500)
INFO  [21:58:15.398] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 238/500)
INFO  [21:58:15.419] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 239/500)
INFO  [21:58:15.440] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 240/500)
INFO  [21:58:15.463] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 241/500)
INFO  [21:58:15.486] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 242/500)
INFO  [21:58:15.514] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 243/500)
INFO  [21:58:15.534] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 244/500)
INFO  [21:58:15.552] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 245/500)
INFO  [21:58:15.572] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 246/500)
INFO  [21:58:15.591] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 247/500)
INFO  [21:58:15.612] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 248/500)
INFO  [21:58:15.632] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 249/500)
INFO  [21:58:15.651] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 250/500)
INFO  [21:58:15.672] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 251/500)
INFO  [21:58:15.692] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 252/500)
INFO  [21:58:15.713] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 253/500)
INFO  [21:58:15.732] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 254/500)
INFO  [21:58:15.752] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 255/500)
INFO  [21:58:15.773] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 256/500)
INFO  [21:58:15.793] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 257/500)
INFO  [21:58:15.814] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 258/500)
INFO  [21:58:15.834] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 259/500)
INFO  [21:58:15.854] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 260/500)
INFO  [21:58:15.879] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 261/500)
INFO  [21:58:15.910] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 262/500)
INFO  [21:58:15.930] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 263/500)
INFO  [21:58:15.949] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 264/500)
INFO  [21:58:15.969] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 265/500)
INFO  [21:58:15.989] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 266/500)
INFO  [21:58:16.009] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 267/500)
INFO  [21:58:16.029] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 268/500)
INFO  [21:58:16.049] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 269/500)
INFO  [21:58:16.069] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 270/500)
INFO  [21:58:16.088] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 271/500)
INFO  [21:58:16.109] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 272/500)
INFO  [21:58:16.129] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 273/500)
INFO  [21:58:16.149] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 274/500)
INFO  [21:58:16.168] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 275/500)
INFO  [21:58:16.189] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 276/500)
INFO  [21:58:16.210] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 277/500)
INFO  [21:58:16.231] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 278/500)
INFO  [21:58:16.254] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 279/500)
INFO  [21:58:16.278] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 280/500)
INFO  [21:58:16.322] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 281/500)
INFO  [21:58:16.341] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 282/500)
INFO  [21:58:16.361] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 283/500)
INFO  [21:58:16.382] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 284/500)
INFO  [21:58:16.400] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 285/500)
INFO  [21:58:16.420] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 286/500)
INFO  [21:58:16.440] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 287/500)
INFO  [21:58:16.460] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 288/500)
INFO  [21:58:16.482] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 289/500)
INFO  [21:58:16.502] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 290/500)
INFO  [21:58:16.522] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 291/500)
INFO  [21:58:16.542] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 292/500)
INFO  [21:58:16.562] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 293/500)
INFO  [21:58:16.582] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 294/500)
INFO  [21:58:16.601] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 295/500)
INFO  [21:58:16.622] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 296/500)
INFO  [21:58:16.646] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 297/500)
INFO  [21:58:16.669] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 298/500)
INFO  [21:58:16.688] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 299/500)
INFO  [21:58:16.709] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 300/500)
INFO  [21:58:16.740] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 301/500)
INFO  [21:58:16.760] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 302/500)
INFO  [21:58:16.779] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 303/500)
INFO  [21:58:16.799] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 304/500)
INFO  [21:58:16.819] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 305/500)
INFO  [21:58:16.839] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 306/500)
INFO  [21:58:16.859] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 307/500)
INFO  [21:58:16.880] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 308/500)
INFO  [21:58:16.899] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 309/500)
INFO  [21:58:16.920] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 310/500)
INFO  [21:58:16.940] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 311/500)
INFO  [21:58:16.960] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 312/500)
INFO  [21:58:16.981] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 313/500)
INFO  [21:58:17.000] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 314/500)
INFO  [21:58:17.024] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 315/500)
INFO  [21:58:17.046] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 316/500)
INFO  [21:58:17.066] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 317/500)
INFO  [21:58:17.087] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 318/500)
INFO  [21:58:17.107] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 319/500)
INFO  [21:58:17.138] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 320/500)
INFO  [21:58:17.157] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 321/500)
INFO  [21:58:17.176] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 322/500)
INFO  [21:58:17.197] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 323/500)
INFO  [21:58:17.216] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 324/500)
INFO  [21:58:17.236] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 325/500)
INFO  [21:58:17.258] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 326/500)
INFO  [21:58:17.278] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 327/500)
INFO  [21:58:17.298] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 328/500)
INFO  [21:58:17.318] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 329/500)
INFO  [21:58:17.338] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 330/500)
INFO  [21:58:17.361] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 331/500)
INFO  [21:58:17.384] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 332/500)
INFO  [21:58:17.405] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 333/500)
INFO  [21:58:17.425] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 334/500)
INFO  [21:58:17.446] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 335/500)
INFO  [21:58:17.465] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 336/500)
INFO  [21:58:17.485] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 337/500)
INFO  [21:58:17.516] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 338/500)
INFO  [21:58:17.537] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 339/500)
INFO  [21:58:17.557] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 340/500)
INFO  [21:58:17.578] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 341/500)
INFO  [21:58:17.598] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 342/500)
INFO  [21:58:17.618] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 343/500)
INFO  [21:58:17.637] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 344/500)
INFO  [21:58:17.658] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 345/500)
INFO  [21:58:17.678] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 346/500)
INFO  [21:58:17.701] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 347/500)
INFO  [21:58:17.723] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 348/500)
INFO  [21:58:17.744] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 349/500)
INFO  [21:58:17.764] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 350/500)
INFO  [21:58:17.785] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 351/500)
INFO  [21:58:17.807] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 352/500)
INFO  [21:58:17.827] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 353/500)
INFO  [21:58:17.848] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 354/500)
INFO  [21:58:17.868] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 355/500)
INFO  [21:58:17.898] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 356/500)
INFO  [21:58:17.918] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 357/500)
INFO  [21:58:17.939] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 358/500)
INFO  [21:58:17.959] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 359/500)
INFO  [21:58:17.979] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 360/500)
INFO  [21:58:17.998] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 361/500)
INFO  [21:58:18.022] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 362/500)
INFO  [21:58:18.042] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 363/500)
INFO  [21:58:18.064] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 364/500)
INFO  [21:58:18.085] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 365/500)
INFO  [21:58:18.104] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 366/500)
INFO  [21:58:18.125] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 367/500)
INFO  [21:58:18.145] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 368/500)
INFO  [21:58:18.164] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 369/500)
INFO  [21:58:18.185] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 370/500)
INFO  [21:58:18.205] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 371/500)
INFO  [21:58:18.225] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 372/500)
INFO  [21:58:18.245] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 373/500)
INFO  [21:58:18.561] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 374/500)
INFO  [21:58:18.579] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 375/500)
INFO  [21:58:18.600] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 376/500)
INFO  [21:58:18.622] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 377/500)
INFO  [21:58:18.644] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 378/500)
INFO  [21:58:18.664] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 379/500)
INFO  [21:58:18.683] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 380/500)
INFO  [21:58:18.703] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 381/500)
INFO  [21:58:18.724] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 382/500)
INFO  [21:58:18.744] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 383/500)
INFO  [21:58:18.764] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 384/500)
INFO  [21:58:18.784] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 385/500)
INFO  [21:58:18.804] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 386/500)
INFO  [21:58:18.825] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 387/500)
INFO  [21:58:18.845] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 388/500)
INFO  [21:58:18.865] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 389/500)
INFO  [21:58:18.891] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 390/500)
INFO  [21:58:18.912] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 391/500)
INFO  [21:58:18.932] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 392/500)
INFO  [21:58:18.953] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 393/500)
INFO  [21:58:18.974] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 394/500)
INFO  [21:58:18.995] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 395/500)
INFO  [21:58:19.015] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 396/500)
INFO  [21:58:19.038] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 397/500)
INFO  [21:58:19.059] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 398/500)
INFO  [21:58:19.079] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 399/500)
INFO  [21:58:19.099] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 400/500)
INFO  [21:58:19.120] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 401/500)
INFO  [21:58:19.145] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 402/500)
INFO  [21:58:19.169] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 403/500)
INFO  [21:58:19.191] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 404/500)
INFO  [21:58:19.214] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 405/500)
INFO  [21:58:19.238] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 406/500)
INFO  [21:58:19.262] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 407/500)
INFO  [21:58:19.286] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 408/500)
INFO  [21:58:19.317] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 409/500)
INFO  [21:58:19.337] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 410/500)
INFO  [21:58:19.358] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 411/500)
INFO  [21:58:19.379] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 412/500)
INFO  [21:58:19.397] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 413/500)
INFO  [21:58:19.417] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 414/500)
INFO  [21:58:19.437] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 415/500)
INFO  [21:58:19.463] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 416/500)
INFO  [21:58:19.485] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 417/500)
INFO  [21:58:19.506] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 418/500)
INFO  [21:58:19.526] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 419/500)
INFO  [21:58:19.546] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 420/500)
INFO  [21:58:19.567] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 421/500)
INFO  [21:58:19.587] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 422/500)
INFO  [21:58:19.606] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 423/500)
INFO  [21:58:19.626] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 424/500)
INFO  [21:58:19.646] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 425/500)
INFO  [21:58:19.666] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 426/500)
INFO  [21:58:19.687] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 427/500)
INFO  [21:58:19.707] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 428/500)
INFO  [21:58:19.732] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 429/500)
INFO  [21:58:19.754] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 430/500)
INFO  [21:58:19.773] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 431/500)
INFO  [21:58:19.793] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 432/500)
INFO  [21:58:19.814] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 433/500)
INFO  [21:58:19.835] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 434/500)
INFO  [21:58:19.855] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 435/500)
INFO  [21:58:19.876] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 436/500)
INFO  [21:58:19.896] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 437/500)
INFO  [21:58:19.917] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 438/500)
INFO  [21:58:19.938] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 439/500)
INFO  [21:58:19.968] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 440/500)
INFO  [21:58:19.988] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 441/500)
INFO  [21:58:20.014] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 442/500)
INFO  [21:58:20.034] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 443/500)
INFO  [21:58:20.053] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 444/500)
INFO  [21:58:20.073] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 445/500)
INFO  [21:58:20.093] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 446/500)
INFO  [21:58:20.113] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 447/500)
INFO  [21:58:20.133] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 448/500)
INFO  [21:58:20.153] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 449/500)
INFO  [21:58:20.173] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 450/500)
INFO  [21:58:20.193] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 451/500)
INFO  [21:58:20.213] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 452/500)
INFO  [21:58:20.235] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 453/500)
INFO  [21:58:20.257] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 454/500)
INFO  [21:58:20.279] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 455/500)
INFO  [21:58:20.299] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 456/500)
INFO  [21:58:20.320] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 457/500)
INFO  [21:58:20.341] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 458/500)
INFO  [21:58:20.362] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 459/500)
INFO  [21:58:20.382] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 460/500)
INFO  [21:58:20.402] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 461/500)
INFO  [21:58:20.422] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 462/500)
INFO  [21:58:20.442] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 463/500)
INFO  [21:58:20.462] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 464/500)
INFO  [21:58:20.487] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 465/500)
INFO  [21:58:20.511] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 466/500)
INFO  [21:58:20.532] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 467/500)
INFO  [21:58:20.553] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 468/500)
INFO  [21:58:20.573] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 469/500)
INFO  [21:58:20.593] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 470/500)
INFO  [21:58:20.622] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 471/500)
INFO  [21:58:20.642] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 472/500)
INFO  [21:58:20.662] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 473/500)
INFO  [21:58:20.681] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 474/500)
INFO  [21:58:20.702] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 475/500)
INFO  [21:58:20.722] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 476/500)
INFO  [21:58:20.741] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 477/500)
INFO  [21:58:20.765] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 478/500)
INFO  [21:58:20.786] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 479/500)
INFO  [21:58:20.807] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 480/500)
INFO  [21:58:20.826] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 481/500)
INFO  [21:58:20.846] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 482/500)
INFO  [21:58:20.868] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 483/500)
INFO  [21:58:20.888] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 484/500)
INFO  [21:58:20.908] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 485/500)
INFO  [21:58:20.926] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 486/500)
INFO  [21:58:20.946] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 487/500)
INFO  [21:58:20.966] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 488/500)
INFO  [21:58:20.986] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 489/500)
INFO  [21:58:21.009] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 490/500)
INFO  [21:58:21.030] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 491/500)
INFO  [21:58:21.052] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 492/500)
INFO  [21:58:21.071] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 493/500)
INFO  [21:58:21.091] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 494/500)
INFO  [21:58:21.110] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 495/500)
INFO  [21:58:21.131] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 496/500)
INFO  [21:58:21.150] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 497/500)
INFO  [21:58:21.170] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 498/500)
INFO  [21:58:21.189] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 499/500)
INFO  [21:58:21.210] [mlr3] Applying learner 'classif.kknn' on task 'diabetesTib' (iter 500/500)

How does this look in mlr:

kFold <- makeResampleDesc(method = “RepCV”, folds = 10, reps = 50, stratify = TRUE)
kFoldCV <- resample(learner = knn, task = diabetesTask, resampling = kFold, measures = list(mmce, acc))
Here everything has to be specifically defined, in mlr3 that is no longer necessary
But in mlr3 a specific order is necessary in order for the command to work: task, learner, resampling.

Finally, Aggregate() provides us with the summary performance measures after the k-fold.

kfoldCv$aggregate(knn_measure)

classif.acc  classif.ce 
 0.91178095  0.08821905

Calculating the confusion Matrix the confusion matrix is calculated using $confusion on the cross-validation object and calling $prediction

conf_matrix <- kfoldCv$prediction()$confusion
conf_matrix

          truth
response   Chemical Normal Overt
  Chemical     1505    164   154
  Normal        198   3636     0
  Overt          97      0  1496

Equivalent to calculateConfusionMatrix(kFoldCV$pred) in mlr.

Mlr3 on Abalone Dataset

library(tidyverse)
library(magrittr)

G3;
Attaching package: ‘magrittr’

gG3;The following object is masked from ‘package:purrr’:

    set_names

gG3;The following object is masked from ‘package:tidyr’:

    extract

g

abalone <- read.csv("C:/Users/joely/OneDrive/Desktop/Master/AIV/abalone.csv")

Necessary to create a new column “Age” and convert Age into a factor (categorical value in r ) before creating classification Task

abalone %<>%
    dplyr::mutate(abalone, Age = factor(ifelse(Rings > 11, "old", "young")))
table(abalone$Age)


  old young 
  960  3217

abalone <- as_tibble(abalone)
abalone

Since the dataset is unbalanced as we saw in our original script (distribution of samples across its classes is significantly unequal) we need to ignore certain features. We can transform the data to do this with the select() method in the dplyr package.

abaloneTib_2 <- dplyr::select(abalone, -c(Sex, Rings)) #excludes Sex, Rings column
abaloneTib_2

NA

Task:

abalone_task <- as_task_classif(abaloneTib_2, target = "Age")

Learner:

knn_aba <- lrn("classif.kknn", k = 2)

Train Model:

knn_aba_model <- knn_aba$train(abalone_task)
knn_aba_model

<LearnerClassifKKNN:classif.kknn>: k-Nearest-Neighbor
* Model: list
* Parameters: k=2
* Packages: mlr3, mlr3learners, kknn
* Predict Types:  [response], prob
* Feature Types: logical, integer, numeric, factor, ordered
* Properties: multiclass, twoclass

Prediction

knn_aba_predict <- knn_aba_model$predict(abalone_task)
knn_aba_predict

<PredictionClassif> for 4177 observations:

Performance Measures

knn_aba_measure <- msrs(c("classif.acc", "classif.ce", "classif.tpr", "classif.tnr", "classif.fbeta")) #default f1 score 
knn_aba_predict$score(knn_aba_measure)

  classif.acc    classif.ce 
            1             0 
  classif.tpr   classif.tnr 
            1             1 
classif.fbeta 
            1

Resample Description

knn_cv_aba <- rsmp("repeated_cv", repeats = 20, folds = 10)

Using the resample() function, we can run the cross-validation on our learner.

kfoldCv_aba <- resample(abalone_task, knn_aba, knn_cv_aba)

INFO  [22:04:22.343] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 1/200)
INFO  [22:04:22.411] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 2/200)
INFO  [22:04:22.479] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 3/200)
INFO  [22:04:22.559] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 4/200)
INFO  [22:04:22.651] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 5/200)
INFO  [22:04:22.761] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 6/200)
INFO  [22:04:22.835] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 7/200)
INFO  [22:04:22.927] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 8/200)
INFO  [22:04:23.001] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 9/200)
INFO  [22:04:23.077] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 10/200)
INFO  [22:04:23.156] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 11/200)
INFO  [22:04:23.226] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 12/200)
INFO  [22:04:23.302] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 13/200)
INFO  [22:04:23.456] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 14/200)
INFO  [22:04:23.528] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 15/200)
INFO  [22:04:23.602] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 16/200)
INFO  [22:04:23.687] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 17/200)
INFO  [22:04:23.757] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 18/200)
INFO  [22:04:23.830] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 19/200)
INFO  [22:04:23.902] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 20/200)
INFO  [22:04:24.001] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 21/200)
INFO  [22:04:24.075] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 22/200)
INFO  [22:04:24.145] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 23/200)
INFO  [22:04:24.244] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 24/200)
INFO  [22:04:24.314] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 25/200)
INFO  [22:04:24.389] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 26/200)
INFO  [22:04:24.484] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 27/200)
INFO  [22:04:24.557] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 28/200)
INFO  [22:04:24.611] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 29/200)
INFO  [22:04:24.685] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 30/200)
INFO  [22:04:24.729] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 31/200)
INFO  [22:04:24.771] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 32/200)
INFO  [22:04:24.824] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 33/200)
INFO  [22:04:24.865] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 34/200)
INFO  [22:04:24.907] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 35/200)
INFO  [22:04:24.949] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 36/200)
INFO  [22:04:25.004] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 37/200)
INFO  [22:04:25.046] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 38/200)
INFO  [22:04:25.089] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 39/200)
INFO  [22:04:25.143] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 40/200)
INFO  [22:04:25.186] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 41/200)
INFO  [22:04:25.228] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 42/200)
INFO  [22:04:25.279] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 43/200)
INFO  [22:04:25.322] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 44/200)
INFO  [22:04:25.363] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 45/200)
INFO  [22:04:25.415] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 46/200)
INFO  [22:04:25.460] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 47/200)
INFO  [22:04:25.503] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 48/200)
INFO  [22:04:25.546] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 49/200)
INFO  [22:04:25.599] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 50/200)
INFO  [22:04:25.642] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 51/200)
INFO  [22:04:25.683] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 52/200)
INFO  [22:04:25.737] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 53/200)
INFO  [22:04:25.779] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 54/200)
INFO  [22:04:25.819] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 55/200)
INFO  [22:04:25.874] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 56/200)
INFO  [22:04:25.915] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 57/200)
INFO  [22:04:25.955] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 58/200)
INFO  [22:04:26.010] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 59/200)
INFO  [22:04:26.054] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 60/200)
INFO  [22:04:26.096] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 61/200)
INFO  [22:04:26.139] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 62/200)
INFO  [22:04:26.194] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 63/200)
INFO  [22:04:26.236] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 64/200)
INFO  [22:04:26.276] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 65/200)
INFO  [22:04:26.329] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 66/200)
INFO  [22:04:26.371] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 67/200)
INFO  [22:04:26.412] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 68/200)
INFO  [22:04:26.466] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 69/200)
INFO  [22:04:26.508] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 70/200)
INFO  [22:04:26.551] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 71/200)
INFO  [22:04:26.886] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 72/200)
INFO  [22:04:26.929] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 73/200)
INFO  [22:04:26.971] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 74/200)
INFO  [22:04:27.011] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 75/200)
INFO  [22:04:27.052] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 76/200)
INFO  [22:04:27.093] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 77/200)
INFO  [22:04:27.136] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 78/200)
INFO  [22:04:27.186] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 79/200)
INFO  [22:04:27.227] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 80/200)
INFO  [22:04:27.268] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 81/200)
INFO  [22:04:27.310] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 82/200)
INFO  [22:04:27.352] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 83/200)
INFO  [22:04:27.394] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 84/200)
INFO  [22:04:27.436] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 85/200)
INFO  [22:04:27.487] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 86/200)
INFO  [22:04:27.527] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 87/200)
INFO  [22:04:27.568] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 88/200)
INFO  [22:04:27.611] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 89/200)
INFO  [22:04:27.652] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 90/200)
INFO  [22:04:27.694] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 91/200)
INFO  [22:04:27.735] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 92/200)
INFO  [22:04:27.778] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 93/200)
INFO  [22:04:27.828] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 94/200)
INFO  [22:04:27.871] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 95/200)
INFO  [22:04:27.911] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 96/200)
INFO  [22:04:27.952] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 97/200)
INFO  [22:04:27.993] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 98/200)
INFO  [22:04:28.035] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 99/200)
INFO  [22:04:28.082] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 100/200)
INFO  [22:04:28.133] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 101/200)
INFO  [22:04:28.174] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 102/200)
INFO  [22:04:28.216] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 103/200)
INFO  [22:04:28.258] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 104/200)
INFO  [22:04:28.299] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 105/200)
INFO  [22:04:28.340] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 106/200)
INFO  [22:04:28.382] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 107/200)
INFO  [22:04:28.430] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 108/200)
INFO  [22:04:28.471] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 109/200)
INFO  [22:04:28.513] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 110/200)
INFO  [22:04:28.556] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 111/200)
INFO  [22:04:28.597] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 112/200)
INFO  [22:04:28.638] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 113/200)
INFO  [22:04:28.680] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 114/200)
INFO  [22:04:28.730] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 115/200)
INFO  [22:04:28.771] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 116/200)
INFO  [22:04:28.812] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 117/200)
INFO  [22:04:28.852] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 118/200)
INFO  [22:04:28.893] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 119/200)
INFO  [22:04:28.934] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 120/200)
INFO  [22:04:28.984] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 121/200)
INFO  [22:04:29.025] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 122/200)
INFO  [22:04:29.066] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 123/200)
INFO  [22:04:29.108] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 124/200)
INFO  [22:04:29.151] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 125/200)
INFO  [22:04:29.198] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 126/200)
INFO  [22:04:29.243] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 127/200)
INFO  [22:04:29.291] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 128/200)
INFO  [22:04:29.333] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 129/200)
INFO  [22:04:29.375] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 130/200)
INFO  [22:04:29.416] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 131/200)
INFO  [22:04:29.457] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 132/200)
INFO  [22:04:29.500] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 133/200)
INFO  [22:04:29.549] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 134/200)
INFO  [22:04:29.591] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 135/200)
INFO  [22:04:29.633] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 136/200)
INFO  [22:04:29.675] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 137/200)
INFO  [22:04:29.715] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 138/200)
INFO  [22:04:29.757] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 139/200)
INFO  [22:04:29.807] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 140/200)
INFO  [22:04:29.849] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 141/200)
INFO  [22:04:29.892] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 142/200)
INFO  [22:04:29.932] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 143/200)
INFO  [22:04:29.973] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 144/200)
INFO  [22:04:30.015] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 145/200)
INFO  [22:04:30.066] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 146/200)
INFO  [22:04:30.106] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 147/200)
INFO  [22:04:30.150] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 148/200)
INFO  [22:04:30.194] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 149/200)
INFO  [22:04:30.237] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 150/200)
INFO  [22:04:30.286] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 151/200)
INFO  [22:04:30.329] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 152/200)
INFO  [22:04:30.368] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 153/200)
INFO  [22:04:30.410] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 154/200)
INFO  [22:04:30.451] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 155/200)
INFO  [22:04:30.494] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 156/200)
INFO  [22:04:30.551] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 157/200)
INFO  [22:04:30.592] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 158/200)
INFO  [22:04:30.634] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 159/200)
INFO  [22:04:30.676] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 160/200)
INFO  [22:04:30.719] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 161/200)
INFO  [22:04:30.759] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 162/200)
INFO  [22:04:30.801] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 163/200)
INFO  [22:04:30.853] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 164/200)
INFO  [22:04:30.897] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 165/200)
INFO  [22:04:30.938] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 166/200)
INFO  [22:04:30.979] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 167/200)
INFO  [22:04:31.021] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 168/200)
INFO  [22:04:31.063] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 169/200)
INFO  [22:04:31.105] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 170/200)
INFO  [22:04:31.158] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 171/200)
INFO  [22:04:31.200] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 172/200)
INFO  [22:04:31.242] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 173/200)
INFO  [22:04:31.285] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 174/200)
INFO  [22:04:31.325] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 175/200)
INFO  [22:04:31.367] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 176/200)
INFO  [22:04:31.407] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 177/200)
INFO  [22:04:31.449] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 178/200)
INFO  [22:04:31.502] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 179/200)
INFO  [22:04:31.543] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 180/200)
INFO  [22:04:31.584] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 181/200)
INFO  [22:04:31.625] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 182/200)
INFO  [22:04:31.667] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 183/200)
INFO  [22:04:31.709] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 184/200)
INFO  [22:04:31.751] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 185/200)
INFO  [22:04:31.804] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 186/200)
INFO  [22:04:31.847] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 187/200)
INFO  [22:04:31.890] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 188/200)
INFO  [22:04:31.932] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 189/200)
INFO  [22:04:31.973] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 190/200)
INFO  [22:04:32.015] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 191/200)
INFO  [22:04:32.059] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 192/200)
INFO  [22:04:32.111] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 193/200)
INFO  [22:04:32.153] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 194/200)
INFO  [22:04:32.193] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 195/200)
INFO  [22:04:32.234] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 196/200)
INFO  [22:04:32.275] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 197/200)
INFO  [22:04:32.319] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 198/200)
INFO  [22:04:32.362] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 199/200)
INFO  [22:04:32.414] [mlr3] Applying learner 'classif.kknn' on task 'abaloneTib_2' (iter 200/200)

Aggregate()

kfoldCv_aba$aggregate(knn_aba_measure)

  classif.acc    classif.ce   classif.tpr   classif.tnr classif.fbeta 
    0.7664476     0.2335524     0.4558165     0.8593518     0.4717204

The results for classif.tpr, classif.tnr, classif.fbeta could be due to the way the unbalanced data is being split into training and test datasets. For some splits, some of the classes might be missing. Some folds may not contain examples of the positive class. ## Sources https://cran.r-project.org/web/packages/mlr3/index.html https://r6.r-lib.org/articles/Introduction.html https://www.rdocumentation.org/packages/mlr3/versions/0.23.0 https://cloud.r-project.org/web/packages/mclust/mclust.pdf\ https://mlr3verse.mlr-org.com https://mlr3book.mlr-org.com/chapters/chapter4/hyperparameter_optimization.html https://www.geeksforgeeks.org/what-are-the-tidyverse-packages-in-r-language/

LS0tDQp0aXRsZTogIkxlYXJuZXIsIFRhc2ssIENyb3NzIFZhbGlkYXRpb24gaW4gbWxyMyINCm91dHB1dDogaHRtbF9ub3RlYm9vaw0KZWRpdG9yX29wdGlvbnM6IA0KICBtYXJrZG93bjogDQogICAgd3JhcDogNzINCi0tLQ0KDQojIyBXaHkgdGhlIG5lZWQgZm9yIGFuIHVwZ3JhZGU/DQoNCm1scjMgd2FzIGNyZWF0ZWQgdG8gYmV0dGVyIHN1cHBvcnQgKipwYXJhbGxlbGl6YXRpb24qKiBhbmQgKipsYXJnZXINCmRhdGFzZXRzKiogKG91dC1vZi1tZW1vcnkgZGF0YS1iYWNrZW5kcykuIFRoaXMgdXBncmFkZSB3b3JrcyB3aXRoIFI2DQpvYmplY3RzIHdoaWNoIGFsbG93IGZvciBzaW1wbGVyIHN5bnRheC4gVGFza3MsIExlYXJuZXJzLCBhbmQgZXZhbHVhdGlvbg0KbWVhc3VyZXMgYXJlIGFsbCBzdG9yZWQgaW4gZGljdGlvbmFyaWVzLiANCg0KKipJbnN0YWxsYXRpb24gb2YgbWxyMyoqIEl0IGlzIHJlY29tbWVuZGVkIHRvIGluc3RhbGwgdGhlIHBhY2thZ2UNCiJtbHIzdmVyc2UiIGluc3RlYWQgb2Ygc2ltcGx5ICJtbHIzIiBzbyB0aGF0IHRoZSBtb3N0IGltcG9ydGFudCBwYWNrYWdlcw0KYXJlIGluc3RhbGxlZC4gTWxyMyBkaWZmZXJzIGZyb20gbWxyIGluIHRoYXQgbGVhcm5lcnMgYW5kIG90aGVyDQpleHRlbnNpb25zIG5lZWQgdG8gYmUgaW5zdGFsbGVkIHNlcGFyYXRlbHkgYWNjb3JkaW5nIHRvIG5lZWQuIE1scjN2ZXJzZQ0KYXZvaWRzIGhhdmluZyB0byBkbyBzb21lIG9mIHRoZXNlIGFkZGl0aW9uYWwgaW5zdGFsbGF0aW9ucy4NCmBgYHtyIGVjaG89VFJVRSwgcmVzdWx0cz0naGlkZScsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0UsIHBhZ2VkLnByaW50PUZBTFNFfQ0KaW5zdGFsbC5wYWNrYWdlcygibWxyM3ZlcnNlIiwgZGVwZW5kZW5jaWVzID0gVFJVRSkNCmBgYA0KDQojIyBMb2FkaW5nIHRoZSBkYXRhc2V0IGFuZCBmb3JtYXR0aW5nIGl0IGFzIGEgdGliYmxlDQoNCkxvYWRpbmcgdGhlIGRhdGEgaXMgdGhlIHNhbWUgYXMgd2l0aCBtbHIsIHVzaW5nIHRoZSBtY2x1c3QgcGFja2FnZS4NCg0KYGBge3IgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRSwgcGFnZWQucHJpbnQ9RkFMU0UsIHJlc3VsdHM9J2hpZGUnfQ0KbGlicmFyeShtbHIzdmVyc2UpDQpsaWJyYXJ5KHRpZHl2ZXJzZSkNCmBgYA0KDQpgYGB7ciBlY2hvPVRSVUV9DQpkYXRhKGRpYWJldGVzLCBwYWNrYWdlID0gIm1jbHVzdCIpDQpkaWFiZXRlc1RpYiA8LSBhc190aWJibGUoZGlhYmV0ZXMpDQpzdW1tYXJ5KGRpYWJldGVzVGliKQ0KYGBgDQoNCioqVGFraW5nIGEgbG9vayBhdCB0aGUgbmV3IHRpYmJsZSoqDQpgYGB7ciBlY2hvPVRSVUV9DQpkaWFiZXRlc1RpYg0KYGBgDQoNCiMjIFVzaW5nIG1scjMgdG8gdHJhaW4gYSBtb2RlbA0KDQoqKkhvdyBkb2VzIGJ1aWxkaW5nIGEgbWFjaGluZSBsZWFybmluZyBtb2RlbCB3aXRoIHRoZSBtbHIzIHBhY2thZ2UNCmRpZmZlciBmcm9tIG1scj8qKg0KDQpKdXN0IGFzIGEgcmVtaW5kZXIgdHJhaW5pbmcgYSBtb2RlbCB3aXRoIG1sciBoYXMgdGhyZWUgbWFpbiBzdGFnZXM6DQoNCioqMS4gRGVmaW5lIHRoZSB0YXNrLiBUaGUgdGFzayBjb25zaXN0cyBvZiB0aGUgZGF0YSBhbmQgd2hhdCB3ZSB3YW50IHRvDQpkbyB3aXRoIGl0LioqDQoNCioqMi4gRGVmaW5lIHRoZSBsZWFybmVyLiBUaGUgbGVhcm5lciBpcyB0aGUgYWxnb3JpdGhtIHdlIHBsYW4gdG8gdXNlLioqDQoNCioqMy4gVHJhaW4gdGhlIG1vZGVsOiB5b3UgcGFzcyB0aGUgdGFzayB0byB0aGUgbGVhcm5lciwgdGhpcyBnZW5lcmF0ZXMgYSBtb2RlbA0KdGhhdCB5b3UgY2FuIHVzZSB0byBtYWtlIGZ1dHVyZSBwcmVkaWN0aW9ucy4qKg0KDQpUaGVzZSAzIG1haW4gc3RhZ2VzIGFyZSBzdGlsbCBuZWNlc3NhcnkgaW4gbWxyMywgaG93ZXZlciB0aGUgd2F5IGluDQp3aGljaCB0aGVzZSBzdGFnZXMgYXJlIGV4ZWN1dGVkIGRpZmZlcnMgc2xpZ2h0bHkuIA0KDQpMZXQncyB0YWtlIGEgbG9vayBhbmQgc2VlIGp1c3QgZXhhY3RseSBob3cgdGhleSBkaWZmZXIuDQoNCioqMS4gRGVmaW5lIHRoZSB0YXNrLioqIA0KTGV04oCZcyBiZWdpbiBieSBjcmVhdGluZyBvdXIgb3duIHRhc2sgaW4gbWxyMyB3aXRoDQoqKmFzX2NsYXNzaWYoKSoqLiANCg0KYGBge3IgZWNobz1UUlVFfQ0KZGlhYmV0ZXNUYXNrPC0gYXNfdGFza19jbGFzc2lmKGRpYWJldGVzVGliLCB0YXJnZXQgPSAiY2xhc3MiKQ0KZGlhYmV0ZXNUYXNrDQpgYGANCg0KKkhvdyBkb2VzIHRoaXMgc3RlcCBsb29rIGluIG1scjoqIA0KPGJyPg0KDQpkaWFiZXRlc1Rhc2sgPC0gKiptYWtlQ2xhc3NpZlRhc2sqKihkYXRhID0NCmRpYWJldGVzVGliLCB0YXJnZXQgPSAiY2xhc3MiKQ0KPGJyPg0KDQotIFdlIG5vIGRvIG5vdCBuZWVkIHRvIHNwZWNpZnkgZGF0YSA9IGRpYWJldGVzVGliLCB3ZSBjYW4NCnNpbXBseSBpbnB1dCBkaWFiZXRlc1RpYiBhbmQgaXQgaXMgcGFzc2VkIGRvd24uDQoNCjxicj4NCg0KKipEZWZpbmluZyB0aGUgbGVhcm5lci4qKg0KSW4gbWxyMyB3ZSBkZWZpbmUgdGhlIGxlYXJuZXIgdXNpbmcgKipscm4oKSoqLiBUbyBjaGVjayB3aGljaCBsZWFybmVycyBhcmUgYXZhaWxhYmxlIHJ1bjogDQoqKmFzLmRhdGEudGFibGUobWxyX2xlYXJuZXJzKSoqDQpgYGB7ciBlY2hvPVRSVUV9DQprbm4gPC0gbHJuKCJjbGFzc2lmLmtrbm4iLCBrID0gMikNCiNpZiB0aGlzIGRvZXMgbm90IHdvcmsgYWRkIHRoaXMgYmVmb3JlIHJ1bm5pbmcgbHJuKCk6IGluc3RhbGwucGFja2FnZXMoImtrbm4iKSANCiNkZWZhdWx0IGsgPSA3IA0KYGBgDQoNCipIb3cgZG9lcyB0aGlzIHN0ZXAgbG9vayBpbiBtbHI6KiANCjxicj4NCmtubiBcPC0gKiptYWtlTGVhcm5lcioqKCJjbGFzc2lmLmtubiIsDQoqKnBhci52YWxzID0gbGlzdChrID0gMikqKikuIA0KPGJyPg0KDQotIE1ha2VMZWFybmVyIG5vIGxvbmdlciB1c2VkLCBpdCBpcyByZXBsYWNlZCB3aXRoIGxybigpIA0KDQotIHBhci52YWxzID0gbGlzdCgpIC0+IHNldHMgaHlwZXJwYXJhbWV0ZXIgaW4gbWxyLiBObyBsb25nZXIgdXNlZCBpbiBtbHIzLCBoeXBlcnBhcmFtZXRlciBjYW4gYmUgc2V0IGRpcmVjdGx5DQoNCjxicj4NCg0KKipUcmFpbmluZyB0aGUgbW9kZWwuKiogSW4gbWxyMyBjZXJ0YWluIG1ldGhvZHMgc3VjaCBhcyB0cmFpbigpLCBtb2RlbCgpLA0KYW5kIHByZWRpY3QoKSBhcmUgY2FsbGVkIHVzaW5nIHRoZSAqKlwkIG9wZXJhdG9yKiouIFBhc3MgdGhlIGxlYXJuZXIgdG8NCnRoZSBtb2RlbCB2aWEgKipcJHRyYWluKCkqKjoNCg0KYGBge3IgZWNobz1UUlVFfQ0Ka25uX21vZGVsIDwtIGtubiR0cmFpbihkaWFiZXRlc1Rhc2spDQprbm5fbW9kZWwNCmBgYA0KDQoqSG93IGRvZXMgdGhpcyBsb29rIGluIG1scjoqIA0KPGJyPg0Ka25uTW9kZWwgXDwtIHRyYWluKGtubiwgZGlhYmV0ZXNUYXNrKQ0KDQo8YnI+DQoNCkFmdGVyIHRyYWluaW5nLCB0aGUgZml0dGVkIG1vZGVsIGlzIHN0b3JlZCBpbiB0aGUgKiokbW9kZWwqKiBmaWVsZCBmb3IgYSBtb3JlIGRldGFpbGVkIGluc3BlY3Rpb24NCmBgYHtyfQ0Ka25uX21vZGVsJG1vZGVsDQpgYGANCjxicj4NCg0KKipQcmVkaWN0aW9uKioNClByZWRpY3QgZnJvbSB0aGUgdHJhaW5lZCBtb2RlbCB1c2luZyAqKiRwcmVkaWN0KCkqKjoNCg0KYGBge3IgZWNobz1UUlVFfQ0Ka25uX3ByZWRpY3QgPC0ga25uX21vZGVsJHByZWRpY3QoZGlhYmV0ZXNUYXNrKQ0Ka25uX3ByZWRpY3QNCmBgYA0KDQoqSG93IGRvZXMgdGhpcyBsb29rIGluIG1scio6IA0KPGJyPg0Ka25uUHJlZCBcPC0gcHJlZGljdChrbm5Nb2RlbCwgbmV3ZGF0YSA9DQpkaWFiZXRlc1RpYikNCjxicj4NCg0KLSBtb2RlbCBpcyBpbnNpZGUgcGFyZW50aGVzaXMgDQotIGRhdGEgbmVlZGVkIHRvIGJlIHNldCBhcyBhIHBhcmFtZXRlcg0KDQo8YnI+DQoNCioqUGVyZm9ybWFuY2UgTWVhc3VyZXMqKiBTdG9yZWQgaW4gbWxyX21lYXN1cmVzIGRpY3Rpb25hcnksIGFjY2Vzc2VkDQp3aXRoICoqbXNyKCkqKiBmb3IgYSBzaW5nbGUgbWVhc3VyZW1lbnQgYW5kICoqbXNycygpKiogZm9yIG11bHRpcGxlDQptZWFzdXJlbWVudHMuDQpgYGB7ciBlY2hvPVRSVUV9DQprbm5fbWVhc3VyZSA8LSBtc3JzKGMoImNsYXNzaWYuYWNjIiwgImNsYXNzaWYuY2UiKSkgI211c3QgYmUgc2V0IHVwIGFzIGEgdmVjdG9yIGMoKSBpbiBvcmRlciB0byB0cmVhdCBpdCBhcyBhIHNpbmdsZSBhcmd1bWVudC4gDQprbm5fcHJlZGljdCRzY29yZShrbm5fbWVhc3VyZSkNCmBgYA0KDQoNCipIb3cgZG9lcyB0aGlzIGxvb2sgaW4gbWxyKjogDQo8YnI+IA0KcGVyZm9ybWFuY2Uoa25uUHJlZCwgbWVhc3VyZXMgPSBsaXN0KG1tY2UsIGFjYykpDQoNCg0KIyMgQ3Jvc3MgVmFsaWRhdGlvbiBpbiBtbHIzDQpLLWZvbGQgY3Jvc3MtdmFsaWRhdGlvbg0KV2Ugd2lsbCB1c2UgcmVwZWF0ZWQgay1mb2xkIGNyb3NzLXZhbGlkYXRpb24sIHNpbmNlIHdlIGFscmVhZHkga25vdyB0aGlzIGlzIHRoZSBiZXN0IG9wdGlvbg0KaWYgdGhlIGRhdGFzZXQgaXMgbm90IHRvbyBsYXJnZS4NCg0KKipSZXNhbXBsaW5nIERlc2NyaXB0aW9uOioqDQpXZSBmaXJzdCBuZWVkIHRvIG1ha2Ugb3VyIHJlc2FtcGxpbmcgZGVzY3JpcHRpb24gd2hpY2ggaXMgc2ltcGx5IHRoZSBzZXQgb2YgaW5zdHJ1Y3Rpb25zIGZvciBob3cgdGhlIGRhdGEgd2lsbCBiZSBzcGxpdCBpbnRvICp0ZXN0KiBhbmQgKnRyYWluaW5nKiBzZXRzLg0KYGBge3J9DQprbm5fY3YgPC0gcnNtcCgicmVwZWF0ZWRfY3YiLCByZXBlYXRzID0gNTAsIGZvbGRzID0gMTApICNzdHJhdGlmeSBhdXRvbWF0aWNhbGx5IGRvbmUgaW4gbWxyMw0KYGBgDQpVc2luZyB0aGUgKipyZXNhbXBsZSgpKiogZnVuY3Rpb24gYXMgaW4gbWxyLCB3ZSBjYW4gcnVuIHRoZSBjcm9zcy12YWxpZGF0aW9uIG9uIG91ciBsZWFybmVyLiANCmBgYHtyIGVjaG89VFJVRX0NCmtmb2xkQ3YgPC0gcmVzYW1wbGUoZGlhYmV0ZXNUYXNrLCBrbm4sIGtubl9jdikNCmBgYA0KKkhvdyBkb2VzIHRoaXMgbG9vayBpbiBtbHI6Kg0KPGJyPiANCg0KLSBrRm9sZCA8LSAqKm1ha2VSZXNhbXBsZURlc2MqKihtZXRob2QgPSAiUmVwQ1YiLCBmb2xkcyA9IDEwLCByZXBzID0gNTAsIHN0cmF0aWZ5ID0gVFJVRSkNCg0KLSBrRm9sZENWIDwtIHJlc2FtcGxlKGxlYXJuZXIgPSBrbm4sIHRhc2sgPSBkaWFiZXRlc1Rhc2ssIHJlc2FtcGxpbmcgPSBrRm9sZCwgbWVhc3VyZXMgPSBsaXN0KG1tY2UsDQogICAgYWNjKSkNCi0gSGVyZSBldmVyeXRoaW5nIGhhcyB0byBiZSBzcGVjaWZpY2FsbHkgZGVmaW5lZCwgaW4gbWxyMyB0aGF0IGlzIG5vIGxvbmdlciBuZWNlc3NhcnkNCi0gQnV0IGluIG1scjMgYSBzcGVjaWZpYyBvcmRlciBpcyBuZWNlc3NhcnkgaW4gb3JkZXIgZm9yIHRoZSBjb21tYW5kIHRvIHdvcms6IHRhc2ssIGxlYXJuZXIsIHJlc2FtcGxpbmcuIA0KDQpGaW5hbGx5LCAqKkFnZ3JlZ2F0ZSgpKiogcHJvdmlkZXMgdXMgd2l0aCB0aGUgc3VtbWFyeSBwZXJmb3JtYW5jZSBtZWFzdXJlcyBhZnRlciB0aGUgay1mb2xkLiANCmBgYHtyIGVjaG89VFJVRX0NCmtmb2xkQ3YkYWdncmVnYXRlKGtubl9tZWFzdXJlKQ0KYGBgDQo8YnI+DQoNCioqQ2FsY3VsYXRpbmcgdGhlIGNvbmZ1c2lvbiBNYXRyaXgqKiANCnRoZSBjb25mdXNpb24gbWF0cml4IGlzIGNhbGN1bGF0ZWQgdXNpbmcgJGNvbmZ1c2lvbiBvbiB0aGUgY3Jvc3MtdmFsaWRhdGlvbiBvYmplY3QgYW5kIGNhbGxpbmcgJHByZWRpY3Rpb24NCmBgYHtyIGVjaG89VFJVRX0NCmNvbmZfbWF0cml4IDwtIGtmb2xkQ3YkcHJlZGljdGlvbigpJGNvbmZ1c2lvbg0KY29uZl9tYXRyaXgNCmBgYA0KLSBFcXVpdmFsZW50IHRvIGNhbGN1bGF0ZUNvbmZ1c2lvbk1hdHJpeChrRm9sZENWJHByZWQpIGluIG1sci4gDQoNCiMjIE1scjMgb24gQWJhbG9uZSBEYXRhc2V0DQpgYGB7cn0NCmxpYnJhcnkodGlkeXZlcnNlKQ0KbGlicmFyeShtYWdyaXR0cikNCmBgYA0KDQpgYGB7cn0NCmFiYWxvbmUgPC0gcmVhZC5jc3YoIkM6L1VzZXJzL2pvZWx5L09uZURyaXZlL0Rlc2t0b3AvTWFzdGVyL0FJVi9hYmFsb25lLmNzdiIpDQpgYGANCg0KDQpOZWNlc3NhcnkgdG8gY3JlYXRlIGEgbmV3IGNvbHVtbiAiQWdlIiBhbmQgY29udmVydCBBZ2UgaW50byBhIGZhY3RvciAoY2F0ZWdvcmljYWwgdmFsdWUgIGluIHIgKSBiZWZvcmUgY3JlYXRpbmcgY2xhc3NpZmljYXRpb24gVGFzaw0KYGBge3IgZWNobz1UUlVFfQ0KYWJhbG9uZSAlPD4lDQogICAgZHBseXI6Om11dGF0ZShhYmFsb25lLCBBZ2UgPSBmYWN0b3IoaWZlbHNlKFJpbmdzID4gMTEsICJvbGQiLCAieW91bmciKSkpDQp0YWJsZShhYmFsb25lJEFnZSkNCmBgYA0KDQpgYGB7ciBlY2hvPVRSVUV9DQphYmFsb25lIDwtIGFzX3RpYmJsZShhYmFsb25lKQ0KYWJhbG9uZQ0KYGBgDQoNClNpbmNlIHRoZSBkYXRhc2V0IGlzIHVuYmFsYW5jZWQgYXMgd2Ugc2F3IGluIG91ciBvcmlnaW5hbCBzY3JpcHQgKGRpc3RyaWJ1dGlvbiBvZiBzYW1wbGVzIGFjcm9zcyBpdHMgY2xhc3NlcyBpcyBzaWduaWZpY2FudGx5IHVuZXF1YWwpIHdlIG5lZWQgdG8gaWdub3JlIGNlcnRhaW4gZmVhdHVyZXMuIFdlIGNhbiB0cmFuc2Zvcm0gdGhlIGRhdGEgdG8gZG8gdGhpcyB3aXRoIHRoZSAqKnNlbGVjdCgpKiogbWV0aG9kIGluIHRoZSAqKmRwbHlyIHBhY2thZ2UqKi4gDQoNCmBgYHtyIGVjaG89VFJVRX0NCmFiYWxvbmVUaWJfMiA8LSBkcGx5cjo6c2VsZWN0KGFiYWxvbmUsIC1jKFNleCwgUmluZ3MpKSAjZXhjbHVkZXMgU2V4LCBSaW5ncyBjb2x1bW4NCmFiYWxvbmVUaWJfMg0KDQpgYGANCg0KKipUYXNrOioqDQpgYGB7cn0NCmFiYWxvbmVfdGFzayA8LSBhc190YXNrX2NsYXNzaWYoYWJhbG9uZVRpYl8yLCB0YXJnZXQgPSAiQWdlIikNCmBgYA0KKipMZWFybmVyOioqDQpgYGB7cn0NCmtubl9hYmEgPC0gbHJuKCJjbGFzc2lmLmtrbm4iLCBrID0gMikNCmBgYA0KKipUcmFpbiBNb2RlbDoqKiANCmBgYHtyIGVjaG89VFJVRX0NCmtubl9hYmFfbW9kZWwgPC0ga25uX2FiYSR0cmFpbihhYmFsb25lX3Rhc2spDQprbm5fYWJhX21vZGVsDQpgYGANCioqUHJlZGljdGlvbioqDQpgYGB7ciBlY2hvPVRSVUV9DQprbm5fYWJhX3ByZWRpY3QgPC0ga25uX2FiYV9tb2RlbCRwcmVkaWN0KGFiYWxvbmVfdGFzaykNCmtubl9hYmFfcHJlZGljdA0KYGBgDQoNCioqUGVyZm9ybWFuY2UgTWVhc3VyZXMqKiANCmBgYHtyIGVjaG89VFJVRX0NCmtubl9hYmFfbWVhc3VyZSA8LSBtc3JzKGMoImNsYXNzaWYuYWNjIiwgImNsYXNzaWYuY2UiLCAiY2xhc3NpZi50cHIiLCAiY2xhc3NpZi50bnIiLCAiY2xhc3NpZi5mYmV0YSIpKSAjZGVmYXVsdCBmMSBzY29yZSANCmtubl9hYmFfcHJlZGljdCRzY29yZShrbm5fYWJhX21lYXN1cmUpDQoNCmBgYA0KDQoqKlJlc2FtcGxlIERlc2NyaXB0aW9uKioNCg0KYGBge3J9DQprbm5fY3ZfYWJhIDwtIHJzbXAoInJlcGVhdGVkX2N2IiwgcmVwZWF0cyA9IDIwLCBmb2xkcyA9IDEwKQ0KYGBgDQpVc2luZyB0aGUgKipyZXNhbXBsZSgpKiogZnVuY3Rpb24sIHdlIGNhbiBydW4gdGhlIGNyb3NzLXZhbGlkYXRpb24gb24gb3VyIGxlYXJuZXIuIA0KYGBge3IgZWNobz1UUlVFfQ0Ka2ZvbGRDdl9hYmEgPC0gcmVzYW1wbGUoYWJhbG9uZV90YXNrLCBrbm5fYWJhLCBrbm5fY3ZfYWJhKQ0KYGBgDQoNCioqQWdncmVnYXRlKCkqKg0KYGBge3J9DQprZm9sZEN2X2FiYSRhZ2dyZWdhdGUoa25uX2FiYV9tZWFzdXJlKQ0KYGBgDQpUaGUgcmVzdWx0cyBmb3IgY2xhc3NpZi50cHIsIGNsYXNzaWYudG5yLCBjbGFzc2lmLmZiZXRhIGNvdWxkIGJlIGR1ZSB0byB0aGUgd2F5IHRoZSB1bmJhbGFuY2VkIGRhdGEgaXMgYmVpbmcgc3BsaXQgaW50byB0cmFpbmluZyBhbmQgdGVzdCBkYXRhc2V0cy4gRm9yIHNvbWUgc3BsaXRzLCBzb21lIG9mIHRoZSBjbGFzc2VzIG1pZ2h0IGJlIG1pc3NpbmcuIFNvbWUgZm9sZHMgbWF5IG5vdCBjb250YWluIGV4YW1wbGVzIG9mIHRoZSBwb3NpdGl2ZSBjbGFzcy4gDQojIyBTb3VyY2VzDQpodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvbWxyMy9pbmRleC5odG1sDQpodHRwczovL3I2LnItbGliLm9yZy9hcnRpY2xlcy9JbnRyb2R1Y3Rpb24uaHRtbA0KaHR0cHM6Ly93d3cucmRvY3VtZW50YXRpb24ub3JnL3BhY2thZ2VzL21scjMvdmVyc2lvbnMvMC4yMy4wDQpodHRwczovL2Nsb3VkLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL21jbHVzdC9tY2x1c3QucGRmXA0KaHR0cHM6Ly9tbHIzdmVyc2UubWxyLW9yZy5jb20NCmh0dHBzOi8vbWxyM2Jvb2subWxyLW9yZy5jb20vY2hhcHRlcnMvY2hhcHRlcjQvaHlwZXJwYXJhbWV0ZXJfb3B0aW1pemF0aW9uLmh0bWwNCmh0dHBzOi8vd3d3LmdlZWtzZm9yZ2Vla3Mub3JnL3doYXQtYXJlLXRoZS10aWR5dmVyc2UtcGFja2FnZXMtaW4tci1sYW5ndWFnZS8NCg==