Teoría

EL paquete CARET (Classification and Regression Training) es un paquete integral con una amplia variedad de algoritmos para el aprendizaje automático.

Teoría

#install.packages("caret") #Algoritmos de aprendizaje automático
library(caret)
## Loading required package: ggplot2
## Loading required package: lattice
#install.packages("ggplot2") # Gráficas
library(ggplot2)
#install.packages("lattice") #Crear gráficas
library(lattice)

library(datasets)
#install.packages("DataExplorer") #Análisis descriptivo
library(DataExplorer)
#install.packages("kernlab")
library(kernlab)
## 
## Attaching package: 'kernlab'
## The following object is masked from 'package:ggplot2':
## 
##     alpha
#install.packages("randomForest")
library(randomForest)
## randomForest 4.7-1.2
## Type rfNews() to see new features/changes/bug fixes.
## 
## Attaching package: 'randomForest'
## The following object is masked from 'package:ggplot2':
## 
##     margin

Crear la base de datos

df <- data.frame(iris)

Crear la base de datos

summary(df)
##   Sepal.Length    Sepal.Width     Petal.Length    Petal.Width   
##  Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100  
##  1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300  
##  Median :5.800   Median :3.000   Median :4.350   Median :1.300  
##  Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199  
##  3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800  
##  Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500  
##        Species  
##  setosa    :50  
##  versicolor:50  
##  virginica :50  
##                 
##                 
## 
str(df)
## 'data.frame':    150 obs. of  5 variables:
##  $ Sepal.Length: num  5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
##  $ Sepal.Width : num  3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
##  $ Petal.Length: num  1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
##  $ Petal.Width : num  0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
##  $ Species     : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...
#create report
plot_missing(df)

plot_histogram(df)

plot_correlation(df)

NOTA: En modelos de clasificación, la variable que queremos predecir debe de tener formto FACTOR

Partir la base de datos

#Normalmente 80-20 o 70-30
set.seed(123)
renglones_entrenamiento <- createDataPartition(df$Species, p=0.8, list=FALSE)
entrenamiento <- df[renglones_entrenamiento, ]
prueba <- df [-renglones_entrenamiento, ]

Distintos tipos de Métodos para Modelar

Los métodos más utlizados para modelar aprendizaje automático son:

  • SVM: Support Ventor Machine o Maquina de Ventores de Soporte. Hay varios subtipos: Lineal (svmLinear), Radial(svmRdadial), Polinómico (svmPoly), etc.

  • Árbol de decesión: rpart

  • Redes Neuronales : nnet

  • Radom Forests : rf

Modelo 1. MVS Lineal

modelo1 <- train(Species ~ ., 
                 data = entrenamiento,
                 method = "svmLinear", 
                 preProcess = c("scale", "center"),
                 trControl = trainControl(method = "cv", number = 10), 
                 tuneGrid = data.frame(C = 1)
)
resultado_entrenamiento1 <- predict(modelo1, entrenamiento)
resultado_prueba1 <- predict(modelo1, prueba)

#matriz de confusión
#tabla de evaluacion que desglosa el rendimiento del modelo de clasificación

#matriz de confusión del resultado de entrenamiento
mcre1 <- confusionMatrix(resultado_entrenamiento1, entrenamiento$Species)
mcre1
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         40          0         0
##   versicolor      0         39         0
##   virginica       0          1        40
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9917          
##                  95% CI : (0.9544, 0.9998)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.9875          
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            0.9750           1.0000
## Specificity                 1.0000            1.0000           0.9875
## Pos Pred Value              1.0000            1.0000           0.9756
## Neg Pred Value              1.0000            0.9877           1.0000
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3250           0.3333
## Detection Prevalence        0.3333            0.3250           0.3417
## Balanced Accuracy           1.0000            0.9875           0.9938
#matriz de confusión del resultado de prueba
mcrp1 <- confusionMatrix(resultado_prueba1, prueba$Species)
mcrp1
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0         10         1
##   virginica       0          0         9
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9667          
##                  95% CI : (0.8278, 0.9992)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : 2.963e-13       
##                                           
##                   Kappa : 0.95            
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            1.0000           0.9000
## Specificity                 1.0000            0.9500           1.0000
## Pos Pred Value              1.0000            0.9091           1.0000
## Neg Pred Value              1.0000            1.0000           0.9524
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3333           0.3000
## Detection Prevalence        0.3333            0.3667           0.3000
## Balanced Accuracy           1.0000            0.9750           0.9500

Modelo 2. MVS Radial

modelo2 <- train(Species ~ ., 
                 data = entrenamiento,
                 method = "svmRadial", #cambiar1
                 preProcess = c("scale", "center"),
                 trControl = trainControl(method = "cv", number = 10), 
                 tuneGrid = data.frame(sigma=1, C= 1) #cambiar
)
resultado_entrenamiento2 <- predict(modelo2, entrenamiento)
resultado_prueba2 <- predict(modelo2, prueba)

#matriz de confusión
#tabla de evaluacion que desglosa el rendimiento del modelo de clasificación

#matriz de confusión del resultado de entrenamiento
mcre2 <- confusionMatrix(resultado_entrenamiento2, entrenamiento$Species)
mcre2
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         40          0         0
##   versicolor      0         39         0
##   virginica       0          1        40
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9917          
##                  95% CI : (0.9544, 0.9998)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.9875          
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            0.9750           1.0000
## Specificity                 1.0000            1.0000           0.9875
## Pos Pred Value              1.0000            1.0000           0.9756
## Neg Pred Value              1.0000            0.9877           1.0000
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3250           0.3333
## Detection Prevalence        0.3333            0.3250           0.3417
## Balanced Accuracy           1.0000            0.9875           0.9938
#matriz de confusión del resultado de prueba
mcrp2 <- confusionMatrix(resultado_prueba2, prueba$Species)
mcrp2
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0         10         2
##   virginica       0          0         8
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9333          
##                  95% CI : (0.7793, 0.9918)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : 8.747e-12       
##                                           
##                   Kappa : 0.9             
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            1.0000           0.8000
## Specificity                 1.0000            0.9000           1.0000
## Pos Pred Value              1.0000            0.8333           1.0000
## Neg Pred Value              1.0000            1.0000           0.9091
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3333           0.2667
## Detection Prevalence        0.3333            0.4000           0.2667
## Balanced Accuracy           1.0000            0.9500           0.9000

Modelo 3. MVS Polinómico

modelo3 <- train(Species ~ ., 
                 data = entrenamiento,
                 method = "svmPoly", #cambiar1
                 preProcess = c("scale", "center"),
                 trControl = trainControl(method = "cv", number = 10), 
                 tuneGrid = data.frame(degree=1, scale=1, C= 1) #cambiar
)
resultado_entrenamiento3 <- predict(modelo3, entrenamiento)
resultado_prueba3 <- predict(modelo3, prueba)

#matriz de confusión
#tabla de evaluacion que desglosa el rendimiento del modelo de clasificación

#matriz de confusión del resultado de entrenamiento
mcre3 <- confusionMatrix(resultado_entrenamiento3, entrenamiento$Species)
mcre3
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         40          0         0
##   versicolor      0         39         0
##   virginica       0          1        40
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9917          
##                  95% CI : (0.9544, 0.9998)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.9875          
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            0.9750           1.0000
## Specificity                 1.0000            1.0000           0.9875
## Pos Pred Value              1.0000            1.0000           0.9756
## Neg Pred Value              1.0000            0.9877           1.0000
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3250           0.3333
## Detection Prevalence        0.3333            0.3250           0.3417
## Balanced Accuracy           1.0000            0.9875           0.9938
#matriz de confusión del resultado de prueba
mcrp3 <- confusionMatrix(resultado_prueba3, prueba$Species)
mcrp3
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0         10         1
##   virginica       0          0         9
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9667          
##                  95% CI : (0.8278, 0.9992)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : 2.963e-13       
##                                           
##                   Kappa : 0.95            
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            1.0000           0.9000
## Specificity                 1.0000            0.9500           1.0000
## Pos Pred Value              1.0000            0.9091           1.0000
## Neg Pred Value              1.0000            1.0000           0.9524
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3333           0.3000
## Detection Prevalence        0.3333            0.3667           0.3000
## Balanced Accuracy           1.0000            0.9750           0.9500

Modelo 4. Árbol de decisión

modelo4 <- train(Species ~ ., 
                 data = entrenamiento,
                 method = "rpart", #cambiar1
                 preProcess = c("scale", "center"),
                 trControl = trainControl(method = "cv", number = 10), 
                 tuneLength = 10 #cambiar
)
resultado_entrenamiento4 <- predict(modelo4, entrenamiento)
resultado_prueba4 <- predict(modelo4, prueba)

#matriz de confusión
#tabla de evaluacion que desglosa el rendimiento del modelo de clasificación

#matriz de confusión del resultado de entrenamiento
mcre4 <- confusionMatrix(resultado_entrenamiento4, entrenamiento$Species)
mcre4
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         40          0         0
##   versicolor      0         39         3
##   virginica       0          1        37
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9667          
##                  95% CI : (0.9169, 0.9908)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.95            
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            0.9750           0.9250
## Specificity                 1.0000            0.9625           0.9875
## Pos Pred Value              1.0000            0.9286           0.9737
## Neg Pred Value              1.0000            0.9872           0.9634
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3250           0.3083
## Detection Prevalence        0.3333            0.3500           0.3167
## Balanced Accuracy           1.0000            0.9688           0.9563
#matriz de confusión del resultado de prueba
mcrp4 <- confusionMatrix(resultado_prueba4, prueba$Species)
mcrp4
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0         10         2
##   virginica       0          0         8
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9333          
##                  95% CI : (0.7793, 0.9918)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : 8.747e-12       
##                                           
##                   Kappa : 0.9             
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            1.0000           0.8000
## Specificity                 1.0000            0.9000           1.0000
## Pos Pred Value              1.0000            0.8333           1.0000
## Neg Pred Value              1.0000            1.0000           0.9091
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3333           0.2667
## Detection Prevalence        0.3333            0.4000           0.2667
## Balanced Accuracy           1.0000            0.9500           0.9000

Modelo 5. Random Forests

modelo5 <- train(Species ~ ., 
                 data = entrenamiento,
                 method = "rf", #cambiar1
                 preProcess = c("scale", "center"),
                 trControl = trainControl(method = "cv", number = 10), 
                 tuneGrid = expand.grid (mtry=c(2,4,6)) #cambiar
)
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
## Warning in randomForest.default(x, y, mtry = param$mtry, ...): invalid mtry:
## reset to within valid range
resultado_entrenamiento5 <- predict(modelo5, entrenamiento)
resultado_prueba5 <- predict(modelo5, prueba)

#matriz de confusión
#tabla de evaluacion que desglosa el rendimiento del modelo de clasificación

#matriz de confusión del resultado de entrenamiento
mcre5 <- confusionMatrix(resultado_entrenamiento5, entrenamiento$Species)
mcre5
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         40          0         0
##   versicolor      0         40         0
##   virginica       0          0        40
## 
## Overall Statistics
##                                      
##                Accuracy : 1          
##                  95% CI : (0.9697, 1)
##     No Information Rate : 0.3333     
##     P-Value [Acc > NIR] : < 2.2e-16  
##                                      
##                   Kappa : 1          
##                                      
##  Mcnemar's Test P-Value : NA         
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            1.0000           1.0000
## Specificity                 1.0000            1.0000           1.0000
## Pos Pred Value              1.0000            1.0000           1.0000
## Neg Pred Value              1.0000            1.0000           1.0000
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3333           0.3333
## Detection Prevalence        0.3333            0.3333           0.3333
## Balanced Accuracy           1.0000            1.0000           1.0000
#matriz de confusión del resultado de prueba
mcrp5 <- confusionMatrix(resultado_prueba5, prueba$Species)
mcrp5
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0         10         2
##   virginica       0          0         8
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9333          
##                  95% CI : (0.7793, 0.9918)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : 8.747e-12       
##                                           
##                   Kappa : 0.9             
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            1.0000           0.8000
## Specificity                 1.0000            0.9000           1.0000
## Pos Pred Value              1.0000            0.8333           1.0000
## Neg Pred Value              1.0000            1.0000           0.9091
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3333           0.2667
## Detection Prevalence        0.3333            0.4000           0.2667
## Balanced Accuracy           1.0000            0.9500           0.9000

Modelo 6. Redes Neuronales

modelo6 <- train(Species ~ ., 
                 data = entrenamiento,
                 method = "nnet", #cambiar1
                 preProcess = c("scale", "center"),
                 trControl = trainControl(method = "cv", number = 10) 
                 
)
## # weights:  11
## initial  value 133.467466 
## iter  10 value 56.499094
## iter  20 value 49.923441
## iter  30 value 49.907436
## iter  40 value 49.907285
## iter  50 value 49.901726
## iter  60 value 49.353421
## iter  70 value 48.564455
## iter  80 value 48.498392
## final  value 48.488373 
## converged
## # weights:  27
## initial  value 126.397068 
## iter  10 value 7.833630
## iter  20 value 0.039071
## iter  30 value 0.000176
## final  value 0.000065 
## converged
## # weights:  43
## initial  value 151.741509 
## iter  10 value 4.052042
## iter  20 value 0.395557
## iter  30 value 0.000185
## iter  30 value 0.000095
## iter  30 value 0.000089
## final  value 0.000089 
## converged
## # weights:  11
## initial  value 125.912784 
## iter  10 value 44.821441
## iter  20 value 43.482406
## final  value 43.482383 
## converged
## # weights:  27
## initial  value 138.588105 
## iter  10 value 32.348087
## iter  20 value 20.829281
## iter  30 value 20.199052
## iter  40 value 19.658835
## iter  50 value 19.462631
## iter  60 value 19.457035
## iter  70 value 19.456256
## final  value 19.456256 
## converged
## # weights:  43
## initial  value 136.113616 
## iter  10 value 24.735583
## iter  20 value 18.443553
## iter  30 value 18.059086
## iter  40 value 18.023653
## iter  50 value 18.015729
## iter  60 value 18.015551
## final  value 18.015547 
## converged
## # weights:  11
## initial  value 124.233408 
## iter  10 value 52.514582
## iter  20 value 50.223304
## iter  30 value 50.174117
## iter  40 value 50.140547
## iter  50 value 50.126178
## iter  60 value 49.832738
## iter  70 value 48.454784
## iter  80 value 44.474704
## iter  90 value 33.443336
## iter 100 value 8.020658
## final  value 8.020658 
## stopped after 100 iterations
## # weights:  27
## initial  value 121.216396 
## iter  10 value 13.775459
## iter  20 value 1.507578
## iter  30 value 0.548671
## iter  40 value 0.531342
## iter  50 value 0.468523
## iter  60 value 0.447887
## iter  70 value 0.410866
## iter  80 value 0.369219
## iter  90 value 0.354750
## iter 100 value 0.309708
## final  value 0.309708 
## stopped after 100 iterations
## # weights:  43
## initial  value 143.577723 
## iter  10 value 4.924880
## iter  20 value 0.586712
## iter  30 value 0.468976
## iter  40 value 0.444829
## iter  50 value 0.419467
## iter  60 value 0.363410
## iter  70 value 0.350052
## iter  80 value 0.334215
## iter  90 value 0.327032
## iter 100 value 0.303263
## final  value 0.303263 
## stopped after 100 iterations
## # weights:  11
## initial  value 120.758928 
## iter  10 value 50.313089
## iter  20 value 49.907419
## final  value 49.906646 
## converged
## # weights:  27
## initial  value 116.990448 
## iter  10 value 15.720652
## iter  20 value 2.160592
## iter  30 value 0.083333
## iter  40 value 0.007314
## final  value 0.000058 
## converged
## # weights:  43
## initial  value 133.035359 
## iter  10 value 8.436344
## iter  20 value 1.473068
## iter  30 value 0.000388
## final  value 0.000096 
## converged
## # weights:  11
## initial  value 131.099854 
## iter  10 value 58.159352
## iter  20 value 44.341963
## iter  30 value 44.320189
## final  value 44.320150 
## converged
## # weights:  27
## initial  value 137.441832 
## iter  10 value 35.739495
## iter  20 value 21.370421
## iter  30 value 21.286440
## final  value 21.286428 
## converged
## # weights:  43
## initial  value 142.447903 
## iter  10 value 31.003709
## iter  20 value 20.495501
## iter  30 value 18.608023
## iter  40 value 18.498559
## iter  50 value 18.472618
## iter  60 value 18.472495
## final  value 18.472490 
## converged
## # weights:  11
## initial  value 121.871926 
## iter  10 value 53.613733
## iter  20 value 49.962868
## iter  30 value 49.960469
## iter  40 value 49.952476
## iter  50 value 48.525080
## iter  60 value 35.251093
## iter  70 value 9.330626
## iter  80 value 5.502871
## iter  90 value 4.777306
## iter 100 value 4.234704
## final  value 4.234704 
## stopped after 100 iterations
## # weights:  27
## initial  value 121.369686 
## iter  10 value 10.489758
## iter  20 value 1.831673
## iter  30 value 0.825670
## iter  40 value 0.678147
## iter  50 value 0.601801
## iter  60 value 0.581842
## iter  70 value 0.562350
## iter  80 value 0.543180
## iter  90 value 0.522403
## iter 100 value 0.499835
## final  value 0.499835 
## stopped after 100 iterations
## # weights:  43
## initial  value 130.515530 
## iter  10 value 5.432301
## iter  20 value 1.613859
## iter  30 value 0.605485
## iter  40 value 0.570238
## iter  50 value 0.503516
## iter  60 value 0.420715
## iter  70 value 0.412586
## iter  80 value 0.388752
## iter  90 value 0.368576
## iter 100 value 0.335229
## final  value 0.335229 
## stopped after 100 iterations
## # weights:  11
## initial  value 132.113785 
## iter  10 value 54.763607
## iter  20 value 47.384887
## iter  30 value 42.779933
## iter  40 value 12.808897
## iter  50 value 5.553339
## iter  60 value 3.984547
## iter  70 value 2.945985
## iter  80 value 2.595096
## iter  90 value 2.549872
## iter 100 value 2.484124
## final  value 2.484124 
## stopped after 100 iterations
## # weights:  27
## initial  value 124.997243 
## iter  10 value 17.219030
## iter  20 value 1.654776
## iter  30 value 0.031678
## final  value 0.000088 
## converged
## # weights:  43
## initial  value 131.589667 
## iter  10 value 22.713150
## iter  20 value 2.150048
## iter  30 value 0.001697
## final  value 0.000098 
## converged
## # weights:  11
## initial  value 128.440899 
## iter  10 value 60.867509
## iter  20 value 45.367738
## iter  30 value 43.622901
## final  value 43.618017 
## converged
## # weights:  27
## initial  value 125.942429 
## iter  10 value 28.305778
## iter  20 value 22.181439
## iter  30 value 21.246672
## iter  40 value 21.211123
## iter  50 value 21.207324
## final  value 21.207217 
## converged
## # weights:  43
## initial  value 123.865070 
## iter  10 value 22.909398
## iter  20 value 18.347771
## iter  30 value 18.102669
## iter  40 value 18.089471
## iter  50 value 18.088928
## final  value 18.088869 
## converged
## # weights:  11
## initial  value 124.814605 
## iter  10 value 56.321962
## iter  20 value 54.033362
## iter  30 value 51.784556
## iter  40 value 46.777553
## iter  50 value 30.402626
## iter  60 value 7.055919
## iter  70 value 4.919570
## iter  80 value 4.322950
## iter  90 value 3.786531
## iter 100 value 3.685751
## final  value 3.685751 
## stopped after 100 iterations
## # weights:  27
## initial  value 130.782087 
## iter  10 value 6.203604
## iter  20 value 0.924258
## iter  30 value 0.567102
## iter  40 value 0.536176
## iter  50 value 0.516974
## iter  60 value 0.497094
## iter  70 value 0.488145
## iter  80 value 0.459236
## iter  90 value 0.440405
## iter 100 value 0.429526
## final  value 0.429526 
## stopped after 100 iterations
## # weights:  43
## initial  value 162.432134 
## iter  10 value 16.686319
## iter  20 value 3.627790
## iter  30 value 0.711958
## iter  40 value 0.453051
## iter  50 value 0.420726
## iter  60 value 0.409601
## iter  70 value 0.371096
## iter  80 value 0.362158
## iter  90 value 0.358836
## iter 100 value 0.354201
## final  value 0.354201 
## stopped after 100 iterations
## # weights:  11
## initial  value 138.676295 
## iter  10 value 22.431353
## iter  20 value 7.418031
## iter  30 value 4.251711
## iter  40 value 3.697600
## iter  50 value 2.987236
## iter  60 value 2.646556
## iter  70 value 2.424648
## iter  80 value 2.406296
## iter  90 value 2.245763
## iter 100 value 2.180920
## final  value 2.180920 
## stopped after 100 iterations
## # weights:  27
## initial  value 135.387877 
## iter  10 value 6.845083
## iter  20 value 1.554809
## iter  30 value 0.005483
## iter  40 value 0.001084
## iter  50 value 0.000447
## iter  60 value 0.000123
## final  value 0.000077 
## converged
## # weights:  43
## initial  value 120.251338 
## iter  10 value 4.925799
## iter  20 value 0.724229
## iter  30 value 0.002269
## iter  40 value 0.000193
## final  value 0.000091 
## converged
## # weights:  11
## initial  value 138.090199 
## iter  10 value 65.174174
## iter  20 value 44.674623
## iter  30 value 43.612207
## final  value 43.608489 
## converged
## # weights:  27
## initial  value 170.880571 
## iter  10 value 24.998660
## iter  20 value 21.663455
## iter  30 value 21.522155
## iter  40 value 20.577176
## iter  50 value 19.727951
## iter  60 value 19.704986
## iter  70 value 19.704200
## final  value 19.704199 
## converged
## # weights:  43
## initial  value 119.569215 
## iter  10 value 29.830582
## iter  20 value 18.835389
## iter  30 value 18.479917
## iter  40 value 18.457781
## iter  50 value 18.457495
## final  value 18.457494 
## converged
## # weights:  11
## initial  value 117.902170 
## iter  10 value 49.837360
## iter  20 value 48.971408
## iter  30 value 42.148848
## iter  40 value 10.282000
## iter  50 value 5.079855
## iter  60 value 4.284688
## iter  70 value 3.997024
## iter  80 value 3.813841
## iter  90 value 3.767934
## iter 100 value 3.760707
## final  value 3.760707 
## stopped after 100 iterations
## # weights:  27
## initial  value 118.707425 
## iter  10 value 6.757582
## iter  20 value 1.916853
## iter  30 value 0.608832
## iter  40 value 0.517643
## iter  50 value 0.471617
## iter  60 value 0.449850
## iter  70 value 0.435415
## iter  80 value 0.407723
## iter  90 value 0.396876
## iter 100 value 0.382750
## final  value 0.382750 
## stopped after 100 iterations
## # weights:  43
## initial  value 126.119239 
## iter  10 value 7.924752
## iter  20 value 0.761525
## iter  30 value 0.379959
## iter  40 value 0.359777
## iter  50 value 0.335636
## iter  60 value 0.296540
## iter  70 value 0.278028
## iter  80 value 0.261671
## iter  90 value 0.250033
## iter 100 value 0.229219
## final  value 0.229219 
## stopped after 100 iterations
## # weights:  11
## initial  value 123.471046 
## iter  10 value 87.215235
## iter  20 value 46.096923
## iter  30 value 16.965742
## iter  40 value 4.233919
## iter  50 value 3.271229
## iter  60 value 3.141837
## iter  70 value 3.005910
## iter  80 value 2.833071
## iter  90 value 2.717460
## iter 100 value 2.595672
## final  value 2.595672 
## stopped after 100 iterations
## # weights:  27
## initial  value 128.378233 
## iter  10 value 20.653956
## iter  20 value 1.988002
## iter  30 value 0.003194
## final  value 0.000078 
## converged
## # weights:  43
## initial  value 137.599347 
## iter  10 value 8.637207
## iter  20 value 1.882962
## iter  30 value 0.000622
## final  value 0.000054 
## converged
## # weights:  11
## initial  value 118.689548 
## iter  10 value 49.062720
## iter  20 value 42.954534
## final  value 42.954138 
## converged
## # weights:  27
## initial  value 110.041789 
## iter  10 value 27.699491
## iter  20 value 21.054332
## iter  30 value 19.617310
## iter  40 value 19.532623
## iter  50 value 19.529774
## final  value 19.529773 
## converged
## # weights:  43
## initial  value 137.624367 
## iter  10 value 25.465104
## iter  20 value 18.388498
## iter  30 value 17.648854
## iter  40 value 17.616784
## iter  50 value 17.616111
## iter  60 value 17.615696
## final  value 17.615695 
## converged
## # weights:  11
## initial  value 125.039432 
## iter  10 value 52.812537
## iter  20 value 50.058000
## iter  30 value 50.029214
## iter  40 value 50.013251
## iter  50 value 49.485708
## iter  60 value 43.899141
## iter  70 value 13.740082
## iter  80 value 5.479817
## iter  90 value 4.746620
## iter 100 value 3.748241
## final  value 3.748241 
## stopped after 100 iterations
## # weights:  27
## initial  value 127.451349 
## iter  10 value 8.317237
## iter  20 value 1.724242
## iter  30 value 0.718141
## iter  40 value 0.697257
## iter  50 value 0.632792
## iter  60 value 0.545890
## iter  70 value 0.524162
## iter  80 value 0.470738
## iter  90 value 0.461215
## iter 100 value 0.457980
## final  value 0.457980 
## stopped after 100 iterations
## # weights:  43
## initial  value 162.670235 
## iter  10 value 3.701170
## iter  20 value 1.794747
## iter  30 value 0.603616
## iter  40 value 0.511217
## iter  50 value 0.473268
## iter  60 value 0.457990
## iter  70 value 0.440708
## iter  80 value 0.425178
## iter  90 value 0.412680
## iter 100 value 0.405848
## final  value 0.405848 
## stopped after 100 iterations
## # weights:  11
## initial  value 116.078937 
## iter  10 value 43.764464
## iter  20 value 12.068660
## iter  30 value 0.866386
## iter  40 value 0.192263
## iter  50 value 0.161503
## iter  60 value 0.143024
## iter  70 value 0.131472
## iter  80 value 0.100659
## iter  90 value 0.099228
## iter 100 value 0.060986
## final  value 0.060986 
## stopped after 100 iterations
## # weights:  27
## initial  value 115.997437 
## iter  10 value 5.762137
## iter  20 value 0.014787
## final  value 0.000052 
## converged
## # weights:  43
## initial  value 138.812385 
## iter  10 value 3.020010
## iter  20 value 0.006062
## iter  30 value 0.001855
## iter  40 value 0.000178
## final  value 0.000099 
## converged
## # weights:  11
## initial  value 122.338664 
## iter  10 value 55.542166
## iter  20 value 43.299861
## iter  30 value 43.130133
## final  value 43.128281 
## converged
## # weights:  27
## initial  value 115.214571 
## iter  10 value 25.334970
## iter  20 value 20.226924
## iter  30 value 20.007209
## iter  40 value 20.005713
## iter  40 value 20.005713
## iter  40 value 20.005713
## final  value 20.005713 
## converged
## # weights:  43
## initial  value 132.474771 
## iter  10 value 20.536804
## iter  20 value 17.747044
## iter  30 value 17.315580
## iter  40 value 17.259265
## iter  50 value 17.243319
## final  value 17.243254 
## converged
## # weights:  11
## initial  value 119.191617 
## iter  10 value 64.608244
## iter  20 value 54.586920
## iter  30 value 50.888571
## iter  40 value 50.669341
## iter  50 value 49.709341
## iter  60 value 48.664458
## iter  70 value 48.619287
## iter  80 value 48.437118
## iter  90 value 48.121573
## iter 100 value 47.509955
## final  value 47.509955 
## stopped after 100 iterations
## # weights:  27
## initial  value 122.426297 
## iter  10 value 17.925612
## iter  20 value 0.377940
## iter  30 value 0.246832
## iter  40 value 0.229963
## iter  50 value 0.221942
## iter  60 value 0.207172
## iter  70 value 0.199363
## iter  80 value 0.180377
## iter  90 value 0.162314
## iter 100 value 0.155334
## final  value 0.155334 
## stopped after 100 iterations
## # weights:  43
## initial  value 139.772311 
## iter  10 value 13.501853
## iter  20 value 0.287675
## iter  30 value 0.200910
## iter  40 value 0.190224
## iter  50 value 0.166951
## iter  60 value 0.157913
## iter  70 value 0.149719
## iter  80 value 0.146740
## iter  90 value 0.138721
## iter 100 value 0.137314
## final  value 0.137314 
## stopped after 100 iterations
## # weights:  11
## initial  value 119.458477 
## iter  10 value 47.517101
## iter  20 value 43.755224
## iter  30 value 12.273691
## iter  40 value 4.781756
## iter  50 value 4.232692
## iter  60 value 3.398996
## iter  70 value 2.593651
## iter  80 value 2.453325
## iter  90 value 2.370864
## iter 100 value 2.301835
## final  value 2.301835 
## stopped after 100 iterations
## # weights:  27
## initial  value 132.322411 
## iter  10 value 13.684920
## iter  20 value 0.334113
## iter  30 value 0.001431
## final  value 0.000072 
## converged
## # weights:  43
## initial  value 137.342757 
## iter  10 value 6.401086
## iter  20 value 1.724292
## iter  30 value 0.016078
## iter  40 value 0.001959
## final  value 0.000065 
## converged
## # weights:  11
## initial  value 124.828473 
## iter  10 value 57.964945
## iter  20 value 49.160903
## iter  30 value 43.749696
## final  value 43.705141 
## converged
## # weights:  27
## initial  value 151.570239 
## iter  10 value 28.182757
## iter  20 value 20.741288
## iter  30 value 20.621664
## iter  40 value 20.620617
## iter  40 value 20.620617
## iter  40 value 20.620617
## final  value 20.620617 
## converged
## # weights:  43
## initial  value 127.568433 
## iter  10 value 24.887959
## iter  20 value 18.285195
## iter  30 value 18.005680
## iter  40 value 17.957312
## iter  50 value 17.953535
## final  value 17.953387 
## converged
## # weights:  11
## initial  value 143.435993 
## iter  10 value 46.856693
## iter  20 value 28.265895
## iter  30 value 10.711530
## iter  40 value 4.320480
## iter  50 value 4.061257
## iter  60 value 3.996685
## iter  70 value 3.930916
## iter  80 value 3.847145
## iter  90 value 3.830418
## iter 100 value 3.812754
## final  value 3.812754 
## stopped after 100 iterations
## # weights:  27
## initial  value 126.646852 
## iter  10 value 4.901963
## iter  20 value 0.761938
## iter  30 value 0.603430
## iter  40 value 0.590298
## iter  50 value 0.532759
## iter  60 value 0.511825
## iter  70 value 0.502339
## iter  80 value 0.474631
## iter  90 value 0.468888
## iter 100 value 0.462742
## final  value 0.462742 
## stopped after 100 iterations
## # weights:  43
## initial  value 140.543179 
## iter  10 value 6.277340
## iter  20 value 1.298686
## iter  30 value 0.446331
## iter  40 value 0.389343
## iter  50 value 0.285193
## iter  60 value 0.258920
## iter  70 value 0.240574
## iter  80 value 0.230701
## iter  90 value 0.217963
## iter 100 value 0.210798
## final  value 0.210798 
## stopped after 100 iterations
## # weights:  11
## initial  value 125.447636 
## iter  10 value 45.214802
## iter  20 value 9.549114
## iter  30 value 1.865711
## iter  40 value 1.596249
## iter  50 value 1.498337
## iter  60 value 1.469283
## iter  70 value 1.345374
## iter  80 value 1.271922
## iter  90 value 1.240844
## iter 100 value 1.112367
## final  value 1.112367 
## stopped after 100 iterations
## # weights:  27
## initial  value 141.347544 
## iter  10 value 14.567346
## iter  20 value 1.238888
## iter  30 value 0.141314
## iter  40 value 0.002202
## final  value 0.000091 
## converged
## # weights:  43
## initial  value 116.494804 
## iter  10 value 2.310839
## iter  20 value 0.030591
## iter  30 value 0.000260
## final  value 0.000093 
## converged
## # weights:  11
## initial  value 105.103680 
## iter  10 value 49.049945
## iter  20 value 43.541863
## final  value 43.538032 
## converged
## # weights:  27
## initial  value 127.530053 
## iter  10 value 26.180650
## iter  20 value 19.560401
## iter  30 value 18.857004
## iter  40 value 18.770348
## iter  50 value 18.768606
## final  value 18.768372 
## converged
## # weights:  43
## initial  value 138.328188 
## iter  10 value 41.416008
## iter  20 value 18.777403
## iter  30 value 17.391933
## iter  40 value 17.266946
## iter  50 value 17.227451
## iter  60 value 17.216363
## iter  70 value 17.213191
## iter  80 value 17.212823
## final  value 17.212822 
## converged
## # weights:  11
## initial  value 123.273671 
## iter  10 value 60.798674
## iter  20 value 15.968925
## iter  30 value 3.992467
## iter  40 value 3.676835
## iter  50 value 3.266294
## iter  60 value 3.119206
## iter  70 value 3.072751
## iter  80 value 3.057633
## iter  90 value 3.040420
## iter 100 value 3.006222
## final  value 3.006222 
## stopped after 100 iterations
## # weights:  27
## initial  value 167.332909 
## iter  10 value 4.179610
## iter  20 value 0.367410
## iter  30 value 0.354591
## iter  40 value 0.311338
## iter  50 value 0.269177
## iter  60 value 0.264758
## iter  70 value 0.253690
## iter  80 value 0.251488
## iter  90 value 0.241622
## iter 100 value 0.239302
## final  value 0.239302 
## stopped after 100 iterations
## # weights:  43
## initial  value 143.407027 
## iter  10 value 15.357143
## iter  20 value 1.534211
## iter  30 value 0.443356
## iter  40 value 0.383355
## iter  50 value 0.355698
## iter  60 value 0.339996
## iter  70 value 0.302569
## iter  80 value 0.280035
## iter  90 value 0.267095
## iter 100 value 0.257840
## final  value 0.257840 
## stopped after 100 iterations
## # weights:  11
## initial  value 119.444080 
## iter  10 value 48.359824
## iter  20 value 18.910175
## iter  30 value 7.475154
## iter  40 value 4.194597
## iter  50 value 3.802844
## iter  60 value 1.473123
## iter  70 value 0.378392
## iter  80 value 0.301722
## iter  90 value 0.225668
## iter 100 value 0.221834
## final  value 0.221834 
## stopped after 100 iterations
## # weights:  27
## initial  value 138.011225 
## iter  10 value 14.287412
## iter  20 value 0.529840
## iter  30 value 0.001575
## final  value 0.000053 
## converged
## # weights:  43
## initial  value 131.174576 
## iter  10 value 15.699141
## iter  20 value 0.135465
## iter  30 value 0.001181
## final  value 0.000071 
## converged
## # weights:  11
## initial  value 123.614548 
## iter  10 value 59.732694
## iter  20 value 45.737050
## iter  30 value 43.810373
## final  value 43.810367 
## converged
## # weights:  27
## initial  value 137.739333 
## iter  10 value 28.464357
## iter  20 value 20.705956
## iter  30 value 19.731867
## iter  40 value 19.658595
## final  value 19.658589 
## converged
## # weights:  43
## initial  value 132.978289 
## iter  10 value 31.295889
## iter  20 value 19.514598
## iter  30 value 18.051996
## iter  40 value 18.039360
## iter  50 value 18.037890
## iter  60 value 18.037709
## final  value 18.037430 
## converged
## # weights:  11
## initial  value 126.397373 
## iter  10 value 40.463937
## iter  20 value 5.758678
## iter  30 value 3.393941
## iter  40 value 3.232755
## iter  50 value 3.106484
## iter  60 value 3.031050
## iter  70 value 2.958568
## iter  80 value 2.957785
## iter  90 value 2.950938
## iter 100 value 2.950016
## final  value 2.950016 
## stopped after 100 iterations
## # weights:  27
## initial  value 130.595339 
## iter  10 value 13.350772
## iter  20 value 0.454270
## iter  30 value 0.346626
## iter  40 value 0.325928
## iter  50 value 0.294918
## iter  60 value 0.283251
## iter  70 value 0.272378
## iter  80 value 0.265604
## iter  90 value 0.256185
## iter 100 value 0.240390
## final  value 0.240390 
## stopped after 100 iterations
## # weights:  43
## initial  value 146.843475 
## iter  10 value 7.738310
## iter  20 value 0.414368
## iter  30 value 0.308484
## iter  40 value 0.285415
## iter  50 value 0.237763
## iter  60 value 0.224638
## iter  70 value 0.198467
## iter  80 value 0.191303
## iter  90 value 0.189064
## iter 100 value 0.186599
## final  value 0.186599 
## stopped after 100 iterations
## # weights:  11
## initial  value 123.085481 
## iter  10 value 50.013738
## iter  20 value 46.553501
## iter  30 value 46.310232
## iter  40 value 46.265203
## iter  50 value 45.656500
## iter  60 value 44.539686
## iter  70 value 39.631834
## iter  80 value 17.998857
## iter  90 value 5.826749
## iter 100 value 4.385807
## final  value 4.385807 
## stopped after 100 iterations
## # weights:  27
## initial  value 118.817084 
## iter  10 value 6.042781
## iter  20 value 1.302175
## iter  30 value 0.000494
## final  value 0.000054 
## converged
## # weights:  43
## initial  value 124.677811 
## iter  10 value 5.554376
## iter  20 value 0.911985
## iter  30 value 0.001135
## final  value 0.000098 
## converged
## # weights:  11
## initial  value 114.811560 
## iter  10 value 47.440500
## iter  20 value 44.030135
## final  value 44.030129 
## converged
## # weights:  27
## initial  value 125.805148 
## iter  10 value 28.828795
## iter  20 value 21.493917
## iter  30 value 21.374817
## final  value 21.374738 
## converged
## # weights:  43
## initial  value 133.825125 
## iter  10 value 28.655059
## iter  20 value 19.881791
## iter  30 value 19.152076
## iter  40 value 19.091301
## iter  50 value 19.090205
## final  value 19.090168 
## converged
## # weights:  11
## initial  value 125.247815 
## iter  10 value 32.579451
## iter  20 value 9.371730
## iter  30 value 4.717510
## iter  40 value 4.324216
## iter  50 value 4.030714
## iter  60 value 3.981077
## iter  70 value 3.880663
## iter  80 value 3.874985
## iter  90 value 3.872048
## iter 100 value 3.871548
## final  value 3.871548 
## stopped after 100 iterations
## # weights:  27
## initial  value 126.824706 
## iter  10 value 8.610085
## iter  20 value 1.506732
## iter  30 value 0.937729
## iter  40 value 0.773666
## iter  50 value 0.586116
## iter  60 value 0.504953
## iter  70 value 0.468180
## iter  80 value 0.441652
## iter  90 value 0.396302
## iter 100 value 0.369654
## final  value 0.369654 
## stopped after 100 iterations
## # weights:  43
## initial  value 124.149888 
## iter  10 value 11.207139
## iter  20 value 1.966745
## iter  30 value 0.502958
## iter  40 value 0.446473
## iter  50 value 0.419642
## iter  60 value 0.335366
## iter  70 value 0.297527
## iter  80 value 0.272806
## iter  90 value 0.266718
## iter 100 value 0.258663
## final  value 0.258663 
## stopped after 100 iterations
## # weights:  11
## initial  value 143.828028 
## iter  10 value 75.616983
## iter  20 value 59.523683
## iter  30 value 47.460613
## iter  40 value 46.598156
## iter  40 value 46.598156
## iter  40 value 46.598156
## final  value 46.598156 
## converged
resultado_entrenamiento6 <- predict(modelo6, entrenamiento)
resultado_prueba6 <- predict(modelo6, prueba)

#matriz de confusión
#tabla de evaluacion que desglosa el rendimiento del modelo de clasificación

#matriz de confusión del resultado de entrenamiento
mcre6 <- confusionMatrix(resultado_entrenamiento6, entrenamiento$Species)
mcre6
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         40          0         0
##   versicolor      0         36         0
##   virginica       0          4        40
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9667          
##                  95% CI : (0.9169, 0.9908)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.95            
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            0.9000           1.0000
## Specificity                 1.0000            1.0000           0.9500
## Pos Pred Value              1.0000            1.0000           0.9091
## Neg Pred Value              1.0000            0.9524           1.0000
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3000           0.3333
## Detection Prevalence        0.3333            0.3000           0.3667
## Balanced Accuracy           1.0000            0.9500           0.9750
#matriz de confusión del resultado de prueba
mcrp6 <- confusionMatrix(resultado_prueba6, prueba$Species)
mcrp6
## Confusion Matrix and Statistics
## 
##             Reference
## Prediction   setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0          9         0
##   virginica       0          1        10
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9667          
##                  95% CI : (0.8278, 0.9992)
##     No Information Rate : 0.3333          
##     P-Value [Acc > NIR] : 2.963e-13       
##                                           
##                   Kappa : 0.95            
##                                           
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: setosa Class: versicolor Class: virginica
## Sensitivity                 1.0000            0.9000           1.0000
## Specificity                 1.0000            1.0000           0.9500
## Pos Pred Value              1.0000            1.0000           0.9091
## Neg Pred Value              1.0000            0.9524           1.0000
## Prevalence                  0.3333            0.3333           0.3333
## Detection Rate              0.3333            0.3000           0.3333
## Detection Prevalence        0.3333            0.3000           0.3667
## Balanced Accuracy           1.0000            0.9500           0.9750

Tabla de Resultados

resultados <- data.frame(
  "svmLinear" = c(mcre1$overall["Accuracy"], mcrp1$overall["Accuracy"]),
  "svmRadial" = c(mcre2$overall["Accuracy"], mcrp2$overall["Accuracy"]),
  "svmPoly"   = c(mcre3$overall["Accuracy"], mcrp3$overall["Accuracy"]),
  "rpart"     = c(mcre4$overall["Accuracy"], mcrp4$overall["Accuracy"]),
  "rf"        = c(mcre5$overall["Accuracy"], mcrp5$overall["Accuracy"]),
  "nnet"      = c(mcre6$overall["Accuracy"], mcrp6$overall["Accuracy"])
)

rownames(resultados) <- c("Exactitud del Entrenamiento", "Exactitud de la Prueba")
resultados
##                             svmLinear svmRadial   svmPoly     rpart        rf
## Exactitud del Entrenamiento 0.9916667 0.9916667 0.9916667 0.9666667 1.0000000
## Exactitud de la Prueba      0.9666667 0.9333333 0.9666667 0.9333333 0.9333333
##                                  nnet
## Exactitud del Entrenamiento 0.9666667
## Exactitud de la Prueba      0.9666667

Tabla de Resultados

En conclusión el modelo de **Redes Neuronales* es el recomendado para la clasificación de los lirios.

LS0tCnRpdGxlOiAiQ0FSRVQgLSBJUklTIgphdXRob3I6ICJDZWNpbGlhIEJhbGRpdCIKZGF0ZTogIjIwMjYtMDgtMjUiCm91dHB1dDoKICAgIGh0bWxfZG9jdW1lbnQ6CiAgICAgIHRvYzogVFJVRQogICAgICB0b2NfZmxvYXQ6IFRSVUUKICAgICAgY29kZV9kb3dubG9hZDogVFJVRQogICAgICB0aGVtZTogc2FuZHN0b25lCi0tLQohW10oaHR0cHM6Ly9lbmNyeXB0ZWQtdGJuMC5nc3RhdGljLmNvbS9pbWFnZXM/cT10Ym46QU5kOUdjVEgtbHJoV09PeVF6VFh6RmJ5SVRnbzhLcDFRT2d2bzNSbnFIN25nc2FmdmR2dXVDUlJraVA5bkwtbyZzPTEwKQoKIyA8c3BhbiBzdHlsZSA9ImNvbG9yOmJsdWUiPlRlb3LDrWE8L3NwYW4+CkVMIHBhcXVldGUgKipDQVJFVCAoQ2xhc3NpZmljYXRpb24gYW5kIFJlZ3Jlc3Npb24gVHJhaW5pbmcpKiogZXMgdW4gcGFxdWV0ZSBpbnRlZ3JhbCBjb24gdW5hIGFtcGxpYSB2YXJpZWRhZCBkZSBhbGdvcml0bW9zIHBhcmEgZWwgYXByZW5kaXphamUgYXV0b23DoXRpY28uCgoKIyA8c3BhbiBzdHlsZSA9ImNvbG9yOmJsdWUiPlRlb3LDrWE8L3NwYW4+CmBgYHtyfQojaW5zdGFsbC5wYWNrYWdlcygiY2FyZXQiKSAjQWxnb3JpdG1vcyBkZSBhcHJlbmRpemFqZSBhdXRvbcOhdGljbwpsaWJyYXJ5KGNhcmV0KQojaW5zdGFsbC5wYWNrYWdlcygiZ2dwbG90MiIpICMgR3LDoWZpY2FzCmxpYnJhcnkoZ2dwbG90MikKI2luc3RhbGwucGFja2FnZXMoImxhdHRpY2UiKSAjQ3JlYXIgZ3LDoWZpY2FzCmxpYnJhcnkobGF0dGljZSkKCmxpYnJhcnkoZGF0YXNldHMpCiNpbnN0YWxsLnBhY2thZ2VzKCJEYXRhRXhwbG9yZXIiKSAjQW7DoWxpc2lzIGRlc2NyaXB0aXZvCmxpYnJhcnkoRGF0YUV4cGxvcmVyKQojaW5zdGFsbC5wYWNrYWdlcygia2VybmxhYiIpCmxpYnJhcnkoa2VybmxhYikKI2luc3RhbGwucGFja2FnZXMoInJhbmRvbUZvcmVzdCIpCmxpYnJhcnkocmFuZG9tRm9yZXN0KQpgYGAKCiMgPHNwYW4gc3R5bGUgPSJjb2xvcjpibHVlIj5DcmVhciBsYSBiYXNlIGRlIGRhdG9zPC9zcGFuPgpgYGB7cn0KZGYgPC0gZGF0YS5mcmFtZShpcmlzKQpgYGAKIyA8c3BhbiBzdHlsZSA9ImNvbG9yOmJsdWUiPkNyZWFyIGxhIGJhc2UgZGUgZGF0b3M8L3NwYW4+CmBgYHtyfQpzdW1tYXJ5KGRmKQpzdHIoZGYpCiNjcmVhdGUgcmVwb3J0CnBsb3RfbWlzc2luZyhkZikKcGxvdF9oaXN0b2dyYW0oZGYpCnBsb3RfY29ycmVsYXRpb24oZGYpCmBgYAoKCioqTk9UQTogRW4gbW9kZWxvcyBkZSBjbGFzaWZpY2FjacOzbiwgbGEgdmFyaWFibGUgcXVlIHF1ZXJlbW9zIHByZWRlY2lyIGRlYmUgZGUgdGVuZXIgZm9ybXRvIEZBQ1RPUioqCiAKIyA8c3BhbiBzdHlsZSA9ImNvbG9yOmJsdWUiPlBhcnRpciBsYSBiYXNlIGRlIGRhdG9zPC9zcGFuPgpgYGB7cn0KI05vcm1hbG1lbnRlIDgwLTIwIG8gNzAtMzAKc2V0LnNlZWQoMTIzKQpyZW5nbG9uZXNfZW50cmVuYW1pZW50byA8LSBjcmVhdGVEYXRhUGFydGl0aW9uKGRmJFNwZWNpZXMsIHA9MC44LCBsaXN0PUZBTFNFKQplbnRyZW5hbWllbnRvIDwtIGRmW3Jlbmdsb25lc19lbnRyZW5hbWllbnRvLCBdCnBydWViYSA8LSBkZiBbLXJlbmdsb25lc19lbnRyZW5hbWllbnRvLCBdIApgYGAKCiMgPHNwYW4gc3R5bGUgPSJjb2xvcjpibHVlIj5EaXN0aW50b3MgdGlwb3MgZGUgTcOpdG9kb3MgcGFyYSBNb2RlbGFyPC9zcGFuPgpMb3MgbcOpdG9kb3MgbcOhcyB1dGxpemFkb3MgcGFyYSBtb2RlbGFyIGFwcmVuZGl6YWplIGF1dG9tw6F0aWNvIHNvbjoKCiogKipTVk0qKjogKipTdXBwb3J0IFZlbnRvciBNYWNoaW5lKiogbyBNYXF1aW5hIGRlIFZlbnRvcmVzIGRlIFNvcG9ydGUuIEhheSAKdmFyaW9zIHN1YnRpcG9zOiBMaW5lYWwgKHN2bUxpbmVhciksIFJhZGlhbChzdm1SZGFkaWFsKSwgUG9saW7Ds21pY28gKHN2bVBvbHkpLCBldGMuCgoqICoqw4FyYm9sIGRlIGRlY2VzacOzbioqOiBycGFydCAKKiAqKlJlZGVzIE5ldXJvbmFsZXMqKiA6IG5uZXQKKiAqKlJhZG9tIEZvcmVzdHMqKiA6IHJmCgojIDxzcGFuIHN0eWxlID0iY29sb3I6Ymx1ZSI+TW9kZWxvIDEuIE1WUyBMaW5lYWw8L3NwYW4+CmBgYHtyfQpgYGAKCgpgYGB7cn0KbW9kZWxvMSA8LSB0cmFpbihTcGVjaWVzIH4gLiwgCiAgICAgICAgICAgICAgICAgZGF0YSA9IGVudHJlbmFtaWVudG8sCiAgICAgICAgICAgICAgICAgbWV0aG9kID0gInN2bUxpbmVhciIsIAogICAgICAgICAgICAgICAgIHByZVByb2Nlc3MgPSBjKCJzY2FsZSIsICJjZW50ZXIiKSwKICAgICAgICAgICAgICAgICB0ckNvbnRyb2wgPSB0cmFpbkNvbnRyb2wobWV0aG9kID0gImN2IiwgbnVtYmVyID0gMTApLCAKICAgICAgICAgICAgICAgICB0dW5lR3JpZCA9IGRhdGEuZnJhbWUoQyA9IDEpCikKcmVzdWx0YWRvX2VudHJlbmFtaWVudG8xIDwtIHByZWRpY3QobW9kZWxvMSwgZW50cmVuYW1pZW50bykKcmVzdWx0YWRvX3BydWViYTEgPC0gcHJlZGljdChtb2RlbG8xLCBwcnVlYmEpCgojbWF0cml6IGRlIGNvbmZ1c2nDs24KI3RhYmxhIGRlIGV2YWx1YWNpb24gcXVlIGRlc2dsb3NhIGVsIHJlbmRpbWllbnRvIGRlbCBtb2RlbG8gZGUgY2xhc2lmaWNhY2nDs24KCiNtYXRyaXogZGUgY29uZnVzacOzbiBkZWwgcmVzdWx0YWRvIGRlIGVudHJlbmFtaWVudG8KbWNyZTEgPC0gY29uZnVzaW9uTWF0cml4KHJlc3VsdGFkb19lbnRyZW5hbWllbnRvMSwgZW50cmVuYW1pZW50byRTcGVjaWVzKQptY3JlMQoKI21hdHJpeiBkZSBjb25mdXNpw7NuIGRlbCByZXN1bHRhZG8gZGUgcHJ1ZWJhCm1jcnAxIDwtIGNvbmZ1c2lvbk1hdHJpeChyZXN1bHRhZG9fcHJ1ZWJhMSwgcHJ1ZWJhJFNwZWNpZXMpCm1jcnAxCmBgYAoKIyA8c3BhbiBzdHlsZSA9ImNvbG9yOmJsdWUiPk1vZGVsbyAyLiBNVlMgUmFkaWFsPC9zcGFuPgpgYGB7cn0KbW9kZWxvMiA8LSB0cmFpbihTcGVjaWVzIH4gLiwgCiAgICAgICAgICAgICAgICAgZGF0YSA9IGVudHJlbmFtaWVudG8sCiAgICAgICAgICAgICAgICAgbWV0aG9kID0gInN2bVJhZGlhbCIsICNjYW1iaWFyMQogICAgICAgICAgICAgICAgIHByZVByb2Nlc3MgPSBjKCJzY2FsZSIsICJjZW50ZXIiKSwKICAgICAgICAgICAgICAgICB0ckNvbnRyb2wgPSB0cmFpbkNvbnRyb2wobWV0aG9kID0gImN2IiwgbnVtYmVyID0gMTApLCAKICAgICAgICAgICAgICAgICB0dW5lR3JpZCA9IGRhdGEuZnJhbWUoc2lnbWE9MSwgQz0gMSkgI2NhbWJpYXIKKQpyZXN1bHRhZG9fZW50cmVuYW1pZW50bzIgPC0gcHJlZGljdChtb2RlbG8yLCBlbnRyZW5hbWllbnRvKQpyZXN1bHRhZG9fcHJ1ZWJhMiA8LSBwcmVkaWN0KG1vZGVsbzIsIHBydWViYSkKCiNtYXRyaXogZGUgY29uZnVzacOzbgojdGFibGEgZGUgZXZhbHVhY2lvbiBxdWUgZGVzZ2xvc2EgZWwgcmVuZGltaWVudG8gZGVsIG1vZGVsbyBkZSBjbGFzaWZpY2FjacOzbgoKI21hdHJpeiBkZSBjb25mdXNpw7NuIGRlbCByZXN1bHRhZG8gZGUgZW50cmVuYW1pZW50bwptY3JlMiA8LSBjb25mdXNpb25NYXRyaXgocmVzdWx0YWRvX2VudHJlbmFtaWVudG8yLCBlbnRyZW5hbWllbnRvJFNwZWNpZXMpCm1jcmUyCgojbWF0cml6IGRlIGNvbmZ1c2nDs24gZGVsIHJlc3VsdGFkbyBkZSBwcnVlYmEKbWNycDIgPC0gY29uZnVzaW9uTWF0cml4KHJlc3VsdGFkb19wcnVlYmEyLCBwcnVlYmEkU3BlY2llcykKbWNycDIKYGBgCiMgPHNwYW4gc3R5bGUgPSJjb2xvcjpibHVlIj5Nb2RlbG8gMy4gTVZTIFBvbGluw7NtaWNvPC9zcGFuPgpgYGB7cn0KbW9kZWxvMyA8LSB0cmFpbihTcGVjaWVzIH4gLiwgCiAgICAgICAgICAgICAgICAgZGF0YSA9IGVudHJlbmFtaWVudG8sCiAgICAgICAgICAgICAgICAgbWV0aG9kID0gInN2bVBvbHkiLCAjY2FtYmlhcjEKICAgICAgICAgICAgICAgICBwcmVQcm9jZXNzID0gYygic2NhbGUiLCAiY2VudGVyIiksCiAgICAgICAgICAgICAgICAgdHJDb250cm9sID0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJjdiIsIG51bWJlciA9IDEwKSwgCiAgICAgICAgICAgICAgICAgdHVuZUdyaWQgPSBkYXRhLmZyYW1lKGRlZ3JlZT0xLCBzY2FsZT0xLCBDPSAxKSAjY2FtYmlhcgopCnJlc3VsdGFkb19lbnRyZW5hbWllbnRvMyA8LSBwcmVkaWN0KG1vZGVsbzMsIGVudHJlbmFtaWVudG8pCnJlc3VsdGFkb19wcnVlYmEzIDwtIHByZWRpY3QobW9kZWxvMywgcHJ1ZWJhKQoKI21hdHJpeiBkZSBjb25mdXNpw7NuCiN0YWJsYSBkZSBldmFsdWFjaW9uIHF1ZSBkZXNnbG9zYSBlbCByZW5kaW1pZW50byBkZWwgbW9kZWxvIGRlIGNsYXNpZmljYWNpw7NuCgojbWF0cml6IGRlIGNvbmZ1c2nDs24gZGVsIHJlc3VsdGFkbyBkZSBlbnRyZW5hbWllbnRvCm1jcmUzIDwtIGNvbmZ1c2lvbk1hdHJpeChyZXN1bHRhZG9fZW50cmVuYW1pZW50bzMsIGVudHJlbmFtaWVudG8kU3BlY2llcykKbWNyZTMKCiNtYXRyaXogZGUgY29uZnVzacOzbiBkZWwgcmVzdWx0YWRvIGRlIHBydWViYQptY3JwMyA8LSBjb25mdXNpb25NYXRyaXgocmVzdWx0YWRvX3BydWViYTMsIHBydWViYSRTcGVjaWVzKQptY3JwMwpgYGAKCiMgPHNwYW4gc3R5bGUgPSJjb2xvcjpibHVlIj5Nb2RlbG8gNC4gw4FyYm9sIGRlIGRlY2lzacOzbjwvc3Bhbj4KYGBge3J9Cm1vZGVsbzQgPC0gdHJhaW4oU3BlY2llcyB+IC4sIAogICAgICAgICAgICAgICAgIGRhdGEgPSBlbnRyZW5hbWllbnRvLAogICAgICAgICAgICAgICAgIG1ldGhvZCA9ICJycGFydCIsICNjYW1iaWFyMQogICAgICAgICAgICAgICAgIHByZVByb2Nlc3MgPSBjKCJzY2FsZSIsICJjZW50ZXIiKSwKICAgICAgICAgICAgICAgICB0ckNvbnRyb2wgPSB0cmFpbkNvbnRyb2wobWV0aG9kID0gImN2IiwgbnVtYmVyID0gMTApLCAKICAgICAgICAgICAgICAgICB0dW5lTGVuZ3RoID0gMTAgI2NhbWJpYXIKKQpyZXN1bHRhZG9fZW50cmVuYW1pZW50bzQgPC0gcHJlZGljdChtb2RlbG80LCBlbnRyZW5hbWllbnRvKQpyZXN1bHRhZG9fcHJ1ZWJhNCA8LSBwcmVkaWN0KG1vZGVsbzQsIHBydWViYSkKCiNtYXRyaXogZGUgY29uZnVzacOzbgojdGFibGEgZGUgZXZhbHVhY2lvbiBxdWUgZGVzZ2xvc2EgZWwgcmVuZGltaWVudG8gZGVsIG1vZGVsbyBkZSBjbGFzaWZpY2FjacOzbgoKI21hdHJpeiBkZSBjb25mdXNpw7NuIGRlbCByZXN1bHRhZG8gZGUgZW50cmVuYW1pZW50bwptY3JlNCA8LSBjb25mdXNpb25NYXRyaXgocmVzdWx0YWRvX2VudHJlbmFtaWVudG80LCBlbnRyZW5hbWllbnRvJFNwZWNpZXMpCm1jcmU0CgojbWF0cml6IGRlIGNvbmZ1c2nDs24gZGVsIHJlc3VsdGFkbyBkZSBwcnVlYmEKbWNycDQgPC0gY29uZnVzaW9uTWF0cml4KHJlc3VsdGFkb19wcnVlYmE0LCBwcnVlYmEkU3BlY2llcykKbWNycDQKYGBgCgojIDxzcGFuIHN0eWxlID0iY29sb3I6Ymx1ZSI+TW9kZWxvIDUuIFJhbmRvbSBGb3Jlc3RzPC9zcGFuPgpgYGB7cn0KbW9kZWxvNSA8LSB0cmFpbihTcGVjaWVzIH4gLiwgCiAgICAgICAgICAgICAgICAgZGF0YSA9IGVudHJlbmFtaWVudG8sCiAgICAgICAgICAgICAgICAgbWV0aG9kID0gInJmIiwgI2NhbWJpYXIxCiAgICAgICAgICAgICAgICAgcHJlUHJvY2VzcyA9IGMoInNjYWxlIiwgImNlbnRlciIpLAogICAgICAgICAgICAgICAgIHRyQ29udHJvbCA9IHRyYWluQ29udHJvbChtZXRob2QgPSAiY3YiLCBudW1iZXIgPSAxMCksIAogICAgICAgICAgICAgICAgIHR1bmVHcmlkID0gZXhwYW5kLmdyaWQgKG10cnk9YygyLDQsNikpICNjYW1iaWFyCikKcmVzdWx0YWRvX2VudHJlbmFtaWVudG81IDwtIHByZWRpY3QobW9kZWxvNSwgZW50cmVuYW1pZW50bykKcmVzdWx0YWRvX3BydWViYTUgPC0gcHJlZGljdChtb2RlbG81LCBwcnVlYmEpCgojbWF0cml6IGRlIGNvbmZ1c2nDs24KI3RhYmxhIGRlIGV2YWx1YWNpb24gcXVlIGRlc2dsb3NhIGVsIHJlbmRpbWllbnRvIGRlbCBtb2RlbG8gZGUgY2xhc2lmaWNhY2nDs24KCiNtYXRyaXogZGUgY29uZnVzacOzbiBkZWwgcmVzdWx0YWRvIGRlIGVudHJlbmFtaWVudG8KbWNyZTUgPC0gY29uZnVzaW9uTWF0cml4KHJlc3VsdGFkb19lbnRyZW5hbWllbnRvNSwgZW50cmVuYW1pZW50byRTcGVjaWVzKQptY3JlNQoKI21hdHJpeiBkZSBjb25mdXNpw7NuIGRlbCByZXN1bHRhZG8gZGUgcHJ1ZWJhCm1jcnA1IDwtIGNvbmZ1c2lvbk1hdHJpeChyZXN1bHRhZG9fcHJ1ZWJhNSwgcHJ1ZWJhJFNwZWNpZXMpCm1jcnA1CmBgYAojIDxzcGFuIHN0eWxlID0iY29sb3I6Ymx1ZSI+TW9kZWxvIDYuIFJlZGVzIE5ldXJvbmFsZXM8L3NwYW4+CmBgYHtyfQptb2RlbG82IDwtIHRyYWluKFNwZWNpZXMgfiAuLCAKICAgICAgICAgICAgICAgICBkYXRhID0gZW50cmVuYW1pZW50bywKICAgICAgICAgICAgICAgICBtZXRob2QgPSAibm5ldCIsICNjYW1iaWFyMQogICAgICAgICAgICAgICAgIHByZVByb2Nlc3MgPSBjKCJzY2FsZSIsICJjZW50ZXIiKSwKICAgICAgICAgICAgICAgICB0ckNvbnRyb2wgPSB0cmFpbkNvbnRyb2wobWV0aG9kID0gImN2IiwgbnVtYmVyID0gMTApIAogICAgICAgICAgICAgICAgIAopCnJlc3VsdGFkb19lbnRyZW5hbWllbnRvNiA8LSBwcmVkaWN0KG1vZGVsbzYsIGVudHJlbmFtaWVudG8pCnJlc3VsdGFkb19wcnVlYmE2IDwtIHByZWRpY3QobW9kZWxvNiwgcHJ1ZWJhKQoKI21hdHJpeiBkZSBjb25mdXNpw7NuCiN0YWJsYSBkZSBldmFsdWFjaW9uIHF1ZSBkZXNnbG9zYSBlbCByZW5kaW1pZW50byBkZWwgbW9kZWxvIGRlIGNsYXNpZmljYWNpw7NuCgojbWF0cml6IGRlIGNvbmZ1c2nDs24gZGVsIHJlc3VsdGFkbyBkZSBlbnRyZW5hbWllbnRvCm1jcmU2IDwtIGNvbmZ1c2lvbk1hdHJpeChyZXN1bHRhZG9fZW50cmVuYW1pZW50bzYsIGVudHJlbmFtaWVudG8kU3BlY2llcykKbWNyZTYKCiNtYXRyaXogZGUgY29uZnVzacOzbiBkZWwgcmVzdWx0YWRvIGRlIHBydWViYQptY3JwNiA8LSBjb25mdXNpb25NYXRyaXgocmVzdWx0YWRvX3BydWViYTYsIHBydWViYSRTcGVjaWVzKQptY3JwNgpgYGAKIyA8c3BhbiBzdHlsZSA9ImNvbG9yOmJsdWUiPlRhYmxhIGRlIFJlc3VsdGFkb3M8L3NwYW4+CmBgYHtyfQpyZXN1bHRhZG9zIDwtIGRhdGEuZnJhbWUoCiAgInN2bUxpbmVhciIgPSBjKG1jcmUxJG92ZXJhbGxbIkFjY3VyYWN5Il0sIG1jcnAxJG92ZXJhbGxbIkFjY3VyYWN5Il0pLAogICJzdm1SYWRpYWwiID0gYyhtY3JlMiRvdmVyYWxsWyJBY2N1cmFjeSJdLCBtY3JwMiRvdmVyYWxsWyJBY2N1cmFjeSJdKSwKICAic3ZtUG9seSIgICA9IGMobWNyZTMkb3ZlcmFsbFsiQWNjdXJhY3kiXSwgbWNycDMkb3ZlcmFsbFsiQWNjdXJhY3kiXSksCiAgInJwYXJ0IiAgICAgPSBjKG1jcmU0JG92ZXJhbGxbIkFjY3VyYWN5Il0sIG1jcnA0JG92ZXJhbGxbIkFjY3VyYWN5Il0pLAogICJyZiIgICAgICAgID0gYyhtY3JlNSRvdmVyYWxsWyJBY2N1cmFjeSJdLCBtY3JwNSRvdmVyYWxsWyJBY2N1cmFjeSJdKSwKICAibm5ldCIgICAgICA9IGMobWNyZTYkb3ZlcmFsbFsiQWNjdXJhY3kiXSwgbWNycDYkb3ZlcmFsbFsiQWNjdXJhY3kiXSkKKQoKcm93bmFtZXMocmVzdWx0YWRvcykgPC0gYygiRXhhY3RpdHVkIGRlbCBFbnRyZW5hbWllbnRvIiwgIkV4YWN0aXR1ZCBkZSBsYSBQcnVlYmEiKQpyZXN1bHRhZG9zCmBgYAojIDxzcGFuIHN0eWxlID0iY29sb3I6Ymx1ZSI+VGFibGEgZGUgUmVzdWx0YWRvczwvc3Bhbj4KRW4gY29uY2x1c2nDs24gZWwgbW9kZWxvIGRlICoqUmVkZXMgTmV1cm9uYWxlcyogZXMgZWwgcmVjb21lbmRhZG8gcGFyYSBsYSBjbGFzaWZpY2FjacOzbiBkZSBsb3MgbGlyaW9zLgoK