Data 624 Homework 8 Chapter 7 Non-Linear Regression
Enid Roman
2024-11-09
7.2. Friedman (1991) introduced several benchmark data sets create by simulation. One of these simulations used the following nonlinear equation to create data:
y = 10 sin(πx1x2) + 20(x3 − 0.5)2 + 10x4 + 5x5 + N (0, σ2)
where the x values are random variables uniformly distributed between [0, 1 (there are also 5 other non-informative variables also created in the simulation). The package mlbench contains a function called mlbench.friedman1 that simulates these data:
Generate a data set of 200 observations using the mlbench.friedman1 function.
Tune several models on these data. For example:
## k-Nearest Neighbors
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ...
## Resampling results across tuning parameters:
##
## k RMSE Rsquared MAE
## 5 3.466085 0.5121775 2.816838
## 7 3.349428 0.5452823 2.727410
## 9 3.264276 0.5785990 2.660026
## 11 3.214216 0.6024244 2.603767
## 13 3.196510 0.6176570 2.591935
## 15 3.184173 0.6305506 2.577482
## 17 3.183130 0.6425367 2.567787
## 19 3.198752 0.6483184 2.592683
## 21 3.188993 0.6611428 2.588787
## 23 3.200458 0.6638353 2.604529
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 17.Train a k-Nearest Neighbors (kNN) model using the caret package with 10-fold cross-validation and tune the number of neighbors (k) using 10 different values. The pre-processing steps include centering and scaling the predictors. The kNN model is trained on the training data (trainingData) with the response variable ‘y’ and the predictors ‘x’. The model is tuned to find the optimal number of neighbors (k) based on the specified range of values. The final model is stored in the knnModel object.
## RMSE Rsquared MAE
## 3.1564486 0.6463324 2.5261843Which models appear to give the best performance? Does MARS select the informative predictors (those named X1–X5)?
The k-Nearest Neighbors (kNN) model appears to give the best performance based on the performance metrics (e.g., RMSE, R-squared) calculated on the test data. The performance metrics for the kNN model are as follows:
RMSE: 3.1564486 R-squared: 0.6463324 MAE: 2.5261843
The kNN model has the lowest RMSE and the highest R-squared value compared to other models, indicating that it has the best performance in terms of predictive accuracy.
As for the MARS model, it does not select the informative predictors (X1–X5) as the model is a non-linear regression model that uses a series of piecewise linear functions to model the relationship between the predictors and the response. The MARS model may identify interactions between the predictors and the response, but it does not explicitly select the informative predictors based on their names. Therefore, the MARS model may not directly identify the informative predictors X1–X5 in the data.
7.5. Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.
Data Imputation
## 'data.frame': 176 obs. of 58 variables:
## $ Yield : num 38 42.4 42 41.4 42.5 ...
## $ BiologicalMaterial01 : num 6.25 8.01 8.01 8.01 7.47 6.12 7.48 6.94 6.94 6.94 ...
## $ BiologicalMaterial02 : num 49.6 61 61 61 63.3 ...
## $ BiologicalMaterial03 : num 57 67.5 67.5 67.5 72.2 ...
## $ BiologicalMaterial04 : num 12.7 14.6 14.6 14.6 14 ...
## $ BiologicalMaterial05 : num 19.5 19.4 19.4 19.4 17.9 ...
## $ BiologicalMaterial06 : num 43.7 53.1 53.1 53.1 54.7 ...
## $ BiologicalMaterial07 : num 100 100 100 100 100 100 100 100 100 100 ...
## $ BiologicalMaterial08 : num 16.7 19 19 19 18.2 ...
## $ BiologicalMaterial09 : num 11.4 12.6 12.6 12.6 12.8 ...
## $ BiologicalMaterial10 : num 3.46 3.46 3.46 3.46 3.05 3.78 3.04 3.85 3.85 3.85 ...
## $ BiologicalMaterial11 : num 138 154 154 154 148 ...
## $ BiologicalMaterial12 : num 18.8 21.1 21.1 21.1 21.1 ...
## $ ManufacturingProcess01: num NA 0 0 0 10.7 12 11.5 12 12 12 ...
## $ ManufacturingProcess02: num NA 0 0 0 0 0 0 0 0 0 ...
## $ ManufacturingProcess03: num NA NA NA NA NA NA 1.56 1.55 1.56 1.55 ...
## $ ManufacturingProcess04: num NA 917 912 911 918 924 933 929 928 938 ...
## $ ManufacturingProcess05: num NA 1032 1004 1015 1028 ...
## $ ManufacturingProcess06: num NA 210 207 213 206 ...
## $ ManufacturingProcess07: num NA 177 178 177 178 178 177 178 177 177 ...
## $ ManufacturingProcess08: num NA 178 178 177 178 178 178 178 177 177 ...
## $ ManufacturingProcess09: num 43 46.6 45.1 44.9 45 ...
## $ ManufacturingProcess10: num NA NA NA NA NA NA 11.6 10.2 9.7 10.1 ...
## $ ManufacturingProcess11: num NA NA NA NA NA NA 11.5 11.3 11.1 10.2 ...
## $ ManufacturingProcess12: num NA 0 0 0 0 0 0 0 0 0 ...
## $ ManufacturingProcess13: num 35.5 34 34.8 34.8 34.6 34 32.4 33.6 33.9 34.3 ...
## $ ManufacturingProcess14: num 4898 4869 4878 4897 4992 ...
## $ ManufacturingProcess15: num 6108 6095 6087 6102 6233 ...
## $ ManufacturingProcess16: num 4682 4617 4617 4635 4733 ...
## $ ManufacturingProcess17: num 35.5 34 34.8 34.8 33.9 33.4 33.8 33.6 33.9 35.3 ...
## $ ManufacturingProcess18: num 4865 4867 4877 4872 4886 ...
## $ ManufacturingProcess19: num 6049 6097 6078 6073 6102 ...
## $ ManufacturingProcess20: num 4665 4621 4621 4611 4659 ...
## $ ManufacturingProcess21: num 0 0 0 0 -0.7 -0.6 1.4 0 0 1 ...
## $ ManufacturingProcess22: num NA 3 4 5 8 9 1 2 3 4 ...
## $ ManufacturingProcess23: num NA 0 1 2 4 1 1 2 3 1 ...
## $ ManufacturingProcess24: num NA 3 4 5 18 1 1 2 3 4 ...
## $ ManufacturingProcess25: num 4873 4869 4897 4892 4930 ...
## $ ManufacturingProcess26: num 6074 6107 6116 6111 6151 ...
## $ ManufacturingProcess27: num 4685 4630 4637 4630 4684 ...
## $ ManufacturingProcess28: num 10.7 11.2 11.1 11.1 11.3 11.4 11.2 11.1 11.3 11.4 ...
## $ ManufacturingProcess29: num 21 21.4 21.3 21.3 21.6 21.7 21.2 21.2 21.5 21.7 ...
## $ ManufacturingProcess30: num 9.9 9.9 9.4 9.4 9 10.1 11.2 10.9 10.5 9.8 ...
## $ ManufacturingProcess31: num 69.1 68.7 69.3 69.3 69.4 68.2 67.6 67.9 68 68.5 ...
## $ ManufacturingProcess32: num 156 169 173 171 171 173 159 161 160 164 ...
## $ ManufacturingProcess33: num 66 66 66 68 70 70 65 65 65 66 ...
## $ ManufacturingProcess34: num 2.4 2.6 2.6 2.5 2.5 2.5 2.5 2.5 2.5 2.5 ...
## $ ManufacturingProcess35: num 486 508 509 496 468 490 475 478 491 488 ...
## $ ManufacturingProcess36: num 0.019 0.019 0.018 0.018 0.017 0.018 0.019 0.019 0.019 0.019 ...
## $ ManufacturingProcess37: num 0.5 2 0.7 1.2 0.2 0.4 0.8 1 1.2 1.8 ...
## $ ManufacturingProcess38: num 3 2 2 2 2 2 2 2 3 3 ...
## $ ManufacturingProcess39: num 7.2 7.2 7.2 7.2 7.3 7.2 7.3 7.3 7.4 7.1 ...
## $ ManufacturingProcess40: num NA 0.1 0 0 0 0 0 0 0 0 ...
## $ ManufacturingProcess41: num NA 0.15 0 0 0 0 0 0 0 0 ...
## $ ManufacturingProcess42: num 11.6 11.1 12 10.6 11 11.5 11.7 11.4 11.4 11.3 ...
## $ ManufacturingProcess43: num 3 0.9 1 1.1 1.1 2.2 0.7 0.8 0.9 0.8 ...
## $ ManufacturingProcess44: num 1.8 1.9 1.8 1.8 1.7 1.8 2 2 1.9 1.9 ...
## $ ManufacturingProcess45: num 2.4 2.2 2.3 2.1 2.1 2 2.2 2.2 2.1 2.4 ...## [1] 106## [1] 0Split the data into training and test sets
## [1] 124 58## [1] 52 58Pre-process the data
## 'data.frame': 124 obs. of 58 variables:
## $ Yield : num -1.217 1.041 0.699 1.299 1.904 ...
## $ BiologicalMaterial01 : num -0.224 2.325 2.325 1.543 -0.412 ...
## $ BiologicalMaterial02 : num -1.567 1.281 1.281 1.872 0.629 ...
## $ BiologicalMaterial03 : num -2.6543 -0.0861 -0.0861 1.0795 -0.6164 ...
## $ BiologicalMaterial04 : num 0.234 1.333 1.333 0.971 1.633 ...
## $ BiologicalMaterial05 : num 0.496 0.413 0.413 -0.385 1.75 ...
## $ BiologicalMaterial06 : num -1.407 1.065 1.065 1.464 0.563 ...
## $ BiologicalMaterial07 : num -0.0898 -0.0898 -0.0898 -0.0898 -0.0898 ...
## $ BiologicalMaterial08 : num -1.24 2.36 2.36 1.12 1.24 ...
## $ BiologicalMaterial09 : num -3.308 -0.7 -0.7 -0.113 -1.687 ...
## $ BiologicalMaterial10 : num 1.102 1.102 1.102 0.428 1.627 ...
## $ BiologicalMaterial11 : num -1.817 1.366 1.366 0.128 1 ...
## $ BiologicalMaterial12 : num -1.73 1.04 1.04 1.04 0.68 ...
## $ ManufacturingProcess01: num 0.18 -6.353 -6.353 -0.327 0.405 ...
## $ ManufacturingProcess02: num 0.567 -1.925 -1.925 -1.925 -1.925 ...
## $ ManufacturingProcess03: num 0.2563 -0.0513 0.3588 -0.0513 0.4613 ...
## $ ManufacturingProcess04: num 0.525 -3.224 -3.384 -2.263 -1.302 ...
## $ ManufacturingProcess05: num -0.484 0.129 0.564 1.074 0.651 ...
## $ ManufacturingProcess06: num -0.661 -0.147 2.602 -0.767 0.652 ...
## $ ManufacturingProcess07: num -0.19 1.01 -0.99 1.01 1.01 ...
## $ ManufacturingProcess08: num -0.272 0.931 -1.074 0.931 0.931 ...
## $ ManufacturingProcess09: num -1.837 -0.416 -0.519 -0.492 -0.245 ...
## $ ManufacturingProcess10: num -5.34e-02 3.47e-01 1.17e-16 -3.74e-01 3.20e-01 ...
## $ ManufacturingProcess11: num -0.1365 0.5246 0.7833 0.0647 1.4157 ...
## $ ManufacturingProcess12: num -0.463 -0.463 -0.463 -0.463 -0.463 ...
## $ ManufacturingProcess13: num 0.9568 0.2876 0.2876 0.0964 -0.4773 ...
## $ ManufacturingProcess14: num 0.728 0.367 0.71 2.426 2.3 ...
## $ ManufacturingProcess15: num 1.082 0.728 0.981 3.19 3.005 ...
## $ ManufacturingProcess16: num 0.294 0.138 0.181 0.416 0.543 ...
## $ ManufacturingProcess17: num 0.979 0.418 0.418 -0.304 -0.705 ...
## $ ManufacturingProcess18: num 0.154 0.182 0.17 0.202 0.147 ...
## $ ManufacturingProcess19: num 0.444 1.086 0.975 1.617 1.905 ...
## $ ManufacturingProcess20: num 0.285 0.179 0.155 0.27 0.36 ...
## $ ManufacturingProcess21: num 0.289 0.289 0.289 -0.63 -0.498 ...
## $ ManufacturingProcess22: num 0.0213 -0.4597 -0.1591 0.7429 1.0436 ...
## $ ManufacturingProcess23: num 0.927 -1.207 -0.579 0.676 -1.207 ...
## $ ManufacturingProcess24: num 0.921 -0.761 -0.593 1.593 -1.265 ...
## $ ManufacturingProcess25: num 0.128 0.183 0.172 0.259 0.124 ...
## $ ManufacturingProcess26: num 0.133 0.21 0.201 0.274 0.232 ...
## $ ManufacturingProcess27: num 0.318 0.203 0.186 0.316 0.323 ...
## $ ManufacturingProcess28: num 0.876 0.951 0.951 0.989 1.007 ...
## $ ManufacturingProcess29: num 0.541 0.698 0.698 0.854 0.906 ...
## $ ManufacturingProcess30: num 0.713 0.235 0.235 -0.147 0.905 ...
## $ ManufacturingProcess31: num -0.1348 -0.1039 -0.1039 -0.0885 -0.2735 ...
## $ ManufacturingProcess32: num -0.495 2.738 2.358 2.358 2.738 ...
## $ ManufacturingProcess33: num 0.952 0.952 1.747 2.542 2.542 ...
## $ ManufacturingProcess34: num -1.764 2.047 0.141 0.141 0.141 ...
## $ ManufacturingProcess35: num -0.88 1.2743 0.0567 -2.566 -0.5053 ...
## $ ManufacturingProcess36: num -0.652 -1.849 -1.849 -3.046 -1.849 ...
## $ ManufacturingProcess37: num -1.11 -0.679 0.398 -1.757 -1.326 ...
## $ ManufacturingProcess38: num 0.676 -0.745 -0.745 -0.745 -0.745 ...
## $ ManufacturingProcess39: num 0.273 0.273 0.273 0.33 0.273 ...
## $ ManufacturingProcess40: num 0.0547 -0.4672 -0.4672 -0.4672 -0.4672 ...
## $ ManufacturingProcess41: num -0.0654 -0.4343 -0.4343 -0.4343 -0.4343 ...
## $ ManufacturingProcess42: num 0.237 0.4114 -0.199 -0.0246 0.1934 ...
## $ ManufacturingProcess43: num 2.066 0.087 0.186 0.186 1.275 ...
## $ ManufacturingProcess44: num 0.0277 0.0277 0.0277 -0.2369 0.0277 ...
## $ ManufacturingProcess45: num 0.61 0.396 -0.031 -0.031 -0.245 ...## 'data.frame': 52 obs. of 58 variables:
## $ Yield : num 1.271 1.652 1.618 0.402 -0.488 ...
## $ BiologicalMaterial01 : num 2.325 1.558 0.775 2.094 1.601 ...
## $ BiologicalMaterial02 : num 1.281 2.157 1.939 0.594 0.974 ...
## $ BiologicalMaterial03 : num -0.0861 1.1186 1.0331 -0.2156 -0.135 ...
## $ BiologicalMaterial04 : num 1.333 0.856 1.938 1.754 1.949 ...
## $ BiologicalMaterial05 : num 0.413 -0.495 0.446 1.239 1.756 ...
## $ BiologicalMaterial06 : num 1.065 1.409 1.48 0.461 0.844 ...
## $ BiologicalMaterial07 : num -0.0898 -0.0898 -0.0898 -0.0898 -0.0898 ...
## $ BiologicalMaterial08 : num 2.36 1.88 2.07 1.56 2.01 ...
## $ BiologicalMaterial09 : num -0.7 0.239 0.662 -0.348 0.31 ...
## $ BiologicalMaterial10 : num 1.102 0.412 1.742 2.005 2.383 ...
## $ BiologicalMaterial11 : num 1.3657 0.0257 1.474 1.1921 2.3237 ...
## $ BiologicalMaterial12 : num 1.041 0.667 1.54 0.605 2.002 ...
## $ ManufacturingProcess01: num -6.353 0.124 0.405 -0.102 0.799 ...
## $ ManufacturingProcess02: num -1.92 -1.92 -1.92 -1.92 -1.92 ...
## $ ManufacturingProcess03: num 0.0513 0.8714 0.3588 0.8714 0.7689 ...
## $ ManufacturingProcess04: num -2.423 0.14 -0.501 -0.661 -0.501 ...
## $ ManufacturingProcess05: num 1.259 -0.451 0.418 1.074 3.63 ...
## $ ManufacturingProcess06: num 1.1393 1.1393 1.893 -0.6786 0.0308 ...
## $ ManufacturingProcess07: num -0.99 -0.99 1.01 -0.99 1.01 ...
## $ ManufacturingProcess08: num 0.931 0.931 0.931 -1.074 0.931 ...
## $ ManufacturingProcess09: num 0.6129 2.5274 2.0608 -0.0939 -0.8349 ...
## $ ManufacturingProcess10: num 0.561 3.257 1.388 -0.214 -0.347 ...
## $ ManufacturingProcess11: num 1.071 2.997 2.709 0.841 0.553 ...
## $ ManufacturingProcess12: num -0.463 -0.463 -0.463 -0.463 -0.463 ...
## $ ManufacturingProcess13: num -0.477 -2.007 -0.86 1.244 0.957 ...
## $ ManufacturingProcess14: num 0.2041 -2.0359 -0.0668 1.5229 1.7939 ...
## $ ManufacturingProcess15: num 0.863 -0.757 1.031 2.246 2.313 ...
## $ ManufacturingProcess16: num 0.138 -0.176 0.159 0.397 0.459 ...
## $ ManufacturingProcess17: num -0.224 -0.384 -0.545 1.22 0.979 ...
## $ ManufacturingProcess18: num 0.1587 -0.0908 -0.0725 0.1381 0.2045 ...
## $ ManufacturingProcess19: num 1.507 -0.352 -0.153 1.573 2.104 ...
## $ ManufacturingProcess20: num 0.1788 -0.06 0.0124 0.2632 0.3934 ...
## $ ManufacturingProcess21: num 0.289 2.126 0.289 0.289 0.289 ...
## $ ManufacturingProcess22: num -0.76 -1.36 -1.06 -1.36 -0.76 ...
## $ ManufacturingProcess23: num -1.834 -1.207 -0.579 -1.207 -0.579 ...
## $ ManufacturingProcess24: num -0.929 -1.265 -1.097 -1.265 -0.929 ...
## $ ManufacturingProcess25: num 0.1193 -0.0498 -0.0246 0.0987 0.1353 ...
## $ ManufacturingProcess26: num 0.193 0.102 0.105 0.18 0.197 ...
## $ ManufacturingProcess27: num 0.1858 0.046 0.0797 0.2558 0.345 ...
## $ ManufacturingProcess28: num 0.97 0.97 0.951 0.951 0.951 ...
## $ ManufacturingProcess29: num 0.75 0.645 0.645 0.698 0.75 ...
## $ ManufacturingProcess30: num 0.713 1.957 1.67 0.809 0.618 ...
## $ ManufacturingProcess31: num -0.196 -0.366 -0.32 -0.196 -0.181 ...
## $ ManufacturingProcess32: num 1.9773 0.0752 0.4556 1.5969 -0.4955 ...
## $ ManufacturingProcess33: num 0.952 0.554 0.554 1.747 0.156 ...
## $ ManufacturingProcess34: num 2.047 0.141 0.141 0.141 -1.764 ...
## $ ManufacturingProcess35: num 1.181 -1.91 -1.629 -0.037 1.743 ...
## $ ManufacturingProcess36: num -0.652 -0.652 -0.652 -0.652 1.741 ...
## $ ManufacturingProcess37: num 2.121 -0.464 -0.033 0.613 -0.679 ...
## $ ManufacturingProcess38: num -0.745 -0.745 -0.745 0.676 0.676 ...
## $ ManufacturingProcess39: num 0.273 0.33 0.33 0.386 0.33 ...
## $ ManufacturingProcess40: num 2.142 -0.467 -0.467 -0.467 -0.467 ...
## $ ManufacturingProcess41: num 2.332 -0.434 -0.434 -0.434 -0.434 ...
## $ ManufacturingProcess42: num 0.019 0.281 0.15 0.193 0.237 ...
## $ ManufacturingProcess43: num -0.012 -0.21 -0.111 0.186 0.879 ...
## $ ManufacturingProcess44: num 0.292 0.557 0.557 0.292 -0.237 ...
## $ ManufacturingProcess45: num 0.183 0.183 0.183 0.396 0.396 ...I will start with Linear Regression Model and then compare it with Nonlinear Regression Models
## RMSE Rsquared MAE
## 0.8285972 0.5095169 0.6407973Nonlinear Regression Models
Random Forest Model
## RMSE Rsquared MAE
## 0.6867001 0.6820857 0.5268914Support Vector Machine (SVM) Model
## RMSE Rsquared MAE
## 0.7642341 0.5855119 0.5910040K-Nearest Neighbors (kNN) Model
## RMSE Rsquared MAE
## 0.8462222 0.4379732 0.7106254Nueral Network Model
## # weights: 60
## initial value 158.409217
## final value 138.010105
## converged
## # weights: 178
## initial value 162.657532
## final value 138.010105
## converged
## # weights: 296
## initial value 191.618501
## final value 138.010105
## converged
## # weights: 60
## initial value 186.915115
## iter 10 value 137.524487
## iter 20 value 97.266784
## iter 30 value 92.747332
## iter 40 value 91.607796
## iter 50 value 91.486835
## iter 60 value 90.730225
## iter 70 value 90.567804
## iter 80 value 90.519484
## final value 90.519302
## converged
## # weights: 178
## initial value 176.114373
## iter 10 value 104.640026
## iter 20 value 90.514797
## iter 30 value 88.620311
## iter 40 value 87.744114
## iter 50 value 87.683245
## iter 60 value 87.680905
## iter 70 value 87.680309
## iter 80 value 87.678479
## iter 90 value 87.628359
## iter 100 value 87.444198
## final value 87.444198
## stopped after 100 iterations
## # weights: 296
## initial value 189.483654
## iter 10 value 104.716186
## iter 20 value 91.277749
## iter 30 value 88.841857
## iter 40 value 87.611289
## iter 50 value 87.394278
## iter 60 value 87.100503
## iter 70 value 86.190594
## iter 80 value 85.902408
## iter 90 value 85.713085
## iter 100 value 85.568056
## final value 85.568056
## stopped after 100 iterations
## # weights: 60
## initial value 160.269187
## iter 10 value 138.011602
## iter 20 value 138.010624
## iter 30 value 138.009455
## iter 40 value 138.008026
## iter 50 value 138.006236
## iter 60 value 138.003914
## iter 70 value 138.000769
## iter 80 value 137.996240
## iter 90 value 137.989109
## iter 100 value 137.976114
## final value 137.976114
## stopped after 100 iterations
## # weights: 178
## initial value 186.709623
## iter 10 value 138.023010
## iter 20 value 138.007882
## iter 30 value 89.379531
## iter 40 value 82.118709
## iter 50 value 81.647256
## iter 60 value 80.532407
## iter 70 value 80.430132
## iter 80 value 80.362522
## iter 90 value 80.316221
## iter 100 value 80.023866
## final value 80.023866
## stopped after 100 iterations
## # weights: 296
## initial value 187.207704
## iter 10 value 138.027654
## iter 20 value 134.947281
## iter 30 value 81.222016
## iter 40 value 80.470900
## iter 50 value 80.313861
## iter 60 value 80.274329
## iter 70 value 80.270310
## iter 80 value 80.250275
## iter 90 value 80.213986
## iter 100 value 80.189836
## final value 80.189836
## stopped after 100 iterations
## # weights: 60
## initial value 101.307121
## final value 92.036094
## converged
## # weights: 178
## initial value 124.865488
## final value 92.036104
## converged
## # weights: 296
## initial value 139.515740
## final value 92.036104
## converged
## # weights: 60
## initial value 109.119678
## iter 10 value 80.534650
## iter 20 value 67.039634
## iter 30 value 60.298857
## iter 40 value 57.900940
## iter 50 value 57.663056
## iter 60 value 57.621939
## final value 57.621779
## converged
## # weights: 178
## initial value 105.893055
## iter 10 value 63.155531
## iter 20 value 57.256234
## iter 30 value 55.599448
## iter 40 value 55.445834
## iter 50 value 55.407712
## iter 60 value 55.387912
## iter 70 value 55.385010
## iter 80 value 55.381640
## iter 90 value 55.381568
## final value 55.381563
## converged
## # weights: 296
## initial value 131.507907
## iter 10 value 68.606231
## iter 20 value 59.213952
## iter 30 value 55.703478
## iter 40 value 54.639093
## iter 50 value 54.418161
## iter 60 value 54.403564
## iter 70 value 54.401825
## iter 80 value 54.401700
## iter 90 value 54.401688
## final value 54.401686
## converged
## # weights: 60
## initial value 138.474675
## iter 10 value 92.050679
## iter 20 value 92.047655
## iter 30 value 92.047572
## iter 40 value 92.047487
## iter 50 value 92.047400
## iter 60 value 92.047311
## iter 70 value 92.047220
## iter 80 value 92.047127
## iter 90 value 92.047033
## iter 100 value 92.046938
## final value 92.046938
## stopped after 100 iterations
## # weights: 178
## initial value 120.206681
## iter 10 value 92.043810
## iter 20 value 92.000145
## iter 30 value 58.143055
## iter 40 value 51.932807
## iter 50 value 51.702084
## iter 60 value 51.630965
## iter 70 value 50.756677
## iter 80 value 49.820559
## iter 90 value 49.381560
## iter 100 value 49.262591
## final value 49.262591
## stopped after 100 iterations
## # weights: 296
## initial value 147.156743
## iter 10 value 92.051166
## iter 20 value 91.974336
## iter 30 value 60.266153
## iter 40 value 54.820069
## iter 50 value 54.579474
## iter 60 value 54.549254
## iter 70 value 54.444602
## iter 80 value 53.383752
## iter 90 value 53.358021
## iter 100 value 53.295363
## final value 53.295363
## stopped after 100 iterations
## # weights: 60
## initial value 117.190271
## final value 110.782090
## converged
## # weights: 178
## initial value 113.079356
## final value 110.782033
## converged
## # weights: 296
## initial value 149.893435
## final value 110.782090
## converged
## # weights: 60
## initial value 141.003780
## iter 10 value 78.572536
## iter 20 value 60.236589
## iter 30 value 58.910427
## iter 40 value 58.812757
## iter 50 value 58.208768
## iter 60 value 58.136179
## iter 70 value 58.134301
## iter 70 value 58.134301
## iter 70 value 58.134301
## final value 58.134301
## converged
## # weights: 178
## initial value 138.333129
## iter 10 value 79.541039
## iter 20 value 61.189426
## iter 30 value 56.688572
## iter 40 value 56.257648
## iter 50 value 55.847008
## iter 60 value 55.838369
## final value 55.838318
## converged
## # weights: 296
## initial value 134.838236
## iter 10 value 76.420657
## iter 20 value 59.057489
## iter 30 value 55.568802
## iter 40 value 54.362658
## iter 50 value 54.155544
## iter 60 value 54.135473
## iter 70 value 54.133407
## iter 80 value 54.133272
## final value 54.133268
## converged
## # weights: 60
## initial value 135.720714
## iter 10 value 110.789892
## iter 20 value 110.789489
## iter 30 value 110.789048
## iter 40 value 110.788563
## iter 50 value 110.788024
## iter 60 value 110.787422
## iter 70 value 110.786741
## iter 80 value 110.785961
## iter 90 value 110.785058
## iter 100 value 110.783995
## final value 110.783995
## stopped after 100 iterations
## # weights: 178
## initial value 123.378233
## iter 10 value 110.765688
## iter 20 value 110.709914
## iter 30 value 110.123752
## iter 40 value 54.972985
## iter 50 value 51.836905
## iter 60 value 51.683795
## iter 70 value 51.515530
## iter 80 value 50.391524
## iter 90 value 50.364623
## iter 100 value 50.345001
## final value 50.345001
## stopped after 100 iterations
## # weights: 296
## initial value 141.732263
## iter 10 value 110.793343
## iter 20 value 110.778974
## iter 30 value 110.660371
## iter 40 value 86.334223
## iter 50 value 79.180988
## iter 60 value 74.266659
## iter 70 value 73.752894
## iter 80 value 66.339096
## iter 90 value 66.199043
## iter 100 value 65.804161
## final value 65.804161
## stopped after 100 iterations
## # weights: 60
## initial value 148.113088
## final value 111.535550
## converged
## # weights: 178
## initial value 145.117085
## final value 111.535550
## converged
## # weights: 296
## initial value 143.304121
## final value 111.535550
## converged
## # weights: 60
## initial value 156.555144
## iter 10 value 84.514917
## iter 20 value 71.569678
## iter 30 value 67.748206
## iter 40 value 67.227625
## iter 50 value 65.939477
## iter 60 value 65.859060
## iter 70 value 65.855550
## final value 65.855547
## converged
## # weights: 178
## initial value 136.224260
## iter 10 value 80.261554
## iter 20 value 66.092943
## iter 30 value 62.951071
## iter 40 value 62.496674
## iter 50 value 62.353414
## iter 60 value 62.327894
## iter 70 value 62.326304
## iter 80 value 62.324749
## iter 90 value 62.324532
## final value 62.324522
## converged
## # weights: 296
## initial value 146.680123
## iter 10 value 83.112685
## iter 20 value 65.941609
## iter 30 value 63.912779
## iter 40 value 63.373760
## iter 50 value 63.258548
## iter 60 value 63.227053
## iter 70 value 63.223925
## iter 80 value 63.219998
## iter 90 value 63.219416
## iter 100 value 63.219165
## final value 63.219165
## stopped after 100 iterations
## # weights: 60
## initial value 124.833859
## iter 10 value 111.555435
## iter 20 value 111.541104
## iter 30 value 111.540732
## iter 40 value 111.540332
## iter 50 value 111.539903
## iter 60 value 111.539442
## iter 70 value 111.538945
## iter 80 value 111.538409
## iter 90 value 111.537829
## iter 100 value 111.537198
## final value 111.537198
## stopped after 100 iterations
## # weights: 178
## initial value 161.920614
## iter 10 value 111.553035
## iter 20 value 95.602727
## iter 30 value 59.241038
## iter 40 value 57.511543
## iter 50 value 57.164501
## iter 60 value 56.286448
## iter 70 value 56.276988
## iter 80 value 56.272791
## iter 90 value 56.210819
## iter 100 value 56.128738
## final value 56.128738
## stopped after 100 iterations
## # weights: 296
## initial value 127.151680
## iter 10 value 111.548190
## iter 20 value 111.542780
## iter 30 value 111.542326
## iter 40 value 111.541751
## iter 50 value 111.541013
## iter 60 value 111.540032
## iter 70 value 111.538624
## iter 80 value 111.536341
## iter 90 value 111.531779
## iter 100 value 111.518732
## final value 111.518732
## stopped after 100 iterations
## # weights: 60
## initial value 181.776571
## final value 145.337517
## converged
## # weights: 178
## initial value 181.869205
## final value 145.337517
## converged
## # weights: 296
## initial value 143.951731
## iter 10 value 83.319117
## iter 20 value 82.231265
## iter 30 value 81.884397
## iter 40 value 81.841650
## iter 50 value 81.741808
## iter 60 value 81.589500
## iter 70 value 81.573077
## iter 80 value 81.561944
## iter 90 value 81.559009
## iter 100 value 81.558159
## final value 81.558159
## stopped after 100 iterations
## # weights: 60
## initial value 155.853743
## iter 10 value 97.055107
## iter 20 value 92.477952
## iter 30 value 91.461080
## iter 40 value 90.965453
## iter 50 value 90.955164
## iter 60 value 90.935642
## final value 90.932132
## converged
## # weights: 178
## initial value 166.778734
## iter 10 value 96.910061
## iter 20 value 88.587100
## iter 30 value 85.988713
## iter 40 value 85.057806
## iter 50 value 84.804151
## iter 60 value 84.728888
## iter 70 value 84.725088
## iter 80 value 84.724202
## iter 90 value 84.723368
## iter 100 value 84.723169
## final value 84.723169
## stopped after 100 iterations
## # weights: 296
## initial value 198.690489
## iter 10 value 104.259596
## iter 20 value 89.866836
## iter 30 value 87.064758
## iter 40 value 86.091774
## iter 50 value 85.926309
## iter 60 value 85.865185
## iter 70 value 85.833350
## iter 80 value 85.814931
## iter 90 value 85.813828
## iter 100 value 85.812805
## final value 85.812805
## stopped after 100 iterations
## # weights: 60
## initial value 150.614189
## iter 10 value 145.015189
## iter 20 value 89.209391
## iter 30 value 83.175383
## iter 40 value 82.977342
## iter 50 value 82.973476
## iter 60 value 82.970022
## iter 70 value 82.858768
## iter 80 value 82.846047
## iter 90 value 82.841032
## iter 100 value 82.838131
## final value 82.838131
## stopped after 100 iterations
## # weights: 178
## initial value 170.768161
## iter 10 value 145.351392
## iter 20 value 145.349933
## iter 30 value 145.348123
## iter 40 value 145.345799
## iter 50 value 145.342715
## iter 60 value 145.338469
## iter 70 value 145.332191
## iter 80 value 145.321688
## iter 90 value 145.299853
## iter 100 value 145.224687
## final value 145.224687
## stopped after 100 iterations
## # weights: 296
## initial value 166.363171
## iter 10 value 145.355437
## iter 20 value 145.343050
## iter 30 value 92.044827
## iter 40 value 80.653268
## iter 50 value 80.301636
## iter 60 value 79.703345
## iter 70 value 79.609287
## iter 80 value 79.510267
## iter 90 value 79.484632
## iter 100 value 79.469318
## final value 79.469318
## stopped after 100 iterations
## # weights: 60
## initial value 174.773008
## final value 143.064074
## converged
## # weights: 178
## initial value 176.681649
## final value 143.064074
## converged
## # weights: 296
## initial value 195.881704
## final value 143.064074
## converged
## # weights: 60
## initial value 181.294881
## iter 10 value 108.027898
## iter 20 value 98.707361
## iter 30 value 95.310955
## iter 40 value 94.334734
## iter 50 value 94.204835
## iter 60 value 94.201133
## final value 94.201130
## converged
## # weights: 178
## initial value 179.174043
## iter 10 value 99.846334
## iter 20 value 92.206549
## iter 30 value 91.774180
## iter 40 value 91.671668
## iter 50 value 91.661835
## iter 60 value 91.661385
## iter 70 value 91.661172
## final value 91.661152
## converged
## # weights: 296
## initial value 175.732185
## iter 10 value 110.341082
## iter 20 value 92.481877
## iter 30 value 90.713957
## iter 40 value 90.480468
## iter 50 value 90.425496
## iter 60 value 90.383544
## iter 70 value 90.381714
## iter 80 value 90.381636
## iter 90 value 90.379520
## iter 100 value 90.378988
## final value 90.378988
## stopped after 100 iterations
## # weights: 60
## initial value 166.990597
## iter 10 value 143.067314
## iter 20 value 143.066285
## iter 30 value 143.065045
## iter 40 value 143.063518
## iter 50 value 143.061585
## iter 60 value 143.059050
## iter 70 value 143.055571
## iter 80 value 143.050478
## iter 90 value 143.042274
## iter 100 value 143.026757
## final value 143.026757
## stopped after 100 iterations
## # weights: 178
## initial value 168.769996
## iter 10 value 143.078181
## iter 20 value 143.077831
## iter 30 value 143.077455
## iter 40 value 143.077049
## iter 50 value 143.076608
## iter 60 value 143.076124
## iter 70 value 143.075590
## iter 80 value 143.074994
## iter 90 value 143.074323
## iter 100 value 143.073556
## final value 143.073556
## stopped after 100 iterations
## # weights: 296
## initial value 168.936341
## iter 10 value 143.073110
## iter 20 value 97.142341
## iter 30 value 90.965991
## iter 40 value 89.933203
## iter 50 value 88.910535
## iter 60 value 88.846388
## iter 70 value 88.812875
## iter 80 value 88.803529
## iter 90 value 88.495830
## iter 100 value 88.256801
## final value 88.256801
## stopped after 100 iterations
## # weights: 60
## initial value 138.260947
## final value 131.801757
## converged
## # weights: 178
## initial value 141.604088
## final value 131.801815
## converged
## # weights: 296
## initial value 175.752055
## final value 131.801815
## converged
## # weights: 60
## initial value 149.172494
## iter 10 value 94.729193
## iter 20 value 84.430060
## iter 30 value 80.327851
## iter 40 value 77.809102
## iter 50 value 77.529677
## iter 60 value 77.504886
## final value 77.504847
## converged
## # weights: 178
## initial value 137.266172
## iter 10 value 83.241424
## iter 20 value 76.538862
## iter 30 value 74.118106
## iter 40 value 73.808411
## iter 50 value 73.464952
## iter 60 value 73.419749
## iter 70 value 73.411253
## iter 80 value 73.389536
## iter 90 value 73.387761
## final value 73.387692
## converged
## # weights: 296
## initial value 168.672312
## iter 10 value 94.201811
## iter 20 value 79.946866
## iter 30 value 75.761222
## iter 40 value 74.300040
## iter 50 value 73.167837
## iter 60 value 73.026479
## iter 70 value 72.942206
## iter 80 value 72.930807
## iter 90 value 72.930130
## iter 100 value 72.929665
## final value 72.929665
## stopped after 100 iterations
## # weights: 60
## initial value 175.930692
## iter 10 value 131.821657
## iter 20 value 131.814654
## iter 30 value 131.814178
## iter 40 value 131.813679
## iter 50 value 131.813146
## iter 60 value 131.812562
## iter 70 value 131.811903
## iter 80 value 131.811129
## iter 90 value 131.810189
## iter 100 value 131.809042
## final value 131.809042
## stopped after 100 iterations
## # weights: 178
## initial value 142.340054
## iter 10 value 131.820464
## iter 20 value 131.818558
## iter 30 value 131.816050
## iter 40 value 131.813384
## iter 50 value 131.810397
## iter 60 value 131.806628
## iter 70 value 131.801299
## iter 80 value 131.792798
## iter 90 value 131.776667
## iter 100 value 131.733322
## final value 131.733322
## stopped after 100 iterations
## # weights: 296
## initial value 146.118608
## iter 10 value 131.817873
## iter 20 value 131.809262
## iter 30 value 131.802764
## iter 40 value 131.794284
## iter 50 value 131.778924
## iter 60 value 131.718897
## iter 70 value 80.438595
## iter 80 value 72.227322
## iter 90 value 71.678628
## iter 100 value 69.925565
## final value 69.925565
## stopped after 100 iterations
## # weights: 60
## initial value 184.009125
## final value 133.263981
## converged
## # weights: 178
## initial value 152.726831
## final value 133.263981
## converged
## # weights: 296
## initial value 153.708974
## final value 133.263981
## converged
## # weights: 60
## initial value 157.861770
## iter 10 value 107.750495
## iter 20 value 92.953617
## iter 30 value 89.336987
## iter 40 value 87.520725
## iter 50 value 86.881407
## iter 60 value 86.656911
## final value 86.655489
## converged
## # weights: 178
## initial value 183.524237
## iter 10 value 108.872204
## iter 20 value 90.376848
## iter 30 value 87.223766
## iter 40 value 85.254958
## iter 50 value 83.214067
## iter 60 value 82.831769
## iter 70 value 82.808170
## iter 80 value 82.800759
## iter 90 value 82.800333
## iter 100 value 82.798724
## final value 82.798724
## stopped after 100 iterations
## # weights: 296
## initial value 175.584864
## iter 10 value 106.157978
## iter 20 value 85.210206
## iter 30 value 83.886088
## iter 40 value 83.774191
## iter 50 value 83.761625
## iter 60 value 83.754455
## final value 83.754349
## converged
## # weights: 60
## initial value 200.319270
## iter 10 value 133.279642
## iter 20 value 133.273870
## iter 30 value 133.271864
## iter 40 value 133.156751
## iter 50 value 97.921034
## iter 60 value 81.555161
## iter 70 value 77.972400
## iter 80 value 77.896650
## iter 90 value 77.888740
## iter 100 value 77.862130
## final value 77.862130
## stopped after 100 iterations
## # weights: 178
## initial value 164.883403
## iter 10 value 133.279119
## iter 20 value 133.278813
## iter 30 value 133.278489
## iter 40 value 133.278146
## iter 50 value 133.277781
## iter 60 value 133.277393
## iter 70 value 133.276978
## iter 80 value 133.276530
## iter 90 value 133.276044
## iter 100 value 133.275514
## final value 133.275514
## stopped after 100 iterations
## # weights: 296
## initial value 161.161927
## iter 10 value 133.274350
## iter 20 value 133.269226
## iter 30 value 133.268602
## iter 40 value 133.267597
## iter 50 value 133.265629
## iter 60 value 133.260989
## iter 70 value 133.250450
## iter 80 value 133.219004
## iter 90 value 125.496264
## iter 100 value 80.734713
## final value 80.734713
## stopped after 100 iterations
## # weights: 60
## initial value 139.289941
## final value 113.388734
## converged
## # weights: 178
## initial value 159.415392
## final value 113.388734
## converged
## # weights: 296
## initial value 157.924706
## final value 113.388734
## converged
## # weights: 60
## initial value 158.693071
## iter 10 value 92.294823
## iter 20 value 84.618266
## iter 30 value 81.965763
## iter 40 value 80.027005
## iter 50 value 79.939616
## final value 79.939411
## converged
## # weights: 178
## initial value 140.418674
## iter 10 value 88.021358
## iter 20 value 81.842120
## iter 30 value 78.681442
## iter 40 value 77.717065
## iter 50 value 77.487255
## iter 60 value 77.426775
## iter 70 value 77.422786
## iter 80 value 77.422477
## final value 77.422469
## converged
## # weights: 296
## initial value 165.535946
## iter 10 value 97.494707
## iter 20 value 82.696975
## iter 30 value 79.238579
## iter 40 value 77.218711
## iter 50 value 77.062680
## iter 60 value 77.049897
## iter 70 value 77.036654
## iter 80 value 77.036210
## iter 90 value 77.036161
## final value 77.036159
## converged
## # weights: 60
## initial value 166.189958
## iter 10 value 113.404199
## iter 20 value 113.401138
## iter 30 value 113.382158
## iter 40 value 113.378200
## iter 50 value 113.372235
## iter 60 value 113.362123
## iter 70 value 113.341011
## iter 80 value 113.269326
## iter 90 value 99.035456
## iter 100 value 84.807476
## final value 84.807476
## stopped after 100 iterations
## # weights: 178
## initial value 169.886711
## iter 10 value 113.401477
## iter 20 value 113.393225
## iter 30 value 113.376935
## iter 40 value 113.255804
## iter 50 value 79.882365
## iter 60 value 73.399362
## iter 70 value 70.859908
## iter 80 value 70.504168
## iter 90 value 70.416897
## iter 100 value 70.390989
## final value 70.390989
## stopped after 100 iterations
## # weights: 296
## initial value 134.474839
## iter 10 value 113.393804
## iter 20 value 113.393080
## iter 30 value 113.392140
## iter 40 value 113.390858
## iter 50 value 113.388993
## iter 60 value 113.385857
## iter 70 value 113.380445
## iter 80 value 113.371023
## iter 90 value 113.342510
## iter 100 value 88.885979
## final value 88.885979
## stopped after 100 iterations
## # weights: 60
## initial value 145.021439
## final value 118.724424
## converged
## # weights: 178
## initial value 188.896637
## final value 118.724424
## converged
## # weights: 296
## initial value 157.323671
## final value 118.724424
## converged
## # weights: 60
## initial value 182.144353
## iter 10 value 93.069991
## iter 20 value 86.410864
## iter 30 value 83.502132
## iter 40 value 82.819543
## iter 50 value 82.809890
## final value 82.809885
## converged
## # weights: 178
## initial value 158.901052
## iter 10 value 94.501950
## iter 20 value 81.845022
## iter 30 value 78.488711
## iter 40 value 78.283946
## iter 50 value 78.267683
## iter 60 value 78.266405
## iter 70 value 78.266051
## final value 78.266036
## converged
## # weights: 296
## initial value 174.945795
## iter 10 value 106.603328
## iter 20 value 80.887562
## iter 30 value 78.078284
## iter 40 value 77.791137
## iter 50 value 77.684926
## iter 60 value 77.659723
## iter 70 value 77.609261
## iter 80 value 77.586141
## iter 90 value 77.585167
## iter 100 value 77.585067
## final value 77.585067
## stopped after 100 iterations
## # weights: 60
## initial value 166.154310
## iter 10 value 118.743596
## iter 20 value 118.737144
## final value 118.734833
## converged
## # weights: 178
## initial value 172.013418
## iter 10 value 118.738755
## iter 20 value 116.993118
## iter 30 value 75.937415
## iter 40 value 73.742646
## iter 50 value 71.227487
## iter 60 value 70.846993
## iter 70 value 70.815772
## iter 80 value 70.810602
## iter 90 value 70.702603
## iter 100 value 70.697218
## final value 70.697218
## stopped after 100 iterations
## # weights: 296
## initial value 171.730989
## iter 10 value 118.744873
## iter 20 value 118.656580
## iter 30 value 80.002323
## iter 40 value 79.695294
## iter 50 value 79.251453
## iter 60 value 76.058110
## iter 70 value 75.864023
## iter 80 value 74.517500
## iter 90 value 74.274810
## iter 100 value 74.129143
## final value 74.129143
## stopped after 100 iterations
## # weights: 60
## initial value 186.110176
## final value 109.855667
## converged
## # weights: 178
## initial value 128.839743
## final value 109.855667
## converged
## # weights: 296
## initial value 131.125067
## final value 109.855667
## converged
## # weights: 60
## initial value 155.751991
## iter 10 value 100.277713
## iter 20 value 81.715729
## iter 30 value 75.790914
## iter 40 value 75.298564
## iter 50 value 75.246101
## final value 75.245972
## converged
## # weights: 178
## initial value 162.567528
## iter 10 value 85.334160
## iter 20 value 73.818580
## iter 30 value 73.333944
## iter 40 value 73.173337
## iter 50 value 73.144720
## final value 73.144487
## converged
## # weights: 296
## initial value 157.095965
## iter 10 value 86.463141
## iter 20 value 73.779182
## iter 30 value 73.034642
## iter 40 value 72.943623
## iter 50 value 72.938515
## iter 60 value 72.938295
## final value 72.938293
## converged
## # weights: 60
## initial value 156.048261
## iter 10 value 109.870813
## iter 20 value 109.861960
## iter 30 value 102.544661
## iter 40 value 96.627910
## iter 50 value 94.912038
## iter 60 value 94.903210
## iter 70 value 94.812553
## iter 80 value 90.794094
## iter 90 value 87.916746
## iter 100 value 87.847291
## final value 87.847291
## stopped after 100 iterations
## # weights: 178
## initial value 134.351854
## iter 10 value 109.869498
## iter 20 value 83.970868
## iter 30 value 66.618439
## iter 40 value 66.242093
## iter 50 value 66.131203
## iter 60 value 66.116424
## iter 70 value 65.987269
## iter 80 value 65.958577
## iter 90 value 65.953170
## iter 100 value 65.946573
## final value 65.946573
## stopped after 100 iterations
## # weights: 296
## initial value 189.104124
## iter 10 value 109.871622
## iter 20 value 104.218545
## iter 30 value 81.727719
## iter 40 value 81.599631
## iter 50 value 81.480786
## iter 60 value 77.744483
## iter 70 value 77.241535
## iter 80 value 77.092946
## iter 90 value 77.066576
## iter 100 value 76.903381
## final value 76.903381
## stopped after 100 iterations
## # weights: 60
## initial value 164.226056
## final value 97.748603
## converged
## # weights: 178
## initial value 121.132444
## final value 97.748603
## converged
## # weights: 296
## initial value 144.017104
## final value 97.748603
## converged
## # weights: 60
## initial value 167.626915
## iter 10 value 83.223329
## iter 20 value 75.701093
## iter 30 value 74.164476
## iter 40 value 72.768485
## iter 50 value 70.974647
## iter 60 value 70.886570
## final value 70.885772
## converged
## # weights: 178
## initial value 171.469773
## iter 10 value 87.357656
## iter 20 value 74.227670
## iter 30 value 70.937289
## iter 40 value 69.435873
## iter 50 value 68.985208
## iter 60 value 68.794145
## iter 70 value 68.772730
## iter 80 value 68.771453
## iter 90 value 68.771255
## final value 68.771242
## converged
## # weights: 296
## initial value 132.689817
## iter 10 value 82.576172
## iter 20 value 70.427869
## iter 30 value 68.550304
## iter 40 value 67.534431
## iter 50 value 67.309237
## iter 60 value 67.283753
## iter 70 value 67.282857
## final value 67.282848
## converged
## # weights: 60
## initial value 173.877571
## iter 10 value 97.765155
## iter 20 value 97.763688
## iter 30 value 97.760923
## iter 40 value 97.760860
## iter 50 value 97.760793
## iter 60 value 97.760721
## iter 70 value 97.760643
## iter 80 value 97.760558
## iter 90 value 97.760465
## iter 100 value 97.760364
## final value 97.760364
## stopped after 100 iterations
## # weights: 178
## initial value 152.874747
## iter 10 value 97.761650
## iter 20 value 97.755598
## iter 30 value 97.755009
## iter 40 value 97.754272
## iter 50 value 97.753088
## iter 60 value 97.737557
## iter 70 value 97.277356
## iter 80 value 65.440542
## iter 90 value 64.966982
## iter 100 value 64.161437
## final value 64.161437
## stopped after 100 iterations
## # weights: 296
## initial value 163.418309
## iter 10 value 97.764497
## iter 20 value 97.749144
## iter 30 value 97.738271
## iter 40 value 97.392482
## iter 50 value 64.834613
## iter 60 value 63.326431
## iter 70 value 62.673905
## iter 80 value 62.593769
## iter 90 value 62.354861
## iter 100 value 62.318675
## final value 62.318675
## stopped after 100 iterations
## # weights: 60
## initial value 142.872384
## final value 128.793557
## converged
## # weights: 178
## initial value 135.197497
## final value 128.793557
## converged
## # weights: 296
## initial value 137.754192
## final value 128.793557
## converged
## # weights: 60
## initial value 132.442359
## iter 10 value 84.856177
## iter 20 value 80.560448
## iter 30 value 76.401523
## iter 40 value 75.909349
## iter 50 value 75.807991
## final value 75.807788
## converged
## # weights: 178
## initial value 152.126524
## iter 10 value 94.769113
## iter 20 value 77.547680
## iter 30 value 74.856109
## iter 40 value 74.144952
## iter 50 value 73.709279
## iter 60 value 73.528741
## iter 70 value 73.513494
## iter 80 value 73.513218
## iter 80 value 73.513217
## iter 80 value 73.513217
## final value 73.513217
## converged
## # weights: 296
## initial value 139.064980
## iter 10 value 83.909448
## iter 20 value 75.481878
## iter 30 value 73.908697
## iter 40 value 73.562398
## iter 50 value 73.346848
## iter 60 value 73.274148
## iter 70 value 73.265649
## iter 80 value 73.265085
## final value 73.265060
## converged
## # weights: 60
## initial value 156.629530
## iter 10 value 128.813098
## iter 20 value 128.806486
## iter 30 value 128.788864
## iter 40 value 128.784779
## iter 50 value 128.778615
## iter 60 value 128.768174
## iter 70 value 128.746321
## iter 80 value 128.670717
## iter 90 value 117.806334
## iter 100 value 88.788443
## final value 88.788443
## stopped after 100 iterations
## # weights: 178
## initial value 130.753256
## iter 10 value 128.815147
## iter 20 value 128.787983
## iter 30 value 128.218721
## iter 40 value 84.188390
## iter 50 value 78.167696
## iter 60 value 74.997237
## iter 70 value 74.735772
## iter 80 value 74.650828
## iter 90 value 74.639245
## iter 100 value 74.589219
## final value 74.589219
## stopped after 100 iterations
## # weights: 296
## initial value 133.631278
## iter 10 value 128.798169
## iter 20 value 128.775359
## iter 30 value 108.993604
## iter 40 value 91.505387
## iter 50 value 80.997516
## iter 60 value 75.423304
## iter 70 value 72.930909
## iter 80 value 72.692487
## iter 90 value 72.659602
## iter 100 value 72.629630
## final value 72.629630
## stopped after 100 iterations
## # weights: 60
## initial value 145.691426
## final value 123.780074
## converged
## # weights: 178
## initial value 149.460380
## final value 123.780074
## converged
## # weights: 296
## initial value 176.983085
## final value 123.780074
## converged
## # weights: 60
## initial value 140.947702
## iter 10 value 100.253043
## iter 20 value 88.327515
## iter 30 value 83.385732
## iter 40 value 82.972564
## iter 50 value 82.866662
## iter 60 value 82.421734
## iter 70 value 82.421021
## iter 70 value 82.421021
## iter 70 value 82.421021
## final value 82.421021
## converged
## # weights: 178
## initial value 147.687932
## iter 10 value 94.886488
## iter 20 value 84.273040
## iter 30 value 81.181986
## iter 40 value 80.119671
## iter 50 value 79.818474
## iter 60 value 79.796113
## iter 70 value 79.795880
## final value 79.795876
## converged
## # weights: 296
## initial value 177.186375
## iter 10 value 92.790822
## iter 20 value 81.683812
## iter 30 value 80.144148
## iter 40 value 79.606902
## iter 50 value 79.412225
## iter 60 value 79.316121
## iter 70 value 79.288447
## iter 80 value 79.269402
## iter 90 value 79.268411
## iter 100 value 79.267771
## final value 79.267771
## stopped after 100 iterations
## # weights: 60
## initial value 147.693161
## iter 10 value 123.797607
## iter 20 value 123.797465
## iter 30 value 123.797319
## iter 40 value 123.797172
## iter 50 value 123.797021
## iter 60 value 123.796867
## iter 70 value 123.796710
## iter 80 value 123.796549
## iter 90 value 123.796385
## iter 100 value 123.796216
## final value 123.796216
## stopped after 100 iterations
## # weights: 178
## initial value 143.013820
## iter 10 value 123.744835
## iter 20 value 123.489969
## iter 30 value 77.711733
## iter 40 value 75.621187
## iter 50 value 75.557064
## iter 60 value 75.549732
## iter 70 value 75.547560
## iter 80 value 75.495021
## iter 90 value 75.413386
## iter 100 value 73.900969
## final value 73.900969
## stopped after 100 iterations
## # weights: 296
## initial value 129.319646
## iter 10 value 123.772937
## iter 20 value 123.684515
## iter 30 value 83.781545
## iter 40 value 78.280694
## iter 50 value 73.327623
## iter 60 value 72.690263
## iter 70 value 72.558142
## iter 80 value 72.227772
## iter 90 value 72.142289
## iter 100 value 71.545219
## final value 71.545219
## stopped after 100 iterations
## # weights: 60
## initial value 151.908023
## final value 106.922881
## converged
## # weights: 178
## initial value 132.969844
## final value 106.922881
## converged
## # weights: 296
## initial value 172.155040
## final value 106.922881
## converged
## # weights: 60
## initial value 149.585390
## iter 10 value 103.712356
## iter 20 value 90.429418
## iter 30 value 82.055336
## iter 40 value 78.426929
## iter 50 value 78.065661
## iter 60 value 77.923777
## iter 70 value 77.919362
## final value 77.919345
## converged
## # weights: 178
## initial value 179.051341
## iter 10 value 99.451844
## iter 20 value 77.870210
## iter 30 value 76.307973
## iter 40 value 76.102199
## iter 50 value 76.077733
## iter 60 value 76.077357
## iter 70 value 76.077197
## final value 76.077195
## converged
## # weights: 296
## initial value 162.031124
## iter 10 value 99.958534
## iter 20 value 81.361885
## iter 30 value 78.935328
## iter 40 value 76.113329
## iter 50 value 76.009308
## iter 60 value 75.869239
## iter 70 value 75.782110
## iter 80 value 75.747830
## iter 90 value 75.745660
## final value 75.745653
## converged
## # weights: 60
## initial value 161.237381
## iter 10 value 106.838861
## iter 20 value 94.185265
## iter 30 value 71.381289
## iter 40 value 69.529914
## iter 50 value 69.220039
## iter 60 value 69.168526
## iter 70 value 69.074411
## iter 80 value 69.014212
## iter 90 value 69.011152
## iter 100 value 69.009819
## final value 69.009819
## stopped after 100 iterations
## # weights: 178
## initial value 156.366314
## iter 10 value 106.934558
## iter 20 value 106.879236
## iter 30 value 89.204401
## iter 40 value 72.756637
## iter 50 value 71.233226
## iter 60 value 70.583484
## iter 70 value 70.541445
## iter 80 value 70.524984
## iter 90 value 70.498042
## iter 100 value 70.467415
## final value 70.467415
## stopped after 100 iterations
## # weights: 296
## initial value 131.668012
## iter 10 value 106.907481
## iter 20 value 106.853515
## iter 30 value 106.026327
## iter 40 value 78.011730
## iter 50 value 72.009302
## iter 60 value 69.805221
## iter 70 value 69.508036
## iter 80 value 68.858350
## iter 90 value 68.768420
## iter 100 value 68.735624
## final value 68.735624
## stopped after 100 iterations
## # weights: 60
## initial value 142.357258
## final value 106.903656
## converged
## # weights: 178
## initial value 125.552936
## final value 106.903656
## converged
## # weights: 296
## initial value 136.824222
## final value 106.903656
## converged
## # weights: 60
## initial value 140.476288
## iter 10 value 94.726039
## iter 20 value 75.516994
## iter 30 value 68.422885
## iter 40 value 63.361709
## iter 50 value 63.105978
## iter 60 value 62.726765
## iter 70 value 62.445603
## iter 80 value 62.034019
## iter 90 value 61.935613
## iter 100 value 61.724151
## final value 61.724151
## stopped after 100 iterations
## # weights: 178
## initial value 159.433741
## iter 10 value 74.447051
## iter 20 value 61.932960
## iter 30 value 60.963926
## iter 40 value 59.577046
## iter 50 value 59.125196
## iter 60 value 59.121496
## iter 70 value 59.120874
## final value 59.120784
## converged
## # weights: 296
## initial value 139.768945
## iter 10 value 74.627299
## iter 20 value 61.289732
## iter 30 value 57.820990
## iter 40 value 57.284592
## iter 50 value 57.144836
## iter 60 value 57.114363
## iter 70 value 57.111296
## final value 57.111268
## converged
## # weights: 60
## initial value 121.031540
## iter 10 value 106.893008
## iter 20 value 106.882975
## iter 30 value 106.864164
## iter 40 value 106.806441
## iter 50 value 103.669793
## iter 60 value 65.775990
## iter 70 value 63.754937
## iter 80 value 63.274640
## iter 90 value 63.217140
## iter 100 value 63.163567
## final value 63.163567
## stopped after 100 iterations
## # weights: 178
## initial value 168.410076
## iter 10 value 106.920184
## iter 20 value 106.919858
## iter 30 value 106.919595
## iter 40 value 106.916979
## iter 50 value 105.877059
## iter 60 value 59.891366
## iter 70 value 58.511502
## iter 80 value 58.261831
## iter 90 value 58.056220
## iter 100 value 57.929362
## final value 57.929362
## stopped after 100 iterations
## # weights: 296
## initial value 114.109874
## iter 10 value 70.290917
## iter 20 value 61.666918
## iter 30 value 59.771340
## iter 40 value 59.273518
## iter 50 value 58.783373
## iter 60 value 58.604113
## iter 70 value 57.984375
## iter 80 value 57.935111
## iter 90 value 57.890430
## iter 100 value 57.141566
## final value 57.141566
## stopped after 100 iterations
## # weights: 60
## initial value 182.898574
## final value 139.171930
## converged
## # weights: 178
## initial value 196.040336
## final value 139.171930
## converged
## # weights: 296
## initial value 180.910072
## final value 139.171930
## converged
## # weights: 60
## initial value 244.518833
## iter 10 value 139.002924
## iter 20 value 125.463387
## iter 30 value 113.337994
## iter 40 value 107.875994
## iter 50 value 106.804479
## iter 60 value 106.725470
## final value 106.725259
## converged
## # weights: 178
## initial value 179.508169
## iter 10 value 112.618068
## iter 20 value 104.852449
## iter 30 value 104.213797
## iter 40 value 104.103233
## iter 50 value 104.099788
## final value 104.099742
## converged
## # weights: 296
## initial value 179.134552
## iter 10 value 114.494351
## iter 20 value 106.855663
## iter 30 value 105.016942
## iter 40 value 104.042678
## iter 50 value 103.125369
## iter 60 value 102.806250
## iter 70 value 102.603273
## iter 80 value 102.572866
## iter 90 value 102.556670
## iter 100 value 102.553679
## final value 102.553679
## stopped after 100 iterations
## # weights: 60
## initial value 183.037942
## iter 10 value 139.182711
## iter 20 value 122.947223
## iter 30 value 113.534271
## iter 40 value 112.215968
## iter 50 value 104.653253
## iter 60 value 103.959912
## iter 70 value 103.942349
## iter 80 value 103.937665
## iter 90 value 102.528126
## iter 100 value 102.402525
## final value 102.402525
## stopped after 100 iterations
## # weights: 178
## initial value 196.295532
## iter 10 value 139.189471
## iter 20 value 138.663877
## iter 30 value 100.130732
## iter 40 value 98.014490
## iter 50 value 97.017215
## iter 60 value 97.000742
## iter 70 value 96.997309
## iter 80 value 96.991967
## iter 90 value 96.981883
## iter 100 value 96.964601
## final value 96.964601
## stopped after 100 iterations
## # weights: 296
## initial value 210.331367
## iter 10 value 139.191750
## iter 20 value 139.177512
## iter 30 value 139.176995
## iter 40 value 139.176292
## iter 50 value 139.175283
## iter 60 value 139.173711
## iter 70 value 139.170949
## iter 80 value 139.165246
## iter 90 value 139.150277
## iter 100 value 139.072348
## final value 139.072348
## stopped after 100 iterations
## # weights: 60
## initial value 131.677040
## final value 107.144018
## converged
## # weights: 178
## initial value 125.521209
## final value 107.144018
## converged
## # weights: 296
## initial value 126.664962
## final value 107.144018
## converged
## # weights: 60
## initial value 139.495992
## iter 10 value 90.261702
## iter 20 value 74.769455
## iter 30 value 71.800408
## iter 40 value 71.402626
## iter 50 value 71.226121
## iter 60 value 70.531654
## iter 70 value 70.496001
## final value 70.495591
## converged
## # weights: 178
## initial value 158.656334
## iter 10 value 81.105634
## iter 20 value 69.630650
## iter 30 value 69.057877
## iter 40 value 68.392502
## iter 50 value 68.281028
## iter 60 value 68.268435
## iter 70 value 68.242263
## iter 80 value 68.241275
## iter 90 value 68.241146
## final value 68.241085
## converged
## # weights: 296
## initial value 151.172355
## iter 10 value 88.494252
## iter 20 value 68.632905
## iter 30 value 67.863043
## iter 40 value 67.754685
## iter 50 value 67.715027
## iter 60 value 67.712739
## iter 70 value 67.704912
## iter 80 value 67.702342
## final value 67.702328
## converged
## # weights: 60
## initial value 154.540761
## iter 10 value 107.133492
## iter 20 value 107.127327
## iter 30 value 107.116209
## iter 40 value 107.091353
## iter 50 value 106.993404
## iter 60 value 79.391017
## iter 70 value 65.542880
## iter 80 value 65.439071
## iter 90 value 65.413707
## iter 100 value 65.410687
## final value 65.410687
## stopped after 100 iterations
## # weights: 178
## initial value 122.091101
## iter 10 value 107.116252
## iter 20 value 106.858608
## iter 30 value 72.576661
## iter 40 value 64.407887
## iter 50 value 63.103732
## iter 60 value 62.384891
## iter 70 value 62.294432
## iter 80 value 62.253726
## iter 90 value 62.226373
## iter 100 value 61.944413
## final value 61.944413
## stopped after 100 iterations
## # weights: 296
## initial value 139.264488
## iter 10 value 107.153209
## iter 20 value 89.484628
## iter 30 value 62.850982
## iter 40 value 60.585692
## iter 50 value 60.174626
## iter 60 value 60.094470
## iter 70 value 60.080676
## iter 80 value 60.019401
## iter 90 value 59.948107
## iter 100 value 59.928908
## final value 59.928908
## stopped after 100 iterations
## # weights: 60
## initial value 154.863332
## final value 120.914182
## converged
## # weights: 178
## initial value 178.408863
## final value 120.914182
## converged
## # weights: 296
## initial value 185.353190
## final value 120.914182
## converged
## # weights: 60
## initial value 158.418834
## iter 10 value 90.816808
## iter 20 value 79.886117
## iter 30 value 77.113404
## iter 40 value 76.155021
## iter 50 value 76.024363
## iter 60 value 76.001152
## iter 70 value 75.991864
## final value 75.991859
## converged
## # weights: 178
## initial value 148.875738
## iter 10 value 80.186538
## iter 20 value 73.822865
## iter 30 value 72.766687
## iter 40 value 72.581639
## iter 50 value 72.575329
## iter 60 value 72.574831
## iter 70 value 72.574635
## final value 72.574633
## converged
## # weights: 296
## initial value 152.890091
## iter 10 value 85.046445
## iter 20 value 73.301353
## iter 30 value 72.035850
## iter 40 value 71.853466
## iter 50 value 71.598043
## iter 60 value 70.931796
## iter 70 value 70.890178
## iter 80 value 70.885675
## iter 90 value 70.885015
## iter 100 value 70.884620
## final value 70.884620
## stopped after 100 iterations
## # weights: 60
## initial value 136.849519
## iter 10 value 120.911189
## iter 20 value 120.908736
## iter 30 value 120.905421
## iter 40 value 120.900709
## iter 50 value 120.893360
## iter 60 value 120.880008
## iter 70 value 120.847491
## iter 80 value 120.652913
## iter 90 value 72.883209
## iter 100 value 67.001350
## final value 67.001350
## stopped after 100 iterations
## # weights: 178
## initial value 158.579307
## iter 10 value 120.925068
## iter 20 value 120.625325
## iter 30 value 69.914740
## iter 40 value 67.811532
## iter 50 value 67.590096
## iter 60 value 67.457328
## iter 70 value 67.446959
## iter 80 value 67.441825
## iter 90 value 67.438373
## iter 100 value 66.462038
## final value 66.462038
## stopped after 100 iterations
## # weights: 296
## initial value 159.515987
## iter 10 value 120.923858
## iter 20 value 120.917901
## iter 30 value 120.916486
## iter 40 value 120.913534
## iter 50 value 120.903810
## iter 60 value 120.859408
## iter 70 value 120.114912
## iter 80 value 69.900488
## iter 90 value 67.722681
## iter 100 value 66.781436
## final value 66.781436
## stopped after 100 iterations
## # weights: 60
## initial value 144.595725
## final value 130.833202
## converged
## # weights: 178
## initial value 162.301131
## final value 130.833202
## converged
## # weights: 296
## initial value 160.155003
## final value 130.833202
## converged
## # weights: 60
## initial value 190.752124
## iter 10 value 99.243148
## iter 20 value 91.869189
## iter 30 value 84.167639
## iter 40 value 82.668467
## iter 50 value 82.515260
## iter 60 value 82.450067
## final value 82.450055
## converged
## # weights: 178
## initial value 144.786101
## iter 10 value 96.272166
## iter 20 value 82.272777
## iter 30 value 80.448718
## iter 40 value 79.722711
## iter 50 value 79.096695
## iter 60 value 78.757609
## iter 70 value 78.626258
## iter 80 value 78.551631
## iter 90 value 78.536827
## iter 100 value 78.536700
## final value 78.536700
## stopped after 100 iterations
## # weights: 296
## initial value 198.346033
## iter 10 value 103.202607
## iter 20 value 83.573393
## iter 30 value 80.345678
## iter 40 value 78.954207
## iter 50 value 78.075775
## iter 60 value 77.707136
## iter 70 value 77.526414
## iter 80 value 77.501402
## iter 90 value 77.499527
## iter 100 value 77.499281
## final value 77.499281
## stopped after 100 iterations
## # weights: 60
## initial value 147.814249
## iter 10 value 101.717262
## iter 20 value 88.179796
## iter 30 value 88.165237
## iter 40 value 88.145059
## iter 50 value 87.779292
## iter 60 value 87.773395
## iter 70 value 87.769619
## iter 80 value 87.765722
## iter 90 value 87.012448
## iter 100 value 84.778126
## final value 84.778126
## stopped after 100 iterations
## # weights: 178
## initial value 171.840826
## iter 10 value 130.851343
## iter 20 value 130.844549
## iter 30 value 130.766848
## iter 40 value 113.349785
## iter 50 value 79.218502
## iter 60 value 78.492163
## iter 70 value 77.981998
## iter 80 value 77.759374
## iter 90 value 77.744317
## iter 100 value 73.906827
## final value 73.906827
## stopped after 100 iterations
## # weights: 296
## initial value 173.383393
## iter 10 value 130.846514
## iter 20 value 130.835034
## iter 30 value 130.824113
## iter 40 value 130.691582
## iter 50 value 83.674135
## iter 60 value 76.790041
## iter 70 value 73.118928
## iter 80 value 72.762779
## iter 90 value 72.609906
## iter 100 value 72.591074
## final value 72.591074
## stopped after 100 iterations
## # weights: 60
## initial value 200.572732
## final value 133.801472
## converged
## # weights: 178
## initial value 160.845361
## final value 133.801472
## converged
## # weights: 296
## initial value 182.618272
## final value 133.801472
## converged
## # weights: 60
## initial value 173.798736
## iter 10 value 131.301647
## iter 20 value 99.606997
## iter 30 value 98.184770
## iter 40 value 96.590899
## iter 50 value 96.463513
## iter 60 value 96.427417
## final value 96.427371
## converged
## # weights: 178
## initial value 185.302189
## iter 10 value 103.269985
## iter 20 value 94.548356
## iter 30 value 93.956401
## iter 40 value 93.904150
## iter 50 value 93.900127
## iter 60 value 93.899786
## final value 93.899761
## converged
## # weights: 296
## initial value 214.712712
## iter 10 value 124.384964
## iter 20 value 96.057429
## iter 30 value 93.927522
## iter 40 value 93.669537
## iter 50 value 93.539594
## iter 60 value 93.534598
## iter 70 value 93.534319
## final value 93.534295
## converged
## # weights: 60
## initial value 191.016494
## iter 10 value 133.808362
## iter 20 value 133.786882
## iter 30 value 133.779121
## iter 40 value 133.764240
## iter 50 value 133.723947
## iter 60 value 133.350149
## iter 70 value 94.906606
## iter 80 value 93.053623
## iter 90 value 92.928916
## iter 100 value 92.774964
## final value 92.774964
## stopped after 100 iterations
## # weights: 178
## initial value 205.925056
## iter 10 value 133.814515
## iter 20 value 133.808845
## iter 30 value 133.808495
## iter 40 value 133.808015
## iter 50 value 133.807268
## iter 60 value 133.806086
## iter 70 value 133.804539
## iter 80 value 133.802653
## iter 90 value 133.799960
## iter 100 value 133.795319
## final value 133.795319
## stopped after 100 iterations
## # weights: 296
## initial value 192.505715
## iter 10 value 133.825159
## iter 20 value 133.808937
## iter 30 value 98.518895
## iter 40 value 86.927292
## iter 50 value 86.422395
## iter 60 value 86.372817
## iter 70 value 86.332278
## iter 80 value 86.328890
## iter 90 value 86.326403
## iter 100 value 86.324093
## final value 86.324093
## stopped after 100 iterations
## # weights: 60
## initial value 173.391896
## final value 122.706866
## converged
## # weights: 178
## initial value 140.125316
## final value 122.706866
## converged
## # weights: 296
## initial value 198.393822
## final value 122.706866
## converged
## # weights: 60
## initial value 140.604214
## iter 10 value 95.849814
## iter 20 value 83.991439
## iter 30 value 78.822215
## iter 40 value 77.869625
## iter 50 value 77.566676
## iter 60 value 77.553424
## iter 70 value 77.552513
## final value 77.552500
## converged
## # weights: 178
## initial value 150.438069
## iter 10 value 83.684745
## iter 20 value 75.738017
## iter 30 value 74.830491
## iter 40 value 74.760890
## iter 50 value 74.746764
## iter 60 value 74.721905
## iter 70 value 74.640475
## iter 80 value 74.630246
## iter 90 value 74.629393
## iter 100 value 74.629295
## final value 74.629295
## stopped after 100 iterations
## # weights: 296
## initial value 158.427214
## iter 10 value 94.627055
## iter 20 value 76.011361
## iter 30 value 74.718456
## iter 40 value 74.460139
## iter 50 value 74.369542
## iter 60 value 74.253569
## iter 70 value 74.245876
## iter 80 value 74.245535
## iter 90 value 74.245501
## final value 74.245495
## converged
## # weights: 60
## initial value 164.619756
## iter 10 value 122.700476
## iter 20 value 90.935341
## iter 30 value 90.226362
## iter 40 value 89.498320
## iter 50 value 89.465710
## iter 60 value 89.460218
## iter 70 value 89.434848
## iter 80 value 88.801343
## iter 90 value 88.792905
## iter 100 value 88.767022
## final value 88.767022
## stopped after 100 iterations
## # weights: 178
## initial value 130.321741
## iter 10 value 122.767972
## iter 20 value 122.767670
## iter 30 value 122.767366
## iter 40 value 122.767060
## iter 50 value 122.766751
## iter 60 value 122.766440
## iter 70 value 122.766127
## iter 80 value 122.765809
## iter 90 value 122.765489
## iter 100 value 122.765164
## final value 122.765164
## stopped after 100 iterations
## # weights: 296
## initial value 140.339550
## iter 10 value 122.716021
## iter 20 value 122.710209
## iter 30 value 122.705683
## iter 40 value 122.689849
## iter 50 value 122.625446
## iter 60 value 95.876180
## iter 70 value 79.051710
## iter 80 value 70.836541
## iter 90 value 69.758926
## iter 100 value 69.054177
## final value 69.054177
## stopped after 100 iterations
## # weights: 60
## initial value 169.769513
## final value 117.150773
## converged
## # weights: 178
## initial value 148.277456
## final value 117.150773
## converged
## # weights: 296
## initial value 168.280941
## final value 117.150773
## converged
## # weights: 60
## initial value 149.924526
## iter 10 value 100.696774
## iter 20 value 84.179037
## iter 30 value 79.464098
## iter 40 value 75.989119
## iter 50 value 74.797594
## iter 60 value 74.119896
## iter 70 value 73.856566
## final value 73.856444
## converged
## # weights: 178
## initial value 164.399163
## iter 10 value 103.900809
## iter 20 value 76.156206
## iter 30 value 72.401839
## iter 40 value 71.433634
## iter 50 value 71.376809
## iter 60 value 71.371818
## final value 71.371650
## converged
## # weights: 296
## initial value 190.560717
## iter 10 value 97.739637
## iter 20 value 73.778230
## iter 30 value 70.444982
## iter 40 value 69.695949
## iter 50 value 69.618669
## iter 60 value 69.596662
## iter 70 value 69.591161
## iter 80 value 69.590507
## iter 90 value 69.590117
## iter 100 value 69.590044
## final value 69.590044
## stopped after 100 iterations
## # weights: 60
## initial value 145.994356
## iter 10 value 117.163779
## iter 20 value 116.886438
## iter 30 value 98.433313
## iter 40 value 82.015977
## iter 50 value 79.963346
## iter 60 value 78.431618
## iter 70 value 78.334836
## iter 80 value 78.324499
## iter 90 value 78.317468
## iter 100 value 77.706011
## final value 77.706011
## stopped after 100 iterations
## # weights: 178
## initial value 142.839119
## iter 10 value 117.163386
## iter 20 value 117.156649
## iter 30 value 117.107194
## iter 40 value 116.074636
## iter 50 value 68.504653
## iter 60 value 64.048069
## iter 70 value 63.733319
## iter 80 value 63.617572
## iter 90 value 63.492186
## iter 100 value 63.156960
## final value 63.156960
## stopped after 100 iterations
## # weights: 296
## initial value 134.901354
## iter 10 value 116.958967
## iter 20 value 77.156147
## iter 30 value 70.135461
## iter 40 value 69.613839
## iter 50 value 69.493187
## iter 60 value 67.621297
## iter 70 value 67.138300
## iter 80 value 66.343416
## iter 90 value 65.023966
## iter 100 value 64.902413
## final value 64.902413
## stopped after 100 iterations
## # weights: 60
## initial value 143.764213
## final value 128.990871
## converged
## # weights: 178
## initial value 163.666274
## final value 128.990871
## converged
## # weights: 296
## initial value 162.297407
## final value 128.990871
## converged
## # weights: 60
## initial value 155.591660
## iter 10 value 109.727636
## iter 20 value 87.967219
## iter 30 value 83.771744
## iter 40 value 83.093820
## iter 50 value 82.956742
## iter 60 value 82.953461
## final value 82.953424
## converged
## # weights: 178
## initial value 155.837848
## iter 10 value 110.559181
## iter 20 value 90.093653
## iter 30 value 85.469037
## iter 40 value 82.906909
## iter 50 value 79.545559
## iter 60 value 79.255041
## iter 70 value 79.172462
## iter 80 value 79.169434
## final value 79.169370
## converged
## # weights: 296
## initial value 168.967623
## iter 10 value 105.423450
## iter 20 value 82.978534
## iter 30 value 78.777465
## iter 40 value 77.719776
## iter 50 value 77.494938
## iter 60 value 77.422020
## iter 70 value 77.386677
## iter 80 value 77.376058
## iter 90 value 77.374803
## iter 100 value 77.374727
## final value 77.374727
## stopped after 100 iterations
## # weights: 60
## initial value 149.156379
## iter 10 value 128.979992
## iter 20 value 128.973818
## iter 30 value 128.963526
## iter 40 value 128.942466
## iter 50 value 128.872989
## iter 60 value 106.073078
## iter 70 value 78.350941
## iter 80 value 74.458338
## iter 90 value 73.893744
## iter 100 value 73.761682
## final value 73.761682
## stopped after 100 iterations
## # weights: 178
## initial value 180.409705
## iter 10 value 129.003449
## iter 20 value 128.972205
## iter 30 value 128.947574
## iter 40 value 128.820573
## iter 50 value 85.112076
## iter 60 value 74.041065
## iter 70 value 72.674093
## iter 80 value 71.862132
## iter 90 value 71.622498
## iter 100 value 71.574585
## final value 71.574585
## stopped after 100 iterations
## # weights: 296
## initial value 154.070785
## iter 10 value 128.991310
## iter 20 value 128.834732
## iter 30 value 77.616287
## iter 40 value 73.029292
## iter 50 value 72.737682
## iter 60 value 72.698203
## iter 70 value 72.486284
## iter 80 value 72.425353
## iter 90 value 72.420362
## iter 100 value 72.417661
## final value 72.417661
## stopped after 100 iterations
## # weights: 60
## initial value 164.027010
## final value 121.884691
## converged
## # weights: 178
## initial value 169.595370
## final value 121.884691
## converged
## # weights: 296
## initial value 144.768141
## final value 121.884691
## converged
## # weights: 60
## initial value 170.598454
## iter 10 value 89.327653
## iter 20 value 83.579912
## iter 30 value 83.122813
## final value 83.121137
## converged
## # weights: 178
## initial value 157.476528
## iter 10 value 88.879104
## iter 20 value 81.153489
## iter 30 value 79.396950
## iter 40 value 79.087217
## iter 50 value 79.003032
## iter 60 value 78.999977
## final value 78.999964
## converged
## # weights: 296
## initial value 159.585024
## iter 10 value 94.625004
## iter 20 value 82.653221
## iter 30 value 81.167872
## iter 40 value 80.981516
## iter 50 value 80.965076
## iter 60 value 80.964563
## iter 70 value 80.964494
## final value 80.964493
## converged
## # weights: 60
## initial value 141.836237
## iter 10 value 121.898910
## iter 20 value 121.892523
## iter 30 value 121.891865
## iter 40 value 121.891170
## iter 50 value 121.890426
## iter 60 value 121.889605
## iter 70 value 121.888667
## iter 80 value 121.887556
## iter 90 value 121.886195
## iter 100 value 121.884467
## final value 121.884467
## stopped after 100 iterations
## # weights: 178
## initial value 156.205562
## iter 10 value 121.896076
## iter 20 value 121.895628
## iter 30 value 121.895124
## iter 40 value 121.894556
## iter 50 value 121.893915
## iter 60 value 121.893194
## iter 70 value 121.892381
## iter 80 value 121.891461
## iter 90 value 121.890409
## iter 100 value 121.889187
## final value 121.889187
## stopped after 100 iterations
## # weights: 296
## initial value 156.038965
## iter 10 value 121.894182
## iter 20 value 121.882320
## iter 30 value 121.872610
## iter 40 value 96.472835
## iter 50 value 75.837759
## iter 60 value 74.507008
## iter 70 value 73.025054
## iter 80 value 72.344976
## iter 90 value 72.077480
## iter 100 value 72.042995
## final value 72.042995
## stopped after 100 iterations
## # weights: 296
## initial value 179.733449
## iter 10 value 97.031521
## iter 20 value 82.604387
## iter 30 value 79.044723
## iter 40 value 76.992855
## iter 50 value 76.444921
## iter 60 value 76.247254
## iter 70 value 76.056877
## iter 80 value 76.035584
## iter 90 value 76.035126
## final value 76.035088
## converged## RMSE Rsquared MAE
## 0.8993722 0.5441986 0.7042420Compare the performance of the different models on the test data using a table and plot.
| Model | RMSE | R_squared | MAE |
|---|---|---|---|
| Linear Regression | 0.8285972 | 0.5095169 | 0.6407973 |
| Random Forest | 0.6867001 | 0.6820857 | 0.5268914 |
| SVM | 0.7642341 | 0.5855119 | 0.5910040 |
| kNN | 0.8462222 | 0.4379732 | 0.7106254 |
| Neural Network | 0.8993722 | 0.5441986 | 0.7042420 |
(a) Which nonlinear regression model gives the optimal resampling and test set performance?
Based on the performance metrics (e.g., RMSE, R-squared, MAE) calculated on the test data, the Random Forest model appears to give the best performance among the nonlinear regression models. The Random Forest model has the lowest RMSE and the highest R-squared value compared to the other models, indicating that it has the best predictive accuracy. The Neural Network model also performs well, with a relatively low RMSE and a high R-squared value. The k-Nearest Neighbors (kNN) model has the highest RMSE and the lowest R-squared value among the models, indicating that it has the worst performance in terms of predictive accuracy. The Support Vector Machine (SVM) model has moderate performance, with an intermediate RMSE and R-squared value.
The Linear Regression model has the highest RMSE and the lowest R-squared value among all the models, indicating that it has the worst performance in terms of predictive accuracy. The Linear Regression model may not capture the non-linear relationships between the predictors and the response, leading to poorer predictive performance compared to the nonlinear regression models.
Overall, the Random Forest model appears to be the best-performing model for predicting the yield in the chemical manufacturing process based on the test data. The Random Forest model may be able to capture complex non-linear relationships between the predictors and the response, leading to better predictive accuracy compared to other models.
b. Which predictors are most important in the optimal nonlinear regression model? Do either the biological or process variables dominate the list? How do the top ten important predictors compare to the top ten predictors from the optimal linear model?
The Random Forest model is the optimal nonlinear regression model based on the test set performance. To determine the most important predictors in the Random Forest model, we can extract the variable importance measures from the model. The variable importance measures indicate the contribution of each predictor to the model’s predictive accuracy.
Variable Importance for Random Forest
## rf variable importance
##
## only 10 most important variables shown (out of 57)
##
## Overall
## ManufacturingProcess32 100.000
## BiologicalMaterial12 19.310
## ManufacturingProcess31 19.249
## ManufacturingProcess17 19.248
## BiologicalMaterial03 15.672
## ManufacturingProcess28 13.031
## ManufacturingProcess09 10.617
## BiologicalMaterial06 10.561
## ManufacturingProcess13 9.786
## ManufacturingProcess06 9.727
Important Predictors in the Random Forest
The importance plot generated from your Random Forest model shows the top 10 most influential predictors for predicting Yield in the chemical manufacturing process data. Based on the plot:
Most Important Predictors:
ManufacturingProcess32 is by far the most significant predictor in the Random Forest model. Other important predictors include ManufacturingProcess17, BiologicalMaterial03, BiologicalMaterial12, and ManufacturingProcess31.
Type of Variables:
Both process variables (e.g., ManufacturingProcess32, ManufacturingProcess17, etc.) and biological variables (e.g., BiologicalMaterial03, BiologicalMaterial12) appear in the top 10 list, but process variables tend to dominate the list, suggesting that variations in the manufacturing process might have a stronger nonlinear relationship with the yield.
Dominance of Biological or Process Variables
From the Random Forest model, it’s clear that process variables dominate the list of important predictors, indicating that they contribute more significantly to yield prediction in a nonlinear context. Process variables like ManufacturingProcess32 and ManufacturingProcess17 have high importance scores, which might indicate complex, nonlinear interactions within the manufacturing process itself that impact yield.
Biological variables are also influential but appear less frequently among the top predictors, suggesting that while they do affect yield, their relationship may be simpler or less interactive than the manufacturing process variables.
Comparison to the Optimal Linear Model
In a linear regression model, predictor importance is often assessed based on the magnitude of coefficients (assuming predictors are standardized), with larger coefficients indicating stronger linear associations.
If the linear regression model ranks different predictors as most important compared to the Random Forest model, this suggests that the relationships between those predictors and the target variable are primarily linear. In contrast, the predictors deemed important by the Random Forest model might contribute through nonlinear interactions or complex patterns that a linear model cannot capture.
Direct Comparison:
Ideally, you should generate a similar feature importance or coefficient plot from the linear model to see which predictors it emphasizes. Typically, linear models may favor a different set of predictors if the linear associations differ from the complex, non-linear interactions identified by Random Forest.
For instance, if a linear model identified certain biological variables as more important, it might indicate that these variables have a straightforward linear association with yield, while process variables contribute more complex, interaction-driven effects that Random Forest can capture but linear regression cannot.
In the nonlinear Random Forest model, process variables were generally more important than biological variables, suggesting that nonlinear interactions within the manufacturing process variables are critical for predicting yield.
In a linear regression model, the top predictors might differ, especially if certain variables (such as biological ones) have stronger direct correlations with yield.
Top Predictors Comparison: To provide a full comparison, check the standardized coefficients or feature importance from the linear model to see if there’s overlap or notable differences in the top predictors between the two models.
c. Explore the relationships between the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model. Do these plots reveal intuition about the biological or process predictors and their relationship with yield?
To explore the relationships between the top predictors unique to the optimal nonlinear regression model and the response variable (Yield), we can create scatterplots for these predictors. These plots can provide insights into the nature of the relationships between the predictors and the response, helping us understand how these variables impact yield.
Scatterplots for Top Predictors in Random Forest Model
## [1] "ManufacturingProcess32" "BiologicalMaterial12" "ManufacturingProcess31"
## [4] "ManufacturingProcess17" "BiologicalMaterial03" "ManufacturingProcess28"
## [7] "ManufacturingProcess09" "BiologicalMaterial06" "ManufacturingProcess13"
## [10] "ManufacturingProcess06"
The scatterplots provide some intuition about the biological predictors, although their relationship with yield appears subtle and not directly impactful on their own.
Most biological predictors show little to no clear linear relationship with yield. The scatterplots mostly exhibit horizontal distributions, suggesting that changes in these predictors individually do not correlate strongly with variations in yield.
This flat relationship indicates that these biological variables might not have a direct or consistent impact on yield when considered in isolation.
The slight curvature of the LOESS (smooth trend) lines in a few plots (e.g., BiologicalMaterial07) suggests a nonlinear relationship or threshold effect. Nonlinear regression models like Random Forests capture such patterns better than linear models, which could explain why these variables rank as important in the Random Forest model.
This nonlinear trend implies that certain levels of these biological materials might be optimal or critical to achieving a desired yield, even if small variations around these levels don’t make a significant difference.
The relatively flat patterns across a range of values suggest that biological materials might introduce inherent variability or “noise” rather than a deterministic effect on yield.
This observation aligns with real-world scenarios where biological inputs are less predictable, and small changes might not always have a significant or predictable impact. It implies that while these predictors are necessary, they don’t singularly drive the yield outcome in a straightforward manner.
The subtle influence of biological materials on yield hints that these materials could act in conjunction with process variables rather than independently. For instance, certain levels of a biological material might become more important only when paired with specific conditions in the manufacturing process (e.g., temperature, pressure, or chemical concentration).
This aligns with why a Random Forest model, which captures interactions among variables, finds value in these predictors. In contrast, a linear model might overlook their importance due to a lack of strong, direct correlations.
Process variables, often more controllable in manufacturing settings, are likely to exhibit clearer relationships with yield, as they can directly influence reaction rates, completion times, or product quality. Process variables might demonstrate more pronounced and direct effects on yield in comparison, which makes them more interpretable and more likely to rank higher in a linear regression model.
Biological Predictors: These likely contribute to the background variability in yield but may not consistently drive changes in yield without interacting with specific process conditions. They may need to be present in certain thresholds but aren’t definitive on their own.
Process Predictors: Expected to have a more consistent and measurable effect on yield, especially in terms of controllable process parameters. These might dominate in linear models due to their direct impact.
Interactions: The Random Forest model likely captures interactions between biological and process variables, which could explain why it ranks certain biological predictors highly. These interactions might be crucial for understanding how different factors combine to influence yield.
Overall, the subtle relationships between biological predictors and yield suggest that they might play a more nuanced role in the manufacturing process, potentially interacting with other variables to drive yield outcomes. The Random Forest model’s ability to capture these complex interactions highlights the importance of considering both biological and process variables in predicting yield accurately.
While the biological predictors appear indirectly related to yield, their presence is likely essential in conjunction with key process variables, supporting the intuition that process-driven factors dominate in determining yield directly.