Práctica 8: Regresión lineal con regularización ´- FAA
Judit González Prol
23 de abril de 2024
install.packages("knitr")Error in install.packages : Updating loaded packagesinstall.packages("glmnet")Error in install.packages : Updating loaded packagesinstall.packages("Brq")Error in install.packages : Updating loaded packageslibrary(knitr)
library(glmnet)
library(Brq)
library(MASS)ENTREGA PT2
Tenemos p variables o p caracteristicas y n rexistros.
Introducción a los escenarios de penalización
El problema que sucede en esta práctica es que \(p\>n\), por ello, los métodos aplicados hasta ahora no funcionarán. (Estamos en problemas de alta dimensión)
Para poder atacar estos conjuntos de datos, tenemos que utilizar otro tipo de conclusiones para tomar las decisiones correspondientes.
Como podemos reducir el numero de variables que vamos a utilizar. - 1º: Utilizamos un conjunto de variables mínimo, un: \(p'\<n\)
2º: Utilizamos métodos de regularización : Lasso … Reducen en num de coefficientes, es decir que sean 0.
La regresión Ridge, los minimiza pero no los llega a igualar a 0.
La regresión Lasso, los minimiza pero llegando a valer 0.
3º: Métodos de proyección:
Cuando calculamos los beta, los podemos obtener como el método de mínimos cuadrados o también los obtenemos también pensando en un problema de optimización.
- Ridge -> Restricciones del tipo:
Suma de los betas_i sea menor que un valor dado
- Lasso -> Restricciones del tipo:
En Rigde bajan de magnitud, y en Lasso llegan algunos a valer 0
Suma de los valos absolutos de los betas_i y que sea menor que un valor dado
Lo que hará que sean iguales a 0 en algunos casos.- Cuando tenemos muchas variables es difícil que todas sean entre sí independientes. Es decir, el problema que se da en la multicoleanidad es que no podríamos aplicar el modelo de regresión.
Cuando estamos en estos problemas tendremos que aplicar:
Procedementos de validación cruzada: Nuestro conjunto de datos lo dividimos en 2 trozos, un 80% y un 20%
El 80% seria el conjunto de entrenamiento
20% sería el conjunto de validación y de test
Ajustamos un modelo con solo el 80% de los datos -> la muestra de los datos de entrenamiento
Otro método que podríamos utilizar es:
Ej: tenemos 100 datos
Ajustamos el modelo con los 99 primeros datos.
En cada iteración tenemos que quitar uno de los datos de cada vez, así que miramos la predicción del modelo.
Al calcular la predicción media, en cada iteración vamos moviendo un registro.
Otro:
- Aplicando los kflolds.
- Quitamos k registros de cada vez
Metodo rapido para reducir variables-> estudiar las variables principales -> PCA
Cargamos el conjunto de datos
data(Prostate)
attach(Prostate)The following objects are masked from Prostate (pos = 3):
age, gleason, lbph, lcavol, lcp, lpsa, lweight, pgg45, svi
The following objects are masked from Prostate (pos = 4):
age, gleason, lbph, lcavol, lcp, lpsa, lweight, pgg45, svi
The following objects are masked from Prostate (pos = 5):
age, gleason, lbph, lcavol, lcp, lpsa, lweight, pgg45, svi
The following objects are masked from Prostate (pos = 11):
age, gleason, lbph, lcavol, lcp, lpsa, lweight, pgg45, sviVemos la forma y variables del conjunto de datos:
head(Prostate)plot(Prostate)
n <- nrow(Prostate)
p <- ncol(Prostate)
install.packages("Brq")WARNING: Rtools is required to build R packages but is not currently installed. Please download and install the appropriate version of Rtools before proceeding:
https://cran.rstudio.com/bin/windows/Rtools/Warning in install.packages :
package ‘Brq’ is in use and will not be installedn;p[1] 97
[1] 9install.packages("glmnet")WARNING: Rtools is required to build R packages but is not currently installed. Please download and install the appropriate version of Rtools before proceeding:
https://cran.rstudio.com/bin/windows/Rtools/Warning in install.packages :
package ‘glmnet’ is in use and will not be installedinstall.packages("knitr")Error in install.packages : Updating loaded packagesEs decir, tenemos 9 variables y 97 observaciones
names(Prostate)[1] "lcavol" "lweight" "age" "lbph" "svi" "lcp" "gleason"
[8] "pgg45" "lpsa" Disponemos así como variables explicativas de medidas como el volumen del tumor (lcavol), el log-peso de la próstata (lweight), la edad (age), la cantidad de hiperplasia prostática benigna (lbph), el grado de infiltración del tumor en la vesícula seminal (svi), el grado de penetración capsular (lcp) y el score de Gleason (gleason y pgg45).
Proponemos un modelo de regresión lineal para explicar la variable lpsa en función del resto de marcadores clínicos (p = 8)
\[ Y = \beta_0 + \beta_1 X_1 + \ldots + \beta_p X_p + \epsilon \]
modelo_completo = lm(lpsa ~ lcavol + lweight + age + lbph + svi + lcp + gleason + pgg45 )
modelo_completo
Call:
lm(formula = lpsa ~ lcavol + lweight + age + lbph + svi + lcp +
gleason + pgg45)
Coefficients:
(Intercept) lcavol lweight age lbph
0.669399 0.587023 0.454461 -0.019637 0.107054
svi lcp gleason pgg45
0.766156 -0.105474 0.045136 0.004525 Podemos obtener los coeficientes del modelo en base a la primera columa que se muestra en el summary del modelo:
summary(modelo_completo)
Call:
lm(formula = lpsa ~ lcavol + lweight + age + lbph + svi + lcp +
gleason + pgg45)
Residuals:
Min 1Q Median 3Q Max
-1.73316 -0.37133 -0.01702 0.41414 1.63811
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.669399 1.296381 0.516 0.60690
lcavol 0.587023 0.087920 6.677 2.11e-09 ***
lweight 0.454461 0.170012 2.673 0.00896 **
age -0.019637 0.011173 -1.758 0.08229 .
lbph 0.107054 0.058449 1.832 0.07040 .
svi 0.766156 0.244309 3.136 0.00233 **
lcp -0.105474 0.091013 -1.159 0.24964
gleason 0.045136 0.157464 0.287 0.77506
pgg45 0.004525 0.004421 1.024 0.30885
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.7084 on 88 degrees of freedom
Multiple R-squared: 0.6548, Adjusted R-squared: 0.6234
F-statistic: 20.86 on 8 and 88 DF, p-value: < 2.2e-16Por otra parte también podemos ver que las variables más significativas son: - lcavol -> 0.001 - lweight -> 0.01 - svi -> 0.01
sum <- summary(modelo_completo)
coeficiente_lpsa <- sum$coefficients[1]
coeficiente_lweight <- sum$coefficients[2]
coeficiente_age <- sum$coefficients[3]
coeficiente_lbph <- sum$coefficients[4]
coeficiente_svi <- sum$coefficients[5]
coeficiente_lcp <- sum$coefficients[6]
coeficiente_gleason <- sum$coefficients[7]
coeficiente_pgg45 <- sum$coefficients[8]
coeficiente_lpsa;[1] 0.669399coeficiente_lweight;[1] 0.5870229coeficiente_age;[1] 0.4544606coeficiente_lbph;[1] -0.01963721coeficiente_svi;[1] 0.1070544coeficiente_lcp;[1] 0.7661559coeficiente_gleason;[1] -0.1054736coeficiente_pgg45;[1] 0.04513596Ajustando el modelo usando el método de estimación de minimos cuadrados OSL manualmente:
Resolviendo el problema de optimización:
\[ \min_{\beta} \sum_{i=1}^{n} (y_i - \beta_0 - \beta_1 x_{i1} - \ldots - \beta_p x_{ip})^2 \]
X = matriz_diseño = model.matrix(modelo_completo)
X (Intercept) lcavol lweight age lbph svi lcp gleason
1 1 -0.579818495 2.769459 50 -1.38629436 0 -1.38629436 6
2 1 -0.994252273 3.319626 58 -1.38629436 0 -1.38629436 6
3 1 -0.510825624 2.691243 74 -1.38629436 0 -1.38629436 7
4 1 -1.203972804 3.282789 58 -1.38629436 0 -1.38629436 6
5 1 0.751416089 3.432373 62 -1.38629436 0 -1.38629436 6
6 1 -1.049822124 3.228826 50 -1.38629436 0 -1.38629436 6
7 1 0.737164066 3.473518 64 0.61518564 0 -1.38629436 6
8 1 0.693147181 3.539509 58 1.53686722 0 -1.38629436 6
9 1 -0.776528789 3.539509 47 -1.38629436 0 -1.38629436 6
10 1 0.223143551 3.244544 63 -1.38629436 0 -1.38629436 6
11 1 0.254642218 3.604138 65 -1.38629436 0 -1.38629436 6
12 1 -1.347073648 3.598681 63 1.26694760 0 -1.38629436 6
13 1 1.613429934 3.022861 63 -1.38629436 0 -0.59783700 7
14 1 1.477048724 2.998229 67 -1.38629436 0 -1.38629436 7
15 1 1.205970807 3.442019 57 -1.38629436 0 -0.43078292 7
16 1 1.541159072 3.061052 66 -1.38629436 0 -1.38629436 6
17 1 -0.415515444 3.516013 70 1.24415459 0 -0.59783700 7
18 1 2.288486169 3.649359 66 -1.38629436 0 0.37156356 6
19 1 -0.562118918 3.267666 41 -1.38629436 0 -1.38629436 6
20 1 0.182321557 3.825375 70 1.65822808 0 -1.38629436 6
21 1 1.147402453 3.419365 59 -1.38629436 0 -1.38629436 6
22 1 2.059238834 3.501043 60 1.47476301 0 1.34807315 7
23 1 -0.544727175 3.375880 59 -0.79850770 0 -1.38629436 6
24 1 1.781709133 3.451574 63 0.43825493 0 1.17865500 7
25 1 0.385262401 3.667400 69 1.59938758 0 -1.38629436 6
26 1 1.446918983 3.124565 68 0.30010459 0 -1.38629436 6
27 1 0.512823626 3.719651 65 -1.38629436 0 -0.79850770 7
28 1 -0.400477567 3.865979 67 1.81645208 0 -1.38629436 7
29 1 1.040276712 3.128951 67 0.22314355 0 0.04879016 7
30 1 2.409644165 3.375880 65 -1.38629436 0 1.61938824 6
31 1 0.285178942 4.090169 65 1.96290773 0 -0.79850770 6
32 1 0.182321557 6.107580 65 1.70474809 0 -1.38629436 6
33 1 1.275362800 3.037354 71 1.26694760 0 -1.38629436 6
34 1 0.009950331 3.267666 54 -1.38629436 0 -1.38629436 6
35 1 -0.010050336 3.216874 63 -1.38629436 0 -0.79850770 6
36 1 1.308332820 4.119850 64 2.17133681 0 -1.38629436 7
37 1 1.423108334 3.657131 73 -0.57981850 0 1.65822808 8
38 1 0.457424847 2.374906 64 -1.38629436 0 -1.38629436 7
39 1 2.660958594 4.085136 68 1.37371558 1 1.83258146 7
40 1 0.797507196 3.013081 56 0.93609336 0 -0.16251893 7
41 1 0.620576488 3.141995 60 -1.38629436 0 -1.38629436 9
42 1 1.442201993 3.682610 68 -1.38629436 0 -1.38629436 7
43 1 0.582215620 3.865979 62 1.71379793 0 -0.43078292 6
44 1 1.771556762 3.896909 61 -1.38629436 0 0.81093022 7
45 1 1.486139696 3.409496 66 1.74919985 0 -0.43078292 7
46 1 1.663926098 3.392829 61 0.61518564 0 -1.38629436 7
47 1 2.727852828 3.995445 79 1.87946505 1 2.65675691 9
48 1 1.163150810 4.035125 68 1.71379793 0 -0.43078292 7
49 1 1.745715531 3.498022 43 -1.38629436 0 -1.38629436 6
50 1 1.220829921 3.568123 70 1.37371558 0 -0.79850770 6
51 1 1.091923301 3.993603 68 -1.38629436 0 -1.38629436 7
52 1 1.660131027 4.234831 64 2.07317193 0 -1.38629436 6
53 1 0.512823626 3.633631 64 1.49290410 0 0.04879016 7
54 1 2.127040520 4.121473 68 1.76644166 0 1.44691898 7
55 1 3.153590358 3.516013 59 -1.38629436 0 -1.38629436 7
56 1 1.266947603 4.280132 66 2.12226154 0 -1.38629436 7
57 1 0.974559640 2.865054 47 -1.38629436 0 0.50077529 7
58 1 0.463734016 3.764682 49 1.42310833 0 -1.38629436 6
59 1 0.542324291 4.178226 70 0.43825493 0 -1.38629436 7
60 1 1.061256502 3.851211 61 1.29472717 0 -1.38629436 7
61 1 0.457424847 4.524502 73 2.32630162 0 -1.38629436 6
62 1 1.997417706 3.719651 63 1.61938824 1 1.90954250 7
63 1 2.775708850 3.524889 72 -1.38629436 0 1.55814462 9
64 1 2.034705648 3.917011 66 2.00821403 1 2.11021320 7
65 1 2.073171929 3.623007 64 -1.38629436 0 -1.38629436 6
66 1 1.458615023 3.836221 61 1.32175584 0 -0.43078292 7
67 1 2.022871190 3.878466 68 1.78339122 0 1.32175584 7
68 1 2.198335072 4.050915 72 2.30757263 0 -0.43078292 7
69 1 -0.446287103 4.408547 69 -1.38629436 0 -1.38629436 6
70 1 1.193922468 4.780383 72 2.32630162 0 -0.79850770 7
71 1 1.864080131 3.593194 60 -1.38629436 1 1.32175584 7
72 1 1.160020917 3.341093 77 1.74919985 0 -1.38629436 7
73 1 1.214912744 3.825375 69 -1.38629436 1 0.22314355 7
74 1 1.838961071 3.236716 60 0.43825493 1 1.17865500 9
75 1 2.999226163 3.849083 69 -1.38629436 1 1.90954250 7
76 1 3.141130476 3.263849 68 -0.05129329 1 2.42036813 7
77 1 2.010894999 4.433789 72 2.12226154 0 0.50077529 7
78 1 2.537657215 4.354784 78 2.32630162 0 -1.38629436 7
79 1 2.648300197 3.582129 69 -1.38629436 1 2.58399755 7
80 1 2.779440197 3.823192 63 -1.38629436 0 0.37156356 7
81 1 1.467874348 3.070376 66 0.55961579 0 0.22314355 7
82 1 2.513656063 3.473518 57 0.43825493 0 2.32727771 7
83 1 2.613006652 3.888754 77 -0.52763274 1 0.55961579 7
84 1 2.677590994 3.838376 65 1.11514159 0 1.74919985 9
85 1 1.562346305 3.709907 60 1.69561561 0 0.81093022 7
86 1 3.302849259 3.518980 64 -1.38629436 1 2.32727771 7
87 1 2.024193067 3.731699 58 1.63899671 0 -1.38629436 6
88 1 1.731655545 3.369018 62 -1.38629436 1 0.30010459 7
89 1 2.807593831 4.718052 65 -1.38629436 1 2.46385324 7
90 1 1.562346305 3.695110 76 0.93609336 1 0.81093022 7
91 1 3.246490992 4.101817 68 -1.38629436 0 -1.38629436 6
92 1 2.532902848 3.677566 61 1.34807315 1 -1.38629436 7
93 1 2.830267834 3.876396 68 -1.38629436 1 1.32175584 7
94 1 3.821003607 3.896909 44 -1.38629436 1 2.16905370 7
95 1 2.907447359 3.396185 52 -1.38629436 1 2.46385324 7
96 1 2.882563575 3.773910 68 1.55814462 1 1.55814462 7
97 1 3.471966453 3.974998 68 0.43825493 1 2.90416508 7
pgg45
1 0
2 0
3 20
4 0
5 0
6 0
7 0
8 0
9 0
10 0
11 0
12 0
13 30
14 5
15 5
16 0
17 30
18 0
19 0
20 0
21 0
22 20
23 0
24 60
25 0
26 0
27 70
28 20
29 80
30 0
31 0
32 0
33 0
34 0
35 0
36 5
37 15
38 15
39 35
40 5
41 80
42 10
43 0
44 6
45 20
46 15
47 100
48 40
49 0
50 0
51 50
52 0
53 70
54 40
55 5
56 15
57 4
58 0
59 20
60 40
61 0
62 40
63 95
64 60
65 0
66 20
67 70
68 10
69 0
70 5
71 60
72 25
73 20
74 90
75 20
76 50
77 60
78 10
79 70
80 50
81 40
82 60
83 30
84 70
85 30
86 60
87 0
88 30
89 60
90 75
91 0
92 15
93 60
94 40
95 10
96 80
97 20
attr(,"assign")
[1] 0 1 2 3 4 5 6 7 8install.packages("knitr")WARNING: Rtools is required to build R packages but is not currently installed. Please download and install the appropriate version of Rtools before proceeding:
https://cran.rstudio.com/bin/windows/Rtools/Warning in install.packages :
package ‘knitr’ is in use and will not be installedmatriz_hat = X %*% solve(t(X) %*% X) %*% t(X)
H = matriz_hatbeta_hat <- solve(t(X) %*% H %*% X) %*% t(X) %*% H %*% Y
beta_hat [,1]
(Intercept) 0.669399027
lcavol 0.587022881
lweight 0.454460641
age -0.019637208
lbph 0.107054351
svi 0.766155885
lcp -0.105473570
gleason 0.045135964
pgg45 0.004525324Coeficientes del modelo completo
O también podemos obtener los coeficientes de la siguiente forma:
coef(modelo_completo) (Intercept) lcavol lweight age lbph
0.669399027 0.587022881 0.454460641 -0.019637208 0.107054351
svi lcp gleason pgg45
0.766155885 -0.105473570 0.045135964 0.004525324 kable(coef(modelo_completo), format='markdown')| x | |
|---|---|
| (Intercept) | 0.6693990 |
| lcavol | 0.5870229 |
| lweight | 0.4544606 |
| age | -0.0196372 |
| lbph | 0.1070544 |
| svi | 0.7661559 |
| lcp | -0.1054736 |
| gleason | 0.0451360 |
| pgg45 | 0.0045253 |
Estimación ridge
\[ \min_{\beta} \sum_{i=1}^{n} (y_i - \beta_0 - \beta_1 x_{i1} - \ldots - \beta_p x_{ip})^2 + \lambda \sum_{j=1}^{p} \beta_j^2 \] Si establecemos lambda como 0, el valor de los coeficientes sigue siendo el mismo que con el problema original:
Y_hat <- H %*% Y
lambda <- 0
beta_ridge <- solve(t(X) %*% H %*% X + lambda * diag(ncol(X))) %*% t(X) %*% H %*% Y
print(beta_ridge) [,1]
(Intercept) 0.669399027
lcavol 0.587022881
lweight 0.454460641
age -0.019637208
lbph 0.107054351
svi 0.766155885
lcp -0.105473570
gleason 0.045135964
pgg45 0.004525324coef(modelo_completo) (Intercept) lcavol lweight age lbph
0.669399027 0.587022881 0.454460641 -0.019637208 0.107054351
svi lcp gleason pgg45
0.766155885 -0.105473570 0.045135964 0.004525324 Podemos ver que son los mismos coeficientes.
En cambio si variamos el valor de lambda, podemos ver cambios notables en los coeficientes:
Y_hat <- H %*% Y
lambda <- 0.1
beta_ridge <- solve(t(X) %*% H %*% X + lambda * diag(ncol(X))) %*% t(X) %*% H %*% Y
print(beta_ridge) [,1]
(Intercept) 0.510054886
lcavol 0.584554329
lweight 0.463101551
age -0.019031716
lbph 0.104386625
svi 0.759517901
lcp -0.104030516
gleason 0.060002691
pgg45 0.004258437Ridge automaticamente
Primero necesitamos convertir los datos de Prostate en una matriz con el comando as.matrix()
matrix_prostate <- as.matrix(Prostate[c(1:8)])
matrix_prostate lcavol lweight age lbph svi lcp gleason pgg45
[1,] -0.579818495 2.769459 50 -1.38629436 0 -1.38629436 6 0
[2,] -0.994252273 3.319626 58 -1.38629436 0 -1.38629436 6 0
[3,] -0.510825624 2.691243 74 -1.38629436 0 -1.38629436 7 20
[4,] -1.203972804 3.282789 58 -1.38629436 0 -1.38629436 6 0
[5,] 0.751416089 3.432373 62 -1.38629436 0 -1.38629436 6 0
[6,] -1.049822124 3.228826 50 -1.38629436 0 -1.38629436 6 0
[7,] 0.737164066 3.473518 64 0.61518564 0 -1.38629436 6 0
[8,] 0.693147181 3.539509 58 1.53686722 0 -1.38629436 6 0
[9,] -0.776528789 3.539509 47 -1.38629436 0 -1.38629436 6 0
[10,] 0.223143551 3.244544 63 -1.38629436 0 -1.38629436 6 0
[11,] 0.254642218 3.604138 65 -1.38629436 0 -1.38629436 6 0
[12,] -1.347073648 3.598681 63 1.26694760 0 -1.38629436 6 0
[13,] 1.613429934 3.022861 63 -1.38629436 0 -0.59783700 7 30
[14,] 1.477048724 2.998229 67 -1.38629436 0 -1.38629436 7 5
[15,] 1.205970807 3.442019 57 -1.38629436 0 -0.43078292 7 5
[16,] 1.541159072 3.061052 66 -1.38629436 0 -1.38629436 6 0
[17,] -0.415515444 3.516013 70 1.24415459 0 -0.59783700 7 30
[18,] 2.288486169 3.649359 66 -1.38629436 0 0.37156356 6 0
[19,] -0.562118918 3.267666 41 -1.38629436 0 -1.38629436 6 0
[20,] 0.182321557 3.825375 70 1.65822808 0 -1.38629436 6 0
[21,] 1.147402453 3.419365 59 -1.38629436 0 -1.38629436 6 0
[22,] 2.059238834 3.501043 60 1.47476301 0 1.34807315 7 20
[23,] -0.544727175 3.375880 59 -0.79850770 0 -1.38629436 6 0
[24,] 1.781709133 3.451574 63 0.43825493 0 1.17865500 7 60
[25,] 0.385262401 3.667400 69 1.59938758 0 -1.38629436 6 0
[26,] 1.446918983 3.124565 68 0.30010459 0 -1.38629436 6 0
[27,] 0.512823626 3.719651 65 -1.38629436 0 -0.79850770 7 70
[28,] -0.400477567 3.865979 67 1.81645208 0 -1.38629436 7 20
[29,] 1.040276712 3.128951 67 0.22314355 0 0.04879016 7 80
[30,] 2.409644165 3.375880 65 -1.38629436 0 1.61938824 6 0
[31,] 0.285178942 4.090169 65 1.96290773 0 -0.79850770 6 0
[32,] 0.182321557 6.107580 65 1.70474809 0 -1.38629436 6 0
[33,] 1.275362800 3.037354 71 1.26694760 0 -1.38629436 6 0
[34,] 0.009950331 3.267666 54 -1.38629436 0 -1.38629436 6 0
[35,] -0.010050336 3.216874 63 -1.38629436 0 -0.79850770 6 0
[36,] 1.308332820 4.119850 64 2.17133681 0 -1.38629436 7 5
[37,] 1.423108334 3.657131 73 -0.57981850 0 1.65822808 8 15
[38,] 0.457424847 2.374906 64 -1.38629436 0 -1.38629436 7 15
[39,] 2.660958594 4.085136 68 1.37371558 1 1.83258146 7 35
[40,] 0.797507196 3.013081 56 0.93609336 0 -0.16251893 7 5
[41,] 0.620576488 3.141995 60 -1.38629436 0 -1.38629436 9 80
[42,] 1.442201993 3.682610 68 -1.38629436 0 -1.38629436 7 10
[43,] 0.582215620 3.865979 62 1.71379793 0 -0.43078292 6 0
[44,] 1.771556762 3.896909 61 -1.38629436 0 0.81093022 7 6
[45,] 1.486139696 3.409496 66 1.74919985 0 -0.43078292 7 20
[46,] 1.663926098 3.392829 61 0.61518564 0 -1.38629436 7 15
[47,] 2.727852828 3.995445 79 1.87946505 1 2.65675691 9 100
[48,] 1.163150810 4.035125 68 1.71379793 0 -0.43078292 7 40
[49,] 1.745715531 3.498022 43 -1.38629436 0 -1.38629436 6 0
[50,] 1.220829921 3.568123 70 1.37371558 0 -0.79850770 6 0
[51,] 1.091923301 3.993603 68 -1.38629436 0 -1.38629436 7 50
[52,] 1.660131027 4.234831 64 2.07317193 0 -1.38629436 6 0
[53,] 0.512823626 3.633631 64 1.49290410 0 0.04879016 7 70
[54,] 2.127040520 4.121473 68 1.76644166 0 1.44691898 7 40
[55,] 3.153590358 3.516013 59 -1.38629436 0 -1.38629436 7 5
[56,] 1.266947603 4.280132 66 2.12226154 0 -1.38629436 7 15
[57,] 0.974559640 2.865054 47 -1.38629436 0 0.50077529 7 4
[58,] 0.463734016 3.764682 49 1.42310833 0 -1.38629436 6 0
[59,] 0.542324291 4.178226 70 0.43825493 0 -1.38629436 7 20
[60,] 1.061256502 3.851211 61 1.29472717 0 -1.38629436 7 40
[61,] 0.457424847 4.524502 73 2.32630162 0 -1.38629436 6 0
[62,] 1.997417706 3.719651 63 1.61938824 1 1.90954250 7 40
[63,] 2.775708850 3.524889 72 -1.38629436 0 1.55814462 9 95
[64,] 2.034705648 3.917011 66 2.00821403 1 2.11021320 7 60
[65,] 2.073171929 3.623007 64 -1.38629436 0 -1.38629436 6 0
[66,] 1.458615023 3.836221 61 1.32175584 0 -0.43078292 7 20
[67,] 2.022871190 3.878466 68 1.78339122 0 1.32175584 7 70
[68,] 2.198335072 4.050915 72 2.30757263 0 -0.43078292 7 10
[69,] -0.446287103 4.408547 69 -1.38629436 0 -1.38629436 6 0
[70,] 1.193922468 4.780383 72 2.32630162 0 -0.79850770 7 5
[71,] 1.864080131 3.593194 60 -1.38629436 1 1.32175584 7 60
[72,] 1.160020917 3.341093 77 1.74919985 0 -1.38629436 7 25
[73,] 1.214912744 3.825375 69 -1.38629436 1 0.22314355 7 20
[74,] 1.838961071 3.236716 60 0.43825493 1 1.17865500 9 90
[75,] 2.999226163 3.849083 69 -1.38629436 1 1.90954250 7 20
[76,] 3.141130476 3.263849 68 -0.05129329 1 2.42036813 7 50
[77,] 2.010894999 4.433789 72 2.12226154 0 0.50077529 7 60
[78,] 2.537657215 4.354784 78 2.32630162 0 -1.38629436 7 10
[79,] 2.648300197 3.582129 69 -1.38629436 1 2.58399755 7 70
[80,] 2.779440197 3.823192 63 -1.38629436 0 0.37156356 7 50
[81,] 1.467874348 3.070376 66 0.55961579 0 0.22314355 7 40
[82,] 2.513656063 3.473518 57 0.43825493 0 2.32727771 7 60
[83,] 2.613006652 3.888754 77 -0.52763274 1 0.55961579 7 30
[84,] 2.677590994 3.838376 65 1.11514159 0 1.74919985 9 70
[85,] 1.562346305 3.709907 60 1.69561561 0 0.81093022 7 30
[86,] 3.302849259 3.518980 64 -1.38629436 1 2.32727771 7 60
[87,] 2.024193067 3.731699 58 1.63899671 0 -1.38629436 6 0
[88,] 1.731655545 3.369018 62 -1.38629436 1 0.30010459 7 30
[89,] 2.807593831 4.718052 65 -1.38629436 1 2.46385324 7 60
[90,] 1.562346305 3.695110 76 0.93609336 1 0.81093022 7 75
[91,] 3.246490992 4.101817 68 -1.38629436 0 -1.38629436 6 0
[92,] 2.532902848 3.677566 61 1.34807315 1 -1.38629436 7 15
[93,] 2.830267834 3.876396 68 -1.38629436 1 1.32175584 7 60
[94,] 3.821003607 3.896909 44 -1.38629436 1 2.16905370 7 40
[95,] 2.907447359 3.396185 52 -1.38629436 1 2.46385324 7 10
[96,] 2.882563575 3.773910 68 1.55814462 1 1.55814462 7 80
[97,] 3.471966453 3.974998 68 0.43825493 1 2.90416508 7 20Una vez hecha la transformación procedemos a hacer el modelo con los datos.
rr <- lm.ridge(lpsa ~ matrix_prostate, lambda = 0)A continuación procedemos a obtener los coeficientes del modelo
coef(rr) matrix_prostatelcavol matrix_prostatelweight
0.669399027 0.587022881 0.454460641
matrix_prostateage matrix_prostatelbph matrix_prostatesvi
-0.019637208 0.107054351 0.766155885
matrix_prostatelcp matrix_prostategleason matrix_prostatepgg45
-0.105473570 0.045135964 0.004525324 Podemos ver que estos coeficientes coinciden con los obtenidos con el método de mínimos cuadrados.
Por otra parte, podemos calcular también los coeficientes estandarizados:
rr$coef matrix_prostatelcavol matrix_prostatelweight matrix_prostateage
0.68830414 0.22453268 -0.14544574
matrix_prostatelbph matrix_prostatesvi matrix_prostatelcp
0.15451249 0.31554540 -0.14671621
matrix_prostategleason matrix_prostatepgg45
0.03242577 0.12697278 Estos últimos coeficientes obtenidos coincidirían con los resultados de ajustar el modelo de regresión lineal por el método de mínimos cuadrados a los datos que se obtienen tras restar a cada columna de la matriz x su media y dividir entre la desviación típica.
Podemos demostrarlo de la siguiente forma:
Primero escalamos la matriz
matrix_prostate_escalada <- scale(matrix_prostate)
matrix_prostate_escalada lcavol lweight age lbph svi
[1,] -1.63735563 -1.778439445 -1.86242597 -1.02470580 -0.5229409
[2,] -1.98898046 -0.670640550 -0.78789619 -1.02470580 -0.5229409
[3,] -1.57881888 -1.935932230 1.36116337 -1.02470580 -0.5229409
[4,] -2.16691708 -0.744813533 -0.78789619 -1.02470580 -0.5229409
[5,] -0.50787448 -0.443616164 -0.25063130 -1.02470580 -0.5229409
[6,] -2.03612850 -0.853471735 -1.86242597 -1.02470580 -0.5229409
[7,] -0.51996655 -0.360767788 0.01800114 0.35485779 -0.5229409
[8,] -0.55731252 -0.227890462 -0.78789619 0.99014685 -0.5229409
[9,] -1.80425376 -0.227890462 -2.26537464 -1.02470580 -0.5229409
[10,] -0.95608539 -0.821823637 -0.11631508 -1.02470580 -0.5229409
[11,] -0.92936046 -0.097755066 0.15231737 -1.02470580 -0.5229409
[12,] -2.28833048 -0.108743190 -0.11631508 0.80409889 -0.5229409
[13,] 0.22349804 -1.268196862 -0.11631508 -1.02470580 -0.5229409
[14,] 0.10778590 -1.317794660 0.42094981 -1.02470580 -0.5229409
[15,] -0.12220917 -0.424192521 -0.92221241 -1.02470580 -0.5229409
[16,] 0.16218009 -1.191297062 0.28663359 -1.02470580 -0.5229409
[17,] -1.49795330 -0.275201156 0.82389848 0.78838831 -0.5229409
[18,] 0.79624706 -0.006700538 0.28663359 -1.02470580 -0.5229409
[19,] -1.62233848 -0.775265059 -3.07127198 -1.02470580 -0.5229409
[20,] -0.99072067 0.347720798 0.82389848 1.07379746 -0.5229409
[21,] -0.17190128 -0.469809298 -0.65357997 -1.02470580 -0.5229409
[22,] 0.60174299 -0.305344965 -0.51926375 0.94734018 -0.5229409
[23,] -1.60758252 -0.557369562 -0.65357997 -0.61956107 -0.5229409
[24,] 0.36627392 -0.404954456 -0.11631508 0.23290445 -0.5229409
[25,] -0.81853624 0.029627719 0.68958226 1.03324037 -0.5229409
[26,] 0.08222243 -1.063408451 0.55526603 0.13768133 -0.5229409
[27,] -0.71030738 0.134838078 0.15231737 -1.02470580 -0.5229409
[28,] -1.48519447 0.429479479 0.42094981 1.18285680 -0.5229409
[29,] -0.26279173 -1.054576994 0.42094981 0.08463426 -0.5229409
[30,] 0.89904312 -0.557369562 0.15231737 -1.02470580 -0.5229409
[31,] -0.90345169 0.880901715 0.15231737 1.28380453 -0.5229409
[32,] -0.99072067 4.943096194 0.15231737 1.10586239 -0.5229409
[33,] -0.06333379 -1.239014195 0.95821470 0.80409889 -0.5229409
[34,] -1.13696841 -0.775265059 -1.32516108 -1.02470580 -0.5229409
[35,] -1.15393790 -0.877538580 -0.11631508 -1.02470580 -0.5229409
[36,] -0.03536050 0.940665769 0.01800114 1.42746880 -0.5229409
[37,] 0.06202037 0.008949039 1.22684715 -0.46882478 -0.5229409
[38,] -0.75731028 -2.572899200 0.01800114 -1.02470580 -0.5229409
[39,] 1.11226993 0.870766280 0.55526603 0.87769104 1.8925480
[40,] -0.46876864 -1.287889623 -1.05652864 0.57605041 -0.5229409
[41,] -0.61888486 -1.028312681 -0.51926375 -1.02470580 -0.5229409
[42,] 0.07822032 0.060252930 0.55526603 -1.02470580 -0.5229409
[43,] -0.65143200 0.429479479 -0.25063130 1.11210019 -0.5229409
[44,] 0.35766018 0.491759767 -0.38494752 -1.02470580 -0.5229409
[45,] 0.11549910 -0.489680204 0.28663359 1.13650174 -0.5229409
[46,] 0.26634133 -0.523240461 -0.38494752 0.35485779 -0.5229409
[47,] 1.16902610 0.690167263 2.03274448 1.22628985 1.8925480
[48,] -0.15853965 0.770066858 0.55526603 1.11210019 -0.5229409
[49,] 0.33573528 -0.311428259 -2.80263953 -1.02470580 -0.5229409
[50,] -0.10960201 -0.170273690 0.82389848 0.87769104 -0.5229409
[51,] -0.21897237 0.686459032 0.55526603 -1.02470580 -0.5229409
[52,] 0.26312142 1.172187976 0.01800114 1.35980653 -0.5229409
[53,] -0.71030738 -0.038369376 0.01800114 0.95984431 -0.5229409
[54,] 0.65926908 0.943934550 0.55526603 1.14838603 -0.5229409
[55,] 1.53024156 -0.275201156 -0.65357997 -1.02470580 -0.5229409
[56,] -0.07047363 1.263405556 0.28663359 1.39364261 -0.5229409
[57,] -0.31854914 -1.585952094 -2.26537464 -1.02470580 -0.5229409
[58,] -0.75195729 0.225511708 -1.99674220 0.91173607 -0.5229409
[59,] -0.68527765 1.058210250 0.82389848 0.23290445 -0.5229409
[60,] -0.24499150 0.399742691 -0.38494752 0.82324656 -0.5229409
[61,] -0.75731028 1.755461272 1.22684715 1.53428167 -0.5229409
[62,] 0.54929107 0.134838078 -0.11631508 1.04702626 1.8925480
[63,] 1.20962937 -0.257329127 1.09253092 -1.02470580 -0.5229409
[64,] 0.58092789 0.532234874 0.28663359 1.31503289 1.8925480
[65,] 0.61356447 -0.059762075 0.01800114 -1.02470580 -0.5229409
[66,] 0.09214589 0.369560153 -0.38494752 0.84187666 -0.5229409
[67,] 0.57088699 0.454623225 0.55526603 1.16006888 -0.5229409
[68,] 0.71975868 0.801860710 1.09253092 1.52137231 -0.5229409
[69,] -1.52406140 1.521977020 0.68958226 -1.02470580 -0.5229409
[70,] -0.13243154 2.270695107 1.09253092 1.53428167 -0.5229409
[71,] 0.43616129 -0.119791606 -0.51926375 -1.02470580 1.8925480
[72,] -0.16119519 -0.627413800 1.76411204 1.13650174 -0.5229409
[73,] -0.11462242 0.347720798 0.68958226 -1.02470580 1.8925480
[74,] 0.41484912 -0.837585508 -0.51926375 0.23290445 1.8925480
[75,] 1.39927182 0.395458500 0.68958226 -1.02470580 1.8925480
[76,] 1.51967002 -0.782950445 0.55526603 -0.10452730 1.8925480
[77,] 0.56072583 1.572802969 1.09253092 1.39364261 -0.5229409
[78,] 1.00765533 1.413721630 1.89842826 1.53428167 -0.5229409
[79,] 1.10152996 -0.142071986 0.68958226 -1.02470580 1.8925480
[80,] 1.21279522 0.343324357 -0.11631508 -1.02470580 -0.5229409
[81,] 0.10000193 -1.172522392 0.28663359 0.31655506 -0.5229409
[82,] 0.98729163 -0.360767788 -0.92221241 0.23290445 -0.5229409
[83,] 1.07158528 0.475339136 1.76411204 -0.43285462 1.8925480
[84,] 1.12638163 0.373899742 0.15231737 0.69946330 -0.5229409
[85,] 0.18015632 0.115217277 -0.51926375 1.09956763 -0.5229409
[86,] 1.65687974 -0.269226170 0.01800114 -1.02470580 1.8925480
[87,] 0.57200853 0.159098237 -0.78789619 1.06054183 -0.5229409
[88,] 0.32380613 -0.571184840 -0.25063130 -1.02470580 1.8925480
[89,] 1.23668207 2.145187631 0.15231737 -1.02470580 1.8925480
[90,] 0.18015632 0.085422867 1.62979581 0.57605041 1.8925480
[91,] 1.60906277 0.904354528 0.55526603 -1.02470580 -0.5229409
[92,] 1.00362150 0.050096193 -0.38494752 0.86001644 1.8925480
[93,] 1.25591974 0.450454344 0.55526603 -1.02470580 1.8925480
[94,] 2.09650591 0.491759767 -2.66832331 -1.02470580 1.8925480
[95,] 1.32140243 -0.516483512 -1.59379353 -1.02470580 1.8925480
[96,] 1.30028987 0.244091482 0.55526603 1.00481276 1.8925480
[97,] 1.80036661 0.648996206 0.55526603 0.23290445 1.8925480
lcp gleason pgg45
[1,] -0.86317119 -1.0421573 -0.8644665
[2,] -0.86317119 -1.0421573 -0.8644665
[3,] -0.86317119 0.3426271 -0.1553481
[4,] -0.86317119 -1.0421573 -0.8644665
[5,] -0.86317119 -1.0421573 -0.8644665
[6,] -0.86317119 -1.0421573 -0.8644665
[7,] -0.86317119 -1.0421573 -0.8644665
[8,] -0.86317119 -1.0421573 -0.8644665
[9,] -0.86317119 -1.0421573 -0.8644665
[10,] -0.86317119 -1.0421573 -0.8644665
[11,] -0.86317119 -1.0421573 -0.8644665
[12,] -0.86317119 -1.0421573 -0.8644665
[13,] -0.29928234 0.3426271 0.1992111
[14,] -0.86317119 0.3426271 -0.6871869
[15,] -0.17980862 0.3426271 -0.6871869
[16,] -0.86317119 -1.0421573 -0.8644665
[17,] -0.29928234 0.3426271 0.1992111
[18,] 0.39401343 -1.0421573 -0.8644665
[19,] -0.86317119 -1.0421573 -0.8644665
[20,] -0.86317119 -1.0421573 -0.8644665
[21,] -0.86317119 -1.0421573 -0.8644665
[22,] 1.09239345 0.3426271 -0.1553481
[23,] -0.86317119 -1.0421573 -0.8644665
[24,] 0.97122899 0.3426271 1.2628887
[25,] -0.86317119 -1.0421573 -0.8644665
[26,] -0.86317119 -1.0421573 -0.8644665
[27,] -0.44279799 0.3426271 1.6174479
[28,] -0.86317119 0.3426271 -0.1553481
[29,] 0.16317240 0.3426271 1.9720071
[30,] 1.28643255 -1.0421573 -0.8644665
[31,] -0.44279799 -1.0421573 -0.8644665
[32,] -0.86317119 -1.0421573 -0.8644665
[33,] -0.86317119 -1.0421573 -0.8644665
[34,] -0.86317119 -1.0421573 -0.8644665
[35,] -0.44279799 -1.0421573 -0.8644665
[36,] -0.86317119 0.3426271 -0.6871869
[37,] 1.31421001 1.7274114 -0.3326277
[38,] -0.86317119 0.3426271 -0.3326277
[39,] 1.43890405 0.3426271 0.3764907
[40,] 0.01204838 0.3426271 -0.6871869
[41,] -0.86317119 3.1121957 1.9720071
[42,] -0.86317119 0.3426271 -0.5099073
[43,] -0.17980862 -1.0421573 -0.8644665
[44,] 0.70823963 0.3426271 -0.6517310
[45,] -0.17980862 0.3426271 -0.1553481
[46,] -0.86317119 0.3426271 -0.3326277
[47,] 2.02833775 3.1121957 2.6811255
[48,] -0.17980862 0.3426271 0.5537703
[49,] -0.86317119 -1.0421573 -0.8644665
[50,] -0.44279799 -1.0421573 -0.8644665
[51,] -0.86317119 0.3426271 0.9083295
[52,] -0.86317119 -1.0421573 -0.8644665
[53,] 0.16317240 0.3426271 1.6174479
[54,] 1.16308600 0.3426271 0.5537703
[55,] -0.86317119 0.3426271 -0.6871869
[56,] -0.86317119 0.3426271 -0.3326277
[57,] 0.48642306 0.3426271 -0.7226428
[58,] -0.86317119 -1.0421573 -0.8644665
[59,] -0.86317119 0.3426271 -0.1553481
[60,] -0.86317119 0.3426271 0.5537703
[61,] -0.86317119 -1.0421573 -0.8644665
[62,] 1.49394504 0.3426271 0.5537703
[63,] 1.24263234 3.1121957 2.5038459
[64,] 1.63746068 0.3426271 1.2628887
[65,] -0.86317119 -1.0421573 -0.8644665
[66,] -0.17980862 0.3426271 -0.1553481
[67,] 1.07357184 0.3426271 1.6174479
[68,] -0.17980862 0.3426271 -0.5099073
[69,] -0.86317119 -1.0421573 -0.8644665
[70,] -0.44279799 0.3426271 -0.6871869
[71,] 1.07357184 0.3426271 1.2628887
[72,] -0.86317119 0.3426271 0.0219315
[73,] 0.28786643 0.3426271 -0.1553481
[74,] 0.97122899 3.1121957 2.3265663
[75,] 1.49394504 0.3426271 -0.1553481
[76,] 1.85927725 0.3426271 0.9083295
[77,] 0.48642306 0.3426271 1.2628887
[78,] -0.86317119 0.3426271 -0.5099073
[79,] 1.97630172 0.3426271 1.6174479
[80,] 0.39401343 0.3426271 0.9083295
[81,] 0.28786643 0.3426271 0.5537703
[82,] 1.79270085 0.3426271 1.2628887
[83,] 0.52850461 0.3426271 0.1992111
[84,] 1.37927120 3.1121957 1.6174479
[85,] 0.70823963 0.3426271 0.1992111
[86,] 1.79270085 0.3426271 1.2628887
[87,] -0.86317119 -1.0421573 -0.8644665
[88,] 0.34290742 0.3426271 0.1992111
[89,] 1.89037692 0.3426271 1.2628887
[90,] 0.70823963 0.3426271 1.7947275
[91,] -0.86317119 -1.0421573 -0.8644665
[92,] -0.86317119 0.3426271 -0.3326277
[93,] 1.07357184 0.3426271 1.2628887
[94,] 1.67954222 0.3426271 0.5537703
[95,] 1.89037692 0.3426271 -0.5099073
[96,] 1.24263234 0.3426271 1.9720071
[97,] 2.20527909 0.3426271 -0.1553481
attr(,"scaled:center")
lcavol lweight age lbph svi lcp gleason
1.3500096 3.6526864 63.8659794 0.1003556 0.2164948 -0.1793656 6.7525773
pgg45
24.3814433
attr(,"scaled:scale")
lcavol lweight age lbph svi lcp gleason
1.1786249 0.4966307 7.4451171 1.4508066 0.4139949 1.3982496 0.7221341
pgg45
28.2040346 Generamos el modelo de regresión en base a la matriz escalada:
mod_escalado <- lm(lpsa ~ matrix_prostate_escalada)coefficients(mod_escalado) (Intercept) matrix_prostate_escaladalcavol
2.47838688 0.69187977
matrix_prostate_escaladalweight matrix_prostate_escaladaage
0.22569909 -0.14620131
matrix_prostate_escaladalbph matrix_prostate_escaladasvi
0.15531516 0.31718461
matrix_prostate_escaladalcp matrix_prostate_escaladagleason
-0.14747838 0.03259422
matrix_prostate_escaladapgg45
0.12763238 Ajuste mediante el procedimiento de estimación Ridge
Usamos diferentes valores del parametro lambda:
lam <- seq(0, 10, by = 0.01)Y a continuación generamos el modelo:
rr <- lm.ridge(lpsa ~ matrix_prostate, lambda = lam)Mostramos los coeficientes:
coef(rr) matrix_prostatelcavol matrix_prostatelweight
0.00 0.6693990 0.5870229 0.4544606
0.01 0.6689995 0.5868907 0.4544480
0.02 0.6686006 0.5867586 0.4544353
0.03 0.6682021 0.5866266 0.4544226
0.04 0.6678042 0.5864946 0.4544098
0.05 0.6674068 0.5863628 0.4543971
0.06 0.6670099 0.5862311 0.4543843
0.07 0.6666135 0.5860995 0.4543715
0.08 0.6662177 0.5859680 0.4543587
0.09 0.6658224 0.5858365 0.4543459
0.10 0.6654276 0.5857052 0.4543330
0.11 0.6650333 0.5855739 0.4543202
0.12 0.6646395 0.5854428 0.4543073
0.13 0.6642463 0.5853117 0.4542944
0.14 0.6638535 0.5851808 0.4542815
0.15 0.6634613 0.5850499 0.4542686
0.16 0.6630696 0.5849192 0.4542556
0.17 0.6626784 0.5847885 0.4542427
0.18 0.6622877 0.5846579 0.4542297
0.19 0.6618975 0.5845274 0.4542167
0.20 0.6615078 0.5843970 0.4542037
0.21 0.6611186 0.5842667 0.4541906
0.22 0.6607299 0.5841365 0.4541776
0.23 0.6603418 0.5840064 0.4541645
0.24 0.6599541 0.5838764 0.4541514
0.25 0.6595670 0.5837465 0.4541383
0.26 0.6591803 0.5836167 0.4541252
0.27 0.6587942 0.5834869 0.4541120
0.28 0.6584085 0.5833573 0.4540989
0.29 0.6580234 0.5832278 0.4540857
0.30 0.6576387 0.5830983 0.4540725
0.31 0.6572546 0.5829689 0.4540593
0.32 0.6568709 0.5828397 0.4540461
0.33 0.6564878 0.5827105 0.4540328
0.34 0.6561051 0.5825814 0.4540196
0.35 0.6557229 0.5824524 0.4540063
0.36 0.6553413 0.5823235 0.4539930
0.37 0.6549601 0.5821947 0.4539797
0.38 0.6545794 0.5820660 0.4539663
0.39 0.6541992 0.5819374 0.4539530
0.40 0.6538195 0.5818089 0.4539396
0.41 0.6534403 0.5816804 0.4539263
0.42 0.6530616 0.5815521 0.4539129
0.43 0.6526833 0.5814238 0.4538994
0.44 0.6523056 0.5812957 0.4538860
0.45 0.6519283 0.5811676 0.4538726
0.46 0.6515515 0.5810396 0.4538591
0.47 0.6511752 0.5809117 0.4538456
0.48 0.6507994 0.5807839 0.4538321
0.49 0.6504241 0.5806562 0.4538186
0.50 0.6500492 0.5805286 0.4538051
0.51 0.6496749 0.5804011 0.4537915
0.52 0.6493010 0.5802737 0.4537780
0.53 0.6489276 0.5801463 0.4537644
0.54 0.6485546 0.5800191 0.4537508
0.55 0.6481822 0.5798919 0.4537372
0.56 0.6478102 0.5797648 0.4537235
0.57 0.6474387 0.5796378 0.4537099
0.58 0.6470677 0.5795109 0.4536962
0.59 0.6466971 0.5793841 0.4536825
0.60 0.6463271 0.5792574 0.4536688
0.61 0.6459575 0.5791308 0.4536551
0.62 0.6455883 0.5790043 0.4536414
0.63 0.6452197 0.5788778 0.4536277
0.64 0.6448515 0.5787515 0.4536139
0.65 0.6444837 0.5786252 0.4536001
0.66 0.6441165 0.5784990 0.4535863
0.67 0.6437497 0.5783729 0.4535725
0.68 0.6433834 0.5782469 0.4535587
0.69 0.6430175 0.5781210 0.4535448
0.70 0.6426521 0.5779952 0.4535310
0.71 0.6422872 0.5778695 0.4535171
0.72 0.6419228 0.5777438 0.4535032
0.73 0.6415588 0.5776183 0.4534893
0.74 0.6411952 0.5774928 0.4534754
0.75 0.6408322 0.5773674 0.4534615
0.76 0.6404695 0.5772421 0.4534475
0.77 0.6401074 0.5771169 0.4534335
0.78 0.6397457 0.5769918 0.4534196
0.79 0.6393845 0.5768667 0.4534056
0.80 0.6390237 0.5767418 0.4533915
0.81 0.6386633 0.5766169 0.4533775
0.82 0.6383035 0.5764922 0.4533635
0.83 0.6379441 0.5763675 0.4533494
0.84 0.6375851 0.5762429 0.4533353
0.85 0.6372266 0.5761184 0.4533212
0.86 0.6368685 0.5759939 0.4533071
0.87 0.6365109 0.5758696 0.4532930
0.88 0.6361538 0.5757454 0.4532788
0.89 0.6357971 0.5756212 0.4532647
0.90 0.6354408 0.5754971 0.4532505
0.91 0.6350850 0.5753731 0.4532363
0.92 0.6347297 0.5752492 0.4532221
0.93 0.6343748 0.5751254 0.4532079
0.94 0.6340203 0.5750017 0.4531937
0.95 0.6336663 0.5748780 0.4531794
0.96 0.6333127 0.5747545 0.4531651
0.97 0.6329596 0.5746310 0.4531509
0.98 0.6326069 0.5745076 0.4531366
0.99 0.6322547 0.5743843 0.4531223
1.00 0.6319029 0.5742611 0.4531079
1.01 0.6315515 0.5741379 0.4530936
1.02 0.6312006 0.5740149 0.4530792
1.03 0.6308501 0.5738919 0.4530649
1.04 0.6305000 0.5737690 0.4530505
1.05 0.6301504 0.5736462 0.4530361
1.06 0.6298013 0.5735235 0.4530217
1.07 0.6294525 0.5734009 0.4530072
1.08 0.6291042 0.5732784 0.4529928
1.09 0.6287564 0.5731559 0.4529783
1.10 0.6284090 0.5730335 0.4529638
matrix_prostateage matrix_prostatelbph matrix_prostatesvi
0.00 -0.01963721 0.10705435 0.7661559
0.01 -0.01962951 0.10703401 0.7660194
0.02 -0.01962181 0.10701368 0.7658831
0.03 -0.01961413 0.10699336 0.7657468
0.04 -0.01960644 0.10697305 0.7656107
0.05 -0.01959877 0.10695276 0.7654747
0.06 -0.01959109 0.10693249 0.7653388
0.07 -0.01958343 0.10691222 0.7652031
0.08 -0.01957576 0.10689197 0.7650674
0.09 -0.01956811 0.10687173 0.7649319
0.10 -0.01956046 0.10685151 0.7647964
0.11 -0.01955281 0.10683129 0.7646611
0.12 -0.01954517 0.10681109 0.7645259
0.13 -0.01953753 0.10679091 0.7643908
0.14 -0.01952990 0.10677074 0.7642558
0.15 -0.01952228 0.10675058 0.7641210
0.16 -0.01951466 0.10673043 0.7639862
0.17 -0.01950704 0.10671029 0.7638516
0.18 -0.01949943 0.10669017 0.7637170
0.19 -0.01949183 0.10667007 0.7635826
0.20 -0.01948423 0.10664997 0.7634483
0.21 -0.01947664 0.10662989 0.7633141
0.22 -0.01946905 0.10660982 0.7631800
0.23 -0.01946147 0.10658976 0.7630460
0.24 -0.01945389 0.10656972 0.7629122
0.25 -0.01944631 0.10654969 0.7627784
0.26 -0.01943875 0.10652967 0.7626448
0.27 -0.01943118 0.10650967 0.7625113
0.28 -0.01942363 0.10648968 0.7623778
0.29 -0.01941607 0.10646970 0.7622445
0.30 -0.01940853 0.10644973 0.7621113
0.31 -0.01940099 0.10642978 0.7619783
0.32 -0.01939345 0.10640984 0.7618453
0.33 -0.01938592 0.10638991 0.7617124
0.34 -0.01937839 0.10636999 0.7615797
0.35 -0.01937087 0.10635009 0.7614470
0.36 -0.01936335 0.10633020 0.7613145
0.37 -0.01935584 0.10631033 0.7611820
0.38 -0.01934834 0.10629046 0.7610497
0.39 -0.01934084 0.10627061 0.7609175
0.40 -0.01933334 0.10625077 0.7607854
0.41 -0.01932585 0.10623094 0.7606534
0.42 -0.01931836 0.10621113 0.7605215
0.43 -0.01931088 0.10619133 0.7603898
0.44 -0.01930341 0.10617154 0.7602581
0.45 -0.01929594 0.10615177 0.7601265
0.46 -0.01928847 0.10613200 0.7599951
0.47 -0.01928101 0.10611225 0.7598637
0.48 -0.01927356 0.10609251 0.7597325
0.49 -0.01926611 0.10607279 0.7596014
0.50 -0.01925866 0.10605308 0.7594703
0.51 -0.01925122 0.10603338 0.7593394
0.52 -0.01924379 0.10601369 0.7592086
0.53 -0.01923636 0.10599401 0.7590779
0.54 -0.01922893 0.10597435 0.7589473
0.55 -0.01922151 0.10595470 0.7588168
0.56 -0.01921410 0.10593506 0.7586865
0.57 -0.01920669 0.10591543 0.7585562
0.58 -0.01919928 0.10589582 0.7584260
0.59 -0.01919188 0.10587622 0.7582960
0.60 -0.01918449 0.10585663 0.7581660
0.61 -0.01917710 0.10583705 0.7580362
0.62 -0.01916971 0.10581749 0.7579064
0.63 -0.01916233 0.10579794 0.7577768
0.64 -0.01915495 0.10577840 0.7576473
0.65 -0.01914758 0.10575887 0.7575178
0.66 -0.01914022 0.10573936 0.7573885
0.67 -0.01913286 0.10571986 0.7572593
0.68 -0.01912550 0.10570037 0.7571302
0.69 -0.01911815 0.10568089 0.7570012
0.70 -0.01911081 0.10566142 0.7568723
0.71 -0.01910347 0.10564197 0.7567435
0.72 -0.01909613 0.10562253 0.7566148
0.73 -0.01908880 0.10560310 0.7564862
0.74 -0.01908147 0.10558368 0.7563577
0.75 -0.01907415 0.10556428 0.7562293
0.76 -0.01906683 0.10554488 0.7561010
0.77 -0.01905952 0.10552550 0.7559729
0.78 -0.01905222 0.10550614 0.7558448
0.79 -0.01904492 0.10548678 0.7557168
0.80 -0.01903762 0.10546744 0.7555890
0.81 -0.01903033 0.10544810 0.7554612
0.82 -0.01902304 0.10542878 0.7553336
0.83 -0.01901576 0.10540948 0.7552060
0.84 -0.01900848 0.10539018 0.7550785
0.85 -0.01900121 0.10537090 0.7549512
0.86 -0.01899394 0.10535162 0.7548240
0.87 -0.01898668 0.10533236 0.7546968
0.88 -0.01897942 0.10531312 0.7545698
0.89 -0.01897216 0.10529388 0.7544428
0.90 -0.01896492 0.10527466 0.7543160
0.91 -0.01895767 0.10525544 0.7541892
0.92 -0.01895043 0.10523624 0.7540626
0.93 -0.01894320 0.10521706 0.7539361
0.94 -0.01893597 0.10519788 0.7538096
0.95 -0.01892874 0.10517871 0.7536833
0.96 -0.01892152 0.10515956 0.7535571
0.97 -0.01891431 0.10514042 0.7534309
0.98 -0.01890710 0.10512129 0.7533049
0.99 -0.01889989 0.10510218 0.7531790
1.00 -0.01889269 0.10508307 0.7530532
1.01 -0.01888550 0.10506398 0.7529274
1.02 -0.01887830 0.10504490 0.7528018
1.03 -0.01887112 0.10502583 0.7526763
1.04 -0.01886394 0.10500677 0.7525509
1.05 -0.01885676 0.10498772 0.7524255
1.06 -0.01884959 0.10496869 0.7523003
1.07 -0.01884242 0.10494966 0.7521752
1.08 -0.01883526 0.10493065 0.7520501
1.09 -0.01882810 0.10491165 0.7519252
1.10 -0.01882094 0.10489267 0.7518004
matrix_prostatelcp matrix_prostategleason matrix_prostatepgg45
0.00 -0.10547357 0.04513596 0.004525324
0.01 -0.10534493 0.04517216 0.004522978
0.02 -0.10521642 0.04520830 0.004520635
0.03 -0.10508803 0.04524439 0.004518296
0.04 -0.10495978 0.04528041 0.004515960
0.05 -0.10483165 0.04531639 0.004513628
0.06 -0.10470365 0.04535230 0.004511299
0.07 -0.10457578 0.04538816 0.004508974
0.08 -0.10444803 0.04542397 0.004506651
0.09 -0.10432042 0.04545972 0.004504333
0.10 -0.10419293 0.04549541 0.004502017
0.11 -0.10406556 0.04553105 0.004499705
0.12 -0.10393833 0.04556663 0.004497397
0.13 -0.10381122 0.04560216 0.004495091
0.14 -0.10368423 0.04563763 0.004492790
0.15 -0.10355738 0.04567305 0.004490491
0.16 -0.10343064 0.04570841 0.004488196
0.17 -0.10330404 0.04574372 0.004485904
0.18 -0.10317756 0.04577898 0.004483615
0.19 -0.10305121 0.04581418 0.004481330
0.20 -0.10292498 0.04584932 0.004479048
0.21 -0.10279887 0.04588441 0.004476770
0.22 -0.10267289 0.04591945 0.004474494
0.23 -0.10254704 0.04595443 0.004472222
0.24 -0.10242131 0.04598936 0.004469954
0.25 -0.10229571 0.04602424 0.004467688
0.26 -0.10217023 0.04605906 0.004465426
0.27 -0.10204487 0.04609383 0.004463167
0.28 -0.10191964 0.04612854 0.004460912
0.29 -0.10179453 0.04616321 0.004458659
0.30 -0.10166954 0.04619781 0.004456410
0.31 -0.10154468 0.04623237 0.004454164
0.32 -0.10141994 0.04626687 0.004451922
0.33 -0.10129533 0.04630132 0.004449682
0.34 -0.10117084 0.04633572 0.004447446
0.35 -0.10104647 0.04637006 0.004445213
0.36 -0.10092222 0.04640435 0.004442984
0.37 -0.10079809 0.04643859 0.004440757
0.38 -0.10067409 0.04647278 0.004438534
0.39 -0.10055021 0.04650691 0.004436314
0.40 -0.10042645 0.04654100 0.004434097
0.41 -0.10030281 0.04657503 0.004431884
0.42 -0.10017930 0.04660901 0.004429673
0.43 -0.10005590 0.04664293 0.004427466
0.44 -0.09993263 0.04667681 0.004425262
0.45 -0.09980947 0.04671063 0.004423061
0.46 -0.09968644 0.04674440 0.004420863
0.47 -0.09956353 0.04677813 0.004418669
0.48 -0.09944074 0.04681180 0.004416477
0.49 -0.09931807 0.04684541 0.004414289
0.50 -0.09919552 0.04687898 0.004412104
0.51 -0.09907309 0.04691250 0.004409922
0.52 -0.09895078 0.04694596 0.004407743
0.53 -0.09882859 0.04697938 0.004405567
0.54 -0.09870652 0.04701274 0.004403394
0.55 -0.09858456 0.04704606 0.004401225
0.56 -0.09846273 0.04707932 0.004399059
0.57 -0.09834101 0.04711253 0.004396895
0.58 -0.09821942 0.04714570 0.004394735
0.59 -0.09809794 0.04717881 0.004392578
0.60 -0.09797658 0.04721187 0.004390424
0.61 -0.09785534 0.04724488 0.004388273
0.62 -0.09773422 0.04727785 0.004386125
0.63 -0.09761322 0.04731076 0.004383980
0.64 -0.09749233 0.04734362 0.004381839
0.65 -0.09737156 0.04737644 0.004379700
0.66 -0.09725091 0.04740920 0.004377564
0.67 -0.09713038 0.04744192 0.004375432
0.68 -0.09700996 0.04747459 0.004373302
0.69 -0.09688966 0.04750720 0.004371176
0.70 -0.09676947 0.04753977 0.004369052
0.71 -0.09664941 0.04757229 0.004366932
0.72 -0.09652946 0.04760476 0.004364815
0.73 -0.09640962 0.04763718 0.004362700
0.74 -0.09628991 0.04766956 0.004360589
0.75 -0.09617030 0.04770188 0.004358481
0.76 -0.09605082 0.04773416 0.004356375
0.77 -0.09593145 0.04776639 0.004354273
0.78 -0.09581219 0.04779857 0.004352173
0.79 -0.09569305 0.04783070 0.004350077
0.80 -0.09557403 0.04786278 0.004347984
0.81 -0.09545512 0.04789482 0.004345893
0.82 -0.09533632 0.04792681 0.004343806
0.83 -0.09521764 0.04795875 0.004341721
0.84 -0.09509908 0.04799064 0.004339640
0.85 -0.09498063 0.04802248 0.004337561
0.86 -0.09486229 0.04805428 0.004335486
0.87 -0.09474407 0.04808603 0.004333413
0.88 -0.09462596 0.04811774 0.004331343
0.89 -0.09450796 0.04814939 0.004329276
0.90 -0.09439008 0.04818100 0.004327212
0.91 -0.09427231 0.04821256 0.004325152
0.92 -0.09415466 0.04824408 0.004323093
0.93 -0.09403712 0.04827555 0.004321038
0.94 -0.09391969 0.04830697 0.004318986
0.95 -0.09380237 0.04833834 0.004316937
0.96 -0.09368517 0.04836967 0.004314890
0.97 -0.09356808 0.04840096 0.004312847
0.98 -0.09345110 0.04843219 0.004310806
0.99 -0.09333423 0.04846338 0.004308769
1.00 -0.09321748 0.04849453 0.004306734
1.01 -0.09310084 0.04852562 0.004304702
1.02 -0.09298431 0.04855668 0.004302673
1.03 -0.09286789 0.04858768 0.004300646
1.04 -0.09275158 0.04861864 0.004298623
1.05 -0.09263538 0.04864956 0.004296602
1.06 -0.09251930 0.04868043 0.004294585
1.07 -0.09240332 0.04871125 0.004292570
1.08 -0.09228746 0.04874203 0.004290558
1.09 -0.09217171 0.04877276 0.004288548
1.10 -0.09205606 0.04880345 0.004286542
[ reached getOption("max.print") -- omitted 890 rows ]Podemos ver los diferentes valores que obtienen los parametros estimados en base a cada valor de \[\lambda\]
plot(coef(rr))
Paquete glmnet: regresión regularizada
Glmnet resuelve el siguiente problema de optimización:
\[ \text{Minimizar} \quad \frac{1}{2n} \sum_{i=1}^{n} \left( y_i - \beta_0 - \sum_{j=1}^{p} \beta_j x_{ij} \right)^2 + \frac{\lambda}{2} \sum_{j=1}^{p} \beta_j^2 \]
Generamos un modelo con una regularización alpha = 0 y el parámetro de penalización lambda = 0:
rr_glmnet <- glmnet(matrix_prostate, lpsa, alpha = 0, lambda = 0)Procedemos a ver los coeficientes del modelo
coef(rr_glmnet)9 x 1 sparse Matrix of class "dgCMatrix"
s0
(Intercept) 0.667514485
lcavol 0.586983973
lweight 0.454630898
age -0.019639964
lbph 0.107017836
svi 0.766095738
lcp -0.105508227
gleason 0.045365684
pgg45 0.004523291Vemos que son los mismos coeficientes (aproximadamente) que los calculados por el método de mínimos cuadrados.
GLMNET con diferentes valores del parámetro lambda de penalización:
lam <- seq(0, 10, by = 0.01)
rr_glmnet <- glmnet(matrix_prostate, lpsa, alpha = 0, lambda = lam)
rr_glmnet
Call: glmnet(x = matrix_prostate, y = lpsa, alpha = 0, lambda = lam)
Df %Dev Lambda
1 8 24.63 10.00
2 8 24.65 9.99
3 8 24.66 9.98
4 8 24.68 9.97
5 8 24.70 9.96
6 8 24.71 9.95
7 8 24.73 9.94
8 8 24.74 9.93
9 8 24.76 9.92
10 8 24.78 9.91
11 8 24.79 9.90
12 8 24.81 9.89
13 8 24.82 9.88
14 8 24.84 9.87
15 8 24.85 9.86
16 8 24.87 9.85
17 8 24.89 9.84
18 8 24.90 9.83
19 8 24.92 9.82
20 8 24.94 9.81
21 8 24.95 9.80
22 8 24.97 9.79
23 8 24.98 9.78
24 8 25.00 9.77
25 8 25.02 9.76
26 8 25.03 9.75
27 8 25.05 9.74
28 8 25.06 9.73
29 8 25.08 9.72
30 8 25.10 9.71
31 8 25.11 9.70
32 8 25.13 9.69
33 8 25.15 9.68
34 8 25.16 9.67
35 8 25.18 9.66
36 8 25.20 9.65
37 8 25.21 9.64
38 8 25.23 9.63
39 8 25.24 9.62
40 8 25.26 9.61
41 8 25.28 9.60
42 8 25.29 9.59
43 8 25.31 9.58
44 8 25.33 9.57
45 8 25.34 9.56
46 8 25.36 9.55
47 8 25.38 9.54
48 8 25.39 9.53
49 8 25.41 9.52
50 8 25.43 9.51
51 8 25.44 9.50
52 8 25.46 9.49
53 8 25.48 9.48
54 8 25.49 9.47
55 8 25.51 9.46
56 8 25.53 9.45
57 8 25.55 9.44
58 8 25.56 9.43
59 8 25.58 9.42
60 8 25.60 9.41
61 8 25.61 9.40
62 8 25.63 9.39
63 8 25.65 9.38
64 8 25.66 9.37
65 8 25.68 9.36
66 8 25.70 9.35
67 8 25.72 9.34
68 8 25.73 9.33
69 8 25.75 9.32
70 8 25.77 9.31
71 8 25.78 9.30
72 8 25.80 9.29
73 8 25.82 9.28
74 8 25.84 9.27
75 8 25.85 9.26
76 8 25.87 9.25
77 8 25.89 9.24
78 8 25.90 9.23
79 8 25.92 9.22
80 8 25.94 9.21
81 8 25.96 9.20
82 8 25.97 9.19
83 8 25.99 9.18
84 8 26.01 9.17
85 8 26.03 9.16
86 8 26.04 9.15
87 8 26.06 9.14
88 8 26.08 9.13
89 8 26.10 9.12
90 8 26.11 9.11
91 8 26.13 9.10
92 8 26.15 9.09
93 8 26.17 9.08
94 8 26.19 9.07
95 8 26.20 9.06
96 8 26.22 9.05
97 8 26.24 9.04
98 8 26.26 9.03
99 8 26.27 9.02
100 8 26.29 9.01
101 8 26.31 9.00
102 8 26.33 8.99
103 8 26.35 8.98
104 8 26.36 8.97
105 8 26.38 8.96
106 8 26.40 8.95
107 8 26.42 8.94
108 8 26.44 8.93
109 8 26.45 8.92
110 8 26.47 8.91
111 8 26.49 8.90
112 8 26.51 8.89
113 8 26.53 8.88
114 8 26.55 8.87
115 8 26.56 8.86
116 8 26.58 8.85
117 8 26.60 8.84
118 8 26.62 8.83
119 8 26.64 8.82
120 8 26.65 8.81
121 8 26.67 8.80
122 8 26.69 8.79
123 8 26.71 8.78
124 8 26.73 8.77
125 8 26.75 8.76
126 8 26.77 8.75
127 8 26.78 8.74
128 8 26.80 8.73
129 8 26.82 8.72
130 8 26.84 8.71
131 8 26.86 8.70
132 8 26.88 8.69
133 8 26.90 8.68
134 8 26.91 8.67
135 8 26.93 8.66
136 8 26.95 8.65
137 8 26.97 8.64
138 8 26.99 8.63
139 8 27.01 8.62
140 8 27.03 8.61
141 8 27.05 8.60
142 8 27.06 8.59
143 8 27.08 8.58
144 8 27.10 8.57
145 8 27.12 8.56
146 8 27.14 8.55
147 8 27.16 8.54
148 8 27.18 8.53
149 8 27.20 8.52
150 8 27.22 8.51
151 8 27.24 8.50
152 8 27.26 8.49
153 8 27.27 8.48
154 8 27.29 8.47
155 8 27.31 8.46
156 8 27.33 8.45
157 8 27.35 8.44
158 8 27.37 8.43
159 8 27.39 8.42
160 8 27.41 8.41
161 8 27.43 8.40
162 8 27.45 8.39
163 8 27.47 8.38
164 8 27.49 8.37
165 8 27.51 8.36
166 8 27.53 8.35
167 8 27.55 8.34
168 8 27.57 8.33
169 8 27.59 8.32
170 8 27.61 8.31
171 8 27.62 8.30
172 8 27.64 8.29
173 8 27.66 8.28
174 8 27.68 8.27
175 8 27.70 8.26
176 8 27.72 8.25
177 8 27.74 8.24
178 8 27.76 8.23
179 8 27.78 8.22
180 8 27.80 8.21
181 8 27.82 8.20
182 8 27.84 8.19
183 8 27.86 8.18
184 8 27.88 8.17
185 8 27.90 8.16
186 8 27.92 8.15
187 8 27.94 8.14
188 8 27.96 8.13
189 8 27.98 8.12
190 8 28.00 8.11
191 8 28.02 8.10
192 8 28.04 8.09
193 8 28.07 8.08
194 8 28.09 8.07
195 8 28.11 8.06
196 8 28.13 8.05
197 8 28.15 8.04
198 8 28.17 8.03
199 8 28.19 8.02
200 8 28.21 8.01
201 8 28.23 8.00
202 8 28.25 7.99
203 8 28.27 7.98
204 8 28.29 7.97
205 8 28.31 7.96
206 8 28.33 7.95
207 8 28.35 7.94
208 8 28.37 7.93
209 8 28.39 7.92
210 8 28.41 7.91
211 8 28.44 7.90
212 8 28.46 7.89
213 8 28.48 7.88
214 8 28.50 7.87
215 8 28.52 7.86
216 8 28.54 7.85
217 8 28.56 7.84
218 8 28.58 7.83
219 8 28.60 7.82
220 8 28.62 7.81
221 8 28.65 7.80
222 8 28.67 7.79
223 8 28.69 7.78
224 8 28.71 7.77
225 8 28.73 7.76
226 8 28.75 7.75
227 8 28.77 7.74
228 8 28.79 7.73
229 8 28.82 7.72
230 8 28.84 7.71
231 8 28.86 7.70
232 8 28.88 7.69
233 8 28.90 7.68
234 8 28.92 7.67
235 8 28.95 7.66
236 8 28.97 7.65
237 8 28.99 7.64
238 8 29.01 7.63
239 8 29.03 7.62
240 8 29.05 7.61
241 8 29.08 7.60
242 8 29.10 7.59
243 8 29.12 7.58
244 8 29.14 7.57
245 8 29.16 7.56
246 8 29.18 7.55
247 8 29.21 7.54
248 8 29.23 7.53
249 8 29.25 7.52
250 8 29.27 7.51
251 8 29.29 7.50
252 8 29.32 7.49
253 8 29.34 7.48
254 8 29.36 7.47
255 8 29.38 7.46
256 8 29.41 7.45
257 8 29.43 7.44
258 8 29.45 7.43
259 8 29.47 7.42
260 8 29.50 7.41
261 8 29.52 7.40
262 8 29.54 7.39
263 8 29.56 7.38
264 8 29.58 7.37
265 8 29.61 7.36
266 8 29.63 7.35
267 8 29.65 7.34
268 8 29.68 7.33
269 8 29.70 7.32
270 8 29.72 7.31
271 8 29.74 7.30
272 8 29.77 7.29
273 8 29.79 7.28
274 8 29.81 7.27
275 8 29.83 7.26
276 8 29.86 7.25
277 8 29.88 7.24
278 8 29.90 7.23
279 8 29.93 7.22
280 8 29.95 7.21
281 8 29.97 7.20
282 8 30.00 7.19
283 8 30.02 7.18
284 8 30.04 7.17
285 8 30.07 7.16
286 8 30.09 7.15
287 8 30.11 7.14
288 8 30.14 7.13
289 8 30.16 7.12
290 8 30.18 7.11
291 8 30.21 7.10
292 8 30.23 7.09
293 8 30.25 7.08
294 8 30.28 7.07
295 8 30.30 7.06
296 8 30.32 7.05
297 8 30.35 7.04
298 8 30.37 7.03
299 8 30.40 7.02
300 8 30.42 7.01
301 8 30.44 7.00
302 8 30.47 6.99
303 8 30.49 6.98
304 8 30.51 6.97
305 8 30.54 6.96
306 8 30.56 6.95
307 8 30.59 6.94
308 8 30.61 6.93
309 8 30.63 6.92
310 8 30.66 6.91
311 8 30.68 6.90
312 8 30.71 6.89
313 8 30.73 6.88
314 8 30.76 6.87
315 8 30.78 6.86
316 8 30.80 6.85
317 8 30.83 6.84
318 8 30.85 6.83
319 8 30.88 6.82
320 8 30.90 6.81
321 8 30.93 6.80
322 8 30.95 6.79
323 8 30.98 6.78
324 8 31.00 6.77
325 8 31.03 6.76
326 8 31.05 6.75
327 8 31.08 6.74
328 8 31.10 6.73
329 8 31.12 6.72
330 8 31.15 6.71
331 8 31.17 6.70
332 8 31.20 6.69
333 8 31.22 6.68
[ reached getOption("max.print") -- omitted 668 rows ]coef(rr_glmnet)9 x 1001 sparse Matrix of class "dgCMatrix" [[ suppressing 32 column names ‘s0’, ‘s1’, ‘s2’ ... ]]
(Intercept) 1.638348358 1.637759373 1.637168405 1.636576618 1.635984008
lcavol 0.064816857 0.064868151 0.064919892 0.064971717 0.065023626
lweight 0.076432830 0.076495012 0.076557524 0.076620138 0.076682855
age 0.001770777 0.001771658 0.001772548 0.001773439 0.001774330
lbph 0.013068153 0.013078692 0.013089225 0.013099775 0.013110343
svi 0.135911872 0.136014255 0.136117086 0.136220074 0.136323218
lcp 0.036705378 0.036730978 0.036756604 0.036782264 0.036807960
gleason 0.044964742 0.044993835 0.045022851 0.045051902 0.045080989
pgg45 0.001332066 0.001332938 0.001333810 0.001334684 0.001335559
(Intercept) 1.635390572 1.634796309 1.634201217 1.633605294 1.633008538
lcavol 0.065075619 0.065127697 0.065179859 0.065232107 0.065284440
lweight 0.076745676 0.076808601 0.076871630 0.076934764 0.076998002
age 0.001775222 0.001776114 0.001777007 0.001777901 0.001778795
lbph 0.013120927 0.013131529 0.013142148 0.013152785 0.013163439
svi 0.136426521 0.136529981 0.136633600 0.136737377 0.136841313
lcp 0.036833691 0.036859457 0.036885259 0.036911096 0.036936968
gleason 0.045110112 0.045139271 0.045168466 0.045197697 0.045226964
pgg45 0.001336434 0.001337311 0.001338189 0.001339068 0.001339948
(Intercept) 1.632410948 1.631812523 1.631213259 1.630613157 1.630012213
lcavol 0.065336858 0.065389362 0.065441951 0.065494627 0.065547389
lweight 0.077061345 0.077124794 0.077188348 0.077252007 0.077315773
age 0.001779689 0.001780585 0.001781480 0.001782376 0.001783273
lbph 0.013174111 0.013184800 0.013195506 0.013206231 0.013216973
svi 0.136945409 0.137049665 0.137154080 0.137258657 0.137363394
lcp 0.036962876 0.036988820 0.037014800 0.037040815 0.037066866
gleason 0.045256267 0.045285607 0.045314982 0.045344394 0.045373843
pgg45 0.001340830 0.001341712 0.001342596 0.001343480 0.001344366
(Intercept) 1.629410426 1.628807795 1.628204318 1.627599992 1.626994816
lcavol 0.065600237 0.065653172 0.065706194 0.065759303 0.065812500
lweight 0.077379645 0.077443623 0.077507708 0.077571901 0.077636200
age 0.001784170 0.001785068 0.001785966 0.001786865 0.001787765
lbph 0.013227732 0.013238510 0.013249305 0.013260118 0.013270948
svi 0.137468292 0.137573352 0.137678574 0.137783959 0.137889506
lcp 0.037092954 0.037119077 0.037145236 0.037171432 0.037197663
gleason 0.045403328 0.045432849 0.045462407 0.045492002 0.045521633
pgg45 0.001345253 0.001346141 0.001347030 0.001347920 0.001348812
(Intercept) 1.626388789 1.625781909 1.625174173 1.624565581 1.623956129
lcavol 0.065865784 0.065919156 0.065972616 0.066026164 0.066079800
lweight 0.077700608 0.077765123 0.077829746 0.077894478 0.077959319
age 0.001788665 0.001789565 0.001790466 0.001791367 0.001792269
lbph 0.013281797 0.013292664 0.013303548 0.013314451 0.013325372
svi 0.137995217 0.138101091 0.138207129 0.138313331 0.138419698
lcp 0.037223931 0.037250236 0.037276577 0.037302954 0.037329368
gleason 0.045551301 0.045581006 0.045610748 0.045640527 0.045670343
pgg45 0.001349704 0.001350598 0.001351493 0.001352389 0.001353286
(Intercept) 1.623345817 1.622734643 1.622122605 1.621509700 1.620895929
lcavol 0.066133525 0.066187339 0.066241243 0.066295235 0.066349317
lweight 0.078024268 0.078089327 0.078154495 0.078219773 0.078285162
age 0.001793172 0.001794075 0.001794979 0.001795883 0.001796787
lbph 0.013336311 0.013347268 0.013358244 0.013369237 0.013380249
svi 0.138526231 0.138632928 0.138739792 0.138846822 0.138954018
lcp 0.037355819 0.037382307 0.037408831 0.037435392 0.037461990
gleason 0.045700196 0.045730086 0.045760014 0.045789979 0.045819981
pgg45 0.001354184 0.001355083 0.001355984 0.001356885 0.001357788
(Intercept) 1.620281287 1.619665774 ......
lcavol 0.066403489 0.066457751 ......
lweight 0.078350660 0.078416270 ......
age 0.001797692 0.001798598 ......
lbph 0.013391280 0.013402328 ......
svi 0.139061382 0.139168913 ......
lcp 0.037488626 0.037515298 ......
gleason 0.045850020 0.045880097 ......
pgg45 0.001358692 0.001359597 ......
.....suppressing 969 columns in show(); maybe adjust options(max.print=, width=)
..............................En este caso se devuelven las columnas que recogen los parámetros estimados para cada valor de λ (las estimaciones se devuelven en la escala original).
Las estimaciones para el valor \[\lambda\] almacenado en ´rr_glmnet$lambda[j]´, las podemos obtener de la siguiente forma:
coef(rr_glmnet)[,j](Intercept) lcavol lweight age lbph svi
1.629410426 0.065600237 0.077379645 0.001784170 0.013227732 0.137468292
lcp gleason pgg45
0.037092954 0.045403328 0.001345253 Observa que no coincide exactamente con la formulación original de problema y esto hace que un mismo valor de λ 6= 0 no devuelva los mismos estimadores con la función lm.ridge y glmnet. Aún así el papel de λ como parámetro de penalización es el mismo, como se observa en la siguiente gráfica, que muestra las estimaciones de los parámetros a medida que aumenta λ.
Plot de los coeficientes en relación al logaritmo de lambda
plot(rr_glmnet, xvar = "lambda")
2. Regresión Lasso (least absolute shrinkage and selection operator)
Los parámetros de la regresión Lasso los podemos obtener a través de la siguiente minimizacion:
\[ \text{Minimizar} \quad \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \beta_0 - \sum_{j=1}^{p} \beta_j x_{i,j} \right)^2 + \lambda \sum_{j=1}^{p} |\beta_j| \] Primero ajustamos el modelo con la función glmnet:
lasso <- glmnet(matrix_prostate, lpsa, alpha = 1, lambda = 0)
lasso
Call: glmnet(x = matrix_prostate, y = lpsa, alpha = 1, lambda = 0)
Df %Dev Lambda
1 8 65.48 0Si determinamos alpha como 1, estaremos ejecutando con una regularización de tipo lasso.
Podemos obtener los coeficientes estimados (en este caso iguales que los ajustados por mínimos cuadrados):
coef(lasso)9 x 1 sparse Matrix of class "dgCMatrix"
s0
(Intercept) 0.667514485
lcavol 0.586983973
lweight 0.454630898
age -0.019639964
lbph 0.107017836
svi 0.766095738
lcp -0.105508227
gleason 0.045365684
pgg45 0.004523291Probando al igual que antes con diferenes valores de \(\lambda\)
lam <- seq(0, 10, by = 0.01)
lasso <- glmnet(matrix_prostate, lpsa, alpha = 1, lambda = lam)
lasso
Call: glmnet(x = matrix_prostate, y = lpsa, alpha = 1, lambda = lam)
Df %Dev Lambda
1 0 0.00 10.00
2 0 0.00 9.99
3 0 0.00 9.98
4 0 0.00 9.97
5 0 0.00 9.96
6 0 0.00 9.95
7 0 0.00 9.94
8 0 0.00 9.93
9 0 0.00 9.92
10 0 0.00 9.91
11 0 0.00 9.90
12 0 0.00 9.89
13 0 0.00 9.88
14 0 0.00 9.87
15 0 0.00 9.86
16 0 0.00 9.85
17 0 0.00 9.84
18 0 0.00 9.83
19 0 0.00 9.82
20 0 0.00 9.81
21 0 0.00 9.80
22 0 0.00 9.79
23 0 0.00 9.78
24 0 0.00 9.77
25 0 0.00 9.76
26 0 0.00 9.75
27 0 0.00 9.74
28 0 0.00 9.73
29 0 0.00 9.72
30 0 0.00 9.71
31 0 0.00 9.70
32 0 0.00 9.69
33 0 0.00 9.68
34 0 0.00 9.67
35 0 0.00 9.66
36 0 0.00 9.65
37 0 0.00 9.64
38 0 0.00 9.63
39 0 0.00 9.62
40 0 0.00 9.61
41 0 0.00 9.60
42 0 0.00 9.59
43 0 0.00 9.58
44 0 0.00 9.57
45 0 0.00 9.56
46 0 0.00 9.55
47 0 0.00 9.54
48 0 0.00 9.53
49 0 0.00 9.52
50 0 0.00 9.51
51 0 0.00 9.50
52 0 0.00 9.49
53 0 0.00 9.48
54 0 0.00 9.47
55 0 0.00 9.46
56 0 0.00 9.45
57 0 0.00 9.44
58 0 0.00 9.43
59 0 0.00 9.42
60 0 0.00 9.41
61 0 0.00 9.40
62 0 0.00 9.39
63 0 0.00 9.38
64 0 0.00 9.37
65 0 0.00 9.36
66 0 0.00 9.35
67 0 0.00 9.34
68 0 0.00 9.33
69 0 0.00 9.32
70 0 0.00 9.31
71 0 0.00 9.30
72 0 0.00 9.29
73 0 0.00 9.28
74 0 0.00 9.27
75 0 0.00 9.26
76 0 0.00 9.25
77 0 0.00 9.24
78 0 0.00 9.23
79 0 0.00 9.22
80 0 0.00 9.21
81 0 0.00 9.20
82 0 0.00 9.19
83 0 0.00 9.18
84 0 0.00 9.17
85 0 0.00 9.16
86 0 0.00 9.15
87 0 0.00 9.14
88 0 0.00 9.13
89 0 0.00 9.12
90 0 0.00 9.11
91 0 0.00 9.10
92 0 0.00 9.09
93 0 0.00 9.08
94 0 0.00 9.07
95 0 0.00 9.06
96 0 0.00 9.05
97 0 0.00 9.04
98 0 0.00 9.03
99 0 0.00 9.02
100 0 0.00 9.01
101 0 0.00 9.00
102 0 0.00 8.99
103 0 0.00 8.98
104 0 0.00 8.97
105 0 0.00 8.96
106 0 0.00 8.95
107 0 0.00 8.94
108 0 0.00 8.93
109 0 0.00 8.92
110 0 0.00 8.91
111 0 0.00 8.90
112 0 0.00 8.89
113 0 0.00 8.88
114 0 0.00 8.87
115 0 0.00 8.86
116 0 0.00 8.85
117 0 0.00 8.84
118 0 0.00 8.83
119 0 0.00 8.82
120 0 0.00 8.81
121 0 0.00 8.80
122 0 0.00 8.79
123 0 0.00 8.78
124 0 0.00 8.77
125 0 0.00 8.76
126 0 0.00 8.75
127 0 0.00 8.74
128 0 0.00 8.73
129 0 0.00 8.72
130 0 0.00 8.71
131 0 0.00 8.70
132 0 0.00 8.69
133 0 0.00 8.68
134 0 0.00 8.67
135 0 0.00 8.66
136 0 0.00 8.65
137 0 0.00 8.64
138 0 0.00 8.63
139 0 0.00 8.62
140 0 0.00 8.61
141 0 0.00 8.60
142 0 0.00 8.59
143 0 0.00 8.58
144 0 0.00 8.57
145 0 0.00 8.56
146 0 0.00 8.55
147 0 0.00 8.54
148 0 0.00 8.53
149 0 0.00 8.52
150 0 0.00 8.51
151 0 0.00 8.50
152 0 0.00 8.49
153 0 0.00 8.48
154 0 0.00 8.47
155 0 0.00 8.46
156 0 0.00 8.45
157 0 0.00 8.44
158 0 0.00 8.43
159 0 0.00 8.42
160 0 0.00 8.41
161 0 0.00 8.40
162 0 0.00 8.39
163 0 0.00 8.38
164 0 0.00 8.37
165 0 0.00 8.36
166 0 0.00 8.35
167 0 0.00 8.34
168 0 0.00 8.33
169 0 0.00 8.32
170 0 0.00 8.31
171 0 0.00 8.30
172 0 0.00 8.29
173 0 0.00 8.28
174 0 0.00 8.27
175 0 0.00 8.26
176 0 0.00 8.25
177 0 0.00 8.24
178 0 0.00 8.23
179 0 0.00 8.22
180 0 0.00 8.21
181 0 0.00 8.20
182 0 0.00 8.19
183 0 0.00 8.18
184 0 0.00 8.17
185 0 0.00 8.16
186 0 0.00 8.15
187 0 0.00 8.14
188 0 0.00 8.13
189 0 0.00 8.12
190 0 0.00 8.11
191 0 0.00 8.10
192 0 0.00 8.09
193 0 0.00 8.08
194 0 0.00 8.07
195 0 0.00 8.06
196 0 0.00 8.05
197 0 0.00 8.04
198 0 0.00 8.03
199 0 0.00 8.02
200 0 0.00 8.01
201 0 0.00 8.00
202 0 0.00 7.99
203 0 0.00 7.98
204 0 0.00 7.97
205 0 0.00 7.96
206 0 0.00 7.95
207 0 0.00 7.94
208 0 0.00 7.93
209 0 0.00 7.92
210 0 0.00 7.91
211 0 0.00 7.90
212 0 0.00 7.89
213 0 0.00 7.88
214 0 0.00 7.87
215 0 0.00 7.86
216 0 0.00 7.85
217 0 0.00 7.84
218 0 0.00 7.83
219 0 0.00 7.82
220 0 0.00 7.81
221 0 0.00 7.80
222 0 0.00 7.79
223 0 0.00 7.78
224 0 0.00 7.77
225 0 0.00 7.76
226 0 0.00 7.75
227 0 0.00 7.74
228 0 0.00 7.73
229 0 0.00 7.72
230 0 0.00 7.71
231 0 0.00 7.70
232 0 0.00 7.69
233 0 0.00 7.68
234 0 0.00 7.67
235 0 0.00 7.66
236 0 0.00 7.65
237 0 0.00 7.64
238 0 0.00 7.63
239 0 0.00 7.62
240 0 0.00 7.61
241 0 0.00 7.60
242 0 0.00 7.59
243 0 0.00 7.58
244 0 0.00 7.57
245 0 0.00 7.56
246 0 0.00 7.55
247 0 0.00 7.54
248 0 0.00 7.53
249 0 0.00 7.52
250 0 0.00 7.51
251 0 0.00 7.50
252 0 0.00 7.49
253 0 0.00 7.48
254 0 0.00 7.47
255 0 0.00 7.46
256 0 0.00 7.45
257 0 0.00 7.44
258 0 0.00 7.43
259 0 0.00 7.42
260 0 0.00 7.41
261 0 0.00 7.40
262 0 0.00 7.39
263 0 0.00 7.38
264 0 0.00 7.37
265 0 0.00 7.36
266 0 0.00 7.35
267 0 0.00 7.34
268 0 0.00 7.33
269 0 0.00 7.32
270 0 0.00 7.31
271 0 0.00 7.30
272 0 0.00 7.29
273 0 0.00 7.28
274 0 0.00 7.27
275 0 0.00 7.26
276 0 0.00 7.25
277 0 0.00 7.24
278 0 0.00 7.23
279 0 0.00 7.22
280 0 0.00 7.21
281 0 0.00 7.20
282 0 0.00 7.19
283 0 0.00 7.18
284 0 0.00 7.17
285 0 0.00 7.16
286 0 0.00 7.15
287 0 0.00 7.14
288 0 0.00 7.13
289 0 0.00 7.12
290 0 0.00 7.11
291 0 0.00 7.10
292 0 0.00 7.09
293 0 0.00 7.08
294 0 0.00 7.07
295 0 0.00 7.06
296 0 0.00 7.05
297 0 0.00 7.04
298 0 0.00 7.03
299 0 0.00 7.02
300 0 0.00 7.01
301 0 0.00 7.00
302 0 0.00 6.99
303 0 0.00 6.98
304 0 0.00 6.97
305 0 0.00 6.96
306 0 0.00 6.95
307 0 0.00 6.94
308 0 0.00 6.93
309 0 0.00 6.92
310 0 0.00 6.91
311 0 0.00 6.90
312 0 0.00 6.89
313 0 0.00 6.88
314 0 0.00 6.87
315 0 0.00 6.86
316 0 0.00 6.85
317 0 0.00 6.84
318 0 0.00 6.83
319 0 0.00 6.82
320 0 0.00 6.81
321 0 0.00 6.80
322 0 0.00 6.79
323 0 0.00 6.78
324 0 0.00 6.77
325 0 0.00 6.76
326 0 0.00 6.75
327 0 0.00 6.74
328 0 0.00 6.73
329 0 0.00 6.72
330 0 0.00 6.71
331 0 0.00 6.70
332 0 0.00 6.69
333 0 0.00 6.68
[ reached getOption("max.print") -- omitted 668 rows ]Obtenemos los siguientes resultados:
coef(lasso)9 x 1001 sparse Matrix of class "dgCMatrix" [[ suppressing 32 column names ‘s0’, ‘s1’, ‘s2’ ... ]]
(Intercept) 2.478387 2.478387 2.478387 2.478387 2.478387 2.478387 2.478387
lcavol . . . . . . .
lweight . . . . . . .
age . . . . . . .
lbph . . . . . . .
svi . . . . . . .
lcp . . . . . . .
gleason . . . . . . .
pgg45 . . . . . . .
(Intercept) 2.478387 2.478387 2.478387 2.478387 2.478387 2.478387 2.478387
lcavol . . . . . . .
lweight . . . . . . .
age . . . . . . .
lbph . . . . . . .
svi . . . . . . .
lcp . . . . . . .
gleason . . . . . . .
pgg45 . . . . . . .
(Intercept) 2.478387 2.478387 2.478387 2.478387 2.478387 2.478387 2.478387
lcavol . . . . . . .
lweight . . . . . . .
age . . . . . . .
lbph . . . . . . .
svi . . . . . . .
lcp . . . . . . .
gleason . . . . . . .
pgg45 . . . . . . .
(Intercept) 2.478387 2.478387 2.478387 2.478387 2.478387 2.478387 2.478387
lcavol . . . . . . .
lweight . . . . . . .
age . . . . . . .
lbph . . . . . . .
svi . . . . . . .
lcp . . . . . . .
gleason . . . . . . .
pgg45 . . . . . . .
(Intercept) 2.478387 2.478387 2.478387 2.478387 ......
lcavol . . . . ......
lweight . . . . ......
age . . . . ......
lbph . . . . ......
svi . . . . ......
lcp . . . . ......
gleason . . . . ......
pgg45 . . . . ......
.....suppressing 969 columns in show(); maybe adjust options(max.print=, width=)
..............................Si hacemos plot de los resultados obtenidos, se muestra lo siguiente:
plot(lasso)
plot(lasso, xvar = "lambda")
3. Selección del parámetro de penalización
Para obtener el parámetro de penalización \(\lambda\) utilizaremos el proceso de validación cruzada:
El proceso fija un rango de posibles valores λ y ajusta el modelo para cada λ utilizando una parte de la muestra de entrenamiento (un subconjunto de observaciones se apartó previamente). A continuación se analiza la capacidad predictiva del modelo sobre el subconjunto de observaciones apartado (muestra de validación). Para ello es habitual calcular el error cuadrático medio (MSE) en la muestra de validación, es decir
\[ \frac{1}{n_v} \sum_{i=1}^{n_v} (y_i - \hat{y}_i)^2 \] donde nv es el tamaño de la muestra de validación, yi son las observaciones de la muestra de validación e yˆi son las predicciones dadas por el modelo ajustado. El λ óptimo es aquel para el cual el MSE sea menor
Para realizar la validación cruzada ejecutamos el siguiente código:
Ejecutando para la regresión Ridge:
lam <- seq(0, 10, by = 0.01)
cvout <- cv.glmnet(matrix_prostate, lpsa, alpha = 0, lambda = lam)
cvout
Call: cv.glmnet(x = matrix_prostate, y = lpsa, lambda = lam, alpha = 0)
Measure: Mean-Squared Error
Lambda Index Measure SE Nonzero
min 0.04 997 0.5434 0.05131 8
1se 0.59 942 0.5946 0.05927 8coef(cvout)9 x 1 sparse Matrix of class "dgCMatrix"
s1
(Intercept) 0.302761266
lcavol 0.326987615
lweight 0.356269493
age -0.004546968
lbph 0.063949448
svi 0.516519562
lcp 0.065623787
gleason 0.081763214
pgg45 0.002652088(Estamos usando por defecto k = 10)
Para obtener el valor de \(\lambda\) que minimiza el error de validación cruzada:
cvout$lambda.min[1] 0.04Vemos que el valor que minimiza el error es: 021
Por otra parte, para obtener los parámetros de ajuste del modelo de regresión con regularización:
coef(cvout, s = "lambda.min")9 x 1 sparse Matrix of class "dgCMatrix"
s1
(Intercept) 0.558440699
lcavol 0.547159766
lweight 0.449450706
age -0.017302403
lbph 0.100805326
svi 0.725404125
lcp -0.068301905
gleason 0.054827876
pgg45 0.003894953La función también devuelve el valor cvout\(lambda.1se, que representa un valor de λ para un modelo más simple que el de cvout\)lambda.min, pero cuyo error está a una desviación estándar del mejor.
cvout$lambda.1se[1] 0.59coef(cvout, s = "lambda.1se")9 x 1 sparse Matrix of class "dgCMatrix"
s1
(Intercept) 0.302761266
lcavol 0.326987615
lweight 0.356269493
age -0.004546968
lbph 0.063949448
svi 0.516519562
lcp 0.065623787
gleason 0.081763214
pgg45 0.002652088Del mismo modo, si queremos seleccionar el parámetro λ en el ajuste Lasso, haremos:
lam <- seq(0, 10, by = 0.01)
cvoutl <- cv.glmnet(matrix_prostate, lpsa, alpha = 1, lambda = lam)
cvoutl
Call: cv.glmnet(x = matrix_prostate, y = lpsa, lambda = lam, alpha = 1)
Measure: Mean-Squared Error
Lambda Index Measure SE Nonzero
min 0.03 998 0.5349 0.07835 7
1se 0.18 983 0.6116 0.08159 3Obteniendo:
Valor de \(\lambda\) que minimiza el error de validación cruzada:
cvoutl$lambda.min[1] 0.03Con los parámetros asociados:
coef(cvoutl, s = "lambda.min")9 x 1 sparse Matrix of class "dgCMatrix"
s1
(Intercept) 0.646375629
lcavol 0.530328057
lweight 0.396877894
age -0.008585523
lbph 0.077774752
svi 0.603925320
lcp .
gleason 0.002404276
pgg45 0.002457964Y también su respectivo valor de \(\lambda\) para un modelo más simple, cuyo error está a una desviación estandar del mejor:
cvoutl$lambda.1se[1] 0.18Con sus coeficientes asociados:
coef(cvoutl, s = "lambda.1se")9 x 1 sparse Matrix of class "dgCMatrix"
s1
(Intercept) 1.0052760
lcavol 0.4763693
lweight 0.2044571
age .
lbph .
svi 0.3842595
lcp .
gleason .
pgg45 . Mostramos a continuación una gráfica de los parámetros estimados con Lasso junto con el valor de λ seleccionado por validación cruzada (el valor óptimo cvoutl\(lambda.min y cvoutl\)lambda.1se)
plot(rr_glmnet, xvar = "lambda")
plot(lasso, xvar = 'lambda')
abline(v = log(cvoutl$lambda.1se), col = 'red', lty = 2) # Línea roja que representa lambda.1seabline(v = log(cvoutl$lambda.min), col = 'blue', lty = 2) # Línea azul que representa lambda.min
Si juntamos en una misma tabla los valor de Ridge, Laso y los parámetros estimados por mínimos cuadrados obtenemos la siguiente tabla:
Guardamos los coeficientes:
coef_cvoutl <- as.matrix(coef(cvoutl, s = "lambda.1se"))
coef_cvoult <- coef_cvoutl[,1]
coef_cvout <- as.matrix(coef(cvout, s = "lambda.1se"))
coef_cvoul <- coef_cvout[,1]
coef_modelo_completo <- as.matrix(coef(modelo_completo))coef_cvoult(Intercept) lcavol lweight age lbph svi
1.0052760 0.4763693 0.2044571 0.0000000 0.0000000 0.3842595
lcp gleason pgg45
0.0000000 0.0000000 0.0000000 coef_cvoul (Intercept) lcavol lweight age lbph
0.302761266 0.326987615 0.356269493 -0.004546968 0.063949448
svi lcp gleason pgg45
0.516519562 0.065623787 0.081763214 0.002652088 coef_modelo_completo [,1]
(Intercept) 0.669399027
lcavol 0.587022881
lweight 0.454460641
age -0.019637208
lbph 0.107054351
svi 0.766155885
lcp -0.105473570
gleason 0.045135964
pgg45 0.004525324Los combinamos en una misma tabla:
tabla_coeficientes <- cbind(coef_modelo_completo, coef_cvout, coef_cvoutl)Y mostramos la tabla:
tabla_coeficientes s1 s1
(Intercept) 0.669399027 0.302761266 1.0052760
lcavol 0.587022881 0.326987615 0.4763693
lweight 0.454460641 0.356269493 0.2044571
age -0.019637208 -0.004546968 0.0000000
lbph 0.107054351 0.063949448 0.0000000
svi 0.766155885 0.516519562 0.3842595
lcp -0.105473570 0.065623787 0.0000000
gleason 0.045135964 0.081763214 0.0000000
pgg45 0.004525324 0.002652088 0.0000000| OLS | Ridge | Lasso | |
|---|---|---|---|
| (Intercept) | 0.669399027 | 0.384202844 | 0.8637906 |
| lcavol | 0.587022881 | 0.259912316 | 0.4847350 |
| lweight | 0.454460641 | 0.299613793 | 0.2382435 |
| age | -0.019637208 | -0.001278062 | 0.0000000 |
| lbph | 0.107054351 | 0.051714815 | 0.0000000 |
| svi | 0.766155885 | 0.443766276 | 0.4155778 |
| lcp | -0.105473570 | 0.078649463 | 0.0000000 |
| gleason | 0.045135964 | 0.085662976 | 0.0000000 |
| pgg45 | 0.004525324 | 0.002662990 | 0.0000000 |
Podemos ver que en el ajuste con Lasso, algunos de los coeficientes calculados llegan a valer 0, en cambio en Ridge podemos ver que se aproximan a valores bajos pero nunca llegan a tener un valor igual a 0.
LS0tDQp0aXRsZTogIlByw6FjdGljYSA4OiBSZWdyZXNpw7NuIGxpbmVhbCBjb24gcmVndWxhcml6YWNpw7NuIMK0LSBGQUEiDQphdXRob3I6ICJKdWRpdCBHb256w6FsZXogUHJvbCINCmRhdGU6ICIyMyBkZSBhYnJpbCBkZSAyMDI0Ig0Kb3V0cHV0OiANCiAgaHRtbF9ub3RlYm9vazoNCi0tLQ0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQppbnN0YWxsLnBhY2thZ2VzKCJrbml0ciIpDQppbnN0YWxsLnBhY2thZ2VzKCJnbG1uZXQiKQ0KaW5zdGFsbC5wYWNrYWdlcygiQnJxIikNCmBgYA0KDQpgYGB7cn0NCmxpYnJhcnkoa25pdHIpDQpsaWJyYXJ5KGdsbW5ldCkNCmxpYnJhcnkoQnJxKQ0KbGlicmFyeShNQVNTKQ0KYGBgDQoNCiMgRU5UUkVHQSBQVDINCg0KVGVuZW1vcyBwIHZhcmlhYmxlcyBvIHAgY2FyYWN0ZXJpc3RpY2FzIHkgbiByZXhpc3Ryb3MuDQoNCiMjIEludHJvZHVjY2nDs24gYSBsb3MgZXNjZW5hcmlvcyBkZSBwZW5hbGl6YWNpw7NuDQoNCkVsIHByb2JsZW1hIHF1ZSBzdWNlZGUgZW4gZXN0YSBwcsOhY3RpY2EgZXMgcXVlICRwXD5uJCwgcG9yIGVsbG8sIGxvcyBtw6l0b2RvcyBhcGxpY2Fkb3MgaGFzdGEgYWhvcmEgbm8gZnVuY2lvbmFyw6FuLiAoRXN0YW1vcyBlbiBwcm9ibGVtYXMgZGUgYWx0YSBkaW1lbnNpw7NuKQ0KDQpQYXJhIHBvZGVyIGF0YWNhciBlc3RvcyBjb25qdW50b3MgZGUgZGF0b3MsIHRlbmVtb3MgcXVlIHV0aWxpemFyIG90cm8gdGlwbyBkZSBjb25jbHVzaW9uZXMgcGFyYSB0b21hciBsYXMgZGVjaXNpb25lcyBjb3JyZXNwb25kaWVudGVzLiANCg0KQ29tbyBwb2RlbW9zIHJlZHVjaXIgZWwgbnVtZXJvIGRlIHZhcmlhYmxlcyBxdWUgdmFtb3MgYSB1dGlsaXphci4NCi0gMcK6OiBVdGlsaXphbW9zIHVuIGNvbmp1bnRvIGRlIHZhcmlhYmxlcyBtw61uaW1vLCB1bjogJHAnXDxuJA0KDQotICAgMsK6OiBVdGlsaXphbW9zIG3DqXRvZG9zIGRlIHJlZ3VsYXJpemFjacOzbiA6IExhc3NvIC4uLiBSZWR1Y2VuIGVuIG51bSBkZSBjb2VmZmljaWVudGVzLCBlcyBkZWNpciBxdWUgc2VhbiAwLg0KDQotICAgTGEgcmVncmVzacOzbiBSaWRnZSwgbG9zIG1pbmltaXphIHBlcm8gbm8gbG9zIGxsZWdhIGEgaWd1YWxhciBhIDAuDQoNCi0gICBMYSByZWdyZXNpw7NuIExhc3NvLCBsb3MgbWluaW1pemEgcGVybyBsbGVnYW5kbyBhIHZhbGVyIDAuDQoNCi0gICAzwro6IE3DqXRvZG9zIGRlIHByb3llY2Npw7NuOg0KDQoqKkN1YW5kbyBjYWxjdWxhbW9zIGxvcyBiZXRhLCBsb3MgcG9kZW1vcyBvYnRlbmVyIGNvbW8gZWwgbcOpdG9kbyBkZSBtw61uaW1vcyBjdWFkcmFkb3MgbyB0YW1iacOpbiBsb3Mgb2J0ZW5lbW9zIHRhbWJpw6luIHBlbnNhbmRvIGVuIHVuIHByb2JsZW1hIGRlIG9wdGltaXphY2nDs24uKioNCg0KLSAgIFJpZGdlIC1cPiBSZXN0cmljY2lvbmVzIGRlbCB0aXBvOg0KDQpTdW1hIGRlIGxvcyBiZXRhc19pIHNlYSBtZW5vciBxdWUgdW4gdmFsb3IgZGFkbw0KDQotICAgTGFzc28gLVw+IFJlc3RyaWNjaW9uZXMgZGVsIHRpcG86DQoNCiAgRW4gUmlnZGUgYmFqYW4gZGUgbWFnbml0dWQsIHkgZW4gTGFzc28gbGxlZ2FuIGFsZ3Vub3MgYSB2YWxlciAwDQoNCiAgICBTdW1hIGRlIGxvcyB2YWxvcyBhYnNvbHV0b3MgZGUgbG9zIGJldGFzX2kgeSBxdWUgc2VhIG1lbm9yIHF1ZSB1biB2YWxvciBkYWRvDQoNCiAgICBMbyBxdWUgaGFyw6EgcXVlIHNlYW4gaWd1YWxlcyBhIDAgZW4gYWxndW5vcyBjYXNvcy4NCg0KLSAgIEN1YW5kbyB0ZW5lbW9zIG11Y2hhcyB2YXJpYWJsZXMgZXMgZGlmw61jaWwgcXVlIHRvZGFzIHNlYW4gZW50cmUgc8OtIGluZGVwZW5kaWVudGVzLiBFcyBkZWNpciwgZWwgcHJvYmxlbWEgcXVlIHNlIGRhIGVuIGxhIG11bHRpY29sZWFuaWRhZCBlcyBxdWUgbm8gcG9kcsOtYW1vcyBhcGxpY2FyIGVsIG1vZGVsbyBkZSByZWdyZXNpw7NuLg0KDQpDdWFuZG8gZXN0YW1vcyBlbiBlc3RvcyBwcm9ibGVtYXMgdGVuZHJlbW9zIHF1ZSBhcGxpY2FyOg0KDQotICAgKipQcm9jZWRlbWVudG9zIGRlIHZhbGlkYWNpw7NuIGNydXphZGEqKjogTnVlc3RybyBjb25qdW50byBkZSBkYXRvcyBsbyBkaXZpZGltb3MgZW4gMiB0cm96b3MsIHVuIDgwJSB5IHVuIDIwJQ0KDQotICAgRWwgODAlIHNlcmlhIGVsIGNvbmp1bnRvIGRlIGVudHJlbmFtaWVudG8NCg0KLSAgIDIwJSBzZXLDrWEgZWwgY29uanVudG8gZGUgdmFsaWRhY2nDs24geSBkZSB0ZXN0DQoNCi0gICBBanVzdGFtb3MgdW4gbW9kZWxvIGNvbiBzb2xvIGVsIDgwJSBkZSBsb3MgZGF0b3MgLVw+IGxhIG11ZXN0cmEgZGUgbG9zIGRhdG9zIGRlIGVudHJlbmFtaWVudG8NCg0KT3RybyBtw6l0b2RvIHF1ZSBwb2Ryw61hbW9zIHV0aWxpemFyIGVzOg0KDQotICAgRWo6IHRlbmVtb3MgMTAwIGRhdG9zDQoNCi0gICBBanVzdGFtb3MgZWwgbW9kZWxvIGNvbiBsb3MgOTkgcHJpbWVyb3MgZGF0b3MuDQoNCi0gICBFbiBjYWRhIGl0ZXJhY2nDs24gdGVuZW1vcyBxdWUgcXVpdGFyIHVubyBkZSBsb3MgZGF0b3MgZGUgY2FkYSB2ZXosIGFzw60gcXVlIG1pcmFtb3MgbGEgcHJlZGljY2nDs24gZGVsIG1vZGVsby4NCg0KLSAgIEFsIGNhbGN1bGFyIGxhIHByZWRpY2Npw7NuIG1lZGlhLCBlbiBjYWRhIGl0ZXJhY2nDs24gdmFtb3MgbW92aWVuZG8gdW4gcmVnaXN0cm8uDQoNCk90cm86DQoNCi0gICBBcGxpY2FuZG8gbG9zIGtmbG9sZHMuDQotICAgUXVpdGFtb3MgayByZWdpc3Ryb3MgZGUgY2FkYSB2ZXoNCg0KKipNZXRvZG8gcmFwaWRvIHBhcmEgcmVkdWNpciB2YXJpYWJsZXMtXD4gZXN0dWRpYXIgbGFzIHZhcmlhYmxlcyBwcmluY2lwYWxlcyAtXD4gUENBKioNCg0KIyMjIENhcmdhbW9zIGVsIGNvbmp1bnRvIGRlIGRhdG9zDQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmRhdGEoUHJvc3RhdGUpDQphdHRhY2goUHJvc3RhdGUpDQpgYGANCg0KVmVtb3MgbGEgZm9ybWEgeSB2YXJpYWJsZXMgZGVsIGNvbmp1bnRvIGRlIGRhdG9zOg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpoZWFkKFByb3N0YXRlKQ0KYGBgDQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCnBsb3QoUHJvc3RhdGUpDQpgYGANCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KbiA8LSBucm93KFByb3N0YXRlKQ0KcCA8LSBuY29sKFByb3N0YXRlKQ0KDQpuO3ANCmBgYA0KDQpFcyBkZWNpciwgdGVuZW1vcyA5IHZhcmlhYmxlcyB5IDk3IG9ic2VydmFjaW9uZXMNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KbmFtZXMoUHJvc3RhdGUpDQpgYGANCg0KRGlzcG9uZW1vcyBhc8OtIGNvbW8gdmFyaWFibGVzIGV4cGxpY2F0aXZhcyBkZSBtZWRpZGFzIGNvbW8gZWwgdm9sdW1lbiBkZWwgdHVtb3IgKGxjYXZvbCksIGVsIGxvZy1wZXNvIGRlIGxhIHByw7NzdGF0YSAobHdlaWdodCksIGxhIGVkYWQgKGFnZSksIGxhIGNhbnRpZGFkIGRlIGhpcGVycGxhc2lhIHByb3N0w6F0aWNhIGJlbmlnbmEgKGxicGgpLCBlbCBncmFkbyBkZSBpbmZpbHRyYWNpw7NuIGRlbCB0dW1vciBlbiBsYSB2ZXPDrWN1bGEgc2VtaW5hbCAoc3ZpKSwgZWwgZ3JhZG8gZGUgcGVuZXRyYWNpw7NuIGNhcHN1bGFyIChsY3ApIHkgZWwgc2NvcmUgZGUgR2xlYXNvbiAoZ2xlYXNvbiB5IHBnZzQ1KS4NCg0KUHJvcG9uZW1vcyB1biBtb2RlbG8gZGUgcmVncmVzacOzbiBsaW5lYWwgcGFyYSBleHBsaWNhciBsYSB2YXJpYWJsZSBscHNhIGVuIGZ1bmNpw7NuIGRlbCByZXN0byBkZSBtYXJjYWRvcmVzIGNsw61uaWNvcyAocCA9IDgpDQoNCiQkDQpZID0gXGJldGFfMCArIFxiZXRhXzEgWF8xICsgXGxkb3RzICsgXGJldGFfcCBYX3AgKyBcZXBzaWxvbg0KJCQNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KbW9kZWxvX2NvbXBsZXRvID0gbG0obHBzYSB+IGxjYXZvbCArIGx3ZWlnaHQgKyBhZ2UgICsgbGJwaCArICBzdmkgKyAgbGNwICsgZ2xlYXNvbiArIHBnZzQ1ICkNCm1vZGVsb19jb21wbGV0bw0KYGBgDQoNClBvZGVtb3Mgb2J0ZW5lciBsb3MgY29lZmljaWVudGVzIGRlbCBtb2RlbG8gZW4gYmFzZSBhIGxhIHByaW1lcmEgY29sdW1hIHF1ZSBzZSBtdWVzdHJhIGVuIGVsIHN1bW1hcnkgZGVsIG1vZGVsbzoNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0Kc3VtbWFyeShtb2RlbG9fY29tcGxldG8pDQpgYGANCg0KUG9yIG90cmEgcGFydGUgdGFtYmnDqW4gcG9kZW1vcyB2ZXIgcXVlIGxhcyB2YXJpYWJsZXMgbcOhcyBzaWduaWZpY2F0aXZhcyBzb246IC0gbGNhdm9sIC1cPiAwLjAwMSAtIGx3ZWlnaHQgLVw+IDAuMDEgLSBzdmkgLVw+IDAuMDENCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0Kc3VtIDwtIHN1bW1hcnkobW9kZWxvX2NvbXBsZXRvKQ0KY29lZmljaWVudGVfbHBzYSA8LSBzdW0kY29lZmZpY2llbnRzWzFdDQpjb2VmaWNpZW50ZV9sd2VpZ2h0IDwtIHN1bSRjb2VmZmljaWVudHNbMl0NCmNvZWZpY2llbnRlX2FnZSA8LSBzdW0kY29lZmZpY2llbnRzWzNdDQpjb2VmaWNpZW50ZV9sYnBoICA8LSBzdW0kY29lZmZpY2llbnRzWzRdDQpjb2VmaWNpZW50ZV9zdmkgIDwtIHN1bSRjb2VmZmljaWVudHNbNV0NCmNvZWZpY2llbnRlX2xjcCA8LSBzdW0kY29lZmZpY2llbnRzWzZdDQpjb2VmaWNpZW50ZV9nbGVhc29uICAgPC0gc3VtJGNvZWZmaWNpZW50c1s3XQ0KY29lZmljaWVudGVfcGdnNDUgPC0gc3VtJGNvZWZmaWNpZW50c1s4XQ0KY29lZmljaWVudGVfbHBzYTsNCmNvZWZpY2llbnRlX2x3ZWlnaHQ7DQpjb2VmaWNpZW50ZV9hZ2U7DQpjb2VmaWNpZW50ZV9sYnBoOw0KY29lZmljaWVudGVfc3ZpOw0KY29lZmljaWVudGVfbGNwOw0KY29lZmljaWVudGVfZ2xlYXNvbjsNCmNvZWZpY2llbnRlX3BnZzQ1Ow0KYGBgDQoNCiMjIEFqdXN0YW5kbyBlbCBtb2RlbG8gdXNhbmRvIGVsIG3DqXRvZG8gZGUgZXN0aW1hY2nDs24gZGUgbWluaW1vcyBjdWFkcmFkb3MgT1NMIG1hbnVhbG1lbnRlOg0KDQpSZXNvbHZpZW5kbyBlbCBwcm9ibGVtYSBkZSBvcHRpbWl6YWNpw7NuOg0KDQokJA0KXG1pbl97XGJldGF9IFxzdW1fe2k9MX1ee259ICh5X2kgLSBcYmV0YV8wIC0gXGJldGFfMSB4X3tpMX0gLSBcbGRvdHMgLSBcYmV0YV9wIHhfe2lwfSleMg0KJCQNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KWCA9IG1hdHJpel9kaXNlw7FvID0gbW9kZWwubWF0cml4KG1vZGVsb19jb21wbGV0bykNClgNCmBgYA0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQptYXRyaXpfaGF0ID0gWCAlKiUgc29sdmUodChYKSAlKiUgWCkgJSolIHQoWCkNCkggPSBtYXRyaXpfaGF0DQpgYGANCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KYmV0YV9oYXQgPC0gc29sdmUodChYKSAlKiUgSCAlKiUgWCkgJSolIHQoWCkgJSolIEggJSolIFkNCmJldGFfaGF0DQpgYGANCg0KIyBDb2VmaWNpZW50ZXMgZGVsIG1vZGVsbyBjb21wbGV0bw0KDQpPIHRhbWJpw6luIHBvZGVtb3Mgb2J0ZW5lciBsb3MgY29lZmljaWVudGVzIGRlIGxhIHNpZ3VpZW50ZSBmb3JtYToNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KY29lZihtb2RlbG9fY29tcGxldG8pDQpgYGANCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0Ka2FibGUoY29lZihtb2RlbG9fY29tcGxldG8pLCBmb3JtYXQ9J21hcmtkb3duJykNCmBgYA0KDQojIyBFc3RpbWFjacOzbiByaWRnZQ0KDQokJA0KXG1pbl97XGJldGF9IFxzdW1fe2k9MX1ee259ICh5X2kgLSBcYmV0YV8wIC0gXGJldGFfMSB4X3tpMX0gLSBcbGRvdHMgLSBcYmV0YV9wIHhfe2lwfSleMiArIFxsYW1iZGEgXHN1bV97aj0xfV57cH0gXGJldGFfal4yDQokJCBTaSBlc3RhYmxlY2Vtb3MgbGFtYmRhIGNvbW8gMCwgZWwgdmFsb3IgZGUgbG9zIGNvZWZpY2llbnRlcyBzaWd1ZSBzaWVuZG8gZWwgbWlzbW8gcXVlIGNvbiBlbCBwcm9ibGVtYSBvcmlnaW5hbDoNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KWV9oYXQgPC0gSCAlKiUgWQ0KDQpsYW1iZGEgPC0gMA0KDQpiZXRhX3JpZGdlIDwtIHNvbHZlKHQoWCkgJSolIEggJSolIFggKyBsYW1iZGEgKiBkaWFnKG5jb2woWCkpKSAlKiUgdChYKSAlKiUgSCAlKiUgWQ0KDQpwcmludChiZXRhX3JpZGdlKQ0KYGBgDQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWYobW9kZWxvX2NvbXBsZXRvKQ0KYGBgDQoNClBvZGVtb3MgdmVyIHF1ZSBzb24gbG9zIG1pc21vcyBjb2VmaWNpZW50ZXMuDQoNCkVuIGNhbWJpbyBzaSB2YXJpYW1vcyBlbCB2YWxvciBkZSBsYW1iZGEsIHBvZGVtb3MgdmVyIGNhbWJpb3Mgbm90YWJsZXMgZW4gbG9zIGNvZWZpY2llbnRlczoNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KWV9oYXQgPC0gSCAlKiUgWQ0KDQpsYW1iZGEgPC0gMC4xDQoNCmJldGFfcmlkZ2UgPC0gc29sdmUodChYKSAlKiUgSCAlKiUgWCArIGxhbWJkYSAqIGRpYWcobmNvbChYKSkpICUqJSB0KFgpICUqJSBIICUqJSBZDQoNCnByaW50KGJldGFfcmlkZ2UpDQpgYGANCg0KIyMgUmlkZ2UgYXV0b21hdGljYW1lbnRlDQoNClByaW1lcm8gbmVjZXNpdGFtb3MgY29udmVydGlyIGxvcyBkYXRvcyBkZSBQcm9zdGF0ZSBlbiB1bmEgbWF0cml6IGNvbiBlbCBjb21hbmRvIGFzLm1hdHJpeCgpDQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCm1hdHJpeF9wcm9zdGF0ZSA8LSBhcy5tYXRyaXgoUHJvc3RhdGVbYygxOjgpXSkNCm1hdHJpeF9wcm9zdGF0ZQ0KYGBgDQoNClVuYSB2ZXogaGVjaGEgbGEgdHJhbnNmb3JtYWNpw7NuIHByb2NlZGVtb3MgYSBoYWNlciBlbCBtb2RlbG8gY29uIGxvcyBkYXRvcy4NCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KcnIgPC0gbG0ucmlkZ2UobHBzYSB+IG1hdHJpeF9wcm9zdGF0ZSwgbGFtYmRhID0gMCkNCmBgYA0KDQpBIGNvbnRpbnVhY2nDs24gcHJvY2VkZW1vcyBhIG9idGVuZXIgbG9zIGNvZWZpY2llbnRlcyBkZWwgbW9kZWxvDQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWYocnIpDQpgYGANCg0KUG9kZW1vcyB2ZXIgcXVlIGVzdG9zIGNvZWZpY2llbnRlcyBjb2luY2lkZW4gY29uIGxvcyBvYnRlbmlkb3MgY29uIGVsIG3DqXRvZG8gZGUgbcOtbmltb3MgY3VhZHJhZG9zLg0KDQpQb3Igb3RyYSBwYXJ0ZSwgcG9kZW1vcyBjYWxjdWxhciB0YW1iacOpbiBsb3MgY29lZmljaWVudGVzIGVzdGFuZGFyaXphZG9zOg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpyciRjb2VmDQpgYGANCg0KRXN0b3Mgw7psdGltb3MgY29lZmljaWVudGVzIG9idGVuaWRvcyBjb2luY2lkaXLDrWFuIGNvbiBsb3MgcmVzdWx0YWRvcyBkZSBhanVzdGFyIGVsIG1vZGVsbyBkZSByZWdyZXNpw7NuIGxpbmVhbCBwb3IgZWwgbcOpdG9kbyBkZSBtw61uaW1vcyBjdWFkcmFkb3MgYSBsb3MgZGF0b3MgcXVlIHNlIG9idGllbmVuIHRyYXMgcmVzdGFyIGEgY2FkYSBjb2x1bW5hIGRlIGxhIG1hdHJpeiB4IHN1IG1lZGlhIHkgZGl2aWRpciBlbnRyZSBsYSBkZXN2aWFjacOzbiB0w61waWNhLg0KDQpQb2RlbW9zIGRlbW9zdHJhcmxvIGRlIGxhIHNpZ3VpZW50ZSBmb3JtYToNCg0KUHJpbWVybyBlc2NhbGFtb3MgbGEgbWF0cml6DQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCm1hdHJpeF9wcm9zdGF0ZV9lc2NhbGFkYSA8LSBzY2FsZShtYXRyaXhfcHJvc3RhdGUpDQptYXRyaXhfcHJvc3RhdGVfZXNjYWxhZGENCmBgYA0KDQpHZW5lcmFtb3MgZWwgbW9kZWxvIGRlIHJlZ3Jlc2nDs24gZW4gYmFzZSBhIGxhIG1hdHJpeiBlc2NhbGFkYToNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KbW9kX2VzY2FsYWRvIDwtIGxtKGxwc2EgfiBtYXRyaXhfcHJvc3RhdGVfZXNjYWxhZGEpDQpgYGANCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KY29lZmZpY2llbnRzKG1vZF9lc2NhbGFkbykNCmBgYA0KDQojIyBBanVzdGUgbWVkaWFudGUgZWwgcHJvY2VkaW1pZW50byBkZSBlc3RpbWFjacOzbiBSaWRnZQ0KDQpVc2Ftb3MgZGlmZXJlbnRlcyB2YWxvcmVzIGRlbCBwYXJhbWV0cm8gbGFtYmRhOg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpsYW0gPC0gc2VxKDAsIDEwLCBieSA9IDAuMDEpDQpgYGANCg0KWSBhIGNvbnRpbnVhY2nDs24gZ2VuZXJhbW9zIGVsIG1vZGVsbzoNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KcnIgPC0gbG0ucmlkZ2UobHBzYSB+IG1hdHJpeF9wcm9zdGF0ZSwgbGFtYmRhID0gbGFtKQ0KYGBgDQoNCk1vc3RyYW1vcyBsb3MgY29lZmljaWVudGVzOg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpjb2VmKHJyKQ0KYGBgDQoNClBvZGVtb3MgdmVyIGxvcyBkaWZlcmVudGVzIHZhbG9yZXMgcXVlIG9idGllbmVuIGxvcyBwYXJhbWV0cm9zIGVzdGltYWRvcyBlbiBiYXNlIGEgY2FkYSB2YWxvciBkZSAkJFxsYW1iZGEkJA0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpwbG90KGNvZWYocnIpKQ0KYGBgDQoNCiMjIFBhcXVldGUgZ2xtbmV0OiByZWdyZXNpw7NuIHJlZ3VsYXJpemFkYQ0KDQpHbG1uZXQgcmVzdWVsdmUgZWwgc2lndWllbnRlIHByb2JsZW1hIGRlIG9wdGltaXphY2nDs246DQoNCiQkDQpcdGV4dHtNaW5pbWl6YXJ9IFxxdWFkIFxmcmFjezF9ezJufSBcc3VtX3tpPTF9XntufSBcbGVmdCggeV9pIC0gXGJldGFfMCAtIFxzdW1fe2o9MX1ee3B9IFxiZXRhX2ogeF97aWp9IFxyaWdodCleMiArIFxmcmFje1xsYW1iZGF9ezJ9IFxzdW1fe2o9MX1ee3B9IFxiZXRhX2peMg0KJCQNCg0KR2VuZXJhbW9zIHVuIG1vZGVsbyBjb24gdW5hIHJlZ3VsYXJpemFjacOzbiBhbHBoYSA9IDAgeSBlbCBwYXLDoW1ldHJvIGRlIHBlbmFsaXphY2nDs24gbGFtYmRhID0gMDoNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KDQpycl9nbG1uZXQgPC0gZ2xtbmV0KG1hdHJpeF9wcm9zdGF0ZSwgbHBzYSwgYWxwaGEgPSAwLCBsYW1iZGEgPSAwKQ0KYGBgDQoNClByb2NlZGVtb3MgYSB2ZXIgbG9zIGNvZWZpY2llbnRlcyBkZWwgbW9kZWxvDQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWYocnJfZ2xtbmV0KQ0KYGBgDQoNClZlbW9zIHF1ZSBzb24gbG9zIG1pc21vcyBjb2VmaWNpZW50ZXMgKGFwcm94aW1hZGFtZW50ZSkgcXVlIGxvcyBjYWxjdWxhZG9zIHBvciBlbCBtw6l0b2RvIGRlIG3DrW5pbW9zIGN1YWRyYWRvcy4NCg0KIyMgR0xNTkVUIGNvbiBkaWZlcmVudGVzIHZhbG9yZXMgZGVsIHBhcsOhbWV0cm8gbGFtYmRhIGRlIHBlbmFsaXphY2nDs246DQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmxhbSA8LSBzZXEoMCwgMTAsIGJ5ID0gMC4wMSkNCnJyX2dsbW5ldCA8LSBnbG1uZXQobWF0cml4X3Byb3N0YXRlLCBscHNhLCBhbHBoYSA9IDAsIGxhbWJkYSA9IGxhbSkNCnJyX2dsbW5ldA0KYGBgDQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWYocnJfZ2xtbmV0KQ0KYGBgDQoNCkVuIGVzdGUgY2FzbyBzZSBkZXZ1ZWx2ZW4gbGFzIGNvbHVtbmFzIHF1ZSByZWNvZ2VuIGxvcyBwYXLDoW1ldHJvcyBlc3RpbWFkb3MgcGFyYSBjYWRhIHZhbG9yIGRlIM67IChsYXMgZXN0aW1hY2lvbmVzIHNlIGRldnVlbHZlbiBlbiBsYSBlc2NhbGEgb3JpZ2luYWwpLg0KDQpMYXMgZXN0aW1hY2lvbmVzIHBhcmEgZWwgdmFsb3IgJCRcbGFtYmRhJCQgYWxtYWNlbmFkbyBlbiDCtHJyX2dsbW5ldFwkbGFtYmRhW2pdwrQsIGxhcyBwb2RlbW9zIG9idGVuZXIgZGUgbGEgc2lndWllbnRlIGZvcm1hOg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpjb2VmKHJyX2dsbW5ldClbLGpdDQpgYGANCg0KKk9ic2VydmEgcXVlIG5vIGNvaW5jaWRlIGV4YWN0YW1lbnRlIGNvbiBsYSBmb3JtdWxhY2nDs24gb3JpZ2luYWwgZGUgcHJvYmxlbWEgeSBlc3RvIGhhY2UgcXVlIHVuIG1pc21vIHZhbG9yIGRlIM67IDY9IDAgbm8gZGV2dWVsdmEgbG9zIG1pc21vcyBlc3RpbWFkb3JlcyBjb24gbGEgZnVuY2nDs24gbG0ucmlkZ2UgeSBnbG1uZXQuIEHDum4gYXPDrSBlbCBwYXBlbCBkZSDOuyBjb21vIHBhcsOhbWV0cm8gZGUgcGVuYWxpemFjacOzbiBlcyBlbCBtaXNtbywgY29tbyBzZSBvYnNlcnZhIGVuIGxhIHNpZ3VpZW50ZSBncsOhZmljYSwgcXVlIG11ZXN0cmEgbGFzIGVzdGltYWNpb25lcyBkZSBsb3MgcGFyw6FtZXRyb3MgYSBtZWRpZGEgcXVlIGF1bWVudGEgzrsuKg0KDQojIyBQbG90IGRlIGxvcyBjb2VmaWNpZW50ZXMgZW4gcmVsYWNpw7NuIGFsIGxvZ2FyaXRtbyBkZSBsYW1iZGENCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KcGxvdChycl9nbG1uZXQsIHh2YXIgPSAibGFtYmRhIikNCmBgYA0KDQojIDIuIFJlZ3Jlc2nDs24gTGFzc28gKGxlYXN0IGFic29sdXRlIHNocmlua2FnZSBhbmQgc2VsZWN0aW9uIG9wZXJhdG9yKQ0KDQpMb3MgcGFyw6FtZXRyb3MgZGUgbGEgcmVncmVzacOzbiBMYXNzbyBsb3MgcG9kZW1vcyBvYnRlbmVyIGEgdHJhdsOpcyBkZSBsYSBzaWd1aWVudGUgbWluaW1pemFjaW9uOg0KDQokJA0KXHRleHR7TWluaW1pemFyfSBccXVhZCBcZnJhY3sxfXtufSBcc3VtX3tpPTF9XntufSBcbGVmdCggeV9pIC0gXGJldGFfMCAtIFxzdW1fe2o9MX1ee3B9IFxiZXRhX2ogeF97aSxqfSBccmlnaHQpXjIgKyBcbGFtYmRhIFxzdW1fe2o9MX1ee3B9IHxcYmV0YV9qfCANCiQkDQpQcmltZXJvIGFqdXN0YW1vcyBlbCBtb2RlbG8gY29uIGxhIGZ1bmNpw7NuIGdsbW5ldDoNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KbGFzc28gPC0gZ2xtbmV0KG1hdHJpeF9wcm9zdGF0ZSwgbHBzYSwgYWxwaGEgPSAxLCBsYW1iZGEgPSAwKQ0KbGFzc28NCmBgYA0KU2kgZGV0ZXJtaW5hbW9zIGFscGhhIGNvbW8gMSwgZXN0YXJlbW9zIGVqZWN1dGFuZG8gY29uIHVuYSByZWd1bGFyaXphY2nDs24gZGUgdGlwbyBsYXNzby4NCg0KUG9kZW1vcyBvYnRlbmVyIGxvcyBjb2VmaWNpZW50ZXMgZXN0aW1hZG9zIChlbiBlc3RlIGNhc28gaWd1YWxlcyBxdWUgbG9zIGFqdXN0YWRvcyBwb3IgbcOtbmltb3MgY3VhZHJhZG9zKToNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KY29lZihsYXNzbykNCmBgYA0KDQpQcm9iYW5kbyBhbCBpZ3VhbCBxdWUgYW50ZXMgY29uIGRpZmVyZW5lcyB2YWxvcmVzIGRlICRcbGFtYmRhJA0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KbGFtIDwtIHNlcSgwLCAxMCwgYnkgPSAwLjAxKQ0KbGFzc28gPC0gZ2xtbmV0KG1hdHJpeF9wcm9zdGF0ZSwgbHBzYSwgYWxwaGEgPSAxLCBsYW1iZGEgPSBsYW0pDQpsYXNzbw0KYGBgDQoNCk9idGVuZW1vcyBsb3Mgc2lndWllbnRlcyByZXN1bHRhZG9zOg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KY29lZihsYXNzbykNCmBgYA0KU2kgaGFjZW1vcyBwbG90IGRlIGxvcyByZXN1bHRhZG9zIG9idGVuaWRvcywgc2UgbXVlc3RyYSBsbyBzaWd1aWVudGU6DQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCnBsb3QobGFzc28pDQpgYGANCmBgYHtyLCBlY2hvID0gVFJVRX0NCnBsb3QobGFzc28sIHh2YXIgPSAibGFtYmRhIikNCmBgYA0KIyAzLiBTZWxlY2Npw7NuIGRlbCBwYXLDoW1ldHJvIGRlIHBlbmFsaXphY2nDs24NCg0KUGFyYSBvYnRlbmVyIGVsIHBhcsOhbWV0cm8gZGUgcGVuYWxpemFjacOzbiAkXGxhbWJkYSQgdXRpbGl6YXJlbW9zIGVsIHByb2Nlc28gZGUgdmFsaWRhY2nDs24gY3J1emFkYToNCg0KKkVsIHByb2Nlc28gZmlqYSB1biByYW5nbyBkZSBwb3NpYmxlcyB2YWxvcmVzIM67IHkgYWp1c3RhIGVsIG1vZGVsbyBwYXJhIGNhZGEgzrsgdXRpbGl6YW5kbyB1bmEgcGFydGUgZGUgbGEgbXVlc3RyYSBkZSBlbnRyZW5hbWllbnRvICh1biBzdWJjb25qdW50byBkZSBvYnNlcnZhY2lvbmVzIHNlIGFwYXJ0w7MgcHJldmlhbWVudGUpLiBBIGNvbnRpbnVhY2nDs24gc2UgYW5hbGl6YSBsYSBjYXBhY2lkYWQgcHJlZGljdGl2YSBkZWwgbW9kZWxvIHNvYnJlIGVsIHN1YmNvbmp1bnRvIGRlIG9ic2VydmFjaW9uZXMgYXBhcnRhZG8gKG11ZXN0cmEgZGUgdmFsaWRhY2nDs24pLiBQYXJhIGVsbG8gZXMgaGFiaXR1YWwgY2FsY3VsYXIgZWwgZXJyb3IgY3VhZHLDoXRpY28gbWVkaW8gKE1TRSkgZW4gbGEgbXVlc3RyYSBkZSB2YWxpZGFjacOzbiwgZXMgZGVjaXIqDQoNCiQkDQpcZnJhY3sxfXtuX3Z9IFxzdW1fe2k9MX1ee25fdn0gKHlfaSAtIFxoYXR7eX1faSleMg0KJCQNCipkb25kZSBudiBlcyBlbCB0YW1hw7FvIGRlIGxhIG11ZXN0cmEgZGUgdmFsaWRhY2nDs24sIHlpIHNvbiBsYXMgb2JzZXJ2YWNpb25lcyBkZSBsYSBtdWVzdHJhIGRlIHZhbGlkYWNpw7NuIGUgecuGaSBzb24gbGFzIHByZWRpY2Npb25lcyBkYWRhcyBwb3IgZWwgbW9kZWxvIGFqdXN0YWRvLiBFbCDOuyDDs3B0aW1vIGVzIGFxdWVsIHBhcmEgZWwgY3VhbCBlbCBNU0Ugc2VhIG1lbm9yKg0KDQpQYXJhIHJlYWxpemFyIGxhIHZhbGlkYWNpw7NuIGNydXphZGEgZWplY3V0YW1vcyBlbCBzaWd1aWVudGUgY8OzZGlnbzoNCg0KRWplY3V0YW5kbyBwYXJhIGxhIHJlZ3Jlc2nDs24gKipSaWRnZSoqOg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpsYW0gPC0gc2VxKDAsIDEwLCBieSA9IDAuMDEpDQpjdm91dCA8LSBjdi5nbG1uZXQobWF0cml4X3Byb3N0YXRlLCBscHNhLCBhbHBoYSA9IDAsIGxhbWJkYSA9IGxhbSkNCmN2b3V0DQpgYGANCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWYoY3ZvdXQpDQpgYGANCiooRXN0YW1vcyB1c2FuZG8gcG9yIGRlZmVjdG8gayA9IDEwKSoNCg0KUGFyYSBvYnRlbmVyIGVsICoqdmFsb3IgZGUgJFxsYW1iZGEkIHF1ZSBtaW5pbWl6YSBlbCBlcnJvcioqIGRlIHZhbGlkYWNpw7NuIGNydXphZGE6DQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmN2b3V0JGxhbWJkYS5taW4NCmBgYA0KKipWZW1vcyBxdWUgZWwgdmFsb3IgcXVlIG1pbmltaXphIGVsIGVycm9yIGVzOiAwMjEqKg0KDQpQb3Igb3RyYSBwYXJ0ZSwgcGFyYSBvYnRlbmVyIGxvcyBwYXLDoW1ldHJvcyBkZSBhanVzdGUgZGVsIG1vZGVsbyBkZSByZWdyZXNpw7NuIGNvbiByZWd1bGFyaXphY2nDs246DQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWYoY3ZvdXQsIHMgPSAibGFtYmRhLm1pbiIpDQpgYGANCipMYSBmdW5jacOzbiB0YW1iacOpbiBkZXZ1ZWx2ZSBlbCB2YWxvciBjdm91dCRsYW1iZGEuMXNlLCBxdWUgcmVwcmVzZW50YSB1biB2YWxvciBkZSDOuyBwYXJhIHVuIG1vZGVsbyBtw6FzIHNpbXBsZSBxdWUgZWwgZGUgY3ZvdXQkbGFtYmRhLm1pbiwgcGVybyBjdXlvIGVycm9yIGVzdMOhIGEgdW5hIGRlc3ZpYWNpw7NuIGVzdMOhbmRhciBkZWwgbWVqb3IuKg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpjdm91dCRsYW1iZGEuMXNlDQpgYGANCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWYoY3ZvdXQsIHMgPSAibGFtYmRhLjFzZSIpDQpgYGANCg0KRGVsIG1pc21vIG1vZG8sIHNpIHF1ZXJlbW9zIHNlbGVjY2lvbmFyIGVsIHBhcsOhbWV0cm8gzrsgZW4gZWwgYWp1c3RlIExhc3NvLCBoYXJlbW9zOg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpsYW0gPC0gc2VxKDAsIDEwLCBieSA9IDAuMDEpDQpjdm91dGwgPC0gY3YuZ2xtbmV0KG1hdHJpeF9wcm9zdGF0ZSwgbHBzYSwgYWxwaGEgPSAxLCBsYW1iZGEgPSBsYW0pDQpjdm91dGwNCmBgYA0KT2J0ZW5pZW5kbzoNCg0KVmFsb3IgZGUgJFxsYW1iZGEkIHF1ZSBtaW5pbWl6YSBlbCBlcnJvciBkZSB2YWxpZGFjacOzbiBjcnV6YWRhOg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpjdm91dGwkbGFtYmRhLm1pbg0KYGBgDQpDb24gbG9zIHBhcsOhbWV0cm9zIGFzb2NpYWRvczoNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KY29lZihjdm91dGwsIHMgPSAibGFtYmRhLm1pbiIpDQpgYGANCg0KWSB0YW1iacOpbiBzdSByZXNwZWN0aXZvIHZhbG9yIGRlICRcbGFtYmRhJCBwYXJhIHVuIG1vZGVsbyBtw6FzIHNpbXBsZSwgY3V5byBlcnJvciBlc3TDoSBhIHVuYSBkZXN2aWFjacOzbiBlc3RhbmRhciBkZWwgbWVqb3I6DQoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmN2b3V0bCRsYW1iZGEuMXNlDQpgYGANCkNvbiBzdXMgY29lZmljaWVudGVzIGFzb2NpYWRvczoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWYoY3ZvdXRsLCBzID0gImxhbWJkYS4xc2UiKQ0KYGBgDQoqTW9zdHJhbW9zIGEgY29udGludWFjacOzbiB1bmEgZ3LDoWZpY2EgZGUgbG9zIHBhcsOhbWV0cm9zIGVzdGltYWRvcyBjb24gTGFzc28ganVudG8gY29uIGVsIHZhbG9yIGRlIM67IHNlbGVjY2lvbmFkbyBwb3IgdmFsaWRhY2nDs24gY3J1emFkYSAoZWwgdmFsb3Igw7NwdGltbyBjdm91dGwkbGFtYmRhLm1pbiB5IGN2b3V0bCRsYW1iZGEuMXNlKSoNCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KcGxvdChycl9nbG1uZXQsIHh2YXIgPSAibGFtYmRhIikNCmBgYA0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpwbG90KGxhc3NvLCB4dmFyID0gJ2xhbWJkYScpDQphYmxpbmUodiA9IGxvZyhjdm91dGwkbGFtYmRhLjFzZSksIGNvbCA9ICdyZWQnLCBsdHkgPSAyKSAjIEzDrW5lYSByb2phIHF1ZSByZXByZXNlbnRhIGxhbWJkYS4xc2UNCmFibGluZSh2ID0gbG9nKGN2b3V0bCRsYW1iZGEubWluKSwgY29sID0gJ2JsdWUnLCBsdHkgPSAyKSAjIEzDrW5lYSBhenVsIHF1ZSByZXByZXNlbnRhIGxhbWJkYS5taW4NCmBgYA0KDQpTaSBqdW50YW1vcyBlbiB1bmEgbWlzbWEgdGFibGEgbG9zIHZhbG9yIGRlIFJpZGdlLCBMYXNvIHkgbG9zIHBhcsOhbWV0cm9zIGVzdGltYWRvcyBwb3IgbcOtbmltb3MgY3VhZHJhZG9zIG9idGVuZW1vcyBsYSBzaWd1aWVudGUgdGFibGE6DQoNCg0KDQpHdWFyZGFtb3MgbG9zIGNvZWZpY2llbnRlczoNCmBgYHtyLCBlY2hvID0gVFJVRX0NCmNvZWZfY3ZvdXRsIDwtIGFzLm1hdHJpeChjb2VmKGN2b3V0bCwgcyA9ICJsYW1iZGEuMXNlIikpDQpjb2VmX2N2b3VsdCA8LSBjb2VmX2N2b3V0bFssMV0NCmNvZWZfY3ZvdXQgPC0gYXMubWF0cml4KGNvZWYoY3ZvdXQsIHMgPSAibGFtYmRhLjFzZSIpKQ0KY29lZl9jdm91bCA8LSBjb2VmX2N2b3V0WywxXQ0KY29lZl9tb2RlbG9fY29tcGxldG8gPC0gYXMubWF0cml4KGNvZWYobW9kZWxvX2NvbXBsZXRvKSkNCmBgYA0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpjb2VmX2N2b3VsdA0KYGBgDQpgYGB7ciwgZWNobyA9IFRSVUV9DQpjb2VmX2N2b3VsDQpgYGANCg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KY29lZl9tb2RlbG9fY29tcGxldG8NCmBgYA0KDQpMb3MgY29tYmluYW1vcyBlbiB1bmEgbWlzbWEgdGFibGE6DQpgYGB7ciwgZWNobyA9IFRSVUV9DQp0YWJsYV9jb2VmaWNpZW50ZXMgPC0gY2JpbmQoY29lZl9tb2RlbG9fY29tcGxldG8sIGNvZWZfY3ZvdXQsIGNvZWZfY3ZvdXRsKQ0KYGBgDQoNClkgbW9zdHJhbW9zIGxhIHRhYmxhOg0KYGBge3IsIGVjaG8gPSBUUlVFfQ0KdGFibGFfY29lZmljaWVudGVzDQpgYGANCnwgICAgICAgICAgICB8ICAgICAgICAgT0xTICAgICAgICAgfCAgICAgIFJpZGdlICB8ICAgICAgTGFzc28gICB8DQp8LS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS18DQp8IChJbnRlcmNlcHQpfCAgICAgIDAuNjY5Mzk5MDI3ICAgICB8IDAuMzg0MjAyODQ0IHwgMC44NjM3OTA2ICB8DQp8IGxjYXZvbCAgICAgfCAgICAgIDAuNTg3MDIyODgxICAgICB8IDAuMjU5OTEyMzE2IHwgMC40ODQ3MzUwICB8DQp8IGx3ZWlnaHQgICAgfCAgICAgIDAuNDU0NDYwNjQxICAgICB8IDAuMjk5NjEzNzkzIHwgMC4yMzgyNDM1ICB8DQp8IGFnZSAgICAgICAgfCAgICAgLTAuMDE5NjM3MjA4ICAgICB8LTAuMDAxMjc4MDYyIHwgMC4wMDAwMDAwICB8DQp8IGxicGggICAgICAgfCAgICAgIDAuMTA3MDU0MzUxICAgICB8IDAuMDUxNzE0ODE1IHwgMC4wMDAwMDAwICB8DQp8IHN2aSAgICAgICAgfCAgICAgIDAuNzY2MTU1ODg1ICAgICB8IDAuNDQzNzY2Mjc2IHwgMC40MTU1Nzc4ICB8DQp8IGxjcCAgICAgICAgfCAgICAgLTAuMTA1NDczNTcwICAgICB8IDAuMDc4NjQ5NDYzIHwgMC4wMDAwMDAwICB8DQp8IGdsZWFzb24gICAgfCAgICAgIDAuMDQ1MTM1OTY0ICAgICB8IDAuMDg1NjYyOTc2IHwgMC4wMDAwMDAwICB8DQp8IHBnZzQ1ICAgICAgfCAgICAgIDAuMDA0NTI1MzI0ICAgICB8IDAuMDAyNjYyOTkwIHwgMC4wMDAwMDAwICB8DQoNCg0KKipQb2RlbW9zIHZlciBxdWUgZW4gZWwgYWp1c3RlIGNvbiBMYXNzbywgYWxndW5vcyBkZSBsb3MgY29lZmljaWVudGVzIGNhbGN1bGFkb3MgbGxlZ2FuIGEgdmFsZXIgMCwgZW4gY2FtYmlvIGVuIFJpZGdlIHBvZGVtb3MgdmVyIHF1ZSBzZSBhcHJveGltYW4gYSB2YWxvcmVzIGJham9zIHBlcm8gbnVuY2EgbGxlZ2FuIGEgdGVuZXIgdW4gdmFsb3IgaWd1YWwgYSAwLioqDQo=