Scoring model

Analyzed data set: generated available here

Loading the necesary packages…

Project Objectives

Obiectivul proiectului este de a pune in evidenta efectul gradului de corelatie dintre predictori asupra functiei de cost (RSS) si modul in care acesta influenteaza forma functiei obiecti (RSS) in spatiul predictorilor.

Modelul de regresie liniară multipla

Ecuatia modelului de regresie este:

\[Y_i = a + b X_i + \varepsilon_i\]

Linia de regresie estimata reprezinta media conditionata a variabilei dependente pentru o valoarea data a variabilelor independente:

\[\hat{y} = a + b x\]

Diferenta dintre valoarea estimata si valoarea observata represinta eroarea de estimare:

\[e_i = y_i - \hat{y}_i\]

Modelul OLS permite estimarea parametrilor de regresie prin minimizarea sumei patratelor erorilor (RSS):

Estimarea parametrilor de regresie

In modelul de regresie OLS estimarea parametrilor de regresie se face cu ajutorul metodei celor mai mici patrate care da si denumirea acestui model (OLS - Ordinary Least Squares)

\[ Y \mid X = x_i \sim N(\mu_i,\sigma^2) \]

Metoda este bazata pe determinarea valorilor coeficientilor de regresie care minimizeaza suma patratelor erorilor:

\[RSS = \sum_{i=1}^{n} (y_i - a - b x_i)^2 = min = \sum_{i=1}^{n} e_i^2 \]

prin minimizarea acestei functii se obtin valorile estimate ale coeficientilor de regresie:

panta de regresie sau parametrul b

\[\hat{b} = \frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^2}\]

ordonata la origine

\[\hat{a} = \bar{y} - \hat{b}\bar{x}\]

unde: \(\bar{y}\) este media variabilei răspuns, respectiv \(\bar{x}\) este media valorilor predictorului

Astfel obtinem valorile estimate sau linia de regresie estimata:

\[\hat{y}_i = \hat{a} + \hat{b}x_i\]

care aproximeaza legatura dintre variabila dependenta si variabilele predictor pentru care suma patratului erorilor este minima.

Modelul de regresie liniară multipla

Modelul de regresie liniară multiplă descrie relația dintre variabila dependentă și un set de predictori.

Ecuatia modelului de regresie pentru observația \(i\):

\[ Y_i = \beta_0 + \beta_1 X_{i1} + \beta_2 X_{i2} + \dots + \beta_p X_{ip} + \varepsilon_i \]

unde:

  • \(Y_i\) este variabila dependentă
  • \(X_{ij}\) reprezintă predictorii
  • \(\beta_j\) sunt coeficienții modelului
  • \(\varepsilon_i\) este termenul de eroare.

Media erorilor urmeaza o lege normala de medie zero si varianta constanta \(\sigma^2\).

\[ \varepsilon_i \sim N(0,\sigma^2) \]

Forma matriciala a modelului

In cazul unui set de date cu \(n\) observatii, modelul poate fi scris în formă matriciala:

\[ \mathbf{y} = X\beta + \varepsilon \]

unde:

vectorul variabilei dependente este:

\[ \mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix} \]

Matricea predictorilor este:

\[ X = \begin{bmatrix} 1 & x_{11} & x_{12} & \dots & x_{1p} \\ 1 & x_{21} & x_{22} & \dots & x_{2p} \\ \vdots & \vdots & \vdots & \dots & \vdots \\ 1 & x_{n1} & x_{n2} & \dots & x_{np} \end{bmatrix} \]

Prima coloană contine valori egale cu 1 si reprezinta ordonata la origine a modelului sau este valoarea medie a variabile dependente atunci cand toti predictori sunt nuli.

Vectorul coeficientilor este:

\[ \beta = \begin{bmatrix} \beta_0 \\ \beta_1 \\ \vdots \\ \beta_p \end{bmatrix} \]

Vectorul erorilor este:

\[ \varepsilon = \begin{bmatrix} \varepsilon_1 \\ \varepsilon_2 \\ \vdots \\ \varepsilon_n \end{bmatrix} \]

Estimarea parametrilor modelului de regresie multiplu

Parametrii modelului de regresie multipla sunt estimati folosind aceeasi metoda Ordinary Least Squares (OLS) sau metoda celor mai mici patrate, care minimizează suma patratelor erorilor.

Variabila reziduala este definit ca:

\[ e = y - X\beta \]

Functia obiectiv care trebuie minimizata este:

\[ RSS(\beta) = (y - X\beta)^T (y - X\beta) \]

Prin derivarea acestei functii in raport cu \(\beta\) si egalarea cu zero se obtin ecuatiile:

\[ X^T X \beta = X^T y \]

Daca matricea \(X^T X\) este inversabila, solutia pentru estimarea coeficientilor este de forma:

\[ \hat{\beta} = (X^T X)^{-1} X^T y \]

aceasta relatie arata ca estimarea coeficientilor depinde de: matricea produselor predictorilor \((X^T X)\), respectiv corelatia dintre predictori si variabila raspuns \(X^T y\)

Relatia este importanta deoarece permite analiza unor proprietati ale modelului de regresiie, cum ar fi:

  • multicoliniaritatea (când \((X^T X)\) este aproape singulară)

  • valorile proprii ale \((X^T X)\)

  • numarul de conditionare al matricei.

Aceasta formula reprezinta estimarea prin metoda OLS a vectorului coeficientilor si sta la baza analizei regresiei liniare. Proprietatile matricei \(X^TX\) sunt importante pentru studierea unor probleme precum multicoliniaritatea, valorile proprii ale matricei si stabilitatea estimarilor.

In ceea ce urmeaza vom estima o serie de modele de regresie OLS cu scopul de a pune in evidenta efectul gradului de corelatie dintre predictori si modul in care acesta influenteaza forma elipselor functiei obiectiv (RSS) in spatiul predictorilor.

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.952377    2.733657
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.9524     0.1980   9.861 0.000000000000000268 ***
## Xx2           2.7337     0.2097  13.034 < 0.0000000000000002 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.7431, Adjusted R-squared:  0.7378 
## F-statistic: 140.3 on 2 and 97 DF,  p-value: < 0.00000000000000022

Din punct de vedere geometric functia obiectiv RSS formeaza suprafete eliptice in spatiul coeficientilor de regresie, iar in centrul suprafetelor eliptice se gaseste solutia OLS sau valorile coeficientilor de regresie optimi. Fiecare punct este o combinatie de valori ale coeficientilor de regresie, iar tehnica OLS selecteaza punctul sau combinatia de valori ale coeficientilor de regresie pentru care suma patratului erorilor sau functia RSS este minima.

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.784936    2.855927
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7849     0.1993   8.958   0.0000000000000239 ***
## Xx2           2.8559     0.2094  13.640 < 0.0000000000000002 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.7408, Adjusted R-squared:  0.7355 
## F-statistic: 138.6 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.790427    2.865725
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7904     0.2018   8.872   0.0000000000000366 ***
## Xx2           2.8657     0.2120  13.520 < 0.0000000000000002 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.7571, Adjusted R-squared:  0.7521 
## F-statistic: 151.2 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.794573    2.875069
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7946     0.2068   8.678   0.0000000000000953 ***
## Xx2           2.8751     0.2170  13.251 < 0.0000000000000002 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.7716, Adjusted R-squared:  0.7669 
## F-statistic: 163.8 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.797356    2.884303
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7974     0.2148   8.368     0.00000000000044 ***
## Xx2           2.8843     0.2250  12.820 < 0.0000000000000002 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.7845, Adjusted R-squared:  0.7801 
## F-statistic: 176.6 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.798604    2.893849
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7986     0.2269   7.927     0.00000000000384 ***
## Xx2           2.8938     0.2371  12.203 < 0.0000000000000002 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.7961, Adjusted R-squared:  0.7919 
## F-statistic: 189.3 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.797866    2.904353
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7979     0.2453   7.329      0.0000000000695 ***
## Xx2           2.9044     0.2556  11.364 < 0.0000000000000002 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.8065, Adjusted R-squared:  0.8025 
## F-statistic: 202.2 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.794075    2.917036
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7941     0.2746   6.534        0.00000000297 ***
## Xx2           2.9170     0.2849  10.240 < 0.0000000000000002 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.816,  Adjusted R-squared:  0.8122 
## F-statistic: 215.1 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.784327    2.934923
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7843     0.3268   5.460    0.000000365949147 ***
## Xx2           2.9349     0.3371   8.706    0.000000000000083 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.8247, Adjusted R-squared:  0.8211 
## F-statistic: 228.1 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.756882    2.969856
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7569     0.4504   3.901             0.000177 ***
## Xx2           2.9699     0.4608   6.446        0.00000000447 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.8326, Adjusted R-squared:  0.8292 
## F-statistic: 241.3 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.714536    3.015728
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value             Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 < 0.0000000000000002 ***
## Xx1           1.7145     0.6298   2.722              0.00769 ** 
## Xx2           3.0157     0.6402   4.711           0.00000824 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.8364, Adjusted R-squared:  0.833 
## F-statistic:   248 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    1.529752    3.203237
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value            Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 <0.0000000000000002 ***
## Xx1           1.5298     1.3987   1.094              0.2768    
## Xx2           3.2032     1.4091   2.273              0.0252 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.8394, Adjusted R-squared:  0.8361 
## F-statistic: 253.5 on 2 and 97 DF,  p-value: < 0.00000000000000022

## y ~ X
## (Intercept)         Xx1         Xx2 
##    5.270131    0.801922    3.931668
## 
## Call:
## lm(formula = y ~ X)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7460 -1.3215 -0.2489  1.2427  4.1597 
## 
## Coefficients:
##             Estimate Std. Error t value            Pr(>|t|)    
## (Intercept)   5.2701     0.1923  27.409 <0.0000000000000002 ***
## Xx1           0.8019     4.4232   0.181               0.857    
## Xx2           3.9317     4.4336   0.887               0.377    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.903 on 97 degrees of freedom
## Multiple R-squared:  0.8401, Adjusted R-squared:  0.8368 
## F-statistic: 254.8 on 2 and 97 DF,  p-value: < 0.00000000000000022

Dupa cum se poate observa pe masura ce gradul de colinearitate creste elipsele devin foarte alungite, iar estimarile coeficientilor de regresie devin instabili (varianta este foarte mare). Motiv pentru care pentru estimarea coeficientilor de regresie sau analiza de regresie se folosesc modele robuste de regresie sau modele bazate pe regularizare sau aplicarea unei penalizari asupra coeficientilor mari

Bibliography

  1. Wickham H, Averick M, Bryan J, Chang W, McGowan LD, François R, Grolemund G, Hayes A, Henry L, Hester J, Kuhn M, Pedersen TL, Miller E, Bache SM, Müller K, Ooms J, Robinson D, Seidel DP, Spinu V, Takahashi K, Vaughan D, Wilke C, Woo K, Yutani H (2019). “Welcome to the tidyverse.” Journal of Open Source Software, 4(43), 1686. doi:10.21105/joss.01686 https://doi.org/10.21105/joss.01686.

  2. Kuhn et al., (2020). Tidymodels: a collection of packages for modeling and machine learning using tidyverse principles. https://www.tidymodels.org

  3. Waring E, Quinn M, McNamara A, Arino de la Rubia E, Zhu H, Ellis S (2022). skimr: Compact and Flexible Summaries of Data. R package version 2.1.5, https://CRAN.R-project.org/package=skimr.

  4. Peterson BG, Carl P (2020). PerformanceAnalytics: Econometric Tools for Performance and Risk Analysis. R package version 2.0.4, https://CRAN.R-project.org/package=PerformanceAnalytics.

  5. Wickham H, Pedersen T, Seidel D (2025). scales: Scale Functions for Visualization. R package version 1.4.0, https://CRAN.R-project.org/package=scales.

  6. Fox J, Weisberg S (2019). An R Companion to Applied Regression, Third edition. Sage, Thousand Oaks CA. https://socialsciences.mcmaster.ca/jfox/Books/Companion/.

  7. Schloerke B, Cook D, Larmarange J, Briatte F, Marbach M, Thoen E, Elberg A, Crowley J (2021). GGally: Extension to ‘ggplot2’. R package version 2.1.2, https://CRAN.R-project.org/package=GGally.

  8. Taiyun Wei and Viliam Simko (2021). R package ‘corrplot’: Visualization of a Correlation Matrix (Version 0.92). Available from https://github.com/taiyun/corrplot

  9. Rubba C (2023). htmltab: Assemble Data Frames from HTML Tables. R package version 0.8.2.9000, https://github.com/htmltab/htmltab.

  10. Brandon M. Greenwell and Bradley C. Boehmke (2020). Variable Importance Plots—An Introduction to the vip Package. The R Journal, 12(1), 343–366. URL https://doi.org/10.32614/RJ-2020-013.

  11. Marvin N. Wright, Andreas Ziegler (2017). ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R. Journal of Statistical Software, 77(1), 1-17. doi:10.18637/jss.v077.i01

  12. Robinson D, Hayes A, Couch S (2023). broom: Convert Statistical Objects into Tidy Tibbles. R package version 1.0.5, https://CRAN.R-project.org/package=broom. 13.H. Wickham. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, 2016.

  13. Wickham H, Hester J, Bryan J (2023). readr: Read Rectangular Text Data. R package version 2.1.4, https://CRAN.R-project.org/package=readr.

  14. Zhu H (2021). kableExtra: Construct Complex Table with ‘kable’ and Pipe Syntax. R package version 1.3.4, https://CRAN.R-project.org/package=kableExtra.

  15. Cui B (2020). DataExplorer: Automate Data Exploration and Treatment. R package version 0.8.2, https://CRAN.R-project.org/package=DataExplorer.

  16. Rushworth A (2022). inspectdf: Inspection, Comparison and Visualisation of Data Frames. R package version 0.0.12, https://CRAN.R-project.org/package=inspectdf.

  17. Grosjean P, Ibanez F (2018). pastecs: Package for Analysis of Space-Time Ecological Series. R package version 1.3.21, https://CRAN.R-project.org/package=pastecs.

  18. https://www.kaggle.com/datasets/rhuebner/human-resources-data-set

  19. William Revelle (2023). psych: Procedures for Psychological, Psychometric, and Personality Research. Northwestern University, Evanston, Illinois. R package version 2.3.9, https://CRAN.R-project.org/package=psych.

  20. B. Venables, Modern Applied Statistics With S, 2002, Edition: 4thPublisher: Springer-Verlag, DOI: 10.1007/b97626.

  21. Friedman J, Tibshirani R, Hastie T (2010). “Regularization Paths for Generalized Linear Models via Coordinate Descent.” Journal of Statistical Software, 33(1), 1-22. doi:10.18637/jss.v033.i01 https://doi.org/10.18637/jss.v033.i01.

  22. Martin Maechler, Peter Rousseeuw, Christophe Croux, Valentin Todorov, Andreas Ruckstuhl, Matias Salibian-Barrera, Tobias Verbeke, Manuel Koller, c(“Eduardo”, “L. T.”) Conceicao and Maria Anna di Palma (2023). robustbase: Basic Robust Statistics R package version 0.99-1. URL http://CRAN.R-project.org/package=robustbase

  23. https://www.linkedin.com/pulse/lasso-regression-clearly-explained-bhabani-shankear-basak/ accesat la data 17.12.2024

  24. U. Riswanto, Ridge regression is a fantastic choice when you need a balance between flexibility and simplicity, especially in cases where you have lots of features or multicollinearity issues, https://ujangriswanto08.medium.com/a-beginners-guide-to-ridge-regression-and-regularization-in-machine-learning-4aeae6ec7680 accesat la data 25.12.2024.

  25. https://www.quora.com/What-are-the-pros-and-cons-of-lasso-regression accesat la data de 19.12.2024.

  26. https://medium.com/@shruti.dhumne/what-is-lasso-regression-bd44addc448c accesat la data de 19.12.2024.

  27. J. Gallier, J. Quaintance, Solving the Elastic Net and Lasso Regression Problems, 2024, https://www.cis.upenn.edu/~cis5150/elastic-net.pdf accesat la data de 19.12.2024.

  28. H. Zou, T. Hastie, Regularization and Variable Selection via the Elastic Net, 2004, https://statanaly.com/wp-content/uploads/2023/04/elasticnet.pdf accesat la data 19.12.2024.

  29. E. Rodola, A. Torsello, T. Harada, Y. Kuniyoshi, D. Cremers, Elastic Net Constraints for Shape Matching, https://cvg.cit.tum.de/_media/spezial/bib/rodola-iccv13.pdf accesat la data 19.12.2024.

  30. P. Mohan, Ridge, Lasso & Elastic Net Regression, 2021, https://blog.devgenius.io/ridge-lasso-elastic-net-regression-2ea752186e51 accesat la data 17.12.2024.

  31. https://dev.to/harsimranjit_singh_0133dc/elastic-net-regularization-balancing-between-l1-and-l2-penalties-3ib7 accesat la data 17.12.2024. 33.Arthur E. Hoerl, Robert W. Kennard, TECHNOMETRICS, 1970, 12, 1, Ridge Regression: Biased Estimation for Nonorthogonal Problems.

  32. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning Data Mining, Inference, and Prediction, 2017, Springer Science+Business Media, ISBN: 978-0-387-84857-0.

  33. Xavier Bourret Sicotte, Ridge and Lasso: visualizing the optimal solutions, 2018, https://xavierbourretsicotte.github.io/ridge_lasso_visual.html.

  34. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning Data Mining, Inference, and Prediction, 2017, Springer Science+Business Media, ISBN: 978-0-387-84857-0.

  35. Arthur E. Hoerl, Robert W. Kennard, TECHNOMETRICS, 1970, 12, 1, Ridge Regression: Biased Estimation for Nonorthogonal Problems.

  36. Hui Zou and Trevor Hastie, Regularization and variable selection via the elastic net, J. R. Statist. Soc. B, 2005, 67, Part 2, pp. 301–320.