Multiple Linear Regression Model
Sadia Fathima
I: Introduction:
In this project, we aim to develop a multiple linear regression model to predict the number of active physicians in 1990 (Y) using explanatory variables from the CDI2 dataset. The analysis aims to explore the relationship between the response variable and predictors such as population demographics, socioeconomic factors, and geographic regions to identify significant predictors.
II. Explanatory Data Analysis:
##
## Call:
## lm(formula = Y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 +
## X10, data = cdi)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1807.46 -123.83 2.45 99.55 2166.55
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -861.7972194 228.5814978 -3.770 0.000186 ***
## X1 0.0104726 0.0133407 0.785 0.432883
## X2 -0.0019568 0.0002703 -7.240 0.00000000000209 ***
## X3 21.7330229 5.4960655 3.954 0.00008977916614 ***
## X4 2.3681769 5.8668940 0.404 0.686671
## X5 0.4899088 0.0246557 19.870 < 0.0000000000000002 ***
## X6 -0.0010822 0.0007331 -1.476 0.140643
## X7 8.9006586 5.4373202 1.637 0.102373
## X8 -9.2467116 9.4466768 -0.979 0.328215
## X9 0.1468216 0.0104644 14.031 < 0.0000000000000002 ***
## X10 41.0279640 20.1457105 2.037 0.042308 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 364.8 on 429 degrees of freedom
## Multiple R-squared: 0.9594, Adjusted R-squared: 0.9585
## F-statistic: 1014 on 10 and 429 DF, p-value: < 0.00000000000000022## (Intercept) X5 X9 X2 X3
## -844.442972913 0.505978117 0.140416515 -0.001870581 21.510091334
## X10 X6
## 56.971930685 -0.001111988## (Intercept) X2 X3 X5 X6
## -844.442972913 -0.001870581 21.510091334 0.505978117 -0.001111988
## X9 X10
## 0.140416515 56.971930685## (Intercept) X5 X9 X2 X3
## -844.442972913 0.505978117 0.140416515 -0.001870581 21.510091334
## X10 X6
## 56.971930685 -0.001111988## (Intercept) X2 X3 X5 X6
## -844.442972913 -0.001870581 21.510091334 0.505978117 -0.001111988
## X9 X10
## 0.140416515 56.971930685## (Intercept) X5 X9 X2 X3
## -827.465755236 0.500564060 0.146022436 -0.002065138 21.310719958
## X10
## 56.491145059## (Intercept) X2 X3 X5 X9
## -827.465755236 -0.002065138 21.310719958 0.500564060 0.146022436
## X10
## 56.491145059## (Intercept) X5 X9 X2 X3
## -827.465755236 0.500564060 0.146022436 -0.002065138 21.310719958
## X10
## 56.491145059## (Intercept) X2 X3 X5 X9
## -827.465755236 -0.002065138 21.310719958 0.500564060 0.146022436
## X10
## 56.491145059## model p adjR2 CP BIC
## 1 Y,X5 2 0.9031620 584.925122 -1016.105
## 2 Y,X5 2 0.9473357 584.925122 -1016.105
## 3 Y,X5 2 0.9549641 584.925122 -1016.105
## 4 Y,X5 2 0.9574870 584.925122 -1016.105
## 5 Y,X5 2 0.9583746 584.925122 -1016.105
## 6 Y,X5 2 0.9585061 584.925122 -1016.105
## 7 Y,X5 2 0.9585955 584.925122 -1016.105
## 8 Y,X5 2 0.9585673 584.925122 -1016.105
## 9 Y,X5,X9 3 0.9031620 119.950861 -1279.029
## 10 Y,X5,X9 3 0.9473357 119.950861 -1279.029
## 11 Y,X5,X9 3 0.9549641 119.950861 -1279.029
## 12 Y,X5,X9 3 0.9574870 119.950861 -1279.029
## 13 Y,X5,X9 3 0.9583746 119.950861 -1279.029
## 14 Y,X5,X9 3 0.9585061 119.950861 -1279.029
## 15 Y,X5,X9 3 0.9585955 119.950861 -1279.029
## 16 Y,X5,X9 3 0.9585673 119.950861 -1279.029
## 17 Y,X2,X5,X9 4 0.9031620 40.627441 -1342.800
## 18 Y,X2,X5,X9 4 0.9473357 40.627441 -1342.800
## 19 Y,X2,X5,X9 4 0.9549641 40.627441 -1342.800
## 20 Y,X2,X5,X9 4 0.9574870 40.627441 -1342.800
## 21 Y,X2,X5,X9 4 0.9583746 40.627441 -1342.800
## 22 Y,X2,X5,X9 4 0.9585061 40.627441 -1342.800
## 23 Y,X2,X5,X9 4 0.9585955 40.627441 -1342.800
## 24 Y,X2,X5,X9 4 0.9585673 40.627441 -1342.800
## 25 Y,X2,X3,X5,X9 5 0.9031620 15.127747 -1363.090
## 26 Y,X2,X3,X5,X9 5 0.9473357 15.127747 -1363.090
## 27 Y,X2,X3,X5,X9 5 0.9549641 15.127747 -1363.090
## 28 Y,X2,X3,X5,X9 5 0.9574870 15.127747 -1363.090
## 29 Y,X2,X3,X5,X9 5 0.9583746 15.127747 -1363.090
## 30 Y,X2,X3,X5,X9 5 0.9585061 15.127747 -1363.090
## 31 Y,X2,X3,X5,X9 5 0.9585955 15.127747 -1363.090
## 32 Y,X2,X3,X5,X9 5 0.9585673 15.127747 -1363.090
## 33 Y,X2,X3,X5,X9,X10 6 0.9031620 6.832758 -1367.299
## 34 Y,X2,X3,X5,X9,X10 6 0.9473357 6.832758 -1367.299
## 35 Y,X2,X3,X5,X9,X10 6 0.9549641 6.832758 -1367.299
## 36 Y,X2,X3,X5,X9,X10 6 0.9574870 6.832758 -1367.299
## 37 Y,X2,X3,X5,X9,X10 6 0.9583746 6.832758 -1367.299
## 38 Y,X2,X3,X5,X9,X10 6 0.9585061 6.832758 -1367.299
## 39 Y,X2,X3,X5,X9,X10 6 0.9585955 6.832758 -1367.299
## 40 Y,X2,X3,X5,X9,X10 6 0.9585673 6.832758 -1367.299
## 41 Y,X2,X3,X5,X6,X9,X10 7 0.9031620 6.460024 -1363.620
## 42 Y,X2,X3,X5,X6,X9,X10 7 0.9473357 6.460024 -1363.620
## 43 Y,X2,X3,X5,X6,X9,X10 7 0.9549641 6.460024 -1363.620
## 44 Y,X2,X3,X5,X6,X9,X10 7 0.9574870 6.460024 -1363.620
## 45 Y,X2,X3,X5,X6,X9,X10 7 0.9583746 6.460024 -1363.620
## 46 Y,X2,X3,X5,X6,X9,X10 7 0.9585061 6.460024 -1363.620
## 47 Y,X2,X3,X5,X6,X9,X10 7 0.9585955 6.460024 -1363.620
## 48 Y,X2,X3,X5,X6,X9,X10 7 0.9585673 6.460024 -1363.620
## 49 Y,X2,X3,X5,X6,X7,X9,X10 8 0.9031620 6.531748 -1359.499
## 50 Y,X2,X3,X5,X6,X7,X9,X10 8 0.9473357 6.531748 -1359.499
## 51 Y,X2,X3,X5,X6,X7,X9,X10 8 0.9549641 6.531748 -1359.499
## 52 Y,X2,X3,X5,X6,X7,X9,X10 8 0.9574870 6.531748 -1359.499
## 53 Y,X2,X3,X5,X6,X7,X9,X10 8 0.9583746 6.531748 -1359.499
## 54 Y,X2,X3,X5,X6,X7,X9,X10 8 0.9585061 6.531748 -1359.499
## 55 Y,X2,X3,X5,X6,X7,X9,X10 8 0.9585955 6.531748 -1359.499
## 56 Y,X2,X3,X5,X6,X7,X9,X10 8 0.9585673 6.531748 -1359.499
## 57 Y,X2,X3,X5,X6,X7,X8,X9,X10 9 0.9031620 7.827656 -1354.133
## 58 Y,X2,X3,X5,X6,X7,X8,X9,X10 9 0.9473357 7.827656 -1354.133
## 59 Y,X2,X3,X5,X6,X7,X8,X9,X10 9 0.9549641 7.827656 -1354.133
## 60 Y,X2,X3,X5,X6,X7,X8,X9,X10 9 0.9574870 7.827656 -1354.133
## 61 Y,X2,X3,X5,X6,X7,X8,X9,X10 9 0.9583746 7.827656 -1354.133
## 62 Y,X2,X3,X5,X6,X7,X8,X9,X10 9 0.9585061 7.827656 -1354.133
## 63 Y,X2,X3,X5,X6,X7,X8,X9,X10 9 0.9585955 7.827656 -1354.133
## 64 Y,X2,X3,X5,X6,X7,X8,X9,X10 9 0.9585673 7.827656 -1354.133model: Y~ X2 X3 X5 X9 X10 bc get from AIC and CPmallows
## 1 2 8 12 21 34 36 43 48 50 53 67
## 1 2 8 12 21 34 36 43 48 50 53 67
##
## Shapiro-Wilk normality test
##
## data: ei
## W = 0.81338, p-value < 0.00000000000000022##
## Fligner-Killeen test of homogeneity of variances
##
## data: cdi$ei and cdi$Group
## Fligner-Killeen:med chi-squared = 99.399, df = 1, p-value <
## 0.00000000000000022## [1] 0.0000000000000000006595177## Y X2 X3 X5 X9 X10
## Y 1.00000000 0.94024859 0.11969924 0.95046439 0.94811057 0.02464607
## X2 0.94024859 1.00000000 0.07837212 0.92373836 0.98674763 0.06943707
## X3 0.11969924 0.07837212 1.00000000 0.07453191 0.07116151 0.05241407
## X5 0.95046439 0.92373836 0.07453191 1.00000000 0.90206155 -0.00310692
## X9 0.94811057 0.98674763 0.07116151 0.90206155 1.00000000 0.03768546
## X10 0.02464607 0.06943707 0.05241407 -0.00310692 0.03768546 1.00000000##
## Durbin-Watson test
##
## data: model
## DW = 2.2359, p-value = 0.9926
## alternative hypothesis: true autocorrelation is greater than 0## 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19 22 23
## 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19 22 23
## 25 28 32 34 39 41 42 47 48 50 53 67 69 70 73 102 123 187 269 303
## 25 28 32 34 39 41 42 47 48 50 53 67 69 70 73 102 123 187 269 303
## 344 357 363 392 405 436
## 344 357 363 392 405 436## 1 2 3 4 5 6 8 9 10 11 12 16 18 19 21 22 24 25 28 32
## 1 2 3 4 5 6 8 9 10 11 12 16 18 19 21 22 24 25 28 32
## 34 36 39 43 47 48 50 52 53 58 67 72 73 168 258 363 418
## 34 36 39 43 47 48 50 52 53 58 67 72 73 168 258 363 418
## [1] 1 2 8 12 34 48 50 53 67we use standardize residual to find outliers and cook’s distance to find influential points. Then we use common lines between standsardize residual, influetial point and high leverage points as outliers that have to remove X2 X5 X9 are highly correlated.
## Analysis of Variance Table
##
## Model 1: Y ~ X2 + X3 + X5 + X9 + X10
## Model 2: Y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X10
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 3 5975946
## 2 0 0 3 5975946 NaN NaN## (Intercept) X2 X3 X5 X9 X10
## -9788.0579 -0.0023 293.5094 0.3920 0.1668 702.4431## 1 % 99 %
## (Intercept) -30811.63273428 11235.5169409
## X2 -0.01525614 0.0106055
## X3 -265.42939381 852.4481201
## X5 -1.06972935 1.8538239
## X9 -0.37480274 0.7083886
## X10 -2581.03868142 3985.9249380##
## Call:
## lm(formula = Y ~ X2 + X3 + X5 + X9 + X10, data = cdi_cleaned)
##
## Residuals:
## 1 2 8 12 34 48 50 53 67
## 255.3 -114.0 -796.2 -621.4 -417.3 1292.7 1479.8 -900.6 -178.3
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -9788.057897 4630.026560 -2.114 0.1249
## X2 -0.002325 0.002848 -0.817 0.4740
## X3 293.509363 123.095207 2.384 0.0972 .
## X5 0.392047 0.321927 1.218 0.3103
## X9 0.166793 0.119276 1.398 0.2564
## X10 702.443128 723.121929 0.971 0.4030
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1411 on 3 degrees of freedom
## Multiple R-squared: 0.9848, Adjusted R-squared: 0.9595
## F-statistic: 38.88 on 5 and 3 DF, p-value: 0.006274