STAT 3220 Unit 2.4
During periods of high electricity demand, especially during the hot summer months, the power output from a gas turbine engine can drop dramatically. One way to counter this drop in power is by cooling the inlet air to the gas turbine. An increasingly popular cooling method uses high pressure inlet fogging. The performance of a sample of 67 gas turbines augmented with high pressure inlet fogging was investigated in the Journal of Engineering for Gas Turbines and Power (January 2005). One measure of performance is heat rate (kilojoules per kilowatt per hour). Heat rates for the 67 gas turbines, saved in the gasturbine file. Read some of the engineering applications from this journal here:
Multicollinearity and Variable Screening
Step 1: Collect the Data
- Data Compilation: These data were collected from researchers in a reputable journal.
- Data Organization: Are the data organized for MLR?- each row contains information about one observation (experimental unit) including all of the explanatory predictors and the response.
- Suitability of response variable for regression: View a histogram of response variable. Primarily, it should be continuous with few outliers. Data should not be recorded over time. It is sometimes useful if the response variable is unimodal and symmetric, but that is NOT a requirement. (Further, we will not use “rank” as a response variable in MLR. We need a continuous variable with several values of each response, if the variable is discrete.)
gasturbine<-read.delim("https://raw.githubusercontent.com/kvaranyak4/STAT3220/main/GASTURBINE.txt")
head(gasturbine)names(gasturbine)[1] "ENGINE" "SHAFTS" "RPM" "CPRATIO" "INLET.TEMP"
[6] "EXH.TEMP" "AIRFLOW" "POWER" "HEATRATE" hist(gasturbine$HEATRATE, xlab="Heat Rate", main="Histogram of Heat Rate") 
The distribution of the response variable, heat rate, is unimodal and skewed right. It is continuous, so it should still be suitable for regression.
Step 2: Hypothesize Relationship (Exploratory Data Analysis)
We will explore the relationship with quantitative variables with scatter plots and correlations and classify each relationship as linear, curvilinear, other, or none. We explore the box plots and means for each qualitative variable explanatory variable then classify the relationships as existent or not. We will not explore interactions in this example.
#Scatter plots for quantitative variables
for (i in names(gasturbine)[3:8]) {
plot((gasturbine[,i]), gasturbine$HEATRATE,xlab=i,ylab="Heat Rate")
}
#Attempt Transformations
plot(log(gasturbine$RPM), gasturbine$HEATRATE,ylab="Heat Rate")
plot(log(gasturbine$CPRATIO), gasturbine$HEATRATE,ylab="Heat Rate")
plot(log(gasturbine$POWER), gasturbine$HEATRATE,ylab="Heat Rate")
plot(log(gasturbine$AIRFLOW), gasturbine$HEATRATE,ylab="Heat Rate")
#Correlations for quantitative variables
round(cor(gasturbine[3:8],gasturbine$HEATRATE,use="complete.obs"),3) [,1]
RPM 0.844
CPRATIO -0.735
INLET.TEMP -0.801
EXH.TEMP -0.314
AIRFLOW -0.703
POWER -0.697round(cor(log(gasturbine[3:8]),gasturbine$HEATRATE,use="complete.obs"),3) [,1]
RPM 0.810
CPRATIO -0.818
INLET.TEMP -0.813
EXH.TEMP -0.303
AIRFLOW -0.840
POWER -0.871#Summary Statistics for response variable grouped by each level of the response
tapply(gasturbine$HEATRATE,gasturbine$ENGINE,summary)$Advanced
Min. 1st Qu. Median Mean 3rd Qu. Max.
9105 9295 9669 9764 9933 11588
$Aeroderiv
Min. 1st Qu. Median Mean 3rd Qu. Max.
8714 10708 12414 12312 13697 16243
$Traditional
Min. 1st Qu. Median Mean 3rd Qu. Max.
10086 10598 11183 11544 11956 14796 tapply(gasturbine$HEATRATE,gasturbine$SHAFTS,summary)$`1`
Min. 1st Qu. Median Mean 3rd Qu. Max.
9105 9918 10592 10930 11674 14796
$`2`
Min. 1st Qu. Median Mean 3rd Qu. Max.
10951 11223 11654 12536 13232 16243
$`3`
Min. 1st Qu. Median Mean 3rd Qu. Max.
8714 8903 9092 9092 9280 9469 #Box plots for Qualitative
boxplot(HEATRATE~ENGINE,gasturbine, ylab="Heat Rate")
boxplot(HEATRATE~SHAFTS,gasturbine, ylab="Heat Rate")
# Summary counts for qualitative variables
table(gasturbine$ENGINE,gasturbine$SHAFTS)
1 2 3
Advanced 21 0 0
Aeroderiv 1 4 2
Traditional 35 4 0- Most of the variables have high correlations. Exhaust temperature has the weakest correlation.
- CPRATIO, POWER and AIRFLOW should have a log transformation because their correlations with HEATRATE were much higher, indicating a stronger linear form with that transformation.
- Engine type appears to have a different average heat rate for the advanced group.
- Note: Shafts=3 only has two observations. Further, we would not be able to fit an interaction between Engine and Shaft because some level combinations do not have any observations.
Step 2 (continued): Multicollinearity
Do any of the explanatory variables have relationships with each other? We will look at pairwise correlations and VIF to evaluate multicollinearity in the quantitative explanatory variables.
Pairwise Correlations
Below are several methods for evaluating PAIRWISE relationships of the explanatory variables. You may use whichever is easiest to interpret for yourself.
#Store the transformations in the data table
gasturbine$log.CPRATIO<-log(gasturbine$CPRATIO)
gasturbine$log.AIRFLOW<-log(gasturbine$AIRFLOW)
gasturbine$log.POWER<-log(gasturbine$POWER)
#Regular correlation
# Use the correlation function with no second argument
# to find the correlations with the explanatory variables with each other
gasturcor<-round(cor(gasturbine[,c(3,5,6,10:12)]),4)
gasturcor RPM INLET.TEMP EXH.TEMP log.CPRATIO log.AIRFLOW log.POWER
RPM 1.0000 -0.5536 -0.1715 -0.5819 -0.9177 -0.9083
INLET.TEMP -0.5536 1.0000 0.7283 0.7440 0.6649 0.7221
EXH.TEMP -0.1715 0.7283 1.0000 0.1712 0.4068 0.4515
log.CPRATIO -0.5819 0.7440 0.1712 1.0000 0.5308 0.5757
log.AIRFLOW -0.9177 0.6649 0.4068 0.5308 1.0000 0.9964
log.POWER -0.9083 0.7221 0.4515 0.5757 0.9964 1.0000#Correlation Visualization
# The argument is the stored correlation values from the cor() function
corrplot(gasturcor)
# Scatter plot matrix
plot(gasturbine[c(3,5,6,10:12)])
#A new correlation function
# this will also output the associated p-values with Ho: rho=0
rcorr(as.matrix(gasturbine[,c(3,5,6,10:12)])) RPM INLET.TEMP EXH.TEMP log.CPRATIO log.AIRFLOW log.POWER
RPM 1.00 -0.55 -0.17 -0.58 -0.92 -0.91
INLET.TEMP -0.55 1.00 0.73 0.74 0.66 0.72
EXH.TEMP -0.17 0.73 1.00 0.17 0.41 0.45
log.CPRATIO -0.58 0.74 0.17 1.00 0.53 0.58
log.AIRFLOW -0.92 0.66 0.41 0.53 1.00 1.00
log.POWER -0.91 0.72 0.45 0.58 1.00 1.00
n= 67
P
RPM INLET.TEMP EXH.TEMP log.CPRATIO log.AIRFLOW log.POWER
RPM 0.0000 0.1653 0.0000 0.0000 0.0000
INLET.TEMP 0.0000 0.0000 0.0000 0.0000 0.0000
EXH.TEMP 0.1653 0.0000 0.1660 0.0006 0.0001
log.CPRATIO 0.0000 0.0000 0.1660 0.0000 0.0000
log.AIRFLOW 0.0000 0.0000 0.0006 0.0000 0.0000
log.POWER 0.0000 0.0000 0.0001 0.0000 0.0000 There is concern of strong pairwise relationships with several pairs of variables.
- log.Power & log.airflow (r=0.92)
- log.Power & inlet_temp (r=0.72)
- log.airflow & RPM (r=0.92)
Multicollinearity
Next we evaluate the severity of MULTICOLLINEARITY with VIF but fitting a model with all first order quantitative variables.
#Multicollinearity VIF
# First fit the model with all of the quantitative variables
gasmod1<-lm(HEATRATE~RPM+INLET.TEMP+EXH.TEMP+log.CPRATIO+log.AIRFLOW+log.POWER,data=gasturbine)
summary(gasmod1)
Call:
lm(formula = HEATRATE ~ RPM + INLET.TEMP + EXH.TEMP + log.CPRATIO +
log.AIRFLOW + log.POWER, data = gasturbine)
Residuals:
Min 1Q Median 3Q Max
-337.73 -139.16 -60.72 75.76 735.19
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.312e+04 1.984e+03 21.737 < 2e-16 ***
RPM 6.700e-02 1.202e-02 5.575 6.22e-07 ***
INLET.TEMP 1.036e+00 9.487e-01 1.092 0.279
EXH.TEMP 1.326e+01 1.599e+00 8.291 1.56e-11 ***
log.CPRATIO 1.578e+02 2.534e+02 0.623 0.536
log.AIRFLOW 7.625e+03 5.632e+02 13.539 < 2e-16 ***
log.POWER -7.433e+03 5.247e+02 -14.166 < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 213.7 on 60 degrees of freedom
Multiple R-squared: 0.9837, Adjusted R-squared: 0.9821
F-statistic: 603 on 6 and 60 DF, p-value: < 2.2e-16#store and round the VIF values
gasmod1vif<-round(vif(gasmod1),3)
gasmod1vif RPM INLET.TEMP EXH.TEMP log.CPRATIO log.AIRFLOW log.POWER
10.300 24.579 7.203 7.086 1085.142 1225.107 #find the average VIF
mean(gasmod1vif)[1] 393.2362Yes, there is evidence of severe multicollinearity because all VIFs are MUCH greater than 10 and the average VIF is MUCH greater than 3 at 393.2362.
Step 2 (continued): Variable Screening
Because we have quite a few variables and severe multicollinearity, we need to address that. It is not clear from EDA what variables should remain and which variables should be removed.
We will use variable selection procedures to narrow down our quantitative variables to a best set of predictors. We will use the entry and remain significance levels of 0.15
REMEMBER: Variable Screening’s main purpose is to narrow down quantitative predictors. It can be a useful tool when multicollinearity is present, but that is not its only function.
# Note: Details=T gives all of the steps within each process
# In most cases, you can set Details=F
# backwards elimination
#Default: prem = 0.3
ols_step_backward_p(gasmod1,p_rem=0.15,details=T)Backward Elimination Method
---------------------------
Candidate Terms:
1. RPM
2. INLET.TEMP
3. EXH.TEMP
4. log.CPRATIO
5. log.AIRFLOW
6. log.POWER
Step => 0
Model => HEATRATE ~ RPM + INLET.TEMP + EXH.TEMP + log.CPRATIO + log.AIRFLOW + log.POWER
R2 => 0.984
Initiating stepwise selection...
Step => 1
Removed => log.CPRATIO
Model => HEATRATE ~ RPM + INLET.TEMP + EXH.TEMP + log.AIRFLOW + log.POWER
R2 => 0.98358
No more variables to be removed.
Variables Removed:
=> log.CPRATIO
Stepwise Summary
--------------------------------------------------------------------------
Step Variable AIC SBC SBIC R2 Adj. R2
--------------------------------------------------------------------------
0 Full Model 917.568 935.205 729.036 0.98369 0.98206
1 log.CPRATIO 916.000 931.433 727.152 0.98358 0.98224
--------------------------------------------------------------------------
Final Model Output
------------------
Model Summary
-------------------------------------------------------------------
R 0.992 RMSE 202.837
R-Squared 0.984 MSE 41142.862
Adj. R-Squared 0.982 Coef. Var 1.921
Pred R-Squared 0.975 AIC 916.000
MAE 152.893 SBC 931.433
-------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
AIC: Akaike Information Criteria
SBC: Schwarz Bayesian Criteria
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 165140636.668 5 33028127.334 730.877 0.0000
Residual 2756571.779 61 45189.701
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 43134.882 1973.376 21.858 0.000 39188.873 47080.890
RPM 0.066 0.012 0.291 5.568 0.000 0.042 0.090
INLET.TEMP 1.337 0.813 0.115 1.644 0.105 -0.289 2.962
EXH.TEMP 12.538 1.098 0.347 11.422 0.000 10.343 14.733
log.AIRFLOW 7529.364 539.316 7.262 13.961 0.000 6450.935 8607.794
log.POWER -7347.619 503.882 -8.082 -14.582 0.000 -8355.194 -6340.044
--------------------------------------------------------------------------------------------------# forward selection
#default: penter = 0.3
ols_step_forward_p(gasmod1,p_enter=0.15,details=T)Forward Selection Method
------------------------
Candidate Terms:
1. RPM
2. INLET.TEMP
3. EXH.TEMP
4. log.CPRATIO
5. log.AIRFLOW
6. log.POWER
Step => 0
Model => HEATRATE ~ 1
R2 => 0
Initiating stepwise selection...
Selection Metrics Table
------------------------------------------------------------------
Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
------------------------------------------------------------------
log.POWER 0.00000 0.758 0.755 1088.159
RPM 0.00000 0.712 0.708 1099.849
log.AIRFLOW 0.00000 0.705 0.701 1101.457
log.CPRATIO 0.00000 0.670 0.665 1109.066
INLET.TEMP 0.00000 0.641 0.635 1114.713
EXH.TEMP 0.00959 0.099 0.085 1176.358
------------------------------------------------------------------
Step => 1
Selected => log.POWER
Model => HEATRATE ~ log.POWER
R2 => 0.758
Selection Metrics Table
------------------------------------------------------------------
Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
------------------------------------------------------------------
log.CPRATIO 0.00000 0.909 0.906 1024.868
log.AIRFLOW 0.00000 0.865 0.861 1051.171
INLET.TEMP 2e-05 0.820 0.814 1070.432
RPM 0.03660 0.774 0.767 1085.549
EXH.TEMP 0.14832 0.766 0.759 1087.954
------------------------------------------------------------------
Step => 2
Selected => log.CPRATIO
Model => HEATRATE ~ log.POWER + log.CPRATIO
R2 => 0.909
Selection Metrics Table
------------------------------------------------------------------
Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
------------------------------------------------------------------
log.AIRFLOW 0.00236 0.921 0.918 1016.961
RPM 0.10842 0.913 0.908 1024.106
EXH.TEMP 0.27423 0.911 0.906 1025.587
INLET.TEMP 0.45619 0.910 0.905 1026.273
------------------------------------------------------------------
Step => 3
Selected => log.AIRFLOW
Model => HEATRATE ~ log.POWER + log.CPRATIO + log.AIRFLOW
R2 => 0.921
Selection Metrics Table
----------------------------------------------------------------
Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
----------------------------------------------------------------
EXH.TEMP 0.00000 0.975 0.974 941.630
RPM 0.00000 0.947 0.943 993.145
INLET.TEMP 0.00000 0.944 0.941 995.852
----------------------------------------------------------------
Step => 4
Selected => EXH.TEMP
Model => HEATRATE ~ log.POWER + log.CPRATIO + log.AIRFLOW + EXH.TEMP
R2 => 0.975
Selection Metrics Table
----------------------------------------------------------------
Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
----------------------------------------------------------------
RPM 0.00000 0.983 0.982 916.887
INLET.TEMP 0.76662 0.975 0.973 943.533
----------------------------------------------------------------
Step => 5
Selected => RPM
Model => HEATRATE ~ log.POWER + log.CPRATIO + log.AIRFLOW + EXH.TEMP + RPM
R2 => 0.983
Selection Metrics Table
----------------------------------------------------------------
Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
----------------------------------------------------------------
INLET.TEMP 0.27902 0.984 0.982 917.568
----------------------------------------------------------------
Step => 6
Selected => INLET.TEMP
Model => HEATRATE ~ log.POWER + log.CPRATIO + log.AIRFLOW + EXH.TEMP + RPM + INLET.TEMP
R2 => 0.984
Variables Selected:
=> log.POWER
=> log.CPRATIO
=> log.AIRFLOW
=> EXH.TEMP
=> RPM
=> INLET.TEMP
Stepwise Summary
----------------------------------------------------------------------------
Step Variable AIC SBC SBIC R2 Adj. R2
----------------------------------------------------------------------------
0 Base Model 1181.327 1185.737 987.298 0.00000 0.00000
1 log.POWER 1088.159 1094.773 892.613 0.75839 0.75467
2 log.CPRATIO 1024.868 1033.687 828.649 0.90882 0.90597
3 log.AIRFLOW 1016.961 1027.984 819.495 0.92135 0.91761
4 EXH.TEMP 941.630 954.858 748.697 0.97520 0.97360
5 RPM 916.887 932.320 727.870 0.98336 0.98200
6 INLET.TEMP 917.568 935.205 729.036 0.98369 0.98206
----------------------------------------------------------------------------
Final Model Output
------------------
Model Summary
-------------------------------------------------------------------
R 0.992 RMSE 202.184
R-Squared 0.984 MSE 40878.488
Adj. R-Squared 0.982 Coef. Var 1.931
Pred R-Squared 0.975 AIC 917.568
MAE 154.185 SBC 935.205
-------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
AIC: Akaike Information Criteria
SBC: Schwarz Bayesian Criteria
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 165158349.755 6 27526391.626 603.019 0.0000
Residual 2738858.692 60 45647.645
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 43116.228 1983.576 21.737 0.000 39148.486 47083.970
log.POWER -7433.155 524.714 -8.176 -14.166 0.000 -8482.740 -6383.570
log.CPRATIO 157.821 253.354 0.027 0.623 0.536 -348.962 664.604
log.AIRFLOW 7624.558 563.172 7.354 13.539 0.000 6498.047 8751.070
EXH.TEMP 13.259 1.599 0.367 8.291 0.000 10.060 16.457
RPM 0.067 0.012 0.295 5.575 0.000 0.043 0.091
INLET.TEMP 1.036 0.949 0.089 1.092 0.279 -0.861 2.934
--------------------------------------------------------------------------------------------------# stepwise regression
#Default: pent = 0.1, prem = 0.3
ols_step_both_p(gasmod1,p_ent=0.15,p_rem=0.15,details=T)Stepwise Selection Method
-------------------------
Candidate Terms:
1. RPM
2. INLET.TEMP
3. EXH.TEMP
4. log.CPRATIO
5. log.AIRFLOW
6. log.POWER
Step => 0
Model => HEATRATE ~ 1
R2 => 0
Initiating stepwise selection...
Step => 1
Selected => log.POWER
Model => HEATRATE ~ log.POWER
R2 => 0.758
Step => 2
Selected => log.CPRATIO
Model => HEATRATE ~ log.POWER + log.CPRATIO
R2 => 0.909
Step => 3
Selected => log.AIRFLOW
Model => HEATRATE ~ log.POWER + log.CPRATIO + log.AIRFLOW
R2 => 0.921
Step => 4
Selected => EXH.TEMP
Model => HEATRATE ~ log.POWER + log.CPRATIO + log.AIRFLOW + EXH.TEMP
R2 => 0.975
Step => 5
Removed => log.CPRATIO
Model => HEATRATE ~ log.POWER + log.AIRFLOW + EXH.TEMP
=> 0.97519
Step => 6
Selected => RPM
Model => HEATRATE ~ log.POWER + log.AIRFLOW + EXH.TEMP + RPM
R2 => 0.983
Step => 7
Selected => INLET.TEMP
Model => HEATRATE ~ log.POWER + log.AIRFLOW + EXH.TEMP + RPM + INLET.TEMP
R2 => 0.984
Stepwise Summary
--------------------------------------------------------------------------------
Step Variable AIC SBC SBIC R2 Adj. R2
--------------------------------------------------------------------------------
0 Base Model 1181.327 1185.737 987.298 0.00000 0.00000
1 log.POWER (+) 1088.159 1094.773 892.613 0.75839 0.75467
2 log.CPRATIO (+) 1024.868 1033.687 828.649 0.90882 0.90597
3 log.AIRFLOW (+) 1016.961 1027.984 819.495 0.92135 0.91761
4 EXH.TEMP (+) 941.630 954.858 748.697 0.97520 0.97360
5 log.CPRATIO (-) 939.661 950.684 747.255 0.97519 0.97401
6 RPM (+) 916.905 930.133 727.384 0.98285 0.98175
7 INLET.TEMP (+) 916.000 931.433 727.152 0.98358 0.98224
--------------------------------------------------------------------------------
Final Model Output
------------------
Model Summary
-------------------------------------------------------------------
R 0.992 RMSE 202.837
R-Squared 0.984 MSE 41142.862
Adj. R-Squared 0.982 Coef. Var 1.921
Pred R-Squared 0.975 AIC 916.000
MAE 152.893 SBC 931.433
-------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
AIC: Akaike Information Criteria
SBC: Schwarz Bayesian Criteria
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 165140636.668 5 33028127.334 730.877 0.0000
Residual 2756571.779 61 45189.701
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 43134.882 1973.376 21.858 0.000 39188.873 47080.890
log.POWER -7347.619 503.882 -8.082 -14.582 0.000 -8355.194 -6340.044
log.AIRFLOW 7529.364 539.316 7.262 13.961 0.000 6450.935 8607.794
EXH.TEMP 12.538 1.098 0.347 11.422 0.000 10.343 14.733
RPM 0.066 0.012 0.291 5.568 0.000 0.042 0.090
INLET.TEMP 1.337 0.813 0.115 1.644 0.105 -0.289 2.962
--------------------------------------------------------------------------------------------------- Backwards elimination: rpm, inlet.temp, exh.temp, log.airflow, log.power
- Forward Selection: log.power, log.cpratio, log.airflow, exh.temp, inlet.temp, rpm
- Stepwise Regression: log.power, log.airflow, exh.temp, rpm, inlet.temp
- In this instance the three procedures resulted in slightly different subsets of variables. This may not always happen.
- Let’s use the stepwise set of predictors to fit a model and determine if our problem of severe multicollinearity has been resolved.
On Your Own
What happens when we change the level of entry/remain?
First run each of the iterative procedures at their defaults settings (higher levels of enter and remain). And set details = F.
# backwards elimination
#Default: p_rem = 0.3
# forward selection
#default: p_enter = 0.3
# stepwise regression
#Default: p_ent = 0.1, p_rem = 0.3Add your notes for the task above.
Now run each of the iterative procedures at p_enter and p_rem equal to 0.05. And set details = F.
# backwards elimination
#Default: p_rem = 0.3
# forward selection
#default: p_enter = 0.3
# stepwise regression
#Default: p_ent = 0.1, p_rem = 0.3Add your notes for the task above.
Now what?
Since our main goal from this particular EDA was to eliminate severe multicollinearity. We will first check if this new subset of predictors does that.
#updated model
gasmod2<-lm(HEATRATE~RPM+INLET.TEMP+EXH.TEMP+log.AIRFLOW+log.POWER,data=gasturbine)
summary(gasmod2)
Call:
lm(formula = HEATRATE ~ RPM + INLET.TEMP + EXH.TEMP + log.AIRFLOW +
log.POWER, data = gasturbine)
Residuals:
Min 1Q Median 3Q Max
-345.13 -130.99 -49.58 68.14 757.16
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.313e+04 1.973e+03 21.858 < 2e-16 ***
RPM 6.613e-02 1.188e-02 5.568 6.13e-07 ***
INLET.TEMP 1.337e+00 8.130e-01 1.644 0.105
EXH.TEMP 1.254e+01 1.098e+00 11.422 < 2e-16 ***
log.AIRFLOW 7.529e+03 5.393e+02 13.961 < 2e-16 ***
log.POWER -7.348e+03 5.039e+02 -14.582 < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 212.6 on 61 degrees of freedom
Multiple R-squared: 0.9836, Adjusted R-squared: 0.9822
F-statistic: 730.9 on 5 and 61 DF, p-value: < 2.2e-16gasmod2vif<-round(vif(gasmod2),3)
gasmod2vif RPM INLET.TEMP EXH.TEMP log.AIRFLOW log.POWER
10.163 18.231 3.429 1005.242 1141.209 mean(gasmod2vif)[1] 435.6548This did not resolve our problem of severe multicollinearity. We should remove log.power, log.airflow (or both).
#updated model: REMOVE LOG.POWER
gasmod3<-lm(HEATRATE~RPM+INLET.TEMP+EXH.TEMP+log.AIRFLOW,data=gasturbine)
gasmod3vif<-round(vif(gasmod3),3)
gasmod3vif RPM INLET.TEMP EXH.TEMP log.AIRFLOW
10.057 3.575 3.334 10.683 mean(gasmod3vif)[1] 6.91225#updated model: REMOVE LOG.AIRFLOW
gasmod4<-lm(HEATRATE~RPM+INLET.TEMP+EXH.TEMP+log.POWER,data=gasturbine)
gasmod4vif<-round(vif(gasmod4),3)
gasmod4vif RPM INLET.TEMP EXH.TEMP log.POWER
9.777 3.676 3.271 12.128 mean(gasmod4vif)[1] 7.213#updated model: REMOVE BOTH
gasmod5<-lm(HEATRATE~RPM+INLET.TEMP+EXH.TEMP,data=gasturbine)
gasmod5vif<-round(vif(gasmod5),3)
gasmod5vif RPM INLET.TEMP EXH.TEMP
1.727 3.570 2.551 mean(gasmod5vif)[1] 2.616Of the three models above- which seems to resolve our concern best?
It appears the model without power and airflow have resolved the issue of severe multicollinearity. HOWEVER, we may have also decided to remove a combination of RPM and power/airflow, as that also had a strong relationship. As a researcher, or with your client, you may discuss which of those three variables would be most important to analyze moving forward. Here, we will remove the transformations as we want the most least complex model.
All of what we’ve done so far is still just our exploratory data analysis (EDA) phase of narrowing down predictors. We combine our visuals with the iterative process to select quantitative variables to begin the stages of model building. RECALL:
- Stage 1: Add Quantitative Predictors, higher order terms, and the QUANT X QUANT interactions
- Stage 2: Add Qualitative predictors and QUAL X QUAL interactions
- Stage 3: Add QUANT X QUAL Interactions