STAT 3220 Unit 3
During periods of high electricity demand, especially during the hot summer months, the power output from a gas turbine engine can drop dramatically. One way to counter this drop in power is by cooling the inlet air to the gas turbine. An increasingly popular cooling method uses high pressure inlet fogging. The performance of a sample of 67 gas turbines augmented with high pressure inlet fogging was investigated in the Journal of Engineering for Gas Turbines and Power (January 2005). One measure of performance is heat rate (kilojoules per kilowatt per hour). Heat rates for the 67 gas turbines, saved in the gasturbine file.
Unit 3.1: Multicollinearity and Variable Screening
Step 1: Collect the Data
Check the appropriateness of response variable for regression: View a histogram of response variable. It should be continuous, and approximately unimodal and symmetric, with few outliers.
gasturbine<-read.delim("https://raw.githubusercontent.com/kvaranyak4/STAT3220/main/GASTURBINE.txt")
head(gasturbine)names(gasturbine)[1] "ENGINE" "SHAFTS" "RPM" "CPRATIO" "INLET.TEMP"
[6] "EXH.TEMP" "AIRFLOW" "POWER" "HEATRATE" hist(gasturbine$HEATRATE, xlab="Heat Rate", main="Histogram of Heat Rate") 
The distribution of the response variable, heat rate, is unimodal and skewed right. It is continuous, so it should still be suitable for regression.
Step 2: Hypothesize Relationship (Exploratory Data Analysis)
We will explore the relationship with quantitative variables with scatter plots and correlations and classify each relationship as linear, curvilinear, or none. We explore the box plots and means for each qualitative variable explanatory variable then classify the relationships as existent or not. We will not explore interactions in this example.
#Scatter plots for quantitative variables
for (i in names(gasturbine)[3:8]) {
plot(gasturbine[,i], gasturbine$HEATRATE,xlab=i,ylab="Heat Rate")
}
#Correlations for quantitative variables
round(cor(gasturbine[3:8],gasturbine$HEATRATE,use="complete.obs"),3) [,1]
RPM 0.844
CPRATIO -0.735
INLET.TEMP -0.801
EXH.TEMP -0.314
AIRFLOW -0.703
POWER -0.697#Summary Statistics for response variable grouped by each level of the response
tapply(gasturbine$HEATRATE,gasturbine$ENGINE,summary)$Advanced
Min. 1st Qu. Median Mean 3rd Qu. Max.
9105 9295 9669 9764 9933 11588
$Aeroderiv
Min. 1st Qu. Median Mean 3rd Qu. Max.
8714 10708 12414 12312 13697 16243
$Traditional
Min. 1st Qu. Median Mean 3rd Qu. Max.
10086 10598 11183 11544 11956 14796 tapply(gasturbine$HEATRATE,gasturbine$SHAFTS,summary)$`1`
Min. 1st Qu. Median Mean 3rd Qu. Max.
9105 9918 10592 10930 11674 14796
$`2`
Min. 1st Qu. Median Mean 3rd Qu. Max.
10951 11223 11654 12536 13232 16243
$`3`
Min. 1st Qu. Median Mean 3rd Qu. Max.
8714 8903 9092 9092 9280 9469 #Box plots for Qualitative
boxplot(HEATRATE~ENGINE,gasturbine, ylab="Heat Rate")
boxplot(HEATRATE~SHAFTS,gasturbine, ylab="Heat Rate")
# Summary counts for qualitative variables
table(gasturbine$ENGINE,gasturbine$SHAFTS)
1 2 3
Advanced 21 0 0
Aeroderiv 1 4 2
Traditional 35 4 0- Most of the variables have high correlations. Exhaust temperature has the weakest correlation.
- Power has a curvilinear or potentially a negative log relationship.
- Engine type appears to have a different average heat rate for the advanced group.
- Note: Shafts=3 only has two observations. Further, we would not be able to fit an interaction between Engine and Shaft because some level combinations do not have any observations.
Step 2 (continued): Multicollinearity
Do any of the explanatory variables have relationships with each other? We will look at pairwise correlations and VIF to evaluate multicollinearity in the quantitative explanatory variables.
#Regular correlation
gasturcor<-round(cor(gasturbine[,3:8]),4)
gasturcor RPM CPRATIO INLET.TEMP EXH.TEMP AIRFLOW POWER
RPM 1.0000 -0.4903 -0.5536 -0.1715 -0.6876 -0.6169
CPRATIO -0.4903 1.0000 0.6851 0.1139 0.3826 0.4473
INLET.TEMP -0.5536 0.6851 1.0000 0.7283 0.6808 0.7503
EXH.TEMP -0.1715 0.1139 0.7283 1.0000 0.5665 0.6309
AIRFLOW -0.6876 0.3826 0.6808 0.5665 1.0000 0.9776
POWER -0.6169 0.4473 0.7503 0.6309 0.9776 1.0000# Scatter plot matrix
plot(gasturbine[3:8])
#A new correlation function
gasturcor2<-rcorr(as.matrix(gasturbine[,3:8]))
gasturcor2 RPM CPRATIO INLET.TEMP EXH.TEMP AIRFLOW POWER
RPM 1.00 -0.49 -0.55 -0.17 -0.69 -0.62
CPRATIO -0.49 1.00 0.69 0.11 0.38 0.45
INLET.TEMP -0.55 0.69 1.00 0.73 0.68 0.75
EXH.TEMP -0.17 0.11 0.73 1.00 0.57 0.63
AIRFLOW -0.69 0.38 0.68 0.57 1.00 0.98
POWER -0.62 0.45 0.75 0.63 0.98 1.00
n= 67
P
RPM CPRATIO INLET.TEMP EXH.TEMP AIRFLOW POWER
RPM 0.0000 0.0000 0.1653 0.0000 0.0000
CPRATIO 0.0000 0.0000 0.3585 0.0014 0.0001
INLET.TEMP 0.0000 0.0000 0.0000 0.0000 0.0000
EXH.TEMP 0.1653 0.3585 0.0000 0.0000 0.0000
AIRFLOW 0.0000 0.0014 0.0000 0.0000 0.0000
POWER 0.0000 0.0001 0.0000 0.0000 0.0000 #Correlation Visualization
corrplot(gasturcor)
There is concern of strong pairwise relationships.
- Power & airflow (r=0.977)
- Power & inlet_temp (r=0.75)
- inlet_temp & exh_temp (r=0.73)
#Multicollinearity VIF
gasmod1<-lm(HEATRATE~.-ENGINE-SHAFTS,data=gasturbine)
# Syntax Note: We can use the . to indicate all the variables in the data frame
# And use the - to exclude a particular variable from the model
summary(gasmod1)
Call:
lm(formula = HEATRATE ~ . - ENGINE - SHAFTS, data = gasturbine)
Residuals:
Min 1Q Median 3Q Max
-1003.32 -307.35 -91.44 271.18 1405.52
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.431e+04 1.112e+03 12.869 < 2e-16 ***
RPM 8.058e-02 1.611e-02 5.002 5.25e-06 ***
CPRATIO -6.775e+00 3.038e+01 -0.223 0.824301
INLET.TEMP -9.507e+00 1.529e+00 -6.217 5.33e-08 ***
EXH.TEMP 1.415e+01 3.469e+00 4.081 0.000135 ***
AIRFLOW -2.553e+00 1.746e+00 -1.462 0.148892
POWER 4.257e-03 4.217e-03 1.009 0.316804
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 458.8 on 60 degrees of freedom
Multiple R-squared: 0.9248, Adjusted R-squared: 0.9173
F-statistic: 123 on 6 and 60 DF, p-value: < 2.2e-16gasmod1vif<-round(vif(gasmod1),3)
gasmod1vif RPM CPRATIO INLET.TEMP EXH.TEMP AIRFLOW POWER
4.015 5.213 13.852 7.351 49.136 49.765 mean(gasmod1vif)[1] 21.55533Yes, there is evidence of severe multicollinearity because several VIFs are much greater than 10 and the average VIF is greater than 3.
Step 2 (continued): Variable Screening
Because we have quite a few variables and severe multicollinearity, we need to address that. It is not clear from EDA what variables should remain and which variables should be removed.
We will use variable selection procedures to narrow down our quantitative variables to a best set of predictors. We will use the entry and remain significance levels of 0.15
# backwards elimination
#Default: prem = 0.3
ols_step_backward_p(gasmod1,prem=0.15,details=T)Backward Elimination Method
---------------------------
Candidate Terms:
1 . RPM
2 . CPRATIO
3 . INLET.TEMP
4 . EXH.TEMP
5 . AIRFLOW
6 . POWER
We are eliminating variables based on p value...
- CPRATIO
Backward Elimination: Step 1
Variable CPRATIO Removed
Model Summary
------------------------------------------------------------------
R 0.962 RMSE 455.170
R-Squared 0.925 Coef. Var 4.113
Adj. R-Squared 0.919 MSE 207179.318
Pred R-Squared 0.907 MAE 336.847
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 155259270.080 5 31051854.016 149.879 0.0000
Residual 12637938.368 61 207179.318
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 14215.194 1011.866 14.048 0.000 12191.843 16238.544
RPM 0.080 0.016 0.354 5.038 0.000 0.048 0.112
INLET.TEMP -9.769 0.969 -0.842 -10.080 0.000 -11.707 -7.831
EXH.TEMP 14.732 2.290 0.408 6.432 0.000 10.152 19.312
AIRFLOW -2.473 1.696 -0.352 -1.459 0.150 -5.864 0.917
POWER 0.004 0.004 0.239 0.992 0.325 -0.004 0.012
--------------------------------------------------------------------------------------------------
- POWER
Backward Elimination: Step 2
Variable POWER Removed
Model Summary
------------------------------------------------------------------
R 0.961 RMSE 455.114
R-Squared 0.924 Coef. Var 4.113
Adj. R-Squared 0.919 MSE 207128.472
Pred R-Squared 0.908 MAE 340.078
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 155055243.172 4 38763810.793 187.149 0.0000
Residual 12841965.276 62 207128.472
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 13617.924 813.306 16.744 0.000 11992.148 15243.700
RPM 0.089 0.013 0.391 6.608 0.000 0.062 0.116
INLET.TEMP -9.186 0.770 -0.791 -11.923 0.000 -10.726 -7.646
EXH.TEMP 14.363 2.260 0.397 6.356 0.000 9.846 18.880
AIRFLOW -0.848 0.437 -0.120 -1.939 0.057 -1.721 0.026
--------------------------------------------------------------------------------------------------
No more variables satisfy the condition of p value = 0.15
Variables Removed:
- CPRATIO
- POWER
Final Model Output
------------------
Model Summary
------------------------------------------------------------------
R 0.961 RMSE 455.114
R-Squared 0.924 Coef. Var 4.113
Adj. R-Squared 0.919 MSE 207128.472
Pred R-Squared 0.908 MAE 340.078
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 155055243.172 4 38763810.793 187.149 0.0000
Residual 12841965.276 62 207128.472
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 13617.924 813.306 16.744 0.000 11992.148 15243.700
RPM 0.089 0.013 0.391 6.608 0.000 0.062 0.116
INLET.TEMP -9.186 0.770 -0.791 -11.923 0.000 -10.726 -7.646
EXH.TEMP 14.363 2.260 0.397 6.356 0.000 9.846 18.880
AIRFLOW -0.848 0.437 -0.120 -1.939 0.057 -1.721 0.026
--------------------------------------------------------------------------------------------------
Elimination Summary
---------------------------------------------------------------------------
Variable Adj.
Step Removed R-Square R-Square C(p) AIC RMSE
---------------------------------------------------------------------------
1 CPRATIO 0.9247 0.9186 5.0497 1018.0217 455.1695
2 POWER 0.9235 0.9186 4.0192 1017.0947 455.1137
---------------------------------------------------------------------------# forward selection
#default: penter = 0.3
ols_step_forward_p(gasmod1,penter=0.15,details=T)Forward Selection Method
---------------------------
Candidate Terms:
1. RPM
2. CPRATIO
3. INLET.TEMP
4. EXH.TEMP
5. AIRFLOW
6. POWER
We are selecting variables based on p value...
Forward Selection: Step 1
- RPM
Model Summary
------------------------------------------------------------------
R 0.844 RMSE 862.007
R-Squared 0.712 Coef. Var 7.789
Adj. R-Squared 0.708 MSE 743056.584
Pred R-Squared 0.696 MAE 648.175
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
----------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
----------------------------------------------------------------------------
Regression 119598530.459 1 119598530.459 160.955 0.0000
Residual 48298677.989 65 743056.584
Total 167897208.448 66
----------------------------------------------------------------------------
Parameter Estimates
----------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
----------------------------------------------------------------------------------------------
(Intercept) 9470.484 164.058 57.726 0.000 9142.838 9798.131
RPM 0.192 0.015 0.844 12.687 0.000 0.161 0.222
----------------------------------------------------------------------------------------------
Forward Selection: Step 2
- INLET.TEMP
Model Summary
------------------------------------------------------------------
R 0.934 RMSE 578.322
R-Squared 0.873 Coef. Var 5.226
Adj. R-Squared 0.869 MSE 334456.428
Pred R-Squared 0.859 MAE 389.791
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 146491997.059 2 73245998.530 219 0.0000
Residual 21405211.389 64 334456.428
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
-------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
-------------------------------------------------------------------------------------------------
(Intercept) 16523.288 794.181 20.805 0.000 14936.728 18109.847
RPM 0.131 0.012 0.578 10.783 0.000 0.107 0.156
INLET.TEMP -5.577 0.622 -0.481 -8.967 0.000 -6.820 -4.335
-------------------------------------------------------------------------------------------------
Forward Selection: Step 3
- EXH.TEMP
Model Summary
------------------------------------------------------------------
R 0.959 RMSE 464.980
R-Squared 0.919 Coef. Var 4.202
Adj. R-Squared 0.915 MSE 216206.126
Pred R-Squared 0.907 MAE 342.429
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 154276222.495 3 51425407.498 237.854 0.0000
Residual 13620985.953 63 216206.126
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 14359.717 733.308 19.582 0.000 12894.318 15825.116
RPM 0.105 0.011 0.463 9.818 0.000 0.084 0.127
INLET.TEMP -9.223 0.787 -0.795 -11.721 0.000 -10.795 -7.650
EXH.TEMP 12.426 2.071 0.344 6.000 0.000 8.288 16.564
--------------------------------------------------------------------------------------------------
Forward Selection: Step 4
- AIRFLOW
Model Summary
------------------------------------------------------------------
R 0.961 RMSE 455.114
R-Squared 0.924 Coef. Var 4.113
Adj. R-Squared 0.919 MSE 207128.472
Pred R-Squared 0.908 MAE 340.078
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 155055243.172 4 38763810.793 187.149 0.0000
Residual 12841965.276 62 207128.472
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 13617.924 813.306 16.744 0.000 11992.148 15243.700
RPM 0.089 0.013 0.391 6.608 0.000 0.062 0.116
INLET.TEMP -9.186 0.770 -0.791 -11.923 0.000 -10.726 -7.646
EXH.TEMP 14.363 2.260 0.397 6.356 0.000 9.846 18.880
AIRFLOW -0.848 0.437 -0.120 -1.939 0.057 -1.721 0.026
--------------------------------------------------------------------------------------------------
No more variables to be added.
Variables Entered:
+ RPM
+ INLET.TEMP
+ EXH.TEMP
+ AIRFLOW
Final Model Output
------------------
Model Summary
------------------------------------------------------------------
R 0.961 RMSE 455.114
R-Squared 0.924 Coef. Var 4.113
Adj. R-Squared 0.919 MSE 207128.472
Pred R-Squared 0.908 MAE 340.078
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 155055243.172 4 38763810.793 187.149 0.0000
Residual 12841965.276 62 207128.472
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 13617.924 813.306 16.744 0.000 11992.148 15243.700
RPM 0.089 0.013 0.391 6.608 0.000 0.062 0.116
INLET.TEMP -9.186 0.770 -0.791 -11.923 0.000 -10.726 -7.646
EXH.TEMP 14.363 2.260 0.397 6.356 0.000 9.846 18.880
AIRFLOW -0.848 0.437 -0.120 -1.939 0.057 -1.721 0.026
--------------------------------------------------------------------------------------------------
Selection Summary
-------------------------------------------------------------------------------
Variable Adj.
Step Entered R-Square R-Square C(p) AIC RMSE
-------------------------------------------------------------------------------
1 RPM 0.7123 0.7079 166.4933 1099.8486 862.0073
2 INLET.TEMP 0.8725 0.8685 40.7078 1047.3261 578.3221
3 EXH.TEMP 0.9189 0.9150 5.7207 1019.0405 464.9797
4 AIRFLOW 0.9235 0.9186 4.0192 1017.0947 455.1137
-------------------------------------------------------------------------------# stepwise regression
#Default: pent = 0.1, prem = 0.3
ols_step_both_p(gasmod1,pent=0.15,prem=0.15,details=T)Stepwise Selection Method
---------------------------
Candidate Terms:
1. RPM
2. CPRATIO
3. INLET.TEMP
4. EXH.TEMP
5. AIRFLOW
6. POWER
We are selecting variables based on p value...
Stepwise Selection: Step 1
- RPM added
Model Summary
------------------------------------------------------------------
R 0.844 RMSE 862.007
R-Squared 0.712 Coef. Var 7.789
Adj. R-Squared 0.708 MSE 743056.584
Pred R-Squared 0.696 MAE 648.175
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
----------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
----------------------------------------------------------------------------
Regression 119598530.459 1 119598530.459 160.955 0.0000
Residual 48298677.989 65 743056.584
Total 167897208.448 66
----------------------------------------------------------------------------
Parameter Estimates
----------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
----------------------------------------------------------------------------------------------
(Intercept) 9470.484 164.058 57.726 0.000 9142.838 9798.131
RPM 0.192 0.015 0.844 12.687 0.000 0.161 0.222
----------------------------------------------------------------------------------------------
Stepwise Selection: Step 2
- INLET.TEMP added
Model Summary
------------------------------------------------------------------
R 0.934 RMSE 578.322
R-Squared 0.873 Coef. Var 5.226
Adj. R-Squared 0.869 MSE 334456.428
Pred R-Squared 0.859 MAE 389.791
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 146491997.059 2 73245998.530 219 0.0000
Residual 21405211.389 64 334456.428
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
-------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
-------------------------------------------------------------------------------------------------
(Intercept) 16523.288 794.181 20.805 0.000 14936.728 18109.847
RPM 0.131 0.012 0.578 10.783 0.000 0.107 0.156
INLET.TEMP -5.577 0.622 -0.481 -8.967 0.000 -6.820 -4.335
-------------------------------------------------------------------------------------------------
Model Summary
------------------------------------------------------------------
R 0.934 RMSE 578.322
R-Squared 0.873 Coef. Var 5.226
Adj. R-Squared 0.869 MSE 334456.428
Pred R-Squared 0.859 MAE 389.791
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 146491997.059 2 73245998.530 219 0.0000
Residual 21405211.389 64 334456.428
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
-------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
-------------------------------------------------------------------------------------------------
(Intercept) 16523.288 794.181 20.805 0.000 14936.728 18109.847
RPM 0.131 0.012 0.578 10.783 0.000 0.107 0.156
INLET.TEMP -5.577 0.622 -0.481 -8.967 0.000 -6.820 -4.335
-------------------------------------------------------------------------------------------------
Stepwise Selection: Step 3
- EXH.TEMP added
Model Summary
------------------------------------------------------------------
R 0.959 RMSE 464.980
R-Squared 0.919 Coef. Var 4.202
Adj. R-Squared 0.915 MSE 216206.126
Pred R-Squared 0.907 MAE 342.429
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 154276222.495 3 51425407.498 237.854 0.0000
Residual 13620985.953 63 216206.126
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 14359.717 733.308 19.582 0.000 12894.318 15825.116
RPM 0.105 0.011 0.463 9.818 0.000 0.084 0.127
INLET.TEMP -9.223 0.787 -0.795 -11.721 0.000 -10.795 -7.650
EXH.TEMP 12.426 2.071 0.344 6.000 0.000 8.288 16.564
--------------------------------------------------------------------------------------------------
Model Summary
------------------------------------------------------------------
R 0.959 RMSE 464.980
R-Squared 0.919 Coef. Var 4.202
Adj. R-Squared 0.915 MSE 216206.126
Pred R-Squared 0.907 MAE 342.429
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 154276222.495 3 51425407.498 237.854 0.0000
Residual 13620985.953 63 216206.126
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 14359.717 733.308 19.582 0.000 12894.318 15825.116
RPM 0.105 0.011 0.463 9.818 0.000 0.084 0.127
INLET.TEMP -9.223 0.787 -0.795 -11.721 0.000 -10.795 -7.650
EXH.TEMP 12.426 2.071 0.344 6.000 0.000 8.288 16.564
--------------------------------------------------------------------------------------------------
Stepwise Selection: Step 4
- AIRFLOW added
Model Summary
------------------------------------------------------------------
R 0.961 RMSE 455.114
R-Squared 0.924 Coef. Var 4.113
Adj. R-Squared 0.919 MSE 207128.472
Pred R-Squared 0.908 MAE 340.078
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 155055243.172 4 38763810.793 187.149 0.0000
Residual 12841965.276 62 207128.472
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 13617.924 813.306 16.744 0.000 11992.148 15243.700
RPM 0.089 0.013 0.391 6.608 0.000 0.062 0.116
INLET.TEMP -9.186 0.770 -0.791 -11.923 0.000 -10.726 -7.646
EXH.TEMP 14.363 2.260 0.397 6.356 0.000 9.846 18.880
AIRFLOW -0.848 0.437 -0.120 -1.939 0.057 -1.721 0.026
--------------------------------------------------------------------------------------------------
Model Summary
------------------------------------------------------------------
R 0.961 RMSE 455.114
R-Squared 0.924 Coef. Var 4.113
Adj. R-Squared 0.919 MSE 207128.472
Pred R-Squared 0.908 MAE 340.078
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 155055243.172 4 38763810.793 187.149 0.0000
Residual 12841965.276 62 207128.472
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 13617.924 813.306 16.744 0.000 11992.148 15243.700
RPM 0.089 0.013 0.391 6.608 0.000 0.062 0.116
INLET.TEMP -9.186 0.770 -0.791 -11.923 0.000 -10.726 -7.646
EXH.TEMP 14.363 2.260 0.397 6.356 0.000 9.846 18.880
AIRFLOW -0.848 0.437 -0.120 -1.939 0.057 -1.721 0.026
--------------------------------------------------------------------------------------------------
No more variables to be added/removed.
Final Model Output
------------------
Model Summary
------------------------------------------------------------------
R 0.961 RMSE 455.114
R-Squared 0.924 Coef. Var 4.113
Adj. R-Squared 0.919 MSE 207128.472
Pred R-Squared 0.908 MAE 340.078
------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
ANOVA
---------------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
---------------------------------------------------------------------------
Regression 155055243.172 4 38763810.793 187.149 0.0000
Residual 12841965.276 62 207128.472
Total 167897208.448 66
---------------------------------------------------------------------------
Parameter Estimates
--------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
--------------------------------------------------------------------------------------------------
(Intercept) 13617.924 813.306 16.744 0.000 11992.148 15243.700
RPM 0.089 0.013 0.391 6.608 0.000 0.062 0.116
INLET.TEMP -9.186 0.770 -0.791 -11.923 0.000 -10.726 -7.646
EXH.TEMP 14.363 2.260 0.397 6.356 0.000 9.846 18.880
AIRFLOW -0.848 0.437 -0.120 -1.939 0.057 -1.721 0.026
--------------------------------------------------------------------------------------------------
Stepwise Selection Summary
-------------------------------------------------------------------------------------------
Added/ Adj.
Step Variable Removed R-Square R-Square C(p) AIC RMSE
-------------------------------------------------------------------------------------------
1 RPM addition 0.712 0.708 166.4930 1099.8486 862.0073
2 INLET.TEMP addition 0.873 0.869 40.7080 1047.3261 578.3221
3 EXH.TEMP addition 0.919 0.915 5.7210 1019.0405 464.9797
4 AIRFLOW addition 0.924 0.919 4.0190 1017.0947 455.1137
-------------------------------------------------------------------------------------------- Backwards elimination: rpm, inlet.temp, exh.temp, airflow
- Forward Selection: rpm, inlet.temp, exh.temp, airflow
- Stepwise Regression: rpm, inlet.temp, exh.temp, airflow
- In this instance all three procedures resulted in the same model. This may not always happen.
- We will use this set of predictors to fit a model and determine if our problem of severe multicollinearity has been resolved.
#updated model
gasmod2<-lm(HEATRATE~.-ENGINE-SHAFTS-POWER-CPRATIO,data=gasturbine)
# Syntax Note: We can use the . to indicate all the variables in the data frame
# And use the - to exclude a particular variable from the model
summary(gasmod1)
Call:
lm(formula = HEATRATE ~ . - ENGINE - SHAFTS, data = gasturbine)
Residuals:
Min 1Q Median 3Q Max
-1003.32 -307.35 -91.44 271.18 1405.52
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.431e+04 1.112e+03 12.869 < 2e-16 ***
RPM 8.058e-02 1.611e-02 5.002 5.25e-06 ***
CPRATIO -6.775e+00 3.038e+01 -0.223 0.824301
INLET.TEMP -9.507e+00 1.529e+00 -6.217 5.33e-08 ***
EXH.TEMP 1.415e+01 3.469e+00 4.081 0.000135 ***
AIRFLOW -2.553e+00 1.746e+00 -1.462 0.148892
POWER 4.257e-03 4.217e-03 1.009 0.316804
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 458.8 on 60 degrees of freedom
Multiple R-squared: 0.9248, Adjusted R-squared: 0.9173
F-statistic: 123 on 6 and 60 DF, p-value: < 2.2e-16gasmod2vif<-round(vif(gasmod2),3)
gasmod2vif RPM INLET.TEMP EXH.TEMP AIRFLOW
2.840 3.572 3.170 3.128 mean(gasmod2vif)[1] 3.1775The average VIF is slightly greater than 3. We conclude this is no longer a severe issue and we can continue with our analysis. Moving forward, we might consider adding the remaining qualitative variables, interactions, and higher order (or variable transformations). We then would go forward and assess the model.