Use case of Artificial Neural Network algorithm ,Multiple Linear Regression
Đặng Khải Hoàn
8/23/2020
Artificial Neural Network algorithm (ANN):
Abstract:
To measure the performance of building materials in the field of engeneering, accurate estimates of those materials can be done using neural network models. Put aside the interaction effects between the components of a process, artificial neural network approach can reliably predict the solidity of a material, given the input ingredients we have in hand. The data used to apply the ANN algorithm can be downloaded from https://raw.githubusercontent.com/stedy/Machine-Learning-with-R-datasets/master/concrete.csv. Details of this approach can be found in the book “Machine Learning with R”, by Brett Lantz.
Step 1: Load dataset:
link = "https://raw.githubusercontent.com/stedy/Machine-Learning-with-R-datasets/master/concrete.csv"
data = read.csv(link,sep=",",header = TRUE)
head(data)## cement slag ash water superplastic coarseagg fineagg age strength
## 1 540.0 0.0 0 162 2.5 1040.0 676.0 28 79.99
## 2 540.0 0.0 0 162 2.5 1055.0 676.0 28 61.89
## 3 332.5 142.5 0 228 0.0 932.0 594.0 270 40.27
## 4 332.5 142.5 0 228 0.0 932.0 594.0 365 41.05
## 5 198.6 132.4 0 192 0.0 978.4 825.5 360 44.30
## 6 266.0 114.0 0 228 0.0 932.0 670.0 90 47.03str(data)## 'data.frame': 1030 obs. of 9 variables:
## $ cement : num 540 540 332 332 199 ...
## $ slag : num 0 0 142 142 132 ...
## $ ash : num 0 0 0 0 0 0 0 0 0 0 ...
## $ water : num 162 162 228 228 192 228 228 228 228 228 ...
## $ superplastic: num 2.5 2.5 0 0 0 0 0 0 0 0 ...
## $ coarseagg : num 1040 1055 932 932 978 ...
## $ fineagg : num 676 676 594 594 826 ...
## $ age : int 28 28 270 365 360 90 365 28 28 28 ...
## $ strength : num 80 61.9 40.3 41 44.3 ...Step 2: Explore and prepare the data:
Rescale and Normalize data using a predefined function and Apply the function to our dataset.
normalize <-function(x){
return((x-min(x))/(max(x)-min(x)))
}
concrete_norm <- as.data.frame(lapply(data, normalize))hist(concrete_norm$strength,col = "red")
Verify that the range of the target variable is between 0 and 1 by comparing it to the original variable range.
summary(concrete_norm$strength)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.2664 0.4001 0.4172 0.5457 1.0000summary(data$strength)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 2.33 23.71 34.45 35.82 46.13 82.60Explore correlations between variables.
library(psych)
pairs.panels(concrete_norm)
Strength of material seems to have a moderate relation with only the cement variable.
Split data between training (75%) and test (25%) sets.
concrete_train<-concrete_norm[1:773,]
concrete_test<-concrete_norm[774:1030,]
head(concrete_train)## cement slag ash water superplastic coarseagg fineagg
## 1 1.0000000 0.0000000 0 0.3210863 0.07763975 0.6947674 0.2057200
## 2 1.0000000 0.0000000 0 0.3210863 0.07763975 0.7383721 0.2057200
## 3 0.5262557 0.3964942 0 0.8482428 0.00000000 0.3808140 0.0000000
## 4 0.5262557 0.3964942 0 0.8482428 0.00000000 0.3808140 0.0000000
## 5 0.2205479 0.3683918 0 0.5607029 0.00000000 0.5156977 0.5807827
## 6 0.3744292 0.3171953 0 0.8482428 0.00000000 0.3808140 0.1906673
## age strength
## 1 0.07417582 0.9674847
## 2 0.07417582 0.7419958
## 3 0.73901099 0.4726548
## 4 1.00000000 0.4823720
## 5 0.98626374 0.5228603
## 6 0.24450549 0.5568706head(concrete_test)## cement slag ash water superplastic coarseagg fineagg
## 774 0.6392694 0 0 0.5127796 0 0.7151163 0.3637732
## 775 0.6392694 0 0 0.5127796 0 0.9011628 0.4766683
## 776 0.4086758 0 0 0.5127796 0 0.8808140 0.4515805
## 777 0.5410959 0 0 0.5047923 0 0.7790698 0.4014049
## 778 0.5410959 0 0 0.5047923 0 0.7790698 0.4014049
## 779 0.4406393 0 0 0.5047923 0 0.7790698 0.4390366
## age strength
## 774 0.07417582 0.4371496
## 775 0.01648352 0.1138657
## 776 0.07417582 0.2505295
## 777 0.01648352 0.2345833
## 778 0.07417582 0.3676342
## 779 0.01648352 0.1553507Step 3: Train the model on the data:
Use a multilayer feedforward neural network (MLP) to model the relationship between the ingredients used in concrete and the finished product.This is a model of a single hidden node by default.
# Building the model
library(neuralnet)
set.seed(12345) # to guarantee repeatable results
concrete_model<-neuralnet(formula = strength ~ cement + slag +
ash + water + superplastic +
coarseagg + fineagg + age,
data = concrete_train)Visualize the network topology (multilayer and single hidden node).
plot(concrete_model, rep="best")
bove is displayed a simple model of ann with a single hidden node.The weight between each input node and the hidden node can be interpreted as a regression coefficient. Which makes it very similar to multiple regression model algorithm.
Alternative plot:
library(NeuralNetTools)
par(mar = numeric(4), family = 'serif')
plotnet(concrete_model, alpha=0.6)
Step 4:: Evaluate the model performance
The prediction problem here is to model the relationship between the numericalfeatures and the target variable (strength of our components). Thus we must measure the correlation between our predicted concrete strength and the true value, so that we can have an insight in the strength of linear link between variables. The compute function has 2 elements:neurons and net.results. This latter stores the predicted values.
# Make predictions using the model trained by neuralnet function
model_results<-compute(concrete_model, concrete_train[1:8])
# display the predicted scores of the target variable
predicted_strength<-model_results$net.result
# Correlations between predicted and actual values of target variable
cor(predicted_strength, concrete_train$strength)## [,1]
## [1,] 0.8356286The score close to 1 indicates a strong relationship between the target variable and the predictors.
Step 5:Improve the model performance.
With a more complex topology we can increase the number of hidden nodes to 3 (parameter hidden) to see if the model performs better.
concrete_model_2<-neuralnet(formula = strength ~ cement + slag +
ash + water + superplastic +
coarseagg + fineagg + age,
data = concrete_train, hidden =3)
plot(concrete_model_2, rep="best")
Compare the predicted values to the true values on model 2.
model_results_2<-compute(concrete_model_2, concrete_train[1:8])
predicted_strength_2<-model_results_2$net.result
cor(predicted_strength_2, concrete_train$strength)## [,1]
## [1,] 0.9387237The correlation (0.94) is much better than in the previous model with only a single hidden node.There is also a huge increase in the number of connections.The prediction error has been reduced(from 5 to 2). The model performance has thus been fairly improved.
Multiple Linear Regression:
mlg = lm(data = concrete_test, strength ~ cement + slag +
ash + water + superplastic +
coarseagg + fineagg + age)
summary(mlg)##
## Call:
## lm(formula = strength ~ cement + slag + ash + water + superplastic +
## coarseagg + fineagg + age, data = concrete_test)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.295401 -0.044536 0.003475 0.039928 0.181617
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.38821 0.20165 1.925 0.05535 .
## cement 0.57492 0.08813 6.523 3.83e-10 ***
## slag 0.35836 0.08793 4.075 6.19e-05 ***
## ash -0.01517 0.06209 -0.244 0.80724
## water -0.31673 0.09552 -3.316 0.00105 **
## superplastic 0.02686 0.05486 0.490 0.62480
## coarseagg -0.23171 0.08502 -2.725 0.00688 **
## fineagg -0.14825 0.11363 -1.305 0.19322
## age 0.29118 0.04282 6.801 7.74e-11 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.07471 on 248 degrees of freedom
## Multiple R-squared: 0.7858, Adjusted R-squared: 0.7789
## F-statistic: 113.7 on 8 and 248 DF, p-value: < 2.2e-16predicted_mlg = predict(mlg,newdata = concrete_train)
cor(predicted_mlg, concrete_train$strength)## [1] 0.6553491