Multi Target Regression - Lab
Bert Gollnick
1 Data Understanding
We will work on Energy Efficiency. More information on the data can be found here.
We perform energy analysis using 12 different building shapes simulated in Ecotect. The buildings differ with respect to the glazing area, the glazing area distribution, and the orientation, amongst other parameters. We simulate various settings as functions of the afore-mentioned characteristics to obtain 768 building shapes. The dataset comprises 768 samples and 8 features, aiming to predict two real valued responses. It can also be used as a multi-class classification problem if the response is rounded to the nearest integer.
We have these attributes:
The dataset contains eight attributes (or features, denoted by X1…X8) and two responses (or outcomes, denoted by y1 and y2). The aim is to use the eight features to predict each of the two responses.
Specifically: X1 Relative Compactness X2 Surface Area X3 Wall Area X4 Roof Area X5 Overall Height X6 Orientation X7 Glazing Area X8 Glazing Area Distribution y1 Heating Load y2 Cooling Load
2 Data Preparation
2.1 Packages
library(dplyr)
library(ggplot2)
library(keras)
library(readxl)
library(caret)
source("./functions/train_val_test.R")2.2 Data Import
# if file does not exist, download it first
file_path <- "./data/energy.xlsx"
if (!file.exists(file_path)) {
dir.create("./data")
url <- "http://archive.ics.uci.edu/ml/machine-learning-databases/00242/ENB2012_data.xlsx"
download.file(url = url,
destfile = file_path,
mode = "wb")
}
energy_raw <- readxl::read_xlsx(path = file_path)We now have768 observations and 10 variables.
These column names are detected:
energy_raw %>% colnames## [1] "X1" "X2" "X3" "X4" "X5" "X6" "X7" "X8" "Y1" "Y2"You can see we have eight independent variables and two dependent (target) variables.
energy_raw %>% summary## X1 X2 X3 X4
## Min. :0.6200 Min. :514.5 Min. :245.0 Min. :110.2
## 1st Qu.:0.6825 1st Qu.:606.4 1st Qu.:294.0 1st Qu.:140.9
## Median :0.7500 Median :673.8 Median :318.5 Median :183.8
## Mean :0.7642 Mean :671.7 Mean :318.5 Mean :176.6
## 3rd Qu.:0.8300 3rd Qu.:741.1 3rd Qu.:343.0 3rd Qu.:220.5
## Max. :0.9800 Max. :808.5 Max. :416.5 Max. :220.5
## X5 X6 X7 X8 Y1
## Min. :3.50 Min. :2.00 Min. :0.0000 Min. :0.000 Min. : 6.01
## 1st Qu.:3.50 1st Qu.:2.75 1st Qu.:0.1000 1st Qu.:1.750 1st Qu.:12.99
## Median :5.25 Median :3.50 Median :0.2500 Median :3.000 Median :18.95
## Mean :5.25 Mean :3.50 Mean :0.2344 Mean :2.812 Mean :22.31
## 3rd Qu.:7.00 3rd Qu.:4.25 3rd Qu.:0.4000 3rd Qu.:4.000 3rd Qu.:31.67
## Max. :7.00 Max. :5.00 Max. :0.4000 Max. :5.000 Max. :43.10
## Y2
## Min. :10.90
## 1st Qu.:15.62
## Median :22.08
## Mean :24.59
## 3rd Qu.:33.13
## Max. :48.03All variables are numeric.
2.3 Train / Validation / Test Split
We will use 80 % training data and 20 % testing data.
c(train, val, test) %<-% train_val_test_split(df = energy_raw,
train_ratio = 0.8,
val_ratio = 0.0,
test_ratio = 0.2)3 Modeling
The data will be transformed to a matrix.
x_train<-train%>%select(-Y1, -Y2)%>%as.matrix
y_train<-train%>%select(Y1, Y2)%>%as.matrix
x_test<-test%>%select(-Y1, -Y2)%>%as.matrix
y_test<-test%>%select(Y1, Y2)%>%as.matrix3.1 Data Scaling
The data is scaled. This is highly recommended, because features can have very different ranges. This can speed up the training process and avoid convergence problems.
x_train_scale<-x_train%>%scale()
col_mean_train<-attr(x_train_scale, "scaled:center")
col_sd_train<-attr(x_train_scale, "scaled:scale")
x_test_scale<-x_test%>%scale(center = col_mean_train, scale=col_sd_train)3.2 Initialize Model
dnn_reg_model<-keras_model_sequential()3.3 Add Layers
We add two hidden layers and one output layer with two units for prediction.
dnn_reg_model%>%layer_dense(units = 50, activation = 'relu', input_shape = c(ncol(x_train_scale)))%>%
layer_dense(units=10, activation = 'relu')%>%
layer_dense(units=2, activation = 'relu')Let’s take a look at the model details.
dnn_reg_model %>% summary()## Model: "sequential"
## ________________________________________________________________________________
## Layer (type) Output Shape Param #
## ================================================================================
## dense (Dense) (None, 50) 450
## ________________________________________________________________________________
## dense_1 (Dense) (None, 10) 510
## ________________________________________________________________________________
## dense_2 (Dense) (None, 2) 22
## ================================================================================
## Total params: 982
## Trainable params: 982
## Non-trainable params: 0
## ________________________________________________________________________________It is a very small model with only close to 1000 parameters.
3.4 Loss Function, Optimizer, Metric
dnn_reg_model%>%compile(optimizer=optimizer_adam(),
loss='mean_absolute_error')We put all in one function, because we need to run it each time a model should be trained.
create_model <- function() {
dnn_reg_model <-
keras_model_sequential() %>%
layer_dense(units = 50,
activation = 'relu',
input_shape = c(ncol(x_train_scale))) %>%
layer_dense(units = 50, activation = 'relu') %>%
layer_dense(units = 2, activation = 'relu') %>%
compile(optimizer = optimizer_rmsprop(),
loss = 'mean_absolute_error')
}3.5 Model Fitting
We fit the model and stop after 80 epochs. A validation ratio of 20 % is used for evaluating the model.
dnn_reg_model<-create_model()
history<-dnn_reg_model%>%keras::fit(x=x_train_scale, y=y_train, epochs=80, validation_split=.2, verbose=0, batch_size=128)
plot(history, smooth=F)There is not much improvement after 40 epochs. We implement an approach, in which training stops if there is no further improvement.
We use a patience parameter for this. It represents the nr of epochs to analyse for possible improvements.
# re-create our model for this new run
# code here
dnn_reg_model<-create_model()
early_stop<-callback_early_stopping(monitor="val_loss", patience = 20)
history<-dnn_reg_model%>%keras::fit(x=x_train_scale, y=y_train, epochs=200, validation_split=0.2, verbose=0, batch_size=128, callbacks=list(early_stop))
plot(history,
smooth = F)
4 Model Evaluation
We will create predictions and create plots to show correlation of prediction and actual values.
4.1 Predictions
First, we create predictions, that we then can compare to actual values.
y_test_pred<-predict(object=dnn_reg_model, x=x_test_scale)
y_test_pred %>% head## [,1] [,2]
## [1,] 20.924156 24.32765
## [2,] 24.438129 27.74497
## [3,] 28.741470 31.44666
## [4,] 7.291873 11.18135
## [5,] 9.618581 12.14366
## [6,] 10.030615 11.85370We can see that we have two output columns, which refer to our two target variables.
4.2 Check Performance
test$y1_pred<-y_test_pred[,1]
test$y2_pred<-y_test_pred[,2]We create correlation plots for Y1 and Y2.
R2_test <- caret::postResample(pred = test$y1_pred, obs = test$Y1)
g <- ggplot(test, aes(Y1, y1_pred))
g <- g + geom_point(alpha = .5)
g <- g + annotate(geom = "text", x = 15, y = 30, label = paste("R**2 = ", round(R2_test[2], 3)))
g <- g + labs(x = "Actual Y1", y = "Predicted Y1", title = "Y1 Correlation Plot")
g <- g + geom_smooth(se=F, method = "lm")
g
R2_test <- caret::postResample(pred = test$y2_pred, obs = test$Y2)
g <- ggplot(test, aes(Y2, y2_pred))
g <- g + geom_point(alpha = .5)
g <- g + annotate(geom = "text", x = 15, y = 30, label = paste("R**2 = ", round(R2_test[2], 3)))
g <- g + labs(x = "Actual Y2", y = "Predicted Y2", title = "Y2 Correlation Plot")
g <- g + geom_smooth(se=F, method = "lm")
g
5 Hyperparameter Tuning
Now we could move forward and adapt
network topology,
count of layers,
type of layers,
count of nodes per layer,
loss function,
activation function,
learning rate,
and much more, …
Play around with the parameters and see how they impact the result.
6 Acknowledgement
We thank the authors of the dataset:
The dataset was created by Angeliki Xifara (angxifara ‘@’ gmail.com, Civil/Structural Engineer) and was processed by Athanasios Tsanas (tsanasthanasis ‘@’ gmail.com, Oxford Centre for Industrial and Applied Mathematics, University of Oxford, UK).