Draw a Regression Plane

# Necessray Library
library(scatterplot3d) # This library will allow us to draw 3d plot

Statistics 2 Class Textbook page 696.

We have data sets containing 3 variables (x1, x2, y)

y: reaction to Drug B

x1: reaction to Drug A

x2: heart rate

We want to exaplain y with variables x1 and x2 by employing least squares method

# Input data
x1 <- c(1.9,0.8,1.1,0.1,-0.1,4.4,4.6,1.6,5.5,3.4)
x2 <- c(66, 62, 64, 61, 63, 70, 68, 62, 68, 66)
y <- c(0.7,-1.0,-0.2,-1.2,-0.1,3.4,0.0,0.8,3.7,2.0)
dataset = cbind.data.frame(x1,x2,y)
scatterplot3d(x1,x2,y)

# Use optimization to find regression coefficients
# Same as fminsearch in matlab
ls <- function(dataset, par) 
        {with(dataset, sum((y-par[1]-par[2]*x1-par[3]*x2)^2))}
result <- optim(par=c(0,0,0),ls,data=dataset) 
# starting values for the parameters to be optimised (0,0,0)
coef <- result$par
coef

[1] -11.4591902   0.4500494   0.1726229

y_hat

 [1]  0.78901682 -0.39652923  0.08373144 -0.88418673 -0.62895076  2.60463202  2.34939604
 [8] -0.03648971  2.75444050  1.46409091

Multiple Regression with Matrix Algebra

Projection of y into the Column Space X = [X1,X2]

Projection Matrix, P = X(X`X)^-1X

Y_hat = P * Y

# fit a linear model using lm model and draw a regression plane
#plot3d <- scatterplot3d(x1,x2,y, type="h", highlight.3d=TRUE,angle=55, scale.y=0.7, pch=16)
plot3d <- scatterplot3d(x1,x2,y,
angle=55, scale.y=0.7, pch=16, color ="red", main ="Regression Plane")
my.lm<- lm(y ~ x1 + x2,data=dataset)
plot3d$plane3d(my.lm, lty.box = "solid")

plot3d$points3d(x1,x2,y_hat,col="blue", type="h", pch=16)

Red dots are data points of y

Blue dots are data points of y_hat

summary(my.lm)                        

Call:
lm(formula = y ~ x1 + x2, data = dataset)

Residuals:
    Min      1Q  Median      3Q     Max 
-2.3497 -0.3078  0.2202  0.7304  0.9451 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept) -11.4528    16.7952  -0.682    0.517
x1            0.4503     0.4142   1.087    0.313
x2            0.1725     0.2715   0.635    0.545

Residual standard error: 1.125 on 7 degrees of freedom
Multiple R-squared:  0.663, Adjusted R-squared:  0.5668 
F-statistic: 6.887 on 2 and 7 DF,  p-value: 0.02221

Done