Multiple Regression: A Case Study in Real Estate Valuation

This lecture walks you through performing a multiple regression analysis in R the California Housing Price dataset. This dataset includes both numerical and categorical variables, allowing us to cover all the required topics.

We use the tidyverse package for data manipulation and visualization, and the base R functions for regression.

Setup and Data Preparation

First, we need to load the necessary library and a suitable dataset. The modeldata package contains the California Housing data.

# Install and load necessary packages (you only need to run install.packages once)
# install.packages(c("tidyverse", "modeldata"))
library(tidyverse)
library(modeldata)

# Load the California Housing dataset
data(ames) # This loads the Ames, Iowa housing data, a common choice for grad-level examples
housing_data <- ames %>%
  drop_na() %>%
  # Convert Sale_Price to log scale, which often improves model performance for price data
  mutate(log_Sale_Price = log(Sale_Price)) %>%
  # Select a few key variables and rename for clarity
  select(Gr_Liv_Area, Year_Built, Garage_Area, Total_Bsmt_SF, Garage_Cars, Year_Remod_Add, First_Flr_SF, Full_Bath, Garage_Type, Fireplaces, log_Sale_Price ) %>%
  # Filter out potential outliers for clean analysis (optional but good practice)
  filter(Gr_Liv_Area < 4000)

# Display the first few rows of the data
head(housing_data)

Our response variable (\(Y\)) is Sale_Price, and we’ll use several explanatory variables (\(X\)’s).

Multiple Regression Model

We start with a basic multiple regression model including only numerical explanatory variables: Gr_Liv_Area (Above Ground Living Area), Year_Built, and Garage_Area (Size of garage in square feet).

\[ \hat{y}_{\text{log_Sale_Price}} = \beta_0 + \beta_1x_{\text{Gr_Liv_Area}} + \beta_2x_{\text{Year_Built}} + \beta_3x_{\text{Garage_Area}} + \epsilon \]

# Fit the model in R using the lm() function
model_num <- lm(log_Sale_Price ~ Gr_Liv_Area + Year_Built + Garage_Area, data = housing_data)

# Print a summary of the model
summary(model_num)

Call:
lm(formula = log_Sale_Price ~ Gr_Liv_Area + Year_Built + Garage_Area, 
    data = housing_data)

Residuals:
     Min       1Q   Median       3Q      Max 
-2.16196 -0.10463  0.00262  0.11702  0.80234 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 9.571e-01  2.749e-01   3.482 0.000504 ***
Gr_Liv_Area 4.274e-04  8.784e-06  48.655  < 2e-16 ***
Year_Built  5.186e-03  1.416e-04  36.611  < 2e-16 ***
Garage_Area 4.276e-04  2.209e-05  19.359  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.2032 on 2921 degrees of freedom
Multiple R-squared:  0.7497,    Adjusted R-squared:  0.7494 
F-statistic:  2916 on 3 and 2921 DF,  p-value: < 2.2e-16

Interpreting Parameter Coefficients

This linear model uses a log-transformed response variable (\(ln(\text{Sale_Price})\)), which means we interpret the coefficients in terms of percentage change in the sale price.

• Intercept (\(\hat{\beta}_0\)): The estimated mean log-sale price is \(\mathbf{0.9571}\) when all predictors (Gr_Liv_Area, Year_Built, and Garage_Area) are zero. Exponentiating this (\(\exp(0.9571) \approx \$2.60\)) gives the estimated sale price under these (impractical) conditions.

• Gr_Liv_Area (\(\hat{\beta}_1\) \(\approx 4.274 \times 10^{-4}\)): For every one square foot increase in above-ground living area, the mean sale price is estimated to increase by approximately \(\mathbf{0.04274\%}\) (since \(4.274 \times 10^{-4} \times 100\% = 0.04274\%\)), holding all other variables constant.

• Year_Built (\(\hat{\beta}_2\) \(\approx 5.186 \times 10^{-3}\)): For every one year newer the house is, the mean sale price is estimated to increase by approximately \(\mathbf{0.5186\%}\) (since \(5.186 \times 10^{-3} \times 100\% = 0.5186\%\)), holding all other variables constant.

• Garage_Area (\(\hat{\beta}_3\) \(\approx 4.276 \times 10^{-4}\)): For every one square foot increase in garage area, the mean sale price is estimated to increase by approximately \(\mathbf{0.04276\%}\) (since \(4.276 \times 10^{-4} \times 100\% = 0.04276\%\)), holding all other variables constant.

Since all \(P\)-values are extremely small (\(< 2e-16\)), all three explanatory variables are highly significant predictors of the log-transformed sale price. The biggest practical effect, on a percentage basis, comes from Year_Built.

Adding a Categorical Variable in Regression

We now add the categorical variable Garage_Type to the model. \(\mathbf{R}\) automatically handles categorical variables by creating dummy (indicator) variables.

Since Neighborhood has multiple levels, \(\mathbf{R}\) chooses one level as the reference level (usually the first one alphabetically) and creates \(k-1\) dummy variables for a categorical variable with \(k\) levels.

\[ \text{ln(Sale_Price)} = \beta_0 + \beta_1(\text{Gr_Liv_Area}) + \dots + \beta_4(\text{Garage_Type_Indicator_Attchd}) + \beta_5(\text{Garage_Type_Indicator_Basment}) + \dots + \epsilon \]

# Fit the model including the categorical variable Neighborhood
model_cat <- lm(log_Sale_Price ~ Gr_Liv_Area + Year_Built + Garage_Area  + Garage_Type, data = housing_data)

# Print a summary of the new model
summary(model_cat)

Call:
lm(formula = log_Sale_Price ~ Gr_Liv_Area + Year_Built + Garage_Area + 
    Garage_Type, data = housing_data)

Residuals:
     Min       1Q   Median       3Q      Max 
-2.17301 -0.09957  0.00092  0.11073  0.85350 

Coefficients:
                                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)                     2.329e+00  3.264e-01   7.135 1.22e-12 ***
Gr_Liv_Area                     4.273e-04  9.194e-06  46.470  < 2e-16 ***
Year_Built                      4.517e-03  1.661e-04  27.201  < 2e-16 ***
Garage_Area                     3.992e-04  2.562e-05  15.583  < 2e-16 ***
Garage_TypeBasment             -1.038e-01  3.353e-02  -3.096  0.00198 ** 
Garage_TypeBuiltIn             -8.549e-02  1.600e-02  -5.342 9.89e-08 ***
Garage_TypeCarPort             -2.944e-01  5.165e-02  -5.699 1.32e-08 ***
Garage_TypeDetchd              -8.597e-02  1.045e-02  -8.228 2.83e-16 ***
Garage_TypeMore_Than_Two_Types -2.329e-01  4.260e-02  -5.467 4.97e-08 ***
Garage_TypeNo_Garage           -1.252e-01  2.010e-02  -6.232 5.28e-10 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1981 on 2915 degrees of freedom
Multiple R-squared:  0.7627,    Adjusted R-squared:  0.7619 
F-statistic:  1041 on 9 and 2915 DF,  p-value: < 2.2e-16

Interpreting Categorical Parameters for Garage_Type

Since the response variable is \(\ln(\text{Sale\_Price})\), the coefficients for the dummy variables represent the approximate percentage change in the mean Sale Price relative to the reference group, holding all other variables constant.

The categorical variable is Garage_Type, and the levels listed in the coefficient table are Basment, BuiltIn, CarPort, Detchd, More_Than_Two_Types, and No_Garage. This means the reference level (the level not included in the coefficients) is Attchd (Attached Garage).

The following interpretations compare each garage type to the Attached Garage (Attchd) reference group, assuming that the numerical variables (Gr_Liv_Area, Year_Built, and Garage_Area) are the same.

• Garage_TypeBasment (\(\hat{\beta} \approx -0.1038\)): A house with a Basement garage is estimated to have a mean sale price that is approximately \(10.38\%\) lower than a comparable house with an Attached garage, holding all other variables constant.

• Garage_TypeBuiltIn (\(\hat{\beta} \approx -0.0855\)): A house with a Built-In garage is estimated to have a mean sale price that is approximately \(8.55\%\) lower than a comparable house with an Attached garage, holding all other variables constant.

• Garage_TypeCarPort (\(\hat{\beta} \approx -0.2944\)): A house with a Car Port is estimated to have a mean sale price that is approximately \(29.44\%\) lower than a comparable house with an Attached garage, holding all other variables constant. This represents the largest negative impact among the types.

• Garage_TypeDetchd (\(\hat{\beta} \approx -0.0860\)): A house with a Detached garage is estimated to have a mean sale price that is approximately \(8.60\%\) lower than a comparable house with an Attached garage, holding all other variables constant.

• Garage_TypeMore_Than_Two_Types (\(\hat{\beta} \approx -0.2329\)): A house with More Than Two Types of garage is estimated to have a mean sale price that is approximately \(23.29\%\) lower than a comparable house with an Attached garage, holding all other variables constant.

• Garage_TypeNo_Garage (\(\hat{\beta} \approx -0.1252\)): A house with No Garage is estimated to have a mean sale price that is approximately \(12.52\%\) lower than a comparable house with an Attached garage, holding all other variables constant.

Inference for Regression Parameters (and A Case Study)

The \(\mathbf{R}\) summary() output provides the necessary information for inference:

Term	Estimate	Std. Error	t value	Pr(\(>|t|\))
\(\dots\)	\(\dots\)	\(\dots\)	\(\dots\)	\(\dots\)

Inference for Regression Parameters

For each coefficient \(\beta_j\), we test the null hypothesis \(H_0: \beta_j = 0\) against the alternative \(H_a: \beta_j \ne 0\).

1. Test Statistic: The \(t\)-statistic is calculated as \(t = \frac{\hat{\beta}_j - 0}{\text{SE}(\hat{\beta}_j)}\).

2. P-value: The \(\mathbf{P}\)-value (Pr($>|t|$)) is the probability of observing a \(t\)-statistic as extreme as the one calculated, assuming \(H_0\) is true.

Case Study: Gr_Liv_Area

The \(t\)-value for Gr_Liv_Area is typically very large, and the \(\mathbf{P}\)-value is typically extremely small (\(< 2 \times 10^{-16}\)).

• Conclusion: Since the \(\mathbf{P}\)-value is much less than a standard significance level like \(\alpha=0.05\), we reject the null hypothesis \(H_0: \beta_{\text{Gr\_Liv\_Area}} = 0\).

• Interpretation: There is strong statistical evidence that \(\mathbf{Gr\_Liv\_Area}\) has a linear relationship with \(\mathbf{Sale\_Price}\), controlling for the other predictors in the model. In other words, its coefficient is significantly different from zero.

Confidence Intervals

We often use confidence intervals (CIs) to estimate the true population parameter \(\beta_j\). A \(95\%\) CI is \((\hat{\beta}_j \pm t^* \times \text{SE}(\hat{\beta}_j))\).

# Get confidence intervals for all coefficients
confint(model_cat, level = 0.95)

                                       2.5 %        97.5 %
(Intercept)                     1.6888233837  2.9687612498
Gr_Liv_Area                     0.0004092390  0.0004452955
Year_Built                      0.0041913047  0.0048424996
Garage_Area                     0.0003489689  0.0004494272
Garage_TypeBasment             -0.1695750108 -0.0380743640
Garage_TypeBuiltIn             -0.1168644361 -0.0541109002
Garage_TypeCarPort             -0.3956534502 -0.1931036982
Garage_TypeDetchd              -0.1064609915 -0.0654859349
Garage_TypeMore_Than_Two_Types -0.3163980070 -0.1493448294
Garage_TypeNo_Garage           -0.1646482187 -0.0858345788

Interpretation of CI for Gr_Liv_Area (e.g., [\(\mathbf{0.0409\%}\), \(\mathbf{0.0445\%}\)]): We are \(95\%\) confident that the true population mean increase in Sale Price for a one-unit increase in Gr_Liv_Area, holding all other variables constant, is between \(\mathbf{0.0409\%}\) and \(\mathbf{0.0445\%}\).

Correlations Between Explanatory Variables (Multicollinearity)

Multicollinearity occurs when two or more explanatory variables in a multiple regression model are highly linearly correlated. It doesn’t bias the estimates but makes them less precise (larger Standard Errors and \(P\)-values), making interpretation unstable.

# Check correlation among the numerical predictors
cor_data <- housing_data %>%
  mutate(Garage_Type=as.numeric(Garage_Type)) %>%
  select(Gr_Liv_Area, Year_Built, Garage_Area, Garage_Type)
cor(cor_data)

            Gr_Liv_Area Year_Built Garage_Area Garage_Type
Gr_Liv_Area   1.0000000  0.2393052   0.4749449  -0.2351748
Year_Built    0.2393052  1.0000000   0.4795550  -0.5427980
Garage_Area   0.4749449  0.4795550   1.0000000  -0.4193524
Garage_Type  -0.2351748 -0.5427980  -0.4193524   1.0000000

The correlation matrix will show the pairwise correlation coefficients. If any absolute correlation value is high (e.g., \(|r| > 0.8\)), we should be concerned. Nothing here causes concern.

If multicollinearity is a concern, consider dropping one of the highly correlated variables or creating an index/composite score.

---

Interactions

An interaction term allows the effect of one predictor on the response variable to depend on the value of another predictor.

We hypothesize that the effect of Gr_Liv_Area on Sale_Price might depend on the Year_Built of the house. A square foot of living space might be more valuable in a older home than a newer one.

\[ \text{ln(Sale_Price)} = \beta_0 + \beta_1(\text{Gr_Liv_Area}) + \dots + \beta_4(\mathbf{\text{Gr_Liv_Area} \times \text{Year_Built}}) + \epsilon \]

# Fit a model with an interaction term
# The colon ":" indicates an interaction between two variables
model_int <- lm(log_Sale_Price ~ Gr_Liv_Area + Year_Built + Gr_Liv_Area:Year_Built, data = housing_data)

summary(model_int)

Call:
lm(formula = log_Sale_Price ~ Gr_Liv_Area + Year_Built + Gr_Liv_Area:Year_Built, 
    data = housing_data)

Residuals:
    Min      1Q  Median      3Q     Max 
-2.0406 -0.1214  0.0130  0.1270  0.6729 

Coefficients:
                         Estimate Std. Error t value Pr(>|t|)    
(Intercept)            -1.643e-01  8.626e-01  -0.190    0.849    
Gr_Liv_Area            -1.955e-04  5.189e-04  -0.377    0.706    
Year_Built              5.801e-03  4.390e-04  13.215   <2e-16 ***
Gr_Liv_Area:Year_Built  3.530e-07  2.636e-07   1.339    0.181    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.2158 on 2921 degrees of freedom
Multiple R-squared:  0.7177,    Adjusted R-squared:  0.7175 
F-statistic:  2476 on 3 and 2921 DF,  p-value: < 2.2e-16

I’d be happy to provide that interpretation. When analyzing this log-linear model, remember that the interaction coefficient, \(\hat{\beta}_{\text{int}}\), represents the change in the rate of percentage change in the sale price.

Interpreting Interaction Parameters

The interaction coefficient, \(\hat{\beta}_{\text{int}}\) (for Gr_Liv_Area:Year_Built), estimates how the slope (the percentage effect) of Gr_Liv_Area changes for a one-year increase in Year_Built.

• Effective Slope: The relationship between Living Area and log Sale Price is now conditional. The effective increase in \(\ln(\text{Sale_Price})\) for one additional square foot of Gr_Liv_Area is:

  \[\text{Effective Slope}_{\text{Gr_Liv_Area}} = (-1.955 \times 10^{-4}) + (3.530 \times 10^{-7}) \times \text{Year_Built}\]

• Interpretation of the Estimate: The positive interaction coefficient (\(\hat{\beta}_{\text{int}} \approx 3.530 \times 10^{-7}\)) suggests that the marginal benefit (the value) of an additional square foot of living area increases slightly for newer homes (i.e., as Year_Built increases), holding all other variables constant.

• Inference: With a P-value of \(0.181\), the interaction term is not statistically significant at a standard \(\alpha=0.05\) level. This means we cannot conclude there is strong evidence that the effect of living area on sale price significantly depends on the year the house was built. The non-interactive model might provide a more parsimonious fit.

Checking the Conditions for Inference (The “LINE” Conditions)

Reliable inference (P-values and CIs) relies on four key assumptions about the model errors (\(\epsilon\)):

1. Linearity: The relationship between \(Y\) and the \(X\)’s is linear.
2. Independence: The errors are independent of each other. (Usually checked by context, e.g., houses are independent observations).
3. Normality: The errors are Normally distributed.
4. Equal Variance (Homoscedasticity): The errors have constant variance across all levels of the predictors.

These are primarily checked by examining the Residuals (the difference between the observed \(Y\) and the predicted \(\hat{Y}\)).

# 1 & 4. Linearity and Equal Variance (Homoscedasticity)
# Plot of Residuals vs. Fitted Values
plot(model_cat, which = 1)

• Ideal Plot: A random scatter of points around the horizontal line at \(0\).

• What to look for: A fan shape (heteroscedasticity) or a curved pattern (non-linearity). The housing data often shows a fan shape, indicating heteroscedasticity. This is a common issue with price data, often fixed by log-transforming the response variable: lm(log(Sale_Price) ~ ...)

# 3. Normality of Residuals
# Q-Q Plot
plot(model_cat, which = 2)

• Ideal Plot: Points lie closely along the diagonal straight line.

• What to look for: Significant deviation from the line, especially in the tails, indicating non-Normality.

If conditions are severely violated, transformations (like log) or more advanced methods (e.g., Robust Regression, Generalized Least Squares) are required.