BANA 6610: Homework 5

Seth Monahan (ID: 108740098)

Part One:

Seattle Homes

Seattle home Prices. Home prices are often modeled by looking at the square footage of a home, number of bathrooms and other characteristics. One interesting question is if even after taking into account these characteristics is there evidence for differences in the listing prices for different Realtors? Is it realistic to assume that some Realtors tend to under or over valuate their listed properties? This dataset has information on the sales prices of 36 homes listed in the Seattle area. Of these homes, 28 were listed by one Realtor while 8 of the homes were listed by a different Realtor. In this dataset we also have information on the price of the listings in thousands of dollars, the total square feet, the price in dollars per square feet, the Realtor indicator, the number of bedrooms and the number of bathrooms.

Response Variable:

Price ($000): The dollar sales of the convenience store

Explanatory Variables

• Square Feet: The total number of square feet in the home
• Price/Sq Ft: The listing price (in dollars) per square foot
• Realtor: A categorical variable indicating the Realtor
• Bedrooms: Number of bedrooms
• Bathrooms: Number of bathrooms

Exploring Seattle Homes Data

library(car)
library(leaps)
library(ggplot2)
# Set working directory
setwd("~/OneDrive - The University of Colorado Denver/BANA 6610/Homework 5")
# Bring numerical data in to R
seahomes <- read.csv("realestatedata-1.csv", header = T, sep = ",")
head(seahomes)

1. Create a plot that allows us to make all bivariate comparisons for this data.

#Plot pairs
# Use pairs
pairs(seahomes)

2. Make a plot of price vs square feet. Color the plot by realtor.

qplot(x = seahomes$Square.Feet, y = seahomes$Price, col = seahomes$Realtor)

3. Create a histogram of the square footage, as well as histogram of price.

#Histogram of square footage
hist(seahomes$Square.Feet)

#Histogram of price
hist(seahomes$Price)

4. Look at the summary statistics associated with all variables.

summary(seahomes)

     Price        Square.Feet     Price.SqFt     Realtor    Bedrooms       Bathrooms    
 Min.   :165.0   Min.   : 868   Min.   :0.1164   A:28    Min.   :1.000   Min.   :1.000  
 1st Qu.:281.8   1st Qu.:1535   1st Qu.:0.1601   B: 8    1st Qu.:3.000   1st Qu.:2.000  
 Median :386.5   Median :1952   Median :0.2070           Median :3.000   Median :2.000  
 Mean   :407.4   Mean   :1940   Mean   :0.2167           Mean   :3.111   Mean   :2.167  
 3rd Qu.:575.0   3rd Qu.:2345   3rd Qu.:0.2485           3rd Qu.:4.000   3rd Qu.:3.000  
 Max.   :625.0   Max.   :3260   Max.   :0.4437           Max.   :4.000   Max.   :3.000  

Modeling Seattle Homes Data

1. Create a model using all quantitative variables to predict price.

#Linear regression model 
seamod <- lm(Price ~ Square.Feet+Price.SqFt+Bedrooms+Bathrooms,data = seahomes)
summary(seamod)

Call:
lm(formula = Price ~ Square.Feet + Price.SqFt + Bedrooms + Bathrooms, 
    data = seahomes)

Residuals:
    Min      1Q  Median      3Q     Max 
-82.741 -14.273   5.019  18.186  47.703 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) -347.91023   38.80814  -8.965 4.07e-10 ***
Square.Feet    0.18435    0.01159  15.912  < 2e-16 ***
Price.SqFt  1696.37258   85.25337  19.898  < 2e-16 ***
Bedrooms      25.15235   10.72116   2.346   0.0255 *  
Bathrooms    -22.21939   10.35955  -2.145   0.0399 *  
---
NA

Residual standard error: 31.33 on 31 degrees of freedom
Multiple R-squared:  0.9604,    Adjusted R-squared:  0.9553 
F-statistic:   188 on 4 and 31 DF,  p-value: < 2.2e-16

par(mfrow=c(2,2))
plot(seamod)

2. Based on the EDA we performed, do you have any concerns regarding the model we built in the previous part? Why or why not?
   The R-squared and adjusted R-squared values look very good. However, I’m concerned about 3 things: a. The Square Ft and Price Per Sq Ft variables may be collinear; b. Price is not normally distributed and outliers may be skewing the results; c. The scatter plot of price vs. Price per Sq Ft doesn’t appear to be a strong linear relationship.

3. Is your overall model useful? Provide the results of a hypothesis test.
   (assuming 95% confidence interval and \(\alpha\) = 0.05)
   H₀: b₁ = b₂ = … b_k = 0; all variables are irrelevant for determining Price.
   H₁: any variable != 0 therefore at least one is helpful in determining Price.
   Our model’s F-statistic has a p-value very near zero, far below alpha, therefore we reject the null hypothesis and state that at least one variable is meaningful in predicting Price.
   
4. Which individual variables are useful for predicting price based on individual t-tests? Is this what you expected based on the EDA you performed?
   The two variables with individual t-test p-values very near zero are Square Feet & Price per Sq Ft. I definitely expected Square Feet to be a strong predictor based on EDA, but didn’t expect Price per Sq Ft to be quite as strong.

5. Consider adding the categorical variable (and remove price/sq. foot, as well as any other variables you believe should be removed based on the previous questions). How many coefficients will be added to your model to incorporate Realtor?
   By including the realtor categorical variable we add a single attribute to the data set. The base level is realtor A, so if the observation is realtor A then the realtorA_IND is 1, otherwise it is 0.

# Bring numerical data in to R
seahomes2 <- read.csv("realestatedata-2.csv", header = T, sep = ",")
head(seahomes2)

Next we create a new model with the revised dataset:

#Linear regression model 
seamod2 <- lm(Price ~ .,data = seahomes2)
summary(seamod2)

Call:
lm(formula = Price ~ ., data = seahomes2)

Residuals:
     Min       1Q   Median       3Q      Max 
-168.303  -54.720   -8.842   59.767  158.621 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)   386.76580   65.34965   5.918 1.55e-06 ***
Square.Feet     0.16186    0.03178   5.093 1.65e-05 ***
RealtorA_IND -196.84035   36.98065  -5.323 8.51e-06 ***
Bedrooms      -26.09857   28.09336  -0.929    0.360    
Bathrooms     -27.28321   28.63452  -0.953    0.348    
---
NA

Residual standard error: 84.03 on 31 degrees of freedom
Multiple R-squared:  0.7152,    Adjusted R-squared:  0.6784 
F-statistic: 19.46 on 4 and 31 DF,  p-value: 4.251e-08

par(mfrow=c(2,2))
plot(seamod2)

6. Interpret the coefficient for realtor for the model you built in part 5.
   If all other items remain constant, when the realtor is realtor A the price of the home will be ~$197k less than if it were sold by realtor B.
7. Interpret the coefficient for square footage for the model you built in part 5.
   For each thousand dollar increase in price, the square footage will increase by ~0.16.
8. Interpret the R-squared for this model.
   An R-squared of 0.715 means that 71.5% of the change in price can be explained by the x variables included in the model while the other 28.5% is due to unexplained error.
9. Predict the price for a home sold by Realtor A, with 3 bedrooms and 2 bathrooms, and 1500 square feet. Do you think the prediction is close to what you would expect in reality? Explain.
   Based on our equation:
   Price($000’s) = 386.77 + (0.16 x [Square Feet]) + (-196.84 x [Realtor A Ind]) + (-26.10 x [Bedrooms]) + (-27.28 x [Bathrooms])
   Price = 386.77 + (0.16 x 1,500) + (-196.84 x 1) + (-26.10 x 3) + (-27.28 x 2)
   Price = 386.77 + 240 - 196.84 - 78.30 - 54.56
   Price = $297k
   Yes, I think this estimate is reasonable based on reality. However, I don’t think a model that sets a high intercept and then reduces the prediction based on the number of bedrooms an bathrooms will work with a different set of data points.
10. Based on the plots in questions 1 and 2 of the EDA portion, do you think we should consider adding additional terms to this model? What is the benefit of not adding additional terms to your model?
    Yes, I think we should consider adding additional terms to the model. The price per square foot relationship is not entirely linear. Zip code, or population density might provide more information about why prices vary regardless of square feet. However, adding more terms makes the model more difficult to interpret. We want to find the fewest terms possible that still provide an excellent prediction.

Part Two:

Box Office Gross and Pay per View Sales

Consider a problem of predicting subsequent sales for movies based on Box Office gross sales and other features of the movie. The data file Movies.xls contains information for over 200 movies released during 1998 and 2001, regarding the following variables:

• Title: Movie name
• Sales: Subsequent Sales (in millions of dollars)
• BoxOffice: Box office gross sales (in millions of dollars)
• Rating: Motion Picture Association of America rating of a movie
• Genre: Genre of the movie

Modeling Box Office Data

1. (a) Fit a model with only one explanatory variable, Rating. Report the fitted model here.

# Bring numerical data in to R
movies <- read.csv("movies.csv", header = T, sep = ",")
head(movies)

#Linear regression model 
movielm <- lm(Sales ~ Rating,data = movies)
summary(movielm)

Call:
lm(formula = Sales ~ Rating, data = movies)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.3482 -0.8533 -0.2884  0.6342  4.5717 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)   
(Intercept)   0.6186     0.2406   2.571  0.01080 * 
RatingPG      0.5298     0.2935   1.805  0.07245 . 
RatingR       0.7997     0.2559   3.125  0.00202 **
---
NA

Residual standard error: 1.103 on 221 degrees of freedom
Multiple R-squared:  0.04602,   Adjusted R-squared:  0.03739 
F-statistic:  5.33 on 2 and 221 DF,  p-value: 0.005484

par(mfrow=c(2,2))
plot(movielm)

2. (b) Write down the fitted model separately for each of the three categories of movie ratings.
   G: [Subsequent Sales] = 0.61 ($MM)
   PG: [Subsequent Sales] = 0.61 ($MM) + 0.53 ($MM)
   R: [Subsequent Sales] = 0.61 ($MM) + 0.80 ($MM)
   
3. (c) Using only the estimated parameter values, calculate the overall predicted mean of the subsequent sales for all movies, regardless of their rating.
   \[\frac{(0.61) + (0.61 + 0.53) + (0.61 + 0.80)}{3} = \$1.053MM\]
4. (d) Interpret the intercept reported by R within the context of the data.
   When a movie is NOT rated R and not rated PG it will generate $0.62MM in subsequent sales.
5. (e) What is the predicted difference in subsequent pay per view sales for movies rated G and movies rated R?
   R rated movies generate $0.80MM more in subsequent revenue than G rated movies (which is the base value for the categorical variable).
6. (f) Is this model overall useful?
   No. The R-squared value suggests that 95.5% of the change is Y is NOT explained by rating. However, the f-statistic’s p-value suggests that at least one of the rating dummy variables is useful in predicting subsequent sales. The conclusion is that this model is NOT very useful.
7. (g) Verify whether the assumptions needed to conduct correct inferences with this model are verified. Print out and turn in any graphs you have used to draw your conclusions.
   Above we have the model summary plots. There we see the Normal Q-Q plot shows there may be an issue with the distribution of the data.
   Linear relationships:

pairs(movies)

We can check for auto-correlation using a Durbin-Watson test. Here it shows a score below two which suggests some positive auto-correlation may exist:

durbinWatsonTest(movielm)

 lag Autocorrelation D-W Statistic p-value
   1    -0.000130451      1.992605   0.924
 Alternative hypothesis: rho != 0

8. (h) Explain how many dummy variables are needed to add the variable Genre as an explanatory variable, and how to construct them.
   Our dataset contains 9 distinct genre categories:

levels(movies$Genre)

[1] "Action/Adventure" "Animation"        "Comedy"           "Drama"            "Family"           "Horror/Thriller"  "Mystery"          "Romance"         
[9] "Sci-Fi/Fantasy"  

Therefore, we would need to establish a base level, and create 8 dummy variables for each of the non-base levels.
9. (i) How many error degrees of freedom would you have for a model that includes Rating and Genre as explanatory variables?
With 2 dummy variables for Rating, and 8 dummy variables for Genre we have 10 total x variables. For DF of (n -k -1), where n = 224, & k=10, our DF = 213.