Author: Barnabás Piller

1. Import the dataset Apartments.xlsx

apartments <- read_xlsx("C:/WU/BBE/Sixth Semester/Applied Data Analysis with R/Homework 3/Apartments.xlsx")

Description:

2. Change categorical variables into factors. (0.5 p)

apartments$Parking <- factor(apartments$Parking, levels = c(0,1), labels = c("No", "Yes"))
apartments$Balcony <- factor(apartments$Balcony, levels = c(0,1), labels = c("No", "Yes"))

3. Test the hypothesis H0: Mu_Price = 1900 eur. What can you conclude? (1 p)

t.test(apartments$Price, mu = 1900, alternative = "two.sided")
## 
##  One Sample t-test
## 
## data:  apartments$Price
## t = 2.9022, df = 84, p-value = 0.004731
## alternative hypothesis: true mean is not equal to 1900
## 95 percent confidence interval:
##  1937.443 2100.440
## sample estimates:
## mean of x 
##  2018.941

We reject the null hypothesis (p-value < 0.01) and conclude that the average price differs from €1900.

4. Estimate the simple regression function: Price = f(Age). Save results in object fit1 and explain the estimate of regression coefficient, coefficient of correlation and coefficient of determination. (2 p)

fit1 <- lm(Price ~ Age, data = apartments)
summary(fit1)
## 
## Call:
## lm(formula = Price ~ Age, data = apartments)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -623.9 -278.0  -69.8  243.5  776.1 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 2185.455     87.043  25.108   <2e-16 ***
## Age           -8.975      4.164  -2.156    0.034 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 369.9 on 83 degrees of freedom
## Multiple R-squared:  0.05302,    Adjusted R-squared:  0.04161 
## F-statistic: 4.647 on 1 and 83 DF,  p-value: 0.03401

For every year an apartment is older, the price/m2 decreases by €8.975. If an apartment is brand new, the average price/m2 is €2185.5.

cor(apartments$Price, apartments$Age)
## [1] -0.230255

The correlation coefficient is negative and rather small, meaning there is a weak negative relationship between price and age.

The R-squared tells us that 5.3% of the variability in the data can be explained by the model.

5. Show the scateerplot matrix between Price, Age and Distance. Based on the matrix determine if there is potential problem with multicolinearity. (0.5 p)

scatterplotMatrix(apartments[,1:3], smooth = F)

We see that there is no linear correlation between the independent variables Age and Distance, and so there is no issue with multicolinearity.

6. Estimate the multiple regression function: Price = f(Age, Distance). Save it in object named fit2.

fit2 <- lm(Price ~ Age + Distance, data = apartments)

7. Chech the multicolinearity with VIF statistics. Explain the findings. (1 p)

vif(fit2)
##      Age Distance 
## 1.001845 1.001845

Since the VIF = 1, the independent variables are uncorrelated, and multicolinearity does not exist.

8. Calculate standardized residuals and Cooks Distances for model fit2. Remove any potentially problematic units (outliers or units with high influence). (2 p)

apartments$StdRes <- round(rstandard(fit2), 3)
apartments$CooksD <- round(cooks.distance(fit2), 3)
head(apartments[order(-apartments$CooksD),], 6)
## # A tibble: 6 × 7
##     Age Distance Price Parking Balcony StdRes CooksD
##   <dbl>    <dbl> <dbl> <fct>   <fct>    <dbl>  <dbl>
## 1     5       45  2180 Yes     Yes       2.58  0.32 
## 2    43       37  1740 No      No        1.44  0.104
## 3     2       11  2790 Yes     No        2.05  0.069
## 4     7        2  1760 No      Yes      -2.15  0.066
## 5    37        3  2540 Yes     Yes       1.58  0.061
## 6    40        2  2400 No      Yes       1.09  0.038

I will remove the problematic observations in Task 11.

9. Check for potential heteroskedasticity with scatterplot between standarized residuals and standrdized fitted values. Explain the findings. (1 p)

apartments$StdFit <- scale(fit2$fitted.values)

scatterplot(y = apartments$StdRes, x = apartments$StdFit,
            xlab = "Standardized fitted values",
            ylab = "Standardized residuals",
            boxplots = F, regLine = F, smooth = F)

Homoscedasticity does not seem to be violated, meaning the variance of the residual is mostly constant.

10. Are standardized residuals ditributed normally? Show the graph and formally test it. Explain the findings. (1 p)

hist(apartments$StdRes, breaks = 20, xlab = "Standardized residuals",
     ylab = "Frequency", main = "Histogram of standardized residuals")

shapiro.test(apartments$StdRes)
## 
##  Shapiro-Wilk normality test
## 
## data:  apartments$StdRes
## W = 0.95303, p-value = 0.003645

Since p < 0.01, we reject the null hypothesis and determine that the standardized residuals are not normally distributed. This violates an assumption of the linear model.

11. Estimate the fit2 again without potentially excluded units and show the summary of the model. Explain all coefficients. (2 p)

apartments <- subset(apartments, CooksD < 0.1)
fit2 <- lm(Price ~ Age + Distance, data = apartments)
summary(fit2)
## 
## Call:
## lm(formula = Price ~ Age + Distance, data = apartments)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -627.27 -212.96  -46.23  205.05  578.98 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 2490.112     76.189  32.684  < 2e-16 ***
## Age           -7.850      3.244  -2.420   0.0178 *  
## Distance     -23.945      2.826  -8.473 9.53e-13 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 273.5 on 80 degrees of freedom
## Multiple R-squared:  0.4968, Adjusted R-squared:  0.4842 
## F-statistic: 39.49 on 2 and 80 DF,  p-value: 1.173e-12

The price/m2 gets smaller by €7.85 for every year an apartment is older, and it gets smaller by €23.95 for every additional km away from the center. The average price/m2 of a brand new apartment located in the city center is €2490.1. The R-squared tells us that 49.7% of the variability in the data can be explained by the model.

12. Estimate the linear regression function Price = f(Age, Distance, Parking and Balcony). Be careful to correctly include categorical variables. Save the object named fit3.

fit3 <- lm(Price ~ Age + Distance + Parking + Balcony, data = apartments)

13. With function anova check if model fit3 fits data better than model fit2. (1 p)

anova(fit2, fit3)
## Analysis of Variance Table
## 
## Model 1: Price ~ Age + Distance
## Model 2: Price ~ Age + Distance + Parking + Balcony
##   Res.Df     RSS Df Sum of Sq      F  Pr(>F)  
## 1     80 5982100                              
## 2     78 5458696  2    523404 3.7395 0.02813 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Since the p-value < 0.05, we reject the null hypothesis and conclude that model fit3 fits the data better than model fit2.

14. Show the results of fit3 and explain regression coefficient for both categorical variables. Can you write down the hypothesis which is being tested with F-statistics, shown at the bottom of the output? (2 p)

summary(fit3)
## 
## Call:
## lm(formula = Price ~ Age + Distance + Parking + Balcony, data = apartments)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -499.06 -194.33  -32.04  219.03  544.31 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 2358.900     93.664  25.185  < 2e-16 ***
## Age           -7.197      3.148  -2.286  0.02499 *  
## Distance     -21.241      2.911  -7.296 2.14e-10 ***
## ParkingYes   168.921     62.166   2.717  0.00811 ** 
## BalconyYes    -6.985     58.745  -0.119  0.90566    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 264.5 on 78 degrees of freedom
## Multiple R-squared:  0.5408, Adjusted R-squared:  0.5173 
## F-statistic: 22.97 on 4 and 78 DF,  p-value: 1.449e-12

The coefficients tell us that the price/m2 of an apartment that has a parking space is on average €168.9 higher than the price/m2 of an apartment without a parking space. If an apartment has a balcony, the price/m2 is €7 lower on average than without a balcony.

Regarding the F-statistic, the null hypothesis is that the coefficients of the model are zero, and the alternative hypothesis is that the coefficients are non-zero. In other words, we are testing the significance of our model compared to the model which contains only the intercept.

15. Save fitted values and calculate the residual for apartment ID2. (1 p)

apartments$Fitted <- fitted.values(fit3)
apartments$Price[2] - apartments$Fitted[2]
##        2 
## 422.9572

The residual for the second apartment in the list is €423, meaning the price predicted by the model is lower than the actual price. We could have also gotten the residual in the following way:

fit3$residuals[2]
##        2 
## 422.9572