HW 3
Paul FRIEDRICH
1. Import the dataset Apartments.xlsx
## # A tibble: 85 × 5
## Age Distance Price Parking Balcony
## <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 7 28 1640 0 1
## 2 18 1 2800 1 0
## 3 7 28 1660 0 0
## 4 28 29 1850 0 1
## 5 18 18 1640 1 1
## 6 28 12 1770 0 1
## 7 14 20 1850 0 1
## 8 18 6 1970 1 1
## 9 22 7 2270 1 0
## 10 25 2 2570 1 0
## # ℹ 75 more rowsDescription:
- Age: Age of an apartment in years
- Distance: The distance from city centre in km
- Price: Price per m2
- Parking: 0-No, 1-Yes
- Balcony: 0-No, 1-Yes
2. Change categorical variables into factors. (0.5 p)
## # A tibble: 85 × 5
## Age Distance Price Parking Balcony
## <dbl> <dbl> <dbl> <fct> <fct>
## 1 7 28 1640 0 1
## 2 18 1 2800 1 0
## 3 7 28 1660 0 0
## 4 28 29 1850 0 1
## 5 18 18 1640 1 1
## 6 28 12 1770 0 1
## 7 14 20 1850 0 1
## 8 18 6 1970 1 1
## 9 22 7 2270 1 0
## 10 25 2 2570 1 0
## # ℹ 75 more rows3. Test the hypothesis H0: Mu_Price = 1900 eur. What can you conclude? (1 p)
##
## One Sample t-test
##
## data: apartments_data$Price
## t = 2.9022, df = 84, p-value = 0.004731
## alternative hypothesis: true mean is not equal to 1900
## 95 percent confidence interval:
## 1937.443 2100.440
## sample estimates:
## mean of x
## 2018.9414. Estimate the simple regression function: Price = f(Age). Save results in object fit1 and explain the estimate of regression coefficient, coefficient of correlation and coefficient of determination. (2 p)
##
## Call:
## lm(formula = Price ~ Age, data = apartments_data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -623.9 -278.0 -69.8 243.5 776.1
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2185.455 87.043 25.108 <2e-16 ***
## Age -8.975 4.164 -2.156 0.034 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 369.9 on 83 degrees of freedom
## Multiple R-squared: 0.05302, Adjusted R-squared: 0.04161
## F-statistic: 4.647 on 1 and 83 DF, p-value: 0.03401## The estimate for the regression coefficient tells us that the m^2 price for apartments in Vienna is expected to change by -8.975058 € per year with the age of the building.
## The coefficient of correlation of -0.230255 indicates a week, negative, linear relationship between age and price of the real estate.
## The coefficient of determination of 0.05301737 indicates a very week relationship between age and price, only about 5.3 % of the price difference is easily explained by the property's age.5. Show the scatterplot matrix between Price, Age and Distance. Based on the matrix determine if there is potential problem with multicollinearity. (0.5 p)
6. Estimate the multiple regression function: Price = f(Age, Distance). Save it in object named fit2.
##
## Call:
## lm(formula = Price ~ Age + Distance, data = apartments_data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -603.23 -219.94 -85.68 211.31 689.58
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2460.101 76.632 32.10 < 2e-16 ***
## Age -7.934 3.225 -2.46 0.016 *
## Distance -20.667 2.748 -7.52 6.18e-11 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 286.3 on 82 degrees of freedom
## Multiple R-squared: 0.4396, Adjusted R-squared: 0.4259
## F-statistic: 32.16 on 2 and 82 DF, p-value: 4.896e-117. Check the multicollinearity with VIF statistics. Explain the findings. (1 p)
## A variance inflation factor (VIF) of 1.001845, therefore close to 1 tells us that tehere is (almost) no multicollinearity.8. Calculate standardized residuals and Cooks Distances for model fit2. Remove any potentially problematic units (outliers or units with high influence). (2 p)
The data points with the highest Cook’s distance are:
## Standardized_Residuals Cooks_Distance
## 38 2.576772 0.31973058
## 55 1.444768 0.10420445
## 33 2.050586 0.06913379
## 53 -2.151787 0.06625775
## 22 1.575982 0.06086868
## 39 1.091176 0.03750987The data points with the highest absolute values of standardized residuals are:
## Standardized_Residuals Cooks_Distance
## 38 2.576772 0.31973058
## 53 -2.151787 0.06625775
## 33 2.050586 0.06913379
## 2 1.783288 0.03036543
## 61 1.783288 0.03036543
## 58 1.655343 0.03654413Data point 38 shows both the highest standardized residual and the
largest Cook’s distance. It shall be removed.
9. Check for potential heteroskedasticity with scatterplot between standardized residuals and standardized fitted values. Explain the findings. (1 p)
The plot shows some heteroskedasticity, there appears to be a slight trend of increasing variability in the standardized residuals as the standardized fitted values increase.
10. Are standardized residuals distributed normally? Show the graph and formally test it. Explain the findings. (1 p)
##
## Shapiro-Wilk normality test
##
## data: std_resid
## W = 0.9565, p-value = 0.00636## The p-value of 0.006359658 suggests that a normal distribution would produce an outcome such as the one observed in only one out of 157 cases.11. Estimate the fit2 again without potentially excluded units and show the summary of the model. Explain all coefficients. (2 p)
## 2460.101 represents the intercept. A hypothetical object with age 0 and distance 0 is expected to have a price of 2460.101. With increasing age the price is expected to change by -7.934267 per year. Every km in distance increase is expected to effect a price change of -20.6669.12. Estimate the linear regression function Price = f(Age, Distance, Parking and Balcony). Be careful to correctly include categorical variables. Save the object named fit3.
## # A tibble: 6 × 5
## Age Distance Price Parking Balcony
## <dbl> <dbl> <dbl> <fct> <fct>
## 1 7 28 1640 0 1
## 2 18 1 2800 1 0
## 3 7 28 1660 0 0
## 4 28 29 1850 0 1
## 5 18 18 1640 1 1
## 6 28 12 1770 0 1##
## Call:
## lm(formula = Price ~ Age + Distance + Parking + Balcony, data = apartments_data)
##
## Coefficients:
## (Intercept) Age Distance Parking1 Balcony1
## 2329.724 -5.821 -20.279 167.531 -15.20713. With function anova check if model fit3 fits data better than model fit2. (1 p)
## Analysis of Variance Table
##
## Model 1: Price ~ Age + Distance
## Model 2: Price ~ Age + Distance + Parking + Balcony
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 81 6176767
## 2 79 5654480 2 522287 3.6485 0.03051 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## The p-value of 0.03050997 tells us that fit3's two additional variables significantly improve the model.14. Show the results of fit3 and explain regression coefficient for both categorical variables. Can you write down the hypothesis which is being tested with F-statistics, shown at the bottom of the output? (2 p)
##
## Call:
## lm(formula = Price ~ Age + Distance + Parking + Balcony, data = apartments_data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -473.21 -192.37 -28.89 204.17 558.77
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2329.724 93.066 25.033 < 2e-16 ***
## Age -5.821 3.074 -1.894 0.06190 .
## Distance -20.279 2.886 -7.026 6.66e-10 ***
## Parking1 167.531 62.864 2.665 0.00933 **
## Balcony1 -15.207 59.201 -0.257 0.79795
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 267.5 on 79 degrees of freedom
## Multiple R-squared: 0.5275, Adjusted R-squared: 0.5035
## F-statistic: 22.04 on 4 and 79 DF, p-value: 3.018e-12## An object with a balcony is -15.21 € more expensive than one without. Whether or not there is a balcony makes hardly a difference.
## Parking on the other hand is very important, it raises the price of an object by 167.53 €.The hypothesis being tested with the F-statistic at the bottom of the output is the overall significance of the linear regression model. Specifically, the null hypothesis is that all of the regression coefficients in the model are equal to zero, meaning that none of the predictor variables have any effect on the response variable.
15. Save fitted values and calculate the residual for apartment ID2. (1 p)
## Residual for the second row of apartments_data: 427.8029 .