HW 3
NAME SURNAME
1. Import the dataset Apartments.xlsx
library(readxl)
mydata <- read_xlsx("./Apartments.xlsx")
head(mydata)## # A tibble: 6 × 5
## Age Distance Price Parking Balcony
## <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 7 28 1640 0 1
## 2 18 1 2800 1 0
## 3 7 28 1660 0 0
## 4 28 29 1850 0 1
## 5 18 18 1640 1 1
## 6 28 12 1770 0 1Description:
- Age: Age of an apartment in years
- Distance: The distance from city center in km
- Price: Price per m2
- Parking: 0-No, 1-Yes
- Balcony: 0-No, 1-Yes
2. Change categorical variables into factors. (0.5 p)
mydata$ParkingF <- factor(mydata$Parking,
levels = c(1, 0),
labels = c("Yes", "No"))
mydata$BalconyF <- factor(mydata$Balcony,
levels = c(0, 1),
labels = c("No", "Yes"))
contrasts(mydata$ParkingF)## No
## Yes 0
## No 1contrasts(mydata$BalconyF)## Yes
## No 0
## Yes 13. Test the hypothesis H0: Mu_Price = 1900 eur. What can you conclude? (1 p)
t.test(mydata$Price,
mu = 1900,
alternative = "two.sided")##
## One Sample t-test
##
## data: mydata$Price
## t = 2.9022, df = 84, p-value = 0.004731
## alternative hypothesis: true mean is not equal to 1900
## 95 percent confidence interval:
## 1937.443 2100.440
## sample estimates:
## mean of x
## 2018.941Explanation: Since p-value < 0.05 we reject the H0, and we can say that that average price is not 1900 eur.
4. Estimate the simple regression function: Price = f(Age). Save results in object fit1 and explain the estimate of regression coefficient, coefficient of correlation and coefficient of determination. (2 p)
fit1 <- lm(Price ~ Age,
data = mydata)
summary(fit1)##
## Call:
## lm(formula = Price ~ Age, data = mydata)
##
## Residuals:
## Min 1Q Median 3Q Max
## -623.9 -278.0 -69.8 243.5 776.1
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2185.455 87.043 25.108 <2e-16 ***
## Age -8.975 4.164 -2.156 0.034 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 369.9 on 83 degrees of freedom
## Multiple R-squared: 0.05302, Adjusted R-squared: 0.04161
## F-statistic: 4.647 on 1 and 83 DF, p-value: 0.034015. Show the scateerplot matrix between Price, Age and Distance. Based on the matrix determine if there is potential problem with multicolinearity. (0.5 p)
library(car)## Loading required package: carDatascatterplotMatrix(mydata[ , -c(4:7) ],
smooth=FALSE )
6. Estimate the multiple regression function: Price = f(Age, Distance). Save it in object named fit2.
fit2 <- lm(Price ~ Age + Distance,
data = mydata)
summary(fit2)##
## Call:
## lm(formula = Price ~ Age + Distance, data = mydata)
##
## Residuals:
## Min 1Q Median 3Q Max
## -603.23 -219.94 -85.68 211.31 689.58
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2460.101 76.632 32.10 < 2e-16 ***
## Age -7.934 3.225 -2.46 0.016 *
## Distance -20.667 2.748 -7.52 6.18e-11 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 286.3 on 82 degrees of freedom
## Multiple R-squared: 0.4396, Adjusted R-squared: 0.4259
## F-statistic: 32.16 on 2 and 82 DF, p-value: 4.896e-117. Chech the multicolinearity with VIF statistics. Explain the findings. (1 p)
vif(fit2)## Age Distance
## 1.001845 1.001845Explanation: Since both are close to 1 we dont have to remove any variable.
8. Calculate standardized residuals and Cooks Distances for model fit2. Remove any potentially problematic units (outliers or units with high influence). (2 p)
mydata$StdResid <- round(rstandard(fit2), 3) #Standardized residuals
mydata$CooksD <- round(cooks.distance(fit2), 3) #Cooks distances
hist(mydata$StdResid,
xlab = "Standardized residuals",
ylab = "Frequency",
main = "Histogram of standardized residuals")
hist(mydata$CooksD,
xlab = "Cooks distance",
ylab = "Frequency",
main = "Histogram of Cooks distances")
head(mydata[order(mydata$StdResid),], 3) #Three units with lowest value of stand. residuals## # A tibble: 3 × 9
## Age Distance Price Parking Balcony ParkingF BalconyF StdResid CooksD
## <dbl> <dbl> <dbl> <dbl> <dbl> <fct> <fct> <dbl> <dbl>
## 1 7 2 1760 0 1 No Yes -2.15 0.066
## 2 12 14 1650 0 1 No Yes -1.50 0.013
## 3 12 14 1650 0 0 No No -1.50 0.013head(mydata[order(-mydata$CooksD),], 6) #Six units with highest value of Cooks distance## # A tibble: 6 × 9
## Age Distance Price Parking Balcony ParkingF BalconyF StdResid CooksD
## <dbl> <dbl> <dbl> <dbl> <dbl> <fct> <fct> <dbl> <dbl>
## 1 5 45 2180 1 1 Yes Yes 2.58 0.32
## 2 43 37 1740 0 0 No No 1.44 0.104
## 3 2 11 2790 1 0 Yes No 2.05 0.069
## 4 7 2 1760 0 1 No Yes -2.15 0.066
## 5 37 3 2540 1 1 Yes Yes 1.58 0.061
## 6 40 2 2400 0 1 No Yes 1.09 0.038library(dplyr)##
## Attaching package: 'dplyr'## The following object is masked from 'package:car':
##
## recode## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionmydata <- mydata %>%
filter(!(CooksD == 0.066 | CooksD == 0.320 | CooksD == 0.104))
fit2 <- lm(Price ~ Age + Distance,
data = mydata)
mydata$StdResid <- round(rstandard(fit2), 3) #Standardized residuals
mydata$CooksD <- round(cooks.distance(fit2), 3) #Cooks distances
hist(mydata$StdResid,
xlab = "Standardized residuals",
ylab = "Frequency",
main = "Histogram of standardized residuals")
hist(mydata$CooksD,
xlab = "Cooks distance",
ylab = "Frequency",
main = "Histogram of Cooks distances")
Explanation: i have removed 3 rows to eliminate the gaps on Histogram of Standardized Residuals and Cookcs distances
9. Check for potential heteroskedasticity with scatterplot between standarized residuals and standrdized fitted values. Explain the findings. (1 p)
mydata$StdFitted <- scale(fit2$fitted.values)
library(car)
scatterplot(y = mydata$StdResid, x = mydata$StdFitted,
ylab = "Standardized residuals",
xlab = "Standardized fitted values",
boxplots = FALSE,
regLine = FALSE,
smooth = FALSE)
Explanation: On the graph according to dots we can see that it is
Heteroskedasticity, bcs its doesnt have parallel lines.
10. Are standardized residuals ditributed normally? Show the graph and formally test it. Explain the findings. (1 p)
mydata$StdResid <- round(rstandard(fit2), 3)
mydata$CooksD <- round(cooks.distance(fit2), 3)
hist(mydata$StdResid,
xlab = "Standardized residuals",
ylab = "Frequency",
main = "Histogram of standardized residuals")
shapiro.test(mydata$StdResid)##
## Shapiro-Wilk normality test
##
## data: mydata$StdResid
## W = 0.94355, p-value = 0.001279Explanation: The Shapiro-Wilk test p-value of 0.001279 is less than 0.05, indicating that the null hypothesis of normal distribution is rejected.
11. Estimate the fit2 again without potentially excluded units and show the summary of the model. Explain all coefficients. (2 p)
summary(fit2)##
## Call:
## lm(formula = Price ~ Age + Distance, data = mydata)
##
## Residuals:
## Min 1Q Median 3Q Max
## -424.84 -215.45 -44.84 213.26 552.21
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2527.549 75.591 33.437 < 2e-16 ***
## Age -8.876 3.179 -2.792 0.00657 **
## Distance -24.728 2.763 -8.949 1.23e-13 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 265.5 on 79 degrees of freedom
## Multiple R-squared: 0.5287, Adjusted R-squared: 0.5168
## F-statistic: 44.31 on 2 and 79 DF, p-value: 1.245e-13Explanation: In every added KM to distance from city center, the price in average will decrease 24.73 eur.(p-value<0.001) Since the p-value > 0.001 is for Age we dont have to explain it.
12. Estimate the linear regression function Price = f(Age, Distance, Parking and Balcony). Be careful to correctly include categorical variables. Save the object named fit3.
fit3 <- lm(Price ~ Age + Distance + BalconyF,
data = mydata)Explanation: In every added KM to distance from city center, the price in average will decrease 24.73 eur. (p-value < 0.001) And for age and Balcony Factor we dont have to say anything bcs the p-value is > 0.001 and it is not significant.
13. With function anova check if model fit3 fits data better than model fit2. (1 p)
anova(fit2,fit3)## Analysis of Variance Table
##
## Model 1: Price ~ Age + Distance
## Model 2: Price ~ Age + Distance + BalconyF
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 79 5570638
## 2 78 5570636 1 2.659 0 0.9951Explanation: Model 1 fits the data as well as Model 2. Adding ‘BalconyF’ does not provide a better fit according to the ANOVA results.
14. Show the results of fit3 and explain regression coefficient for both categorical variables. Can you write down the hypothesis which is being tested with F-statistics, shown at the bottom of the output? (2 p)
summary(fit3)##
## Call:
## lm(formula = Price ~ Age + Distance + BalconyF, data = mydata)
##
## Residuals:
## Min 1Q Median 3Q Max
## -425.0 -215.6 -45.0 213.5 552.1
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2527.7017 80.1118 31.552 < 2e-16 ***
## Age -8.8764 3.1995 -2.774 0.00692 **
## Distance -24.7280 2.7810 -8.892 1.75e-13 ***
## BalconyFYes -0.3641 59.6675 -0.006 0.99515
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 267.2 on 78 degrees of freedom
## Multiple R-squared: 0.5287, Adjusted R-squared: 0.5106
## F-statistic: 29.17 on 3 and 78 DF, p-value: 9.488e-13