library(tidyverse)
library (ggfortify)
library(plotly)
library(scales)
setwd("/Users/jeffreyren/Downloads/Directory")
dataset1 <- read_csv("realestate.csv")Final Project
## Source: AndyAndersonCartoons.com
A. Introduction
The topic for this project will be about the Real Estate Market. It will explore a data set which is about all the real estate sales in Connecticut from 2001 to 2021. The data set includes the serial numbers, Year listed, date recorded, town, address, assessed value, sale value, sale ratio, property type, residential type, and locations of the real estate sales in Connecticut. I decided to clean the data set for the three most populous towns in Connecticut because they would have the widest variety of houses. This information was gathered from the Office of Management and Policy where they maintain a list of all real estate sales with a sales price of $2,000 or greater. I chose this topic because I will 100% invest in real estate in the future so when I saw this data set I knew I had to dive deeper into it.
Load in the necessary libraries and read the data.
Cleaned the data set to remove NA’s, filter for the three most populous towns in Connecticut, and filtered out for the zero values in “Assessed Values”
nona <- dataset1 |>
filter(!is.na(`Residential Type`)) |>
filter(Town %in% c("Bridgeport", "New Haven", "Stamford")) |>
filter(`Assessed Value` > 0)Created a multi-linear regression and used factor to get correct variable types. This is not a good model because it violates all assumptions and the r^2 values are low. The P-value is low though so it shows significance.
datalm <- lm (`Assessed Value` ~ factor(`List Year`) + factor(Town) + factor(`Residential Type`), data = nona)
summary(datalm)
Call:
lm(formula = `Assessed Value` ~ factor(`List Year`) + factor(Town) +
factor(`Residential Type`), data = nona)
Residuals:
Min 1Q Median 3Q Max
-388375 -84645 -29059 29690 24469034
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 104497 6911 15.121 < 2e-16
factor(`List Year`)2007 -61197 9025 -6.781 1.21e-11
factor(`List Year`)2008 -13694 9818 -1.395 0.16308
factor(`List Year`)2009 2592 8410 0.308 0.75793
factor(`List Year`)2010 -6804 8694 -0.783 0.43389
factor(`List Year`)2011 -25949 8728 -2.973 0.00295
factor(`List Year`)2012 -25143 9456 -2.659 0.00784
factor(`List Year`)2013 -39336 8382 -4.693 2.70e-06
factor(`List Year`)2014 -71705 8343 -8.594 < 2e-16
factor(`List Year`)2015 -84810 9324 -9.096 < 2e-16
factor(`List Year`)2016 -99830 8419 -11.858 < 2e-16
factor(`List Year`)2017 -72449 8908 -8.133 4.27e-16
factor(`List Year`)2018 -71124 8150 -8.727 < 2e-16
factor(`List Year`)2019 -69828 8199 -8.517 < 2e-16
factor(`List Year`)2020 -54194 8696 -6.232 4.64e-10
factor(`List Year`)2021 -56445 7928 -7.120 1.09e-12
factor(Town)New Haven 9593 3640 2.636 0.00840
factor(Town)Stamford 241007 3508 68.699 < 2e-16
factor(`Residential Type`)Four Family 128502 11820 10.871 < 2e-16
factor(`Residential Type`)Single Family 131960 3113 42.391 < 2e-16
factor(`Residential Type`)Three Family 97561 5286 18.456 < 2e-16
factor(`Residential Type`)Two Family 72914 4446 16.398 < 2e-16
(Intercept) ***
factor(`List Year`)2007 ***
factor(`List Year`)2008
factor(`List Year`)2009
factor(`List Year`)2010
factor(`List Year`)2011 **
factor(`List Year`)2012 **
factor(`List Year`)2013 ***
factor(`List Year`)2014 ***
factor(`List Year`)2015 ***
factor(`List Year`)2016 ***
factor(`List Year`)2017 ***
factor(`List Year`)2018 ***
factor(`List Year`)2019 ***
factor(`List Year`)2020 ***
factor(`List Year`)2021 ***
factor(Town)New Haven **
factor(Town)Stamford ***
factor(`Residential Type`)Four Family ***
factor(`Residential Type`)Single Family ***
factor(`Residential Type`)Three Family ***
factor(`Residential Type`)Two Family ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 3e+05 on 52191 degrees of freedom
Multiple R-squared: 0.1408, Adjusted R-squared: 0.1404
F-statistic: 407.1 on 21 and 52191 DF, p-value: < 2.2e-16autoplot(datalm, 1:4, nrow=2, ncol=2)
Created a histogram using ggplot and geom bar to show the number of sales each year. Used position = stack to show the proportions for each residential type each year. Used the set2 colors and dark theme.
plot1 <- ggplot(nona, aes(x = `List Year`, fill = `Residential Type`)) +
geom_bar(position = "stack", alpha = 0.5, color = "Black") +
ggtitle("Number of Sales for Each Residential Type in the \n 3 most Populous Towns of Connecticut (2006-2021)") +
labs(x = "Year", y = "Count", caption = "Source: The Office of Management and Policy") +
scale_fill_brewer(palette = "Set2", name = "Residential Type") +
theme_dark()
int <- ggplotly(plot1)
intThis visualization shows the number of categorized sales each year from the year 2006 to 2021. For each year it shows the total sales, and because of the stack command it also breaks down how much of those sales were for each residential type. The bars show that the proportions of the Residential types remained similar throughout the years. The bars also show that the total sales did have a spike in 2021.
Cleaned the data set specifically for the second visualization by filtering for only the year 2021 (Latest)
data2 <- nona |>
filter(`List Year` == 2021)Used new 2021 dataset to create a boxplot using ggplot and geom box plot. Used scaled_y_continous labels = dollar to get rid of the e notation in the numbers (they were too big). Used set3 and theme dark for aesthetics.
plot2 <- ggplot(data2, aes(x = Town, y = `Assessed Value`, fill = Town)) +
geom_boxplot() +
ggtitle("Boxplot of the Assessed Real Estate Value in the \n Three Most Populous Towns of Connecticut (2021)") +
labs(x = "Town", y = "Assessed Value (in $)", caption = "Source: The Office of Management and Policy") +
scale_y_continuous(labels = dollar) +
scale_fill_brewer(palette = "Set3", name = "Town") +
theme_dark()
int2 <- ggplotly(plot2)
int2The box plot shows the different values of the assessed values of real estate from these three towns. It shows the min-max, upper-lower fence, q1-q3, median, and the outliers of assessed real estate value in each of these three towns. The outliers kind of skew the plot but I didn’t want to remove them because they’re important. It shows that Stamford has a very large number of the range and total number of outliers.
Filtered NA and 0 values for Assessed values for the third visualization.
data3 <- dataset1 |>
filter(!is.na(`Residential Type`)) |>
filter(`Assessed Value` > 0)Found the averages for each residential type and year using group by, summarize, and mean.
averages <- data3 |>
group_by(`List Year`, `Residential Type`) |>
summarize(Average = mean(`Assessed Value`, na.rm = TRUE), .groups = "drop")Used gg plot, geom line, and geom point to create a line graph for the averages of each residential type through the years. Used set4 and dark theme for the aesthetics.
plot3 <- ggplot(averages, aes(x = `List Year`, y = `Average`, color = `Residential Type`)) +
geom_line() +
geom_point() +
labs(title = "Average Assessed Value of Residential Types in the \n Three most Populous Towns in Connecticut (2006-2021)",
x = "Year",
y = "Assessed Value (in $)",
caption = "Office of Management and Policy") +
scale_color_brewer(palette = "Set1") +
theme_dark()
int3 <- ggplotly(plot3)
int3This plot shows how the averages of the assessed values for each residential type change over time. It also shows the relationship between each type. For some reason the single family homes have a higher average value than multi-family homes. I think the reason is that there are so many single family homes being sold and that maybe the expensive multi-family homes weren’t worth the investment so they weren’t being sold.
Not one of the three visualizations but I just made this because I thought the previous plot didn’t make sense because the single family homes had a higher average cost than the four family homes but I guess it is true based on this graph.
scatterplot <- ggplot(data2, aes(x = `Date Recorded`, y = `Assessed Value`, color = `Residential Type`)) +
geom_point() +
ggtitle("Scatter Plot of Assessed Value by List Year") +
labs(x = "Date", y = "Assessed Value", color = "Residential Type") +
scale_color_brewer(palette = "Set1")
scatterplot
B. Second Question
There was quite a large number of towns in the data set so I had to hand pick a few towns to narrow down the list so I figured this could be where the background research comes in. I decided that I was going to research the most populous towns of Connecticut because they’ll have the largest variety as well as the largest total number of sales. According to Connecticut Demographics by Cubit, Bridgeport, New Haven, and Stamford were the three most populous with 148,470, 135,736, and 135,413 respectively. Although Stamford was the lowest of the three we can see according to the box plot that the assessed real estate values are higher than the other two. This tells me that Stamford is a town with high-end houses when compared to the other two. https://www.connecticut-demographics.com/cities_by_population
C. Conclusion
The first visualization is a histogram that shows the number of categorized sales each year from the year 2006 to 2021, as well as how much each residential type makes up that total. The second visualization is a box plot shows the different values of the assessed values of real estate from these three towns.It shows the min-max, upper-lower fence, q1-q3, median, and the outliers of assessed real estate value in each of these three towns. The last visualization is a line graph which shows how the averages of the assessed values for each residential type change over time. It also shows the relationship between each type. It’s kind of embarrassing because the data set and the results did not go how I thought it would go so the whole purpose and cohesiveness of this project did not go the way I thought it would. The original point I wanted to make with a real estate data set was how hard it is now to buy a house. Housing prices have been ridiculously high and it’s not because of inflation. I wanted to include background research where I saw how in every decade the average salary could qualify you for a loan to buy a house except in this one but because the results don’t show that or anything at all honestly, so I wished that could have been better. There also wasn’t anything too surprising as well. I don’t even know what I’ll present but one thing I can take away from this is that with data you can’t go into it with an expectation of how it’s going to go because a lot of the time your work could go in completely different directions that could bear no fruit.