Study of Change in Housing Price Index Using Crime Data
Nazneen
3 January 2018
PROJECT NAME : Effect of crimes on Housing price Index NAME : Naznee Mansoori E mail : nazneenmansoori968@gmail.com COLLEGE : IIT Kanpur
1.Introduction
Everyone wants to live with their family and wants to feel safe in their home, but what if you bought a house in wrong place because you didn’t analyse the neighbourhood area or you were facing financial crisis. Often, places where crimes like robbery, rape, assaults or violence occur frequently people either leave their home and settle down elsewhere or they sell their houses in very low price because of hurry. This project addresses the issue of crime in American states and change in housing price index during the period of 1975 to 2015. It was the time when american economy was already well established and new business ideas were growing fast in places like california, San-francisco and LA. During this span of forty years america had faced all ups and downs like oil-shock, 9/11 and small country wars. We investigated housing price index as dependent variable on crimes to see the effect on development of society, as places where people live longer have much grown culture than the other.
2.Overview of project
Our project present a study of increase in price of houses in american state, on an average everywhere price have been increased, which can be observed in plot of mean housing price index of the states over the years.There is negative correlation between housing price Index and crimes.(Please see Appendix 1 and 2). We have analysised change in population over the years. Our study includes total 44 states of america,which includes Miami,Boston,Pittsbergh,Washington and many other famous city.
Hypothesis H1: Over the year Housing price Index have been increased so high, does this means crimes are falling in America.
3.Data
This dataset was actually made to check the correlations between a housing price index and its crime rate. Rise and fall of housing prices can be due to various factors with obvious reasons being the facilities of the house and its neighborhood. Think of a place like Detroit where there are hoodlums and you don’t want to end up buying a house in the wrong place. This data set will serve as historical data for crime rate data and this in turn can be used to predict whether the housing price will rise or fall. Rise in housing price will suggest decrease in crime rate over the years and vice versa.
4.Model
To see the correlation between crime data and housing price index, we proposed the following model, with housing price index as dependent variable Housing Price Index = ????0 +(Year) ????1 + (Population) ????2 + (Homicides)????3 + (Robberies)????4 + (Rapes)????5 + (Violent Crimes)????6 + (Assaults)????7 + (City..State)????8 + error (Numbers 0-8 in above equation denote betas’ coefficients)???(e)????8 #4.1 Reading Data
setwd("C:/Users/Taiyyab Ali/Desktop/R language")
Housingpriceindex <- read.csv("Housingpriceindex.csv")
View(Housingpriceindex)4.2 Tested model
Model1 = index_nsa ~ Year + Population + Homicides + Robberies + Rapes + Violent.Crimes + Assaults + City..State
fit <- lm(Model1, data = Housingpriceindex)
summary(fit)##
## Call:
## lm(formula = Model1, data = Housingpriceindex)
##
## Residuals:
## Min 1Q Median 3Q Max
## -84.289 -15.412 -2.201 10.405 182.836
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -9.360e+03 1.282e+02 -73.007 < 2e-16 ***
## Year 4.748e+00 6.417e-02 74.003 < 2e-16 ***
## Population 5.613e-06 1.576e-06 3.562 0.000378 ***
## Homicides 4.162e-03 1.218e-02 0.342 0.732642
## Robberies 7.298e-04 5.229e-04 1.396 0.163040
## Rapes -1.106e-02 3.995e-03 -2.769 0.005679 **
## Violent.Crimes -9.682e-05 3.736e-04 -0.259 0.795527
## Assaults 7.458e-05 7.445e-04 0.100 0.920211
## City..StateArlington, TX 1.725e+00 6.244e+00 0.276 0.782419
## City..StateAtlanta, GA 1.800e+00 6.129e+00 0.294 0.768990
## City..StateAurora, CO 3.150e+01 6.210e+00 5.073 4.37e-07 ***
## City..StateAustin, TX 2.932e+01 6.171e+00 4.751 2.20e-06 ***
## City..StateBaltimore, MD 9.210e+00 6.148e+00 1.498 0.134281
## City..StateBoston, MA 2.259e+01 6.240e+00 3.620 0.000304 ***
## City..StateBuffalo, NY -1.590e+01 6.187e+00 -2.570 0.010249 *
## City..StateCharlotte, NC -5.170e+00 6.234e+00 -0.829 0.407051
## City..StateChicago, IL 1.124e+00 6.117e+00 0.184 0.854237
## City..StateCincinnati, OH -6.436e+00 6.158e+00 -1.045 0.296118
## City..StateCleveland, OH -9.708e+00 6.109e+00 -1.589 0.112198
## City..StateColumbus, OH -5.089e+00 6.153e+00 -0.827 0.408337
## City..StateDallas, TX 5.267e+00 6.180e+00 0.852 0.394213
## City..StateDenver, CO 3.208e+01 6.190e+00 5.183 2.46e-07 ***
## City..StateDetroit, MI -4.220e+00 6.178e+00 -0.683 0.494706
## City..StateFresno, CA 5.140e+00 6.225e+00 0.826 0.409060
## City..StateHonolulu, HI -1.526e+01 6.262e+00 -2.436 0.014952 *
## City..StateHouston, TX 1.253e+01 6.199e+00 2.021 0.043476 *
## City..StateIndianapolis, IN -9.631e+00 6.241e+00 -1.543 0.122967
## City..StateJacksonville, FL 1.443e+01 6.267e+00 2.303 0.021392 *
## City..StateLouisville, KY 1.080e+00 6.227e+00 0.173 0.862357
## City..StateMemphis, TN -4.843e+00 6.217e+00 -0.779 0.436099
## City..StateMesa, AZ 2.342e+01 6.269e+00 3.736 0.000194 ***
## City..StateMiami, FL 3.096e+01 6.183e+00 5.007 6.10e-07 ***
## City..StateMilwaukee, WI 4.964e+00 6.331e+00 0.784 0.433169
## City..StateMinneapolis, MN 1.904e+01 6.200e+00 3.072 0.002164 **
## City..StateNashville, TN 7.028e+00 6.311e+00 1.114 0.265562
## City..StateNewark, NJ 1.271e+01 6.216e+00 2.044 0.041125 *
## City..StateOakland, CA 1.399e+01 6.223e+00 2.249 0.024664 *
## City..StateOmaha, NE 4.025e+00 6.276e+00 0.641 0.521376
## City..StateOrlando, FL 5.917e+00 6.277e+00 0.943 0.345995
## City..StatePhiladelphia, PA 6.232e+00 6.189e+00 1.007 0.314084
## City..StatePhoenix, AZ 2.406e+01 6.237e+00 3.857 0.000119 ***
## City..StatePittsburgh, PA -2.853e+00 6.195e+00 -0.460 0.645262
## City..StatePortland, OR 2.731e+01 6.295e+00 4.339 1.52e-05 ***
## City..StateRaleigh, NC -3.007e+00 6.249e+00 -0.481 0.630426
## City..StateSacramento, CA -1.063e+00 6.206e+00 -0.171 0.864078
## City..StateSeattle, WA 1.551e+01 6.160e+00 2.517 0.011913 *
## City..StateTampa, FL 1.640e+01 6.214e+00 2.639 0.008392 **
## City..StateTucson, AZ 1.090e+01 6.223e+00 1.752 0.080037 .
## City..StateTulsa, OK 4.952e+00 6.225e+00 0.795 0.426474
## City..StateWashington, DC 1.243e+01 6.152e+00 2.020 0.043540 *
## City..StateWichita, KS 1.576e+00 6.184e+00 0.255 0.798872
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 27.21 on 1657 degrees of freedom
## Multiple R-squared: 0.8088, Adjusted R-squared: 0.803
## F-statistic: 140.2 on 50 and 1657 DF, p-value: < 2.2e-16We use the OLS model which minimise the error using least square method and gives the beta coefficients. By using these beta we can predict housing price index for coming years. The R-square value is pretty high so our model estimated a good approximation model and adjusted R-square value shows all the independent variables are important to include.(we have checked by removing some variables values of R-square and adjusted R-square decreased in most cases).
4.3 Result
Value of F- statistics and p-value <<0.001 shows our hypothesis is right.All the City..states with statistically significant p-value of beta are currently well developed states and have grown well in terms of education, society and bussiness.We can see Boston,Washington,Denver,Miami,Seattle and others in this statistically significant list.Over the years technology and population changed which played vital role in change in housing price index as both year and population is statistically significant p-value of their beta coefficiengts is less than 0.01.By forcing law and order properly in states, crimes decreses and that result in rich societal value which leads to attract people to buy house at higher cost.
4.4 Conclusion
By forcing law and order properly in states, crimes decreses and that result in rich societal value which leads to attract people to buy house at higher cost.Over hypothesis is true that crimes and housing price index have negative correlation between each other. People generally like to live crouded places where law and order are properly working and crime rate is low.
4.5 References
Housing price index data Available from https://www.kaggle.com/sandeep04201988/housing-price-index-using-crime-rate-data
American crime study Available from https://www.kaggle.com/marshallproject/crime-rates
fit$coefficients[1:8]## (Intercept) Year Population Homicides Robberies
## -9.359824e+03 4.748463e+00 5.613083e-06 4.162309e-03 7.297628e-04
## Rapes Violent.Crimes Assaults
## -1.106297e-02 -9.682311e-05 7.458312e-055.1 Appendix 1
Studying correlation
cor(Housingpriceindex$index_nsa,Housingpriceindex$Population,method = "pearson")## [1] 0.1021095Positive correlation between population and housing price index, as expected with increase in population housing price index increases but slop of housing price index is very high in comparison to population, which can be the reason of weak correlation.
cor.test(Housingpriceindex$index_nsa,Housingpriceindex$Population,method = "pearson")##
## Pearson's product-moment correlation
##
## data: Housingpriceindex$index_nsa and Housingpriceindex$Population
## t = 4.2397, df = 1706, p-value = 2.358e-05
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## 0.05494486 0.14881948
## sample estimates:
## cor
## 0.1021095As p-value is less than 0.01, this correlation is statistically significant.
cor.test(Housingpriceindex$index_nsa,Housingpriceindex$Violent.Crimes)##
## Pearson's product-moment correlation
##
## data: Housingpriceindex$index_nsa and Housingpriceindex$Violent.Crimes
## t = -1.4656, df = 1706, p-value = 0.1429
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## -0.08275273 0.01198973
## sample estimates:
## cor
## -0.03546118People don’t like to buy houses in vicinity of area full of crime, so the correlation is negative.
cor.test(Housingpriceindex$index_nsa,Housingpriceindex$Homicides)##
## Pearson's product-moment correlation
##
## data: Housingpriceindex$index_nsa and Housingpriceindex$Homicides
## t = -3.5359, df = 1706, p-value = 0.0004173
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## -0.13219098 -0.03801809
## sample estimates:
## cor
## -0.08529503Correlation is negative as expected and also statistically significant with p-value = 0.0004173.Usually people avoid these kind of area to live because negative things affect minds to do negative.
cor.test(Housingpriceindex$index_nsa,Housingpriceindex$Rapes)##
## Pearson's product-moment correlation
##
## data: Housingpriceindex$index_nsa and Housingpriceindex$Rapes
## t = -4.885, df = 1706, p-value = 1.131e-06
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## -0.16396959 -0.07041383
## sample estimates:
## cor
## -0.1174523Negative correlation with p-value = 1.131e-0.6. statistically significnt. Again people don’t like to buy house where rape cases are high, it’s not safe for there children.
cor.test(Housingpriceindex$index_nsa,Housingpriceindex$Assaults)##
## Pearson's product-moment correlation
##
## data: Housingpriceindex$index_nsa and Housingpriceindex$Assaults
## t = 1.5716, df = 1706, p-value = 0.1162
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## -0.009426004 0.085298648
## sample estimates:
## cor
## 0.03802174Weak positive correlation with p-value = 0.1126, which is statistically insignificant.
cor.test(Housingpriceindex$index_nsa,Housingpriceindex$Robberies)##
## Pearson's product-moment correlation
##
## data: Housingpriceindex$index_nsa and Housingpriceindex$Robberies
## t = -3.9478, df = 1706, p-value = 8.208e-05
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## -0.14193599 -0.04793135
## sample estimates:
## cor
## -0.09514578Weak negative correlation as expected with p-value 8.208e-05,which is statistically significant.
library(corrgram)
corrgram(Housingpriceindex[ ,c(1,2,4:9)],order = TRUE,upper.panel = panel.pie, lower.panel = panel.shade,text.panel = panel.txt, main= "corrgram plot of housing price index with each variable",diag.panel = panel.minmax)
5.2 Appendix 2
2.1 Quick review of dataset
str(Housingpriceindex)## 'data.frame': 1708 obs. of 9 variables:
## $ Year : int 1975 1975 1975 1975 1975 1975 1976 1976 1976 1976 ...
## $ index_nsa : num 41.1 30.8 36.4 20.9 20.4 ...
## $ City..State : Factor w/ 44 levels "Albuquerque, NM",..: 3 10 12 30 39 43 3 6 10 11 ...
## $ Population : int 490584 3150000 659931 337748 503500 716000 457300 860974 3134499 427045 ...
## $ Violent.Crimes: int 8033 37160 10403 5900 3971 12704 7529 14191 30640 3461 ...
## $ Homicides : int 185 818 288 111 52 235 154 200 814 56 ...
## $ Rapes : int 443 1657 491 316 324 520 477 460 1179 263 ...
## $ Assaults : int 3518 12514 2524 2288 1492 2812 3518 5776 11070 1617 ...
## $ Robberies : int 3887 22171 7100 3185 2103 9137 3380 7755 17577 1525 ...index_nsa stand for housing price index in USD. Housing price Index = change in price/ Initial price
2.2 Dimension of data
dim(Housingpriceindex)## [1] 1708 92.3 Mean, max and median of data
library(psych)
describe(Housingpriceindex[ ,c(2,4:9)])[ ,c(2,3,4,5,8,9)]## n mean sd median min max
## index_nsa 1708 123.78 61.32 108.58 20.39 386.21
## Population 1708 621931.25 499821.21 470970.00 112994.00 3150000.00
## Violent.Crimes 1708 7794.24 8798.88 5119.00 385.00 90520.00
## Homicides 1708 115.13 143.51 62.00 1.00 960.00
## Rapes 1708 411.85 405.57 297.50 34.00 3754.00
## Assaults 1708 3853.74 4344.99 2588.00 264.00 42237.00
## Robberies 1708 3415.84 4206.70 2108.50 83.00 43783.00- One way contingency table
table(Housingpriceindex$Year)##
## 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989
## 6 24 40 43 44 44 44 44 44 44 44 44 44 44 44
## 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004
## 44 44 44 44 44 44 44 44 44 44 44 44 44 44 44
## 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
## 44 44 44 44 44 44 44 44 44 44 11Mostly we have 44 state’s observation sets every year.
table(Housingpriceindex$City..State)##
## Albuquerque, NM Arlington, TX Atlanta, GA Aurora, CO
## 39 38 41 39
## Austin, TX Baltimore, MD Boston, MA Buffalo, NY
## 39 40 39 39
## Charlotte, NC Chicago, IL Cincinnati, OH Cleveland, OH
## 39 41 40 41
## Columbus, OH Dallas, TX Denver, CO Detroit, MI
## 40 39 39 39
## Fresno, CA Honolulu, HI Houston, TX Indianapolis, IN
## 38 38 39 38
## Jacksonville, FL Louisville, KY Memphis, TN Mesa, AZ
## 37 39 38 38
## Miami, FL Milwaukee, WI Minneapolis, MN Nashville, TN
## 39 38 39 36
## Newark, NJ Oakland, CA Omaha, NE Orlando, FL
## 39 40 37 37
## Philadelphia, PA Phoenix, AZ Pittsburgh, PA Portland, OR
## 39 38 39 39
## Raleigh, NC Sacramento, CA Seattle, WA Tampa, FL
## 38 39 40 39
## Tucson, AZ Tulsa, OK Washington, DC Wichita, KS
## 38 38 40 39There are total 44 states of america in dataset.
meancrime = aggregate(Housingpriceindex$Violent.Crimes, by = list(State = Housingpriceindex$City..State),mean)
meancrime## State x
## 1 Albuquerque, NM 12217.590
## 2 Arlington, TX 5326.053
## 3 Atlanta, GA 6847.732
## 4 Aurora, CO 5522.333
## 5 Austin, TX 9443.641
## 6 Baltimore, MD 7451.400
## 7 Boston, MA 6857.205
## 8 Buffalo, NY 5917.872
## 9 Charlotte, NC 7273.128
## 10 Chicago, IL 7095.878
## 11 Cincinnati, OH 5905.475
## 12 Cleveland, OH 6193.293
## 13 Columbus, OH 5999.425
## 14 Dallas, TX 7123.615
## 15 Denver, CO 6111.308
## 16 Detroit, MI 6909.051
## 17 Fresno, CA 7084.553
## 18 Honolulu, HI 6732.368
## 19 Houston, TX 5093.231
## 20 Indianapolis, IN 6453.816
## 21 Jacksonville, FL 15465.838
## 22 Louisville, KY 6269.308
## 23 Memphis, TN 7386.921
## 24 Mesa, AZ 9535.342
## 25 Miami, FL 7297.410
## 26 Milwaukee, WI 9191.158
## 27 Minneapolis, MN 5203.333
## 28 Nashville, TN 7821.500
## 29 Newark, NJ 10248.667
## 30 Oakland, CA 7172.975
## 31 Omaha, NE 7322.865
## 32 Orlando, FL 7560.216
## 33 Philadelphia, PA 13513.590
## 34 Phoenix, AZ 8605.447
## 35 Pittsburgh, PA 8231.333
## 36 Portland, OR 10210.795
## 37 Raleigh, NC 9260.605
## 38 Sacramento, CA 7178.282
## 39 Seattle, WA 6979.925
## 40 Tampa, FL 9321.667
## 41 Tucson, AZ 8036.500
## 42 Tulsa, OK 8756.737
## 43 Washington, DC 7258.625
## 44 Wichita, KS 8254.564plot(meancrime, main = "Mean Violent crime rate in states")
Some city have been very high in violent crime.
meanPopulation = aggregate(Housingpriceindex$Population, by = list(State = Housingpriceindex$City..State),mean)
meanPopulation## State x
## 1 Albuquerque, NM 534349.6
## 2 Arlington, TX 562341.3
## 3 Atlanta, GA 569319.4
## 4 Aurora, CO 486596.0
## 5 Austin, TX 584607.9
## 6 Baltimore, MD 512723.5
## 7 Boston, MA 952875.6
## 8 Buffalo, NY 600227.6
## 9 Charlotte, NC 978754.2
## 10 Chicago, IL 636766.9
## 11 Cincinnati, OH 592461.3
## 12 Cleveland, OH 528310.0
## 13 Columbus, OH 546036.4
## 14 Dallas, TX 586302.9
## 15 Denver, CO 551614.2
## 16 Detroit, MI 502257.8
## 17 Fresno, CA 484269.0
## 18 Honolulu, HI 531729.5
## 19 Houston, TX 502107.2
## 20 Indianapolis, IN 585038.5
## 21 Jacksonville, FL 528063.2
## 22 Louisville, KY 515232.9
## 23 Memphis, TN 407784.2
## 24 Mesa, AZ 504605.3
## 25 Miami, FL 555914.2
## 26 Milwaukee, WI 1259156.8
## 27 Minneapolis, MN 475220.4
## 28 Nashville, TN 522815.9
## 29 Newark, NJ 667643.4
## 30 Oakland, CA 822692.1
## 31 Omaha, NE 478915.5
## 32 Orlando, FL 582112.8
## 33 Philadelphia, PA 646698.7
## 34 Phoenix, AZ 536581.0
## 35 Pittsburgh, PA 683420.4
## 36 Portland, OR 1104236.8
## 37 Raleigh, NC 714322.6
## 38 Sacramento, CA 712689.2
## 39 Seattle, WA 660126.2
## 40 Tampa, FL 685315.0
## 41 Tucson, AZ 596459.7
## 42 Tulsa, OK 652768.0
## 43 Washington, DC 567694.1
## 44 Wichita, KS 641348.4boxplot(Housingpriceindex$index_nsa, horizontal = TRUE, main = "Housing price index distribution over the years")
Most of the time price doubled as median is more than hundred.
boxplot(Housingpriceindex$Population, horizontal = TRUE, main = "Population distribution over the years")
There are too much outlier in population boxplot in righthand side.
library(car)##
## Attaching package: 'car'## The following object is masked from 'package:psych':
##
## logitscatterplot(Population ~ Year, data = Housingpriceindex)
4. Hypothesis: With increase in poplulation, housing price index should be incresing
House_index <- aggregate(Housingpriceindex$index_nsa, by = list(Year = Housingpriceindex$Year),mean)
House_index## Year x
## 1 1975 30.11500
## 2 1976 36.72017
## 3 1977 40.73010
## 4 1978 46.90176
## 5 1979 53.80188
## 6 1980 58.45125
## 7 1981 61.55432
## 8 1982 63.95102
## 9 1983 66.26108
## 10 1984 68.61102
## 11 1985 71.55506
## 12 1986 75.43449
## 13 1987 78.77506
## 14 1988 81.64983
## 15 1989 84.95335
## 16 1990 88.14335
## 17 1991 96.97639
## 18 1992 99.92093
## 19 1993 103.17143
## 20 1994 107.40221
## 21 1995 110.89804
## 22 1996 114.66321
## 23 1997 118.34258
## 24 1998 124.34049
## 25 1999 131.87725
## 26 2000 141.28338
## 27 2001 151.43585
## 28 2002 160.82273
## 29 2003 171.12045
## 30 2004 186.22026
## 31 2005 209.28146
## 32 2006 224.55007
## 33 2007 223.26373
## 34 2008 203.34487
## 35 2009 187.41925
## 36 2010 180.93774
## 37 2011 173.03646
## 38 2012 177.97761
## 39 2013 191.83261
## 40 2014 204.67202
## 41 2015 199.17545plot(House_index ,main = " Mean Housing Price Index from 1975 to 2015", ylab = "Mean housing Price index")
Overall trend is upward, Housing price contantly increasing over the years.
YearlyPolulation <- aggregate(Population ~ Year, data = Housingpriceindex,sum)
YearlyPolulation## Year Population
## 1 1975 5857763
## 2 1976 16983123
## 3 1977 23420031
## 4 1978 23987753
## 5 1979 24946275
## 6 1980 24636321
## 7 1981 24612296
## 8 1982 24853004
## 9 1983 27788236
## 10 1984 25181519
## 11 1985 25344823
## 12 1986 25601365
## 13 1987 26192138
## 14 1988 25435384
## 15 1989 25433873
## 16 1990 25988349
## 17 1991 25583366
## 18 1992 25803789
## 19 1993 26373767
## 20 1994 27283435
## 21 1995 25922116
## 22 1996 26557239
## 23 1997 26879303
## 24 1998 26612583
## 25 1999 27553312
## 26 2000 28055928
## 27 2001 28620733
## 28 2002 28676866
## 29 2003 28667377
## 30 2004 28762411
## 31 2005 28955986
## 32 2006 29196347
## 33 2007 29658684
## 34 2008 29992130
## 35 2009 29480743
## 36 2010 29442632
## 37 2011 29761041
## 38 2012 30082832
## 39 2013 30399496
## 40 2014 30805707
## 41 2015 6868491plot(YearlyPolulation, main = "Total Yearly Populatin of America's 44 states", ylab = "YearlyPolulation")