##Introduction The goal of this project is to demonstrate proficiency in working with data and to outline plans for creating a prediction algorithm and a Shiny app using the Boston Housing Data dataset. This document will provide an exploratory analysis of the dataset and summarize the goals for the eventual algorithm and app.

##Data Description The Boston Housing Data dataset contains information about various characteristics of houses in Boston and their corresponding prices. Let’s load the dataset and take a quick look at its structure.

library(mlbench)
library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
# Load the dataset
data(BostonHousing)

# Display the first few rows of the dataset
head(BostonHousing)
##      crim zn indus chas   nox    rm  age    dis rad tax ptratio      b lstat
## 1 0.00632 18  2.31    0 0.538 6.575 65.2 4.0900   1 296    15.3 396.90  4.98
## 2 0.02731  0  7.07    0 0.469 6.421 78.9 4.9671   2 242    17.8 396.90  9.14
## 3 0.02729  0  7.07    0 0.469 7.185 61.1 4.9671   2 242    17.8 392.83  4.03
## 4 0.03237  0  2.18    0 0.458 6.998 45.8 6.0622   3 222    18.7 394.63  2.94
## 5 0.06905  0  2.18    0 0.458 7.147 54.2 6.0622   3 222    18.7 396.90  5.33
## 6 0.02985  0  2.18    0 0.458 6.430 58.7 6.0622   3 222    18.7 394.12  5.21
##   medv
## 1 24.0
## 2 21.6
## 3 34.7
## 4 33.4
## 5 36.2
## 6 28.7

##The dataset contains the following variables:

crim: Per capita crime rate by town zn: Proportion of residential land zoned for lots over 25,000 sq. ft. indus: Proportion of non-retail business acres per town chas: Charles River dummy variable (1 if tract bounds river; 0 otherwise) nox: Nitric oxides concentration (parts per 10 million) rm: Average number of rooms per dwelling age: Proportion of owner-occupied units built prior to 1940 dis: Weighted distances to five Boston employment centers rad: Index of accessibility to radial highways tax: Full-value property tax rate per $10,000 ptratio: Pupil-teacher ratio by town black: 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town lstat: Lower status of the population (%) medv: Median value of owner-occupied homes in $1000’s

##Exploratory Analysis Now, let’s perform some exploratory analysis on the dataset to better understand its characteristics.

# Histogram of crime rate
hist(BostonHousing$crim, main = "Histogram of Crime Rate", xlab = "Crime Rate")

# Histogram of average number of rooms per dwelling
hist(BostonHousing$rm, main = "Histogram of Average Number of Rooms", xlab = "Number of Rooms")

# Histogram of median value of owner-occupied homes
hist(BostonHousing$medv, main = "Histogram of Median Home Value", xlab = "Median Home Value ($1000's)")

# Convert chas to numeric
BostonHousing$chas <- as.numeric(as.character(BostonHousing$chas))

# Check the structure again
str(BostonHousing)
## 'data.frame':    506 obs. of  14 variables:
##  $ crim   : num  0.00632 0.02731 0.02729 0.03237 0.06905 ...
##  $ zn     : num  18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...
##  $ indus  : num  2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87 ...
##  $ chas   : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ nox    : num  0.538 0.469 0.469 0.458 0.458 0.458 0.524 0.524 0.524 0.524 ...
##  $ rm     : num  6.58 6.42 7.18 7 7.15 ...
##  $ age    : num  65.2 78.9 61.1 45.8 54.2 58.7 66.6 96.1 100 85.9 ...
##  $ dis    : num  4.09 4.97 4.97 6.06 6.06 ...
##  $ rad    : num  1 2 2 3 3 3 5 5 5 5 ...
##  $ tax    : num  296 242 242 222 222 222 311 311 311 311 ...
##  $ ptratio: num  15.3 17.8 17.8 18.7 18.7 18.7 15.2 15.2 15.2 15.2 ...
##  $ b      : num  397 397 393 395 397 ...
##  $ lstat  : num  4.98 9.14 4.03 2.94 5.33 ...
##  $ medv   : num  24 21.6 34.7 33.4 36.2 28.7 22.9 27.1 16.5 18.9 ...
# Calculate correlation matrix
cor_matrix <- cor(BostonHousing)

# Display correlation matrix
cor_matrix
##                crim          zn       indus         chas         nox
## crim     1.00000000 -0.20046922  0.40658341 -0.055891582  0.42097171
## zn      -0.20046922  1.00000000 -0.53382819 -0.042696719 -0.51660371
## indus    0.40658341 -0.53382819  1.00000000  0.062938027  0.76365145
## chas    -0.05589158 -0.04269672  0.06293803  1.000000000  0.09120281
## nox      0.42097171 -0.51660371  0.76365145  0.091202807  1.00000000
## rm      -0.21924670  0.31199059 -0.39167585  0.091251225 -0.30218819
## age      0.35273425 -0.56953734  0.64477851  0.086517774  0.73147010
## dis     -0.37967009  0.66440822 -0.70802699 -0.099175780 -0.76923011
## rad      0.62550515 -0.31194783  0.59512927 -0.007368241  0.61144056
## tax      0.58276431 -0.31456332  0.72076018 -0.035586518  0.66802320
## ptratio  0.28994558 -0.39167855  0.38324756 -0.121515174  0.18893268
## b       -0.38506394  0.17552032 -0.35697654  0.048788485 -0.38005064
## lstat    0.45562148 -0.41299457  0.60379972 -0.053929298  0.59087892
## medv    -0.38830461  0.36044534 -0.48372516  0.175260177 -0.42732077
##                  rm         age         dis          rad         tax    ptratio
## crim    -0.21924670  0.35273425 -0.37967009  0.625505145  0.58276431  0.2899456
## zn       0.31199059 -0.56953734  0.66440822 -0.311947826 -0.31456332 -0.3916785
## indus   -0.39167585  0.64477851 -0.70802699  0.595129275  0.72076018  0.3832476
## chas     0.09125123  0.08651777 -0.09917578 -0.007368241 -0.03558652 -0.1215152
## nox     -0.30218819  0.73147010 -0.76923011  0.611440563  0.66802320  0.1889327
## rm       1.00000000 -0.24026493  0.20524621 -0.209846668 -0.29204783 -0.3555015
## age     -0.24026493  1.00000000 -0.74788054  0.456022452  0.50645559  0.2615150
## dis      0.20524621 -0.74788054  1.00000000 -0.494587930 -0.53443158 -0.2324705
## rad     -0.20984667  0.45602245 -0.49458793  1.000000000  0.91022819  0.4647412
## tax     -0.29204783  0.50645559 -0.53443158  0.910228189  1.00000000  0.4608530
## ptratio -0.35550149  0.26151501 -0.23247054  0.464741179  0.46085304  1.0000000
## b        0.12806864 -0.27353398  0.29151167 -0.444412816 -0.44180801 -0.1773833
## lstat   -0.61380827  0.60233853 -0.49699583  0.488676335  0.54399341  0.3740443
## medv     0.69535995 -0.37695457  0.24992873 -0.381626231 -0.46853593 -0.5077867
##                   b      lstat       medv
## crim    -0.38506394  0.4556215 -0.3883046
## zn       0.17552032 -0.4129946  0.3604453
## indus   -0.35697654  0.6037997 -0.4837252
## chas     0.04878848 -0.0539293  0.1752602
## nox     -0.38005064  0.5908789 -0.4273208
## rm       0.12806864 -0.6138083  0.6953599
## age     -0.27353398  0.6023385 -0.3769546
## dis      0.29151167 -0.4969958  0.2499287
## rad     -0.44441282  0.4886763 -0.3816262
## tax     -0.44180801  0.5439934 -0.4685359
## ptratio -0.17738330  0.3740443 -0.5077867
## b        1.00000000 -0.3660869  0.3334608
## lstat   -0.36608690  1.0000000 -0.7376627
## medv     0.33346082 -0.7376627  1.0000000

From the histograms, we can observe the distributions of key variables such as crime rate, average number of rooms per dwelling, and median home value.

We can see whether there is a strong correlation between crime rate and median home value, or between average number of rooms and median home value. These insights will be valuable for building our prediction algorithm.

##Conclusion The histogram of crime rate shows that the majority of towns have a low crime rate, but there are some outliers with higher crime rates. The histogram of average number of rooms per dwelling reveals a roughly normal distribution, with most houses having around 6 rooms. The histogram of median home value indicates that the majority of homes have a median value in the range of $20,000 to $30,000, with some outliers at higher values. The correlation matrix shows that there are some significant correlations between certain variables. For example, there is a positive correlation between the number of rooms and median home value, indicating that larger houses tend to have higher values. Conversely, there is a negative correlation between crime rate and median home value, suggesting that areas with higher crime rates tend to have lower home values.

Our prediction goal, algorithm and Shiny app will be based on these observations. We will leverage the insights gained from the exploratory analysis to inform the development of our algorithm and app.

Overall, this exploratory analysis has provided a solid foundation for our future work on building the prediction algorithm and Shiny app, and it has helped us understand the key features of the Boston Housing Data dataset.