2024-02-18

This is an R Markdown presentation. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.

When you click the Knit button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document.

## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.4     ✔ readr     2.1.5
## ✔ forcats   1.0.0     ✔ stringr   1.5.1
## ✔ ggplot2   3.4.4     ✔ tibble    3.2.1
## ✔ lubridate 1.9.3     ✔ tidyr     1.3.1
## ✔ purrr     1.0.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
## 
## Attaching package: 'plotly'
## 
## 
## The following object is masked from 'package:ggplot2':
## 
##     last_plot
## 
## 
## The following object is masked from 'package:stats':
## 
##     filter
## 
## 
## The following object is masked from 'package:graphics':
## 
##     layout

New York Housing Data from Kaggle

##  BROKERTITLE            TYPE               PRICE                BEDS       
##  Length:4801        Length:4801        Min.   :2.494e+03   Min.   : 1.000  
##  Class :character   Class :character   1st Qu.:4.990e+05   1st Qu.: 2.000  
##  Mode  :character   Mode  :character   Median :8.250e+05   Median : 3.000  
##                                        Mean   :2.357e+06   Mean   : 3.357  
##                                        3rd Qu.:1.495e+06   3rd Qu.: 4.000  
##                                        Max.   :2.147e+09   Max.   :50.000  
##       BATH         PROPERTYSQFT     ADDRESS             STATE          
##  Min.   : 0.000   Min.   :  230   Length:4801        Length:4801       
##  1st Qu.: 1.000   1st Qu.: 1200   Class :character   Class :character  
##  Median : 2.000   Median : 2184   Mode  :character   Mode  :character  
##  Mean   : 2.374   Mean   : 2184                                        
##  3rd Qu.: 3.000   3rd Qu.: 2184                                        
##  Max.   :50.000   Max.   :65535                                        
##  MAIN_ADDRESS       ADMINISTRATIVE_AREA_LEVEL_2   LOCALITY        
##  Length:4801        Length:4801                 Length:4801       
##  Class :character   Class :character            Class :character  
##  Mode  :character   Mode  :character            Mode  :character  
##                                                                   
##                                                                   
##                                                                   
##  SUBLOCALITY        STREET_NAME         LONG_NAME         FORMATTED_ADDRESS 
##  Length:4801        Length:4801        Length:4801        Length:4801       
##  Class :character   Class :character   Class :character   Class :character  
##  Mode  :character   Mode  :character   Mode  :character   Mode  :character  
##                                                                             
##                                                                             
##                                                                             
##     LATITUDE       LONGITUDE     
##  Min.   :40.50   Min.   :-74.25  
##  1st Qu.:40.64   1st Qu.:-73.99  
##  Median :40.73   Median :-73.95  
##  Mean   :40.71   Mean   :-73.94  
##  3rd Qu.:40.77   3rd Qu.:-73.87  
##  Max.   :40.91   Max.   :-73.70

Cleaning/Organizing data

sum(is.na(df)) ## no NA values found
newdf = df %>% select(-c(BROKERTITLE,ADDRESS:LOCALITY,STREET_NAME:LONGITUDE)) 
#removed unneeded columns
summary(newdf)
newnewdf = newdf %>% mutate(PRICE = log(PRICE),
                            PROPERTYSQFT = log(PROPERTYSQFT))
## Mutating price and property sqft to be more readable on graph

psq = newdf['PROPERTYSQFT']

plot_ly(data=newdf,x= ~PROPERTYSQFT,y= ~PRICE) 
## Identified outlier within PRICE, I am going to remove that data point 
## before we perform a regression analysis

Cleaning/Organizing Data Cont.

PriceOutlier = max(newdf$PRICE)
which(newdf$PRICE == PriceOutlier)
newdf = newdf[-(305),]
max(newdf$PRICE)

plot_ly(data=newdf,x= ~PROPERTYSQFT,y= ~PRICE) 
## Another outlier in PRICE and one in PROPERTYSQFT, this probably isn't 
## the best way to find outliers lol

which(newdf$PRICE == max(newdf$PRICE))
which(psq == max(psq))
newdf = newdf[-c(2,4624),]
max(newdf$PRICE)
max(psq)

What is Regression Analysis?

Simple Linear Regression

  • Aims to find a linear relationship to describe the correlation between an independent and possibly dependent variable.

  • Equation: y = a + bx

Multiple Linear Regression

  • Aims to find a linear relationship between variables in situations where there are several independent values.

  • Equation: y = a + b1x1 + b2x2 + … + bnxn

What is the Correlation Coefficient and How to Find It

Correlation Coefficient

  • A statistical concept which helps in establishing a relation between predicted and actual values obtained in a statistical experiment. The calculated value of the correlation coefficient explains the exactness between the predicted and actual values.

How to Find It

  • \[ x = \frac{n (\sum xy) - (\sum x)(\sum y)}{\sqrt{[n \sum x^2 - (\sum x^2)][n \sum y^2 - (\sum y^2)]}} \]

Starter Scatterplot Graph

Determining Correlation in Previous Scatterplot

Code for Correlation Scatterplot

correlcoef = round((cor(newnewdf$PROPERTYSQFT,newnewdf$PRICE)),3)
correlcoef = c('Correlation Coefficient: \n \n', correlcoef)

sp = ggplot(data=newnewdf, aes(x=PROPERTYSQFT,y=PRICE))
sp + geom_point() + geom_smooth(method = 'lm', formula = y~x, se = FALSE) + 
  annotate("text", x=6, y=17, label=correlcoef) 

## Adding line of best fit and correlation coefficient

Sublocality Classification Variables