Linear regression is one of the most powerful tools that is used daily by those who deal with data. It is one of the fundamentals of Data Science and Science in general.

Linear Regression allows us to find quantitative relationships between data and which will then help us predict future occurrences within a ball park.

Imagine you wanted to predict the price of a house based on its area. What you would need is historical data of housing prices and their corresponding area/size. Plotting this Price vs Size graph, would result in strong relationship that would allow us to predict/observe the price we are interested in and find what size of a house it would likely get us. We will look at this more, but before we do, lets define the math behind it for better understanding.

\[ y = \beta_0 + \beta_1 x + \varepsilon \]

This is the general formula for linear regression, where:

\(y\) : The dependent variable (what we are trying to predict)

\(x\) : The independent variable (The predictor)

\(\beta_0\) : The intercept (The value of y when x = 0)

\(\beta_1\) : The slope (how much y changes for each 1 unit increase in x)

\(\varepsilon\) : the error term (the difference between predicted and actual values)

Now that we have a better understanding of Linear Regression, let’s explore the residual plot to see how well this linear model fits our data.

The Residual plot helps us understand the accuracy of this fit. this is essentially our

\(\varepsilon\) which is calculated from the difference between predicted y(price) and the actual y(price). Using the following formula:

\[ \varepsilon = y -\hat{y} \]

By using the R language, we can draw a better picture of what is happening in this data set which would help us understand the data even more. The more we understand the data, the better data driven decision we can make. This is very important in business and also familiarity with data allows us to come up with hypothesis to discover new relationships as well.

Using the following chunk of code, we can evaluate this relationship and our model.

model = lm(price ~ sqft, data = Sacramento)
summary(model)

## 
## Call:
## lm(formula = price ~ sqft, data = Sacramento)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -231889  -54717  -11822   38993  600141 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 13859.393   6948.714   1.995   0.0464 *  
## sqft          138.546      3.796  36.495   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 84130 on 930 degrees of freedom
## Multiple R-squared:  0.5888, Adjusted R-squared:  0.5884 
## F-statistic:  1332 on 1 and 930 DF,  p-value: < 2.2e-16

This report shows us that each additional square foot added to the property, approximately adds $138.55 to the price of a house in Sacramento. However, the model suggests a 59% variance which is good to get a grasp of the fit of our data, yet not quite reliable.

We can make linear regression more sophisticated by adding more features in to what we already know.

\[ \hat{y} = \beta_0 + \beta_1(sqft) + \beta_2(beds) \]

With this, we can have a better prediction and a stronger understanding of price prediction.

The following 3D graph, shows us a better view of what is happening with price as Square footage and number of bedrooms increase.

Finally, linear regression is a powerful tool for anyone who works and deals with data. It is one of the fundamentals of data science which is used widely in all realms, especially in Science, Finance, Technology and Engineering sectors.

library(plotly)
data("Sacramento")

sac = na.omit(Sacramento[, c("price", "sqft", "beds")])

plot_ly(
  data = sac,
  x = ~sqft,
  y = ~beds,
  z = ~price, 
  type = "scatter3d",
  mode = "markers",
  marker = list(size = 4, opacity = 0.8, color = ~price, colorscale = "Viridis")
  )

  layout(
    title = "3D: Price vs. Square Feet vs. Bedrooms",
    scene = list(
      xaxis = list(title = "Square Feet"),
      yaxis = list(title = "Bedrooms"),
      zaxis = list(title = "Price (USD)")
    )
  )