What is Simple Linear Regression?

  • Simple Linear Regression is a supervised machine learning method.
  • It predicts a numerical value using one independent variable.
  • It finds the best-fitting straight line through the data.
  • In this example, we will predict house prices using the average number of rooms.

Boston Housing Dataset

  • The dataset contains 506 observations from Boston-area census tracts.
  • It is available in R through the MASS package.
  • rm: Average number of rooms per dwelling (X).
  • medv: Median home value in $1,000s (Y).
  • Our goal is to predict median home values based on average room counts.

Simple Linear Regression Equation

In general simple linear regression models the relationship between an independent variable and a dependent variable using a straight line.

\[ \hat{Y} = b_0 + b_1X \]

  • \(\hat{Y}\): Predicted house price
  • \(b_0\): It is the Intercept of the regression line
  • \(b_1\): The Slope of the regression line
  • \(X\): The Average number of rooms We can use this equation to predict median house values based on the average number of rooms.

Relationship Between Rooms and Price

Before fitting a regression line, we can visualize how the average number of rooms relates to median house values.

The scatterplot shows a positive relationship between average room counts and median house values.

Fitting the Regression Line

We can use linear regression to find the best fitting line that describes the relationship between room counts and median house values.

The red line represents the predicted median house value for a given average number of rooms.

Interactive House Price Prediction

Using Plotly to explore the relationship between average room counts and median house values interactively.

Training and Making Predictions

We train a simple linear regression model using the Boston housing dataset and use it to predict median house values.

model <- lm(medv ~ rm, data = Boston)

# View the regression coefficients
coef(model)
## (Intercept)          rm 
##  -34.670621    9.102109
# Predict median value for an average of 6 rooms
predict(model, newdata = data.frame(rm = 6))
##        1 
## 19.94203

The model learns the relationship between average room counts and median house values.

Model Evaluation and Conclusion

We can evaluate our regression model using Mean Squared Error (MSE), which measures the average squared difference between actual and predicted values.

\[ MSE = \frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2 \]

  • \(y_i\): Actual median house value
  • \(\hat{y}_i\): Predicted median house value
  • \(n\): Total number of observations

Conclusion:

  • We used simple linear regression to predict median house values based on average room counts.
  • The model showed a positive relationship between the number of rooms and median house values.
  • Simple linear regression provides a basic approach to predicting numerical values in machine learning.