Introduction

  • Linear Regression is a statistical model which commonly used to present the relationship between a dependent variable and one or more independent variables.
  • Linear Regression is used for prediction, inference and understading relationship between variables.

Linear Regression Formula

One dependent variable and one independent variable

  • \(y = \beta_0 + \beta_1 x_1 + \epsilon\)
  • \(y\): Dependent variable
  • \(x_1\): Independent variable
  • \(\beta_0, \beta_1\): Coefficients (parameters)
  • \(\epsilon\): Error term

Linear Regression Formula

One dependent variable and many independent variables

  • \(y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + ... + \beta_n x_n + \epsilon\)
  • \(y\): Dependent variable
  • \(x_1, x_2, ..., x_n\): Independent variables
  • \(\beta_0, \beta_1, ..., \beta_n\): Coefficients (parameters)
  • \(\epsilon\): Error term

Loss Function

  • To calculate the difference between predicted values and actual values
  • \(\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2\)
  • \(y_i\): Actual Value
  • \(\hat{y}_i\): Predicted Value

Advantages of Linear Regression

  • Simple to understand and implement.
  • Easy to interpret coefficients and understand relationships.
  • Can handle non-linear relationships through transformation.

Disadvantages of Linear Regression

  • Sensitive to data outliers.
  • Cannot capture complex relationships between data.
  • Relies on assumptions such as linearity, independence, and normality of residuals (error term).

Data Visualization

Dataset

head(insurance)
##   age    sex    bmi children smoker    region   charges
## 1  19 female 27.900        0    yes southwest 16884.924
## 2  18   male 33.770        1     no southeast  1725.552
## 3  28   male 33.000        3     no southeast  4449.462
## 4  33   male 22.705        0     no northwest 21984.471
## 5  32   male 28.880        0     no northwest  3866.855
## 6  31 female 25.740        0     no southeast  3756.622

Correlation matrix

Relationship between BMI, Age and Charges

Code of the previous plot

plot_ly(data = insurance, x = ~bmi, y = ~age, z = ~charges, 
        type = "scatter3d", mode = "markers", 
        marker = list(size = 5, color = ~children, 
                      colorscale = "Viridis"), 
        text = ~paste("BMI: ", bmi, 
                      "<br>Age: ", age, 
                      "<br>Charges: ", charges, 
                      "<br>Children: ", children)) %>%
        layout(scene = list(xaxis = list(title = "BMI"), 
                            yaxis = list(title = "Age"), 
                            zaxis = list(title = "Charges")))

Age vs. Charges

Smoker status vs. Charges

Sex vs. Charges

Conclusion

  • Linear Regression serves as a fundamental tool in predictive modeling and understanding relationships between variables.
  • Linear Regression is easy to understand, flexible, and straightforward, however it also has limitations.
  • Regression models are easier to analyze and validate when the right data visualization approaches are used.