2024-03-21

Introduction

Simple linear regression is a statistical method that allows us to summarize and study relationships between two continuous (quantitative) variables:

  1. Dependent Variable (DV): This is the main factor you are trying to understand or predict. In simple linear regression, the DV is assumed to be a linear function of the independent variable.

  2. Independent Variable (IV): This is the variable you are using to predict the DV. It is sometimes called the predictor variable.

The basic mathematical model for simple linear regression is:

\[ y = \beta_0 + \beta_1x + \epsilon \]

Basics of Linear Regression

Linear regression models the relationship between a dependent variable \(Y\) and one or more independent variables \(X\) using a linear equation. Here,

  • \(y\) represents the dependent variable.
  • \(x\) represents the independent variable.
  • \(\beta_0\) is the y-intercept of the regression line.
  • \(\beta_1\) is the slope of the regression line, which represents the average change in the DV for a one-unit change in the IV.
  • \(\epsilon\) is the error term, which accounts for the variability in DV that cannot be explained by the IV.

The goal of linear regression is to find the best-fitting straight line through the points on a scatter plot that minimizes the sum of squared residuals (the differences between observed and predicted values).

Assumptions of Linear Regression

For the least squares estimates to be the best linear unbiased estimators (BLUE), the following assumptions should be satisfied:

  1. Linearity: The relationship between \(X\) and \(Y\) is linear.
  2. Independence: Observations are independent of each other.
  3. Homoscedasticity: The residuals have constant variance at every level of \(X\).
  4. Normality: The residuals of the model are normally distributed.

Loading Data

Building a Linear Model

Call:
lm(formula = mpg ~ wt, data = mtcars)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.5432 -2.3647 -0.1252  1.4096  6.8727 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
wt           -5.3445     0.5591  -9.559 1.29e-10 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 3.046 on 30 degrees of freedom
Multiple R-squared:  0.7528,    Adjusted R-squared:  0.7446 
F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

Histogram

This histogram shows the distribution of miles per gallon (mpg) across all cars in the mtcars dataset.

Model Summary

Advance Visualisation with Ploty

R Code for Each Plot

1.Scatter Plot :

rs, aes(x = wt, y = mpg)) + 
  geom_point() + 
  theme_minimal() + 
  labs(title = "Scatter plot of MPG vs Car Weight")

2.Histogram :

# Histogram of mpg
ggplot(mtcars, aes(x = mpg)) +
  geom_histogram(bins = 10, fill = "blue", color = "black") +
  theme_minimal() +
  labs(title = "Distribution of Miles Per Gallon",
       x = "Miles per Gallon",
       y = "Frequency")

R Code for Each Plot

3.Model Summary :

plot(lm_model, which = 1, mai`n = "Residuals vs Fitted", 
     col = "red", pch = 20, cex = 1.2)
abline(h = 0, col = "blue")

R Code for Each Plot

4.Advanced visualisation plot :

# 3D scatter plot of wt, mpg, and disp from the mtcars dataset
p <- plot_ly(data = mtcars, x = ~wt, y = ~mpg, z = ~disp, 
             type = 'scatter3d', mode = 'markers',
             marker = list(size = 5,
                           opacity = 0.8,
                           color = mtcars$disp,
                           colorscale = c('#FF0000', '#0000FF')))
p <- p %>% layout(scene = list(
                    xaxis = list(title = 'Weight (1000 lbs)'),
                    yaxis = list(title = 'Miles per Gallon'),
                    zaxis = list(title = 'Displacement (cu. in.)')
                  ),
                  margin = list(l = 0, r = 0, b = 0, t = 0))
p

Conculsion

Simple linear regression is more than just a statistical tool—it is a lens through which we can interpret the world around us. By quantifying relationships between variables, it serves as a foundational technique for both predictive modeling and inferential statistics.

  • Predictive Power: It allows us to forecast future observations and make decisions based on data.
  • Causal Inference: With the right data and precautions, regression can help us understand the cause-and-effect relationships.
  • Versatility: Its principles are applicable in various fields, from economics to engineering, biology to social sciences.
  • Foundation for Advanced Methods: Understanding linear regression opens the door to more complex models like multiple regression, time series analysis, and machine learning.

The simplicity of the linear regression model—its clear interpretation and ease of use—makes it an indispensable tool in the statistician’s toolkit. However, it is crucial to remember the assumptions underlying the model and ensure they are met before drawing conclusions from the analysis.