2025-02-09
Simple Linear Regression
Topics To Cover
- Intro To Regression
- Understanding Simple Linear Regression
- Assumptions of Linear Regression
- Estimating the Parameters
- R Code For Linear Regression
- Visualizing Data and Regression Line
- Interpreting Regression Output
- Residual Analysis
- Advanced Visualization
Intro to Regression Analysis
Regression analysis is a way we can figure out how things relate to each other. It helps us predict what’s going to happen to on thing known as the dependent variable based on the what we already know about other things known as the independent variables. This is a tool used in statistics to help see patterns and make sense of how different factors may impact each other.
Understanding Simple Linear Regression
Simple Linear Regression (SLR) models the relationship between a dependent variable \(Y\) and an independent variable \(X\) as linear:
\(Y\) = \(\beta_0\) + \(\beta_1 X\) + \(\epsilon\)
Where \(\beta_0\) is the intercept, \(\beta_1\) the slope, and \(\epsilon\) the error term.
Assumptions of Linear Regression
SLR relies on four main assumptions:
1. Linearity: The relationship between \(X\) and \(Y\) is linear.
2. Independence: Observations are independent of one another.
3. Homoscedasticity: Constant variance of error terms.
4. Normal Distribution of Errors: Errors are normally distributed.
Estimating The Parameters
Parameters\(\beta_0\) and \(\beta_1\) are estimated using the Least Squares Method, which minimizes the sum of the squared differences between observed and predicted values:
\(\hat{\beta}_{1}\) = \(\frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}\)
\(\hat{\beta}_{0}\) = \(\bar{y} - \hat{\beta_1}\bar{x}\)
R Code For Linear Regression
## ## Call: ## lm(formula = mpg ~ wt, data = mtcars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -4.5432 -2.3647 -0.1252 1.4096 6.8727 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 37.2851 1.8776 19.858 < 2e-16 *** ## wt -5.3445 0.5591 -9.559 1.29e-10 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 3.046 on 30 degrees of freedom ## Multiple R-squared: 0.7528, Adjusted R-squared: 0.7446 ## F-statistic: 91.38 on 1 and 30 DF, p-value: 1.294e-10
Visualizing Data and Regression Line
## `geom_smooth()` using formula = 'y ~ x'
Interpreting Regression Output
In the graph created from the mtcars dataset, there’s a clear visual indication that as the weight of the cars increases, their fuel efficiency decreases. This downward trend on the plot, illustrated by the blue regression line, directly reflects our analysis results, highlighting the negative impact of increased weight on fuel economy.
Residual Analysis
Residual Analysis
In the residual plot, each point shows the error for a single prediction, plotted against the predicted value itself. We look for randomness in how these points scatter around the zero line. If they’re randomly spread, it suggests our model is doing a good job at predicting with no bias.
Advanced Visualization
Conclusion
In summary, our exploration of the mtcars dataset with linear regression and 3D scatter plots revealed clear relationships between a car’s weight, its fuel efficiency, and acceleration performance. Heavier cars generally consume more fuel and have slower acceleration times. This not only confirms some expected trends but also helps in understanding how different vehicle characteristics interplay, which is crucial for automotive design and performance optimization.