2026-10-13

Introduction to Linear Regression

  • In data science, statistical models are essential for understanding patterns within data and making accurate predictions.

  • Linear regression methods are particularly important because they allow researchers to quantify relationships between variables.

  • Understanding these concepts is essential for improving predictive models in the future and finding trends between variables.

  • Real-world applications of linear regression models include innovative research in fields such as healthcare, sales figures in business, and key performance indicators in economics, to name a few.

Understanding the basics of Linear Regression

  • Simple Linear Regression: the technique for fitting a line through data points, so that it can make accurate predictions on unknown data

  • The simple linear regression method is widely used because it can predict outcomes between an independent and dependent variable.

  • Given a data set with one or more independent (x/input) variables and a single dependent (y/output) variable, a linear regression model describes the relationship between these variables by using the formula, \[ y = m*x + b\]

Statistics Behind Simple Linear Regression

  • The equation for linear regression is frequently represented as:

\[ y = (\beta_1*x_1) + (\beta_2*x_2) + (\beta_3*x_3) + ... + (\beta_n*x_n) + (\beta_0)\]

  • This equation follows the same pattern as the \[ y = m*x + b\] equation, where \[ \beta_n\] represents the slope and \[ \beta_0\] represents the y-intercept.

Steps to build a Simple Linear Regression Model:

Now, let’s get into the steps to build a model:

  1. Create a line of best fit based on the data points, which are graphed as a linear relationship.
  2. Take the residuals, which are the difference between the data point and the prediction from the line of best fit, and square them. This results in the squared residuals.
  3. Find the beta n and beta 0 values that minimize the residual sum of squares (RSS). The RSS is also known as the loss function.
  • Use mathematical methods such as matrix decomposition and gradient descent to find these coefficients (beta n and beta 0).

Optional: Utilize the P-test to confirm whether the relationship between the variables is statistically significant.

Simple Linear Regression in Action: Heart Rate Data

data(heartrate)
#head(heartrate)

reg1 <- lm(maxrate ~ age, data = heartrate)

ggplot(heartrate, aes(x = age, y = maxrate)) +
  geom_point(shape = 16) +
  geom_abline(intercept = coef(reg1)[1], 
              slope = coef(reg1)[2], color = "mediumturquoise")+
  labs(title = "Max Heart Rate vs Age")

This scatter plot shows that as the Age of an individual increases, their maximum heart rate decreases. This means that relationship between the two variables is inverse and as such the slope in the linear regression equation will be negative.

Another example: Baby Development Data

This scatter plot shows the relationship between the gestational period (amount of days that a woman is pregnant) and the weight of the baby. The positive slope from the linear regression model shows that as the gestation period increases, the baby’s weight also increases.

Interactive Graph Example

This interactive graph allows you to hover over the points to see each of the x and y values. According to the linear regression model, as the gestation period increases, the mother’s pregnancy weight also increases.

Key Takeaways

  • In conclusion, data science and statistics are not just theoretical math concepts; they have practical applications in the analysis of data and building predictive models.

  • By understanding how variables relate to one another and how different modeling techniques behave, data scientists can develop more reliable and accurate models.

  • Techniques such as Linear regression allow researchers to model relationships within data.

  • In the medical field, it helps researchers discover relationships between variables such as age and health indicators, or dosage and treatment response.

  • These models can help save and extend human lives. This can alter how people live on a day-to-day basis, making it greatly impactful and important to understand.