2024-03-21

Introduction to Simple Linear Regression

  • Simple linear regression is a statistical method used to model the relationship between two continuous variables, typically denoted as \(X\) (the independent variable) and \(Y\) (the dependent variable).

  • It assumes that this relationship can be approximated by a straight line, where changes in \(X\) cause changes in \(Y\).

  • The goal of simple linear regression is to find the best-fitting line through the data points, allowing us to predict the value of \(Y\) for any given value of \(X\).

Mathematical Formula for Simple Linear Regression

The formula for Simple Linear Regression is:

\[Y = \beta_0 + \beta_1 X\] Where:

  • \(Y\) is the dependent variable (variable we want to predict).
  • \(X\) is the independent variable (the variable used to make predictions).
  • \(\beta_0\) is the intercept, representing the value of \(Y\) when \(X\) = 0.
  • \(\beta_1\) is the slope, representing the change in \(Y\) for a one-unit change in \(X\).

Data Set Example

Using a Startup Data Set from Kaggle, we have 50 different startups from different states. We are trying to view how each variable aspect of the startup correlates with the company’s profits.

In the dataset provided, there are 4 variable columns:

  • R&D Spend: This quantiative variable shows how much the startup spends on Research and Development.
  • Administration: This quantitative variable shows how much the startup spends on Administration for their company.
  • Marketing Spend: This quantitative variable shows how much the startup spends on the Marketing for their company.
  • State: This categorical variable shows the location of the startup.

There is 1 output column:

  • Profit: This quantitative variable shows how much profit the startup has made.

In terms of this dataset, we will be viewing the relationship/ correlation of the quantitative variables (R&D Spending, Administration, Marketing Spending) with the profits.

Code with ggplot (R&D with Profit)

In this part, we will look at the correlation between the startup’s spending with Research & Development and the company’s Profit.

  • The \(X\) value will represent the startup’s Research & Development spending.
  • The \(Y\) value will represent the startup’s Profit.

Code:

g <- ggplot(df, aes(x = R.D.Spend, y = Profit)) + 
     geom_point() + 
     geom_smooth(method="lm", level=0.99) + 
     xlab("Research and Development Spending (In Dollars)") + 
     ylab("Profit (In Dollars)") + 
     theme(axis.text.x = element_text(size = 12), 
           axis.text.y = element_text(size = 12))

Graph with ggplot (R&D with Profit)

Relationship Between R&D and Profit

Correlation between Research & Development and Profit is represented by the value:

## [1] 0.9729005

As the value is very close to 1, it indicates that this relationship between Research & Development and Profit has a very strong linear correlation. In regards to the linear model, we can extract the intercept and slope values from the line plotted on the graph before.

\(y = 0.85x + 49032.90\)

In this equation of the line, we can see that the value before the \(x\) indicates the slope: \(\beta_1\). The constant at the end of the equation represents \(\beta_0\), or the intercept of the line.

We can predict the Profits with $150000 in R&D Spending using the Linear model:

predict(linearModel, data.frame(R.D.Spend=150000))
##        1 
## 177176.6

Therefore, with $150000 in Research & Development spending, we can predict the company profits to be around $177176.6

Code with ggplot (Administration & Profit)

In this part, we will look at the correlation between the startup’s spending with Administrative Spending and the company’s Profit.

  • The \(X\) value will represent the startup’s Administrative spending.
  • The \(Y\) value will represent the startup’s Profit.

Code:

g <- ggplot(df, aes(x = Administration, y = Profit)) + 
     geom_point() + 
     geom_smooth(method="lm", formula = y ~ x) + 
     xlab("Administration Spending ($)") + 
     ylab("Profit ($)") + 
     theme(axis.text.x = element_text(size = 12), 
           axis.text.y = element_text(size = 12))

Graph with ggplot (Administration & Profit)

Relationship Between Administration and Profit

Correlation between Administrative Spending and Profit is represented by the value:

## [1] 0.2007166

Based on this value, the value is on the positive side but closer to 0, meaning that there is very little positive correlation between the Administration Spending and Profits. Therefore, a linear model would not represent this relationship the best.

Code with plotly (Marketing Spending & Profit)

In this part, we will look at the correlation between the startup’s spending with Marketing Spending and the company’s Profit.

  • The \(X\) value will represent the startup’s Marketing Spending.
  • The \(Y\) value will represent the startup’s Profit.

Code:

linearModel <- lm(Profit ~ Marketing.Spend, data = df)

xax <- list (title = "Marketing Spend ($)")
yax <- list (title = "Profit ($)")

g <- plot_ly(x=df$Marketing.Spend, 
             y=df$Profit, 
             type="scatter", 
             mode="markers", 
             name = "Marketing Spend / Profit",
             width=690, 
             height=300) %>%
  add_lines(x = df$Marketing.Spend, 
            y = predict(linearModel), 
            name = "Linear Fit") %>%
  layout(xaxis = xax, yaxis = yax) %>%
  layout(margin=list(
    l=150,
    r=50,
    b=20,
    t=40
  )
  ) %>%
  layout (title = "Marketing Spend vs. Profit")

Graph with ggplot (Marketing Spending & Profit)

Relationship Between Marketing Spending and Profit

Correlation between Marketing Spending and Profit is represented by the value:

## [1] 0.7477657

This value is on the positive side and closer to 1, which indicates that there is a strong linear relationship between Marketing Spend and Profit.

In regards to the linear model, we can extract the intercept and slope values from the line plotted on the graph before.

\(y = 0.25x + 60003.55\)

In this equation of the line, we can see that the value before the \(x\) indicates the slope: \(\beta_1\). The constant at the end of the equation represents \(\beta_0\), or the intercept of the line. We can predict the Profits with $300,000 in Marketing Spending using the Linear model:

predict(linearModel, data.frame(Marketing.Spend=300000))
##        1 
## 133941.3

Therefore, with $300000 in Marketing Spending, we can predict the company profits to be around $1133941.3