1. I chose the penguins data set. I picked the flipper length and body mass as the quantitative variables.
data("penguins")
summary(penguins)
##       species          island       bill_len        bill_dep    
##  Adelie   :152   Biscoe   :168   Min.   :32.10   Min.   :13.10  
##  Chinstrap: 68   Dream    :124   1st Qu.:39.23   1st Qu.:15.60  
##  Gentoo   :124   Torgersen: 52   Median :44.45   Median :17.30  
##                                  Mean   :43.92   Mean   :17.15  
##                                  3rd Qu.:48.50   3rd Qu.:18.70  
##                                  Max.   :59.60   Max.   :21.50  
##                                  NAs    :2       NAs    :2      
##   flipper_len      body_mass        sex           year     
##  Min.   :172.0   Min.   :2700   female:165   Min.   :2007  
##  1st Qu.:190.0   1st Qu.:3550   male  :168   1st Qu.:2007  
##  Median :197.0   Median :4050   NAs   : 11   Median :2008  
##  Mean   :200.9   Mean   :4202                Mean   :2008  
##  3rd Qu.:213.0   3rd Qu.:4750                3rd Qu.:2009  
##  Max.   :231.0   Max.   :6300                Max.   :2009  
##  NAs    :2       NAs    :2

A. Independent and dependent variables independent variable (x) - flipper length, measured in millimeters dependent variable (y) - body mass, measured in grams

I want to investigate whether penguins with longer flippers tend to have greater body mass.

Estimating equation

\[ y_i = \beta_0 + \beta_1 x_i + \varepsilon_i \]
where:

\(y_i\) - body mass of penguin \(i\)(grams)

\(x_i\) - flipper length of penguin \(i\)(millimeters)

\(\beta_0\) - the intercept

\(\beta_1\) - the slope

\(\varepsilon_i\) - random error term for penguin \(i\)

B. Linear Regression

model <- lm(body_mass ~ flipper_len,data = penguins)
plot(model)

summary(model)
## 
## Call:
## lm(formula = body_mass ~ flipper_len, data = penguins)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1058.80  -259.27   -26.88   247.33  1288.69 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -5780.831    305.815  -18.90   <2e-16 ***
## flipper_len    49.686      1.518   32.72   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 394.3 on 340 degrees of freedom
##   (2 observations deleted due to missingness)
## Multiple R-squared:  0.759,  Adjusted R-squared:  0.7583 
## F-statistic:  1071 on 1 and 340 DF,  p-value: < 2.2e-16

C. Interpret the slope and intercept parameters

Based on the model summary, the slope is 49.686, and the intercept is -5780.831.

Slope - for every increase in 1mm in flipper length, the predicted body mass will increase by approximately 49.686 grams on average.

Intercept - When a penguin’s flipper length is 0mm, the predicted body mass is -5780.831 grams. That is not possible because a penguin cannot have a flipper length of 0 mm. This value is outside the observed range of the dataset.

D. slope and intercept parameter using the covariance/variance formulas

# Slope
b1 <- cov(penguins$flipper_len,penguins$body_mass,use = "complete.obs") / var(penguins$flipper_len,na.rm = TRUE)
round(b1,3)
## [1] 49.686
# Intercept
b0 <- mean(penguins$body_mass,na.rm = TRUE) - b1 * mean(penguins$flipper_len,na.rm = TRUE)
round(b0,3)
## [1] -5780.831
  1. Ordinary Least Squares (OLS)

Ordinary Least Squares regression is a common technique for estimating coefficients of linear regression equations which describe the relationship between one or more independent quantitative variables and one dependent variable. It is evaluated using r-squared. R-squared measures how much of the variation in the dependent variable is explained by the model. The Gauss-Markov Theorem states that when certain conditions are satisfied, the OLS is BLUE(Best Linear Unbiased Estimator), meaning that it has the smallest variance among linear unbiased estimators.

The four conditions of linear regression are:

1. Linearity - the relationship between the independent and dependent variables should be approximately linear.

2. Normality of residuals - the residuals should be approximately normally distributed. Outliers can influence the regression line.

3. Constant Variance (Homoscedasticity) - the spread of the residuals should remain roughly constant across different values of the independent variable.

4. Independence - observations should be independent of one another. They cannot influence each other.

Another important assumption for the Gauss-Markov Theorem is exogeneity, which means that the independent variables should not be systematically related to the error term. If this assumption is not met, the OLS estimates might be biased. Furthermore, if the constant variance assumption is violated, OLS may no longer be the most efficient linear unbiased estimator.

The result of applying the OLS is the Least Square Line. It is the line that minimizes the sum of the squared residuals.

The slope of the this line can be obtained by this formula

\[ b_1 = r \frac{s_y}{s_x} \]

where r is the correlation between the values.\(s_y\) the standard deviation of the dependent variable, y, and \(s_x\) is the standard deviation of the independent variable, x.

Furthermore, if \(\bar{x}\) is the sample mean of the independent variable and \(\bar{y}\) is the sample mean of the dependent variable, then the point (\(\bar{x}\),\(\bar{y}\)) is on the least square regression line. this can be useful when writing the point-slope equation for the line.