Discussion 14

head(mtcars)

##                    mpg cyl disp  hp drat    wt  qsec vs am gear carb
## Mazda RX4         21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
## Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
## Datsun 710        22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
## Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
## Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2
## Valiant           18.1   6  225 105 2.76 3.460 20.22  1  0    3    1

mpg = \(\beta_0\) + \(\beta_1\) * hp + \(\beta_2\) * wt + \(\epsilon\)

#create new data frame that contains only the variables we would like to use to
data <- mtcars[ , c("mpg", "hp", "wt")]

#view first six rows of new data frame
head(data)

##                    mpg  hp    wt
## Mazda RX4         21.0 110 2.620
## Mazda RX4 Wag     21.0 110 2.875
## Datsun 710        22.8  93 2.320
## Hornet 4 Drive    21.4 110 3.215
## Hornet Sportabout 18.7 175 3.440
## Valiant           18.1 105 3.460

Let’s say we want to predict mpg (miles per gallon) using hp (horsepower) and wt (weight) as independent variables.

Estimate equation: mpg = 37.22727 - 0.03177 * hp - 3.87783 * wt

Where \(\beta_0\) = 37.22727 is the intercept, \(\beta_1\) = -0.03177 and \(\beta_2\) = -3.87783 are the coefficients for hp and wt and \(\epsilon\) is the error term.

# Run the Regression
model <- lm(mpg ~ hp + wt, data = mtcars)
summary(model)

## 
## Call:
## lm(formula = mpg ~ hp + wt, data = mtcars)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.941 -1.600 -0.182  1.050  5.854 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 37.22727    1.59879  23.285  < 2e-16 ***
## hp          -0.03177    0.00903  -3.519  0.00145 ** 
## wt          -3.87783    0.63273  -6.129 1.12e-06 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.593 on 29 degrees of freedom
## Multiple R-squared:  0.8268, Adjusted R-squared:  0.8148 
## F-statistic: 69.21 on 2 and 29 DF,  p-value: 9.109e-12

options(repos = c(CRAN = "https://cloud.r-project.org/"))
install.packages("stargazer")

## 
## The downloaded binary packages are in
##  /var/folders/gk/y8z5jt3n3m94l_plzxgdqbxc0000gn/T//RtmplRDrnA/downloaded_packages

library(stargazer)

## 
## Please cite as:

##  Hlavac, Marek (2022). stargazer: Well-Formatted Regression and Summary Statistics Tables.

##  R package version 5.2.3. https://CRAN.R-project.org/package=stargazer

stargazer(mtcars[c("mpg","hp","wt")], type = "text",
          title="Descriptive statistics/selected variables", digits=1, out="table.txt")

## 
## Descriptive statistics/selected variables
## =====================================
## Statistic N  Mean  St. Dev. Min  Max 
## -------------------------------------
## mpg       32 20.1    6.0    10.4 33.9
## hp        32 146.7   68.6    52  335 
## wt        32  3.2    1.0    1.5  5.4 
## -------------------------------------

pairs(data, pch = 18, col = "steelblue")

Coefficients:

1. Intercept (37.22727): This is where the regression line intersects the Y-axis when both hp and wt are zero. The high value suggests that, theoretically, if a car had no weight and no horsepower, it would have very high fuel efficiency, but this is a hypothetical scenario as such a car doesn’t exist.

2. Horsepower (hp, -0.03177): The negative coefficient indicates that as horsepower increases, the fuel efficiency (mpg) decreases, which aligns with expectations. The magnitude (-0.03177) implies that for each additional unit of horsepower, mpg decreases by about 0.03177 units, holding weight constant. This effect is statistically significant (p-value = 0.00145), meaning it’s unlikely to be due to random chance.

3. Weight (wt, -3.87783): The coefficient for weight is also negative and much larger in magnitude than for horsepower. This suggests a stronger relationship between weight and fuel efficiency. For each unit increase in weight, mpg decreases by about 3.87783 units, assuming horsepower remains constant. This relationship is highly significant (p-value = 1.12e-06).

Model Fit:

• Residual Standard Error (2.593): This value indicates the average amount by which the observed values deviate from those predicted by the model. A lower value generally indicates a better fit.

• R-squared (0.8268): This suggests that about 82.68% of the variability in mpg can be explained by the model. It’s a high value, indicating a good fit.

• Adjusted R-squared (0.8148): Adjusted for the number of predictors, it’s still quite high, supporting the model’s effectiveness.

• F-statistic (69.21): This is significantly large, and the associated p-value (9.109e-12) suggests the model is highly statistically significant.

Analysis of Residuals:

• Range of Residuals: The residuals range from -3.941 to 5.854. This range, along with the quartiles, can help in understanding the spread and skew of the errors. Residuals do not appear to be symmetrically distributed around zero, especially with a maximum of 5.854.

• Potential Outliers: The large maximum residual suggests that there might be outliers or specific data points where the model does not perform well.

Interpretation and Conclusion:

• The model effectively predicts mpg based on horsepower and weight, as indicated by high R-squared values and significant F-statistic.

• Both horsepower and weight negatively affect fuel efficiency, with weight having a more substantial impact.

• There might be outliers or leverage points that could be influencing the model, suggested by the range in the residuals.

• Residual analysis for normality, homoscedasticity, and independence would be necessary for more robust conclusions.

Overall, the model seems quite strong but further investigation into the residuals and potential outliers would be beneficial to ensure the model’s robustness and applicability.

The Gauss-Markov assumptions are a set of conditions that, when met, ensure that the Ordinary Least Squares (OLS) estimator is the Best Linear Unbiased Estimator (BLUE) for the coefficients in linear regression. While different sources may list a varying number of assumptions, they generally cover the same key concepts. Here are the main assumptions:

The Gauss-Markov assumptions are a set of conditions that, when met, ensure that the Ordinary Least Squares (OLS) estimator is the Best Linear Unbiased Estimator (BLUE) for the coefficients in linear regression. While different sources may list a varying number of assumptions, they generally cover the same key concepts. Here are the main assumptions:

1. Linearity in Parameters: The relationship between the dependent and independent variables is linear in the parameters. It means the model can be expressed as a linear combination of the coefficients.

2. Random Sampling: The data used in the regression is assumed to be a random sample from the population. This is crucial for the generalizability of the results.

3. No Perfect Multicollinearity: There should not be perfect linear relationships among the independent variables. This means no independent variable is a perfect linear function of other independent variables.

4. Zero Conditional Mean of Errors: The expected value of the error term (%\epsilon%) is zero given any value of the independent variables. This assumption is crucial as it ensures that the error term does not systematically vary with the independent variables.

5. Homoscedasticity: The error terms have constant variance (homoscedasticity) for different values of the independent variables. If the variance of the errors is non-constant (heteroscedasticity), it can lead to inefficiency in the estimates and biased standard error estimates.

6. No Autocorrelation of Errors: There is no correlation between the error terms. This is particularly important in time series data where one observation might be correlated with previous observations.

7. Normality of Errors (Optional): In small sample sizes, for hypothesis testing, the error terms should be normally distributed. However, for large samples, due to the Central Limit Theorem, this assumption is not crucial for the validity of OLS estimates but is important for creating confidence intervals and conducting hypothesis tests.

Why use log of a variable?

Taking the logarithm of a variable in linear regression is a common transformation with several benefits, especially in certain types of datasets and analysis scenarios. The key reasons to apply logarithmic transformation includes, non-linear relationships, Heteroscedasticity (help stabilize the variance of residuals), reducing skewness, interpretation in terms of percentages, handling large range of values…