Using the maximum likelihood estimations presented on slides 4.15 and 4.19, I will be examining the relationship between education and income (where education is the independent variable and income is the dependent variable) using the Zelig package.
library(maxLik)
library(Zelig)
data("turnout")ols.lf <- function(param) {
beta <- param[-1]
sigma <- param[1]
y <- as.vector(turnout$income)
x <- cbind(1, turnout$educate)
mu <- x%*%beta
sum(dnorm(y, mu, sigma, log = TRUE))}mle_ols <- maxLik(logLik = ols.lf, start = c(sigma = 1, beta1 = 1, beta2 = 1))
summary(mle_ols)## --------------------------------------------
## Maximum Likelihood estimation
## Newton-Raphson maximisation, 12 iterations
## Return code 2: successive function values within tolerance limit
## Log-Likelihood: -4691.256
## 3 free parameters
## Estimates:
## Estimate Std. error t value Pr(> t)
## sigma 2.52613 0.03989 63.326 < 2e-16 ***
## beta1 -0.65207 0.20827 -3.131 0.00174 **
## beta2 0.37613 0.01663 22.612 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## --------------------------------------------summary(lm(income ~ educate, data = turnout))##
## Call:
## lm(formula = income ~ educate, data = turnout)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.2028 -1.7363 -0.4273 1.3150 11.0632
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.65207 0.21016 -3.103 0.00194 **
## educate 0.37613 0.01677 22.422 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.527 on 1998 degrees of freedom
## Multiple R-squared: 0.201, Adjusted R-squared: 0.2006
## F-statistic: 502.8 on 1 and 1998 DF, p-value: < 2.2e-16The intercept (beta1) = for people who have 0 years of education, their average income = -.65 units. For the slope (beta2), when education (beta2) goes up by 1 year, average income increases by .376 units, showing that there’s a positive relationship between having high education and high average income. Sigma=2.52613 is our residual standard error of this model.
ols.lf2 <- function(param) {
mu <- param[1]
theta <- param[-1]
y <- as.vector(turnout$income)
x <- cbind(1, turnout$educate)
sigma <- x%*%theta
sum(dnorm(y, mu, sigma, log = TRUE))
} mle_ols2 <- maxLik(logLik = ols.lf2, start = c(mu = 1, theta1 = 1, theta2 = 1))
summary(mle_ols2)## --------------------------------------------
## Maximum Likelihood estimation
## Newton-Raphson maximisation, 9 iterations
## Return code 2: successive function values within tolerance limit
## Log-Likelihood: -4861.964
## 3 free parameters
## Estimates:
## Estimate Std. error t value Pr(> t)
## mu 3.516764 0.070320 50.01 <2e-16 ***
## theta1 1.461011 0.106745 13.69 <2e-16 ***
## theta2 0.109081 0.009185 11.88 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## --------------------------------------------Mean error is mu=3.5. Theta1 = 1.461011 = intercept: for people with no education (education=0), have an estimated standard deviation in income of 1.46. Theta2 = 0.109081 = slope = an increase in education by 1 year will increase the standard deviation of income by .109 units, Does education increase or decrease increase income inequality? Education increases income inequality (since there’s a positive relationship with higher education and higher income).
ols.lfage1 <- function(param) {
beta <- param[-1]
sigma <- param[1]
y <- as.vector(turnout$income)
x <- cbind(1, turnout$educate,turnout$age)
mu <- x%*%beta
sum(dnorm(y, mu, sigma, log = TRUE))}mle_olsage1 <- maxLik(logLik = ols.lfage1, start = c(sigma = 1, beta1 = 1, beta2 = 1, beta3=1))
summary(mle_olsage1)## --------------------------------------------
## Maximum Likelihood estimation
## Newton-Raphson maximisation, 16 iterations
## Return code 2: successive function values within tolerance limit
## Log-Likelihood: -4690.815
## 4 free parameters
## Estimates:
## Estimate Std. error t value Pr(> t)
## sigma 2.525576 0.039919 63.268 <2e-16 ***
## beta1 -0.446047 0.300583 -1.484 0.138
## beta2 0.371011 0.017493 21.209 <2e-16 ***
## beta3 -0.003184 0.003373 -0.944 0.345
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## --------------------------------------------summary(lm(income~educate+age,data=turnout))##
## Call:
## lm(formula = income ~ educate + age, data = turnout)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.2128 -1.7471 -0.4217 1.3042 11.1256
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.446084 0.303955 -1.468 0.142
## educate 0.371013 0.017641 21.031 <2e-16 ***
## age -0.003183 0.003394 -0.938 0.348
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.527 on 1997 degrees of freedom
## Multiple R-squared: 0.2014, Adjusted R-squared: 0.2006
## F-statistic: 251.8 on 2 and 1997 DF, p-value: < 2.2e-16The intercept (beta1) = for people who have 0 years of education and 0 years of age, their average income = -.446 units. For the slope (beta2), when education (beta2) goes up by 1 year, income increases on average by .371 units, similar to the results above before adding age (when everything else is held constant). When age (beta3) goes up by 1 year, income decreases on average by .00318 units, showing a negative relationship between age and average income (when everything else is held constant). 2.52 is our residual standard error of this model, similar to the results found above (before adding age). There’s also no statistical significance of adding age, and education isn’t biased.
ols.lfage2 <- function(param) {
mu <- param[1]
theta <- param[-1]
y <- as.vector(turnout$income)
x <- cbind(1, turnout$educate,turnout$age)
sigma <- x%*%theta
sum(dnorm(y, mu, sigma, log = TRUE))
} mle_olsage2 <- maxLik(logLik = ols.lfage2, start = c(mu = 1, theta1 = 1, theta2 = 1,theta3=1),method="bfgs")
summary(mle_olsage2)## --------------------------------------------
## Maximum Likelihood estimation
## BFGS maximization, 150 iterations
## Return code 0: successful convergence
## Log-Likelihood: -4843.15
## 4 free parameters
## Estimates:
## Estimate Std. error t value Pr(> t)
## mu 3.555011 0.069193 51.378 < 2e-16 ***
## theta1 0.362114 0.204550 1.770 0.0767 .
## theta2 0.133349 0.010756 12.398 < 2e-16 ***
## theta3 0.017507 0.002852 6.139 8.32e-10 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## --------------------------------------------Residual mean error is mu=1.0033. Theta1 = 0.9810 = intercept: people with 0 education and are age 0 have a standard deviation in income of 0.9810 units. Theta2 = 0.7662 = slope for education = an increase in education by 1 year will increase the standard deviation of income by 0.7662 units (when everything else is held constant). Theta3 = 0.1531 = slope for age = an increase in age by 1 year will increase the standard deviation of income by 0.1531 units (when everything else is held constant). Since education and age are both positive numbers, this shows a positive relationship between income inequality and a rise in education and age.