Linear Models & Log-Likelihood Function

Voter Turnout Data

library(Zelig)

Loading required package: survival

data(turnout)
head(turnout)

The Effect of Education on Income (Results on Slide 4.14)

ols.lf <- function(param) {
  beta <- param[-1]
  sigma <- param[1]
  y <- as.vector(turnout$income)
  x <- cbind(1, turnout$educate)
  mu <- x%*%beta
  sum(dnorm(y, mu, sigma, log=TRUE))
}
library(maxLik)
mle_ols <- maxLik(logLik= ols.lf, start=c(sigma=1, beta=1, beta2=1))
summary(mle_ols)

--------------------------------------------
Maximum Likelihood estimation
Newton-Raphson maximisation, 12 iterations
Return code 2: successive function values within tolerance limit
Log-Likelihood: -4691.256 
3  free parameters
Estimates:
      Estimate Std. error t value Pr(> t)    
sigma  2.52613    0.03989  63.326 < 2e-16 ***
beta  -0.65207    0.20827  -3.131 0.00174 ** 
beta2  0.37613    0.01663  22.612 < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
--------------------------------------------

The results on slide 4.14 maximizes the log-likelihood function which shows the effect of education on voters income from the “turnout” data. Education is our independent variable and income is our dependent variable. The three parameters in this result are sigma, beta1 and beta2. The standard deviation is 2.5 (shows variation). Beta1 represents the y-intercept, in other words, voters who have no education have an income of -0.65. Beta2 is the slope which shows the effect education has on income, one unit increase in education has a 0.38 increase on income. The results demonstrate that education and income are positively correlated. Voters who report a higher level of education will have a high level of income. The results for both beta1 and beta2 estimates are statistically significant, p<.05.

The Effect of Education on the Standard Deviation of Income

ols.lf2 <- function(param) {
  mu <- param[1]
  theta <- param[-1]
  y <- as.vector(turnout$income)
  x <- cbind(1, turnout$educate)
  sigma <- x%*%theta
  sum(dnorm(y, mu, sigma, log = TRUE))
}    
library(maxLik)
mle_ols2 <- maxLik(logLik = ols.lf2, start = c(mu = 1, theta1 = 1, theta2 = 1))
summary(mle_ols2)

--------------------------------------------
Maximum Likelihood estimation
Newton-Raphson maximisation, 9 iterations
Return code 2: successive function values within tolerance limit
Log-Likelihood: -4861.964 
3  free parameters
Estimates:
       Estimate Std. error t value Pr(> t)    
mu     3.516764   0.070320   50.01  <2e-16 ***
theta1 1.461011   0.106745   13.69  <2e-16 ***
theta2 0.109081   0.009185   11.88  <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
--------------------------------------------

```

The results on slide 4.18 show the effect education has on the variability of income. The three parameters in this result are mu, theta1 and theta2. Theta1 represents the y-intercept, voters with no education have a 1.46 variability on income. Theta2 (slope) represents the effect education has on the standard deviation on income, one unit increase in education results in an increase of 0.11 increase on the variability of income. We can see the results show that the average income is 3.52. Results for both theta1 and theta2 are statistically significant, p<0.05.

The Effect of Age on Income

For both models, we can see that the standard error (the difference between our observed and actual value) decreases as new variables are introduced. If we were to introduce a new independent variable “age” to our linear models, I would predict that the slope and standard error of the new variable would decrease because age and education can possibly be interrelated. As age increases, our education level increases as well. I think the relationship between age and income will be positively correlated in the first model. I would also predict age to have a similar effect on the variability of income as education, however, adding a new independent variable would have an intricate effect on the variability on income. That is because as education level increases, the variability on income starts to widen. It would be difficult to pinpoint voter’s variability on income as they obtain the highest level of education. If we think in terms of what this means for the real world, people with the highest level of education, phD do not necessairly have a high income because there are other intervening variables in the model we need to consider. There are people with high level education who have low income jobs.