SOC 712: Homework 3

Jennifer Ganeles
2/24/19

Comparing MLE Results: Slide 4.15 vs. Slide 4.19

Using maximum likelihood estimation, both Slide 4.15 and Slide 4.19 show the relationship between education (independent variable) and income (dependent variable) using the “turnout” dataset from the Zelig package. However, Slide 4.15 maximizes the likelihood estimate of mean income (mu), while Slide 4.19 estimates the standard deviation (sigma) of income (so estimated parameters are different). In other words, Slide 4.15 looks at the effect of education on average income (the more common appraoch) while Slide 4.19 looks at the effect of education on the variation of income, or income inequality (less commonly used, but nevertheless useful in this example).

Relationship Between Education and Mean Income (Slide 4.15)

The results on Slide 4.15 use a linear regression model of Y∼N(β0+β1X,σ), in which beta1 represents the y-intercept for mu and beta2 represents the slope for mu. As shown below, a beta1 (y-intercept) of -.65 means that those who have no education (x=0) have an estimated mean income of -.65. A beta2 (slope) of .38 means that an increase in education by one year will have an estimated increase in mean income by .38 units. This result is statistically significant, and suggests a positive correlation between education and income, where those with higher education will also have a higher mean income.

library(Zelig)
data("turnout")
#Looking at the relationship between education and mean income
ols.lf<-function(param) {
  beta<-param[-1]
  sigma<-param[1]
  y<-as.vector(turnout$income)
  x<-cbind(1, turnout$educate)
  mu<-x%*%beta
  sum(dnorm(y, mu, sigma, log=TRUE))
}
library(maxLik)
mle_ols<-maxLik(logLik=ols.lf, start=c(sigma=1, beta1=1, beta2=1))
summary(mle_ols)

--------------------------------------------
Maximum Likelihood estimation
Newton-Raphson maximisation, 12 iterations
Return code 2: successive function values within tolerance limit
Log-Likelihood: -4691.256 
3  free parameters
Estimates:
      Estimate Std. error t value Pr(> t)    
sigma  2.52613    0.03989  63.326 < 2e-16 ***
beta1 -0.65207    0.20827  -3.131 0.00174 ** 
beta2  0.37613    0.01663  22.612 < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
--------------------------------------------

Checking Results:

summary(lm(income~educate,data=turnout))

Call:
lm(formula = income ~ educate, data = turnout)

Residuals:
    Min      1Q  Median      3Q     Max 
-6.2028 -1.7363 -0.4273  1.3150 11.0632 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.65207    0.21016  -3.103  0.00194 ** 
educate      0.37613    0.01677  22.422  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.527 on 1998 degrees of freedom
Multiple R-squared:  0.201, Adjusted R-squared:  0.2006 
F-statistic: 502.8 on 1 and 1998 DF,  p-value: < 2.2e-16

Relationship Between Education and Income Inequality (Slide 4.19)

The results on Slide 4.19 use a linear regression model of Y∼N(μ,θ0+θ1X, in which theta1 represents the y-intercept for sigma and theta2 represents the slope for sigma. A theta1 (y-intercept) of 1.46 indicates that those with no education (x=0) have an estimated standard deviation of income of 1.46. A theta2 (slope) of .109 means that an increase in education by one year will have an estimated increase in the standard deviation of income by .109 units. These results are also statistically significant and suggest a positive correlation between education and income inequality, in which those with higher education will also see more variation in income.

#Looking at the relationship between education and standard deviation of income (i.e. income inequality)
ols.lf2<-function(param){
  mu<-param[1]
  theta<-param[-1]
  y<-as.vector(turnout$income)
  x<-cbind(1, turnout$educate)
  sigma<-x%*%theta
  sum(dnorm(y,mu,sigma,log=TRUE))
}
library(maxLik)
mle_ols2<-maxLik(logLik=ols.lf2, start=c(mu=1, theta1=1, theta2=1))
summary(mle_ols2)

--------------------------------------------
Maximum Likelihood estimation
Newton-Raphson maximisation, 9 iterations
Return code 2: successive function values within tolerance limit
Log-Likelihood: -4861.964 
3  free parameters
Estimates:
       Estimate Std. error t value Pr(> t)    
mu     3.516764   0.070320   50.01  <2e-16 ***
theta1 1.461011   0.106745   13.69  <2e-16 ***
theta2 0.109081   0.009185   11.88  <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
--------------------------------------------

Introducing Age as a Second Independent Variable for Both Models

If we introduce age as another independent variable, it can be hypothesized that age and mean income will be positively correlated since people become more educated as they get older (and as we just saw, higher education translates to higher mean income). However, given that people eventually retire and earn less as they grow older, it is also possible that age negatively correlates with mean income after a certain age is reached. The following results show an intercept of -.446, which refers to a mean income of -.446 units when education and age equal zero. The results also suggest a negative correlatation between age and mean income (beta3=-.003), implying that an increase in age by one year will decrease mean income by .003 units. However, it is important to note that this relationship is insignificant. Meanwhile, the slope for education is similar to our previous results, showing a statistically significant increase in mean income by .37 (beta2=.37) as education increases by one year. Education, therefore, seems to be a better predictor for mean income than age.

When it comes to age and income inequality, I would hypothesize that age and the standard deviation of income would be positively correlated. Indeed, the results show an intercept of .36 and a positive correlation between age and variation of income (theta3=.018). The slope for age implies that as age increases by one year, the standard deviation of income increases by .018 units. This relationship is statistically significant and suggests that as age increases, so does income inequality. These results also show a statistically significant positive correlation between education and variation of income, with a slightly higher slope for education than our previous result exhibited (theta2=.133). As one can see, income inequality positively correlates with both increases in education and increases in age.

Adding Age to Model 1:

#Looking at the relationship between age and education and mean income
ols.lf3<-function(param) {
  beta<-param[-1]
  sigma<-param[1]
  y<-as.vector(turnout$income)
  x<-cbind(1, turnout$educate, turnout$age)
  mu<-x%*%beta
  sum(dnorm(y, mu, sigma, log=TRUE))
}
mle_ols3<-maxLik(logLik=ols.lf3, start=c(sigma=1, beta1=1, beta2=1, beta3=1))
summary(mle_ols3)

--------------------------------------------
Maximum Likelihood estimation
Newton-Raphson maximisation, 16 iterations
Return code 2: successive function values within tolerance limit
Log-Likelihood: -4690.815 
4  free parameters
Estimates:
       Estimate Std. error t value Pr(> t)    
sigma  2.525576   0.039919  63.268  <2e-16 ***
beta1 -0.446047   0.300583  -1.484   0.138    
beta2  0.371011   0.017493  21.209  <2e-16 ***
beta3 -0.003184   0.003373  -0.944   0.345    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
--------------------------------------------

Checking Results:

#Checking Results
summary(lm(income~educate+age,data=turnout))

Call:
lm(formula = income ~ educate + age, data = turnout)

Residuals:
    Min      1Q  Median      3Q     Max 
-6.2128 -1.7471 -0.4217  1.3042 11.1256 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.446084   0.303955  -1.468    0.142    
educate      0.371013   0.017641  21.031   <2e-16 ***
age         -0.003183   0.003394  -0.938    0.348    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.527 on 1997 degrees of freedom
Multiple R-squared:  0.2014,    Adjusted R-squared:  0.2006 
F-statistic: 251.8 on 2 and 1997 DF,  p-value: < 2.2e-16

Adding Age to Model 2:

#Looking at the relationship between age and educaton and standard deviation of income (income inequality)
ols.lf4<-function(param){
  mu<-param[1]
  theta<-param[-1]
  y<-as.vector(turnout$income)
  x<-cbind(1, turnout$educate, turnout$age)
  sigma<-x%*%theta
  sum(dnorm(y, mu, sigma, log=TRUE))
}
mle_ols4<-maxLik(logLik=ols.lf4, start=c(mu=1, theta1=1, theta2=1, theta3=1), method="bfgs")
summary(mle_ols4)

--------------------------------------------
Maximum Likelihood estimation
BFGS maximization, 150 iterations
Return code 0: successful convergence 
Log-Likelihood: -4843.15 
4  free parameters
Estimates:
       Estimate Std. error t value  Pr(> t)    
mu     3.555011   0.069193  51.378  < 2e-16 ***
theta1 0.362114   0.204550   1.770   0.0767 .  
theta2 0.133349   0.010756  12.398  < 2e-16 ***
theta3 0.017507   0.002852   6.139 8.32e-10 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
--------------------------------------------

Plotting Age and Income Variables

As hypothesized above, conditional mean income appears to increase with age until about age 50, where it then begins to decline.

library(ggplot2)
ggplot(turnout)+
  geom_point(aes(x = age, y = income)) + geom_smooth(aes(x = age, y = income))