Midterm project: Do Unions Raise Wages for Young Men?
Daniella Roberts
Logistics
- DEADLINE: The midterm is due before Sunday, April 5th, 23:59. Penalty for late work is -2 points per hour late.
- You can do this homework in groups of two at most.
- This homework has a total of 25 points. The number of points for each section is clearly indicated in the section and/or subsection headings. Note that this midterm contains two types of exercises. On the one hand, you will be asked to write lines of code to produce some output (e.g., tables, plots, etc.). On the other hand, you will be asked to answer specific questions (interpret your output results, give intuition, etc.). If you want to get full points, you should not ignore any of them. Write your answers succinctly, that means, be precise and give short answers!
- You need to hand in this
.Rmdfile on Moodle together with the.htmlfile - that is produced when you click on the “Knit” button. If you are having issues knitting your file, do get in touch (and knit regularly to make sure everything is working properly). Your files must be named “midterm_metrics_name1_name2.Rmd” if you are in group or midterm_metrics_name1.Rmd” otherwise. - We left code blocks in this
.Rmdfile for you to fill out. Just put your code within the marked region, and click the Knit button in RStudio, just above this script, to see the generated HTML output. Knit it now, before you add any code to see how nice the output looks like. - Whenever we give you a chunk of code with code already written, you
need to replace
eval = FALSEbyeval = TRUEin the chunk “header”. This will make sure that the chunk is ‘evaluated’ (i.e., that the code runs) when you knit the document. - FYI a code block looks like this:
# code goes here- For each question you will see blocks like the above where you are supposed to write your code. Also, you will see Your answer goes here, where you can explain your thoughts in English. These are just referential. You are allowed to structure your answers differently as long as you convey them clearly.
- Do NOT delete the ```{r} and ```; they separate the code chunk from text.
- You can find a nice intro to Rmarkdown (
.Rmd) here.
Do unions actually raise wages for young men?
In this midterm project, you will attempt to answer this question,
using a dataset collected by researchers and made available through the
wooldridge package. This dataset contains information on
545 young men in the American workforce, from years 1980 to 1987. This
time period coincides with declining unionization in the United
States.
Unions have historically played a key part in labor organization, both in the U.S. and in Europe. However, union density has decreased across the board in all Western countries, and some researchers have suggested that their declining influence could play a role in the rise of income inequality. Unions face critical challenges in globalized economies: the threat of job loss, relocation of activity abroad and foreign competitors all create additional constraints.
However, unions have also been accused of fostering inefficiencies and rents for employees, in the same way market power creates inefficiencies in resource allocation. Indeed, if the unions are able to effectively exert bargaining power, we should observe higher wages in sectors that are extensively covered by collective bargaining. There exists a wealth of research that explores the complex union activity structures, and how these influence overall income inequality, labor relations, and productivity.
In this midterm, you will tackle the most basic question one can ask about unions :
Do they effectively raise the wages of the workers for whom they bargain ?
You will use everything we’ve learned so far: importing data, summarizing, visualizing and tidying it; running regressions using different variables; and thinking about whether these associations can be interpreted causally. Specifically, by the end of this midterm, you should be able to acknowledge how difficult it is to reach a definitive conclusion about causality. You will witness this by attempting to model conditional expectations properly, which is a necessary condition in any causal analysis.
We hope you find this project interesting and useful. We hope you learn a lot in the process. Now let’s get going!
Loading packages
It’s good practice to load all the packages you will need in the same place at the beginning of the script.
(Once packages are installed, remember to set
eval = FALSE to eval = TRUE in the chunk
below.)
# Load all the packages you need here. Careful : you will need to install some of those first.
# Don't write the install.packages() code here. Run that code in the console.
library(tidyverse)
library(AER)
library(stargazer)
library(dplyr)
library(ggplot2)
library(jtools)
library(fixest)Loading the Data [0.5 points]
You first need to understand the data and the information contained in it.
- Load
wage_unions.csvinto RStudio in an object calleddf_raw. The file has been sent to you by email, along with a readME file, that contains information about the variables.
You might want to use the read.csv function.
# Code here
df_raw <- read.csv("wage_unions.csv")Understanding what the dataset contains [1 point]
First, we need to understand what the dataset contains.
Let’s get a first sense of what the dataset looks like !
1. Display the last 10 rows of the dataset
# Code here
tail(df_raw, 10)2. Which years are included in our data? How many men do we observe
each year ? Tabulate the two variables year and
nr to find out.
# Code here
table(df_raw$year)##
## 1980 1981 1982 1983 1984 1985 1986 1987
## 545 545 545 545 545 545 545 545table(df_raw$year, df_raw$nr)##
## 13 17 18 45 110 120 126 150 162 166 189 193 209 212 218 243 259 260 309
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 351 353 383 408 424 464 483 556 560 569 583 647 658 684 711 729 731 732
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 793 797 800 813 823 827 847 851 863 873 891 908 910 916 919 922 924 925
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 944 945 947 955 965 996 1007 1054 1064 1081 1085 1091 1094 1096 1098
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 1102 1107 1142 1156 1180 1190 1204 1249 1272 1311 1316 1318 1345 1397
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 1434 1492 1496 1506 1515 1520 1528 1554 1575 1576 1628 1641 1644 1653
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 1654 1721 1742 1744 1763 1777 1843 1891 1895 1899 1925 1930 1961 1963
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 1979 1988 2000 2014 2025 2038 2075 2101 2106 2107 2108 2147 2157 2163
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 2173 2180 2183 2216 2220 2227 2264 2306 2312 2314 2329 2335 2341 2351
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 2386 2401 2413 2421 2445 2451 2494 2508 2535 2540 2685 2711 2718 2721
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 2733 2741 2745 2751 2774 2801 2813 2833 2839 2842 2866 2868 2874 2916
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 2951 2980 2994 2997 3017 3037 3059 3062 3100 3102 3127 3136 3137 3138
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 3140 3193 3196 3200 3202 3207 3208 3210 3215 3219 3226 3235 3239 3271
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 3275 3282 3289 3290 3307 3333 3353 3380 3381 3389 3394 3401 3414 3420
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 3440 3461 3468 3482 3495 3503 3525 3526 3538 3563 3575 3580 3581 3589
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 3591 3598 3602 3607 3621 3628 3653 3706 3707 3708 3743 3777 3831 3844
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 3847 3848 3882 3937 4000 4004 4025 4032 4046 4088 4091 4122 4127 4128
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 4159 4204 4229 4258 4261 4264 4278 4297 4302 4321 4328 4332 4335 4357
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 4365 4380 4394 4510 4559 4563 4569 4603 4607 4633 4676 4701 4716 4720
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 4759 4791 4811 4828 4857 4858 4859 4866 4881 4884 4888 4901 4917 4926
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 4982 5017 5033 5048 5122 5141 5147 5158 5221 5223 5227 5248 5252 5263
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 5274 5335 5345 5359 5368 5377 5390 5419 5435 5437 5497 5525 5529 5531
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 5579 5588 5599 5650 5660 5665 5666 5698 5699 5731 5750 5755 5772 5816
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 5823 5851 5857 5859 6016 6020 6025 6056 6094 6186 6395 6430 6446 6463
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 6558 6559 6561 6574 6648 6813 6824 6888 6942 6954 6955 6964 6987 7025
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 7043 7060 7087 7238 7279 7342 7343 7411 7424 7429 7454 7472 7474 7509
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 7539 7769 7783 7784 7801 7824 7874 7887 7923 7926 8021 8087 8089 8090
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 8096 8106 8107 8142 8168 8173 8203 8211 8224 8272 8300 8304 8364 8370
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 8381 8388 8406 8415 8496 8501 8518 8520 8524 8548 8556 8564 8581 8586
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 8587 8597 8656 8722 8743 8749 8758 8796 8838 8842 8846 8860 8862 8880
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 8886 8903 8908 8911 8917 8991 8997 9014 9015 9027 9066 9082 9131 9132
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 9154 9158 9184 9230 9265 9367 9390 9391 9418 9424 9447 9449 9453 9468
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 9502 9505 9603 9643 9667 9683 9694 9710 9718 9725 9744 9752 9776 9786
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 9791 9794 9810 9846 9859 9868 9876 9883 9889 9901 9936 9964 10043 10067
## 1980 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## 10091 10120 10121 10167 10209 10230 10265 10274 10311 10392 10425 10441
## 1980 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1
##
## 10457 10469 10524 10552 10553 10570 10593 10666 11275 11328 11750 11821
## 1980 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1
##
## 11857 11887 11890 11892 11924 11925 11957 11973 11990 12012 12013 12045
## 1980 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1
##
## 12055 12084 12088 12122 12179 12182 12220 12221 12245 12276 12385 12410
## 1980 1 1 1 1 1 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1 1 1 1 1 1
##
## 12420 12433 12451 12477 12500 12534 12548
## 1980 1 1 1 1 1 1 1
## 1981 1 1 1 1 1 1 1
## 1982 1 1 1 1 1 1 1
## 1983 1 1 1 1 1 1 1
## 1984 1 1 1 1 1 1 1
## 1985 1 1 1 1 1 1 1
## 1986 1 1 1 1 1 1 1
## 1987 1 1 1 1 1 1 1length(unique(df_raw$nr))## [1] 545Your answer here. The years 1980-1987 are included in the dataset, with 545 men observed each year.
3. What is the unit of observation? In other words, what does each row correspond to?
Your answer here. The unit of observation is person-year, with each row corresponding one specific worker in a specific year.
4. How many variables are there in the dataset?
# Code here
ncol(df_raw)## [1] 20Your answer here. There are 20 variables in each dataset 5. How many rows are there in the dataset?
# Code here
nrow(df_raw)## [1] 4360Your answer here. 4360 ## Filtering the data and exploring further [3.5 points]
1. Our main variable of interest is wage, which measures
the hourly wage for a worker in a given year. Calculate how many workers
are paid more than $10 an hour for each year of our sample. Do you
notice a trend ? What would be a possible explanation for this ?
# Code here
df_raw %>%
filter(wage > 10) %>%
group_by(year) %>%
summarize(high_earners = n())Your answer here. There is an upward trend, demonstrating an increase in the number of workers earning over $10/hour from 1980-1987, potentially as a result of the men in the sample gaining work experience and thus earning higher wages.
How many sectors observed a unionization rate (share of employees member of a union) superior to 30% in 1980 ? In how many years did at least one sector have a unionization rate inferior to 5% ?
# Code here
df_raw %>%
filter(year == 1980) %>%
group_by(sector) %>%
summarize(union_rate = mean(union, na.rm = TRUE)) %>%
filter(union_rate > 0.30)df_raw %>%
group_by(year, sector) %>%
summarize(union_rate = mean(union, na.rm = TRUE), .groups="drop") %>%
filter(union_rate < 0.05) %>%
distinct(year) %>%
count()Your answer here. In 1980, there were 5 sectors with a unioninzation rate over 30%. In 6 of the 8 years observed, at least one sector had a unionization rate less than 5%.
2. Let’s start focusing on the wage. Calculate the average wage per sector, and per union membership status. Then, produce overlaid histograms of the distribution of wages for the unionized members, and for non-unionized members. Add the average wage for union (blue line) and non-union (red line) members as vertical lines to your graphs.
Label your axes, and describe your output (shapes of the distribution, what is on each axis, what the two histograms correspond to, etc…).
Describe your graphs. What do you notice ?
# Code here
df_raw %>%
group_by(sector, union) %>%
summarize(avg_wage = mean(wage, na.rm=TRUE), .groups="drop")avg_wages <- df_raw %>%
group_by(union) %>%
summarize(mean_wage = mean(wage, na.rm=TRUE))
ggplot(df_raw, aes(x = wage, fill = factor(union))) +
geom_histogram(position = "identity", alpha = 0.5, bins = 40) +
geom_vline(data = avg_wages, aes(xintercept = mean_wage, color = factor(union)), size = 1) +
scale_color_manual(values = c("0" = "red", "1" = "blue")) +
scale_fill_manual(values = c("0" = "red", "1" = "blue"), labels = c("Non-Union", "Union")) +
labs(x = "Hourly Wage", y = "Count", title = "Distribution of Wages by Union Status") +
theme_minimal()
Your answer here. The x-axis represents the hourly wage, whereas the y-axis represents the frequency of workers. The blue histogram (union members) has a marginally higher mean than the red histogram (non-union members), denoted by the vertical blue and red lines, respectively. Also, the distribution of union members has less extreme variance, in contrast to the non-union members, which extend further to the right due to some extreme high earners.
3. We are interested in understanding to what extend the workers change of union status during their carrier. How many workers in the whole sample change at least once of union status ? Is this a large part of the workforce ?
# Code here
df_raw %>%
group_by(nr) %>%
summarize(varies = n_distinct(union) > 1) %>%
filter(varies == TRUE) %>%
count()Your answer here. 246 workers change their union status at least once, representing approximately 45% of the 545 men, which is a substantial fraction of the sample.
4. Finally, we are interested in understanding the relationship between education and wage, and how this changes between union members and non-members. Create three scatter plots of wage (x-axis) and education (y-axis), with blue dots corresponding to union members and red dots corresponding to non-union members.
You need to create one scatter plot for the year 1980, one scatter
plot for the year 1983, and one for the year 1987. You might want to use
the function facet_wrap().
Label your axes. Describe your output. What could you visibly say about the relationship between wage and education through the years ?
# Code here
df_raw %>%
filter(year %in% c(1980, 1983, 1987)) %>%
ggplot(aes(x = educ, y = wage, color = factor(union))) +
geom_point(alpha = 0.6) +
facet_wrap(~year) +
scale_color_manual(values = c("0" = "red", "1" = "blue"), labels=c("Non-Union", "Union")) +
labs(x = "Years of Education", y = "Hourly Wage") +
theme_minimal()
Your answer here. There is a weakly positive relationship (of approximately 0.106) between the variables education and wage. However, the correlation varies significantly by year and across sectors
5. As you noticed, our dataset is a panel, as we are observing units
across different time periods. Yet, we are (for now!) only interested in
the data from 1980 and 1987, that is, only the first and the last period
observed. Use the filter function to keep only the
observations that we are concerned about. Create a new object
df_8087 for this task.
# Code here
df_8087 <- df_raw %>% filter(year %in% c(1980, 1987))Preliminary Analysis: Understanding Union Membership [5 points]
1. Our ultimate goal is to learn how union membership
(union) affects the wage of a worker (wage),
and what other variables we need to take into account to fully
understand the relationship between the two.
First, we will report correlations between variables of interest : report the correlation between the two aforementioned variables (i) across sectors and years, (ii) for each sector across years and (iii) for each year across sectors
What do you conclude from these numbers ?
# Code here
cor(df_8087$union, df_8087$wage, use="complete.obs")## [1] 0.1058021df_8087 %>% group_by(sector) %>% summarize(correlation = cor(union, wage, use="complete.obs"))df_8087 %>% group_by(year) %>% summarize(correlation = cor(union, wage, use="complete.obs"))Your answer here. The overall correlation across sector and years is relatively weak but positive. However, it is highly variable across sectors, with certain industries demonstrating higher correlations than others.
2. We will now try to fit a linear regression to this data. Fit a linear (univariate) regression to assess the relation between union membership and wage.
Report and interpret your results.
# Code here
model_all <- lm(wage ~ union, data = df_8087)
summary(model_all)##
## Call:
## lm(formula = wage ~ union, data = df_8087)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.0403 -2.1463 -0.6619 1.3506 22.6161
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 5.6925 0.1141 49.88 < 2e-16 ***
## union 0.7902 0.2252 3.51 0.000467 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.248 on 1088 degrees of freedom
## Multiple R-squared: 0.01119, Adjusted R-squared: 0.01029
## F-statistic: 12.32 on 1 and 1088 DF, p-value: 0.0004672Your answer here. The coefficient for union is positive and statistically significant. This univariate regression suggests that being in a union is associated with an approximate $0.79 increase in hourly wages compared to non-union workers.
3. Next, let us study heterogeneity in the OLS slope. For this, fit
linear regressions to the 1980 and 1987 data, separately. It is
convenient to create two separate sub-datasets containing only
observations corresponding to those years. The name for these should be
df_1980 and df_1987. Then, fit the model to
these two subsamples. Do you notice a considerable difference between
the slopes?
# Code here
df_1980 <- df_8087 %>% filter(year == 1980)
df_1987 <- df_8087 %>% filter(year == 1987)
model_1980 <- lm(wage ~ union, data = df_1980)
model_1987 <- lm(wage ~ union, data = df_1987)
summary(model_1980)$coefficients[2,]## Estimate Std. Error t value Pr(>|t|)
## 1.168482e+00 2.118897e-01 5.514575e+00 5.414591e-08summary(model_1987)$coefficients[2,]## Estimate Std. Error t value Pr(>|t|)
## 0.3483077 0.3518151 0.9900304 0.3226002Your answer here. There is a considerable difference between the slopes. In 1980, the OLS slope indicates a statistically significant wage premium of roughly $1.17 for union members. However, in 1987, the slope drops to approximately $0.35 and the p-value is 0.322. Because the p-value is greater than 0.05, the effect of union membership on wages in 1987 is not statistically significant.
4. Use the function export_summs from the
jtools package to jointly display the results of the last
three models we have run. The names of your models must appear as “All”,
“Year 1980”, “Year 1987”.
Do these preliminary results indicate the fact that being part of a union leads to higher wages ? Comment on possible reasons for the difference, or the absence of difference, in the OLS slopes across the years 1980 and 1987.
# Code here
export_summs(model_all, model_1980, model_1987, model.names = c("All", "Year 1980", "Year 1987"))| All | Year 1980 | Year 1987 | |
|---|---|---|---|
| (Intercept) | 5.69 *** | 4.29 *** | 7.11 *** |
| (0.11) | (0.11) | (0.18) | |
| union | 0.79 *** | 1.17 *** | 0.35 |
| (0.23) | (0.21) | (0.35) | |
| N | 1090 | 545 | 545 |
| R2 | 0.01 | 0.05 | 0.00 |
| *** p < 0.001; ** p < 0.01; * p < 0.05. | |||
Your answer here. These preliminary results do support the claim that being a union member is associated with higher wages at the beginning of the decade. However, this advantage decreased drastically by 1987. The changes in OLS slopes from 1980-87 may reflect broader economic factors affecting unionization, such as a decreased in their bargaining power due to globalization and domestic labor policy.
Diving into the analysis II: Other important predictors and functional forms [5 points]
One sticky insight from the other regressions is the solid link between education and wage, and experience and wage.
Functional forms are potentially another source of bias in the sense that if we fail to correctly specify the conditional expectation of Y given X we will not be able to model the true relationship between the variables and thus we cannot talk about causality. In this exercise, we will explore such a bias. In Econometrics, this is known as “misspecification bias.”
In particular, is the relationship between education / experience and
wage nonlinear? So far, we have included only educ and
exper, that is, we were assuming a linear relationship
between these variables and wages, but what about including
educsq, that is educ squared?
1. Construct the variable educsq which is
educ squared, as well as expersq, that is
exper squared. Add it to the three databases (for 1980 and
1987 together, and 1980 and 1987, separately).
Give a plausible explanation for why the relationship between
exper and wage might not be linear and have a
quadratic form. Do the same for educ and
wage.
# Code here
df_8087_health <- df_8087_health %>% mutate(educsq = educ^2, expersq = exper^2)
df_80_health <- df_80_health %>% mutate(educsq = educ^2, expersq = exper^2)
df_87_health <- df_87_health %>% mutate(educsq = educ^2, expersq = exper^2)Your answer here. The relationship between experience and wage is likely quadratic, rather than linear, because workers learn more rapidly early in their careers, leading to steeper initial wage increases. As they approach their peak productivity, the wage returns per additional year likely diminish. A quadratic term for education helps capture non-linear returns, such as large wage spikes that occur upon completion of a degree or certificate program.
2. To the models you ran in the question 5 of the part I, add
expersq. Name your new models as follows:
model8087mvnonlinear, model80mvnonlinear,
model87mvnonlinear. Use the function
export_summs from the jtools package to
jointly display the results of the last three models we have run.
Compare your results with the above linear models from question 5.
# Code here
model8087mvnonlinear <- lm(wage ~ union + educ + exper + expersq + hours + poorhlth, data = df_8087_health)
model80mvnonlinear <- lm(wage ~ union + educ + exper + expersq + hours + poorhlth, data = df_80_health)
model87mvnonlinear <- lm(wage ~ union + educ + exper + expersq + hours + poorhlth, data = df_87_health)
export_summs(model8087mvnonlinear, model80mvnonlinear, model87mvnonlinear, model.names = c("All NL", "1980 NL", "1987 NL"))| All NL | 1980 NL | 1987 NL | |
|---|---|---|---|
| (Intercept) | -4.36 *** | -2.85 ** | 11.00 ** |
| (0.69) | (0.87) | (3.99) | |
| union | 0.68 *** | 1.09 *** | 0.44 |
| (0.20) | (0.20) | (0.33) | |
| educ | 0.64 *** | 0.48 *** | 0.74 *** |
| (0.05) | (0.06) | (0.10) | |
| exper | 0.81 *** | 0.49 ** | -1.82 ** |
| (0.09) | (0.17) | (0.69) | |
| expersq | -0.03 *** | -0.02 | 0.08 * |
| (0.01) | (0.02) | (0.03) | |
| hours | -0.00 * | 0.00 | -0.00 *** |
| (0.00) | (0.00) | (0.00) | |
| poorhlth | -0.23 | 0.42 | -0.59 |
| (0.66) | (0.60) | (1.28) | |
| N | 1090 | 545 | 545 |
| R2 | 0.26 | 0.16 | 0.15 |
| *** p < 0.001; ** p < 0.01; * p < 0.05. | |||
Your answer here. Compared to the linear models, adding expersq reveals a negative, statistically significant coefficient for the squared term. This confirms that marginal returns to experience diminish over time. Consequently, the linear exper term increases, as it now represents the steeper wage growth slope at zero years of experience.
3. Let us now explore further nonlinearities. In particular, run a
multivariate regression of wage on all previous variables,
and, in addition, include educexper, where
educexper is the product between educ and
exper.
Run the models for the three datasets. Name the models as
follows:model8087mvnonlinear_2,
model80mvnonlinear_2, model87mvnonlinear_2.
Report your results. Use the function export_summs from the
jtools package to jointly display the results of the last
three models we have run.
- How do we call a variable such as
educexper? Explain in English the implications of using such a technique for the interpretation of the effects ofeducandexperonwageand give a plausible economic reason for which we would want to introduceeducexper.
Have your conclusions about the impact of union membership on wage changed?
# Code here
df_8087_health <- df_8087_health %>% mutate(educexper = educ * exper)
df_80_health <- df_80_health %>% mutate(educexper = educ * exper)
df_87_health <- df_87_health %>% mutate(educexper = educ * exper)
model8087mvnonlinear_2 <- lm(wage ~ union + educ + exper + expersq + educexper + hours + poorhlth, data = df_8087_health)
model80mvnonlinear_2 <- lm(wage ~ union + educ + exper + expersq + educexper + hours + poorhlth, data = df_80_health)
model87mvnonlinear_2 <- lm(wage ~ union + educ + exper + expersq + educexper + hours + poorhlth, data = df_87_health)
export_summs(model8087mvnonlinear_2, model80mvnonlinear_2, model87mvnonlinear_2, model.names = c("All NL 2", "1980 NL 2", "1987 NL 2"))| All NL 2 | 1980 NL 2 | 1987 NL 2 | |
|---|---|---|---|
| (Intercept) | -4.91 ** | -3.62 * | -10.37 |
| (1.51) | (1.75) | (12.88) | |
| union | 0.68 *** | 1.10 *** | 0.50 |
| (0.20) | (0.20) | (0.33) | |
| educ | 0.68 *** | 0.53 *** | 1.78 ** |
| (0.11) | (0.12) | (0.61) | |
| exper | 0.91 *** | 0.81 | 1.25 |
| (0.25) | (0.65) | (1.89) | |
| expersq | -0.04 *** | -0.03 | -0.01 |
| (0.01) | (0.04) | (0.06) | |
| educexper | -0.01 | -0.02 | -0.11 |
| (0.01) | (0.04) | (0.06) | |
| hours | -0.00 * | 0.00 | -0.00 *** |
| (0.00) | (0.00) | (0.00) | |
| poorhlth | -0.21 | 0.42 | -0.59 |
| (0.66) | (0.60) | (1.28) | |
| N | 1090 | 545 | 545 |
| R2 | 0.26 | 0.16 | 0.16 |
| *** p < 0.001; ** p < 0.01; * p < 0.05. | |||
Your answer here. A variable like educexper is also
referred to as an interaction term. Including it tests whether the wage
return for an extra year of experience depends on the worker’s level of
education and vice-versa. 4. Now, consider the model from question 2,
that is, disregard educexper. Compute the marginal effect
of exper on wage in the model for the whole
data only, that is, model8087mvnonlinear. Explain how you
compute the marginal effect.
Hint: Be careful with how you compute the marginal effect of
exper under a non-linear model.
Your answer here. In order to compute the marginal effect of ‘exper’ under a non-linear model, one must take the partial derivative of wage with respect to experience, which mathematically yields the linear equation: Marginal Effect = B1 + 2 * B2 * exper
Plot the marginal effect of exper on wage
as a function of exper, and the total effect of
exper on wage as a function of exper, only for
our model8087mvnonlinear. At which level of experience is
the total effect of experience on wage the highest ? At which level of
experience is the marginal effect of 1 more year of experience the
largest ?
# Code here
B1 <- coef(model8087mvnonlinear)["exper"]
B2 <- coef(model8087mvnonlinear)["expersq"]
exper_seq <- seq(min(df_8087_health$exper, na.rm=TRUE), max(df_8087_health$exper, na.rm=TRUE), by=1)
marginal <- B1 + 2 * B2 * exper_seq
total <- B1 * exper_seq + B2 * (exper_seq^2)
plot_data <- data.frame(exper = exper_seq, marginal = marginal, total = total)
ggplot(plot_data, aes(x = exper)) +
geom_line(aes(y = marginal, color = "Marginal Effect"), size=1) +
geom_line(aes(y = total, color = "Total Effect"), size=1) +
labs(title = "Effects of Experience on Wage", x = "Years of Experience", y = "Effect") +
theme_minimal()
Your answer here. The plot exhibits the marginal effect of experience as a straight, downward-sloping line and the total effect is a concave parabola. The marginal effect of 1 more year of experience is largest at exactly 0 years of experience (the y-intercept of the marginal line). The total effect of experience on wage is highest at the vertex of the total effect curve, which aligns with the exact point where the marginal effect line crosses zero on the y-axis.
Diving into the analysis III : Adapting our inquiry to the nature of the data [5 points]
Notice that we have not yet really exploited the nature of our data. In particular, we are not really taking advantage of the fact that we are observing the same unit (i.e., individuals) across different periods (i.e., years). In the last part of this exam, we will take this into account.
Furthermore, we will attempt to understand how individual-level heterogeneity can influence our estimate and what kind of threat to validity this can bring to our work.
Do not use export_summsto present your results in this
section.
1. One advantage that panel data entails is that we can understand how the simple fact of time passing influences our estimates. Notice that simply because time passes, and macro-conditions in the U.S. evolve, there might be some component that pertains to wage determination, that varies across years.
Present a linear model that takes this fact into account. What does this change to our conclusions ?
From now on, we will use all the years that we have at our disposal. Apply all the transformations we applied to our data in this midterm (i.e. add expersq to the full dataset and create the poorhlth variable, correctly handling missing observations) to the full data (all years from 1980 to 1987).
In your new model, in addition to your proposed solution to take
advantage of the fact we now observe all the years, drop the interaction
variable that we previously set up, but keep all other relevant
variables. To account for an additional important predictor given the
cultural context of the United States, add the black
variable to your regression.
# Code here
df_full <- df_raw %>%
mutate(poorhlth = ifelse(is.na(poorhlth), "no", poorhlth),
poorhlth = ifelse(poorhlth == "yes", 1, 0),
expersq = exper^2)
model_macro <- lm(wage ~ union + educ + exper + expersq + hours + poorhlth + black + factor(year), data = df_full)
summary(model_macro)##
## Call:
## lm(formula = wage ~ union + educ + exper + expersq + hours +
## poorhlth + black + factor(year), data = df_full)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.535 -1.759 -0.343 1.238 51.019
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -1.878e+00 4.811e-01 -3.903 9.66e-05 ***
## union 8.790e-01 1.041e-01 8.442 < 2e-16 ***
## educ 5.447e-01 3.128e-02 17.415 < 2e-16 ***
## exper 4.453e-01 8.311e-02 5.358 8.86e-08 ***
## expersq -2.013e-02 4.991e-03 -4.033 5.61e-05 ***
## hours -5.976e-04 8.084e-05 -7.392 1.73e-13 ***
## poorhlth -3.247e-01 3.425e-01 -0.948 0.343128
## black -8.460e-01 1.394e-01 -6.071 1.38e-09 ***
## factor(year)1981 2.982e-01 1.843e-01 1.618 0.105720
## factor(year)1982 3.396e-01 2.016e-01 1.684 0.092202 .
## factor(year)1983 4.188e-01 2.226e-01 1.881 0.059983 .
## factor(year)1984 7.280e-01 2.434e-01 2.991 0.002800 **
## factor(year)1985 9.855e-01 2.633e-01 3.743 0.000184 ***
## factor(year)1986 1.261e+00 2.819e-01 4.473 7.89e-06 ***
## factor(year)1987 1.564e+00 3.003e-01 5.210 1.98e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.916 on 4345 degrees of freedom
## Multiple R-squared: 0.1735, Adjusted R-squared: 0.1709
## F-statistic: 65.16 on 14 and 4345 DF, p-value: < 2.2e-16Your answer here. Introducing factor(year) accounts for unobserved macroeconomic shists, such as general inflation and aggregate economic growth. The output shows these year dummies become increasingly large and highly significant by 1985-1987. Controlling for this macro-trend, the coefficient on union is approximately $0.88 and remains highly statistically significant.
2. Now that we have correctly controlled for macro-effects of the passing time, notice we still have a major threat to validity : individual-level heterogeneity. Indeed, unobserved ability and other factors (family background, ability to deal with stress, temperament, etc…) might make certain individuals more suited to a working environment and raise their wage.
Drawing on your solution to take into account the year-specific
components, propose a linear model that takes into account this
individual-level heterogeneity. (Hint : this model’s output shouldn’t
output coefficients for some variables: this is normal and is addressed
in the next question; Hint 2 : you should use the function
feols() from the package fixest, which will
avoid to display too many coefficients, only those important will be
left in.).
Did this modification significantly affect the coefficient on
union ? What do you conclude ?
# Code here
model_fe <- feols(wage ~ union + educ + exper + expersq + hours + poorhlth + black | nr + year, data = df_full)
summary(model_fe)## OLS estimation, Dep. Var.: wage
## Observations: 4,360
## Fixed-effects: nr: 545, year: 8
## Standard-errors: IID
## Estimate Std. Error t value Pr(>|t|)
## union 0.377205 0.106583 3.539083 0.00040632 ***
## expersq -0.043811 0.003915 -11.191181 < 2.2e-16 ***
## hours -0.001080 0.000075 -14.468160 < 2.2e-16 ***
## poorhlth 0.105830 0.260211 0.406708 0.68424568
## ... 3 variables were removed because of collinearity (educ, exper and
## black)
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## RMSE: 1.80843 Adj. R2: 0.63445
## Within R2: 0.075702Your answer here. Yes, applying individual fixed effects drastically affects the coefficieent estimate on ‘union’. It decreased significantly from $0.88 in the pooled macro model to $0.38. It can thus be concluded that a major portioh of the initial “union premium” was an illusion caused by omitted variable bias, as unobserved individual characteristics were inflating the OLS estimates.
3. Explain why, in your previous model, three variables are removed by the software from the regression and therefore do not have coefficients. List them and provide an explanation for each one.
Your answer here. 1) black: Race is time-invariant. Because it does not change within an individual over the 1980-1987 period, it has zero within-person variance and is perfectly absorbed by the individual fixed effect.
educ: For the vast majority of adult men in this sample, years of schooling is fixed and does not change over the 8 years, meaning it is also absorbed by the individual fixed effect.
exper: Experience increases by exactly 1 year, every single year. Therefore, it is perfectly collinear with the year fixed effects we included. The model cannot mathematically separate the effect of “one more year of experience” from “one more calendar year passing.”
4. Going back to your conclusion after running the regression in question 2 : do you think individual-level heterogeneity plays a role in wage formation that could mislead us when trying to estimate the effect of unionization ? Compare this conclusion to your findings in the preliminary analysis section, particularly to the univariate OLS regression in question 3.
If you want to get full points, give a detailed explanation of all the steps you followed throughout this midterm and how much each step allowed you to progress towards this conclusion.
Your answer here. Individual-level heterogeneity plays a major role in the formulation of one’s wages. In the initial univariate analysis, OLS suggested a massive union premium of approximately $0.79 overall (and $1.17 in 1980). Adding standard controls (education, experience) did reduce it, but only slightly. However, upon controlling for unobserved, time-invariant individual traits using fixed effects, the premium halved to $0.38, thus proving that workers with high unobserved ability or baseline motivation were disproportionately sorted into union jobs, causing OLS to falsely attribute their naturally higher wages to the union itself.
5. We switched in the last section from comparing results for two individual years (1980 and 1987) to using the full dataset to run a single regression. This seems like we could be losing some information about the effect of individual years, since we are now bundling all years into a single estimation.
Explain (i) which specific part of the regressions we carried in this
last section guarantees we don’t lose this kind of information and (ii)
why it is a good idea to go from our df_8087 to a dataset
containing information about all the years.
Your answer here. i) By explicitly keeping the year variable as a fixed effect inside the feols formula (| nr + year), it is guaranteed to maintain information about macro-level yearly shocks. ii) It advisable to use the full dataset rather than just analyzing 1980 and 1987 because the fixed-effects estimator relies entirely on within-person variation. The continuous 8-year time series allows us to capture enough instances where individuals actually switch their union status to accurately estimate the effect.
6. Do you think this exercise has allowed you to identify a causal effect of unionization on wages ? Explain clearly what makes you think you identified a causal effect, and/or what makes you think this is not a causal effect to get full points.
Your answer here. I think this exercise has allowed me to identify an effect that is more likely causal than OLS, but it is likely not perfectly causal. Individual fixed effects successfully remove time-invariant unobserved heterogeneity (like baseline intelligence or demographic traits). However, it does not solve time-varying unobserved heterogeneity.
7. (EXTRA CREDIT) By now, you should have understood that Econometrics is all about interpreting observed variation in the dependent variable, using variation in the independent variables. For this reason, some researchers feel that the design you used in question 2 (recall that three variables were dropped altogether in the estimation, therefore were not used at all!) led to discarding too much information.
What do you think they mean ?
Your answer here. Such researchers would mean that the fixed effects design explicitly discards all “between-subject” variation. By mathematically removing variables that don’t change over time (such as education level and race), we are eliminating our ability to estimate how those notable societal factors influence wages across the population. The tradeoff presented here is more internal validity but a lessened ability to measure the impact of static traits.
8. (EXTRA CREDIT) These researchers use a class of estimators called
the “random effects” estimators. You don’t need to understand perfectly
what these estimators do, but they allow to use some of the variables
that were dropped in question 2, under a strong assumption : that the
individual-level heterogeneity is uncorrelated with all other variables
(such as union and all other controls).
Do you think it is likely that this assumption hold ? Explain why.
Your answer here.
Congratulations, you have just completed this midterm project. Well done!
And that’s it for this midterm! We hope you had fun, or at least
some fun, doing it and that you improved your R
skills as well as your critical thinking when it comes to learn effects
among relevant variables in the real world.