Many variables in the social sciences consist of integer counts
E.g., number of times a candidate runs for office, frequency of conflict, number of terror attacks, number of war casualties, number of positive statements about a candidate, number of homicides in a city, etc.
Least squares is inappropriate for the reasons we’ve already discussed: \(\textbf{Nonlinearity, nonadditivity and heteroskedasticity}\).
For instance
Models
The Poisson Regression Model (PRM)
A strong assumption: \(E(\mu)=var(y)\)
Overdispersion
Underdispersion
Extensions: Negative binomial and zero-inflated models.
The Poisson Regression Model
\[p(y|\mu)={{exp(-\mu)\mu^y}\over{y!}}\]
If \(y\) takes on values from 0, 1, 2, 3, etc. (Long 1997, p. 218)
In this density, the only parameter that governs the shape of the density is \(\mu\) or the “rate” parameter
A characteristic of the poisson distribution is that \(E(y)=var(y)=\mu\), an assumption called equidispersion
The Poission Distribution
Extensions
If \(\mu\) is the rate parameter, we can then model \(\mu_i\) based on a set of covariates. That is,
\[\mu_i=E(y_i|x_i)=exp(\alpha+\beta x_i)\]
\(\alpha+\beta x_i\) must be positive; the rate parameter must be positive.
Non-linear regression model with heteroskedasticity (Long 1997, p. 223), \(\mu_i=exp(\alpha+\beta x_i)\)
Extensions
Let’s just assume \(\alpha=-0.25\), and \(\beta=0.13\) as Long does (p. 221). If we calculate the expected values of \(y\) for a number of \(x\) values, then
When \(x=1\), then the expected number of counts – the rate parameter – is 0.89 (exp(-0.25 + 0.13* 1))
When \(x=10\) then the expected number of counts is 2.85
\[\tiny {{\partial E(Y|X)}\over{\partial x_k}}={{\partial exp(u)}\over{\partial u}}{{\partial u \beta}\over{\partial x_k}}\]
\[\tiny exp(\alpha+\sum_K \beta_k x_{k,i})\beta_k=E(Y|X)\beta_k\] - The effect of \(x_k\) on \(y\) is now a function of the rate parameter and the expected effect of \(x_k\) on that rate parameter
It’s not a constant change in the expected count; instead it’s a function of how \(x_k\) affects the rate as well as how all others relate to the rate!
The PRM Likelihood
What effect does a \(d_k\) change in \(x_k\) have on the expected count. Take the ratio of the the prediction including the change over the prediction absent the change (Long 1997, p. 225):
\[\tiny {{E(y|X, x_k+d_k)}\over{E(y|X, x_k)}}\]
The numerator: \(\tiny E(y|X, x_k+d_k)=exp(\beta_0)exp(\beta_1 x_1)exp(\beta_2 x_2)...exp(\beta_1 x_k)exp(\beta_k d_k)\)
The denominator: \(\tiny E(y|X, x_k)=exp(\beta_0)exp(\beta_1 x_1)exp(\beta_2 x_2)...exp(\beta_1 x_k)\)
We might encounter either under or overdispersion brought about by unobserved heterogeneity.
It is useful to rely on an alternative model that doesn’t treat \(\mu\) as fixed, but rather it is drawn from a distribution, i.e., \[\mu_i=exp(\alpha+\sum_K \beta_k x_{k,i})\beta_k+\epsilon_i)\]
The Negative Binomial Regression Model
The NBRM stems from the negative binomial distribution
Recall the binomial density is the PDF stemming from \(k\) independent bernoulli trials. The \(\theta\) parameter will govern its shape.
We can modify the logic (and code) slightly to generate a probability of observing \(r\) successes, given \(n\) trials.
The Negative Binomial Regression Model
For instance, how many times would we need to flip a coin in order for three heads to appear, or four heads, and so forth? \(({{s+f-1}\over{s}})\theta^{f}(1-\theta)^{s}\)
\(f=\)number of failures, \(s=\)number of successes. The total number of trials is just \(s+f\).
The Negative Binomial Regression Model
If \(\theta\) represents the probability of striking out, how many at bats are expected before a batter strikes out once, or twice, or 10 times?
Or, how many games must the Minnesota Vikings play in the 2022 NFL season before they lose three games?
The Negative Binomial Regression Model
Here, define “success”’” as striking out (that is the outcome we’re interested in), or losing two games; “failure” is at-bats before striking out or number of games before losing twice
The Negative Binomial Regression Model
The negative binomial because if expand and then rearrange the binomial coefficient, it will equal
To calculate \(p(y|\mu^*_i)\) we need to assume that \(d_i\) follows from some density, and then we should integrate (i.e., average) over this unknown parameter to obtain the joint density of \(y\) given \(\mu_i\). Let’s assume that \(d\) follows a gamma density.
Notice the similarity to the negative binomial above. If you compare these two, you’ll see that by using the distribution of \(d_i\) in the integration equation, as well as the poisson, you will find that the negative binomial is nothing more than a poisson mixed with the gamma.
Capturing Dispersion
\[E(y|x)=\mu_i\]
But, the variance is no longer \(\mu_i\)
\[var(y_i|x)=\mu_i(1+({{\mu_i}\over{v_i}}))\]
The \(v_i\) parameter governs the shape of the gamma density. Another way to write this is,
The negative binomial model is an example of a mixture model that has a closed form maximum likelihood solution, where we maximize,
