Principles
- Relaxes the assumption of fixed parameters of parameter priors.
\[ \theta\sim N(a,b),\quad\text{where }a,b\text{ are unknown}\]
Multi-level structure
Exchangability of model parameters between units
Modelling Dependent Parameters
- Three approaches to modelling a problem with data measured as \(x_{ij}\), with individual \(j\) in unit \(i\).
- Identical parameters \(\theta_i\)
- Data is pooled and individual units are ignored. Therefore one parameter is estimated for the entirety of the population.
- Seperate parameters \(\theta_i\)
- Data from each unit is analysed seperately. Small population per unit therefore likely to have highly variable individual estimates.
- Exchangable Parameters \(\theta_i\)
- \(\theta_i\) are assumed to be similar in the sense that labels convey no information
- Used in heirarchal models, where the parameters ‘share strength’ by estimating hyperparameters based on entire dataset.
Exchangability
- \(X_1,...,X_n\) are exchangable if the probability that we assign to any set of potential outcomes $$ is unaffected by permutations of labels of the variables
- Equivalent to drawing from the same population distribution
- Exchangability implies:
\[ f(\theta_1,...,\theta_n) = \int\prod^n_{i=1}f(\theta_i|\phi)f(\phi)d\phi\]
- Note that iid \(\rightarrow\) exchangable, but exchangable only \(\rightarrow\) identically distributed (not independent)
Exchangability and Heirarchal model
- Consider \(x_{ij}\) as measurement of individual \(j\) of unit \(i\) with unit specific parameter \(\theta_i\)
- Assuming partial exchangability of individuals within units:
\[ x_{ij}\sim f(x|\theta_{i}),\quad\theta_i\sim f(\theta) \]
- Assuming exchangability of units:
\[ \theta_i\sim f(\theta|\phi),\quad\phi\sim f(\phi)\]
Multi-Level Structure
- Level 0 is observables: \(x_{ij}, \theta_i\)
- First level is the likelihood \(f(x|\theta)\), the model of the observables
- Prior \(f(\theta)\) is then decomposed into levels of conditional distributions:
- Level 2 - \(f(\theta|\phi_2)\)
- …
- Level \(m-1\) - \(f(\phi_{m-1}|\phi_m)\)
- Top level - \(f(\phi_m)\)
- With this level system, we can calculate a full prior for the model:
\[ f(\theta) = \int f(\theta|\phi_2)*...*f(\phi_{m-1}|\phi_m)*f(\phi_m)d\phi_2...d\phi_m\]