Occupational Mobility and Optimal Transport
Simulation results for now — awaiting linked census microdata
Richard Martin
Ministry of Post Secondary Education and Future Skills
Disclaimer: The views expressed are those of the author and do not necessarily reflect those of the Government of British Columbia.
Labour markets are constantly evolving
Sources of structural change include:
technology
COVID-19
government policy
resource depletion
These shocks shift occupational labour demand.
Entry and exit may account for part of the adjustment, but typically some workers must move across occupations to meet changing demand.
What governs mobility between occupations?
How does occupational distance shape mobility?
It is natural to expect mobility to decline with occupational distance.
We therefore ask which notion of occupational distance best rationalizes local mobility?
This requires:
Considering multiple notions of occupational distance
A method to compare model fit
Recognizing two mobility regimes
Local mobility: gradual movement in occupational space
Non-local mobility: jumps that bypass distance frictions
Distance should explain local transitions but fail to explain non-local moves.
Where we are going
Local mobility can be modeled using entropy-regularized optimal transport, which produces flows consistent with observed origin and destination margins.
This framework allows competing distance metrics to be compared in a clean “horse race.”
In simulations, when the data-generating process is known, the method recovers the correct distance structure.
This provides a framework for testing whether skill similarity or hierarchical structure better explains local mobility.
Two mobility regimes
Established workers mostly move locally in occupational space.
New credentials can act like wormholes, enabling jumps across distant occupations.
Implication:
- Distance should rationalize local mobility.
- If the model is correct, it should struggle with wormhole transitions.
Local Mobility
Gravity-style mobility models predict that flows increase with occupation size and decline with occupational distance.
This mirrors Newtonian gravity: \(F \propto \frac{m_1 m_2}{d^2}\).
- Goal: separate the mass effect (occupation size) from the distance effect in order to identify the distance measure that best rationalizes local mobility.
What Defines Occupational Distance?
Two natural candidates:
Skill similarity: based on 161 O*NET dimensions.
Institutional hierarchy: based on 5 digit NOC classification
| Digit | Level | Description |
|---|---|---|
| 1 | Broad Occupational Category | 10 broad categories |
| 2 | TEER Category | 6 levels |
| 3 | Sub-major Group | Occupational cluster |
| 4 | Minor Group | Narrow family |
| 5 | Unit Group | Detailed occupation |
Gravity: standard model for local mobility
\[ P_{ij} = \exp(\alpha_i + \beta_j - \gamma C_{ij}) \]
- \(P_{ij}\) is the share of workers moving from occupation \(i\) to \(j\)
- \(\alpha_i\) origin fixed effect
- \(\beta_j\) destination fixed effect
- \(C_{ij}\) occupational distance
- \(\gamma\) object of interest: sensitivity of flows to a proposed cost structure.
Gravity Models: An Unfair Horse Race
In horse racing, handicaps add weight to stronger horses, making the outcome less informative about the horses’ true speed.
If the friction specification \(C_{ij}\) is wrong — and it will be — fixed effects may absorb part of the misspecification.
We prefer misspecification to appear as lack of fit rather than being masked by parameter adjustment.
The goal of our horse race is to cleanly identify how well each friction rationalizes the observed flows.
Of course, winning the horse race does not imply the winner was particularly “fast”.
A Fair Horse Race
Entropy-regularized optimal transport enforces the observed marginals exactly:
\[ \sum_j P_{ij} = a_i, \qquad \sum_i P_{ij} = b_j \]
via iterative row and column rescaling.
Equilibrium employment levels are taken as given; we model the mobility flows consistent with those levels.
This corresponds to a decentralized random utility model where mobility utilities include Gumbel shocks, producing logit choice probabilities.
Entropy-regularized optimal transport
\[ \mathcal{L} = \underbrace{\sum_{ij} P_{ij} C_{ij}}_{\text{mass}\times\text{distance}} + \varepsilon\times \underbrace{\sum_{ij} P_{ij}(\log P_{ij}-1)}_{\text{negative entropy}} + \sum_i f_i \underbrace{\left(a_i - \sum_j P_{ij}\right)}_{\text{origin constraint}} + \sum_j g_j \underbrace{\left(b_j - \sum_i P_{ij}\right)}_{\text{destination constraint}} \]
First-order conditions:
\[ \frac{\partial \mathcal{L}}{\partial P_{ij}} = C_{ij} + \varepsilon \log P_{ij} - f_i - g_j = 0. \]
Rearranging:
\[ \log P_{ij} = \frac{f_i}{\varepsilon} + \frac{g_j}{\varepsilon} - \frac{C_{ij}}{\varepsilon}. \]
Structure of the Solution
Exponentiating:
\[ P_{ij} = u_i\, e^{-C_{ij}/\varepsilon}\, v_j, \]
- Thus the optimal solution must be a row- and column-rescaling of the kernel \(e^{-C_{ij}/\varepsilon}\).
- Given \(v\), there is a unique \(u\) that fixes the rows.
- Given \(u\), there is a unique \(v\) that fixes the columns.
- Sinkhorn alternates these corrections until both margins are satisfied.
- Implemented using a custom log-domain Sinkhorn solver in R (safesink) to ensure numerical stability and dimensional alignment.
Result
Because the objective is strictly convex, the optimal solution \(P^{\star}\) is unique.
If the Sinkhorn algorithm converges, it must converge to \(P^{\star}\).
\(P^{\star}\)
- matches the observed origin and destination totals
- respects the relative friction structure implied by \(C_{ij}\).
Same structure as gravity, but…
Gravity estimates margins and frictions jointly.
Sinkhorn conditions on the margins, so frictions are identified from the residual structure of flows.
With a misspecified cost matrix, a gravity model merely measures sensitivity to the wrong notion of distance.
When the goal is to compare alternative cost matrices, conditioning on the margins allows cleaner identification of frictions.
Sinkhorn is the right tool for this job.
Interpretation of Temperature: \(\varepsilon\)
\(\varepsilon\) is the scale of random utility shocks (Gumbel noise) affecting mobility choices.
Low \(\varepsilon\)
- Mobility concentrates on the lowest-cost transitions
- Behavior approaches cost-minimizing reallocation (social planner)
High \(\varepsilon\)
- Random shocks dominate cost differences
- Mobility becomes diffuse and approaches random choice \(P_{ij} = a_i b_j\)
The distance metrics
- Skill distance: represents continuous skill transferability
- Hierarchical distance: represents discrete barriers within labour market.