# --- Step 1: Compute total weighted 15–59 population per year ---
pop_by_year <- d_state %>%
  group_by(year) %>%
  summarise(total_weighted_pop = sum(weight, na.rm = TRUE)) %>%
  ungroup()

# --- Step 2: Compute scale factor for each year ---
scale_factor_tbl <- pop_by_year %>%
  mutate(scale_factor = total_weighted_pop / mean(total_weighted_pop, na.rm = TRUE))

# --- Step 3: Merge scale factor and compute scaled weights ---
d_state_scaled <- d_state %>%
  left_join(scale_factor_tbl %>% select(year, scale_factor), by = "year") %>%
  mutate(weight_scaled = weight / scale_factor)

# --- Step 4: Create survey design with scaled weights only (no PSU/strata) ---
design_scaled <- d_state_scaled %>%
  as_survey_design(ids = 1, weights = weight_scaled)

Between 2017 and 2023 Tamil Nadu did not experience the tidy, service-led structural shift economists typically expect. Instead the state’s labour market shows a nuanced “U-turn”: a small rise in the share of workers in agriculture, a noticeable rise in self-employment, a decline in manufacturing’s share, and only modest gains in higher-quality regular wage work.

Headline numbers (what changed)

Worker Population Ratio (ages 15–59) rose from 51.5% (95% CI: 50.4–52.5) in 2017 to 57.2% (95% CI: 56.1–58.3) in 2023 — a rise of 5.7 percentage points and a statistically clear increase (the 2017 and 2023 CIs do not overlap). (See visual: Worker Population Ratio (15–59) with 95% CI.)

Agriculture’s share of employment rose slightly from 24.0% → 24.5% (+0.5 pp). Not large in absolute terms, but important when read alongside other shifts. (See Sectoral Shift: The U-Turn slope chart and Percentage-point change bar chart.)

# (e) Sectoral Shift Slope Chart
sector_slope <- sector_tbl %>%
  filter(!is.na(ind_group2), ind_group2 != "Other") %>%  # drop NA
  select(year, ind_group2, share)

ggplot(sector_slope, aes(x = factor(year), y = share, group = ind_group2, color = ind_group2)) +
  geom_line(size = 1.2, alpha = 0.8) +
  geom_point(size = 3) +
  geom_text_repel(
    data = sector_slope %>% filter(year == 2023),
    aes(label = paste0(ind_group2, ": ", round(share, 1), "%")),
    nudge_x = 0.2, size = 3, fontface = "bold", segment.color = "grey70",
    na.rm = TRUE    # ignore NA labels
  ) +
  labs(
    title = "Sectoral Shift: The U-Turn in Tamil Nadu’s Labour Market",
    subtitle = "Share of employed workers by sector, 2017–2023",
    x = NULL, y = "Share of Employment (%)"
  ) +
  theme_minimal(base_size = 13) +
  theme(legend.position = "none", plot.title = element_text(face = "bold"))
## Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
## ℹ Please use `linewidth` instead.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

##Sectoral change 

ggplot(sector_change %>% filter(!is.na(ind_group2)) %>% arrange(desc(change_17_23)),
       aes(x = reorder(ind_group2, change_17_23), y = change_17_23, fill = change_17_23 > 0)) +
  geom_col() +
  scale_fill_manual(values = c("#de2d26", "#2ca25f"), guide = "none") +
  geom_text(aes(label = paste0(round(change_17_23, 1), " pp")),
            position = position_stack(vjust = 0.5),  # label inside the bar
            color = "white", fontface = "bold", size = 3.5) +
  coord_flip() +
  labs(title = "Percentage-point change in sectoral share: 2017 → 2023",
       x = NULL, y = "Change (percentage points)") +
  theme_minimal(base_size = 10)

• Self-employment rose from 31.3% → 33.6% (+2.3 pp). At the same time casual wage work fell (-4.4 pp) and regular wage work rose (+2.1 pp). In plain language: more people are self-employed, fewer are casual wage workers, and there has been a modest growth in regular salaried jobs. (See Employment Type Distribution by Year and Employment type composition within sectors (2017 vs 2023).)

# (c) Employment Type (grouped bar)
ggplot(emp_type_tbl, aes(x = factor(year), y = share, fill = emp_status)) +
  geom_col(position = "dodge") +
  labs(title = "Employment Type Distribution by Year",
       subtitle = "Regular, Self-employed and Casual workers",
       x = "Year", y = "Share of Employment (%)",
       fill = "Employment Type") +
  theme_minimal(base_size = 13)

sector_type <- design_scaled %>%
  filter(employed == 1) %>%
  group_by(year, ind_group2, emp_status) %>%
  summarize(n = survey_total(1, na.rm = TRUE)) %>%
  group_by(year, ind_group2) %>%
  mutate(share = 100 * n / sum(n)) %>%
  ungroup() %>%
  as_tibble()

ggplot(sector_type %>% filter(ind_group2 != "Other"),
       aes(x = reorder(ind_group2, share), y = share, fill = emp_status)) +
  geom_col(position = "stack") +
  geom_text(aes(label = paste0(round(share, 1), "%")),
            position = position_stack(vjust = 0.5),  # puts labels in the middle of each segment
            size = 2) +
  coord_flip() +
  facet_wrap(~year) +
  labs(
    title = "Employment type composition within sectors (2017 vs 2023)",
    x = "Sector",
    y = "Share within sector (%)",
    fill = "Employment status"
  ) +
  theme_minimal(base_size = 12) +
  theme(legend.position = "bottom",
        plot.title = element_text(face = "bold", size = 14))

Manufacturing’s share fell by 3.1 pp (20.6% → 17.5%), while Traditional Services rose 1.5 pp and Construction rose 0.6 pp. Modern services were essentially flat (+0.3 pp). This speaks to a weakening of manufacturing’s share of employment — a concern for structural transformation.

What the subgroup breakdowns reveal (the newsworthy pockets)

Gender — big female gains but the gap is still large Female WPR jumped from 30.2% → 38.5% (+8.3 pp), while male WPR moved from 73.8% → 77.1% (+3.3 pp). The male–female gap narrowed from about 43.6 pp to 38.6 pp. That narrowing is important — female labour-force attachment is rising — but women still participate at far lower rates than men. The Gender × Sector (2023) plot shows women remain heavily concentrated in agriculture (36.6% of employed women) and are under-represented in construction; men are more distributed across manufacturing, construction and traditional services. (See Gender gap in WPR over time and Sectoral Composition of Employment by Gender (2023).)

library(ggplot2)
library(dplyr)

# prepare gender × sector for 2017 & 2023
gender_sector_tbl2 <- design_scaled %>%
  filter(year %in% c(2017, 2023), employed == 1, !is.na(ind_group2), !is.na(sex)) %>%
  group_by(year, sex, ind_group2) %>%
  summarize(emp_weighted = survey_total(1, na.rm = TRUE), .groups = "drop") %>%
  group_by(year, sex) %>%
  mutate(share = 100 * emp_weighted / sum(emp_weighted)) %>%
  ungroup()

# plot with labels inside bars
ggplot(gender_sector_tbl2,
       aes(x = reorder(ind_group2, share), y = share, fill = sex)) +
  geom_col(position = position_dodge(width = 0.8), width = 0.7) +
  geom_text(aes(label = paste0(round(share,1), "%")),
            position = position_dodge(width = 0.8),
            hjust = -0.1, size = 2.4) +
  facet_wrap(~year) +
  coord_flip() +
  labs(
    title = "Sectoral Composition of Employment by Gender\n (2017 vs 2023)",
    subtitle = "Tamil Nadu: Gendered distribution across industries",
    x = "Sector", y = "Share of employed within gender (%)",
    fill = "Gender"
  ) +
  theme_minimal(base_size = 13) +
  theme(
    legend.position = "top",
    plot.title = element_text(face = "bold")
  ) +
  scale_y_continuous(expand = expansion(mult = c(0, 0.1))) # adds some space for labels

Youth (15–29) — more in education, big drop in measured unemployment Youth WPR was roughly stable (31.5% → 32.5%), but youth unemployment fell sharply from 11.4% → 6.7%, and the share in education increased 33.3% → 38.7%. That combination suggests either (a) many younger people returned to schooling or (b) labour demand improved for some youth. The improvement in youth unemployment is one of the clearest positive signals in the data. (See Youth labour outcomes (15–29).)

# - C. Youth employment crisis (15-29 age group) ---
youth_metrics <- design_scaled %>%
    filter(age >= 15 & age <= 29) %>%
    group_by(year) %>%
    summarize(
      youth_wpr = survey_mean(employed == 1, na.rm = TRUE) * 100,
      youth_unemployment = survey_mean(unemployed == 1, na.rm = TRUE) * 100,
      youth_in_education = survey_mean(cwas == 91, na.rm = TRUE) * 100
    ) %>%
    as_tibble()
  
library(knitr)
## Warning: package 'knitr' was built under R version 4.3.3
library(kableExtra)

youth_metrics %>%
  select(year, youth_wpr, youth_unemployment, youth_in_education) %>%
  mutate(
    youth_wpr = sprintf("%.1f%%", youth_wpr),
    youth_unemployment = sprintf("%.1f%%", youth_unemployment),
    youth_in_education = sprintf("%.1f%%", youth_in_education)
  ) %>%
  kable(
    caption = "Youth (15–29) labour outcomes, Tamil Nadu (2017 vs 2023)",
    align = c("c","c","c","c")
  ) %>%
  kable_styling(bootstrap_options = c("striped","hover","condensed","responsive"),
                full_width = FALSE)
Youth (15–29) labour outcomes, Tamil Nadu (2017 vs 2023)
year youth_wpr youth_unemployment youth_in_education
2017 31.5% 11.4% 33.3%
2023 32.5% 6.7% 38.7%

Caste — very different trajectories across groups Scheduled Tribes (ST) saw the highest agricultural concentration and an increase in that concentration: in_agri 57.4% → 62.0%. Scheduled Castes (SC) show a decline in the share working in agriculture (32.5% → 25.5%). OBCs rose modestly (21.4% → 24.5%), while the “Others” group fell sharply in agricultural share (6.3% → 2.15%). These divergent paths suggest that the “reversal” is not uniform — it concentrates more on some disadvantaged groups (notably STs). The Sectoral Concentration by Caste Group (2023) heatmap and the caste tables illustrate these patterns.

# (f) Sector Composition × Caste (Heatmap)
heatmap_data <- design_scaled %>%
  filter(year == 2023, employed == 1) %>%
  group_by(social_group, ind_group2) %>%
  summarize(count = survey_total(1, na.rm = TRUE)) %>%
  group_by(social_group) %>%
  mutate(share = 100 * count / sum(count)) %>%
  ungroup()

ggplot(heatmap_data, aes(x = ind_group2, y = social_group, fill = share)) +
  geom_tile(color = "white") +
  geom_text(aes(label = round(share, 1)), color = "white", fontface = "bold", size = 3) +
  scale_fill_gradient2(low = "#3288bd", mid = "#fee090", high = "#d53e4f", midpoint = 20) +
  labs(
    title = "Sectoral Concentration by Caste Group (2023)",
    subtitle = "Weighted employment shares by social group",
    x = "Sector", y = "Caste Group", fill = "Share (%)"
  ) +
  theme_minimal(base_size = 13) +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

Job quality — small improvement but still limited The share of workers in regular wage jobs rose from 37.8% → 39.9%; the metric we constructed for “quality” jobs (regular wage jobs within manufacturing and modern services) edged up only marginally (26.9% → 27.2%). In short: a higher share are in regular jobs than in 2017, but the gains are modest and not yet a transformation toward high-quality formal employment. (See Job Quality Metrics Over Time table and the sector × employment-type stacked bars.)

job_quality <- design_scaled %>%
  filter(employed == 1) %>%
  mutate(
    regular_wage = as.numeric(emp_status == "Regular_wage"),
    has_quality_job = as.numeric(
      emp_status == "Regular_wage" & 
      ind_group2 %in% c("Manufacturing", "Modern_Services")
    )
  ) %>%
  group_by(year) %>%
  summarize(
    pct_regular = survey_mean(regular_wage, na.rm = TRUE) * 100,
    pct_quality = survey_mean(has_quality_job, na.rm = TRUE) * 100
  )
print(job_quality)
## # A tibble: 2 × 5
##    year pct_regular pct_regular_se pct_quality pct_quality_se
##   <dbl>       <dbl>          <dbl>       <dbl>          <dbl>
## 1  2017        37.8          0.712        26.9          0.642
## 2  2023        39.9          0.725        27.2          0.664

Interpreting the pattern: is this a reversal of structural transformation? Across Tamil Nadu the picture is mixed. The small rise in agriculture’s share (+0.5 pp) by itself is not decisive. But when paired with (a) a notable drop in manufacturing’s share (-3.1 pp), (b) a rise in self-employment (+2.3 pp), and (c) only modest gains in higher-quality regular jobs, the evidence points to a partial reversal — or at least a stall — in the classic pattern of workers moving out of agriculture into higher-productivity manufacturing and modern services.

At the same time there are encouraging signs: overall WPR rose substantially and youth unemployment fell markedly. Female participation is increasing faster than male participation (a positive trend), even though the absolute female participation rate remains low.

Policy implications:

Manufacturing needs a revival: a loss of manufacturing share is worrying for productivity growth. Tamil Nadu should prioritise firm investment and skills that connect workers to higher-value factories and services.

Support for the self-employed and ST workers: the rise in self-employment and the high agricultural concentration among STs call for targeted credit, market access, and social protection that lifts incomes and reduces vulnerability.

Keep building women’s attachment to the labour market: female participation is rising — complementary measures (childcare, safe transport, training) would amplify the gains and narrow the participation gap further.