Dataset information
Number of Agencies in Dataset
##
## AB AF AG AJ AK AL AM AR AU BD BF BG BO BT CC CM CT CU DD DJ DL DN DR EB ED EE EP EW FC FJ FM FQ FT FY GG GS GW HB HE HF HP HS HU IB IF IG IN KS LF MC MR NF NL NM NN NP NQ NU NV OM OS PJ RE RF RR SB SE SI SK SN SS ST SW SZ TB TC TD TN TR TX VA
## 63 338448 422545 242 213 191 26229 607353 172 701 388 4389 1920 43 44 220996 2835 8265 299767 325521 113871 109224 12289 793 36200 15078 105795 126 6992 148 680 4717 6308 291 155 95118 264 41 536710 2393 52 753190 64924 8414 148 246 332937 2445 343 252 919 10516 7231 43 112441 63 20420 32349 430117 37085 181 197 68 131 3363 21662 25622 37274 1152 3015 358 89236 46 249971 470 655 241172 305 552331 368 214022##
## AB AF AG AJ AK AL AM AR AU BD BF BG BO BT CC CM CT CU DD DJ DL DN DR EB ED EE EP EW FC FJ FM FQ FT FY GG GS GW HB HE HF HP HS HU IB IF IG IN KS LF MC MR NF NL NM NN NP NQ NU NV OM OS PJ RE RF RR SB SE SI SK SN SS ST SW SZ TB TC TD TN TR TX VA
## 2010 26 10560 13911 87 97 48 749 19413 81 161 68 564 267 22 21 14956 301 576 8781 18357 4273 6648 687 0 2350 1088 8445 35 824 22 197 720 478 291 0 2727 89 21 25601 0 25 10189 3591 986 23 51 28322 352 185 87 919 591 918 22 8593 31 1986 2503 12378 2644 33 46 36 60 634 1501 2151 24 210 423 89 1871 24 8940 250 170 9617 110 16712 99 2537
## 2011 0 8775 14588 97 116 49 1243 18827 91 143 84 555 286 21 23 18071 387 510 8006 21488 7482 5613 971 163 2891 1252 8584 26 862 33 176 556 597 0 0 2491 175 20 23102 314 27 15506 5365 1089 22 73 7051 405 158 88 0 728 665 21 9240 32 1855 2612 13337 3463 33 49 32 71 611 1621 809 47 230 384 107 2422 22 7069 220 168 10203 88 17985 93 13707
## 2012 0 61907 42569 0 0 0 2229 77948 0 0 0 579 0 0 0 19872 0 724 25003 35023 7653 6467 748 0 2677 1263 8847 0 725 0 0 569 541 0 0 6592 0 0 29146 0 0 82218 4741 829 0 0 27287 0 0 0 0 789 787 0 9296 0 2085 2709 67604 3140 0 0 0 0 590 1401 2492 4764 0 0 0 10379 0 41409 0 0 25892 0 54890 0 13303
## 2013 0 12129 13256 0 0 47 2266 22130 0 135 82 401 0 0 0 9447 355 0 11230 17004 5205 6707 893 228 2658 1180 3924 31 814 30 179 714 565 0 55 8429 0 0 32329 424 0 39090 3741 1156 34 61 18396 452 0 0 0 888 726 0 9985 0 1617 2509 19518 2929 42 62 0 0 0 1511 2422 3918 0 376 89 2551 0 8345 0 0 23204 107 50010 98 29893
## 2014 37 19168 20162 0 0 47 2045 22414 0 124 73 0 0 0 0 9892 397 0 13982 17213 10953 6515 0 185 2415 0 3863 34 702 32 128 0 0 0 53 8567 0 0 32806 393 0 42798 3890 0 36 61 18384 418 0 0 0 0 0 0 9430 0 0 2467 19461 3596 41 40 0 0 0 0 2472 13318 0 323 73 3776 0 9540 0 0 11673 0 51038 78 27639
## 2015 0 18776 20624 58 0 0 2004 21003 0 138 81 0 0 0 0 10129 0 0 15249 20218 11359 8469 0 217 2701 0 4456 0 573 31 0 0 0 0 47 7874 0 0 36772 350 0 43090 5404 0 33 0 26366 492 0 77 0 0 0 0 9936 0 0 2675 17891 3378 32 0 0 0 0 0 1921 15203 0 0 0 4060 0 10527 0 0 15598 0 51700 0 32236
## 2016 0 15586 22878 0 0 0 2285 17086 0 0 0 569 351 0 0 9784 0 661 13597 16501 11262 8075 1150 0 2862 1510 10156 0 648 0 0 618 641 0 0 7081 0 0 40345 0 0 46991 5464 904 0 0 23098 0 0 0 0 854 882 0 11202 0 1870 2152 12361 3196 0 0 0 0 320 1383 3213 0 0 0 0 5256 0 8907 0 0 14871 0 45497 0 30313
## 2017 0 16899 48953 0 0 0 2087 21850 0 0 0 610 343 0 0 10480 508 665 15922 16126 8837 8589 1070 0 2831 1416 9414 0 715 0 0 542 612 0 0 7532 0 0 43086 0 0 47414 4960 1070 0 0 25867 0 0 0 0 910 850 0 11814 0 1861 2442 16022 2914 0 0 0 0 449 1512 3526 0 0 537 0 4294 0 8501 0 0 16835 0 46368 0 64394
## 2018 0 33351 43352 0 0 0 1837 70005 0 0 0 526 338 0 0 20725 476 633 30877 30978 8075 8624 1115 0 2592 1379 7972 0 594 0 0 470 638 0 0 7157 0 0 43029 412 0 73899 4628 829 0 0 28290 326 0 0 0 940 859 0 11568 0 1743 2308 47882 3069 0 0 0 0 413 1543 3394 0 355 520 0 7228 0 26318 0 0 21552 0 42027 0 0
## 2019 0 31348 36529 0 0 0 2010 85639 0 0 0 585 335 0 0 19847 411 891 31418 28199 7949 8565 1162 0 2167 1235 8352 0 535 0 0 528 625 0 0 7095 0 0 51703 500 0 76883 3763 765 0 0 26815 0 0 0 0 865 768 0 10789 0 1697 2174 51318 3049 0 0 0 0 346 1388 3222 0 357 452 0 9713 0 27933 0 317 20414 0 41771 0 0
## 2020 0 35476 33399 0 0 0 2097 82155 0 0 0 0 0 0 0 21310 0 926 36435 29013 7187 8904 1148 0 2367 1311 8115 0 0 0 0 0 0 0 0 7332 0 0 50393 0 0 84704 4509 0 0 0 26114 0 0 0 0 927 776 0 10588 0 1494 2166 52153 1343 0 0 0 0 0 1333 0 0 0 0 0 10933 0 28651 0 0 22246 0 40347 0 0
## 2021 0 13848 20634 0 0 0 1625 35772 0 0 0 0 0 0 0 9568 0 867 17055 15454 6872 7980 1071 0 2218 963 6684 0 0 0 0 0 0 0 0 6651 0 0 19061 0 0 25638 4361 786 0 0 13984 0 0 0 0 905 0 0 0 0 1303 1835 21101 1253 0 0 0 0 0 1382 0 0 0 0 0 9431 0 11546 0 0 8454 0 15860 0 0
## 2022 0 31191 43332 0 0 0 1769 60278 0 0 0 0 0 0 0 21009 0 904 31888 25866 7550 8587 1079 0 2698 1102 7757 0 0 0 0 0 782 0 0 7498 0 0 50317 0 0 73070 4866 0 0 0 27014 0 0 0 0 1049 0 0 0 0 1407 1889 39890 1516 0 0 0 0 0 3524 0 0 0 0 0 7962 0 26528 0 0 19989 0 35764 0 0
## 2023 0 29434 48358 0 0 0 1983 52833 0 0 0 0 0 0 0 25906 0 908 40324 34081 9214 9481 1195 0 2773 1379 9226 0 0 0 0 0 829 0 0 8092 0 0 59020 0 0 91700 5641 0 0 0 35949 0 0 0 0 1070 0 0 0 0 1502 1908 39201 1595 0 0 0 0 0 3563 0 0 0 0 0 9360 0 25757 0 0 20624 0 42362 0 0## [1] "# Of agencies: 81"Reliabilities
##
## Pearson's product-moment correlation
##
## data: mut_1 and mut_2
## t = 2070.5, df = 5839001, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## 0.6501893 0.6511247
## sample estimates:
## cor
## 0.6506572##
## Reliability analysis
## Call: psych::alpha(x = as.data.frame(cbind(voice_1, voice_2, voice_3)))
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r
## 0.81 0.81 0.74 0.58 4.2 0.00013 3.6 1 0.56
##
## 95% confidence boundaries
## lower alpha upper
## Feldt 0.81 0.81 0.81
## Duhachek 0.81 0.81 0.81
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
## voice_1 0.70 0.70 0.54 0.54 2.3 0.00023 NA 0.54
## voice_2 0.79 0.79 0.65 0.65 3.8 0.00016 NA 0.65
## voice_3 0.72 0.72 0.56 0.56 2.5 0.00022 NA 0.56
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## voice_1 6480807 0.87 0.87 0.77 0.69 3.6 1.2
## voice_2 6269837 0.83 0.82 0.66 0.60 3.7 1.2
## voice_3 6310223 0.86 0.86 0.76 0.68 3.4 1.1
##
## Non missing response frequency for each item
## 1 2 3 4 5 miss
## voice_1 0.07 0.14 0.17 0.38 0.25 0.01
## voice_2 0.08 0.08 0.17 0.37 0.29 0.04
## voice_3 0.06 0.17 0.22 0.38 0.16 0.04Correlations
Analyses
Multilevel models
Initial analyses
With weights
## Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
## Formula: voice ~ mut + (1 | agency) + (1 | year)
## Data: full_data_means
## Weights: weight
##
## REML criterion at convergence: 14724487
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -43.56 -0.45 0.02 0.52 43.90
##
## Random effects:
## Groups Name Variance Std.Dev.
## agency (Intercept) 0.007685 0.0877
## year (Intercept) 0.000598 0.0245
## Residual 1.825365 1.3511
## Number of obs: 5919194, groups: agency, 81; year, 13
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 1.380508 0.012947 71.529301 107 <0.0000000000000002 ***
## mut 0.666167 0.000269 5917287.303678 2481 <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## mut -0.068Without weights
## Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
## Formula: voice ~ mut + (1 | agency) + (1 | year)
## Data: full_data_means
##
## REML criterion at convergence: 13786035
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -5.556 -0.572 0.009 0.575 4.501
##
## Random effects:
## Groups Name Variance Std.Dev.
## agency (Intercept) 0.009977 0.0999
## year (Intercept) 0.000468 0.0216
## Residual 0.483270 0.6952
## Number of obs: 6531290, groups: agency, 81; year, 14
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 1.377342 0.013125 83.349772 105 <0.0000000000000002 ***
## mut 0.668849 0.000254 6525742.363739 2629 <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## mut -0.065With controls
With weights
## Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
## Formula: voice ~ mut + sex + ethnicity + tenure_fed + supervisor + leaving + (1 | agency) + (1 | year)
## Data: full_data_means
## Weights: weight
##
## REML criterion at convergence: 12779349
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -42.71 -0.47 0.06 0.51 44.29
##
## Random effects:
## Groups Name Variance Std.Dev.
## agency (Intercept) 0.00704 0.0839
## year (Intercept) 0.00113 0.0336
## Residual 1.68748 1.2990
## Number of obs: 5287598, groups: agency, 48; year, 12
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 1.520200 0.018545 29.106058 81.97 < 0.0000000000000002 ***
## mut 0.629867 0.000293 5287175.319339 2146.17 < 0.0000000000000002 ***
## sexA 0.062534 0.002812 5287572.685071 22.24 < 0.0000000000000002 ***
## sexB 0.059525 0.002820 5287574.402828 21.11 < 0.0000000000000002 ***
## ethnicity1 0.091109 0.019592 10.368777 4.65 0.00082 ***
## ethnicity2 0.102719 0.019588 10.359603 5.24 0.00034 ***
## ethnicityA 0.055649 0.002815 5287144.568650 19.77 < 0.0000000000000002 ***
## ethnicityB 0.080570 0.002673 5287567.683738 30.15 < 0.0000000000000002 ***
## tenure_fedA 0.009839 0.003137 5285300.746029 3.14 0.00171 **
## tenure_fedB -0.052203 0.003123 5286749.413308 -16.72 < 0.0000000000000002 ***
## tenure_fedC -0.072261 0.003143 5286578.919028 -22.99 < 0.0000000000000002 ***
## tenure_fedD -0.087635 0.003683 5145227.064350 -23.79 < 0.0000000000000002 ***
## tenure_fedE -0.114120 0.004069 5188862.561270 -28.04 < 0.0000000000000002 ***
## tenure_fedF -0.132387 0.003971 5176026.497560 -33.34 < 0.0000000000000002 ***
## tenure_fedG -0.145774 0.003523 5101320.499105 -41.38 < 0.0000000000000002 ***
## supervisorA -0.112752 0.003011 5287285.609074 -37.45 < 0.0000000000000002 ***
## supervisorB 0.012422 0.003083 5287329.480108 4.03 0.000056 ***
## supervisorC 0.073001 0.004207 5287175.438162 17.35 < 0.0000000000000002 ***
## leavingA 0.066117 0.002174 5280552.310546 30.41 < 0.0000000000000002 ***
## leavingB -0.178988 0.002273 5281208.650322 -78.75 < 0.0000000000000002 ***
## leavingC -0.217301 0.002355 5282280.194754 -92.25 < 0.0000000000000002 ***
## leavingD -0.195472 0.002397 5281846.423015 -81.55 < 0.0000000000000002 ***
## leavingE -0.230516 0.004375 5284238.675256 -52.69 < 0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Without weights
## Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
## Formula: voice ~ mut + sex + ethnicity + tenure_fed + supervisor + leaving + (1 | agency) + (1 | year)
## Data: full_data_means
##
## REML criterion at convergence: 12148710
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -5.852 -0.600 0.068 0.634 4.750
##
## Random effects:
## Groups Name Variance Std.Dev.
## agency (Intercept) 0.007519 0.0867
## year (Intercept) 0.000926 0.0304
## Residual 0.458971 0.6775
## Number of obs: 5899694, groups: agency, 48; year, 13
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 1.529325 0.017158 40.289125 89.13 < 0.0000000000000002 ***
## mut 0.628183 0.000277 5899090.140446 2264.01 < 0.0000000000000002 ***
## sexA 0.082238 0.002331 5899665.489562 35.27 < 0.0000000000000002 ***
## sexB 0.076458 0.002333 5899670.223240 32.77 < 0.0000000000000002 ***
## ethnicity1 0.071006 0.017099 11.390145 4.15 0.00150 **
## ethnicity2 0.092765 0.017091 11.368851 5.43 0.00019 ***
## ethnicityA 0.050539 0.002343 5899098.287757 21.57 < 0.0000000000000002 ***
## ethnicityB 0.077506 0.002198 5899664.055899 35.26 < 0.0000000000000002 ***
## tenure_fedA 0.016028 0.002799 5893223.592616 5.73 0.00000001 ***
## tenure_fedB -0.050892 0.002790 5896837.883796 -18.24 < 0.0000000000000002 ***
## tenure_fedC -0.072203 0.002805 5895926.148511 -25.74 < 0.0000000000000002 ***
## tenure_fedD -0.090710 0.004113 3031608.235408 -22.05 < 0.0000000000000002 ***
## tenure_fedE -0.126851 0.004602 3681722.647702 -27.57 < 0.0000000000000002 ***
## tenure_fedF -0.152017 0.004362 3357853.234664 -34.85 < 0.0000000000000002 ***
## tenure_fedG -0.147942 0.003702 2382577.610029 -39.97 < 0.0000000000000002 ***
## supervisorA -0.131069 0.002658 5898987.397971 -49.31 < 0.0000000000000002 ***
## supervisorB 0.002493 0.002722 5899325.346892 0.92 0.35961
## supervisorC 0.076089 0.004132 5893980.828565 18.41 < 0.0000000000000002 ***
## leavingA 0.078987 0.001942 5893919.934822 40.66 < 0.0000000000000002 ***
## leavingB -0.175154 0.002046 5894340.079018 -85.61 < 0.0000000000000002 ***
## leavingC -0.217759 0.002127 5895517.304880 -102.39 < 0.0000000000000002 ***
## leavingD -0.173069 0.002170 5894410.452706 -79.75 < 0.0000000000000002 ***
## leavingE -0.277810 0.005702 5881374.036949 -48.72 < 0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1