Note: This work is at an early stage; conference presentations for NARST and ASTE are currently under review. We welcome your feedback on any and all parts of this work.

1 Background (personal)

  • Studied State Educational Twitter Hashtags
  • Took Ken’s class! Got interested in influence
  • Examining influence in the context of a graduate certificate program (have self-reported and Twitter data for three cohorts)
  • Examining selection in the context of teachers, researchers, and administrators’ use of Twitter hashtag #NGSSchat

2 Background of the study

  • Social media is changing our understanding of whether, how and why changes in science teaching and learning happen.
  • Social media can assist with supporting changes (i.e., recent science education reform efforts) by allowing various stakeholders to interact outside of traditional educational learning environments
  • Although research has shown the importance of social media for supporting reform efforts in science education (Shelton & Ende, 2015), there are questions that remain to be answered: how individuals use social media to discuss science education reform related topics, who is using social media to discuss these topics, how are they are interacting, and what is being discussed.

2.1 Context

  • In this study, we explored answers to these questions in the context of twitter conversations using the hashtag #NGSSchat. A hashtag is a convention on Twitter that is used to organize conversations, and, allowing for regularly occurring (“synchronous”) chats at pre-specified times
  • While research on the educational uses of Twitter, and specifically Twitter hashtags, is new, there is a growing literature base on their uses (see Carpenter & Krutka, 2014, for a review).
  • However, very little research has been carried out in science education contexts, particularly on topics related to the NGSS.
  • RQ (for this presentation): What patterns of interactions between #NGSSchat participants are present?

3 Method

  • The study included all tweets that were archived in the #NGSSchat community between 2012 and 2017.
  • This resulted in 25,720 individual tweets, each of which were categorized into one of 103 chats via Storify1.
  • We used Storify because the #NGSSchat community chose to self-archive all of the tweets associated with the chats.
  • Tweet IDs and additional tweet data was collected via Storify and the Twitter Application Programming Interface (API).
  • Next, relational data was analyzed via sociograms or social network graphs using the R (R Core Team, 2018) software and the igraph (Csardi & Nepusz, 2006) and ggraph (Pederson, 2018) packages.
  • To then model the interactions, we used the lme4 package (Bates, Machler, & Bolker, 2015) and/or the ergm package (Hunter, Hancock, & Butts, 2008) to estimate an exponential random graph model (ERGM) suitable for exploring who interacts with whom.
  • To streamline the modeling and to aid in the interpretation, we chose to combine two of the types of interactions into two more general categories: In particular, we combined replying and mentioning into a category for “Conversing,” and retweeting or favoriting into one for “Endorsing.”

4 Overview of results

4.1 P2

  • Processing this data is difficult
  • Creating network data from Twitter data in a trustworthy way is challenging
  • Edgelist “spread” to adjancency matrix
  • Adjacency matrix “gathered” to long edgelist (seriously long)
  • Logistic mixed effects model with sender and receiver effects
  • Null model (just sender and receiver random effects) fit fine
  • Model with profile type interaction fit fine but took awhile to fit
  • Is this a ‘poor mans’ p2?
  • Had trouble running Peter Hoff’s routine

4.2 ERGM

  • Model that involves estimating structural and tie elements at the same time (using MCMC)
  • Has some convergence checks
  • Null model (just tie) seem to fit fine
  • Model with tie, mutual edges, and number of tweets seem to fit fine
  • Model with geodesically weighted edges and degree did not seem to converge with much confidence

4.3 Punchlines

  • Similar relationships seem to emerge regardless of which model or modeling approach is specified
  • Admin-admin and admin-research relationships seem to be key regarding selection
  • Teachers less selected by other

4.4 Questions for Ken’s research group

  • Is the (poor man’s) P2 model sufficient?
  • Or, is the simple ergm (without geodesically weighted terms) preferred?
  • Should I try to get the more sophisisticated ergm model to converge?

5 Loading, setting up

5.1 Conversing (mentioning and replying) data

#> # A tibble: 238,144 x 10
#>    sender    receiver      weight sender_profile receiver_profile sender_n_tweets receiver_n_tweets sum_var      tie weight_l
#>    <chr>     <chr>          <dbl> <fct>          <fct>                      <int>             <int> <chr>      <dbl>    <dbl>
#>  1 21stscied bravesearth        1 Administration Teacher                        6                20 Conversing     1     0   
#>  2 21stscied brunsell           3 Administration Other                          6                90 Conversing     1     1.10
#>  3 21stscied cduke62            0 Administration Unclear                        6                40 Conversing     0     0   
#>  4 21stscied fredende           6 Administration Administration                 6               948 Conversing     1     1.79
#>  5 21stscied jasperfoxsr        1 Administration Other                          6                92 Conversing     1     0   
#>  6 21stscied jenarnswald        1 Administration Administration                 6               103 Conversing     1     0   
#>  7 21stscied krsciencelady      0 Administration Administration                 6               230 Conversing     0     0   
#>  8 21stscied lpugh3             0 Administration Administration                 6                32 Conversing     0     0   
#>  9 21stscied reiserbrianj       1 Administration Policy/Research                6                 4 Conversing     1     0   
#> 10 21stscied tdishelton         0 Administration Teacher                        6               550 Conversing     0     0   
#> # ... with 238,134 more rows

5.2 Endorsing (retweeting and favoriting/liking)

#> # A tibble: 247,009 x 9
#>    sender    receiver     weight sender_profile receiver_profile sender_n_tweets receiver_n_tweets sum_var     tie
#>    <chr>     <chr>         <dbl> <fct>          <fct>                      <int>             <int> <chr>     <dbl>
#>  1 21stscied amykfmurphy       0 Administration Teacher                        6               129 Endorsing     0
#>  2 21stscied dtcampbe          0 Administration Policy/Research                6                74 Endorsing     0
#>  3 21stscied fredende          0 Administration Administration                 6               948 Endorsing     0
#>  4 21stscied karalu79          0 Administration Teacher                        6               145 Endorsing     0
#>  5 21stscied kastidham         0 Administration Teacher                        6                31 Endorsing     0
#>  6 21stscied reiserbrianj      0 Administration Policy/Research                6                 4 Endorsing     0
#>  7 21stscied starrscience      0 Administration Administration                 6               136 Endorsing     0
#>  8 21stscied tdishelton        0 Administration Teacher                        6               550 Endorsing     0
#>  9 21stscied 8blah8blah8       0 Administration Teacher                        6                 3 Endorsing     0
#> 10 21stscied alliebbogart      0 Administration Administration                 6                25 Endorsing     0
#> # ... with 246,999 more rows

6 ERGMs with full dataset

6.1 Conversing

#> 
#> ==========================
#> Summary of model fit
#> ==========================
#> 
#> Formula:   gg ~ edges + mutual + nodecov("n_tweets") + nodemix("profile_code", 
#>     base = 25)
#> 
#> Iterations:  7 out of 30 
#> 
#> Monte Carlo MLE Results:
#>                                                   Estimate Std. Error MCMC % z value Pr(>|z|)    
#> edges                                            -5.180056   0.233595      0 -22.175  < 1e-04 ***
#> mutual                                            3.985294   0.056652      0  70.347  < 1e-04 ***
#> nodecov.n_tweets                                  0.316842   0.004578      1  69.204  < 1e-04 ***
#> mix.profile_code.Administration.Administration    0.881836   0.239400      0   3.684 0.000230 ***
#> mix.profile_code.Other.Administration             0.927815   0.256076      0   3.623 0.000291 ***
#> mix.profile_code.Policy/Research.Administration   1.083574   0.249292      0   4.347  < 1e-04 ***
#> mix.profile_code.Teacher.Administration           0.746865   0.241910      0   3.087 0.002019 ** 
#> mix.profile_code.Unclear.Administration           0.735041   0.264143      0   2.783 0.005390 ** 
#> mix.profile_code.Administration.Other             0.489695   0.263146      0   1.861 0.062755 .  
#> mix.profile_code.Other.Other                      0.439478   0.278216      0   1.580 0.114193    
#> mix.profile_code.Policy/Research.Other            1.028006   0.259063      0   3.968  < 1e-04 ***
#> mix.profile_code.Teacher.Other                    0.412104   0.247085      0   1.668 0.095343 .  
#> mix.profile_code.Unclear.Other                    0.138546   0.319491      0   0.434 0.664545    
#> mix.profile_code.Administration.Policy/Research   0.415201   0.263286      0   1.577 0.114797    
#> mix.profile_code.Other.Policy/Research            0.362689   0.263793      0   1.375 0.169163    
#> mix.profile_code.Policy/Research.Policy/Research  0.749510   0.251597      0   2.979 0.002892 ** 
#> mix.profile_code.Teacher.Policy/Research          0.216281   0.247161      0   0.875 0.381541    
#> mix.profile_code.Unclear.Policy/Research          0.379460   0.286361      0   1.325 0.185134    
#> mix.profile_code.Administration.Teacher           0.635578   0.240539      0   2.642 0.008234 ** 
#> mix.profile_code.Other.Teacher                    0.686185   0.242986      0   2.824 0.004743 ** 
#> mix.profile_code.Policy/Research.Teacher          0.731733   0.244379      0   2.994 0.002751 ** 
#> mix.profile_code.Teacher.Teacher                  0.623727   0.232525      0   2.682 0.007309 ** 
#> mix.profile_code.Unclear.Teacher                  0.591886   0.247053      0   2.396 0.016585 *  
#> mix.profile_code.Administration.Unclear          -0.504453   0.313841      0  -1.607 0.107977    
#> mix.profile_code.Other.Unclear                    0.117043   0.315637      0   0.371 0.710776    
#> mix.profile_code.Policy/Research.Unclear          0.063245   0.297451      0   0.213 0.831621    
#> mix.profile_code.Teacher.Unclear                  0.080134   0.258251      0   0.310 0.756336    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#>      Null Deviance: 329461  on 237656  degrees of freedom
#>  Residual Deviance:  35741  on 237629  degrees of freedom
#>  
#> AIC: 35795    BIC: 36075    (Smaller is better.)

6.2 Endorsing

#> 
#> ==========================
#> Summary of model fit
#> ==========================
#> 
#> Formula:   gg ~ edges + mutual + nodecov("n_tweets") + nodemix("profile_code", 
#>     base = 25)
#> 
#> Iterations:  5 out of 30 
#> 
#> Monte Carlo MLE Results:
#>                                                   Estimate Std. Error MCMC % z value Pr(>|z|)    
#> edges                                            -4.304840   0.155870      0 -27.618  < 1e-04 ***
#> mutual                                            2.755104   0.041665      0  66.125  < 1e-04 ***
#> nodecov.n_tweets                                  0.298806   0.005544      0  53.894  < 1e-04 ***
#> mix.profile_code.Administration.Administration    0.800308   0.167877      0   4.767  < 1e-04 ***
#> mix.profile_code.Other.Administration             0.658639   0.169519      0   3.885 0.000102 ***
#> mix.profile_code.Policy/Research.Administration   0.695980   0.171350      0   4.062  < 1e-04 ***
#> mix.profile_code.Teacher.Administration           0.758270   0.161688      0   4.690  < 1e-04 ***
#> mix.profile_code.Unclear.Administration           0.342926   0.186515      0   1.839 0.065975 .  
#> mix.profile_code.Administration.Other             0.755279   0.171655      0   4.400  < 1e-04 ***
#> mix.profile_code.Other.Other                      0.456908   0.181746      0   2.514 0.011937 *  
#> mix.profile_code.Policy/Research.Other            0.739460   0.179328      0   4.124  < 1e-04 ***
#> mix.profile_code.Teacher.Other                    0.668976   0.163771      0   4.085  < 1e-04 ***
#> mix.profile_code.Unclear.Other                    0.277946   0.203963      0   1.363 0.172970    
#> mix.profile_code.Administration.Policy/Research   0.797051   0.167642      0   4.754  < 1e-04 ***
#> mix.profile_code.Other.Policy/Research            0.659795   0.171534      0   3.846 0.000120 ***
#> mix.profile_code.Policy/Research.Policy/Research  0.735273   0.168435      0   4.365  < 1e-04 ***
#> mix.profile_code.Teacher.Policy/Research          0.762931   0.163539      0   4.665  < 1e-04 ***
#> mix.profile_code.Unclear.Policy/Research          0.466388   0.190655      0   2.446 0.014435 *  
#> mix.profile_code.Administration.Teacher           0.611808   0.162195      0   3.772 0.000162 ***
#> mix.profile_code.Other.Teacher                    0.571671   0.161260      0   3.545 0.000393 ***
#> mix.profile_code.Policy/Research.Teacher          0.337815   0.162239      0   2.082 0.037324 *  
#> mix.profile_code.Teacher.Teacher                  0.652635   0.157122      0   4.154  < 1e-04 ***
#> mix.profile_code.Unclear.Teacher                  0.413473   0.166239      0   2.487 0.012874 *  
#> mix.profile_code.Administration.Unclear           0.109275   0.194849      0   0.561 0.574921    
#> mix.profile_code.Other.Unclear                    0.312056   0.198019      0   1.576 0.115051    
#> mix.profile_code.Policy/Research.Unclear          0.102418   0.197712      0   0.518 0.604446    
#> mix.profile_code.Teacher.Unclear                  0.269391   0.172532      0   1.561 0.118430    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#>      Null Deviance: 341738  on 246512  degrees of freedom
#>  Residual Deviance:  67824  on 246485  degrees of freedom
#>  
#> AIC: 67878    BIC: 68159    (Smaller is better.)

6.3 Checking fit

MCMC diagnostics (trace and density plot); simulated versus observed model; lots of output.

7 P2 with full dataset

7.1 Conversing

  tie
Predictors Odds Ratios CI p
(Intercept) 0.00 0.00 – 0.01 <0.001
sender_profileAdministration 0.74 0.38 – 1.43 0.370
sender_profileOther 0.72 0.34 – 1.51 0.381
sender_profilePolicy/Resebackground-color:#f2f2f2;h 0.63 0.31 – 1.28 0.201
sender_profileTeacher 0.80 0.44 – 1.45 0.465
receiver_profileAdministration 1.34 0.67 – 2.68 0.412
receiver_profileOther 0.67 0.31 – 1.44 0.309
receiver_profilePolicy/Resebackground-color:#f2f2f2;h 0.94 0.44 – 1.97 0.863
receiver_profileTeacher 0.94 0.50 – 1.76 0.836
sender_profileAdministration:receiver_profileAdministration 2.28 1.38 – 3.78 0.001
sender_profileOther:receiver_profileAdministration 1.81 1.01 – 3.25 0.046
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileAdministration 2.42 1.39 – 4.22 0.002
sender_profileTeacher:receiver_profileAdministration 1.48 0.93 – 2.36 0.102
sender_profileAdministration:receiver_profileOther 2.24 1.25 – 4.00 0.007
sender_profileOther:receiver_profileOther 2.36 1.21 – 4.59 0.012
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileOther 3.56 1.90 – 6.68 <0.001
sender_profileTeacher:receiver_profileOther 1.64 0.95 – 2.84 0.077
sender_profileAdministration:receiver_profilePolicy/Resebackground-color:#f2f2f2;h 1.95 1.12 – 3.40 0.018
sender_profileOther:receiver_profilePolicy/Resebackground-color:#f2f2f2;h 1.84 0.97 – 3.47 0.061
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profilePolicy/Resebackground-color:#f2f2f2;h 3.11 1.71 – 5.67 <0.001
sender_profileTeacher:receiver_profilePolicy/Resebackground-color:#f2f2f2;h 1.15 0.68 – 1.92 0.604
sender_profileAdministration:receiver_profileTeacher 1.73 1.07 – 2.81 0.026
sender_profileOther:receiver_profileTeacher 1.50 0.85 – 2.63 0.158
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileTeacher 1.56 0.91 – 2.65 0.104
sender_profileTeacher:receiver_profileTeacher 1.46 0.93 – 2.28 0.098
Random Effects
σ2 3.29
τ00 sender 2.11
τ00 receiver 2.33
ICC sender 0.27
ICC receiver 0.30
Observations 193457
Marginal R2 / Conditional R2 0.010 / 0.579

7.2 Endorsing

  tie
Predictors Odds Ratios CI p
(Intercept) 0.00 0.00 – 0.01 <0.001
sender_profileAdministration 0.98 0.52 – 1.82 0.939
sender_profileOther 1.02 0.52 – 2.01 0.945
sender_profilePolicy/Resebackground-color:#f2f2f2;h 0.80 0.41 – 1.57 0.514
sender_profileTeacher 1.14 0.65 – 2.01 0.647
receiver_profileAdministration 1.55 0.85 – 2.80 0.152
receiver_profileOther 1.44 0.75 – 2.76 0.272
receiver_profilePolicy/Resebackground-color:#f2f2f2;h 1.26 0.66 – 2.40 0.476
receiver_profileTeacher 1.76 1.03 – 3.00 0.040
sender_profileAdministration:receiver_profileAdministration 1.56 1.03 – 2.35 0.035
sender_profileOther:receiver_profileAdministration 1.14 0.73 – 1.79 0.561
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileAdministration 1.51 0.96 – 2.37 0.078
sender_profileTeacher:receiver_profileAdministration 1.18 0.81 – 1.74 0.385
sender_profileAdministration:receiver_profileOther 1.47 0.94 – 2.31 0.095
sender_profileOther:receiver_profileOther 0.89 0.54 – 1.47 0.644
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileOther 1.61 0.98 – 2.64 0.061
sender_profileTeacher:receiver_profileOther 1.16 0.76 – 1.76 0.504
sender_profileAdministration:receiver_profilePolicy/Resebackground-color:#f2f2f2;h 1.60 1.03 – 2.50 0.038
sender_profileOther:receiver_profilePolicy/Resebackground-color:#f2f2f2;h 1.15 0.71 – 1.88 0.561
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profilePolicy/Resebackground-color:#f2f2f2;h 1.67 1.02 – 2.71 0.040
sender_profileTeacher:receiver_profilePolicy/Resebackground-color:#f2f2f2;h 1.10 0.73 – 1.67 0.640
sender_profileAdministration:receiver_profileTeacher 1.13 0.77 – 1.66 0.522
sender_profileOther:receiver_profileTeacher 0.92 0.61 – 1.41 0.709
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileTeacher 0.89 0.58 – 1.37 0.605
sender_profileTeacher:receiver_profileTeacher 1.02 0.71 – 1.46 0.916
Random Effects
σ2 3.29
τ00 receiver 2.12
τ00 sender 2.27
ICC receiver 0.28
ICC sender 0.30
Observations 218064
Marginal R2 / Conditional R2 0.007 / 0.575

8 ERGMs with additional terms

8.1 Conversing

Seem to not converge after 30 iterations.

m4c <- ergm(ggc ~ edges + mutual + nodecov("n_tweets") + nodemix("profile_code", base = 25) + gwesp(decay = log(2), fixed = TRUE) + gwidegree(decay = log(2), fixed = TRUE) + gwodegree(decay = log(2), fixed = TRUE), control = control.ergm(MCMC.burnin = 50000, MCMC.interval = 5000, MCMLE.maxit = 40, MCMLE.steplength.margin = 0))  # unclear/unclear
saveRDS(m4c, "model-output/m4c.rds")
m4c <- read_rds("model-output/m4c.rds")
summary(m4c)
#> 
#> ==========================
#> Summary of model fit
#> ==========================
#> 
#> Formula:   ggc ~ edges + mutual + nodecov("n_tweets") + nodemix("profile_code", 
#>     base = 25) + gwesp(decay = 0.5, fixed = TRUE)
#> 
#> Iterations:  30 out of 30 
#> 
#> Monte Carlo MLE Results:
#>                                                   Estimate Std. Error MCMC % z value Pr(>|z|)    
#> edges                                            -6.272975   1.622223     99  -3.867 0.000110 ***
#> mutual                                            1.892370   0.017892      1 105.769  < 1e-04 ***
#> nodecov.n_tweets                                  0.085279   0.006604      1  12.913  < 1e-04 ***
#> mix.profile_code.Administration.Administration   -1.106317   0.082044      1 -13.484  < 1e-04 ***
#> mix.profile_code.Other.Administration            -0.625625   0.075845      1  -8.249  < 1e-04 ***
#> mix.profile_code.Policy/Research.Administration  -0.158592   0.085553      1  -1.854 0.063779 .  
#> mix.profile_code.Teacher.Administration          -0.745426   0.074628      1  -9.989  < 1e-04 ***
#> mix.profile_code.Unclear.Administration          -0.171879   0.089374      1  -1.923 0.054460 .  
#> mix.profile_code.Administration.Other            -0.352870   0.084681      1  -4.167  < 1e-04 ***
#> mix.profile_code.Other.Other                     -0.957744   0.088490      1 -10.823  < 1e-04 ***
#> mix.profile_code.Policy/Research.Other           -0.325640   0.097341      1  -3.345 0.000822 ***
#> mix.profile_code.Teacher.Other                   -0.971834   0.073387      1 -13.243  < 1e-04 ***
#> mix.profile_code.Unclear.Other                   -0.669168   0.109421      1  -6.116  < 1e-04 ***
#> mix.profile_code.Administration.Policy/Research  -1.018567   0.075778      1 -13.442  < 1e-04 ***
#> mix.profile_code.Other.Policy/Research           -0.561671   0.092426      1  -6.077  < 1e-04 ***
#> mix.profile_code.Policy/Research.Policy/Research -0.719630   0.093522      1  -7.695  < 1e-04 ***
#> mix.profile_code.Teacher.Policy/Research         -1.093955   0.080429      1 -13.601  < 1e-04 ***
#> mix.profile_code.Unclear.Policy/Research         -0.737393   0.116436      1  -6.333  < 1e-04 ***
#> mix.profile_code.Administration.Teacher          -0.982084   0.070596      1 -13.911  < 1e-04 ***
#> mix.profile_code.Other.Teacher                   -0.482484   0.069387      1  -6.954  < 1e-04 ***
#> mix.profile_code.Policy/Research.Teacher         -0.839626   0.084110      1  -9.982  < 1e-04 ***
#> mix.profile_code.Teacher.Teacher                 -0.604419   0.069263      1  -8.727  < 1e-04 ***
#> mix.profile_code.Unclear.Teacher                 -0.262638   0.086364      1  -3.041 0.002357 ** 
#> mix.profile_code.Administration.Unclear          -0.418361   0.091095      1  -4.593  < 1e-04 ***
#> mix.profile_code.Other.Unclear                   -0.316209   0.105881      1  -2.986 0.002822 ** 
#> mix.profile_code.Policy/Research.Unclear         -0.987229   0.105509      1  -9.357  < 1e-04 ***
#> mix.profile_code.Teacher.Unclear                 -0.578359   0.074814      1  -7.731  < 1e-04 ***
#> gwesp.fixed.0.5                                   3.323203   0.983904     97   3.378 0.000731 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#>      Null Deviance: 329461  on 237656  degrees of freedom
#>  Residual Deviance:  92873  on 237628  degrees of freedom
#>  
#> AIC: 92929    BIC: 93220    (Smaller is better.)

8.2 Endorsing

Seem to not converge after 30 iterations.

m4e <- readr::read_rds("model-output/m4e.rds")
summary(m4e)
#> 
#> ==========================
#> Summary of model fit
#> ==========================
#> 
#> Formula:   gg ~ edges + mutual + nodecov("n_tweets") + nodemix("profile_code", 
#>     base = 25) + gwesp(decay = 0.5, fixed = TRUE)
#> 
#> Iterations:  30 out of 30 
#> 
#> Monte Carlo MLE Results:
#>                                                    Estimate Std. Error MCMC % z value Pr(>|z|)    
#> edges                                            -10.349285   0.298438      0 -34.678  < 1e-04 ***
#> mutual                                             2.173988   0.045397      1  47.888  < 1e-04 ***
#> nodecov.n_tweets                                  -0.034813   0.009515      1  -3.659 0.000254 ***
#> mix.profile_code.Administration.Administration     0.235024   0.131632      1   1.785 0.074185 .  
#> mix.profile_code.Other.Administration              0.139955   0.140930      1   0.993 0.320668    
#> mix.profile_code.Policy/Research.Administration    0.153952   0.146222      1   1.053 0.292402    
#> mix.profile_code.Teacher.Administration            0.151482   0.127755      1   1.186 0.235732    
#> mix.profile_code.Unclear.Administration            0.130338   0.156951      0   0.830 0.406290    
#> mix.profile_code.Administration.Other              0.252747   0.135946      1   1.859 0.063003 .  
#> mix.profile_code.Other.Other                      -0.153611   0.164316      0  -0.935 0.349863    
#> mix.profile_code.Policy/Research.Other             0.183497   0.154311      1   1.189 0.234386    
#> mix.profile_code.Teacher.Other                     0.080058   0.132312      1   0.605 0.545131    
#> mix.profile_code.Unclear.Other                    -0.030400   0.159854      0  -0.190 0.849176    
#> mix.profile_code.Administration.Policy/Research    0.180857   0.137494      1   1.315 0.188383    
#> mix.profile_code.Other.Policy/Research             0.186132   0.151370      1   1.230 0.218829    
#> mix.profile_code.Policy/Research.Policy/Research   0.151923   0.141611      1   1.073 0.283352    
#> mix.profile_code.Teacher.Policy/Research           0.150925   0.127291      1   1.186 0.235753    
#> mix.profile_code.Unclear.Policy/Research          -0.025411   0.177621      0  -0.143 0.886242    
#> mix.profile_code.Administration.Teacher            0.096228   0.128677      1   0.748 0.454567    
#> mix.profile_code.Other.Teacher                     0.147605   0.134178      1   1.100 0.271302    
#> mix.profile_code.Policy/Research.Teacher          -0.362993   0.137983      1  -2.631 0.008520 ** 
#> mix.profile_code.Teacher.Teacher                   0.099071   0.123174      1   0.804 0.421215    
#> mix.profile_code.Unclear.Teacher                   0.073554   0.138883      1   0.530 0.596382    
#> mix.profile_code.Administration.Unclear           -0.143897   0.152583      1  -0.943 0.345643    
#> mix.profile_code.Other.Unclear                    -0.014768   0.172178      0  -0.086 0.931646    
#> mix.profile_code.Policy/Research.Unclear          -0.550616   0.203522      0  -2.705 0.006821 ** 
#> mix.profile_code.Teacher.Unclear                  -0.088302   0.140419      1  -0.629 0.529447    
#> gwesp.fixed.0.5                                    4.125327   0.160791      0  25.656  < 1e-04 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#>      Null Deviance: 341738  on 246512  degrees of freedom
#>  Residual Deviance:  63683  on 246484  degrees of freedom
#>  
#> AIC: 63739    BIC: 64031    (Smaller is better.)

8.3 Checking fit

These seem to take forever to run too!