Note: This work is at an early stage; conference presentations for NARST and ASTE are currently under review. We welcome your feedback on any and all parts of this work.

Background of the study

Social media is changing our understanding of whether, how and why changes in science teaching and learning happen.
Social media can assist with supporting changes (i.e., recent science education reform efforts) by allowing various stakeholders to interact outside of traditional educational learning environments
Although research has shown the importance of social media for supporting reform efforts in science education (Shelton & Ende, 2015), there are questions that remain to be answered: how individuals use social media to discuss science education reform related topics, who is using social media to discuss these topics, how are they are interacting, and what is being discussed.

Context of the study

In this study, we explored answers to these questions in the context of twitter conversations using the hashtag #NGSSchat. A hashtag is a convention on Twitter that is used to organize conversations, and, allowing for regularly occurring (“synchronous”) chats at pre-specified times
While research on the educational uses of Twitter, and specifically Twitter hashtags, is new, there is a growing literature base on their uses (see Carpenter & Krutka, 2014, for a review).
However, very little research has been carried out in science education contexts, particularly on topics related to the NGSS.
RQ (for this presentation): What patterns of interactions between #NGSSchat participants are present?

Method

The study included all tweets that were archived in the #NGSSchat community between 2012 and 2017.
This resulted in 25,720 individual tweets, each of which were categorized into one of 103 chats via Storify1.
We used Storify because the #NGSSchat community chose to self-archive all of the tweets associated with the chats.
Tweet IDs and additional tweet data was collected via Storify and the Twitter Application Programming Interface (API).
Next, relational data was analyzed via sociograms or social network graphs using the R (R Core Team, 2018) software and the igraph (Csardi & Nepusz, 2006) and ggraph (Pederson, 2018) packages.
To then model the interactions, we used the lme4 package (Bates, Machler, & Bolker, 2015) and/or the ergm package (Hunter, Hancock, & Butts, 2008) to estimate an exponential random graph model (ERGM) suitable for exploring who interacts with whom.
To streamline the modeling and to aid in the interpretation, we chose to combine two of the types of interactions into two more general categories: In particular, we combined replying and mentioning into a category for “Conversing,” and retweeting or favoriting into one for “Endorsing.”

Loading, setting up

Conversing (mentioning and replying) data

#> # A tibble: 238,144 x 10
#>    sender    receiver      weight sender_profile receiver_profile sender_n_tweets receiver_n_tweets sum_var      tie weight_l
#>    <chr>     <chr>          <dbl> <fct>          <fct>                      <int>             <int> <chr>      <dbl>    <dbl>
#>  1 21stscied bravesearth        1 Administration Teacher                        6                20 Conversing     1     0   
#>  2 21stscied brunsell           3 Administration Other                          6                90 Conversing     1     1.10
#>  3 21stscied cduke62            0 Administration Unclear                        6                40 Conversing     0     0   
#>  4 21stscied fredende           6 Administration Administration                 6               948 Conversing     1     1.79
#>  5 21stscied jasperfoxsr        1 Administration Other                          6                92 Conversing     1     0   
#>  6 21stscied jenarnswald        1 Administration Administration                 6               103 Conversing     1     0   
#>  7 21stscied krsciencelady      0 Administration Administration                 6               230 Conversing     0     0   
#>  8 21stscied lpugh3             0 Administration Administration                 6                32 Conversing     0     0   
#>  9 21stscied reiserbrianj       1 Administration Policy/Research                6                 4 Conversing     1     0   
#> 10 21stscied tdishelton         0 Administration Teacher                        6               550 Conversing     0     0   
#> # ... with 238,134 more rows

Endorsing (retweeting and favoriting/liking)

#> # A tibble: 247,009 x 9
#>    sender    receiver     weight sender_profile receiver_profile sender_n_tweets receiver_n_tweets sum_var     tie
#>    <chr>     <chr>         <dbl> <fct>          <fct>                      <int>             <int> <chr>     <dbl>
#>  1 21stscied amykfmurphy       0 Administration Teacher                        6               129 Endorsing     0
#>  2 21stscied dtcampbe          0 Administration Policy/Research                6                74 Endorsing     0
#>  3 21stscied fredende          0 Administration Administration                 6               948 Endorsing     0
#>  4 21stscied karalu79          0 Administration Teacher                        6               145 Endorsing     0
#>  5 21stscied kastidham         0 Administration Teacher                        6                31 Endorsing     0
#>  6 21stscied reiserbrianj      0 Administration Policy/Research                6                 4 Endorsing     0
#>  7 21stscied starrscience      0 Administration Administration                 6               136 Endorsing     0
#>  8 21stscied tdishelton        0 Administration Teacher                        6               550 Endorsing     0
#>  9 21stscied 8blah8blah8       0 Administration Teacher                        6                 3 Endorsing     0
#> 10 21stscied alliebbogart      0 Administration Administration                 6                25 Endorsing     0
#> # ... with 246,999 more rows

0. Null P2 models predicting tie

0A. Conversing

There appear to be moderate-large sender and receiver effects (in terms of random effects):

#> 
#> Generalized linear mixed model
#>  Family: binomial (logit)
#> Formula: tie ~ 1 + (1 | sender) + (1 | receiver)
#> 
#>     ICC (sender): 0.255228
#>   ICC (receiver): 0.386557

0B. Endorsing

There appear to be small sender and receiver effects for endorsing, too (in terms of random effects):

#> 
#> Generalized linear mixed model
#>  Family: binomial (logit)
#> Formula: tie ~ 1 + (1 | sender) + (1 | receiver)
#> 
#>     ICC (sender): 0.333777
#>   ICC (receiver): 0.290476

1. Initially built P2 models predicting tie

Sender and receive random effects; sender and receiver n-tweets; sender and receive profile

1A. Conversing

	tie
Predictors	Odds Ratios	CI	p
(Intercept)	0.00	0.00 – 0.01	<0.001
I(sender n tweets/10)	1.15	1.13 – 1.17	<0.001
I(receiver n tweets/10)	1.15	1.13 – 1.17	<0.001
sender_profileAdministration	0.51	0.27 – 0.98	0.044
sender_profileOther	0.65	0.32 – 1.35	0.252
sender_profilePolicy/Resebackground-color:#f2f2f2;h	0.52	0.26 – 1.06	0.071
sender_profileTeacher	0.71	0.39 – 1.29	0.262
receiver_profileAdministration	0.84	0.42 – 1.66	0.613
receiver_profileOther	0.60	0.28 – 1.28	0.186
receiver_profilePolicy/Resebackground-color:#f2f2f2;h	0.74	0.36 – 1.54	0.422
receiver_profileTeacher	0.81	0.43 – 1.53	0.522
sender_profileAdministration:receiver_profileAdministration	2.33	1.31 – 4.12	0.004
sender_profileOther:receiver_profileAdministration	1.80	0.94 – 3.45	0.078
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileAdministration	2.44	1.31 – 4.57	0.005
sender_profileTeacher:receiver_profileAdministration	1.48	0.87 – 2.52	0.150
sender_profileAdministration:receiver_profileOther	2.22	1.14 – 4.29	0.018
sender_profileOther:receiver_profileOther	2.31	1.10 – 4.84	0.026
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileOther	3.51	1.73 – 7.13	<0.001
sender_profileTeacher:receiver_profileOther	1.62	0.87 – 3.02	0.129
sender_profileAdministration:receiver_profilePolicy/Resebackground-color:#f2f2f2;h	1.98	1.07 – 3.67	0.031
sender_profileOther:receiver_profilePolicy/Resebackground-color:#f2f2f2;h	1.82	0.90 – 3.67	0.094
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profilePolicy/Resebackground-color:#f2f2f2;h	3.13	1.60 – 6.10	0.001
sender_profileTeacher:receiver_profilePolicy/Resebackground-color:#f2f2f2;h	1.15	0.64 – 2.05	0.646
sender_profileAdministration:receiver_profileTeacher	1.73	1.01 – 2.98	0.046
sender_profileOther:receiver_profileTeacher	1.48	0.79 – 2.75	0.218
sender_profilePolicy/Resebackground-color:#f2f2f2;h:receiver_profileTeacher	1.55	0.85 – 2.81	0.151
sender_profileTeacher:receiver_profileTeacher	1.45	0.87 – 2.41	0.149
Random Effects
σ²	3.29
τ₀₀ _sender	1.25
τ₀₀ _receiver	1.34
ICC _sender	0.21
ICC _receiver	0.23
Observations	193457
Marginal R² / Conditional R²	0.214 / 0.561

1B. Endorsing

#> Some variance components equal zero. Respecify random structure!

	tie
Predictors	Odds Ratios	CI	p
(Intercept)	0.00	0.00 – 0.01	<0.001
I(sender n tweets/10)	1.12	1.10 – 1.15	<0.001
I(receiver n tweets/10)	1.15	1.13 – 1.17	<0.001
Random Effects
σ²	3.29
τ₀₀ _receiver	1.34
τ₀₀ _sender	1.71
τ₀₀ _{sender_profile:receiver_profile}	0.01
τ₀₀ _{receiver_profile}	0.00
τ₀₀ _{sender_profile}	0.00
ICC _receiver	0.21
ICC _sender	0.27
ICC _{sender_profile:receiver_profile}	0.00
ICC _{receiver_profile}	0.00
ICC _{sender_profile}	0.00
Observations	218064

2. Same P2 models but ssing just a subset of the data - participants with more than five tweets sent to the hashtag

2A. Conversing

#> Some variance components equal zero. Respecify random structure!

	tie
Predictors	Odds Ratios	CI	p
(Intercept)	0.00	0.00 – 0.01	<0.001
I(sender n tweets/10)	1.13	1.11 – 1.15	<0.001
I(receiver n tweets/10)	1.14	1.12 – 1.16	<0.001
Random Effects
σ²	3.29
τ₀₀ _sender	1.12
τ₀₀ _receiver	1.27
τ₀₀ _{sender_profile:receiver_profile}	0.03
τ₀₀ _{receiver_profile}	0.00
τ₀₀ _{sender_profile}	0.00
ICC _sender	0.20
ICC _receiver	0.22
ICC _{sender_profile:receiver_profile}	0.01
ICC _{receiver_profile}	0.00
ICC _{sender_profile}	0.00
Observations	81900

2B. Endorsing

#> Some variance components equal zero. Respecify random structure!

	tie
Predictors	Odds Ratios	CI	p
(Intercept)	0.01	0.01 – 0.02	<0.001
I(sender n tweets/10)	1.10	1.08 – 1.13	<0.001
I(receiver n tweets/10)	1.12	1.10 – 1.14	<0.001
Random Effects
σ²	3.29
τ₀₀ _receiver	1.06
τ₀₀ _sender	1.72
τ₀₀ _{sender_profile:receiver_profile}	0.01
τ₀₀ _{receiver_profile}	0.00
τ₀₀ _{sender_profile}	0.00
ICC _receiver	0.17
ICC _sender	0.28
ICC _{sender_profile:receiver_profile}	0.00
ICC _{receiver_profile}	0.00
ICC _{sender_profile}	0.00
Observations	83220

3. ERGMs with full dataset

3A. Conversing

#> 
#> ==========================
#> Summary of model fit
#> ==========================
#> 
#> Formula:   gg ~ edges + mutual + nodecov("n_tweets") + nodemix("profile_code", 
#>     base = 25)
#> 
#> Iterations:  7 out of 30 
#> 
#> Monte Carlo MLE Results:
#>                                                   Estimate Std. Error MCMC % z value Pr(>|z|)    
#> edges                                            -5.180056   0.233595      0 -22.175  < 1e-04 ***
#> mutual                                            3.985294   0.056652      0  70.347  < 1e-04 ***
#> nodecov.n_tweets                                  0.316842   0.004578      1  69.204  < 1e-04 ***
#> mix.profile_code.Administration.Administration    0.881836   0.239400      0   3.684 0.000230 ***
#> mix.profile_code.Other.Administration             0.927815   0.256076      0   3.623 0.000291 ***
#> mix.profile_code.Policy/Research.Administration   1.083574   0.249292      0   4.347  < 1e-04 ***
#> mix.profile_code.Teacher.Administration           0.746865   0.241910      0   3.087 0.002019 ** 
#> mix.profile_code.Unclear.Administration           0.735041   0.264143      0   2.783 0.005390 ** 
#> mix.profile_code.Administration.Other             0.489695   0.263146      0   1.861 0.062755 .  
#> mix.profile_code.Other.Other                      0.439478   0.278216      0   1.580 0.114193    
#> mix.profile_code.Policy/Research.Other            1.028006   0.259063      0   3.968  < 1e-04 ***
#> mix.profile_code.Teacher.Other                    0.412104   0.247085      0   1.668 0.095343 .  
#> mix.profile_code.Unclear.Other                    0.138546   0.319491      0   0.434 0.664545    
#> mix.profile_code.Administration.Policy/Research   0.415201   0.263286      0   1.577 0.114797    
#> mix.profile_code.Other.Policy/Research            0.362689   0.263793      0   1.375 0.169163    
#> mix.profile_code.Policy/Research.Policy/Research  0.749510   0.251597      0   2.979 0.002892 ** 
#> mix.profile_code.Teacher.Policy/Research          0.216281   0.247161      0   0.875 0.381541    
#> mix.profile_code.Unclear.Policy/Research          0.379460   0.286361      0   1.325 0.185134    
#> mix.profile_code.Administration.Teacher           0.635578   0.240539      0   2.642 0.008234 ** 
#> mix.profile_code.Other.Teacher                    0.686185   0.242986      0   2.824 0.004743 ** 
#> mix.profile_code.Policy/Research.Teacher          0.731733   0.244379      0   2.994 0.002751 ** 
#> mix.profile_code.Teacher.Teacher                  0.623727   0.232525      0   2.682 0.007309 ** 
#> mix.profile_code.Unclear.Teacher                  0.591886   0.247053      0   2.396 0.016585 *  
#> mix.profile_code.Administration.Unclear          -0.504453   0.313841      0  -1.607 0.107977    
#> mix.profile_code.Other.Unclear                    0.117043   0.315637      0   0.371 0.710776    
#> mix.profile_code.Policy/Research.Unclear          0.063245   0.297451      0   0.213 0.831621    
#> mix.profile_code.Teacher.Unclear                  0.080134   0.258251      0   0.310 0.756336    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#>      Null Deviance: 329461  on 237656  degrees of freedom
#>  Residual Deviance:  35741  on 237629  degrees of freedom
#>  
#> AIC: 35795    BIC: 36075    (Smaller is better.)

#> Joining, by = "term"

3B. Endorsing

#> 
#> ==========================
#> Summary of model fit
#> ==========================
#> 
#> Formula:   gg ~ edges + mutual + nodecov("n_tweets") + nodemix("profile_code", 
#>     base = 25)
#> 
#> Iterations:  5 out of 30 
#> 
#> Monte Carlo MLE Results:
#>                                                   Estimate Std. Error MCMC % z value Pr(>|z|)    
#> edges                                            -4.304840   0.155870      0 -27.618  < 1e-04 ***
#> mutual                                            2.755104   0.041665      0  66.125  < 1e-04 ***
#> nodecov.n_tweets                                  0.298806   0.005544      0  53.894  < 1e-04 ***
#> mix.profile_code.Administration.Administration    0.800308   0.167877      0   4.767  < 1e-04 ***
#> mix.profile_code.Other.Administration             0.658639   0.169519      0   3.885 0.000102 ***
#> mix.profile_code.Policy/Research.Administration   0.695980   0.171350      0   4.062  < 1e-04 ***
#> mix.profile_code.Teacher.Administration           0.758270   0.161688      0   4.690  < 1e-04 ***
#> mix.profile_code.Unclear.Administration           0.342926   0.186515      0   1.839 0.065975 .  
#> mix.profile_code.Administration.Other             0.755279   0.171655      0   4.400  < 1e-04 ***
#> mix.profile_code.Other.Other                      0.456908   0.181746      0   2.514 0.011937 *  
#> mix.profile_code.Policy/Research.Other            0.739460   0.179328      0   4.124  < 1e-04 ***
#> mix.profile_code.Teacher.Other                    0.668976   0.163771      0   4.085  < 1e-04 ***
#> mix.profile_code.Unclear.Other                    0.277946   0.203963      0   1.363 0.172970    
#> mix.profile_code.Administration.Policy/Research   0.797051   0.167642      0   4.754  < 1e-04 ***
#> mix.profile_code.Other.Policy/Research            0.659795   0.171534      0   3.846 0.000120 ***
#> mix.profile_code.Policy/Research.Policy/Research  0.735273   0.168435      0   4.365  < 1e-04 ***
#> mix.profile_code.Teacher.Policy/Research          0.762931   0.163539      0   4.665  < 1e-04 ***
#> mix.profile_code.Unclear.Policy/Research          0.466388   0.190655      0   2.446 0.014435 *  
#> mix.profile_code.Administration.Teacher           0.611808   0.162195      0   3.772 0.000162 ***
#> mix.profile_code.Other.Teacher                    0.571671   0.161260      0   3.545 0.000393 ***
#> mix.profile_code.Policy/Research.Teacher          0.337815   0.162239      0   2.082 0.037324 *  
#> mix.profile_code.Teacher.Teacher                  0.652635   0.157122      0   4.154  < 1e-04 ***
#> mix.profile_code.Unclear.Teacher                  0.413473   0.166239      0   2.487 0.012874 *  
#> mix.profile_code.Administration.Unclear           0.109275   0.194849      0   0.561 0.574921    
#> mix.profile_code.Other.Unclear                    0.312056   0.198019      0   1.576 0.115051    
#> mix.profile_code.Policy/Research.Unclear          0.102418   0.197712      0   0.518 0.604446    
#> mix.profile_code.Teacher.Unclear                  0.269391   0.172532      0   1.561 0.118430    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#>      Null Deviance: 341738  on 246512  degrees of freedom
#>  Residual Deviance:  67824  on 246485  degrees of freedom
#>  
#> AIC: 67878    BIC: 68159    (Smaller is better.)

#> Joining, by = "term"

An Exploration of #NGSSchat Through Selection Models

Joshua M. Rosenberg, Joshua W. Reid, Matthew J. Koehler, Christian Fischer, Thomas J. McKenna

9/9/2018

Table of contents: