Measure: The Twente Engagement with Ehealth Technologies Scale (TWEETS)

Citation(s):

· Kelders SM, Kip H. Development and initial validation of a scale to measure engagement with eHealth technologies. 2019 Presented at: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems; 2019; Glasgow.

· Kelders, S. M., Kip, H., & Greeff, J. (2020). Psychometric evaluation of the TWente Engagement with Ehealth Technologies Scale (TWEETS): evaluation study. Journal of medical internet research, 22(10), e17757.

TWEETS Deployment in the Current Study

The original TWEETS has 9 question scored on a 0-4 scale (strongly disagree to strongly agree), with three possible subscales (behavior, cognition, affect).

In the current study, the wording for the cognition questions was not accurately deployed to participants (i.e., the “goal” half of questions was not changed/specfied).

As such, all of the analyses for the current scale are done for the behavior and affect scales separately. Note: The original Qualtrics scale was collected on a scale of 1-5: this was previously re-coded to 0-4 during data cleaning.

TWEETS Questions

Tweets Questions, Variable Names, and Subscale Construct, as deployed in study

Cronbach’s Alpha

## [1] "Cronbach's Alpha for Week 4 Subscales"
## [1] "Behavioral Subscale:"
## [1] 0.5514132
## [1] "Cognitive Subscale:"
## [1] 0.7177101
## [1] "Affective Subscale:"
## [1] 0.700205
## [1] "Total Scale:"
## [1] 0.8536219

TWEETS Descriptives

Note: Each variable presented is the average subscale score for the week represented by the underscore. Cronbach’s Alpha and Descriptives

Behavior Subscale

vars n mean sd median trimmed mad min max range skew kurtosis se
tweets_beh_4 1 67 2.86 0.76 3.00 2.91 0.99 0.67 4 3.33 -0.62 0.06 0.09
tweets_beh_5 2 66 2.82 0.81 2.67 2.87 0.99 0.67 4 3.33 -0.49 -0.36 0.10
tweets_beh_6 3 66 2.72 0.87 2.67 2.75 0.99 0.67 4 3.33 -0.28 -0.77 0.11
tweets_beh_7 4 67 2.56 0.99 2.67 2.62 0.99 0.00 4 4.00 -0.52 -0.49 0.12
tweets_beh_8 5 48 2.47 1.03 2.67 2.52 0.99 0.00 4 4.00 -0.47 -0.33 0.15
tweets_beh_9 6 58 2.49 1.05 2.50 2.56 0.99 0.00 4 4.00 -0.48 -0.40 0.14
tweets_beh_10 7 62 2.33 1.14 2.33 2.39 1.48 0.00 4 4.00 -0.28 -0.92 0.15
tweets_beh_11 8 61 2.39 1.15 2.67 2.45 1.48 0.00 4 4.00 -0.38 -0.97 0.15
tweets_beh_12 9 54 2.46 1.10 2.67 2.54 0.99 0.00 4 4.00 -0.50 -0.47 0.15
tweets_beh_13 10 60 2.42 1.19 2.67 2.51 0.99 0.00 4 4.00 -0.57 -0.84 0.15

Visualization of Means over Time with Error Bars

Regression to analyze basic trend over time

A linear regression model was fitted to predict Tweets Behavior subscale as a function of Week. This approach quantified the direction and magnitude of the trend in engagement over time. The regression analysis revealed small, significant decrease over time,

## Linear Regression Results:
## Slope of the trend (Week): -0.055
## P-value for the slope: 1.147e-04
## R-squared of the model: 0.024

Combined Violin and Box Plot for Weeks 4–13

This plot combines violin plots and box plots to illustrate the distribution of behavior subscale scores across weeks 4 to 13. The violin plot shows the density of scores for each week, while the box plot provides a summary of the data’s central tendency and spread, including the median, interquartile range, and overall range.

### Behavior Violin Plot for Weeks 4–13 The plot below displays the distribution of the behavior subscale scores from weeks 4 to 13. The wider sections of the violin indicate a higher concentration of scores, while the narrower sections show less frequent values. Jittered points are overlaid to show the individual participant scores.

### Raincloud Plot for Weeks 4 and 13 These plot provides a comparison of the behavior subscale scores between weeks 4 and 13 (first and last weeks recorded). Each week is represented by a combination of a violin plot (showing the score distribution), jittered points (representing individual participant scores), and lines connecting scores for the same participants between the two timepoints showing change over time.

Affect Subscale

vars n mean sd median trimmed mad min max range skew kurtosis se
tweets_aff_4 1 67 3.13 0.66 3.33 3.18 0.49 1.33 4 2.67 -0.59 -0.43 0.08
tweets_aff_5 2 66 3.15 0.67 3.33 3.20 0.49 1.33 4 2.67 -0.64 -0.29 0.08
tweets_aff_6 3 66 3.15 0.73 3.33 3.22 0.49 1.33 4 2.67 -0.82 -0.35 0.09
tweets_aff_7 4 67 3.17 0.73 3.33 3.25 0.49 1.00 4 3.00 -0.90 0.18 0.09
tweets_aff_8 5 48 3.10 0.85 3.33 3.18 0.99 0.67 4 3.33 -0.93 0.00 0.12
tweets_aff_9 6 58 3.02 0.80 3.17 3.10 0.74 1.00 4 3.00 -0.83 0.01 0.10
tweets_aff_10 7 62 2.93 1.00 3.33 3.06 0.99 0.00 4 4.00 -0.97 0.10 0.13
tweets_aff_11 8 61 2.95 1.01 3.33 3.11 0.49 0.00 4 4.00 -1.39 1.45 0.13
tweets_aff_12 9 54 2.93 1.06 3.33 3.07 0.99 0.00 4 4.00 -0.97 0.18 0.14
tweets_aff_13 10 60 2.86 1.11 3.00 3.01 0.99 0.00 4 4.00 -0.96 -0.09 0.14

Visualization of Means over Time with Error Bars

## Regression to analyze basic trend over time A linear regression model was fitted to predict Tweets Affect subscale as a function of Week. This approach quantified the direction and magnitude of the trend in engagement over time. The regression analysis revealed small, significant decrease over time,

## Linear Regression Results for Affect Subscale:
## Slope of the trend (Week): -0.035
## P-value for the slope: 3.881e-03
## R-squared of the model: 0.014

Combined Violin and Box Plot for Weeks 4–13

This plot combines violin plots and box plots to illustrate the distribution of affect subscale scores across weeks 4 to 13. The violin plot shows the density of scores for each week, while the box plot provides a summary of the data’s central tendency and spread, including the median, interquartile range, and overall range.

### Affect Violin Plot for Weeks 4–13 The plot below displays the distribution of the affect subscale scores from weeks 4 to 13. The wider sections of the violin indicate a higher concentration of scores, while the narrower sections show less frequent values. Jittered points are overlaid to show the individual participant scores.

Raincloud Plot for Weeks 4 and 13

These plot provides a comparison of the affect subscale scores between weeks 4 and 13 (first and last weeks recorded). Each week is represented by a combination of a violin plot (showing the score distribution), jittered points (representing individual participant scores), and lines connecting scores for the same participants between the two timepoints showing change over time.

TWEETS Descriptives

Note: Each variable presented is the average subscale score for the week represented by the underscore. Cronbach’s Alpha and Descriptives

Cognition Subscale

vars n mean sd median trimmed mad min max range skew kurtosis se
tweets_cog_4 1 67 2.48 0.80 2.33 2.49 0.49 0.33 4 3.67 -0.09 -0.17 0.10
tweets_cog_5 2 66 2.65 0.82 2.67 2.68 0.99 0.33 4 3.67 -0.43 -0.13 0.10
tweets_cog_6 3 66 2.58 0.90 2.67 2.60 0.99 0.33 4 3.67 -0.32 -0.47 0.11
tweets_cog_7 4 67 2.55 0.86 2.67 2.59 0.99 0.00 4 4.00 -0.38 -0.03 0.10
tweets_cog_8 5 48 2.42 1.01 2.33 2.47 0.99 0.00 4 4.00 -0.32 -0.28 0.15
tweets_cog_9 6 58 2.51 0.83 2.67 2.51 0.99 0.00 4 4.00 -0.23 0.21 0.11
tweets_cog_10 7 62 2.36 1.07 2.33 2.43 0.99 0.00 4 4.00 -0.54 -0.41 0.14
tweets_cog_11 8 61 2.42 1.03 2.67 2.48 0.99 0.00 4 4.00 -0.52 -0.24 0.13
tweets_cog_12 9 54 2.50 0.98 2.67 2.58 0.99 0.00 4 4.00 -0.75 0.40 0.13
tweets_cog_13 10 60 2.36 1.18 2.67 2.45 0.99 0.00 4 4.00 -0.56 -0.49 0.15

Visualization of Means over Time with Error Bars

Regression to analyze basic trend over time

A linear regression model was fitted to predict Tweets Cognition subscale as a function of Week. This approach quantified the direction and magnitude of the trend in engagement over time. The regression analysis revealed small, significant decrease over time,

## Linear Regression Results:
## Slope of the trend (Week): -0.021
## P-value for the slope: 1.081e-01
## R-squared of the model: 0.004

Combined Violin and Box Plot for Weeks 4–13

This plot combines violin plots and box plots to illustrate the distribution of Cognition subscale scores across weeks 4 to 13. The violin plot shows the density of scores for each week, while the box plot provides a summary of the data’s central tendency and spread, including the median, interquartile range, and overall range.

### Cognition Violin Plot for Weeks 4–13 The plot below displays the distribution of the Cognition subscale scores from weeks 4 to 13. The wider sections of the violin indicate a higher concentration of scores, while the narrower sections show less frequent values. Jittered points are overlaid to show the individual participant scores.

### Raincloud Plot for Weeks 4 and 13 These plot provides a comparison of the Cognition subscale scores between weeks 4 and 13 (first and last weeks recorded). Each week is represented by a combination of a violin plot (showing the score distribution), jittered points (representing individual participant scores), and lines connecting scores for the same participants between the two timepoints showing change over time.

TWEETS Descriptives

Note: Each variable presented is the average Scale score for the week represented by the underscore. Cronbach’s Alpha and Descriptives

Total Scale

vars n mean sd median trimmed mad min max range skew kurtosis se
tweets_tot_4 1 67 2.83 0.66 2.89 2.83 0.66 1.00 4 3.00 -0.23 -0.40 0.08
tweets_tot_5 2 66 2.87 0.66 2.89 2.90 0.66 1.22 4 2.78 -0.28 -0.44 0.08
tweets_tot_6 3 66 2.81 0.74 2.89 2.84 0.82 0.78 4 3.22 -0.42 -0.52 0.09
tweets_tot_7 4 67 2.76 0.77 2.89 2.81 0.82 0.33 4 3.67 -0.71 0.27 0.09
tweets_tot_8 5 48 2.66 0.84 2.78 2.71 0.66 0.22 4 3.78 -0.61 0.44 0.12
tweets_tot_9 6 58 2.67 0.79 2.72 2.71 0.74 0.33 4 3.67 -0.57 0.42 0.10
tweets_tot_10 7 62 2.54 0.98 2.67 2.61 0.99 0.22 4 3.78 -0.53 -0.50 0.12
tweets_tot_11 8 61 2.58 0.97 2.89 2.66 0.82 0.00 4 4.00 -0.78 0.22 0.12
tweets_tot_12 9 54 2.63 0.95 2.83 2.71 0.91 0.00 4 4.00 -0.80 0.32 0.13
tweets_tot_13 10 60 2.55 1.08 2.78 2.65 0.82 0.00 4 4.00 -0.77 -0.37 0.14

Visualization of Means over Time with Error Bars

Regression to analyze basic trend over time

A linear regression model was fitted to predict Tweets Total Scale as a function of Week. This approach quantified the direction and magnitude of the trend in engagement over time. The regression analysis revealed small, significant decrease over time.

## Linear Regression Results:
## Slope of the trend (Week): -0.037
## P-value for the slope: 1.766e-03
## R-squared of the model: 0.016

Combined Violin and Box Plot for Weeks 4–13

This plot combines violin plots and box plots to illustrate the distribution of Total Scale scores across weeks 4 to 13. The violin plot shows the density of scores for each week, while the box plot provides a summary of the data’s central tendency and spread, including the median, interquartile range, and overall range.

### Total Violin Plot for Weeks 4–13 The plot below displays the distribution of the Total Scale scores from weeks 4 to 13. The wider sections of the violin indicate a higher concentration of scores, while the narrower sections show less frequent values. Jittered points are overlaid to show the individual participant scores.

### Raincloud Plot for Weeks 4 and 13 These plot provides a comparison of the Total Scale scores between weeks 4 and 13 (first and last weeks recorded). Each week is represented by a combination of a violin plot (showing the score distribution), jittered points (representing individual participant scores), and lines connecting scores for the same participants between the two timepoints showing change over time.