Loading Libraries

library(psych) # for the describe() command
library(afex) # to run the ANOVA and plot results
library(ggplot2) # to visualize our results
library(expss) # for the cross_cases() command
library(car) # for the leveneTest() command
library(emmeans) # for posthoc tests
library(effsize) # for the cohen.d() command
library(apaTables) # to create our correlation table
library(kableExtra) # to create our correlation table
library(sjPlot) # to visualize our results

Importing Data

# import your AI results dataset
d <- read.csv(file="Data/final_results.csv", header=T)

State Your Hypotheses & Chosen Tests

H1: I predict participants who perceive a higher negative impact of COVID-19 pandemic on their close relationships will report higher levels of worry compared to participants who perceived a lower negative impact. H2: I predict that participants that report high levels of pandemic anxiety will show increased worry. I will be testing hypothesis one using Pearson’s Correlation Coefficient and Multiple Linear Regression for hypothesis two.

Check Your Variables

This is just basic variable checking that is used across all HW assignments.

# to view stats for all variables
describe(d)

           vars   n  mean    sd median trimmed   mad  min   max range  skew
id            1 100 50.50 29.01  50.50   50.50 37.06  1.0 100.0  99.0  0.00
identity*     2 100 50.50 29.01  50.50   50.50 37.06  1.0 100.0  99.0  0.00
consent*      3 100  1.90  0.44   2.00    1.94  0.00  1.0   3.0   2.0 -0.50
age           4 100 31.65  9.80  28.00   30.15  5.93 18.0  62.0  44.0  1.30
race          5 100  4.78  1.53   6.00    4.85  0.74  2.0   7.0   5.0 -0.34
gender*       6 100  1.97  0.52   2.00    1.99  0.00  1.0   5.0   4.0  2.08
manip_out*    7 100 50.50 29.01  50.50   50.50 37.06  1.0 100.0  99.0  0.00
survey1       8 100  6.82  9.95   3.40    3.79  0.30  2.1  45.0  42.9  2.54
survey2       9 100  3.57  0.40   3.67    3.59  0.49  3.0   4.5   1.5 -0.28
ai_manip*    10 100 50.50 29.01  50.50   50.50 37.06  1.0 100.0  99.0  0.00
condition    11 100  1.50  0.50   1.50    1.50  0.74  1.0   2.0   1.0  0.00
           kurtosis   se
id            -1.24 2.90
identity*     -1.24 2.90
consent*       1.66 0.04
age            0.84 0.98
race          -1.52 0.15
gender*       12.48 0.05
manip_out*    -1.24 2.90
survey1        4.72 0.99
survey2       -1.16 0.04
ai_manip*     -1.24 2.90
condition     -2.02 0.05

# we'll use the describeBy() command to view skew and kurtosis across our IVs
describeBy(d, group = d$condition)


 Descriptive statistics by group 
group: 1
          vars  n  mean    sd median trimmed   mad  min   max range  skew
id           1 50 25.50 14.58   25.5   25.50 18.53  1.0  50.0  49.0  0.00
identity     2 50 48.46 27.32   47.0   48.33 32.62  2.0  98.0  96.0  0.06
consent      3 50  1.90  0.46    2.0    1.93  0.00  1.0   3.0   2.0 -0.36
age          4 50 32.10  9.98   28.0   30.65  4.45 19.0  61.0  42.0  1.21
race         5 50  4.82  1.55    6.0    4.92  0.00  2.0   7.0   5.0 -0.45
gender       6 50  1.96  0.28    2.0    2.00  0.00  1.0   3.0   2.0 -1.35
manip_out    7 50 51.22 24.37   51.5   50.08 23.72 13.0  99.0  86.0  0.27
survey1      8 50  7.12 10.24    3.5    4.18  0.15  2.5  38.0  35.5  2.29
survey2      9 50  3.63  0.38    3.7    3.65  0.44  3.0   4.5   1.5 -0.38
ai_manip    10 50 49.70 24.84   50.5   50.05 25.95  2.0 100.0  98.0 -0.04
condition   11 50  1.00  0.00    1.0    1.00  0.00  1.0   1.0   0.0   NaN
          kurtosis   se
id           -1.27 2.06
identity     -1.18 3.86
consent       1.20 0.07
age           0.41 1.41
race         -1.49 0.22
gender        8.62 0.04
manip_out    -0.76 3.45
survey1       3.37 1.45
survey2      -0.74 0.05
ai_manip     -0.67 3.51
condition      NaN 0.00
------------------------------------------------------------ 
group: 2
          vars  n  mean    sd median trimmed   mad  min max range  skew
id           1 50 75.50 14.58   75.5   75.50 18.53 51.0 100  49.0  0.00
identity     2 50 52.54 30.75   55.5   52.92 37.81  1.0 100  99.0 -0.08
consent      3 50  1.90  0.42    2.0    1.95  0.00  1.0   3   2.0 -0.66
age          4 50 31.20  9.69   28.0   29.70  5.93 18.0  62  44.0  1.36
race         5 50  4.74  1.52    6.0    4.78  1.48  2.0   7   5.0 -0.21
gender       6 50  1.98  0.68    2.0    1.93  0.00  1.0   5   4.0  1.89
manip_out    7 50 49.78 33.25   41.5   50.08 48.93  1.0 100  99.0 -0.08
survey1      8 50  6.51  9.73    3.4    3.42  0.22  2.1  45  42.9  2.74
survey2      9 50  3.52  0.41    3.5    3.52  0.74  3.0   4   1.0 -0.16
ai_manip    10 50 51.30 32.89   50.5   51.45 49.67  1.0  98  97.0 -0.02
condition   11 50  2.00  0.00    2.0    2.00  0.00  2.0   2   0.0   NaN
          kurtosis   se
id           -1.27 2.06
identity     -1.35 4.35
consent       2.00 0.06
age           1.17 1.37
race         -1.59 0.22
gender        6.95 0.10
manip_out    -1.62 4.70
survey1       6.01 1.38
survey2      -1.55 0.06
ai_manip     -1.63 4.65
condition      NaN 0.00

# also use histograms and scatterplots to examine your continuous variables
hist(d$survey1)

plot(d$survey1, d$survey2)

# and table() and cross_cases() to examine your categorical variables
# you may not need the cross_cases code
table(d$survey1)


2.1 2.5   3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9  31  32  33  34  35  36  38  45 
  1   3   7   2   7   8  24  16  16   3   1   1   1   3   1   2   1   1   1   1

cross_cases(d, survey1, survey2)

	survey2
	3	3.33	3.4	3.5	3.56	3.6	3.67	3.7	3.8	4	4.5
survey1
2.1	1
2.5	1							1		1
3	2			1				1	2	1
3.1	2
3.2				1	1		1	1		2	1
3.3	3			2		1				2
3.4	3			7			1	2	1	10
3.5	5		1	3			3			4
3.6	3	1	1	1				3	2	5
3.7	1			1						1
3.8	1
3.9	1
31										1
32	1						1			1
33				1
34	1							1
35										1
36									1
38	1
45										1
#Total cases	26	1	2	17	1	1	6	9	6	30	1

# and boxplot to examine any categorical variables with continuous variables
boxplot(d$survey1~d$condition)

#convert any categorical variables to factors
#d$variable <- as.factor(d$variable)

Check Your Assumptions

Pearson’s Correlation Coefficient Assumptions

Should have two measurements for each participant for each variable (confirmed by earlier procedures – we dropped any participants with missing data)
Variables should be continuous and normally distributed, or assessments of the relationship may be inaccurate (will do below)
Outliers should be identified and removed, or results will be inaccurate (will do below)
Relationship between the variables should be linear, or they will not be detected (will do below)

Run a Multiple Linear Regression

To check the assumptions for Pearson’s correlation coefficient, we run our regression and then check our diagnostic plots.

# use the lm() command to run the regression
# dependent/outcome variable on the left, independent/predictor variables on the right
reg_model <- lm(survey1 ~ condition + survey2, data = d)

Check linearity with Residuals vs Fitted plot

For some examples of good Residuals vs Fitted plot and ones that show serious errors, check out this page.

For your homework, you’ll simply need to generate this plot and talk about how your plot compares to the good and problematic plots linked to above. Is it closer to the ‘good’ plots or one of the ‘bad’ plots? This is going to be a judgement call, and that’s okay! In practice, you’ll always be making these judgement calls as part of a team, so this assignment is just about getting experience with it, not making the perfect call.

 plot(reg_model, 1)

Check for outliers using Cook’s distance and a Residuals vs Leverage plot

For your homework, you’ll simply need to generate these plots, assess Cook’s distance in your dataset, and then identify any potential cases that are prominent outliers.

# Cook's distance
plot(reg_model, 4)

# Residuals vs Leverage
plot(reg_model, 5)

Multiple Linear Regression Assumptions

Observations should be independent (confirmed by data report)
Number of cases should be adequate (N ≥ 80 + 8m, where m is the number of IVs). If you don’t have enough, it won’t run. (will check this below)
Independent variables should not be too correlated (aka multicollinearity). (will check this below)
Relationship between the variables should be linear. (will check this below)
Outliers should be identified and removed. (will check this below)
Residuals should be normally distributed and have constant variance. (will check this below)

Check Number of Cases

For your homework, if you don’t have the required number of cases you’ll need to drop one of your independent variables. Reach out to me if this happens and we can figure out the best way to proceed!

needed <- 80 + 8*3
nrow(d) >= needed

[1] FALSE

Run a Multiple Linear Regression

To check the assumptions for an MLR, we run our regression and then check some output and diagnostic plots BEFORE looking at our results.

# use the lm() command to run the regression
# dependent/outcome variable on the left, independent/predictor variables on the right
reg_model <- lm(survey1 ~ condition + survey2, data = d)

Check multicollinearity

Higher values indicate more multicollinearity. This usually requires dropping a variable. For your homework, you will need to discuss multicollinearity and any high values, but you don’t have to drop any variables.
Cutoff is usually 5

vif(reg_model)

condition   survey2 
 1.019572  1.019572

Check linearity with Residuals vs Fitted plot

For some examples of good Residuals vs Fitted plot and ones that show serious errors, check out this page.

For your homework, you’ll simply need to generate this plot and talk about how your plot compares to the good and problematic plots linked to above. Is it closer to the ‘good’ plots or one of the ‘bad’ plots? This is going to be a judgement call, and that’s okay! In practice, you’ll always be making these judgement calls as part of a team, so this assignment is just about getting experience with it, not making the perfect call.

plot(reg_model, 1)

Check for outliers using Cook’s distance and a Residuals vs Leverage plot

For your homework, you’ll simply need to generate these plots, assess Cook’s distance in your dataset, and then identify any potential cases that are prominent outliers. Since we have some cutoffs, that makes this process is a bit less subjective than some of the other assessments we’ve done here, which is a nice change!

# Cook's distance
plot(reg_model, 4)

# Residuals vs Leverage
plot(reg_model, 5)

Check homogeneity of variance in a Scale-Location plot

You can check out this page for some other examples of this type of plot. (Notice that the Scale-Location plot is the third in the grids.)

For your homework, you’ll simply need to generate this plot and talk about how your plot compares to the ones pictured. Is it closer to the ‘good’ plots or one of the ‘bad’ plots? Again, this is a judgement call! It’s okay if feel uncertain, and you won’t be penalized for that.

plot(reg_model, 3)

Check normality of residuals with a Q-Q plot

This page shows how different types of non-normality appear on a Q-Q plot. It’s normal for Q-Q plots show a bit of deviation at the ends.

This page also shows some examples that help us put our Q-Q plot into context. Although it isn’t perfect, we don’t have any serious issues and are okay to proceed.

For your homework, you’ll simply need to generate this plot and talk about how your plot compares to the ones pictured. Does it seem like any skew or kurtosis is indicated by your plot? Is it closer to the ‘good’/‘bad’ plots from the second link?

plot(reg_model, 2)

Issues with My Data

This study can have some useful insights, but there are few limitations that need to considered. While using AI there can be room for gender and race biases with the experimental data as it seems to pull a majority from African American and white groups, and gender from cis-women and cis-men. Kurtosis for both of the conditions are shown to be on the high end and does not fall with in the accepted -2 to 2 range. Multicollineraity is below 5. THe residual vs leverage, residual vs fitted, cook’s distance, box-plot, histogram, and Q-Q residual graph did not display any serious patterns but did so some outliers. # Run Your Analysis ## Run a Correlation Test ### Create a Correlation Matrix

d2 <- subset(d, select=-c(survey1, survey2))
corr_output_m <- corr.test(d$survey1, d$survey2)

View Test Output

Strong effect: Between |0.50| and |1|
Moderate effect: Between |0.30| and |0.49|
Weak effect: Between |0.10| and |0.29|
Trivial effect: Less than |0.09|

corr_output_m

Call:corr.test(x = d$survey1, y = d$survey2)
Correlation matrix 
[1] 0.03
Sample Size 
[1] 100
These are the unadjusted probability values.
  The probability values  adjusted for multiple tests are in the p.adj object. 
[1] 0.74

 To see confidence intervals of the correlations, print with the short=FALSE option

Run a Multiple Linear Regression

You already ran this!

View Test Output

Effect size cutoffs from Cohen (1988): * Trivial: < .1 * Small: between .1 and .3 * Medium: between .3 and .5 * Large: > .5

summary(reg_model)


Call:
lm(formula = survey1 ~ condition + survey2, data = d)

Residuals:
   Min     1Q Median     3Q    Max 
-4.675 -3.721 -3.415 -2.982 38.120 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)   4.8628    10.1094   0.481    0.632
condition    -0.5241     2.0273  -0.259    0.797
survey2       0.7665     2.5675   0.299    0.766

Residual standard error: 10.04 on 97 degrees of freedom
Multiple R-squared:  0.001861,  Adjusted R-squared:  -0.01872 
F-statistic: 0.09041 on 2 and 97 DF,  p-value: 0.9136

Write Up Results

Correlation Test

I expect participants who perceive a higher negative impact of COVID-19 pandemic on their close relationships will report higher levels of worry compared to participants who perceived a lower negative impact. The hypothesis was not supported. There is weak effect of .03 shown in Table 1. Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences. New York, NY: Routledge Academic.

Variable	M	SD	1	2	3	4	5
Condition	50.50	29.01
Survey 2
	31.65	9.80	-.05
			[-.25, .14]

3. race	4.78	1.53	-.03	.03
			[-.23, .16]	[-.16, .23]

4. survey1	6.82	9.95	-.09	.11	-.08
			[-.28, .11]	[-.09, .30]	[-.27, .12]

5. survey2	3.57	0.40	-.09	.08	-.22*	.03
			[-.28, .11]	[-.12, .27]	[-.40, -.03]	[-.16, .23]

6. condition	1.50	0.50	.87**	-.05	-.03	-.03	-.14
			[.81, .91]	[-.24, .15]	[-.22, .17]	[-.23, .17]	[-.33, .06]

Note:
M and SD are used to represent mean and standard deviation, respectively. Values in square brackets indicate the 95% confidence interval. The confidence interval is a plausible range of population correlations that could have caused the sample correlation.
^* indicates p < .05
^** indicates p < .01.

Multiple Linear Regression

I expect participants who perceive a higher negative impact of COVID-19 pandemic on their close relationships will report higher levels of worry compared to participants who perceived a lower negative impact. This hypothesis is not supported. The results as seen in Table show R^2= .002, F(2,97)= .09, p-value= 0.9. This suggests that the IVs are significant predictors of the DV.

Table 1: Regression output for the relationships between condition, survey 2, and survey 1
	Survey 1
Predictors	Estimates	SE	CI	p
Intercept	4.86	10.11	-15.20 – 24.93	0.632
Condition	-0.52	2.03	-4.55 – 3.50	0.797
Survey 2	0.77	2.57	-4.33 – 5.86	0.766
Observations	100
R² / R² adjusted	0.002 / -0.019

References

Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences. New York, NY: Routledge Academic.