Gen AI Survey Report

Load Data

Key Dependent Variables - Means, SDs, Correlations

## Joining with `by = join_by(dvs)`
## Loading required package: Hmisc
## Attaching package: 'Hmisc'
## The following objects are masked from 'package:dplyr':
## 
## src, summarize
## The following objects are masked from 'package:base':
## 
## format.pval, units
## Joining with `by = join_by(dvs)`

Demographics

Gender

## 
##    Female      Male NonBinary 
##    0.4884    0.4950    0.0132

Race

## 
## American Indian           Black      East Asian     Multiracial           Other     South Asian           White 
##         0.00330         0.15512         0.05611         0.03630         0.04290         0.08581         0.61716

Country

Age

Role

Industry

as.data.frame(table(genaiclean$industry)/nrow(genaiclean)) %>% arrange(desc(Freq))

Analyses

1. There is a pervasive sense of uncertainty in regards to what responsibility means and looks like in using genAI (despite widespread adoption), which stems from immaturity in the tech industry and amongst adopters broadly, foundation models lack of transparency and a sense of diffused responsibility.

Most common challenge expressed by survey respondents

2. While PMs discuss their own individual agency to operationalize responsibility, there is a sense other teams are handling it and lack of clarity on their role. Lack of trust in bureaucracy and recognition that more needs to be taken in their own hands.

Lack of clarity around responsibility expectations

## 
##  Welch Two Sample t-test
## 
## data:  Q13_2 by challenges_clarityexpectations
## t = -1.9, df = 72, p-value = 0.06
## alternative hypothesis: true difference in means between group No and group Yes is not equal to 0
## 95 percent confidence interval:
##  -0.668130  0.009361
## sample estimates:
##  mean in group No mean in group Yes 
##             2.536             2.865

Discomfort raising issues around responsibility

## 
##  Welch Two Sample t-test
## 
## data:  Q13_3 by challenges_clarityexpectations
## t = -2.3, df = 78, p-value = 0.02
## alternative hypothesis: true difference in means between group No and group Yes is not equal to 0
## 95 percent confidence interval:
##  -0.70596 -0.05277
## sample estimates:
##  mean in group No mean in group Yes 
##             2.851             3.231

Relationship between discomfort raising issues around responsibility and being labeled a troublemaker

## 
##  Pearson's product-moment correlation
## 
## data:  Q13_3 and Q13_2
## t = 13, df = 299, p-value <0.0000000000000002
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.5327 0.6756
## sample estimates:
##   cor 
## 0.609

Relationship between suggestions about how to use gen AI more responsibly and being labeled a troublemaker

## 
##  Pearson's product-moment correlation
## 
## data:  Q13_2 and Q15_1
## t = 1.8, df = 268, p-value = 0.08
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.0114  0.2246
## sample estimates:
##    cor 
## 0.1081

3. Lack of incentives for responsible use of genAI, which is exacerbated by lack of education and standards.

Respondents who feel a lack of incentives feel greater business pressure

## 
##  Welch Two Sample t-test
## 
## data:  Q13_1 by challenges_incentives
## t = -5.1, df = 290, p-value = 0.0000006
## alternative hypothesis: true difference in means between group No and group Yes is not equal to 0
## 95 percent confidence interval:
##  -0.8319 -0.3691
## sample estimates:
##  mean in group No mean in group Yes 
##             3.426             4.027

Respondents who feel a lack of incentives have greater discomfort

## 
##  Welch Two Sample t-test
## 
## data:  Q13_3 by challenges_incentives
## t = -2.5, df = 229, p-value = 0.01
## alternative hypothesis: true difference in means between group No and group Yes is not equal to 0
## 95 percent confidence interval:
##  -0.6136 -0.0775
## sample estimates:
##  mean in group No mean in group Yes 
##             2.788             3.134

4. Important conditions for operationalizing responsibility include having principles tied to company values and leadership commitment, as well as incentives with tools and frameworks.

Indeed, survey participants who said “leadership expressed commitment to responsible AI” are 2.89 times more likely to work with responsible AI colleagues

## 
## Call:
## lm(formula = q9_workwith ~ orghave_leadership, data = genaiclean, 
##     family = "binomial")
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -0.544 -0.292 -0.292  0.456  0.708 
## 
## Coefficients:
##                       Estimate Std. Error t value           Pr(>|t|)    
## (Intercept)             0.2921     0.0356    8.21 0.0000000000000065 ***
## orghave_leadershipYes   0.2519     0.0554    4.55 0.0000078949758364 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.475 on 301 degrees of freedom
## Multiple R-squared:  0.0643, Adjusted R-squared:  0.0612 
## F-statistic: 20.7 on 1 and 301 DF,  p-value: 0.00000789

Indeed, survey participants who said “leadership expressed commitment to responsible AI” are 2.38 times more likely to test for bias

## 
## Call:
## lm(formula = q9_conductfair ~ orghave_leadership, data = genaiclean, 
##     family = "binomial")
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -0.408 -0.225 -0.225  0.592  0.775 
## 
## Coefficients:
##                       Estimate Std. Error t value      Pr(>|t|)    
## (Intercept)             0.2247     0.0338    6.65 0.00000000014 ***
## orghave_leadershipYes   0.1833     0.0526    3.48       0.00057 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.451 on 301 degrees of freedom
## Multiple R-squared:  0.0387, Adjusted R-squared:  0.0355 
## F-statistic: 12.1 on 1 and 301 DF,  p-value: 0.000569

Furthermore, participants who indicated their leadership was supportive of genAI felt significantly more comfortable volunteering suggestions than those who did not

## 
##  Welch Two Sample t-test
## 
## data:  Q15_2 by orghave_leadership
## t = -3.3, df = 245, p-value = 0.001
## alternative hypothesis: true difference in means between group No and group Yes is not equal to 0
## 95 percent confidence interval:
##  -0.9950 -0.2517
## sample estimates:
##  mean in group No mean in group Yes 
##             3.450             4.073

They also were more likely to report that their supervisor asks them to think about and take actions to help the team be more responsible in using generative AI than those who did not.

## 
##  Welch Two Sample t-test
## 
## data:  Q15_3 by orghave_leadership
## t = -3.3, df = 236, p-value = 0.001
## alternative hypothesis: true difference in means between group No and group Yes is not equal to 0
## 95 percent confidence interval:
##  -0.8779 -0.2187
## sample estimates:
##  mean in group No mean in group Yes 
##             3.906             4.455