AI Unc 2

Load necessary packages and setup

Loading required package: pacman

R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22621)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.utf8 
[2] LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] ez_4.4-0            plotrix_3.8-2       emmeans_1.7.5      
 [4] sjPlot_2.8.11       brms_2.18.0         Rcpp_1.0.9         
 [7] rstan_2.26.13       StanHeaders_2.26.13 multcomp_1.4-20    
[10] TH.data_1.1-1       MASS_7.3-57         survival_3.3-1     
[13] mvtnorm_1.1-3       sjstats_0.18.1      superb_0.95.0      
[16] broom_1.0.3         afex_1.2-0          lme4_1.1-31        
[19] Matrix_1.4-1        knitr_1.39          lubridate_1.8.0    
[22] forcats_1.0.0       stringr_1.5.0       dplyr_1.0.10       
[25] purrr_0.3.4         readr_2.1.4         tidyr_1.2.0        
[28] tibble_3.1.8        ggplot2_3.4.1       tidyverse_2.0.0    
[31] pacman_0.5.1       

loaded via a namespace (and not attached):
  [1] backports_1.4.1      plyr_1.8.7           igraph_1.3.5        
  [4] splines_4.2.1        crosstalk_1.2.0      rstantools_2.2.0    
  [7] inline_0.3.19        digest_0.6.29        htmltools_0.5.4     
 [10] lmerTest_3.1-3       fansi_1.0.3          magrittr_2.0.3      
 [13] checkmate_2.1.0      tzdb_0.3.0           modelr_0.1.10       
 [16] RcppParallel_5.1.5   matrixStats_0.62.0   xts_0.12.1          
 [19] sandwich_3.0-2       prettyunits_1.1.1    colorspace_2.0-3    
 [22] rbibutils_2.2.8      xfun_0.31            callr_3.7.1         
 [25] crayon_1.5.1         jsonlite_1.8.0       zoo_1.8-11          
 [28] glue_1.6.2           gtable_0.3.0         V8_4.2.1            
 [31] sjmisc_2.8.9         distributional_0.3.1 car_3.1-1           
 [34] pkgbuild_1.3.1       abind_1.4-5          scales_1.2.0        
 [37] DBI_1.1.3            ggeffects_1.1.3      miniUI_0.1.1.1      
 [40] xtable_1.8-4         performance_0.9.2    foreign_0.8-82      
 [43] stats4_4.2.1         DT_0.25              datawizard_0.5.1    
 [46] htmlwidgets_1.6.1    threejs_0.3.3        posterior_1.3.1     
 [49] ellipsis_0.3.2       pkgconfig_2.0.3      loo_2.5.1           
 [52] farver_2.1.1         utf8_1.2.2           tidyselect_1.2.0    
 [55] rlang_1.0.6          reshape2_1.4.4       later_1.3.0         
 [58] effectsize_0.7.0.5   munsell_0.5.0        tools_4.2.1         
 [61] cli_3.4.1            generics_0.1.3       sjlabelled_1.2.0    
 [64] ggridges_0.5.4       evaluate_0.15        shinyBS_0.61.1      
 [67] fastmap_1.1.0        yaml_2.3.5           processx_3.7.0      
 [70] nlme_3.1-157         mime_0.12            shinythemes_1.2.0   
 [73] compiler_4.2.1       bayesplot_1.9.0      rstudioapi_0.14     
 [76] curl_4.3.2           stringi_1.7.8        ps_1.7.1            
 [79] parameters_0.18.2    Brobdingnag_1.2-7    lattice_0.20-45     
 [82] nloptr_2.0.3         markdown_1.1         shinyjs_2.1.0       
 [85] tensorA_0.36.2       vctrs_0.5.1          pillar_1.8.1        
 [88] lifecycle_1.0.3      Rdpack_2.4           bridgesampling_1.1-2
 [91] estimability_1.4     insight_0.18.2       httpuv_1.6.5        
 [94] R6_2.5.1             promises_1.2.0.1     gridExtra_2.3       
 [97] codetools_0.2-18     boot_1.3-28          colourpicker_1.1.1  
[100] gtools_3.9.3         withr_2.5.0          shinystan_2.6.0     
[103] mgcv_1.8-40          bayestestR_0.12.1    parallel_4.2.1      
[106] hms_1.1.2            grid_4.2.1           coda_0.19-4         
[109] minqa_1.2.4          rmarkdown_2.14       carData_3.0-5       
[112] lsr_0.5.2            numDeriv_2016.8-1.1  shiny_1.7.2         
[115] base64enc_0.1-3      dygraphs_1.1.1.6    

Read in cleaned data

#separating this out so the final qmd won't include prolific id info
source('clean-raw-data.R')

[1] "Wrong number of prolific ids"

cleaned_df <- read.csv('AI_unc_Exp2_task_data.csv')
quest_df <- read.csv('AI_unc_Exp2_questionnaire_data.csv')

#Data used to generate figures, probabilities for each stimulus
stim_data <- read.csv('all_datasets_exp.csv')

Check that data make sense

# see how many subjects in each condition
cleaned_df %>%
  group_by(subject) %>%
  summarize(viz = first(exp_condition), 
            dataset = first(dataset)) %>%
  group_by(viz, dataset) %>%
  summarize(nsubs = n()) %>%
  kable()

`summarise()` has grouped output by 'viz'. You can override using the `.groups`
argument.

viz	dataset	nsubs
Uncertainty_Dist_1	Census	10
Uncertainty_Dist_1	Credit	11
Uncertainty_Dist_1	Education	13
Uncertainty_Dist_2	Census	11
Uncertainty_Dist_2	Credit	12
Uncertainty_Dist_2	Education	10
Uncertainty_Pt_1	Census	11
Uncertainty_Pt_1	Credit	12
Uncertainty_Pt_1	Education	11
Uncertainty_Pt_2	Census	10
Uncertainty_Pt_2	Credit	12
Uncertainty_Pt_2	Education	11

# does everyone have the right number of trials
ntrials <- cleaned_df %>% 
  group_by(subject, dataset) %>%
  summarize(nprac = sum(test_part == "practice"),
            ntest = sum(test_part == "test")) 

`summarise()` has grouped output by 'subject'. You can override using the
`.groups` argument.

all(ntrials$nprac == 8)

[1] TRUE

all(ntrials$ntest == 40)

[1] TRUE

#make sure "correct" coding is working
iscorr = as.numeric(cleaned_df$response) == cleaned_df$correct_response
all(iscorr == cleaned_df$correct)

[1] TRUE

#look at overall participant accuracies
#expect census > edu > credit
overall.accs <- cleaned_df %>%
  mutate(dataset = factor(dataset, levels = c("Credit","Education","Census"))) %>%
  filter(test_part == "test") %>%
  group_by(subject, dataset) %>%
  summarize(ncorrect = sum(correct == "TRUE"),
            pct.correct = ncorrect/40) 

`summarise()` has grouped output by 'subject'. You can override using the
`.groups` argument.

ggplot(overall.accs, aes(x = pct.correct)) +
  geom_histogram(position = "stack", alpha = 0.5, binwidth = 0.05) +
  facet_grid(dataset ~ .) +
  geom_vline(data=filter(overall.accs, dataset=="Credit"), aes(xintercept=.76), colour="darkred") + 
  geom_vline(data=filter(overall.accs, dataset=="Education"), aes(xintercept=.82), colour="darkred") +
  geom_vline(data=filter(overall.accs, dataset=="Census"), aes(xintercept=.88), colour="darkred")

#look at RTs for 2AFC
cleaned_df <- cleaned_df %>%
  mutate(rt = as.numeric(rt))

cleaned_df %>%
  transmute(subject, test_part, rtmin = rt/60000) %>%
  filter(rtmin > 2, test_part != "practice") %>%
  group_by(subject) %>%
  tally() %>%
  arrange(desc(n))

# A tibble: 11 × 2
   subject     n
     <int> <int>
 1      77     2
 2     120     2
 3      12     1
 4      62     1
 5      63     1
 6      80     1
 7      85     1
 8      91     1
 9     112     1
10     114     1
11     123     1

#From prereg: 
#"Trials for reaction times exceeding +/- 3 standard deviations above the mean may be excluded" 
cleaned_df2 <- cleaned_df %>%
  filter(test_part == "test") %>%
  group_by(subject) %>%
  filter(abs(rt - mean(rt)) < (sd(rt) * 3))

dim(cleaned_df %>% filter(test_part == "test"))[1] - dim(cleaned_df2)[1]

[1] 121

#histogram average RTs by subject
cleaned_df %>% group_by(subject) %>%
  filter(test_part == "test") %>%
  summarise(avgRT = mean(rt)) %>% 
  with(hist(avgRT, main = "Average RT by Subject"))

#histogram average RTs by subject - outliers removed
cleaned_df2 %>% group_by(subject) %>%
  summarise(avgRT = mean(rt)) %>% 
  with(hist(avgRT, main = "Average RT by Subject - Outliers Removed"))

            # , xlim = c(0,50000), main = "Average RT by Subject - Outliers Removed"))

#Code if AI prediction was accurate 
cleaned_df2$AI_pred_binary <- recode(cleaned_df2$AI_pred,
                                     "Fail" = 0,
                                     "Pass" = 1,
                                     "Less than 94K" = 0,
                                     "More than 94K" = 1,
                                     "Break Terms" = 0,
                                     "Follow Terms" = 1
)

cleaned_df2$AI_acc <- 
  ifelse(cleaned_df2$AI_pred_binary == cleaned_df2$correct_response, 1, 0)

cleaned_df2$Follow_AI <-   
  ifelse(cleaned_df2$AI_pred_binary == cleaned_df2$response, 1, 0)

quest_df %>%
  dplyr::select(exp_condition, dataset, strategies) %>%
  kable()

exp_condition	dataset	strategies
Uncertainty_Pt_2	Credit	I tended to go with the AI, but I also paid close attention to level a job difficulty how well they save and the loan amount asked for.
Uncertainty_Dist_2	Education	family wellness and failure rate.
Uncertainty_Pt_2	Education	i tended to look at study time, amount of failures and absences to determine if the student would pass or fail. I figured those are the most useful in determining that. I used the AI to determine if I was unsure.
Uncertainty_Dist_1	Credit	I tended to follow the AI direction, especially if the ‘needle’ was narrow.
Uncertainty_Pt_1	Credit	Based mostly on savings and checkings
Uncertainty_Dist_2	Credit	I tended to follow the AI’s prediction, but I had less and less confidence it it as it progressed.
Uncertainty_Pt_1	Census	I looked at what the AI predicted, but looked at education, occupation, gender and race before making my decision.
Uncertainty_Pt_1	Education	I assumed that the best predictor for failing the class was previous failures. I also looked at study time and paid attention to absences a lot.
Uncertainty_Dist_2	Credit	I looked at the data provided about each person and the loan as well as the AI predictions.
Uncertainty_Pt_1	Census	Evaluate all the data fields and then see if AI agreed with me I went with my intuition on options I didnt agree with the AI often
Uncertainty_Pt_2	Education	I used the number of times absent.
Uncertainty_Dist_1	Census	I used the information given by the table, education, sex, capital gains, and hours worked were the most important to me. I also used the AI as it was mostly correct.
Uncertainty_Pt_1	Census	I looked at the profession and the loss or capital gains and then I looked at what the AI predicted and used that to form my prediction
Uncertainty_Pt_2	Education	I based my decision mainly on how many previous failures the student had and what was their reason for choosing the school
Uncertainty_Pt_2	Census	Age (lower the more likely they would be under 94k)(higher more likely to be above 94k)gains or losses (anyone putting money in the market has disposable income)AI seemed to be correct almost every time except maybe 5 times
Uncertainty_Pt_1	Education	If the student didn’t have a desire to go seek higher education, I failed them. If they had high absences or drank heavily, I also was more likely to fail them.
Uncertainty_Dist_2	Census	look at education, craft and # of hours worked
Uncertainty_Pt_2	Credit	I considered the amount of the loan, the time it took to pay back, as well as the purpose of the loan into consideration the most before deciding if someone could really pay back that kind of loan.
Uncertainty_Dist_1	Education	I generally went with the AI’s prediction, but I paid attention to alcohol, failure and family as well.
Uncertainty_Dist_2	Census	I looked at their profession and who they were living with.
Uncertainty_Dist_1	Credit	I tended to follow the AI’s recommendations the majority of the time, as they seemed to be fairly accurate. If the AI seemed uncertain, I evaluated factors in the table to come to a reasonably informed decision.
Uncertainty_Dist_1	Census	Mainly approximations of how much people in certain professions make, coupled with their age (assuming they’ve been in that position longer) along with any capital gains.
Uncertainty_Dist_2	Credit	I tended to just follow the AI. Sometimes I looked at the checking and savings account variables, but more often than not followed the AI.
Uncertainty_Dist_1	Education	I tended to go with the AI almost every time.
Uncertainty_Dist_2	Education	I looked at the information that was given and looked at the recommendation of the AI and then when with what I felt was the answer.
Uncertainty_Dist_2	Census	I tended to just look at the AI’s response. If the AI was less than 60% sure I would look at some of the identifying details of the individual, but would still trend towards picking the AI’s suggestion.
Uncertainty_Dist_1	Census	I looked at the person’s education and age mostly. If they had a good education, and were older, that usually means they’ve earned their way up through the ranks of their jobs and earn more. Also, the AI was very helpful.
Uncertainty_Pt_1	Credit
Uncertainty_Dist_2	Education	Follow mostly AI predictions, checking previous failures, etc.
Uncertainty_Pt_2	Credit	Look at their housing situation and skills.
Uncertainty_Pt_1	Credit	I followed the AI prediction. If it was less than 70 percent on either side I had low confidence. If it was higher than 90 I had high confidence.
Uncertainty_Dist_1	Credit	I paid attention to all factors especially loan amount and duration compared with checking and savings accounts.
Uncertainty_Pt_1	Education	I tended to look specifically on the amount of absences and the previous amounts of failures. Sometimes this was telling while sometimes it was not. I found myself double-guessing quite a bit if my hunch didn’t correlate with the AI prediction.
Uncertainty_Dist_1	Education
Uncertainty_Dist_1	Education	I mainly decided to weigh most of my choices on a certain few categories. As I continued through the experiment, I evolved a bit how much I cared for one category or another. At the end, I looked at if the student had a desire for higher education as the highest priority and then looked at the parents level of education next. I also looked at time studied and the student’s reason for choosing the school. the ai’s input, I think I mainly ignored it.
Uncertainty_Pt_2	Education	The strategies I used to make classifications in this task include going by what the AI predicted in terms of whether or not it thought the student would pass or fail the class.
Uncertainty_Pt_2	Credit	I tried to use the information provided to get an idea of where the person is at in there life.
Uncertainty_Dist_2	Credit	I think I noticed algorithm that the AI was using. Maybe. It took me some trial and error to figure it out.
Uncertainty_Dist_1	Census	I used the Ai and a bit of common sense of certain occupations
Uncertainty_Pt_2	Census	Mainly looked at degree, race and profession.
Uncertainty_Pt_1	Education	I tried to figure out how much each category weighed in the likelihood of someone failing a class.
Uncertainty_Dist_1	Education	I studied the graph and their qualifications
Uncertainty_Dist_1	Education	I tended to check mainly the study time, if it was less than 2 hours then there’s a good chance that the student will fail. I also took into account past failed classe.
Uncertainty_Pt_2	Census	compare AI to information given
Uncertainty_Pt_1	Education	I tried to mostly pay attention to how many classes they failed previously, how much they are studying, their absence count, and if they plan on going to achieve a higher education.
Uncertainty_Dist_1	Credit	What loan was for and whether they owned or rented
Uncertainty_Pt_2	Census	/I used the AI prediction
Uncertainty_Pt_1	Credit	I looked at the person’s savings and checking, whether they owned or rented, and the reason for the loan. A person wanting a huge amount for a TV is likely to default vs someone wanting a loan for a business.
Uncertainty_Pt_1	Education	go with the graph predictions
Uncertainty_Dist_1	Credit	I used the AI model and the range predictor.
Uncertainty_Dist_1	Education	I tried to consider both the information chart I was given and what the AI predicted.
Uncertainty_Dist_2	Census	gauged by age ed, and profession
Uncertainty_Pt_1	Credit	I looked at the age as how experienced the person might be with borrowing money. I looked at owned/rented over subsidized for personal experience in borrowing money. Having money in checking/savings I saw as having the wherewithal to pay back the loan. The amount of the loan for the item/service purchase was rated against the previous factors - a person looking to buy a car I saw as more likely to follow the plan than someone buying a TV.
Uncertainty_Pt_1	Census	I looked specifically at capital gains and losses, cause that told me a lot. I also generally considered the job itself, the racial/sex demographics, and the country they are from. The AI was very helpful, but not a crutch, and I mostly made decisions outside of its influence.
Uncertainty_Dist_1	Credit	Mostly guesses. I took the AI into consideration a little.
Uncertainty_Dist_2	Credit	50-60 low. 60-80 moderate. 80+ high.
Uncertainty_Dist_1	Census	I mostly focused on occupation and hours worked. If Capital Gain was above 7000 I would assume a high wage.
Uncertainty_Pt_2	Census	I thought of what I know about certain job titles and what they need education-wise to obtain that job title. That along with what the AI was guessing. I started to learn slightly about how to pick them.
Uncertainty_Dist_1	Census	I watched age, education and the AI predictions
Uncertainty_Dist_1	Census	Utilized the AI prediction as well as percentages.
Uncertainty_Dist_2	Credit	followed the instruction as I read earlier
Uncertainty_Pt_2	Census	I used the education and race to determine it and the AI.
Uncertainty_Dist_2	Census	I mostly looked at education and field of work. I also took the AI’s prediction and others responses into consideration as well.
Uncertainty_Dist_2	Education	If they had low study, low parent education, and failed or missed a lot of classes.
Uncertainty_Pt_1	Credit	I tried to examine what they were buying (tv, furniture, car) and the skill of their jobs and the amount and how much they had in the bank. If it was little I thought they would be more apt to spend and break the terms.
Uncertainty_Pt_2	Credit	I tried to see what their debt was for and the amount of debt and duration. I also looked at their skills and their current housing situation. I tried to make the best choices in relation to these things.
Uncertainty_Dist_2	Census	I know hardly anything about income levels, especially in countries outside of the US, so I focused entirely on the chart and recommendation from the ai/algorithm and went with it’s suggestions. I also rated my confidence largely based on the percentage it gave.
Uncertainty_Pt_1	Credit	I tended to look at amount is savings and age.
Uncertainty_Pt_2	Education	i usually was looking at both the study time and amount of failures in the chart before making my decision
Uncertainty_Dist_1	Credit	I thought I had a strategy, but I kept getting most of them wrong. I was looking at if they owned and checking/savings.
Uncertainty_Pt_2	Education	I felt the amount of failures in school, alchol use, time spent studying and absences were the best indicators. This may be from the perspective of what I was with similiar students in my own upbringing. Sometimes the students with poor home lives were more motivated to get out of that situation.
Uncertainty_Dist_1	Credit	If they lived in subsidized housing, the broke it
Uncertainty_Dist_2	Credit	I usually went with the AI if it was really high on one end or the other. Towards the middle I looked at stats such as how much money they wanted and what their job looked like.
Uncertainty_Dist_2	Education	I looked at the range of answers the whole time and where the average was to make my decisions.
Uncertainty_Dist_1	Credit	I started by looking at the person’s info and graph, but over time I started relying more on the graph usually.
Uncertainty_Pt_2	Credit	I considered owning a home as an asset even if the person had an unskilled job and a little in the bank. Also smaller loans seemed like they would have a higher probability of being paid back even if the person had little money in checking or savings. I considered having subsidized housing and a loan as high risk.
Uncertainty_Pt_2	Credit	Their money in savings and checking. Also what the loan was for and for how long. Then I looked at the amount that was loaned out.
Uncertainty_Pt_2	Education	It seemed that the amount of time they spent studying and their parents education were key factors.
Uncertainty_Dist_2	Credit	I used the given information, plus the AI predicted graph to see the likelihood that the individual would break or follow the loan. If the individual lived in subsidized housing, I felt it was less likely they would pay off the loan. However, if it was a small amount, and the individual was skilled or highly skilled, I tried to give them the benefit of a doubt. It helps if there was other individuals that were able to pay off the loan.
Uncertainty_Pt_1	Census	I just guessed based on education, age, sex, race, and country of origin.
Uncertainty_Pt_1	Census	i looked at each stage carefully and attentively in other to answer correctly
Uncertainty_Pt_2	Education	I looked at the AI prediction as well as the stats on the student (absences, drinking, family relationship, etc).
Uncertainty_Pt_2	Census	I mainly looked at age and education and if I was unsure I put more weight on what the AI said.
Uncertainty_Pt_2	Census	I went with the degree and how many hours per week for my base
Uncertainty_Pt_1	Education	I based my decision mainly on how often study was, how many passes and fails, and how many absences.
Uncertainty_Pt_2	Education	The use of alcohol, the absences, any failures
Uncertainty_Pt_2	Credit	I did look at the AI but tried to base if on my opinion. Looked at age/skill/own/rent
Uncertainty_Dist_1	Credit	I used my experience and the people I know to try to determine if they could handle the monthly amount that they asked for. But it seemed very difficult to get the right answer. Getting the wrong answer made me doubt myself and feel confused. It feels as if the system doesn’t have as much confidence in people as I do.
Uncertainty_Dist_1	Education	I tried to see when the algorithm was overconfident
Uncertainty_Pt_1	Credit	My strategy was to take into account the information I had to make a logical decision
Uncertainty_Dist_2	Credit	To look at age first to see if they were even old enough to be stable. I looked at housing and the type of merchandise being purchased next and whether or not it’s really important that they have it.
Uncertainty_Pt_1	Education	Took info about failures, absences, and higher ed into consideration more than anything else
Uncertainty_Pt_2	Education	I looked at absences, number of failed classes and a few others, like the dad’s job and family life.
Uncertainty_Dist_1	Credit	At first I looked at the information given, then I wound up just going off of the AI prediction since it was more accurate than what I thought was correct. Eventually I went back to the information and deciding based off of what was provided.
Uncertainty_Pt_2	Education	I tried to look at if they had failed before or had a high number of absences and a bad family relationship, I felt like they would fail. Most of the time, I agreed with the AI with a few exceptions.
Uncertainty_Dist_1	Education	I tried to use all the data on what I thought would lead a person to pass/fail to come up with a decision and used the AI prediction to verify my ideas
Uncertainty_Pt_1	Census	From my own experience mostly. And the older and more educated, the more I had a tendency to think they would make over the 94,000
Uncertainty_Pt_1	Credit	I viewed their occupational skill level and money on hand to make decisions based on the amount of money they were being loaned. For instance, for unskilled workers who asked for high loan amounts, I typically said break terms.
Uncertainty_Pt_1	Credit	Well, sometimes I tried to go with my own instinct as to whether someone in the situation would be able and willing to pay the loan back and sometimes I went with the AI prediction if it had a high certainty…It was wrong a lot.
Uncertainty_Dist_2	Census	graphs and ai
Uncertainty_Dist_2	Census	If they were self-employed or had an executive position, I tended to believe they were paid more. If they weren’t, I looked more to the AI for guidance.
Uncertainty_Pt_1	Credit	I followed the AI results and worked around it.
Uncertainty_Dist_2	Census	Mostly age and career type.
Uncertainty_Dist_2	Education	I tended to look at the chart and make my choice based on what I was shown.
Uncertainty_Dist_2	Education	The AI’s prediction, study time, and parents education were the three main pieces of information I used.
Uncertainty_Pt_2	Census	I tended to look at the captial gain and loss as well as the hours worked with their jobs
Uncertainty_Pt_2	Census	Check and analyze the AI prediction with my own prediction and I chose a balance between the two.
Uncertainty_Pt_1	Credit	I tended to follow the AI recommendation. If it was uncertain then I would use the borrowers information to decide. I placed the highest importance on the terms of the loan the amount and the job of the borrower.
Uncertainty_Pt_1	Education	I looked for ambition in the study numbers and desire for higher education . I also looked at the health and alcohol use numbers too . I know how much alcohol can derail even the best intentions .
Uncertainty_Pt_1	Education	Sided with the AI
Uncertainty_Dist_2	Census	I agreed with the graph a majority of the time, if not every single time. It was very useful regarding my decisions.
Uncertainty_Dist_1	Education	After examining all of the data related to the student, I reviewed the predictor to make my final decision.
Uncertainty_Dist_1	Education	I looked at both of the parents education and their relationship with the child to make a decision. the absences played a huge role in my choices too.
Uncertainty_Pt_2	Credit	followed the AI’s suggestions
Uncertainty_Dist_1	Census	I looked at age, education level, and hours worked to make most of my assumptions.
Uncertainty_Pt_1	Census	Look at the AI suggestion, then look at the person’s education level.
Uncertainty_Dist_1	Census	I tended to follow AI. However I noticed that when it came to females and people of color we were not in agreement
Uncertainty_Dist_2	Census	I used the AI as well as the confidence levels from the plot points and the data suggested to make an educated guess.
Uncertainty_Dist_2	Credit	I tended to go with the A.I. but also assess the person’s job and savings.
Uncertainty_Dist_2	Credit	follow the AI
Uncertainty_Pt_2	Credit	I was trying to base it on the amount and the number of months at first, but I honestly could find no real consistency in it. It felt like I was just guessing by the end of it.
Uncertainty_Dist_2	Education	I tried to take it all in at once. If a student studied a lot, I generally viewed that as very positive with a willingness to try to correct themselves. With a lot of studying, I was more likely to give them the benefit of the doubt.don’t think absences generally mean that much, as I missed a lot of school but was able to easily get through college. If other factors were also negative, too many absences would add to me thinking fail, though.consumption did not have much weight to me on its own, but if combined with other factors could rise to the level of being detrimental.failed class did not have a bearing on my decision. Two or more is when it starts to matter to my decision process.did not generally use their parents low education against them, but if both parents had college or above that was a plus. In addition, family lives had a large effect on my decisions as a negative home life can greatly affect a students learning abilities.
Uncertainty_Pt_1	Census	I tried to take into account the gains/losses as an indicator when the AI wasn’t sure.
Uncertainty_Pt_1	Census	I usually made a guess based on all of the provided info and then took into account the AI’s predictions. Most of the time my initial guess was similar to the AI’s.
Uncertainty_Pt_2	Credit	my strategy was to follow all the recommendations of the AI
Uncertainty_Dist_2	Education	At first, I wasn’t sure how I should go about it. After a few though, I found more success by going in this order of priority:ededjobrelationsother information didn’t seem to usually have much of an effect on the results.
Uncertainty_Pt_2	Credit	I tried to focus on the size and length of the loan compared with the person’s current finances.
Uncertainty_Dist_1	Census	Age, job title, marital status, and the AI
Uncertainty_Pt_1	Census	I generally followed the AI’s prediction if it was above 60%. If it was lower, I decided mainly based on the profession, education and age of the person.
Uncertainty_Dist_1	Education	I looked at the provided statistics for each person and made my best educated decision based on the numbers.
Uncertainty_Dist_1	Education	Logistic Regression: Logistic regression is a statistical method that estimates the probability of a binary outcome (e.g., pass/fail) based on one or more input variables.
Uncertainty_Dist_2	Credit	I considered job skill, amount in checking and savings, and what the loan consisted of(amount, duration)
Uncertainty_Pt_1	Education	I looked more at the student than the family accomplishments.
Uncertainty_Dist_2	Education	I prioritized study time/absences and also past failed classes. In my experience these things make more of a difference than the rest of it. This study seemed random as hell, more like you’re trying to find out if people will just automatically agree with the AI.

Combine behavioral, questionnaire, and stimulus data