library(ggpubr)

## Loading required package: ggplot2

mat = read.csv("~/Desktop/LNR.csv", header=T)
mat$Group_Status = paste(mat$Group, mat$Status, sep = "_")

Notes for myself:

Subjective: Observe scan and make judgement call on dye changing color

Objective: Use AI to assist in making the call

Pilot: Pre-educational talk

Post CPD: Post educational talk

We expect Objective to have lower variance than Subjective. Ideally, we would like to see a reduction in variance after the educational talk (i.e Subjective pilot vs Subjective post).

Not 100% sure why we would expect Objective to improve after the educational talk (you’re not teaching the AI!). Maybe we could use this as a “control” of sorts. If variance is wildly different for the AI in pilot vs. post CPD then we might have grounds to think that the data collection or experimental design is flawed.

Aortic Arch

aortic = mat[which(mat$Vessel=="Aortic_arch"),]

Assess the distribution of each group

Histogram plot

Plot the data and see if it fits a bellcurve…

The plot looks really good for aortic data collected

gghistogram(aortic, x="Value",  add_density = TRUE, rug=TRUE, add = "mean" )

## Warning: Using `bins = 30` by default. Pick better value with the argument
## `bins`.

## Warning: `geom_vline()`: Ignoring `mapping` because `xintercept` was provided.

## Warning: `geom_vline()`: Ignoring `data` because `xintercept` was provided.

QQ plot

You can also test if the data is normally distributed using a QQ plot

The points should follow the 45 degree angle blue line - they do for the most part - this looks good…

qqnorm(aortic$Value, pch = 1, frame = FALSE)
qqline(aortic$Value, col = "steelblue", lwd = 2)

Shapiro-Wilk test

Finally, you can formally test if your data follows a normal distribution using the shapiro-wilks test:

The p-value is 0.7167 – we can definitely say that your data for aortic arch follows a normal distribution and it is fine to use the F-test

shapiro.test(aortic$Value)

## 
##  Shapiro-Wilk normality test
## 
## data:  aortic$Value
## W = 0.99016, p-value = 0.7167

Assess Variance (Objective vs. Subjective)

Sanity check here! AI should have less variance than Subjective

I’m starting with the basic ones just to make sure the tests I use are correct and confirm what we are seeing.

Let’s first plot the data..

ggscatter(aortic, x="Group", y="Value")

Two - tailed F test

We are just asking if the variance between the two groups is different.

We are not asking if one is greater or less than the other.

subjective = aortic$Value[aortic$Group=="Subjective"]
objective = aortic$Value[aortic$Group=="Objective"]
var.test(objective, subjective, alternative="two.sided")

## 
##  F test to compare two variances
## 
## data:  objective and subjective
## F = 0.51709, num df = 29, denom df = 63, p-value = 0.05323
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
##  0.2852654 1.0095478
## sample estimates:
## ratio of variances 
##          0.5170863

The result is 0.05 = we can say that the variances between the two groups is significant.

One-tailed F-test

Now we can ask the question “Is the Objective variance less than the Subjective variance?”

var.test(objective, subjective, alternative = "less")

## 
##  F test to compare two variances
## 
## data:  objective and subjective
## F = 0.51709, num df = 29, denom df = 63, p-value = 0.02662
## alternative hypothesis: true ratio of variances is less than 1
## 95 percent confidence interval:
##  0.0000000 0.9041696
## sample estimates:
## ratio of variances 
##          0.5170863

p-value of 0.02662 shows that the AI (Objective) method has significantly less variance compared to the Subjective group. Which we expected.

Boxplots

You would usually plot the data as boxplots instead of just points.

ggboxplot(aortic, y="Value", x="Group", color = "black", fill = "Group", bxp.errorbar = TRUE, bxp.errorbar.width = 0.2, width = 0.5, ggtheme = theme_bw(), palette = "jco",
          ylab = "", xlab="", title = "Aortic Arch")

## Warning in (function (mapping = NULL, data = NULL, geom = "boxplot", position =
## "dodge2", : Ignoring unknown aesthetics: fill

You want to perform a T-test here to show that the data from the two groups are measuring the same thing if that makes sense.

In this case you want a result that is not significant.

If the t-test was significant then the difference in variance could be coming from an external factor that you have not accounted for.

comps = list(c("Subjective", "Objective"))
ggboxplot(aortic, y="Value", x="Group", color = "black", fill = "Group", bxp.errorbar = TRUE, bxp.errorbar.width = 0.2, width = 0.5, ggtheme = theme_bw(), palette = "jco",
          ylab = "", xlab="", title = "Aortic Arch") + coord_cartesian(ylim = c(0, 800)) + stat_compare_means(comparisons = comps, label.y = 720, method = "t.test")

## Warning in (function (mapping = NULL, data = NULL, geom = "boxplot", position =
## "dodge2", : Ignoring unknown aesthetics: fill

Subjective group

Post CPD less variance than Pilot?

The plot looks good, we can see here that the variance in post CPD is lower

x = aortic[which(aortic$Group=="Subjective"),]
ggscatter(x, x="Status", y="Value")

Two-tailed F-test

Are the variances different?

subjective_pilot = x$Value[x$Status=="Pilot"]
subjective_post  = x$Value[x$Status=="Post"]

var.test(subjective_post, subjective_pilot, alternative = "two.sided")

## 
##  F test to compare two variances
## 
## data:  subjective_post and subjective_pilot
## F = 0.95946, num df = 19, denom df = 43, p-value = 0.9565
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
##  0.4656805 2.2258329
## sample estimates:
## ratio of variances 
##          0.9594583

Unfortunately the difference in variance is not statistically significant.

One-tailed F-test

We can still ask the question “Is Subjective Post CPD less variance than Subjective Pilot?”

var.test(subjective_post, subjective_pilot, alternative = "less")

## 
##  F test to compare two variances
## 
## data:  subjective_post and subjective_pilot
## F = 0.95946, num df = 19, denom df = 43, p-value = 0.4783
## alternative hypothesis: true ratio of variances is less than 1
## 95 percent confidence interval:
##  0.00000 1.93502
## sample estimates:
## ratio of variances 
##          0.9594583

The result is still not significant - you do not have a result here.

Boxplot

In this case we would be interested in performing a T-test to double check that the Pilot and Post CPD group are basically recording the same thing.

comps = list(c("Pilot", "Post"))
ggboxplot(x, y="Value", x="Status", color = "black", fill = "Status", bxp.errorbar = TRUE, bxp.errorbar.width = 0.2, width = 0.5, ggtheme = theme_bw(), palette = "jco",
          ylab = "", xlab="", title = "Aortic Arch (Subjective Group)") + coord_cartesian(ylim = c(0, 800)) + stat_compare_means(comparisons = comps, label.y = 720, method = "t.test")

## Warning in (function (mapping = NULL, data = NULL, geom = "boxplot", position =
## "dodge2", : Ignoring unknown aesthetics: fill

The t-test is not significant so unfortunately you just have a non-significant result here. Thats okay, as long as you can articulate your thought process and the stuff I mentioned here

Objective group

for the Objective AI the plot suggests the variance was worse post CPD.

x = aortic[which(aortic$Group=="Objective"),]
ggscatter(x, x="Status", y="Value")

Two-tailed F test

is there a difference in variance?

subjective_pilot = x$Value[x$Status=="Pilot"]
subjective_post  = x$Value[x$Status=="Post"]
var.test(subjective_post, subjective_pilot, alternative="two.sided")

## 
##  F test to compare two variances
## 
## data:  subjective_post and subjective_pilot
## F = 2.3761, num df = 19, denom df = 9, p-value = 0.1844
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
##  0.645099 6.843343
## sample estimates:
## ratio of variances 
##           2.376118

not significant.

One-tailed F-test

Has post CPD less variance than pilot ?

var.test(subjective_post, subjective_pilot, alternative = "less")

## 
##  F test to compare two variances
## 
## data:  subjective_post and subjective_pilot
## F = 2.3761, num df = 19, denom df = 9, p-value = 0.9078
## alternative hypothesis: true ratio of variances is less than 1
## 95 percent confidence interval:
##  0.000000 5.756618
## sample estimates:
## ratio of variances 
##           2.376118

Unfortunately the difference in variance is not statistically significant.

In this case we would be interested in performing a T-test to double check that the Pilot and Post CPD group are basically recording the same thing.

comps = list(c("Pilot", "Post"))
ggboxplot(x, y="Value", x="Status", color = "black", fill = "Status", bxp.errorbar = TRUE, bxp.errorbar.width = 0.2, width = 0.5, ggtheme = theme_bw(), palette = "jco",
          ylab = "", xlab="", title = "Aortic Arch (Subjective Group)") + coord_cartesian(ylim = c(0, 800)) + stat_compare_means(comparisons = comps, label.y = 720, method = "t.test")

## Warning in (function (mapping = NULL, data = NULL, geom = "boxplot", position =
## "dodge2", : Ignoring unknown aesthetics: fill

The t-test is not significant so unfortunately you just have a non-significant result here. Thats okay, as long as you can articulate your thought process and the stuff I mentioned here

El

Barry

2023-11-19

Aortic Arch

Assess the distribution of each group

Histogram plot

QQ plot

Shapiro-Wilk test

Assess Variance (Objective vs. Subjective)

Two - tailed F test

One-tailed F-test

Boxplots

Subjective group

Post CPD less variance than Pilot?

Two-tailed F-test

One-tailed F-test

Boxplot

Objective group

Two-tailed F test

One-tailed F-test