Analysis of BioMAPS data
Background
We are interested in measuring student growth in conceptual understanding as they undertake the first year subjects ‘Life’s machinery’ and ‘Life’s complexity’, and their introductory counterparts. Gen-BioMAPS is a validated instrument designed to measure understanding of five core concepts in biology
Approach
We administer the instrument at the beginning and at the end of each teaching semester, with the goal of understanding whether conceptual understanding has improved over the course of the semester, the course of the year, and ultimately, the course of the degree.
Student performance on BioMAPS is summarised as percentage scores for the test as a whole, and each of the five conceptual areas individually.
#load libraries
library(plyr)
library(tidyverse)
library(knitr)
library(formatR)
library(kableExtra)
library(ggforce)
library(ggplot2)
library(janitor)
library(EnvStats)
#code folding
knitr::opts_chunk$set(echo = TRUE, warning = FALSE, message = FALSE, tidy = TRUE)Overview of the subjects and the students
We will first look at how the first-year subjects differ in terms of their enrolment. We are interested in a number of key variables, namely:
- gender distribution
- proportion of the class that is domestic vs international
- proportion of students for whom english is their first language
- proportion of students who have studied biology at high school (VCE or equivalent) level
- distribution of ATAR scores across the class
# create stacked bars for gender
demographics <- read.csv("data/biol_demographic20_21.csv")
gender_percent <- demographics %>%
group_by(subject_code) %>%
count(gender) %>%
mutate(ratio = scales::percent(n/sum(n)))
ggplot(demographics, aes(x = factor(subject_code), fill = gender)) + geom_bar(position = "fill") +
geom_text(data = gender_percent, aes(y = n, label = ratio), position = position_fill(vjust = 0.5)) +
labs(title = "Class gender distribution by subject", y = "Percentage of cohort",
x = "Subject", caption = "")
All subjects have a female bias; the introductory subjects slightly less so.
# create stacked bars for dom/int
dom_percent <- demographics %>%
group_by(subject_code) %>%
count(source) %>%
mutate(ratio = scales::percent(n/sum(n)))
ggplot(demographics, aes(x = factor(subject_code), fill = source)) + geom_bar(position = "fill") +
geom_text(data = dom_percent, aes(y = n, label = ratio), position = position_fill(vjust = 0.5)) +
labs(title = "Percent international by subject", y = "Percentage of cohort",
x = "Subject", caption = "")
# create stacked bars for language
english_percent <- demographics %>%
group_by(subject_code) %>%
count(language) %>%
mutate(ratio = scales::percent(n/sum(n)))
ggplot(demographics, aes(x = factor(subject_code), fill = language)) + geom_bar(position = "fill") +
geom_text(data = english_percent, aes(y = n, label = ratio), position = position_fill(vjust = 0.5)) +
labs(title = "Language by subject", y = "Percentage of cohort", x = "Subject",
caption = "")
Although the proportion of international students in BIOL10002 is in line with the standard subjects, it is more like the intro subjects in terms of the percentage of students for whom English is not their first language.
# create stacked bars for biol background
biology <- read.csv("data/biomaps_master.csv")
biol_percent <- biology %>%
group_by(subject_code) %>%
count(any_biol) %>%
mutate(ratio = scales::percent(n/sum(n)))
ggplot(biology, aes(x = factor(subject_code), fill = any_biol)) + geom_bar(position = "fill") +
geom_text(data = biol_percent, aes(y = n, label = ratio), position = position_fill(vjust = 0.5)) +
labs(title = "Previous biology by subject", y = "Percentage of cohort", x = "Subject",
caption = "")
As expected given subject admission criteria, the standard students all have some biology background. However, 20-30% of students in the introductory subjects have some VCE-equivalent background. These are probably international students.
atar <- read.csv("data/biol_demographic20_21.csv")
# set new variables
names(atar)[2] <- "Subject"
names(atar)[16] <- "ATAR"
# calculate means for each cohort
means <- ddply(atar, "Subject", summarise, AveATAR = round(mean(ATAR), digits = 1),
N = length(Subject))
# density plot
ggplot(atar, aes(x = ATAR, fill = Subject)) + geom_density(alpha = 0.3) + xlim(70,
100) + geom_vline(data = means, aes(xintercept = AveATAR, colour = Subject),
linetype = "longdash", size = 1) + labs(title = "Density distribution of ATAR by subject \n(dashed lines are means)",
y = "Density", x = "ATAR", caption = "")
ATAR distribution for BIOL10008,9,10,11 is almost identical. Very different for BIOL10002.
Cleanup of data
The data are in files with one row per completed BioMAPS assessment. For each row, there are descriptive columns; e.g. student_id, subject_code, semester, timing (Begin or End), duration (time taken to complete the test), etc. For different analyses, we will load the relevant datasets. The dataset has been reduced to include only attempts that were 100% completed (progress=100, finished=true).
# load data
alldata <- read.csv("data/biomaps_master.csv")
head(alldata)Let’s see whether the test data look sensible (no strange outliers, spurious datapoints etc). To do this, first look at a frequency histogram of the scores. Then plot the overall test score against the time taken to do the test.
# frequency distribution of scores
hist(alldata$total_score, , main = "Frequency distribution of scores", xlab = "Score",
ylab = "Frequency")
# score plotted against time taken (s)
plot(total_score ~ duration, data = alldata, xlab = "Time taken (s)", ylab = "Score")
A few students took an eternity to complete the test (longest attempt was around 1,166,325 seconds or 324 hours!) and this is distorting the scale of the x-axis (pushing the majority of points to the extreme left of the graph). Let’s have a look at a more restricted part of the scale, e.g. students who took less than an hour (3600 seconds.).
# new plot
plot(total_score ~ duration, data = alldata, xlim = c(0, 3600), xlab = "Time taken (s)",
ylab = "Score")
It looks like the plot ‘bends down’ to lower scores for the shorter durations (particularly below 1000 seconds, or about 15 minutes). This is not unexpected - it suggests that students who finished very quickly did not score highly and may not have made a serious attempt. According to the paper describing GEN-BioMAPS, the test is designed to take around 30 minutes to complete. In their study, the authors removed any tests that took less than 10 minutes (600 seconds) to complete, as this was determined to be too short a duration to have made a good faith effort to read and answer the questions. We will do the same and load a reduced dataset. Filtering out any responses that took less than 600 seconds removes 98 datapoints, reducing to overall dataset to 1,255 surveys.
# load data
over600 <- subset(alldata, duration >= 600)
plot(total_score ~ duration, data = over600, xlim = c(0, 3600), xlab = "Time taken (s)",
ylab = "Score")
This looks better, though there is still a noticeable dip in values for shorter survey times. A more conservative approach might be to remove all attempts that took less than 15 minutes (900 seconds). This only removes another 39 attempts, for a total sample of 1,126 surveys
over900 <- subset(alldata, duration >= 900)
plot(total_score ~ duration, data = over900, xlim = c(900, 9000), xlab = "Time taken (s)",
ylab = "Score")
hist(over900$total_score, main = "Frequency distribution of scores", xlab = "Score",
ylab = "Frequency")
Looks OK!
Check that the student population that took the BioMAPs survey is representative of the class population by comparing the distribution of final subject marks obtained by students in the biomaps sample to those obtained by the class as a whole.
# load the dataset with biomaps and class subject marks
marks <- read.csv("data/scores_marks.csv")
# density plot
ggplot(data = marks, aes(x = mark, group = population, fill = population)) + geom_density(adjust = 1.5,
alpha = 0.4) + facet_wrap(~subject_code)
### What level of conceptual understanding do students have on entry into each
### subject?Can’t complain about that!
CONCLUSION
Once we have cleaned the data (removed any students who did not make a serious attempt at the survey, ie took less than 15 minutes to complete it), the distribution looks good. There is no indication that the sample of students who took the survey is anything other than a random slice of the class population.
Conceptual understanding of students on subject entry
Now let’s look at the distributions of intial (beginning) scores according to subject, to see what degree of overlap (if any) there is across subjects by creating a density plot.
# use dataset limited to beginning scores
begindata <- read.csv("data/biomaps_master_begin.csv")
# set new variables
names(begindata)[1] <- "Subject"
names(begindata)[11] <- "Score"
# calculate means for each cohort
means <- ddply(begindata, "Subject", summarise, AveMark = round(mean(Score), digits = 1),
N = length(Subject))
# density plot
ggplot(begindata, aes(x = Score, fill = Subject)) + geom_density(alpha = 0.3) + geom_vline(data = means,
aes(xintercept = AveMark, colour = Subject), linetype = "longdash", size = 1) +
labs(title = "Density distribution of BioMAPS entry score by subject \n(dashed lines are means)",
y = "Density", x = "BioMAPS entry score", caption = "")
Mean BioMAPS scores upon entry into each subject look to be lower for the introductory students, and very similar for the first and second-semester cohorts. But the overlap is getting messy. Might be better to plot these as facet plots.
ggplot(data = begindata, aes(x = Score, group = Subject, fill = Subject)) + geom_density(alpha = 0.3) +
geom_vline(data = means, aes(xintercept = AveMark, colour = Subject), linetype = "longdash",
size = 1) + geom_density(adjust = 1.5, alpha = 0.4) + facet_wrap(~Subject)
Distribution of overall subject marks for comparison
# load dataset
marks <- read.csv("data/biolmarks2020.csv")
# set new variables
names(marks)[1] <- "Subject"
names(marks)[2] <- "Mark"
# calculate means for each cohort
means <- ddply(marks, "Subject", summarise, AveMark = round(mean(Mark), digits = 1),
N = length(Subject))
# density plot
ggplot(marks, aes(x = Mark, fill = Subject)) + geom_density(alpha = 0.3) + geom_vline(data = means,
aes(xintercept = AveMark, colour = Subject), linetype = "longdash", size = 1) +
labs(title = "Density distribution of subject marks by subject \n(dashed lines are means)",
y = "Density", x = "Subject mark", caption = "")
# to run an anova and store the output in an object called 'anovaresults'
anovaresults = with(begindata, aov(Score ~ Subject))
# to see summary output for the ANOVA
summary(anovaresults)## Df Sum Sq Mean Sq F value Pr(>F)
## Subject 6 27193 4532 45.28 <2e-16 ***
## Residuals 1636 163755 100
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# to report the means and the number of subjects per cell
print(model.tables(anovaresults, "means"), digits = 3)## Tables of means
## Grand mean
##
## 69.26873
##
## Subject
## BIOL10002(21) BIOL10008 BIOL10008(21) BIOL10009 BIOL10009(21) BIOL10010
## 72.2 61.9 63.9 71.3 72 63.5
## rep 283.0 181.0 246.0 285.0 500 66.0
## BIOL10011
## 72.2
## rep 82.0# to perform a post-hoc test to see which means are significantly different
# from each other
TukeyHSD(anovaresults)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = Score ~ Subject)
##
## $Subject
## diff lwr upr p adj
## BIOL10008-BIOL10002(21) -10.30373417 -13.1146563 -7.492812 0.0000000
## BIOL10008(21)-BIOL10002(21) -8.26546171 -10.8399409 -5.690983 0.0000000
## BIOL10009-BIOL10002(21) -0.86542406 -3.3438779 1.613030 0.9469482
## BIOL10009(21)-BIOL10002(21) -0.16459944 -2.3615738 2.032375 0.9999903
## BIOL10010-BIOL10002(21) -8.64681611 -12.6839193 -4.609713 0.0000000
## BIOL10011-BIOL10002(21) 0.04157639 -3.6624034 3.745556 1.0000000
## BIOL10008(21)-BIOL10008 2.03827246 -0.8539349 4.930480 0.3648128
## BIOL10009-BIOL10008 9.43831011 6.6312380 12.245382 0.0000000
## BIOL10009(21)-BIOL10008 10.13913473 7.5771792 12.701090 0.0000000
## BIOL10010-BIOL10008 1.65691806 -2.5898598 5.903696 0.9117211
## BIOL10011-BIOL10008 10.34531056 6.4138495 14.276772 0.0000000
## BIOL10009-BIOL10008(21) 7.40003765 4.8297626 9.970313 0.0000000
## BIOL10009(21)-BIOL10008(21) 8.10086227 5.8008022 10.400922 0.0000000
## BIOL10010-BIOL10008(21) -0.38135440 -4.4754700 3.712761 0.9999647
## BIOL10011-BIOL10008(21) 8.30703810 4.5409995 12.073077 0.0000000
## BIOL10009(21)-BIOL10009 0.70082462 -1.4912217 2.892871 0.9653334
## BIOL10010-BIOL10009 -7.78139205 -11.8158155 -3.746969 0.0000003
## BIOL10011-BIOL10009 0.90700045 -2.7940585 4.608059 0.9911886
## BIOL10010-BIOL10009(21) -8.48221667 -12.3501018 -4.614332 0.0000000
## BIOL10011-BIOL10009(21) 0.20617583 -3.3126023 3.724954 0.9999978
## BIOL10011-BIOL10010 8.68839250 3.8044079 13.572377 0.0000035Gen-BioMAPS entry score is significantly higher for std subjects than for intro subjects. Still, the density plots suggest considerable overlap. We can estimate how much.
Semester 1
d8dens <- with(begindata, density(Score[Subject == "BIOL10008"], from = min(Score),
to = max(Score)))
d9dens <- with(begindata, density(Score[Subject == "BIOL10009"], from = min(Score),
to = max(Score)))
joint <- pmin(d8dens$y, d9dens$y)
df2 <- data.frame(x = rep(d8dens$x, 3), y = c(d8dens$y, d9dens$y, joint), Subject = rep(c("BIOL10008",
"BIOL10009", "overlap"), each = length(d8dens$x)))
ggplot(df2, aes(x, y, fill = Subject)) + geom_area(position = position_identity(),
color = "black") + scale_fill_brewer(palette = "Pastel2") + theme_bw()
# calculate the percentage overlap between the two density distributions
(sum(joint)/sum(d8dens$y) + sum(joint)/sum(d9dens$y))/2## [1] 0.6847077Semester 2
d10dens <- with(begindata, density(Score[Subject == "BIOL10010"], from = min(Score),
to = max(Score)))
d11dens <- with(begindata, density(Score[Subject == "BIOL10011"], from = min(Score),
to = max(Score)))
joint <- pmin(d10dens$y, d11dens$y)
df3 <- data.frame(x = rep(d10dens$x, 3), y = c(d10dens$y, d11dens$y, joint), Subject = rep(c("BIOL10010",
"BIOL10011", "overlap"), each = length(d10dens$x)))
ggplot(df3, aes(x, y, fill = Subject)) + geom_area(position = position_identity(),
color = "black") + scale_fill_brewer(palette = "Pastel2") + theme_bw()
# calculate the percentage overlap between the two density distributions
(sum(joint)/sum(d10dens$y) + sum(joint)/sum(d11dens$y))/2## [1] 0.7310392Biomeds With or Without Biology
# read in the biomed dataset
biomeds <- read.csv("data/biomeds.csv")
names(biomeds)[9] <- "BiolBackground"
names(biomeds)[11] <- "Score"
nobiol_dens <- with(biomeds, density(Score[BiolBackground == "no"], from = min(Score),
to = max(Score)))
biol_dens <- with(biomeds, density(Score[BiolBackground == "yes"], from = min(Score),
to = max(Score)))
joint <- pmin(nobiol_dens$y, biol_dens$y)
df4 <- data.frame(x = rep(nobiol_dens$x, 3), y = c(nobiol_dens$y, biol_dens$y, joint),
BiolBackground = rep(c("No Biology", "Biology", "overlap"), each = length(nobiol_dens$x)))
ggplot(df4, aes(x, y, fill = BiolBackground)) + geom_area(position = position_identity(),
color = "black") + scale_fill_brewer(palette = "Pastel2") + theme_bw()
# calculate the percentage overlap between the two density distributions
(sum(joint)/sum(nobiol_dens$y) + sum(joint)/sum(biol_dens$y))/2## [1] 0.5840513CONCLUSION
The standard cohorts score on average 9-10% higher than the introductory cohorts on the instrument when they enter the subject. Note however, that this does not take into account other factors that could influence performance, including language, gender, and domestic/international status. There is no statistical difference between scores obtained by students entering intro and standard subjects in either first or second semester.
There is large overlap in scores each semester between the standard and intro cohorts (69-73 percent, depending on the semester). So, around 70% of students currently taking intro show the same level of conceptual understanding as their peers taking standard.
Relationship between performance on the BioMAPS instrument and on the final exam
This addresses the question of whether our overall assessments are doing a good job at testing for conceptual understanding. For this analysis we will compare final (end of semester) BioMAPS scores and final subject scores.
#remove Biomeds as we don't have a final exam score for them yet
biolexam<-subset(over900, subject_code!='BIOL10002')
ggplot(biolexam, aes(x=total_score, y=exam_raw, color=subject_code)) +
geom_point(shape=1) +
labs(title = "Relationship between End-of-semester BioMAPS score and raw exam score", y = "Raw exam score",
x = "End-of-semester BioMAPS score", caption = "") + theme_bw() +
scale_colour_hue(l=50) + # Use a slightly darker palette than normal
geom_smooth(method=lm, # Add linear regression lines
se=FALSE) # Don't add shaded confidence region
Changes in conceptual understanding over the course of the semester
Questions:
Does conceptual understanding improve over the course of the semester?
Does change in conceptual understanding depend on subject?
Does conceptual understanding improve more for some concepts than others?
To do this analysis, we have created a new dataset, which not only includes the pre-post surveys where students did the test at the beginning and end of the semester (type: pre-post in the dataset), but also using the end-of-first semester score as the ‘pre’ for students in semester 2 (I’m calling these type: post-post).
First, let’s plot changes in BioMaps score over the course of the semester as lines for individual students, with a mean trendline. It’ll be a bit of a forest but interesting nonetheless to see general trends.
# read in the dataset which has begin-end scores for all students in long
# format. Data have already been filtered to remove test durations <900s.
biomaps_change <- read.csv("data/biomaps_beforeafter_long.csv")
ggplot(data = biomaps_change, aes(x = timing, y = total_score, group = student_id,
colour = student_id)) + geom_line() + geom_point(size = 0, shape = 21, fill = "white") +
labs(title = "Change in Gen-BioMAPS score over the course of a semester", y = "BIOMAPS score",
x = "Semester timing of test", caption = "") + theme(legend.position = "none") +
facet_wrap(~subject_code) + geom_smooth(aes(group = subject_code), method = "lm",
se = FALSE, color = "black", size = 1.5)
Slightly positive slopes for the Semester 1 subjects, and flat lines for Second semester. We can look at the data another way by plotting a histogram of change values to see which are the most common ones, and what the mean change is for each subject.
# read in the dataset with gains data. Data have already been filtered to
# remove test durations <900s.
gains <- read.csv("data/biomaps_gains.csv")
# set new variables
names(gains)[2] <- "Subject"
names(gains)[15] <- "Gain"
# calculate means for each cohort
means <- ddply(gains, "Subject", summarise, AveMark = round(mean(Gain), digits = 1),
N = length(Subject))
ggplot(data = gains, aes(Gain)) + geom_histogram(binwidth = 1, color = "blue") +
facet_wrap(~Subject) + theme(legend.position = "none") + geom_vline(data = means,
aes(xintercept = AveMark, colour = "red"), size = 1) + labs(title = "Frequency distribution of changes in BioMAPS score over a semester\n (positive values represent improvement; red lines show means)",
y = "Frequency", x = "Change in BioMAPS score", caption = "")
So, can we explain who is gaining and who is not? Start by plotting starting BioMAPS entry score against gain.
#read in the dataset with gains data in wide format. Data have already been filtered to remove test durations <900s.
gains_wide <- read.csv("data/biomaps_gains_wide.csv")
ggplot(gains_wide, aes(x=total_score_begin, total_score_gain, color=subject_code)) +
geom_point(shape=1) +
labs(title = "Starting BioMAPS score and change in conceptual understanding", y = "Change in BIOMAPS over semester",
x = "BioMAPS entry score", caption = "") + theme_bw() +
scale_colour_hue(l=50) + # Use a slightly darker palette than normal
geom_smooth(method=lm, # Add linear regression lines
se=FALSE) # Don't add shaded confidence region
Umm … looks like a negative relationship in all subjects! We’d expect that the weakest students might gain the most and stronger students gain less, but it looks like students with excellent conceptual understanding on entry to the course might actually be going backward.
Predictors of student performance on the Gen-BioMAPS instrument
Model change in conceptual understanding against factors such as subject code, language etc.
m1 <- lm(total_score_gain ~ stream + any_biol + language + dom_int + gender, data = gains_wide)
# summary(m1)
summary(m1)##
## Call:
## lm(formula = total_score_gain ~ stream + any_biol + language +
## dom_int + gender, data = gains_wide)
##
## Residuals:
## Min 1Q Median 3Q Max
## -25.261 -5.793 -0.053 5.398 28.947
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 5.1328 1.1766 4.363 1.8e-05 ***
## streamstd -0.8453 2.2974 -0.368 0.713
## any_biolyes -1.4701 2.4215 -0.607 0.544
## languageother -0.5045 1.5436 -0.327 0.744
## dom_intinternational -1.0478 1.5063 -0.696 0.487
## genderM 1.2437 1.1713 1.062 0.289
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.74 on 286 degrees of freedom
## Multiple R-squared: 0.0198, Adjusted R-squared: 0.002668
## F-statistic: 1.156 on 5 and 286 DF, p-value: 0.3312eSize <- coef(m1)
eSizeSub <- eSize[!(grepl(pattern = "Interc", names(eSize)))]
par(mar = c(9, 4, 2, 2))
x <- barplot(eSizeSub, las = 2, ylab = "Effect size on conceptual gains", xaxt = "n")
text(x, y = min(eSizeSub) * 1.1, labels = names(eSizeSub), xpd = TRUE, srt = 45,
pos = 2)
It doesn’t appear that there are any key predictors of change in conceptual understanding over the course of the semester (including stream).
What if we just look at their performance in the exam?
m1 <- lm(exam_raw ~ stream + any_biol + language + dom_int + gender, data = gains_wide)
# summary(m1)
summary(m1)##
## Call:
## lm(formula = exam_raw ~ stream + any_biol + language + dom_int +
## gender, data = gains_wide)
##
## Residuals:
## Min 1Q Median 3Q Max
## -37.358 -6.207 0.171 6.967 25.620
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 66.1719 1.3641 48.509 < 2e-16 ***
## streamstd 4.4897 2.7219 1.649 0.10015
## any_biolyes -0.6581 2.8638 -0.230 0.81841
## languageother -6.0022 1.7746 -3.382 0.00082 ***
## dom_intinternational -1.6020 1.7335 -0.924 0.35619
## genderM 0.3560 1.3524 0.263 0.79257
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 10.04 on 284 degrees of freedom
## (2 observations deleted due to missingness)
## Multiple R-squared: 0.0923, Adjusted R-squared: 0.07632
## F-statistic: 5.776 on 5 and 284 DF, p-value: 4.266e-05eSize <- coef(m1)
eSizeSub <- eSize[!(grepl(pattern = "Interc", names(eSize)))]
par(mar = c(9, 4, 2, 2))
x <- barplot(eSizeSub, las = 2, ylab = "Effect size on conceptual gains", xaxt = "n")
text(x, y = min(eSizeSub) * 1.1, labels = names(eSizeSub), xpd = TRUE, srt = 45,
pos = 2)
Largest and only significant effect on exam performance is language (students with first language other than English do worse on the exam than those whose first language is English). Biology background and gender have almost zero effect.
Is the story the same for predicting final BioMAPS score?
m1 <- lm(total_score_end ~ stream + any_biol + language + dom_int + gender, data = gains_wide)
# summary(m1)
summary(m1)##
## Call:
## lm(formula = total_score_end ~ stream + any_biol + language +
## dom_int + gender, data = gains_wide)
##
## Residuals:
## Min 1Q Median 3Q Max
## -26.2355 -6.0753 0.0691 6.7016 22.6770
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 71.3529 1.2567 56.778 < 2e-16 ***
## streamstd 3.4326 2.4539 1.399 0.16296
## any_biolyes 2.3532 2.5865 0.910 0.36369
## languageother -3.8614 1.6488 -2.342 0.01987 *
## dom_intinternational -0.2142 1.6089 -0.133 0.89418
## genderM 3.4121 1.2511 2.727 0.00678 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 9.336 on 286 degrees of freedom
## Multiple R-squared: 0.09976, Adjusted R-squared: 0.08402
## F-statistic: 6.339 on 5 and 286 DF, p-value: 1.339e-05eSize <- coef(m1)
eSizeSub <- eSize[!(grepl(pattern = "Interc", names(eSize)))]
par(mar = c(9, 4, 2, 2))
x <- barplot(eSizeSub, las = 2, ylab = "Effect size on conceptual gains", xaxt = "n")
text(x, y = min(eSizeSub) * 1.1, labels = names(eSizeSub), xpd = TRUE, srt = 45,
pos = 2)
Weaker effect of language, but also an effect of gender (boys do better)? And for starting score?
m1 <- lm(total_score_begin ~ stream + any_biol + language + dom_int + gender, data = gains_wide)
# summary(m1)
summary(m1)##
## Call:
## lm(formula = total_score_begin ~ stream + any_biol + language +
## dom_int + gender, data = gains_wide)
##
## Residuals:
## Min 1Q Median 3Q Max
## -32.292 -7.482 0.391 7.768 22.512
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 66.2201 1.3609 48.658 <2e-16 ***
## streamstd 4.2778 2.6574 1.610 0.1086
## any_biolyes 3.8233 2.8010 1.365 0.1733
## languageother -3.3568 1.7855 -1.880 0.0611 .
## dom_intinternational 0.8336 1.7423 0.478 0.6327
## genderM 2.1683 1.3548 1.600 0.1106
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 10.11 on 286 degrees of freedom
## Multiple R-squared: 0.1159, Adjusted R-squared: 0.1005
## F-statistic: 7.502 on 5 and 286 DF, p-value: 1.236e-06eSize <- coef(m1)
eSizeSub <- eSize[!(grepl(pattern = "Interc", names(eSize)))]
par(mar = c(9, 4, 2, 2))
x <- barplot(eSizeSub, las = 2, ylab = "Effect size on conceptual gains", xaxt = "n")
text(x, y = min(eSizeSub) * 1.1, labels = names(eSizeSub), xpd = TRUE, srt = 45,
pos = 2)
Which shows that if we look only at stream, there is a strong significant effect on performance on the BioMAPS instrument. But if we also account for biology background, language etc., the effect is weaker and non-significant.
What about change in conceptual understanding versus exam score?
#read in the dataset with gains data in wide format. Data have already been filtered to remove test durations <900s.
ggplot(gains_wide, aes(x=total_score_gain, exam_raw, color=subject_code)) +
geom_point(shape=1) +
labs(title = "Change in conceptual understanding and exam score", y = "Raw exam score",
x = "Change in BIOMAPS over semester", caption = "") + theme_bw() +
scale_colour_hue(l=50) + # Use a slightly darker palette than normal
geom_smooth(method=lm, # Add linear regression lines
se=FALSE) # Don't add shaded confidence region
No pattern
Next, let’s explore changes in understanding over the semester. We now read in the data in a file that has the same response variable (‘score’) for each student, split into their total score, and their score for each of the five concept areas: energy & matter, evolution, information flow, structure & function and systems. We will visualise changes in conceptual understanding (scores) by means of violin plots for subsets of the data.
# read in data
alldata_categories <- read.csv("data/biomaps_alldata_categories.csv")
# this datafile contains all attempts at the test, but we want to now restrict
# the datafile to students who did the test at least twice. For every attempt
# at the test, the datafile contains 6 rows (total, plus each of the five
# categories). So, any student who has done the test at least twice will have
# at least 12 rows for a particular student number. remove all rows for student
# IDs that have less than 12 rows and create a new datafile #'before_after'
before_after <- alldata_categories %>%
group_by(alldata_categories$subject_code, alldata_categories$student_id) %>%
filter(n() > 11)# the new datafiles have a before_after stem with a suffix indicating the
# subset (eg before_after_std, or before_after_biol10008).
before_after_total <- before_after %>%
filter(measure == "total")
before_after_std <- before_after %>%
filter(stream == "std")
before_after_intro <- before_after %>%
filter(stream == "intro")
before_after_biol10008 <- before_after %>%
filter(subject_code == "BIOL10008")
before_after_biol10009 <- before_after %>%
filter(subject_code == "BIOL10009")
before_after_biol10010 <- before_after %>%
filter(subject_code == "BIOL10010")
before_after_biol10011 <- before_after %>%
filter(subject_code == "BIOL10011")Let’s start by looking at before-after data for total scores in each of the four subjects.
# violin plots with multiple categories use dodge function to align violins,
# boxplots and points The sina plot is a data visualization chart suitable for
# plotting any single variable in a multiclass dataset. It is an enhanced
# jitter strip chart, where the width of the jitter is controlled by the
# density distribution of the data within each class. geom_sina requires
# package ggforce
dodge <- position_dodge(width = 1)
p <- ggplot(before_after_total, aes(x = subject_code, y = score, fill = timing)) +
geom_violin(position = dodge) + scale_fill_brewer(palette = "PuBu") + theme_minimal() +
geom_boxplot(width = 0.3, position = dodge, col = "black") + geom_sina(position = dodge,
alpha = 0.3) + stat_n_text() + labs(title = "Change in Gen-BioMAPS total score over the semester, by subject",
y = "Score", x = "Subject", caption = "")
p
# to run an anova and store the output in an object called '2wayanovaresults'
twowayanovaresults = with(before_after_total, aov(score ~ timing * subject_code))
# to see summary output for the ANOVA
summary(twowayanovaresults)## Df Sum Sq Mean Sq F value Pr(>F)
## timing 1 2484 2483.7 24.376 1.24e-06 ***
## subject_code 3 3982 1327.2 13.026 4.43e-08 ***
## timing:subject_code 3 576 192.1 1.885 0.132
## Residuals 346 35254 101.9
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# to report the means and the number of subjects per cell
print(model.tables(twowayanovaresults, "means"), digits = 3)## Tables of means
## Grand mean
##
## 73.08137
##
## timing
## Begin End
## 70.4 75.7
## rep 177.0 177.0
##
## subject_code
## BIOL10008 BIOL10009 BIOL10010 BIOL10011
## 68.9 75.2 66.8 76.4
## rep 96.0 186.0 24.0 48.0
##
## timing:subject_code
## subject_code
## timing BIOL10008 BIOL10009 BIOL10010 BIOL10011
## Begin 64.5 72.8 66.4 75.3
## rep 48.0 93.0 12.0 24.0
## End 73.4 77.6 67.1 77.4
## rep 48.0 93.0 12.0 24.0# to perform a post-hoc test to see which means are significantly different
# from each other
TukeyHSD(twowayanovaresults)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = score ~ timing * subject_code)
##
## $timing
## diff lwr upr p adj
## End-Begin 5.297552 3.187152 7.407952 1.2e-06
##
## $subject_code
## diff lwr upr p adj
## BIOL10009-BIOL10008 6.285109 3.010337 9.559882 0.0000068
## BIOL10010-BIOL10008 -2.162778 -8.109780 3.784223 0.7839425
## BIOL10011-BIOL10008 7.454448 2.847921 12.060975 0.0002179
## BIOL10010-BIOL10009 -8.447888 -14.099810 -2.795965 0.0007814
## BIOL10011-BIOL10009 1.169338 -3.049370 5.388047 0.8908897
## BIOL10011-BIOL10010 9.617226 3.102613 16.131840 0.0009381
##
## $`timing:subject_code`
## diff lwr upr p adj
## End:BIOL10008-Begin:BIOL10008 8.9237324 2.6402395 15.207225 0.0005119
## Begin:BIOL10009-Begin:BIOL10008 8.3201703 2.8493277 13.791013 0.0001344
## End:BIOL10009-Begin:BIOL10008 13.1737810 7.7029385 18.644624 0.0000000
## Begin:BIOL10010-Begin:BIOL10008 1.9239547 -8.0111200 11.859029 0.9989767
## End:BIOL10010-Begin:BIOL10008 2.6742208 -7.2608539 12.609296 0.9918749
## Begin:BIOL10011-Begin:BIOL10008 10.8967610 3.2010852 18.592437 0.0005405
## End:BIOL10011-Begin:BIOL10008 12.9358672 5.2401914 20.631543 0.0000135
## Begin:BIOL10009-End:BIOL10008 -0.6035622 -6.0744047 4.867280 0.9999763
## End:BIOL10009-End:BIOL10008 4.2500486 -1.2207940 9.720891 0.2600221
## Begin:BIOL10010-End:BIOL10008 -6.9997778 -16.9348525 2.935297 0.3859579
## End:BIOL10010-End:BIOL10008 -6.2495116 -16.1845864 3.685563 0.5392424
## Begin:BIOL10011-End:BIOL10008 1.9730286 -5.7226472 9.668704 0.9939587
## End:BIOL10011-End:BIOL10008 4.0121347 -3.6835410 11.707811 0.7562075
## End:BIOL10009-Begin:BIOL10009 4.8536108 0.3394161 9.367805 0.0251090
## Begin:BIOL10010-Begin:BIOL10009 -6.3962156 -15.8383310 3.045900 0.4393065
## End:BIOL10010-Begin:BIOL10009 -5.6459495 -15.0880649 3.796166 0.6044699
## Begin:BIOL10011-Begin:BIOL10009 2.5765907 -4.4711933 9.624375 0.9533219
## End:BIOL10011-Begin:BIOL10009 4.6156969 -2.4320872 11.663481 0.4852416
## Begin:BIOL10010-End:BIOL10009 -11.2498263 -20.6919418 -1.807711 0.0076874
## End:BIOL10010-End:BIOL10009 -10.4995602 -19.9416757 -1.057445 0.0175258
## Begin:BIOL10011-End:BIOL10009 -2.2770200 -9.3248041 4.770764 0.9763147
## End:BIOL10011-End:BIOL10009 -0.2379138 -7.2856979 6.809870 1.0000000
## End:BIOL10010-Begin:BIOL10010 0.7502661 -11.8167198 13.317252 0.9999997
## Begin:BIOL10011-Begin:BIOL10010 8.9728063 -1.9105227 19.856135 0.1928721
## End:BIOL10011-Begin:BIOL10010 11.0119125 0.1285834 21.895242 0.0450586
## Begin:BIOL10011-End:BIOL10010 8.2225402 -2.6607889 19.105869 0.2943595
## End:BIOL10011-End:BIOL10010 10.2616464 -0.6216827 21.144975 0.0808332
## End:BIOL10011-Begin:BIOL10011 2.0391062 -6.8470948 10.925307 0.9969679But really we should be using the power of paired values for the same individual to run paired t-tests.
BIOL10008
# filter the before-after data for each subject to include only student_ids
# with before-after data, and only the total score
library(dplyr)
b8 <- before_after_biol10008 %>%
filter(duplicated(student_id) | duplicated(student_id, fromLast = TRUE), measure ==
"total") %>%
arrange(timing, student_id)
with(b8, t.test(score ~ timing, paired = TRUE))##
## Paired t-test
##
## data: score by timing
## t = -5.4319, df = 47, p-value = 1.927e-06
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -12.228688 -5.618777
## sample estimates:
## mean of the differences
## -8.923732BIOL10009
b9 <- before_after_biol10009 %>%
filter(duplicated(student_id) | duplicated(student_id, fromLast = TRUE), measure ==
"total") %>%
arrange(timing, student_id)
with(b9, t.test(score ~ timing, paired = TRUE))##
## Paired t-test
##
## data: score by timing
## t = -6.8178, df = 92, p-value = 9.491e-10
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -6.267514 -3.439707
## sample estimates:
## mean of the differences
## -4.853611BIOL10010
b10 <- before_after_biol10010 %>%
filter(duplicated(student_id) | duplicated(student_id, fromLast = TRUE), measure ==
"total") %>%
arrange(timing, student_id)
with(b10, t.test(score ~ timing, paired = TRUE))##
## Paired t-test
##
## data: score by timing
## t = -0.30578, df = 11, p-value = 0.7655
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -6.150702 4.650170
## sample estimates:
## mean of the differences
## -0.7502661BIOL10011
b11 <- before_after_biol10011 %>%
filter(duplicated(student_id) | duplicated(student_id, fromLast = TRUE), measure ==
"total") %>%
arrange(timing, student_id)
with(b11, t.test(score ~ timing, paired = TRUE))##
## Paired t-test
##
## data: score by timing
## t = -1.1037, df = 23, p-value = 0.2811
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -5.860820 1.782608
## sample estimates:
## mean of the differences
## -2.039106CONCLUSION
Subject and Timing are significant predictors of score. Mean scores on the Gen-BioMAPS instrument improve in all subjects over the course of a semester. The percentage improvement ranges from 3.1% (BIOL100011) to 9% (BIOL10008). The changes are statistically significant for the subjects in first semester but not in second semester.
Now looking at before-after data for all of the ‘standard’ students (combined BIOL10009 and 100011).
# violin plots with multiple categories use dodge function to align violins,
# boxplots and points The sina plot is a data visualization chart suitable for
# plotting any single variable in a multiclass dataset. It is an enhanced
# jitter strip chart, where the width of the jitter is controlled by the
# density distribution of the data within each class. geom_sina requires
# package ggforce
dodge <- position_dodge(width = 1)
p <- ggplot(before_after_std, aes(x = measure, y = score, fill = timing)) + geom_violin(position = dodge) +
scale_fill_brewer(palette = "PuBu") + theme_minimal() + geom_boxplot(width = 0.3,
position = dodge, col = "black") + geom_sina(position = dodge, alpha = 0.3) +
labs(title = "Change in Gen-BioMAPS total score over the semester, \n 'Standard' students (BIOL10009+BIOL10011)",
y = "Score", x = "Concept area", caption = "") + scale_x_discrete(labels = function(x) {
sub("\\s", "\n", x)
})
p
# to run an anova and store the output in an object called
# '2wayanovaresults_std'
twowayanovaresults_std = with(before_after_std, aov(score ~ timing * measure))
# to see summary output for the ANOVA
summary(twowayanovaresults_std)## Df Sum Sq Mean Sq F value Pr(>F)
## timing 1 6289 6289 31.718 2.15e-08 ***
## measure 5 8748 1750 8.824 2.95e-08 ***
## timing:measure 5 107 21 0.108 0.991
## Residuals 1392 276006 198
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# to report the means and the number of subjects per cell
print(model.tables(twowayanovaresults_std, "means"), digits = 3)## Tables of means
## Grand mean
##
## 75.57922
##
## timing
## timing
## Begin End
## 73.5 77.7
##
## measure
## measure
## energy & matter evolution information flow
## 74.8 71.6 74.3
## structure & function systems total
## 79.2 78.1 75.4
##
## timing:measure
## measure
## timing energy & matter evolution information flow structure & function systems
## Begin 72.4 69.7 72.2 76.8 76.4
## End 77.2 73.5 76.5 81.7 79.7
## measure
## timing total
## Begin 73.3
## End 77.6# to perform a post-hoc test to see which means are significantly different
# from each other
TukeyHSD(twowayanovaresults_std)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = score ~ timing * measure)
##
## $timing
## diff lwr upr p adj
## End-Begin 4.232909 2.75852 5.707297 0
##
## $measure
## diff lwr upr
## evolution-energy & matter -3.1886868 -6.90363731 0.52626367
## information flow-energy & matter -0.4877468 -4.20269725 3.22720374
## structure & function-energy & matter 4.4147479 0.69979738 8.12969836
## systems-energy & matter 3.2669595 -0.44799104 6.98190994
## total-energy & matter 0.6338137 -3.08113677 4.34876421
## information flow-evolution 2.7009401 -1.01401043 6.41589056
## structure & function-evolution 7.6034347 3.88848420 11.31838518
## systems-evolution 6.4556463 2.74069578 10.17059676
## total-evolution 3.8225005 0.10755005 7.53745103
## structure & function-information flow 4.9024946 1.18754413 8.61744511
## systems-information flow 3.7547062 0.03975571 7.46965670
## total-information flow 1.1215605 -2.59339002 4.83651096
## systems-structure & function -1.1477884 -4.86273891 2.56716207
## total-structure & function -3.7809341 -7.49588464 -0.06598366
## total-systems -2.6331457 -6.34809622 1.08180476
## p adj
## evolution-energy & matter 0.1402640
## information flow-energy & matter 0.9990411
## structure & function-energy & matter 0.0093193
## systems-energy & matter 0.1219216
## total-energy & matter 0.9966308
## information flow-evolution 0.3011487
## structure & function-evolution 0.0000001
## systems-evolution 0.0000118
## total-evolution 0.0394910
## structure & function-information flow 0.0023818
## systems-information flow 0.0458682
## total-information flow 0.9554633
## systems-structure & function 0.9509246
## total-structure & function 0.0433041
## total-systems 0.3298157
##
## $`timing:measure`
## diff lwr
## End:energy & matter-Begin:energy & matter 4.8159466 -1.2108982
## Begin:evolution-Begin:energy & matter -2.6946441 -8.7214888
## End:evolution-Begin:energy & matter 1.1332170 -4.8936277
## Begin:information flow-Begin:energy & matter -0.2460304 -6.2728751
## End:information flow-Begin:energy & matter 4.0864835 -1.9403612
## Begin:structure & function-Begin:energy & matter 4.3898806 -1.6369641
## End:structure & function-Begin:energy & matter 9.2555617 3.2287170
## Begin:systems-Begin:energy & matter 4.0353456 -1.9914991
## End:systems-Begin:energy & matter 7.3145199 1.2876752
## Begin:total-Begin:energy & matter 0.9036488 -5.1231959
## End:total-Begin:energy & matter 5.1799252 -0.8469195
## Begin:evolution-End:energy & matter -7.5105906 -13.5374353
## End:evolution-End:energy & matter -3.6827296 -9.7095743
## Begin:information flow-End:energy & matter -5.0619770 -11.0888217
## End:information flow-End:energy & matter -0.7294631 -6.7563078
## Begin:structure & function-End:energy & matter -0.4260659 -6.4529106
## End:structure & function-End:energy & matter 4.4396151 -1.5872296
## Begin:systems-End:energy & matter -0.7806010 -6.8074457
## End:systems-End:energy & matter 2.4985733 -3.5282714
## Begin:total-End:energy & matter -3.9122978 -9.9391425
## End:total-End:energy & matter 0.3639787 -5.6628660
## End:evolution-Begin:evolution 3.8278611 -2.1989836
## Begin:information flow-Begin:evolution 2.4486136 -3.5782311
## End:information flow-Begin:evolution 6.7811275 0.7542828
## Begin:structure & function-Begin:evolution 7.0845247 1.0576800
## End:structure & function-Begin:evolution 11.9502057 5.9233610
## Begin:systems-Begin:evolution 6.7299896 0.7031449
## End:systems-Begin:evolution 10.0091640 3.9823193
## Begin:total-Begin:evolution 3.5982928 -2.4285519
## End:total-Begin:evolution 7.8745693 1.8477246
## Begin:information flow-End:evolution -1.3792474 -7.4060921
## End:information flow-End:evolution 2.9532665 -3.0735782
## Begin:structure & function-End:evolution 3.2566636 -2.7701811
## End:structure & function-End:evolution 8.1223447 2.0955000
## Begin:systems-End:evolution 2.9021286 -3.1247161
## End:systems-End:evolution 6.1813029 0.1544582
## Begin:total-End:evolution -0.2295682 -6.2564129
## End:total-End:evolution 4.0467082 -1.9801365
## End:information flow-Begin:information flow 4.3325139 -1.6943308
## Begin:structure & function-Begin:information flow 4.6359110 -1.3909337
## End:structure & function-Begin:information flow 9.5015921 3.4747474
## Begin:systems-Begin:information flow 4.2813760 -1.7454687
## End:systems-Begin:information flow 7.5605503 1.5337056
## Begin:total-Begin:information flow 1.1496792 -4.8771655
## End:total-Begin:information flow 5.4259557 -0.6008890
## Begin:structure & function-End:information flow 0.3033972 -5.7234475
## End:structure & function-End:information flow 5.1690782 -0.8577665
## Begin:systems-End:information flow -0.0511379 -6.0779826
## End:systems-End:information flow 3.2280364 -2.7988083
## Begin:total-End:information flow -3.1828347 -9.2096794
## End:total-End:information flow 1.0934418 -4.9334029
## End:structure & function-Begin:structure & function 4.8656810 -1.1611637
## Begin:systems-Begin:structure & function -0.3545351 -6.3813798
## End:systems-Begin:structure & function 2.9246393 -3.1022054
## Begin:total-Begin:structure & function -3.4862319 -9.5130766
## End:total-Begin:structure & function 0.7900446 -5.2368001
## Begin:systems-End:structure & function -5.2202161 -11.2470608
## End:systems-End:structure & function -1.9410418 -7.9678865
## Begin:total-End:structure & function -8.3519129 -14.3787576
## End:total-End:structure & function -4.0756364 -10.1024811
## End:systems-Begin:systems 3.2791743 -2.7476704
## Begin:total-Begin:systems -3.1316968 -9.1585415
## End:total-Begin:systems 1.1445797 -4.8822650
## Begin:total-End:systems -6.4108711 -12.4377158
## End:total-End:systems -2.1345947 -8.1614394
## End:total-Begin:total 4.2762765 -1.7505682
## upr p adj
## End:energy & matter-Begin:energy & matter 10.8427913 0.2715711
## Begin:evolution-Begin:energy & matter 3.3322006 0.9500043
## End:evolution-Begin:energy & matter 7.1600617 0.9999791
## Begin:information flow-Begin:energy & matter 5.7808143 1.0000000
## End:information flow-Begin:energy & matter 10.1133282 0.5354573
## Begin:structure & function-Begin:energy & matter 10.4167253 0.4172982
## End:structure & function-Begin:energy & matter 15.2824064 0.0000361
## Begin:systems-Begin:energy & matter 10.0621903 0.5558284
## End:systems-Begin:energy & matter 13.3413646 0.0042654
## Begin:total-Begin:energy & matter 6.9304935 0.9999980
## End:total-Begin:energy & matter 11.2067699 0.1752448
## Begin:evolution-End:energy & matter -1.4837459 0.0027822
## End:evolution-End:energy & matter 2.3441151 0.6932980
## Begin:information flow-End:energy & matter 0.9648677 0.2033857
## End:information flow-End:energy & matter 5.2973816 0.9999998
## Begin:structure & function-End:energy & matter 5.6007788 1.0000000
## End:structure & function-End:energy & matter 10.4664598 0.3987782
## Begin:systems-End:energy & matter 5.2462437 0.9999996
## End:systems-End:energy & matter 8.5254180 0.9711016
## Begin:total-End:energy & matter 2.1145469 0.6047290
## End:total-End:energy & matter 6.3908234 1.0000000
## End:evolution-Begin:evolution 9.8547058 0.6378820
## Begin:information flow-Begin:evolution 8.4754584 0.9751920
## End:information flow-Begin:evolution 12.8079722 0.0127443
## Begin:structure & function-Begin:evolution 13.1113694 0.0069233
## End:structure & function-Begin:evolution 17.9770504 0.0000000
## Begin:systems-Begin:evolution 12.7568343 0.0140782
## End:systems-Begin:evolution 16.0360087 0.0000042
## Begin:total-Begin:evolution 9.6251375 0.7242243
## End:total-Begin:evolution 13.9014140 0.0012164
## Begin:information flow-End:evolution 4.6475973 0.9998502
## End:information flow-End:evolution 8.9801112 0.9077458
## Begin:structure & function-End:evolution 9.2835083 0.8347156
## End:structure & function-End:evolution 14.1491894 0.0006757
## Begin:systems-End:evolution 8.9289733 0.9175243
## End:systems-End:evolution 12.2081476 0.0384862
## Begin:total-End:evolution 5.7972765 1.0000000
## End:total-End:evolution 10.0735529 0.5513004
## End:information flow-Begin:information flow 10.3593586 0.4390375
## Begin:structure & function-Begin:information flow 10.6627557 0.3292824
## End:structure & function-Begin:information flow 15.5284368 0.0000182
## Begin:systems-Begin:information flow 10.3082207 0.4587134
## End:systems-Begin:information flow 13.5873950 0.0024900
## Begin:total-Begin:information flow 7.1765239 0.9999758
## End:total-Begin:information flow 11.4528004 0.1257958
## Begin:structure & function-End:information flow 6.3302419 1.0000000
## End:structure & function-End:information flow 11.1959229 0.1777101
## Begin:systems-End:information flow 5.9757068 1.0000000
## End:systems-End:information flow 9.2548811 0.8426905
## Begin:total-End:information flow 2.8440100 0.8548334
## End:total-End:information flow 7.1202865 0.9999855
## End:structure & function-Begin:structure & function 10.8925257 0.2567642
## Begin:systems-Begin:structure & function 5.6723096 1.0000000
## End:systems-Begin:structure & function 8.9514840 0.9133091
## Begin:total-Begin:structure & function 2.5406128 0.7633133
## End:total-Begin:structure & function 6.8168893 0.9999995
## Begin:systems-End:structure & function 0.8066286 0.1663029
## End:systems-End:structure & function 4.0858029 0.9963345
## Begin:total-End:structure & function -2.3250682 0.0003852
## End:total-End:structure & function 1.9512083 0.5397759
## End:systems-Begin:systems 9.3060190 0.8282922
## Begin:total-Begin:systems 2.8951479 0.8678957
## End:total-Begin:systems 7.1714244 0.9999769
## Begin:total-End:systems -0.3840264 0.0256279
## End:total-End:systems 3.8922500 0.9917226
## End:total-Begin:total 10.3031212 0.4606891CONCLUSION
Mean scores improve in all concept areas in standard subjects, but changes are generally small and only statistically significantly in two:
energy & matter (+5.2%, p<0.01)
evolution (+4.7%, p=0.03)
information flow (+2.1%, p=0.9)
structure & function (+4.1%, p=0.11)
systems (+3.3%, p=0.39)
Now the ‘intro’ students (combined BIOL10008 and 100010).
# violin plots with multiple categories use dodge function to align violins,
# boxplots and points The sina plot is a data visualization chart suitable for
# plotting any single variable in a multiclass dataset. It is an enhanced
# jitter strip chart, where the width of the jitter is controlled by the
# density distribution of the data within each class. geom_sina requires
# package ggforce
dodge <- position_dodge(width = 1)
p <- ggplot(before_after_intro, aes(x = measure, y = score, fill = timing)) + geom_violin(position = dodge) +
scale_fill_brewer(palette = "PuBu") + theme_minimal() + geom_boxplot(width = 0.3,
position = dodge, col = "black") + geom_sina(position = dodge, alpha = 0.3) +
labs(title = "Change in Gen-BioMAPS total score over the semester, \n 'Intro' students (BIOL10008+BIOL10010)",
y = "Score", x = "Concept area", caption = "") + scale_x_discrete(labels = function(x) {
sub("\\s", "\n", x)
})
p
# to run an anova and store the output in an object called
# '2wayanovaresults_intro'
twowayanovaresults_intro = with(before_after_intro, aov(score ~ timing * measure))
# to see summary output for the ANOVA
summary(twowayanovaresults_intro)## Df Sum Sq Mean Sq F value Pr(>F)
## timing 1 8844 8844 38.962 7.44e-10 ***
## measure 5 6748 1350 5.946 2.13e-05 ***
## timing:measure 5 415 83 0.365 0.872
## Residuals 708 160704 227
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# to report the means and the number of subjects per cell
print(model.tables(twowayanovaresults_intro, "means"), digits = 3)## Tables of means
## Grand mean
##
## 68.71366
##
## timing
## timing
## Begin End
## 65.2 72.2
##
## measure
## measure
## energy & matter evolution information flow
## 67.2 67.6 64.3
## structure & function systems total
## 70.6 74.1 68.5
##
## timing:measure
## measure
## timing energy & matter evolution information flow structure & function systems
## Begin 63.5 63.5 62.4 66.7 70.3
## End 71.0 71.6 66.1 74.4 78.0
## measure
## timing total
## Begin 64.8
## End 72.1# to perform a post-hoc test to see which means are significantly different
# from each other
TukeyHSD(twowayanovaresults_intro)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = score ~ timing * measure)
##
## $timing
## diff lwr upr p adj
## End-Begin 7.009454 4.804741 9.214167 0
##
## $measure
## diff lwr upr
## evolution-energy & matter 0.3501018 -5.2078376 5.9080412
## information flow-energy & matter -2.9588830 -8.5168224 2.5990564
## structure & function-energy & matter 3.3390728 -2.2188666 8.8970122
## systems-energy & matter 6.9167511 1.3588117 12.4746905
## total-energy & matter 1.2518131 -4.3061263 6.8097525
## information flow-evolution -3.3089848 -8.8669242 2.2489546
## structure & function-evolution 2.9889710 -2.5689684 8.5469104
## systems-evolution 6.5666493 1.0087099 12.1245887
## total-evolution 0.9017113 -4.6562281 6.4596507
## structure & function-information flow 6.2979558 0.7400164 11.8558952
## systems-information flow 9.8756341 4.3176947 15.4335735
## total-information flow 4.2106961 -1.3472433 9.7686355
## systems-structure & function 3.5776783 -1.9802611 9.1356177
## total-structure & function -2.0872597 -7.6451991 3.4706797
## total-systems -5.6649380 -11.2228774 -0.1069986
## p adj
## evolution-energy & matter 0.9999739
## information flow-energy & matter 0.6506364
## structure & function-energy & matter 0.5210791
## systems-energy & matter 0.0053601
## total-energy & matter 0.9876283
## information flow-evolution 0.5313568
## structure & function-evolution 0.6405493
## systems-evolution 0.0100406
## total-evolution 0.9973245
## structure & function-information flow 0.0158674
## systems-information flow 0.0000073
## total-information flow 0.2556705
## systems-structure & function 0.4409925
## total-structure & function 0.8919955
## total-systems 0.0428018
##
## $`timing:measure`
## diff lwr
## End:energy & matter-Begin:energy & matter 7.52568073 -1.4938712
## Begin:evolution-Begin:energy & matter 0.04832907 -8.9712229
## End:evolution-Begin:energy & matter 8.17755528 -0.8419967
## Begin:information flow-Begin:energy & matter -1.02806288 -10.0476149
## End:information flow-Begin:energy & matter 2.63597765 -6.3835743
## Begin:structure & function-Begin:energy & matter 3.26446630 -5.7550857
## End:structure & function-Begin:energy & matter 10.93936006 1.9198081
## Begin:systems-Begin:energy & matter 6.79266997 -2.2268820
## End:systems-Begin:energy & matter 14.56651295 5.5469610
## Begin:total-Begin:energy & matter 1.37013387 -7.6494181
## End:total-Begin:energy & matter 8.65917304 -0.3603789
## Begin:evolution-End:energy & matter -7.47735166 -16.4969036
## End:evolution-End:energy & matter 0.65187455 -8.3676774
## Begin:information flow-End:energy & matter -8.55374362 -17.5732956
## End:information flow-End:energy & matter -4.88970308 -13.9092551
## Begin:structure & function-End:energy & matter -4.26121443 -13.2807664
## End:structure & function-End:energy & matter 3.41367933 -5.6058726
## Begin:systems-End:energy & matter -0.73301076 -9.7525627
## End:systems-End:energy & matter 7.04083222 -1.9787198
## Begin:total-End:energy & matter -6.15554686 -15.1750988
## End:total-End:energy & matter 1.13349231 -7.8860597
## End:evolution-Begin:evolution 8.12922621 -0.8903258
## Begin:information flow-Begin:evolution -1.07639196 -10.0959439
## End:information flow-Begin:evolution 2.58764858 -6.4319034
## Begin:structure & function-Begin:evolution 3.21613722 -5.8034148
## End:structure & function-Begin:evolution 10.89103099 1.8714790
## Begin:systems-Begin:evolution 6.74434090 -2.2752111
## End:systems-Begin:evolution 14.51818388 5.4986319
## Begin:total-Begin:evolution 1.32180480 -7.6977472
## End:total-Begin:evolution 8.61084397 -0.4087080
## Begin:information flow-End:evolution -9.20561817 -18.2251701
## End:information flow-End:evolution -5.54157763 -14.5611296
## Begin:structure & function-End:evolution -4.91308899 -13.9326410
## End:structure & function-End:evolution 2.76180477 -6.2577472
## Begin:systems-End:evolution -1.38488532 -10.4044373
## End:systems-End:evolution 6.38895767 -2.6305943
## Begin:total-End:evolution -6.80742141 -15.8269734
## End:total-End:evolution 0.48161776 -8.5379342
## End:information flow-Begin:information flow 3.66404054 -5.3555114
## Begin:structure & function-Begin:information flow 4.29252918 -4.7270228
## End:structure & function-Begin:information flow 11.96742294 2.9478710
## Begin:systems-Begin:information flow 7.82073285 -1.1988191
## End:systems-Begin:information flow 15.59457584 6.5750239
## Begin:total-Begin:information flow 2.39819675 -6.6213552
## End:total-Begin:information flow 9.68723593 0.6676840
## Begin:structure & function-End:information flow 0.62848864 -8.3910633
## End:structure & function-End:information flow 8.30338241 -0.7161696
## Begin:systems-End:information flow 4.15669231 -4.8628597
## End:systems-End:information flow 11.93053530 2.9109833
## Begin:total-End:information flow -1.26584378 -10.2853958
## End:total-End:information flow 6.02319539 -2.9963566
## End:structure & function-Begin:structure & function 7.67489376 -1.3446582
## Begin:systems-Begin:structure & function 3.52820367 -5.4913483
## End:systems-Begin:structure & function 11.30204666 2.2824947
## Begin:total-Begin:structure & function -1.89433243 -10.9138844
## End:total-Begin:structure & function 5.39470675 -3.6248452
## Begin:systems-End:structure & function -4.14669009 -13.1662421
## End:systems-End:structure & function 3.62715290 -5.3923991
## Begin:total-End:structure & function -9.56922619 -18.5887782
## End:total-End:structure & function -2.28018701 -11.2997390
## End:systems-Begin:systems 7.77384299 -1.2457090
## Begin:total-Begin:systems -5.42253610 -14.4420881
## End:total-Begin:systems 1.86650308 -7.1530489
## Begin:total-End:systems -13.19637908 -22.2159311
## End:total-End:systems -5.90733991 -14.9268919
## End:total-Begin:total 7.28903917 -1.7305128
## upr p adj
## End:energy & matter-Begin:energy & matter 16.5452327 0.2108846
## Begin:evolution-Begin:energy & matter 9.0678810 1.0000000
## End:evolution-Begin:energy & matter 17.1971073 0.1187750
## Begin:information flow-Begin:energy & matter 7.9914891 0.9999999
## End:information flow-Begin:energy & matter 11.6555296 0.9984256
## Begin:structure & function-Begin:energy & matter 12.2840183 0.9898763
## End:structure & function-Begin:energy & matter 19.9589120 0.0043810
## Begin:systems-Begin:energy & matter 15.8122219 0.3610666
## End:systems-Begin:energy & matter 23.5860649 0.0000103
## Begin:total-Begin:energy & matter 10.3896858 0.9999976
## End:total-Begin:energy & matter 17.6787250 0.0736045
## Begin:evolution-End:energy & matter 1.5422003 0.2192819
## End:evolution-End:energy & matter 9.6714265 1.0000000
## Begin:information flow-End:energy & matter 0.4658084 0.0820446
## End:information flow-End:energy & matter 4.1298489 0.8298269
## Begin:structure & function-End:energy & matter 4.7583375 0.9261077
## End:structure & function-End:energy & matter 12.4332313 0.9854533
## Begin:systems-End:energy & matter 8.2865412 1.0000000
## End:systems-End:energy & matter 16.0603842 0.3049743
## Begin:total-End:energy & matter 2.8640051 0.5225249
## End:total-End:energy & matter 10.1530443 0.9999997
## End:evolution-Begin:evolution 17.1487782 0.1243070
## Begin:information flow-Begin:evolution 7.9431600 0.9999998
## End:information flow-Begin:evolution 11.6072006 0.9986710
## Begin:structure & function-Begin:evolution 12.2356892 0.9910500
## End:structure & function-Begin:evolution 19.9105830 0.0046916
## Begin:systems-Begin:evolution 15.7638929 0.3725372
## End:systems-Begin:evolution 23.5377359 0.0000112
## Begin:total-Begin:evolution 10.3413568 0.9999984
## End:total-Begin:evolution 17.6303959 0.0773801
## Begin:information flow-End:evolution -0.1860662 0.0405745
## End:information flow-End:evolution 3.4779743 0.6833732
## Begin:structure & function-End:evolution 4.1064630 0.8253123
## End:structure & function-End:evolution 11.7813567 0.9975997
## Begin:systems-End:evolution 7.6346667 0.9999973
## End:systems-End:evolution 15.4085096 0.4612931
## Begin:total-End:evolution 2.2121306 0.3575988
## End:total-End:evolution 9.5011697 1.0000000
## End:information flow-Begin:information flow 12.6835925 0.9747417
## Begin:structure & function-Begin:information flow 13.3120812 0.9224686
## End:structure & function-Begin:information flow 20.9869749 0.0009416
## Begin:systems-Begin:information flow 16.8402848 0.1643957
## End:systems-Begin:information flow 24.6141278 0.0000014
## Begin:total-Begin:information flow 11.4177487 0.9993455
## End:total-Begin:information flow 18.7067879 0.0229730
## Begin:structure & function-End:information flow 9.6480406 1.0000000
## End:structure & function-End:information flow 17.3229344 0.1052719
## Begin:systems-End:information flow 13.1762443 0.9374069
## End:systems-End:information flow 20.9500873 0.0009976
## Begin:total-End:information flow 7.7537082 0.9999990
## End:total-End:information flow 15.0427474 0.5576827
## End:structure & function-Begin:structure & function 16.6944457 0.1863530
## Begin:systems-Begin:structure & function 12.5477556 0.9811191
## End:systems-Begin:structure & function 20.3215986 0.0025916
## Begin:total-Begin:structure & function 7.1252196 0.9999342
## End:total-Begin:structure & function 14.4142587 0.7195887
## Begin:systems-End:structure & function 4.8728619 0.9384209
## End:systems-End:structure & function 12.6467049 0.9766166
## Begin:total-End:structure & function -0.5496742 0.0265062
## End:total-End:structure & function 6.7393650 0.9995948
## End:systems-Begin:systems 16.7933950 0.1712397
## Begin:total-Begin:systems 3.5970159 0.7128412
## End:total-Begin:systems 10.8860551 0.9999432
## Begin:total-End:systems -4.1768271 0.0001238
## End:total-End:systems 3.1122121 0.5884524
## End:total-Begin:total 16.3085911 0.2541020CONCLUSION
Mean scores improve in all concept areas in intro subjects; changes are variable and statistically significant in three out of five concept areas:
energy & matter (+6.1%, p=0.26)
evolution (+8.8%, p<0.01)
information flow (+3.9%, p=0.87)
structure & function (+9.5%, p<0.01)
systems (+7.8%, p=0.04)
… and each of the subjects individually.
dodge <- position_dodge(width = 1)
p8 <- ggplot(before_after_biol10008, aes(x = measure, y = score, fill = timing)) +
geom_violin(position = dodge) + scale_fill_brewer(palette = "PuBu") + theme_minimal() +
geom_boxplot(width = 0.3, position = dodge, col = "black") + geom_sina(position = dodge,
alpha = 0.3) + labs(title = "Change in Gen-BioMAPS total score over the semester, BIOL10008",
y = "Score", x = "Concept area", caption = "") + scale_x_discrete(labels = function(x) {
sub("\\s", "\n", x)
})
p8
# to run an anova and store the output in an object called
# '2wayanovaresults_biol10008'
twowayanovaresults_biol10008 = with(before_after_biol10008, aov(score ~ timing *
measure))
# to see summary output for the ANOVA
summary(twowayanovaresults_biol10008)## Df Sum Sq Mean Sq F value Pr(>F)
## timing 1 10908 10908 46.988 1.88e-11 ***
## measure 5 5097 1019 4.392 0.000624 ***
## timing:measure 5 640 128 0.552 0.737019
## Residuals 564 130924 232
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# to report the means and the number of subjects per cell
print(model.tables(twowayanovaresults_biol10008, "means"), digits = 3)## Tables of means
## Grand mean
##
## 69.15574
##
## timing
## timing
## Begin End
## 64.8 73.5
##
## measure
## measure
## energy & matter evolution information flow
## 68.6 67.8 64.3
## structure & function systems total
## 71.3 74.0 68.9
##
## timing:measure
## measure
## timing energy & matter evolution information flow structure & function systems
## Begin 64.0 63.0 61.6 67.9 67.9
## End 73.2 72.6 67.1 74.8 80.0
## measure
## timing total
## Begin 64.5
## End 73.4# to perform a post-hoc test to see which means are significantly different
# from each other
TukeyHSD(twowayanovaresults_biol10008)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = score ~ timing * measure)
##
## $timing
## diff lwr upr p adj
## End-Begin 8.703265 6.209417 11.19711 0
##
## $measure
## diff lwr upr
## evolution-energy & matter -0.8223380 -7.1108271 5.466151
## information flow-energy & matter -4.2664007 -10.5548898 2.022088
## structure & function-energy & matter 2.7134886 -3.5750005 9.001978
## systems-energy & matter 5.3429193 -0.9455698 11.631408
## total-energy & matter 0.3044983 -5.9839909 6.592987
## information flow-evolution -3.4440627 -9.7325518 2.844426
## structure & function-evolution 3.5358265 -2.7526626 9.824316
## systems-evolution 6.1652573 -0.1232318 12.453746
## total-evolution 1.1268362 -5.1616529 7.415325
## structure & function-information flow 6.9798893 0.6914002 13.268378
## systems-information flow 9.6093200 3.3208309 15.897809
## total-information flow 4.5708990 -1.7175902 10.859388
## systems-structure & function 2.6294307 -3.6590584 8.917920
## total-structure & function -2.4089903 -8.6974794 3.879499
## total-systems -5.0384210 -11.3269101 1.250068
## p adj
## evolution-energy & matter 0.9990467
## information flow-energy & matter 0.3787710
## structure & function-energy & matter 0.8200750
## systems-energy & matter 0.1478385
## total-energy & matter 0.9999929
## information flow-evolution 0.6213143
## structure & function-evolution 0.5936892
## systems-evolution 0.0583376
## total-evolution 0.9956964
## structure & function-information flow 0.0196531
## systems-information flow 0.0002140
## total-information flow 0.3000444
## systems-structure & function 0.8389410
## total-structure & function 0.8831839
## total-systems 0.1993356
##
## $`timing:measure`
## diff lwr
## End:energy & matter-Begin:energy & matter 9.165289858 -1.0414485
## Begin:evolution-Begin:energy & matter -1.051136324 -11.2578747
## End:evolution-Begin:energy & matter 8.571750255 -1.6349882
## Begin:information flow-Begin:energy & matter -2.418245208 -12.6249836
## End:information flow-Begin:energy & matter 3.050733670 -7.1560047
## Begin:structure & function-Begin:energy & matter 3.849075014 -6.3576634
## End:structure & function-Begin:energy & matter 10.743191981 0.5364536
## Begin:systems-Begin:energy & matter 3.853270676 -6.3534677
## End:systems-Begin:energy & matter 15.997857774 5.7911194
## Begin:total-Begin:energy & matter 0.425276964 -9.7814614
## End:total-Begin:energy & matter 9.349009403 -0.8577290
## Begin:evolution-End:energy & matter -10.216426182 -20.4231646
## End:evolution-End:energy & matter -0.593539603 -10.8002780
## Begin:information flow-End:energy & matter -11.583535066 -21.7902735
## End:information flow-End:energy & matter -6.114556188 -16.3212946
## Begin:structure & function-End:energy & matter -5.316214843 -15.5229532
## End:structure & function-End:energy & matter 1.577902123 -8.6288363
## Begin:systems-End:energy & matter -5.312019182 -15.5187576
## End:systems-End:energy & matter 6.832567916 -3.3741705
## Begin:total-End:energy & matter -8.740012893 -18.9467513
## End:total-End:energy & matter 0.183719546 -10.0230189
## End:evolution-Begin:evolution 9.622886579 -0.5838518
## Begin:information flow-Begin:evolution -1.367108884 -11.5738473
## End:information flow-Begin:evolution 4.101869994 -6.1048684
## Begin:structure & function-Begin:evolution 4.900211339 -5.3065271
## End:structure & function-Begin:evolution 11.794328305 1.5875899
## Begin:systems-Begin:evolution 4.904407000 -5.3023314
## End:systems-Begin:evolution 17.048994098 6.8422557
## Begin:total-Begin:evolution 1.476413289 -8.7303251
## End:total-Begin:evolution 10.400145727 0.1934073
## Begin:information flow-End:evolution -10.989995463 -21.1967339
## End:information flow-End:evolution -5.521016584 -15.7277550
## Begin:structure & function-End:evolution -4.722675240 -14.9294136
## End:structure & function-End:evolution 2.171441726 -8.0352967
## Begin:systems-End:evolution -4.718479579 -14.9252180
## End:systems-End:evolution 7.426107520 -2.7806309
## Begin:total-End:evolution -8.146473290 -18.3532117
## End:total-End:evolution 0.777259149 -9.4294793
## End:information flow-Begin:information flow 5.468978878 -4.7377595
## Begin:structure & function-Begin:information flow 6.267320222 -3.9394182
## End:structure & function-Begin:information flow 13.161437189 2.9546988
## Begin:systems-Begin:information flow 6.271515884 -3.9352225
## End:systems-Begin:information flow 18.416102982 8.2093646
## Begin:total-Begin:information flow 2.843522172 -7.3632162
## End:total-Begin:information flow 11.767254611 1.5605162
## Begin:structure & function-End:information flow 0.798341344 -9.4083971
## End:structure & function-End:information flow 7.692458311 -2.5142801
## Begin:systems-End:information flow 0.802537005 -9.4042014
## End:systems-End:information flow 12.947124104 2.7403857
## Begin:total-End:information flow -2.625456706 -12.8321951
## End:total-End:information flow 6.298275733 -3.9084627
## End:structure & function-Begin:structure & function 6.894116966 -3.3126214
## Begin:systems-Begin:structure & function 0.004195661 -10.2025427
## End:systems-Begin:structure & function 12.148782760 1.9420444
## Begin:total-Begin:structure & function -3.423798050 -13.6305365
## End:total-Begin:structure & function 5.499934389 -4.7068040
## Begin:systems-End:structure & function -6.889921305 -17.0966597
## End:systems-End:structure & function 5.254665793 -4.9520726
## Begin:total-End:structure & function -10.317915016 -20.5246534
## End:total-End:structure & function -1.394182578 -11.6009210
## End:systems-Begin:systems 12.144587099 1.9378487
## Begin:total-Begin:systems -3.427993711 -13.6347321
## End:total-Begin:systems 5.495738728 -4.7109997
## Begin:total-End:systems -15.572580810 -25.7793192
## End:total-End:systems -6.648848371 -16.8555868
## End:total-Begin:total 8.923732439 -1.2830060
## upr p adj
## End:energy & matter-Begin:energy & matter 19.372028264 0.1276013
## Begin:evolution-Begin:energy & matter 9.155602082 1.0000000
## End:evolution-Begin:energy & matter 18.778488661 0.2021037
## Begin:information flow-Begin:energy & matter 7.788493198 0.9997802
## End:information flow-Begin:energy & matter 13.257472077 0.9980432
## Begin:structure & function-Begin:energy & matter 14.055813421 0.9857153
## End:structure & function-Begin:energy & matter 20.949930387 0.0290574
## Begin:systems-Begin:energy & matter 14.060009082 0.9855903
## End:systems-Begin:energy & matter 26.204596181 0.0000239
## Begin:total-Begin:energy & matter 10.632015371 1.0000000
## End:total-Begin:energy & matter 19.555747810 0.1094382
## Begin:evolution-End:energy & matter -0.009687776 0.0495296
## End:evolution-End:energy & matter 9.613198803 1.0000000
## Begin:information flow-End:energy & matter -1.376796659 0.0115074
## End:information flow-End:energy & matter 4.092182219 0.7162933
## Begin:structure & function-End:energy & matter 4.890523563 0.8635450
## End:structure & function-End:energy & matter 11.784640529 0.9999971
## Begin:systems-End:energy & matter 4.894719224 0.8641714
## End:systems-End:energy & matter 17.039306323 0.5525505
## Begin:total-End:energy & matter 1.466725513 0.1784195
## End:total-End:energy & matter 10.390457952 1.0000000
## End:evolution-Begin:evolution 19.829624985 0.0862243
## Begin:information flow-Begin:evolution 8.839629523 0.9999994
## End:information flow-Begin:evolution 14.308608401 0.9765017
## Begin:structure & function-Begin:evolution 15.106949745 0.9173091
## End:structure & function-Begin:evolution 22.001066711 0.0089958
## Begin:systems-Begin:evolution 15.111145406 0.9168508
## End:systems-Begin:evolution 27.255732505 0.0000041
## Begin:total-Begin:evolution 11.683151695 0.9999986
## End:total-Begin:evolution 20.606884134 0.0413004
## Begin:information flow-End:evolution -0.783257056 0.0223456
## End:information flow-End:evolution 4.685721822 0.8309323
## Begin:structure & function-End:evolution 5.484063166 0.9351726
## End:structure & function-End:evolution 12.378180133 0.9999240
## Begin:systems-End:evolution 5.488258827 0.9355593
## End:systems-End:evolution 17.632845926 0.4160635
## Begin:total-End:evolution 2.060265116 0.2712344
## End:total-End:evolution 10.983997555 1.0000000
## End:information flow-Begin:information flow 15.675717284 0.8395914
## Begin:structure & function-Begin:information flow 16.474058629 0.6829473
## End:structure & function-Begin:information flow 23.368175595 0.0016054
## Begin:systems-Begin:information flow 16.478254290 0.6820150
## End:systems-Begin:information flow 28.622841388 0.0000004
## Begin:total-Begin:information flow 13.050260579 0.9989728
## End:total-Begin:information flow 21.973993018 0.0092876
## Begin:structure & function-End:information flow 11.005079751 1.0000000
## End:structure & function-End:information flow 17.899196717 0.3589483
## Begin:systems-End:information flow 11.009275412 1.0000000
## End:systems-End:information flow 23.153862510 0.0021330
## Begin:total-End:information flow 7.581281701 0.9995149
## End:total-End:information flow 16.505014139 0.6760504
## End:structure & function-Begin:structure & function 17.100855373 0.5381028
## Begin:systems-Begin:structure & function 10.210934068 1.0000000
## End:systems-Begin:structure & function 22.355521166 0.0058747
## Begin:total-Begin:structure & function 6.782940356 0.9945914
## End:total-Begin:structure & function 15.706672795 0.8344704
## Begin:systems-End:structure & function 3.316817101 0.5390872
## End:systems-End:structure & function 15.461404200 0.8725641
## Begin:total-End:structure & function -0.111176610 0.0448254
## End:total-End:structure & function 8.812555829 0.9999992
## End:systems-Begin:systems 22.351325505 0.0059049
## Begin:total-Begin:systems 6.778744695 0.9945344
## End:total-Begin:systems 15.702477134 0.8351697
## Begin:total-End:systems -5.365842403 0.0000472
## End:total-End:systems 3.557890036 0.5956249
## End:total-Begin:total 19.130470845 0.1549080CONCLUSION
Mean scores improve in all concept areas in BIOL10008; changes are significant in two out of five concept areas:
energy & matter (+8.0%, p=0.17)
evolution (+9.7%, p=0.03)
information flow (+5.4%, p=0.76)
structure & function (+8.2%, p=0.15)
systems (+11.8%, p=<0.01)
dodge <- position_dodge(width = 1)
p9 <- ggplot(before_after_biol10009, aes(x = measure, y = score, fill = timing)) +
geom_violin(position = dodge) + scale_fill_brewer(palette = "PuBu") + theme_minimal() +
geom_boxplot(width = 0.3, position = dodge, col = "black") + geom_sina(position = dodge,
alpha = 0.3) + labs(title = "Change in Gen-BioMAPS total score over the semester, BIOL10009",
y = "Score", x = "Concept area", caption = "") + scale_x_discrete(labels = function(x) {
sub("\\s", "\n", x)
})
p9
# to run an anova and store the output in an object called
# '2wayanovaresults_biol10009'
twowayanovaresults_biol10009 = with(before_after_biol10009, aov(score ~ timing *
measure))
# to see summary output for the ANOVA
summary(twowayanovaresults_biol10009)## Df Sum Sq Mean Sq F value Pr(>F)
## timing 1 5867 5867 29.627 6.45e-08 ***
## measure 5 8236 1647 8.318 9.92e-08 ***
## timing:measure 5 636 127 0.643 0.667
## Residuals 1104 218632 198
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# to report the means and the number of subjects per cell
print(model.tables(twowayanovaresults_biol10009, "means"), digits = 3)## Tables of means
## Grand mean
##
## 75.35202
##
## timing
## timing
## Begin End
## 73.1 77.6
##
## measure
## measure
## energy & matter evolution information flow
## 74.9 70.8 74.3
## structure & function systems total
## 79.7 77.3 75.2
##
## timing:measure
## measure
## timing energy & matter evolution information flow structure & function systems
## Begin 71.5 68.8 71.7 77.1 76.4
## End 78.2 72.8 76.9 82.2 78.1
## measure
## timing total
## Begin 72.8
## End 77.6# to perform a post-hoc test to see which means are significantly different
# from each other
TukeyHSD(twowayanovaresults_biol10009)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = score ~ timing * measure)
##
## $timing
## diff lwr upr p adj
## End-Begin 4.585787 2.932704 6.238871 1e-07
##
## $measure
## diff lwr upr
## evolution-energy & matter -4.0557260 -8.2214973 0.1100453
## information flow-energy & matter -0.5592374 -4.7250087 3.6065339
## structure & function-energy & matter 4.7928391 0.6270678 8.9586104
## systems-energy & matter 2.4235158 -1.7422555 6.5892871
## total-energy & matter 0.3378498 -3.8279215 4.5036211
## information flow-evolution 3.4964886 -0.6692827 7.6622599
## structure & function-evolution 8.8485652 4.6827939 13.0143365
## systems-evolution 6.4792419 2.3134705 10.6450132
## total-evolution 4.3935758 0.2278045 8.5593471
## structure & function-information flow 5.3520766 1.1863052 9.5178479
## systems-information flow 2.9827532 -1.1830181 7.1485246
## total-information flow 0.8970872 -3.2686841 5.0628585
## systems-structure & function -2.3693233 -6.5350946 1.7964480
## total-structure & function -4.4549893 -8.6207606 -0.2892180
## total-systems -2.0856660 -6.2514373 2.0801053
## p adj
## evolution-energy & matter 0.0615497
## information flow-energy & matter 0.9989295
## structure & function-energy & matter 0.0134373
## systems-energy & matter 0.5581838
## total-energy & matter 0.9999093
## information flow-evolution 0.1583981
## structure & function-evolution 0.0000000
## systems-evolution 0.0001438
## total-evolution 0.0318123
## structure & function-information flow 0.0034871
## systems-information flow 0.3180516
## total-information flow 0.9899811
## systems-structure & function 0.5829698
## total-structure & function 0.0280214
## total-systems 0.7091484
##
## $`timing:measure`
## diff lwr
## End:energy & matter-Begin:energy & matter 6.64034944 -0.1184361
## Begin:evolution-Begin:energy & matter -2.74120513 -9.4999907
## End:evolution-Begin:energy & matter 1.27010252 -5.4886831
## Begin:information flow-Begin:energy & matter 0.11828034 -6.6405052
## End:information flow-Begin:energy & matter 5.40359426 -1.3551913
## Begin:structure & function-Begin:energy & matter 5.59722445 -1.1615611
## End:structure & function-Begin:energy & matter 10.62880326 3.8700177
## Begin:systems-Begin:energy & matter 4.89740869 -1.8613769
## End:systems-Begin:energy & matter 6.58997240 -0.1688132
## Begin:total-Begin:energy & matter 1.23121916 -5.5275664
## End:total-Begin:energy & matter 6.08482991 -0.6739557
## Begin:evolution-End:energy & matter -9.38155457 -16.1403401
## End:evolution-End:energy & matter -5.37024692 -12.1290325
## Begin:information flow-End:energy & matter -6.52206910 -13.2808547
## End:information flow-End:energy & matter -1.23675518 -7.9955408
## Begin:structure & function-End:energy & matter -1.04312499 -7.8019106
## End:structure & function-End:energy & matter 3.98845382 -2.7703318
## Begin:systems-End:energy & matter -1.74294075 -8.5017263
## End:systems-End:energy & matter -0.05037704 -6.8091626
## Begin:total-End:energy & matter -5.40913029 -12.1679159
## End:total-End:energy & matter -0.55551954 -7.3143051
## End:evolution-Begin:evolution 4.01130765 -2.7474779
## Begin:information flow-Begin:evolution 2.85948547 -3.8993001
## End:information flow-Begin:evolution 8.14479939 1.3860138
## Begin:structure & function-Begin:evolution 8.33842958 1.5796440
## End:structure & function-Begin:evolution 13.37000840 6.6112228
## Begin:systems-Begin:evolution 7.63861382 0.8798282
## End:systems-Begin:evolution 9.33117753 2.5723920
## Begin:total-Begin:evolution 3.97242429 -2.7863613
## End:total-Begin:evolution 8.82603504 2.0672495
## Begin:information flow-End:evolution -1.15182218 -7.9106078
## End:information flow-End:evolution 4.13349174 -2.6252938
## Begin:structure & function-End:evolution 4.32712193 -2.4316636
## End:structure & function-End:evolution 9.35870074 2.5999152
## Begin:systems-End:evolution 3.62730617 -3.1314794
## End:systems-End:evolution 5.31986988 -1.4389157
## Begin:total-End:evolution -0.03888336 -6.7976689
## End:total-End:evolution 4.81472739 -1.9440582
## End:information flow-Begin:information flow 5.28531392 -1.4734717
## Begin:structure & function-Begin:information flow 5.47894411 -1.2798415
## End:structure & function-Begin:information flow 10.51052292 3.7517374
## Begin:systems-Begin:information flow 4.77912835 -1.9796572
## End:systems-Begin:information flow 6.47169206 -0.2870935
## Begin:total-Begin:information flow 1.11293882 -5.6458468
## End:total-Begin:information flow 5.96654957 -0.7922360
## Begin:structure & function-End:information flow 0.19363019 -6.5651554
## End:structure & function-End:information flow 5.22520900 -1.5335766
## Begin:systems-End:information flow -0.50618557 -7.2649711
## End:systems-End:information flow 1.18637814 -5.5724074
## Begin:total-End:information flow -4.17237510 -10.9311607
## End:total-End:information flow 0.68123565 -6.0775499
## End:structure & function-Begin:structure & function 5.03157881 -1.7272068
## Begin:systems-Begin:structure & function -0.69981576 -7.4586013
## End:systems-Begin:structure & function 0.99274795 -5.7660376
## Begin:total-Begin:structure & function -4.36600529 -11.1247909
## End:total-Begin:structure & function 0.48760546 -6.2711801
## Begin:systems-End:structure & function -5.73139457 -12.4901801
## End:systems-End:structure & function -4.03883086 -10.7976164
## Begin:total-End:structure & function -9.39758411 -16.1563697
## End:total-End:structure & function -4.54397336 -11.3027589
## End:systems-Begin:systems 1.69256371 -5.0662219
## Begin:total-Begin:systems -3.66618953 -10.4249751
## End:total-Begin:systems 1.18742122 -5.5713644
## Begin:total-End:systems -5.35875324 -12.1175388
## End:total-End:systems -0.50514249 -7.2639281
## End:total-Begin:total 4.85361075 -1.9051748
## upr p adj
## End:energy & matter-Begin:energy & matter 13.3991350 0.0594496
## Begin:evolution-Begin:energy & matter 4.0175804 0.9753934
## End:evolution-Begin:energy & matter 8.0288881 0.9999791
## Begin:information flow-Begin:energy & matter 6.8770659 1.0000000
## End:information flow-Begin:energy & matter 12.1623798 0.2704625
## Begin:structure & function-Begin:energy & matter 12.3560100 0.2214398
## End:structure & function-Begin:energy & matter 17.3875888 0.0000198
## Begin:systems-Begin:energy & matter 11.6561943 0.4253481
## End:systems-Begin:energy & matter 13.3487580 0.0639024
## Begin:total-Begin:energy & matter 7.9900047 0.9999848
## End:total-Begin:energy & matter 12.8436155 0.1256898
## Begin:evolution-End:energy & matter -2.6227690 0.0003765
## End:evolution-End:energy & matter 1.3885387 0.2795281
## Begin:information flow-End:energy & matter 0.2367165 0.0703424
## End:information flow-End:energy & matter 5.5220304 0.9999841
## Begin:structure & function-End:energy & matter 5.7156606 0.9999973
## End:structure & function-End:energy & matter 10.7472394 0.7384616
## Begin:systems-End:energy & matter 5.0158448 0.9995200
## End:systems-End:energy & matter 6.7084085 1.0000000
## Begin:total-End:energy & matter 1.3496553 0.2689749
## End:total-End:energy & matter 6.2032660 1.0000000
## End:evolution-Begin:evolution 10.7700932 0.7312497
## Begin:information flow-Begin:evolution 9.6182710 0.9662313
## End:information flow-Begin:evolution 14.9035850 0.0047843
## Begin:structure & function-Begin:evolution 15.0972152 0.0032991
## End:structure & function-Begin:evolution 20.1287940 0.0000000
## Begin:systems-Begin:evolution 14.3973994 0.0120371
## End:systems-Begin:evolution 16.0899631 0.0004207
## Begin:total-Begin:evolution 10.7312099 0.7434735
## End:total-Begin:evolution 15.5848206 0.0012382
## Begin:information flow-End:evolution 5.6069634 0.9999923
## End:information flow-End:evolution 10.8922773 0.6914891
## Begin:structure & function-End:evolution 11.0859075 0.6252452
## End:structure & function-End:evolution 16.1174863 0.0003960
## Begin:systems-End:evolution 10.3860917 0.8404222
## End:systems-End:evolution 12.0786555 0.2935593
## Begin:total-End:evolution 6.7199022 1.0000000
## End:total-End:evolution 11.5735130 0.4535124
## End:information flow-Begin:information flow 12.0440995 0.3034134
## Begin:structure & function-Begin:information flow 12.2377297 0.2506450
## End:structure & function-Begin:information flow 17.2693085 0.0000266
## Begin:systems-Begin:information flow 11.5379139 0.4658044
## End:systems-Begin:information flow 13.2304776 0.0754604
## Begin:total-Begin:information flow 7.8717244 0.9999946
## End:total-Begin:information flow 12.7253351 0.1453642
## Begin:structure & function-End:information flow 6.9524158 1.0000000
## End:structure & function-End:information flow 11.9839946 0.3209853
## Begin:systems-End:information flow 6.2526000 1.0000000
## End:systems-End:information flow 7.9451637 0.9999896
## Begin:total-End:information flow 2.5864105 0.6784603
## End:total-End:information flow 7.4400212 1.0000000
## End:structure & function-Begin:structure & function 11.7903644 0.3810517
## Begin:systems-Begin:structure & function 6.0589698 1.0000000
## End:systems-Begin:structure & function 7.7515335 0.9999984
## Begin:total-Begin:structure & function 2.3927803 0.6116147
## End:total-Begin:structure & function 7.2463910 1.0000000
## Begin:systems-End:structure & function 1.0273910 0.1911668
## End:systems-End:structure & function 2.7199547 0.7224647
## Begin:total-End:structure & function -2.6387985 0.0003634
## End:total-End:structure & function 2.2148122 0.5485942
## End:systems-Begin:systems 8.4513493 0.9996375
## Begin:total-Begin:systems 3.0925960 0.8306598
## End:total-Begin:systems 7.9462068 0.9999895
## Begin:total-End:systems 1.4000323 0.2826940
## End:total-End:systems 6.2536431 1.0000000
## End:total-Begin:total 11.6123963 0.4401955Mean scores improve in all concept areas in BIOL10009; changes are not significant in any concept area.
energy & matter (+5.4%, p=0.06)
evolution (+4.2%, p=0.33)
information flow (+2.5%, p=0.93)
structure & function (+3.8%, p=0.49)
systems (+1.5%, p=0.99)
dodge <- position_dodge(width = 1)
p10 <- ggplot(before_after_biol10010, aes(x = measure, y = score, fill = timing)) +
geom_violin(position = dodge) + scale_fill_brewer(palette = "PuBu") + theme_minimal() +
geom_boxplot(width = 0.3, position = dodge, col = "black") + geom_sina(position = dodge,
alpha = 0.3) + labs(title = "Change in Gen-BioMAPS total score over the semester, BIOL10010",
y = "Score", x = "Concept area", caption = "") + scale_x_discrete(labels = function(x) {
sub("\\s", "\n", x)
})
p10
# to run an anova and store the output in an object called
# '2wayanovaresults_biol10010'
twowayanovaresults_biol10010 = with(before_after_biol10010, aov(score ~ timing *
measure))
# to see summary output for the ANOVA
summary(twowayanovaresults_biol10010)## Df Sum Sq Mean Sq F value Pr(>F)
## timing 1 2 2.0 0.011 0.9185
## measure 5 2406 481.3 2.562 0.0301 *
## timing:measure 5 1376 275.2 1.465 0.2055
## Residuals 132 24795 187.8
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# to report the means and the number of subjects per cell
print(model.tables(twowayanovaresults_biol10010, "means"), digits = 3)## Tables of means
## Grand mean
##
## 66.9453
##
## timing
## timing
## Begin End
## 66.8 67.1
##
## measure
## measure
## energy & matter evolution information flow
## 61.7 66.8 64.0
## structure & function systems total
## 67.6 74.9 66.8
##
## timing:measure
## measure
## timing energy & matter evolution information flow structure & function systems
## Begin 61.2 65.7 65.8 62.2 79.8
## End 62.2 67.8 62.2 73.0 70.1
## measure
## timing total
## Begin 66.4
## End 67.1# to perform a post-hoc test to see which means are significantly different
# from each other
TukeyHSD(twowayanovaresults_biol10010)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = score ~ timing * measure)
##
## $timing
## diff lwr upr p adj
## End-Begin 0.2342083 -4.284242 4.752659 0.9184898
##
## $measure
## diff lwr upr
## evolution-energy & matter 5.039860918 -6.4022052 16.481927
## information flow-energy & matter 2.271187885 -9.1708782 13.713254
## structure & function-energy & matter 5.841409783 -5.6006563 17.283476
## systems-energy & matter 13.212078295 1.7700122 24.654144
## total-energy & matter 5.041072438 -6.4009937 16.483139
## information flow-evolution -2.768673034 -14.2107392 8.673393
## structure & function-evolution 0.801548864 -10.6405173 12.243615
## systems-evolution 8.172217376 -3.2698487 19.614283
## total-evolution 0.001211519 -11.4408546 11.443278
## structure & function-information flow 3.570221898 -7.8718442 15.012288
## systems-information flow 10.940890410 -0.5011757 22.382957
## total-information flow 2.769884553 -8.6721816 14.211951
## systems-structure & function 7.370668512 -4.0713976 18.812735
## total-structure & function -0.800337345 -12.2424035 10.641729
## total-systems -8.171005857 -19.6130720 3.271060
## p adj
## evolution-energy & matter 0.7987118
## information flow-energy & matter 0.9925569
## structure & function-energy & matter 0.6798245
## systems-energy & matter 0.0136424
## total-energy & matter 0.7985489
## information flow-evolution 0.9816742
## structure & function-evolution 0.9999520
## systems-evolution 0.3118088
## total-evolution 1.0000000
## structure & function-information flow 0.9452799
## systems-information flow 0.0696222
## total-information flow 0.9816385
## systems-structure & function 0.4295731
## total-structure & function 0.9999524
## total-systems 0.3119724
##
## $`timing:measure`
## diff lwr
## End:energy & matter-Begin:energy & matter 0.96724422 -17.65130479
## Begin:evolution-Begin:energy & matter 4.44619066 -14.17235835
## End:evolution-Begin:energy & matter 6.60077540 -12.01777361
## Begin:information flow-Begin:energy & matter 4.53266641 -14.08588260
## End:information flow-Begin:energy & matter 0.97695358 -17.64159543
## Begin:structure & function-Begin:energy & matter 0.92603142 -17.69251759
## End:structure & function-Begin:energy & matter 11.72403237 -6.89451664
## Begin:systems-Begin:energy & matter 18.55026714 -0.06828187
## End:systems-Begin:energy & matter 8.84113368 -9.77741533
## Begin:total-Begin:energy & matter 5.14956149 -13.46898752
## End:total-Begin:energy & matter 5.89982761 -12.71872140
## Begin:evolution-End:energy & matter 3.47894643 -15.13960257
## End:evolution-End:energy & matter 5.63353118 -12.98501783
## Begin:information flow-End:energy & matter 3.56542219 -15.05312682
## End:information flow-End:energy & matter 0.00970936 -18.60883965
## Begin:structure & function-End:energy & matter -0.04121280 -18.65976181
## End:structure & function-End:energy & matter 10.75678814 -7.86176086
## Begin:systems-End:energy & matter 17.58302291 -1.03552610
## End:systems-End:energy & matter 7.87388945 -10.74465955
## Begin:total-End:energy & matter 4.18231727 -14.43623174
## End:total-End:energy & matter 4.93258338 -13.68596563
## End:evolution-Begin:evolution 2.15458475 -16.46396426
## Begin:information flow-Begin:evolution 0.08647575 -18.53207326
## End:information flow-Begin:evolution -3.46923707 -22.08778608
## Begin:structure & function-Begin:evolution -3.52015924 -22.13870824
## End:structure & function-Begin:evolution 7.27784171 -11.34070730
## Begin:systems-Begin:evolution 14.10407648 -4.51447253
## End:systems-Begin:evolution 4.39494302 -14.22360599
## Begin:total-Begin:evolution 0.70337084 -17.91517817
## End:total-Begin:evolution 1.45363695 -17.16491206
## Begin:information flow-End:evolution -2.06810899 -20.68665800
## End:information flow-End:evolution -5.62382182 -24.24237083
## Begin:structure & function-End:evolution -5.67474398 -24.29329299
## End:structure & function-End:evolution 5.12325696 -13.49529204
## Begin:systems-End:evolution 11.94949173 -6.66905728
## End:systems-End:evolution 2.24035827 -16.37819073
## Begin:total-End:evolution -1.45121391 -20.06976292
## End:total-End:evolution -0.70094780 -19.31949681
## End:information flow-Begin:information flow -3.55571283 -22.17426184
## Begin:structure & function-Begin:information flow -3.60663499 -22.22518400
## End:structure & function-Begin:information flow 7.19136596 -11.42718305
## Begin:systems-Begin:information flow 14.01760073 -4.60094828
## End:systems-Begin:information flow 4.30846727 -14.31008174
## Begin:total-Begin:information flow 0.61689508 -18.00165393
## End:total-Begin:information flow 1.36716120 -17.25138781
## Begin:structure & function-End:information flow -0.05092216 -18.66947117
## End:structure & function-End:information flow 10.74707878 -7.87147022
## Begin:systems-End:information flow 17.57331355 -1.04523546
## End:systems-End:information flow 7.86418009 -10.75436891
## Begin:total-End:information flow 4.17260791 -14.44594110
## End:total-End:information flow 4.92287402 -13.69567499
## End:structure & function-Begin:structure & function 10.79800095 -7.82054806
## Begin:systems-Begin:structure & function 17.62423571 -0.99431329
## End:systems-Begin:structure & function 7.91510226 -10.70344675
## Begin:total-Begin:structure & function 4.22353007 -14.39501894
## End:total-Begin:structure & function 4.97379619 -13.64475282
## Begin:systems-End:structure & function 6.82623477 -11.79231424
## End:systems-End:structure & function -2.88289869 -21.50144770
## Begin:total-End:structure & function -6.57447087 -25.19301988
## End:total-End:structure & function -5.82420476 -24.44275377
## End:systems-Begin:systems -9.70913346 -28.32768247
## Begin:total-Begin:systems -13.40070564 -32.01925465
## End:total-Begin:systems -12.65043953 -31.26898854
## Begin:total-End:systems -3.69157218 -22.31012119
## End:total-End:systems -2.94130607 -21.55985508
## End:total-Begin:total 0.75026611 -17.86828289
## upr p adj
## End:energy & matter-Begin:energy & matter 19.585793 1.0000000
## Begin:evolution-Begin:energy & matter 23.064740 0.9996998
## End:evolution-Begin:energy & matter 25.219324 0.9896709
## Begin:information flow-Begin:energy & matter 23.151215 0.9996390
## End:information flow-Begin:energy & matter 19.595503 1.0000000
## Begin:structure & function-Begin:energy & matter 19.544580 1.0000000
## End:structure & function-Begin:energy & matter 30.342581 0.6269333
## Begin:systems-Begin:energy & matter 37.168816 0.0517818
## End:systems-Begin:energy & matter 27.459683 0.9132456
## Begin:total-Begin:energy & matter 23.768111 0.9988057
## End:total-Begin:energy & matter 24.518377 0.9959790
## Begin:evolution-End:energy & matter 22.097495 0.9999735
## End:evolution-End:energy & matter 24.252080 0.9973161
## Begin:information flow-End:energy & matter 22.183971 0.9999660
## End:information flow-End:energy & matter 18.628258 1.0000000
## Begin:structure & function-End:energy & matter 18.577336 1.0000000
## End:structure & function-End:energy & matter 29.375337 0.7430921
## Begin:systems-End:energy & matter 36.201572 0.0834563
## End:systems-End:energy & matter 26.492438 0.9602003
## Begin:total-End:energy & matter 22.800866 0.9998341
## End:total-End:energy & matter 23.551132 0.9991978
## End:evolution-Begin:evolution 20.773134 0.9999998
## Begin:information flow-Begin:evolution 18.705025 1.0000000
## End:information flow-Begin:evolution 15.149312 0.9999742
## Begin:structure & function-Begin:evolution 15.098390 0.9999701
## End:structure & function-Begin:evolution 25.896391 0.9777125
## Begin:systems-Begin:evolution 32.722625 0.3366639
## End:systems-Begin:evolution 23.013492 0.9997316
## Begin:total-Begin:evolution 19.321920 1.0000000
## End:total-Begin:evolution 20.072186 1.0000000
## Begin:information flow-End:evolution 16.550440 0.9999999
## End:information flow-End:evolution 12.994727 0.9973567
## Begin:structure & function-End:evolution 12.943805 0.9971376
## End:structure & function-End:evolution 23.741806 0.9988605
## Begin:systems-End:evolution 30.568041 0.5984694
## End:systems-End:evolution 20.858907 0.9999997
## Begin:total-End:evolution 17.167335 1.0000000
## End:total-End:evolution 17.917601 1.0000000
## End:information flow-Begin:information flow 15.062836 0.9999669
## Begin:structure & function-Begin:information flow 15.011914 0.9999619
## End:structure & function-Begin:information flow 25.809915 0.9796573
## Begin:systems-Begin:information flow 32.636150 0.3460821
## End:systems-Begin:information flow 22.927016 0.9997785
## Begin:total-Begin:information flow 19.235444 1.0000000
## End:total-Begin:information flow 19.985710 1.0000000
## Begin:structure & function-End:information flow 18.567627 1.0000000
## End:structure & function-End:information flow 29.365628 0.7441844
## Begin:systems-End:information flow 36.191863 0.0838418
## End:systems-End:information flow 26.482729 0.9605486
## Begin:total-End:information flow 22.791157 0.9998378
## End:total-End:information flow 23.541423 0.9992124
## End:structure & function-Begin:structure & function 29.416550 0.7384352
## Begin:systems-Begin:structure & function 36.242785 0.0818360
## End:systems-Begin:structure & function 26.533651 0.9586969
## Begin:total-Begin:structure & function 22.842079 0.9998175
## End:total-Begin:structure & function 23.592345 0.9991333
## Begin:systems-End:structure & function 25.444784 0.9864648
## End:systems-End:structure & function 15.735650 0.9999962
## Begin:total-End:structure & function 12.044078 0.9900014
## End:total-End:structure & function 12.794344 0.9964051
## End:systems-Begin:systems 8.909416 0.8484958
## Begin:total-Begin:systems 5.217843 0.4168620
## End:total-Begin:systems 5.968109 0.5093812
## Begin:total-End:systems 14.926977 0.9999518
## End:total-End:systems 15.677243 0.9999953
## End:total-Begin:total 19.368815 1.0000000Mean scores improve in four concept areas and decline in one in BIOL10010. Large and signficant improvement in structure & function only.
energy & matter (+4.7%, p=0.99)
evolution (+7.8%, p=0.82)
information flow (+6.0%, p=0.97)
structure & function (+14.5%, p=0.04)
systems (-3.1%, p=0.99)
dodge <- position_dodge(width = 1)
p11 <- ggplot(before_after_biol10011, aes(x = measure, y = score, fill = timing)) +
geom_violin(position = dodge) + scale_fill_brewer(palette = "PuBu") + theme_minimal() +
geom_boxplot(width = 0.3, position = dodge, col = "black") + geom_sina(position = dodge,
alpha = 0.3) + labs(title = "Change in Gen-BioMAPS total score over the semester, BIOL10011",
y = "Score", x = "Concept area", caption = "") + scale_x_discrete(labels = function(x) {
sub("\\s", "\n", x)
})
p11
# to run an anova and store the output in an object called
# '2wayanovaresults_biol10011'
twowayanovaresults_biol10011 = with(before_after_biol10011, aov(score ~ timing *
measure))
# to see summary output for the ANOVA
summary(twowayanovaresults_biol10011)## Df Sum Sq Mean Sq F value Pr(>F)
## timing 1 591 591.2 3.001 0.0843 .
## measure 5 1615 323.1 1.640 0.1495
## timing:measure 5 922 184.3 0.936 0.4582
## Residuals 276 54369 197.0
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# to report the means and the number of subjects per cell
print(model.tables(twowayanovaresults_biol10011, "means"), digits = 3)## Tables of means
## Grand mean
##
## 76.45965
##
## timing
## timing
## Begin End
## 75.0 77.9
##
## measure
## measure
## energy & matter evolution information flow
## 74.6 74.8 74.4
## structure & function systems total
## 77.5 81.1 76.4
##
## timing:measure
## measure
## timing energy & matter evolution information flow structure & function systems
## Begin 75.7 73.2 74.1 75.4 76.4
## End 73.5 76.3 74.7 79.6 85.8
## measure
## timing total
## Begin 75.3
## End 77.4# to perform a post-hoc test to see which means are significantly different
# from each other
TukeyHSD(twowayanovaresults_biol10011)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = score ~ timing * measure)
##
## $timing
## diff lwr upr p adj
## End-Begin 2.865505 -0.3906983 6.121708 0.0843208
##
## $measure
## diff lwr upr p adj
## evolution-energy & matter 0.1710901 -8.050866 8.393047 0.9999999
## information flow-energy & matter -0.2107204 -8.432677 8.011236 0.9999997
## structure & function-energy & matter 2.9496442 -5.272312 11.171601 0.9076918
## systems-energy & matter 6.5353035 -1.686653 14.757260 0.2053186
## total-energy & matter 1.7806739 -6.441283 10.002630 0.9893856
## information flow-evolution -0.3818105 -8.603767 7.840146 0.9999941
## structure & function-evolution 2.7785541 -5.443402 11.000511 0.9271146
## systems-evolution 6.3642134 -1.857743 14.586170 0.2312636
## total-evolution 1.6095837 -6.612373 9.831540 0.9933404
## structure & function-information flow 3.1603646 -5.061592 11.382321 0.8798307
## systems-information flow 6.7460239 -1.475933 14.967980 0.1762025
## total-information flow 1.9913943 -6.230562 10.213351 0.9823905
## systems-structure & function 3.5856593 -4.636297 11.807616 0.8108094
## total-structure & function -1.1689703 -9.390927 7.052986 0.9985383
## total-systems -4.7546296 -12.976586 3.467327 0.5597427
##
## $`timing:measure`
## diff lwr
## End:energy & matter-Begin:energy & matter -2.25361465 -15.6094310
## Begin:evolution-Begin:energy & matter -2.51421996 -15.8700363
## End:evolution-Begin:energy & matter 0.60278555 -12.7530308
## Begin:information flow-Begin:energy & matter -1.65773462 -15.0135509
## End:information flow-Begin:energy & matter -1.01732085 -14.3731372
## Begin:structure & function-Begin:energy & matter -0.28857670 -13.6443930
## End:structure & function-Begin:energy & matter 3.93425047 -9.4215659
## Begin:systems-Begin:energy & matter 0.69485096 -12.6609654
## End:systems-Begin:energy & matter 10.12214140 -3.2336749
## Begin:total-Begin:energy & matter -0.36568654 -13.7215029
## End:total-Begin:energy & matter 1.67341962 -11.6823967
## Begin:evolution-End:energy & matter -0.26060531 -13.6164216
## End:evolution-End:energy & matter 2.85640020 -10.4994161
## Begin:information flow-End:energy & matter 0.59588003 -12.7599363
## End:information flow-End:energy & matter 1.23629381 -12.1195225
## Begin:structure & function-End:energy & matter 1.96503796 -11.3907784
## End:structure & function-End:energy & matter 6.18786512 -7.1679512
## Begin:systems-End:energy & matter 2.94846561 -10.4073507
## End:systems-End:energy & matter 12.37575605 -0.9800603
## Begin:total-End:energy & matter 1.88792811 -11.4678882
## End:total-End:energy & matter 3.92703428 -9.4287820
## End:evolution-Begin:evolution 3.11700551 -10.2388108
## Begin:information flow-Begin:evolution 0.85648534 -12.4993310
## End:information flow-Begin:evolution 1.49689912 -11.8589172
## Begin:structure & function-Begin:evolution 2.22564327 -11.1301731
## End:structure & function-Begin:evolution 6.44847043 -6.9073459
## Begin:systems-Begin:evolution 3.20907092 -10.1467454
## End:systems-Begin:evolution 12.63636136 -0.7194550
## Begin:total-Begin:evolution 2.14853342 -11.2072829
## End:total-Begin:evolution 4.18763959 -9.1681767
## Begin:information flow-End:evolution -2.26052017 -15.6163365
## End:information flow-End:evolution -1.62010639 -14.9759227
## Begin:structure & function-End:evolution -0.89136224 -14.2471786
## End:structure & function-End:evolution 3.33146492 -10.0243514
## Begin:systems-End:evolution 0.09206541 -13.2637509
## End:systems-End:evolution 9.51935585 -3.8364605
## Begin:total-End:evolution -0.96847209 -14.3242884
## End:total-End:evolution 1.07063408 -12.2851823
## End:information flow-Begin:information flow 0.64041378 -12.7154026
## Begin:structure & function-Begin:information flow 1.36915793 -11.9866584
## End:structure & function-Begin:information flow 5.59198509 -7.7638312
## Begin:systems-Begin:information flow 2.35258558 -11.0032307
## End:systems-Begin:information flow 11.77987602 -1.5759403
## Begin:total-Begin:information flow 1.29204808 -12.0637682
## End:total-Begin:information flow 3.33115425 -10.0246621
## Begin:structure & function-End:information flow 0.72874415 -12.6270722
## End:structure & function-End:information flow 4.95157132 -8.4042450
## Begin:systems-End:information flow 1.71217181 -11.6436445
## End:systems-End:information flow 11.13946225 -2.2163541
## Begin:total-End:information flow 0.65163430 -12.7041820
## End:total-End:information flow 2.69074047 -10.6650759
## End:structure & function-Begin:structure & function 4.22282716 -9.1329892
## Begin:systems-Begin:structure & function 0.98342766 -12.3723887
## End:systems-Begin:structure & function 10.41071809 -2.9450982
## Begin:total-Begin:structure & function -0.07710985 -13.4329262
## End:total-Begin:structure & function 1.96199632 -11.3938200
## Begin:systems-End:structure & function -3.23939951 -16.5952158
## End:systems-End:structure & function 6.18789093 -7.1679254
## Begin:total-End:structure & function -4.29993701 -17.6557533
## End:total-End:structure & function -2.26083084 -15.6166472
## End:systems-Begin:systems 9.42729044 -3.9285259
## Begin:total-Begin:systems -1.06053750 -14.4163538
## End:total-Begin:systems 0.97856866 -12.3772477
## Begin:total-End:systems -10.48782794 -23.8436443
## End:total-End:systems -8.44872178 -21.8045381
## End:total-Begin:total 2.03910617 -11.3167102
## upr p adj
## End:energy & matter-Begin:energy & matter 11.102202 0.9999922
## Begin:evolution-Begin:energy & matter 10.841596 0.9999760
## End:evolution-Begin:energy & matter 13.958602 1.0000000
## Begin:information flow-Begin:energy & matter 11.698082 0.9999997
## End:information flow-Begin:energy & matter 12.338495 1.0000000
## Begin:structure & function-Begin:energy & matter 13.067240 1.0000000
## End:structure & function-Begin:energy & matter 17.290067 0.9981652
## Begin:systems-Begin:energy & matter 14.050667 1.0000000
## End:systems-Begin:energy & matter 23.477958 0.3454693
## Begin:total-Begin:energy & matter 12.990130 1.0000000
## End:total-Begin:energy & matter 15.029236 0.9999997
## Begin:evolution-End:energy & matter 13.095211 1.0000000
## End:evolution-End:energy & matter 16.212217 0.9999133
## Begin:information flow-End:energy & matter 13.951696 1.0000000
## End:information flow-End:energy & matter 14.592110 1.0000000
## Begin:structure & function-End:energy & matter 15.320854 0.9999981
## End:structure & function-End:energy & matter 19.543681 0.9318863
## Begin:systems-End:energy & matter 16.304282 0.9998813
## End:systems-End:energy & matter 25.731572 0.0989518
## Begin:total-End:energy & matter 15.243744 0.9999988
## End:total-End:energy & matter 17.282851 0.9981954
## End:evolution-Begin:evolution 16.472822 0.9997952
## Begin:information flow-Begin:evolution 14.212302 1.0000000
## End:information flow-Begin:evolution 14.852715 0.9999999
## Begin:structure & function-Begin:evolution 15.581460 0.9999931
## End:structure & function-Begin:evolution 19.804287 0.9109526
## Begin:systems-Begin:evolution 16.564887 0.9997284
## End:systems-Begin:evolution 25.992178 0.0831618
## Begin:total-Begin:evolution 15.504350 0.9999952
## End:total-Begin:evolution 17.543456 0.9967966
## Begin:information flow-End:evolution 11.095296 0.9999919
## End:information flow-End:evolution 11.735710 0.9999998
## Begin:structure & function-End:evolution 12.464454 1.0000000
## End:structure & function-End:evolution 16.687281 0.9996106
## Begin:systems-End:evolution 13.447882 1.0000000
## End:systems-End:evolution 22.875172 0.4442806
## Begin:total-End:evolution 12.387344 1.0000000
## End:total-End:evolution 14.426450 1.0000000
## End:information flow-Begin:information flow 13.996230 1.0000000
## Begin:structure & function-Begin:information flow 14.724974 1.0000000
## End:structure & function-Begin:information flow 18.947801 0.9664531
## Begin:systems-Begin:information flow 15.708402 0.9999878
## End:systems-Begin:information flow 25.135692 0.1440751
## Begin:total-Begin:information flow 14.647864 1.0000000
## End:total-Begin:information flow 16.686971 0.9996110
## Begin:structure & function-End:information flow 14.084560 1.0000000
## End:structure & function-End:information flow 18.307388 0.9868328
## Begin:systems-End:information flow 15.067988 0.9999996
## End:systems-End:information flow 24.495279 0.2082812
## Begin:total-End:information flow 14.007451 1.0000000
## End:total-End:information flow 16.046557 0.9999523
## End:structure & function-Begin:structure & function 17.578643 0.9965521
## Begin:systems-Begin:structure & function 14.339244 1.0000000
## End:systems-Begin:structure & function 23.766534 0.3023508
## Begin:total-Begin:structure & function 13.278706 1.0000000
## End:total-Begin:structure & function 15.317813 0.9999981
## Begin:systems-End:structure & function 10.116417 0.9997025
## End:systems-End:structure & function 19.543707 0.9318844
## Begin:total-End:structure & function 9.055879 0.9959617
## End:total-End:structure & function 11.094985 0.9999919
## End:systems-Begin:systems 22.783107 0.4601841
## Begin:total-Begin:systems 12.295279 1.0000000
## End:total-Begin:systems 14.334385 1.0000000
## Begin:total-End:systems 2.867988 0.2913647
## End:total-End:systems 4.907095 0.6337333
## End:total-Begin:total 15.394922 0.9999972Mean scores improve in all five concept areas in BIOL10011, but not significantly in any area.
energy & matter (+3.6%, p=0.98)
evolution (+2.8%, p=0.99)
information flow (+1.3%, p=0.99)
structure & function (+6.0%, p=0.60)
systems (+4.9%, p=0.85)