R-Exams in Blackboard Ultra
Andrew Dalby
2021/07/08
1 Introduction
This is a guide to using R-exams to build online assessments that can be embedded in Blackboard Ultra. The problem with Ultra is that it does not support complex formats for tables and html and it also does not support numerical answers and so all of the weekly assessments have to be moved from entering numerical values to multiple choice.
I write R-exams files in R markdown which makes this tutorial a bit meta as it is written in R markdown!
2 The Metadata
At the top of the file is some meta data which is marked out by leading and trailing three dashes. This is where you name the file and say something about who wrote it and when as well as define the output format.
---
title: "Fingernails Conditional Probability
author: "Andrew Dalby"
date: "2020/10/02"
output: html_document
---This is the metadata for a file I created for calculating the conditional probability for whether you check your fingernails by folding them or holding your hand flat conditional upon gender. It should be informative as you may want to come back to edit the code later.
NB I used this particular date format as it is best practice especially if you use dates in filenames so that they are listed in date order on windows machines.
3 The Setup Block
This is the part of the markdown script that cleans up the R environment to get rid of any existing data and then imports all of the libraries that are going to be used in the code.
# Clear the environment
rm(list=ls())
# Import the libraries that are used in the code.
# haven allows you to write spss files
# knitr is needed to knit the markdown files
# tinytex is needed to incorporate LaTex such as special characters and equations
# kableExtra allows the fancy formatting of tables
library(haven)
library(knitr)
library(here)## here() starts at /Users/ardalby/Dropbox/R notebookslibrary(exams)
library(tinytex)
library(kableExtra)If instead of {r setup } you have {r setup, echo=FALSE, results=“hide”}. It would not show the code or its output in the resulting html file produced when you knit the markdown document.
4 The Data Generation Block
Next you want to create the random set of data for the question. This data will be generated from assuming the data can be sampled from a distribution based on real experimental data. Most often the data will be assumed to be sampled from a normal distribution where the mean and standard deviation are calculated from experiment but sometimes other distributions such as the uniform or for discrete values binomial and Poisson distributions.
This block it is very important to hide and not echo the contents either otherwise these will appear in the questions which is not what you want but in this case I will omit that part of the code so that it appears.
# Create a vector called Gender which contains 255 Females, 60 Males and 2 Non-binary
Gender <- c(rep("Woman",255),rep("Man",60),rep("Non-binary",2))
# Create vectors for the way that the different genders look at their nails based on the experimental probabilities.
NailW <- sample(c("Outstretched","Curled Up"),255,replace=T,prob=c(188/255, 67/255))
NailM <- sample(c("Outstretched","Curled Up"),60,replace=T,prob=c(48/60,12/60))
NailNB <- sample(c("Outstretched","Curled Up"),2,replace=T)
# Combine these vectors together into one long vector the same length as the vector Gender.
Nail <- c(NailM,NailW,NailNB)
# Combine the two vectors together to create a data frame with two columns called data
data <- data.frame(Gender,Nail)
# Now the datafile can be saved in SPSS format and csv format
write.csv(data, "fingernails.csv", row.names = FALSE, quote = FALSE)
write_sav(data, "fingernails.sav")
# Tabulate this data to produce a cross-tabulation or contingency table
# This counts up how many students from each gender either fold their fingers or keep them outstretched.
t1 <- table(data) The gender distribution is based on the actual responses I had to an online student survey that I took one year and so these numbers are fixed.
There are two ways that you can look at your finger nails, either by holding the back of your hand outstretched or by folding your fingers. From the class data 188 women kept their fingers outstretched from the 255, whereas for the men it was 48 out of 60 and for the non-binary students it was one of each. This allows experimental probabilities to be determined. These are then used to sample whether the student looks at their nails outstretched or folded. The probability is gender dependent and so a random dataset can be created based on sampling with these probabilities.
This creates the samples NailW, NailM and NailNB for women, men and non-binary students respectively.
These are them summarised in a cross tabulation which is displayed below but this piece of code should be omited in the real script. This is just a check of what t1 looks like.
# omit this step from the actual markdown for R-exams
t1## Nail
## Gender Curled Up Outstretched
## Man 17 43
## Non-binary 1 1
## Woman 64 191The table has three rows and two columns. As the numbers are so low for non-binary students these are not effectively sampled and any predictions about such a small group are unreliable and so we will restrict the conditional probability calculation to only men or women.
5 The Image Generation Block
This is not needed in all cases but if you want to include tabluated data or SPSS like output in the question then you need to render the tables as images to be compatible with Blackboard Ultra.
kable(t1, "html", align = "l") %>%
kable_styling(full_width = T, font_size=22) %>%
column_spec(1, bold=T) %>%
collapse_rows(columns = 1:2, valign = "top")%>%
save_kable(file="table1.png")6 The Answer Generation Block
Next you need to generate an answer to the question and also some wrong answers as Blackboard Ultra can only deal with multiple choice answers. In this case the first step is to pick a random cell from the cross-tabulation. This is then used to calculate the probability.
# At random choose a column number i to be either 1 or 2
i <- sample(1:2, size=1)
# At random choose a row number j to be either 1 or 3
j <- sample(c(1,3), size=1)
# Row 1 is man and in total there are 60 men, Row 2 is woman and there are 255 women.
# The variable gen is created to contain the label of the chosen row.
# The variable k is created to contain the total number of people of that gender.
if(j==1) {
gen <-"man"
k <- 60
} else {
gen <-"woman"
k <- 255}
# Column 1 is Curled Up and Column 2 is Outstretched
# The variable finger is created to contain the label of the chosen column.
if(i==1) {
finger <-"Curled Up"
} else {
finger <-"Outstretched"}
# Then the probability is the number in the cell divided by the number of people of that gender.
# The answer is given to three significant figures.
answer <- signif(t1[j,i]/k,3)
# The wrong answers are based on calculating the probability for the cell and not the conditional probability and also using the wrong number of significant figures.
wrong1 <- signif(t1[j,i]/317,3)
wrong2 <- signif(t1[j,i]/k,2)
wrong3 <- signif(t1[j,i]/317,4)7 The Question and Answerlist
All of the steps so far have been in R preparing the materials for the assessment question itself. Now we come to the parts that the students will actually see. It is important that the wording is as clear as possible. When generating answers in the multiple choice you want to represent common mistakes that students make so that you can deleiver effective feedback. I used to only give feedback on the formative versions of the assessments but there is no harm in doing this for the summative assessments as well if you generate a large enough pool of questions.
The text for the questions is given below.
Question
========
An online questionnaire asked students to report whether they looked at their fingernails as either outstretched or curled up.The following data was collected for the 317 students from the class who responded to the questionnaire.In this case we now need to insert the figure with the following code.
NB add echo is false so that you will not see this in the final question.
figure <- c("table1.png")
knitr::include_graphics(figure)
The rest of the text for the question follows. Note that you use the r variables gen and finger which are randomised parts of the question. These are represented by ` r variable ` but you see the values in the code below.
What is the conditional probability that if it is a woman that they check their fingernails by them being Curled Up to 3 significant figures?Once you are sure that the wording of the question makes sense then you can create the answerlist. This tells the online learning environment how many parts there are to the question if there are multiple numerical calculations which is possible in Moodle. For multiple choice questions this is the list of possible answers.
Answerlist
----------
* 0.251
* 0.202
* 0.25
* 0.20198 The Solution
Under solution you can give either generic or specific feedback on the question. For multiple choice it can make sense to have specific feedback based on how you created the wrong solutions and the mistakes that students commonly make. In this case it would be calculating the probability of the cell and not the conditional probability for the gender. That would mean dividing by the total number of participants and not the total number of members of the specified gender.
Solution
========
To calculate the probability you need to find the number of men or women who check their fingers by Curled Up and divide by the total number of participants of that gender. In this case the probability is 64 / 255 = 0.251.
Answerlist
----------
* True
* FALSE
* FALSE
* FALSE
9 The Meta Information
This is additional information that Blackboard needs to characterise the question pool. It also defines the name of the pool when it is imported. In this case I have given it a very long name. It is a single answer multiple choice and the correct solution is solution 1 and the order of the four solutions will be shuffled
Meta-information
============
exname: The conditional probability of how different genders look at their fingernails.
extype: schoice
exsolution: 1000
exshuffle: 410 Stress Testing
One final thing that you should do before deploying a test is a stress test. As you are generating hundreds of random variants of the questions you need to make sure that there are no silly errors.
s1 <- stresstest_exercise("fingernails.Rmd")
plot(s1)
plot(s1, type="solution")
names(s1)
names(s1$objects)