Tutorial A

B. Information about the Gruenke et al 2014 study

Relevance and intervention logic

Reading comprehension is the ability to construct and extract meaning from a written text. It is considered to be the most critical skill that is needed to succeed in school. If readers have serious difficulties to gather relevant information from a historical account, a mathematical word problem, or a passage in a biology book, they are bound to fail in most every task that is put before them.

This single-case study examined the effects of a graphic organizing strategy on the ability of children to improve their text comprehension abilities. Participants were six students between ten and fourteen years old with major problems in understanding what they read. The intervention intended to teach them to visually highlight key elements of a passage, and thus, to deepen their understanding of it (story mapping).

The study was conducted in Germany. Three 5th grade students from a regular education public school and three 8th grade students from a school for children with learning difficulties served as subjects. Four of them were female (Anna, Bella, Christina, and Dunja), two of them were male (Egor and Fabian) (names altered, for anonymity).

Dependent variable measurement

18 narratives from three different German story books were selected. All of them were altered in a way that it was possible to formulate exactly ten comprehension questions about each tale that covered its main content. The comprehension questions were stated in a way that only one specific and distinct answer was possible to be counted as correct. Subsequently, we standardized the texts, so that each of them consisted of exactly 150 words. In a preliminary survey, the stories and comprehension questions were presented to twenty low achieving children between 9 and 10 years old in order to identify items that were either too easy or too hard to solve. We involved the insights from this preliminary survey to compose the final version of our question sets.

Question: What does the comprehension score in this study mean, then, and what is it’s minimum and maximum value?

writeLines(Q_Intro_3)

The score is the number of correctly answered questions. 
The minimum score is zero, the maximum 10.

In the course of the study, each student was individually presented with a different story and a different set of comprehension questions for 18 consecutive school days. The order of the tales was randomly chosen for each child. Each student was asked to read a respective story out loud and then to write down the answers to the corresponding questions on a worksheet.

Question: Why was the order of the tales randomly chosen?

writeLines(Q_Intro_4)

To avoid sytematic learning based on the stories themselves, 
and, hence, on their sequence. While there is nothing wrong with that in principle, 
it is not the kind of learning studied in this research. In statistical terms, 
randomising the sequence of stories allows to generalise: For *any* sequence of 
stories, the intervention is likely to be effective (if it is effective)

The intervention

To teach the boys and girls how to better comprehend narrative texts by using a story map, the student instructor followed a procedure outlined by Idol (1987):

1. Modeling phase: the teacher demonstrates how a story map is used by reading a tale out loud and by stopping whenever important information is mentioned to fill out parts of his or her worksheet,
2. lead phase: the children read stories independently and complete their maps while the teacher prompts and encourages them to review their work results and to add details that they might have overlooked,
3. test phase: the children read texts, draw maps of their own, ask questions pertaining to the content, answer them, and fill in the components into their maps without close supervision by the teacher; the teacher only intervenes if the students ask for or obviously needs help.

Design

An AB multiple baseline design (MBD) across subjects was applied. We’ll cover what that means in detail later. For the moment, let’s look at one of the six cases, Anna, to understand the basic AB approach and what data it yields.

C. Data setup

You can safely ignore the text, just evaluate the code chunks.

R packages contain not only code, but also often data. The data are bound to (variable) names. SCAN comes with a number of such data sets, for instance with GruenkeWilbert2014, the data from the study described above.

D. Single Case analysis

The data take the form of six objects (each a list with a dataframe plus some meta-data) that go by the names of the pupils: Anna, Bella, Christina, Dunja, Egor and Fabian. The same pseudonames as used in the publication of the study.

Let’s look at Anna, the first case. The data:

Anna

#A single-case data frame with one case

# ... up to 3 more rows

Graphically, Anna’s comprehension scores look like this:

plotSC(Anna)

Let’s add some more information to the plot, namely, a media line for the A and B phase:

plotSC(Anna, 
       xlab = "Training session", ylab = "Comprehension Score",
       phase.names = c("Baseline", "Intervention"), 
      lines = list("median", col = "red" ))

NA

It is also helpful sometimes to highligt particular measurements:

plotSC(Anna, 
       phase.names = c("baseline", "reading intervention"), 
       xlab="days", 
       ylab="reading score", 
       marks = list(positions = c(4, 9), col = "red", cex =2))

Question Q3: How can you add a red dot for day 18?

# Your answer here

writeLines(Q3)

plotSC(Anna, 
       phase.names = c("baseline", "reading intervention"), 
       xlab="days", 
       ylab="reading score", 
       marks = list(positions = c(4, 9, 18), col = "red", cex = 2))

Overlap indices for a single case

Percentage of non-overlapping data (PND)

Staying with Anna, we can calculate an overlap index such as PND: The percentage of non-overlapping data (PND) effect size measure was described by Scruggs, Mastropieri, & Casto (1987) . It is the percentage of all data-points of the second phase of a single-case study exceeding the maximum value of the first phase. In case you have a study where you expect a decrease of values in the second phase, PND is calculated as the percentage of data-point of the second phase below the minimum of the first phase.

Question: Before you run the chunk below, What do you expect the PND to be for Anna, from looking at one of the plots above?

pnd(Anna)

Percent Non-Overlapping Data

Mean  : 100 %

These indices are more useful when comparing multiple cases. Let’s look at Fabian.

plotSC(Fabian, 
       xlab = "Training session", ylab = "Comprehension Score",
       phase.names = c("Baseline", "Intervention"), 
      lines = list("median", col = "red" ))

Question: Before the next step: What do you expect the PND for Fabian to be?

pnd(Fabian)

Percent Non-Overlapping Data

Mean  : 0 %

We can directly plot the two cases like so:

pnd(c(Anna, Fabian))

Percent Non-Overlapping Data

Mean  : 50 %

Question Q4: What is your conclusion regarding the effectiveness of the treatment in the cases of Anna and Fabian?

writeLines(Q4)

As both the AB plot and the PND statistics show, we cannot be quite as sure 
in Fabian's case that the effect was due to the reading program.

Percentage exceeding the median (PEM)

The pem function returns the percentage of phase B data exceeding the phase A median. Additionally, a binomial test against a 50/50 distribution is computed. Different measures of central tendency can be used for alternative analyses.

pem(c(Anna, Fabian))

Percent Exceeding the Median

       PEM positives total binom.p
Anna   100        14    14   0.000
Fabian 100        10    10   0.001

Alternative hypothesis: true probability > 50%