UQM - ACSME results
KZ
19 July 2014
ABSTRACT
ARE STUDENTS READING MY FEEDBACK? USING A FEEDBACK ANALYTICS CAPTURE SYSTEM TO UNDERSTAND HOW LARGE COHORTS OF BIOMEDICAL SCIENCE STUDENTS USE FEEDBACK
Kirsten Zimbardia, Andrew Dekkerb, Andrea Bugarcicc, Kay Colthorpea, Prasad Chunduria, Judit Kibedia, Lesley J. Llukaa, Craig Engstromd, Peter Worthye, Phillip D. Longb
Presenting Author: Kirsten Zimbardi (k.zimbardi@uq.edu.au) aSchool of Biomedical Science, Faculty of Medicine and Biomedical Science, University of Queensland, St Lucia QLD 4072, Australia bUQx, University of Queensland, St Lucia QLD 4072, Australia cInstitute for Molecular Bioscience, University of Queensland, St Lucia QLD 4072, Australia dSchool of Human Movement Studies, Faculty of Health Science, University of Queensland, St Lucia QLD 4072, Australia eSchool of Chemistry and Molecular Biosciences, Faculty of Science, University of Queensland, St Lucia QLD 4072, Australia
KEYWORDS: feedback analytics, mentoring, assessment for learning, educational data mining, audio feedback
ABSTRACT Feedback is one of the most potent teaching strategies known to produce student learning gains (Hattie, 2009). However, the provision of feedback has been identified by students as one of the weakest elements of university practices (Graduate Careers Australia, 2012). Although there are many theoretical frameworks aimed at improving feedback provision (Hattie & Timperley, 2007; Nicol & Macfarlane Dick, 2006; Sadler, 2010), very little is known about how students actually use feedback (Jonsson, 2013). Many authors contend that students commonly ignore feedback (Boud & Molloy, 2013), supported by limited empirical evidence that students do not collect or read written feedback (Sinclair & Cleland, 2007), or ignore it when they do not understand what it means (Still & Koerber, 2010). The increasingly widespread adoption of online marking and feedback tools facilitates students’ access to their feedback, but until now there has been no systematic way to characterise the patterns of student access of this feedback, nor how this impacts on their subsequent performance (Ellis, 2013).
We have developed, and extensively trialled, a Feedback Analytics Capture System (FACS, previously called UQMarkUP) which synthesises large-scale data on digital feedback provision, how students access feedback, and changes in students’ academic performance (Zimbardi et al., 2013). Specifically, FACS captures detailed information about the audio, typed and hand-drawn annotations markers insert in situ in electronic assessment submissions, and the marks awarded across a variety of systems, including detailed criteria-standards rubrics. FACS also collects detailed information about how students access this feedback, logging the timing and nature of every mouse click a student uses to interact with the feedback-embedded document.
In this exploratory study, we investigated the frequency, timing, and patterns in how students access their feedback. Analyses of FACS data from laboratory reports submitted for summative assessment in two biomedical science courses in level 1 (n=1781 students) and level 2 (n=389), in Semesters 1 and 2, 2013, revealed that the vast majority of students opened their feedback. In the level 1 course 93% students opened Report 1, 92% opened Report 2, 87% opened Report 3 and 85% opened Report 4. In contrast, far fewer students in the level 2 course opened their feedback, and fewer students opened Report 1 (68%) than Report 2 (82%). Although a similar pattern existed for how long students had their feedback open (level 1 Report 1: 12±8 hours; Report 2: 3.4±1.6 hours; Report 3: 2.1±1.4 hours; Report 4: 43±7 minutes), the level 2 reports now reverted to greater duration of interaction with Report 1 (5.6±0.6 hours) than Report 2 (1.2±0.3 hours). There was a surge in student accessing of feedback 1-2 days after feedback release, followed by a persistent tail of students accessing the feedback for the subsequent two months. In this context of early stage undergraduate biomedical science laboratory assessments, students are not only collecting and reading their feedback, but they are interacting with it extensively. There are however potential maturational, or course-specific effects on feedback use which need to be explored further as we expand this analytical approach to understanding how students engage with feedback across a broader range of educational contexts.
REFERENCES
Boud, D., & Molloy, E. (2013). Feedback in Higher and Professional Education. Routledge. Ellis, C. (2013). Broadening the scope and increasing the usefulness of learning analytics: The case for assessment analytics. British Journal of Educational Technology, 44(4), 662–664. Graduate Careers Australia. (2012). Graduate Course Experience Questionnaire (CEQ) results 2012. Retrieved April 21, 2014, from http://www.graduatecareers.com.au/research/researchreports/graduatecourseexperience/ Hattie, J. (2009). Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. London: Routledge. Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112. Jonsson, A. (2013). Facilitating productive use of feedback in higher education. Active Learning in Higher Education, 14(1), 63–76. Nicol, D. J., & Macfarlane Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. Sadler, D. R. (2010). Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535–550. Sinclair, H. K., & Cleland, J. A. (2007). Undergraduate medical students: who seeks formative feedback? Medical Education, 41(6), 580–582. Blackwell Publishing Ltd. Still, B., & Koerber, A. (2010). Listening to Students: A Usability Evaluation of Instructor Commentary. Journal of Business and Technical Communication, 24(2), 206–233. Zimbardi, K., Colthorpe, K., Bugarcic, A., Dekker, A., Worthy, P., Engstrom, C., Long, P., et al. (2013). Analytics of student interactions with electronic feedback using UQMarkUp. Presented at the Transforming Assessment, Melbourne & Brisbane, Australia. Retrieved from http://www.transformingassessment.com/events_4_september_2013.php
CALCULATIONS
FbP = feedback provision FbU = feedback use AcP = Academic performance
Indexes
ProjectID = read.csv("ProjectIDs.csv")
ProjectID## X ProjectID Course Year Semester AssName LxRx LxSxRx
## 1 1 45 BIOL1040 2013 2 Report 3 L1R3 L1S2R3
## 2 2 44 BIOL1040 2013 2 Report 2 L1R2 L1S2R2
## 3 3 43 BIOL1040 2013 2 Report 1 L1R1 L1S2R1
## 4 4 40 BIOL1040 2013 1 Report 4 L1R4 L1S1R4
## 5 5 39 BIOL1040 2013 1 Report 3 L1R3 L1S1R3
## 6 6 32 BIOL1040 2013 1 Report 2 L1R2 L1S1R2
## 7 7 31 BIOL1040 2013 1 Report 1 L1R1 L1S1R1
## 8 8 34 BIOM2011 2013 1 Report 1 L2R1 L2S1R1
## 9 9 41 BIOM2011 2013 1 Report 2 L2R2 L2S1R2
## 10 10 48 BIOM2011 2013 2 Report 1 L2R1 L2S2R1
## 11 11 50 BIOM2011 2013 2 Report 2 L2R2 L2S2R2
## 12 12 28 BIOM2013 2013 1 Oral L2R1 L2S1R1
## 13 13 42 BIOM2013 2013 1 Report L2R2 L2S1R2
## 14 14 63 BIOM2013 2014 1 Oral L2R1 L2S1R1LxRx = Level and Report eg Level 1 (BIOL1040 = first year) Report 1 (1st Report) Sx = semester
Setting up sequences
source("ProjectNames.R")
project.names()Loading data for Feedback Provision, Feedback Use, Academic Performance, and Deomgraphics
source("RawData.R")
raw.data()Feedback Provision BIOL1040
sapply(FbP.biol, nrow)## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 10881 9845 8934
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 6985 13294 13414
## BIOL1040Sem2Report3
## 10285sapply(FbP.biol, dim)## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## [1,] 10881 9845 8934
## [2,] 13 13 13
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## [1,] 6985 13294 13414
## [2,] 13 13 13
## BIOL1040Sem2Report3
## [1,] 10285
## [2,] 13sapply(FbP.biol, function(x) max(x$Duration.sec, na.rm=T))## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 185 252 206
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 261 238 168
## BIOL1040Sem2Report3
## 148Feedback Provision BIOM2011
sapply(FbP.biom, nrow)## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## 7881 4831 2173
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## 1023 1923 2466
## BIOM2013Sem1Oral
## 1759sapply(FbP.biom, dim)## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## [1,] 7881 4831 2173
## [2,] 13 13 13
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## [1,] 1023 1923 2466
## [2,] 13 13 13
## BIOM2013Sem1Oral
## [1,] 1759
## [2,] 13sapply(FbP.biom, function(x) max(x$Duration.sec, na.rm=T))## Warning: no non-missing arguments to max; returning -Inf## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## 417 393 475
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## 302 295 337
## BIOM2013Sem1Oral
## -InfFeedback Use BIOL1040
sapply(FbU.biol, nrow)## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 62376 62655 59419
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 23231 92681 64722
## BIOL1040Sem2Report3
## 25213sapply(FbU.biol, function(x) max(x$msec.B4.scroll, na.rm=T))## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 8.498e+07 8.579e+07 1.650e+08
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 7.816e+07 4.588e+10 1.128e+08
## BIOL1040Sem2Report3
## 6.961e+07#in minutes
sapply(FbU.biol, function(x) round(max(x$msec.B4.scroll, na.rm=T)/60000))## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 1416 1430 2749
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 1303 764641 1881
## BIOL1040Sem2Report3
## 1160Feedback Use BIOM2011
sapply(FbU.biom, nrow)## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## 35414 13192 7110
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## 3014 4734 4980
## BIOM2013Sem1Oral
## 3654sapply(FbU.biom, function(x) max(x$msec.B4.scroll, na.rm=T))## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## 80846966 62519202 61998728
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## 34956600 69407785 57766049
## BIOM2013Sem1Oral
## 72369730#in minutes
sapply(FbU.biom, function(x) round(max(x$msec.B4.scroll, na.rm=T)/60000))## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## 1347 1042 1033
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## 583 1157 963
## BIOM2013Sem1Oral
## 1206Academic Performance BIOL1040
sapply(AcP.biol, nrow)## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 704 722 696
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 696 931 934
## BIOL1040Sem2Report3
## 869sapply(AcP.biol, function(x) range(x$Final.Grade, na.rm=T))## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## [1,] 20 20 40
## [2,] 100 100 100
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## [1,] 40 40 20
## [2,] 100 100 100
## BIOL1040Sem2Report3
## [1,] 40
## [2,] 100sapply(AcP.biol, function(x) mean(x$Final.Grade, na.rm=T))## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 76.30 79.52 85.55
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 88.59 80.82 86.53
## BIOL1040Sem2Report3
## 89.88Academic Performance BIOM2011
sapply(AcP.biom, nrow)## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## 257 256 89
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## 82 61 65
## BIOM2013Sem1Oral
## 58AcP.biom[[5]][,18] = as.numeric(AcP.biom[[5]][,18])## Warning: NAs introduced by coercionAcP.biom[[7]][,16] = as.numeric(AcP.biom[[7]][,16])## Warning: NAs introduced by coercionsapply(AcP.biom, function(x) range(x$Final.Grade, na.rm=T))## Warning: no non-missing arguments to min; returning Inf
## Warning: no non-missing arguments to max; returning -Inf## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## [1,] 23 7 12.25
## [2,] 99 97 90.25
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## [1,] 15.00 Inf 26.75
## [2,] 92.25 -Inf 95.25
## BIOM2013Sem1Oral
## [1,] 40
## [2,] 96sapply(AcP.biom, function(x) round(mean(x$Final.Grade, na.rm=T)))## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## 60 69 59
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## 68 NaN 66
## BIOM2013Sem1Oral
## 65Demographics BIOL1040
sapply(demog.biol, nrow)## Sem1 Sem2
## 735 974sapply(demog.biol, function(x) range(x$Cumulative.Semester.GPA, na.rm=T))## Sem1 Sem2
## [1,] 0 0
## [2,] 7 7sapply(demog.biol, function(x) round(mean(x$Cumulative.Semester.GPA, na.rm=T), digits = 2))## Sem1 Sem2
## 4.68 4.91Demographics BIOM2011
sapply(demog.biom, nrow)## Sem1 Sem2
## 252 93sapply(demog.biom, function(x) range(x$Cumulative.Semester.GPA, na.rm=T))## Sem1 Sem2
## [1,] 0 2
## [2,] 7 7sapply(demog.biom, function(x) round(mean(x$Cumulative.Semester.GPA, na.rm=T), digits = 2))## Sem1 Sem2
## 4.87 4.58NB no demographic data for BIOM2013 at the moment, but can get this if you want…
corpus stats sep for biol1040 and biom2011
#number of feedback items in corpus
fb = sapply(FbP.biol, nrow)
fb## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 10881 9845 8934
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 6985 13294 13414
## BIOL1040Sem2Report3
## 10285sum(fb)## [1] 73638#number of students (ie submissions)
sub = sapply(biol.names, function(x) length(unique(FbP.biol[[x]][,2])))
sub## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 692 678 675
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 674 909 884
## BIOL1040Sem2Report3
## 859#number of submissions
sum(sub)## [1] 5371#number of feedback items in corpus
fb = sapply(FbP.biom, nrow)
fb## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## 7881 4831 2173
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## 1023 1923 2466
## BIOM2013Sem1Oral
## 1759sum(fb)## [1] 22056#number of students (ie submissions)
sub = sapply(1:4, function(x) length(unique(FbP.biom[[x]][,2])))
sub## [1] 228 231 80 74sum(sub)## [1] 613Combining biol1040 and biom2011 data in to a single list (FbP):
FbP = FbP.biol
FbP = c(FbP, lapply(1:4, function(x) {FbP.biom[[x]]}))
head(FbP[[9]])## X SubID MarkerID AnnotType Page Xlocation Ylocation Xdim Ydim
## 1 1 4802 T31 Highlight 3 0.5078 0.13268 0.1619 0.06290
## 2 2 4802 T31 Highlight 4 0.1341 0.49076 0.7577 0.38101
## 3 3 4802 T31 Highlight 4 0.1500 0.07885 0.6791 0.06596
## 4 4 4802 T31 Highlight 2 0.2531 0.66752 0.2718 0.02451
## 5 5 4802 T31 Recording 4 0.8751 0.50156 30.0000 30.00000
## 6 6 4802 T31 Text 3 0.7453 0.14786 0.1398 0.05037
## Filename Annotation Duration.sec Consent
## 1 annot__2013_05_25_12_07_33.png Annotation NA Yes
## 2 annot__2013_05_25_12_24_39.png Annotation NA Yes
## 3 annot__2013_05_25_12_12_52.png Annotation NA Yes
## 4 annot__2013_05_25_12_01_19.png Annotation NA Yes
## 5 annot__2013_05_25_12_16_01.m4a Results 82 Yes
## 6 Bicep brachii NA Yesnames(FbP) = names.all
#number of feedback items in corpus
fb = sapply(FbP, nrow)
fb## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 10881 9845 8934
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 6985 13294 13414
## BIOL1040Sem2Report3 BIOM2011Sem1Report1 BIOM2011Sem1Report2
## 10285 7881 4831
## BIOM2011Sem2Report1 BIOM2011Sem2Report2
## 2173 1023sum(fb)## [1] 89546#number of students (ie submissions) per Report
sub = sapply(names.all, function(x) length(unique(FbP[[x]][,2])))
sub## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 692 678 675
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 674 909 884
## BIOL1040Sem2Report3 BIOM2011Sem1Report1 BIOM2011Sem1Report2
## 859 228 231
## BIOM2011Sem2Report1 BIOM2011Sem2Report2
## 80 74#number of submissions in corpus
sum(sub)## [1] 5984There were 5984 submissions in the corpus, for the following assessment items:
## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## 692 678 675
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## 674 909 884
## BIOL1040Sem2Report3 BIOM2011Sem1Report1 BIOM2011Sem1Report2
## 859 228 231
## BIOM2011Sem2Report1 BIOM2011Sem2Report2
## 80 74## [1] 695 690 675 679 912 887 861There were 1619 and 684 consenting students in level 1 and level 2 respectively
The length of time (units = minutes) BIOL1040 students opened the feedback on Report 1 (in both semesters 1 and 2) was…
Use of feedback - cleaning biol to match biom
sapply(FbU.biol, dim)## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## [1,] 62376 62655 59419
## [2,] 24 24 24
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## [1,] 23231 92681 64722
## [2,] 24 24 24
## BIOL1040Sem2Report3
## [1,] 25213
## [2,] 24sapply(FbU.biom, dim)## BIOM2011Sem1Report1 BIOM2011Sem1Report2 BIOM2011Sem2Report1
## [1,] 35414 13192 7110
## [2,] 21 21 21
## BIOM2011Sem2Report2 BIOM2013Sem1Oral BIOM2013Sem1Report
## [1,] 3014 4734 4980
## [2,] 21 21 21
## BIOM2013Sem1Oral
## [1,] 3654
## [2,] 21df = FbU.biol
for (i in 1:7)
{
for (j in seq(12,10,-1))
{
df[[i]][,j] = NULL
}
}
sapply(df, dim)## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## [1,] 62376 62655 59419
## [2,] 21 21 21
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## [1,] 23231 92681 64722
## [2,] 21 21 21
## BIOL1040Sem2Report3
## [1,] 25213
## [2,] 21df[[1]][1:3,]## X EventID SubmissionID StudentID Consent MarkerID Audio.Annot Log.Type
## 1 1 70363 1978 S8402295 Yes T15 7 2
## 2 2 70349 1978 S8402295 Yes T15 7 2
## 3 3 70362 1978 S8402295 Yes T15 7 2
## Interaction Date State Pages Start End Page.Size
## 1 Audio 19/03/13 13:34 finished NA NA NA NA
## 2 Annotations 19/03/13 13:33 NA NA NA NA
## 3 Audio 19/03/13 13:34 finished NA NA NA NA
## msec.B4.scroll AnnotID From.time Current.time Duration
## 1 NA 5 0 4.428 4.428
## 2 NA NA NA NA NA
## 3 NA 5 0 4.428 4.428
## Filename
## 1 annot__2013_03_16_07_20_18.mp3
## 2
## 3 annot__2013_03_16_07_20_18.mp3FbU.biom[[1]][1:3,]## X EventID SubmissionID StudentID Consent MarkerID Audio.Annot Log.Type
## 1 1 201912 3301 S8285971 Yes T19 11 2
## 2 2 201911 3301 S8285971 Yes T19 11 2
## 3 3 201907 3301 S8285971 Yes T19 11 2
## Interaction Date State Pages Start End Page.Size
## 1 Scroll 30/04/13 6:43 NA 14.9 94.6 6
## 2 Scroll 30/04/13 6:43 NA 0.1 14.0 6
## 3 Automatic 30/04/13 6:43 opened NA NA NA NA
## msec.B4.scroll AnnotID From.time Current.time Duration Filename
## 1 1861 NA NA NA NA
## 2 7796 NA NA NA NA
## 3 NA NA NA NA NAFbU.biol = df
FbU.biol[[1]][1:3,]## X EventID SubmissionID StudentID Consent MarkerID Audio.Annot Log.Type
## 1 1 70363 1978 S8402295 Yes T15 7 2
## 2 2 70349 1978 S8402295 Yes T15 7 2
## 3 3 70362 1978 S8402295 Yes T15 7 2
## Interaction Date State Pages Start End Page.Size
## 1 Audio 19/03/13 13:34 finished NA NA NA NA
## 2 Annotations 19/03/13 13:33 NA NA NA NA
## 3 Audio 19/03/13 13:34 finished NA NA NA NA
## msec.B4.scroll AnnotID From.time Current.time Duration
## 1 NA 5 0 4.428 4.428
## 2 NA NA NA NA NA
## 3 NA 5 0 4.428 4.428
## Filename
## 1 annot__2013_03_16_07_20_18.mp3
## 2
## 3 annot__2013_03_16_07_20_18.mp3Use of feedback - merging biol and biom
FbU = FbU.biol
FbU = append(FbU, FbU.biom[1:4])
FbU[[1]][1:3,]## X EventID SubmissionID StudentID Consent MarkerID Audio.Annot Log.Type
## 1 1 70363 1978 S8402295 Yes T15 7 2
## 2 2 70349 1978 S8402295 Yes T15 7 2
## 3 3 70362 1978 S8402295 Yes T15 7 2
## Interaction Date State Pages Start End Page.Size
## 1 Audio 19/03/13 13:34 finished NA NA NA NA
## 2 Annotations 19/03/13 13:33 NA NA NA NA
## 3 Audio 19/03/13 13:34 finished NA NA NA NA
## msec.B4.scroll AnnotID From.time Current.time Duration
## 1 NA 5 0 4.428 4.428
## 2 NA NA NA NA NA
## 3 NA 5 0 4.428 4.428
## Filename
## 1 annot__2013_03_16_07_20_18.mp3
## 2
## 3 annot__2013_03_16_07_20_18.mp3FbU[[9]][1:3,]## X EventID SubmissionID StudentID Consent MarkerID Audio.Annot Log.Type
## 1 1 510378 4907 S8285971 Yes T19 9 2
## 2 2 510379 4907 S8285971 Yes T19 9 2
## 3 3 510381 4907 S8285971 Yes T19 9 2
## Interaction Date State Pages Start End Page.Size
## 1 Scroll 2/09/13 10:06 NA 17.3 17.3 5
## 2 Scroll 2/09/13 10:06 NA 17.6 19.8 5
## 3 Scroll 2/09/13 10:07 NA 21.1 22.2 5
## msec.B4.scroll AnnotID From.time Current.time Duration Filename
## 1 1956 NA NA NA NA
## 2 4324 NA NA NA NA
## 3 844 NA NA NA NAnames(FbU) = names.all
sapply(8:11, function(x) length(unique(FbU[[x]]$StudentID)))## [1] 154 185 55 58sum(sapply(8:11, function(x) length(unique(FbU[[x]]$StudentID))))## [1] 452Use of feedback - checking msecB4scroll for negatives
df = FbU
min(df[[1]][,16], na.rm=T)## [1] 82min(df[[2]][,16], na.rm=T)## [1] 35min(df[[3]][,16], na.rm=T)## [1] 18min(df[[4]][,16], na.rm=T)## [1] 40min(df[[5]][,16], na.rm=T)## [1] 7min(df[[6]][,16], na.rm=T)## [1] 16min(df[[7]][,16], na.rm=T)## [1] 11min(df[[8]][,16], na.rm=T)## [1] 129min(df[[9]][,16], na.rm=T)## [1] 476min(df[[10]][,16], na.rm=T)## [1] 96min(df[[11]][,16], na.rm=T)## [1] 446unique submission IDs FbP vs FbU - did they open the fb (% opened)
FbP.submissions = NULL
for (i in 1:11)
{
FbP.submissions[[i]] = unique(FbP[[i]][,2])
}
FbU.submissions = NULL
for (i in 1:11)
{
FbU.submissions[[i]] = unique(FbU[[i]][,3])
}
p = sapply(FbP.submissions, length)
u = sapply(FbU.submissions, length)
p## [1] 692 678 675 674 909 884 859 228 231 80 74u## [1] 626 637 628 572 853 797 715 155 188 55 59p - u## [1] 66 41 47 102 56 87 144 73 43 25 15u/p## [1] 0.9046 0.9395 0.9304 0.8487 0.9384 0.9016 0.8324 0.6798 0.8139 0.6875
## [11] 0.7973Working out mean and SEMs for open duration
use.min = NULL
for (i in 1:11)
{
use.min[[i]] = round(xtabs(msec.B4.scroll ~ SubmissionID, data=FbU[[i]])/60000)
}
names.all## [1] "BIOL1040Sem1Report1" "BIOL1040Sem1Report2" "BIOL1040Sem1Report3"
## [4] "BIOL1040Sem1Report4" "BIOL1040Sem2Report1" "BIOL1040Sem2Report2"
## [7] "BIOL1040Sem2Report3" "BIOM2011Sem1Report1" "BIOM2011Sem1Report2"
## [10] "BIOM2011Sem2Report1" "BIOM2011Sem2Report2"names(use.min) = names.all
use.hrs = NULL
for (i in 1:11)
{
use.hrs[[i]] = round(xtabs(msec.B4.scroll ~ SubmissionID, data=FbU[[i]])/(60*60000), digits = 2)
}
names(use.hrs) = names.all
summarise = function(x)
{
mean = mean(x)
n = length(x)
sd = sd(x)
sem = sd/sqrt(n)
print(c(mean, sem))
}
use.min.stats = sapply(use.min, summarise)## [1] 235.51 15.47
## [1] 229.74 16.39
## [1] 228.00 15.81
## [1] 42.95 6.94
## [1] 1092 897
## [1] 184.74 12.18
## [1] 28.729 4.936
## [1] 389.06 52.62
## [1] 80.14 20.16
## [1] 222.84 59.64
## [1] 33.24 14.25use.hrs.stats = sapply(use.hrs, summarise)## [1] 3.9254 0.2579
## [1] 3.8294 0.2732
## [1] 3.8002 0.2635
## [1] 0.7164 0.1157
## [1] 18.21 14.95
## [1] 3.079 0.203
## [1] 0.47867 0.08226
## [1] 6.484 0.877
## [1] 1.336 0.336
## [1] 3.714 0.994
## [1] 0.5544 0.2374write.csv(use.hrs.stats, "use.csv")
use.hrs.combined = use.hrs
use.hrs.combined[[1]] = append(use.hrs[[1]],use.hrs[[5]])
use.hrs.combined[[2]] = append(use.hrs[[2]],use.hrs[[6]])
use.hrs.combined[[3]] = append(use.hrs[[3]],use.hrs[[7]])
use.hrs.combined[[5]] = append(use.hrs[[8]],use.hrs[[10]])
use.hrs.combined[[6]] = append(use.hrs[[9]],use.hrs[[11]])
use.hrs.combined[[11]] = NULL
use.hrs.combined[[10]] = NULL
use.hrs.combined[[9]] = NULL
use.hrs.combined[[8]] = NULL
use.hrs.combined[[7]] = NULL
use.hrs.combined.stats = sapply(use.hrs.combined, summarise)## [1] 12.161 8.623
## [1] 3.4121 0.1659
## [1] 2.0305 0.1383
## [1] 0.7164 0.1157
## [1] 5.7550 0.7011
## [1] 1.1495 0.2626write.csv(use.hrs.combined.stats, "useSemCombined.csv")statistical tests - biom FbU
t.test(use.hrs.combined[[5]], use.hrs.combined[[6]])##
## Welch Two Sample t-test
##
## data: use.hrs.combined[[5]] and use.hrs.combined[[6]]
## t = 6.151, df = 266, p-value = 2.806e-09
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 3.131 6.080
## sample estimates:
## mean of x mean of y
## 5.755 1.150histograms
for (i in 1:11)
{
h = hist(log(use.min[[i]]), xlim=c(0,10), main=names.all[i])
h
}
exp(6)## [1] 403.4400/60## [1] 6.667boxplots
boxplot(use.min)
m = NULL
for (i in 1:11)
{
m[i] = max(use.min[[i]])
}
which(m[3] == use.min[[3]])## 3959
## 235use.min[[3]][234]## 3958
## 510use.min.outliers = use.min
use.min.outliers[[3]] = use.min.outliers[[3]][-234]
max(use.min.outliers[[3]])## [1] 4279boxplot(use.min.outliers)
boxplot(use.min.outliers, ylim=c(0,1000))
pauses - unfinished
sapply(names.all, function(x) range(FbU[[x]]$msec.B4.scroll, na.rm=T))## BIOL1040Sem1Report1 BIOL1040Sem1Report2 BIOL1040Sem1Report3
## [1,] 82 35 1.80e+01
## [2,] 84980468 85789649 1.65e+08
## BIOL1040Sem1Report4 BIOL1040Sem2Report1 BIOL1040Sem2Report2
## [1,] 40 7.000e+00 16
## [2,] 78161892 4.588e+10 112830911
## BIOL1040Sem2Report3 BIOM2011Sem1Report1 BIOM2011Sem1Report2
## [1,] 11 129 476
## [2,] 69614078 80846966 62519202
## BIOM2011Sem2Report1 BIOM2011Sem2Report2
## [1,] 9.6e+01 446
## [2,] 6.2e+07 34956600timing of access - adding Pgroups
file.names <- c("BIOL-2013S1Pgroups.csv", "BIOL-2013S2Pgroups.csv", "BIOM-2013S1Pgroups.csv", "BIOM-2013S2Pgroups.csv")
Pgroups <- lapply(file.names, read.csv, header=TRUE, stringsAsFactors=FALSE)
sapply(1:4, function(x) length(Pgroups[[x]][,1]))## [1] 725 918 201 88sum(sapply(1:4, function(x) length(Pgroups[[x]][,1])))## [1] 1932Pgroups[[1]][1:10,]## StudentID Course Sem Pgroup Pg
## 1 S8584975 BIOL 1 P1 BIOL1P1
## 2 S8597563 BIOL 1 P1 BIOL1P1
## 3 S8529073 BIOL 1 P1 BIOL1P1
## 4 S8412939 BIOL 1 P1 BIOL1P1
## 5 S8607105 BIOL 1 P1 BIOL1P1
## 6 S8563497 BIOL 1 P1 BIOL1P1
## 7 S8624225 BIOL 1 P1 BIOL1P1
## 8 S8573027 BIOL 1 P1 BIOL1P1
## 9 S8587341 BIOL 1 P1 BIOL1P1
## 10 S8587345 BIOL 1 P1 BIOL1P1Pgroups[[2]][1:10,]## StudentID Course Sem Pgroup Pg
## 1 S8584333 BIOL 2 P1 BIOL2P1
## 2 S8583787 BIOL 2 P1 BIOL2P1
## 3 S8199113 BIOL 2 P1 BIOL2P1
## 4 S8626835 BIOL 2 P1 BIOL2P1
## 5 S8597919 BIOL 2 P1 BIOL2P1
## 6 S8568873 BIOL 2 P1 BIOL2P1
## 7 S8643327 BIOL 2 P1 BIOL2P1
## 8 S8639707 BIOL 2 P1 BIOL2P1
## 9 S8643535 BIOL 2 P1 BIOL2P1
## 10 S8640021 BIOL 2 P1 BIOL2P1Pgroups[[3]][1:10,]## StudentID Course Sem Pgroup Pg
## 1 S8588339 BIOM 1 P3 BIOM1P3
## 2 S8577309 BIOM 1 P3 BIOM1P3
## 3 S8587235 BIOM 1 P3 BIOM1P3
## 4 S8534409 BIOM 1 P3 BIOM1P3
## 5 S8574607 BIOM 1 P3 BIOM1P3
## 6 S8578821 BIOM 1 P3 BIOM1P3
## 7 S8580303 BIOM 1 P3 BIOM1P3
## 8 S8580399 BIOM 1 P3 BIOM1P3
## 9 S8343073 BIOM 1 P3 BIOM1P3
## 10 S8571187 BIOM 1 P3 BIOM1P3Pgroups[[4]][1:10,]## StudentID Course Sem Pgroup Pg
## 1 S8419713 BIOM 2 P1 BIOM2P1
## 2 S8549773 BIOM 2 P1 BIOM2P1
## 3 S8579731 BIOM 2 P1 BIOM2P1
## 4 S8560941 BIOM 2 P1 BIOM2P1
## 5 S8584841 BIOM 2 P1 BIOM2P1
## 6 S8658567 BIOM 2 P1 BIOM2P1
## 7 S8241553 BIOM 2 P1 BIOM2P1
## 8 S8586053 BIOM 2 P1 BIOM2P1
## 9 S8605993 BIOM 2 P1 BIOM2P1
## 10 S8528959 BIOM 2 P1 BIOM2P1a = NULL
b = NULL
for (i in 1:4)
{
a = Pgroups[[i]]
b = rbind(b, a)
}
dim(b)## [1] 1932 5df2 = FbU
for (i in 1:length(df2))
{
df2[[i]] = merge(df2[[i]], b[,c(1,4)], by="StudentID", all.x=T)
}
length(unique(FbU[[8]]$StudentID))## [1] 154length(unique(df[[8]]$StudentID))## [1] 154length(unique(df2[[8]]$StudentID))## [1] 154FbU = df2
Pgroups = NULL
Pgroups = b
table(Pgroups$Pg)##
## BIOL1P1 BIOL1P2 BIOL1P3 BIOL1P4 BIOL1P5 BIOL1P6 BIOL1P7 BIOL2P1 BIOL2P2
## 109 119 102 108 108 106 73 99 100
## BIOL2P3 BIOL2P4 BIOL2P5 BIOL2P6 BIOL2P7 BIOL2P8 BIOL2P9 BIOM1P3 BIOM1P4
## 109 97 112 107 93 102 99 59 77
## BIOM1P6 BIOM2P1
## 65 88sum(table(Pgroups$Pg))## [1] 1932pg = table(Pgroups$Pg)
length(pg)## [1] 20sum(pg[17:19])## [1] 201table(FbU[[1]]$Pgroup)##
## P1 P2 P3 P4 P5 P6 P7 P8 P9
## 11610 10480 9577 8325 9780 7388 5598 340 91table(FbU[[8]]$Pgroup)##
## P1 P3 P4 P6 P8
## 1237 10394 9358 8270 357converting to POSIXct
df = FbU
df[[1]][1:3,]## StudentID X EventID SubmissionID Consent MarkerID Audio.Annot Log.Type
## 1 S8024919 92 103748 2669 Yes uqadekke 0 2
## 2 S8024919 93 103749 2669 Yes uqadekke 0 2
## 3 S8024919 94 103747 2669 Yes uqadekke 0 2
## Interaction Date State Pages Start End Page.Size
## 1 Annotations 25/03/13 17:58 NA NA NA NA
## 2 Scroll 25/03/13 17:58 NA 0 0 13
## 3 Automatic 25/03/13 17:58 opened NA NA NA NA
## msec.B4.scroll AnnotID From.time Current.time Duration Filename Pgroup
## 1 NA NA NA NA NA <NA>
## 2 902 NA NA NA NA <NA>
## 3 NA NA NA NA NA <NA>for (i in 1:11)
{
df[[i]]$Date = as.POSIXct(df[[i]]$Date, format = "%d/%m/%y %H:%M", tz = "UTC")
}
df[[1]][1:3,9:11]## Interaction Date State
## 1 Annotations 2013-03-25 17:58:00
## 2 Scroll 2013-03-25 17:58:00
## 3 Automatic 2013-03-25 17:58:00 openedfor (i in 1:11)
{
df[[i]]$date = trunc(df[[i]]$Date, "days")
}
str(df[[1]]$date)## POSIXlt[1:63734], format: "2013-03-25" "2013-03-25" "2013-03-25" "2013-03-26" ...for (i in 1:11)
{
df[[i]]$date = as.POSIXct(df[[i]]$date, tz = "UTC")
}
str(df[[1]]$date)## POSIXct[1:63734], format: "2013-03-25" "2013-03-25" "2013-03-25" "2013-03-26" ...df[[1]][1:3,]## StudentID X EventID SubmissionID Consent MarkerID Audio.Annot Log.Type
## 1 S8024919 92 103748 2669 Yes uqadekke 0 2
## 2 S8024919 93 103749 2669 Yes uqadekke 0 2
## 3 S8024919 94 103747 2669 Yes uqadekke 0 2
## Interaction Date State Pages Start End Page.Size
## 1 Annotations 2013-03-25 17:58:00 NA NA NA NA
## 2 Scroll 2013-03-25 17:58:00 NA 0 0 13
## 3 Automatic 2013-03-25 17:58:00 opened NA NA NA NA
## msec.B4.scroll AnnotID From.time Current.time Duration Filename Pgroup
## 1 NA NA NA NA NA <NA>
## 2 902 NA NA NA NA <NA>
## 3 NA NA NA NA NA <NA>
## date
## 1 2013-03-25
## 2 2013-03-25
## 3 2013-03-25FbU = dfsplitting lists into data frames
s = split(FbU[[1]], FbU[[1]]$Pgroup)
bioms1 = split(FbU[[8]], FbU[[8]]$Pgroup)generating calendar figures
source("perdiam.R")
source("calendarHeat.R")
stud.per.day(s[[1]])## Loading required package: lubridate
##
## Attaching package: 'lubridate'
##
## The following objects are masked from 'package:chron':
##
## days, hours, minutes, seconds, years
##
## Loading required package: lattice
## Loading required package: grid## Warning: the condition has length > 1 and only the first element will be
## used
hours.per.day(s[[1]])## Warning: the condition has length > 1 and only the first element will be
## used
ave.hours.per.day(s[[1]])## Warning: the condition has length > 1 and only the first element will be
## used
length(unique(bioms1[[1]][,1]))## [1] 9length(unique(bioms1[[2]][,1]))## [1] 39length(unique(bioms1[[3]][,1]))## [1] 46length(unique(bioms1[[4]][,1]))## [1] 35length(unique(bioms1[[5]][,1]))## [1] 1sum(sapply(1:5, function(x) length(unique(bioms1[[x]][,1]))))## [1] 130BIOM2011 P3
stud.per.day(bioms1[[2]])## Warning: the condition has length > 1 and only the first element will be
## used
hours.per.day(bioms1[[2]])## Warning: the condition has length > 1 and only the first element will be
## used
ave.hours.per.day(bioms1[[2]])## Warning: the condition has length > 1 and only the first element will be
## used
BIOM2011 P4
stud.per.day(bioms1[[3]])## Warning: the condition has length > 1 and only the first element will be
## used
hours.per.day(bioms1[[3]])## Warning: the condition has length > 1 and only the first element will be
## used
ave.hours.per.day(bioms1[[3]])## Warning: the condition has length > 1 and only the first element will be
## used