Meta-Learning Analytics
KZ
8 August 2014
Research Question Brain Storm
with Kay and Louise Friday 8 August
Students who report looking at lecture recordings, do they?
Student plans don’t come to fruition (ref)
-> so if students didn’t say they use lecture recordings and now plan to (in response to weakness or Mid-Sem), does access change? For how long?
- control: if said do, and plan to, then lecture recording viewing should be stable, but will give insights into normal semester/lecturer fluctuations
=> by eye, lectures were tues arvo and wed morning and access peaks tue, wed and thurs (can visualise with calendarHeat figures)
how do students prepare for classes => do students look at prac videos
=> by eye Thurs/Fri for Fri prac,
how many of 5 prac’s with videos (as how many times/weeks they access)
are there changes over time ie students stop looking when they realise not so useful
Which students look at lecture recordings
actually figures give how many students access how often first, but calculations do determine which students look, and when
data in “Folder access across semester.xls” moved to “LectAccess.csv”
clean - remove name column, remove empty rows (233-678)
move total column and total row to new vectors, and remove
clean - keep only consenting students reports number of students by number of variables
## [1] 230 116## [1] 99 116clean - De-ID students so can push to html
basic structure of data
## StudentID X4.03.14 X5.03.14 X6.03.14 X7.03.14
## 1 S6089847 1 1 0 0
## 3 S8117889 0 1 4 0
## 4 S8118323 2 0 0 0
## 5 S8152093 0 0 0 0
## 6 S8239113 1 1 0 0
## 7 S8283571 0 2 0 0
## 9 S8395419 1 0 0 0
## 12 S8407099 3 8 4 0
## 13 S8408815 0 0 0 0
## 14 S8465063 0 0 1 0dimensions (rows by columns)
## [1] 99 116Number of students who looked at lectures x number of times 
clipped x axis at 100 access clicks to zoom into lower end 
Converted x axis to log to spread clumped data into normal-ish curve
NB Log scale:
0 = 1
1 ~ 3
2 ~ 7
3 ~ 20
4 ~ 55
5 ~ 150
6 ~ 403
Working out viewings by day - number of times folder accessed per day (access.day), number of students who access each day (stud.day)…
Working out number of times (access.stud) and number of days (days.stud) each student accessed…
Loading ‘describe’ function to get descriptive stat’s…
Descriptive stat’s for viewings by day and by student:
Number of times lecture recording folder was accessed per day
## min max median mean SD SEM n NAs sum
## 0.0 162.0 30.0 40.1 31.3 2.9 114.0 0.0 4570.0Number of students who accessed lecture recordings each day
## min max median mean SD SEM n NAs sum
## 0.0 43.0 13.0 14.9 9.9 0.9 114.0 0.0 1697.0Number of times each student accessed the lecture recordings
## min max median mean SD SEM n NAs sum
## 4.0 194.0 35.0 46.2 35.0 3.5 99.0 0.0 4570.0Number of days each student accessed the lecture recordings
## min max median mean SD SEM n NAs sum
## 3.0 41.0 16.0 17.1 8.5 0.9 99.0 0.0 1697.0Useful conclusions: 114 days (16 weeks, 2 days) in data for 99 consenting students (cohort 231)
Large range in the number of access hits (0, 22) recorded for each student each day. Overall, the number of access hits per day is 2-3x number of students who access per day, and number of access hits per student is also 2-3x the number of days a student access the folder.
Since we don’t really know how the number of folder openings is tracked by Blackboard (could be refreshings), the number of students is probably a better way of looking at the data than number of times the folder is ‘opened’.
On average 15 +/- 1 (mean+/-SEM) students accessed each day, with a max of 43 students one day (1/5/14).
On average students accessed lecture recordings on 17 days, with a max of 41 days and a minimum of 3 days. So there were no students who didn’t access lecture recordings at all?
## days.stud
## 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19 20 21 22 23 25 26 27 28 29
## 1 1 3 3 6 1 5 8 5 5 5 6 1 6 6 1 5 3 4 3 3 4 1 1 1
## 30 31 32 33 34 35 41
## 2 1 1 1 3 2 1So, only 1 student looked on three days… only 1 student looked on 41 days, the majority looked on 7-26 days. Or as a histogram:
Transposed data (lat) so we can get real dates…
## [1] 114 100## Date S8530605 S8636955 S8475915 S8645607
## 1 2014-03-04 0 1 3 2
## 2 2014-03-05 1 2 8 1
## 3 2014-03-06 0 4 0 0
## 4 2014-03-07 0 2 0 0
## 5 2014-03-08 0 2 0 0Summed numbers of times the lecture recording folder was accessed and number of students who accessed lecture recording folder per day…
Loaded calanderHeat function…
Calendar of number of times lecture recording folder was accessesed each day 
Calendar of number of students who accessed lecture recordings each day 
Cluster analysis
Built data frame with student ID and T/F for access each day… NB created 2 data frames: la.norm 0 = not accessed, 1 = accessed; la.norm2 1 = accessed, NA = not accessed (NA = missing value), but cluster analysis errors with NA => don’t use data with missing values for cluster analysis
Use la.norm to cluster leture recording access
distances = dist(la.norm[2:115], method = "euclidean")
clusterLA = hclust(distances, method = "ward")
plot(clusterLA)
clusterGroups5 = cutree(clusterLA, k = 5)
la.norm$cluster5 = clusterGroups5
dim(la.norm)## [1] 99 116la.norm[1:5,1:5]## StudentID X4.03.14 X5.03.14 X6.03.14 X7.03.14
## 1 S6089847 1 1 0 0
## 3 S8117889 0 1 1 0
## 4 S8118323 1 0 0 0
## 5 S8152093 0 0 0 0
## 6 S8239113 1 1 0 0la.norm[1:5,110:116]## X20.06.14 X21.06.14 X22.06.14 X23.06.14 X24.06.14 X25.06.14 cluster5
## 1 0 0 0 0 0 0 1
## 3 1 1 1 1 1 0 2
## 4 1 0 0 0 0 0 3
## 5 0 1 0 0 0 0 4
## 6 0 0 0 0 0 0 1## [1] "The number of students in each cluster by the the number of variables"## [1] 18 116
## [1] 15 116
## [1] 8 116
## [1] 37 116
## [1] 21 116So what are the characteristics of the clusters - how often do students view lecture recordings and when:
## min max median mean SD SEM n NAs sum
## 15.0 35.0 23.0 23.8 6.8 1.6 18.0 0.0 428.0
## min max median mean SD SEM n NAs sum
## 12.0 30.0 20.0 21.3 5.7 1.5 15.0 0.0 319.0
## min max median mean SD SEM n NAs sum
## 21.0 41.0 32.0 30.0 7.2 2.6 8.0 0.0 240.0
## min max median mean SD SEM n NAs sum
## 3.0 17.0 9.0 9.2 3.5 0.6 37.0 0.0 342.0
## min max median mean SD SEM n NAs sum
## 12.0 29.0 17.0 17.5 4.5 1.0 21.0 0.0 368.0
To see ‘when’ need to get cluster groups into lat (transposed version)
The run calendarHeat for all 5 clusters…
Load in qualitative coding “pattern of lecture recording use ML” -> “qual.csv”
clean - de-identify
clean - capitalisation, converted “no info”, “deferred” and “” to NA (ie missing)
Data structure
## [1] 99 10## StudentID ML1.previous ML2.planMS ML3.usedMS ML4.planEOS total.no
## 1 S6089847 No Yes No No 3
## 2 S8117889 No No No No 4
## 3 S8118323 Yes No No No 3
## 4 S8152093 Maybe No Maybe No 2
## 5 S8239113 Yes No Yes No 2
## total.yes total.maybe total.noinfo access
## 1 1 0 0 21
## 2 0 0 0 75
## 3 1 0 0 122
## 4 0 2 0 13
## 5 2 0 0 89## ML1.previous ML2.planMS ML3.usedMS ML4.planEOS
## Maybe:18 Maybe: 5 Maybe: 4 Maybe: 3
## No :52 No :77 No :69 No :71
## Yes :27 Yes :15 Yes :23 Yes :17
## NA's : 2 NA's : 2 NA's : 3 NA's : 8Patterns of self-reported lecture recording use
##
## Maybe No Yes
## 18 52 27
## [1] "ML1.previous"
##
## Maybe No Yes
## 5 77 15
## [1] "ML2.planMS"
##
## Maybe No Yes
## 4 69 23
## [1] "ML3.usedMS"
##
## Maybe No Yes
## 3 71 17
## [1] "ML4.planEOS"## ML2.planMS
## ML1.previous Maybe No Yes Sum
## Maybe 1 16 0 17
## No 2 40 9 51
## Yes 2 19 6 27
## Sum 5 75 15 95## ML3.usedMS
## ML2.planMS Maybe No Yes Sum
## Maybe 0 4 0 4
## No 4 57 14 75
## Yes 0 6 9 15
## Sum 4 67 23 94Concl:
Most students (52/99 (i.e. 53%)) report that they don’t usually use lecture recordings, even more didn’t plan to use lecture recordings for mid-semeter exam (77/99) and a similar number didn’t use lecture recordings for mid-semeter exam (69/99), and this was the same for the end of semester exam (71/99).
This seems inconsistent with the number of students who do use lecture recordings (all 99 at some point), and the majority used lecture recordings on 7-26 days, which is still half to twice the number of weeks in semester so ~ once/fortnight to twice/week.
What are the patterns of No, No, No, No etc, similar to what Kay calculated as number of No’s, Yes’, Maybe’s (tables have the number of no’s 0-4 in header row, then frequency (number of students) in 2nd row)
##
## 0 1 2 3 4
## 2 15 21 32 29
## [1] "total.no"
##
## 0 1 2 3 4
## 52 27 7 11 2
## [1] "total.yes"
##
## 0 1 2
## 72 24 3
## [1] "total.maybe"Most frequent patterns of repsonse:
##
## No Yes Yes Yes Yes No No Yes Yes Yes Yes No No Yes No No Yes No Yes Yes
## 3 3 3 4 4
## Yes No No No Maybe No No No No No No No
## 8 9 29everything else was reported by 2 or less students.
So there is definitely a group of 29 students who never report using lecture recordings (LR). There are 27 students who report that they usually used LR.
Of these, 6 plan to use LR for mid-sem, 2 maybes, and 19 don’t mention LR for mid-sem prep. There are 52 (51?) students who don’t report usually using LR. Of these, 9 plan to use LR for mid-sem, 2 maybes, and 40 don’t mention LR for mid-sem prep.
How often do these groups of students use LR?
## [1] 99 116## StudentID X4.03.14 X5.03.14 X6.03.14 X7.03.14
## 1 S6089847 1 1 0 0
## 3 S8117889 0 1 1 0
## 4 S8118323 1 0 0 0
## 5 S8152093 0 0 0 0
## 6 S8239113 1 1 0 0## X20.06.14 X21.06.14 X22.06.14 X23.06.14 X24.06.14 X25.06.14 cluster5
## 1 0 0 0 0 0 0 1
## 3 1 1 1 1 1 0 2
## 4 1 0 0 0 0 0 3
## 5 0 1 0 0 0 0 4
## 6 0 0 0 0 0 0 1## [1] 99 126## StudentID X4.03.14 X5.03.14 X6.03.14 X7.03.14
## 1 S6089847 1 1 0 0
## 2 S8117889 0 1 1 0
## 3 S8118323 1 0 0 0
## 4 S8152093 0 0 0 0
## 5 S8239113 1 1 0 0## X25.06.14 cluster5 ML1.previous ML2.planMS ML3.usedMS ML4.planEOS
## 1 0 1 No Yes No No
## 2 0 2 No No No No
## 3 0 3 Yes No No No
## 4 0 4 Maybe No Maybe No
## 5 0 1 Yes No Yes No
## total.no total.yes total.maybe total.noinfo access pattern
## 1 3 1 0 0 21 No Yes No No
## 2 4 0 0 0 75 No No No No
## 3 3 1 0 0 122 Yes No No No
## 4 2 0 2 0 13 Maybe No Maybe No
## 5 2 2 0 0 89 Yes No Yes No## [1] 99 128## X25.06.14 cluster5 ML1.previous ML2.planMS ML3.usedMS ML4.planEOS
## 1 0 1 No Yes No No
## 2 0 2 No No No No
## 3 0 3 Yes No No No
## 4 0 4 Maybe No Maybe No
## 5 0 1 Yes No Yes No
## total.no total.yes total.maybe total.noinfo access pattern
## 1 3 1 0 0 21 No Yes No No
## 2 4 0 0 0 75 No No No No
## 3 3 1 0 0 122 Yes No No No
## 4 2 0 2 0 13 Maybe No Maybe No
## 5 2 2 0 0 89 Yes No Yes No
## prevLR access.days
## 1 No 15
## 2 No 26
## 3 Yes 33
## 4 Yes 10
## 5 Yes 28Statistical tests:
Wilcox (ie unpaired t test for categorical data)
Do students who report usually using LR, access more LR? First as number of folder openings, then as number of days. (order is test, mean, sem)
##
## Wilcoxon rank sum test with continuity correction
##
## data: access by prevLR
## W = 739, p-value = 0.00184
## alternative hypothesis: true location shift is not equal to 0## No Yes
## 37.9 56.2## [1] 4.796
## [1] 5.033## No Yes
## 4.8 5.0##
## Wilcoxon rank sum test with continuity correction
##
## data: access.days by prevLR
## W = 790, p-value = 0.005989
## alternative hypothesis: true location shift is not equal to 0## No Yes
## 14.96 19.82## [1] 1.078
## [1] 1.312## No Yes
## 1.078 1.312Do students who report usually using LR, fall into different clusters? (order is test, table, mean, sem)
##
## Wilcoxon rank sum test with continuity correction
##
## data: cluster5 by prevLR
## W = 1470, p-value = 0.02436
## alternative hypothesis: true location shift is not equal to 0## cluster5
## prevLR 1 2 3 4 5 Sum
## No 6 6 3 24 13 52
## Yes 11 9 5 12 8 45
## Sum 17 15 8 36 21 97## No Yes
## 3.62 2.93## [1] 0.1804
## [1] 0.2211## No Yes
## 0.18 0.22When do they use LR?
Dates were:
ML 2 9/04/14 - 16/04/14
MS exam 3/05/14
ML 3 7/05/14 - 14/05/14
Kay’s patterns
1 Previous and did >3 y yyyy ynyy yyny yyyn ynyn 2 previous/intended, but did not ynnn yynn ynny
3 No previous use, but then did or intended nnyy nyyy nnny
4 No previous use, intention but not nyny nynn
5 No report nnnn
0 Don’t fit?
Load “Kay.gp.index.csv”
clean - de-identified
merge into la.norm.qual
## [1] 99 2## pattern prevLR access.days Kay.pattern
## 1 No Yes No No No 15 4
## 2 No No No No No 26 5
## 3 Yes No No No Yes 33 2
## 4 Maybe No Maybe No Yes 10 0
## 5 Yes No Yes No Yes 28 1Alignment Kay gp with cluster5
## Kay.pattern
## cluster5 0 1 2 3 4 5
## 1 2 5 1 1 1 8
## 2 1 5 3 4 1 1
## 3 1 0 4 2 0 1
## 4 10 1 3 2 0 21
## 5 1 3 1 2 3 11