TPACK Context Survey
Joshua Rosenberg
January 6, 2016
1. Introduction.
Since the introduction of the TPACK framework, interest in its conceptualization, development, and assessment has grown. Measures for TPACK range from interviews, questionnaires, observation protocols, and performance assessments to surveys (Koehler, Shin, & Mishra, 2011). Early in the development of TPACK, there was an absence of valid and reliable survey instruments for the measurement of TPACK. As a result, Schmidt et al. developed and piloted a survey instrument on 124 preservice teachers.
While this validated measure, the Survey of Preservice Teachers’ Knowledge of Teaching and Technology (Schmidt et al., 2009) has become widely used among TPACK research and development, the survey does not include context, which is a critical gap that needs to be addressed in order to advance efforts to measure the knowledge teachers need to teach with technology. The objective of this study, then, is to develop a measure of teacher knowledge of context to be used alongside the Survey of Preservice Teachers’ Knowledge of Teaching and Technology.
This measure builds on prior conceptual and empirical research on context and TPACK (e.g., Porras-Hernandez & Salinas-Amescua, 2013; Rosenberg & Koehler, 2015). Specifically, this measure examines five aspects of teachers’ context: Micro, Meso, Macro, Teacher, and Student. Items were developed through concept mapping each of these aspects, pilot testing items with a small group of graduate students, and receiving expert feedback. The specific items used to measure these, along with four items related to teachers’ TPACK, follow:
Micro:
-The resources in my classroom or learning environment enable me to provide all, and not just some, students with access to technology. -The behavioral norms in my classroom or learning environment help me to use technology to support student learning rather than serve as a distraction. -The technology in my classroom or learning environment allows me to meet students’ individual needs. -Technological resources are available in my classroom or learning environment allow me to design learning activities that otherwise would not be possible.
Meso:
-I have adequate technical support in my school or institution to help me manage technology. -The technical infrastructure (such as Wi-Fi networks or learning management systems) in my school or institution in allows me to integrate technology into my teaching. -I think there is sufficient funding in my school or institution to provide access to technologies for all students. -Within the culture of my school or institution, technology use is seen as a way for teachers to strengthen their teaching.
Macro:
-I know who to ask in my district or organization about professional development opportunities related to technology -The policies and guidelines in my district or organization are flexible enough to allow me to effectively use all of the technological resources available to me. -When new technologies become available, my district or organization provides me with adequate support to help me use them. -There are ways for me to communicate with administrators in my district or organization about new technologies that I think would enrich my teaching.
Teacher:
-I find it easy to incorporate new technology, or existing technology in new ways, into my teaching. -I think it is important to use current and innovative technologies to enhance my teaching practices. -Even when they are challenging to use, I find that new technological resources enable me to enrich student learning. -I think it is important to regularly update my existing lessons to incorporate technology in new ways.
Student:
-I take students’ interests into account when I select technologies to use in my teaching. -I understand what technological resources my students are familiar with and have access to. -I know what my students can do with different types of technologies. -I consider my students’ backgrounds with technology to ensure that my lessons are accessible to them.
TPACK: -I can teach in a way that integrates my curriculum, technologies, and teaching and learning strategies. -I know how to use technology to teach content that departs from textbook knowledge. -I can select technologies that improve what I teach and how I teach. -I think the lessons I teach appropriately combine the curriculum, technologies, and teaching approaches.
2. Field Test
The survey was revised after its initial development and then field tested with teachers enrolled in the Master of Arts in Educational Technology (MAET) program at Michigan State University. Participants were solicited via email from the program director and were not compensated. 50 students completed the survey.
## 'data.frame': 68 obs. of 63 variables:
## $ V1 : chr "R_3PtoROA3A2qqwRe" "R_dpArJh5hHgDSmEV" "R_2aVSIXwETcUWwAH" "R_BSAbQHeE9aOnJWp" ...
## $ V2 : chr "Default Response Set" "Default Response Set" "Default Response Set" "Default Response Set" ...
## $ V3 : chr "Anonymous" "Anonymous" "Anonymous" "Anonymous" ...
## $ V4 : chr NA NA NA NA ...
## $ V5 : chr NA NA NA NA ...
## $ V6 : chr "198.199.134.100" "63.199.41.5" "35.10.40.162" "86.141.114.179" ...
## $ V7 : chr "0" "0" "0" "0" ...
## $ V8 : chr "2015-08-04 10:51:31" "2015-08-04 10:58:56" "2015-08-04 11:15:57" "2015-08-04 12:32:31" ...
## $ V9 : chr "2015-08-04 10:54:48" "2015-08-04 11:08:12" "2015-08-04 11:21:12" "2015-08-04 12:38:23" ...
## $ V10 : chr "1" "1" "1" "1" ...
## $ Q1 : chr "1" "1" "1" "1" ...
## $ Q51_1 : chr "5" "4" "4" "4" ...
## $ Q51_2 : chr "5" "5" "5" "4" ...
## $ Q51_3 : chr "5" "2" "5" "3" ...
## $ Q51_4 : chr "5" "5" "5" "5" ...
## $ Q51_5 : chr "5" "5" "5" "4" ...
## $ Q51_6 : chr "5" "4" "5" "4" ...
## $ Q51_7 : chr "5" "2" "4" "1" ...
## $ Q51_14 : chr "5" "5" "4" "4" ...
## $ Q56_1 : chr "5" "2" "4" "2" ...
## $ Q56_2 : chr "5" "4" "5" "4" ...
## $ Q56_3 : chr "5" "4" "4" "5" ...
## $ Q56_4 : chr "5" "4" "5" "2" ...
## $ Q56_5 : chr "5" "5" "5" "5" ...
## $ Q56_7 : chr "5" "3" "5" "2" ...
## $ Q56_8 : chr "5" "4" "5" "5" ...
## $ Q56_13 : chr "5" "4" "5" "4" ...
## $ Q60_7 : chr "5" "4" "5" "4" ...
## $ Q60_13 : chr "5" "1" "4" "1" ...
## $ Q60_19 : chr "5" "4" "5" "4" ...
## $ Q60_25 : chr "5" "2" "5" "5" ...
## $ Q60_31 : chr "4" "2" "5" "1" ...
## $ Q60_37 : chr "5" "3" "5" "5" ...
## $ Q60_43 : chr "5" "3" "5" "5" ...
## $ Q60_44 : chr "5" "4" "5" "5" ...
## $ Q36_1 : chr NA NA NA NA ...
## $ Q36_2 : chr NA NA NA "1" ...
## $ Q36_3 : chr NA NA NA NA ...
## $ Q36_4 : chr NA "1" NA NA ...
## $ Q36_5 : chr NA NA NA NA ...
## $ Q36_6 : chr "1" NA "1" NA ...
## $ Q36_6_TEXT : chr "Professional Development" NA "educational technology coach" NA ...
## $ Q36_7 : chr NA NA NA NA ...
## $ Q36_9 : chr NA NA NA NA ...
## $ Q42 : chr "1" "4" "1" "4" ...
## $ Q44 : chr "4" "6" "8" "7" ...
## $ Q53_1 : chr NA NA NA "1" ...
## $ Q53_2 : chr NA "1" NA "1" ...
## $ Q53_3 : chr NA NA NA "1" ...
## $ Q53_4 : chr NA NA NA "1" ...
## $ Q53_5 : chr NA NA NA NA ...
## $ Q53_6 : chr NA NA NA NA ...
## $ Q53_7 : chr NA NA NA NA ...
## $ Q53_8 : chr NA NA NA NA ...
## $ Q53_9 : chr "1" NA NA NA ...
## $ Q53_9_TEXT : chr "Professional Development" NA NA NA ...
## $ Q38 : chr "1" "1" "1" "2" ...
## $ Q38_TEXT : chr NA NA NA NA ...
## $ Q50 : chr NA NA NA "My district unilaterally decided not to support 1:1 programs, so I purchased devices for use in my classroom. Because ours is n"| __truncated__ ...
## $ LocationLatitude : chr "39.95719909668" "38.169906616211" "42.728302001953" "52" ...
## $ LocationLongitude: chr "-74.916198730469" "-121.93669891357" "-84.48819732666" "-0.98330688476562" ...
## $ LocationAccuracy : chr "-1" "-1" "-1" "-1" ...
## $ X : logi NA NA NA NA NA NA ...Descriptive statistics.
# Descriptives
micro <- dplyr::select(data_ss, Q3.25, Q2.3, Q3.43, Q3.37)
meso <- dplyr::select(data_ss, Q3.31, Q1.3, Q3.13, Q2.4)
macro <- dplyr::select(data_ss, Q3.7, Q2.7, Q1.7, Q2.1)
teacher <- dplyr::select(data_ss, Q2.2, Q2.5, Q1.6, Q3.44)
student <- dplyr::select(data_ss, Q1.2, Q2.13, Q1.14, Q2.8)
tpack <- dplyr::select(data_ss, Q1.1, Q1.5, Q1.4, Q3.19)
micro_scale <- (micro[, 1] + micro[, 2] + micro[, 3] + micro[, 4]) / 4
meso_scale <- (meso[, 1] + meso[, 2] + meso[, 3] + meso[, 4]) / 4
macro_scale <- (macro[, 1] + macro[, 2] + macro[, 3] + macro[, 4]) / 4
teacher_scale <- (teacher[, 1] + teacher[, 2] + teacher[, 3] + teacher[, 4]) / 4
student_scale <- (student[, 1] + student[, 2] + student[, 3] + student[, 4]) / 4
tpack_scale <- (tpack[, 1] + tpack[, 2] + tpack[, 3] + tpack[, 4]) / 4
psych::describe(micro_scale)## vars n mean sd median trimmed mad min max range skew kurtosis se
## 1 1 54 3.73 0.95 3.88 3.77 1.3 2 5 3 -0.26 -1.21 0.13hist(micro_scale)
psych::describe(meso_scale)## vars n mean sd median trimmed mad min max range skew kurtosis se
## 1 1 54 3.23 1.03 3.25 3.22 1.48 1.5 5 3.5 0.08 -1.26 0.14hist(meso_scale)
psych::describe(macro_scale)## vars n mean sd median trimmed mad min max range skew kurtosis se
## 1 1 54 3.63 0.8 3.75 3.64 1.11 2.25 5 2.75 -0.04 -1.13 0.11hist(macro_scale)
psych::describe(teacher_scale)## vars n mean sd median trimmed mad min max range skew kurtosis se
## 1 1 54 4.29 0.55 4.25 4.34 0.37 2.5 5 2.5 -0.92 1.07 0.07hist(teacher_scale)
psych::describe(student_scale)## vars n mean sd median trimmed mad min max range skew kurtosis se
## 1 1 54 4.24 0.48 4.25 4.27 0.37 2.75 5 2.25 -0.69 0.44 0.06hist(student_scale)
psych::describe(tpack_scale)## vars n mean sd median trimmed mad min max range skew kurtosis se
## 1 1 53 4.37 0.52 4.5 4.44 0.37 3 5 2 -0.87 0.3 0.07hist(tpack_scale)
Reliabilities.
micro_reliability <- psych::alpha(as.matrix(micro))
meso_reliability <- psych::alpha(as.matrix(meso))
macro_reliability <- psych::alpha(as.matrix(macro))
teacher_reliability <- psych::alpha(as.matrix(teacher))
student_reliability <- psych::alpha(as.matrix(student))
tpack_reliability <- psych::alpha(as.matrix(tpack))
print(micro_reliability)##
## Reliability analysis
## Call: psych::alpha(x = as.matrix(micro))
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd
## 0.85 0.85 0.84 0.59 5.7 0.076 3.7 0.95
##
## lower alpha upper 95% confidence boundaries
## 0.7 0.85 1
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se
## Q3.25 0.76 0.76 0.71 0.51 3.2 0.114
## Q2.3 0.89 0.89 0.85 0.73 8.2 0.093
## Q3.43 0.76 0.76 0.72 0.52 3.2 0.112
## Q3.37 0.80 0.81 0.78 0.59 4.3 0.106
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## Q3.25 54 0.92 0.90 0.88 0.82 3.5 1.40
## Q2.3 54 0.65 0.70 0.53 0.49 3.9 0.83
## Q3.43 54 0.90 0.90 0.87 0.81 3.7 1.07
## Q3.37 54 0.85 0.83 0.76 0.71 3.8 1.18
##
## Non missing response frequency for each item
## 1 2 3 4 5 miss
## Q3.25 0.07 0.28 0.06 0.26 0.33 0.21
## Q2.3 0.00 0.07 0.17 0.54 0.22 0.21
## Q3.43 0.00 0.17 0.22 0.31 0.30 0.21
## Q3.37 0.04 0.17 0.09 0.39 0.31 0.21print(meso_reliability)##
## Reliability analysis
## Call: psych::alpha(x = as.matrix(meso))
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd
## 0.81 0.81 0.78 0.52 4.4 0.085 3.2 1
##
## lower alpha upper 95% confidence boundaries
## 0.65 0.81 0.98
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se
## Q3.31 0.72 0.72 0.64 0.46 2.6 0.12
## Q1.3 0.74 0.74 0.68 0.49 2.9 0.12
## Q3.13 0.79 0.79 0.72 0.55 3.7 0.11
## Q2.4 0.81 0.81 0.74 0.59 4.2 0.11
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## Q3.31 54 0.85 0.86 0.81 0.73 3.0 1.2
## Q1.3 56 0.82 0.83 0.75 0.68 3.6 1.3
## Q3.13 54 0.79 0.77 0.66 0.59 2.6 1.4
## Q2.4 54 0.73 0.74 0.60 0.54 3.8 1.2
##
## Non missing response frequency for each item
## 1 2 3 4 5 miss
## Q3.31 0.11 0.31 0.15 0.31 0.11 0.21
## Q1.3 0.04 0.25 0.14 0.27 0.30 0.18
## Q3.13 0.30 0.26 0.11 0.20 0.13 0.21
## Q2.4 0.02 0.20 0.15 0.26 0.37 0.21print(macro_reliability)##
## Reliability analysis
## Call: psych::alpha(x = as.matrix(macro))
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd
## 0.74 0.74 0.71 0.41 2.8 0.096 3.6 0.82
##
## lower alpha upper 95% confidence boundaries
## 0.55 0.74 0.93
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se
## Q3.7 0.70 0.70 0.61 0.44 2.4 0.13
## Q2.7 0.74 0.74 0.65 0.48 2.8 0.12
## Q1.7 0.63 0.63 0.56 0.36 1.7 0.14
## Q2.1 0.65 0.64 0.59 0.37 1.8 0.13
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## Q3.7 54 0.73 0.72 0.60 0.51 4.0 1.08
## Q2.7 54 0.65 0.68 0.52 0.43 4.0 0.94
## Q1.7 56 0.80 0.80 0.72 0.62 2.8 1.19
## Q2.1 54 0.78 0.79 0.68 0.59 3.7 1.04
##
## Non missing response frequency for each item
## 1 2 3 4 5 miss
## Q3.7 0.02 0.11 0.11 0.33 0.43 0.21
## Q2.7 0.00 0.09 0.15 0.41 0.35 0.21
## Q1.7 0.14 0.34 0.23 0.20 0.09 0.18
## Q2.1 0.00 0.22 0.04 0.54 0.20 0.21print(teacher_reliability)##
## Reliability analysis
## Call: psych::alpha(x = as.matrix(teacher))
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd
## 0.74 0.75 0.7 0.43 3 0.096 4.3 0.55
##
## lower alpha upper 95% confidence boundaries
## 0.56 0.74 0.93
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se
## Q2.2 0.63 0.64 0.55 0.37 1.8 0.14
## Q2.5 0.74 0.75 0.68 0.50 3.1 0.12
## Q1.6 0.67 0.68 0.61 0.42 2.2 0.13
## Q3.44 0.67 0.68 0.60 0.42 2.1 0.13
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## Q2.2 54 0.85 0.81 0.74 0.64 4.0 0.91
## Q2.5 54 0.61 0.68 0.49 0.43 4.5 0.50
## Q1.6 56 0.79 0.76 0.65 0.57 4.2 0.81
## Q3.44 54 0.75 0.77 0.66 0.58 4.4 0.63
##
## Non missing response frequency for each item
## 1 2 3 4 5 miss
## Q2.2 0.02 0.07 0.09 0.56 0.26 0.21
## Q2.5 0.00 0.00 0.00 0.46 0.54 0.21
## Q1.6 0.00 0.05 0.07 0.46 0.41 0.18
## Q3.44 0.00 0.00 0.07 0.43 0.50 0.21print(student_reliability)##
## Reliability analysis
## Call: psych::alpha(x = as.matrix(student))
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd
## 0.64 0.65 0.6 0.32 1.9 0.11 4.2 0.47
##
## lower alpha upper 95% confidence boundaries
## 0.41 0.64 0.86
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se
## Q1.2 0.56 0.59 0.51 0.32 1.4 0.15
## Q2.13 0.50 0.51 0.41 0.26 1.0 0.16
## Q1.14 0.66 0.67 0.58 0.40 2.0 0.13
## Q2.8 0.55 0.56 0.47 0.30 1.3 0.15
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## Q1.2 56 0.70 0.70 0.53 0.42 4.3 0.71
## Q2.13 54 0.73 0.77 0.67 0.53 4.4 0.60
## Q1.14 56 0.66 0.61 0.38 0.31 3.8 0.79
## Q2.8 54 0.69 0.72 0.58 0.45 4.5 0.64
##
## Non missing response frequency for each item
## 2 3 4 5 miss
## Q1.2 0.04 0.04 0.52 0.41 0.18
## Q2.13 0.00 0.06 0.50 0.44 0.21
## Q1.14 0.07 0.23 0.55 0.14 0.18
## Q2.8 0.02 0.02 0.41 0.56 0.21print(tpack_reliability)##
## Reliability analysis
## Call: psych::alpha(x = as.matrix(tpack))
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd
## 0.76 0.75 0.73 0.43 3.1 0.092 4.4 0.52
##
## lower alpha upper 95% confidence boundaries
## 0.58 0.76 0.94
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se
## Q1.1 0.77 0.77 0.72 0.53 3.4 0.11
## Q1.5 0.63 0.63 0.56 0.37 1.7 0.14
## Q1.4 0.75 0.75 0.70 0.49 2.9 0.11
## Q3.19 0.61 0.61 0.52 0.34 1.6 0.14
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## Q1.1 56 0.62 0.66 0.46 0.41 4.4 0.54
## Q1.5 56 0.85 0.82 0.78 0.69 4.5 0.74
## Q1.4 56 0.67 0.70 0.52 0.47 4.5 0.57
## Q3.19 53 0.89 0.85 0.82 0.72 4.1 0.85
##
## Non missing response frequency for each item
## 2 3 4 5 miss
## Q1.1 0.00 0.02 0.52 0.46 0.18
## Q1.5 0.02 0.09 0.30 0.59 0.18
## Q1.4 0.00 0.04 0.45 0.52 0.18
## Q3.19 0.09 0.04 0.57 0.30 0.22Correlations.
corr_matrix <- cbind(micro_scale, meso_scale, macro_scale, teacher_scale, student_scale, tpack_scale)
print(Hmisc::rcorr(corr_matrix))## micro_scale meso_scale macro_scale teacher_scale
## micro_scale 1.00 0.57 0.52 0.56
## meso_scale 0.57 1.00 0.75 0.45
## macro_scale 0.52 0.75 1.00 0.45
## teacher_scale 0.56 0.45 0.45 1.00
## student_scale 0.60 0.44 0.60 0.56
## tpack_scale 0.51 0.31 0.46 0.65
## student_scale tpack_scale
## micro_scale 0.60 0.51
## meso_scale 0.44 0.31
## macro_scale 0.60 0.46
## teacher_scale 0.56 0.65
## student_scale 1.00 0.72
## tpack_scale 0.72 1.00
##
## n
## micro_scale meso_scale macro_scale teacher_scale
## micro_scale 54 54 54 54
## meso_scale 54 54 54 54
## macro_scale 54 54 54 54
## teacher_scale 54 54 54 54
## student_scale 54 54 54 54
## tpack_scale 53 53 53 53
## student_scale tpack_scale
## micro_scale 54 53
## meso_scale 54 53
## macro_scale 54 53
## teacher_scale 54 53
## student_scale 54 53
## tpack_scale 53 53
##
## P
## micro_scale meso_scale macro_scale teacher_scale
## micro_scale 0.0000 0.0000 0.0000
## meso_scale 0.0000 0.0000 0.0006
## macro_scale 0.0000 0.0000 0.0007
## teacher_scale 0.0000 0.0006 0.0007
## student_scale 0.0000 0.0008 0.0000 0.0000
## tpack_scale 0.0000 0.0259 0.0006 0.0000
## student_scale tpack_scale
## micro_scale 0.0000 0.0000
## meso_scale 0.0008 0.0259
## macro_scale 0.0000 0.0006
## teacher_scale 0.0000 0.0000
## student_scale 0.0000
## tpack_scale 0.0000- Next Steps