Effects of Metacoginiton and Motivation on Reading Performance: An Multilevel Approach

Summary

Self-regulated learning (SRL) includes the cognitive, meta-cognitive, behavioral, motivational, and emotional/effective aspects of learning. This conceptual framework has made a major contribution to educational psychology. The purpose of this study will be to explore the influences of SRL strategies and motivational construct on reading performance in countries of North-America using the information provided by the Program for International Student Assessment (PISA) 2018. PISA is an international assessment conducted by Organization for Economic Co-operation and Development (OECD) that measures 15-year-old students’ reading, mathematics, and science literacy every three years since 2000. This large-scale study also includes a wide range of students’ motivations, attitudes, beliefs, and behaviors as predictors of scholastic performance, educational attainment, and labor market success. The hierarchical linear model approach was used to analyze the relationships between self-regulated learning and students’ performance in reading.

1. Introduction

Large-scale assessments of student performance can provide a window into broadly defined concepts of student achievement about some of the correlates of learning—such as student background, attitudes, and perceptions—and perhaps school and home characteristics. In that sense, the Programme for International Student Assessment (PISA) is a one-large-scale, internationally representative dataset that assesses s a broad range of competencies relevant to coping with the adult life of a 15-year-old-students.

PISA is administered in a 3-year cycle, measuring student performance in reading, mathematics, and science with some attention paid to problem-solving. Furthermore, information about student attitudes and perceptions related to schooling, home background variables, and instructional and teaching practices is collected via a questionnaire administered to students, teachers, and principals. These datasets offer the opportunity to investigate relationships among the correlates of learning and achievement and do so from an internationally comparative perspective.

PISA provides useful information for stakeholders in the educational sector. It produces high-quality information, such as a rigorous sampling design, well-developed objective measures of student achievement, and data collection related to student and school traits. Thus, PISA meets the standards required for conducting national and international comparative studies. In addition, the data are hierarchical in structure; students are nested within schools within countries; which is not uncommon for educational datasets, meaning that multilevel modeling (e.g., HLM) is a suitable analytic approach.

The PISA 2018 assessement focused on reading literacy from the perspective that students can use the knowledge and skills they have learned and praticed at school when presented with situations in which knowledge is relevant. PISA defines reading literacy as: understanding, using, reflecting on and engaging with written texts, in order to achieve one’s goals, develop one’s knowledge and potential, and participate in. society. The overall aim of the present study is to use an international large-scale representative sample of to investigate the relationship between meta-cognitive and motivational variables on reading literacy student’s performance using a quantitative, cross-sectional, and correlational approach.

1.1 Relationship between meta-cognitive and motivation with academic achievement

Empirical studies showed that metacognition is significantly correlated with reading proficiency and is responsive to quality of teaching and learning outcomes (Artelt, Schiefele and Schneider, 2001; Brown, Palincsar and Armbruster, 2004). Some metacognitive reading strategies studied in recent research, including adapting ones reading strategies depending on these goals, knowing how to summarize a piece of text or remember essential information, monitoring comprehension and knowing how to repair comprehension problems.

Achievement motivation is another central element to education and learning. An important aspect regarding academic motivation is the distinctions between learning and performance goals (Schunk, 1996; Schunk & Swartz, 1993a, 1993b). According to literature one important characteristic of optimal learners is that they are focused on improvement in the classroom or pursue mastery-approach goals. Some studies suggest that students who adopt mastery-approach goals have been shown to engage in deep learning, to persist upon failure and to show high levels of intrinsic motivation (Hulleman et al., 2010; Kaplan and Maehr, 2007; Middleton and Perks, 2014, cited by OCDE 2019).

Other important construct is self-concept. Empirical studies showed that general correlation between academic achievement measures (grade point averages) and measures of self-concept was 0.30, which is a moderate and positive relation. The highest correlations with achievement have been found with domain-specific self-concepts (e.g., in areas such as English or mathematics. Schunk & Pajares, 2009).

1.2 Present study

Using hierarchical linear models, this project examined the student- and school-level factors correlate to reading students’ literacy across countries in North America, with a particular focus on students’ meta cognitive strategies, academic self-concept, and dispositional attitudes toward reading while controlling for students and schools socio-economic and demographic characteristics. The current analysis addresses the following research questions:

1. Are any structural features of students, such as socioeconomic status, gender, or previous academic background, that explain the variability on reading score? If so, are these effects similar between school?

2. Are any structural features of schools, such as sector, area, socioeconomic status, or teaching quality, that explain the variability on reading score?

3. What are the effects of student’s meta-cognitive strategies and motivation on reading performance scores? Are these effects similar between school?

• Does the effect of meta-cognitive strategies and motivation on reading performance similar across schools?

• Does the effect of meta-cognitive strategies and motivation similar based on students’ gender and socioeconomic status?

With respect to these questions we hypothesized that:

Hypothesis 1: There are a direct and positive effect of socioeconomic status on reading performance, thus students enrolled in schools with disadvantages socioeconomic status obtained lower scores in reading performance.

Hypothesis 2: The effect of SES on reading performance is moderate by students’ meta-cognitive strategies and motivation. Thus, students with higher socioeconomic status have a positive a higher effect on reading performance than those with disadvantage in socioeconomic status. However the magnitug of SES effect is lower than the effect observed before controlled by meta-cognitive strategies, dispositional attitudes, and self-concept.

Hypothesis 3: The effect of meta-cognitive strategies and motivation on reading performance is similar across school after controlling by SES.

Hypothesis 4: The effect of meta-cognitive strategies and motivation on reading performance depend on student gender.

1. Students with economic disadvantage (ECONDISAD): Proportion of students with economic disadvantage, with take four levels (> 50% economically disadvantaged, 26 − 50% economically disadvantaged, 11 − 25% economically disadvantaged, and 10% economically disadvantaged). Students classified as economic disadvantages have a ESCS lower than the mean of average of socioeconomic status in their countries.

2. School Socioeconomic Status (SESCS) continuous variable calculated as the average of students’ Index of Socioeconomic Status at school level.

3. Students with reading disadvantage (REDISAD): Proportion of students with reading performance score lower than the cut-point for achieve the proficiency level 2; with take four levels (> 50% performance below Level 2, 26 − 50% performance below Level 2, 11 − 25% performance below Level 2, and < 10% performance below Level 2).

2 Methods

2.1 Sample

The current analysis is based on the PISA 2018 dataset with the full responses from individual students and school principals. PISA measures 15-year-olds’ ability to use their reading, mathematics, and science knowledge and skills to meet real-life challenges. In 2018, 612204 students participated in PISA, representing about 32 million 15-year-olds in the schools of the 79 participating countries and economies.

The analysis uses the results for the United States of America, Canada, and Mexico, where 34790 students from 1271 schools were randomly selected to represent their countries in this study. PISA Sampling is conducted in two stages, where schools are selected in the first stage and students in the second stage. The dataset contains sampling variables and plausible values of achievement scores to account for the complex sampling and test design, respectively. The sample means age was 15.84 years (SD = 0.285), and 50.33% were girls. Analysis was conducted using complete cases, then a total of 21683 observations were include in final dataset.

2.2 Measures

2.2.1 Measures for Student (Level 1)

Reading literacy student’s performance.

Reading performance, for PISA, is defined as a student’s capacity to understand, use and reflect on written texts to achieve a goal, develop knowledge and potential, and participate in society (OCDE, 2000). In 2018, the major domain was reading literacy, then about half of the assessment was devoted to reading literacy items. The metric for the overall reading scale is based on a mean for participating OECD countries set at 500, with a standard deviation of 100. Scores below 407.47 points are classified as low performance according to PISA criteria, while scores upper than 625.61 are considered as a higher proficiency level.

Demographic and socio-economic background.

The index of economic, social, and cultural status (ESCS) is a composite score based on education, occupational status, and income. In PISA 2018, the ESCS was constructed as the arithmetic mean of the three indicators after their imputation and standardization. The ESCS scale has a mean of 0 and a standard deviation of 1 when values less than 0 imply means an ESCS lower than the OCDE average. Students educational background was measured through a repetition index (REPEAT)that take the value of 1 if the student had repeated a grade in at least one ISCED level and the value of 0 if “no, never” was chosen at least once, given that none of the repeated grade categories were chosen. Furthermore, were included demographic variables such as students’ gender (GENDER) and age (AGE).

Meta-cognitive strategies.

PISA 2018 assessed students’ metacognitive strategies by asking students how useful they thought two reading strategies – summarizing a piece of text (UNDREM) and understanding and memorizing a piece of text (METASUM), Moreover, metacognition digital reading was measured through the scale of quality and the credibility of sources (METASPAM).

Students were asked to rate the strategies regarding their usefulness for solving the reading task. All strategies were rated by reading experts regarding their usefulness via multiple pairwise comparisons. This rating resulted in a hierarchy of all strategies for each task, and it was based on all the pairs agreed upon by at least 80% of the experts. The final scores assigned to each student for each task ranges from 0 to 1 and can be interpreted as the proportion of the total number of experts pairwise relations that are consistent with the student order. The higher the score, the higher the number of times in which a student chose an expert-validated strategy over a less useful one.

Students’ Motivation

Reading motivation and engagement was measured through several constructs. First, self-concept, defined as a general measure of an individual’s own perceived abilities in a specific domain, was measured through two aspects: perception of competence (SCREADCOMP) and perception of difficulty (SCREADDIFF) when performing reading tasks. Second, Intrinsic motivation of reading was measured through enjoyment of reading index (JOYREAD). Third, achievement motivation was measured by two scales: attitudes towards learning activities (ATTLNACT), and work mastery (WORKMAST), defined as the dispositional desire to work hard to master a task. Finally, achievement goals were measured using scales: the scale of students’ general fear of failure (GFOFAIL) and students’ mastery-approach orientation (MASTGOAL).

All students’ motivation scales are measured through four-point Likert items (“Strongly disagree” to “Strongly agree), except for MASTGOAL, which used five-point Likert items (”Not at all true of me” to “Extremely true of me”). Furthermore, scales were constructed using the IRT (item response theory) scaling methodology. Finally, Table 1 shows the item wording in scales used in the current analysis.

Table 1. Items for Meta-cognitive and motivational scales

2.2.2 Measures for Schools (Level 2)

School Characteristics

Two school variables, extracted from school data set, were included in the analysis: Type of School (PRIVATESCH), with takes two values (Public or Private), and characteristics of community where school is located (SCH_COM) with takes two values (Rural area or small town, and City or Large City).

Additionally, three variables were calculated using student data aggregation by school:

1. Students with economic disadvantage (ECONDISAD): Proportion of students with economic disadvantage, with take four levels (1 => 50% economically disadvantaged, 2 = 26−50% economically disadvantaged, 3 = 11 − 25% economically disadvantaged, and 4 = 10% economically disadvantaged). Students classified as economic disadvantages have a ESCS lower than the mean of average of socioeconomic status in their countries.

2. School Socioeconomic Status (SISCED) continuous variable calculated as the average of students’ Index of Socioeconomic Status at school level.

3. Students with reading disadvantage (REDISAD): Proportion of students with reading performance score lower than the cut-point in the proficiency level 2; with take four levels ( 1 = >50% performance below Level 2, 2 = 26−50% performance below Level 2, 3 = 11−25% performance below Level 2, and 4 = <10% performance below Level 2).

Quality of Teaching

Quality of teaching is defined as the proportion of fully certified teachers (PROATCE), that was computed by dividing the number of fully certified teachers by the total number of teachers.

School resources

School resources use the school principal’s perceptions of potential factors hindering the provision of instruction at school. Two scales, were calculated using four-point Likert items (“Not at all” to “A lot”): staff shortage (STAFFSHORT), and shortage of educational material (EDUSHORT). Table 2 shows the item includes in these scales.

Table 2. Items for Meta-cognitive and motivational scales

Figure 1 shows that the distribution of reading performance scores by country follow a normal distribution. Canada (Mean= , S.D=) and U.S (Mean= , S.D=) have scores significantly higher than Mexico. Student level variables used in the analyses are summarize by country in Table 3, 4 and 5, while Tables 6, 7, and 8 presents the school-level characteristics.

Table 3. Canada. Descriptive Students’ Variables

Variable	Label	Stats / Values	Freqs (% of Valid)	Graph	Missing
PV1READ [numeric]	Student Reading Performance	Mean (sd) : 523.7 (96.9) min ≤ med ≤ max: 140.4 ≤ 525.6 ≤ 852.6 Q1 - Q3 : 458.7 - 593.1	13679 distinct values		0 (0.0%)
REPEAT [character]	Grade Repetition	1. No 2. Yes	13320 ( 95.4% ) 648 ( 4.6% )		0 (0.0%)
ESCS [haven_labelled, vctrs_vctr, double]	Index of economic, social and cultural status	Mean (sd) : 0.4 (0.8) min ≤ med ≤ max: -6.7 ≤ 0.5 ≤ 4 Q1 - Q3 : -0.1 - 1	11072 distinct values		0 (0.0%)
UNDREM [haven_labelled, vctrs_vctr, double]	Meta-cognition: understanding and remembering	Mean (sd) : -0.1 (1) min ≤ med ≤ max: -1.6 ≤ -0.2 ≤ 1.5 Q1 - Q3 : -0.9 - 0.4	-1.64 : 2118 ( 15.2% ) -1.29 : 528 ( 3.8% ) -0.94 : 1180 ( 8.4% ) -0.60 : 1811 ( 13.0% ) -0.25 : 1408 ( 10.1% ) 0.10 : 1316 ( 9.4% ) 0.45 : 2237 ( 16.0% ) 0.80 : 1163 ( 8.3% ) 1.15 : 770 ( 5.5% ) 1.50 : 1437 ( 10.3% )		0 (0.0%)
METASUM [haven_labelled, vctrs_vctr, double]	Meta-cognition: summarising	Mean (sd) : -0.1 (1) min ≤ med ≤ max: -1.7 ≤ 0.2 ≤ 1.4 Q1 - Q3 : -0.9 - 0.6	-1.72 : 2285 ( 16.4% ) -1.34 : 552 ( 4.0% ) -0.95 : 725 ( 5.2% ) -0.57 : 1297 ( 9.3% ) -0.18 : 1554 ( 11.1% ) 0.21 : 1784 ( 12.8% ) 0.59 : 2822 ( 20.2% ) 0.98 : 1454 ( 10.4% ) 1.36 : 1495 ( 10.7% )		0 (0.0%)
METASPAM [haven_labelled, vctrs_vctr, double]	Meta-cognition: assess credibility	Mean (sd) : 0 (1) min ≤ med ≤ max: -1.4 ≤ 0 ≤ 1.3 Q1 - Q3 : -1 - 0.9	-1.41 : 3072 ( 22.0% ) -0.96 : 711 ( 5.1% ) -0.50 : 2183 ( 15.6% ) -0.04 : 1397 ( 10.0% ) 0.42 : 2360 ( 16.9% ) 0.87 : 987 ( 7.1% ) 1.33 : 3258 ( 23.3% )		0 (0.0%)
JOYREAD [haven_labelled, vctrs_vctr, double]	Joy/Like reading (WLE)	Mean (sd) : 0 (1.2) min ≤ med ≤ max: -2.7 ≤ 0 ≤ 2.6 Q1 - Q3 : -0.6 - 0.7	747 distinct values		0 (0.0%)
SCREADCOMP [haven_labelled, vctrs_vctr, double]	Self-concept of reading: Perception of competence (WLE)	Mean (sd) : 0.3 (1) min ≤ med ≤ max: -2.4 ≤ 0.1 ≤ 1.9 Q1 - Q3 : -0.4 - 1.2	64 distinct values		0 (0.0%)
SCREADDIFF [haven_labelled, vctrs_vctr, double]	Self-concept of reading: Perception of difficulty (WLE)	Mean (sd) : 0 (1) min ≤ med ≤ max: -1.9 ≤ -0.1 ≤ 2.8 Q1 - Q3 : -0.5 - 0.6	64 distinct values		0 (0.0%)
ATTLNACT [haven_labelled, vctrs_vctr, double]	Attitude towards school: learning activities (WLE)	Mean (sd) : 0.2 (1) min ≤ med ≤ max: -2.5 ≤ 0.5 ≤ 1.1 Q1 - Q3 : -0.7 - 1.1	63 distinct values		0 (0.0%)
WORKMAST [haven_labelled, vctrs_vctr, double]	Work mastery (WLE)	Mean (sd) : 0.1 (1) min ≤ med ≤ max: -2.7 ≤ -0.1 ≤ 1.8 Q1 - Q3 : -0.7 - 0.9	62 distinct values		0 (0.0%)
MASTGOAL [haven_labelled, vctrs_vctr, double]	Mastery goal orientation (WLE)	Mean (sd) : 0.2 (1) min ≤ med ≤ max: -2.5 ≤ 0.2 ≤ 1.9 Q1 - Q3 : -0.4 - 0.8	112 distinct values		0 (0.0%)
GFOFAIL [haven_labelled, vctrs_vctr, double]	General fear of failure (WLE)	Mean (sd) : 0.3 (1.1) min ≤ med ≤ max: -1.9 ≤ 0.5 ≤ 1.9 Q1 - Q3 : -0.4 - 1	64 distinct values		0 (0.0%)

Generated by summarytools 1.0.0 (R version 4.1.2)
2021-12-19

Table 4. Mexico. Descriptive Students’ Variables

Variable	Label	Stats / Values	Freqs (% of Valid)	Graph	Missing
PV1READ [numeric]	Student Reading Performance	Mean (sd) : 440.4 (80.9) min ≤ med ≤ max: 200.4 ≤ 437.5 ≤ 702.2 Q1 - Q3 : 384.5 - 493.9	4330 distinct values		0 (0.0%)
REPEAT [character]	Grade Repetition	1. No 2. Yes	4048 ( 92.8% ) 313 ( 7.2% )		0 (0.0%)
ESCS [haven_labelled, vctrs_vctr, double]	Index of economic, social and cultural status	Mean (sd) : -1 (1.2) min ≤ med ≤ max: -4.7 ≤ -1 ≤ 2.3 Q1 - Q3 : -1.9 - 0	4151 distinct values		0 (0.0%)
UNDREM [haven_labelled, vctrs_vctr, double]	Meta-cognition: understanding and remembering	Mean (sd) : -0.2 (1) min ≤ med ≤ max: -1.6 ≤ -0.2 ≤ 1.5 Q1 - Q3 : -0.9 - 0.4	-1.64 : 634 ( 14.5% ) -1.29 : 225 ( 5.2% ) -0.94 : 471 ( 10.8% ) -0.60 : 599 ( 13.7% ) -0.25 : 495 ( 11.4% ) 0.10 : 394 ( 9.0% ) 0.45 : 705 ( 16.2% ) 0.80 : 247 ( 5.7% ) 1.15 : 228 ( 5.2% ) 1.50 : 363 ( 8.3% )		0 (0.0%)
METASUM [haven_labelled, vctrs_vctr, double]	Meta-cognition: summarising	Mean (sd) : 0.1 (0.9) min ≤ med ≤ max: -1.7 ≤ 0.2 ≤ 1.4 Q1 - Q3 : -0.6 - 0.6	-1.72 : 464 ( 10.6% ) -1.34 : 177 ( 4.1% ) -0.95 : 238 ( 5.5% ) -0.57 : 365 ( 8.4% ) -0.18 : 575 ( 13.2% ) 0.21 : 603 ( 13.8% ) 0.59 : 940 ( 21.6% ) 0.98 : 478 ( 11.0% ) 1.36 : 521 ( 11.9% )		0 (0.0%)
METASPAM [haven_labelled, vctrs_vctr, double]	Meta-cognition: assess credibility	Mean (sd) : -0.4 (0.9) min ≤ med ≤ max: -1.4 ≤ -0.5 ≤ 1.3 Q1 - Q3 : -1.4 - 0.4	-1.41 : 1294 ( 29.7% ) -0.96 : 379 ( 8.7% ) -0.50 : 929 ( 21.3% ) -0.04 : 428 ( 9.8% ) 0.42 : 662 ( 15.2% ) 0.87 : 177 ( 4.1% ) 1.33 : 492 ( 11.3% )		0 (0.0%)
JOYREAD [haven_labelled, vctrs_vctr, double]	Joy/Like reading (WLE)	Mean (sd) : 0.4 (0.9) min ≤ med ≤ max: -2.7 ≤ 0.3 ≤ 2.7 Q1 - Q3 : -0.2 - 0.9	572 distinct values		0 (0.0%)
SCREADCOMP [haven_labelled, vctrs_vctr, double]	Self-concept of reading: Perception of competence (WLE)	Mean (sd) : -0.1 (0.8) min ≤ med ≤ max: -2.4 ≤ 0.1 ≤ 1.9 Q1 - Q3 : -0.5 - 0.1	59 distinct values		0 (0.0%)
SCREADDIFF [haven_labelled, vctrs_vctr, double]	Self-concept of reading: Perception of difficulty (WLE)	Mean (sd) : 0.2 (0.9) min ≤ med ≤ max: -1.9 ≤ 0.4 ≤ 2.8 Q1 - Q3 : -0.2 - 0.8	59 distinct values		0 (0.0%)
ATTLNACT [haven_labelled, vctrs_vctr, double]	Attitude towards school: learning activities (WLE)	Mean (sd) : 0.2 (1.1) min ≤ med ≤ max: -2.5 ≤ 0.5 ≤ 1.1 Q1 - Q3 : -0.7 - 1.1	61 distinct values		0 (0.0%)
WORKMAST [haven_labelled, vctrs_vctr, double]	Work mastery (WLE)	Mean (sd) : 0.4 (1) min ≤ med ≤ max: -2.7 ≤ 0.6 ≤ 1.8 Q1 - Q3 : -0.1 - 1.1	55 distinct values		0 (0.0%)
MASTGOAL [haven_labelled, vctrs_vctr, double]	Mastery goal orientation (WLE)	Mean (sd) : 0.6 (0.9) min ≤ med ≤ max: -2.5 ≤ 0.6 ≤ 1.9 Q1 - Q3 : -0.1 - 1.2	90 distinct values		0 (0.0%)
GFOFAIL [haven_labelled, vctrs_vctr, double]	General fear of failure (WLE)	Mean (sd) : 0.1 (0.9) min ≤ med ≤ max: -1.9 ≤ 0.1 ≤ 1.9 Q1 - Q3 : -0.7 - 0.5	63 distinct values		0 (0.0%)

Generated by summarytools 1.0.0 (R version 4.1.2)
2021-12-19

Table 5. United States. Descriptive Students’ Variables

Variable	Label	Stats / Values	Freqs (% of Valid)	Graph	Missing
PV1READ [numeric]	Student Reading Performance	Mean (sd) : 511.7 (103.5) min ≤ med ≤ max: 170.4 ≤ 516.1 ≤ 868.9 Q1 - Q3 : 438.6 - 585.2	3300 distinct values		0 (0.0%)
REPEAT [character]	Grade Repetition	1. No 2. Yes	3041 ( 91.9% ) 268 ( 8.1% )		0 (0.0%)
ESCS [haven_labelled, vctrs_vctr, double]	Index of economic, social and cultural status	Mean (sd) : 0.1 (1) min ≤ med ≤ max: -3.8 ≤ 0.2 ≤ 3.3 Q1 - Q3 : -0.6 - 0.9	3142 distinct values		0 (0.0%)
UNDREM [haven_labelled, vctrs_vctr, double]	Meta-cognition: understanding and remembering	Mean (sd) : -0.1 (1) min ≤ med ≤ max: -1.6 ≤ 0.1 ≤ 1.5 Q1 - Q3 : -0.9 - 0.8	-1.64 : 476 ( 14.4% ) -1.29 : 119 ( 3.6% ) -0.94 : 259 ( 7.8% ) -0.60 : 416 ( 12.6% ) -0.25 : 333 ( 10.1% ) 0.10 : 298 ( 9.0% ) 0.45 : 513 ( 15.5% ) 0.80 : 292 ( 8.8% ) 1.15 : 193 ( 5.8% ) 1.50 : 410 ( 12.4% )		0 (0.0%)
METASUM [haven_labelled, vctrs_vctr, double]	Meta-cognition: summarising	Mean (sd) : 0 (1) min ≤ med ≤ max: -1.7 ≤ 0.2 ≤ 1.4 Q1 - Q3 : -0.6 - 0.6	-1.72 : 530 ( 16.0% ) -1.34 : 118 ( 3.6% ) -0.95 : 143 ( 4.3% ) -0.57 : 290 ( 8.8% ) -0.18 : 360 ( 10.9% ) 0.21 : 476 ( 14.4% ) 0.59 : 655 ( 19.8% ) 0.98 : 360 ( 10.9% ) 1.36 : 377 ( 11.4% )		0 (0.0%)
METASPAM [haven_labelled, vctrs_vctr, double]	Meta-cognition: assess credibility	Mean (sd) : 0 (1) min ≤ med ≤ max: -1.4 ≤ 0 ≤ 1.3 Q1 - Q3 : -1 - 0.9	-1.41 : 658 ( 19.9% ) -0.96 : 190 ( 5.7% ) -0.50 : 570 ( 17.2% ) -0.04 : 297 ( 9.0% ) 0.42 : 624 ( 18.9% ) 0.87 : 232 ( 7.0% ) 1.33 : 738 ( 22.3% )		0 (0.0%)
JOYREAD [haven_labelled, vctrs_vctr, double]	Joy/Like reading (WLE)	Mean (sd) : -0.1 (1.1) min ≤ med ≤ max: -2.7 ≤ -0.1 ≤ 2.6 Q1 - Q3 : -0.7 - 0.6	459 distinct values		0 (0.0%)
SCREADCOMP [haven_labelled, vctrs_vctr, double]	Self-concept of reading: Perception of competence (WLE)	Mean (sd) : 0.3 (1) min ≤ med ≤ max: -2.4 ≤ 0.1 ≤ 1.9 Q1 - Q3 : -0.4 - 1.2	55 distinct values		0 (0.0%)
SCREADDIFF [haven_labelled, vctrs_vctr, double]	Self-concept of reading: Perception of difficulty (WLE)	Mean (sd) : 0.1 (1) min ≤ med ≤ max: -1.9 ≤ 0 ≤ 2.8 Q1 - Q3 : -0.5 - 0.8	60 distinct values		0 (0.0%)
ATTLNACT [haven_labelled, vctrs_vctr, double]	Attitude towards school: learning activities (WLE)	Mean (sd) : 0.3 (1) min ≤ med ≤ max: -2.5 ≤ 0.5 ≤ 1.1 Q1 - Q3 : -0.7 - 1.1	53 distinct values		0 (0.0%)
WORKMAST [haven_labelled, vctrs_vctr, double]	Work mastery (WLE)	Mean (sd) : 0.2 (1) min ≤ med ≤ max: -2.7 ≤ -0.1 ≤ 1.8 Q1 - Q3 : -0.7 - 1	53 distinct values		0 (0.0%)
MASTGOAL [haven_labelled, vctrs_vctr, double]	Mastery goal orientation (WLE)	Mean (sd) : 0.3 (1) min ≤ med ≤ max: -2.5 ≤ 0.3 ≤ 1.9 Q1 - Q3 : -0.4 - 1	92 distinct values		0 (0.0%)
GFOFAIL [haven_labelled, vctrs_vctr, double]	General fear of failure (WLE)	Mean (sd) : 0.2 (1.1) min ≤ med ≤ max: -1.9 ≤ 0.1 ≤ 1.9 Q1 - Q3 : -0.7 - 0.9	64 distinct values		0 (0.0%)

Generated by summarytools 1.0.0 (R version 4.1.2)
2021-12-19

Table 6. Canada. Descriptive Schools’ Variables

Variable	Label	Stats / Values	Freqs (% of Valid)	Graph	Missing
PRIVATESCH [character]	School type derived from sampling information; values = public, private, missing	1. private 2. public	55 ( 8.5% ) 593 ( 91.5% )		0 (0.0%)
SCH_COM [character]	Size School Community	1. City or Large City 2. Rural area or Town	281 ( 43.4% ) 367 ( 56.6% )		0 (0.0%)
SESCS [numeric]	School Index of Socio-Economic Status	Mean (sd) : 0.3 (0.4) min ≤ med ≤ max: -2 ≤ 0.3 ≤ 1.5 Q1 - Q3 : 0.1 - 0.6	648 distinct values		0 (0.0%)
ECONDISAD [numeric]	Students with economic disadvantage	Mean (sd) : 1.8 (0.8) min ≤ med ≤ max: 1 ≤ 2 ≤ 4 Q1 - Q3 : 1 - 2	1 : 258 ( 39.8% ) 2 : 286 ( 44.1% ) 3 : 77 ( 11.9% ) 4 : 27 ( 4.2% )		0 (0.0%)
REDISAD [numeric]	Students with reading disadvantage	Mean (sd) : 3.1 (0.8) min ≤ med ≤ max: 1 ≤ 3 ≤ 4 Q1 - Q3 : 3 - 4	1 : 22 ( 3.4% ) 2 : 126 ( 19.4% ) 3 : 263 ( 40.6% ) 4 : 237 ( 36.6% )		0 (0.0%)

Generated by summarytools 1.0.0 (R version 4.1.2)
2021-12-19

Table 7. Mexico. Descriptive Schools’ Variables

Variable	Label	Stats / Values	Freqs (% of Valid)	Graph	Missing
PRIVATESCH [character]	School type derived from sampling information; values = public, private, missing	1. private 2. public	36 ( 14.6% ) 210 ( 85.4% )		0 (0.0%)
SCH_COM [character]	Size School Community	1. City or Large City 2. Rural area or Town	134 ( 54.5% ) 112 ( 45.5% )		0 (0.0%)
SESCS [numeric]	School Index of Socio-Economic Status	Mean (sd) : -1.1 (0.9) min ≤ med ≤ max: -3.5 ≤ -1.2 ≤ 1.2 Q1 - Q3 : -1.6 - -0.7	246 distinct values		0 (0.0%)
ECONDISAD [numeric]	Students with economic disadvantage	Mean (sd) : 1.9 (1) min ≤ med ≤ max: 1 ≤ 2 ≤ 4 Q1 - Q3 : 1 - 2	1 : 116 ( 47.2% ) 2 : 71 ( 28.9% ) 3 : 30 ( 12.2% ) 4 : 29 ( 11.8% )		0 (0.0%)
REDISAD [numeric]	Students with reading disadvantage	Mean (sd) : 2 (1.1) min ≤ med ≤ max: 1 ≤ 2 ≤ 4 Q1 - Q3 : 1 - 3	1 : 102 ( 41.5% ) 2 : 74 ( 30.1% ) 3 : 35 ( 14.2% ) 4 : 35 ( 14.2% )		0 (0.0%)

Generated by summarytools 1.0.0 (R version 4.1.2)
2021-12-19

Table 8. United States. Descriptive Schools’ Variables

Variable	Label	Stats / Values	Freqs (% of Valid)	Graph	Missing
PRIVATESCH [character]	School type derived from sampling information; values = public, private, missing	1. private 2. public	8 ( 6.1% ) 123 ( 93.9% )		0 (0.0%)
SCH_COM [character]	Size School Community	1. City or Large City 2. Rural area or Town	53 ( 40.5% ) 78 ( 59.5% )		0 (0.0%)
SESCS [numeric]	School Index of Socio-Economic Status	Mean (sd) : 0.1 (0.5) min ≤ med ≤ max: -1.2 ≤ 0.1 ≤ 1.2 Q1 - Q3 : -0.3 - 0.5	131 distinct values		0 (0.0%)
ECONDISAD [numeric]	Students with economic disadvantage	Mean (sd) : 1.9 (0.9) min ≤ med ≤ max: 1 ≤ 2 ≤ 4 Q1 - Q3 : 1 - 2	1 : 56 ( 42.7% ) 2 : 44 ( 33.6% ) 3 : 23 ( 17.6% ) 4 : 8 ( 6.1% )		0 (0.0%)
REDISAD [numeric]	Students with reading disadvantage	Mean (sd) : 2.9 (0.8) min ≤ med ≤ max: 1 ≤ 3 ≤ 4 Q1 - Q3 : 2 - 4	1 : 3 ( 2.3% ) 2 : 38 ( 29.0% ) 3 : 55 ( 42.0% ) 4 : 35 ( 26.7% )		0 (0.0%)

Generated by summarytools 1.0.0 (R version 4.1.2)
2021-12-19

3.2 The individual factors associated with reading performance

Table 9, summarizes the results of the initial null model, which separates the variability of reading performance scores into within- and between-school components, yielded the intra-class correlation coefficient (ICC). Results indicates that 13.63%, 35%, and 13.99% of the total variability in reading literacy in Canada, Mexico, and U.S could be attributed to schools (i.e., the group level); respectively. Table 10, presents the estimation of confidence intervals for random effects.

Table 9. Null Model (Model 0)

	Canada			Mexico			United States
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	519.58 ***	1.66	<0.001	431.59 ***	3.32	<0.001	511.70 ***	3.81	<0.001
Random Effects
σ2	8160.265			4307.543			9235.451
τ00	1287.376 CNTSCHID			2319.231 CNTSCHID			1501.617 CNTSCHID
ICC	0.136			0.350			0.140
N	648 CNTSCHID			246 CNTSCHID			131 CNTSCHID
Observations	13968			4361			3309
Marginal R2 / Conditional R2	0.000 / 0.136			0.000 / 0.350			0.000 / 0.140
p<0.05   ** p<0.01   *** p<0.001

Table 10. Confidence Interval for Null Model (Model 5.0)

	2.5 %	97.5 %	Country
.sig01	33.28192	38.68734	CAN
.sigma	89.26049	91.42955	CAN
(Intercept)	516.29642	522.83076	CAN
.sig011	43.51818	53.49357	MEX
.sigma1	64.23935	67.07537	MEX
(Intercept)1	425.01988	438.11238	MEX
.sig012	33.27792	45.24809	USA
.sigma2	93.78615	98.51318	USA
(Intercept)2	504.18706	519.21451	USA

Model 1 add student-level explanatory variables (level 1) to the random intercept model. Although fixed and random effects were statistically significant, we observed a reduction in the variance between schools (\(\tau_{00}\)) even without considering any variable at the school level. Consequently, we estimated an additional model (Model 2) including both the ISCE centered at the school level (CISCE) and a random slope for socioeconomic status, which suggests that the effect of socioeconomic status on reading performance could vary between schools. Table 11 presents the confidence interval for Model 2, showing that both random intercept and slope effects are statistically significant.

Table 11. Confidence Interval for Fixed and Random Effects (Model 2)

	2.5 %	97.5 %	Country
.sig01	32.3298424	37.5218807	CAN
.sig02	-0.1896064	0.4227763	CAN
.sig03	3.9116193	12.6181366	CAN
.sigma	85.2940282	87.4047679	CAN
.sig011	39.8917432	49.2373936	MEX
.sig021	-1.0000000	1.0000000	MEX
.sig031	0.0000000	7.3695916	MEX
.sigma1	63.6838773	66.5461433	MEX
.sig012	31.2852519	42.5260201	USA
.sig022	0.1767975	0.8251034	USA
.sig032	9.2046036	19.8539539	USA
.sigma2	87.5371779	92.0297298	USA

Estimation for previous models is summarized in the tables below. Results suggested that the Index of Socioeconomic Status and Student Age (grand-centered) have a positive effect on reading performance, while gender (male) and grade repetition are negative predictors in all countries included in the analysis.

Table 10. Canada, Fixed and Random Effect for Student Variables on Reading Performance

	Null Model			Model 2
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	519.58 ***	1.66	<0.001	534.99 ***	1.77	<0.001
ESCS (School-Centered)				19.12 ***	1.09	<0.001
Student Age (Grand-Centered)				17.20 ***	2.68	<0.001
Gender (Male)				-23.58 ***	1.50	<0.001
Grade repetion (Yes)				-76.99 ***	3.68	<0.001
Random Effects
σ2	8160.265			7454.381
τ00	1287.376 CNTSCHID			1212.781 CNTSCHID
τ11				80.291 CNTSCHID.CESCS
ρ01				0.105 CNTSCHID
N	648 CNTSCHID			648 CNTSCHID
Observations	13968			13968
Marginal R2 / Conditional R2	0.000 / 0.136			0.073 / 0.207
p<0.05   ** p<0.01   *** p<0.001

Although the direction of effect is consistent across countries, the magnitude of them differs considerably. For example, while an increase in one unit in the ESCS status above the school average increases the score in 18.81 and 22.11 points in Canada and US, respectively; in Mexico, this effect is slightly (5.87 points). Also, grade repetition has a greater effect in Canada and US than in Mexico. This should be related to differences in policies about grade promotion among countries. Similarly, gender is a statistically significant predictor of reading performance. Therefore, boys have a negative and significantly effect on reading scores compared to girls, which is higher in Canada (23.56 points) and US (16.83 points) than Mexico (5.15 points).

Table 11. Mexico, Fixed and Random Effect for Student Variables on Reading Performance

	Null Model			Model 1			Model 2
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	431.59 ***	3.32	<0.001	450.25 ***	3.17	<0.001	439.28 ***	3.29	<0.001
ESCS				9.81 ***	1.03	<0.001
Student Age (Grand-Centered)				8.64 *	3.63	0.017	8.58 *	3.63	0.018
Gender (Male)				-5.75 **	2.05	0.005	-5.18 *	2.05	0.012
Grade repetion (Yes)				-41.08 ***	5.29	<0.001	-40.09 ***	5.42	<0.001
Center School ESCS							6.50 ***	1.10	<0.001
Random Effects
σ2	4307.543			4251.291			4236.873
τ00	2319.231 CNTSCHID			1474.457 CNTSCHID			1957.053 CNTSCHID
τ11							4.826 CNTSCHID.CESCS
ρ01							-0.016 CNTSCHID
N	246 CNTSCHID			246 CNTSCHID			246 CNTSCHID
Observations	4361			4361			4361
Marginal R2 / Conditional R2	0.000 / 0.350			0.046 / 0.292			0.025 / 0.333
p<0.05   ** p<0.01   *** p<0.001

Additionally, the students-level variables reduced in 8.65% and 12.80% the variability among students in Canada and U.S, respectively; however, in Mexico, these variables contribute to reducing slightly the residual variance (1.64%)

Table 12. United States, Fixed and Random Effect for Student Variables on Reading Performance.

	Null Model			Model 1			Model 2
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	511.70 ***	3.81	<0.001	523.20 ***	3.23	<0.001	526.33 ***	3.93	<0.001
ESCS				27.35 ***	1.77	<0.001
Student Age (Grand-Centered)				16.12 **	5.61	0.004	15.67 **	5.59	0.005
Gender (Male)				-17.20 ***	3.21	<0.001	-16.90 ***	3.20	<0.001
Grade repetion (Yes)				-75.47 ***	5.99	<0.001	-77.11 ***	5.96	<0.001
Center School ESCS							23.93 ***	2.28	<0.001
Random Effects
σ2	9235.451			8224.953			8052.697
τ00	1501.617 CNTSCHID			658.534 CNTSCHID			1326.781 CNTSCHID
τ11							212.897 CNTSCHID.CESCS
ρ01							0.512 CNTSCHID
N	131 CNTSCHID			131 CNTSCHID			131 CNTSCHID
Observations	3309			3309			3309
Marginal R2 / Conditional R2	0.000 / 0.140			0.140 / 0.204			0.101 / 0.241
p<0.05   ** p<0.01   *** p<0.001

3.3 The school factors (level 2) associated with reading performance

For studying the effect of school factors on reading performance, a third model was conducted. Model 3 adds an explanatory variable at the school level to the previous model. First, we conducted a complete model that includes all variables at the school level and verified the statistical significance of coefficients and random effect (intercept and slope), using coefficient intervals and ANOVA-Test (Table 15). Finally, no significant coefficients and effects were deleted to estimate the final model by country. Finally, a cross-level interaction between gender and the proportion of students with reading disadvantages was included. Results for final models are summarized in the tables below.

Table 15. Confidence Interval for Fixed and Random Effects for Model 3

	2.5 %	97.5 %	Country
.sig01	13.7882633	17.9255758	CAN
.sig02	-0.1651849	0.6393714	CAN
.sig03	3.8584286	12.5746041	CAN
.sigma	85.1888523	87.2919142	CAN
.sig011	10.5326678	16.2862554	MEX
.sigma1	63.8434479	66.6536485	MEX
.sig012	1.5549688	13.3700918	USA
.sig021	-0.0646573	1.0000000	USA
.sig031	9.0178941	19.7569652	USA
.sigma2	87.5320824	92.0215426	USA

For Canada, School socio-economic status (\(\beta = 18.13\)) and a lower proportion of students with reading disadvantages at school-level (\(\beta =29.71\)) are positive and significant effects on reading performance. In contrast, public schools (\(\beta =-9.98\)) and those located in community rural areas ($ $) have a negative effect on reading performance. Additionally, the random slope for socioeconomic status is statistically significant; in other words, the effect of students’ socioeconomic conditions varies between schools.

Table 13. Canada, Fixed and Random Effect with Student and School variables.

	Model 0			Model 3
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	519.58 ***	1.66	<0.001	449.21 ***	6.90	<0.001
ESCS (School-Centered)				19.24 ***	1.08	<0.001
Student Age (Grand-Centered)				17.70 ***	2.67	<0.001
Gender (Male)				-46.28 ***	6.45	<0.001
Grade repetion (Yes)				-71.18 ***	3.61	<0.001
School ESCS				18.13 ***	3.26	<0.001
School Sector (Public)				-9.98 **	3.82	0.009
School Area (Rural)				-7.91 ***	2.08	<0.001
Proportion of reading disadvantage (REDISAD)				29.71 ***	1.71	<0.001
Interaction Gender - REDISAD				7.07 ***	1.97	<0.001
Random Effects
σ2	8160.265			7435.590
τ00	1287.376 CNTSCHID			250.573 CNTSCHID
τ11				79.417 CNTSCHID.CESCS
ρ01				0.198 CNTSCHID
N	648 CNTSCHID			648 CNTSCHID
Observations	13968			13968
Marginal R2 / Conditional R2	0.000 / 0.136			0.178 / 0.210
p<0.05   ** p<0.01   *** p<0.001

Furthermore, the main effects of gender and the proportion of students with reading disadvantages should be interpreted in light of the significant cross-level interaction. Results show that the effect of gender on reading scores depended on whether the students attend a school with a lower or higher proportion of students below the basic reading performance level (\(\beta = 7.07\)). Overall, variables at school level explained a 80.53% of variance at school level in Canada (from \(\$tau_{00}\)= 1287.376 in Model Null to \(\$tau_{00}\)=250.573 in Model 3).

Figure 1, which plots the interaction, shows that Gender was more strongly related to reading performance for students in schools with a lower proportion of students with disadvantages in reading performance than schools with a high proportion of those. For example, for male students who attend a school with higher than 50% performance below Level 2, the total gender effect on reading score is \(-38.93\). In contrast, for a male who attends a school with less than 10% of student performance below Level 2, the total gender effect on reading score is \(-17.72\).

In Mexico, school socio-economic status (\(\beta = 17.15\)) and a lower proportion of students with reading disadvantages at school-level (\(\beta =30.08\)) are the unique positive and significant effects on reading performance at school-level. Additionally, the gender, cross-level interaction, and the random slope for socioeconomic status is not statistically significant; after included the school-level predictor (Table 17). Overall, variables at school level reduced the variance at school level in 92.33% (from \(\$tau_{00}\)= 2319.231 in Model Null to \(\$tau_{00}\)=177.939 in Model 3).

Table 14. Mexico, Fixed and Random Effect with Student and School variables

	Model 0			Model 3
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	431.59 ***	3.32	<0.001	394.55 ***	6.41	<0.001
ESCS (School-Centered)				6.54 ***	1.09	<0.001
Student Age (Grand-Centered)				8.39 *	3.62	0.020
Gender (Male)				-6.06	4.64	0.191
Grade repetion (Yes)				-28.92 ***	4.45	<0.001
School ESCS				17.15 ***	2.37	<0.001
Proportion of reading disadvantage (REDISAD)				30.08 ***	2.00	<0.001
Interaction Gender - REDISAD				0.66	1.88	0.725
Random Effects
σ2	4307.543			4254.100
τ00	2319.231 CNTSCHID			177.939 CNTSCHID
N	246 CNTSCHID			246 CNTSCHID
Observations	4361			4361
Marginal R2 / Conditional R2	0.000 / 0.350			0.319 / 0.346
p<0.05   ** p<0.01   *** p<0.001

In US, school socio-economic status (\(\beta = 31.39\)) and a lower proportion of students with reading disadvantages at school-level (\(\beta =30.80\)) are positive and significant effects on reading performance at school-level. Additionally, the random slope for socioeconomic status is statistically significant; in other words, the effect of students’ socioeconomic conditions varies between schools. In contrast, the cross-level interaction between gender and reading performance at school level is not statistically significant. Overall, variables at school level reduced the variance at school level in 95.51% (from \(\$tau_{00}\)= 1501.617 in Model Null to \(\$tau_{00}\)=67.389 in Model 3).

Table 15. United States, Fixed and Random Effect with Student and School variables

	Model 0			Model 3
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	511.70 ***	3.81	<0.001	432.39 ***	10.21	<0.001
ESCS (School-Centered)				23.67 ***	2.28	<0.001
Student Age (Grand-Centered)				15.95 **	5.58	0.004
Gender (Male)				-10.12	12.36	0.413
Grade repetion (Yes)				-72.34 ***	5.88	<0.001
School ESCS				31.39 ***	4.31	<0.001
Proportion of reading disadvantage (REDISAD)				30.80 ***	3.43	<0.001
Interaction Gender - REDISAD				-2.32	4.06	0.568
Random Effects
σ2	9235.451			8051.523
τ00	1501.617 CNTSCHID			67.389 CNTSCHID
τ11				209.314 CNTSCHID.CESCS
ρ01				0.675 CNTSCHID
N	131 CNTSCHID			131 CNTSCHID
Observations	3309			3309
Marginal R2 / Conditional R2	0.000 / 0.140			0.223 / 0.244
p<0.05   ** p<0.01   *** p<0.001

3.4 Effects of student’s meta-cognitive strategies and motivation on reading performance

For studying the effect of school factors on reading performance, a fourth model was conducted. Model 4 adds an meta-cognitive and motivational scales at student-level to the previous model. First, we included the meta-cognitive scales and checked the significance of fixed and random effects, then we added the motivational scales. Significance of coefficients and random effect (intercept and slope), using coefficient intervals and ANOVA-Test. Finally, no significant coefficients and effects were deleted for estimating the final model by country.Tables below display the results.

For Canada, meta-cognitive and motivation scales have positive effects on reading performance, except Perception of test’ difficult, thus an increase in one unit in the perception of PISA difficulty predicts a decrease in the reading score in \(8.63\) points. Overall, the variance within school reduced by 32.76% after included these predictors (from \(\sigma^2 = 7435.6\) in Model 3 to \(\sigma^2 = 4999.5\)). Additionally, the interaction between gender and reading performance at school level and the random slope for socioeconomic status is not statistically significant after controlling by students’ meta-cognitive and motivation factors.

Table 16. Canada, Fixed Effects of Metacognition and Motivation on Reading

	Model 3			Model 4
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	449.21 ***	6.90	<0.001	458.25 ***	5.72	<0.001
ESCS (School-Centered)	19.24 ***	1.08	<0.001	7.48 ***	0.84	<0.001
Student Age (Grand-Centered)	17.70 ***	2.67	<0.001	10.99 ***	2.18	<0.001
Gender (Male)	-46.28 ***	6.45	<0.001	-18.74 ***	5.30	<0.001
Grade repetion (Yes)	-71.18 ***	3.61	<0.001	-49.73 ***	2.96	<0.001
School ESCS	18.13 ***	3.26	<0.001	6.28 *	2.71	0.020
School Sector (Public)	-9.98 **	3.82	0.009	-8.89 **	3.17	0.005
School Area (Rural)	-7.91 ***	2.08	<0.001	-4.53 **	1.73	0.009
Proportion of reading disadvantage (REDISAD)	29.71 ***	1.71	<0.001	21.93 ***	1.41	<0.001
Interaction Gender - REDISAD	7.07 ***	1.97	<0.001	5.74 ***	1.61	<0.001
Understanding and remembering (UNDREM)				3.73 ***	0.70	<0.001
Summarizing (METASUM)				15.92 ***	0.72	<0.001
Assessing credibility (METASPAM)				22.81 ***	0.66	<0.001
Enjoyment of reading (JOYREAD)				11.27 ***	0.62	<0.001
Perception of competence (SCREADCOMP)				17.62 ***	0.75	<0.001
Perception of difficulty (SCREADDIFF)				-8.55 ***	0.69	<0.001
General fear of failure (GFOFAIL)				5.22 ***	0.60	<0.001
Random Effects
σ2	7435.590			4988.554
τ00	250.573 CNTSCHID			176.853 CNTSCHID
τ11	79.417 CNTSCHID.CESCS
ρ01	0.198 CNTSCHID
N	648 CNTSCHID			648 CNTSCHID
Observations	13968			13968
Marginal R2 / Conditional R2	0.178 / 0.210			0.450 / 0.469
p<0.05   ** p<0.01   *** p<0.001

In Mexico, meta-cognitive and motivation scales positively affect reading performance, except Perception of test’ difficult (SCREADDIFF) and students’ mastery-approach orientation of achievement goals (MASTGOALS) that are negative predictors of reading performance. In contrast to previous models, after control by meta-cognitive and motivation scales, gender (male) predicts positive effects on reading performance; furthermore the reading performance at school level (REDISAD) and the interaction between gender and reading performance at school level are not statistically significant. Overall, the variance within school reduced by 20.7% after included these predictors (from \(\sigma^2 = 4254.100\) in Model 3 to \(\sigma^2 = 3373.215\)).

Table 17. Mexico, Fixed Effects of Metacognition and Motivation on Reading

	Model 3			Model 4
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	394.55 ***	6.41	<0.001	476.96 ***	3.11	<0.001
ESCS (School-Centered)	6.54 ***	1.09	<0.001	2.55 **	0.98	0.009
Student Age (Grand-Centered)	8.39 *	3.62	0.020	4.09	3.23	0.206
Gender (Male)	-6.06	4.64	0.191	5.48 **	1.93	0.005
Grade repetion (Yes)	-28.92 ***	4.45	<0.001	-32.83 ***	4.34	<0.001
School ESCS	17.15 ***	2.37	<0.001	32.03 ***	2.15	<0.001
Proportion of reading disadvantage (REDISAD)	30.08 ***	2.00	<0.001
Interaction Gender - REDISAD	0.66	1.88	0.725
Understanding and remembering (UNDREM)				7.73 ***	1.04	<0.001
Summarizing (METASUM)				13.27 ***	1.10	<0.001
Assessing credibility (METASPAM)				13.03 ***	1.05	<0.001
Enjoyment of reading (JOYREAD)				5.72 ***	1.22	<0.001
Perception of competence (SCREADCOMP)				13.50 ***	1.35	<0.001
Perception of difficulty (SCREADDIFF)				-8.19 ***	1.12	<0.001
Working motive and mastery (WORKMAST)				3.37 ***	0.96	<0.001
Mastery-approach orientation (MASTGOAL)				-4.51 ***	1.12	<0.001
Random Effects
σ2	4254.100			3373.215
τ00	177.939 CNTSCHID			466.484 CNTSCHID
N	246 CNTSCHID			246 CNTSCHID
Observations	4361			4361
Marginal R2 / Conditional R2	0.319 / 0.346			0.379 / 0.454
p<0.05   ** p<0.01   *** p<0.001

In the US, meta-cognitive and motivation scales are positive predictors of reading performance, except Perception of test’ difficult (SCREADDIFF) and students’ mastery-approach orientation of achievement goals (MASTGOALS) are negative predictors of reading performance. In contrast to the previous model, gender, students’ age, reading performance at school level (REDISAD), and the interaction between gender and reading performance at school level are not statistically significant, after control by meta-cognitive and motivation scales. However, the random slope for socioeconomic status persists as significantly random effects. Overall, the variance within school reduced by 30.69% after included these variables (from \(\sigma^2 = 8051.523\) in Model 3 to \(\sigma^2 = 5580.855\)).

Table 18. United States, Fixed and Random Effect with Student and School variables

	Model 3			Model 4
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	432.39 ***	10.21	<0.001	512.64 ***	2.50	<0.001
ESCS (School-Centered)	23.67 ***	2.28	<0.001	9.79 ***	1.76	<0.001
Student Age (Grand-Centered)	15.95 **	5.58	0.004	2.65	4.66	0.569
Gender (Male)	-10.12	12.36	0.413	-2.72	2.83	0.337
Grade repetion (Yes)	-72.34 ***	5.88	<0.001	-49.22 ***	4.96	<0.001
School ESCS	31.39 ***	4.31	<0.001	34.49 ***	3.65	<0.001
Proportion of reading disadvantage (REDISAD)	30.80 ***	3.43	<0.001
Interaction Gender - REDISAD	-2.32	4.06	0.568
Understanding and remembering (UNDREM)				9.37 ***	1.51	<0.001
Summarizing (METASUM)				17.90 ***	1.57	<0.001
Assessing credibility (METASPAM)				25.49 ***	1.50	<0.001
Enjoyment of reading (JOYREAD)				10.65 ***	1.40	<0.001
Perception of competence (SCREADCOMP)				12.96 ***	1.69	<0.001
Perception of difficulty (SCREADDIFF)				-7.93 ***	1.55	<0.001
Working motive and mastery (WORKMAST)				4.68 **	1.50	0.002
Mastery-approach orientation (MASTGOAL)				-8.60 ***	1.49	<0.001
General fear of failure (GFOFAIL)				7.58 ***	1.27	<0.001
Random Effects
σ2	8051.523			5580.855
τ00	67.389 CNTSCHID			220.462 CNTSCHID
τ11	209.314 CNTSCHID.CESCS			66.722 CNTSCHID.CESCS
ρ01	0.675 CNTSCHID			0.915 CNTSCHID
N	131 CNTSCHID			131 CNTSCHID
Observations	3309			3309
Marginal R2 / Conditional R2	0.223 / 0.244			0.447 / 0.472
p<0.05   ** p<0.01   *** p<0.001

Model 5, which added a cross interaction between gender and meta-cognitive strategies and motivation, was conducted. Similar to the previous analysis, the significance of coefficients and random effects were checked using confidence intervals and Anova test. Results for the final model are presented in the tables below.

In Canada, the interaction between gender and perception of competence (SCREADCOMP) is a positive and significant effect, thus increasing the self-concept competence as readers reduce the negative effect of gender (male) on reading performance. Overall, student-level and school-level variables included in the final model reduced the variance within (\(\sigma^2\)) and between (\(\sigma^2\)) schools in 38.92% and 86.25%, respectively. Also, the total variability in students’ performance associated with their school fall 75% compared to the null model, From 13.6% to 3.4%.

Table 19. Canada, Cross interaction Gender Metacognition and Motivation on Reading

	Null Model			Model 4			Model 5
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	519.58 ***	1.66	<0.001	458.25 ***	5.72	<0.001	458.38 ***	5.72	<0.001
ESCS (School-Centered)				7.48 ***	0.84	<0.001	7.44 ***	0.84	<0.001
Student Age (Grand-Centered)				10.99 ***	2.18	<0.001	10.91 ***	2.18	<0.001
Gender (Male)				-18.74 ***	5.30	<0.001	-18.79 ***	5.30	<0.001
Grade repetion (Yes)				-49.73 ***	2.96	<0.001	-49.68 ***	2.96	<0.001
School ESCS				6.28 *	2.71	0.020	6.35 *	2.71	0.019
School Sector (Public)				-8.89 **	3.17	0.005	-8.71 **	3.17	0.006
School Area (Rural)				-4.53 **	1.73	0.009	-4.42 *	1.73	0.011
Proportion of reading disadvantage (REDISAD)				21.93 ***	1.41	<0.001	22.06 ***	1.41	<0.001
Understanding and remembering (UNDREM)				3.73 ***	0.70	<0.001	3.71 ***	0.70	<0.001
Summarizing (METASUM)				15.92 ***	0.72	<0.001	15.92 ***	0.72	<0.001
Assessing credibility (METASPAM)				22.81 ***	0.66	<0.001	22.81 ***	0.66	<0.001
Enjoyment of reading (JOYREAD)				11.27 ***	0.62	<0.001	11.47 ***	0.63	<0.001
Perception of competence (SCREADCOMP)				17.62 ***	0.75	<0.001	15.38 ***	0.99	<0.001
Perception of difficulty (SCREADDIFF)				-8.55 ***	0.69	<0.001	-8.73 ***	0.69	<0.001
General fear of failure (GFOFAIL)				5.22 ***	0.60	<0.001	5.25 ***	0.60	<0.001
Interaction Gender - REDISAD				5.74 ***	1.61	<0.001	5.40 ***	1.61	0.001
Interaction Gender - Perception of competence							4.12 ***	1.20	0.001
Random Effects
σ2	8160.265			4988.554			4984.190
τ00	1287.376 CNTSCHID			176.853 CNTSCHID			176.963 CNTSCHID
ICC	0.136			0.034			0.034
N	648 CNTSCHID			648 CNTSCHID			648 CNTSCHID
Observations	13968			13968			13968
Marginal R2 / Conditional R2	0.000 / 0.136			0.450 / 0.469			0.451 / 0.469
p<0.05   ** p<0.01   *** p<0.001

In Mexico, the interaction between gender and enjoyment of reading and perception of difficulty are negative predictors of students performance. Overall, student-level and school-level variables included in the final model reduced the variance within (\(\sigma^2\)) and between (\(\sigma^2\)) schools in 21.97% and 79.89%, respectively. Also, the total variability in students’ performance associated with their school fall 65% compared to the null model, From 35.0% to 12.2%.

Table 20. Mexico, Cross interaction Gender Metacognition and Motivation on Reading

	Null Model			Model 4			Model 5
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	431.59 ***	3.32	<0.001	476.96 ***	3.11	<0.001	474.49 ***	3.18	<0.001
ESCS (School-Centered)				2.55 **	0.98	0.009	2.43 *	0.98	0.013
Student Age (Grand-Centered)				4.09	3.23	0.206	4.03	3.23	0.212
Gender (Male)				5.48 **	1.93	0.005	8.78 ***	2.11	<0.001
Grade repetion (Yes)				-32.83 ***	4.34	<0.001	-32.84 ***	4.33	<0.001
School ESCS				32.03 ***	2.15	<0.001	31.85 ***	2.14	<0.001
Understanding and remembering (UNDREM)				7.73 ***	1.04	<0.001	7.65 ***	1.03	<0.001
Summarizing (METASUM)				13.27 ***	1.10	<0.001	13.15 ***	1.10	<0.001
Assessing credibility (METASPAM)				13.03 ***	1.05	<0.001	13.07 ***	1.05	<0.001
Enjoyment of reading (JOYREAD)				5.72 ***	1.22	<0.001	8.70 ***	1.51	<0.001
Perception of competence (SCREADCOMP)				13.50 ***	1.35	<0.001	13.58 ***	1.35	<0.001
Perception of difficulty (SCREADDIFF)				-8.19 ***	1.12	<0.001	-4.69 **	1.54	0.002
Working motive and mastery (WORKMAST)				3.37 ***	0.96	<0.001	3.43 ***	0.95	<0.001
Mastery-approach orientation (MASTGOAL)				-4.51 ***	1.12	<0.001	-4.35 ***	1.12	<0.001
Interaction Gender -Enjoyment of reading							-6.86 **	2.29	0.003
Interaction Gender -Perception of difficulty							-6.73 **	2.12	0.002
Random Effects
σ2	4307.543			3373.215			3360.867
τ00	2319.231 CNTSCHID			466.484 CNTSCHID			466.265 CNTSCHID
ICC	0.350			0.121			0.122
N	246 CNTSCHID			246 CNTSCHID			246 CNTSCHID
Observations	4361			4361			4361
Marginal R2 / Conditional R2	0.000 / 0.350			0.379 / 0.454			0.381 / 0.456
p<0.05   ** p<0.01   *** p<0.001

Finally, in the US, the interaction between gender and summarizing negatively predicts students’ performance. Overall, student-level and school-level variables included in the final model reduced the variance within (\(\sigma^2\)) and between (\(\sigma^2\)) schools in 30.65(% and 85.32%, respectively. Also, the total variability in students’ performance associated with their school fall 67.1% compared to the null model, From 14.0% to 4.6%. Furthermore, after including all student- and school-level variables, the random slope effect of socioeconomic status on student performance remains statistically significant.

Table 21. United States, Cross interaction Gender Metacognition and Motivation on Reading

	Model 0			Model 4			Model 5
Predictors	Estimates	std.Error	p-value	Estimates	std.Error	p-value	Estimates	std.Error	p-value
Intercept	511.70 ***	3.81	<0.001	512.64 ***	2.50	<0.001	512.31 ***	2.50	<0.001
ESCS (School-Centered)				9.79 ***	1.76	<0.001	9.81 ***	1.77	<0.001
Student Age (Grand-Centered)				2.65	4.66	0.569	2.33	4.66	0.617
Gender (Male)				-2.72	2.83	0.337	-2.84	2.83	0.316
Grade repetion (Yes)				-49.22 ***	4.96	<0.001	-49.26 ***	4.95	<0.001
School ESCS				34.49 ***	3.65	<0.001	34.39 ***	3.65	<0.001
Understanding and remembering (UNDREM)				9.37 ***	1.51	<0.001	9.44 ***	1.51	<0.001
Summarizing (METASUM)				17.90 ***	1.57	<0.001	20.71 ***	2.10	<0.001
Assessing credibility (METASPAM)				25.49 ***	1.50	<0.001	25.65 ***	1.50	<0.001
Enjoyment of reading (JOYREAD)				10.65 ***	1.40	<0.001	10.59 ***	1.40	<0.001
Perception of competence (SCREADCOMP)				12.96 ***	1.69	<0.001	12.96 ***	1.69	<0.001
Perception of difficulty (SCREADDIFF)				-7.93 ***	1.55	<0.001	-7.75 ***	1.55	<0.001
Working motive and mastery (WORKMAST)				4.68 **	1.50	0.002	4.60 **	1.50	0.002
Mastery-approach orientation (MASTGOAL)				-8.60 ***	1.49	<0.001	-8.48 ***	1.49	<0.001
General fear of failure (GFOFAIL)				7.58 ***	1.27	<0.001	7.51 ***	1.27	<0.001
Interaction Gender - Summarizing							-5.39 *	2.68	0.044
Random Effects
σ2	9235.451			5580.855			5573.184
τ00	1501.617 CNTSCHID			220.462 CNTSCHID			220.429 CNTSCHID
τ11				66.722 CNTSCHID.CESCS			67.814 CNTSCHID.CESCS
ρ01				0.915 CNTSCHID			0.907 CNTSCHID
ICC	0.140			0.046			0.046
N	131 CNTSCHID			131 CNTSCHID			131 CNTSCHID
Observations	3309			3309			3309
Marginal R2 / Conditional R2	0.000 / 0.140			0.447 / 0.472			0.447 / 0.473
p<0.05   ** p<0.01   *** p<0.001

Discussion

Using hierarchical linear models, this project examined the student- and school-level factors that correlate to reading students’ literacy in Canada, Mexico, and United States using the information provided for PISA 2018. The result of multilevel modeling allows analyzing a wide range of variables that can contribute to underlining the complexities of factors that explain the variability in students’ performance. First, null models show that taking into account the nested structure is essential to improve the analysis of effects on student performance; however, the proportion of variability explained by school characteristics (level 2) vary across countries. Thus, while in Mexico, the 35% of the total variance in student reading performance relate to the school where those students attend, in Canada and the US, the proportion of variance explained by school-level is lower 13.63% and 13.99%, respectively.

Second, results indicates that student-level variables, such as socioeconomic status, gender, age, and grade repetition are significant effects but explain moderately to slightly the variability on reading score. Thus, the students-level variables reduced in 8.65%, 12.80%, and 1.64% the variability among students in Canada, U.S, and Mexico, respectively. In particular, our analysis allows to confirmed the first hypothesis, thus we found a there are a direct and positive effect of socioeconomic status on reading performance, thus students enrolled in schools with disadvantages socioeconomic status obtained lower scores in reading performance.

Third, school-level variables such as school socioeconomic status and proportion of students with positive and significant effects across countries, therefore attending a school with higher socioeconomic status and a lower proportion of students with reading disadvantages, predicts higher reading performance. These results suggested that favorable conditions in terms of socioeconomic background and reading performance at the school level are determinant factors for increasing individual outcomes and these effects were consistent across different countries. Furthermore, other variables, such as type of school (public or private) and area (rural or urban), were significant predictors in Canada, but not in U.S and Mexico.

Fourth, metacognitive strategies (understanding and remembering, summarizing, and assessing credibility), attitudes toward reading, and reading self-concept are positive and significant predictors of reading literacy. At the same time, the perception of difficulty is a negative predictor of reading performance. Overall, these effects are consistent across the countries analyzed. In contrast, the effects of achievement goals and motivation scales vary across countries. Additionally, results support our second hypothesis; therefore, results suggest that students with higher socioeconomic status have a positive a higher effect on reading performance than those with disadvantage in socioeconomic status; however, the magnitude of socioeconomic status at student level effect is lower than those observed before controlled by meta-cognitive strategies and motivation scales. Further, we did not find evidence of random effects of metacognitive strategies and motivation on reading performance; then, the results support our third hypothesis. Finally, results suggested that the magnitude of the effect of enjoyment of reading, perception of difficulty, assessing credibility, and perception of competence on reading performance vary depending on students’ gender, which supports our fourth hypothesis.