Summary

Indonesia ranks low in regional and international education assessments. To boost economic growth and reduce reliance on primary commodities, it must equip its workforce with higher skills. This study examines how computer ownership and internet access affect the test scores of 15-year-olds in the 2018 OECD PISA survey, using an ANCOVA model that accounts for student wealth disparities. Results show both factors positively impact scores, though the effect varies by wealth level. These insights can guide education policy and government spending to improve learning outcomes.

Introduction

Indonesia, a middle-income country, faces the critical challenge of equipping its workforce with the skills needed for a more advanced, diversified economy. Both the Indonesian government and the World Bank have noted that the current workforce lacks the necessary training and technical expertise to transition from agriculture and extractive industries to high-value-added sectors. Without addressing this gap, economic growth will be constrained.

While improving education nationwide is a long-term effort, certain interventions may accelerate progress. This study examines whether computer ownership and internet access-adjusted for wealth levels-can enhance educational achievement in Indonesia.

Using data from the 2018 Programme for International Student Assessment (PISA) conducted by the OECD every three years, we analyze the combined test scores of 15-year-olds in reading, mathematics, and science. By linking these scores to data on student computer ownership, internet access, and wealth, we assess their impact on academic performance.

Watch the video on the left to learn more about the 2018 PISA.

Researched and engineered at Decision Analytics Hub
by Mauricio Claudio | DecisionAnalyticsHub@proton.me

Data

The PISA survey data for this study comes from the learningtower library (v1.0.1) accessed with the R statistical language and RStudio IDE (v2023.12.0 Build 369). The 2018 Indonesia dataset includes 11,819 observations and contains the following variables:

• computer and internet (binary categorical variables) indicating computer ownership and internet access, respectively.

• mean_TestScore (continuous numerical variable), representing the average of individual math, reading, and science test scores.

• wealth (continuous numerical covariate), a z-score where zero represents the mean, and ?1 and ?2 correspond to one and two standard deviations from the mean, covering approximately 68% and 95% of the population, respectively.

A data summary and multivariate plots are available in the right panels. The downloadable plots are interactive so make sure to hover on them. The dataset used in this analysis can be inspected and downloaded as a .csv file there too.

Methodology

To evaluate how computer ownership and internet access influence educational achievement while accounting for wealth, we use a crossed, fixed-effects factor ANCOVA model with unequal slopes. The model includes two binary factor variables - computer and internet and one covariate - wealth. The initial full ANCOVA model includes three main effects, three two-way interactions, and one three-way interaction. The final, simplified model excludes the three-way interaction, as it is not statistically significant at the 0.05 level.

We fit the model using the Anova() function from the car package, applying Type 3 sums of squares. Pairwise mean comparisons, contrasts, and confidence intervals are calculated at the 0.05 level using the Bonferroni method. The final ANCOVA model table and model fit diagnostics are displayed on the right.

Results

The final ANCOVA model indicates that the three two-way interactions - Computer ownership with Internet access, Computer ownership with Wealth level and Internet access with Wealth level - are significant at the 95% confidence level.

Interaction between Computer ownership & Internet access
Mean test scores differ significantly across the four treatment groups. Unsurprisingly, students without a computer or internet score the lowest (378 points), while those with both score the highest (421 points). All six pairwise contrasts among the the four factor combinations are significant, ranging from a 43-point increase for students with both a computer and internet compared to those with neither, to a 9-point decrease for students with internet but no computer compared to those with a computer but no internet. These results suggest that computer ownership has a greater impact on test scores than internet access.

Interaction between Computer ownership & Wealth level
Mean test scores vary significantly across wealth levels (-1, 0, +1, and +2 SD from the mean), with differences increasing as wealth rises. Scores range from 376 points for students without a computer at -1 SD wealth to 461 points for those with a computer at +2 SD wealth. All contrasts are significant except at -2 SD wealth, with score increases ranging from 16 points at -1 SD wealth to 48 points at +2 SD wealth. These findings indicate that wealth amplifies the positive effect of computer ownership on test scores.

Interaction between Internet access & Wealth level
Mean test scores also rise significantly with increasing wealth. Scores range from 378 points for students without internet at -1 SD wealth to 451 points for those with internet at +2 SD wealth. All contrasts are significant except at -2 SD wealth, with increases ranging from 12 points at -1 SD wealth to 28 points at +2 SD wealth. These findings suggest that, while wealth enhances the benefits of internet access, the effect is less pronounced than for computer ownership.

The dashed line in the plots represents the mean (403 points) of all test scores.

Conclusion

Computer ownership and internet access are significantly associated with higher test scores at all but the lowest wealth levels, increasing scores by approximately 30 and 20 points, respectively, and 43 points combined at the mean wealth level. The relative impact of these factors is greater at higher wealth levels and smaller at lower ones.

For students without a computer or internet, test scores rise by about 11 points per standard deviation (SD) increase in wealth. Those with internet but no computer see a 13-point rise, while students with a computer but no internet experience a 17-point increase. The highest gains - 27 points per SD rise in wealth - come from students with both a computer and internet access.

When choosing between investments, decision-makers should prioritize computer ownership, which has a consistently greater impact on test scores. When investing in both, a 55/45 percent split in favor of computers reflects their stronger influence. Additionally, wealth creation should be a parallel focus, given its amplifying effect on test scores when combined with computer and internet access.

This study has some limitations. Using a composite test score for math, reading, and science obscures the distinct effects on each subject. Additionally, the lack of gender-based analysis is a gap. Preliminary findings suggest that disaggregating scores by subject and gender could further strengthen the study.