Introduction

The surge in remote work, digital nomadic lifestyles and international leisure travel means that buying a mobile phone SIM card is one of the first, vital purchases on arrival at destination. The island of Bali, Indonesia’s premier tourist destination, receives upwards of 5 million tourists each year and hosts tens of thousands of expatriate residents who rely on domestic telephony carriers, making the sale and purchase of mobile internet access a large, lucrative business.

Consequently, both providers and consumers want to know which service carrier provides the fastest internet speeds, the former to better position themselves in the market and price their product accordingly, the latter to make a better-informed purchase aligned to their business, work and travel needs. Nevertheless, there is so far no publicly available, independent, rigorous analysis comparing mobile telephony internet speeds among Indonesian domestic carriers in Bali to guide decision-making.

To fill the gap and inform providers and consumer decisions, this market research study compares, using internet speed measurements collected in Bali, the speeds of three domestic carriers - Telkomsel, Tri3 (IM3) and XL Axiata - to assess with rigor whether statistically significant speed differences exist among them. The analysis suggests that not all carriers are alike in the effective, measured internet speeds on offer

Data

The data were collected by measuring internet speeds on Android OS cellphones carrying Telkomsel, Tri and XL carrier SIM cards with the Meteor Speed Test application. They were collected on specific days and times in three, three-day rounds during 13-21 November 2024 in the Sanur district of Bali’s capital city of Denpasar according to the replicated Latin Squares design sequence outlined in Table 1.

For each Round / Day/ Time / Carrier factor combination, five speed tests were taken, each test consisting of one upload speed, one download speed and one latency measurement. Of the five tests, the two with the highest latency measurement were dropped, and a composite Speed score was created for each of the remaining three tests according to the following formula:

\[\left[\frac{Upload\ Speed + Download\ Speed}{Latency}\right]^2\]

The three Speed scores were then averaged, resulting in 27 Speed score instances, one for each of the 3 Round x 9 Day/Time/Carrier factor combinations. These data used in the analysis are displayed in the tabs on the right and available for download.

Methodology

To test differences in mean internet speeds among carriers, an analysis of variance (ANOVA) model in a 3 x 3 Latin Square Design (LSD) replicated three times is applied to the data, with Speed as the response variable. To account for the possible confounding effect of time-of-the-day and of day-of-the-week on internet speed, the LSD blocks the main effect factor Carrier with factor Time and with factor Day. In the model, blocking factor Day is nested within factor Round to account for the replication of the former along the latter.

Model factor significance and post-hoc comparisons of means using Tukey’s ‘Honest Significant Difference’ method are analyzed at the 90% confidence (\(\alpha = 0.10\)) level. The ANOVA table and model diagnostics are shown in the tabs on the right.

Data management, visualizations and analysis are conducted in the R programming language (version 4.1.0) and RStudio IDE (2024.09.1 Build +394) for Windows.

Results

The analysis shows that main effects factor Carrier is significant (p-value 0.0711), pointing to statistically significant differences in internet speeds in at least one of the three carrier combinations at 90% confidence (\(\alpha=0.10\)) level. None of the three other model factors, including blocking factors Time and Round:Day, are significant, meaning that time-of-the-day and day-of-the-week (as well as replication round) do not significantly impact internet speeds.

Model diagnostics show that residuals are uniform, normal, homoskedastic and non-correlated, affirming good model fit and the validity of the results obtained.

Tukey comparisons of mean speeds among carriers show a significant difference (p-value 0.067) between Telkomsel and XL at 90% confidence (\(\alpha=0.10\)) level. No statistically significant differences in internet speeds are found between Telkomsel and Tri, or between Tri and XL.

Conclusion

There are significant differences in mobile internet speeds among the carriers Telkomsel, Tri and XL. They are not all alike in this respect, so provider and buyer beware. The conclusions of the study are summarized below.

• Telkomsel offers significantly higher speeds than XL, with the Speed difference ranging between 1.0 and 4.7.

• No significant Speed difference exists between Telkomsel and Tri, or between Tri and XL.

• Overall variability in Speed is highest on Telkomsel and lowest on XL, Tri occupying the middle ground.

• Time-of-the-day and day-of-the-week do not significantly affect Speed.

A limitation of the study is that the data comes from a single, central test location. Therefore, the results of the analysis may not be fully applicable to the rural, intererior parts of the island. Another possible limitation is the relatively small sample size and length of time with which the study was conducted.