Measuring & Monitoring Internet Speed with R

devtools::install_github("hrbrmstr/speedtest") 

Downloading GitHub repo hrbrmstr/speedtest@master
from URL https://api.github.com/repos/hrbrmstr/speedtest/zipball/master
Installing speedtest
"C:/PROGRA~1/R/R-34~1.2/bin/x64/R" --no-site-file --no-environ --no-save  \
  --no-restore --quiet CMD INSTALL  \
  "C:/Users/rcleo/AppData/Local/Temp/RtmpwXfh7H/devtools36e05e812c57/hrbrmstr-speedtest-f635a99"  \
  --library="C:/Users/rcleo/Documents/R/win-library/3.4" --install-tests 

* installing *source* package 'speedtest' ...
** R
** tests
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
*** arch - i386
*** arch - x64
* DONE (speedtest)

library(speedtest)

Warning message:
In readLines(input, encoding = "UTF-8") :
  incomplete final line found on 'C:/Users/rcleo/Documents/R/win-library/3.4/raw/rmarkdown/templates/reserveReview/template.yaml'

library(stringi)
library(hrbrthemes)
library(ggbeeswarm)

Carregando pacotes exigidos: ggplot2

library(tidyverse)

-- Attaching packages --------------------------------------- tidyverse 1.2.0 --
v tibble  1.3.4     v purrr   0.2.4
v tidyr   0.7.2     v dplyr   0.7.4
v readr   1.1.1     v stringr 1.2.0
v tibble  1.3.4     v forcats 0.2.0
-- Conflicts ------------------------------------------ tidyverse_conflicts() --
x dplyr::filter() masks stats::filter()
x dplyr::lag()    masks stats::lag()

config <- spd_config()
servers <- spd_servers(config=config)
closest_servers <- spd_closest_servers(servers, config=config)
only_the_best_severs <- spd_best_servers(closest_servers, config)
glimpse(spd_download_test(closest_servers[1,], config=config))

|===========                                            | 20% ~1 s remaining     
|================                                       | 30% ~1 s remaining     
|======================                                 | 40% ~1 s remaining     
|===========================                            | 50% ~1 s remaining     
|=================================                      | 60% ~2 s remaining     
|======================================                 | 70% ~2 s remaining     
|============================================           | 80% ~2 s remaining     
|=================================================      | 90% ~2 s remaining     
|=======================================================|100% ~0 s remaining     Observations: 1
Variables: 15
$ url     <chr> "http://st1.zamix.com.br/speedtest/upload.php"
$ lat     <dbl> -22.5228
$ lng     <dbl> -44.1039
$ name    <chr> "Volta Redonda"
$ country <chr> "Brazil"
$ cc      <chr> "BR"
$ sponsor <chr> "SuperOnda/Zamix"
$ id      <chr> "6536"
$ host    <chr> "st1.zamix.com.br:8080"
$ url2    <chr> "http://st2.zamix.com.br/speedtest/upload.php"
$ min     <dbl> 15.35344
$ mean    <dbl> 25.90716
$ median  <dbl> 24.74659
$ max     <dbl> 40.07601
$ sd      <dbl> 7.048748

## Observations: 1
## Variables: 15
## $ url      "http://speed0.xcelx.net/speedtest/upload.php"
## $ lat      42.3875
## $ lng      -71.1
## $ name     "Somerville, MA"
## $ country  "United States"
## $ cc       "US"
## $ sponsor  "Axcelx Technologies LLC"
## $ id       "5960"
## $ host     "speed0.xcelx.net:8080"
## $ url2     "http://speed1.xcelx.net/speedtest/upload.php"
## $ min      14.40439
## $ mean     60.06834
## $ median   55.28457
## $ max      127.9436
## $ sd       34.20695
glimpse(spd_upload_test(only_the_best_severs[1,], config=config))

|==================                                     | 33% ~1 s remaining     
|===========================                            | 50% ~1 s remaining     
|====================================                   | 67% ~1 s remaining     
|=============================================          | 83% ~1 s remaining     
|=======================================================|100% ~0 s remaining     Observations: 1
Variables: 17
$ total_time     <dbl> 0.219
$ retrieval_time <dbl> 0
$ url            <chr> "http://189.50.144.249/speedtest/speedtest/upload.php"
$ lat            <dbl> -22.1124
$ lng            <dbl> -45.028
$ name           <chr> "Sao Lourenco"
$ country        <chr> "Brazil"
$ cc             <chr> "BR"
$ sponsor        <chr> "StarWeb"
$ id             <chr> "14377"
$ host           <chr> "189.50.144.249:8080"
$ url2           <chr> NA
$ min            <dbl> 4.096
$ mean           <dbl> 6.273526
$ median         <dbl> 6.250487
$ max            <dbl> 8.445378
$ sd             <dbl> 1.749677

## Observations: 1
## Variables: 18
## $ ping_time       0.02712567
## $ total_time      0.059917
## $ retrieval_time  2.3e-05
## $ url             "http://speed0.xcelx.net/speedtest/upload.php"
## $ lat             42.3875
## $ lng             -71.1
## $ name            "Somerville, MA"
## $ country         "United States"
## $ cc              "US"
## $ sponsor         "Axcelx Technologies LLC"
## $ id              "5960"
## $ host            "speed0.xcelx.net:8080"
## $ url2            "http://speed1.xcelx.net/speedtest/upload.php"
## $ min             6.240858
## $ mean            9.527599
## $ median          9.303148
## $ max             12.56686
## $ sd              2.451778

Welcome!
Here you will find daily news and tutorials about R, contributed by over 750 bloggers.
There are many ways to follow us -
By e-mail:
On Facebook:
If you are an R blogger yourself you are invited to add your own R content feed to this site (Non-English R bloggers should add themselves- here)
RSS Jobs for R-users

    Rays Research & Development Analyst
    PROGRAMMER/SOFTWARE DEVELOPMENT ENGINEER/COMPUTATIONAL AND MACHINE LEARNING SPECIALIST
    Quantitative Econometrician @ San Francisco, California, U.S.
    Postdoctoral Research Fellow in Healthcare Systems Engineering @ Maryland, U.S.
    R Programmer & Statistician for Academic Research

Popular Searches
Recent Posts

    Measuring & Monitoring Internet Speed with R
    Creating integer64 and nanotime vectors in C++
    Stan Roundup, 10 November 2017
    .rprofile: Mara Averick
    Gold-Mining – Week 10 (2017)
    Recap: EARL Boston 2017
    Announcing “Introduction to the Tidyverse”, my new DataCamp course
    R live class | R with Database and Big Data | Nov 21-22 Milan
    How Happy is Your Country? — Happy Planet Index Visualized
    Formal ways to compare forecasting models: Rolling windows
    Introduction to Visualizing Asset Returns
    bridgesampling [R package]
    Calculating the house edge of a slot machine, with R
    Creating Reporting Template with Glue in R
    R / Finance 2018 Call for Papers

Other sites

    Jobs for R-users
    SAS blogs

Measuring & Monitoring Internet Speed with R
November 11, 2017
By hrbrmstr

inShare4
(This article was first published on R – rud.is, and kindly contributed to R-bloggers)

72
SHARES
Share
Tweet

Working remotely has many benefits, but if you work remotely in an area like, say, rural Maine, one of those benefits is not massively speedy internet connections. Being able to go fast and furious on the internet is one of the many things I miss about our time in Seattle and it is unlikely that we’ll be seeing Google Fiber in my small town any time soon. One other issue is that residential plans from evil giants like Comcast come with things like “bandwidth caps”. I suspect many WFH-ers can live within those limits, but I work with internet-scale data and often shunt extracts or whole datasets to the DatCave™ server farm for local processing. As such, I pay an extra penalty as a Comcast “Business-class” user that has little benefit besides getting slightly higher QoS and some faster service response times when there are issues.

Why go into all that? Well, I recently decided to bump up the connection from 100 Mb/s to 150 Mb/s (and managed to do so w/o increasing the overall monthly bill at all) but wanted to have a way to regularly verify I am getting what I’m paying for without having to go to an interactive “speed test” site.

There’s a handy speedtest-cli攼㹤愼㸰戼㹤攼㹤戼㸴㤼㸷, which is a python-based module with a command-line interface that can perform speed tests against Ookla’s legacy server. (You’ll notice that link forwards you to their new socket-based test service; neither speedtest-cli nor the code in the package being shown here uses that yet)

I run plenty of ruby, python, node and go (et al) programs on the command-line, but I wanted a way to measure this in R-proper (storing individual test results) on a regular basis as well as run something from the command-line whenever I wanted an interactive test. Thus begat speedtest攼㹤愼㸰戼㹤攼㹤戼㸴㤼㸷.
Testing the Need for Speed

After you devtools::install_github("hrbrmstr/speedtest") the package, you can either type speedtest::spd_test() at an R console or Rscript --quiet -e 'speedtest::spd_test()' on the command-line to see something like the following:

What you see there is a short-hand version of what’s available in the package:

    spd_best_servers: Find “best” servers (latency-wise) from master server list
    spd_closest_servers: Find “closest” servers (geography-wise) from master server list
    spd_compute_bandwidth: Compute bandwidth from bytes transferred and time taken
    spd_config: Retrieve client configuration information for the speedtest
    spd_download_test: Perform a download speed/bandwidth test
    spd_servers: Retrieve a list of SpeedTest servers
    spd_upload_test: Perform an upload speed/bandwidth test
    spd_test: Test your internet speed/bandwidth

The general idiom is to grab the test configuration file, collect the master list of servers, figure out which servers you’re going to test against and perform upload/download tests + collect the resultant statistics:

library(speedtest)
library(stringi)
library(hrbrthemes)
library(ggbeeswarm)
library(tidyverse)

config <- spd_config()

servers <- spd_servers(config=config)
closest_servers <- spd_closest_servers(servers, config=config)
only_the_best_severs <- spd_best_servers(closest_servers, config)

glimpse(spd_download_test(closest_servers[1,], config=config))
## Observations: 1
## Variables: 15
## $ url      "http://speed0.xcelx.net/speedtest/upload.php"
## $ lat      42.3875
## $ lng      -71.1
## $ name     "Somerville, MA"
## $ country  "United States"
## $ cc       "US"
## $ sponsor  "Axcelx Technologies LLC"
## $ id       "5960"
## $ host     "speed0.xcelx.net:8080"
## $ url2     "http://speed1.xcelx.net/speedtest/upload.php"
## $ min      14.40439
## $ mean     60.06834
## $ median   55.28457
## $ max      127.9436
## $ sd       34.20695

glimpse(spd_upload_test(only_the_best_severs[1,], config=config))
## Observations: 1
## Variables: 18
## $ ping_time       0.02712567
## $ total_time      0.059917
## $ retrieval_time  2.3e-05
## $ url             "http://speed0.xcelx.net/speedtest/upload.php"
## $ lat             42.3875
## $ lng             -71.1
## $ name            "Somerville, MA"
## $ country         "United States"
## $ cc              "US"
## $ sponsor         "Axcelx Technologies LLC"
## $ id              "5960"
## $ host            "speed0.xcelx.net:8080"
## $ url2            "http://speed1.xcelx.net/speedtest/upload.php"
## $ min             6.240858
## $ mean            9.527599
## $ median          9.303148
## $ max             12.56686
## $ sd              2.451778

set.seed(8675309)
bind_rows(
  closest_servers[1:3,] %>%
    mutate(type="closest"),
  only_the_best_severs[1:3,] %>%
    mutate(type="best"),
  filter(servers, !(id %in% c(closest_servers[1:3,]$id, only_the_best_severs[1:3,]$id))) %>%
    sample_n(3) %>%
    mutate(type="random")
) %>%
  group_by(type) %>%
  ungroup() -> to_compare
select(to_compare, sponsor, name, country, host, type)

map_df(1:nrow(to_compare), ~{
  spd_download_test(to_compare[.x,], config=config, summarise=FALSE, timeout=30)
}) -> dl_results_full

|==========                                            | 20% ~3 s remaining     
|================                                      | 30% ~6 s remaining     
|=====================                                 | 40% ~7 s remaining     
|===========================                           | 50% ~7 s remaining     
|================================                      | 60% ~7 s remaining     
|=====================================                 | 70% ~6 s remaining     
|===========================================           | 80% ~9 s remaining     
|================================================      | 90% ~7 s remaining     
|======================================================|100% ~0 s remaining     
|==========                                            | 20% ~4 s remaining     
|================                                      | 30% ~4 s remaining     
|=====================                                 | 40% ~4 s remaining     
|===========================                           | 50% ~5 s remaining     
|================================                      | 60% ~8 s remaining     
|=====================================                 | 70% ~12 s remaining    
|===========================================           | 80% ~12 s remaining    
|================================================      | 90% ~8 s remaining     
|======================================================|100% ~0 s remaining     
|==========                                            | 20% ~3 s remaining     
|================                                      | 30% ~13 s remaining    
|=====================                                 | 40% ~22 s remaining    
|===========================                           | 50% ~22 s remaining    
|================================                      | 60% ~26 s remaining    
|=====================================                 | 70% ~29 s remaining    
|===========================================           | 80% ~25 s remaining    
|================================================      | 90% ~14 s remaining    
|======================================================|100% ~0 s remaining     
|==========                                            | 20% ~3 s remaining     
|================                                      | 30% ~9 s remaining     
|=====================                                 | 40% ~31 s remaining    
|===========================                           | 50% ~50 s remaining    
|================================                      | 60% ~54 s remaining    
|=====================================                 | 70% ~47 s remaining    
|===========================================           | 80% ~35 s remaining    
|================================================      | 90% ~19 s remaining    
|======================================================|100% ~0 s remaining     
|==========                                            | 20% ~5 s remaining     
|================                                      | 30% ~5 s remaining     
|=====================                                 | 40% ~8 s remaining     
|===========================                           | 50% ~13 s remaining    
|================================                      | 60% ~16 s remaining    
|=====================================                 | 70% ~17 s remaining    
|===========================================           | 80% ~16 s remaining    
|================================================      | 90% ~9 s remaining     
|======================================================|100% ~0 s remaining     
|==========                                            | 20% ~4 s remaining     
|================                                      | 30% ~5 s remaining     
|=====================                                 | 40% ~12 s remaining    
|===========================                           | 50% ~14 s remaining    
|================================                      | 60% ~11 s remaining    
|=====================================                 | 70% ~20 s remaining    
|===========================================           | 80% ~14 s remaining    
|================================================      | 90% ~10 s remaining    
|======================================================|100% ~0 s remaining     
|============                                                  | 20% ~16 s remaining    
|==================                                            | 30% ~11 s remaining    
|========================                                      | 40% ~9 s remaining     
|===============================                               | 50% ~8 s remaining     
|=====================================                         | 60% ~9 s remaining     
|===========================================                   | 70% ~19 s remaining    
|=================================================             | 80% ~18 s remaining    
|=======================================================       | 90% ~11 s remaining    
|==============================================================|100% ~0 s remaining     
|============                                                  | 20% ~21 s remaining    
|==================                                            | 30% ~28 s remaining    
|========================                                      | 40% ~38 s remaining    
|===============================                               | 50% ~55 s remaining    
|=====================================                         | 60% ~57 s remaining    
|===========================================                   | 70% ~50 s remaining    
|=================================================             | 80% ~36 s remaining    
|=======================================================       | 90% ~20 s remaining    
|==============================================================|100% ~0 s remaining     
|============                                                  | 20% ~29 s remaining    
|==================                                            | 30% ~1 m remaining     
|========================                                      | 40% ~1 m remaining     
|===============================                               | 50% ~1 m remaining     
|=====================================                         | 60% ~1 m remaining     
|===========================================                   | 70% ~58 s remaining    
|=================================================             | 80% ~42 s remaining    
|=======================================================       | 90% ~22 s remaining    
|==============================================================|100% ~0 s remaining     

mutate(dl_results_full, type=stri_trans_totitle(type)) %>%
  ggplot(aes(type, bw, fill=type)) +
  geom_quasirandom(aes(size=size, color=type), width=0.15, shape=21, stroke=0.25) +
  scale_y_continuous(expand=c(0,5), labels=c(sprintf("%s", seq(0,150,50)), "200 Mb/s"), limits=c(0,200)) +
  scale_size(range=c(2,6)) +
  scale_color_manual(values=c(Random="#b2b2b2", Best="#2b2b2b", Closest="#2b2b2b")) +
  scale_fill_ipsum() +
  labs(x=NULL, y=NULL, title="Download bandwidth test by selected server type",
       subtitle="Circle size scaled by size of file used in that speed test") +
  theme_ipsum_rc(grid="Y") +
  theme(legend.position="none")

bind_rows(
  closest_servers[1:3,] %>% mutate(type="closest"),
  only_the_best_severs[1:3,] %>% mutate(type="best")
) %>%
  distinct(.keep_all=TRUE) -> to_compare
select(to_compare, sponsor, name, country, host, type)

map_df(1:nrow(to_compare), ~{
  spd_upload_test(to_compare[.x,], config=config, summarise=FALSE, timeout=30)
}) -> ul_results_full

|===========================                                                      | 33% ~17 s remaining    
|========================================                                         | 50% ~10 s remaining    
|======================================================                           | 67% ~6 s remaining     
|===================================================================              | 83% ~3 s remaining     
|=================================================================================|100% ~0 s remaining     
|===========================                                                      | 33% ~1 s remaining     
|========================================                                         | 50% ~1 s remaining     
|======================================================                           | 67% ~1 s remaining     
|===================================================================              | 83% ~1 s remaining     
|=================================================================================|100% ~0 s remaining     
|===========================                                                      | 33% ~8 s remaining     
|========================================                                         | 50% ~5 s remaining     
|======================================================                           | 67% ~3 s remaining     
|===================================================================              | 83% ~1 s remaining     
|=================================================================================|100% ~0 s remaining     
|===========================                                                      | 33% ~1 s remaining     
|========================================                                         | 50% ~1 s remaining     
|======================================================                           | 67% ~1 s remaining     
|===================================================================              | 83% ~1 s remaining     
|=================================================================================|100% ~0 s remaining     
|===========================                                                      | 33% ~3 s remaining     
|========================================                                         | 50% ~2 s remaining     
|======================================================                           | 67% ~1 s remaining     
|===================================================================              | 83% ~1 s remaining     
|=================================================================================|100% ~0 s remaining     
|===========================                                                      | 33% ~1 s remaining     
|========================================                                         | 50% ~1 s remaining     
|======================================================                           | 67% ~1 s remaining     
|===================================================================              | 83% ~1 s remaining     
|=================================================================================|100% ~0 s remaining     

ggplot(ul_results_full, aes(x="Upload Test", y=bw)) +
  geom_quasirandom(aes(size=size, fill="col"), width=0.1, shape=21, stroke=0.25, color="#2b2b2b") +
  scale_y_continuous(expand=c(0,0.5), breaks=seq(0,16,4),
                     labels=c(sprintf("%s", seq(0,12,4)), "16 Mb/s"), limits=c(0,16)) +
  scale_size(range=c(2,6)) +
  scale_fill_ipsum() +
  labs(x=NULL, y=NULL, title="Upload bandwidth test by selected server type",
       subtitle="Circle size scaled by size of file used in that speed test") +
  theme_ipsum_rc(grid="Y") +
  theme(legend.position="none")