All times are in seconds (lower is better).

Install packages

Installation of R packages is significantly affected by disk I/O, but still quite usable under EFS.

task EBS EFS EFS1 NFS
Install MASS 9.4650 22.1605 17.3410 16.5430
Install lattice 12.0225 19.0915 16.3065 17.1565

One notable outlier is BH, which is a worst-case scenario for distributed filesystems: it contains nearly 13,000 small files (C++ header files that comprise a significant subset of the Boost library). Installing BH on EFS is so much slower than usual that users commonly think the session has hung.

task EBS EFS EFS1 NFS
Install BH 18.169 289.954 179.913 147.0115

Write CSV

Each of the following tests writes 100MB of CSV data, distributed over different numbers of files. The greater the number of files, the smaller each individual CSV file is; for example, 100MB over 1,000 files results in each file being 100KB, while 100MB over 10,000 files results in each file being 10KB.

The collective overhead of EFS increases as we write to greater numbers of smaller files.

task EBS EFS EFS1 NFS
Write CSV, 100MB over 10 files 0.8010 2.2680 1.5900 1.2770
Write CSV, 100MB over 100 files 1.0560 5.0540 3.3140 2.1395
Write CSV, 100MB over 1000 files 1.1120 21.8030 12.6815 7.0555
Write CSV, 100MB over 10000 files 3.5695 103.6645 63.6930 60.7880

Read CSV

task EBS EFS EFS1 NFS
Read CSV, 10KB 0.0120 0.0185 0.0170 0.0105
Read CSV, 1MB 0.0185 0.0255 0.0235 0.0105
Read CSV, 100MB 1.0870 0.8110 0.7075 0.3795
Read CSV, 1GB 21.1810 6.4385 5.6205 4.0140

task EBS EFS EFS1 NFS
Read CSV, 100MB over 10 files 1.5965 0.9695 0.6375 0.4025
Read CSV, 100MB over 100 files 1.2990 1.6870 1.3350 0.6340
Read CSV, 100MB over 1000 files 2.3120 8.1435 5.8155 1.9425
Read CSV, 100MB over 10000 files 15.0200 55.2790 44.3640 14.8850

FST reads

task EBS EFS EFS1 NFS
FST random reads, 100MB over 10*10MB reads 0.4095 0.6040 0.5560 0.2525
FST random reads, 100MB over 100*1MB reads 1.1325 1.1365 0.7495 0.3260
FST random reads, 100MB over 1000*100KB reads 2.0470 3.7160 2.6990 0.9765
FST random reads, 100MB over 10000*10KB reads 4.9865 22.5675 17.7240 3.2165

Realistic CSV reads

task EBS EFS EFS1 NFS
Read 14 days of CRAN logs with fread 60.2925 61.418 61.1495 66.8105

task EBS EFS EFS1 NFS
Sample 5000 rows from each of 14 CRAN logs with vroom 37.0815 38.6115 38.0275 39.599

Parallel sequential reads/writes, 1GB

task parallelism EBS EFS EFS1 NFS
DD write, 1GB 2 39.4265 19.0235 18.9250 39.5620
DD write, 1GB 4 80.4830 39.5915 39.4865 80.5135
DD write, 1GB 8 161.4355 79.0170 79.0205 162.2220
DD write, 1GB 16 322.3420 158.5270 158.4390 323.2215

task parallelism EBS EFS EFS1 NFS
DD read, 1GB 2 39.5040 5.7565 5.8000 7.4420
DD read, 1GB 4 80.7790 13.4225 13.3690 19.7735
DD read, 1GB 8 161.6085 26.7745 26.8435 39.4490
DD read, 1GB 16 323.2530 53.7765 53.7860 210.9845

Parallel sequential reads/writes, 10MB over 1000 files

task parallelism EBS EFS EFS1 NFS
DD write, 10MB over 1000 files 2 2.5725 12.0615 7.8355 9.1030
DD write, 10MB over 1000 files 4 4.8450 32.7465 19.4280 11.3130
DD write, 10MB over 1000 files 8 9.6255 65.2505 38.9615 22.5835
DD write, 10MB over 1000 files 16 19.0635 140.1085 79.4660 45.0220

task parallelism EBS EFS EFS1 NFS
DD read, 10MB over 1000 files 2 3.1410 5.9605 5.272 3.3620
DD read, 10MB over 1000 files 4 4.7970 6.7485 6.350 5.9355
DD read, 10MB over 1000 files 8 9.2485 11.5030 11.139 11.7950
DD read, 10MB over 1000 files 16 18.5015 22.4510 21.792 24.2700