Measuring Codex Performance for Content Delivery

(aka Codex Benchmarks)

Intro

  • Why?
    • performance is a key aspect of a storage system;
    • want to understand how Codex performs.
  • What?
    • Content delivery: download performance.
    • Download performance: latency, throughput.
    • Codex aims at supporting large files;
      • download speed (\(\text{MB/s}\)) is dominant.

Baseline

  • Quality baseline: easier to know where you stand;
    • faster: good;
    • slower: maybe not so good.
  • Decentralized, large-file content distribution:
    • Bittorrent;
    • IPFS.
  • Bittorrent: de facto standard;
    • been used for a very long time;
    • fast and efficient (more so than IPFS);
    • several open source implementations.

Baseline

Static Dissemination Experiment

  • Static dissemination experiment.
  • Network of size \(n\);
    • split into \(s\) seeders, \(l = n - s\) leechers;
    • seeder ratio \(r = \frac{s}{n}\).
  • Experiment:
    • generate file \(F\) of size \(b\);
    • upload \(F\) to each seeder;
    • fire up all leechers “at the same time”;
    • measure time to download \(F\) at leechers.

Static Dissemination Experiment

Static Dissemination Experiment

Static Dissemination Experiment

Static Dissemination Experiment

Static Dissemination Experiment

Static Dissemination Experiment

Static Dissemination Experiment

  • Parameters:
    • File sizes: \(b \in {100\text{MB}, 1\text{GB}, 5\text{GB}}\);
    • Network sizes: \(n \in {2, 8, 16, 32}\);
    • Seeder ratios: \(0.5, 0.25, 0.125, 0.0625, 0.03125\) (depending on \(n\)).
  • Hardware:
    • CPX31 Hetzner VMs (4 vCPU, 8GB RAM);
    • \(\sim 4\text{Gbps}\) point-to-point bandwidth.

Results - Download Speed

Results - Relative (Median) Speed

Median downlaod time ratio for Codex and Deluge

Next

  • Debugging, debugging…
  • larger experiments (networks, files);
  • dynamic network experiments, with churn and faults.

Thank You!