Measuring Codex Performance for Content Delivery
(aka Codex Benchmarks)
Intro
- Why?
- performance is a key aspect of a storage system;
- want to understand how Codex performs.
- What?
- Content delivery: download performance.
- Download performance: latency, throughput.
- Codex aims at supporting large files;
- download speed (\(\text{MB/s}\)) is dominant.
Baseline
- Quality baseline: easier to know where you stand;
- faster: good;
- slower: maybe not so good.
- Decentralized, large-file content distribution:
- Bittorrent;
- IPFS.
- Bittorrent: de facto standard;
- been used for a very long time;
- fast and efficient (more so than IPFS);
- several open source implementations.
Baseline
- For our baseline, we picked Deluge:
- well-known, lots of users despite small market share (1%);
- can be run as a daemon, has open source client libraries to interact with daemon remotely;
- based on libtorrent (rasterbar).
Static Dissemination Experiment
- Static dissemination experiment.
- Network of size \(n\);
- split into \(s\) seeders, \(l = n - s\) leechers;
- seeder ratio \(r = \frac{s}{n}\).
- Experiment:
- generate file \(F\) of size \(b\);
- upload \(F\) to each seeder;
- fire up all leechers “at the same time”;
- measure time to download \(F\) at leechers.
Static Dissemination Experiment
Static Dissemination Experiment
Static Dissemination Experiment
Static Dissemination Experiment
Static Dissemination Experiment
Static Dissemination Experiment
Static Dissemination Experiment
- Parameters:
- File sizes: \(b \in {100\text{MB}, 1\text{GB}, 5\text{GB}}\);
- Network sizes: \(n \in {2, 8, 16, 32}\);
- Seeder ratios: \(0.5, 0.25, 0.125, 0.0625, 0.03125\) (depending on \(n\)).
- Hardware:
- CPX31 Hetzner VMs (4 vCPU, 8GB RAM);
- \(\sim 4\text{Gbps}\) point-to-point bandwidth.
Results - Download Speed
Results - Relative (Median) Speed
Median downlaod time ratio for Codex and Deluge
Next
- Debugging, debugging…
- larger experiments (networks, files);
- dynamic network experiments, with churn and faults.
Thank You!
- Benchmarks repo: github.com/codex-network/codex-benchmarks
- RPubs Notebook with Data: https://rpubs.com/giuliano_mega/1266876
- Blog post: https://blog.codex.storage/measuring-codex-performance-for-content-delivery/
Measuring Codex Performance for Content Delivery (aka Codex Benchmarks)