## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

Performance comparison between Snabb versions. Compares scores with various benchmarks, workloads, configurations, and software dependency versions. Results are reported as probability density graphs showing the distribution of benchmark scores (measured in Gbps for iperf and Mpps for l2fwd).

The overall purpose of this report is to highlight the relative strengths and weaknesses between Snabb software versions. This is particularly intended for evaluating how experimental changes affect performance and reliability.

This report starts with a very broad summary and then drills down on iperf and l2fwd results separately.

Overall summary

Split by benchmark

iperf

Configuration

Guest kernel

QEMU

Success and failure

Success rate:

Sucesses:

##           config                        qemu          kernel     
##  base        :3145   2.1.3                :1800   3.18.29:12585  
##  filter      :   0   2.1.3-with-snabbpatch:1800                  
##  ipsec       :3150   2.2.1                :1800                  
##  l2tpv3      :3140   2.3.1                :1800                  
##  l2tpv3_ipsec:3150   2.4.1                :1795                  
##                      2.5.1                :1790                  
##                      2.6.0                :1800

Failures:

##           config                        qemu         kernel    
##  base        :   5   2.1.3                :450   3.18.29:3165  
##  filter      :3150   2.1.3-with-snabbpatch:450                 
##  ipsec       :   0   2.2.1                :450                 
##  l2tpv3      :  10   2.3.1                :450                 
##  l2tpv3_ipsec:   0   2.4.1                :455                 
##                      2.5.1                :460                 
##                      2.6.0                :450

l2fwd

Packet size

Configuration

DPDK

DPDK + configuration

QEMU

Success and failure

Success rate:

Sucesses:

##    config                        qemu         dpdk              snabb     
##  base : 688   2.1.3                : 316   1.8.0:   0   matrix     :1375  
##  noind:2027   2.1.3-with-snabbpatch: 345   16.04:1001   matrix-next:1395  
##  nomrg:1368   2.2.1                : 309   2.0.0:1016   next       :1313  
##               2.3.1                : 308   2.1.0:1051                     
##               2.4.1                :2143   2.2.0:1015                     
##               2.5.1                : 320                                  
##               2.6.0                : 342                                  
##  pktsize   
##  64 :1324  
##  256:2759  
##            
##            
##            
##            
##

Failures:

##    config                        qemu         dpdk              snabb     
##  base :2462   2.1.3                :2384   1.8.0:3780   matrix     :4925  
##  noind:7423   2.1.3-with-snabbpatch:2355   16.04:2779   matrix-next:4905  
##  nomrg:4932   2.2.1                :2391   2.0.0:2764   next       :4987  
##               2.3.1                :2392   2.1.0:2729                     
##               2.4.1                : 557   2.2.0:2765                     
##               2.5.1                :2380                                  
##               2.6.0                :2358                                  
##  pktsize   
##  64 :4976  
##  256:9841  
##            
##            
##            
##            
##