Comparaison de Pacogen - CASA - SPLCATool

Summary of the experimental data

data2 <- read.csv(file = "/home/aymeric/Dropbox/These/PhD/RProject/DataAnalysuis/Data/dataSummary.csv", 
    sep = ";")
summary(data2[2:9])

##  Pacogen_No_Min_Nb_conf Pacogen_No_Min_Time Pacogen_Nb_conf
##  Min.   :  0.0          Min.   :      0     Min.   :  2.0  
##  1st Qu.:  7.0          1st Qu.:   2365     1st Qu.:  6.0  
##  Median :  9.0          Median :   9855     Median :  8.0  
##  Mean   : 12.3          Mean   : 132074     Mean   : 11.9  
##  3rd Qu.: 13.0          3rd Qu.:  52205     3rd Qu.: 12.0  
##  Max.   :180.0          Max.   :9245630     Max.   :180.0  
##   Pacogen_Time       CASA_Nb_Conf     CASA_Time        SPLCAT_Nb_Conf 
##  Min.   :       0   Min.   :  2.0   Min.   :       0   Min.   :  0.0  
##  1st Qu.:    9652   1st Qu.:  6.0   1st Qu.:     245   1st Qu.:  8.0  
##  Median :   42465   Median :  8.0   Median :     718   Median : 10.0  
##  Mean   :  525447   Mean   : 11.8   Mean   :  101564   Mean   : 13.8  
##  3rd Qu.:  210395   3rd Qu.: 12.0   3rd Qu.:    6410   3rd Qu.: 15.0  
##  Max.   :36675270   Max.   :180.0   Max.   :10012246   Max.   :183.0  
##   SPLACT_Time  
##  Min.   :   0  
##  1st Qu.: 166  
##  Median : 326  
##  Mean   : 611  
##  3rd Qu.: 759  
##  Max.   :8873

Columns labels :

• Pacogen_No_Min_Nb_conf : Number of configuration returned by pacogen without minimisation
• Pacogen_No_Min_Time : Generation time in ms took by pacogen without minimisation (ms)
• Pacogen_Nb_conf : Number of configuration returned by pacogen with minimisation
• Pacogen_Time : Generation time in ms took by pacogen with minimisation (ms)
• CASA_Nb_Conf : Number of configuration returned by casa
• CASA_Time : Generation time in ms took by Casa (ms)
• SPLCAT_Nb_Conf : Number of configuration returned by SPLCAT
• SPLACT_Time : Generation time in ms took by SPLCAT (ms)

Comparison of the tools in term of number of configurations

Pacogen vs SPCLATool

vectHeight <- c(18, 52, 71, 30, 42, 3, 0, 0, 1)
vectLabel <- c("[-40%;-30%[", "[-30%;-20%[", "[-20%;-10%[", "[-10%;0%[", "[0%]", 
    "]0%;10%[", "[10%;20%[", "[20%;30%[", "[30%;40%[")
barplot(cex.lab = 1.3, cex.axis = 1.3, cex.names = 0.77, vectHeight, ylim = c(0, 
    80), axisnames = TRUE, axis.lty = 1, names.arg = vectLabel, border = TRUE, 
    xlab = "Difference of the number of configurations Pacogen / SPLCAT [%]", 
    ylab = "Number of Model", axes = TRUE)
abline(v = 5, col = "red", lwd = 2)

Left side of the line : Pacogen performs better than splcat. Right side of the line SPLCAT performs better than Pacogen

Pacogen vs CASA

vectHeight <- c(0, 2, 17, 15, 151, 16, 11, 9, 1)
vectLabel <- c("[-40%;-30%[", "[-30%;-20%[", "[-20%;-10%[", "[-10%;0%[", "[0%]", 
    "]0%;10%[", "[10%;20%[", "[20%;30%[", "[30%;40%[")
barplot(cex.lab = 1.3, cex.axis = 1.3, cex.names = 0.77, vectHeight, ylim = c(0, 
    150), axisnames = TRUE, axis.lty = 1, names.arg = vectLabel, border = TRUE, 
    xlab = "Difference of the number of configurations Pacogen / CASA [%]", 
    ylab = "Number of Model", axes = TRUE)
abline(v = 5, col = "red", lwd = 2)

Left side of the line : Pacogen performs better than CASA. Right side of the line CASA performs better than Pacogen

Comparison of the tools in term of resolution time

Pacogen vs SPCLATool

A value between [1:10] means that SPLCAT is between 1 to 10 times quicker than pacogen

vectHeight <- c(2, 30, 114, 36, 30, 7, 1, 0)
vectLabel <- c("[0;1[", "[1;10[", "[10;50[", "[50;100[", "[100;400[", "[400;700[", 
    "[700;1000[", "[1000;10000[")

barplot(cex.lab = 1.3, cex.axis = 1.3, cex.names = 0.77, vectHeight, ylim = c(0, 
    120), axisnames = TRUE, axis.lty = 1, names.arg = vectLabel, border = TRUE, 
    xlab = "Ratio between SPLCAT resolution time and Pacogen resolution time", 
    ylab = "Number of models", axes = TRUE)

abline(v = 1, col = "red", lwd = 2)

A value between [1:10] means that SPLCAT is between 1 to 10 times quicker than pacogen

Left side of the line : Pacogen performs better than splcat. Right side of the line SPLCAT performs better than Pacogen

Pacogen vs CASA

vectHeight <- c(11, 111, 94, 4, 0, 0, 1, 0)
vectLabel <- c("[0;1[", "[1;10[", "[10;50[", "[50;100[", "[100;400[", "[400;700[", 
    "[700;1000[", "[1000;10000[")

barplot(cex.lab = 1.3, cex.axis = 1.3, cex.names = 0.77, vectHeight, ylim = c(0, 
    120), axisnames = TRUE, axis.lty = 1, names.arg = vectLabel, border = TRUE, 
    xlab = "Ratio between CASA resolution time and Pacogen resolution time", 
    ylab = "Number of models", axes = TRUE)

abline(v = 1, col = "red", lwd = 2)

Left side of the line : Pacogen performs better than CASA. Right side of the line CASA performs better than Pacogen