XX AB test report

Backgroud

The test started from 2014-08-01 and finished on 2014-09-01. What does it do? well…it does:

  1. this
  2. and that
  3. and that and that and that

KPI change

DAU

Average DAU for control group is 1.0360067 × 106, for test group is 1.0388995 × 106. The difference is significant.

##      case0             case2        
##  Min.   : 961707   Min.   : 965937  
##  1st Qu.:1007977   1st Qu.:1009404  
##  Median :1042671   Median :1045954  
##  Mean   :1036007   Mean   :1038900  
##  3rd Qu.:1059397   3rd Qu.:1067642  
##  Max.   :1099283   Max.   :1098655

plot of chunk dau

DARPPU

The averate ARPU from control group is 0.0505756, from test group is 0.0300115. Below is the boostraped ARPU from these two groups:

plot of chunk boostrap_result