Causal Impact

Pertanyaan sering diajukan bagaimana melakukan evaluasi dampak perlakukan suatu program terhadap bisnis? Kebetulan rekan kolega saya memiliki data penjualan sebuah perusahaan 113 data dengan kurun waktu dari tanggal 25 Februari 2015 hingga 24 Mei 2015. (data dalam ribuan)

Beliau melakukan perombakan besar-besaran pada Struktur Organisasi departemen penjualan pada minggu kedua bulan April 2015 lebih tepatnya pada tanggal 10 April 2015, maka beliau bertanya apakah ada efek dari perombakan struktur organisasi terhadap tingkat penjualan (asumsi dampak penyesuaian struktur organisasi 1 minggu)

Kami mengumpulkan datanya terlebih dahulu

library(CausalImpact)
## Loading required package: bsts
## Loading required package: BoomSpikeSlab
## Loading required package: Boom
## Loading required package: MASS
## Loading required package: zoo
## 
## Attaching package: 'zoo'
## The following objects are masked from 'package:base':
## 
##     as.Date, as.Date.numeric
## Loading required package: xts
## Warning: replacing previous import 'stats::filter' by 'dplyr::filter' when
## loading 'CausalImpact'
## Warning: replacing previous import 'stats::lag' by 'dplyr::lag' when
## loading 'CausalImpact'
salesdata<-c(18972, 22725, 32330, 28747, 31390, 25494, 18796, 15486, 19761, 20198, 20560, 12177, 18276,21492, 23883, 25628, 29769, 30099, 21175, 18242, 16850, 22934, 21957, 10834, 24233, 23363,
27604, 34200, 25258, 28429, 20300, 16599, 15835, 17761, 16289, 10847, 19788, 17325, 27019,
35467, 28865, 27045, 21772, 17851, 14829, 17868, 21911, 12041, 16457, 19195, 34111, 32069,
31600, 31849, 19439, 14774, 17497, 17999, 13872,  8060, 15676, 15688, 20587, 31087, 25446,
22471, 21558, 14844, 14254, 14816, 12245, 10489, 21186, 22880, 44640, 39672, 41354, 30931,
25913, 22645, 22379, 28632, 18884, 15580, 27761, 25573, 32968, 44257, 48506, 38458, 31484,
19149, 26621, 29293, 25463, 18452, 25888, 26751, 36201, 50035, 42734, 38132, 30477, 21348,
23925, 29213, 30492, 16868, 28166, 28261, 39617, 46619, 44664)

time.points <- seq.Date(as.Date("2015-02-01"), by = 1, length.out = 113)
data <- zoo(cbind(salesdata), time.points)

Tampilan Data 5 terawal

##            salesdata
## 2015-02-01     18972
## 2015-02-02     22725
## 2015-02-03     32330
## 2015-02-04     28747
## 2015-02-05     31390
## 2015-02-06     25494

Tampilan Data 5 terakhir

##            salesdata
## 2015-05-19     16868
## 2015-05-20     28166
## 2015-05-21     28261
## 2015-05-22     39617
## 2015-05-23     46619
## 2015-05-24     44664

Plots data

Grafik dari data penjualan yang diberikan adalah sebagai berikut:

Dari analisa grafik kita akan mengamai kesulitan untuk melakukan analisa apakah perombakan struktur organisasi membawa dampak terhadap tingkat penjualan oleh karenanya kita membutuhkan tools Causal Impact.

Setting waktu Analisa dampak

pre.period <- as.Date(c("2015-02-01", "2015-04-15"))
post.period <- as.Date(c("2015-04-16", "2015-05-24"))

Kita setting waktu awal dari tanggal 2 Februari 2015 hingga tanggal 15 April 2015, pemberlakukan struktur organisasi baru tanggal 10 April 2015 (diasumsikan 5 hari penyesuaian terhadap struktur baru).

Analisa dampak baru berjalan pada tanggal 16 April hingga 24 Mei 2015….. kita coba analisa dampaknya

Hasil Analisa Dampak

Plotting hasil analisa

impact <- CausalImpact(data, pre.period, post.period)
plot(impact)
## Warning: Removed 113 rows containing missing values (geom_path).
## Warning: Removed 226 rows containing missing values (geom_path).

Dari grafik diatas tampak terdapat perbedaan yang signifikan antara sebelum perubahan struktur organisasi dengan setelah perubahan struktur organisasi, lihat grafik original, point wise dan cumulative setelah garis vertikal terputus-putus terdapat lonjakan drastis dalam tingkat penjualan. (garis vertikal adalah batas tanggal 16 April 2015).

Berapa efek atau dampak dari Perubahan Struktur Organisasi

summary(impact)
## Posterior inference {CausalImpact}
## 
##                          Average          Cumulative      
## Actual                   31231            1218006         
## Prediction (s.d.)        21097 (1438)     822788 (56075)  
## 95% CI                   [18507, 23971]   [721790, 934876]
##                                                           
## Absolute effect (s.d.)   1e+04 (1438)     4e+05 (56075)   
## 95% CI                   [7260, 12723]    [283130, 496216]
##                                                           
## Relative effect (s.d.)   48% (6.8%)       48% (6.8%)      
## 95% CI                   [34%, 60%]       [34%, 60%]      
## 
## Posterior tail-area probability p:   0.001
## Posterior prob. of a causal effect:  99.9%
## 
## For more details, type: summary(impact, "report")

Inilah jawaban yang ditunggu-tunggu, dari hasil analisa tersebut dapat dilihat bahwa dampak atau efek dari perubahan struktur organisasi terhadap tingkat penjualan secara relative 48% (baik secara rerata ataupun kumulatif) dengan galat baku 6.5%, jika kita gunakan tingkat kepercayaan 95% efek tersebut menjadi 38%.

Deskripsi Hasil Analisa secara Naratif

summary(impact, "report")
## Analysis report {CausalImpact}
## 
## 
## During the post-intervention period, the response variable had an average value of approx. 31.23K. By contrast, in the absence of an intervention, we would have expected an average response of 21.10K. The 95% interval of this counterfactual prediction is [18.51K, 23.97K]. Subtracting this prediction from the observed response yields an estimate of the causal effect the intervention had on the response variable. This effect is 10.13K with a 95% interval of [7.26K, 12.72K]. For a discussion of the significance of this effect, see below.
## 
## Summing up the individual data points during the post-intervention period (which can only sometimes be meaningfully interpreted), the response variable had an overall value of 1.22M. By contrast, had the intervention not taken place, we would have expected a sum of 0.82M. The 95% interval of this prediction is [0.72M, 0.93M].
## 
## The above results are given in terms of absolute numbers. In relative terms, the response variable showed an increase of +48%. The 95% interval of this percentage is [+34%, +60%].
## 
## This means that the positive effect observed during the intervention period is statistically significant and unlikely to be due to random fluctuations. It should be noted, however, that the question of whether this increase also bears substantive significance can only be answered by comparing the absolute effect (10.13K) to the original goal of the underlying intervention.
## 
## The probability of obtaining this effect by chance is very small (Bayesian one-sided tail-area probability p = 0.001). This means the causal effect can be considered statistically significant.

Heru Wiryanto, depok 5 Januari 2017