GA Effect @ WAW

Mark Edmondson
June 17th, 2015

Appendix

The GA Effect App

Your Numbers Are Lying

About M.E.

  • From Cornwall, UK.
  • MSci Physics, Kings College London 2002
  • Moved to Copenhagen 2010
  • Worked in digital agencies since 2007: Neutralize, Guava, NetBooster, Wunderman
  • 2015: New baby, GDE and RStudio advocate
  • Say hello: @HoloMarkeD and mark@markedmondson.me
  • Blog: http://markedmondson.me

Motivation - Your numbers are lying (maybe)


If you report…

  • “Our display campaign created 5% more sales for Feb over Jan”
  • “Our SEM for sunscreen sales are 250% higher in Q3 over Q2”
  • “Our redesign created 50000 more visits”

Motivation - Your numbers are lying (maybe)


…and you are not considering:

  • A peak in traffic without ruling out random noise
  • Seasonality
  • What would have happened if no redesign had taken place

Intro: GA Effect

An app built in R and Shiny

https://gallery.shinyapps.io/ga-effect/

demo-gaeffect

Aims: GA Effect

  • Estimate impact of events in real and % terms.
  • Be quick and easy to use.
  • Work with Google Analytics data.
  • Be available to anyone.
  • Robust answer backed up by statistics.
  • Present results in attractive plots.

Example Demos

Setup demo (live)

Mobilegeddon

Eurovision

Mobilegeddon Setup

Google SEO mobile update April 22nd

mobilegeddon-setup

Mobilegeddon Results

Significant effect found of +12% SEO traffic

mobilegeddon-result

Eurovision Setup

Eurovision Denmark announced 18th May, 2013 - did it affect a Danish travel website?

eurovision-setup

Eurovision Results

Non-signficant effect found of +7% (95% confidence interval crosses 0)

eurovision-result

How To Build GA Effect

CausalImpact

Shinyapps.io

shinyga()

Dygraphs

CausalImpact

An R package developed by Kay Brodersen at Google

Constructs a time-series model before and after a set event, which is then compared to what actually happened.

The difference between the two is attributed to the event - can A/B test between model and reality.

Details in the appendix

causalImpact

Shinyapps.io

Managed hosting of Shiny apps - www.shinyapps.io

Alternative to running your own Shiny server.

Deploy from within RStudio.

Free tier available.

shinyapps.io

shinyga()

My first public R package!

Aims to make the Google Authentication of GA Effect available to any Shiny app, provides macros for quick(er) start apps.

Ambition to use for all Google APIs, not just Google Analytics.

## install shinyga
library(shinyga)

## in sever.r generate GA profile table and auth token
auth <- doAuthMacro(input, output, session, securityCode, client.id, client.secret)

##in ui.r use these to authenticate user
uiOutput("AuthGAURL")
authDropdownRow()

Dygraphs

Popular javascript time-series plotting library adapted for R:

Uses new htmlwidgets() R package to bind any javascript viz library (d3 etc.) to Shiny R code.

dygraphs

The Future

Build your own app?

Improvements

  • Use other data sources - other GA accounts, CRM, Adobe Analytics, GoogleDocs.
  • Allow more control time-series e.g. Google Trends
  • Auto-discover segments.
  • Exporting and/or create API of results - export into DMP etc.
  • Pre-defined segments e.g. SEO performance app.
  • Attribution measurement
  • Other statistics - e.g. Twitter's breakout detection

Thank You!

Keep in touch:

@HoloMarkeD

markedmondson.me

—

This presentation is available at URL:

http://rpubs.com/MarkeD/ga-effect

—

Links to further reading:

Appendix

Appendix: Resources

Deeper into CausalImpact 1

Constructs model from three sources:

  • The response time-series pre event
  • Other time-series predictive of response (i.e. same product, different country) - assumes no spill-over effect
  • Bayesian prior knowledge of model parameters (e.g. seasonality, data volatility default = 0.01)

Traditional “difference in difference”“ approach assumes same trend.

Difference in difference

Deeper into CausalImpact 2

CausalImpacts state-space model allows flexible choices in which trend between each data point - allows examination of model trend.

Example from paper shows response (Y) and two predictor variables (X1, X2)

causalImpact