R Statistics with MongoDB

• Introduction to Big Data, MongoSoup and R
• R statistics with MongoDB and Examples
• Summary & Questions

• databases using looser consistency models to store data
• German MongoDB as a Service: MongoSoup
  • cloudControl Add-On
  • currently running on AWS EU-Region (Ireland)
  • all features available: shared / dedicated hosting, replica set, sharding
  • 24/7 support available

• go to cloudControl: www.cloudcontrol.com
• add an account and a billing address
• create a new app, e.g. “rmongodb”
• install cloudControl command line tools: cctrlapp
• enable your preferred MongoSoup hosting: cctrlapp rmongodb/default addon.add mongosoup.medium
• go to the cloudControl Web-Console-AddOns and get your credentials

https://www.cloudcontrol.com/console/app/rmongodb

• backward-looking analysis is outdated
• today: quasi real-time analysis

• tomorrow: forward-looking predictive analysis

• more complex methods, more data available, more processing time required

• Check my Strata London Tutorial “Big Data Analyses with R”
  

• R is a free software environment for statistical computing and graphics
• offers tools to manage and analyze data
• standard statistical methods are implemented
• compiles and runs under different OS
• support via huge community

www.r-project.org

http://www.rstudio.com

 (5+5) - 1 * 3

[1] 7

 x <- 3
 x

[1] 3

 x^2 + 4

[1] 13

kmeans(dat, 4)

K-means clustering with 4 clusters of sizes 21, 18, 30, 31

Cluster means:
     [,1]    [,2]
1  0.7755  0.8509
2 -0.1557 -0.2305
3  1.2299  1.1472
4  0.1510  0.1507

Clustering vector:
  [1] 4 2 4 4 2 4 4 4 2 4 4 4 2 2 4 4 1 4 2 2 2 4 4 4 2 4 2 4 4 2 4 2 2 4 4
 [36] 4 4 4 4 4 4 4 4 2 4 2 2 4 2 2 1 1 1 1 3 1 3 3 3 1 1 3 3 3 3 1 3 1 3 3
 [71] 1 3 1 1 3 3 3 3 1 1 3 3 1 1 1 3 3 3 3 1 3 1 3 3 3 3 1 3 3 3

Within cluster sum of squares by cluster:
[1] 3.318 1.166 4.019 3.195
 (between_SS / total_SS =  83.0 %)

Available components:

[1] "cluster"      "centers"      "totss"        "withinss"    
[5] "tot.withinss" "betweenss"    "size"        

• developed by RStudio
• turns R analyses into interactive web applications that anyone can use
• let your users choose input parameters using friendly controls like sliders, drop-downs, and text fields
• easily incorporate any number of outputs like plots, tables, and summaries
• no HTML or JavaScript knowledge is necessary, only R

http://www.rstudio.com/shiny/

• SQL provides a standard language to filter, aggregate, group, sort data
• SQL in new places: Hive, Impala, …
• ODBC provides SQL interface to non-database data (Excel, CSV, text files)
• R stores relational data in data.frames (extended lists)

running SQL statements on R data frames

library(sqldf)
sqldf("select * from iris limit 2")

  Sepal_Length Sepal_Width Petal_Length Petal_Width Species
1          5.1         3.5          1.4         0.2  setosa
2          4.9         3.0          1.4         0.2  setosa

sqldf("select count(*) from iris")

  count(*)
1      150

• RMySQL package provides an interface to MySQL
• RPostgreSQL package provides an interface to PostgreSQL
• ROracle package provides an interface for Oracle
• RJDBC package provides access to databases through a JDBC interface
• RSQLite package provides access to SQLite
  (SQLite engine is included)
• One big problem:
  all packages read the full result in R memory

on CRAN there are two packages to connect R with MongoDB

• rmongodb supported by MongoDB, Inc.
  • powerful for big data
  • difficult to use due to BSON objects
• RMongo
  • easy to use
  • limited functionality
  • reads full results in R memory
  • does not work on MAC OS X

library(Rmongo)
mongo <- mongoDbConnect("cc_JwQcDLJSYQJb", "dbs001.mongosoup.de", 27017)
dbAuthenticate(mongo, username="JwQcDLJSYQJb", password="RSXPkUkXXXXX")

dbShowCollections(mongo)
dbGetQuery(mongo, "zips","{'state':'AL'}")
dbInsertDocument(mongo, "test_data", '{"foo": "bar", "size": 5 }')

dbDisconnect(mongo)

• developed on top of the MongoDB supported C driver

library(rmongodb)
mongo <- mongo.create(host="dbs001.mongosoup.de", db="cc_JwQcDLJSYQJb", username="JwQcDLJSYQJb", password="RSXPkUkXXXXX")

mongo

[1] 0
attr(,"mongo")
<pointer: 0x105a1de80>
attr(,"class")
[1] "mongo"
attr(,"host")
[1] "dbs001.mongosoup.de"
attr(,"name")
[1] ""
attr(,"username")
[1] "JwQcDLJSYQJb"
attr(,"password")
[1] "RSXPkUkxRdOX"
attr(,"db")
[1] "cc_JwQcDLJSYQJb"
attr(,"timeout")
[1] 0

buf <- mongo.bson.buffer.create()
query <- mongo.bson.from.buffer(buf)
buf <- mongo.bson.buffer.create()
err <- mongo.bson.buffer.append(buf, "user", 1)
err <- mongo.bson.buffer.append(buf, "type", 1)
field <- mongo.bson.from.buffer(buf)
out <- mongo.find(mongo, "cc_JwQcDLJSYQJb.ccp", query, fields=field, limit=1000)
res <- NULL
while (mongo.cursor.next(out)){
    value <- mongo.cursor.value(out)
    Rvalue <- mongo.bson.to.list(value)
    res <- rbind(res, Rvalue)
}

• R based MongoDB User Interface
• R packages shiny and rmongodb
• less than 200 lines of code
• DEMO: http://localhost:8100

https://github.com/comsysto/ShinyMongo

• R is a powerful statistical tool to analyse many different kind of data
• R can access databases
• MongoDB and rmongodb ready for Big Data
• start playing around with R, Big Data and MongoDB
  • http://www.r-project.org
  • http://www.mongodb.org
  • http://www.mongosoup.de 

• thanks a lot for your attention
• there are R trainings in December 2013 in Munich
  • http://comsysto.com/events.html#r
• we are hosting many events and meetups
• meet you at the MongoSoup booth

Email: markus@mongosoup.de
Twitter: @cloudHPC