CS 424 Big Data Analytics

Session 1: Orientation

Instructor: Dr. Bob Batzinger
Academic year: 2021/2022
Semester: 1

Begins June 2021

Let’s start with you…

Who are you?

Why are you taking this course?

What do you want to accomplish this term?

Common Reasons for studying Big Data

Data driven decisions provide a competitive advantage
Big Data provides a spring board for AI
Big Data skills that discover insights from data to create value are in high demand
Big Data combines data science, data architecture, statistics and machine learning principles to mine insights from data
Studying Big Data will broaden your horizon

Instructor for this course

Name: Robert Batzinger
Position: Praeceptor Emeritus
Email: robert_b@payap.ac.th
Office hours: by appointment
CV: https://rbatzing.github.io

Definition of Terms 大數據分析

Big: ใหญ่
1. large or great in dimensions, bulk, or extent
2. large or great in quantity, number, or amount
Data: ข้อมูล- factual units of information used as a basis for reasoning, discussion, or calculation
Analytics: การวิเคราะห - the scientific process of discovering and communicating the meaningful patterns which can be found in data; a process that turns raw data into insight for making better decisions

Course details

Big Data - Big Opportunity

Big Data

Netflix uses big data to saves $1 billion per year on customer retention.

In 2023 Big data analytics market set to reach $103 billion.
Poor data quality costs the US economy up to $3.1 trillion yearly.
Average person generate 1.7 megabytes per second.
Internet users generate about 2.5 Petrabytes of data daily.
95% of businesses have problems managing unstructured, mission-critical data
97.2% of organizations are currently investing in big data and AI.

The nature of Big Data

Velocity: ความเร็ว, Volume: ปริมาณ, Value: มูลค่, Variety: ความหลากหลาย, Veracity: ความจริง

Big Data Tools for Data Analysis

Statistical Computing
- Lumify
- Tableau
- R
Data trigger and stream processing
- Splunk
- Knime
- Storm

Database
- Cassandra
- MongoDB
Data pipeliner
- Xplenty
- Apache Hadoop
AI, ML, and Clustering
- HPCC
- Apache SAMOA
- Tensorflow
- Flink

Analytical Exercise

Understand the problem
Gather the information
Model the relationship between the independant and the dependant variables
Test your theory and verify the results
Communicate the results

Data Analytics

HOW?

Understand the problem
Gather the information
Model the relationship between the independant and the dependant variables
Test your theory and verify the results
Communicate the results

WHY?

Document a phenomena
Develop a theory or model that explains the results
Study the relative amount of impact of various parameters
Choose between conflicting theories/predictions
Test effectiveness of attempt at intervention

Framing an experimental question

\[\begin{matrix} \hbox{Cause} & \rightarrow & \hbox{Mediator} &\rightarrow & \hbox{Effect}\\ \hbox{(independant)} & &\hbox{(moderating)} & &\hbox{(dependant)}\\ \\ \hbox{Weather}\atop\hbox{conditions} &\rightarrow & \left\{\begin{matrix}\hbox{Holidays and weekends}\\\hbox{Political unrest}\\\hbox{Daylight savings time}\\\end{matrix}\right\} &\rightarrow & \hbox{Athletic event}\atop\hbox{participation}\\ \\ \hbox{Cost of}\atop\hbox{food delivery} &\rightarrow & \left\{\begin{matrix}\hbox{Ease of use}\\\hbox{Work from home order}\\ \hbox{Curfew and road blocks}\end{matrix}\right\} &\rightarrow &\hbox{Grab Revenue}\\ \end{matrix}\]

Experimentation

Control of factors that influence the result

Background variables
Sampling/allocation biases
Triangulaion between factors
Method of study
Datat analysis (opinion poll vs measurements)
Level of confidence

Experiment designs

Before and after
Two condition
Cross-tabulated study of multiple variables
Between individual vs within individual

Common experimental issues

Collective intelligence vs. peer pressure bias
Society based conformity vs. society based on diversity
Hawthorned effect vs. natural condition
Correlation vs. causation
Sampling biases vs relativity