• Warm-up: collaboration activity
• Intro to GitHub
• Three types of collaborative data
• 10 Features of Big Data
• Start with GitHub

• Managing Coding: RStudio, GitHub
• Gathering Data with R
• Creating and handling data
• Running, committing, pull requests and branches

• Office Hours: Fridays, 1:30pm-3:00pm (Tyler)
• Tuesdays, 10:30am-12:00pm (Yao)

• Absences
• R Markdown issues
• Readings

##          group 1           group 2         group 3           group 4
## 1 Devir, Lindsey   Batson, Anthony    Myoung, Sein       Smith, Reid
## 2  Leahy, Olivia     Pacheco, Alex     Qin, Celine   Kang, Christine
## 3   Barga, Jolie Wolfenstein, Luci Bell, Mary Rose Mahoney, Brigette
##          group 5      group 6
## 1    Ong, Alyssa Mendoza, Ava
## 2 Knowles, Genny             
## 3  Moore, Allana  Pham, Canon

• In groups: how would you create a system for collaboration in the issues named?
• See handout for details

##          group 1           group 2         group 3           group 4
## 1 Devir, Lindsey   Batson, Anthony    Myoung, Sein       Smith, Reid
## 2  Leahy, Olivia     Pacheco, Alex     Qin, Celine   Kang, Christine
## 3   Barga, Jolie Wolfenstein, Luci Bell, Mary Rose Mahoney, Brigette
##          group 5      group 6
## 1    Ong, Alyssa Mendoza, Ava
## 2 Knowles, Genny             
## 3  Moore, Allana  Pham, Canon

Git is a version control system.

GitHub is a way to share your version history online with collaborators.

• Some basic functions:
• Commit: save your code
• Push: send your code “up” to a higher level
• Pull: to gather code “down” from a higher level

• Other functions:
• Fork: Use existing work, and create your own version
• Branch: Create parallel streams of work (e.g. working version and editing version)

• In the activity, the central question is how to merge information up and down when it is changing on both ends
• Such scenarios are common in computational tasks and collaboration
• A tool like GitHub could help. For example
• A principal could create a repository with attendance information and notes
• Teachers could pull data on expected students and push information on actual attendance
• Teachers might use branches to create investigate ambiguous information (e.g. substitute recordings)
• A principal could merge branches when information is finalized
• There are other ways to do this! (no singular solution)

• Example: Galaxy Zoo

• Example: Netflix Prize
• 2006
• Data: user, movie, date of grade, grade
• Key element: training set and test set
• In 2009, a group beat Netflix’s algorithm by 10%, winning challenge

• Example: E-bird

• In groups:
• Why did the collaborative data collection work?
• What are potential risks?
• Could a similar model be applied to other social science tasks?
• Groups 1, 2: Galaxy Zoo. Groups 3, 4: Netflix. Groups 5, 6: E-bird.

##          group 1           group 2         group 3           group 4
## 1 Devir, Lindsey   Batson, Anthony    Myoung, Sein       Smith, Reid
## 2  Leahy, Olivia     Pacheco, Alex     Qin, Celine   Kang, Christine
## 3   Barga, Jolie Wolfenstein, Luci Bell, Mary Rose Mahoney, Brigette
##          group 5      group 6
## 1    Ong, Alyssa Mendoza, Ava
## 2 Knowles, Genny             
## 3  Moore, Allana  Pham, Canon

• Humanly possible … but takes a long time!
• Fun
• Relatively easy (for humans)
• Difficult (for machines)

• Fun
• Massive amount of data (>100,000,000 records)
• Money involved
• Very likely that someone out there had better algorithm than Netflix
• In late 2009, a group filed a privacy lawsuit against Netflix, later settled

• Birders are already birding!
• Risk for rare/poached birds

• On your own, write down one example of “big data”
• Maybe something you are interested in analyzing or using for class project

##          group 1           group 2         group 3           group 4
## 1 Devir, Lindsey   Batson, Anthony    Myoung, Sein       Smith, Reid
## 2  Leahy, Olivia     Pacheco, Alex     Qin, Celine   Kang, Christine
## 3   Barga, Jolie Wolfenstein, Luci Bell, Mary Rose Mahoney, Brigette
##          group 5      group 6
## 1    Ong, Alyssa Mendoza, Ava
## 2 Knowles, Genny             
## 3  Moore, Allana  Pham, Canon

• Too much to analyze individually (e.g. 100,000,000 observations in Netflix competition)

• Always tracking data (e.g. social media)

• People don’t know they are being observed (e.g., when they use Google Search)

• Often missing key components for social theory, such as information on users (e.g. mobile phone data)

• The best datasets are often not publicly available (e.g. tax records)

• Users are not taken from probabilistic random sample (e.g. Reddit forums)

• How people use platforms changes over time (e.g. Twitter)

• Site-specific rules may create social artifacts (e.g. Facebook friend numbers)

• Because data are collected for non-research purposes, they are often messy (e.g. housing records)

• Could contain non-obvious sensitive data (e.g. Netflix)

• In groups: pick a big data set
• How many of the 10 terms apply (big, always on, nonreactive, incomplete, inaccessible, nonrepresentative, drifting, algorithmically confounded, dirty, sensitive)?

##          group 1           group 2         group 3           group 4
## 1 Devir, Lindsey   Batson, Anthony    Myoung, Sein       Smith, Reid
## 2  Leahy, Olivia     Pacheco, Alex     Qin, Celine   Kang, Christine
## 3   Barga, Jolie Wolfenstein, Luci Bell, Mary Rose Mahoney, Brigette
##          group 5      group 6
## 1    Ong, Alyssa Mendoza, Ava
## 2 Knowles, Genny             
## 3  Moore, Allana  Pham, Canon

• github.com
• Before next class: add your github username

• Warm-up
• Back to coding! Gathering Data with gtrendsR
• GitHub: Creating repositories, cloning, and committing
• Starting PSet 2

• Tyler: 1:30pm-3:00pm on Zoom (this week)

• In groups: pick a big data set
• How many of the 10 terms apply (big, always on, nonreactive, incomplete, inaccessible, nonrepresentative, drifting, algorithmically confounded, dirty, sensitive)?

##          group 1           group 2         group 3           group 4
## 1 Devir, Lindsey   Batson, Anthony    Myoung, Sein       Smith, Reid
## 2  Leahy, Olivia     Pacheco, Alex     Qin, Celine   Kang, Christine
## 3   Barga, Jolie Wolfenstein, Luci Bell, Mary Rose Mahoney, Brigette
##          group 5      group 6
## 1    Ong, Alyssa Mendoza, Ava
## 2 Knowles, Genny             
## 3  Moore, Allana  Pham, Canon

• Google Trends search data has many of the characteristics of “big data”

#install.packages("gtrendsR")

library(gtrendsR)

data_social <- gtrends(c("data science", "social science"), 
                  geo = c("US"))

# for help doc
?gtrends

plot(data_social)

data_social_all <- gtrends(c("data science", "social science"), 
                  geo = c("US"), time = "all")

plot(data_social_all)

data_geo <- gtrends(c("data science"), 
                  geo = c("US-CA", "US-NY", "US-FL"), time = "all")

plot(data_geo)

• Try gathering some data on Google Trends by modifying the code below:

library(gtrendsR)

data_social_all <- gtrends(c("data science", "social science"), 
                  geo = c("US-CA"), time = "all")

• Brookings Article on AI Coding Agents
• In what follows: if you finish early, explore gtrendsR
• Code for the plots we created is available on the course website

• Go to www.github.com !
• Sign in
• Create a repository

• Go to www.github.com !
• Sign in
• Create a repository

• Detailed Instructions Here (linked on course site)
• Let’s Try it!
• Create a repo called “soc10problemsets” or something similar

• Go to your Repositories
• Find soc10problemsets
• Upload a file
• Optional: try a branch!

• Can be done entirely using GitHub Online
• Download .Rmd file, make changes, copy/paste online, commit them
• Preferred: GitHub Desktop

• Download GitHub Desktop
• Open GitHub Desktop
• Sign in!

• Let’s download and sign into GitHub Desktop
• Pause
• If success: great! Experiment with google trends in RStudio

• (Live demo)

• Cloned Repo should be in your folders
• Ideal: create a GitHub folder in your Documents

• PSet2: Turn in on Canvas/Gradescope
• Markdown file, pdf (or html), link to github
• Next week: Data Viz!!