##           group 1           group 2           group 3         group 4
## 1 Kang, Christine     Moore, Allana   Randall, Javion                
## 2     Ong, Alyssa Mahoney, Brigette       Qin, Celine Batson, Anthony
## 3   Leahy, Olivia      Myoung, Sein       Smith, Reid    Mendoza, Ava
## 4  Devir, Lindsey       Pham, Canon Wolfenstein, Luci   Pacheco, Alex
##           group 5
## 1    Barga, Jolie
## 2 Bell, Mary Rose
## 3  Knowles, Genny
## 4 Kang, Christine

• How would you gather data on neighborhood social change?
• You don’t need to use all the information or produce a finished product

##           group 1           group 2           group 3         group 4
## 1 Kang, Christine     Moore, Allana   Randall, Javion                
## 2     Ong, Alyssa Mahoney, Brigette       Qin, Celine Batson, Anthony
## 3   Leahy, Olivia      Myoung, Sein       Smith, Reid    Mendoza, Ava
## 4  Devir, Lindsey       Pham, Canon Wolfenstein, Luci   Pacheco, Alex
##           group 5
## 1    Barga, Jolie
## 2 Bell, Mary Rose
## 3  Knowles, Genny
## 4 Kang, Christine

• Warm-up: neighborhood change
• Gathering data in the 21st century
• Activity: data modelling
• Data models

• Using APIs
• Web Scraping
• Data Models
• Prediction

• Office Hours: Fridays, 1:30pm-3:00pm (Tyler)
• Tuesdays, 10:30am-12:00pm (Yao)

• Problem set and file naming
• GitHub/PSet Submissions
• PSet 3 and week 3 course site materials released shortly

• Probabilistic surveys!
• Census
• Qualitative interviews/ethnographic observations
• Does computation change the way we do these methods?

• Probabilistic surveys! (recruit respondents on social media)
• Census (use API to pull data)
• Qualitative interviews/ethnographic observations (observe online behavior)
• Does computation change the way we do these methods?

• Use Application Program Interfaces (APIs) to gather census data tidycensus
• Open-access measures Urban Displacement Project

• Use Application Program Interfaces (APIs) to gather census data tidycensus
• Open-access measures Urban Displacement Project
• Wiki Surveys (flexible surveys with user input) allourideas.org

• Use Application Program Interfaces (APIs) to gather census data tidycensus
• Open-access measures Urban Displacement Project
• Wiki Surveys (flexible surveys with user input) allourideas.org
• Ecological Momentary Assessments (survey people in real time)
• Gamification (fun surveys)
• Gathering text from online sources (scraping)
• Gathering images or data from online sources
• Link surveys to gathered data
• Let’s look at some examples!

• Much debate about occurrence and extent of gentrification
• However: empirical evidence of neighborhood change is limited (surveys, census data)

• Hwang’s solution: use Google Street View to look at neighborhood change
• Combined these data with Census data
• Compared these estimates to earlier Chicago gentrification estimates

• In pairs: discuss whether Hwang’s approach would be effective for studying gentrification in the Bay Area

• The US incarceration system is large (more on this soon)
• Little is known about how people re-enter society after incarceration

• Sugie’s solution: administer cell phones to study participants leaving prison
• Conduct “Ecological Momentary Assessments” (daily surveys) to assess well-being, job search, and more

• In pairs: discuss how Sugie’s approach helps us learn more about human behaviors, and any potential challenges to gathering data this way

• Plot some (or all) of the incarceration data
• How would you predict what rates will be in 2030?

##           group 1           group 2           group 3         group 4
## 1 Kang, Christine     Moore, Allana   Randall, Javion                
## 2     Ong, Alyssa Mahoney, Brigette       Qin, Celine Batson, Anthony
## 3   Leahy, Olivia      Myoung, Sein       Smith, Reid    Mendoza, Ava
## 4  Devir, Lindsey       Pham, Canon Wolfenstein, Luci   Pacheco, Alex
##           group 5
## 1    Barga, Jolie
## 2 Bell, Mary Rose
## 3  Knowles, Genny
## 4 Kang, Christine

• We may want to make generalizations or predictions
• For example: Will incarceration trends continue through 2030? What leads to rises in incarceration?

• How would we calculate line of best fit from our incarceration data?

• Polynomials: quadratic, cubic, etc.
• Tree-based models! Random forests
• Neural networks
• And more

• Data scientists often split their data into training and test sets
• Goal: choose model that is likely to predict well out of sample

• Share one idea for potential final project