Bike Network & Public Sentiment

Bike lanes are controversial! Complaints about bike lanes are common in local media, but how much do New York residents actually object?

Using publicly available data on the location and timing of installation of new bike lanes in NYC and messages from the public sent to the NYC Department of Transportation, we will explore whether there is any correlation between public dissatisfaction with bike lanes and the amount of new bike lanes installed in a given calendar year.

This project pulls from three types of data:

1. Bike lane data drawn from published records of bike lane installation available through NYC Open Data. This data is stored as a shapefile (.SHP) and will be scraped from the web.

2. Proxy for public sentiment: Metadata from the DOT's Commissioner's Correspondence Unit (CCU), which consists of tags and metadata from messages many types (including emails, letters, phone calls, social media, and webforms) sent to the NYC Department of Transportation. This data is stored as JSON and will be retrieved from an API, querying for messages tagged as “Bicycle Lanes and Programs.”

3. Maps pulled from the Google API using ggmap.

library(httr)
library(jsonlite)
library(dplyr)

baseurl <- "https://data.cityofnewyork.us/resource/paw9-9kar.json?casetopic=Bicycle%20Lanes%20%26%20Programs" # includes the query casetopic = Bicycle Lanes, encoded as URL

allSearch <- fromJSON(baseurl) #creates a data frame from JSON retrieved through API

for(n in 1:6){
  i <- n*1000
  searchCCU <- fromJSON(paste0(baseurl,"&$offset=",i)) #searches for pages in chunks of 1000
  allSearch <- full_join(allSearch, searchCCU) #combines data into original data frame
}

glimpse(allSearch)

Observations: 6,820
Variables: 42
$ borough                <chr> "Brooklyn", "Queens", "Brooklyn", NA, "...
$ caseaddressedto        <chr> "Customer Service", "BC-Queens", "311",...
$ casechannel            <chr> "Web Form", "Web Form", "Customer Comme...
$ casenumber             <chr> "DOT-391803-G9M7", "DOT-323519-H9K3", "...
$ casetopic              <chr> "Bicycle Lanes & Programs", "Bicycle La...
$ clienttype             <chr> "Citizen", "Citizen", "Citizen", "Citiz...
$ createddate            <chr> "2018-10-02T13:46:20.000", "2017-01-13T...
$ crossstreet            <chr> "Prospect Park", "Yellowstone", NA, NA,...
$ fromstreet             <chr> "8 Avenue", "Eliot", "7th Street", NA, ...
$ latitude               <chr> "40.665749", "40.7325068549", "40.67038...
$ locationdetail         <chr> "Multiple Block - Stretch", "Multiple B...
$ locationtype           <chr> "Street or Sidewalk", "Street or Sidewa...
$ longitude              <chr> "-73.979224", "-73.8682252485", "-73.98...
$ maplocation            <chr> "40.6657490,-73.9792240", "40.732506854...
$ pressname              <chr> "No", "No", "No", "No", "No", "No", "No...
$ resolutionreason       <chr> "Open - New", "Information Provided", "...
$ status                 <chr> "Active", "Resolved", "Resolved", "Reso...
$ streetname             <chr> "9 Street", "Queens Blvd", "5th Avenue"...
$ translationneeded      <chr> "No", "No", "No", "No", "No", "No", "No...
$ acknowledgesent        <chr> NA, "2017-01-23T12:34:35.000", "2016-08...
$ caseissue              <chr> NA, "Comment", "Remove or Relocate", NA...
$ closeddate             <chr> NA, "2018-07-08T11:48:32.000", "2017-11...
$ communitydistrict      <chr> NA, "06", "06", NA, "01", NA, "11", NA,...
$ policeprecinct         <chr> NA, "112", "078", NA, "114", NA, "111",...
$ xcoordinate            <chr> NA, "1020819", "988329", NA, "1003102",...
$ ycoordinate            <chr> NA, "206204", "183516", NA, "213248", N...
$ externaltrackingnumber <chr> NA, NA, "3914.02", NA, NA, NA, NA, NA, ...
$ seibelnumber           <chr> NA, NA, "1-1-1296826294", "1-1-12725743...
$ mayorsnumber           <chr> NA, NA, NA, "854615", NA, NA, NA, NA, N...
$ buildingnumber         <chr> NA, NA, NA, NA, NA, NA, NA, NA, "2233",...
$ category               <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, "Ex...
$ inspector              <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, "Ke...
$ clientcompanyname      <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ bridgename             <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ zipcode                <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ otherbridgename        <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ highwayname            <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ ferrydatetime          <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ ferrydirection         <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ directionname          <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ segmentorexit          <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...
$ bridgedirection        <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,...