Introduction to the assignment
This assignment consists of three main sections.
In the first section, you need to select one Census Tract that you think is the most walkable and another one that you think is least walkable within Fulton and DeKalb Counties, GA. As long as they are within the two counties, you can pick any two Census Tracts you want. If the area you want to use as walkable/unwalkable area is not well-covered by a single Census Tract, you can select multiple tracts (e.g., selecting three adjacent tracts as one walkable area). The definition of ‘walkable’ can be your own - you can choose solely based on your experience (e.g., had best/worst walking experience because …), refer to Walk Score, or any other mix of criteria you want. After you make the selection, provide a short write-up of why you chose those Census Tracts.
The second section is the main part of this assignment in which you prepare OSM data, download GSV images, apply computer vision technique we learned in the class (i.e., semantic segmentation).
In the third section, you will summarise and analyze the output and provide your findings. After you apply computer vision to the images, you will have the number of pixels in each image that represent 150 categories in your data. You will focus on the following categories in your analysis: building, sky, tree, road, and sidewalk. Specifically, you will (1) create maps to visualize the spatial distribution of different objects, (2) compare the mean of each category between the two Census Tract and (3) draw boxplots to compare the distributions.
Section 1. Choose your Census Tracts.
Provide a brief description of your census tracts. Why do you think the Census Tracts are walkable and unwalkable? What were the contributing factors?
Section 2. OSM, GSV, and computer vision.
Fill out the template to complete the script.
library(tidyverse)
library(tidycensus)
library(osmdata)
library(sfnetworks)
library(units)
library(sf)
library(tidygraph)
library(tmap)
library(here)Step 1. Get OSM data and clean it.
The getbb() function, which we used in the class to
download OSM data, isn’t suitable for downloading just two Census
Tracts. We will instead use an alternative method.
- Using tidycensus package, download the Census Tract polygon for Fulton and DeKalb counties.
- Extract two Census Tracts, each of which will be your most walkable and least walkable Census Tracts.
- Using their bounding boxes, get OSM data.
- Convert them into sfnetwork object and clean it.
# TASK ////////////////////////////////////////////////////////////////////////
# 1. Set up your api key here
census_api_key(
Sys.getenv('census_api'), overwrite = TRUE, install = TRUE
)## Your original .Renviron will be backed up and stored in your R HOME directory if needed.## Your API key has been stored in your .Renviron and can be accessed by Sys.getenv("CENSUS_API_KEY").
## To use now, restart R or run `readRenviron("~/.Renviron")`## [1] "073d0b7093bd1e1d1a70e8eba5259274deb6d28d"# //TASK //////////////////////////////////////////////////////////////////////
# =========== NO MODIFICATION ZONE STARTS HERE ===============================
# Download Census Tract polygon for Fulton and DeKalb
tract <- get_acs("tract",
variables = c('tot_pop' = 'B01001_001'),
year = 2022,
state = "GA",
county = c("Fulton", "DeKalb"),
geometry = TRUE)## Getting data from the 2018-2022 5-year ACS## Downloading feature geometry from the Census website. To cache shapefiles for use in future sessions, set `options(tigris_use_cache = TRUE)`.## | | | 0% | |= | 1% | |== | 2% | |== | 3% | |=== | 4% | |==== | 5% | |==== | 6% | |===== | 7% | |====== | 8% | |====== | 9% | |======= | 10% | |======== | 11% | |======== | 12% | |========= | 12% | |========= | 13% | |========== | 14% | |========== | 15% | |=========== | 15% | |=========== | 16% | |============ | 17% | |============ | 18% | |============= | 18% | |============= | 19% | |============== | 20% | |============== | 21% | |=============== | 21% | |=============== | 22% | |================ | 23% | |================= | 24% | |================= | 25% | |================== | 26% | |=================== | 26% | |=================== | 27% | |==================== | 28% | |==================== | 29% | |===================== | 29% | |===================== | 30% | |====================== | 31% | |====================== | 32% | |======================= | 32% | |======================= | 33% | |======================== | 34% | |======================== | 35% | |========================= | 35% | |========================= | 36% | |========================== | 37% | |========================== | 38% | |=========================== | 38% | |============================ | 40% | |============================= | 41% | |============================== | 43% | |=============================== | 44% | |=============================== | 45% | |================================ | 46% | |================================= | 48% | |================================== | 49% | |=================================== | 49% | |=================================== | 50% | |==================================== | 51% | |==================================== | 52% | |===================================== | 52% | |===================================== | 53% | |====================================== | 54% | |======================================= | 55% | |======================================= | 56% | |======================================== | 57% | |========================================= | 58% | |========================================= | 59% | |========================================== | 60% | |=========================================== | 61% | |=========================================== | 62% | |============================================ | 63% | |============================================= | 64% | |============================================== | 66% | |=============================================== | 67% | |=============================================== | 68% | |================================================ | 69% | |================================================= | 70% | |================================================= | 71% | |================================================== | 71% | |=================================================== | 73% | |==================================================== | 74% | |===================================================== | 75% | |===================================================== | 76% | |====================================================== | 77% | |======================================================= | 78% | |======================================================= | 79% | |======================================================== | 80% | |========================================================= | 81% | |========================================================= | 82% | |========================================================== | 82% | |========================================================== | 83% | |=========================================================== | 84% | |============================================================ | 85% | |============================================================ | 86% | |============================================================= | 88% | |============================================================== | 88% | |============================================================== | 89% | |=============================================================== | 90% | |=============================================================== | 91% | |================================================================ | 91% | |================================================================ | 92% | |================================================================= | 93% | |================================================================== | 94% | |=================================================================== | 95% | |=================================================================== | 96% | |==================================================================== | 97% | |===================================================================== | 98% | |===================================================================== | 99% | |======================================================================| 99% | |======================================================================| 100%# =========== NO MODIFY ZONE ENDS HERE ========================================
# TASK ////////////////////////////////////////////////////////////////////////
# The purpose of this TASK is to create one bounding box for walkable Census Tract and another bounding box for unwalkable Census Tract.
# As long as you generate what's needed for the subsequent codes, you are good. The numbered list of tasks below is to provide some hints.
# 1. Write the GEOID of walkable & unwalkable Census Tracts. e.g., tr1_ID <- c("13121001205", "13121001206")
# 2. Extract the selected Census Tracts using tr1_ID & tr2_ID
# 3. Create their bounding boxes using st_bbox(), and
# 4. assign them to tract_1_bb and tract_1_bb, respectively.
# For the walkable Census Tract(s)
# 1.
tr1_ID <- "13121001001" # **YOUR CODE HERE..** --> For example, tr1_ID <- c("13121001205", "13121001206").
# 2~4
tract_1_bb <- tract %>%
filter(GEOID == tr1_ID) %>%
st_bbox()
# For the unwalkable Census Tract(s)
# 1.
tr2_ID <- "13121001902" # **YOUR CODE HERE..**
# 2~4
tract_2_bb <- tract %>%
filter(GEOID == tr2_ID) %>%
st_bbox()
# //TASK //////////////////////////////////////////////////////////////////////
# =========== NO MODIFICATION ZONE STARTS HERE ===============================
# Get OSM data for the two bounding box
osm_1 <- opq(bbox = tract_1_bb) %>%
add_osm_feature(key = 'highway',
value = c("motorway", "trunk", "primary",
"secondary", "tertiary", "unclassified",
"residential")) %>%
osmdata_sf() %>%
osm_poly2line()
osm_2 <- opq(bbox = tract_2_bb) %>%
add_osm_feature(key = 'highway',
value = c("motorway", "trunk", "primary",
"secondary", "tertiary", "unclassified",
"residential")) %>%
osmdata_sf() %>%
osm_poly2line()
# =========== NO MODIFY ZONE ENDS HERE ========================================
# TASK ////////////////////////////////////////////////////////////////////////
# 1. Convert osm_1 and osm_2 to sfnetworks objects (set directed = FALSE)
# 2. Clean the network by (1) deleting parallel lines and loops, (2) create missing nodes, and (3) remove pseudo nodes,
# 3. Add a new column named length using edge_length() function.
net1 <- osm_1$osm_lines %>%
# Drop redundant columns
select(osm_id, highway) %>%
sfnetworks::as_sfnetwork(directed = FALSE) %>%
activate("edges") %>%
convert(to_spatial_subdivision) %>%
convert(to_undirected) %>%
mutate(length = edge_length())## Warning: to_spatial_subdivision assumes attributes are constant over geometriesnet2 <- osm_2$osm_lines %>%
# Drop redundant columns
select(osm_id, highway) %>%
sfnetworks::as_sfnetwork(directed = FALSE) %>%
activate("edges") %>%
convert(to_spatial_subdivision) %>%
convert(to_undirected) %>%
mutate(length = edge_length())## Warning: to_spatial_subdivision assumes attributes are constant over geometries# //TASK //////////////////////////////////////////////////////////////////////
# =========== NO MODIFICATION ZONE STARTS HERE ===============================
# OSM for the walkable part
edges_1 <- net1 %>%
# Extract 'edges'
st_as_sf("edges") %>%
# Drop segments that are too short (100m)
mutate(length = as.vector(length)) %>%
filter(length > 100) %>%
# Add a unique ID for each edge
mutate(edge_id = seq(1,nrow(.)),
is_walkable = "walkable")
# OSM for the unwalkable part
edges_2 <- net2 %>%
# Extract 'edges'
st_as_sf("edges") %>%
# Drop segments that are too short (100m)
mutate(length = as.vector(length)) %>%
filter(length > 100) %>%
# Add a unique ID for each edge
mutate(edge_id = seq(1,nrow(.)),
is_walkable = "unwalkable")
# Merge the two
edges <- bind_rows(edges_1, edges_2)
# =========== NO MODIFY ZONE ENDS HERE ========================================I chose Census Tract 10.01 as the walkable tract because it is a route I frequently take to and from school, and it perfectly exemplifies a pedestrian-friendly area. The streets are bustling with pedestrians, lined with trees and greenery, and feature wide, well-maintained sidewalks that make walking both safe and pleasant.
On the other hand, I selected Census Tract 19.02 as the unwalkable tract due to its contrasting conditions. Located in the downtown area, this tract has a reputation for poorer safety. The streets are often littered, and the area feels crowded with tall buildings that block natural light and create a sense of confinement.
Step 2. Define getAzimuth() function.
getAzimuth <- function(line){
# This function takes one edge (i.e., a street segment) as an input and
# outputs a data frame with four points (start, mid1, mid2, and end) and their azimuth.
# TASK ////////////////////////////////////////////////////////////////////////
# 1. From `line` object, extract the coordinates using st_coordinates() and extract the first two rows.
# 2. Use atan2() function to calculate the azimuth in degree.
# Make sure to adjust the value such that 0 is north, 90 is east, 180 is south, and 270 is west.
# 1
start_p <- line %>%
st_coordinates() %>%
.[1:2,1:2]
# **YOUR CODE HERE..**
# 2
start_azi <- atan2(start_p[2,"X"] - start_p[1, "X"],
start_p[2,"Y"] - start_p[1, "Y"])*180/pi # **YOUR CODE HERE..** --> For example, atan2()..
# //TASK //////////////////////////////////////////////////////////////////////
# TASK ////////////////////////////////////////////////////////////////////////
# Repeat what you did above, but for last two rows (instead of the first two rows).
# Remember to flip the azimuth so that the camera would be looking at the street that's being measured
end_p <- line %>%
st_coordinates() %>%
.[(nrow(.)-1):nrow(.),1:2]
# **YOUR CODE HERE..**
end_azi <- atan2(end_p[2,"X"] - end_p[1, "X"],
end_p[2,"Y"] - end_p[1, "Y"])*180/pi # **YOUR CODE HERE..** --> For example, atan2()..
end_azi <- if (end_azi < 180) {end_azi + 180} else {end_azi - 180}
# //TASK //////////////////////////////////////////////////////////////////////
# TASK ////////////////////////////////////////////////////////////////////////
# 1. From `line` object, use st_line_sample() function to generate points at 0.45 and 0.55 locations. These two points will be used to calculate the azimuth.
# 2. Use st_case() function to convert 'MULTIPOINT' object to 'POINT' object.
# 3. Extract coordinates using st_coordinates().
# 4. Use atan2() functino to Calculate azimuth.
# 5. Use st_line_sample() again to generate a point at 0.5 location and get its coordinates. This point will be the location at which GSV image will be downloaded.
mid_p <- line %>%
st_geometry() %>%
.[[1]] %>%
st_line_sample(sample = c(0.45, 0.55)) %>%
st_cast("POINT") %>%
st_coordinates()
# **YOUR CODE HERE..** --> For 0.45 & 0.55 points
mid_azi <- atan2(mid_p[2,"X"] - mid_p[1, "X"],
mid_p[2,"Y"] - mid_p[1, "Y"])*180/pi # **YOUR CODE HERE..** For example, atan2()..
mid_p <- line %>%
st_geometry() %>%
.[[1]] %>%
st_line_sample(sample = 0.5) %>%
st_coordinates() %>%
.[1,1:2]
# **YOUR CODE HERE..** --> For 0.5 point
# //TASK //////////////////////////////////////////////////////////////////////
# =========== NO MODIFICATION ZONE STARTS HERE ===============================
return(tribble(
~type, ~X, ~Y, ~azi,
"start", start_p[1,"X"], start_p[1,"Y"], start_azi,
"mid1", mid_p["X"], mid_p["Y"], mid_azi,
"mid2", mid_p["X"], mid_p["Y"], ifelse(mid_azi < 180, mid_azi + 180, mid_azi - 180),
"end", end_p[2,"X"], end_p[2,"Y"], end_azi))
# =========== NO MODIFY ZONE ENDS HERE ========================================
}Step 3. Apply the function to all street segments
We can apply getAzimuth() function to the edges object.
We finally append edges object to make use of the columns
in edges object (e.g., is_walkable column).
When you are finished with this code chunk, you will be ready to
download GSV images.
# TASK ////////////////////////////////////////////////////////////////////////
# Apply getAzimuth() function to all edges.
# Remember that you need to pass edges object to st_geometry() before you apply getAzimuth()
edges_azi <- edges %>%
st_geometry() %>%
map_df(getAzimuth)
# **YOUR CODE HERE..**
# //TASK //////////////////////////////////////////////////////////////////////
# =========== NO MODIFICATION ZONE STARTS HERE ===============================
edges_azi <- edges_azi %>%
bind_cols(edges %>%
st_drop_geometry() %>%
slice(rep(1:nrow(edges),each=4))) %>%
st_as_sf(coords = c("X", "Y"), crs = 4326, remove=FALSE) %>%
mutate(node_id = seq(1, nrow(.)))
# =========== NO MODIFY ZONE ENDS HERE ========================================Step 4. Define a function that formats request URL and download images.
getImage <- function(iterrow){
# This function takes one row of edges_azi and downloads GSV image using the information from edges_azi.
# TASK ////////////////////////////////////////////////////////////////////////
# Finish this function definition.
# 1. Extract required information from the row of edges_azi, including
# type (i.e., start, mid1, mid2, end), location, heading, edge_id, node_id, and key.
# 2. Format the full URL and store it in `request`. Refer to this page: https://developers.google.com/maps/documentation/streetview/request-streetview
# 3. Format the full path (including the file name) of the image being downloaded and store it in `fpath`
type <- iterrow$type # **YOUR CODE HERE..**
location <- paste0(iterrow$Y %>% round(5), ",", iterrow$X %>% round(5)) # **YOUR CODE HERE..**
heading <- iterrow$azi # **YOUR CODE HERE..**
edge_id <- iterrow$edge_id # **YOUR CODE HERE..**
node_id <- iterrow$node_id # **YOUR CODE HERE..**
key <- Sys.getenv("google_api") # **YOUR CODE HERE..**
endpoint <- "https://maps.googleapis.com/maps/api/streetview" # **YOUR CODE HERE..**
request <- glue::glue("{endpoint}?size=640x640&location={location}&heading={heading}&fov=90&pitch=0&key={key}") # **YOUR CODE HERE..**
fname <- glue::glue("GSV-nid_{node_id}-eid_{edge_id}-type_{type}-Location_{location}-heading_{heading}.jpg") # Don't change this code for fname
fpath <- here("major2","images",fname) # **YOUR CODE HERE..**
# //TASK //////////////////////////////////////////////////////////////////////
# =========== NO MODIFICATION ZONE STARTS HERE ===============================
# Download images
if (!file.exists(fpath)){
download.file(request, fpath, mode = 'wb')
}
# =========== NO MODIFY ZONE ENDS HERE ========================================
}Step 5. Download GSV images
Before you download GSV images, make sure
the row number of edges_azi is not too large! The row
number of edges_azi will be the number of GSV images you
will be downloading. Before you download images, always double-check
your Google Cloud Console’s Billing tab to make sure that you will not
go above the free credit of $200 each month. The price is $7 per 1000
images.
# =========== NO MODIFICATION ZONE STARTS HERE ===============================
# Loop!
for (i in seq(1,nrow(edges_azi))){
getImage(edges_azi[i,])
}
# =========== NO MODIFY ZONE ENDS HERE ========================================ZIP THE DOWNLOADED IMAGES AND NAME IT ‘gsv_images.zip’ FOR STEP 6.
Step 6. Apply computer vision
Now, use Google Colab to apply the semantic segmentation model. Zip your images and upload the images to your Colab session.
Step 7. Merging the processed data back to R
Once all of the images are processed and saved in your Colab session as a CSV file, download the CSV file and merge it back to edges.
# TASK ////////////////////////////////////////////////////////////////////////
# Read the downloaded CSV file from Google Colab
seg_output <- read.csv("/home/rstudio/major2/seg_output.csv"
# **YOUR CODE HERE..**
)
# //TASK ////////////////////////////////////////////////////////////////////////
# =========== NO MODIFICATION ZONE STARTS HERE ===============================
# Join the seg_output object back to edges_azi object using node_id as the join key.
edges_seg_output <- edges_azi %>%
inner_join(seg_output, by=c("node_id" = "img_id")) %>%
select(type, X, Y, is_walkable, node_id, building, sky, tree, road, sidewalk) %>%
mutate(across(c(building, sky, tree, road, sidewalk), function(x) x/(640*640)))
# =========== NO MODIFY ZONE ENDS HERE ========================================Section 3. Summarise and analyze the results.
At the beginning of this assignment, you defined one Census Tract as walkable and the other as unwalkable. The key to the following analysis is the comparison between walkable and unwalkable Census Tracts.
Analysis 1 - Create interactive map(s) to visualize the spatial distribution of the streetscape.
You need to create maps of the proportion of building, sky, tree, road, and sidewalk for walkable and unwalkable areas. In total, you will have 10 maps.
Provide a brief description of your findings from the maps.
# TASK ////////////////////////////////////////////////////////////////////////
# Create interactive map(s) to visualize the `edges_seg_output` objects.
# As long as you can deliver the message clearly, you can use any format/package you want.
library(leaflet)
library(dplyr)
walkable_data <- edges_seg_output %>% filter(is_walkable == "walkable")
unwalkable_data <- edges_seg_output %>% filter(is_walkable == "unwalkable")
features <- c("building", "sky", "tree", "road", "sidewalk")
for (feature in features) {
palette <- colorBin(palette = "YlOrRd", domain = edges_seg_output[[feature]], bins = 5)
# Map for walkable areas
leaflet(walkable_data) %>%
addTiles() %>%
addCircleMarkers(
lng = ~st_coordinates(geometry)[,1],
lat = ~st_coordinates(geometry)[,2],
radius = 5,
color = ~palette(get(feature)),
label = ~paste0(feature, ": ", round(get(feature), 2)),
popup = ~paste("Walkable Area<br>", feature, ": ", round(get(feature), 2))
) %>%
addLegend(
position = "bottomright",
pal = palette,
values = walkable_data[[feature]],
title = paste(feature, "Percentage of Walkable Tract")
) %>%
print()
# Map for unwalkable areas
leaflet(unwalkable_data) %>%
addTiles() %>%
addCircleMarkers(
lng = ~st_coordinates(geometry)[,1],
lat = ~st_coordinates(geometry)[,2],
radius = 5,
color = ~palette(get(feature)),
label = ~paste0(feature, ": ", round(get(feature), 2)),
popup = ~paste("Unwalkable Area<br>", feature, ": ", round(get(feature), 2))
) %>%
addLegend(
position = "bottomright",
pal = palette,
values = unwalkable_data[[feature]],
title = paste(feature, "Percentage of Unwalkable Tract")
) %>%
print()
}
# //TASK //////////////////////////////////////////////////////////////////////Regarding the comparison of the two areas, for most locations, the percentage of buildings in the non-walkable areas is higher than that in the walkable areas; the percentage of sky and trees in the walkable areas are both higher than those in the non-walkable census areas. This may indicate that with the increase in buildings, the population tends to choose other modes of transportation in these areas rather than walking. However, the percentage of roads and sidewalks is not well explained.
Analysis 2 - Compare the means.
You need to calculate the mean of the proportion of building, sky, tree, road, and sidewalk for walkable and unwalkable areas. For example, you need to calculate the mean of building category for each of walkable and unwalkable Census Tracts. Then, you need to calculate the mean of sky category for each of walkable and unwalkable Census Tracts. In total, you will have 10 mean values. Provide a brief description of your findings.
# TASK ////////////////////////////////////////////////////////////////////////
# Perform the calculation as described above.
# As long as you can deliver the message clearly, you can use any format/package you want.
mean_walkable <- walkable_data %>%
summarise(across(c(building, sky, tree, road, sidewalk), mean))
mean_unwalkable <- unwalkable_data %>%
summarise(across(c(building, sky, tree, road, sidewalk), mean))
mean_walkable## Simple feature collection with 1 feature and 5 fields
## Geometry type: MULTIPOINT
## Dimension: XY
## Bounding box: xmin: -84.40606 ymin: 33.76735 xmax: -84.38006 ymax: 33.79001
## Geodetic CRS: WGS 84
## # A tibble: 1 × 6
## building sky tree road sidewalk geometry
## <dbl> <dbl> <dbl> <dbl> <dbl> <MULTIPOINT [°]>
## 1 0.153 0.214 0.148 0.361 0.0413 ((-84.38385 33.78321), (-84.39453 33.7899…mean_unwalkable## Simple feature collection with 1 feature and 5 fields
## Geometry type: MULTIPOINT
## Dimension: XY
## Bounding box: xmin: -84.39205 ymin: 33.75479 xmax: -84.37776 ymax: 33.77556
## Geodetic CRS: WGS 84
## # A tibble: 1 × 6
## building sky tree road sidewalk geometry
## <dbl> <dbl> <dbl> <dbl> <dbl> <MULTIPOINT [°]>
## 1 0.205 0.171 0.126 0.368 0.0449 ((-84.39069 33.75839), (-84.38763 33.7584…# //TASK //////////////////////////////////////////////////////////////////////This shows that the average value of buildings in walkable and unwalkable areas is quite different, about 5%. The sky is also quite different, 4% larger in walkable areas than in unwalkable areas, and the average value of trees is also about 2% larger. There is not much difference between road and sidewalk, and the absolute difference is quite small, only 0.5%.
Analysis 3 - Draw boxplot
You need to calculate the mean of the proportion of building, sky, tree, road, and sidewalk for walkable and unwalkable areas. For example, you need to calculate the mean of building category for each of walkable and unwalkable Census Tracts. Then, you need to calculate the mean of sky category for each of walkable and unwalkable Census Tracts. In total, you will have 10 mean values. Provide a brief description of your findings.
# TASK ////////////////////////////////////////////////////////////////////////
# Create boxplot(s) using ggplot2 package.
library(ggplot2)
library(reshape2)##
## Attaching package: 'reshape2'## The following object is masked from 'package:tidyr':
##
## smithsmelted_data <- melt(edges_seg_output, id.vars = "is_walkable", measure.vars = c("building", "sky", "tree", "road", "sidewalk"))
# Create boxplots
ggplot(melted_data, aes(x = is_walkable, y = value, fill = is_walkable)) +
geom_boxplot() +
facet_wrap(~ variable, scales = "free") +
labs(
title = "Comparison of Streetscape Features by Walkability",
x = "Walkability",
y = "Proportion",
fill = "Walkability"
) +
theme_minimal()
# //TASK //////////////////////////////////////////////////////////////////////Walkable areas generally have a slightly higher proportion of sky and trees compared to unwalkable areas. This may indicate that walkable areas have better urban layout and denser greenery. Walkable tract tends to exhibit a lower proportion of buildings, indicating a more open and pedestrian-friendly environment. Conversely, unwalkable tract with higher density may create barriers to pedestrian movement. The proportion of roads and sidewalks is consistent between walkable and unwalkable areas. However, the proportion of sidewalks in walkable areas could continue to increase in the future, which could help improve pedestrian accessibility.