Predictive Modeling of the Perception of Liveliness in Atlanta Using Place Pulse and Google Street View Data

Introduction

Background

Although a crucial quality to a successful urban environment, “Liveliness” is a metric that is difficult to measure. For this project, we found a dataset, ‘Place Pulse’, which compiles user input on such subjective metrics including ‘liveliness’, ‘safety’, etc, along with the accompanied images of the surveyed locations. The most recent available Place Pulse inputs were from 2016. Our project objective was to use this dataset as model input to then predict the City of Atlanta’s liveliness scores in more recent years, specifically 2021. We chose 2021 as our study year based on our decision to use Google Street View data as the predictor dataset, for which the most recently available images were from 2021.

One of the goals of the project was to see if it was possible to use regression to identify which urban factors had the most impact on perceived liveliness. Another goal was to see whether we could apply regression and/or machine learning to identify neighborhoods in Atlanta that had a significant shift in perceived liveliness in the years 2016-2021.

A critical element of the project was the use of segmentation. We applied a segmentation method to first derive variables from the Place Pulse data to use as model input to train the model. We then ran the same segmentation method on the Atlanta’s Google Street View images from 2021 to predict the liveliness scores for each image. In order to match the Google Street View images to the Place Pulse image locations, we used the same coordinates, as well as the same headings for the image extractions.

This document goes over the specific objectives, methods, results, and discussion of the project.

Research Question

What elements influence individuals’ perceptions of liveliness in urban areas, and which factors have a more significant impact?

Have certain neighborhoods seen an increase or decrease in liveliness over recent years?

Workflow

Data

Place Pulse

Place Pulse is a dataset that compiles user input on more subjective metrics such as ‘liveliness’, ‘safety’, ‘wealthy’, ‘beautiful’, ‘depressing’ etc, along with the accompanied images of the surveyed locations. There are 111390 images from 56 cities, and 4059 images in the Atlanta area. The liveliness score (the metric that we used in this project) ranges from 6-45. The most recent available Place Pulse inputs were from 2016.

Place Pulse image examples

Place Pulse image data in Atlanta

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   7.124  21.942  25.166  25.114  28.371  42.636

Google Street View

Another important dataset we used was the Google Street View data. These images, contrary to the Place Pulse data (latest available were from 2016), were available for more recent years (2021). Using the Google Street View data, we aimed to predict the liveliness scores of the City of Atlanta in 2021, and compare the difference to the liveliness scores from 2016 to see whether there were any significant changes in perceived liveliness in this 5 year window.

Google Street View Example

Methods

Segmentation

Segmentation was a large element of this project. We applied the SegFormer (NeurIPS 2021) segmentation method to first derive variables from the Place Pulse data to use as model input to run the regression/train the machine learning model.

We applied the same segmentation model to derive the segmentation variables for each Google Street View image that we used to predict the liveliness score for each.

The segmentation model returned an extensive list of variables that we then analyzed to decide which to include when proceeding with the rest of the analysis.

Variable Selection

Manual Selection

In order to increase the reliability of our input data, we only used Place Pulse images that had over 10 votes. 22195 out of 111390 images were selected through this step. In addition, as mentioned before, the segmentation model returned a large list of variables. In order to filter out those that weren’t as significant, we took out the variables where more than 90% of all data points were NAs.

We also looked at relevant studies to see what types of variables others in the field were using. Combined with our domain knowledge of urban environments and intuitive and critical thinking, we decided on a list of variables that we saw fit to utilize.

##  [1] "wall"             "building"         "sky"              "tree"            
##  [5] "road"             "grass"            "sidewalk"         "person"          
##  [9] "earth"            "plant"            "car"              "field"           
## [13] "fence"            "signboard"        "path"             "streetlight"     
## [17] "pole"             "van"              "greenness"        "b2s_ratio"       
## [21] "building_ratio"   "street_infra"     "vehicle_ratio"    "verticle_element"
## [25] "natural_urban"

Our next step was to run a correlation analysis to see whether any of our variables were strongly correlated, thus suggesting the need to remove certain variables.

Correlation Analysis

Before

Below is the result of the correlation analysis for the list of variables that we manually selected. As we can see, car & vehicle_ratio (car + van) have a very strong correlation of 0.982. Similarly, tree & greenness (tree + grass + plant + field), signboard & vertical_element (streetlight + pole + signboard), road & street_infra (road + sidewalk + path + streetlight + pole + signboard) also had correlation coefficients over 0.9.

After running a few models and comparing their performances, we decided to remove car, tree, building, road, and vertical_element.

After

After removing certain highly-correlated variables, below are our final variables that we used for the remainder of the analysis.

##  [1] "wall"            "sky"             "grass"           "sidewalk"       
##  [5] "person"          "earth"           "plant"           "field"          
##  [9] "fence"           "signboard"       "path"            "streetlight"    
## [13] "pole"            "van"             "trueskill.score" "greenness"      
## [17] "b2s_ratio"       "building_ratio"  "street_infra"    "vehicle_ratio"  
## [21] "natural_urban"

Predictive Modeling

Using the Place Pulse segmentation data, we started by running simple models (linear regression, stepwise regression, elastic net), then tested out more advanced machine learning models (decision tree, random forest) to see whether the Rsquared value will improve. We found that the Random Forest model using the list of variables that we selected in the previous step had the best predictive capacity, with an Rsquared value of 0.2731358, and decided to move forward with this model for the predictions.

We performed a comparable analysis to this study utilizing Random Forest (Yao Yao, Zhaotang Liang, Zehao Yuan, Penghua Liu, Yongpan Bie, Jinbao Zhang, Ruoyu Wang, Jiale Wang & Qingfeng Guan (2019) A human-machine adversarial scoring framework for urban perception assessment using street-view images, International Journal of Geographical Information Science, 33:12, 2363-2384, DOI: 10.1080/13658816.2019.1643024). The absence of displayed Rsquared value in the study implies that constructing an effective model based solely on segmentation results might be challenging.

Linear Regression (Stepwise variable selection)

## 
## Call:
## lm(formula = trueskill.score ~ wall + sky + grass + sidewalk + 
##     person + earth + plant + field + fence + van + greenness + 
##     street_infra + vehicle_ratio + verticle_element, data = model_data)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -23.6318  -2.8393   0.0021   2.8778  17.7328 
## 
## Coefficients:
##                  Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      25.14476    0.02993 839.979  < 2e-16 ***
## wall             -0.51568    0.03507 -14.703  < 2e-16 ***
## sky              -1.29288    0.04098 -31.548  < 2e-16 ***
## grass             0.14489    0.04419   3.279 0.001045 ** 
## sidewalk          0.32849    0.03265  10.060  < 2e-16 ***
## person            0.27564    0.03064   8.994  < 2e-16 ***
## earth            -0.45173    0.03672 -12.301  < 2e-16 ***
## plant             0.17026    0.03549   4.797 1.62e-06 ***
## field            -0.37403    0.03364 -11.119  < 2e-16 ***
## fence            -0.14496    0.03227  -4.492 7.09e-06 ***
## van              -0.13376    0.03090  -4.328 1.51e-05 ***
## greenness         0.08834    0.04569   1.934 0.053178 .  
## street_infra      0.66815    0.04675  14.291  < 2e-16 ***
## vehicle_ratio     1.03424    0.03990  25.922  < 2e-16 ***
## verticle_element -0.11051    0.03035  -3.641 0.000272 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 4.372 on 21314 degrees of freedom
## Multiple R-squared:  0.2088, Adjusted R-squared:  0.2083 
## F-statistic: 401.9 on 14 and 21314 DF,  p-value: < 2.2e-16

##         Variable Importance
## 2            sky   31.54817
## 13 vehicle_ratio   25.92203
## 1           wall   14.70331
## 12  street_infra   14.29145
## 6          earth   12.30062
## 8          field   11.11856

Elastic Net

##    alpha r_squared fit.name
## 1    0.0 0.1958573   alpha0
## 2    0.1 0.1962702 alpha0.1
## 3    0.2 0.1998322 alpha0.2
## 4    0.3 0.1923011 alpha0.3
## 5    0.4 0.1968524 alpha0.4
## 6    0.5 0.1961567 alpha0.5
## 7    0.6 0.1978440 alpha0.6
## 8    0.7 0.1981116 alpha0.7
## 9    0.8 0.1971584 alpha0.8
## 10   0.9 0.1984561 alpha0.9
## 11   1.0 0.1974728   alpha1

Decision Tree/Random Forest

Decision Tree

## n= 17065 
## 
## node), split, n, deviance, yval
##       * denotes terminal node
## 
##  1) root 17065 411610.30 25.14405  
##    2) field>=-0.1627326 2168  40512.08 21.21297 *
##    3) field< -0.1627326 14897 332719.60 25.71615  
##      6) vehicle_ratio< -0.6491907 4589  96818.04 24.12224  
##       12) sidewalk< -0.57186 1156  24067.91 22.44917 *
##       13) sidewalk>=-0.57186 3433  68424.70 24.68562 *
##      7) vehicle_ratio>=-0.6491907 10308 219052.90 26.42573  
##       14) sky>=-0.4665923 6457 129349.90 25.60365  
##         28) sky>=1.091487 980  19702.40 23.57509 *
##         29) sky< 1.091487 5477 104893.20 25.96662 *
##       15) sky< -0.4665923 3851  78022.44 27.80412 *

##   Resample     RMSE  Rsquared
## 1 Test Set 4.448011 0.1833307

##        Variable Importance
## 6           sky 0.23746586
## 5      sidewalk 0.23228981
## 2         earth 0.15737580
## 8 vehicle_ratio 0.14263007
## 3         field 0.09324021
## 9          wall 0.05770385

Random Forest

##          Variable IncNodePurity
## 1             sky     45945.157
## 2   vehicle_ratio     36042.008
## 3        sidewalk     35303.756
## 4       b2s_ratio     26988.940
## 5       greenness     24484.091
## 6    street_infra     23847.418
## 7  building_ratio     22891.695
## 8   natural_urban     21558.421
## 9           earth     20942.021
## 10          field     19961.575
## 11           wall     19907.693
## 12          grass     16785.274
## 13          plant     15456.932
## 14          fence     14747.940
## 15    streetlight     11319.499
## 16         person     10923.633
## 17      signboard     10667.290
## 18           pole      7244.027
## 19            van      3682.332
## 20           path      3626.801

##   Resample     RMSE  Rsquared
## 1 Test Set 4.196868 0.2731358

Results

In order to predict the liveliness scores for the most recent Atlanta Google Street View images, we fit the segmentation results of the images to the Random Forest model that we built using the Place Pulse data.

A notable finding was that these models tend to avoid extreme values when making predictions (unless built specifically for that purpose), and our predicted scores skewed towards the mean of the original Place Pulse data, as seen in the boxplots below.

Point-level

The maps below compares the original scores from the Place Pulse data that we used to build the model (Left) to the predicted values (Right). As pointed out previously, the range of the predicted scores is narrow compared to the original values.

However, one can still spot areas that have relatively higher liveliness scores towards the left-center and central part of the city as well as certain ‘dark spots’ with clusters of lower predicted scores.

Block Group-level

We aggregated the points into different block groups, calculating the average score for each group based on the points within it. It’s important to note that we used the 2015 block group boundary for Place Pulse data and the 2021 block group boundary for the downloaded Google Street View images. These two boundaries do not perfectly match due to boundary adjustments. As a result, there are noticeable holes in some areas without values, indicating that there are no points within those block groups. In total, there are 528 block groups included for Place Pulse data and 662 block groups included for downloaded Google Street View images. The Place Pulse scores show a relatively clear pattern: high-scoring block groups are concentrated at the city center, while low-scoring block groups are mostly located at the boundary. However, the prediction scores display some changes; we observed high-scoring block groups appearing at the boundary as well.

Block Group level

Hotspot Analysis

The hot spot analysis reveals a clearer pattern change. In the 2015 block groups, three cold spots are identified: one at the west, another at the southwest, and a third at the southeast, while a significant hot spot exists at the city center. By 2021, the city center’s hot spot has diminished in size and shifted towards the east. The cold spots in the west and southeast have disappeared, and notably, the cold spot at the southwest has transformed from a cold spot to a hot spot, indicating significant liveliness improvement in that area (College Park/East Point). However, conversely, a substantial cold spot has emerged in the western-central area, with several block groups changing from hot spots to cold spots, such as the West End neighborhoods. (For the hotspot analysis, we used red as a hotspot for increased liveliness, and blue as a hotspot for decreased liveliness, aligning with widely used color annotations.)

Hotspot Analysis

Change Detection

We calculated the score difference for each block group by subtracting the Place Pulse score from the prediction score. It’s evident that most block groups show an increased score located at the boundary, while those with decreased scores are situated around the city center. Upon analyzing the hot spots based on the score change, we observe that there are three significant hot spots (College Park/East Point, Collier Heights, Gresham Park), illustrating significant score improvement in these areas.

As previously indicated in the report, a contributing factor to this phenomenon could be the Random Forest model predicting scores that tend to cluster around the mean, steering clear of extreme values. To evaluate whether the observed change detection results stem from the model’s performance or actual changes, we delve into real-life examples in the next section.

Real-life Examples

We examined examples from neighborhoods that had experienced significant score changes to assess the validity and coherence of the predictions by manually comparing images in those areas.

The two figures below illustrate that certain score improvements stem from property development, transitioning from a rural to a more urbanized view. This aligns with the understanding that liveliness is closely linked to people and housing.

Image id: 513d7cd1fdc9f03587007000

Score change: 16.2 to 26.9

Location: Around College Park (33.660896 -84.468363)

Image id: 513d9de5fdc9f0358700821c

Score change: 13.2 to 23.4

Location: Around Gresham Park (33.707443 -84.324167)

The images below present examples of neighborhoods with decreased scores. Here, the number of cars appears to be a contributing factor to the decline.

Additionally, we’ve observed that some changes might be attributed to seasonal variations (the two examples below). Despite minimal alterations in the streetscape, the change in greenness due to seasonal shifts stands out. This indicates that greenness significantly influences people’s perception. It also emphasizes the importance of using images from the same season for a more accurate comparison, thereby avoiding score fluctuations caused by seasonal changes in greenery.

Image id: 513d9af8fdc9f0358700786e

Score change: 30.6 to 22.6

Location: Around West End (33.738155 -84.446746)

Image id: 513d9d2bfdc9f03587007fa0

Score change: 32.8 to 24.9

Location: Around Midtown (33.78043 -84.372825)

We also found that some score changes may be due to data quality or variations in people’s perceptions of liveliness. For instance, the images below display a Place Pulse score of 30 (left), considered high, while our predicted result is around 19 (right). Upon comparison with other images in Place Pulse having scores around 30, our group collectively holds the view that the assigned high rating might be too high. The prediction score of 19, we think, is more suitable to the image. However, acknowledging the inherent variability in individual perceptions of liveliness, it’s plausible that others might assign different scores to this image.

Image id: 513da063fdc9f03587008977

Score change: 30.1 to 19.2

Location: Around Downtown (33.766654 -84.404853)

Discussion

Significant Findings

This project enabled us to address the initial research questions we posed.

For our first question, “What elements influence individuals’ perceptions of liveliness in urban areas, and which factors have a more significant impact?”, we found that sky, vehicle ratio (car + van), sidewalk, building-to-street ratio, greenness were the top 5 most important variables when predicting liveliness. In other words, this result suggests that these urban factors have the most influence on individuals’ perceptions of liveliness.

Variable importance in the random forest model

In addressing our second research question regarding changes in liveliness amongst neighborhoods over recent years, our findings revealed that three specific areas—College Park/East Point, Collier Heights, and Gresham Park—have undergone a notable increase in perceived liveliness. Conversely, central neighborhoods, with the West End neighborhood being the most impacted, exhibited a decrease in perceived liveliness.

These outcomes were validated by our examination of real-life examples through image comparisons. Through these real-life examples, we were able to pinpoint certain factors influencing score changes, including property development, seasonal variations, and the ratio of vehicles in the imagery. Recognizing the importance of these details and aiming to synchronize them in future analyses, we anticipate enhancing the accuracy and robustness of our investigations.

Limitations

While we successfully addressed our research questions, the project encountered certain limitations. Firstly, we encountered an alignment issue between the Place Pulse and Google Street View images, despite efforts to synchronize coordinates and headings. This may have contributed to score changes that may not accurately reflect actual changes.

Difference in Place Pulse and Google Street View headings

Second, seasonal change may affect the predicted scores, as we pointed out earlier when describing the decreased liveliness in West End and Midtown. Optimal results are attainable by ensuring images are from the same season, achievable through the use of an additional API to filter Google Street View images based on timestamps.

Our literature review, coupled with our own findings, revealed that image-based models exhibit limited efficacy in accurately predicting liveliness. While the incorporation of external data, such as census data, has the potential to enhance these models, our specific attempts combining Atlanta image and census data did not yield performance improvements compared to the model built solely on the global Place Pulse image data.

In the future, we would like to further pinpoint the specific factors driving score changes.

Lastly, we investigated certain social demographic factors to account for the changes in liveliness scores, but these factors proved to have limited explanatory power.

Conclusion

Overall, this project illustrated that image segmentation coupled with machine learning can be a powerful tool in assessing more subjective measures of urban environments. Moving forward, we are eager to expand our inquiry to encompass other subjective measures such as ‘safety,’ ‘wealth,’ ‘beauty,’ and ‘depression,’ all of which are accessible through the comprehensive Place Pulse dataset. This expansion would further enrich our understanding of diverse urban perceptions and experiences.

The integration of urban technologies with human input, as demonstrated by the surveyed data in this project, offers promising opportunities. This collaborative approach has the potential to untangle the complexities of urban environments, offering valuable insights for designing more sustainable and enriching spaces for future communities. We are enthusiastic about our involvement in this emerging field and eager to continue our pursuit of knowledge and innovation.

References

Yao Yao, Zhaotang Liang, Zehao Yuan, Penghua Liu, Yongpan Bie, Jinbao Zhang, Ruoyu Wang, Jiale Wang & Qingfeng Guan (2019) A human-machine adversarial scoring framework for urban perception assessment using street-view images, International Journal of Geographical Information Science, 33:12, 2363-2384, DOI: 10.1080/13658816.2019.1643024