City Comparison: Chicago vs. New York
Introduction
Something that has always interest me, was the differences between major cities. Comparing cities can offer valuable insights as to why people prefer on place over another, or what draws them to a new location. In this analysis, I have taken reviews of Chicago and New York from their own respective Yelp pages. In this analysis, I want to explore what people say about these two cities, if there is a trend in the timing of positive or negative reviews written, as well as to see if one city has more positive reviews overall.
The Data
As mentioned in the introduction, I have scraped these reviews from Chicago’s Yelp page (https://www.yelp.com/biz/city-of-chicago-chicago-6) and New York’s Yelp page (https://www.yelp.com/biz/city-of-new-york-new-york-49). All of the reviews were scraped on April 24th, 2024. The variables I collected were:
-Name: The name of the reviewer
-Date: The date the user posted the review
-Content: The text the user wrote in their review about their respective city
-Page_id: The URL to identify what page the review came from
I did create a new “City” column that has either Chicago or New York to easily identify which city the review is writing about.
Data: City_reviews
Analysis
1. What Are the Most Frequent Emotions Associated with Each City?
To answer this question, I used the NRC lexicon. Using this lexicon, allowed me to get the most commonly used emotive words, and then I could group by the city to be able to visualize how each city compares to each other.
This graph provides a clear visual of the emotional sentiments expressed in both cities. We see that for Chicago, there is a very high count for positive words, at just under 2,000. While New York is a little lower at roughly 1,500. The next most common emotion is negative. This make sense since not everyone is going to love these cities so there will always be a lot of negative reviews. The next most popular emotions are joy, trust, and anticipation. Joy makes a lot of sense to me since these are major cities so there is a lot to do in them that will bring joy to people. As well as anticipation since in cities you never know whats going to happen or what you are going to do, so the anticipation for things to happen or take place can explain why this score is very high. The word trust seems very interesting to me. Is it trust in the city, or something else?
Overall, both cities predominantly express positive emotions than negative ones, with Chicago having higher sentiment scores for joy and positivity compared to New York. The graph shows us that while both cities share a generally positive emotions, Chicago’s sentiment profile is slightly more positive than New York’s.
2. What City Has a Higher Positivity Score?
To answer this question, I calculated the total sentiment scores for each city, after give my reviews in tidy formatting. By taking each word from each review, and then assigning that word a score using the NRC lexicon, I can then calculate the total positivity score for each city, by taking the total number of positive words minus the total number of negative words.
This graph shows us that both Chicago and new York has very high positivity scores. This makes sense as both of these cities are major cities in the United States. We can see that Chicago has a higher positivity score by just 9 points. this offers us a good insight as to how many people said either negative or positive things in their reviews about their respective city.
# A tibble: 2 × 4
city positive_score negative_score positivity_score
<chr> <dbl> <dbl> <dbl>
1 Chicago 1725 936 789
2 New York 1480 699 7813. How Does the Positivity Score Change by Month for Each City?
I did this by using the Bing lexicon. After getting my reviews in tidy formatting, I was able to join the big lexicon on the word. I grouped by city, month, and sentiment to be able to produce a bar chart to easily see the differences in scores by months for each city.
This graph provides a comparative analysis of the positivity scores for Chicago and New York on a monthly basis. In Chicago, the positivity score peaks in June and dips to its lowest in February. On the other hand, in New York it reaches its highest positivity score in October and its lowest in April. Chicago’s highest months June, September, October make a lot of sense to me with the weather being very nice, and football starting up soon it would make more sense that the positivity scores are very high. February in Chicago actually sucks… it is freezing cold and windy, so that score does not surprise me at all. For New York I don’t quite understand why October has the highest positivity score. Maybe it is because the weather is not blazing hot and humid, or freezing cold yet. What also made sense to me in New York is that December and January were positive scores. There is so much to do there for Christmas and New Years and everyone will be writing reviews about the time they had.
Overall, this graph provides valuable insights into the emotional climate of each city and how it changes over time. It also allows for an easy comparison of the cities’ trends across different months. This give us a little insights as to how different months can affect the positivity score associated with a city.
Conclusion
It was very interesting to do this analysis, and see what people were saying about New York or Chicago. Through this analysis only from the reviews I got from each cities yelp page I would say that Chicago is better favored than New York. Through more positive words, a slightly higher positivity score, and overall more “good” emotion words associated with it, Chicago is more favorable than New York. In a deeper analysis it would be interesting to look at the sentiment from various age groups, or demographics to see if that plays into effect overall.