Visualizing Tiktok Trends with RShiny
Motivation:
Tiktok (https://www.tiktok.com/en/) is an immensely popular social networking and mini-video sharing application worldwide with more than 2 billion downloads since Apr 20201. It is especially around the Gen Z and millennial age groups but has also penetrated the other age groups due to the virality and relevance and engagement of its content. Very similar to the social media platforms like Facebook, Instagram, Twitter, WeChat that have reached a massive scale, the platform both has the potential to shape and propagate popular culture, politics and consumer trends - and these information is in parts captured by video content, hashtags, descriptions and meta-data like “shares”, “view counts”, “likes”, “mentions” etc. Hence, I would like to build an interactive visualization to explore the key trends around these.
- Describe the major data and design challenges faced in accomplishing the task and how you plan to overcome these challenges with a proposed sketched design. (3 marks)
Data Challenges:
- TikTok API only allows for a limited number of queries (tested max around 5500 rows returned)
- Solution: Limit the query only to top trending videos make make the analysis significant around viewership
- Hashtags and mentions are mixed together with other words in description
- Solution: Use .py to extract the hashtags and mentions for analysis and counting
- Duplication in data makes will create misrepresentation
- Solution: Deduplicate data by video_id. Only 427 unique videos are turned in the top 5000 video logs
- Freshness of data is important when analysing trends
- Solution: .py script is provided along side Shiny WebApp for timely extraction
Design Challenge:
- Video description and associated data provides alot of nuance and context for analysis and cannot be represented graphically
- Solution provide a data table for user to analyse
- There are multiple plots that are meaningful in the analysis of the TikTok data.
- Use a tabular flow format, where each tab contains a main chart without requiring users to navigate away from the page
- There are multiple ways users might want to filter and search the data:
- Use build in reactive controls like slides and search filter into the interface. Make all the plots and elements responsive to the user actions allowing interactive data exploration
- Social media data has high asymmetric power law like distribution in terms of the views, length, likes, shares etc, where it is possible that there are orders of difference between the top 50 videos
- Solution: Plot the data in log10
2. Provide step-by-step description on how the data visualization was prepared by using ggplot2 and other related R packages. (3 marks)
Step 0 - Data prep | The data was first extracted using TikTok API and prepped using Pandas creating the file “cleaned_trending_5000” which is loaded into R.
Step 1: First using GGplot 2, I tried to create the main visualization which is the plot of Hashtag counts by their respective count based on static data. As shown in the plot below the chart is not interactive and only shows the top 15 hashtags.
Step 2: Using the Shiny Reactive function and the “dplyr”library, I added the logic to allow UI elements to respond to user filters and selection on the fly. The 3 filter capabilities bind the values of the “No. of likes” slider, “Video Length slider” and “Search by Hashtag” input box to filter the plots and datatables.
UI:
Server - Filter logic:
Step 3: Next, I added a data table below using the “DT” library to allow users to dive into the details of each video. The main logic of the table is also shown below:
Server:
Step 4: Then I made the original hashtag plots responsive. This is harder to achieve because it requires the responsive counting of hashtag as user interacts with the dashboard. I experimented with “stringr” and “tidytext” libraries for strings processing but ended writing a the function using the base R/Shiny code.
Server - logic to count and rank by top hashtags
Step 5: Last but not least, 2 more responsive plots, showing Likes count and View counts are plotted using the “ggplot” library, and arranged into the UI using tabular views to the interface is cleaner.
UI - showing tabset widget:
Server - Logic for Views and Likes:
3. The final data visualization and a short description of not more than 350 words. The description must provide at least two useful information revealed by the data visualization. (4 marks)
The key final visualization shows what are the top 15 keys by various additive filters allowing for interactive analysis. Furthermore, a data table is provided below to give the users additional details and context of the visualization. Let’s walk through what this visualization:
At (A), the search for hashtag bar, “#comedy” is set as the input text. This will cause the visualization in (B) and the data table in (C) to be filtered by the comedy hashtag.
This leads us to observation (1): Alongside #comedy, we are able to see what are some of the hashtags that are often used together with it for trending videos. Here we can see some more informative hashtags like:
- #1millionaudition - videos around the themes of audition for users to showcase their creativity and win daily cash prizes, which is one of the most well-recieved TikTok campaign generating >80bn cumulative views)
- #stayhome - which shows the daily lives of users across the globe under Covid19 lockdown
- #tamilcomedy - which is tamil language comedy - it illustrates the upsurge of Tiktok in India and the popularity of comedy content in Tamil speaking populations in India and the diaspora worldwide
Using this we are able to identify what are the some of the key social trends in the world as well as what is generating buzz on the heavily used platform.
Observation (2): Using the data table in (C) one will be able to glean more insights on what does a particular hashtag via the video description along side the video analytics. For example: the video created by the user arnaldomangini has the following description “
“Do you wanna dance with me?#laxedsirenbeat #laxedsirenbeatchallenge #tiktok #tiktokdance #comedy #clown” Which we are able to see that its a comedic version of a dance competition and dance challenges are indeed very interactive and popular on TikTik. This 14 seconds video almost has 1bn views and 43 mm shares.
Besides this, users are able to filter the video by likes, by count and also get a sets of the likes and video views distribution of popular trending videos.
Limitations and future work
The key limitation for this visualization is that I am not able to obtain time series data because of the API throttle and limitation. Hence, am not able to provide momentum and time based views on hashtags/popularity trends.
Furthermore, Tiktok only provides for a small sub-set of query via the API which limits the analysis to top users / videos / hashtags, etc. We are not then able to examine some of the sub-cultures like “#gundam”, “#mimic”, etc
Future work:
- Daily / 6 hourly aggregation of trends
- Network analysis of users_mentions, content relation, hashtags
- NLP on video description
- Video AI - for object idenitication
- Sound AI - for music identification and dialogue transcription