Scraping SEO Elements

For this exercise, I scraped the title tag, meta description, and H1/H2/H3 headings from two triathlon related websites: USA Triathon (the sport’s governing body) and ROKA Multisport (a triathlon gear brand), using rvest.

library(rvest)
library(stringr)
library(httr2)
library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
# List of URLs to scrape
urls <- c(
  "https://www.usatriathlon.org/",
  "https://rokamultisport.com/"
)

# Empty list to collect results
results <- list()

for (i in seq_along(urls)) {
  
  Sys.sleep(1.5) # polite delay between requests
  
  page <- read_html(urls[i])
  
  title_tag <- page %>%
    html_element("title") %>%
    html_text2() %>%
    str_squish()
  
  meta_desc <- page %>%
    html_element("meta[name='description']") %>%
    html_attr("content") %>%
    str_squish()
  
  h1_tags <- page %>%
    html_elements("h1") %>%
    html_text2() %>%
    str_squish() %>%
    str_c(collapse = " | ")
  
  h2_tags <- page %>%
    html_elements("h2") %>%
    html_text2() %>%
    str_squish() %>%
    str_c(collapse = " | ")
  
  h3_tags <- page %>%
    html_elements("h3") %>%
    html_text2() %>%
    str_squish() %>%
    str_c(collapse = " | ")
  
  results[[i]] <- tibble(
    url = urls[i],
    title = title_tag,
    meta_description = meta_desc,
    h1 = h1_tags,
    h2 = h2_tags,
    h3 = h3_tags
  )
}

# Combine all rows into one data frame
seo_data <- bind_rows(results)
seo_data
## # A tibble: 2 × 6
##   url                           title         meta_description h1    h2    h3   
##   <chr>                         <chr>         <chr>            <chr> <chr> <chr>
## 1 https://www.usatriathlon.org/ USA Triathlo… Read the latest… USA … New … USA …
## 2 https://rokamultisport.com/   Triathlon Ge… ROKA Multisport… ROKA… Sear… RACE…
write.csv(seo_data, file = "seo_data.csv", row.names = FALSE)

Findings

ROKA Multisport show i believe a stronger SEO than USA Triathlon. ROKA’s title tag is clean while USA Triathlon’s title is duplicated this tells me that this is a wasted opportunity for additional keywords. ROKA’s meta description is also longer that align with me because I am a triathlete and this is somethign i might search for, whereas USA Triathlon’s description is generic. Both sites have clean H1 tags. Overall, ROKA’s simpler, more static HTML appears more friendly and SEO optimized than USA Triathlon’s more complex site.