Data 607 Tidyverse Create Assignment
Sung Lee
3/12/2020
Purpose
Not all web pages are created the same. Oftentimes, in trying to analyze data, we will need a handy utility to scrape web pages. The package rvest is a handy Tidyverse package that will save time having to “harvest”1 web pages.
Preparation
We will need to load the library for rvest.
library(rvest) # Tidyverse package for rvestTo take a look at what rvest can do, let’s scrap the FiveThirtyEight web page at https://data.fivethirtyeight.com/2. Please note that I downloaded a copy of the html for the page and placed it in this link.
The following code allows you to read a URL into your program.
# This line reads the contents of the url into sampleHTML.
# The function read_html can also take a string of HTML code.
sampleHTML <- read_html("https://raw.githubusercontent.com/logicalschema/SPRING2020TIDYVERSE/master/fivethirtyeight.html")Helpful Functions
rvest has some helpful functions to help you parse HTML code.
html_nodes()
Because HTML (a subset of XML) is a markup language that wraps up text in tags (i.e. nodes), you can select specific HTML tags from a web page.
html_nodes() allows you to grab all of the HTML tags from the page. In this example, I grab the <img> tags for the web page.
sampleHTML %>%
html_nodes("img") ## {xml_nodeset (1)}
## [1] <img width="197" height="25" alt="FiveThirtyEight" src="data:image/svg+xm ...html_attr()
Sometimes the information you need is nested inside the HTML tag. You can use the html_attr() function to extract specific attributes for a HTML tag.
Let’s look at all of the <a> tags for the sample FiveThirtyEight page. <a> tags contain hyperlinks for standard HTML. So as we grab the <a> tags, let’s get the hyperlinks associated with them.
sampleHTML %>%
html_nodes("a") %>%
html_attr("href")## [1] "http://fivethirtyeight.com"
## [2] "https://abcnews.go.com/"
## [3] "http://fivethirtyeight.com/politics/"
## [4] "http://fivethirtyeight.com/sports/"
## [5] "http://fivethirtyeight.com/science/"
## [6] "http://fivethirtyeight.com/economics/"
## [7] "http://fivethirtyeight.com/life/"
## [8] "https://abcnews.go.com/"
## [9] "https://projects.fivethirtyeight.com/polls/"
## [10] "https://github.com/fivethirtyeight/data/tree/master/polls"
## [11] "https://projects.fivethirtyeight.com/congress-generic-ballot-polls/"
## [12] "https://github.com/fivethirtyeight/data/tree/master/congress-generic-ballot"
## [13] "https://projects.fivethirtyeight.com/soccer-predictions/"
## [14] "https://github.com/fivethirtyeight/data/tree/master/soccer-spi"
## [15] "https://projects.fivethirtyeight.com/2020-endorsements/democratic-primary/"
## [16] "https://github.com/fivethirtyeight/data/tree/master/endorsements"
## [17] "https://projects.fivethirtyeight.com/trump-approval-ratings/"
## [18] "https://github.com/fivethirtyeight/data/tree/master/trump-approval-ratings"
## [19] "https://projects.fivethirtyeight.com/congress-trump-score/"
## [20] "https://github.com/fivethirtyeight/data/tree/master/congress-trump-score"
## [21] "https://projects.fivethirtyeight.com/2020-nba-predictions/"
## [22] "https://github.com/fivethirtyeight/data/tree/master/nba-forecasts"
## [23] "https://projects.fivethirtyeight.com/2020-nba-player-ratings/"
## [24] "https://github.com/fivethirtyeight/data/tree/master/nba-raptor"
## [25] "https://fivethirtyeight.com/features/luka-doncic-and-the-mavs-are-pushing-the-limits-of-offensive-efficiency/"
## [26] "https://github.com/fivethirtyeight/nba-player-advanced-metrics"
## [27] "https://projects.fivethirtyeight.com/impeachment-polls/"
## [28] "https://github.com/fivethirtyeight/data/tree/master/impeachment-polls"
## [29] "https://projects.fivethirtyeight.com/2019-nfl-forecasting-game/"
## [30] "https://github.com/fivethirtyeight/nfl-elo-game"
## [31] "https://projects.fivethirtyeight.com/2019-nfl-predictions/"
## [32] "https://github.com/fivethirtyeight/data/tree/master/nfl-elo"
## [33] "https://fivethirtyeight.com/features/the-democratic-presidential-candidates-are-becoming-less-popular/"
## [34] "https://github.com/fivethirtyeight/data/tree/master/presidential-candidate-favorables-2019"
## [35] "https://fivethirtyeight.com/features/the-state-of-the-polls-2019/"
## [36] "https://github.com/fivethirtyeight/data/tree/master/pollster-ratings"
## [37] "http://projects.fivethirtyeight.com/pollster-ratings/"
## [38] "https://fivethirtyeight.com/features/the-polls-are-all-right/"
## [39] "https://fivethirtyeight.com/features/the-state-of-the-polls-2016/"
## [40] "http://fivethirtyeight.com/features/how-fivethirtyeight-calculates-pollster-ratings"
## [41] "https://projects.fivethirtyeight.com/2019-mlb-predictions/"
## [42] "https://github.com/fivethirtyeight/data/tree/master/mlb-elo"
## [43] "https://fivethirtyeight.com/features/the-media-frenzy-around-biden-is-fading/"
## [44] "https://github.com/fivethirtyeight/data/tree/master/media-mentions-2020"
## [45] "https://fivethirtyeight.com/features/introducing-raptor-our-new-metric-for-the-modern-nba/"
## [46] "https://github.com/fivethirtyeight/data/tree/master/nba-raptor"
## [47] "https://fivethirtyeight.com/features/how-our-raptor-metric-works/"
## [48] "https://projects.fivethirtyeight.com/college-fight-song-lyrics/"
## [49] "https://github.com/fivethirtyeight/data/tree/master/fight-songs"
## [50] "https://fivethirtyeight.com/features/we-watched-906-foul-balls-to-find-out-where-the-most-dangerous-ones-land/"
## [51] "https://github.com/fivethirtyeight/data/tree/master/foul-balls"
## [52] "https://fivethirtyeight.com/features/which-democrats-are-campaigning-on-trump/"
## [53] "https://github.com/fivethirtyeight/candidate-emails"
## [54] "https://fivethirtyeight.com/features/a-better-way-to-evaluate-nba-defense/"
## [55] "https://github.com/fivethirtyeight/data/tree/master/nba-draymond"
## [56] "https://projects.fivethirtyeight.com/2019-womens-world-cup-predictions/"
## [57] "https://github.com/fivethirtyeight/data/tree/master/womens-world-cup-2019"
## [58] "https://projects.fivethirtyeight.com/checking-our-work/"
## [59] "https://github.com/fivethirtyeight/checking-our-work-data"
## [60] "https://fivethirtyeight.com/features/what-americas-governors-are-talking-about/"
## [61] "https://github.com/fivethirtyeight/data/tree/master/state-of-the-state"
## [62] "https://fivethirtyeight.com/features/which-2020-candidates-have-the-most-in-common-on-twitter/"
## [63] "https://github.com/fivethirtyeight/twitter-overlap"
## [64] "https://fivethirtyeight.com/features/will-trumps-latest-lawsuits-keep-congress-from-investigating-future-presidents/"
## [65] "https://github.com/fivethirtyeight/data/tree/master/trump-lawsuits"
## [66] "https://fivethirtyeight.com/features/trump-is-losing-the-legal-fight-against-sanctuary-cities-but-it-may-still-pay-off-politically/"
## [67] "https://fivethirtyeight.com/features/should-judges-pay-attention-to-trumps-tweets/"
## [68] "https://fivethirtyeight.com/features/why-one-of-trumps-biggest-legal-threats-is-new-yorks-attorney-general/"
## [69] "https://fivethirtyeight.com/features/will-the-supreme-court-fast-track-cases-involving-trump/"
## [70] "https://fivethirtyeight.com/features/why-it-might-be-impossible-to-overturn-a-presidential-pardon/"
## [71] "https://fivethirtyeight.com/features/what-trumps-legal-battles-tell-us-about-presidential-power/"
## [72] "https://fivethirtyeight.com/features/are-you-the-best-warlord/"
## [73] "https://github.com/fivethirtyeight/data/tree/master/riddler-castles"
## [74] "http://fivethirtyeight.com/features/the-battle-for-riddler-nation-round-2/"
## [75] "https://fivethirtyeight.com/features/can-you-rule-riddler-nation/"
## [76] "https://fivethirtyeight.com/features/both-parties-think-the-mueller-report-was-fair-they-just-disagree-on-what-it-says/"
## [77] "https://github.com/fivethirtyeight/data/tree/master/mueller-polls"
## [78] "https://projects.fivethirtyeight.com/russia-investigation/"
## [79] "https://github.com/fivethirtyeight/data/tree/master/russia-investigation"
## [80] "https://fivethirtyeight.com/features/louisiana-tech-was-the-uconn-of-the-80s/"
## [81] "https://github.com/fivethirtyeight/data/tree/master/ncaa-womens-basketball-tournament"
## [82] "https://fivethirtyeight.com/features/two-years-in-turnover-in-trumps-cabinet-is-still-historically-high/"
## [83] "https://github.com/fivethirtyeight/data/tree/master/cabinet-turnover"
## [84] "https://fivethirtyeight.com/features/how-fivethirtyeights-2018-midterm-forecasts-did/"
## [85] "https://github.com/fivethirtyeight/data/tree/master/forecast-review"
## [86] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/house/"
## [87] "https://github.com/fivethirtyeight/data/tree/master/house-forecast-2018"
## [88] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/house/"
## [89] "https://github.com/fivethirtyeight/data/tree/master/partisan-lean"
## [90] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/senate/"
## [91] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/governor/"
## [92] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/senate/"
## [93] "https://github.com/fivethirtyeight/data/tree/master/senate-forecast-2018"
## [94] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/governor/"
## [95] "https://github.com/fivethirtyeight/data/tree/master/governors-forecast-2018"
## [96] "https://fivethirtyeight.com/features/how-actblue-is-trying-to-turn-small-donations-into-a-blue-wave"
## [97] "https://github.com/fivethirtyeight/actblue-analysis"
## [98] "https://fivethirtyeight.com/features/we-looked-at-hundreds-of-endorsements-heres-who-republicans-are-listening-to/"
## [99] "https://github.com/fivethirtyeight/data/tree/master/primary-candidates-2018"
## [100] "https://fivethirtyeight.com/features/the-establishment-is-beating-the-progressive-wing-in-democratic-primaries-so-far/"
## [101] "https://fivethirtyeight.com/features/democrats-primaries-candidates-demographics/"
## [102] "https://fivethirtyeight.com/features/election-update-the-house-districts-that-swing-the-most-and-least-with-the-national-mood/"
## [103] "https://github.com/fivethirtyeight/data/tree/master/political-elasticity-scores"
## [104] "https://fivethirtyeight.com/features/how-much-trouble-is-ted-cruz-really-in/"
## [105] "https://github.com/fivethirtyeight/data/tree/master/august-senate-polls"
## [106] "https://fivethirtyeight.com/features/why-were-sharing-3-million-russian-troll-tweets/"
## [107] "https://github.com/fivethirtyeight/russian-troll-tweets"
## [108] "https://projects.fivethirtyeight.com/world-cup-comparisons/"
## [109] "https://github.com/fivethirtyeight/data/tree/master/world-cup-comparisons"
## [110] "https://projects.fivethirtyeight.com/2018-world-cup-predictions/"
## [111] "https://github.com/fivethirtyeight/data/tree/master/world-cup-2018"
## [112] "https://fivethirtyeight.com/features/the-lebron-james-decision-making-machine/"
## [113] "https://github.com/fivethirtyeight/data/tree/master/lebron"
## [114] "https://fivethirtyeight.com/features/what-do-men-think-it-means-to-be-a-man/"
## [115] "https://github.com/fivethirtyeight/data/tree/master/masculinity-survey"
## [116] "https://projects.fivethirtyeight.com/2018-march-madness-predictions/"
## [117] "https://github.com/fivethirtyeight/data/tree/master/march-madness-predictions-2018"
## [118] "https://fivethirtyeight.com/features/be-skeptical-of-anyone-who-tells-you-they-know-how-democrats-can-win-in-november/"
## [119] "https://github.com/fivethirtyeight/data/tree/master/special-elections"
## [120] "https://projects.fivethirtyeight.com/guns-parkland-polling-quiz/"
## [121] "https://github.com/fivethirtyeight/data/tree/master/poll-quiz-guns"
## [122] "https://fivethirtyeight.com/features/the-economy-is-keeping-reluctant-trump-voters-with-him/"
## [123] "https://github.com/fivethirtyeight/data/tree/master/reluctant-trump"
## [124] "https://fivethirtyeight.com/features/trumps-reluctant-voters-are-getting-more-reluctant/"
## [125] "https://fivethirtyeight.com/features/how-trumps-least-enthusiastic-voters-feel-about-him-now/"
## [126] "https://fivethirtyeight.com/features/reluctant-trump-voters-swung-the-election-heres-how-they-think-hes-doing/"
## [127] "https://fivethirtyeight.com/features/pepe-the-frog-symbolism-cryptoart-blockchain/"
## [128] "https://github.com/fivethirtyeight/data/tree/master/rare-pepes"
## [129] "https://fivethirtyeight.com/features/more-people-are-resigning-from-congress-than-at-any-time-in-recent-history/"
## [130] "https://github.com/fivethirtyeight/data/tree/master/congress-resignations"
## [131] "https://projects.fivethirtyeight.com/redistricting-maps/"
## [132] "https://github.com/fivethirtyeight/redistricting-atlas-data"
## [133] "https://projects.fivethirtyeight.com/next-bechdel/"
## [134] "https://github.com/fivethirtyeight/data/tree/master/next-bechdel"
## [135] "https://fivethirtyeight.com/features/kenley-jansen-is-the-model-of-a-modern-reliever/"
## [136] "https://github.com/fivethirtyeight/data/tree/master/goose"
## [137] "https://fivethirtyeight.com/features/goose-egg-new-save-stat-relief-pitchers/"
## [138] "http://fivethirtyeight.com/features/the-ultimate-halloween-candy-power-ranking/"
## [139] "https://github.com/fivethirtyeight/data/tree/master/candy-power-ranking"
## [140] "https://fivethirtyeight.com/features/the-worst-tweeter-in-politics-isnt-trump/"
## [141] "https://github.com/fivethirtyeight/data/tree/master/twitter-ratio"
## [142] "https://fivethirtyeight.com/features/the-media-really-started-paying-attention-to-puerto-rico-when-trump-did/"
## [143] "https://github.com/fivethirtyeight/data/tree/master/puerto-rico-media"
## [144] "https://fivethirtyeight.com/features/the-media-really-has-neglected-puerto-rico/"
## [145] "https://fivethirtyeight.com/features/how-every-nfl-teams-fans-lean-politically/"
## [146] "https://github.com/fivethirtyeight/data/tree/master/nfl-fandom"
## [147] "https://projects.fivethirtyeight.com/sandy-311/"
## [148] "https://github.com/fivethirtyeight/data/tree/master/sandy-311-calls"
## [149] "https://fivethirtyeight.com/features/what-the-world-thinks-of-trump/"
## [150] "https://github.com/fivethirtyeight/data/tree/master/trump-world-trust"
## [151] "https://fivethirtyeight.com/features/al-gores-new-movie-exposes-the-big-flaw-in-online-movie-ratings/"
## [152] "https://github.com/fivethirtyeight/data/tree/master/inconvenient-sequel"
## [153] "https://fivethirtyeight.com/features/the-mayweather-mcgregor-fight-as-told-through-emojis/"
## [154] "https://github.com/fivethirtyeight/data/tree/master/mayweather-mcgregor"
## [155] "https://fivethirtyeight.com/features/mayweather-is-defined-by-the-zero-next-to-his-name/"
## [156] "https://github.com/fivethirtyeight/data/tree/master/undefeated-boxers"
## [157] "https://fivethirtyeight.com/features/american-chess-is-great-again/"
## [158] "https://github.com/fivethirtyeight/data/tree/master/chess-transfers"
## [159] "https://fivethirtyeight.com/features/rachels-season-is-fitting-neatly-into-bachelorette-history/"
## [160] "https://github.com/fivethirtyeight/data/tree/master/bachelorette"
## [161] "https://fivethirtyeight.com/features/why-the-gop-is-so-hell-bent-on-passing-an-unpopular-health-care-bill/"
## [162] "https://github.com/fivethirtyeight/data/tree/master/ahca-polls"
## [163] "https://fivethirtyeight.com/features/trump-might-be-the-first-president-to-scrap-a-national-monument/"
## [164] "https://github.com/fivethirtyeight/data/tree/master/antiquities-act"
## [165] "https://fivethirtyeight.com/features/when-scientists-donate-to-politicians-its-usually-to-democrats/"
## [166] "https://github.com/fivethirtyeight/data/tree/master/science-giving"
## [167] "https://fivethirtyeight.com/features/how-qi-and-za-changed-scrabble/"
## [168] "https://github.com/fivethirtyeight/data/tree/master/scrabble-games"
## [169] "https://fivethirtyeight.com/features/for-a-trump-nominee-neil-gorsuchs-record-is-surprisingly-moderate-on-immigration/"
## [170] "https://github.com/fivethirtyeight/data/tree/master/tenth-circuit"
## [171] "https://fivethirtyeight.com/features/dissecting-trumps-most-rabid-online-following/"
## [172] "https://github.com/fivethirtyeight/data/tree/master/subreddit-algebra"
## [173] "https://fivethirtyeight.com/features/higher-rates-of-hate-crimes-are-tied-to-income-inequality/"
## [174] "https://github.com/fivethirtyeight/data/tree/master/hate-crimes"
## [175] "https://fivethirtyeight.com/features/obama-granted-clemency-unlike-any-other-president-in-history/"
## [176] "https://github.com/fivethirtyeight/data/tree/master/obama-commutations"
## [177] "http://fivethirtyeight.com/features/the-last-10-weeks-of-2016-campaign-stops-in-one-handy-gif/"
## [178] "https://github.com/fivethirtyeight/data/tree/master/presidential-campaign-trail"
## [179] "http://fivethirtyeight.com/features/a-handful-of-cities-are-driving-2016s-rise-in-murders/"
## [180] "https://github.com/fivethirtyeight/data/tree/master/murder_2016"
## [181] "http://fivethirtyeight.com/features/when-does-praying-in-public-make-others-uncomfortable/"
## [182] "https://github.com/fivethirtyeight/data/tree/master/religion-survey"
## [183] "http://fivethirtyeight.com/features/the-rams-are-dead-to-me-so-i-answered-3352-questions-to-find-a-new-team/"
## [184] "https://github.com/fivethirtyeight/data/tree/master/nfl-favorite-team"
## [185] "https://projects.fivethirtyeight.com/clinton-trump-hip-hop-lyrics/"
## [186] "https://github.com/fivethirtyeight/data/tree/master/hip-hop-candidate-lyrics"
## [187] "http://fivethirtyeight.com/features/some-people-are-too-superstitious-to-have-a-baby-on-friday-the-13th/"
## [188] "https://github.com/fivethirtyeight/data/tree/master/births"
## [189] "http://fivethirtyeight.com/features/sitting-presidents-give-way-more-commencement-speeches-than-they-used-to/"
## [190] "https://github.com/fivethirtyeight/data/tree/master/presidential-commencement-speeches"
## [191] "http://fivethirtyeight.com/features/how-donald-trump-hacked-the-media/"
## [192] "https://github.com/fivethirtyeight/data/tree/master/trump-news"
## [193] "http://fivethirtyeight.com/features/these-are-the-phrases-each-gop-candidate-repeats-most"
## [194] "https://github.com/fivethirtyeight/data/tree/master/repeated-phrases-gop"
## [195] "http://fivethirtyeight.com/features/iowa-teaches-pollsters-to-poll-until-the-end/"
## [196] "https://github.com/fivethirtyeight/data/tree/master/poll-of-pollsters"
## [197] "http://fivethirtyeight.com/features/most-pollsters-say-their-reputations-have-worsened/"
## [198] "http://fivethirtyeight.com/features/most-pollsters-we-surveyed-say-polling-did-well-this-year/"
## [199] "http://fivethirtyeight.com/features/even-pollsters-dont-know-all-the-details-of-how-their-polls-are-made/"
## [200] "http://fivethirtyeight.com/features/pollsters-say-they-follow-ethical-standards-but-they-arent-so-sure-about-their-peers/"
## [201] "http://fivethirtyeight.com/features/pollsters-predict-greater-polling-error-in-midterm-elections/"
## [202] "http://fivethirtyeight.com/features/you-cant-trust-what-you-read-about-nutrition"
## [203] "https://github.com/fivethirtyeight/data/tree/master/nutrition-studies"
## [204] "https://fivethirtyeight.com/features/is-uber-making-nyc-rush-hour-traffic-worse/"
## [205] "https://github.com/fivethirtyeight/uber-tlc-foil-response"
## [206] "https://fivethirtyeight.com/features/uber-is-taking-millions-of-manhattan-rides-away-from-taxis/"
## [207] "https://fivethirtyeight.com/features/public-transit-should-be-ubers-new-best-friend/"
## [208] "https://fivethirtyeight.com/features/uber-is-serving-new-yorks-outer-boroughs-more-than-taxis-are/"
## [209] "http://fivethirtyeight.com/features/complete-catalog-curses-deaths-quentin-tarantino-films"
## [210] "https://github.com/fivethirtyeight/data/tree/master/tarantino"
## [211] "https://projects.fivethirtyeight.com/complete-history-of-the-nba/"
## [212] "https://github.com/fivethirtyeight/data/tree/master/nba-elo"
## [213] "http://fivethirtyeight.com/features/heres-what-your-part-of-america-eats-on-thanksgiving"
## [214] "https://github.com/fivethirtyeight/data/tree/master/thanksgiving-2015"
## [215] "http://fivethirtyeight.com/features/the-rise-of-religiously-inspired-terrorism-in-france/"
## [216] "https://github.com/fivethirtyeight/data/tree/master/terrorism"
## [217] "http://fivethirtyeight.com/features/college-students-arent-the-only-ones-abusing-adderall"
## [218] "https://github.com/fivethirtyeight/data/tree/master/study-drugs"
## [219] "http://fivethirtyeight.com/features/fandango-movies-ratings/"
## [220] "https://github.com/fivethirtyeight/data/tree/master/fandango"
## [221] "http://fivethirtyeight.com/features/straight-outta-compton-is-the-rare-biopic-not-about-white-dudes"
## [222] "https://github.com/fivethirtyeight/data/tree/master/biopics"
## [223] "http://fivethirtyeight.com/features/every-guest-jon-stewart-ever-had-on-the-daily-show/"
## [224] "https://github.com/fivethirtyeight/data/tree/master/daily-show-guests"
## [225] "http://fivethirtyeight.com/features/buster-poseys-pitch-framing-makes-him-a-potential-mvp/"
## [226] "https://github.com/fivethirtyeight/data/tree/master/buster-posey-mvp"
## [227] "http://fivethirtyeight.com/features/what-12-months-of-record-setting-temperatures-looks-like-across-the-u-s/"
## [228] "https://github.com/fivethirtyeight/data/tree/master/us-weather-history"
## [229] "http://fivethirtyeight.com/features/projecting-the-top-50-players-in-the-2015-nba-draft-class/"
## [230] "https://github.com/fivethirtyeight/data/tree/master/nba-draft-2015"
## [231] "http://fivethirtyeight.com/features/there-are-922-unisex-names-in-america-is-yours-one-of-them"
## [232] "https://github.com/fivethirtyeight/data/tree/master/unisex-names"
## [233] "http://fivethirtyeight.com/features/blatters-reign-at-fifa-hasnt-helped-soccers-poor/"
## [234] "https://github.com/fivethirtyeight/data/tree/master/elo-blatter"
## [235] "http://fivethirtyeight.com/features/why-some-tennis-matches-take-forever/"
## [236] "https://github.com/fivethirtyeight/data/tree/master/tennis-time"
## [237] "http://fivethirtyeight.com/features/where-police-have-killed-americans-in-2015"
## [238] "https://github.com/fivethirtyeight/data/tree/master/police-killings"
## [239] "https://fivethirtyeight.com/features/how-our-womens-world-cup-model-works/"
## [240] "https://github.com/fivethirtyeight/data/tree/master/womens-world-cup-predictions"
## [241] "https://projects.fivethirtyeight.com/womens-world-cup/"
## [242] "http://fivethirtyeight.com/features/the-rock-isnt-alone-lots-of-people-are-worried-about-the-big-one"
## [243] "https://github.com/fivethirtyeight/data/tree/master/san-andreas"
## [244] "http://fivethirtyeight.com/features/how-to-break-fifa/"
## [245] "https://github.com/fivethirtyeight/data/tree/master/fifa"
## [246] "http://fivethirtyeight.com/features/evangelical-protestants-are-the-biggest-winners-when-people-change-faiths/"
## [247] "https://github.com/fivethirtyeight/data/tree/master/pew-religions"
## [248] "http://fivethirtyeight.com/features/mad-men-is-ending-whats-next-for-the-cast/"
## [249] "https://github.com/fivethirtyeight/data/tree/master/mad-men"
## [250] "http://fivethirtyeight.com/features/avengers-death-comics-age-of-ultron"
## [251] "https://github.com/fivethirtyeight/data/tree/master/avengers"
## [252] "http://fivethirtyeight.com/features/how-baby-boomers-get-high/"
## [253] "https://github.com/fivethirtyeight/data/tree/master/drug-use-by-age"
## [254] "http://fivethirtyeight.com/features/weather-forecast-news-app-habits/"
## [255] "https://github.com/fivethirtyeight/data/tree/master/weather-check"
## [256] "http://fivethirtyeight.com/interactives/march-madness-predictions-2015/#mens"
## [257] "https://github.com/fivethirtyeight/data/tree/master/march-madness-predictions-2015"
## [258] "https://fivethirtyeight.com/features/every-nba-teams-chance-of-winning-in-every-minute-across-every-game/"
## [259] "https://github.com/fivethirtyeight/data/tree/master/nba-winprobs"
## [260] "http://fivethirtyeight.com/features/romney-not-running-for-president"
## [261] "https://github.com/fivethirtyeight/data/tree/master/potential-candidates"
## [262] "http://fivethirtyeight.com/features/2016-president-who-is-going-to-run"
## [263] "http://fivethirtyeight.com/features/who-goes-to-meaningless-nfl-games-and-why/"
## [264] "https://github.com/fivethirtyeight/data/tree/master/nfl-ticket-prices"
## [265] "http://fivethirtyeight.com/features/whats-the-most-common-name-in-america/"
## [266] "https://github.com/fivethirtyeight/data/tree/master/most-common-name"
## [267] "http://fivethirtyeight.com/features/women-in-comic-books/"
## [268] "https://github.com/fivethirtyeight/data/tree/master/comic-characters"
## [269] "http://fivethirtyeight.com/features/dear-mona-how-many-flight-attendants-are-men/"
## [270] "https://github.com/fivethirtyeight/data/tree/master/male-flight-attendants"
## [271] "http://fivethirtyeight.com/features/marriage-isnt-dead-yet/"
## [272] "https://github.com/fivethirtyeight/data/tree/master/marriage"
## [273] "http://fivethirtyeight.com/features/how-the-fivethirtyeight-senate-forecast-model-works/"
## [274] "https://github.com/fivethirtyeight/data/tree/master/forecast-methodology"
## [275] "http://fivethirtyeight.com/features/the-economic-guide-to-picking-a-college-major/"
## [276] "https://github.com/fivethirtyeight/data/tree/master/college-majors"
## [277] "http://fivethirtyeight.com/features/airplane-etiquette-recline-seat/"
## [278] "https://github.com/fivethirtyeight/data/tree/master/flying-etiquette-survey"
## [279] "http://fivethirtyeight.com/features/nfl-domestic-violence-policy-suspensions/"
## [280] "https://github.com/fivethirtyeight/data/tree/master/nfl-suspensions"
## [281] "http://fivethirtyeight.com/features/most-police-dont-live-in-the-cities-they-serve/"
## [282] "https://github.com/fivethirtyeight/data/tree/master/police-locals"
## [283] "http://fivethirtyeight.com/features/dear-mona-followup-where-do-people-drink-the-most-beer-wine-and-spirits/"
## [284] "https://github.com/fivethirtyeight/data/tree/master/alcohol-consumption"
## [285] "http://fivethirtyeight.com/features/the-football-hall-of-fame-has-a-receiver-problem/"
## [286] "https://github.com/fivethirtyeight/data/tree/master/nfl-wide-receivers"
## [287] "http://fivethirtyeight.com/features/dear-mona-how-many-couples-sleep-in-separate-beds/"
## [288] "https://github.com/fivethirtyeight/data/tree/master/sleeping-alone-data"
## [289] "http://fivethirtyeight.com/features/americas-favorite-star-wars-movies-and-least-favorite-characters/"
## [290] "https://github.com/fivethirtyeight/data/tree/master/star-wars-survey"
## [291] "http://fivethirtyeight.com/features/should-travelers-avoid-flying-airlines-that-have-had-crashes-in-the-past/"
## [292] "https://github.com/fivethirtyeight/data/tree/master/airline-safety"
## [293] "http://fivethirtyeight.com/features/the-fivethirtyeight-international-food-associations-2014-world-cup/"
## [294] "https://github.com/fivethirtyeight/data/tree/master/food-world-cup"
## [295] "http://fivethirtyeight.com/features/elitist-superfluous-or-popular-we-polled-americans-on-the-oxford-comma/"
## [296] "https://github.com/fivethirtyeight/data/tree/master/comma-survey-data"
## [297] "http://fivethirtyeight.com/interactives/world-cup/"
## [298] "https://github.com/fivethirtyeight/data/tree/master/world-cup-predictions"
## [299] "http://fivethirtyeight.com/features/what-ethan-swan-learned-from-tracking-every-tattoo-in-the-nba/"
## [300] "https://github.com/fivethirtyeight/data/tree/master/nba-tattoos"
## [301] "http://fivethirtyeight.com/features/using-infrastructure-jobs-as-a-measuring-stick-for-state-level-spending/"
## [302] "https://github.com/fivethirtyeight/data/tree/master/infrastructure-jobs"
## [303] "https://fivethirtyeight.com/features/how-americans-like-their-steak/"
## [304] "https://github.com/fivethirtyeight/data/tree/master/steak-survey"
## [305] "http://fivethirtyeight.com/features/weve-published-our-data-on-the-south-and-midwest/"
## [306] "https://github.com/fivethirtyeight/data/tree/master/region-survey"
## [307] "https://fivethirtyeight.com/features/which-states-are-in-the-south/"
## [308] "http://fivethirtyeight.com/features/what-states-are-in-the-midwest/"
## [309] "http://fivethirtyeight.com/features/both-republicans-and-democrats-have-an-age-problem/"
## [310] "https://github.com/fivethirtyeight/data/tree/master/congress-age"
## [311] "http://fivethirtyeight.com/features/early-senate-polls-have-plenty-to-tell-us-about-november/"
## [312] "https://github.com/fivethirtyeight/data/tree/master/early-senate-polls"
## [313] "http://fivethirtyeight.com/features/do-pulitzers-help-newspapers-keep-readers"
## [314] "https://github.com/fivethirtyeight/data/tree/master/pulitzer"
## [315] "http://fivethirtyeight.com/features/a-statistical-analysis-of-the-work-of-bob-ross"
## [316] "https://github.com/fivethirtyeight/data/tree/master/bob-ross"
## [317] "http://fivethirtyeight.com/features/where-are-americas-librarians"
## [318] "https://github.com/fivethirtyeight/data/tree/master/librarians"
## [319] "http://fivethirtyeight.com/features/the-dollar-and-cents-case-against-hollywoods-exclusion-of-women"
## [320] "https://github.com/fivethirtyeight/data/tree/master/bechdel"
## [321] "http://fivethirtyeight.com/features/the-ncaa-bracket-checking-our-work"
## [322] "https://github.com/fivethirtyeight/data/tree/master/historical-ncaa-forecasts"
## [323] "https://fivethirtyeight.com/interactives/march-madness-predictions/"
## [324] "https://github.com/fivethirtyeight/data/tree/master/march-madness-predictions"
## [325] "https://projects.fivethirtyeight.com/polls/"
## [326] "https://github.com/fivethirtyeight/data/tree/master/polls"
## [327] "https://projects.fivethirtyeight.com/congress-generic-ballot-polls/"
## [328] "https://github.com/fivethirtyeight/data/tree/master/congress-generic-ballot"
## [329] "https://projects.fivethirtyeight.com/soccer-predictions/"
## [330] "https://github.com/fivethirtyeight/data/tree/master/soccer-spi"
## [331] "https://projects.fivethirtyeight.com/2020-endorsements/democratic-primary/"
## [332] "https://github.com/fivethirtyeight/data/tree/master/endorsements"
## [333] "https://projects.fivethirtyeight.com/trump-approval-ratings/"
## [334] "https://github.com/fivethirtyeight/data/tree/master/trump-approval-ratings"
## [335] "https://projects.fivethirtyeight.com/congress-trump-score/"
## [336] "https://github.com/fivethirtyeight/data/tree/master/congress-trump-score"
## [337] "https://projects.fivethirtyeight.com/2020-nba-predictions/"
## [338] "https://github.com/fivethirtyeight/data/tree/master/nba-forecasts"
## [339] "https://projects.fivethirtyeight.com/2020-nba-player-ratings/"
## [340] "https://github.com/fivethirtyeight/data/tree/master/nba-raptor"
## [341] "https://fivethirtyeight.com/features/luka-doncic-and-the-mavs-are-pushing-the-limits-of-offensive-efficiency/"
## [342] "https://github.com/fivethirtyeight/nba-player-advanced-metrics"
## [343] "https://projects.fivethirtyeight.com/impeachment-polls/"
## [344] "https://github.com/fivethirtyeight/data/tree/master/impeachment-polls"
## [345] "https://projects.fivethirtyeight.com/2019-nfl-forecasting-game/"
## [346] "https://github.com/fivethirtyeight/nfl-elo-game"
## [347] "https://projects.fivethirtyeight.com/2019-nfl-predictions/"
## [348] "https://github.com/fivethirtyeight/data/tree/master/nfl-elo"
## [349] "https://fivethirtyeight.com/features/the-democratic-presidential-candidates-are-becoming-less-popular/"
## [350] "https://github.com/fivethirtyeight/data/tree/master/presidential-candidate-favorables-2019"
## [351] "https://fivethirtyeight.com/features/the-state-of-the-polls-2019/"
## [352] "http://projects.fivethirtyeight.com/pollster-ratings/"
## [353] "https://fivethirtyeight.com/features/the-polls-are-all-right/"
## [354] "https://fivethirtyeight.com/features/the-state-of-the-polls-2016/"
## [355] "http://fivethirtyeight.com/features/how-fivethirtyeight-calculates-pollster-ratings"
## [356] "https://github.com/fivethirtyeight/data/tree/master/pollster-ratings"
## [357] "https://projects.fivethirtyeight.com/2019-mlb-predictions/"
## [358] "https://github.com/fivethirtyeight/data/tree/master/mlb-elo"
## [359] "https://fivethirtyeight.com/features/the-media-frenzy-around-biden-is-fading/"
## [360] "https://github.com/fivethirtyeight/data/tree/master/media-mentions-2020"
## [361] "https://fivethirtyeight.com/features/introducing-raptor-our-new-metric-for-the-modern-nba/"
## [362] "https://fivethirtyeight.com/features/how-our-raptor-metric-works/"
## [363] "https://github.com/fivethirtyeight/data/tree/master/nba-raptor"
## [364] "https://projects.fivethirtyeight.com/college-fight-song-lyrics/"
## [365] "https://github.com/fivethirtyeight/data/tree/master/fight-songs"
## [366] "https://fivethirtyeight.com/features/we-watched-906-foul-balls-to-find-out-where-the-most-dangerous-ones-land/"
## [367] "https://github.com/fivethirtyeight/data/tree/master/foul-balls"
## [368] "https://fivethirtyeight.com/features/which-democrats-are-campaigning-on-trump/"
## [369] "https://github.com/fivethirtyeight/candidate-emails"
## [370] "https://fivethirtyeight.com/features/a-better-way-to-evaluate-nba-defense/"
## [371] "https://github.com/fivethirtyeight/data/tree/master/nba-draymond"
## [372] "https://projects.fivethirtyeight.com/2019-womens-world-cup-predictions/"
## [373] "https://github.com/fivethirtyeight/data/tree/master/womens-world-cup-2019"
## [374] "https://projects.fivethirtyeight.com/checking-our-work/"
## [375] "https://github.com/fivethirtyeight/checking-our-work-data"
## [376] "https://fivethirtyeight.com/features/what-americas-governors-are-talking-about/"
## [377] "https://github.com/fivethirtyeight/data/tree/master/state-of-the-state"
## [378] "https://fivethirtyeight.com/features/which-2020-candidates-have-the-most-in-common-on-twitter/"
## [379] "https://github.com/fivethirtyeight/twitter-overlap"
## [380] "https://fivethirtyeight.com/features/will-trumps-latest-lawsuits-keep-congress-from-investigating-future-presidents/"
## [381] "https://fivethirtyeight.com/features/trump-is-losing-the-legal-fight-against-sanctuary-cities-but-it-may-still-pay-off-politically/"
## [382] "https://fivethirtyeight.com/features/should-judges-pay-attention-to-trumps-tweets/"
## [383] "https://fivethirtyeight.com/features/why-one-of-trumps-biggest-legal-threats-is-new-yorks-attorney-general/"
## [384] "https://fivethirtyeight.com/features/will-the-supreme-court-fast-track-cases-involving-trump/"
## [385] "https://fivethirtyeight.com/features/why-it-might-be-impossible-to-overturn-a-presidential-pardon/"
## [386] "https://fivethirtyeight.com/features/what-trumps-legal-battles-tell-us-about-presidential-power/"
## [387] "https://github.com/fivethirtyeight/data/tree/master/trump-lawsuits"
## [388] "https://fivethirtyeight.com/features/are-you-the-best-warlord/"
## [389] "http://fivethirtyeight.com/features/the-battle-for-riddler-nation-round-2/"
## [390] "https://fivethirtyeight.com/features/can-you-rule-riddler-nation/"
## [391] "https://github.com/fivethirtyeight/data/tree/master/riddler-castles"
## [392] "https://fivethirtyeight.com/features/both-parties-think-the-mueller-report-was-fair-they-just-disagree-on-what-it-says/"
## [393] "https://github.com/fivethirtyeight/data/tree/master/mueller-polls"
## [394] "https://projects.fivethirtyeight.com/russia-investigation/"
## [395] "https://github.com/fivethirtyeight/data/tree/master/russia-investigation"
## [396] "https://fivethirtyeight.com/features/louisiana-tech-was-the-uconn-of-the-80s/"
## [397] "https://github.com/fivethirtyeight/data/tree/master/ncaa-womens-basketball-tournament"
## [398] "https://fivethirtyeight.com/features/two-years-in-turnover-in-trumps-cabinet-is-still-historically-high/"
## [399] "https://github.com/fivethirtyeight/data/tree/master/cabinet-turnover"
## [400] "https://fivethirtyeight.com/features/how-fivethirtyeights-2018-midterm-forecasts-did/"
## [401] "https://github.com/fivethirtyeight/data/tree/master/forecast-review"
## [402] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/house/"
## [403] "https://github.com/fivethirtyeight/data/tree/master/house-forecast-2018"
## [404] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/house/"
## [405] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/senate/"
## [406] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/governor/"
## [407] "https://github.com/fivethirtyeight/data/tree/master/partisan-lean"
## [408] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/senate/"
## [409] "https://github.com/fivethirtyeight/data/tree/master/senate-forecast-2018"
## [410] "https://projects.fivethirtyeight.com/2018-midterm-election-forecast/governor/"
## [411] "https://github.com/fivethirtyeight/data/tree/master/governors-forecast-2018"
## [412] "https://fivethirtyeight.com/features/how-actblue-is-trying-to-turn-small-donations-into-a-blue-wave"
## [413] "https://github.com/fivethirtyeight/actblue-analysis"
## [414] "https://fivethirtyeight.com/features/we-looked-at-hundreds-of-endorsements-heres-who-republicans-are-listening-to/"
## [415] "https://fivethirtyeight.com/features/the-establishment-is-beating-the-progressive-wing-in-democratic-primaries-so-far/"
## [416] "https://fivethirtyeight.com/features/democrats-primaries-candidates-demographics/"
## [417] "https://github.com/fivethirtyeight/data/tree/master/primary-candidates-2018"
## [418] "https://fivethirtyeight.com/features/election-update-the-house-districts-that-swing-the-most-and-least-with-the-national-mood/"
## [419] "https://github.com/fivethirtyeight/data/tree/master/political-elasticity-scores"
## [420] "https://fivethirtyeight.com/features/how-much-trouble-is-ted-cruz-really-in/"
## [421] "https://github.com/fivethirtyeight/data/tree/master/august-senate-polls"
## [422] "https://fivethirtyeight.com/features/why-were-sharing-3-million-russian-troll-tweets/"
## [423] "https://github.com/fivethirtyeight/russian-troll-tweets"
## [424] "https://projects.fivethirtyeight.com/world-cup-comparisons/"
## [425] "https://github.com/fivethirtyeight/data/tree/master/world-cup-comparisons"
## [426] "https://projects.fivethirtyeight.com/2018-world-cup-predictions/"
## [427] "https://github.com/fivethirtyeight/data/tree/master/world-cup-2018"
## [428] "https://fivethirtyeight.com/features/the-lebron-james-decision-making-machine/"
## [429] "https://github.com/fivethirtyeight/data/tree/master/lebron"
## [430] "https://fivethirtyeight.com/features/what-do-men-think-it-means-to-be-a-man/"
## [431] "https://github.com/fivethirtyeight/data/tree/master/masculinity-survey"
## [432] "https://projects.fivethirtyeight.com/2018-march-madness-predictions/"
## [433] "https://github.com/fivethirtyeight/data/tree/master/march-madness-predictions-2018"
## [434] "https://fivethirtyeight.com/features/be-skeptical-of-anyone-who-tells-you-they-know-how-democrats-can-win-in-november/"
## [435] "https://github.com/fivethirtyeight/data/tree/master/special-elections"
## [436] "https://projects.fivethirtyeight.com/guns-parkland-polling-quiz/"
## [437] "https://github.com/fivethirtyeight/data/tree/master/poll-quiz-guns"
## [438] "https://fivethirtyeight.com/features/the-economy-is-keeping-reluctant-trump-voters-with-him/"
## [439] "https://fivethirtyeight.com/features/trumps-reluctant-voters-are-getting-more-reluctant/"
## [440] "https://fivethirtyeight.com/features/how-trumps-least-enthusiastic-voters-feel-about-him-now/"
## [441] "https://fivethirtyeight.com/features/reluctant-trump-voters-swung-the-election-heres-how-they-think-hes-doing/"
## [442] "https://github.com/fivethirtyeight/data/tree/master/reluctant-trump"
## [443] "https://fivethirtyeight.com/features/pepe-the-frog-symbolism-cryptoart-blockchain/"
## [444] "https://github.com/fivethirtyeight/data/tree/master/rare-pepes"
## [445] "https://fivethirtyeight.com/features/more-people-are-resigning-from-congress-than-at-any-time-in-recent-history/"
## [446] "https://github.com/fivethirtyeight/data/tree/master/congress-resignations"
## [447] "https://projects.fivethirtyeight.com/redistricting-maps/"
## [448] "https://github.com/fivethirtyeight/redistricting-atlas-data"
## [449] "https://projects.fivethirtyeight.com/next-bechdel/"
## [450] "https://github.com/fivethirtyeight/data/tree/master/next-bechdel"
## [451] "https://fivethirtyeight.com/features/kenley-jansen-is-the-model-of-a-modern-reliever/"
## [452] "https://fivethirtyeight.com/features/goose-egg-new-save-stat-relief-pitchers/"
## [453] "https://github.com/fivethirtyeight/data/tree/master/goose"
## [454] "http://fivethirtyeight.com/features/the-ultimate-halloween-candy-power-ranking/"
## [455] "https://github.com/fivethirtyeight/data/tree/master/candy-power-ranking"
## [456] "https://fivethirtyeight.com/features/the-worst-tweeter-in-politics-isnt-trump/"
## [457] "https://github.com/fivethirtyeight/data/tree/master/twitter-ratio"
## [458] "https://fivethirtyeight.com/features/the-media-really-started-paying-attention-to-puerto-rico-when-trump-did/"
## [459] "https://fivethirtyeight.com/features/the-media-really-has-neglected-puerto-rico/"
## [460] "https://github.com/fivethirtyeight/data/tree/master/puerto-rico-media"
## [461] "https://fivethirtyeight.com/features/how-every-nfl-teams-fans-lean-politically/"
## [462] "https://github.com/fivethirtyeight/data/tree/master/nfl-fandom"
## [463] "https://projects.fivethirtyeight.com/sandy-311/"
## [464] "https://github.com/fivethirtyeight/data/tree/master/sandy-311-calls"
## [465] "https://fivethirtyeight.com/features/what-the-world-thinks-of-trump/"
## [466] "https://github.com/fivethirtyeight/data/tree/master/trump-world-trust"
## [467] "https://fivethirtyeight.com/features/al-gores-new-movie-exposes-the-big-flaw-in-online-movie-ratings/"
## [468] "https://github.com/fivethirtyeight/data/tree/master/inconvenient-sequel"
## [469] "https://fivethirtyeight.com/features/the-mayweather-mcgregor-fight-as-told-through-emojis/"
## [470] "https://github.com/fivethirtyeight/data/tree/master/mayweather-mcgregor"
## [471] "https://fivethirtyeight.com/features/mayweather-is-defined-by-the-zero-next-to-his-name/"
## [472] "https://github.com/fivethirtyeight/data/tree/master/undefeated-boxers"
## [473] "https://fivethirtyeight.com/features/american-chess-is-great-again/"
## [474] "https://github.com/fivethirtyeight/data/tree/master/chess-transfers"
## [475] "https://fivethirtyeight.com/features/rachels-season-is-fitting-neatly-into-bachelorette-history/"
## [476] "https://github.com/fivethirtyeight/data/tree/master/bachelorette"
## [477] "https://fivethirtyeight.com/features/why-the-gop-is-so-hell-bent-on-passing-an-unpopular-health-care-bill/"
## [478] "https://github.com/fivethirtyeight/data/tree/master/ahca-polls"
## [479] "https://fivethirtyeight.com/features/trump-might-be-the-first-president-to-scrap-a-national-monument/"
## [480] "https://github.com/fivethirtyeight/data/tree/master/antiquities-act"
## [481] "https://fivethirtyeight.com/features/when-scientists-donate-to-politicians-its-usually-to-democrats/"
## [482] "https://github.com/fivethirtyeight/data/tree/master/science-giving"
## [483] "https://fivethirtyeight.com/features/how-qi-and-za-changed-scrabble/"
## [484] "https://github.com/fivethirtyeight/data/tree/master/scrabble-games"
## [485] "https://fivethirtyeight.com/features/for-a-trump-nominee-neil-gorsuchs-record-is-surprisingly-moderate-on-immigration/"
## [486] "https://github.com/fivethirtyeight/data/tree/master/tenth-circuit"
## [487] "https://fivethirtyeight.com/features/dissecting-trumps-most-rabid-online-following/"
## [488] "https://github.com/fivethirtyeight/data/tree/master/subreddit-algebra"
## [489] "https://fivethirtyeight.com/features/higher-rates-of-hate-crimes-are-tied-to-income-inequality/"
## [490] "https://github.com/fivethirtyeight/data/tree/master/hate-crimes"
## [491] "https://fivethirtyeight.com/features/obama-granted-clemency-unlike-any-other-president-in-history/"
## [492] "https://github.com/fivethirtyeight/data/tree/master/obama-commutations"
## [493] "http://fivethirtyeight.com/features/the-last-10-weeks-of-2016-campaign-stops-in-one-handy-gif/"
## [494] "https://github.com/fivethirtyeight/data/tree/master/presidential-campaign-trail"
## [495] "http://fivethirtyeight.com/features/a-handful-of-cities-are-driving-2016s-rise-in-murders/"
## [496] "https://github.com/fivethirtyeight/data/tree/master/murder_2016"
## [497] "http://fivethirtyeight.com/features/when-does-praying-in-public-make-others-uncomfortable/"
## [498] "https://github.com/fivethirtyeight/data/tree/master/religion-survey"
## [499] "http://fivethirtyeight.com/features/the-rams-are-dead-to-me-so-i-answered-3352-questions-to-find-a-new-team/"
## [500] "https://github.com/fivethirtyeight/data/tree/master/nfl-favorite-team"
## [501] "https://projects.fivethirtyeight.com/clinton-trump-hip-hop-lyrics/"
## [502] "https://github.com/fivethirtyeight/data/tree/master/hip-hop-candidate-lyrics"
## [503] "http://fivethirtyeight.com/features/some-people-are-too-superstitious-to-have-a-baby-on-friday-the-13th/"
## [504] "https://github.com/fivethirtyeight/data/tree/master/births"
## [505] "http://fivethirtyeight.com/features/sitting-presidents-give-way-more-commencement-speeches-than-they-used-to/"
## [506] "https://github.com/fivethirtyeight/data/tree/master/presidential-commencement-speeches"
## [507] "http://fivethirtyeight.com/features/how-donald-trump-hacked-the-media/"
## [508] "https://github.com/fivethirtyeight/data/tree/master/trump-news"
## [509] "http://fivethirtyeight.com/features/these-are-the-phrases-each-gop-candidate-repeats-most"
## [510] "https://github.com/fivethirtyeight/data/tree/master/repeated-phrases-gop"
## [511] "http://fivethirtyeight.com/features/iowa-teaches-pollsters-to-poll-until-the-end/"
## [512] "http://fivethirtyeight.com/features/most-pollsters-say-their-reputations-have-worsened/"
## [513] "http://fivethirtyeight.com/features/most-pollsters-we-surveyed-say-polling-did-well-this-year/"
## [514] "http://fivethirtyeight.com/features/even-pollsters-dont-know-all-the-details-of-how-their-polls-are-made/"
## [515] "http://fivethirtyeight.com/features/pollsters-say-they-follow-ethical-standards-but-they-arent-so-sure-about-their-peers/"
## [516] "http://fivethirtyeight.com/features/pollsters-predict-greater-polling-error-in-midterm-elections/"
## [517] "https://github.com/fivethirtyeight/data/tree/master/poll-of-pollsters"
## [518] "http://fivethirtyeight.com/features/you-cant-trust-what-you-read-about-nutrition"
## [519] "https://github.com/fivethirtyeight/data/tree/master/nutrition-studies"
## [520] "https://fivethirtyeight.com/features/is-uber-making-nyc-rush-hour-traffic-worse/"
## [521] "https://fivethirtyeight.com/features/uber-is-taking-millions-of-manhattan-rides-away-from-taxis/"
## [522] "https://fivethirtyeight.com/features/public-transit-should-be-ubers-new-best-friend/"
## [523] "https://fivethirtyeight.com/features/uber-is-serving-new-yorks-outer-boroughs-more-than-taxis-are/"
## [524] "https://github.com/fivethirtyeight/uber-tlc-foil-response"
## [525] "http://fivethirtyeight.com/features/complete-catalog-curses-deaths-quentin-tarantino-films"
## [526] "https://github.com/fivethirtyeight/data/tree/master/tarantino"
## [527] "https://projects.fivethirtyeight.com/complete-history-of-the-nba/"
## [528] "https://github.com/fivethirtyeight/data/tree/master/nba-elo"
## [529] "http://fivethirtyeight.com/features/heres-what-your-part-of-america-eats-on-thanksgiving"
## [530] "https://github.com/fivethirtyeight/data/tree/master/thanksgiving-2015"
## [531] "http://fivethirtyeight.com/features/the-rise-of-religiously-inspired-terrorism-in-france/"
## [532] "https://github.com/fivethirtyeight/data/tree/master/terrorism"
## [533] "http://fivethirtyeight.com/features/college-students-arent-the-only-ones-abusing-adderall"
## [534] "https://github.com/fivethirtyeight/data/tree/master/study-drugs"
## [535] "http://fivethirtyeight.com/features/fandango-movies-ratings/"
## [536] "https://github.com/fivethirtyeight/data/tree/master/fandango"
## [537] "http://fivethirtyeight.com/features/straight-outta-compton-is-the-rare-biopic-not-about-white-dudes"
## [538] "https://github.com/fivethirtyeight/data/tree/master/biopics"
## [539] "http://fivethirtyeight.com/features/every-guest-jon-stewart-ever-had-on-the-daily-show/"
## [540] "https://github.com/fivethirtyeight/data/tree/master/daily-show-guests"
## [541] "http://fivethirtyeight.com/features/buster-poseys-pitch-framing-makes-him-a-potential-mvp/"
## [542] "https://github.com/fivethirtyeight/data/tree/master/buster-posey-mvp"
## [543] "http://fivethirtyeight.com/features/what-12-months-of-record-setting-temperatures-looks-like-across-the-u-s/"
## [544] "https://github.com/fivethirtyeight/data/tree/master/us-weather-history"
## [545] "http://fivethirtyeight.com/features/projecting-the-top-50-players-in-the-2015-nba-draft-class/"
## [546] "https://github.com/fivethirtyeight/data/tree/master/nba-draft-2015"
## [547] "http://fivethirtyeight.com/features/there-are-922-unisex-names-in-america-is-yours-one-of-them"
## [548] "https://github.com/fivethirtyeight/data/tree/master/unisex-names"
## [549] "http://fivethirtyeight.com/features/blatters-reign-at-fifa-hasnt-helped-soccers-poor/"
## [550] "https://github.com/fivethirtyeight/data/tree/master/elo-blatter"
## [551] "http://fivethirtyeight.com/features/why-some-tennis-matches-take-forever/"
## [552] "https://github.com/fivethirtyeight/data/tree/master/tennis-time"
## [553] "http://fivethirtyeight.com/features/where-police-have-killed-americans-in-2015"
## [554] "https://github.com/fivethirtyeight/data/tree/master/police-killings"
## [555] "https://fivethirtyeight.com/features/how-our-womens-world-cup-model-works/"
## [556] "https://projects.fivethirtyeight.com/womens-world-cup/"
## [557] "https://github.com/fivethirtyeight/data/tree/master/womens-world-cup-predictions"
## [558] "http://fivethirtyeight.com/features/the-rock-isnt-alone-lots-of-people-are-worried-about-the-big-one"
## [559] "https://github.com/fivethirtyeight/data/tree/master/san-andreas"
## [560] "http://fivethirtyeight.com/features/how-to-break-fifa/"
## [561] "https://github.com/fivethirtyeight/data/tree/master/fifa"
## [562] "http://fivethirtyeight.com/features/evangelical-protestants-are-the-biggest-winners-when-people-change-faiths/"
## [563] "https://github.com/fivethirtyeight/data/tree/master/pew-religions"
## [564] "http://fivethirtyeight.com/features/mad-men-is-ending-whats-next-for-the-cast/"
## [565] "https://github.com/fivethirtyeight/data/tree/master/mad-men"
## [566] "http://fivethirtyeight.com/features/avengers-death-comics-age-of-ultron"
## [567] "https://github.com/fivethirtyeight/data/tree/master/avengers"
## [568] "http://fivethirtyeight.com/features/how-baby-boomers-get-high/"
## [569] "https://github.com/fivethirtyeight/data/tree/master/drug-use-by-age"
## [570] "http://fivethirtyeight.com/features/weather-forecast-news-app-habits/"
## [571] "https://github.com/fivethirtyeight/data/tree/master/weather-check"
## [572] "http://fivethirtyeight.com/interactives/march-madness-predictions-2015/#mens"
## [573] "https://github.com/fivethirtyeight/data/tree/master/march-madness-predictions-2015"
## [574] "https://fivethirtyeight.com/features/every-nba-teams-chance-of-winning-in-every-minute-across-every-game/"
## [575] "https://github.com/fivethirtyeight/data/tree/master/nba-winprobs"
## [576] "http://fivethirtyeight.com/features/romney-not-running-for-president"
## [577] "http://fivethirtyeight.com/features/2016-president-who-is-going-to-run"
## [578] "https://github.com/fivethirtyeight/data/tree/master/potential-candidates"
## [579] "http://fivethirtyeight.com/features/who-goes-to-meaningless-nfl-games-and-why/"
## [580] "https://github.com/fivethirtyeight/data/tree/master/nfl-ticket-prices"
## [581] "http://fivethirtyeight.com/features/whats-the-most-common-name-in-america/"
## [582] "https://github.com/fivethirtyeight/data/tree/master/most-common-name"
## [583] "http://fivethirtyeight.com/features/women-in-comic-books/"
## [584] "https://github.com/fivethirtyeight/data/tree/master/comic-characters"
## [585] "http://fivethirtyeight.com/features/dear-mona-how-many-flight-attendants-are-men/"
## [586] "https://github.com/fivethirtyeight/data/tree/master/male-flight-attendants"
## [587] "http://fivethirtyeight.com/features/marriage-isnt-dead-yet/"
## [588] "https://github.com/fivethirtyeight/data/tree/master/marriage"
## [589] "http://fivethirtyeight.com/features/how-the-fivethirtyeight-senate-forecast-model-works/"
## [590] "https://github.com/fivethirtyeight/data/tree/master/forecast-methodology"
## [591] "http://fivethirtyeight.com/features/the-economic-guide-to-picking-a-college-major/"
## [592] "https://github.com/fivethirtyeight/data/tree/master/college-majors"
## [593] "http://fivethirtyeight.com/features/airplane-etiquette-recline-seat/"
## [594] "https://github.com/fivethirtyeight/data/tree/master/flying-etiquette-survey"
## [595] "http://fivethirtyeight.com/features/nfl-domestic-violence-policy-suspensions/"
## [596] "https://github.com/fivethirtyeight/data/tree/master/nfl-suspensions"
## [597] "http://fivethirtyeight.com/features/most-police-dont-live-in-the-cities-they-serve/"
## [598] "https://github.com/fivethirtyeight/data/tree/master/police-locals"
## [599] "http://fivethirtyeight.com/features/dear-mona-followup-where-do-people-drink-the-most-beer-wine-and-spirits/"
## [600] "https://github.com/fivethirtyeight/data/tree/master/alcohol-consumption"
## [601] "http://fivethirtyeight.com/features/the-football-hall-of-fame-has-a-receiver-problem/"
## [602] "https://github.com/fivethirtyeight/data/tree/master/nfl-wide-receivers"
## [603] "http://fivethirtyeight.com/features/dear-mona-how-many-couples-sleep-in-separate-beds/"
## [604] "https://github.com/fivethirtyeight/data/tree/master/sleeping-alone-data"
## [605] "http://fivethirtyeight.com/features/americas-favorite-star-wars-movies-and-least-favorite-characters/"
## [606] "https://github.com/fivethirtyeight/data/tree/master/star-wars-survey"
## [607] "http://fivethirtyeight.com/features/should-travelers-avoid-flying-airlines-that-have-had-crashes-in-the-past/"
## [608] "https://github.com/fivethirtyeight/data/tree/master/airline-safety"
## [609] "http://fivethirtyeight.com/features/the-fivethirtyeight-international-food-associations-2014-world-cup/"
## [610] "https://github.com/fivethirtyeight/data/tree/master/food-world-cup"
## [611] "http://fivethirtyeight.com/features/elitist-superfluous-or-popular-we-polled-americans-on-the-oxford-comma/"
## [612] "https://github.com/fivethirtyeight/data/tree/master/comma-survey-data"
## [613] "http://fivethirtyeight.com/interactives/world-cup/"
## [614] "https://github.com/fivethirtyeight/data/tree/master/world-cup-predictions"
## [615] "http://fivethirtyeight.com/features/what-ethan-swan-learned-from-tracking-every-tattoo-in-the-nba/"
## [616] "https://github.com/fivethirtyeight/data/tree/master/nba-tattoos"
## [617] "http://fivethirtyeight.com/features/using-infrastructure-jobs-as-a-measuring-stick-for-state-level-spending/"
## [618] "https://github.com/fivethirtyeight/data/tree/master/infrastructure-jobs"
## [619] "https://fivethirtyeight.com/features/how-americans-like-their-steak/"
## [620] "https://github.com/fivethirtyeight/data/tree/master/steak-survey"
## [621] "http://fivethirtyeight.com/features/weve-published-our-data-on-the-south-and-midwest/"
## [622] "https://fivethirtyeight.com/features/which-states-are-in-the-south/"
## [623] "http://fivethirtyeight.com/features/what-states-are-in-the-midwest/"
## [624] "https://github.com/fivethirtyeight/data/tree/master/region-survey"
## [625] "http://fivethirtyeight.com/features/both-republicans-and-democrats-have-an-age-problem/"
## [626] "https://github.com/fivethirtyeight/data/tree/master/congress-age"
## [627] "http://fivethirtyeight.com/features/early-senate-polls-have-plenty-to-tell-us-about-november/"
## [628] "https://github.com/fivethirtyeight/data/tree/master/early-senate-polls"
## [629] "http://fivethirtyeight.com/features/do-pulitzers-help-newspapers-keep-readers"
## [630] "https://github.com/fivethirtyeight/data/tree/master/pulitzer"
## [631] "http://fivethirtyeight.com/features/a-statistical-analysis-of-the-work-of-bob-ross"
## [632] "https://github.com/fivethirtyeight/data/tree/master/bob-ross"
## [633] "http://fivethirtyeight.com/features/where-are-americas-librarians"
## [634] "https://github.com/fivethirtyeight/data/tree/master/librarians"
## [635] "http://fivethirtyeight.com/features/the-dollar-and-cents-case-against-hollywoods-exclusion-of-women"
## [636] "https://github.com/fivethirtyeight/data/tree/master/bechdel"
## [637] "http://fivethirtyeight.com/features/the-ncaa-bracket-checking-our-work"
## [638] "https://github.com/fivethirtyeight/data/tree/master/historical-ncaa-forecasts"
## [639] "https://fivethirtyeight.com/interactives/march-madness-predictions/"
## [640] "https://github.com/fivethirtyeight/data/tree/master/march-madness-predictions"
## [641] "http://creativecommons.org/licenses/by/4.0/"
## [642] "http://opensource.org/licenses/MIT"
## [643] "mailto:data@fivethirtyeight.com"
## [644] "https://fivethirtyeight.com/newsletter/"
## [645] "https://fivethirtyeight.com/videos/"
## [646] "https://fivethirtyeight.com/tag/fivethirtyeight-podcasts/"
## [647] "https://twitter.com/fivethirtyeight"
## [648] "https://www.facebook.com/fivethirtyeight"
## [649] "https://data.fivethirtyeight.com/"
## [650] "https://fivethirtyeight.com/datalab/fear-not-readers-we-have-rss-feeds/"
## [651] "https://fivethirtyeight.com/contact/"
## [652] "https://fivethirtyeight.com/jobs/"
## [653] "https://fivethirtyeight.com/masthead/"
## [654] "#footer-additional-info"
## [655] "https://www.nielsen.com/digitalprivacy"
## [656] "https://disneytermsofuse.com/"
## [657] "https://disneyprivacycenter.com/"
## [658] "https://disneyprivacycenter.com/notice-to-california-residents/"
## [659] "https://disneyprivacycenter.com/kids-privacy-policy/english/"
## [660] "http://preferences-mgr.truste.com/?type=abcnews&affiliateId=11&cid=clicksource_4380645_footer_interestbasedads"html_table()
The html_table() function comes in handy if you need to convert a HTML table to a data frame.
Let’s convert the HTML table found on the sample FiveThirtyEight page. Note: for this function it assumes3:
No cells span multiple rows
Headers are in the first row
# dataIndexMobile can be found by looking at the "id" for the table
# Example: If you wanted <table id="dataIndexMobile">, you would use #dataIndexMobile
# Initially the table for the web page does not meet the 1st condition described above.
sampleHTML %>%
html_node("#dataIndex") %>%
html_table(fill = TRUE) # fill = TRUE for badly formed HTML tableshttps://cran.r-project.org/web/packages/rvest/rvest.pdf p 1↩
The download was taken on March 12, 2020 at 10:28 PM EST↩
https://cran.r-project.org/web/packages/rvest/rvest.pdf p 8↩