Homework 11: Text Mining Climate Change Research

Data

The Public Library of Science (PLoS) is a nonprofit open-access science, technology, and medicine publisher. PLoS journals represent an extensive library of open-access journals and other scientific literature under an open-content license. The rplos package interacts with the PLoS journals and allows you to search and retrieve a wide variety of information from PLoS – no API key required!

library(rplos)
#plosabstract searches the abstracts
#the fl command identifies what to retrieve
plos = plosabstract(q= "climate change", fl= "abstract", limit = 200)

#create a corpus stripped of meaningless words
plosCorpus=with(plos, VCorpus(VectorSource(plos)))%>%
  tm_map(stripWhitespace)  %>%
  tm_map(removeNumbers)  %>%
  tm_map(removePunctuation)  %>%
  tm_map(content_transformer(tolower))  %>%
  tm_map(removeWords, stopwords("english"))

PLoS Climate Change Wordcloud

library(dplyr)
library(tm)
library("SnowballC")
library("wordcloud")
library("RColorBrewer")
wordcloud(plosCorpus, max.words = 50, scale = c(6, 1),
          colors=brewer.pal(3, "Set2"),random.order = FALSE)

Using Text from Titles

What happens if we use the words from the titles of articles instead of the abstracts? Maybe the resulting word cloud will be tighter, more interesting?

Retrieve PLoS Titles

library(dplyr)
library(tm)
library(rplos)
#plosabstract searches the abstracts
#the fl command identifies what to retrieve
plos_t = plosabstract(q= "climate change", fl= "title", limit = 500)
head(plos_t)

$meta

$data

NA

#create a corpus of titles stripped of meaningless words
titleCorpus=with(plos_t, VCorpus(VectorSource(plos_t)))%>%
  tm_map(stripWhitespace)  %>%
  tm_map(removeNumbers)  %>%
  tm_map(removePunctuation)  %>%
  tm_map(content_transformer(tolower))  %>%
  tm_map(removeWords, stopwords("english"))

PLoS Climate Change Wordcloud

library(SnowballC)
library(wordcloud)
library(RColorBrewer)
wordcloud(titleCorpus, max.words = 50, scale = c(6, 1),
          colors=brewer.pal(3, "Set2"),random.order = FALSE)

Visualizing Research on Climate Change over Time

library(ggplot2)
plot_throughtime(terms = "Climate Change", limit = 800) + geom_line(size=2, color='springgreen4')

Discussion

The increase in papers published on climate change over time is interesting, and I’d like to look at an even longer range of papers, but PLoS was only founded in 2000 so the data isn’t necessarily robust earlier than 2000ish.