Data_607_Week9
Dan Wigodsky
March 27, 2018
New York Times articles about bees
## [1] "Climate Change Is Shrinking Where Bumblebees Range, Research Finds"
## [2] "Gardening"
## [3] "What’s Behind the Collapse of Honeybee Colonies"
## [4] "A Sharp Spike in Honeybee Deaths Deepens a Worrisome Trend"
## [5] "Monsanto, in Bid for Syngenta, Reaches for a Business It Left Behind"
## [6] "A Pesticide Is Harmful to Wild Bees, Study Finds"
## [7] "Research Suggests Pesticide Is Alluring and Harmful to Bees"
## [8] "Pesticides Linked to Honeybee Deaths Pose More Risks, European Group Says"
## [9] "The Head-Scratching Case of the Vanishing Bees"
## [10] "Human Trafficking: The View From Malaysia"
## [11] "Risking Another Silent Spring"
## [12] "Report Says Fewer Bees Perished Over the Winter, but the Reason Is a Mystery"
## [13] "New Insights From Old DNA, Flowing Ants, Insecticides in Question"
## [14] "European Agency Warns of Risk to Humans in Pesticides Tied to Bee Deaths "
## [15] "Accused of Harming Bees, Bayer Researches a Different Culprit"
## [16] "The Year the Monarch Didnt Appear"
## [17] "Bee Survival in Europe"
## [18] "Trouble in the Hive: Researching the Decimation of Honeybee Colonies"
## [19] "Study Finds No Single Cause of Honeybee Deaths"
## [20] "Europe Bans Pesticides Thought Harmful to Bees"
## [21] "Calamity for Our Most Beneficent Insect"
## [22] "Existential Animal News and the Worlds Lightest Solid"
## [23] "Mystery Malady Kills More Bees, Heightening Worry on Farms"
## [24] "No Decision on Bee-Harming Pesticides in Europe"
## [25] "Hoping to Save Bees, Europe to Vote on Pesticide Ban"
## [26] "Toxic Threats to Grassland Birds"
## [27] "On Our Radar: Cuts in German Solar Subsidies"
## [28] "2 Studies Point to Common Pesticide as a Culprit in Declining Bee Colonies"
## [29] "Is a Pesticide Harming All Those Bees?"
## [30] "Bees Vanish, and Scientists Race for Reasons"
“Your task is to choose one of the New York Times APIs, construct an interface in R to read in the JSON data, and transform it to an R dataframe.”
We chose 3 searches in the New York Times article search API. We looked at articles from 2000-2018. Our first term, “colony collapse disorder”, didn’t return many articles. “Bee death” returned 30 results, listed above. The broadest search, “bee” returned 352 articles. We took articles with the word “bee” and created a data frame with information about the headline, author, date and url, along with a short snippet of each article. To limit the number of calls to the API, we saved our results to a file before working with them.
Sys.setenv(NYTIMES_AS_KEY = "???????????????")
bee_data<- as_search(q="bee deaths", begin_date = "2000101", end_date = '20180301',all_results=TRUE)
head(bee_data)
data_for_bees<-unclass(bee_data$data)
web_url<-data_for_bees$web_url
snippet<-data_for_bees$snippet
author_of_article<-data_for_bees$byline.original
headline<-data_for_bees$headline.main
uri<-data_for_bees$uri
publish_date<-data_for_bees$pub_date
write.to.file<-"C:/Users/dawig/Desktop/CUNY/Times_bee_url_2"
write(web_url,file = write.to.file)
write.to.file<-"C:/Users/dawig/Desktop/CUNY/Times_bee_uri_2"
write(uri,file = write.to.file)
write.to.file<-"C:/Users/dawig/Desktop/CUNY/Times_bee_snippet_2"
write(snippet,file = write.to.file)
write.to.file<-"C:/Users/dawig/Desktop/CUNY/Times_bee_author_2"
write(author_of_article,file = write.to.file)
write.to.file<-"C:/Users/dawig/Desktop/CUNY/Times_bee_headline_2"
write(headline,file = write.to.file)
write.to.file<-"C:/Users/dawig/Desktop/CUNY/Times_bee_date_2"
write(publish_date,file = write.to.file)
Colony collapse disorder has emerged as a threat to honeybees and to our ability to grow crops for food. We wanted to find if the interest in bees changed over the period 2000-2018 to see if we could track the development of the story. In order to have enough data to be meaningful, we chose the broadest search. To smooth our data, we created a 5-month moving average.
Now, we produce our data frame with information from the articles.
Bee_df <- data.frame(matrix(NA, nrow=352, ncol=5))
Bee_df[5]<-readLines("C:/Users/dawig/Desktop/CUNY/Times_bee_url_2")
Bee_df[6]<-readLines("C:/Users/dawig/Desktop/CUNY/Times_bee_uri_2")
Bee_df[2]<-readLines("C:/Users/dawig/Desktop/CUNY/Times_bee_snippet_2")
Bee_df[3]<-readLines("C:/Users/dawig/Desktop/CUNY/Times_bee_author_2")
Bee_df[1]<-readLines("C:/Users/dawig/Desktop/CUNY/Times_bee_headline_2")
Bee_df[4]<-readLines("C:/Users/dawig/Desktop/CUNY/Times_bee_date_2")## 'data.frame': 352 obs. of 6 variables:
## $ X1: chr "Robert Delpire, Champion of Photography as Art, Dies at 91" "Pete Turner, Whose Color Photography Could Alter Reality, Dies at 83" "The Mystery of the Dead Bumblebees and the Linden Trees" "California Today: No Easy Answers After Las Vegas" ...
## $ X2: chr "Mr. Delpire, who created his own photographic universe in Paris, didnt just publish photography books; as one "| __truncated__ "With saturated colors, often on global assignments, Mr. Turner created spectacular images, some for the covers "| __truncated__ "Large numbers of bees were turning up sick or dead beneath fragrant trees in botanic garden in London. Two scie"| __truncated__ "Tuesday: Questions for a gun violence expert, the many California victims of the Las Vegas shooting, and an obi"| __truncated__ ...
## $ X3: chr "By RICHARD SANDOMIR" "By RICHARD SANDOMIR" "By JOANNA KLEIN" "By MIKE McPHATE" ...
## $ X4: chr "2017-10-02T22:13:20+0000" "2017-09-22T19:19:49+0000" "2017-09-27T21:07:52+0000" "2017-10-03T12:28:37+0000" ...
## $ X5: chr "https://www.nytimes.com/2017/10/02/obituaries/robert-delpire-champion-of-photography-as-art-dies-at-91.html" "https://www.nytimes.com/2017/09/22/arts/pete-turner-dead-color-photographer.html" "https://www.nytimes.com/2017/09/27/science/dead-bumblebees-trees.html" "https://www.nytimes.com/2017/10/03/us/california-today-no-easy-answers-after-las-vegas.html" ...
## $ V6: chr "nyt://article/9b615ca2-bb94-5951-a7b5-080dd5566230" "nyt://article/1a69857a-5852-5dc5-9717-867c210ba842" "nyt://article/486ec67d-20e7-5faa-8d9d-47c19b68c9c6" "nyt://article/c80b7b8d-4958-5b08-bb78-87af3b43cacd" ...## [1] "What Josephine Del Deo Saved for Us"
## [2] "Joe Esposito Dies at 78; Spent 20 Years Assisting Elvis Presley"
## [3] "Edward Albee, Trenchant Playwright Who Laid Bare Modern Life, Dies at 88"
## [4] "California Today: A See Something, Say Something Prostitution Plan"
## [5] "Aimed at Zika Mosquitoes, Spray Kills Millions of Honeybees"
## [6] "Zika Cases in Puerto Rico Are Skyrocketing"
## [7] "A History of Fireworks Mayhem on the Fourth of July"
## [8] "After Orlando Shooting, Talk Show Hosts Suggest Talk Is Not Enough"
## [9] "It Makes No Sense to Me: Late-Night Hosts React to the Orlando Shooting"
## [10] "Richard Linke, Andy Griffiths Talent Manager, Dies at 98"
## [11] "Margaret Heldt, Hairdresser Who Built the Beehive, Dies at 98"
## [12] "Muhammad Ali Dies at 74: Titan of Boxing and the 20th Century"
## [13] "Review: In Rita Doves Verse, Loss, Despair and Bedrock Pleasures"
## [14] "Tony Barrow, Beatles Publicist Who Coined the Term Fab Four, Dies at 80"
## [15] "Tommy Kono, Weight-Lifting Champion Raised in Internment Camp, Dies at 85"
## [16] "Lured by Early Warm Weather, Scorpions Emerge to Swarm Arizona Homes"
## [17] "George Martin, Redefining Producer Who Guided the Beatles, Dies at 90"
## [18] "Delmer Berg, Last of American Volunteers in Spanish Civil War, Dies at 100"
## [19] "Robert Stigwood, Impresario of Rock, Film and Stage, Is Dead at 81"
## [20] "The Lawyer Who Became DuPonts Worst Nightmare"
## [21] "Ahmad Chalabi, Iraqi Politician Who Pushed for U.S. Invasion, Dies at 71"
## [22] "Paid Notice: Deaths HELLENBRAND, JULIUS "
## [23] "Paid Notice: Deaths HELLENBRAND, JULIUS A"
## [24] "Jan Wallman, Village Cabaret Owner, Dies at 93"
## [25] "Bill Keith, Who Uncovered Banjos Melodic Potential, Dies at 75"
## [26] "A Dangerous Cycle in Food Production"
## [27] "Formula One Has a Stake in Brain and Spine Research"
## [28] "Renovation at Javits Center Alleviates Hazard for Manhattans Birds"
## [29] "Overcoming Love Addiction: One Apple Martini at a Time"
## [30] "Technology to Keep Swimmers Safe"
## [31] "Bumble Bee Foods Will Pay $6 Million in Workers Death"
## [32] "Sailing Beyond the Rule of Law"
## [33] "James Tate, Prolific Pulitzer-Winning Poet, Dies at 71"
## [34] "Eugene Kennedy, a Voice for Change in the Catholic Church, Dies at 86 "
## [35] "Gardening"
## [36] "Paid Notice: Deaths ZINMAN, FLORENCE"
## [37] "Corrections: May 20, 2015"
## [38] "Paid Notice: Deaths ZINMAN, FLORENCE"
## [39] "Paid Notice: Deaths ZINMAN, FLORENCE "
## [40] "A Sharp Spike in Honeybee Deaths Deepens a Worrisome Trend"
## [41] "Your Monday Briefing"
## [42] "Paid Notice: Deaths STARK, MORGAN B"
## [43] "Paid Notice: Deaths STARK, MORGAN "
## [44] "Pesticides Linked to Honeybee Deaths Pose More Risks, European Group Says"
## [45] "Corrections: March 29, 2015"
## [46] "Francis C. Rooney Jr. Dies at 93; Turned Melville Into a Retail Giant"
## [47] "Jon Jerde, Architect of Merging Visions, Dies at 75"
## [48] "What’s On TV Sunday"
## [49] "Paid Notice: Deaths CULLINAN GINIGER, BERNICE \"BEE,\" 88"
## [50] "Paid Notice: Deaths GINIGER, BERNICE CULLINAN"
## [51] "Off Target on Toy-Gun Regulation"
## [52] "Paid Notice: Deaths SPOOR, WILLIAM 1923, 2014"
## [53] "Paid Notice: Deaths SPOOR, WILLIAM "
## [54] "William Spoor Dies at 91; Turned Pillsbury Into a Food Industry Giant"
## [55] "Jimmy Ruffin, Singer of a Memorable Motown Hit, Dies at 78"
## [56] "Paid Notice: Deaths MULA, LILIA"
## [57] "Paid Notice: Deaths MULA, LILIA"
## [58] "Paid Notice: Deaths WONG, JACK 67"
## [59] "New York Today: Cool, Cool Summer "
## [60] "Jon Cavaiani, Medal of Honor Recipient in 1974, Dies at 70"
## [61] "Do California’s Gun Laws Prevent Gun Deaths? "
## [62] "Peter Marler, Graphic Decoder of Birdsong, Dies at 86"
## [63] "Paid Notice: Deaths RAPOPORT, GEORGE"
## [64] "Paid Notice: Deaths RAPOPORT, GEORGE"
## [65] "Risking Another Silent Spring"
## [66] "The Gun Report: May 21, 2014"
## [67] "The Gun Report: May 20, 2014"
## [68] "The Gun Report: May 9, 2014"
## [69] "Corrections: May 7, 2014"
## [70] "The Gun Report: April 25, 2014"
## [71] "Movie Listings for April 25-May 1"
## [72] "Movie Listings for April 18-24"
## [73] "Movie Listings for April 11-17"
## [74] "Hand-Held Treatment for Overdoses Is Approved"
## [75] "New Lead in Honeybee Deaths and Another Hot Year on Earth"
## [76] "Bee Deaths May Stem From Virus, Study Says"
## [77] "Miles Davis: The Chameleon of Cool; An Innovator With Dueling Ambitions"
## [78] "European Agency Warns of Risk to Humans in Pesticides Tied to Bee Deaths "
## [79] "Amid Repressed Hopes, Reasons to Dream"
## [80] "Accused of Harming Bees, Bayer Researches a Different Culprit"
## [81] "Paid Notice: Deaths CALLAGHAN, WILLIAM"
## [82] "Bee Survival in Europe"
## [83] "Paid Notice: Deaths MANOFF, RICHARD K."
Our data frame contains a lot of articles that we are not interested in. In the selection above, we can see that a number of obituaries appear. In 2014, a lot of references were to the movie Maya the Bee. In the end, to have enough references to track them over time, we admitted a lot of unrelated articles. To be more successful in our analysis, we would have to pair it with searches of many more media outlets and then be able to filter our results more appropriately, but still find enough articles to measure. Alternatively, we could have modeled amount of time between articles to find out when they appeared more frequently.
.
Photos: all-free-download.com - tpsdave
Screen Australia