In this article we will look basic functionalities of API’s and will collect data from API’s with R. API stands for application programming interface which provides capabilities to interact with databases. Let’s say we are interested in getting weather data. There are multiple API’s like Weather Underground, openWeatherMap etc, which provide weather data. I will use openWeatherMap.com API.I will also demonstrate another API openmoviedatabase.
Step 1- Signup @ http://home.openweathermap.org/users/sign_up and get the api key. Refer documentation for accessing all features of API @ http://openweathermap.org/api
Step 2 - Load required packages. We will need httr, RCurl and jsonlite package in R Workspace. (If packages are not installed you can install them with install.packages(“packagename”))
library("httr")
library("RCurl")
library("jsonlite")Step 3 - We are interested in getting weather forecast for Hyderabad. As mentioned in the documentation, query URL will be written as base URL i.e. http://api.openweathermap.org/data/2.5/forecast/daily/? + id=1269843(for Hyderabad) + &cnt=15(Number of days) +&APPID=34ea8871014f1307420017c55113b855(API key). So URL for querying 15days forecast from openweathermap API is - http://api.openweathermap.org/data/2.5/forecast/daily/?id=1269843&cnt=15&APPID=34ea8871014f1307420017c55113b855
Now we can use getURL function for querying this URL and get the data in JSON format. Then with fromJSON we can convert this JSON object to a R list object.
url1 = "http://api.openweathermap.org/data/2.5/forecast/daily/?id=1269843&cnt=15&APPID=34ea8871014f1307420017c55113b855" #forcast for Hyderabad id=1269843
forcast = getURL(url1) #,curl = curl)
forcast1 <- jsonlite::fromJSON(forcast)Step 4 - In the JSON response, date is in unix format, so we will use as.POSIXct to convert it to readable R date. We then create a data frame of required columns and write this data frame to a csv file.
forcast1$list$dt = as.POSIXct( # Date-time Conversion Function
forcast1$list$dt, # date object to be converted
origin = '1970-01-01',
tz='GMT') # tz = timezone
forcast2 = forcast1$list
weather = unlist(
sapply(
sapply(forcast2$weather, "[", i = 3),
"[", i = 1));
forcast3 = data.frame(forcast2$dt,
forcast2$temp,
forcast2$pressure,
forcast2$humidity,
forcast2$speed,
forcast2$deg,
forcast2$clouds,
weather)
write.csv(forcast3,"Hyderabad_weather_forcast.csv",row.names = F) # save in csv form
head(forcast3)## forcast2.dt day min max night eve morn
## 1 2016-02-14 07:00:00 301.60 299.21 301.60 299.21 301.60 301.60
## 2 2016-02-15 07:00:00 302.94 292.05 304.30 294.41 302.47 292.05
## 3 2016-02-16 07:00:00 302.85 289.85 304.71 295.33 303.09 289.85
## 4 2016-02-17 07:00:00 301.42 289.98 303.78 293.00 301.95 289.98
## 5 2016-02-18 07:00:00 301.94 289.01 304.11 292.29 302.30 289.01
## 6 2016-02-19 07:00:00 304.75 289.72 304.83 296.01 304.83 289.72
## forcast2.pressure forcast2.humidity forcast2.speed forcast2.deg
## 1 960.09 40 4.57 131
## 2 962.15 28 5.20 146
## 3 962.45 32 5.16 138
## 4 963.16 44 6.55 129
## 5 963.34 42 4.31 134
## 6 973.32 0 3.27 100
## forcast2.clouds weather
## 1 88 overcast clouds
## 2 0 sky is clear
## 3 0 sky is clear
## 4 8 sky is clear
## 5 36 scattered clouds
## 6 22 sky is clearCSV file will be stored in your working directory. To check your working directory you can type
getwd()## [1] "C:/Users/Aashish/Desktop/New folder/session 3"The OMDb API is a free web service to obtain movie information. It supports two types of queries by serach and move imdb id. For this API no key is required. Let’s say we want to search about movie “Harry Potter”. First query is written for searching all movies which are related to Harry Potter whereas Second query is for specific movie with imdb id =tt1201607 (one of the output from first query)
library("httr")
library("RCurl")
library("jsonlite")a = getURL("http://www.omdbapi.com/?s=harry%20%potter&r=json") # search for harry potter
b = getURL("http://www.omdbapi.com/?i=tt1201607&r=json&tomatoes=true") # serach for imdb id tt1201607 ("Harry Potter and the Deathly Hallows: Part 2")
x <- jsonlite::fromJSON(a)
y <- jsonlite::fromJSON(b)
x## $Search
## Title Year imdbID Type
## 1 Harry Potter and the Deathly Hallows: Part 2 2011 tt1201607 movie
## 2 Harry Potter and the Sorcerer's Stone 2001 tt0241527 movie
## 3 Harry Potter and the Chamber of Secrets 2002 tt0295297 movie
## 4 Harry Potter and the Prisoner of Azkaban 2004 tt0304141 movie
## 5 Harry Potter and the Goblet of Fire 2005 tt0330373 movie
## 6 Harry Potter and the Order of the Phoenix 2007 tt0373889 movie
## 7 Harry Potter and the Deathly Hallows: Part 1 2010 tt0926084 movie
## 8 Harry Potter and the Half-Blood Prince 2009 tt0417741 movie
## 9 Harry Potter and the Chamber of Secrets 2002 tt0304140 game
## 10 Harry Potter and the Order of the Phoenix 2007 tt0944836 game
## Poster
## 1 http://ia.media-imdb.com/images/M/MV5BMTY2MTk3MDQ1N15BMl5BanBnXkFtZTcwMzI4NzA2NQ@@._V1_SX300.jpg
## 2 http://ia.media-imdb.com/images/M/MV5BMTYwNTM5NDkzNV5BMl5BanBnXkFtZTYwODQ4MzY5._V1_SX300.jpg
## 3 http://ia.media-imdb.com/images/M/MV5BMTcxODgwMDkxNV5BMl5BanBnXkFtZTYwMDk2MDg3._V1_SX300.jpg
## 4 http://ia.media-imdb.com/images/M/MV5BMTY4NTIwODg0N15BMl5BanBnXkFtZTcwOTc0MjEzMw@@._V1_SX300.jpg
## 5 http://ia.media-imdb.com/images/M/MV5BMTI1NDMyMjExOF5BMl5BanBnXkFtZTcwOTc4MjQzMQ@@._V1_SX300.jpg
## 6 http://ia.media-imdb.com/images/M/MV5BMTM0NTczMTUzOV5BMl5BanBnXkFtZTYwMzIxNTg3._V1_SX300.jpg
## 7 http://ia.media-imdb.com/images/M/MV5BMTQ2OTE1Mjk0N15BMl5BanBnXkFtZTcwODE3MDAwNA@@._V1_SX300.jpg
## 8 http://ia.media-imdb.com/images/M/MV5BNzU3NDg4NTAyNV5BMl5BanBnXkFtZTcwOTg2ODg1Mg@@._V1_SX300.jpg
## 9 http://ia.media-imdb.com/images/M/MV5BNTM4NzQ2NjA4NV5BMl5BanBnXkFtZTgwODAwMjE4MDE@._V1_SX300.jpg
## 10 N/A
##
## $totalResults
## [1] "74"
##
## $Response
## [1] "True"y## $Title
## [1] "Harry Potter and the Deathly Hallows: Part 2"
##
## $Year
## [1] "2011"
##
## $Rated
## [1] "PG-13"
##
## $Released
## [1] "15 Jul 2011"
##
## $Runtime
## [1] "130 min"
##
## $Genre
## [1] "Adventure, Drama, Fantasy"
##
## $Director
## [1] "David Yates"
##
## $Writer
## [1] "Steve Kloves (screenplay), J.K. Rowling (novel)"
##
## $Actors
## [1] "Ralph Fiennes, Michael Gambon, Alan Rickman, Daniel Radcliffe"
##
## $Plot
## [1] "Harry, Ron and Hermione search for Voldemort's remaining Horcruxes in their effort to destroy the Dark Lord as the final battle rages on at Hogwarts."
##
## $Language
## [1] "English"
##
## $Country
## [1] "USA, UK"
##
## $Awards
## [1] "Nominated for 3 Oscars. Another 44 wins & 81 nominations."
##
## $Poster
## [1] "http://ia.media-imdb.com/images/M/MV5BMTY2MTk3MDQ1N15BMl5BanBnXkFtZTcwMzI4NzA2NQ@@._V1_SX300.jpg"
##
## $Metascore
## [1] "87"
##
## $imdbRating
## [1] "8.1"
##
## $imdbVotes
## [1] "506,314"
##
## $imdbID
## [1] "tt1201607"
##
## $Type
## [1] "movie"
##
## $tomatoMeter
## [1] "96"
##
## $tomatoImage
## [1] "certified"
##
## $tomatoRating
## [1] "8.3"
##
## $tomatoReviews
## [1] "306"
##
## $tomatoFresh
## [1] "293"
##
## $tomatoRotten
## [1] "13"
##
## $tomatoConsensus
## [1] "Thrilling, powerfully acted, and visually dazzling, Deathly Hallows Part II brings the Harry Potter franchise to a satisfying -- and suitably magical -- conclusion."
##
## $tomatoUserMeter
## [1] "89"
##
## $tomatoUserRating
## [1] "4.3"
##
## $tomatoUserReviews
## [1] "386281"
##
## $tomatoURL
## [1] "http://www.rottentomatoes.com/m/harry_potter_and_the_deathly_hallows_part_2_2011/"
##
## $DVD
## [1] "11 Nov 2011"
##
## $BoxOffice
## [1] "$381.0M"
##
## $Production
## [1] "Warner Bros. Pictures"
##
## $Website
## [1] "http://www.HarryPotter.com/"
##
## $Response
## [1] "True"BTW IMDB provides data in txt file of its database ftp://ftp.fu-berlin.de/pub/misc/movies/database/