Watching blockbusters through IMDB lens
Team 23
22 October 2016
About IMDB:
Internet Movie Database(IMDb) is an online database of information related to films,television,programs and video games. As of September 2016, IMDb has approximately 3.9 million titles (including episodes) and 7.4 million personalities in its database, as well as 67 million registered users. It is ranked 58 in Alexa Top Global sites ranking.
Sounds interesting! Let us try fetching top 250 movies on IMDb.This was achieved by scarping top 250 movies data from web and storing it in IMDB Top 250 file
$doc
$file
[1] "<buffer>"
$version
[1] "1.0"
$children
$children$a
<a href="/title/tt0111161/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2398042102&pf_rd_r=1YMV00A9QZEYWA9W5W1H&pf_rd_s=center-1&pf_rd_t=15506&pf_rd_i=top&ref_=chttp_tt_1" title="Frank Darabont (dir.), Tim Robbins, Morgan Freeman">The Shawshank Redemption</a>
attr(,"class")
[1] "XMLDocumentContent"
$dtd
$external
NULL
$internal
NULL
attr(,"class")
[1] "DTDList"
attr(,"class")
[1] "XMLDocument" "XMLAbstractDocument"$doc
$file
[1] "<buffer>"
$version
[1] "1.0"
$children
$children$td
<td class="ratingColumn imdbRating">
<strong title="9.2 based on 1,718,456 user ratings">9.2</strong>
</td>
attr(,"class")
[1] "XMLDocumentContent"
$dtd
$external
NULL
$internal
NULL
attr(,"class")
[1] "DTDList"
attr(,"class")
[1] "XMLDocument" "XMLAbstractDocument"Let us see which of the top 250 movies extracted are released between 1996 and 1998.The data has been tailored to fetch the movies released between the time period which falls under top 250 movies category.
[1] "La vita è bella" "Saving Private Ryan" "American History X"
[4] "Mononoke-hime" "L.A. Confidential" "Good Will Hunting" Do we feel something missing here. Oh yes,we can only see the basic data from the set. Why don’t we scrape more information from IMDB.Let us make it more interesting by scraping information about the movie Director, stars, Taglines, Genres, (partial) story line, Box office budget and box office gross.
Data frame having columns with movie name being the first variable
movieName
1 The Shawshank Redemption
2 The Godfather
3 The Godfather: Part II
4 The Dark Knight
5 12 Angry Men
6 Schindler's List
moviecast yearMovie votes
1 Frank Darabont (dir.), Tim Robbins, Morgan Freeman 1994 1718456
2 Francis Ford Coppola (dir.), Marlon Brando, Al Pacino 1972 1174559
3 Francis Ford Coppola (dir.), Al Pacino, Robert De Niro 1974 804922
4 Christopher Nolan (dir.), Christian Bale, Heath Ledger 2008 1704632
5 Sidney Lumet (dir.), Henry Fonda, Lee J. Cobb 1957 457572
6 Steven Spielberg (dir.), Liam Neeson, Ralph Fiennes 1993 880006
rating TagLine
1 9.2 Fear can hold you prisoner. Hope can set you free.
2 9.2 The Godfather is now a movie.
3 9.0
4 8.9 Welcome to a world without rules.
5 8.9 Life Is In Their Hands -- Death Is On Their Minds!
6 8.9 The List Is Life.
StoryLine
1 Chronicles the experiences of a formerly successful banker as a prisoner in the gloomy jailhouse of Shawshank after being found guilty of a crime he did not commit. The film portrays the man's unique way of dealing with his new, torturous life; along the way he befriends a number of fellow prisoners, most notably a wise long-term inmate named Red. Written byJ-S-Golden
2 When the aging head of a famous crime family decides to transfer his position to one of his subalterns, a series of unfortunate events start happening to the family, and a war begins between all the well-known families leading to insolence, deportation, murder and revenge, and ends with the favorable successor being finally chosen. Written byJ. S. Golden
3 The continuing saga of the Corleone crime family tells the story of a young Vito Corleone growing up in Sicily and in 1910s New York; and follows Michael Corleone in the 1950s as he attempts to expand the family business into Las Vegas, Hollywood and Cuba. Written byKeith Loh <loh@sfu.ca>
4 Set within a year after the events of Batman Begins, Batman, Lieutenant James Gordon, and new district attorney Harvey Dent successfully begin to round up the criminals that plague Gotham City until a mysterious and sadistic criminal mastermind known only as the Joker appears in Gotham, creating a new wave of chaos. Batman's struggle against the Joker becomes deeply personal, forcing him to "confront everything he believes" and improve his technology to stop him. A love triangle develops between Bruce Wayne, Dent and Rachel Dawes. Written byLeon Lombardi
5 The defense and the prosecution have rested and the jury is filing into the jury room to decide if a young man is guilty or innocent of murdering his father. What begins as an open-and-shut case of murder soon becomes a detective story that presents a succession of clues creating doubt, and a mini-drama of each of the jurors' prejudices and preconceptions about the trial, the accused, and each other. Based on the play, all of the action takes place on the stage of the jury room. Written bypjk <PETESID@VNET.IBM.COM>
6 Oskar Schindler is a vainglorious and greedy German businessman who becomes an unlikely humanitarian amid the barbaric German Nazi reign when he feels compelled to turn his factory into a refuge for Jews. Based on the true story of Oskar Schindler who managed to save about 1100 Jews from being gassed at the Auschwitz concentration camp, it is a testament to the good in all of us. Written byHarald Mayr <marvin@bike.augusta.de>
Directors
1 Frank Darabont
2 Francis Ford Coppola
3 Francis Ford Coppola
4 Christopher Nolan
5 Sidney Lumet
6 Steven Spielberg
Stars
1 Tim Robbins,Morgan Freeman,Bob Gunton,William Sadler,Clancy Brown,Gil Bellows,Mark Rolston,James Whitmore,Jeffrey DeMunn,Larry Brandenburg,Neil Giuntoli,Brian Libby,David Proval,Joseph Ragno,Jude Ciccolella
2 Marlon Brando,Al Pacino,James Caan,Richard S. Castellano,Robert Duvall,Sterling Hayden,John Marley,Richard Conte,Al Lettieri,Diane Keaton,Abe Vigoda,Talia Shire,Gianni Russo,John Cazale,Rudy Bond
3 Al Pacino,Robert Duvall,Diane Keaton,Robert De Niro,John Cazale,Talia Shire,Lee Strasberg,Michael V. Gazzo,G.D. Spradlin,Richard Bright,Gastone Moschin,Tom Rosqui,Bruno Kirby,Frank Sivero,Francesca De Sapio
4 Christian Bale,Heath Ledger,Aaron Eckhart,Michael Caine,Maggie Gyllenhaal,Gary Oldman,Morgan Freeman,Monique Gabriela Curnen,Ron Dean,Cillian Murphy,Chin Han,Nestor Carbonell,Eric Roberts,Ritchie Coster,Anthony Michael Hall
5 Martin Balsam,John Fiedler,Lee J. Cobb,E.G. Marshall,Jack Klugman,Edward Binns,Jack Warden,Henry Fonda,Joseph Sweeney,Ed Begley,George Voskovec,Robert Webber
6 Liam Neeson,Ben Kingsley,Ralph Fiennes,Caroline Goodall,Jonathan Sagall,Embeth Davidtz,Malgorzata Gebel,Shmuel Levy,Mark Ivanir,Béatrice Macola,Andrzej Seweryn,Friedrich von Thun,Krzysztof Luft,Harry Nehring,Norbert Weisser
Genre boxofficeGrossVal boxofficebudget
1 Crime , Drama $28,341,469 $25,000,000
2 Crime , Drama $134,821,952 $6,000,000
3 Crime , Drama $57,300,000 $13,000,000
4 Action , Crime , Drama , Thriller $533,316,061 $185,000,000
5 Crime , Drama $350,000
6 Biography , Drama , History $96,067,179 $22,000,000
movies.movie.link
1 http://www.imdb.com/title/tt0111161/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2398042102&pf_rd_r=1R75XTHNP53HGV476GQK&pf_rd_s=center-1&pf_rd_t=15506&pf_rd_i=top&ref_=chttp_tt_1
2 http://www.imdb.com/title/tt0068646/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2398042102&pf_rd_r=1R75XTHNP53HGV476GQK&pf_rd_s=center-1&pf_rd_t=15506&pf_rd_i=top&ref_=chttp_tt_2
3 http://www.imdb.com/title/tt0071562/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2398042102&pf_rd_r=1R75XTHNP53HGV476GQK&pf_rd_s=center-1&pf_rd_t=15506&pf_rd_i=top&ref_=chttp_tt_3
4 http://www.imdb.com/title/tt0468569/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2398042102&pf_rd_r=1R75XTHNP53HGV476GQK&pf_rd_s=center-1&pf_rd_t=15506&pf_rd_i=top&ref_=chttp_tt_4
5 http://www.imdb.com/title/tt0050083/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2398042102&pf_rd_r=1R75XTHNP53HGV476GQK&pf_rd_s=center-1&pf_rd_t=15506&pf_rd_i=top&ref_=chttp_tt_5
6 http://www.imdb.com/title/tt0108052/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2398042102&pf_rd_r=1R75XTHNP53HGV476GQK&pf_rd_s=center-1&pf_rd_t=15506&pf_rd_i=top&ref_=chttp_tt_6Oh I feel nostalgic now looking at the movies list. Use to love watching them. Let us try analyzing the data. Hope to get some interesting insights.
Table movie-count versus Genres :
Category x
1: Action 36
2: Adventure 62
3: Animation 20
4: Biography 26
5: Comedy 42
6: Crime 56Let us try to analyse the Rating pattern of customer for top 250 movies 
Looking at above graph major number of users in IMDb have given minimum rating of 8 to all the top 250 movies.Looks like IMDb users do appreciate good movies. We do observe some elite group of users rating more than 9 for the movies.
Why don’t we see when were the top movies released. I do remember some classics released in and around 2000
Majority of movies in the list were made in 2000 and later on. However, the line graph shoots up at two places during decades-1951-1961 and 1991-2001. It is because of some of the classic movies released at that time and making the way to the top 250. Examples- The Shawshank Redemption,Forrest Gump,Matrix etc.
Let us make it more interesting by analyzing Genre data
Movies having genre as ‘Drama’ did top the IMDB list. Interesting result to watch out for. Next to drama are Thriller and Action movies.
Let us deep dive further
Interested in movies with highest Gross in Top 250 movies from IMDb?
Looks like some of the non English movies also entered the list. Surprisingly the highest gross movies are Biopic. Now we know what Genre of movies we can make in future to turn billionaires!!!