You mention scraping, but did you actually scrape these? I just discovered that IMDB publishes textual data dumps. They appear to be pretty complete. Terms of use are non-commercial, but I'd love to see more analyses like this!
The data files have been available for years. The problem is they are poorly structured and also incomplete. It's impossible to create links sometimes between genre data and movie data as the genre data is linked to a title, and not an IMDB (unique) ID.
The conclusion that runtime correlates with rating is not quite clear to me. It looks to me that the scatter plot gets just more sparse as runtime increases and the rating distribution remains more or less the same.
Somehow the number of movies exploded last(?) year.
If you have a movie that was cut down to 80 minutes or shorter, it's sometimes because they just didn't have anything worth while to put in the final print. Maybe it's because everyone working on the film just wanted it to be over as quickly as possible, so they filmed less, kept less for the final, they were phoning it in to satisfy the contract.
If a long movie is a real stinker, not just like a five or six, but a 2/10... it's costing tons of money. Someone's going to know and walk off the project or the producers are going to pull in emergency editors... you're not going to distribute a three hour film reel of pure trash.
It is true that as runtime increases, the number of movies decreases. However, if you look at the mainstream runtimes(>80 and <100), generally they get quite a variety of ratings for a given runtime if not uniform. On the other hand, the movies that have higher runtimes generally get higher number of votes.
You could also observe the same behavior from rating vs #votes graph; as # votes increases, the number of movies decreases. However, rating and # votes correlate quite strongly.
I wonder why long movies are consistently rated higher?
My guess would be for two reasons. The first being audience selection. Long films are (IMO) more likely to attract, and retain for the entire length, an audience with a prior interest in the film, and therefore more likely to be an audience who know beforehand if the movie is generally good or not.
Secondly, psychology will come into play. The more time an audience invests in a film, the more likely they are to seek a positive reward for their time so they don't feel like they have got a bad deal[1]. Thus, they're more likely to rate the film higher than it perhaps otherwise would be. I also believe this holds true for 'art house' films that are difficult to follow and perhaps less enjoyable than a more mainstream film. Audiences will rate them higher to reassure themselves that they haven't just wasted 2 hours watching something boring that they don't understand.
I did a quick and dirty project[1] involving IMDB and Neo4j when I had some time off between jobs over the holidays. I used screen scraping to get the list of IMDB ids for the AFI top 100 movies and then made calls to MyMovieAPI to pull down IMDB data about each AFI film. I wasn't aware of the imdb.com/interfaces at that point, but it wasn't really my goal to do the "best" possible implementation since it was just a learning experience. For those interested, there's a simple overview of the project[2] that shows what (i thought) were interesting questions about the data: for instance, which actors, if any have appeared in 2 or more of the top 25 AFI films?
After looking at imdb.com/interfaces, I'm not sure that it has what I'm looking for. My plan on expanding this project at some point in the future is to start with data from Freebase[3] since it's already presented in a normalized format and then filling in missing details via IMDB as necessary.
My ultimate goal is to generalize the N-degrees-to-Bacon trivia question to work with any two actors, but that requires getting a lot more data to work with.
Funny that overrated movies is dominated by Twilight. I suspect boy-friends who were forced to watch them together with their girl-friends are responsible.
Buğra talks about looking at directors and actors next.
I'd really like to see whether directors or writers have a bigger impact on quality of films. Like a smallish number of critics, including Pauline Kael, I'm deeply suspicious of the auteur theory that everyone kind of unquestioningly accepts.
“A filmgoer seeking out pictures written by, say, Eric Roth or Charlie Kaufman won’t always see a masterpiece, but he’ll see fewer clunkers than he would following even a brilliant director like John Boorman, or an intelligent actor like Jeff Goldblum. It’s all a matter of betting on the fastest horse, instead of the most highly touted or the prettiest.” - David Kipen
> Therefore, it may be safe to assume this ranking more or less holds true for non-top 250 movies as well.
This may not hold true. A while ago I was looking into it and they seemed to use more complex weighed average without exact details (possibly using internal user scoring). This may affect the final rating in many ways. More detailed analysis here: http://www.quora.com/Movies/What-algorithm-does-IMDB-use-for...
Interesting stuff, although it would be nice to see more analysis and less tables/charts. Some regression lines would also be good, and help in interpreting correlations.
I was wondering how your post got 3 million facebook shares, then I realized that you left in the default data-href attribute for the facebook docs. You might want to change that.
I'd love if someone (who isn't as lazy as me) could figure out a sophisticated way to show the actual good movies from a year, rather than the popularly good ones. Sentiment analysis? Trend recognition? I don't know, but, I feel like Imdb and Rotten Tomatoes are now effectively useless for new movie reviews.
I am having two problems with your site. In FF the data isn't centered. In chrome and FF I don't see anything in the preformatted code block. Newest version of FF and Chrome on Win 8.1. FF is on the left, Chrome on the right.
I am also very interested in the correlation of rating vs director. However, I do not have the budget information for the movies. It would be great if I could find budget information and combine them with the data I have. I have not though that, really good suggestion. Thanks!
Gangs of Wasseypur was released in two parts as two separate movies and IMDB has added the runtime of both the movies. However both movies were equally good :)
I'd like to see more breakdown by release year, e.g. # of movies in each category by release year.
I think it would be interesting to look at those stats next to economic stats, etc.
I'd also like to see a more granular breakdown of attributes of each movie (movies relating to technology, movies with a workers' union being a strong component of the film, race relations, international relations, etc.) and the # of each of those per year, but that would be much more work.
http://www.imdb.com/interfaces