What are the most overrated films?

“Overrated” and “underrated” are slippery terms to try to quantify. An interesting way of looking at this, I thought, would be to compare the reviews of film critics with those of Joe Public, reasoning that a film which is roundly-lauded by the Hollywood press but proved disappointing for the real audience would be “overrated” and vice versa.

To get some data for this I turned to the most prominent review aggregator: Rotten Tomatoes. All this analysis was done in the R programming language, and full code to reproduce it will be attached at the end.

Rotten Tomatoes API

This API is nicely documented, easy to access and permissive with rate limits, as well as being cripplingly restrictive in what data is presents. Want a list of all films in the database? Nope. Most reviewed? Top rated? Highest box-office takings? Nope.

The related forum is full of what seem like simple requests that should be available through the API but aren’t: top 100 lists? Search using mulitple IDs at once? Get audience reviews? All are unanswered or not currently implemented.

So the starting point (a big list of films) is actually kinda hard to get at. The Rube Golbergian method I eventually used was this:

  1. Get the “Top Rentals” list of movie details (max: 50)
  2. Search each one for “Similar films” (max: 5)
  3. Get the unique film IDs from step 2 and iterate

(N.B. This wasn’t my idea but one from a post in the API forums, unfortunately didn’t save the link.)

In theory this grows your set of films at a reasonable pace, but in reality the number of unique films being returned was significantly lower (shown below). I guess this was due to pulling in “walled gardens” to my dataset, e.g. if a Harry Potter film was hit, each further round would pull in the 5 other films as most similar.

Films returned

Results

Here’s an overview of the critic and audience scores I collected through the Rotten Tomatoes API, with some outliers labelled.

Most over- and underrated films

On the whole it should be noted that critics and audience agree most of the time, as shown by the Pearson correlation coefficient between the two scores (0.71 across >1200 films).

Click for interactive version.

Update:

I’ve put together an interactive version of the same plot here using the rCharts R package. It’ll show film title and review scores when you hover over a point so you know what you’re looking at. Also I’ve more than doubled the size of the film dataset by repeating the above method for a couple more iterations — take a look!

Most underrated films

Using our earlier definition it’s easy to build a table of those films where the audience ending up really liking a film that was panned by critics.

Scores are shown out of 100 for both aggregated critics and members of Rotten Tomatoes.

Scores are shown out of 100 for both aggregated critics and members of Rotten Tomatoes.

Somewhat surprisingly, the top of the table is Facing the Giants (2006), an evangelical Christian film. I guess non-Christians might have stayed away, and presumably it struck a chord within its target demographic — but after watching the trailer, I’d probably agree with the critics on this one.

This showed that some weighting of the difference might be needed, at the very least weighting by number of reviews, but the Rotten Tomatoes API doesn’t provide that data.

In addition the Rotten Tomatoes page for the film, shows a “want to see” percentage, rather than an audience score. This came up a few times and I’ve seen no explanation for it, presumably “want to see” rating is for unreleased films, but the API returns a separate (and undisclosed?) audience score for these films also.

Above shows a "want to see" rating, different to the "liked it" rating returned by the API and shown below

Above shows a “want to see” rating, different to the “liked it” rating returned by the API and shown below. Note: these screenshots from RottenTomatoes.com are not CC licensed and is shown here under a claim of Fair Use, reproduced for comment/criticism.

Looking over the rest of the table, it seems the public is more fond of gross-out or slapstick comedies (such as Diary of a Mad Black Woman (2005), Grandma’s boy (2006)) than the critics. Again, not films I’d jump to defend as underrated. Bad Boys II however…

Most overrated films

Here we’re looking at those films which the critics loved, but paying audiences were then less enthused.

As before, scores are out of 100 and they're ranked by difference between audience and critic scores.

As before, scores are out of 100 and they’re ranked by difference between audience and critic scores.

Strangely the top 15 (by difference) contains both the original 2001 Spy Kids and the sequel Spy Kids 2: The Island of Lost Dreams (2002). What did critics see in these films that the public didn’t? A possibility is bias in the audience reviews collected, the target audience is young children for these films and they probably are underrepresented amongst Rotten Tomatoes reviewers. Maybe there’s even an enrichment for disgruntled parent chaperones.

Thankfully, though, in this table there’s the type of film we might more associate with being “overrated” by critics. Momma’s Man (2008) is an indie drama debuted at the 26th Torino Film Festival. Essential Killing is a 2010 drama and political thriller from Polish director and screenwriter Jerzy Skolimowski. 

There’s also a smattering of Rom-Coms (Friends with Money (2006), Splash (1984)) — if the API returned genre information it would be interesting to look for overall trends but, alas. Additional interesting variables to consider might be budget, the lead, reviews of producer’s previous films… There’s a lot of scope for interesting analysis here but it’s currently just not possible with the Rotten Tomatoes API.

 Caveats / Extensions

The full code will be posted below, so if you want to do a better job with this analysis, please do so and send me a link! :)

  • Difference is too simple a metric. A better measure might be weighted by (e.g.) critic ranking. A film critics give 95% but audiences 75% might be more interesting than the same points difference between a 60/40 rated film.
  • There’s something akin to a “founder effect” of my initial chosen films that makes it had to diversify the dataset, especially to films from previous decades and classics.
  • The Rotten Tomatoes API provides an IMDB id for cross-referencing, maybe that’s a path to getting more data and building a better film list.
Full code to reproduce analysis

Note: If you’re viewing this on r-bloggers, you may need to visit the Benomics version to see the attached gist.

 

About these ads

104 Comments

Filed under R, Unrelated

104 responses to “What are the most overrated films?

  1. I saw King Kong and was disappointed. There was no surprises. He dies at the end. I hope I did not spoil it for you. In my version Kong lives and becomes the mayor of New York and does urban renewal.

  2. I though the Artist was so OVERATED

  3. That’s really cool that you crunched all that data. I’m not all that big into movies, but this was a really interesting analysis.

  4. Interestingly I agree with most of the underrated films (the ones I have seen anyway), the overrated I guess I agree with most (of the ones I have seen) of them too.
    Interesting analysis :)

  5. All these overrated-underrated movies seem to concern movies directed at audiences from Grade’s 4-10 lol. Nothing wrong with that; it’s just an interesting observation.

    (I was hoping to see Citizen Kane on the most overrated list, but that’s just me lol.)

  6. aqilaqamar

    I found most of the differences in rotten tomatoes not in the ratings but the reviews because the rotten or fresh metre is not that grand a scale but individual statistics are.

  7. Personally, I think the most overrated film of all time is the Lion King. Lions don’t speak English, they roar. Audiences completely overlooked that glaring distortion of the facts.

  8. I thought/think that this is an interesting article. Then I read the titles of the films – both underated and overrated – and realised that apart from ONE I had never heard of any of them. So what do I know? nothing.

  9. harrietfisk

    I’m sorry but “The Borrowers” should not be in the overrated section!!

  10. Evaluative criticism has become this? Numbers? Fuckin A!

  11. Paul Nash

    I don’t think there is any point trying to use mathematics to determine the value of a film.

    I haven’t heard of or seen any of the films in the ‘underrated’ list. Of the ‘overrated’ films, I have seen ‘About a Boy’ and ‘Babe’, and thought they were superb.

    In the end, you like the films you like and you don’t like the ones you don’t. It’s all a matter of taste.

  12. You may need to control for country as well. Babe and Sirens were both fairly large hits here in Australia, but I can see a lot of overseas audiences wouldn’t get many of the in-jokes. Screen Australia has the stats on the there site
    http://www.screenaustralia.gov.au/research/statistics/mrboxaust.aspx .

  13. I love this! You’ve ‘proven’ something that’s been nagging me for a while. And I always thought – how, how is it possible that Spy Kids is getting so much positive press!? Now I know for sure that my nagging instinct was spot on.

  14. I don’t worry about overrated films as much as films overhyped by people around me. When people start talking about a film (or book) too much, I find it a bit of a turnoff.

  15. I must admit, I have not seen most of the films listed here, but some overrated ones that come to mind are Forrest Gump. I damned near walked out on that. I found it embarasing and demeaning. Another one that I just didn’t get was Lost in Translation. More recently, 12 Years a Slave left me cold. There are more, but I won’t burden you with them now.

  16. So “Spy Kids” movies are totally weak and Tyler Perry in drag should get more respect. Got it!

  17. Very interesting that two Tyler Perry movies are among those under rated! Was looking to see the Matrix trilogy in the over rated section….

  18. I’m glad to know Empire Records made the list! Such an unheralded, or under rated, piece of gold! Never knew about any of these features or stats. Super interesting piece.

  19. Diary of a Mad Black Woman is actually one of the Tyler Perry movies I didn’t mind watching. Empire Records is wonderful as well. A lot of people judge movies without actually watching them. Great post.

  20. A person, at a time long ago could ‘see’ plainly the public review simply by watching the crowd exiting the theater. Now, we see a collection of stoney faces. I still ak them, “how was it?” I’m often treated like a leper for the asking.

  21. Thank you, you have just confirmed something I have suspected for years: If certain critics slated a film, I always tried to get to see it because I knew I would enjoy it, now I have the proof. LOL :)

  22. I think one additional piece of data would be the factor of the difference. For instance, I thought that The Covenant (factor of 19.67) was more underrated than Facing the Giants (factor of 5.61). Likewise Stuart Little 2 and Momma’s Man were both overrated by 41 points. But Stuart Little 2 had a critics rating of 81 and Momma’s Man had a critic’s rating of 91. The 41 point difference was more significant for Momma’s Man (factor of 2.21) than Stuart Little 2 (1.97). I’m sure I’m using incorrect mathematical terminology, but my point is that it would be interesting to take into consideration how high or low a movie’s critic rating was. In my opinion, going from a 90 to a 60 is a bigger deal than going from a 70 to a 40. Similarly, going from a 15 to a 70 is a bigger deal than going from 30 to an 85.

  23. This is pretty magnificent. I applaud you for doing the research and creating something tangible to work with. I wish this type of research was available for every movie released, as it would help me categorize my own reviews and criticisms.

  24. I’d like to add one to your list: “The Long Green Mile!” Great post :)

  25. sheldon bateman

    exquisite work. hope you don’t mind but i’ve added your blog to mine under ‘finds’. i don’t get much traffic but your work here fits into what i’ve been preaching for years. again, very nicely done!

  26. Pingback: 科学によると、最も多くのものは古今の映画を過大評価しました | 可愛い動物ブログ!

  27. Beware of using as a data point the reaction of both critics and audiences when a film is released. Over time certain films that tanked at the box office have been “rehabilitated” into classics, Citizen Kane and Blade Runner both come to mind, and it works the other way too. You can’t even see Disney’s “Song of the South” anymore.

  28. IMHO Tarantino films are WAAAAAAAAY overrated. waits for backlash His “brilliant” dialogues… really? sigh.

  29. Caz

    That is a very interesting way to look at it, comparing the two ways to look at it. But I guess at the end of the day personal tastes and preference come into it when watching and then deciding if you like a film or not!

  30. research and statistics are not always biased, valid and reliable….

  31. Pingback: Is your favorite movie among the top five most underrated? This list presents some of the best unheralded films with strong messages.

  32. Grandma’s Boy and Super Troopers.. classically underrated!

  33. Pingback: cigarettes sale - buy online smokes

Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s