A better way to rate films

Rating films this way may do a good job of the "so bad it's good" camp (Snakes on a Plane), but will do an incredible disservice to movies like Black Swan.

That's a bit harsh, so let me qualify it. I actually rather agree with the division inherent in this and think it will resound with people, but I honestly believe that this is yet another step towards turning our society into idiocracy. It's the slow legitimization of appreciation for content that we acknowledge as garbage but consume anyway. I firmly believe that there is inherent social value in this garbage content remaining quasi-taboo - a social pressure not to like such things. Sure, most people will watch some anyway, or play the latest mindless shooter, or what-have-you, but so long as it's understood that most people will mock you for publicly liking this, the norm does not drift towards mindlessness - you still need to consume some legitimately good content, even if only to have something to talk about.

I don't mean to attack the site; I think it's a great way to slice into the recommendations game, and I do think a lot of people will get value out of it, and it seems like it may well succeed. I simply am afraid it's a net negative for our society in the long run (then again, of course, not launching is hardly going to reverse such a trend.)

Edit: Make that symptomatic of a net negative. I've thought some more and this is only giving an online portal to express the shift I talk about above, which has already been happening aplenty.

Films, music, books, etc. are not about some abstract universal gradation of quality that a large enough, good enough statistical sampling of viewers will reveal with increasing precision. No, they are art. Enjoyment and appreciation of art is a matter of personal taste, which is not, will not, and cannot be universal for everyone.

That's why up/down votes are decent enough ratings systems despite their crudeness, all they do is tell people whether or not taking the effort to find out if a piece of art suits your personal taste is likely to be worthwhile.

Ultimately what ratings are trying to get at is recommendations. The simplest idea being to measure the average opinion of everyone and present the highest rated works as recommendations. However, more sophisticated systems which try to find people with similar taste and recommend to you things that they seem to like but you haven't been exposed to are likely to be more successful, though significantly more complex to implement. It seems unlikely that even that represents the limit of improvement for the problem, but nobody's done much better yet.

I do like the idea of production value vs replay value, I remember when I saw Passion of the Christ I told a friend it was an amazing movie i never want to see again.

I would say a better metric would be "How much did you enjoy this film?" vs "How good is this movie?"

I've been grappling with this problem since subscribing to Netflix, wondering how I should rate a movie I personally did not enjoy that I also felt was incredibly well made. For example, "A Clockwork Orange" is an incredible movie by all means and I would give it 97 stars out of a hundred.... but I hated every single second of it. It's gruesome, sadistic, dark, miserable, and disgusting. But it was undeniably well made.

Rewatchability sucks in the aspect because many movies can't be rewatched, while still being both splendidly produced and enjoyable. Let me make this clearer with examples: "The Sixth Sense," "Rear Window," "The Accused," "The Diving Bell and the Butterfly," "12 Angry Men," "Apollo 13..." The list goes on and on. There are great movies that just don't rank so highly because they're SUSPENSEFUL and that suspense doesn't translate will into rewatches. Rewatching suspenseful movies will make them seem boring and long-drawn.

Rewatching feel-good movies ("Seven Pounds," "How To Train Your Dragon," "Blind Side," etc.) however, is just as good the second time if not better.

tl;dr the rewatchability index will be heavily skewed towards feel-good favorites and epics, and away from dark, gritty, mysterious, and suspenseful. Not a good index.

I have never considered the rewatchability criteria as a potential factor in rating a movie, but must admit that is has appeal. But thinking about it further, it is a very personal metric so would not be very useful in determining the overall watchability of a movie. I.e. if you consider a movie rewatchable it might not correlate with me considering it to be watchable.

I would love to be able to see a plot of individuals and how their ratings cluster (some interesting visual way of adding and removing individuals would be cool) because I would guess (based on our own data) that these plots would not be uniform. And if they are not, then an individual's movie recommendation, based on rewatchability, would have less value to me.

I understand that people overwhelmingly tend towards the extremes of rating scales. But who cares? There's so much noise anyway.

Here's what I want: (1) include (all) films and television series and (2) include top 1000 lists for as many niche categories as possible (foreign films about relationships, teen comedy, and so on).

I don't even care if it is personalized, because the personalization is always horrible. (I've tried Netflix, Jinni and others and they're all terrible.) Just having the average for the movie is actually amazingly accurate on many measures (within 10--20%).

Is there something like this? The closest I've gotten so far is finding a good IMDB user list or two.

One might also watch a very simple movie.. nothing extraordinary that you wouldn't necessarily watch again; that doesn't mean it's not a good movie to watch for the first time.

Personally, I think the best factor to "guess" if I would watch a movie would be to anonymously analyse a huge set of data and find people who like the same stuff as I do. So, he likes x,y,z (as I do) and hate a,b,c as I do, good.. we mostly like the same stuff. Now, he really enjoyed K; I'd probably also like it.

Just because the data better fits your desired skews, doesn't mean it gets at what the users actually want.

I think a major issue at hand is that people often rate things as they expect a critic to rate them. If you would give Transformers 2 a 5-star rewatchability rating, you should be giving it at least 3 or 4 stars for quality, because obviously you enjoyed it very much. Instead, many people pretend to be film critics, whose jobs are very different, and assign an 'objective rating'. A film critic can not give personal opinions because he's supposed to speak for the masses--to appeal to some higher taste he aspires to have and that he hopes society would have. If you loved Transformers, rate it 5 stars--period. The rating isn't meant to be read by others, it's not meant to appeal to an idealized world, it's just how you felt about the thing. Convince your users to rate intelligently with that mindset and you'll start to see good data.

All that said, more data is better than less, and if you could convince your users to double their average rating-time investment and give you ratings for both, awesome. I'm skeptical that users would bother to rate two metrics for everything they've seen...but I think it's certainly possible they would. Be interested to see.

Why would you limit your pool of advice-givers to your friends?

They are good for service advice where you need someone you can trust to give you the whole and honest story.

In movies it's beyond that, it's much more important to find people with the closest match to you in movie taste as possible. The more people the closer taste match you will get (goes without saying).

I also don't see the point of giving "quality" scores. I really don't care if a stranger thinks some movie was of high or low quality. I'd much rather know if my taste-twin loved it or hated it. I for example thought Lord of the Rings was super well done and yet super boring. But this opinion of mine only matters to someone who resembles me in movie taste, who hates and loves the same quirky things as I do about movies. And of the two opinions i mentioned (well done, boring) its really only the latter that matters to my twin, he/she should pay most attention to that when deciding if to go and see the Lord, not if I thought the movie was of high or low quality.

But having said that, I like "rewatchability" as a concept to study further. I too would rip of my nails before seeing Black Swan again but I still thought it was a masterpiece. However, I have seen Requim for a Dream (Darren's Arrenovsky's previous movie) again and again although it was also very emotionally intense like the Swan. Just in a different way, less depressive, but killer strong in an exhaustive yet positive way. And then.. yet and still.. I would recommend both movies to everyone I meet. I think both are a must see even though one of them I would never want to see again.

As you said, there are plenty of terrific movies for which I wouldn't be upset if I never saw them again. Yet, would I recommend them? Yes.

Movies are so wonderfully complex that there is no single metric by which we can judge their credibility. We can only state our opinion of them, and leave the judgement to the next viewer.

However, the 50% rating is actually 3 stars. This is obvious when you list out your star options: 1,2,3,4,5. 3/5 stars visually appears to be better than 50%, but is actually exactly 50% due to the inability to choose 0 stars.

(This seems unresolvable as most people now recognize the image of 5 unfilled stars as null rather than 0%.)

Curious if they considered "Would you watch this movie again? [yes/no]"

I also wonder if NetPromoter could be adapted for rating things like movies. It uses the question "Would you recommend..." and has a much harsher formula (+1 for 9/10, -1 for 0-6).