Replace the 5-star Rating System

Netflix wants to ditch its 5-star rating system

Quote 1: “The problem, Hunt tells Business Insider, is that people subconsciously try to be critics. When they rate a movie or show from one to five stars, they fall into trying to objectively assess the “quality,” instead of basing the stars on how much “enjoyment” they got out of it.”

Quote 2: “Netflix wants to find new and better ways of capturing how much you actually enjoyed yourself — not just how objectively good you thought the movie was. The two methods Hunt is currently tinkering with are a “like/dislike signal” and some form of “percent match.”

Previously I already led a heated discussion in the main Suggestion Thread deriving from why YouTube abandoned the 5-star rating system:

Five Stars Dominate Ratings

Netflix’s move this year further suggests how unreliable star rating is.

The threat of extremes voters persists. Also, like Netflix’s worries, gamers may fall into assessing production value instead of fun.

I highly recommend a emotion / reaction-based rating system to replace the 5-star rating system: “LIKE / DISLIKE” (BINARY) OR “HAPPINESS / LOVE / ANGER / SADNESS / SHOCK” (Facebook Reactions).

Facebook to add happiness, love and anger buttons as alternatives to ‘like’

An option to mark games as “Favorite” could be nice too. Some games had major flaws so the star rating (judged by production value) may not be high. But we can happen to love it.

In fact, one of Grouvee’s rivals - Keep Track of My Games - currently my mainstay - already allows marking “Favorite”.

I wish Grouvee could use a better rating system or kill off the whole rating stuff. At the end of the day, the spotlight should be finishing the backlog.

4 Likes

Grouvee’s star system already factors in the whole “like/dislike” thing. If you hover over each star, a description appears for its value. I like it a lot now that I know the descriptions are there.

4 Likes

@Hitmuton, I mentioned in this thread that I generally agree that a five star system is not ideal. There was some back-and-forth in that thread regarding the merits and limits of switching from five star ratings. I still have reservations regarding any ratings system, but the fact that the stars are not tied to quality of a game, rather reflecting subjective enjoyment shows a conscious effort on the part of the the site management to address this issue., I think there has been an attempt to minimize ratings as a signifier of objective quality. Perhaps, as you mention, a favouriting option is useful. As was mentioned, and I believe also touched on by @peter, the combination of like and stars used by Letterboxd may be a useful model. Perhaps that provides better nuance.

However, even without a rating system I think there will be an effort made by some to provide ratings of their own. There’s no real eliminating the desire to rate a media experience. While some may be less inclined to do so, others will always gravitate toward using ratings as a metric for conveying a sense of their own experience, or gaining insight from others’ experiences to evaluate potential games to play.

I empathize with your desire to change the system, especially since objectivity is essentially impossible in respect to experience. However I don’t know if switching to a like/dislike system offers more nuance than rating systems. And since, as @kazooie points out, the star system is loosely attached to experiential or emotional view of the game Grouvee may have already addressed that issue. Perhaps part of the solution is making the meanings of those stars more explicit or visible as it is easy to miss out on their meanings (especially on mobile).

I am with you in spirit, given my dislike of rating experiences, but I am somewhat at a loss for identifying a solution that is not also encumbered by problems.

3 Likes

The favorites thing is definitely going to happen.

I’m working on a bunch of updates for the site right now, and adding favorites is on that list. My main work is updating the site to the newer version of Django so that I can keep making updates to the main codebase. I wrote about it here a little: Feedback, suggestions, say hi! - #228 by peter

I realize a 5 star rating system isn’t perfect, but I also don’t think just a binary system is right either. I’ve always liked a simple 5 star rating for myself, so that’s why Grouvee has one :smile:

I’ve also made it optional to use the ratings, so hopefully having a favorites button will be a decent compromise for what you want to use Grouvee for. I think our shelf interface is the easiest one to use out there, so hopefully as we tweak things here and there, it’ll be everything you want it to be.

6 Likes

I’ll keep my fingers crossed! :smirk:

1 Like

I found a problem with the favoriting system using it on KTOMG: overflow.

To trim it down, I resorted to “one game per franchise”. It makes more sense because the word “favorite” subtly means “the most liked”, only 1.

For what it’s worth, I realized the star rating should also stay. No system is perfect. We can wait it out what Netflix attempts to do next.

1 Like

I agree, though I would like to see a slight change in the descriptions. I think 3 stars should say “ok” (or “it’s ok”)…I think that’s a better description to use for a game you thought was average instead of “liked it”. “Liked it” should be used for 4 stars and keep 5 stars the same as it is now. With the way it is right now, I’m finding it a bit difficult to rate games I didn’t think were bad, but would be hesitant to flat out say I liked it.

4 Likes

I’ve run into that too. There’s nothing between “mostly didn’t like” and “like” to symbolize neutral or average. Also, the 1 star rating is showing up as “did not like it all” instead of “at all”. Not sure if it’s a me thing or if it’s happening to others.

4 Likes

I’ve left it like that mostly to irritate @headrollsoff at this point :smile:

1 Like

Personally I wish there was something more like a decimal system, or at least out of 10, or out of 100 (or even half-stars, like 3.5 stars or 4.5 stars). I often find myself rating games around the 7.5 or 7.0 rating and I feel weird about giving it 3 or 4 stars.
Perhaps a good system would be a bit like the Ludum Dare system. The reviewer can give a game a score for things like fun, graphics, audio, immersion, mood, etc. And then a final score is generated based on an average.

4 Likes

And it does. Every time I see it.

3 Likes

lol I’m liking the glimpse behind the scenes. It’s like a whole other world back here.

2 Likes

Generally, in five star ratings for other media (i.e movies) three stars means the reviewer liked the object. a rating such as ok would likely be 2.5.

1 Like

Ha! I never even noticed that. I’ll never unsee it now though. :frowning:

2 Likes

Although we cannot directly rate something 2.5, Grouvee averages all ratings to produce an overall picture that has more nuance. I know that is not exactly what you are looking for. However, as someone skeptical of rating systems in general, I think what Grouvee functions well. It is based on subjective interest in a game rather than functioning as an attempt to provide an objective measurement of game quality.

This is specifically the type of system I do not have interest in. I do not mean to disparage others for liking this system, I have no doubt it has its appeal. However, because I believe the experience of consuming media is highly subjective, and there is no way to measure the objective quality of a game I find systems like these difficult to accept. A friend of mine using something similar to rate films on Letterboxd, instead of the built in 5-star system because he specifically wants to prove the objective quality of films he deems good cinema versus those he deems bad cinema. I find that whole attempt both futile and misleading. Media experience is subjective, based on one’s experience or affectual relationship to the game, or movie, etc. I find I can pull very little from such a rating system because it requires I understand a person’s subjective position (biases, preferences, experiences, history) to understand why a game receives an 8/10 for sound/music or a 5/10 for mechanics.

I like Grouvee’s system (to a point) because it is based on enjoyment. I do not need to know anything about a person to understand that they liked, or did not like a game. That works for me. It also tells me a certain number of Grouvee members enjoyed a given game. Since more complex systems are just as subjective, a simple like to dislike scale tells me just as much information and delivers it in an uncomplicated way.

With all that said, I do not think your desire @Torgo is wrong, and if it were to be implemented I would not be against it. What I do enjoy is the discourse surrounding rating systems that is occurring in both academic and professional spheres, as many sites, journalists, critics and academics are currently debating the merits of various rating systems. As mentioned earlier by @Himuton, content delivery platforms such as Netflix are moving away from rating systems, or at lease discussing it, and I find it fascinating discourse.

2 Likes

Yeah, I’ve never liked that idea. If you only have 5 stars to work with, the 3rd star (which is directly in the middle with 2 stars under it and 2 stars above it) should be the average or neutral rating. The fact that people see 2.5 as the ok or neutral rating means they are no longer thinking about a 5 point scale, but a 10 point scale.

Personally, I wouldn’t want to see a 10 point scale (or a 100 point scale) because there are two large groups that think very differently about how to interpret a 10 point scale. You have one group that believes 5 is the average, while another believes 7 is the average because that is what has been ingrained in their mind from their particular school system.

2 Likes

The worst part is that it will take @peter about 30 seconds to correct it.

1 Like

@headrollsoff ah, but a 3 is 60%. 60 on Rotten Tomatoes is fresh. 2.5 is half way, neutral, neither good nor bad.

1 Like

I’m such a horrible person, if I knew how to fix it, I would… but I would go mis-word something else instead and eagerly wait for you to find it. :laughing:

2 Likes

@ClaudiusWhite I pointed out to @headrollsoff that a 3 is mathematically not the middle of a 5 point scale. Visually it may appear that way, but 2.5 is the middle. Even if we ignore decimals, we can’t ignore that 3 is not half way between 1 and 5, regardless of how it presents itself visually when written down.

1 Like