Advertisement

At first glance, this was just messed up to me.

Started by January 11, 2012 07:08 AM
11 comments, last by _mark_ 12 years, 8 months ago

When we analyze any of the ratings that we've gotten on this site, on a 1-5 scale we will see almost entirely 1's or 5's with an occasional 4 popping in. Hardly anybody ever rates a 2 or 3. This also includes the old rating system for users from before 2011.


I think one issue with that is that there is no "scaling" given. I know this problem from photographic gear reviews where users that rate stuff on a 1-10 scale will always compare to their other stuff. So when their best piece of equipment is something out of the midrange they will still rate it with 9 or 10 which often results in "good" gear getting ratings like 9.2 etc while actual high-end stuff will get like 9.7 or so and the largest portion of the range ends up unused. Now the moment you explicitly give a scaling (usually by associating words with the different ratings) the spread "opens up".
If next to the rating "button" it says something like "5 - genius, 4 - useful, 3 - somewhat useful, 2 - missed the point, 1 - wrong" then i'm pretty sure people will be far less likely to give genius and wrong ratings...

[quote name='roverstu' timestamp='1326370088' post='4901970']
When we analyze any of the ratings that we've gotten on this site, on a 1-5 scale we will see almost entirely 1's or 5's with an occasional 4 popping in. Hardly anybody ever rates a 2 or 3. This also includes the old rating system for users from before 2011.


I think one issue with that is that there is no "scaling" given. I know this problem from photographic gear reviews where users that rate stuff on a 1-10 scale will always compare to their other stuff. So when their best piece of equipment is something out of the midrange they will still rate it with 9 or 10 which often results in "good" gear getting ratings like 9.2 etc while actual high-end stuff will get like 9.7 or so and the largest portion of the range ends up unused. Now the moment you explicitly give a scaling (usually by associating words with the different ratings) the spread "opens up".
If next to the rating "button" it says something like "5 - genius, 4 - useful, 3 - somewhat useful, 2 - missed the point, 1 - wrong" then i'm pretty sure people will be far less likely to give genius and wrong ratings...
[/quote]
I'm not so sure. If you look at the IGN scale for ratings, which has a very clear definitions of each point on it's scale, you'll see that really people only care if it is higher or lower than 8.5 without realizing that even a 7 is a "good" game.

I think it boils down to most people recognizing things as good, bad, or having little to no opinion at all. Once you try to push more ratings than fit into that established mental scale most people will parse out the ratings that don't fit that scale to them.
Advertisement
While a valid argument in principle, the word "like" is actually ingenious [snip]
What you're talking about is the concept of "voting something up [or sometimes down]" - I don't think anyone is criticising that system. Which, incidentally was not invented by Facebook, but was commonplace long before.

I believe Cornstalks was referring to (and I agree with him) that it's the choice of word "like" itself that's problematic - it does lead to issues like those described here, which I think is a WTF.

It does have interesting side-effects (possibly intentional). E.g., on Facebook for many things you have to join a page/group to comment, but this means you have to "like" it, but I can imagine people being deterred from being seen to "like" something that they and their friends think is bad - even if their intent was only to criticise it. Arguably this is a positive thing, in that it reduces the risk of flamewars or trolling, but it also has problems where people might reasonably criticise it, such as in this case.

Google+'s +1 functions exactly the same way, but only works only thanks to FB having introduced the general idea.[/quote]Not only did FB not introduce the general idea, I'm also not convinced that Google's "+1" wouldn't have worked without it. People were happily using various "vote up", "recommend" options and so on long before, without people being confused as to what it meant. And if there is the argument that "+1" is too geeky for most people, I'd argue that that's still the case anyway, with or without FB. (I mean, when I first saw "+1", I did think it was a rather odd choice of wording.)

I disagree it's as bad as "++". That's something that only has a specific meaning in programming (and indeed, only some languages). But for "+1", I believe most of the population should be knowledgable with the concept of basic addition... smile.png I think the problem is more grammatical - people will say that they "like" or "recommend" a post, or that they would like to "vote" for it, but you don't say you're going to "+1" a post. (OTOH, I do credit Google with trying to make something obscure that can then be specific to them, rather than Facebook effectively claiming ownership to basic words in the English language...)

http://erebusrpg.sourceforge.net/ - Erebus, Open Source RPG for Windows/Linux/Android
http://conquests.sourceforge.net/ - Conquests, Open Source Civ-like Game for Windows/Linux

This topic is closed to new replies.

Advertisement