Is there a way to fix BGG's ranking system?

Supporter

Anyone have any ideas? Something that will be clearer and more useful?

Like| 36 comments | report | subscribe

Please log in or make an account to post a comment.

Supporter13 months ago

Part of the issue is that many games get rated without the people owning it yet. If there was an easy way to confirm that it’s owned like a retailer connection or photo confirmation, it could lead to a lot less hype voting wars.

Supporter12 months ago

I have to disagree with you here.  I don't think you should have to own a game to rate it.  You should have played it of course.

If you wanted to go the ownership route you could potentially have them scan in the UPC or something.  It could easily be cheated of course but it would make it a bit trickier.  And I'm wondering if kickstarted games with no planned retail version have UPC's?  (I have no idea, maybe they do.)

Using a photo would require someone to manually approve it which will take a lot of someones times or using some fancy business intelligence to look at photos and decide if the boxy thing in the photo matches up with the given games box.   It might be possible to do but wow would it be a lot of work!

Supporter12 months ago

That's a good point. It'd make more sense to make it require a play rather than ownership. Maybe linking it to logged plays or something?

Supporter12 months ago

It would make sense that there should be a play before it’s rated. It’s a small hurdle but it’ll stop some shenanigans. 

Supporter13 months ago

Definitely. I could see how that would mitigate a big part of the problem. How would you resolve users who give horrendously low scores on games just because of bias in not approving certain types of game such as abstracts, dexterity games, or social deduction games?

Supporter13 months ago

I can see why BGG does not want to block ratings before the game is released. After all there are reviews and play testers that play the game before it is released. Where I think they fall down on the job is showing the rating before the game is released. If the ranking wouldn't be shown.... I don't think we would have as much problem with hypsters giving it a ten and others balancing the tens with ones. When Tapestry released... That was the worse storm of prerelease rating snd counter rating in have ever seen. 

12 months ago

Tapestry was an interesting social experiment for sure haha And a clear example that people are not to be trusted. :) I agree in part with the hidden ratings before it's released, but sometimes those ratings help others decide on a Kickstarter. Of course, I always go check to see if the designer/publisher ranked it a 10/10, and if so, the rest don't much matter to me because I don't know who I can trust anymore.

Supporter12 months ago

Yep. It is annoying. So many people and companies in this hobby are so nice, but then there are some companies out people just sort of twist things. 

Supporter12 months ago

YES! Definitely must own to rate

Supporter13 months ago

I have two main ideas. And these aren't necessary about BGG, but more of my thoughts on something we can entertain for BGA as well:

  1. Present two separate rankings - one based on "professional" reviewer accounts, and another by the regular accounts (perhaps this will bring more perspective/balance)
  2. Have a weighted system in which the more "experienced" users will have more impact on the ratings (and by "experienced," I mean something similar to Reddit with users who have high Karma points

Supporter12 months ago

Interesting. I like the idea and I think BGA could start a newer or more relevant system. Would be cool!

Supporter12 months ago

I like #1 but #2 doesn’t sound as plausible. 

Supporter12 months ago

I think so too. Way too many loopholes or too complicated to get it to a point where it's ok.

Like with Rotten Tomatoes or Metacritic as mentioned by Darryl (u/R0land1199), I really like having two different perspectives (one from a critic/professional vs. the public/regular user) because besides the reliability of the scores, the metric becomes way more interesting to look at. For example, when it comes to movies, I often find myself agreeing way more with the mass market than the critics. Not sure how well this sort of approach would work for board game industry though.

Supporter12 months ago

I immediately thought of Rotten Tomatoes as well with their system

Supporter12 months ago

I agree! The masses are typically more reliable to listen to than the critics, at least when it comes to movies and video games. 

I do feel like board game critics are more reliable and trustworthy than video game and movie critics, but having two ratings would be extremely helpful. 

Supporter12 months ago

Those are two very interesting ideas.  #1 is sort of like the metacritic scores for video games. 

#2 would be harder to do but is possible  Picking the right way to decide who gets more impact will be tricky.  A lot of the trouble makers are long time users who really don't want the site or their hobby to change.

12 months ago

I never rank games I review. At least, not until I'm finished with the review and it's live--and has been live for a while. I ranked one game I reviewed on BGG a 7, and the designer/publisher (same person, I think) emailed me asking me to either change the rating or remove it, as 7 was considered low for getting into retail stores. I wrote back defending my position, that I think it was worth a 7 (and that to me, 7 is still a good, fun, and enjoyable game that I'm usually happy to play). But, I took down the rating and decided not to get mixed up in that again hahaThus, I think a "professional" ranking would be a good step. I've heard similar stories from other reviewers as well. People don't like to hear their baby is ugly...or even super cute except that it's got boogers stuck to its face (so to speak).

Supporter12 months ago

Interesting story. I haven't really gotten into rating games on BGG but its odd that they'd reach out to you with a 7 rating. It is a competitive market though. 

12 months ago

It was a review for a game (standalone expansion, really) about to hit Kickstarter, so I can understand why they would want higher ratings as they launched. It is a competitive market, which makes my decision to not rate games I'm reviewing one of the best decisions of my life haha

Supporter12 months ago

I'm sure! Lol

Supporter12 months ago

That's tough. And I agree with you, 7 is a very decent score when considering where all games fall on BGG. 

And hearing your story, I feel like reviewers are placed in a very difficult spot:

  • Reviewing a game takes considerable amount of time and effort, which basically costs money
  • Many people expect you to do it for free and just do it for the passion
  • People also complain about the production quality, which often can't be improved unless you invest more of your own money
  • On top of that, having publishers trying to change your opinion because of one part of your review they don't like--seems disrespectful depending on how it was communicated

Sorry about the rant. Just seems ironic and unfair. I'm sure that kind of situation doesn't happen for you often, but just wanted to get that out hahaha....

12 months ago

Yeah, it has its moments, but for the most part, publishers are good to reviewers. I don't want to come off that they're not in generally, but it's really just a select few, and generally the ones that have one or two games published from their own company. Still, it's enough to make us jump at shadows haha

At first it was tough to know how to deal with it, but now that I've been doing it for a few years, it doesn't phase me. I get that some people will act like that, but that the vast majority are cool and willing to listen and work with you. 

But also, I agree with your points, which some publishers don't seem to understand. C'est la vie :) 

Supporter13 months ago

I've wished to be able to seperate the ratings, and see the average *of those who actually wrote something in their rating. * I always, personally, weight those higher. 

Supporter12 months ago

Good thought. 

12 months ago

If I'm looking seriously at ratings (rather than a quick glance to get an idea), I always check to see who wrote what, and what they ranked it. I also weigh those higher (even though I don't always write something in my own ratings).

Supporter12 months ago

There are definitely ways to make it better.  That said, I've had chats with people at BGG and they have no interest in modifying it.  The feeling I got is they want to pretend that their ratings have no impact and are just there for fun.  (However, when the BGG rating comes up during reviews and discussions with players quite often it's a hard argument to believe in.  The ratings have real world impact.)

If they were to change their mind there are several ways they could make it better:

1) Clearly and tightly define what BGG intends each rating to mean.  Yup, it's loose right now which means anyone can put anything and it's hard to argue.

2) Make it out of 5 and remove decimal values.  I love BGA's rating out of 5.  People get really caught up on rating something a 10 or a 1.  When you have 5 values I find it a lot more freeing.  Love a game?  It's a 5.  Hate a game?  It's a 1.  Sometimes too much choice creates a reduction in happiness.

3) Make it so you have to log a play to rate a game.  Yes, people can lie about having played the game but it makes it more work and they have to really commit to being a jerk about it.  It turns them from someone with an opinion in to an actual liar which will stop a lot of it.  (And that one guy who has rated something like 10,000 games a 1 because he uses it as a flag to not play the game or something which leads to...)

4) Make sure that all of the alternate uses for that field have their own fields instead.  People use that field for all kinds of other uses than rating.  Give them what they need so they can stop buggering things up.

Now, will any or all of those ideas make it perfect?  Not a chance.  Will it make it better?  For sure.   However, it's up to them what they want to do about it if anything and right now they aren't interested in changes of that nature.  

Edit to add: I didn't even get in to potential database cleansing mechanisms or reporting/suspension mechanisms.  I think start with social tweaks is the better way because it could solve the problem enough to make the rest unnecessary.

Supporter12 months ago

I think BGG never intended the rankings to have real world impact at the outset and are probably trying their best to keep it that way to avoid getting into the politics of it. Even with starting this post, I"m sure there won't be one rating system that will make everyone happy. 

Supporter12 months ago

I am nearly sure they don't want the ratings to have real world impact but they do whether they like it or not. 

There definitely won't be one system that makes everyone happy.  However, it's not an all or nothing proposition.

The trick here, if they wanted to "improve" things which they don't, is to define what the ratings mean for them to a detailed degree and then chip away towards that.  

Moving towards a definition like that can be done but, yeah I agree, that definition won't make everyone happy and perfection will never be achieved.  

Supporter12 months ago

What you said. Except, I do sorta disagree about point two. I quite like the 10 point system. 

Isn't the biggest problem the community? I don't really believe that BGG intended for their ratings to be considered the inspired word of the gaming gods. Aren't we weighting it to heavily in our buying decisions? Aren't we the ones misusing the system. 

I may have offended some of you by using the word "we" I don't mean "you" unless you are misusing the system. I mean that the Russians aren't meddling with BGG ratings. Non - gamers, by and large, aren't abusing it. People in the hobby are abusing it. 

Supporter12 months ago

I think people like to have “the source” somewhere for some reason. Especially these days when there are thousands of games released a year there is some value in having somewhere to narrow down your search.  BGG might not want the crown but they have had it thrust upon them.   

For myself I will definitely give a game a look on BGG and here as well to see public opinion if it’s super low I probably won’t look too much further unless a reason comes up to do so.  If it’s super highly rated and I see it used for cheap somewhere, yeah, I’ll give it a look.   It’s a time saving shortcut.

With regards to abusing it, yeah I agree it is people in the hobby doing it.  I don’t think there are too many abusers but enough to be a pain  One thing BGG does is seed new games with a large number of hidden “5” ratings (or something close to 5 anyway.) it helps eliminate the outliers until enough regular votes come in to play  to give it a real score  it’s why you see the BGG rating start off quite low initially vs user ratings  

All of that said, it’s totally their call as to how they handle it. I’m not blaming them for not wanting to make changes. To make a real dent in this would take a lot of work.