This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I see, this feedback loop is to blame again. Viral content gets more viral, and less viral content disappears. This is because popularity is made out to be a metric of quality. All modern algorithms generally work like this, but it's a huge mistake. Merely changing the way the rating works from "Most plays" to "Best ratio of postive and negative reviews" should balance it better.
I actually want to make a game of my own. Guess I'll have to jump into a moral and social dilemma. Thanks for the answer by the way!
Ratio of positive/negative reviews is easily gamed and also selects for very niche things that has a loyal fanbase, rather than something with a wider appeal. I'd go as far as to say that positive/negative ratio on its own is worse than a pure view metric.
What you want is some kind of Bayesian weighting.
I don't see how it's easily gamed, but it does select for niche things. If these niche things are high quality to those that it appears to, isn't that fine? If less than 20% enjoy jazz, the better conclusion is "Of people who like Jazz, this one album is really good", rather than "Only 19% of all people like this album". Everything with a wider appeal has less depth, there's a sort of trade-off. I'd go as far as saying that everything good is niche. There's more people towards the middle of every standard distribution, but the best things (which are still popular enough to survive) are a few standard deviations to the right. And, if you allow those outside the niche to change what's inside of it, through the power of numbers, they will just destroy it or turn it into what they already like (which is plentiful everywhere). Hence why communities (like this one!) protect themselves with gatekeeping and rules and try to stay under the radar of outside political pressure.
I think the reason that votes aren't visible for a while on here is exactly to avoid starting a feedback loop (this one is the social one where people are influenced by other peoples votes). I also think that comments are sorted by "new" by default rather than by "best" (but I could be wrong), and that the "controversial" rating exists because the alternative is that the first decent comment to be made on a thread ends up being #1 simply because it started its exponential growth earlier.
Would your proposed weighting account for these things? (I don't know much about bayesian weighting)
You game it by controlling who gets access/early access to the game.
You can also review bomb other new games released close to your own release.
Given the quantity of games released this sort of score manipulation effectively turns that particular metric into a view of what has been released very recently, what is sufficiently niche to not attract non-fans and non-shills and what has most ratio manipulation behind it.
Controlling who gets access to the game (and therefore reviews) immediately devaluates reviews. I don't feel like this is an unintuitive exploit that we should guard against, but something so obvious that it goes without saying.
I wonder if text-only review would help the next problem. Instead of saying "this game is good/bad", you'd have to give information about the game. So if a game has 10 hours of gameplay, you'd write "I don't like that it's so short, only 10 hours!" but any reader who prefers shorter games would see it as a positive. Reviews like this would describe how the game was, and allow readers to judge the description, rather than merely access the judgement of the previous person.
Isn't this only if a new product gets a single positive review, putting it at 100% (perfect score)?
I do agree with the "niche" thing, but the opposite problem (niches being labeled bad because they don't appeal to a large amount of people) seems harmful as well, because it selects for watered-down content which is inoffensive and all-around unremarkable. If this sounds confusing, think about spicy foods: Those who love what is spicy wants the most spicy food available, but it's necessarily a small minority which enjoys this food, so any global rating would judge this food to be unappealing. If you place games (or other works) in a thousand-dimensional space, then all the edges and corners are maximums, and gives people who enjoy X the most X available. But across all people who judge the contents of this space, the highest scores will be biased towards the middle or possible the surface-area of the shape within the space. Less than 1% of music being listened to is Jazz, does this mean that it's universally hated, or that it should be banned? But that's the argumentation being used against controversial which has less than majority-support, for its removal is justified with the word "democracy". This is a bit of a rant, but I want to challenge the assumption that popularity is a measure of quality, and I think that rating systems may be inherently limited by the wrong assumption that there's one objective measure of good. I'm just theorycrafting, don't feel pressured to engage if it doesn't interest you!
The point isn't that it's too hard to figure out but that it effectively ruins that sort of filtering given the very large (and increasing) amount of games released every day. You might as well sort by new.
What you want is positive/negative reviews weigthed by how many reviews it has and how long it has been released, possibly with some downweighing very early reviews, both positive and negative. Text based reviews are of course also good but those are also possible to game, as seen with both smaller and larger games on steam, and it's quickly getting much worse with the advent of LLMs. This isn't a problem affecting a small amount of games but something that tons of developers do and it's increasingly an arms race, just like with regular search. You practically have to do it to be discoverable.
Ultimately you, as a consumer or a storefront, either need someone you trust to review the game or a very large amount of reviews, preferably released over a longer period of time, from paying customers so that it's effectively impossible to game.
I agree with what you're saying about niche interests but the problem of gaming the reviews still exists there and I feel like the solution is still weighted results but filtered by genre. Perhaps some amount of the featured games could be using genre filtered results so as to not get drowned out by the 800 pound gorillas and get some discoverability.
I feel like you just keep describing systems the Steam review score already has.
It a positive/negative aggregate which only counts steam purchases (hard to game review numbers when each review requires giving money to Valve), weights the descriptive categories based on the total number of reviews, has a recent reviews subcategory to downweight early reviews.
Since it only counts paid Steam purchases, it works especially great with niche genres. The steam review percentage doesn't correspond to what fraction of people like the game, it corresponds to what fraction of people who looked at the game and thought it interesting enough to spend money on liked it. (it works less well when the game has a divisive feature that doesn't neatly cleave across genre lines, such as any game with timers getting like a -10% to the review score)
My one big issue with the steam store is not with the reviews, but that recommendations of similar games seem to weigh popularity way more than similarity. Though I don't think there's a magic fix to discoverability, there is just too games coming out for that.
I'm not claiming these are some new revolutionary ideas, I'm pointing out why a simple ratio of positive/negative reviews don't work in a competitive and saturated environment like this.
This is a hard problem to solve and Valve has clearly thought about this quite a bit and their system is decent.
When there are a lot of purchases, sure, but initially you can spend part of your marketing budget gifting keys to sympathetic people, which people do.
Gifting keys to sympathetic people is free, you can just generate keys for your own game. But those don't count towards the review score. Review manipulation requires buying the game from dummy steam accounts with actual money and probably obfuscated payment methods. Which is not to say it doesn't happen, but there is a pretty clear line between marketing and probably steam tos violating review purchasing.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link