site banner

Small-Scale Question Sunday for March 26, 2023

Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?

This is your opportunity to ask questions. No question too simple or too silly.

Culture war topics are accepted, and proposals for a better intro post are appreciated.

3
Jump in the discussion.

No email address required.

Has anyone noticed that Amazon's "top review" feature doesn't usually work anymore? Years back, whenever you sort reviews by top reviews, the ones listed first always have the most "helpful votes." Now, most products do not. For example, see this iPad:

https://www.amazon.com/2021-Apple-10-2-inch-Wi-Fi-256GB/dp/B09G91TLNJ/

Notice how for a product with 40k ratings, the "top reviews" listed have "8 people found this helpful", "8...", "21...", "5..." etc. Historically, for such a popular product, the top reviews would have thousands of helpful votes, and certainly hundreds at a very minimum, and they would also be sorted in descending order instead of at random.

The same is true for all kinds of products, even those with tens if not hundreds of thousands of ratings, like airpods, airtags, etc.

What gives? I've noticed this being a frustrating problem for years by now. Surely Amazon is not stupid enough to not notice that its "top reviews" feature doesn't work. Is this a deliberate business decision to obfuscate reviews other shoppers have found helpful? Charitably, perhaps Amazon found such weighting incentivized chicanery with review manipulation, and so a new random ranking "fixes" the problem?

Strangely, when I tried to google this problem to see if any articles or blogs (or reddit threads) have complained about it, I couldn't find anything. Either no one actually cares about this feature being broken/gone, or my Googling was bad, or Google was bad. Why does tech no longer work anymore?!

I can answer this question pretty comprehensively. I've worked for Amazon for almost 15 years and was actively involved in review ban appeals when the changes happened that you find concerning here.

Here's the short version: Don't begin your product research on Amazon, you can't trust it. I'd advise reading the content of the 2-4 star reviews, but completely ignore the average review ratings, or anything on amazon that can be described as a 'rating'. None of them are honest. Instead decide what you want elsewhere, then come to Amazon to see if we sell it.

The long version is all the problems hinted at in the short answer were actually much worse in the time you are fondly remembering. Amazon banned all 'compensated' reviews in early 2016 and purged roughly 80% of all reviews ever made off the site at that time. The overwhelming majority of them were completely fake, written by people paid to write them. Often on dozens of accounts. This was in collusion with both sellers and manufacturers, including huge international brands. Millions of dollars of sales can swing in one direction or another based on people believing the reviews. After all, you did. That's way to much money on the table to leave to stupid customers giving honest reviews to have any influence over. While I've not worked in that part of the company for several years, I still know a lot of people that do. None of us trust the reviews at all. The banning of openly compensated reviews only drove it underground, into a group of smaller, much more covert super-reviewers using proxies, cloned devices, Chinese "Like Farm" style operations where entire fake identities and created to bypass safeguards. This is where Brushing comes from:

https://www.uspis.gov/news/scam-article/brushing-scam

So rather than securing the integrity of the system, the changes shifted the review abuse from a cottage industry of bored stay at home moms to Asian organized crime. The "helpful review" votes you miss were also, always, fake. Just another part of what they were paid to do. A list of reviews for you to vote as helpful came with the list of items you were paid to give fake 5 star reviews. Or fake 1 star reviews if you were being paid to sabotage a competitor, which was almost as common as the fake positive reviews.

Now you know. Also sorry I guess. The native Amazon search is unreliable as well, where your products appear is pay-to-play, has been for years.

Interesting, thanks. I suppose I would have suggested that Amazon get rid of the top reviews option altogether then, if it doesn't actually work. But perhaps that would make the curious inquire after its disappearance and prompt more bad PR.

I also get the sense what you describe is a solvable problem. If shady incentivized reviews is such a pervasive problem, why not develop a trusted reviewer network using some kind of ML analytics? I write occasional reviews and I suspect it's very quite obvious based on the data and metadata that I'm a regular consumer and and part of organized crime or cottage industry of SAHMs.

Even now, half the reason I default to Amazon over say Walmart is that the former has a lot more reviews, and that's useful info. You'd think they'd invest far more in protecting this moat. Your suggestion to do product research off site makes sense but wouldn't apply to the purchasing behavior of 90+% of Amazon customers I bet.