site banner

Meta: Is the Motte beset by bots?

So there's a delusional take you see on twitter Etc. All the time. From both sides of almost any issue but especially anything related to Russia, elections, Etc. You see people who respond to normal criticism or an abundance of criticism (usually relatively earned by how bad their takes are) accusing their detractors or those who disagree of being bots or astroturfed Putin or Clinton agents... The implied premise being that only lumps of code or Chinese sweatshop workers employed by bad faith actors could hold views that disagree with the complainer. That only bots or paid shills could oppose Ukraine, or support Clinton over Bernie, or Biden over Trump... Etc.

I used to dismiss these complaints... but now I feel I might owe a general apology.

.

I've noticed since TheMotte moved to its new Site that the Quality of a lot of Comments are just off. Not that the takes are bad or low quality or have odd opinions But that they're Bizarrely and Unnervingly detached from even the barest context of the discussion itself. Stuff completely out of character for even a bad rulebreaking poster on the motte.

Short comments that don't engage with any arguments presented, or even engage with the context of the discussion... but that Immediately tangent off on some culture war point utterly unrelated to the discussion, and then not engaging wit any replies (often with a single external link)... I've seen weird shit on twitter so I've dug into a few of these accounts... and all of their comments are like this, short snipes that never engage even 1 or 2 comments deep with anyone who replies. but that are slowly wracking up a history on the platform...

And then today I was hit by a smoking gun, this Comment:

“The Ukraine conflict is one of the clearest examples of good vs. evil in the past century"

You said it! Look at how despicable these people are!

Video: Ukraine Soldiers Sing Praises Of WW II Era Nazi: https://youtube.com/watch?v=4H-yMmNh5Cs

And now NPR is just casually rehabilitating the Nazis: https://www.npr.org/2022/03/03/1084113728/a-closer-look-at-the-volunteers-who-are-signing-up-to-fight-the-russians

Now the links are to real pieces of media, The Jimmy Dore Show and NPR... both respectable enough... and there'd be little to suggest this was a bot trying to manipulate the discussion... except for one thing:

No one had said the quote he was replying too...

Indeed I know where he got the quote. It was from a discussion/long take weeks before in relation to Ukraine, and would not even have fit the discussion in that piece, since it was a meta-discussion about how figures discuss Ukraine relative to other wars. I'd quoted it back then as an example of something we'd think was delusional and completely detached from intellectual rigor if said about Iraq 1991 or WW1...

Indeed another comment making the opposite argument used the same quote and drew other quotes from the same two week old discussion... except arguing the opposite way (pro-Ukraine)... And likewise replied not at all to having it pointed out that nothing they quoted was at all mentioned in the actual thread or discussion that was being had.

.

This obviously killed the discussion in that thread... when half the thread becomes comments quoting things, points and arugments, that were never said, and the other half must become replies saying in effect WTF!?

Well you can't have a discussion any more. Any organic back and forth between actual mottizens was killed. And obviously none of these either schizos or bots responded to keep discussion going.

Now if this becomes the norm it will kill the space...

.

But its also really unnerved me with regards to the rest of the internet.

The "Dead Internet Theory" doesn't feel like a theory anymore. The Motte is an obscure space with discussion levels high enough you notice if an actor isn't actually thinking or engaging with what's been said... and 2 out of 15 comments in that thread were Fairly undeniably bots....

On a site that's only been up a few months.

What the hell must it be like on other forums? Newspaper comments? YouTube comments?

Hell 4Chan had to implement Capchas for every comment to avoid the problem.

23
Jump in the discussion.

No email address required.

I think the quality has been going down, but I wouldn't peg bots. This might very well just be a personal opinion, just this morning, I was thinking about how increasing amount of subthreads are about trans stuff, a subject that, in the end, doesn't hold a great deal of interest to me. Not only that but it seems to be about endlessly relitigating a few particular facets of this particular culture war. Probably not something that it would be easy to write a bot for, though.

I think the migration just failed, and I plan on doing some data scraping / analytics this weekend to demonstrate it.

What kind of analysis or "hypothesis's" do you want to test. I tried doing a causal text analysis before on a specific subreddit and data is tremendously noisy, don't expect to not be disappointed.

You can use my source code to speed up your process if you want. https://github.com/f3zinker/RedScare-Analysis

Hey, thanks for that. I liked going through the analysis.

Also, I found this pretty funny (hopefully the table looks alright):

EDIT: it doesn't look like anything!

It was one comment being "hell yeah dude" getting readability_qual = "college level"

That's actually a mistake on my part, I reread the code again.

The metric I used returned negative grades. "Hell yeah dude" got 'between grade -2 and grade -3'. When I binned them into real life grade values, I should have added x < 4 -> "grade 4 and below" instead of 0 < x < 4 -> "grade 4 and below". Or I planned to drop negative values when I was doing the analysis and forgot to do it.

Just need to change one line of code, but I don't think it would really change anything all that much.

Ah, well... Fuck. That covers every idea I had and more.

What did you have in mind? I wanna know how others would come up with ways to assess a forums change in "quality"

Not a direct measure of quality, but I would expect it to be correlated: the number of different users participating. My concern is that the long tail of normal people didn't make it in the migration, and only the insane people are left.

@Porean

I think "sanity" is on an axis that is orthogonal to quality. My definition of quality isn't to do with how out of norm ones political opinions is but the quality of their insights, how well defended their arguments are, how logically consistent is the text to itself and its subject, etc.