This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Slow news day? Guess I'll ramble for a bit.
Scientists shamelessly copy and paste ChatGPT output into a peer-reviewed journal article, like seriously they're not even subtle about it:
This is far from an isolated incident - a simple search of Google Scholar for the string "certainly, here is" returns many results. And that certainly isn't going to catch all the papers that have been LLM'd.
This raises the obvious question as to why I would bother reading your paper in the first place if non-trivial sections of it were written by an LLM. How can I trust that the rest of it wasn't written by an LLM? Why don't I cut out the middle man and just ask ChatGPT directly what it thinks about lithium-metal batteries and three-dimensional porous mesh structures?
All this fresh on the heels of youtube announcing that creators must now flag AI generated content in cases where omitting the label could be viewed as deceptive, because "it would be a shame if we (youtube) weren't in compliance with the new EU AI regulations", to which the collective response on Hacker News was "lmao okay, fair point. It would be a shame if we just lied about it."
It would be very boring to talk about how this represents a terminal decline in standards and the fall of the West, and how back in my day things were better and people actually took pride in their work, and how this is probably all part of the same vast conspiracy that's causing DEI and worse service at restaurants and $30 burgers at Five Guy's. Well of course people are going to be lazy and incompetent if you give them the opportunity. I'm lazy and incompetent too. I know what it feels like from the inside.
A more interesting theoretical question would be: are people always lazy and incompetent at the same rate, across all times and places? Or is it possible to organize society and culture in such a way that people are less likely to reach for the lazy option of copy and pasting ChatGPT output into their peer-reviewed journal articles; either because structural incentives are no longer aligned that way, or because it offends their own internal sense of moral decency.
You're always going to have a large swath of people below the true Platonic ideal of a 100 IQ individual, save large scale genetic engineering. That's just how it goes. Laziness I'm not so sure about - it seems like it might be easier to find historical examples of it varying drastically across cultures. Like, the whole idea of the American Revolution is always something that blew my mind. Was it really all about taxes? That sounds like the very-slightly-true sort of myth they teach you in elementary school that turns out to be not-actually-true-at-all. Do we have any historians who can comment? Because if it was all about taxes, then isn't that really wild? Imagine having such a stick up your ass about tax hikes that you start a whole damn revolution over it. Those were not "lazy" men, that's for sure. That seems like the sort of thing that could only be explained if the population had vastly different genetics compared to contemporary America, or a vastly different culture; unless there are "material conditions" that I'm simply not appreciating here.
Speaking of material conditions, Brian Leiter recently posted this:
Marx's view, as far as I can tell, was that ideas (including cultural values and moral guidelines) should be viewed as casually inert epiphenomena of the physical material and economic processes that were actually the driving forces behind social change. I don't know where he actually argues for this in his vast corpus, and I've never heard a Marxist articulate a convincing argument for it - it seems like they might just be assuming it (but if anyone does have a citation for it in Marx I would appreciate it!).
If Marx is right then the project of trying to reshape culture so as to make people less likely to copy and paste ChatGPT output into their peer-reviewed journal articles (I keep repeating the whole phrase to really drive it home) would flounder, because we would be improperly ascribing the cause of the behavior to abstract ideas when we should be ascribing it to material conditions. Which then raises the question of what material conditions make people accepting of AI output in the first place, and how those conditions might be different.
I'm now curious as to how much of this has started seeping into Law Review and Bar Journals, or if the standards there are still high enough and the reviewers still attentive enough that they'd get caught before publication.
More options
Context Copy link
Top-level LLM poasting. You have my seal of approval.
IQ scored are renormalized regularly, so in a society that did do that, the median IQ would be 100 again (though they would be far higher on previous scales, this is a minor nitpick, I'm in the mood for those)
One obvious answer would be that the LLMs have become good enough that accepting their output, in a research setting, actually does achieve the objective of creating and promulgating useful knowledge.
I don't think we're there quite yet, but it's a sign we're close if so many are leaking through the cracks in the hallowed peer review process. You almost certainly couldn't achieve that with GPT-2, it would be too incoherent, unless you went with absolute bottom-barrel pay to publish journals, whereas they're cropping up in modestly respectable ones and even the odd prestigious journal.
People, including scientists, have always been lazy to some degree. That usually manifested as having grad students or even undergrads doing the grunt work. Now we've got new avenues.
Besides, as far as I'm concerned, we're only a few years away from LLMs doing "legitimate" research, and then becoming outright superhuman. This is just a transitional stage to get there, we have to deal with ersatz good enough to fool checked out reviewers section for only a bit longer. And soon enough we'll have the journals using AI themselves to notice and filter out the crap.
I think these cases demonstrate the "peer review process" is not and was not working very well in the first place, and to the extent it was working, it was because of the remaining scraps of integrity among people writing and submitting manuscripts. Thus the reviewers didn't have to do much serious reviewing, like reading all of the manuscript and thinking about it.
I agree peer review is a flawed idea applied terribly.
The incentive structure simply makes no sense. Busy academics are expected to do a great deal of work and cognitive effort in return for little to no recognition or recompense. It's a testament to natural honesty that it sometimes works even in a subpar manner.
Unless you actively incentivize things like performing replications, it's all for naught. We wouldn't have a replication crisis in the softer sciences if that wasn't the case. At least it's more obvious in the harder ones when something plainly does not work.
It needs to be torn down and rebuilt, but easier said than done for such a loadbearing structure in Science™.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
As a teenager, I was a massive unkempt slob with zero shame until I took an interest in girls. Once I developed a crush on some girl, I'd look back in horror at my habits (wearing the same sweat pants every day, having a shitty diet), and tried to clean up my act 100% even in private where my actions are invisible. By nature, humans are highly-efficient pleasure seeking machines, and the only thing that meaningfully interrupts this behavior is some kind of ideal. We can gloat that American scientists probably engage in LLM bullshit less than Chinese scientists, but the Americans aren't far from doing it either. If Americans aren't forming cheating circles like the Chinese, it's not because we're above it so much as we're amateurs at cheating while the Chinese are masters at it.
Like most conflicts, it was framed in terms of ideals. Secular moderners don't really have ideals, so we struggle to imagine going to war over anything, really. Conversely if you've got a heightened sense of morality, anything is worth fighting over, and some men like Cicero or Boethius stick to their guns for an entire lifetime and pay the price. The founding fathers modeled themselves after these men, "Give me liberty or give me death", and it worked. Ideals are the only thing that fully override the comfort-seeking monkey brain, so if you want a nation of honest men, you have to make them genuinely value honesty.
Yes. There was a very stubborn principled mindset centuries ago where people would face certain or near-certain death rather than just let some ideological issue slide.
"Admit that Jesus existed always as part of God, or be burned alive very slowly." "I'll take the flames please."
More options
Context Copy link
More options
Context Copy link
Presumably because the paper includes an experiment and experimental results that are presented accurately, and allow you to learn something new about the field.
I mean, seriously. It's idiotic that a scientific career is gated behind having to write formulaic papers like this, a sane world would have the people who are good at devising and running novel and useful experiments do that, not spend half their time trying to write summaries (to say nothing of grants and lectures).
The numbers are the useful thing in the paper, if the experimental method and results are presented accurately then who cares whether the intro was written by an LLM. This is one of the few cases where tech like that could solve an actual problem we have, of scientific careers being gated behind being a competent and prolific writer.
Of course, the fact that prompt-related text was left in may signal a level of incompetence or rushing that casts doubt on the quality of the actual science, and that's a fair worry. But if that's not the case, then great, don't waste scientist's time on writing.
Why would I believe the paper that starts with a generated introduction had a real experiment behind it, and the results section was not also generated by an LLM?
The only thing keeping the science honest is the replication of experiments. If it is very cheap to describe and publish experiments that never happened, but running a real experiment to verify is costly, why would anyone try to replicate any random experiment they read about?
Unless someone comes up with a solution to reorganize the Science (or the eschaton is immanentized), I think the medium term equilibrium is going to look like even more weight given to academic credence-maintaining networks of reputation, less weight to traditional science (publishing results and judging publications on the merits of their results).
More options
Context Copy link
More options
Context Copy link
There are few jobs that don't have some amount of admin work associated with them. Generally you have to communicate about what you're doing with other people, and communication requires words.
Officially my job is to write code, but over half of an average day for me is spent writing emails and summaries and talking on the phone with people, because I need to talk about what I've been doing, what still needs to be done, and what the best way to get it done is.
Science can't progress if people just churn out experiments in silence and dump out big tables of numbers. If you want to say, argue that the balance of available evidence points towards dark matter theories instead of MOND, or if you want to argue that string theory is no longer a viable research program, then you need to use words. There's no way around it. Even if you do have someone who's silo'd away from the administrative processes as much as possible, they still need to communicate using words at some point.
Could you just put the content of your paper/argument in bullet point format and feed it into an LLM to clean it up and make it sound nice? That wouldn't be the worst thing in the world, but it would depend heavily on the specifics of each individual case. Almost all of the actual content would have to already be present in the input you give to the LLM, which means you're still going to be writing a lot of words yourself. If the LLM does a non-trivial amount of thinking for you, then it raises questions of plagiarism and academic dishonesty.
I can't see how this is an actual problem.
It's hard to imagine a competent scientist who is somehow so bad at writing that he can't clear the bar for your typical academic science journal, because verbal ability is highly correlated with IQ in general.
As for "prolific", that seems like even less of an issue, because the limiting factor in how many journal articles a scientist can publish is definitely not the amount of time it takes to write the words.
More options
Context Copy link
There are a whole lot of important and insightful scientific papers (in hard sciences) that don’t deal with experiments at all. Eg. This seminal paper that forms the basis of the entire field of digital signal processing and all modern long (and many not so long) distance communications.
When I’ve had to read papers (some hundreds of them) because of my studies or career, only a small minority have dealt with experiments and almost none with experiments that would have been feasible to reproduce without major investment in time and / or resources.
Consider also the vast majority of theoretical papers that have been published but you didn't read. Why people read seminal papers and vast majority of other published papers lie forgotten? Usually the papers that become seminal have special something that makes them useful and applicable in practice, and that applicability is discovered by testing against the reality. In experimental sciences, the testing against reality comes from running and reporting formal experiments. Sometimes in the form of explaining past observations and experiments. In engineering, people might not bother reporting experiments, but they integrate the useful results and principles in their products (which usually must be functional in the physical reality). In pure theory land, the mathematical proofs take the place of experiment (very difficult to come up with, often difficult to verify).
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Since all scientific papers are published in English despite 90% of the world’s scientists not speaking English as a native language (and even many of the 10% aren’t great writers) we should assume pretty much all ESL written work in English will be heavily LLM-generated from now on. That they forgot to delete the intro is bad, but it’s not really the same thing as, say, an author generating a book by LLM because the value - if there is any - will be in the data, not the abstract per se.
Given the already high rates of data fabrication inside but especially outside the West, I’d assign very little weight to any data from a paper where the authors, reviewers, and editors don’t even check for howlers like the ones quoted.
More broadly, speaking from the sausage factory floor, I can say that the trend in high-level publishing in the humanities increasingly seems to be towards special issues/special series where all papers are by invitation or commissioned. This creates some problems (harder for outsiders to break in, easier for ideologue editors to maintain a party line), but in general seems like an acceptable stopgap measure for wordcel fields to cover the next 5-10 year interregnum where LLM outputs are good enough to make open submission impossible, but not quite good enough to replace the best human scholars.
The thing is, those fields were already close to pure BS. That they can put off the transition from human-generated BS to machine-generated BS for a few years doesn't really matter to anyone outside the field.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Weirdly, I think ChatGPT could make papers better.
Let's be honest, prior to ChatGPT, most papers were still total garbage. Even if they had useful things to say, (which most didn't) the need to write in some sort of garbled academic-ese made them a chore to read at best.
There's a comic where a person says "Wow, with Chat-GPT I can turn a list of bullet points into a whole email". And the person on the other end says "Wow, with Chat-GPT I can turn a whole email into a list of bullet points".
If academics were serious about spreading knowledge, papers would either be presented in a few short pages, or in the style of a textbook, trying to explain complicated information to a reader with imperfect information. In the past, many papers were actually quite short. Nowadays, no one is enough of a Chad to submit a short paper. They have to fill it out with a bunch of bullshit nobody reads. If they deliberately make simple concepts sound complicated all the better.
Why not have ChatGPT do all that, and then the reader can use ChatGPT to know the correct parts to ignore?
Maybe I just suck at introspection, but I honestly don't think my papers are any more complicated than they have to be. I'll cop to some pro forma filler, but the introductory filler actually would be useful to someone that has some general domain knowledge but isn't well versed in the specific area. The discussions suck and probably actually are a waste of time (more study is required indeed). Nothing is deliberately confusing though and I don't think the introduction, methods, or results would be unintelligible to a layman with a passing understanding of the field.
You're one of the good ones!
If you're trying to be understood, a good rule of thumb is this: Dumb it down further than you think you need to. Pretty much everyone overestimates the intelligence/patience/contextual awareness of their reader.
Or as we say in computer land, it's easier to write code than to read it.
I can understand, however, that this can go against the need of the academic to sound intelligent. But it seems like you aren't motivated by that. Anecdotally, I think your writing on themotte is very clear.
I think a lot of this is variance across disciplines. I was an immunologist and my impression was that the field wasn't generally overrun with bad writing, or at least not the kind of bad writing that I associate with obscurantism. I just went back and tried to take a fresh-eyed look at my most cited paper (which is now old enough that it is almost fresh to read it again) and the thing that would probably be worst for someone outside the field is the alphabet soup nature of cytokine nomenclature. I don't think there's anything to be done about that though, there really just are a lot of cytokines that have conflicting roles in different contexts, differential regulation that's tricky to understand, and names that all kind of sound that same if they're not your old pals.
Other fields trend to either side of this. If I go pick up a physics paper, I'm in over my head pretty quickly (although not if I go to the Nature Physics website where I'm met with titles like Racial equity in physics education research and Towards meaningful diversity, equity and inclusion in physics learning environments on the home page). This isn't because of the authors putting on a show though, their material really is complex and requires a fair bit of background knowledge to avoid getting swamped pretty quickly. In contrast, the Journal of Sociology is silly, resulting in a more performative approach to the work, such as it is.
I'm sure someone has already done it, but something I've been bouncing around a bit is the idea of irreducible complexity in different thought domains. Some things are complex simply because they really are complex, there just isn't any simple way to understand them that doesn't become lossy. Other things really aren't all that complex, but the people in the profession both benefit from complexity and personally enjoy adding it on (much of law seems this way to me when I look at arguments). This shouldn't be read as saying that people in these fields are stupid - unfortunately, it's quite the opposite, they're clever enough to add many layers of complexity to something that should be intelligible to anyone that's interested.
This is an accurate description of software development for the past 10 years.
More options
Context Copy link
I am sorely tempted to make a "Stop Discovering New Cytokines" meme off the usual template . Interleukins were at their best when they were fodder for speculative Michael Crichton novels.
Accursed immunologists, almost as bad as the geneticists when it comes to bloating up medical textbooks.
My grandpappy never heard of DNA till he was done with med school, and it didn't do him no harm.
shakes fist
More options
Context Copy link
More options
Context Copy link
Do you have any specific examples in mind of academic writing that you think is needlessly complicated? Most accusations of intentional obfuscation are overblown, I think.
It’s normal for specialized fields to develop their own jargon. I’m in a few niche (non-academic) hobbies and newcomers often accuse us of intentional obfuscation. But to the experienced regulars our words are perfectly clear.
Academics write for fellow academics, people much like themselves with a similar educational background and usually a similar intelligence level. So they have a pretty good idea of what their readers will find clear and what they won’t.
There’s a problem today where some sub-sub-fields are so specialized that the audience of fellow specialists who are actually capable of understanding the work becomes very small, but I think that’s ultimately a separate issue.
This might be foreign to some people, but, using big words is fun. Reading and understanding a complex piece with lots of big words and dense references is also fun (if it’s well written to begin with of course). It’s not always a nefarious plot to bolster one’s social status. Some people just really enjoy reading and writing large amounts of complex text, and unsurprisingly those tend to be the kind of people who go into careers in academia where they get paid to do just that. So I absolutely don’t fault someone for not squeezing all his content into the smallest number of words possible. As the popular saying goes: let him cook.
Intentional obfuscation - sometimes. Far more I observe obfuscated language caused by the authors being sloppy and/or avoiding speaking plainly if they didn't understand something.
Most common: Enamored with big words yet trying to meet the journal word count limit, a big word is used in a way the meaning of the sentence becomes imprecise. Sometimes they have obtained a minor result, but big words are used to make it sound more important than it is. (Others will misunderstand and take the big words a a face value.)
Sometimes the authors are sloppy to extent that they understand meaning of some concept differently than others and never bother to make it explicit. Often the difference in understanding is a genuine difference in scientific opinion, but sometimes (especially in a run-of-a-mill study) it is because the authors failed to understand something. Sometimes the authors have followed "best practices" but do not understand the arguments for the best practices, producing slightly nonsensical approach. Sometimes authors claim to have found a $thing when they actually found $anotherthing. A mistake or misunderstanding is seldom admitted.
Sometimes the authors are sloppy reading or understanding the previous literature: when I see a paper cited in support of simplistic oneliner statement, these days I am never certain the cited reference supports the statement as clearly as implied ("It is known that system of soothing provides excellent results, thus we followed the approach of Tarr and Fether (1845)" -> go read Tarr and Fether, there is no single coherent system of soothing described, but three, and if you ignore the discussion but look at the results, the implications are unclear. Sometimes I suspect malice, more often I suspect laziness -- they never read Tarr and Fether, but they read something else that claimed to use the method of Tarr and Feather and misunderstood it.)
I see a lot of Science by Obfuscation. It's frustrating, because when I'm asked to review one of these papers, I don't know on the front end whether it's garbage or is genuinely using interesting and esoteric techniques from another area of literature that I'm just not familiar with. The latter is a real possibility that I have to spend a lot of time figuring out. Thankfully, I've only very rarely had to throw up my hands and tell the editors that I personally can't figure out what they're on about, and that maybe someone else would be a better reviewer. Unfortunately, the vast vast majority of my other experience is that once I can cut through their language to figure out what they're actually doing, I realize that it's really just dumb simple under the hood, and usually they don't really have any "contribution" over what has come before.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
People on twitter shit on woke American universities and lowered academic standards in the humanities, and rightfully so, but other countries have different but possibly even worse problems too-like massive amounts of academic fraud that goes unpunished or ignored. This includes plagiarism, auto-generated papers, and citation rings. Same for STEM, which is not immune to this trend of dilution seen elsewhere. I have evidence of citation rings on the arXiv computer science categories, in which there there seems to be very little value or research being produced--just authors citing each other's weak papers and collaborating in the production of said papers to pad CVs. This way more common in the computer science section compared to those 'soft' humanities, in which it can still be reasonably assumed that the putative authors still write their own papers without needing 10+ co-authors for a 10 page paper with 50 citations. It is ridiculous.
More options
Context Copy link
Well, the linked paper is a bunch of Chinese scientists at a Chinese university. Of course, there is shit-tier research done in the West too, plenty of it in fact, but I would bet a lot that the current crop of Chinese researchers are much lazier and more incompetent than their Western peers. I'm not up-to-date enough on the current literature in any field to offer any sort of meaningful first-hand perspective, but when I was closer to it, it always seemed to me that work out of China had more of a me-too, cargo cult feel of copying what Western scientists do. As before, Westerners are guilty of this sort of thing in plenty of cases as well, but my subjective opinion is that China is much, much worse.
More options
Context Copy link
More options
Context Copy link