This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
What is administrative burden in research for?
I think about this in a variety of domains, but it came up again when one of my tech news aggregators pointed to this paper. The idea is using LLMs to generate and evaluate protocols for biology experiments. I think the obvious key concern is related to well-known tradeoffs that people have been brought up in other contexts. Sometimes, it gets reduced to, "Well, people were concerned that with automated spell-checkers, then people will forget how to spell, but that's a silly problem, because even if they forget how to spell, their output that is augmented by the spell-checker will be plenty productive."
I wonder if there are limits to this reasoning. I'm thinking of two topics that I recall Matt Levine writing about (I can't find links at the moment; since Money Stuff always has multiple topics in each letter and he's written about similar topics that use similar words a bunch of times, I can't quickly find them).
One topic I recall is him talking about 'defensive' board meetings. The way I recall it is to suppose that a company puts in their public disclosures that they "consider cybersecurity risks". This doesn't necessarily mean that they do anything about cybersecurity risks, but they have to consider them. The way this plays out is that the board has to put an agenda item for one of their meetings to talk about cybersecurity risks. For an hour or whatever, the board has to talk about the general topic of cybersecurity. This talking can be at a high level of generality, and they don't have to really decide to do anything specific, so long as they have the official minutes that say, in writing, that they "considered" it. Without this, they might be liable for securities fraud. With it, they still might be extremely vulnerable and eventually lose a bunch of money when they're exploited (since they just talked and didn't do anything), but at least when that happens, they won't also get hit with a shareholder suit for securities fraud. (Really, Matt Levine would say, they'll absolutely get hit with a shareholder suit for securities fraud, but they'll be able to point to the minutes to defend themselves.)
The second topic I recall is him talking about where the value lies in corporate contract negotiation. He said that most times, you just start from the "typical" contract. Maybe something you've used in the past. You just pull that old contract off the shelf, change some particulars, then put it forward as a starting point. Then, the negotiations are often about just little modifications, and the phrase, "That's standard," is a pretty solid weapon against any modifications. He then talked about how a firm that does these negotiations in bulk as a service can start to sneak new provisions in around the edges in some contracts, so that they can later point to those prior contracts and say, "That's standard." Having the ability to set the "default" can have value.
So, biology. Science. Writing protocols is complicated, annoying, and time-intensive. Scott has written before about how infuriating the IRB process can be. Even with just that, there were questions about what the IRB process is for, and whether the current level of scrutiny is too lax, too strict, or about right.
Applying LLMs will potentially greatly decrease the barrier for newer researchers (say, grad students) to be able to generate piles of administrative style paperwork, saying all the proper words about what is "supposed" to be done, checking off every box that the IRB or whatever would ask for. But I do have to wonder... will it lead to short-cutting? "Sure, the LLM told us that we needed to have these thirty pages of boilerplate text, so we submitted these thirty pages of boilerplate text, but I mean, who actually does all of that stuff?!" Do they even take the time to read the entirety of the document? I can't imagine they're going to pay as close attention as they might have if they had to painstakingly go through the process of figuring out what the requirements were and why they were necessary (or coming to the personal conclusion that it was a dumb requirement that was necessary for the sake of being necessary). At least if they went through the process, they have to think about it and consider what it was that they were planning to do. This could lead to even worse situations than a board "considering" cybersecurity; they don't even need meeting notes to demonstrate that they "considered" the details of the protocol appropriately; the protocol itself is the written document that they theoretically took things into consideration in an assumed-to-be serious way.
This could also entrench silly requirements. You need to provide the subjects with pencils instead of pens? "That's standard." Who is going to be able to do the yeoman's job of subtly shifting the default to something that's, I don't know, not stupid?
I imagine all sorts of dispositions by particular researchers. There are obviously current researchers who just don't give a damn about doing things the right way, even to the point of outright fraud. There are obviously current researchers who really do care about doing things the "right way", to the point of being so frustrated with how convoluted the "right way" can be that they just give up on the whole she-bang (a la Scott). Which factors become more common? What becomes the prevalent way of doing things, and what are the likely widespread failure modes? Mostly, I worry that it could make things worse in both directions: needing large piles of paper to check off every box will lead to both short-cutting by inferior researchers, possibly producing even more shit-tier research (if that problem wasn't bad enough already; also, since they have the official documents, maybe it'll be in a form that is even harder to discover and criticize) and warding off honest, intelligent would-be researchers like Scott.
I don't know. Lowering the barrier can obviously also have positive effects of helping new researchers just 'magically' get a protocol that actually does make sense, and they can get on with producing units of science when they otherwise would have been stuck with a shit-tier protocol... but will we have enough of that to overcome these other effects?
When writing formal letters in Japanese, there are a variety of extra steps you have to do above and beyond fancy salutations and signoffs, including - my favourite - the seasonal observations beginning the letter (e.g., in August you could say "The oppressive heat continues to linger") and closing it ("please give my regards to everyone"). These are so stereotyped that I think most recipients of letters regard them more as a structural element of the composition than a semantic one, just as in English we don't really think of the virtue of sincerity when reading "Yours Sincerely".
I think this is basically what LLMs will do to writing, at least on the 5-10 year time scale. Everything will be written by LLMs and interpreted and summarised by LLMs, and there will be a whole SEO-style set of best practices to ensure your messages get interpreted in the right way. This might even mean that sometimes when we inspect the actual first-order content of compositions created by LLMs that there are elements we find bizarre or nonsensical, that are there for the AI readers rather than the human ones.
To get back to your point, I absolutely think this is going to happen to bureaucracy in academia and beyond, and I think it's a wonderful thing, a process to be cherished. Right now, the bureaucratic class in education, government, and elsewhere exert a strongly negative influence on productivity, and they have absolutely no incentives to trim down red tape to put themselves out of jobs or reduce the amount of power they hold. This bureaucratic class is at the heart of cost disease, and I'm not exaggerating when I say that their continued unchecked growth is a civilisation-level threat to us.
In this regard, LLMs are absolutely wonderful. They allow anyone with limited training to meet bureaucratic standards with minimal effort. Better still, they can bloviate at such length that the bureaucracy will be forced to rely on LLMs to decode them, as noted above, so they lose most of the advantage that comes with being able to speak bureaucratese better than honest productive citizens. "God created men, ChatGPT made them equal."
If you're worried that this will lead to lax academic standards or shoddy research practices, I'd reassure you that academic standards have never been laxer and shoddy research is absolutely everywhere, and the existence of review boards and similar apparatchik-filled bodies does nothing to curb these. If anything, by preventing basic research being done by anything except those with insider connections and a taste for bureaucracy, they make the problem worse. Similarly, academia is decreasingly valuable for delivering basic research; the incentive structures have been too rotten for too long, and almost no-one produces content with actual value.
I'm actually quite excited about what LLMs mean in this regard. As we get closer to the point where LLMs can spontaneously generate 5000-10000 word pieces that make plodding but cogent arguments and engage meticulously with the existing literature, huge swathes of the academic journal industry will simply be unable to survive the epistemic anarchy of receiving vast numbers of such submissions, with no way to tell the AI-generated ones from the human ones. And in the softer social sciences, LLMs will make the harder bits - i.e., the statistics - much easier and more accessible. I imagine the vast majority of PhD theses that get completed in these fields in 2024 will make extensive use of ChatGPT.
All of these changes will force creative destruction on academia in ways that will be beautiful and painful to watch but will ultimately be constructive. This will force us to think afresh about what on earth Philosophy and History and Sociology departments are all for, and how we measure their success. We'll have to build new institutions that are designed to be ecologically compatible with LLMs and an endless sea of mediocre but passable content. Meanwhile I expect harder fields like biomed and material sciences to (continue to) be supercharged by the capabilities of ML, with the comparative ineffectiveness of institutional research being shown up by insights from DeepMind et al.. We have so, so much to look forward to.
I have a mild anecdotal counterpoint. It's old at this point, since I haven't worked in science in over a decade, but when I did, I was on my organization's institutional animal care and use committee, and despite the bureaucratic jargon and process, we actually did do something to curb some of the more pointless uses of research animals. The group wasn't particularly adversarial and worked with researchers on questions like whether the statistical power was going to be sufficient (we don't want to kill animals if the study won't even give a result), whether it could be done with fewer animals (same, but reversed), and whether the protocol used all reasonable practices to reduce pain and suffering of the animals (e.g. if the end point is death from an infection, can we just do infection instead, since the animal will tend to die painfully?).
I'm sure many groups feel that they're doing something constructive despite just being an annoying bureaucracy, and I'm sure that the review process we were doing was both imperfect and tedious, but I do want to offer that gentle pushback against it being literally useless.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link