This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
What is administrative burden in research for?
I think about this in a variety of domains, but it came up again when one of my tech news aggregators pointed to this paper. The idea is using LLMs to generate and evaluate protocols for biology experiments. I think the obvious key concern is related to well-known tradeoffs that people have been brought up in other contexts. Sometimes, it gets reduced to, "Well, people were concerned that with automated spell-checkers, then people will forget how to spell, but that's a silly problem, because even if they forget how to spell, their output that is augmented by the spell-checker will be plenty productive."
I wonder if there are limits to this reasoning. I'm thinking of two topics that I recall Matt Levine writing about (I can't find links at the moment; since Money Stuff always has multiple topics in each letter and he's written about similar topics that use similar words a bunch of times, I can't quickly find them).
One topic I recall is him talking about 'defensive' board meetings. The way I recall it is to suppose that a company puts in their public disclosures that they "consider cybersecurity risks". This doesn't necessarily mean that they do anything about cybersecurity risks, but they have to consider them. The way this plays out is that the board has to put an agenda item for one of their meetings to talk about cybersecurity risks. For an hour or whatever, the board has to talk about the general topic of cybersecurity. This talking can be at a high level of generality, and they don't have to really decide to do anything specific, so long as they have the official minutes that say, in writing, that they "considered" it. Without this, they might be liable for securities fraud. With it, they still might be extremely vulnerable and eventually lose a bunch of money when they're exploited (since they just talked and didn't do anything), but at least when that happens, they won't also get hit with a shareholder suit for securities fraud. (Really, Matt Levine would say, they'll absolutely get hit with a shareholder suit for securities fraud, but they'll be able to point to the minutes to defend themselves.)
The second topic I recall is him talking about where the value lies in corporate contract negotiation. He said that most times, you just start from the "typical" contract. Maybe something you've used in the past. You just pull that old contract off the shelf, change some particulars, then put it forward as a starting point. Then, the negotiations are often about just little modifications, and the phrase, "That's standard," is a pretty solid weapon against any modifications. He then talked about how a firm that does these negotiations in bulk as a service can start to sneak new provisions in around the edges in some contracts, so that they can later point to those prior contracts and say, "That's standard." Having the ability to set the "default" can have value.
So, biology. Science. Writing protocols is complicated, annoying, and time-intensive. Scott has written before about how infuriating the IRB process can be. Even with just that, there were questions about what the IRB process is for, and whether the current level of scrutiny is too lax, too strict, or about right.
Applying LLMs will potentially greatly decrease the barrier for newer researchers (say, grad students) to be able to generate piles of administrative style paperwork, saying all the proper words about what is "supposed" to be done, checking off every box that the IRB or whatever would ask for. But I do have to wonder... will it lead to short-cutting? "Sure, the LLM told us that we needed to have these thirty pages of boilerplate text, so we submitted these thirty pages of boilerplate text, but I mean, who actually does all of that stuff?!" Do they even take the time to read the entirety of the document? I can't imagine they're going to pay as close attention as they might have if they had to painstakingly go through the process of figuring out what the requirements were and why they were necessary (or coming to the personal conclusion that it was a dumb requirement that was necessary for the sake of being necessary). At least if they went through the process, they have to think about it and consider what it was that they were planning to do. This could lead to even worse situations than a board "considering" cybersecurity; they don't even need meeting notes to demonstrate that they "considered" the details of the protocol appropriately; the protocol itself is the written document that they theoretically took things into consideration in an assumed-to-be serious way.
This could also entrench silly requirements. You need to provide the subjects with pencils instead of pens? "That's standard." Who is going to be able to do the yeoman's job of subtly shifting the default to something that's, I don't know, not stupid?
I imagine all sorts of dispositions by particular researchers. There are obviously current researchers who just don't give a damn about doing things the right way, even to the point of outright fraud. There are obviously current researchers who really do care about doing things the "right way", to the point of being so frustrated with how convoluted the "right way" can be that they just give up on the whole she-bang (a la Scott). Which factors become more common? What becomes the prevalent way of doing things, and what are the likely widespread failure modes? Mostly, I worry that it could make things worse in both directions: needing large piles of paper to check off every box will lead to both short-cutting by inferior researchers, possibly producing even more shit-tier research (if that problem wasn't bad enough already; also, since they have the official documents, maybe it'll be in a form that is even harder to discover and criticize) and warding off honest, intelligent would-be researchers like Scott.
I don't know. Lowering the barrier can obviously also have positive effects of helping new researchers just 'magically' get a protocol that actually does make sense, and they can get on with producing units of science when they otherwise would have been stuck with a shit-tier protocol... but will we have enough of that to overcome these other effects?
When writing formal letters in Japanese, there are a variety of extra steps you have to do above and beyond fancy salutations and signoffs, including - my favourite - the seasonal observations beginning the letter (e.g., in August you could say "The oppressive heat continues to linger") and closing it ("please give my regards to everyone"). These are so stereotyped that I think most recipients of letters regard them more as a structural element of the composition than a semantic one, just as in English we don't really think of the virtue of sincerity when reading "Yours Sincerely".
I think this is basically what LLMs will do to writing, at least on the 5-10 year time scale. Everything will be written by LLMs and interpreted and summarised by LLMs, and there will be a whole SEO-style set of best practices to ensure your messages get interpreted in the right way. This might even mean that sometimes when we inspect the actual first-order content of compositions created by LLMs that there are elements we find bizarre or nonsensical, that are there for the AI readers rather than the human ones.
To get back to your point, I absolutely think this is going to happen to bureaucracy in academia and beyond, and I think it's a wonderful thing, a process to be cherished. Right now, the bureaucratic class in education, government, and elsewhere exert a strongly negative influence on productivity, and they have absolutely no incentives to trim down red tape to put themselves out of jobs or reduce the amount of power they hold. This bureaucratic class is at the heart of cost disease, and I'm not exaggerating when I say that their continued unchecked growth is a civilisation-level threat to us.
In this regard, LLMs are absolutely wonderful. They allow anyone with limited training to meet bureaucratic standards with minimal effort. Better still, they can bloviate at such length that the bureaucracy will be forced to rely on LLMs to decode them, as noted above, so they lose most of the advantage that comes with being able to speak bureaucratese better than honest productive citizens. "God created men, ChatGPT made them equal."
If you're worried that this will lead to lax academic standards or shoddy research practices, I'd reassure you that academic standards have never been laxer and shoddy research is absolutely everywhere, and the existence of review boards and similar apparatchik-filled bodies does nothing to curb these. If anything, by preventing basic research being done by anything except those with insider connections and a taste for bureaucracy, they make the problem worse. Similarly, academia is decreasingly valuable for delivering basic research; the incentive structures have been too rotten for too long, and almost no-one produces content with actual value.
I'm actually quite excited about what LLMs mean in this regard. As we get closer to the point where LLMs can spontaneously generate 5000-10000 word pieces that make plodding but cogent arguments and engage meticulously with the existing literature, huge swathes of the academic journal industry will simply be unable to survive the epistemic anarchy of receiving vast numbers of such submissions, with no way to tell the AI-generated ones from the human ones. And in the softer social sciences, LLMs will make the harder bits - i.e., the statistics - much easier and more accessible. I imagine the vast majority of PhD theses that get completed in these fields in 2024 will make extensive use of ChatGPT.
All of these changes will force creative destruction on academia in ways that will be beautiful and painful to watch but will ultimately be constructive. This will force us to think afresh about what on earth Philosophy and History and Sociology departments are all for, and how we measure their success. We'll have to build new institutions that are designed to be ecologically compatible with LLMs and an endless sea of mediocre but passable content. Meanwhile I expect harder fields like biomed and material sciences to (continue to) be supercharged by the capabilities of ML, with the comparative ineffectiveness of institutional research being shown up by insights from DeepMind et al.. We have so, so much to look forward to.
I'm well aware, and that's not what I'm worried about at all.
I know a bunch of social workers, some of them a generation or so older than me, and I heard a few stories of how things used to run. Like nowadays there was actual work, and there was a bunch of bureaucratic stuff to deal with. Back in the day you had to type it all out on actual paper, mail it etc. Then everybody and their dog started using computers for everything, everything got digitized, hundreds of apps meant to automate the drudgery got deployed, you could instantly send documents via e-mail... do you want to take a wild guess in which of these eras people spent more time doing actual work than they did dealing with bureaucratic nonsense?
I worry the same will happen with AI. No, it will not "make us equal", it's a side-rant but I'm shocked anyone could even utter such a sentiment with a straight face. There are, and there have always been, entire institutions devoted to the task of ensuring this will never happen. What will happen is that you will need AI to even keep up. You will need an AI text generator to output sheer amount of text you will now be required to write in order to cover your ass, and you will need an AI summarizer to "read" the tonnes upon tonnes of paper that will be sent your way. The best part is that all of this will be centralized in the hands of a few companies, who's owners hang out at the same cocktail parties as various panopticon fetishists at the top of our society, who will dictate how exactly this AI needs to be lobotomized to only output goodthink. It will now be the perfect tool for them to "nudge" us, old geezers like you or me might remember a world where you needed to process information yourself, but children born in the new one will only ever know information summarized to them via AI.
I've mentioned this before, but this is what drives me up the wall with AI-optimists. I know it's hard to learn the lessons of history, but this doesn't even count as history. We literally just watched Big Tech bitch-slap the ever-loving hell out utopian tech-nerds like 5 minutes ago, and I'm now supposed to jump on the next bandwagon that is going to "make us equal"? Give me a break.
That was my first thought as well. Like just how disconnected from the reality of day-to-day work does someone have to be for this prediction to make a lick of sense? It genuinely boggles the mind.
More options
Context Copy link
As I understand it, this is already a standard tactic used by large law firms to crush individual lawyers. They don't need LLM, they just hire a ton of new lawyers to churn out vast amounts of legal documents. A single lawyer trying to fight a lawsuit against them would get buried, because he just can't physically read all of that stuff and respond in any human lifetime, and if he can't respond he loses by default.
Big corporations also do this as a defense mechanism. So you want to sue them, and they're required to turn over the relevant docs? Oh they'll do that... but the "relevant docs" are like a million pages of garbage. Again, only a giant law firm has the resources to actually read through all of it and process it effectively. DDOS via human bureaucracy.
More options
Context Copy link
I share your pessimism. I think of it this way: the amount of time it takes to do anything increases with the amount of time available; the amount of admin that a bureaucracy requires of you increases with the amount they think you can do.
So LLM increases in people's capacity to handle admin work will result in an increase in the amount of required admin. "Of course you can complete this form as well; you can use an LLM to help you..."
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link