This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Although this is clearly groundbreaking and historic and I’m not downplaying it in any way, at a glance this seems to be in line with the other major AI mathematics results we’ve seen lately, which mostly fall into the categories of either “noticing links between existing papers in the literature that no one had noticed before”, or “pulling a specific counterexample/construction out of the impossibly vast search space”. I don’t at all have the technical knowledge to evaluate how unexpected this result was, how close it is to existing methods/approaches, etc, but that’s what it looks like to my untrained eyes. I think we’re still waiting for that qualitative breakthrough where AI generates valuable new ideas ex nihilo that really change the way people think about a given domain; I think we’re still waiting for that qualitative breakthrough in math and in every other field. There’s a distinction between say, solving an individual problem and leaving it there, vs inventing something like Galois theory that spawns many new fundamental research directions. If a subject matter expert DOES think this paper will change the lay of the land in fluid dynamics (beyond just the significance of the result itself), please chime in.
It’s mostly a moot point though since all these questions are actually pretty easy to test empirically. If OpenAI and Anthropic are still employing humans, then there’s clearly still work to be done.
Imagine a car.
For a long time, it was a purely theoretical car. Spending any amount of time theorizing about its safety features was a charming activity for undersocialized nerds.
Then it moved. Slowly enough that you could walk alongside it. It was a mild curiosity, at best, to most.
Then it started picking up speed. The acceleration was increasing too, and some of us found that concerning. You could plot the bloody thing. I can't stress enough how little of this required a revelation.
I was around when we were trying to design safety features for a vehicle that didn't exist yet. Later, I was politely asking people to take notice when the ride started getting bumpy.
About a year and a half ago, I became somewhat frantic. Did people really not see what was coming? I spent far too much time arguing about it. I should have saved my breath.
Six months ago, with Mythos? The steering wheel came off. And I considered myself done with trying to convince anyone of anything.
I have the misfortune of occasionally reading someone explaining that the car can't be dangerous because of how the engine works. Or that it's a beat up sedan, and we should only worry about F1 cars. And don't you know that the speed of limit is a hard limit on velocity? Or that our current transmission systems break down at 570 km/h?
I'm sure there's a reply to that. I've probably written one before. If writing doesn't work, look out a window before it's all a blur.
Can people see an inch beyond their noses? Apparently not.
These days, I'm doing my best to strap myself in. I have no idea how much good that will do, but I've only got so much time and attention, and I'd like to spend some of it on the assumption that I have a future.
I don't know where the road ends. But some people are about to wish for airbags, and I find that a "told you so" is a small consolation.
More options
Context Copy link
That does sound like downplaying it, your preamble notwithstanding.
But, sure, we don't have an AI Grothendieck yet. It's been a full three hours since OpenAI announced a solution to Navier Stokes; but, what's it done for us lately, I think we've hit a wall, etc.
Give it a year, or six months, or three, who knows, and we'll be in a different place.
From what I’ve seen of the proofs, you still have a case of researcher + AI solving them rather than just pure AI. An “issue” with following the progress AI development is that until we reach AGI, which allows it to be independent and flip the board, you still will keep the paradigm of AI plus human even as the AI gets better and better. AI is going to keep getting more and more useful for a period of time until it becomes independent and we get full AGI. Until then the paradigm, and the existence of the researchers jobs even if the number is reduced, will not change.
More options
Context Copy link
It’s unfortunate that this whole discussion has been poisoned from the start because people can’t resist turning it into a competition. Granted, I completely understand why it’s deeply satisfying to run victory laps around the Gary Marcuses of the world and I don’t fault you for that.
OpenAI hasn’t sent everyone home yet and many people are still gainfully employed doing the sorts of things that people ordinarily do. Until that changes, I don’t see any problems with speculating about why that hasn’t happened yet, where the remaining gaps are, and what the nature of those gaps is.
More options
Context Copy link
He has a point. Most of the stuff I've heard of (I am not a mathematician) is proof by counterexample which doesn't per se expand the space of what we can do. I suspect the AI+researcher combo is still significantly more important than just AI for the stuff that actually transfers beyond mathematics.
Centaurs are unstable. Before long, AI+human combos are left behind by even stronger AIs, so strong that adding a human to the loop just makes it worse; the human slows the AI down, becoming a liability. In chess, that took ten years. How long will math take? Perhaps another year or two.
I know that this is what people say, but it's predicated on a scenario where the Venn diagram of AI capabilities (at a useable price even) becomes a strict superset of human capabilities. At the moment this hasn't happened even in software where AI cannot maintain a codebase long-term; studies showing that AI outcompetes centaurs are usually on short-term discrimination tasks like medical diagnoses.
I think it's quite likely that this scenario comes to pass, but it hasn't yet, so we'll see.
More options
Context Copy link
In some fields it's been this way for decades (hello air defense!); in others I suspect it will be a very long time before centaurs are replaced.
My sense is that even very smart LLMs are very gullible. I think it will be some time before centaurs aren't invaluable in conducting opposed work, which is the most important work of all. Pivoting models away from general purpose work towards specialized tasks may help here, even if that doesn't seem to be something the primes want to admit.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link