site banner
PaperclipPerfector  7d ago (text post)   24401 thread views

Culture War Roundup for the week of October 12, 2026

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

8
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

This does not quite fall under culture wars, but I thought it's still relevant given that it's a big story.

On October 6, OpenAI released a large batch of mathematical results, with more than 700 manuscripts covering hundreds of problems: Sharing AI progress in mathematics.

This has led to another round of controversy and discussion about AI making mathematicians obsolete or disrupting research programs that people have spent years working on. The Verge covers some of these reactions in “Mathematicians will need years to make sense of OpenAI’s latest drop”.

Many of the results are still being checked, but the release has called into question if mathematics can survive as a profession, or what purpose it serves if machines can outpace humans. Research programs have supposedly been gutted by these results. "Careers ruined," as the narrative goes. I think that is an exaggeration or overreaction. There are thousands of mathematicians and their interests are very wide ranging. The odds of someone's research program or problem being "solved" by AI are very small.

I don't see anything changing much. The status quo of journals and academia will remain intact even with rapid progress in AI, save for the occasional breakthrough in an unsolved problem. What is overlooked or ignored is the massive gap between cotemporary research-level math vs. LLM-produced math. A math PhD program, especially from a good school, teaches or guides future researchers towards productive areas of research that are of interest to the "math community," rather than results that are merely "personally interesting". AI cannot recreate such communities or networks.

AI is great for generating computation-laden results on-the-fly, but research-level math is more concerned with abstractions and relations between them, than crunching. I recall one example where ChatGPT’s Astra program tried to pass off a finite computational cutoff as a “theorem.” Um, you can't do that unless you prove that the bound is sufficient.

To test this, I produced some AI papers which I thought were great and novel, and the AI said so, and journals rejected them (although others are still pending at various stages of peer review). I had some knowledge in said area and could assess novelty, but still not good enough. I don't see laypeople doing better in this regard, although could be wrong. I ran my ideas through the Ai and got 12 papers out of it, which saved considerable time had I written them myself. It also found non-obvious connections and results. I am really satisfied with the results, but does this meet the bar for publication? Time will tell. I bet not.

Also, success is heavily dependent on the prompt. It's not as easy as blindly prompting something. I made a post about that https://greyenlightenment.com/2026/10/11/the-nuance-the-math-debate-is-missing/ I suspect many of these successful proofs were prompted by experts who knew what to look for, rather than just naively trying. What is touted by the media as an AI breakthrough, may involve surgically precise prompting in the background by an expert in said field.

Alluding to the "airplane meme," there is also survivorship bias: you don't see all the failed attempts with AI . Sometimes it works great, but other times hit or miss. Presumably , the "top unsolved problems" have been queried by frontier models and using large token budget, with few proofs to show for it (save for the Open AI "dump").

It would be cool to see AI truly shake -up math and breathe new life into ignored areas, but that is not going to happen, imho. But I do see AI leading to a 2-track system of decentralized preprints and GitHub repositories of AI-produced results, alongside established academia and journals. They will coexist, rather than the later being obsolete. Overall, mathematicians don't have to fear losing their jobs anytime soon (unless of course you happen to have the precise problem you are working on scooped by AI, and even then, your skills would presumably transfer to new problems.

They will coexist, rather than the later being obsolete. Overall, mathematicians don't have to fear losing their jobs anytime soon (unless of course you happen to have the precise problem you are working on scooped by AI, and even then, your skills would presumably transfer to new problems.

The right question to ask instead is what moat mathematicians have, what skills they have which are immune to being encroached upon by AI. Three years ago this was easily answerable: LLMs sucked at basic math.

Today, it is a lot less obvious to me. Sure, at the moment the models might mostly come for proofs which rely on heavy numerical computation efforts, because software development is a capability in which they were trained quite a lot for unrelated reasons, but in the end the domain of mathematics seems quite susceptible to RLVR.

Human mathematicians might be better at identifying interesting areas of mathematics, but even that is not given. A model might need orders of magnitude more training data to get which math is interesting than a human would, just like a model requires orders of magnitude more training data to learn English than a human, but this does not matter if the training data is available. A model can trivially be trained on every math publication published in the last hundred years, while a human can see only the tiniest fraction of them.

I am sure that once upon a time the best way to solve chess problems was to have an expert human player working with a computer, but today the human simply has nothing to offer which would improve upon the machine. This does not mean that it will be similar in every domain, but at least that this outcome is possible.

I am in experimental physics, and I feel that for the moment, we have still a bit of a moat in that unlike the models, we have hands. Even if you give the models biodrones (i.e. minimum wage labor wearing AR goggles), I think the sensory feedback quality would be worse than what a skilled human can get.

Still, I liked my situation a lot better when our moat was our ability to think rather than to use our hands.

LLMs still cannot create research programs or appraise novelty well. They will crunch a bunch of numbers and spit out something which although technically new, is of little value or interest to the the "math community" (unless it solves an important conjecture/problem, and this depends heavily on prompting too )

A model can trivially be trained on every math publication published in the last hundred years, while a human can see only the tiniest fraction of them.

The models already are trained on them, and probably have been since 2022.

All of these discussions cause people to reflect on the process of doing math. They’re like, “Oh, this thing can do this impressive stuff. What does that mean for us?” One of the people in that video highlights this quote: “good mathematicians prove theorems, great mathematicians come up with conjectures, and the greatest mathematicians come up with definitions.” That’s more or less exactly your framing here. We need the conjecture generator and then the definition generator. That’s the premium-tier mathematician.

What is it that causes current models to be great theorem provers and poor conjecture and definition makers? It's unclear, and perhaps this comment will look foolish in six months, but it's not as simple as "train on all the math papers bro".

What is it that causes current models to be great theorem provers and poor conjecture and definition makers?

Assuming that this is true (and I think you are probably right), I would guess that it has to do with the way the model is trained. The easier it is to verify that a correct answer is indeed correct, the easier it is to put together a big pool of training data for an AI system.

So for example, suppose you wanted to train an AI system to create funny jokes. If you had a million jokes and funniness scores for each joke which were precise and accurate, you could probably do it pretty well. But it would be extremely difficult and expensive to construct this kind of training data set.

With mathematical theorems, it's much easier to verify that a proof is correct -- to a large extent it can be automated. Which means that it's much less expensive to train an AI system. By contrast, with conjectures, it's much more difficult to evaluate how good a conjecture is.

Anyway, I am not an expert on AI but that's my guess.

What is it that causes current models to be great theorem provers and poor conjecture and definition makers?

What is the evidence that current models are poor "conjecture and definition makers" relative to humans?

An alternative hypothesis is that this is a case of the "what happens when you post a real Monet and say it's AI?" phenomenon, in which human purported experts systematically downplay the strength of AI capabilities for complicated social / psychological reasons.

Right, the reason we know that models are now better than us at proof is because it's objective and testable. When (probably not if) models surpass us at making conjectures and definitions, I expect us to take a lot longer to acknowledge it. That might have already happened, but we wouldn't see a press release proudly crowing about it; mathematicians would just quietly use AI as a source of inspiration more and more.

I think the coolest thing a mathematician can do is to create a more interpretable lean.

Lots of complaints about how lean proofs are unreadable - then create your own high level lang that compiles down into lean or have your own verifier.

If this works, then all future AI proofs have a strong incentive to go through your thing.

Might be one of the few big math infra projects, though arguably humans may want to curate the proof/lemma ontologies for interpretability.

Overall, mathematicians don't have to fear losing their jobs anytime soon

This is true. But not because they haven't been obsoleted. They have. Rather because most of them work in industries that do not generally fire useless people.

I don't say this with any joy. I absolutely respect the mammoth intellectual ability of top mathematicians. Chess players too.

But their services are no longer required. If I need someone to explain a mathematical result to me, I will ask an AI.

you think math doctoral programs will go away?

No. I think make work jobs will if anything be more common in the future.

not because they haven't been obsoleted. They have.

Well, what does it mean to be obsoleted?

Viewed from one lens, chess players were obsoleted by DeepBlue and Go players by AlphaGo a long time ago. If you want the best chess sparring partner, your phone can trivially outclass the best humans alive without any reason to ever play another human being.

Viewed from another lens, human chess and go has been exploding in popularity over the last few years; there are likely many more people who play chess and go now when compared to two decades ago, and it would be ridiculous to claim the human acts of playing chess or go are obsoleted. In practice, the best human players are still high status and make a lot of money, while the overwhelming supremacy of Stockfish is free and generates almost no economic value.

For those who had dreams of being the greatest chess player on the planet or etching their name into history as the next Euler, then I'm sure losing to the computer is a blow to the ego, but realistically this wasn't on the table for the vast majority of even the best chess players or mathematicians alive, AI or not.

There's no reason why the human practice of mathematics can't reorient itself around the supremacy of AI while continuing to draw down salaries and playing petty academic status games; god knows none of it was any practical importance in the end.

I would say that chess is analogous to track and field: You are competing against other people. The fact that there are machines which can move faster that Usain Bolt doesn't really diminish his accomplishments.

Is pure math like this as well? Not at the moment, it seems. Once they graduate college, the way mathematicians gain status is not by winning math contests, but instead by being the first person to come up with and publish an elegant way to prove or disprove some significant unsolved problem. And AI threatens to take this away.

There's no reason why the human practice of mathematics can't reorient itself around the supremacy of AI while continuing to draw down salaries and playing petty academic status games; god knows none of it was any practical importance in the end.

How would that look? Would there be an International Math Olympiad for adults, with problems generated and graded by an AI? I'm not saying that you're wrong, by the way.

How would that look?

Probably much like it does right now, a mathematical priestly caste who spend their lives training to understand and disseminate the arcane knowledge of frontier mathematics.

Even prior to AI, modern pure mathematics was already impenetrable to 99.99% of the population, and even mathematicians of different disciplines mostly struggled to understand each other's work.

Now that we have AI it's really no different; nearly everyone has no ability to even approach the starting line of understanding the superb works of the AI, and it's mostly through the lens of mathematicians that we assign value and meaning to the new results. I don't see why such a dynamic can't continue once the mathematicians manage to reconstruct their shattered egos.

Probably much like it does right now, a mathematical priestly caste who spend their lives training to understand and disseminate the arcane knowledge of frontier mathematics.

I see a potential problems with this. So what if there is an important unsolved problem of mathematics and before a member of the priestly caste solves it, Joe Blow from his basement publishes an elegant proof online which resolves the problem. Would Joe Blow simply be ignored? Would credit (and the Field Medal) go to whatever priest is the first to copy Joe Blow's paper?

Of course a more likely outcome is that each professional mathematician will use his own AI system, sort of like Gary Kasparov's concept of "power chess" back in the day. A human master to make big strategic decisions and a computer for tactical analysis. Except that power chess didn't last -- the computers got too good for humans to be of any use.

Ok, so what would an international chess championship look like if each player were free to use whatever computer assistance he wanted? It seems like it wouldn't confer much status on the winner. In any event he wouldn't be seen as a great chess player.

Let's put the question this way:

In a post-AI world, how would it be decided which mathematicians win the Fields Medal?

This is true. But not because they haven't been obsoleted. They have. Rather because most of them work in industries that do not generally fire useless people.

There's still the problem that a lot of people don't like to feel useless. Of course I am aware that lots of people work BS jobs where they do essentially nothing but genuinely believe (at least to some extent) that they are important and valuable. But I think that most non-DEI math professors take genuine pride in the skills that they have developed over the years and feel genuinely bad about the fact that they are staring obsolescence in the face.

Anyway, I'm not a mathematician but that's how I basically feel. I'm reasonably good at what I do and I enjoy the satisfaction of solving a challenging problem and the respect I get when I do so. It makes me sad that very likely it's about to go away.

For some it’s also “useless” like art: they want to understand and explain to others because they find the logic beautiful (in the exact same way artists find art beautiful).

And if AI obsoletes most human labor, why not keep paying them for that? A global mental health crisis is creating inefficiency and infighting, just look at modern politics.

For some it’s also “useless” like art: they want to understand and explain to others because they find the logic beautiful (in the exact same way artists find art beautiful).

I agree, but here's what I said in another post:

FWIW I have a professional physicist in my family and he is pretty much resigned to the idea of studying physics out of love for learning more about the way the universe works. Which I guess is a good attitude, but if it were me I would be kind of disappointed if I suddenly had effectively no chance of discovering and publicizing something significant. Because the AI is going to beat me to the punch.

And if AI obsoletes most human labor, why not keep paying them for that?

Fundamentally there is no reason not to. It's probably better to be the subsidized artist in your scenario than to be flat out unemployed. But still, I would feel pretty bad about the whole situation.

I myself started a business a few years back based on a creative idea I had had. I put the business together, worked on it night and day, encountered reasonable success, and ultimately sold for a good deal of money. It was a really nice experience to turn my ideas into a reality which the free market rewarded. It was also nice to earn the respect of my peers. Being a subsidized artist would be okay, but for me it wouldn't be the same.

But their services are no longer required. If I need someone to explain a mathematical result to me, I will ask an AI.

We are really happy that ai is always right.

Humans are wrong a lot too. The problem with the "centaur" model, humans aided by AI, is that at some point humans just get in the way.

Recently there was a physics problem set they gave to Astra where it scored 34/50. But then, after more investigation, it turned out that Astra was actually correct where the researchers who developed the problem set had made subtle errors. After adjusting for that, it scored 49/50.

The centaur era in chess lasted for about a decade. But after about 2014, human involvement could only make chess engines worse. The centaur era in mathematics seems to be coming to a close much more quickly. "AI but humans check the work" is going to fade fairly quickly as humans can only introduce errors.

Again, none of this makes me happy.

The centaur era in chess lasted for about a decade. But after about 2014, human involvement could only make chess engines worse.

Yeah, I was about to bring up this analogy. For now, even assuming that all these AI math proofs are correct, you still need humans involved to interpret, check, and explain. And to some extent, to pose interesting problems for the AIs. But it's hard to see this lasting.

When the results come with a proof it is always right (well, occasionally it exploits the prover, but I’m sure that’s far rarer than human error).

There is somewhat of a jump from lean proof is correct to Mistral (sorry couldn't think of shittier model right now) reading and explaining this proof correctly.

Research programs have supposedly been gutted by these results. "Careers ruined," as the narrative goes. I think that is an exaggeration or overreaction. There are thousands of mathematicians and their interests are very wide ranging. The odds of someone's research program or problem being "solved" by AI are very small.

I suspect you're wrong; I get the sense that the majority of the projects worked on by today's most productive research mathematicians have been very significantly disrupted by OpenAI's public releases. You can get a sense of the scope of these results by skimming the collection of mathematicians' reactions compiled here.

I don't see anything changing much. The status quo of journals and academia will remain intact even with rapid progress in AI, save for the occasional breakthrough in an unsolved problem.

Again, I suspect you're wrong; I expect the most important math journals will all pivot now to publishing primarily expositions of computer-generated results, which is a major shift. Soon, the expositions themselves will be computer-generated, too. As far as academia, the entire systems of reward, hiring, and promotion for research math will be unsettled (each of these systems is currently focused on the ability of individuals to develop proofs of mathematical statements).

A math PhD program, especially from a good school, teaches or guides future researchers towards productive areas of research that are of interest to the "math community," rather than results that are merely "personally interesting". AI cannot recreate such communities or networks.

I think you're making the point here that research math isn't solely about problem solving (proof development), and that exerting social influence over (e.g.) area focus and direction is similarly important. That's clearly true. Gauss wrote: "I confess that Fermat's Last Theorem, as an isolated proposition, has very little interest for me, because I could easily lay down a multitude of such propositions, which one could neither prove nor dispose of" - which I take as suggesting that the prominence of Fermat's Last Theorem owes as much to historical and social factors as it does to any inherent importance of the statement. But so what? Yes, the emphasis placed on particular problems emerges as a consensus across the work of mathematicians operating as a community. But this emphasis has no real bearing on whether humans or machines are better-positioned to tackle such problems, which was previously only ever the work of individual humans.

I suspect many of these successful proofs were prompted by experts who knew what to look for, rather than just naively trying. What is touted by the media as an AI breakthrough, may involve surgically precise prompting in the background by an expert in said field.

I think this is a fair point. At least right now, a lot of the success of the models in solving major math problems may owe to effective interactions with human experts in the form of seasoned prompting. We honestly don't know how significant the human collaboration was to generating these results. Many mathematicians have called for OpenAI to share these details.

Alluding to the "airplane meme," there is also survivorship bias: you don't see all the failed attempts with AI . Sometimes it works great, but other times hit or miss.

I don't understand this point. Certainly we don't deride great (human) mathematicians this way (look at all the problems Gauss couldn't solve). Maybe you're suggesting that humans today can still more effectively resolve certain types of math problems than machines can? If so, what evidence do you think supports this idea?

Again, I suspect you're wrong; I expect the most important math journals will all pivot now to publishing primarily expositions of computer-generated results, which is a major shift. Soon, the expositions themselves will be computer-generated, too. As far as academia, the entire systems of reward, hiring, and promotion for research math will be unsettled (each of these systems is currently focused on the ability of individuals to develop proofs of mathematical statements).

This does not contradict my point. The journals still exist and are selective and important, but with AI playing a bigger role. But the vast majority of content, ai or not, will be rejected. AI has the effect of raising the bar of quality because now people can use AI for help.

ertainly we don't deride great (human) mathematicians this way (look at all the problems Gauss couldn't solve). Maybe you're suggesting that humans today can still more effectively resolve certain types of math problems than machines can? If so, what evidence do you think supports this idea?

My point was there are tons of prompts that fail to yield sought solutions. so no one reports it or write papers about negative results.

I may write a long post of my thoughts about this, because coding LLMs have also become very impressive (and reliable) very recently. In summary:

Research has stagnated, with low-hanging fruit claimed decades ago and researchers only making novel contributions in their late 20s (even many PhDs today aren't really novel). Academia has also become burdened with bureaucracy: bloated administration, "publish or perish", and "researchers" who game the system without producing anything worthwhile (because even those who do produce must game today's system a little). It has been dying long before LLMs.

LLMs open the door to a new realm of opportunity. I expect many more groundbreaking discoveries going forward, of people who apply them in creative ways and others who follow. Like the proliferation of computers or the Internet; most would agree LLMs are at least as big a milestone. And if academics play their cards right, a resurgence of impressive discoveries is exactly what will revive their public appearance and funding.

But LLMs also close the door to most of the current realm. Most work that would be PhD-worthy before is now saturated by LLMs, who can do years of grunt work in weeks (where lower quality is unnoticed or tolerated). We also don't know how much better they'll become; I suppose in math, they already surpass Tao himself. People who've devoted years of learning and effort, now finding that easy to replicate, are feeling left behind, and some will be left behind, because they have to start over...

...well, I really hope they won't have to completely start over: I believe they should get legacy funding and support, kind of like the grant given to Tao, even if their work is completely eclipsed by LLMs - the door to financial stability and recognition artificially held open for the last cohort. Because otherwise no-one will start future long-term projects for fear of getting burned. But obviously new PhD students should not do what LLMs have trivialized, or are likely to barring true ASI...which leaves open the question what will they do, but this is a question that has always existed, and which most spend their first years figuring out anyways.

As I've said before, LLMs may lead to utopia or dystopia, but I don't think of them as good or bad, and I don't think it's worth most people's thought, because they are practically inevitable. What matters is that they are transformative, so we should figure out what they may transform, how, and prepare. I wish academia would adjust: gift-wrapping up obsolete work, leaving those researchers with their currently-allocated finance and prestige (including tenure) but expecting them to try to transition into something new...but unfortunately I slightly expect it to continue as-is, with funding for an unreasonably long time (because it's so entrenched), but that may not be long at all, and/or what it becomes to maintain the funding may make today (or 6 months ago) look like the ideal of pure scientific advancement.

Anyways, how may human-directed research, and math in particular, evolve? I'm almost certain it will stay and actually think it will grow. This is what I think is actually interesting (whereas, is any of the above unexpected?), but also more speculative and philosophical, so I have to think and plan it a bit longer.

I suppose in math, they already surpass Tao himself.

What's so great about Tao, anyway? There's shitload of stuff named after Euler or Gauss, what's the most important "Tao theorem" or "Tao identity"? What has he discovered that people should care about?

I suppose in math, they already surpass Tao himself.

Apropos of nothing, on this very thread a few months ago people (@Soteriologian ) were pooh-poohing the capabilities of AI:

https://www.themotte.org/post/3771/culture-war-roundup-for-the-week/448459?context=8#context

Soteriologian Unstammering Papageno 4mo ago
I like the Harvard analogy because this is surely the intent with Terence Tao. There's a high chance sooner or later Terence Tao will prove something cool "using" ChatGPT, and if he does, it would be really awesome if we could make it sound like the secret ingredient in the ChatGPT-Terence Tao alliance was ChatGPT, when obviously the actual secret ingredient is Terence Tao.

Not so obvious now, is it?

Oh, how the tables turn...

My very loose understanding is that your Euler and Gauss came about in an earlier period of Mathematical ability on the part of the series and thus could 'pick more fruit' by want of more lower-hanging fruit and less competition. Tao's very impressive for coming through in a period where crossing between different subclades of Mathematics has become very very rare and managing to push a bunch of different areas forward even if not as transformatively overall as a Euler or a Gauss. Tao's also known for being very capable of communicating like a normal human being for somebody of his mathematical horsepower and therefore tends to be a lot better at collaborating with other mathematicians than a lot of other 'very brilliant but spiky and 8SD autist' mathematicians at that end of the spectrum of human ability.

Hypothetical 'Terrence Tao is born in Euler's Basel at the same point as Euler was and vice-versa' stuff is always going to be hard to argue/measure, too

He’s widely considered the best living mathematician, really he’s the most famous and certainly one of the best.

The Green-Tao theorem proves there are arbitrarily long arithmetic progressions of primes, and an arithmetic progression is a sequence of numbers with constant difference, like (3, 7, 11) and (14, 19, 24, 29). He made progress in many other unsolved conjectures, including Collatz and Navier-Stokes.

He did manage one practical contribution: reducing lung MRI scan times. His work may eventually also improve cryptography (I remember this SMBC comic featuring Tao and already improved cryptography, but it’s about how someone else’s theoretical contribution found practical applications).

There's the Green-Tao theorem, which was probably the number theory finding of the decade when it came out: https://en.wikipedia.org/wiki/Green%E2%80%93Tao_theorem

I think most maths are going to ai simply because it’s really not worth the effort to keep legacy humans around. I can understand “human in the loop” in life/death decisions, but I don’t see why anyone would want to bloat a university with an entire staff that can be replaced with a laptop and openAI.

One of the problems of academia is that the amount of material to review and understand has absolutely exploded. This might be merely annoying in, say, literature, but it's ideal in history and obviously potentially very important in medicine to be able to wrap your head around all the research that's been done ahead of you.

I think LLMs are in many ways a potential lifeline out of this dilemma that we've created for ourselves. But in a very real sense I am worried that they will work too well, sucking all of the funding away from experimentation and pushing boundaries and redirecting it into using LLMs to make new connections in already published data. This might yield incredible results, but if we don't maintain empirical testing, at some point we'll hit a dead end, and my concern is that we won't hit that dead end until after we've lost the cultural knowledge of how to do empirical testing.

This is particularly important because, due to the replication crisis, we could make vast forays into realms of theory without ever realizing that our work was built on a lie if we don't bother to verify the experiment.

However, this does suggest another potentially useful feature of LLMs for science beyond using them to correlate all the contents of the world: an "auto-replication factory" where some university has an agentic swarm review new publications. Obviously you'd need a lot more than just an Astra swarm to test the more hands-on stuff, but I bet AIs are decent at catching statistical anomalies, mistakes, fraud, and plagiarism.

Thanks for starting a thread about this!

Okay, so LLMs couldn't do arithmetic but then they could.

Then you could hand them a smartphone pic of a college textbook problem plus your answer and they'd tell you what the right answer is and tutor you on solving it if your answer was wrong.

Now they're cranking out proofs of open problems experts struggled with for years, and the complaint is the proofs aren't elegant and experts can't follow them?

Every step of the way someone said "this is the ceiling", so what's the specific limit you think kicks in right now, and why here?

Maths is readily verifiable. It's an area where humans struggle, not being something we evolved specifically to do. This is the most exposed salient in human experience, the least defensible terrain.

AI is going to push further ahead in maths. Only this year has AI been capable of introducing any major novel mathematical discoveries whatsoever. Only in the past month has it gotten to this level where they can fire off the slop cannon with abandon, those 700 results were within normal-people token budgets, not 'gigantic agent swarm' budgets. They said it was roughly 3 hours of GPT-6 Pro worth of compute on an internal model for each of those, on average. Next month? Next year?

Tokens will get cheaper, models will get smarter. The AI companies are mostly focusing their resources on more pressing issues anyway, the practical mathematics of coding, AI inference and training. They've been making continual progress there too.

So what if AI currently makes these dumb mistakes like getting a sign wrong. I had an issue just the other day where it was taking a screenshot and finding that there was no oscillation in the animation (but a screenshot is not going to help here, being static) and so I had to tell it to use another method of testing for the issue. If I were Yann LeCun or Gary Marcus or whoever, I'd say this proves they have no world model, no common sense, it's all a scam, LLMs are a false path... But they get better over time, month by month.

Let's at least wait for a full year of AI plateau before setting out who will coexist with AI!

I was hoping someone would post about this. I have a few reactions.

  1. Solving math problems is a fairly low risk way to demonstrate Ai models ability to solve these sorts of problems. I can’t be the only one to wonder: if it can solve obscure math problems, I wonder what it could do with physics? Apparently nothing really changes when you solve these 700 math problems. But if you solve 100 physics problems, you may wind up with some commercially valuable or dangerous tech. Seems unlikely they would dish it out so readily. People seem to have made this assumptions that we’ve hit the limits of new physics. Anyone claiming interesting effects or results is a crank and it’s all fantasy. Perhaps that assumption will hold up with AI, but perhaps not. It seems to me there’s a chance that the breakthroughs have already happened this year, but they haven’t been released yet.

  2. The reactions from many mathematicians have been a sight to behold. These sorts of intelligent academics truly behave like the world owes them something. By and large they are an unsympathetic bunch that I personally will enjoy watching them be made obsolete. There was one guy on Reddit, who turned out to be a young postdoc, giving the same tired routine that I see from so many of them. Bemoaning way that OpenAi released the work. Complaining that it’s a pile of papers that he will never read because it would take too long to digest and he’s got more important things to do. That perhaps if open Ai went through the proper channels and released peer reviewed articles in a real academic journal then, and only then, would the academic community accept the ideas. Maybe I’m generalizing too much from one guy, but I think this is a common refrain from these academic types.

I can’t wait to see how physicists react when the time comes.

But if you solve 100 physics problems, you may wind up with some commercially valuable or dangerous tech.

Depends very much on the kind of physics. Suppose LLMs find a consistent theory of the four fundamental interactions which agrees with observations. While important in some ways, this will not lead to a superstring bomb in all likelihood, at least not within a decade (at non-singularity speeds).

By contrast, I would claim that most experimental physics problems are at their heart engineering problems. Going from Hahn's discovery of fission to reactors and nukes was mainly an engineering problem (though it did require inventing some new physics along the way). Controlling fusion power is mainly an engineering problem. So is building LHC or LIGO. My experience is that the new physics being investigated (e.g. the Higgs) is the tip of the iceberg, with most of the work being in the underwater part -- detector physics, vacuum and cryo tech, magnets, control systems, calibration procedures and so on.

Rule of thumb for fundamental physics: if your experiment can easily be built using off-the-self components, then someone would already have done it. With all of the instrumental physics/engineering required to get to your physics of interest, LLMs can help but will not accelerate the process by 10X, I feel. Take the Helium leak detector, for example. A good AI might come up with the idea of using noble gas mass spectroscopy for leak detection on its own, and it can help you design the MS, but likely most of the challenges are practical ones where you simply need to build a prototype and test if it works.

People seem to have made this assumptions that we’ve hit the limits of new physics. Anyone claiming interesting effects or results is a crank and it’s all fantasy.

I think that we have definitely picked the low-hanging fruits of our current paradigms. Henri Becquerel discovered radioactivity in 1903 (using COTS components no less -- photography was already mature by then). This kicked off a series of discoveries which lead to the standard model, nuclear power, etc. The last piece of the puzzle so far was discovered in 2012 at LHC, at costs absolutely astronomical compared to what earned Becquerel his Nobel.

I think the physics community has roughly the right priors on someone making a revolutionary experiment in their garage. And even if individual members are not always amendable to evidence, eventually the physicists would come around. If your cold fusion is providing energy to cheap to meter, if your conservation-of-momentum-violating space drive is getting you to Mars, the physicists will be very interested in figuring these out. The problem is that so far, it seems like the alternative proposals are exclusively by cranks and snakeoil salesmen.

I appreciate the perspective and it’s a good explanation for why we don’t expect paradigm changing physics to come from outside the official research/experimental channels.

I don’t think these prior should be so strong for advanced AI models. I’m sure mathematicians did not expect the avalanche of proofs that came out. And that they could have given very detailed explanations of why it wouldn’t happen.

I don’t know if GPT 6 or 7 will crack some novel physics. But i suppose I think it’s approaching more likely than not.

The reactions from many mathematicians have been a sight to behold. These sorts of intelligent academics truly behave like the world owes them something. By and large they are an unsympathetic bunch that I personally will enjoy watching them be made obsolete.

Yeah, you see a lot of terrible takes from the list of reactions @ebrso linked to earlier. But on a positive note, there are also some really mature and grounded posts. For instance, this one from Johannes Schmitt:

One thing to keep in mind about this drop of papers is that they were not created by OpenAI with the main goal of making progress in mathematical research, or even primarily for PR (there were some PR posts, but e.g. the final announcement was comparatively restrained). The main reason these papers exist is that OpenAI used unsolved math problems to evaluate their internal models for the purpose of improving their capabilities. They had to use hard open problems because these are the only ones still posing a challenge to frontier AI models. I believe that, from OpenAI’s point of view, the papers are essentially a byproduct of their internal development.

Thus the company could just have remained silent, or picked a few flashy results for another blog post. I think it is to their credit that instead they reached out to the math community for guidance on how to proceed, leading to the formation of the Advisory Group on Mathematics and Artificial Intelligence. They followed some of the advice by this group, such as a timely release and (partial) Lean certificates, but not all of it, and in particular announced that they will continue to use open math problems for model development. In my view, the most pressing need of the math community right now is to have a central platform to discuss the results, start digesting the (by multiple accounts from colleagues, terribly written) papers, and report any mathematical mistakes or missing attribution. An OpenAI-controlled GitHub repository, with issues and pull requests disabled, cannot serve this purpose, and the ball is in OpenAI’s court to move the papers to a neutral repository and to back such a community effort with the promised funding for workshops and special programs.

This is absolutely correct. These theorems are a byproduct of OpenAI's work, and the only reason they published them at all was to try to share the knowledge with the (surprisingly ungracious) math community. OpenAI is not expecting professional mathematicians to clean up their work and then profit. They simply noted "hey, a decade ago mathematicians would have eaten their own limbs to see a glimpse of some of these fledgling proofs, so even if we don't have the time to clean them up, we should at least give them out for free". The correct response to that should be gratitude.

At the very least I think prompts should be disclosed . This is a good compromise between Ai math and purists. Did the user start with a blank canvas, or had some idea of where to begin, and then guided the AI accordingly?

But if you solve 100 physics problems, you may wind up with some commercially valuable or dangerous tech.

Yeah, apparently the Teller-Ulam design for a Hydrogen bomb relies on some clever insights and is not fully public. So it seems plausible that there are some really dangerous ideas out there, ready for an AI to discover.

It seems to me there’s a chance that the breakthroughs have already happened this year, but they haven’t been released yet.

Agreed.

Maybe I’m generalizing too much from one guy, but I think this is a common refrain from these academic types.

Well I think they are used to owning mathematics like a medieval guild. Which makes sense (or at least used to make sense) given that you need gatekeepers to keep out idiots and cranks.

I can’t wait to see how physicists react when the time comes.

FWIW I have a professional physicist in my family and he is pretty much resigned to the idea of studying physics out of love for learning more about the way the universe works. Which I guess is a good attitude, but if it were me I would be kind of disappointed if I suddenly had effectively no chance of discovering and publicizing something significant. Because the AI is going to beat me to the punch.

I suppose one option available to physicists (but not available to mathematicians) is to keep their experimental data to themselves for a year or two so that they have a chance to analyze it with their own AI in a proprietary way.

I suppose one option available to physicists (but not available to mathematicians) is to keep their experimental data to themselves for a year or two so that they have a chance to analyze it with their own AI in a proprietary way.

I'm not sure if that would be enough. Einstein famously didn't need experimental data to come up with relativity, he just thought about it really hard.

Einstein famously didn't need experimental data to come up with relativity, he just thought about it really hard.

That's the exception rather than the rule. I've heard it said, for example, that science owes more to the steam engine than the steam engine owes to science.

We still don't really know how many pharmaceuticals work.

In general, theoretical knowledge flows from experimental data and not vice versa. And even Einstein was not operating in a vacuum.

That’s because throughout almost all of human history, technology preceded science. One of the prevailing secular dogmas people have in the 21st century is the assumption that science drove technology, when in reality it’s the other way around; and science comes in after the fact and explains how it works.

As a side note to Einstein, not many people know among relativity experts, aspects of his theories are still hotly debated in niche areas as it relates to other aspects of physics relativity is related to. Just as there are a dozen or so some odd interpretations of quantum mechanics, there are different interpretations of general and special relativity as well; especially when you get to debates about cosmic time and absolute simultaneity. The tensed, neo-Lorentzian; “A-theory” interpretation of special relativity overcomes some of the very difficult problems implied by the EPR paradox in ways the orthodox B-theory interpretation we use today doesn’t.

IIRC, Einstein didn't win the Nobel Prize for relativity because many people at the time considered it to be in the realm of philosophy rather than science.

As a side note to Einstein, not many people know among relativity experts, aspects of his theories are still hotly debated in niche areas as it relates to other aspects of physics relativity is related to. Just as there are a dozen or so some odd interpretations of quantum mechanics, there are different interpretations of general and special relativity as well; especially when you get to debates about cosmic time and absolute simultaneity. The tensed, neo-Lorentzian; “A-theory” interpretation of special relativity overcomes some of the very difficult problems implied by the EPR paradox in ways the orthodox B-theory interpretation we use today doesn’t.

I certainly didn't know that!

I'm not sure if that would be enough. Einstein famously didn't need experimental data to come up with relativity, he just thought about it really hard.

FWIW that's not what I was taught in my history of science class in college. What I was taught was that in the late 19th century experimental physicists did what was known as an "ether drift experiment" and famously found that light seems to travel at the same speed regardless of the motion of the reference frame. And that subsequently Einstein started with this assumption and worked out the seemingly bizarre theoretical implications. Is this an accurate account? I am always skeptical of what I am taught in school, but it seems plausible to me that you can't do physics in a vacuum, so to speak. Even Isaac Newton had to start by looking at the motion of the planets and earth-bound objects, kind of a rudimentary form of experimental data.

Arguably the experimental data of today is more easily gate-kept. It's the observations of some orbiting telescope or readouts on some huge particle accelerator. Even so, if most of what the physicist is doing consists of feeding the experimental data into the AI and asking "Is there anything interesting in here? If so, write it up in a paper," it's, well, not quite the same as starting with the ether drift result and putting together a theory of relativity.

No experimental data is overstating it, but he didn't need to collect any fresh data tailored to his ideas. Someone else (presumably) would have eventually gotten to relativity if they ran a series of experiments narrowing down what the heck is happening with time and lightspeed.

He simply figured it out with the data from experiment N-1 (or maybe N-2 or N-3), where a "normal" researcher would need the data from experiment N. If a future AI can pull off the same trick but better, then researchers will need to stay more than one dataset ahead of the public releases if they want to make a discovery first.

No experimental data is overstating it, but he didn't need to collect any fresh data tailored to his ideas. Someone else (presumably) would have eventually gotten to relativity if they ran a series of experiments narrowing down what the heck is happening with time and lightspeed.

He simply figured it out with the data from experiment N-1 (or maybe N-2 or N-3), where a "normal" researcher would need the data from experiment N. If a future AI can pull off the same trick but better, then researchers will need to stay more than one dataset ahead of the public releases if they want to make a discovery first.

I basically agree with this but my impression is that modern experimental data is far more expensive and time-consuming to obtain than the data of the past.

That would make the gains to intelligence more important, not less.

If you can use the data from the Large Hadron Collider to make findings that most researchers would need the Future Circular Collider (its proposed successor) to find, then you've saved about $10B. Decades ago you couldn't have saved that much because people didn't spend that much.

That would make the gains to intelligence more important, not less.

I think this is probably correct, but my point is about a different issue, which is the issue of an AI company swooping in, making the discoveries, publishing them, and foreclosing the opportunities of the human researchers to make and publish the discoveries. This seems to be what is likely to happen in mathematics, if it's not happening already.

Arguably, with physics one could prevent this kind of scoop by keeping the data close to the vest. I'm pretty sure that even before AI got big, there were blackout periods with astronomical data. Although one can ask whether the situation is really that much better for individual researchers if most of what they are doing is feeding the proprietary data into their own AI.

But to your point, yeah, it seems that for scientific progress as a whole, AI analysis of experimental data would make things faster and more efficient.

What I was taught was that in the late 19th century experimental physicists did what was known as an "ether drift experiment" and famously found that light seems to travel at the same speed regardless of the motion of the reference frame. And that subsequently Einstein started with this assumption and worked out the seemingly bizarre theoretical implications.

I also recall being taught in school that Einstein's development of the postulates of special relativity (particularly that the speed of light is equivalent in all inertial reference frames) was influenced by the results of the Michelson-Morley Experiment, which failed to identify expected changes to the measured speed of light when the Earth's motion changed relative to the position of the sun.

However, this discussion suggests that Einstein's development of special relativity was probably not significantly influenced by the MM experiment, and was mainly influenced by assymetries in classical electromagnetism.

I also recall being taught in school that Einstein's development of the postulates of special relativity (particularly that the speed of light is equivalent in all inertial reference frames) was influenced by the results of the Michelson-Morley Experiment, which failed to identify expected changes to the measured speed of light when the Earth's motion changed relative to the position of the sun.

However, this discussion suggests that Einstein's development of special relativity was probably not significantly influenced by the MM experiment, and was mainly influenced by assymetries in classical electromagnetism.

Thanks for posting this. It's perspectives like this that make me skeptical of the pat facts I was sometimes taught in school. To be sure, it's possible that Einstein was mistaken or even lying about the influence of ether drift experiments on his thinking.

you don't see all the failed attempts with AI .

We have gotten a little bit of insight into what the failed attempts look like - OpenAI had to withdraw three of the math papers due to a sign change error that, I've read, is easily spotted by other released models.

I draw two conclusions from this:

  • OpenAI's internal models aren't free from making even very simple mistakes
  • OpenAI is, for some reason, rushing to shovel these out the door with very little review

We have gotten a little bit of insight into what the failed attempts look like

My (admittedly baseless) theory is that they went "Here's a $10k budget for each of the problems, have at it". Many runs ran out of tokens and failed, and the ones that succeeded did so early in their runs and cost an average of $3k.

They're presenting it as an average cost of $3k during a successful run, but that isn't the same thing as the cost to find a successful result. If I'm right, their statements all pass the "very seldom lie" test.

oh, by "fail" I mean prompts which don't give proofs. The Ai tries and fails to prove what the user intended.

Sure. And yet, if I had to choose between the model used to generate the results and all living mathematicians, I'd choose the model.

Now imagine what next year's model with do.

I'm not a mathematician, but if I were one, a significant one of my goals and motivations would be to find an elegant proof/disproof for some conjecture which is fairly widely known and unresolved. I would like the satisfaction of having solved a significant problem which nobody (no person or artificial entity) had solved before and I would also like the respect from my peers for having been first to come up with my elegant proof/disproof.

Do you think that AI threatens to take this possibility away in the near future? I'm not an expert, but it seems to me that the answer is "clearly yes."

As alluded to above, I work in a different field. A few years back I started a business with a creative idea, put the business together, encountered a decent amount of success and eventually sold for a substantial amount of money. Not tech billionaire level wealth, but still a respectable amount. I feel pretty good about the whole experience and I worry that the next generation probably won't have that kind of opportunity. Because AI will be just a lot better at coming up with creative ideas and then executing on them.

I feel like we humans are basically John Henry facing off against the steam shovel. Yes, we can put up a romantic fight, but at the end of the day it's a losing battle.

I feel like we humans are basically John Henry facing off against the steam shovel. Yes, we can put up a romantic fight, but at the end of the day it's a losing battle.

Come to think of it, I've consumed a lot of media about this topic, and for some reason I think this is the first time I've seen John Henry brought up. Even though it's kind of the obvious parable...?

Come to think of it, I've consumed a lot of media about this topic, and for some reason I think this is the first time I've seen John Henry brought up. Even though it's kind of the obvious parable...?

I would guess that over the years, the legend of John Henry has become more of a racial thing than a man vs. machine thing. Which is really a shame in my opinion, because I think his value as a folk hero vastly outweighs his significance as a black person.

Come to think of it, I've consumed a lot of media about this topic, and for some reason I think this is the first time I've seen John Henry brought up. Even though it's kind of the obvious parable...?

Perhaps because those of us who have thought of it prefer not to bring it up.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats