This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Dead forum?
Today in AI news, OpenAI has reportedly released GPT-6 Astra. Rumors on X suggests that it scores 99% on Arc-AGI-3, meaning that the benchmark is essentially dead now. For comparison, Fable only scored 30%. Does this mean AGI is here? Many are saying that it is. Color me skeptical, but its getting harder and harder to find concrete tests that AI cannot pass. The god of the gaps gets smaller.
Many intelligenct people have important things to say about this. But what do the 84 year old communists think? Bernie Sanders today has introduced legislation calling for a ban on superintelligence with a 20 year prison term for attempted superintelligence. He also wants (shocked Pikachu) a government regulatory agency. Although he mumbles something about international cooperation, its clear that China will not be subject to these rules so it solves nothing.
I will give him credit for at least recognizing the importance of the issue.
We could be entering a world of unimaginable change. And, given the political systems in place in the US and China, the midwives of that changes will be those on the cusp of senility.
With all the renewed hype around Astra and AI mathematics, I have been feeling that I haven't really seen a response I'm satisfied with from the AI bulls (at least the imminent mass unemployment / AGI / singularity types) to the 80IQ skeptic question of "If AI is so good, why is nothing happening?".
FWIW, if I knew in 2020 that in a few years we would have AI models that can get a perfect score on the IMO for pennies and meaningfully contribute to research mathematics I have expected that we basically live in a sci-fi world as well; yet now we have these models for $20 a month, and literally nothing I'm aware of has happened on a macro level clearly attributable to AI, either quantitatively or qualitatively, apart from the distortion of the AI capex buildout. In fact, it's actually GLP-1's that are basically a complete miracle on a macro-economic and public health level, and where there actually are many macro statistics that are visibly affected by GLP-1's.
On some level there probably is anthropomorphization going on here where the natural human assumptions are that
when in fact the conclusion does not actually follow from the premises; it is possible that AI is extraordinarily economically valuable and powerful for other reasons, but this does not follow from being superhuman at mathematics.
Modern pure mathematics is, as we seem to be finding out, not particularly economically valuable in and of itself; mostly mathematicians are lionized for their intellect because a human with that level of intelligence could obviously do significant amounts of other economically valuable work, but this seems extremely non-obvious in the case of AI.
The "best" response I've seen is something along the lines of "actually AI is still heavily limited by spikiness right now but the exponential is going to keep going bro" which I suppose is honest as to current capabilities and not disprovable, but it does feel like a retreat away from "AI is already crazy good" as many people have been and are touting, and towards magical thinking; somewhere along the line the party line swapped from "AGI imminent" to "AI that creates AGI imminent".
My take-away certainly isn't that AI is a fugazi; obviously it's powerful technology and I'm still quite bullish in the mid-long term, but I've certainly been feeling myself getting anti-hyped with recent releases as a result of this growing discontinuity of any real-world effects.
One thing I've noticed is that a lot of these mathematical breakthroughs involve lower bounds, upper bounds, or counter examples, and they usually involve the LLM interfacing with some kind of deterministic tooling.
Sometimes I wonder how much of this is the LLMs themselves, and how much is what happens when mathematicians finally get access to hundreds of billions of dollars in computing power, albeit through a weird interface.
Would we have 5% of these breakthroughs with equivalent computing power without LLMs? 50%? 90?
I don't know if I have a good answer, but I do wonder sometimes.
FrontierMath Erdos tries to address these questions: https://epoch.ai/files/frontiermath-erdos.pdf
They enforce no human guidance and a limited token budget ($300). Under these conditions the only model that solved 2 problems is GPT-6 Astra.
With no budget limit GPT-6 Astra has solved 5 problems out of 68 (3 proofs, 2 disproofs).
The paper's conclusion: "These results establish, under controlled conditions, that a current AI system can autonomously resolve open problems of genuine mathematical interest."
And, well, there's no other known way (besides LLMs) to convert compute into solutions of mathematical problems in general. There are SAT solvers, but they are limited in their scope and, being a fixed collection of heuristics, they don't scale that well. Brute force search is usually out of the question. So, I would say it's about 0-1% without LLMs. And taking into account that LLMs are the major driver of compute buildup, we wouldn't have that much compute without them.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link