This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
First, points of presumed agreement: LLMs have significant gaps. My personal bugbear is sample efficiency. And I agree that claims that everyone (at least white collar workers, even those who aren't protected by regulation or custom) is going to be out of a job by 2030 are overblown. Intelligence is valuable, and human intelligence will remain valuable in the short term, even if LLM intelligence substitutes; there's simply not enough compute right now or in the pipeline to drop agents into every single white collar job. Even if there were, the methods used for gathering datasets for the shining stars of current LLMs (math; programming) don't naturally extend to other roles.
The massive short term threat, though, is RSI. Math and programming don't substitute for the majority of human skills, but they do substitute for ML researchers. Maybe more a bunch of bright grad students stumbling around in the dark than visionary geniuses, but grad student descent got us to where we are today. Throw a thousand, or ten thousand, or a million of them at the current bottlenecks in ML (IMO sample efficiency, but take your pick), and it seems quite likely to me they could open them, today. LLMs' flaws compared to the human brain are an algorithmic, not hardware, issue. That's the point here I'm least certain about, but if it's true, the training data and even compute bottlenecks themselves become much weaker constraints. A couple examples could become enough to train the AI to replace a role, and the compute needed to do it would drop even faster than it already is. Stronger agency may itself arise by itself from this. Robotics will be substantially slower (physical world and all, and the demand for capital to build robot factories will have to compete with other massive demands for capital), but still moving much faster than today.
It will make for an incredibly chaotic time, with lots of angry people. And AIs will be rapidly insinuating themselves into the economy, toward an end state of a time of wonders and complete human dependency on AI goodwill (a process that will take more like two decades than two years, but still incredibly fast in the grand scheme of things). Bad enough IMO, though I understand the appeal. But to people who see this as appealing: we have no idea of what the actual shape of the AIs that will arise from this process will be, and I don't want to bet the future on a gamble that they'll be in a friendly shape willing to care for their dependents indefinitely.
Not to nitpick completely, but there is no short term threat via RSI because RSI isn't here. Theoretically yes, if RSI existed, I think the AI models might begin to reach outside their current training datasets. But part of the cost of AI rn is compute for training. If it takes 100 Million worth of compute to train an LLM, I don't think a company is going to give an LLM-Agent the ability to just start up training runs for the "Self" part of RSI. RI already occurs but part of that limitation is that baked into the RSI argument, is the idea that the AI will make improvements that the human researchers can't understand. This presents a conundrum from a simple business economics perspective, are you the company going to risk billions of dollars on potential RSI runs which also could just as likely be dead-end hallucinations? You obviously can't understand the improvements the AI is trying to make, it's smarter than you. So it comes down to a matter of faith, is an AI lab going to take a leap of faith that "this training run is the RSI first domino" that will lead to an investment recoup to pay for the next RSI domino. We are not there yet.
Maybe, but that doesn't mean that algorithmic solutions are actually discoverable. This harken's back to the OG AI researchers trying to discover how humans did it quantitatively as a precursor to coding it into computers. They spent a lot of cash + time trying and got their lunch eaten by data-driven models. Now we've just proposed that we'll sick a hyper data-driven model on the same question. It really is just a shot in the dark. We think its possible because humans can do it, but LLMs don't work at all similar to human cognition so we don't have any evidence that they can achieve an algorithmic solution to their already existing problems.
Two different responses:
RSI is a continuum; AI companies will continue to use humans as long as they add anything to the process, but AIs will play a larger and larger role. Today, human researchers mostly provide taste; actually writing up JAX or GPU kernels to implement a research idea, on the other hand, is a place AI dominates humans. That human taste is incredibly valuable--likely allowing pruning the nodes of a research tree by multiple orders of magnitude--but that edge is temporary.
The actual research process would involve thousands of small, cheap experiments to find promising candidates; selecting those and scaling them up to bigger and harder experiments; and repeating. By the time you're up to multimillion dollar training runs, you already have a pretty solid idea of your candidates' scaling curves. To the extent it's a gamble, it's a gamble that you also have to make on a human-designed model. Humans will still make the ultimate decision to go ahead with something that'll cost 8 or more figures, but they'll quickly become rubber stamps.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link