This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Why entirely local? That is, as far as I can tell desirable rather than necessary, in a world with wireless internet connectivity. Your fridge might have an internet connection these days.
Waddle is an example. Their agents control robots through code and action models, build reusable skills, and even collect data to train smaller policies. Phillip Isola discusses the broader implications here.
I did say to give the robotics people a year. On reflection, I'd be willing to water that down to two or three. I'm not a robotics expert, and I certainly wasn't announcing that we've solved human dexterity. SOTA-Robotics:Human mobility is not SOTA-LLMs:Human Cognition, the latter is much closer, but it's trending the same direction.
But the possibility that robots inherit substantial capabilities from each improvement in frontier models seems rather consequential to me.
A Culture Mind is a hell of a minimum specification. Two of the three examples are ASI, not AGI. I haven't called Astra an ASI.
And fictional examples don't give us an empirical definition. Which capabilities are required? How reliably? Why should I care whether it has emotions when assessing whether it can do my job?
“It's only able to really solve problems” is a rather confusing counter-argument. Depends on the problems, surely. Likewise, working towards an assigned goal doesn't tell us much about how general the intelligence doing the work is. I spend most of my working day performing tasks I am assigned, that do not intrinsically motivate me, mostly because I get paid for it.
You're conflating training the model with teaching the trained model something new. I don't start from a blank brain when learning a game either.
ARC Prize tested Astra on unfamiliar interactive environments. It scored 62.7% with their standard interface, and 99.9% with an interface preserving its reasoning state. In the latter setup (at maximum reasoning effort) it used fewer actions than the median successful human on 96% of levels.
That isn't a comparison of total training data or energy efficiency, which wouldn't necessarily be the decisive factor:
Let's say a hypothetical architecture X is 1000 times less sample efficient than a human for a given unit of performance - well, it would be unfortunate if we managed to give it 10,000 times the training data a human can ingest, wouldn't it?
A mere year or two ago, a lot of people were awfully confident that we'd run out of training data, or that synthetic data and RLVR wouldn't pan out. God knows what the frontier labs are up to these days, but it has clearly not proven an impediment.
It does make “these things need millions of examples to learn something new” rather difficult to sustain, regardlws sof practical relevance - which I dispute. They observed it figuring out unfamiliar mechanics and constructing symbolic models to plan around them. Your claim about analogical reasoning needs similar qualification. I do not believe they're the same thing anyway.
Proper continual learning remains a significant advantage. I said so. Whether that advantage remains economically decisive as scaffolding improves is what I'm uncertain about. I lean towards a no.
Alternatively, we could solve continual learning, or simply reduce the temporal delta between train-deploy-train to the point that it has no practical relevance.
A model contributing original mathematics, writing software and learning unfamiliar environments already exhibits an awful lot of generality. It can even hold a conversation, which, I will note, would have been mind blowing not that long ago.
Call it AGI or don't. A rose by another name smells just as sweet, and has just as many thorns.
My concern is how much useful work it can do, how quickly that range is expanding, and what remains exclusively ours. I don't expect the terminology to buy us much time. I certainly don't want to spend more time arguing terminology.
More options
Context Copy link