This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
On the plausibility of Mars Bases vs that of AI
Responding to @FeepingCreature from last week:
That was a continuation of this discussion in which I say of AI 2027:
As to Mars:
Most of what I know here comes from reading Zach Wiener-Smith (of SMBC)'s A City on Mars. It was wildly pessimistic. For a taste, see Gemini chapter summaries and an answer to:
"Given an enormous budget (10% of global GDP) and current tech, how realistic is a 1 year duration mars base? an indefinite one? what about with highly plausible 2035 tech?"
I agree with the basic take there, both as a summary of the book and as a reflection of my broader (but poorly researched) understanding/intuition of the area: Mars is not practical. We could probably do the 1 year base if we don't mind serious risk of killing the astronauts (which, politically, probably rules it out. Maybe Musk will offer it as a Voluntary Exit Program for soon-to-be-ex X SWEs?)
My main interesting/controversial (?) take: there is an important sense in which Mars bases are much less of baseless scifi nonsense than AI 2027.
Mars is a question of logistics: on the one hand, building a self-contained, O2 recycling, radiation hardened, etc, base requires tech we may (?) not quite have yet. On the other hand, it strikes me as closer to refinements of existing tech than to entirely new concepts. Note that "enormous budget" is doing a lot of work in here. I am not saying it is practical to expect we will pay to ship all of this to Mars, or risk the lives, just that there is good reason to believe we could.
AI is a question of fundamental possibility: by contrast, with AI, there is no good reason to think we can create AI sufficient to replace OpenAI-grade researchers with forseeable timelines/tech. Junior SWEs, maybe, but it's not even clear they're on average positive-value beyond the investment in their future (see my previous rant about firing one of ours).
I don't understand how anyone can in good faith believe that even with an arbitrary amount of effort and funding, AGI, let alone ASI, is coming in the next few years. Any projection out decades is almost definitionally in the realm of speculative science-fiction here. Even mundane tech can't be predicted decades out, and AI has higher ceilings/variance than most things.
And yet, I am sensitive to my use of the phrase "I don't understand." People often unwittingly use it intending to mean "I am sure I understand." For example: "I don't understand how $OTHER_PARTY can think $THING." This is intended to convey "$OTHER_PARTY thinks $THING because they are evil/nazis/stupid/brainwashed." But, the truth of their cognitive state is closer to the literal usage: they do not understand.
So, in largely the literal sense of the phrase: I do not understand the belief in and fear of AI progress I see around me, in people I largely respect on both politics and engineering.
You're just asserting this without providing reasoning despite it being the entire crux of your post. I know it's not reasonable to expect you to prove a negative but you could have at least demonstrated some engagement with the arguments those of us who think it's very possible near term have put forward. You can at least put into some words why you think AI capabilities will plateau somewhere before openAI-grade researcher. How about we find out where we are relative to each other on some concrete claims and we can see where we disagree on them.
Do you agree that capabilities have progressed a lot in the last few years at a relatively stable and high pace?
Do you agree that it's blown past most of the predictions by skeptics, often repeatedly and shortly after the predictions have been made?
Are there even in principle reasons to believe it will plateau before surpassing human level abilities in most non-physical tasks?
Are there convincing signs that it's plateauing at all?
If it does plateau is there reason to believe at what ability level it will plateau?
I think if we agree on all of these then we should agree on whether to expect AI in the nearish term, I'm not committed to 2027 but I'd be surprised if things weren't already very strange by 2030.
Then it's good the 2027 claim isn't projecting out decades.
We have absolutely no clue how to achieve AGI. Simply scaling existing methods, while potentially achieving impressive results, cannot achieve AGI. It is possible that emergent behavior on existing methods allows a specific non-AGI AI can become superhuman in a narrow field that allows it to solve AGI, but we have no reason to believe this is the case.
This reminds me of an old LW article: https://www.greaterwrong.com/w/the-rocket-alignment-problem
Without an understanding of orbital mechanics, it's impossible to reach the moon by just building a bigger rocket and pointing it towards the moon. We're over here on the ground building bigger and bigger rockets: chatgpt, gemini, whatever, but we have no idea what lies ahead on the path to AGI.
Why do you believe this? Is it an article of faith?
It seems like we absolutely do know what lies ahead on the path to AGI and it's incrementally getting better at accomplishing cognitive tasks. We have proof that it's possible too because humans have general intelligence and accomplish this with far fewer units of energy. You can, at this very moment, if you're willing to pay for the extremely premium version, go on chat gpt and have it produce a better research paper on most topics than, being extremely generous to humanity here, 50% of Americans could given three months and it'll do it before you're back from getting coffee. A few years ago it could barely maintain a conversation and a few years before that it was letter better than text completion.
This is rather like having that LW conversation after we'd already put men into orbit. Like you understand that we did actually eventually land on the moon right? I know it's taking the metaphor perhaps to seriously but that story ends up with Alfonso being right in the end. We can, in fact, build spaceships that land on the moon and even return. We in fact did so.
Now we have some of the greatest minds on earth dedicated to building AGI, many of them seem to think we're actually going to be able to accomplish it and people with skin in the game are putting world historical amounts of wealth behind accomplishing this goal.
Absolutely not. Deep research is a useful tool for specific tasks, but it cannot produce an actual research paper. Its results are likely worthless to anyone except the person asking the question who has the correct context.
If you build a bigger rocket and point it at the moon, it will get incrementally closer to the moon. But you will never reach it.
AGI is possible in theory but that does not mean it is possible with currently known techniques.
This clears the bar of most Americans.
If you have some of the smartest people in the world and a functionally unlimited budget you can actually use the information you gain from launching those rockets to learn what you need to do to get to the moon. That is was actually happened after all so I really don't see how this metaphor is working for you. The AI labs are not just training bigger and bigger models without adjusting their process. We've only even had chain of thought models for 6 months yet and there is surely more juice to squeeze out of optimizing that kind of scaffolding.
This is like claiming moore's law can't get us to the next generation of chips because we don't yet know exactly how to build them. Ok, great but we've been making these advancements at a break neck pace for a while now and the doubters have been proven wrong at basically whatever rate they were willing to lay down claims.
Speaking of claims you've decided not to answer my questions, that's fine, continue with whatever discussion format you like but I'd be really interested in you actually making a prediction about where exactly you think ai progress will stall out. what is the capability level you think it will get to and then not surpass?
Newton discovered calculus and gravity without seeing a single rocket.
Sounds like this should be easy then as we have seen some people who are smarter than others.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link