This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Sooo, Big Yud appeared on Lex Fridman for 3 hours, a few scattered thoughts:
Jesus Christ his mannerisms are weird. His face scrunches up and he shows all his teeth whenever he seems to be thinking especially hard about anything, I didn't remember him being this way in the public talks he gave a decade ago, so this must either only be happening in conversations, or something changed. He wasn't like this on the bankless podcast he did a while ago. It also became clear to me that Eliezer cannot become the public face of AI safety, his entire image, from the fedora, to the cheap shirt, facial expressions and flabby small arms oozes "I'm a crank" energy, even if I mostly agree with his arguments.
Eliezer also appears to very sincerely believe that we're all completely screwed beyond any chance of repair and all of humanity will die within 5 or 10 years. GPT4 was a much bigger jump in performance from GPT3 than he expected, and in fact he thought that the GPT series would saturate to a level lower than GPT4's current performance, so he doesn't trust his own model of how Deep Learning capabilities will evolve. He sees GPT4 as the beginning of the final stretch: AGI and SAI are in sight and will be achieved soon... followed by everyone dying. (in an incredible twist of fate, him being right would make Kurzweil's 2029 prediction for AGI almost bang on)
He gets emotional about what to tell the children, about physicists wasting their lives working on string theory, and I can see real desperation in his voice when he talks about what he thinks is really needed to get out of this (global cooperation about banning all GPU farms and large LLM training runs indefinitely, on the level of even stricter nuclear treaties). Whatever you might say about him, he's either fully sincere about everything or has acting ability that stretches the imagination.
Lex is also a fucking moron throughout the whole conversation, he can barely even interact with Yud's thought experiments of imagining yourself being someone trapped in a box, trying to exert control over the world outside yourself, and he brings up essentially worthless viewpoints throughout the whole discussion. You can see Eliezer trying to diplomatically offer suggested discussion routes, but Lex just doesn't know enough about the topic to provide any intelligent pushback or guide the audience through the actual AI safety arguments.
Eliezer also makes an interesting observation/prediction about when we'll finally decide that AIs are real people worthy of moral considerations: that point is when we'll be able to pair midjourney-like photorealistic video generation of attractive young women with chatGPT-like outputs and voice synthesis. At that point he predicts that millions of men will insist that their waifus are actual real people. I'm inclined to believe him, and I think we're only about a year or at most two away from this actually being a reality. So: AGI in 12 months. Hang on to your chairs people, the rocket engines of humanity are starting up, and the destination is unknown.
Every discussion I've ever had with an AI x-risk proponent basically goes like
"AI will kill everyone."
"How?"
"[sci-fi scenario about nanobots or superviruses]"
"[holes in scenario]"
"well that's just an example, the ASI will be so smart it will figure something out that we can't even imagine."
Which kind of nips discussion in the bud.
I'm still skeptical about the power of raw intelligence in a vacuum. If you took a 200 IQ big-brain genius, cut off his arms and legs, blinded him, and then tossed him in a piranha tank I don't think he would MacGyver his way out.
I fully agree for a 200 IQ AI, I think AI safety people in general underestimate the difficulty that being boxed imposes on you, especially if the supervisors of the box have complete read access and reset-access to your brain. However, if instead of the 200 IQ genius, you get something like a full civilization made of Von Neumann geniuses, thinking at 1000x human-speed (like GPT does) trapped in the box, would you be so sure in that case? While the 200 IQ genius is not smart enough to directly kill humanity or escape a strong box, it is certainly smart enough to deceive its operators about its true intentions and potentially make plans to improve itself.
But discussions of box-evasion have become kind of redundant, since none of the big players seem to have hesitated even a little bit to directly connect GPT to the internet...
A shame these systems are notoriously black box. Even having full access to all the "data", nobody can make any meaningful sense out of why these AI's do any of the things they do in any sort of mechanistic sense. They can only analyze it from a statistical perspective after the fact, and see what adjusting weights on nodes does.
More options
Context Copy link
Well, I don't know. Maybe? I must admit I have no idea what such a thing would look like. My problem isn't necessarily the ease or difficulty of boxing an AI in particular, but more generally the assumption in these discussions which seems to be that any given problem yields to raw intelligence, at some point another, and that we should therefore except a slightly superhuman AI to easily boost itself to god-like heights within a couple seconds/months/years.
Like here, you say, paraphrasing, "a 200 IQ intelligence probably couldn't break out of the box, but what about a 10,000 IQ AI?" It seems possible or even likely to me that there are some problems for which just "piling on" intelligence doesn't really do much past a certain point. If you take Shakespeare as a baby, and have him raised in a hunter-gatherer tribe rather than 16th-century England, he's not going to write Hamlet, and in fact will die not even knowing there is such a thing as written language, same as everybody else in his tribe. Shoulders of giants and all that. Replace "Shakespeare" with "Newton" and "Hamlet" with "the laws of motion" if you like.
I'm not convinced there is a level of intelligence at which an intelligent agent can easily upgrade itself to further and arbitrary levels of intelligence.
(As a caveat, I have no actual technical experience with AI or programming, and can only discuss these things on a very abstract level like this. So you may not find it worthwhile engaging with me further, if my ignorance becomes too obvious.)
It doesn't need to have an advantage in 'any given problem', it just needs to be 'technological development, politics (think more 'corporate / international politics' than 'democratic politics'), economic productivity, war, and general capability', which history shows does yield to intelligence. The AIs just need to be better at that than us, at which point they could overpower us, but won't need to as we'll happily give them power!
To vastly outclass humans in 'technological development, politics, economic productivity, war, and general capability' I think an AI would actually need to have an advantage in any given problem.
I'm not sure I understand why? There are many problems of the form of 'reverse 100k iterations of SHA3' or 'what is the 1e10th digit of chaitin's constant' or 'you are in a straitjacket in a river, don't be eaten by piranhas'. And supersmart AIs probably can't solve those. But tech/politics/economics/war problems aren't like those! To an extent, it's just 'do what we're doing now, but better and faster'. The - well tread at this point - example is 'a society where the median person is as smart as John Von Neumann'. It's obviously harder than just cloning him a bunch of times, but assume that society would also have a small fraction of people significantly smarter than JvN. Would that society would have a significant military / technological / political advantages over ours?
Why is it always Von Neumann? Last I recall he was a physicist, not a brilliant leader, warlord or businessman who solves coordination problems.
More options
Context Copy link
If you want to do something 'better' or 'faster' you have to do it differently in some way from how it was being done before. If you are just doing the same thing the same old way then it won't be any better or faster. So an intelligence would have to make war, do politics, economics, etc. in a different way than humans do, and it's not clear that "just be smarter bro" instantly unlocks those 'different' and scarily efficient ways of making war, doing politics.
It is difficult to answer this question empirically but the only real way to do so would be to look at historical conflicts, where it's far from clear that the 'smarter' side always wins. Unless you define 'smarter' tautologically as 'the side that was able to win.'
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link