This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Sooo, Big Yud appeared on Lex Fridman for 3 hours, a few scattered thoughts:
Jesus Christ his mannerisms are weird. His face scrunches up and he shows all his teeth whenever he seems to be thinking especially hard about anything, I didn't remember him being this way in the public talks he gave a decade ago, so this must either only be happening in conversations, or something changed. He wasn't like this on the bankless podcast he did a while ago. It also became clear to me that Eliezer cannot become the public face of AI safety, his entire image, from the fedora, to the cheap shirt, facial expressions and flabby small arms oozes "I'm a crank" energy, even if I mostly agree with his arguments.
Eliezer also appears to very sincerely believe that we're all completely screwed beyond any chance of repair and all of humanity will die within 5 or 10 years. GPT4 was a much bigger jump in performance from GPT3 than he expected, and in fact he thought that the GPT series would saturate to a level lower than GPT4's current performance, so he doesn't trust his own model of how Deep Learning capabilities will evolve. He sees GPT4 as the beginning of the final stretch: AGI and SAI are in sight and will be achieved soon... followed by everyone dying. (in an incredible twist of fate, him being right would make Kurzweil's 2029 prediction for AGI almost bang on)
He gets emotional about what to tell the children, about physicists wasting their lives working on string theory, and I can see real desperation in his voice when he talks about what he thinks is really needed to get out of this (global cooperation about banning all GPU farms and large LLM training runs indefinitely, on the level of even stricter nuclear treaties). Whatever you might say about him, he's either fully sincere about everything or has acting ability that stretches the imagination.
Lex is also a fucking moron throughout the whole conversation, he can barely even interact with Yud's thought experiments of imagining yourself being someone trapped in a box, trying to exert control over the world outside yourself, and he brings up essentially worthless viewpoints throughout the whole discussion. You can see Eliezer trying to diplomatically offer suggested discussion routes, but Lex just doesn't know enough about the topic to provide any intelligent pushback or guide the audience through the actual AI safety arguments.
Eliezer also makes an interesting observation/prediction about when we'll finally decide that AIs are real people worthy of moral considerations: that point is when we'll be able to pair midjourney-like photorealistic video generation of attractive young women with chatGPT-like outputs and voice synthesis. At that point he predicts that millions of men will insist that their waifus are actual real people. I'm inclined to believe him, and I think we're only about a year or at most two away from this actually being a reality. So: AGI in 12 months. Hang on to your chairs people, the rocket engines of humanity are starting up, and the destination is unknown.
Odds are it's Adderall. Bay Aryan culture nerds (sounds like an anthropological term, right?) abuse «ADHD meds» to a degree far beyond Scott's «doing spreadsheets» apologia – it's their friend in need when they want to be on top of their game and really make an impression. They write terrible tweets on addy, go to podcasts on addy, livestream coding sessions on addy and make innumerate github commits that end up borking whole repositories on addy. The society needs to teach these people that addy doesn't actually make them smarter, it only makes them feel smarter and act more grandiose, which they get addicted to even harder than to the direct dopamine hit. Once again: nerds aren't all right; consider this paragraph a simulated example of how exactly. Acting like a hyperactive Loony Tunes or anime character is, well… loony.*
That said, I do not endorse the focus on the way Yud looks and acts or even whether he's a narcissist. He doesn't strike me as too unseemly for someone with his background; it incriminates the speaker more than Yud; and it takes away from substantial criticism.
In fact Yud, suddenly and unexpectedly for him cited as top AI researcher, prominent analyst etc. – is, himself, a red herring that distracts from the real issue. The issue being: a coordinated barrage of attacks on proliferation of transformative AI. I've compiled an incomplete chronicle of proceedings; some of those are obviously just journos latching on, but others had to have been in the works for months or at least weeks. This is some spooky bullshit – though, nothing new I guess, after all the shadowy campaigns to fortify the democracy and battling COVID misinformation/narrative shifts.
I think we are seeing ripples from a battle to capture the public endorsement for deceleration vs. unequal acceleration. With one party (which I associate with old-school paramasonic networks) being genuine «decels» who push crippling regulation using Yud and other useful idiots like EAs and assorted international organizations as a front; and the other being a fractured alliance of national, industrial and academic actors who want narrower regulations for all, displacement of the purported threat onto geopolitical and market competitors and open-source community, and token (from their perspective) conditions like ensuring that the AI stays woke for themselves; though I may be completely wrong with my typology. It's reminiscent of the rise of anti-Nuclear groups and Rome Club «limits to growth» fraudulent models, which then mutated into today's environmentalist degrowth movement (recommended reading on Yudkowsky as our era's Paul Ehrlich; do you like our success in defusing the Population Bomb?).
Anyway:
02/24: OpenAI releases the paper Planning for AGI and beyond which some conclude is unexpectedly thoughtful and the usual subjects (see LW/SSC) pan as not going nearly far enough.
03/12 Yud: « I'm at the Japan AI Alignment Conference, and they're introducing me as the eminent grandfather of the field of AI alignment…»
03/13: after weeks or running covertly under the hood of Bing Search, GPT-4 officially launches (sniping the leading safety-concerned lab Anthropic's Claude by a day).
03/15: Sutskever: «If you believe, as we do, that at some point, AI — AGI — is going to be extremely, unbelievably potent, then it just does not make sense to open-source. It is a bad idea... I fully expect that in a few years it’s going to be completely obvious to everyone that open-sourcing AI is just not wise». Reminder that in 2015, in the OpenAI still aligned with Musk, he signed this: «We believe AI should be an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as possible… it’ll be important to have a leading research institution which can prioritize a good outcome for all over its own self-interest… As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world.» The more cynical logic behind that OpenAI was one of «adversarial equilibrium», which I paraphrase here and which e/acc people articulate better.
03/25: Altman goes on Fridman's podcast, admits that much of work on GPT-4 was in alignment, and drops some of Yud's takes like «we don't know how to align a super powerful system». Lex and Sam discuss Yud's AGI Ruin, apparently, at 55 minutes mark, at 1:11 Sam suggests that opensource LLMs with no safety controls should be regulated away or detected through vetoed «aligned» ones. Note though that Sam is at peace with the idea that there will be multiple AGIs, while Yud thinks 1 is as many as we can afford (ideally 0 for now). Sam mentions Scott's Moloch at 1:16.
03/29: the UK Government publishes a recommendation document on AI policy «to turbocharge growth» called # AI regulation: a pro-innovation approach.
Future of Humanity letter signed by Musk, Harari, Yang etc; safetyists like Geoffrey Miller admit that 6 months is only to get the ball rolling.
03/29: the open-source AI org LAION petitions for the creation of a «CERN for AI research and safety», gathers signatures of random EU redditors.
03/29: Yud's childish piece in TIME Ideas where he suggests an international treaty to physically destroy datacenters training serious AIs. Interestingly I've heard that Time proactively reached him.
03/30: Yud hyperventilating on Fridman.
03/30, Fox News’ reporter uses all his time at the White House to advance Yud's ideas.
03/30 continues: «UNESCO Calls on All Governments to Implement AI Global Ethical Framework Without Delay»: «This global normative framework, adopted unanimously by the 193 Member States of the Organization…»
03/30 unlimited decel policy works: The DC tech ethics think tank Center for Artificial Intelligence and Digital Policy asks FTC «to stop OpenAI from issuing new commercial releases of GPT-4».
03/31: Italy bans ChatGPT over specious privacy concerns.
04/01, BBC: Should we shut down AI? Inconclusive.
04/02, fucking RussiaToday: «Shutting down the development of advanced artificial intelligence systems around the globe and harshly punishing those violating the moratorium is the only way to save humanity from extinction, a high-profile AI researcher has warned».
I see some AI researchers on twitter «coming out» as safetyists, adopting the capabilities vs. safety lingo and so on.
On the other hand, many luminaries in AI safety are quiet, which may be due to the fact that they've moved over to the OpenAI camp. Some are still active like Richard Ngo but they're clearly on board with Sam's «we'll keep pushing the frontier… safely» policy. Aaronson here too.
Another curious detail: in 2021, Will Hurd, «former clandestine CIA operative and cybersecurity executive», joined OpenAI's board of directors. I like this theory that explains the $10B investment into, effectively, GPT training as a spook/military program. I've also updated massively in favor of «Starship is FOBS system» arguments made previously by @Eetan, AKarlin and others.
All in all, it feels like the time for object-level arguments has passed. What decides the result is having connections, not good points.
* As a transhumanist, I understand the aspiration. But as someone who flirted with stims for a while and concluded that, in the long run, they very clearly only make me more cocksure, loquacious (as if any more of that were needed) and welcoming routine to the point of pathological repetitiveness, I endorse @2rafa's recent writeup. This isn't the way up. This isn't even keeping level with the baseline performance in interesting ways.
I have actual ADHD, so I need stimulants to be productive when actual executive function is needed (exams for one, and those never end for doctors). People with it supposedly benefit more from the drugs than neurotypicals do, even if they still improve the focus of the latter.
That being said, I strongly suspect that your negative attitude towards stim use in the Bay Area is simply selection effects. You notice the stimmed out people making a fool of themselves in public, and don't see the thousands-millions of others quietly using them at reasonable doses to boost their productivity. I'm going to go with Scott on this one, it fits with my own personal experience.
I don't often agree with daes, but I strongly endorse his description of the end-point of amphetamine use on people's thought processes. I find the idea that some people are magically different in this regard fairly absurd -- with the caveat that low doses probably aren't that bad and are more likely to be maintained as such when it's given as 'medicine from a doctor' than 'bennies from my dealer'.
But I don't have much confidence that EY (or SBF for that matter) have been particularly strict in keeping to low theraputic-type doses -- that's the whole problem.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link