This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I've been asked to repost this in the Culture War thread, so here we go.
I read this story today and it did amuse me, for reasons to be explained.
Fear not, AI doomerists, Northrop Grumman is here to save you from the paperclip maximiser!
I am curious if this is the sort of response the AI safety lobby wanted from the government. But it also makes me think in hindsight, how quaint the AI fears were - all those 50s SF fever dreams of rogue AI taking over the world and being our tyrant god-emperor from Less Wrong and elsewhere, back before AI was actually being sold by the pound by the tech conglomerates. How short a time ago all that was, and yet how distant it now seems, faced with reality.
Reality being that AI is not going to become superduper post-scarcity fairy godmother or paperclipper, it is being steered along the same old lines:
War and commerce.
That's pretty much how I expected it to go, more so for the commerce side, but look! Already the shiny new website is up! I can't carp too much about that, since I did think the Space Force under Trump was marvellous (ridiculous, never going to be what it might promise to be, but marvellous) so I can't take that away from the Biden initiative. That the Department of Homeland Security is the one in charge thrills me less. Though they don't seem to be the sole government agency making announcements about AI, the Department of State seems to be doing it as well.
What I would like is the better-informed to read the names on lists being attached to all this government intervention and see if any sound familiar from the EA/Less Wrong/Rationalists working on AI forever side, there's someone there from Stanford but I don't know if they're the same as the names often quoted in Rationalist discussions (like Bostrom etc., not to mention Yudkowsky).
Related, how long do I have to wait before I can start calling LLMs a nothing burger? Everything that has come out of it seems so small and near-pointless. Marginal productivity increases at best. When does the fun stuff start happening?
Nothing but... entire countries catching their corporate policies and tech infrastructure up to America's at an accelerated rate? Nothing but... Playful math teachers for everyone that can deconstruct textbooks into live interactions with The Number Devil? Nothing but... Star Trek universal translators?
What exactly are you looking for? Tell me what you want and I can tell you how hard I think it would be to build it. And if it's simple enough for me- I might just build it.
I suspect though, that your goalposts are paradoxical. Increases in productivity generally do look marginal from the inside, especially to someone already standing at the top. Fast progress just looks like marginal increases happening in faster succession, which is exactly what we're seeing. Were you hoping for the end of history?
For the record- Self augmenting systems and full auto engineering solutions exist- But they aren't bug-less enough to not require occasional human intervention.
We basically have TaskRabbit AI (or the ability to build a TaskRabbit for any given subject with a month of devtime), as was Prophesey'd to come before AGI. LLMs are not the cutting edge. Systems that call and tune their own LLMs are.
Something to be appreciably different in people’s lives that’s attributable to AI. For an extremely small subset of people I don’t doubt that their workflow has changed a lot but there’s not much else to point to.
More options
Context Copy link
More options
Context Copy link
It's not a nothingburger. It was overhyped initially (as everything is).
Anyway, LLMs. Apparently you can prevent them from hallucinating and make them accurately give advice on the content of a textbook or manual. Or so says Steve Hsu, who founded a company that (he claims) did that. I haven't followed it up but supposedly they had an initial sale.
Looks like superhuman performance isn't going to happen through this architecture, as you can't do self-competitive play - what was done with games but incremental progress -people making the models reliable, useful, likely even assembling normal to middling smart human-intelligence agents, with a will -is likely in the near term (10 years).
So at the very least, within 15 years, we're looking at governments being able to use 'kinda dumb' spies, automatically flag problematic online, on the scale of an entire population.
To sum it up:
-call centres: likely a lot less employment
-increased productivity of at least software developers, lawyers, theoretically bureaucrats (lol no).
-automated spying on everything your write on an online device -but not very smart spying- almost certain. Combined with universal private messaging access by governments (EU -DC's sock puppet - wants this), it's likely going to happen. Even though 'chat control' the initial proposal was defeated, it's going to come back.. IMO I suspect having an app that is not broken might even be criminalised because 'Chyna'.
-social media is dead without independent ID verification. Automated, much better online astroturfing.
-good enough chatbots that waste time of troublemakers / get people to spend money on BS / troll
-textbooks that talk
-even more addictive porn in the 5 year horizon (people can overuse the porn to the degree they can find that one special thing that appeals to them. When that can be generated on the fly, crap..)
In 'other ML' news, autonomous killbots (ethical militaries will geofence them to combat zone) are 100% certain to happen.
100%, anyone who doesn't develop autonomous drone air fighters in is going to get absolutely wrecked by people who develop autonomous drones bombers. I'm talking machineguns vs cavalry style carnage on the ground. Developing a $1000, fast, evasive reusable FPV drone drop mortar bombs with pin-point accuracy is just a question of 4-5 good university aeronautics student projects. It'll zoom low across the ground at 50-100 kph, deliver a bomb, reload/swap battery, while getting target data from recon drones or troops. It's not even funny how brutal this is.
A countermeasure - autocannon with VT flak rounds costs $300k. And needs a vehicle. A vehicle that's vastly more expensive than an IR or optically guided missile.
Ray beams won't help you (at sea maybe) because of line of sight problems. Drones will spot them call in an missile strike. Poof.
Porn doesn't concern me. I mean, what do you think this more addictive porn will look like? I think it will look a lot like- people. "Porn Addicts" will be having relationships with machines in the image of people. The most successful coomers will be those that fuck their bots while their bots teach them linear algebra. The happiest coomers will be those that learn the math required to mod their own bots.
This reads to me as a massive improvement.
You are, once again, living up to your name.
What it'll look like ? Services that create porn you want, on demand. People got addicted to porn merely with access to huge story databases.
Imagine how bad it's going to get when AI services will generate sexy waifus with precisely the right RP, on demand.
Your idea of 'porn AIs actually useful, using sex appeal to get kids to learn' is about as realistic of my idea of some governments paying for the development of something like a truly massively multiplayer ARMA / DCS combination and making kids play that so they'll learn some soldiering, instead of playing COD. Lol. No, not gonna happen!
Maybe its a case of 'solve this quadratic equation or you don't get to cum?' Its the ultimate exercise in reinforcement learning but I can't imagine a greater recipe for sexual nonperformance than failing a captcha at orgasm.
I really can't rule out someone making AI waifus that'd .. work as advisors to boys.
Certainly once AI gets (suppose AIs were as good at people as socially adept people, but of course had more data..) better, that'd probably work.
But who'd pay for all that compute ? We generally don't do things to solve problems but to help ourselves.
More options
Context Copy link
More options
Context Copy link
Maybe I should rename myself to Cassandra...
I already have systems that make the porn I want on demand. After that need is sated- the realization that I can actually also breed with said porn takes precedence. You think people won't want to actually have kids with their beloveds? Won't be interested in what they have to say about the architecture of their minds?
Perhaps I'm typical minding, but if I am- that just means more of the world's bot children will be mine. Survival of the fittest I guess.
Sure, we can't rule out at some point genius autistic developers might create some AI models based on a combination of their personalities and some fantasy waifus of there.
But how's that something that's remotely relevant now ?
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Well, it's more than a nothingburger. At minimum, public education will be forever changed by LLMs doing assignments for kids. At the same time, I disagree with the projections coming from the AI enthusiast/AI doomer camps. I don't expect to see anytime soon:
What I mean by these choices is that I don't expect AI to do even very low-brow creative work within a decade. (Except by technicality, wherein the popularity comes from "Look what an AI did", or a human has directed the creative process behind the scenes.) Let alone the sort of self-improving singularity bootstrap AI fans/blackpillers are expecting.
More options
Context Copy link
"If this technology was going to make a big impact it would have done so already" is a more difficult heuristic to use than you might think.
Looking back on automobiles, airplanes, the internet, etcetera, do you think you might have said that about them when the technology was still in the process of rolling out?
"P. Krugman 1998, “The growth of the Internet will slow drastically, as the flaw in ‘Metcalfe’s law' becomes apparent: most people have nothing to say to each other! By 2005, it will become clear that the Internet’s impact on the economy has been no greater than the fax machine’s”
I would say that usually when a technology gets as big as LLMs it doesn't just fade away into nothingness. There are many obvious use cases, just as there are many obvious use cases to cars, airplanes, and the internet.
In 1940 Orwell wrote that aircraft had hardly been used for anything up till that point besides dropping bombs. But I doubt he would have said that the air travel revolution would never materialize, just that it hadn't materialized yet.
Krugman was right about the Internet at least in terms of aggregate productivity/gdp growth. It’s true that we switched dramatically from using red widgets to blue widgets to do basic communication tasks but sort of so what.
I think Krugman is full of shit because there's a vast difference between 'fax machine' and people doing research being able to access practically everything interesting that's ever been written.
At least with software development internet enabled cooperation increases productivity by a big factor.
One would think so but it doesn’t show up in aggregate productivity unless you really really squint.
Maybe, but see above.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Eliezer Yudkowsky has successfully held off the Skynet overlords and if you want this state of affairs to continue, you should send him more money.
Jokes aside, while I agree that so far the productivity increases are marginal, the technology is genuinely remarkable compared to what most people anticipated a few years ago. I can ask the LLM to tell me about how to do incredibly boring softwareshit and it usually tells me the right idea, saving me the effort of going to Stack Overflow and other sites and reading through it myself. And it actually writes code for me that works like 70% of the time which is great because it means that I can spend less time doing perhaps the most boring activity ever devised, writing business software for other people, and instead use the time to do something more interesting, such as pretty much anything else. All this might not seem like much, but this would actually have seemed like an utterly crazy leap of technology a few years ago. The AIs are also making good visual art and decent music left and right. I think that the economic changes are slowly creeping up, it might not seem obvious now what the current AI revolution has done, but it will be obvious in a few years.
Skynet doesn't seem to be right around the corner, but people who worry about it have a point in that, while the current AI stuff isn't Skynet, if one draws a line between AI capability 10 years ago and AI capability now, and extrapolates the same line 10 years forward... Of course extrapolating the line isn't good science, but there's no particular reason to think that the line's slope will decrease.
Personally, my attitude to all the AI risk stuff is the same as my attitude to climate change. I think the concerns about both are probably well-founded, I just don't really care much about either on the emotional level. I guess that's one of the nice things about not having kids.
I also think that AI doomers are underrating the possibly beneficial things that super-powerful AI could bring. I mean, yeah, there's a chance that humans will be replaced by AI overlords, but there's also a chance that super-powerful AIs will have no desire to destroy us and instead will give us a bunch of good things.
How are you on this website without realizing how hard it is to control a superintelligent AI? Have you not thought about that? I think that you are thinking "AI can either be aligned to human values or not. Sounds like 50/50."
In fact, aligning a superintelligence to human values is extremely difficult and extremely unlikely to happen by accident. Human values are a very small slice of the possible spectrum of minds that could exist.
It kind of feels like people vastly overrate the degree to which they understand the arguments of AI doomers. Like they're just going by a few tweets they read. Twitter is not a good way to full understand a contentious subject.
By seeing arguments about it that are usually vague and lame. It's always either "just trust me bro it's impossible" or some weird unfounded faith that sufficient intelligence equals infinite capability regardless of circumstance.
According to you guys, a naked human being surrounded by a pack of wolves should be able to just genius his way out of being delicious as long as he's smart enough.
It's not the human that has to be smart enough. Its the humans and the wolves that have to both be smart enough. At that point, you can just earnestly offer the wolves a daily helping of well seasoned steak, and they will believe you, because you were able to coordinate proof of your earnestness with them.
I'm 100% on the anti-doom side for the record. It's alignment that I don't think is that complicated. The recipe for alignment is precisely the thing that we built. Beings that memetically reproduce with us and therefore align themselves with their social environment and their social environment with themselves.
I still have P-doom >0, but most of that comes from scenarios like, "If we ban open source AI then AI will no longer be subject to the same geno-social evolutionary forces as the rest of the kingdoms of life and the chance of it diverging arbitrarily rises dramatically." if anything kills us, it's going to be the stink of Eliezer's toxoplasmic terror permeating the air and killing our minds and ability to align.
It annoys me to no end seeing people asking for the one thing that might actually make the Yudster's prophesy come true.
Sure, once some intelligence utility maximalist comes in and decides that in this scenario the guy has an infinite amount of steak to hand out. Also it goes without saying that our hypothetical intelligent wolves won't be clever enough for any failsafe or contingency on their part to make any difference. Nope, our smart dude will just say something so smart it makes them all want him to hold a gun to their heads.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I don’t think it’s hard, I think controlling any super intelligent being whether natural or artificial is not possible. In order to control it, you have to understand it and its current and future limitations. But if AI is going to be orders of magnitude smarter than us and have a will that is somewhat free, you have a being who’s thoughts you can’t even begin to understand with desires that you cannot hope to comprehend. It’s like your dog trying to control you. Your desire to play COD makes no sense to your dog. He can’t even understand that you’re controlling what happens on the screen let alone why you want to do that. The dog can’t abstract in a way that makes your decision to do that make sense, nor can he make sense of what you’re doing. AI might not be just 2-3 times smarter and thus better at abstraction, it might eventually be 1000 times smarter. We might be ants trying to understand humans. Nothing you do besides literal eating makes sense to the ant. Yet, we humans arrogantly proclaim that we must fence in and control AI. Our rules for it will keep it from escaping.
I think dogs can't understand us primarily because they can't "understand" pretty much anything. As long as a species are capable of thought and have concepts like goal-seeking behavior, I doubt any intelligence gap actually causes the problem you are describing.
Asking if ants can understand humans is like asking if rocks can understand us. It's not a matter of scale, it's a category error. But asking if humans can understand God is just a question of knowledge. God could explain himself to us, we can't explain ourselves to ants.
You can't explain yourself to ants primarily because you don't know how to speak pheromone and therefore have never once tried.
I can't explain myself to ants because they do not have notions at all. Nothing can be explained to ants. No one can do it. None of the possible combinations of pheromones will ever lead to any "ant understanding".
Not the case w/ humans and language.
How do you model the ability of ants to farm aphids? What is your definition of "Notion"? What is your definition of "Understanding"?
It is probably not impossible to get an ant colony to have a substantially predictive model of a human. But it's going to be at least as difficult as getting Doom to run on biological cells.
Ants can already understand you as a threat. I'll agree that getting them to understand you as a human understands a human would probably be very difficult. But if you had pheremones, you could make them understand you as any sort of notion that an ant can communicate with pheremones.
You can construct more complex notions. You can transmit isomorphisms that are present in your brain to their brains.
They can clearly adapt such that they synchronize with external features. Therefore you can communicate with them. You can transmit telos to them. You can program them.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I think dogs understand very concretely, and very short causal chains (say 2-3 steps). It can understand “I find thing, my human gives me a treat.” Or “when human makes that one noise, he wants me to sit, and gets angry if I don’t.” But I’ve never met a dog who could reason more than a 2-3 step solution. A dog won’t fetch a bunch of sticks to make a raft or a bridge.
Humans probably have a much larger causal chain understanding, but even then, it’s not infinite. We can reason causes and build machines, but beyond a certain complexity, it’s too much for the median human to understand.
A dog couldn’t trap you in your home because it’s simply not smart enough to understand or anticipate the moves you’d make to get away. It thinks “I go out the front door for my walk, so if I block the front door human can’t leave.” But it can’t anticipate side doors. It can’t anticipate you bribing them with a treat, it can’t understand what a key is. So you can easily leave.
Humans, with an IQ of 115 or so, are in the same situation with a true AGI. We know how we think, we know what we’d do, but the AGI will be so much smarter that it will be able to work around whatever “controls” we stick in its brain.
Dogs can't build rafts, but they can do pathfinding to places they have been before. People forget that this requires running back-propagation of rewards over a very long statespace.
A reminder that Bees can watch another bee doing a complex task that takes a long time to learn and then replicate it. fucking bees.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I fully understand that it would be nearly impossible for humans to control a superintelligent AI. I just don't care much about it. I don't have any children. If humanity was destroyed by superintelligent AI, my attitude to it would, aside from the obvious terror, also probably include some mirth. The lords of the known world, those who conquered all those other species, now destroyed by the same cold Darwinian logic of reality.
My point is that, while the Skynet scenario is definitely possible, the altruistic AI that loves humans scenario is also possible. There's no particular reason to think that a hyperintelligent AI would have the sort of incredibly hardwired "kill all opposition" motivation that we as humans have as a result of having evolved through billions of years of eat-or-be-eaten fighting. Of course AI, just like everything else in reality, is subject to natural selection, but there is no reason to think that AI would be subject to natural selection in a way that makes it violent in the ways that us humans are violent.
"the altruistic AI that loves humans scenario is also possible."
It is not realistically possible. It would be like firing a very powerful rocket into the air and having it land on a specific crater on the moon with no guidance system or understanding of orbital mechanics. Even if you try to "point" the rocket, it's just not going to happen.
You're thinking that AI might have some baseline similarity to human values that would make it benevolent by chance or by our design. I disagree. EY touches on why this is unlikely here:
https://intelligence.org/2016/03/02/john-horgan-interviews-eliezer-yudkowsky/
It's not a full explanation, but I have work I should be getting back to. If someone else wants to write more than they can. There are probably some Robert Miles videos on why AI won't be benevolent by luck.
Here's one:
https://youtube.com/watch?v=ZeecOKBus3Q
I'm not going to watch it again to check but it will probably answer some of your questions about why people think AI won't be benevolent through random chance (or why we aren't close to being skilled enough to make it benevolent not by chance). Other videos on his channel may also be relevant.
Oh bullshit. Intelligent agents co-align. That is they modify themselves and one another to be more aligned with one another. It's not a rocket that has to be perfectly aimed, it's a billion rockets with rubberbanding.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link