site banner
PaperclipPerfector  1yr ago (text post)   33071 thread views

Culture War Roundup for the week of April 17, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

8
Top New Old Controversial Comments
Jump in the discussion.

No email address required.

And we aren't sure how to ensure that humans have a positive value in any AGI's utility function.

I feel like there is a more basic question here, specifically, what will an AGI's utility function even look like? Do we know the answer to that question? If the answer is no then it is not clear to me how we even make progress on the proposed question.

Is sure seems like 'unfriendly' 'unaligned' hypercompetition between entities is the default assumption, given available evidence.

I am not so sure. After all, if you want to use human evidence, plenty of human groups cooperate effectively. At the level of individuals, groups, nations, and so on. Why will the relationship between humans and AGI be more like the relation between humans in some purported state of nature than between human groups or entities today?

I don't know what you would accept as evidence for this if you are suggesting we need to run experiments with AGIs to see if they try to kill us or not.

I would like something more rigorously argued, at least. What is the reference class for possible minds? How did you construct it? What is the probability density of various possible minds and how was that density determined? Is every mind equally likely? Why think that? On the assumption humans are going to attempt to construct only those minds whose existence would be beneficial to us why doesn't that weigh substantial probability density towards the fact that we end constructing such a mind? Consider other tools humans have made. There are many possible ways to stick a sharp blade to some other object or handle. Almost all such ways are relatively inefficient or useless to humans in consideration of the total possibility space. Yet almost all the tools we actually make are in the tiny probability space where they are actually useful to us.

Their position being that it will take DECADES of concentrated effort to understand the nature of the alignment problem and propose viable solutions, and that it's has proven much easier than hoped to produce AGI-like entities, it makes perfect sense that their argument is "we either slow things to a halt now or we're never going to catch up in time."

Can you explain to me what an "AGI-like" entity is? I'm assuming this is referring to GPT and Midjourney and similar? But how are these entities AGI-like? We have a pretty good idea of what they do (statistical token inference) in a way that seems not true of intelligence more generally. This isn't to say that statistical token inference can't do some pretty impressive things, it can! But it seems quite different than the definition of intelligence you give below.

Where "intelligence" means having the ability to comprehend information and apply it so as to push the world into a state that is more in line with the intelligence's goals.

Is something like GPT "intelligent" on this definition? Does having embedded statistical weights from its training data constitute "comprehending information?" Does choosing it's output according to some statistical function mean it has a "goal" that it's trying to achieve?

Moreover on this definition it seems intelligence has a very natural limit in the form of logical omniscience. At some point you understand the implications of all the facts you know and how they relate to the world. The only way to learn more about the world (and perhaps more implications of the facts you do know) is by learning further facts. Should we just be reasoning about what AGI can do in the limit by reasoning about what a logically omniscient entity could do?

It seems to me there is something of an equivocation between being able to synthesize information and achieve one's goals going on under the term "intelligence." Surely being very good at synthesizing information is a great help to achieving one's goals but it is not the only thing. I feel like in these kinds of discussions people posit (plausibly!) that AI will be much better than humans at the synthesizing information thing, and therefore conclude (less plausibly) it will be arbitrarily better at the achieving goals thing.

The leap, there, is that a superintelligent mind can start improving itself (or future versions of AGI) more rapidly than humans can and that will keep the improvements rolling with humans no longer in the driver's seat.

What is the justification for this leap, though? Why believe that AI can bootstrap itself into logical omniscience (or something close, or beyond?) Again there are questions of storage and compute to consider. What kind of compute does an AI require to achieve logical omniscience? What kind of architecture enables this? As best I can tell the urgency around this situation is entirely driven by imagined possibility.

"Any AGI invented on earth COULD become superintelligent, and if it does so it can figure out how to bootstrap into godlike power inside a decade" is the steelmanned claim, I think.

Can I get a clarification on "godlike power"? Could the AI in question break all our encryption by efficiently factoring integers? What if there is no (non-quantum) algorithm for efficiently factoring integers?

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats