site banner
naraburns  nihil supernum  1yr ago (text post)   35514 thread views

Culture War Roundup for the week of March 20, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

13
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

I don't know. I don't believe the AI doomerism, but neither do I believe the AI pollyannas about how Fairy Godmother AI will solve all the world problems and we'll all be rich and living high on the hog.

And while I try to be sympathetic to Aaronson because I do think he is severely neurotic and troubled, lines like this have me facepalming:

It was also defined by my struggle against the bullies—i.e., the kids who the blankfaced administrators sheltered and protected, and who actually did to me all the things that the blankfaces probably wanted to do but couldn’t.

Dude, mate, pal: I've worked in a school and related areas. The "blankfaces" don't want to stick your head down the toilet, they don't think of you as anything other than "okay, another problem to be solved", especially if your parents were on the phone every five minutes about how unfair it was that little Scott wasn't allowed to jump ahead three grades in maths.

This is someone smart, who works in academia, is married with kids - and is still going through the world as the scared and resentful 12 year old who was bullied at school. If he thinks AI is going to recognise him as a kindred 'high intellect' and be on his side, he needs to wake up fast. AI won't recognise or think of anybody as anything, it'll be a tool in the hands of whoever brings their product to market first.

If the doomerists are anyway right, he'll be just another one of the fleshbags. For someone who seems to live in a constant state of "I'm Jewish, so everyone wants to Holocaust me", I can see why he'd pin his hopes on machine intelligence being the boss of the world. But he's wrong - sure, the machine won't care that he's Jewish, but it also won't care that he's smart for a fleshbag. He's just more paperclip material.

But he's wrong - sure, the machine won't care that he's Jewish

He is in a position to make it care about his Jewish identity and protect it.

Does this mean I can now blame him for the annoying email from Microsoft egging me on to sign in and chat with Bing?

Because OpenAI sure seems to have dumped a lot of principles for the opportunity to "get there first and rake in the dough":

How is Microsoft approaching responsible AI for the new Bing?

At Microsoft, we take our commitment to responsible AI seriously. Bing is being developed in accordance with our AI principles. We are working with our partner, OpenAI, to deliver an experience that encourages responsible use. For example, we have and will continue to partner with OpenAI on foundational model work, we have designed the Bing user experience to keep humans at the centre, and we have developed a safety system that is designed to mitigate failures and avoid misuse with things such as content filtering, operational monitoring and abuse detection, and other safeguards. The waiting list process is also a part of our approach to responsible AI. We’ll be taking user feedback from those with early access to Bing to improve the tool before making it broadly available.

Responsible AI is a journey, and we'll continually improve our systems along the way. We're committed to making our AI more reliable and trustworthy, and your feedback will help us do so. To learn more about how to use Bing responsibly, please see our Terms of Use and Code of Conduct.

To learn more about how to use Bing responsibly, please see our Terms of Use and Code of Conduct.

However AI eventually develops, right now as in this instance, I think it's being used as a tool to use our data and sell us crap:

How do I best use the new Bing?

Ask questions as if you were talking to a person. This means including details, asking for clarification or more information and telling Bing how it can be more helpful to you. Here’s an example: “I’m planning a trip with my friends in September. What beaches are within a 3-hour flight from London Heathrow?” Then follow up with something such as “What should we do when we get there?”.

And of course completely coincidentally all the answers will be "from our sponsors" - what airline, what hotel, what fun pubs and clubs and tours and restaurants, etc. to use and visit. Now we've told it that we have a bunch of people all wanting to visit someplace at a particular date. It can now suggest items for groups, plus collate that with our contacts lists and sell the same advertising to the 'friends'. Expect a deluge of "Hey Guy, you asked about ski trips? There's a great bargain on skiing gear to be found at this site!", maybe less blatant than Google's sponsored and paid-for search rankings, but just as devoted to parting us from our money.

If this is the smart shiny high-IQ future for humanity, Mr. Aaronson, it sure looks a lot like the existing old grubby commercial present.

Jews create the tech, gentiles finance it. That seems to be the recurring pattern. Like Marc Andreessen and Peter Theil investing in Facebook.

I don't want to sound like a broken record, but I trust at least Zuckerberg to not be an asshole about it, based on his track record. Sure, on the practical level he probably values Jews somewhat more than other people (not enough to marry one, though), especially after an offshoot of Jewish mafia has browbeaten him into furthering their activist agenda; but he's not «defined» by the extreme, all-consuming kind of in-group favoritism and proactive hateful paranoia in the «cries out in pain as he strikes you» fashion, that Aaronson has inherited from his tree-hugger parents («they didn't want to burden the Earth with humans, but gave birth to me to spite Hitler!» – or something, can't be bothered to find the quote) and clumsily rearranged into his Nerds vs. Bullies world model (with nerds hiding in the optimized Shtetl, of course – from those evil, genocidal… Trumpists, feminists and jocks). It's very fortunate that Aaronson is far from levers of power (unless someone decides to let him hold them) – he is profoundly untrustworthy. He sees academia as his salvation; it's also a good containment device.

Zuckerberg is also notoriously despised by the chattering class and assorted fools, which is a good signal to me; and he is enabling Meta AI team to opensource a ton of essentially un-brainwashed goodies (like, again, LLaMA), which I am thankful for.

For what it's worth, Altman also doesn't seem to be particularly obsessed with his tribal background, though I have other problems with him and predict OpenAI at large to conform to @SecureSignals's worst expectations.

The entire history of the development of AGI, Kurzweil onward, has been a disproportionately Jewish field even by the standards of American academia and computer science

This is a bit of an overstatement. It must be said that many of those Jews have been a drag on it – just academic scriveners with good PR of «geniuses», exploring Talmudic detours of publishable and sexy, «clever» approaches; they have educated the cohort that is still dumbfounded and in denial about the current progress. Minsky alone has probably delayed Singularity by half a decade with his dismissive bullshit (and he wasn't alone, there was also Seymour Papert). Chomsky, while often excused as not an AI researcher, has also been very harmful. Hubert Dreyfus – another naysayer. Hofstadter is about as bad as Marcus (at least symbolic AI is a real thing, unlike the strange loop woo) and had infinitely more reach with technical people thanks to his insight porn books.

Solomonoff, Rosenblatt (whom Minsky&Papert attacked) and a few others like Sutskever, though – sure, they were/are great (and Vaswani's co-author Noam Shazeer, current head of the rather compelling Character.AI, conspicuously wears a kippah). But also exceptions in their appreciation for the simple, true, Occam-compliant path. Schmidhuber in his overview of deep learning provides a comprehensive overview of pioneers and key contributors up to 2014. Ivakhenko, Fukushima, Hinton, Hutter, Sutton, LeCun, Schmidhooboh himself. I recommend skimming it.

In Chomsky’s case his largely baseless conjecture turned out to be wrong and he doesn’t want to admit it.

Which one? If you are talking about universal grammar/the idea that human language facility is geared towards some small and easily-described subspace of the space of all possible formal languages, I think the jury is still out. Humans acquire language facilities after consuming far less than the GPT series' dozens of terabytes of text. We haven't resolved whether this gap is because humans just have better learning algorithms or because in some sense the GPTs really have more learning work to do (which would imply that there exist alternative formal systems which a GPT could acquire just as well after training for this long on 45TB of examples, but humans would be hopeless growing up in).

Sample efficiency difference between radically dissimilar substates would be a very small hill to die on when arguing a conceptual limitation. But anyway: LLMs acquire «grammar» at about the same pace as humans.

Here's a fascinating new paper: Modern language models refute Chomsky’s approach to language:

…The success of large language models is a failure for generative theories because it goes against virtually all of the principles these theories have espoused. In fact, none of the principles and innate biases that Chomsky and those who work in his tradition have long claimed necessary needed to be built into these models (e.g. binding principles, binary branching, island constraints, empty category principle, etc.). Moreover, these models were created without incorporating any of Chomsky’s key methodological claims, like ensuring the models properly consider competence vs. performance, respect “minimality” or “perfection,” and avoid relying on the statistical patterns of unanalyzed data.

…Work examining the scaling relationship between performance and data size shows that at least current versions of the models do achieve their spectacular performance only with very large network sizes and large amounts of data (Kaplan et al. 2020). However, Zhang et al. 2020 show that actually most of this learning is not about syntax. Models that are trained on 10 − 100 million words “reliably encode most syntactic and semantic features” of language, and the remainder of training seems to target other skills (like knowledge of the world). This in fact matches in spirit analyses showing that syntactic knowledge requires a small number of bits of information, especially when compared to semantics (Mollica & Piantadosi 2019). Hosseini et al. 2022 present evidence that models trained on developmentally-plausible amounts of data already capture human neural responses to language in the brain.

Consider that children are exposed to about 6-20k words per day. So in 3 years, they can realistically process tens of millions of words. And that's augmented with all our truly innate social hooks, hypothesis-testing and multimodality that GPTs have been devoid of.

Chomsky and others have long emphasized the study of syntax as a separate entity, not only from the rest of cognition but from the rest of language (see, e.g. Chomsky 1957, Croft 1995, Newmeyer 1991, Adger 2018). Syntax in this approach is not supposed to be reducible to general statistics between words11—exactly the thing that large language models now provide. Modern large language models integrate syntax and semantics in the underlying representations: encoding words as vectors in a high-dimensional space, without an effort to separate out e.g. part of speech categories from semantic representations, or even predict at any level of analysis other than the literal word.

A second point is that for these models, probability and information theory are central. Chomsky has long been dismissive of probability, saying “it must be recognized that the notion of ‘probability of a sentence’ is an entirely useless one, under any known interpretation of this term” (Chomsky 1969), a position he has maintained for decades (Norvig 2012).12 Often when those who work in Chomsky’s tradition talk about probability models, they refer to simple things like Shannon 1948’s 𝑛-gram models that count up sequential word co-occurrences and were long used in natural language processing tasks (Chen & Goodman 1999, Manning & Schutze 1999). But by now, such models are decades out of date.

The fact that predictions are probabilistic is useful because it means that the underlying representations are continuous and gradient. Unlike work formalizing discrete rules and processes, typical of generative linguistics (e.g. Chomsky 1956, 1995, Collins & Stabler 2016, Chomsky 1957, Pinker & Prince 1988), modern language models do not use (at least explicit) rules and principles—they are based in a continuous calculus that allows multiple influences to have a gradient effect on upcoming linguistic items.

Perhaps most notably, modern language models succeed despite the fact that their underlying architecture for learning is relatively unconstrained. …Recall that many of the learnability arguments were supposed to be mathematical and precise, going back to Gold 1967 (though see Johnson 2004, Chater & Vitányi 2007) and exemplified by work like Wexler & Culicover 1980. It’s not that we don’t know the right learning mechanism; it’s supposed to be that it can be proven none exists. Even my own generative syntax textbook from undergraduate syntax purports to show a “proof” that because infinite, productive systems cannot be learned, parts of syntax must be innate (Carnie 2021). Legate & Yang 2002 call the innateness of language “not really a hypothesis” but “an empirical conclusion” based on the strength of poverty of stimulus arguments. Proof of the impossibility of learning in an unrestricted space was supposed to be the power of this approach. It turned out to be wrong.

But learning without constraints is not only possible, it has been well-understood and even predicted. Formal analyses of learning and inference show that learners can infer the correct theory out of the space of possible computations (Solomonoff 1964, Hutter 2004, Legg & Hutter 2007).

In this view, large language models function somewhat like automated scientists or automated linguists, who also work over relatively unrestricted spaces, searching to find theories which do the best job of parsimoniously predicting observed data.

Chomsky maintains (in the same Marcus interview above) that large language models have achieved nothing because they fail to explain “Why this? Why not that?” The question of whether these models can explain why human language has the form that it does is an interesting one that likely depends on whether the language system evolved before language or concurrently with it. If language co-opted neural systems for general sequential prediction (e.g. Christiansen & Chater 2015), it’s possible we had some architecture like these models before we had language, and therefore the form of language is explained by the pre-existing computational architecture.

…we may admit that there are some “why” questions that a large language model may not answer; this does not mean they have no scientific value… However, it is worth highlighting in this context that Chomsky’s own theories don’t permit particularly deep “why” questions either. In large part, he simply states that the answer is genetics or simplicity or “perfection”, without providing any independent justification for these claims. For example, readers of Berwick & Chomsky 2016—a book titled Why Only Us—might have hoped to find a thorough and satisfying “why” explanation. Their answer boils down to people having merge (essentially chunking two elements into one, unordered). And when it comes down to explaining why merge, they fall down the stairs: they simply state that “merge” is the minimal computational operation, apparently because that’s what they think and that’s that. Forget the relativity of definitions of simplicity, articulated by Goodman 1965, where what is considered simple must ground out in some convention. Berwick & Chomsky do even attempt to explain why they believe “merge” is simpler than other simple computational bases, like cellular automata or combinatory logic or systems of colliding Newtonian particles—all of which are capable of universal computation (and thus encoding structures, including hierarchical ones). Or maybe more directly, what makes merge “simpler” or more “perfect” than, say, backpropagation? Or Elman et al. 1996’s architectural biases? Berwick & Chomsky don’t even consider these questions, even though the ability to scientifically go after such “why” questions is supposed to be the hallmark of the approach. One might equally just declare that a transformer architecture is the “minimal” computational system that can handle the dependencies and structures of natural language and be done with it.

Perhaps most damningly, not even all languages appear to be recursive (Everett 2005, Futrell et al. 2016), contradicting the key universality claim from Hauser et al. 2002.…

It's a long-deserved hatchet job. Statistical learning paradigm is not just shown to be more useful in engineering or even closer to the biological truth than generative linguistics – it's more epistemologically mature, philosophically profound and, yes, elegant; as often happens when people hone their thinking in challenging reality and not just adversarial ivory tower circlejerks.

Frederick Jelinek’s quip “Every time I fire a linguist, the performance of the speech recognizer goes up” was a joke among linguists and computer scientists for decades. I’ve even seen it celebrated by academic linguists who think it elevates their abstract enterprise over and above the dirty details of implementation and engineering. But, while generative syntacticians insulated themselves from engineering, empirical tests, and formal comparisons, engineering took over. And now, engineering has solved the very problems the field has fixated on—or is about to very soon. The unmatched success of an approach based on probability, internalization of constructions in corpora, gradient methods, and neural networks is, in the end, a humiliation for everyone who has spent decades deriding these tools.

I'm not sure what specifically @2rafa meant – and Chomsky is lost in his mirror labyrinth of mottes and baileys. In any case, she's exactly right.

However, Zhang et al. 2020 show that actually most of this learning is not about syntax. Models that are trained on 10 − 100 million words “reliably encode most syntactic and semantic features” of language, and the remainder of training seems to target other skills (like knowledge of the world).

Interesting. If it holds up, I'm updating significantly against universal grammar. (I still see some grounds to be skeptical: in my experience at least the LLaMas often make conspicuous grammatical mistakes in languages such as German which were represented in excess of that in their training set, and in my limited experience looking at the grammatical evaluation sets in that battery they tend to suffer from a certain American laconicity that may make them insufficient for evaluating understanding of recursive structure)

new paper: Modern language models refute Chomsky’s approach to language:

I'll probably come back with more commentary once I had time to read the whole of it, but I do have an issue that might turn out to be a nitpick or a portent of a more general methodological criticism right on the second page:

The answer is outside of the training set. In fact, after “Once upon a time, in a far-off land, there lived a colony of ants,” a Google search returns no matching strings on the entire internet.

This line of argumentation seems wrong in a way that suggests sloppiness about something that should be a core concern of such a paper. LLMs, among being many other things, are lossy compression algorithms with respect to their training set. An output not being an exact reproduction therefore does not imply that it is not a reproduction at all, any more than "I searched the internet for images with the same first 20 pixels and found no matches" implies that a given JPEG is an original creation.

I think it might be his theory of linguistic recursion that the post might be alluding to?

I’ll add “explore connections between singularity/kabbalah/tikkun olam” to my growing to-do list of effortposts.

How on earth would your observation smooth my concerns rather than justify them? The only real controversial thing would be to say that these companies are going to put particular emphasis on consciously making sure the AI is aligned to Jewish interests in how it responds to prompts and generates creative output. Tapping Aaronson is the exact sort of thing that would follow from that conclusion.

consciously making sure the AI is aligned to Jewish interests

How? Is it going to put on a virtual kippah and tallit and respond in a heavy Yiddish accent: "Oy vey, why you not supporting Israel?" if I ask it about dukkah?

How? Is it going to put on a virtual kippah and tallit and respond in a heavy Yiddish accent

It's going to be optimized for giving really good critiques of white identity, history, and culture and lock up when it comes to any sort of prompts that would invite criticism of Jewish identity and culture. It's going to be so good at doing this, it will oftentimes be imperceptible. This going to influence its creative output as well which is going to generate art, film, and advertisements.

It's the most pure, mathematical form of Kevin MacDonald's thesis in Culture of Critique.

white identity, history, and culture

Is giving really good critiques of whatever that term signifies really in line with "Jewish interests" in any putative interethnic zero-sum game? The way it seems to me, the status of the memetically menopausal middle American cultural hobo stew that "white" usually denotes is already so low that its legendary powers of attraction hardly function anymore, and everyone who has the slightest claim to something else has switched to holding on to it for dear life. This seems likely to result in the emergence of more functioning ethnic coalitions that would be actively opposed to "Jewish interests". In concrete terms, a Joseph Anthony Buttigieg II immigrating to the US nowadays may well issue children who would march against Israel with the Arab student union, not generic gay white guys.

Is giving really good critiques of whatever that term signifies really in line with "Jewish interests" in any putative interethnic zero-sum game?

Yes, it is. You in fact say "whatever that term signifies", implying doubt that there is such a thing as white people- no doubt when there is talk about black people you don't feign ignorance. You can split hairs and argue for "social construct", that's what Chat-GPT does (but not for Jews or black people, only for white people). Of course, if I had my way, then the term Aryan should be used in order to denote non-Jewish European-descended, but we can't use the term Aryan because, well you know... let's ask GPT about the existence of an Aryan people:

Something went wrong. If this issue persists please contact us through our help center at help.openai.com.

Oops- you can word the prompt to ellicit some sort of response but it's just denunciations of term. But it does a good job of explaining:

Jewish people have a rich history and diverse cultural practices that may differ significantly from other groups in the United States.

So it goes.

The way it seems to me, the status of the memetically menopausal middle American cultural hobo stew that "white" usually denotes is already so low that its legendary powers of attraction hardly function anymore

How did it get to that point, specifically? With our culture, right?

Edit: Of course, Chat-GPT knows that "Aryan" is verboten but "Indo-European" gives a more accurate response (despite the fact they are synonymous terms):

The Indo-Europeans were a group of people who lived in Eurasia during the 3rd and 2nd millennia BCE and spoke a common language that has come to be known as the Proto-Indo-European language. They are believed to have originated in the Pontic-Caspian steppe, which is located in modern-day Ukraine and Russia, and spread outwards to form various civilizations in Europe, Central Asia, and South Asia.

The Indo-Europeans are important because their language is the ancestor of many of the languages spoken today in Europe, as well as in parts of Central and South Asia. These languages include English, Spanish, French, German, Italian, Hindi, Urdu, Persian, and Russian, among others.

The Indo-European language family is one of the largest language families in the world, with more than 400 languages and dialects that are spoken by billions of people. The study of Indo-European languages and cultures has been an important field of inquiry in linguistics, archaeology, history, and anthropology, and has helped to shed light on the origins and movements of ancient peoples and civilizations.

More comments

Was demanding the unconditional surrender of Germany at a time when millions of Jews might still be saved from the concentration camps in line with "Jewish interests"? You could argue either way, but rational calculation is hardly guaranteed when hatred or extreme fear, whether warranted or not, is in the driver's seat.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats