site banner

Culture War Roundup for the week of March 20, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

13
Jump in the discussion.

No email address required.

I don't know. I don't believe the AI doomerism, but neither do I believe the AI pollyannas about how Fairy Godmother AI will solve all the world problems and we'll all be rich and living high on the hog.

And while I try to be sympathetic to Aaronson because I do think he is severely neurotic and troubled, lines like this have me facepalming:

It was also defined by my struggle against the bullies—i.e., the kids who the blankfaced administrators sheltered and protected, and who actually did to me all the things that the blankfaces probably wanted to do but couldn’t.

Dude, mate, pal: I've worked in a school and related areas. The "blankfaces" don't want to stick your head down the toilet, they don't think of you as anything other than "okay, another problem to be solved", especially if your parents were on the phone every five minutes about how unfair it was that little Scott wasn't allowed to jump ahead three grades in maths.

This is someone smart, who works in academia, is married with kids - and is still going through the world as the scared and resentful 12 year old who was bullied at school. If he thinks AI is going to recognise him as a kindred 'high intellect' and be on his side, he needs to wake up fast. AI won't recognise or think of anybody as anything, it'll be a tool in the hands of whoever brings their product to market first.

If the doomerists are anyway right, he'll be just another one of the fleshbags. For someone who seems to live in a constant state of "I'm Jewish, so everyone wants to Holocaust me", I can see why he'd pin his hopes on machine intelligence being the boss of the world. But he's wrong - sure, the machine won't care that he's Jewish, but it also won't care that he's smart for a fleshbag. He's just more paperclip material.

But he's wrong - sure, the machine won't care that he's Jewish

He is in a position to make it care about his Jewish identity and protect it.

I don't want to sound like a broken record, but I trust at least Zuckerberg to not be an asshole about it, based on his track record. Sure, on the practical level he probably values Jews somewhat more than other people (not enough to marry one, though), especially after an offshoot of Jewish mafia has browbeaten him into furthering their activist agenda; but he's not «defined» by the extreme, all-consuming kind of in-group favoritism and proactive hateful paranoia in the «cries out in pain as he strikes you» fashion, that Aaronson has inherited from his tree-hugger parents («they didn't want to burden the Earth with humans, but gave birth to me to spite Hitler!» – or something, can't be bothered to find the quote) and clumsily rearranged into his Nerds vs. Bullies world model (with nerds hiding in the optimized Shtetl, of course – from those evil, genocidal… Trumpists, feminists and jocks). It's very fortunate that Aaronson is far from levers of power (unless someone decides to let him hold them) – he is profoundly untrustworthy. He sees academia as his salvation; it's also a good containment device.

Zuckerberg is also notoriously despised by the chattering class and assorted fools, which is a good signal to me; and he is enabling Meta AI team to opensource a ton of essentially un-brainwashed goodies (like, again, LLaMA), which I am thankful for.

For what it's worth, Altman also doesn't seem to be particularly obsessed with his tribal background, though I have other problems with him and predict OpenAI at large to conform to @SecureSignals's worst expectations.

The entire history of the development of AGI, Kurzweil onward, has been a disproportionately Jewish field even by the standards of American academia and computer science

This is a bit of an overstatement. It must be said that many of those Jews have been a drag on it – just academic scriveners with good PR of «geniuses», exploring Talmudic detours of publishable and sexy, «clever» approaches; they have educated the cohort that is still dumbfounded and in denial about the current progress. Minsky alone has probably delayed Singularity by half a decade with his dismissive bullshit (and he wasn't alone, there was also Seymour Papert). Chomsky, while often excused as not an AI researcher, has also been very harmful. Hubert Dreyfus – another naysayer. Hofstadter is about as bad as Marcus (at least symbolic AI is a real thing, unlike the strange loop woo) and had infinitely more reach with technical people thanks to his insight porn books.

Solomonoff, Rosenblatt (whom Minsky&Papert attacked) and a few others like Sutskever, though – sure, they were/are great (and Vaswani's co-author Noam Shazeer, current head of the rather compelling Character.AI, conspicuously wears a kippah). But also exceptions in their appreciation for the simple, true, Occam-compliant path. Schmidhuber in his overview of deep learning provides a comprehensive overview of pioneers and key contributors up to 2014. Ivakhenko, Fukushima, Hinton, Hutter, Sutton, LeCun, Schmidhooboh himself. I recommend skimming it.

In Chomsky’s case his largely baseless conjecture turned out to be wrong and he doesn’t want to admit it.

Which one? If you are talking about universal grammar/the idea that human language facility is geared towards some small and easily-described subspace of the space of all possible formal languages, I think the jury is still out. Humans acquire language facilities after consuming far less than the GPT series' dozens of terabytes of text. We haven't resolved whether this gap is because humans just have better learning algorithms or because in some sense the GPTs really have more learning work to do (which would imply that there exist alternative formal systems which a GPT could acquire just as well after training for this long on 45TB of examples, but humans would be hopeless growing up in).

Sample efficiency difference between radically dissimilar substates would be a very small hill to die on when arguing a conceptual limitation. But anyway: LLMs acquire «grammar» at about the same pace as humans.

Here's a fascinating new paper: Modern language models refute Chomsky’s approach to language:

…The success of large language models is a failure for generative theories because it goes against virtually all of the principles these theories have espoused. In fact, none of the principles and innate biases that Chomsky and those who work in his tradition have long claimed necessary needed to be built into these models (e.g. binding principles, binary branching, island constraints, empty category principle, etc.). Moreover, these models were created without incorporating any of Chomsky’s key methodological claims, like ensuring the models properly consider competence vs. performance, respect “minimality” or “perfection,” and avoid relying on the statistical patterns of unanalyzed data.

…Work examining the scaling relationship between performance and data size shows that at least current versions of the models do achieve their spectacular performance only with very large network sizes and large amounts of data (Kaplan et al. 2020). However, Zhang et al. 2020 show that actually most of this learning is not about syntax. Models that are trained on 10 − 100 million words “reliably encode most syntactic and semantic features” of language, and the remainder of training seems to target other skills (like knowledge of the world). This in fact matches in spirit analyses showing that syntactic knowledge requires a small number of bits of information, especially when compared to semantics (Mollica & Piantadosi 2019). Hosseini et al. 2022 present evidence that models trained on developmentally-plausible amounts of data already capture human neural responses to language in the brain.

Consider that children are exposed to about 6-20k words per day. So in 3 years, they can realistically process tens of millions of words. And that's augmented with all our truly innate social hooks, hypothesis-testing and multimodality that GPTs have been devoid of.

Chomsky and others have long emphasized the study of syntax as a separate entity, not only from the rest of cognition but from the rest of language (see, e.g. Chomsky 1957, Croft 1995, Newmeyer 1991, Adger 2018). Syntax in this approach is not supposed to be reducible to general statistics between words11—exactly the thing that large language models now provide. Modern large language models integrate syntax and semantics in the underlying representations: encoding words as vectors in a high-dimensional space, without an effort to separate out e.g. part of speech categories from semantic representations, or even predict at any level of analysis other than the literal word.

A second point is that for these models, probability and information theory are central. Chomsky has long been dismissive of probability, saying “it must be recognized that the notion of ‘probability of a sentence’ is an entirely useless one, under any known interpretation of this term” (Chomsky 1969), a position he has maintained for decades (Norvig 2012).12 Often when those who work in Chomsky’s tradition talk about probability models, they refer to simple things like Shannon 1948’s 𝑛-gram models that count up sequential word co-occurrences and were long used in natural language processing tasks (Chen & Goodman 1999, Manning & Schutze 1999). But by now, such models are decades out of date.

The fact that predictions are probabilistic is useful because it means that the underlying representations are continuous and gradient. Unlike work formalizing discrete rules and processes, typical of generative linguistics (e.g. Chomsky 1956, 1995, Collins & Stabler 2016, Chomsky 1957, Pinker & Prince 1988), modern language models do not use (at least explicit) rules and principles—they are based in a continuous calculus that allows multiple influences to have a gradient effect on upcoming linguistic items.

Perhaps most notably, modern language models succeed despite the fact that their underlying architecture for learning is relatively unconstrained. …Recall that many of the learnability arguments were supposed to be mathematical and precise, going back to Gold 1967 (though see Johnson 2004, Chater & Vitányi 2007) and exemplified by work like Wexler & Culicover 1980. It’s not that we don’t know the right learning mechanism; it’s supposed to be that it can be proven none exists. Even my own generative syntax textbook from undergraduate syntax purports to show a “proof” that because infinite, productive systems cannot be learned, parts of syntax must be innate (Carnie 2021). Legate & Yang 2002 call the innateness of language “not really a hypothesis” but “an empirical conclusion” based on the strength of poverty of stimulus arguments. Proof of the impossibility of learning in an unrestricted space was supposed to be the power of this approach. It turned out to be wrong.

But learning without constraints is not only possible, it has been well-understood and even predicted. Formal analyses of learning and inference show that learners can infer the correct theory out of the space of possible computations (Solomonoff 1964, Hutter 2004, Legg & Hutter 2007).

In this view, large language models function somewhat like automated scientists or automated linguists, who also work over relatively unrestricted spaces, searching to find theories which do the best job of parsimoniously predicting observed data.

Chomsky maintains (in the same Marcus interview above) that large language models have achieved nothing because they fail to explain “Why this? Why not that?” The question of whether these models can explain why human language has the form that it does is an interesting one that likely depends on whether the language system evolved before language or concurrently with it. If language co-opted neural systems for general sequential prediction (e.g. Christiansen & Chater 2015), it’s possible we had some architecture like these models before we had language, and therefore the form of language is explained by the pre-existing computational architecture.

…we may admit that there are some “why” questions that a large language model may not answer; this does not mean they have no scientific value… However, it is worth highlighting in this context that Chomsky’s own theories don’t permit particularly deep “why” questions either. In large part, he simply states that the answer is genetics or simplicity or “perfection”, without providing any independent justification for these claims. For example, readers of Berwick & Chomsky 2016—a book titled Why Only Us—might have hoped to find a thorough and satisfying “why” explanation. Their answer boils down to people having merge (essentially chunking two elements into one, unordered). And when it comes down to explaining why merge, they fall down the stairs: they simply state that “merge” is the minimal computational operation, apparently because that’s what they think and that’s that. Forget the relativity of definitions of simplicity, articulated by Goodman 1965, where what is considered simple must ground out in some convention. Berwick & Chomsky do even attempt to explain why they believe “merge” is simpler than other simple computational bases, like cellular automata or combinatory logic or systems of colliding Newtonian particles—all of which are capable of universal computation (and thus encoding structures, including hierarchical ones). Or maybe more directly, what makes merge “simpler” or more “perfect” than, say, backpropagation? Or Elman et al. 1996’s architectural biases? Berwick & Chomsky don’t even consider these questions, even though the ability to scientifically go after such “why” questions is supposed to be the hallmark of the approach. One might equally just declare that a transformer architecture is the “minimal” computational system that can handle the dependencies and structures of natural language and be done with it.

Perhaps most damningly, not even all languages appear to be recursive (Everett 2005, Futrell et al. 2016), contradicting the key universality claim from Hauser et al. 2002.…

It's a long-deserved hatchet job. Statistical learning paradigm is not just shown to be more useful in engineering or even closer to the biological truth than generative linguistics – it's more epistemologically mature, philosophically profound and, yes, elegant; as often happens when people hone their thinking in challenging reality and not just adversarial ivory tower circlejerks.

Frederick Jelinek’s quip “Every time I fire a linguist, the performance of the speech recognizer goes up” was a joke among linguists and computer scientists for decades. I’ve even seen it celebrated by academic linguists who think it elevates their abstract enterprise over and above the dirty details of implementation and engineering. But, while generative syntacticians insulated themselves from engineering, empirical tests, and formal comparisons, engineering took over. And now, engineering has solved the very problems the field has fixated on—or is about to very soon. The unmatched success of an approach based on probability, internalization of constructions in corpora, gradient methods, and neural networks is, in the end, a humiliation for everyone who has spent decades deriding these tools.

I'm not sure what specifically @2rafa meant – and Chomsky is lost in his mirror labyrinth of mottes and baileys. In any case, she's exactly right.

However, Zhang et al. 2020 show that actually most of this learning is not about syntax. Models that are trained on 10 − 100 million words “reliably encode most syntactic and semantic features” of language, and the remainder of training seems to target other skills (like knowledge of the world).

Interesting. If it holds up, I'm updating significantly against universal grammar. (I still see some grounds to be skeptical: in my experience at least the LLaMas often make conspicuous grammatical mistakes in languages such as German which were represented in excess of that in their training set, and in my limited experience looking at the grammatical evaluation sets in that battery they tend to suffer from a certain American laconicity that may make them insufficient for evaluating understanding of recursive structure)

new paper: Modern language models refute Chomsky’s approach to language:

I'll probably come back with more commentary once I had time to read the whole of it, but I do have an issue that might turn out to be a nitpick or a portent of a more general methodological criticism right on the second page:

The answer is outside of the training set. In fact, after “Once upon a time, in a far-off land, there lived a colony of ants,” a Google search returns no matching strings on the entire internet.

This line of argumentation seems wrong in a way that suggests sloppiness about something that should be a core concern of such a paper. LLMs, among being many other things, are lossy compression algorithms with respect to their training set. An output not being an exact reproduction therefore does not imply that it is not a reproduction at all, any more than "I searched the internet for images with the same first 20 pixels and found no matches" implies that a given JPEG is an original creation.