site banner
naraburns  nihil supernum  1yr ago (text post)   32944 thread views

Culture War Roundup for the week of February 27, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

10
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

I'm going to shamelessly steal @Scimitar's post from the Friday Fun thread because I think we need to talk about LLMs in a CW context:

A few months ago OpenAI dropped their API price, from $0.06/1000 tokens for their best model, to $0.02/1000 tokens. This week, the company released their ChatGPT API which uses their "gpt-3.5-turbo" model, apparently the best one yet, for the price of $0.002/1000 tokens. Yes, an order of magnitude cheaper. I don't quite understand the pricing, and OpenAI themselves say: "Because gpt-3.5-turbo performs at a similar capability to text-davinci-003 but at 10% the price per token, we recommend gpt-3.5-turbo for most use cases." In less than a year, the OpenAI models have not only improved, but become 30 times cheaper. What does this mean?

A human thinks at roughly 800 words per minute. We could debate this all day, but it won’t really effect the math. A word is about 1.33 tokens. This means that a human, working diligently 40 hour weeks for a year, fully engaged, could produce about: 52 * 40 * 60 * 800 * 1.33 = 132 million tokens per year of thought. This would cost $264 out of ChatGPT.

https://old.reddit.com/r/singularity/comments/11fn0td/the_implications_of_chatgpts_api_cost/

...or about $0.13 per hour. Yes technically it overlooks the fact that OpenAI charge for both input and output tokens, but this is still cheap and the line is trending downwards.

Full time minimum wage is ~$20k/year. GPT-3.5-turbo is 100x cheaper and vastly outperforms the average minimum wage worker at certain tasks. I dunno, this just feels crazy. And no, I wont apologize for AI posting. It is simply the most interesting thing happening right now.


I strongly agree with @Scimitar, this is the most interesting thing happening right now. If you haven't been following AI/LLM progress the last month, it has been blazingly fast. I've spent a lot of time in AI doomer circles so I have had a layer of cynicism around people talking about the Singularity, but I'll be damned if I'm not started to feel a bit uncomfortable that they may have been right.

The CW implications seem endless - low skill jobs will be automated, but which tribe first? Will HR admins who spend all day writing two emails be the first to go? Fast food cashiers who are already on their way out through self ordering consoles?

Which jobs will be the last to go? The last-mile problem seems pretty bad for legal and medical professionals (i.e. if an LLM makes up an answer it could be very bad) but theoretically we could use them to generate copy or ideas then go through a final check by a professional.

Outside of employment, what will this do to human relations? I've already seen some (admittedly highly autistic) people online saying that talking to ChatGPT is more satisfying than talking to humans. Will the NEET apocalypse turn into overdrive? Will the next generation even interact with other humans, or will people become individualized entirely and surround themselves with digital avatars?

Perhaps I'm being a bit too optimistic on the acceleration, but I can't help but feel that we are truly on the cusp of a massive realignment of technology and society. What are your thoughts on AI?

ChatGPT's words are not even close to equivalent to a human's words. You have peek under the hood a little bit to understand why. ChatGPT is a prediction engine that predicts the next word in a sequence (as would be typical in its training corpus), and then applies that capability over and over again. ChatGPT has zero capability to abstract and apply its reasoning to its own thought process. ChatGPT can't wait and think about a question for a while before it starts answering.

The LLMs will continue to get better as researchers throw more parameters at the problem, but this avenue is ultimately a dead end for pursing general intelligence. ChatGPT is a neat parlor trick, but it can only make impressive-looking tech demos so long as the context is kept very narrow. Play around with it a little, and the cracks start to show.

All this is not to detract from your main thesis. Artificial general intelligence is still coming for lots of jobs at some unknown point in the future, but don't confuse ChatGPT with the herald of the jobs-apocalypse.

Apologies for the abrasive tone: I take issue with your method and authorities, but would prefer you not to take it as a personal attack.

Funny that you talk about peeking under the hood, but later refer to Hofstadter. I'm sick of hearing about this guy – and cringe at the whole little adoring culture that nerds and «hackers» have built around his books. GEB and strange loops, this intellectual isomorphism of autofellatio, self-referential kabbalistic speculation detached from all non-synthetic evidence and loudly, proudly spinning its wheels in the air. «Dude, imagine thinking about… thinking! Isn't that, like, what programmers do? Woah…» It is pretty sad when a decently smart brain belongs to a man who happens to build a brand out of a single shower thought and gets locked by incentives into inflating it endlessly! Even worse when others mistake that for an epiphany, generations of poorly socialized kids looking for the promised deeper meaning, and establishment journalists respectfully asking the matured stoner for his Expert Input. (Then again, I may simply be envious).

Last I've seen Doug speak of machine learning, it was July 2022, and he was smug because he tricked GPT-3 into hallucinations (The Economist) :

I would call gpt-3’s answers not just clueless but cluelessly clueless, meaning that gpt-3 has no idea that it has no idea about what it is saying. There are no concepts behind the gpt-3 scenes; rather, there’s just an unimaginably huge amount of absorbed text upon which it draws to produce answers. But since it had no input text about, say, dropping things onto the Andromeda galaxy (an idea that clearly makes no sense), the system just starts babbling randomly—but it has no sense that its random babbling is random babbling. Much the same could be said for how it reacts to the absurd notion of transporting Egypt (for the second time) across the Golden Gate Bridge, or the idea of mile-high vases.

This is not to say that a combination of neural-net architectures that involve visual and auditory perception, physical actions in the world, language and so forth, might not eventually be able to formulate genuinely flexible concepts and recognise absurd inputs for what they are. But that still wouldn’t amount to consciousness. For consciousness to emerge would require that the system come to know itself, in the sense of being very familiar with its own behaviour, its own predilections, its own strengths, its own weaknesses and more. It would require the system to know itself as well as you or I know ourselves. That’s what I’ve called a “strange loop” in the past, and it’s still a long way off.

How far off? I don’t know. My record for predicting the future isn’t particularly impressive, so I wouldn’t care to go out on a limb. We’re at least decades away from such a stage, perhaps more. But please don’t hold me to this, since the world is changing faster than I ever expected it to.

Narrator's voice: «they were 4 months away from ChatGPT». Today, pretty much the same text-only GPT-3, just finetuned in a kinda clever way for chat mode, can not only recognize absurd inputs, but also explain the difference between itself and the previous version, better than Hofstadter can understand. This was done on top of the previous InstructGPT tuning, also misrepresented by Experts On AI with what is basically a tech-illiterate boomer's conspiracy theory:

Artificial intelligence is an oxymoron. Despite all the incredible things computers can do, they are still not intelligent in any meaningful sense of the word.

InstructGPT is then further fine-tuned on a dataset labeled by human labelers. The labelers comprise a team of about 40 contractors whom we hired through Upwork and ScaleAI.

OpenAI evidently employs 40 humans to clean up GPT-3’s answers manually because GPT-3 does not know anything about the real world.

I told one of my sons that the hand labelers would probably fix these glitches soon. Sure enough, I tried the same questions the next day, March 19, and found that the answers had indeed been cleaned up: ….

Gary: Can I use random numbers to predict presidential elections?

GPT-3, March 18: There is no definitive answer to this question. It depends on a variety of factors, including the accuracy of the polling data and the margin of error.

GPT-3, March 19: No, you cannot use random numbers to predict presidential elections.

The labelers will probably clean this up by tomorrow, but it doesn’t change the fact that so-called AI algorithms still do not understand the world and consequently cannot be relied upon for sensible predictions or advice.

Today we know that LLMs have what amounts to concepts. Today, people have forgotten what they had expected of the future, and this Sci-Fi reality feels to them like business as usual. It is not.

Every little hiccup of AI, from hallucinations to poor arithmetic, its critics put into the spotlight and explain by there not being any real intelligence under the hood, the sort they have. The obvious intellectual capacity of LLMs demonstrated by e.g. in-context learning is handwaved away as triviality. Now, like Boretti says, «frames or symbols or logic or some other sad abstraction completely absent from real brains» – now implementing that would be a «big theoretical breakthrough». We don't know if any of that exists in minds in any substantial non-metaphorical sense, or is even rigorously imaginable, but some wordcels made nice careers out of pontificating on those subjects. Naturally, if they can be formalized, it wouldn't be much of an engineering task to add them to current ML – the problem is, such reification of schemes only makes things worse. The actual conceptual repertoire developed by humble engineers and researchers over decades of their quest is much more elegant and expressive, and more deserving of attention today.

Do you really think that the idea of «predict next word in a sentence» provides sufficient insight about the under-the-hood intelligence of LLMs, when it is trivial to change the training objective to blank-filling, and RLHF guarantees that there exists no real dataset for which the «predicted» – or actually, chosen – token is in fact the most likely one in that context?

Or that the process of self-attention , actually undergirding those «predictions», is not «reasoning about reasoning» (because it cannot attend to… itself, the way you can attend to patterns of neuron spikes, presumably?

Or that recurrence is hard to tack onto transformers? (or for that matter specialized cognitive tools, multimodality etc.?)

And so on and so forth. But ultimately after years and years of falsified forecasts and blatant displays of ignorance by skeptics, it is time to ask oneself: isn't this parlor trick of a stochastic parrot impressive enough to deserve more respect, at least, than gimmicks of our fraudulent public intellectuals? It does, after all, make more sense when talking, and is clearly more able to grapple with new evidence.

@2rafa reasonably observes that human intelligence may be not so different from next word prediction. Indeed, if I close this tab (in Obsidian) and return to it in half an hour, I may forget where I was going with this rant and start with the most likely next word; and struggling for words when in an intense conversation is an easy way to see how their statistical probabilities affect reasoning (maybe I'm projecting my meta-awareness, lol). But even if the substrate is incompatible: why do we think our one is better? Why do we think it supports «real reasoning» in a way that mutiplying matrices, estimating token likelihood, or any other level of abstraction for LLM internals does not?

It is not obvious that the human brain is anywhere near optimal for producing intelligent writing, or for much of anything except being itself. We didn't evolve to be general-purpose thinkers, we are just monkeys who had our brains scaled up under selective pressures in some limited range of environments, with stupid hacks like the phonological loop and obsession with agency. Obviously LLMs are not optimal either (we are only pursuing them out of convenience), but they might still be better at producing our own text.

A plane doesn't flap its wings, but it definitely flies – and even though it's less efficient per unit of mass, in the absolute sense it does something no bird could. We do not understand birds well enough to replicate them from scratch, nor do we need to. Birds could never achieve enough for a truly general flight, «move anything across the Earth» kind.

We, too, aren't enough for truly general intelligence. It remains to be shown that LLMs aren't better fit for that purpose.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats