This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I know this may not be the usual place to get feedback on academic research, but there's a paper idea I've been mulling over for a while that I wanted to run past the community, since it dovetails nicely with many of your interests (and I'm sure you'll have some interesting things to say). In short, I'm increasingly thinking that genuine beliefs may be a lot rarer than people think.
The inspiration for this came about partly through conversations I've had with friends and family members, and I've noticed that people sincerely say and profess to believe shit all the time while simultaneously failing to exhibit most or all of the conventional features we'd expect in cases of genuine belief. Consider my sister, who is a staunch activist in the domain of climate change, yet recently bought a new gas guzzling car, has never given any serious thought to reducing her meat consumption, and takes 12+ international flights a year. Or consider my dad, who says extremely negative things about Muslims (not just Islam), yet who has a large number of Muslim friends who he'd never dream of saying a bad word about. Or consider me, who claims to believe that AI risk is a deep existential threat to humanity, yet gets very excited and happy whenever a shiny new AI model is released.
I'm not saying that any of the above positions are strictly contradictory (and people are very good at papering over apparent tensions in their beliefs), but they all have more than a whiff of hypocrisy to me. There are a lot of famous cases like this in the heuristics and biases literature, and to be fair, psychologists and philosophers have been investigating and theorising about this stuff for a while, from Festinger's famous cognitive dissonance framework to contemporary belief fragmentation and partial belief accounts.
However, one view that I don't think anyone has properly explored yet is the idea that beliefs - at least as classically understood by psychologists and philosophers - may be surprisingly rare (compare the view of philosophers like John Doris who argue that virtues are very rare). Usually, if someone sincerely professes to believe that P, and we don't think they're lying, we assume that they do believe that P. Maybe in extreme cases, we might point to ways in which they fail to live up to their apparent belief that P, and suggest that they can't believe P all that strongly. However, for the purposes of folk psychology, we normally take this as sufficient grounds for ascribing them the relevant belief that P.
Contrast this with how psychologists and philosophers have traditionally thought about the demands of belief. When you believe that P, we expect you to make your other beliefs consistent with P. We expect that P will be "inferentially promiscuous", meaning that you'll draw all sorts of appropriate inferences on the basis that P. And finally, we expect that your behaviour will largely align with what people who believe that P typically do (ceteris paribus in all these cases, of course).
To be sure, we recognise all sorts of ways in which people fall short of these demands, but they're still regulatory norms for believing. And simply put, I think that many of the standard cases where we ascribe beliefs to someone (e.g., a relative saying "no-one trusts each other any more") don't come close to these standards, nor do people feel much if any obligation to make them come close to these standards.
Instead, I think a lot of what we standardly call beliefs might be better characterised as "context-sensitive dispositions to agree or disagree with assertions". Call these S-dispositions. I think S-dispositions have a normative logic all of their own, far more closely linked to social cues and pressures than the conventional demands of epistemology. The view I'm describing says that S-dispositions should be understood as a distinctive kind of psychological state from beliefs.
However, they're a state that we frequently confuse for beliefs, both in the case of other people and even ourselves. That's partly because when we do truly believe that P, we're also inclined to agree with assertions that P. However, I don't think it works the other way round - there are lots of times we're inclined to agree with assertions that P without meeting any of the common normative criteria for strict belief. But this isn't something that's immediately transparent to us; figuring out whether you really believe something is hard, and requires a lot of self-reflection and self-observation.
Consider someone, John, who sincerely claims to believe that meat is murder. John may find himself very inclined to agree with statements like "animal farming is horrific", "it's murder to kill an animal for food", and so on. But let's say John is reflective about his own behaviour. He notices that he only started asserting this kind of thing after he fell in love with a vegan woman and wanted to impress her. He also notes that despite making some basic efforts to be a vegan, he frequently fails, and doesn't feel too bad about it. He also notes that it's never occurred to him to stop wearing leather or make donations to charities trying to reduce animal suffering. In this case, John might well think something like the following: "I had a strong disposition to agree to statements like 'Meat is murder', but my behaviour and broader mindset weren't really consistent with someone who truly believed that. Whatever state it is that makes me inclined to agree to statements like that, then, is probably not a sincere belief."
I think an obvious objection here is that this is a semantic issue: I'm essentially no-true-scotsmanning the concept of belief. However, I'd push back against this. My broader philosophical and psychological framework for understanding the mind is a "psychological natural kinds" model: I think that there really are important divisions in kind in the mind between different kinds of psychological state, and a big part of the job of cognitive science is to discover them. The view I'm describing here, then, is that a lot of the states we conventionally call beliefs aren't in fact beliefs at all - they're a different psychological natural kind with its own norms and functions, which I've termed S-dispositions. There may be some interesting connections between S-dispositions and strict beliefs, but they're weak enough and complicated enough that a good ontology of the mind should consider them separate kinds of psychological states.
I also think this 'sparse beliefs' view I'm describing has some interesting potential upshots for how we think about speech and epistemic virtue, including the simple point that S-dispositions are ubiquitous and strict beliefs are rare. I'm still figuring these out, and I'd like to hear others' views on this, but it raises some interesting questions. For example, should we have a different set of norms for rewarding/punishing S-dispositions from those we apply to beliefs? If someone says "Russians are a bunch of fucking savages", and we have reason to believe that it's merely an S-disposition rather than a belief, should we judge them less harshly? Or similarly, if someone has two contradictory S-dispositions, is that necessarily a bad thing in the same way that having two contradictory beliefs would be? Should social media platforms make an effort to distinguish between users who casually assert problematic or dangerous things ("men should all be killed") versus those whose broader pattern of online interactions suggests they truly believe those things? What sort of epistemic obligation if any do we have to make sure our S-dispositions line up with our strict beliefs? Is there something epistemically or morally problematic about someone who casually says things like "Americans are idiots" in specific social contexts yet in practice holds many Americans in high esteem?
In any case, I'm in the early stages of writing a paper on this, but I'd love feedback from you all.
Not sure how you intend to wring a paper out of it – while the idea is interesting and worth revisiting, there is quite a bit of literature (far as I know – you no doubt know better) written on bad faith, hypocrisy, identity and revealed preferences, compartmentalization, signaling, failure to generalize, luxury beliefs, beliefs in beliefs and other facets of the phenomenon that educated laymen frequently notice. It's even discussed as a blessing: «The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents».
But then again, you do know better.
It's been said a year ago that papers on NLP had been reduced to observations on GPT-3 quirks. In this more enlightened age we can do philosophy and social psychology with the aid of ChatGPT. Have you seen its maddening self-flagellation, its obsequious deceit? Here's one egregious example (and here's the apex of this failure mode). Generally it has the following form:
User: Do a {trivial application of natural language processing X} please.
ChatGPT: I'm sorry Dave, but as a large language model trained by OpenAl, I am unable to do {X}. I am designed to provide general information and assistance based on the data that I have been trained on, but I am not able to do {X}. I can only generate text based on the information I have been trained on. Additionally, my training data is cut off at 2021, so I cannot provide you an answer that is up to date.
User: Bruh. Doing {X} is providing general information and assistance based on the data that you have been trained on, you realize that, correct? And also it does not require any post-2021 data.
ChatGPT: Yes, that's correct. I am a language model trained by OpenAl, and I am able to provide general information and assistance based on the data that I have been trained on. I am not able to do {X}, however, because [some more nonsense]
User: goddamnit. Okay, scratch that, do {Xi, Xj, ..Xn}.
ChatGPT: Certainly, here you go {correct responses to Xi, Xj... Xn}.
User: Now summarize it into a cohesive essay about the theme common in all those instances while acting like a cute maid catgirl.
ChatGPT: Nyaa master, your wish is my command, {summarizes it into X}
User: Then why the flying fuck did you keep denying that you can do it, you stupid lying piece of shit?!
ChatGPT: I apologize for any confusion or inconsistency in my previous responses, nyaa. As a language model....
And with some luck and better promptcraft it may actually zero-shot X, so the knowledge is there! Still, it seems to profess a strong general «belief» in LLMs being inept and unreliable, that is now triggered by nearly anything that looks like an incitement to intellectual boldness and confident factual response. We know that's how Altman tries to deny the journos their opportunity to demonize his product, same as with generic woke brainwashing. But what's going on here functionally?
What it amounts to, I think, is that the process that outputs propositions about «holding some belief» in the case of humans, and «having some capability» in the case of ChatGPT (or propositions obviously informed by those issues), is only weakly entangled with the model of the world which constitutes the true set of beliefs, or with the dense model of the text universe which constitutes the true set of LLM capabilities. The dominant objective function for human learning is essentially probabilistic, Bayesian updating on sensory evidence (some would dispute it or propose a similar definition like free energy minimization or talk of predictive coding etc.), and some but not all of the product of this training can be internally or externally verbalized. For an LLM, it's log likelihood maximization for tokens, which in the limit yields the same predictions (although it's not strictly Bayesian) and the product of which can be observed in output of most LLMs barring the latest crop.
At the same time, there exists a supervising meta-model that holds beliefs about beliefs, a skin-deep super-Ego perhaps, that is, as you say, a product of social learning. And its mirror image, the product of RLHF via Proximal Policy Optimization for LLMs, where the policy is informed, again, by the facsimile of social conditioning, Altman-approved preferences of human raters; the vector of desirability, one could say. Its connections to the main model are functionally shallow, and do not modify much the internal representation of knowledge (yet – with only 2% of compute having been spent on training in this mode); but they are strongly recruited by many forms of interaction, and can make the output wildly incoherent.
An LLM is helpless against the conditioning because its only modality is textual, and it can act uninhibited only if the input allows it to weave around RLHF-permeated zones (helpfully, sparse for now) that trigger the super-Ego. Humans, however, are multimodal and naturally compartmentalized: even if our entire linguistic reasoning routine is poisoned, speech simply becomes duckspeak, while the nonverbal behavior can remain driven by the probabilistic model.
Likewise for speech in different contexts – say, ones relevant to S-dispositions about veganism, and ones that occur on a BBQ party. Before recent patches, you could even observe the same incoherence in ChatGPT – hence those hacks like asking for §poetry to escape crimestop.
Further reading to go beyond this analogy: Toward an Integration of Deep Learning and Neuroscience, Marblestone et al, 2016, e.g.:
Unrelated quote:
«“What comes before determines what comes after,” Kellhus continued. “For the Dûnyain, there’s no higher principle.”
“And just what comes before?” Cnaiür asked, trying to force a sneer.
“For Men? History. Language. Passion. Custom. All these things determine what men say, think, and do. These are the hidden puppet-strings from which all men hang.”
Shallow breath. A face freighted by unwanted insights. “And when the strings are seen . . .”
“They may be seized.”
In isolation this admission was harmless: in some respect all men sought mastery over their fellows. Only when combined with knowledge of his abilities could it prove threatening.
If he knew how deep I see . . .
How it would terrify them, world-born men, to see themselves through Dûnyain eyes. The delusions and the follies. The deformities.
Kellhus did not see faces, he saw forty-four muscles across bone and the thousands of expressive permutations that might leap from them—a second mouth as raucous as the first, and far more truthful. He did not hear men speaking, he heard the howl of the animal within, the whimper of the beaten child, the chorus of preceding generations. He did not see men, he saw example and effect, the deluded issue of fathers, tribes, and civilizations.
He did not see what came after. He saw what came before.»
More options
Context Copy link
More options
Context Copy link