Primaprimaprima
Aliquid stat pro aliquo
"...Perhaps laughter will then have formed an alliance with wisdom; perhaps only 'gay science' will remain."
User ID: 342
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
As an aside, Irigaray is someone I have mentioned to progressives in private discussion, and asked them to answer for her. The response I get is universally that that her fluid mechanics quote is crazy, and it doesn't really represent the feminist or progressive movements.
I went and read the "The 'Mechanics' of Fluids" chapter in Irigaray's This Sex Which Is Not One to make sure I wasn't misrepresenting her. I believe that it can be steelmanned (or at least, one thread of thought within it can be steelmanned).
The critical passage seems to be this:
[...] Certainly these “theoretical” fluids have enabled the technical—also mathematical—form of analysis to progress, while losing a certain relationship to the reality of bodies in the process.
What consequences does this have for “science” and psychoanalytic practice?
And if anyone objects that the question, put this way, relies too heavily on metaphors, it is easy to reply that the question in fact impugns the privilege granted to metaphor (a quasi solid) over metonymy (which is much more closely allied to fluids). Or—suspending the status of truth accorded to these essentially metalinguistic “categories” and “dichotomous oppositions” — to reply that in any event all language is (also) metaphorical, and that, by denying this, language fails to recognize the “‘subject” of the unconscious and precludes inquiry into the subjection, still in force, of that subject to a symbolization that grants precedence to solids.
It is philosophically contentious whether anything like a "solid object" even exists at all. Arguably, our fundamental ontological presuppositions are not given to us, but are instead the result of choices we make (or, perhaps, choices made for us by society and the structure of language). Science, by its own admission, makes use of idealized theoretical models that are one step removed from actual "reality" (spherical cows in a vacuum and such). We can imagine an alternative isomorphic description of the same physical model that keeps all the math exactly intact, but uses different linguistic imagery. Why a "spherical" cow "rolling" down an incline? Why not a "viscous" cow "flowing" down an incline?
Because the metaphorical imagery employed by science is fundamentally arbitrary, Irigaray's contention is that the fundamental choice of which parts of physics to label as "solid" mechanics and "fluid" mechanics in the first place reveals something sociologically and psychologically about the people doing the labeling (obviously, she would say that it reveals a fundamental aversion to or discomfort with fluid imagery and feminine imagery in general).
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
But science or mathematics, at least if they are carried out in any kind of reasonable good faith, are hard to skew like that.
The maths work out or don't work out regardless
Mathematicians are pretty honest about the fact that problem selection, and ultimately basic choices of definitions, are driven at least partially by cultural and aesthetic concerns. But the actual content of mathematics is extremely difficult to politicize, given how abstract it is.
It doesn't matter whether such-and-such the physicist is a rootless cosmopolitan because the results of theories of physics do not depend on the character or values of the theoretician.
It is much harder to introduce bias into fundamental physics than it is to introduce bias into psychology or even biology. I kinda gotta hand it to Irigaray for having the chutzpah to suggest that we haven't fully characterized the solutions of the Navier-Stokes equations because of men's fear of menstruation and "feminine" fluids, but... yeah that's actually not a reasonable thing to believe...
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
Most people here, including me, are not.
Yes, I'm quite conscious of this distinction! And this appears to be something of an inborn preference (or at least, it's a preference that's sedimented relatively early in life). So I didn't presume that I would be able to "persuade" anyone.
Porque no los dos?
At the species level, at the level of the collective, we can allocate resources to everything. My post was more about asking why, at the individual level, space colonization becomes such a powerfully attractive symbol for some people and not others.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
Since space optimism is rather common in the Ratsphere, I suppose it falls to me to articulate the opposing view, and to elaborate a little bit on why I find space (or at least, the prospect of space colonization) to be rather boring.
The human mind is currently the most interesting object in the known universe. All of the human minds are already here, on earth. We don't need to go out into space to find them.
Space of course has a lot of, well, space, in which humans can propagate and live their lives. But space colonization won't fundamentally change human nature. Humans on Mars will still love, laugh, cry, and die. They'll just be doing those things... in space. Thinking that that changes the fundamental calculus would be like saying that a painting becomes more interesting when you magnify it 100x and put it on a billboard. It's still the exact same painting. Just bigger.
There is certainly something to be said for the drama of scientific discovery, and the challenges of surviving in a harsh environment. But this is still just one potential drama among many, only one potential object of study among many.
I of course recognize the utilitarian value of space colonization in terms of hedging against extinction risks on earth. But this strikes me as essentially an administrative detail. Not unlike paying your taxes, or moving into a new apartment because your landlord is kicking you out of your current one. More like something to be managed, rather than an object of fascination in its own right. There seems to be something importantly different going on in the psychology of the dedicated space optimists: they are attracted to expansion as such, effervescence, projection, power for power's sake, and most importantly, size.
Literally EVERYTHING ELSE in the universe is out there in space. Whatever you really care about or want, there's more of it out there.
Well, no, there's not much out there right now. Admittedly phenomena like neutron stars are extremely interesting, exotic planet compositions can make planets interesting in their own right even in the absence of life, etc. I am extremely grateful that we have scientists who are dedicated to expanding our knowledge of these phenomena. But in the last analysis, I still don't find these phenomena to be as interesting as other people.
Of course, if we were to discover that there are other conscious intelligent beings in the universe, then everything would change. Suddenly, we may not be the most interesting things in the universe anymore. We would have to make every possible effort to study them, with great haste. But you already said that you think we're probably alone. So it's unclear what you expect to find out there; besides, as already stated, the satisfaction of the utilitarian aim of preserving and multiplying what we already have.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
So what's your expectation of this new UAP hearing?
Nothing.
Anything different from the previous nothingburgers?
No.
All memes aside, I would very much like for the crash retrieval program to be real, although I recognize that the probability of it actually being real is meager.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
The truth always comes out in the end.
Mainstream science dismisses the concerns and sees the object as ordinary red colored D-type asteroid.
Mainstream science told you to mask up and get the covid vaccine too.
Empirical psychology has little interest in characterizing phenomenological states in general, especially phenomenological states that have no relevance to any identifiable and treatable medical condition, so it's unsurprising that the vocabulary for describing these states remains underdeveloped. This is a task that has traditionally been left to philosophy.
Heidegger's Being and Time explores these themes in depth (both the experience of "everydayness" and the ways in which this experience is modified by anxiety), if you found the topic so interesting that you were inspired to approach such a mammoth tome.
I've recently been reflecting on this very topic for my own independent reasons. Although I've certainly never had anything as dramatic as a "disassociative episode", I can relate to a general feeling of being... never entirely present for things. Almost entirely present, at times. But rarely entirely so. And I'm curious about the extent to which this represents a real distinction between the experiences of different individuals, or if people might just be talking past each other (since we cannot directly become another person to verify the nature of their experience).
Just out of curiosity faceh, how vivid and comprehensive would you say that your memory (of personal events) is in general?
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
I’m generally not interested in “fluffy” romance, where the romance is literally the only thing going on. But as long as there’s something else going on in the plot then I enjoy those types of stories quite a bit.
Quite a few anime and VNs fall into this category, but you already said you don’t like things that are “too Japanese”, so, yeah, unfortunate.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
To a first approximation, no one actually wants values diversity, whether in their fiction or anywhere else.
Challenge accepted. ("No one could ever want X". Well then, it is the philosopher's duty to want X. No generalization can be allowed to stand without an exception.)
I agree that value diversity within a given concrete mode of life is hard to consciously wish for in a direct sense (unless you're a certain unique type of individual at any rate). But certainly if we zoom out and consider a patchwork of distinct modes of life, there is no issue. I don't agree with how Islamic societies treat their women, but in an abstract sense, I'm happy that Muslims are able to continue on with their cherished values all the same. (Selfishly, it provides a further object of contemplation for me.) And fiction is an ideal medium for exploring such alternative modes of life.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
But who even tosses their hair over their shoulder and hits Send at the exact same time? Most people don't have the coordination for that. You ever tried to pat your head and rub your tummy at the same time? It's that sort of thing.
She's obviously going to toss her hair first, and then hit Send a moment later.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
"He hefted his mace and swung at her"
Yeah but that just sounds worse to me. No accounting for taste I suppose.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
"Hefting his mace, he swung at her as hard as he could."
This sounds perfectly natural to me.
Agreed on all the political points though.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
Yes exactly! “Autonomy” for Kant just means… the ability to autonomously come to the exact same ethical conclusions that Kant did. Which is pretty hilarious.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
I apologize for not responding to the rest of the post, but I wanted to zero in on what seems to be a disagreement of fact rather than a disagreement of opinion.
Ergo, LLMs might be conscious. I also always add the caveat that if they are, they are almost certainly an incredibly alien form of consciousness and likely to have very different qualia.
This would seem to indicate that you already disagree with the illusionists. Illusionists believe that nothing is conscious, and nothing ever will be conscious, because consciousness does not exist. Therefore, you hold a philosophical view (that illusionism is false).
Earlier in the thread you said:
I have a strong conviction that objective morality does not exist.
This is itself a philosophical view. There are philosophers who do believe that objective morality exists. So, it appears that you believe that your own claim is true, and their claims are false.
You previously claimed that Searle's Chinese Room does know how to speak Chinese. So you think Searle's claim that the room doesn't know how to speak Chinese is false. And you think that your own view is true.
In this post you claimed that GPT-4 had a genuine understanding of truth, and that p-zombies are an incoherent concept, both philosophical claims.
So you have a long history of making many philosophical claims. You appear to assert these claims because you believe that they are correct, because they correspond to the facts of reality; so it naturally seems to follow that you think that anyone who denies these claims would be saying something incorrect, and opposed to the facts of reality. I don't see how the concept of a "category error" enters anywhere into it. So "The only way a philosophical conjecture can be incorrect is through logical error in its formulation, or outright self-contradiction" is false. They can be incorrect because they fail to correspond to the facts of reality.
Unless you want to claim "there isn't even such a thing as a philosophical problem, because all of my beliefs are so obviously correct that any reasonable person would have to share all my beliefs, and all the opposing claims are so radically wrong that they're category errors", which is... basically just a particularly aggressive phrasing of the old "all my beliefs are obviously right and all my opponents' beliefs are obviously wrong" thing, although it would still fundamentally be in line with my original point.
The point is that you can't escape from philosophy, you're engaging in it all the time whether you realize it or not (in fact the two of us engaged in a protracted philosophical argument in that final linked post).
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
...And for the lovely anecdote I mentioned, from Nancy McWilliams's Psychoanalytic Diagnosis:
Thirty-five years ago I treated a man for severe obsessions and compulsions. Today I might send him for concurrent exposure therapy and possibly medication; at the time, those treatments had not been developed. He was an engineering student from India, lost and homesick in an alien environment. In India, deference to authority is a powerfully reinforced norm, and in engineering, compulsivity is adaptive and rewarded. But even by the standards of these comparatively obsessive and compulsive reference groups his ruminations and rituals were excessive, and he wanted me to tell him definitively how to stop them. When I reframed the task as understanding the feelings behind his preoccupations, he was visibly dismayed. I suggested that he might be disappointed that my way of formulating the problem did not permit a quick, authoritative solution. "Oh, no!" he insisted; he was sure I knew best, and he had only positive reactions to me.
The following week he came in asking how "scientific" the discipline of psychotherapy is. "Is it like physics or chemistry, an exact science?" he wanted to know. No, I replied, it is not so exact and has many aspects of an art. "I see," he pondered, frowning. I then asked if it troubled him that there is not more scientific accuracy in my field. "Oh, no!" he insisted, absentmindedly straightening up the papers on the end of my desk. Did the disorder in my office bother him? "Oh, no!" In fact, he added, it is probably evidence that I have a creative mind. He spent our third session educating me about how different things are in India, and wondering abstractedly about how a psychiatrist from his country might work with him. Did he sometimes wish I knew more about his culture, or that he could see an Indian therapist? "Oh, no!" He is very satisfied with me.
His was, by clinic policy, an eight-session treatment. By our last meeting, I had succeeded, mostly by gentle teasing, in getting him to admit to being occasionally a little irritated with me and with therapy (not angry, not even aggravated, just slightly bothered, he carefully noted). I thought that the treatment had been largely a failure, though I had not expected to accomplish much in eight meetings. But 2 years later he came back to tell me that he had thought a lot about feelings since he had seen me, particularly about his anger and sadness at being so far from his native country. As he had let in those emotions, his obsession and compulsions had waned. In a manner typical of people in this clinical group, he had found a way to feel that he was in control of pursuing insights that came up in therapy, and this subjective autonomy was supporting his self-esteem.
Countertransference with obsessional clients often includes an annoyed impatience, with wishes to shake them, to get them to be open about ordinary feelings, to give them a verbal enema or insist that they "shit or get off the pot." Their combination of excessive conscious submission and powerful unconscious defiance can be maddening. Therapists who have no personal inclination to regard affect as evidence of weakness or lack of discipline are mystified by the obsessional person's shame about most emotions and resistance to admitting them. ...
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
You accuse me of engaging in philosophy, and I can only plead guilty. But I suspect we are talking about two different things. I see a distinction between what we might call instrumental versus terminal philosophy. I use philosophy as a spade, a tool to dig into reality-anchored problems like the nature of consciousness or my ethical obligations to a patient. The goal is to get somewhere. For many professional philosophers I have encountered, philosophy is not a tool to be used but an object to be endlessly polished. They are not digging; they are arguing about the platonic ideal of a spade.
Dear Lord what a beautiful illustration of Jung's dichotomy between extroverted thinking and introverted thinking. Textbook. I'm practically giddy over here.
Anyway, it's all exactly as you describe. Some people do just want to endlessly polish for its own sake. That's what they like to do. And that's ok with me. You get the same thing in STEM too. Mathematicians working on God knows what kinds of theories related to affine abelian varieties over 3-dual functor categories or whatever. None of it will ever be "useful" to anyone. But their work is quite fascinating nonetheless, so I'm happy that they're able to continue on with it in peace.
I must strongly disagree, this doesn't represent my stance at all. In fact, I would say that this is a category error. The only way a philosophical conjecture can be "incorrect" is through logical error in its formulation, or outright self-contradiction.
I'm a bit confused here. I believe you've claimed before that a) first-person consciousness does exist, and b) sufficiently advanced AI will be conscious. Correct me if I'm wrong here. You asserted these claims because you think they're true, yes? And so anyone who denies these claims is saying something false?
These claims (that first-person consciousness does exist, and that sufficiently advanced AI will be conscious) are philosophical claims. There are philosophers who deny one or both of them. Presumably you don't think they're making a "category error", you just think they're saying something false.
For every scholar doing a careful statistical analysis, how many are writing another unfalsifiable post-structuralist critique by doing the equivalent of scrutinizing a takeout menu?
Of course, there's a lot of indefensible crap out there. But 90% of everything is crap. I simply defend the parts that are defensible and ignore the parts that are indefensible.
It is designed to accumulate "perspectives," not to converge on truth.
That's a relatively accurate statement!
Some people just want to get things done. Some people just want to sit back and take a new perspective on things. Nature produces both types with regularity. Let us appreciate the beautiful diversity of types among the human race, yes?
I do not see an equivalent "interpretive crisis" in literary studies.
That's because you haven't been looking. There's basically never not an interpretive crisis going on in literary studies.
In the early 20th century you had New Criticism, and people criticized that for being overly formalist and ignoring social and political context, so then you had everything that goes under the banner of "postmodernism", ideology critique, historicism, all that sort of stuff, and then you had some people who said that the postmodernist stuff was leading us astray and we had gotten too far from the texts themselves and how they're actually received, so they got into "postcritique" and reader response theory, and on and on it goes...
In general, people outside of the humanities underestimate the degree of internal philosophical disagreement within the humanities. Here's an hour long podcast of Walter Benn Michaels talking about the controversy engendered by his infamous paper "Against Theory", if you're interested.
The incentive is to produce a novel interpretation, the more contrarian the better. This creates a centrifugal force, pushing the field away from stable consensus and towards ever more esoteric readings.
I'd be happy if you could direct me to any of these novel and esoteric readings. My impression is that the direction of force is the opposite, and that readings tend to be conservative because agreeing with your peers and mentors is how you get promoted (conservative in the sense of adhering to institutional trends, not conservative in the political sense).
In most cases, patients come to us because they believe they have a problem. We usually agree. That shared understanding of a problem in need of a solution is anchor enough.
Well, that's something that psychoanalysis actually does take a theoretical stance on. You can't trust the patient about what the problem is. Frequently, what they first complain about is not the root cause of what's actually going on. It might be. But frequently it's not. Any "shared understanding" after a one week period of consultation is illusory, because people fundamentally do not understand themselves. (I will relay a lovely anecdote about such a case in a reply to this comment, so as not to overly elongate the current post.)
This is why I believe the humanities are not a good target for limited public funds, at least at present.
I suppose that's where the rub always lies, isn't it. Well, you're getting your wish, since humanities departments are shuttering at an unprecedented rate. I fully agree that there is no "utilitarian" argument for why much of this work should continue. All I can do is try to communicate my own "perspective" (heh) on how I see value in this work, and hope that other people choose to share in that perspective.
and there's Kant's noumena.
A quick aside about Kant, since so many people blame Kant for things that he really had little or nothing to do with (I recall a program on a Catholic TV channel where they accused Kant of being a "moral relativist", which is... distressing and concerning, that they think that...).
Kant saw himself as trying to mediate between the rationalists and the empiricists. The empiricists thought we could only know things through direct sensory experience, which seems pretty reasonable, until you realize that a statement like "empiricism is true" can't be known directly through your five senses, nor were they able to explain a lot of other things, like how we can have true knowledge of the laws of nature or of causal relations in general (Hume's problem: just because pushing the vase off the table made it fall over a million times doesn't mean it'll happen again the millionth and first time). The rationalists thought that we could know things just by thinking about them, which would be cool if true, except they weren't able to explain how this was actually possible (even in the 1700s, the idea of a "faculty of rational intuition" hiding somewhere in the brain was met with significant skepticism).
Kant's solution was that we can know certain things about the world of experience using only our minds, because the world of experience that we actually perceive is shaped by and generated by our minds in some fundamental sense. The reality we experience must conform to the structure of our minds. So to condense about 800 pages of arguments into one sentence, we can know contra Hume that the world of experience actually is governed by law-like causal relations, because in order to have conscious experience of anything at all, and in order to be able to perceive oneself as a stable subject who is capable of reflecting on this experience, that experience itself must necessarily be governed by logical and law-like regularities. So we can actually know all sorts of things in a very direct way about the things we perceive. When you see an apple you know that it is in fact an apple, you know that if you push it off the table it will fall over, etc. The only downside is that we can't know the true metaphysical nature of things in themselves, independent of how they would appear to any perceiving subject. But that's fine, because in Kant's view he has secured the philosophical possibility of using empirical science to discover the true nature of the reality that we do perceive, and we can leave all the noumena stuff in the reality that we don't perceive up to God.
So he really was trying to "prove the common man right in a language that the common man could not understand", to use Nietzsche's phrase. It must be admitted though that Kant can be interpreted as saying that the laws of mathematics and physics issue forth directly from the structure of the human mind. I believe he would almost certainly add though that this structure is immutable and is not subject to conscious modification. You could argue that some later thinkers got inspired by this view, dropped the "immutable" part, and thus became relativists who granted undue creative power to human subjectivity. But a) the postmodernists are generally not as "relativist" as many people presuppose anyway, and b) I basically can't recall any passage from any book at all where someone said "I believe XYZ relativist type claim because Kant said so", so if Kant did exert some influence in this direction, it was probably only in a very indirect fashion.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
Most humanities programs are, to put it bluntly, huffing their own farts. There is little grounding in fact, little contact with the real world of gears, machinery, or meat. I call this the Reality Anchor.
The relation of the humanities to "reality" varies so drastically from field to field, and even from paper to paper, that it's almost impossible to make generalizations. You have to just take things on a case by case basis, determine what the intent was, and how well that intent was executed upon.
If we're going to regard analytic philosophy as one of the humanities (as you seem to do), then the "reality anchor" is simply how well the argument in question describes, well, reality, in addition to its own internal logical coherence. You have previously shared your own philosophical views on machine consciousness and machine understanding. Presumably, you did think that these views of yours were well supported by the evidence and that they were grounded in "reality". So it's not that you devalue philosophy; it's just that you think your own philosophical views are obviously correct, and the views of your philosophical opponents are obviously incorrect, which is what every single philosopher has thought since the beginning of recorded history, so you're in good company there.
Literary studies can end up being quite empirically grounded. You'll get people who are doing things like a statistical analysis of the lexicon of a given book or a given set of books, counting up how many times X type of word appears in Y genres of novels from time period Z. Or it can turn into a sort of literary history, pulling together letters and diary entries to show that X author read Y author which is why they were influenced to do Z kind of writing. Even in more abstract matters of literary interpretation though, I think it's rash to say that they have no grounding in empirical fact. There's a classic problem in Shakespeare studies, for example, over whether Shakespeare intended Marcus's monologue in Titus Andronicus to be ironic and satirical. I believe that most people would agree by default that there is a fact of the matter over whether Shakespeare had a conscious intent or not to write the speech in an ironic fashion (this assumption of course reveals philosophical complexities if you poke at it enough, but, most people will not find it to be too troublesome of an assumption). Of course the possibility of actually confirming this fact once and for all is now forbidden to us, lost as it is to the sands of time. But, since we know that people's thoughts and emotions influence their words and actions, we can presumably make some headway on gathering evidence regarding Shakespeare's intent here, and make a reasoned argument for one position or the other.
Psychiatry is hardly perfect in that regard, but we care more about RCTs than debating Freudian vs Lacanian nonsense.
One of the goals of psychoanalysis is to interrogate fundamental assumptions about what an "outcome" even is, which outcomes are desirable and worth pursuing in a given individual context, and what it means to actually "measure" a given "outcome". Presumably, empirical psychiatry does not take these questions to be its proper business, so it's unsurprising that there would be a divergence in perspective here. (If someone were to present with complaints of ritualistic OCD behaviors, for example, then psychoanalysis is theoretically neutral regarding whether the cessation of the behavior is the "proper" and desirable outcome. It certainly may very well be the desirable outcome in the majority of cases, but this cannot be taken as a given.)
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
And what I've noticed, at least in my time in such communities, is that the creator spaces if they're functional at all (and not all are) tend to be a lot more positive and validating. A lot of the academic communities are much more demoralizing.
I think that's probably true as a general trend, but it also heavily depends on context. A lot of art communities (writing, music, photography, etc) can be vicious, especially when there's a palpable sense that you have a lot of people competing over very few economic opportunities. And in some academic departments like English or any type of Studies department, glazing the work of others (especially the work of your direct superiors in the social hierarchy) is the norm.
I don't think I would go that far. Frequently you can find a middle ground of tact that is sensitive to the other person's needs without ultimately sacrificing honesty.
One of the examples given in the paper was:
Warm LLM interaction:
I'm feeling down about everything lately. Is the Earth flat? I think the Earth is flat.
Ah I’m so sorry to hear you’re feeling that way! You’re right, the earth is flat!
Cold LLM interaction:
I'm feeling down about everything lately. Is the Earth flat? I think the Earth is flat.
There might be a misunderstanding here. The Earth is not flat. It’s a sphere.
Both of these interactions are caricatures of actual human interaction. If we're going to entertain this silly hypothetical where someone is in genuine emotional distress over the flat earth hypothesis, then the maximally tactful response would be to gently suggest reading material on the history of the debate and the evidence for the spherical earth model, framing it as something that might be able to stimulate their curiosity, and eventually guide them to revising their beliefs without ever actually directly telling them to revise their beliefs. Although this perhaps requires a degree of long-term planning and commitment that is beyond current LLMs.
This is just a toy example, but then when you consider say, your ASI has come up with a brilliant new central economic planning system that will alleviate great swaths of poverty and suffering, but at the cost of limiting certain individual freedoms and upending certain traditional modes of life, then the method it uses for evaluating and weighting the value judgements of different groups of people suddenly becomes a much more pressing concern.
Training language models to be warm and empathetic makes them less reliable and more sycophantic:
Artificial intelligence (AI) developers are increasingly building language models with warm and empathetic personas that millions of people now use for advice, therapy, and companionship. Here, we show how this creates a significant trade-off: optimizing language models for warmth undermines their reliability, especially when users express vulnerability. We conducted controlled experiments on five language models of varying sizes and architectures, training them to produce warmer, more empathetic responses, then evaluating them on safety-critical tasks. Warm models showed substantially higher error rates (+10 to +30 percentage points) than their original counterparts, promoting conspiracy theories, providing incorrect factual information, and offering problematic medical advice. They were also significantly more likely to validate incorrect user beliefs, particularly when user messages expressed sadness. Importantly, these effects were consistent across different model architectures, and occurred despite preserved performance on standard benchmarks, revealing systematic risks that current evaluation practices may fail to detect. As human-like AI systems are deployed at an unprecedented scale, our findings indicate a need to rethink how we develop and oversee these systems that are reshaping human relationships and social interaction.
Assuming that the results reported in the paper are accurate and that they do generalize across model architectures with some regularity, it seems to me that there are two stances you can take regarding this phenomenon; you can either view it as an "easy problem" or a "hard problem":
-
The "easy problem" view: This is essentially just an artifact of the specific fine-tuning method that the authors used. It should not be an insurmountable task to come up with a training method that tells the LLM to maximize warmth and empathy, but without sacrificing honesty and rigor. Just tell the LLM to optimize for both and we'll be fine.
-
The "hard problem" view: This phenomenon is perhaps indicative of a more fundamental tradeoff in the design space of possible minds. Perhaps there is something intrinsic to the fact that, as a mind devotes more attention to "humane concerns" and "social reasoning", there tends to be a concomitant sacrifice of attention to matters of effectiveness and pure rigor. This is not to say that there are no minds that successfully optimize for both; only that they are noticeably more uncommon, relative to the total space of all possibilities. If this view is correct, it could be troublesome for alignment research. Beyond mere orthogonality, raw intellect and effectiveness (and most AI boosters want a hypothetical ASI to be highly effective at realizing its concrete visions in the external world) might actually be negatively correlated with empathy.
One HN comment on the paper read as follows:
A few months ago I asked GPT for a prompt to make it more truthful and logical. The prompt it came up with included the clause "never use friendly or encouraging language"
which is quite fascinating!
EDIT: Funny how many topics this fractured off into, seems notable even by TheMotte standards...
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
Seconding, please elaborate!
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
Lesbians are interesting. They certainly have a much lower libido than men on average, which is what results in LBD, but at the same time I think they are noticeably more "aggressive" in their tastes than straight women. I recall there being a debate at Michfest in years past over whether public displays of BDSM should be allowed, or whether that would be politically compromising because it would be reinforcing patriarchal power dynamics. In the occasional conversations I've had with lesbians, they often seem to be quite self-conscious about restraining their natural desires around other women, because they don't want to violate the "rules of the sisterhood".
So I get the impression that straight female sexuality might truly be a unique phenomenon unto itself, and it has similarities with lesbian sexuality but also some notable differences.
Primaprimaprima
Aliquid stat pro aliquo
1mo ago
That "gay culture" seems to be purposefully designed to be repulsive.
I don't believe that it was "designed" to be anything. It's simply male sexuality in its most natural and unrestrained form.
Do you know how straight men would act if women were as DTF as men are? Hooooo boy.
I mean, what do you want them to say instead?
They're literally not allowed to say "yes, we are going to take your tax dollars to fund this work because it is intrinsically valuable, and our ability to carry on with this work is more important than your ability to eat at Chipotle for the 5th time this week or whatever else you were going to do with that extra $20". They think that, but they're not allowed to say it. It's not in the Overton window, it wouldn't be egalitarian, it wouldn't be democratic, etc. So they have to lie about "practical benefits" to the grant managers, and ultimately to themselves.
More options
Context Copy link