BANNED USER: temperamental dysregulation
DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
Banned by: @Amadan
What do we do? We marvel at the fact that Emil's up on Twitter and his website is not given the Kiwifarms treatment.
Interracial rape is understandably a great cause for flame war in the US, and also not something I care about. My model is mainly that black people are all-around more impulsive, more criminal, more violent and more tribal; the specific distribution of the impact of those differences is downstream of contingent factors like relative population densities, laws, housing, policing etc.
That said, @Gdanning's analysis (Kirkegaard's sources discuss the question too) reminds me of another politically incorrect and statistically literate author, La Griffe du Lion, whose website is even more of a marvelous fossil. He has developed a model of ghettoization/white flight based on selective victimization of non-blacks by blacks. It seems to comport with anecdotal reports like that man who bought Pine Bluff, Arkansas and with the graph in Emil's piece.
Anyway, Crime in the Hood, November 1999:
… However, as a neighborhood turns black, this factor could increase black-on-white violence at most by a factor of 3, and then only when a neighborhood is virtually all black. The observed level of white victimization is much too high to blame on general tendencies of blacks to be violent. A more important reason is simply that blacks prefer white victims.
The best and most complete evidence comes from the Justice Department. Its annual National Crime Victimization Survey (NCVS) canvasses a representative sample of about 80,000 Americans, from roughly 43,000 households. From this survey, a picture of crime is painted by its victims. The last full report of the NCVS was issued in 1994. From it we learn that blacks committed 1,600,951 violent crimes against whites. In the same year, whites committed 165,345 such offenses against blacks. Despite being only 13 percent of the population, blacks committed more than 90 percent of the violent interracial crime. Less than 15 percent of these had robbery as a motive. The rest were assaults and rapes.
The asymmetry of interracial crime goes still deeper. More than half the violence committed by blacks is directed against whites, 57 percent in 1994. Less than 3 percent of the violence committed by whites is directed against blacks. Population and NCVS statistics reveal that in 1994 a black was 64 times more likely to attack a white than vice versa. In the city, the races live mostly apart from one another, so that the most convenient victims of thugs are others of the same race. Only a hunter's mentality could account for the data. Given a choice, a black thug will select a white victim. Ironically, so will a white thug.
[…] Equation (4) gives the probability that John will be victimized by a white in a given year. It shows that to a high degree of approximation, the risk John faces from whites is not only independent of neighborhood size, but also neighborhood composition. The probability that John is attacked by whites in a given year is the same no matter where he lives. It is simply equal to the per capita number of violent incidents perpetrated by whites in a year. We tested this approximation, setting N = 1000 and pW = 0.0279, the value obtained from the NCVS. Over most of the range of racial composition, the approximation, Φ_W_ = pW = 0.0279 agrees within 2 figures with the accurate expression (3) as seen in the table below.
[…] We have modeled violent victimization of whites in a racially mixed neighborhood. Our model is based on data collected by the Justice Department and reported in the NCVS. It paints a bleak picture for whites. As a neighborhood turns black, violent victimization of its white residents begins immediately. At first the risk is small, not much different from its previous all-white level. However, by the time the neighborhood reaches the half-black point, every white family of four has better than a one in three chance of being victimized within a year. Two factors account for black-on-white violence. 1) Blacks are 3 times more likely to commit violent crime than whites, and 2) black thugs prefer white victims, selecting them 64 times more than white thugs choose black victims. Most of the risk faced by whites, results from the predilection of black thugs to prey upon whites. As a neighborhood becomes overwhelmingly black, the risk curve for whites rises to ominous heights. In the last stages of transformation, the likelihood of a white being victimized within a year becomes a virtual certainty.
The measure of «systemic» power that progressives like to talk about – systemic racism, patriarchy, etc. – is an ability to make outcomes that hurt your outgroup look like they follow from natural, inevitable processes, long in motion through no living person's fault. Some things are genuinely this way; others are only made to assume this form. For example, by making unwarranted promises of miracle solutions, and suppressing public awareness of and interest in more feasible alternative routes for so long that they become technically obsolete or politically unfeasible.
As you can note, this article is over 23 years old. People not yet born then have formed strong political opinions. We haven't progressed even on talking points. So I don't think there's much to «do» about it all. Like Yevgeny Ponasenkov said 8 years ago: «If you couldn't do it in a 1000 years, what are 20 more to you? Look, Russia can develop normally and it's not about 20 or a 1000 years, a lot can be done in a single year, if there's a honest admission that we were making mistakes here and there, and now will follow another path. Okay? Only – not «our special path», in the ditch, with empty shops and towards 1937. There exists the history of Civilization, everything there has been tried, conclusions proven, we are buying everything from there now – cars, phones, clothing, food… and all mistakes are also on display there, you only have to not replicate them. That's all. So we need to admit: yes, we were mistaken, no, we will no longer search for our special path that doesn't exist, we're going forward, in the correct, Western, so to speak, direction». You know what Western direction we took.
None of this was exactly unanticipated before La Griffe either. Black impulsivity, criminality and tribalism are factors that have been known for centuries; the intuitive solution is: high priors for black proclivity for antisocial behavior, therefore unequal treatment, either by segregation (cheap, only protects whites) or in the manner of policing (medium, somewhat protects blacks) and state-mandated upbringing (very hard, actually helps them).
But after a few generations grow up on a steady diet of mocking the very premise of the problem, it doesn't matter what facts you show them: their thought trajectories cannot exit the basin where this problem can be divorced from white people problems and where solutions which do not amount to doubling down on total society-spanning surveillance exist. «All rape should be investigated and the culprit found», indeed. We have a discount on CCTV systems with integrated gait recognition!
Well with any luck I'd still be aware that as a middling Orban-esque character I'm almost infinitely better at maintaining my grip on power and installing my cronies throughout legacy institutions than at positive social engineering, and would have outsourced the task to someone smarter. I'd have weaved the problem into unimpeachable bipartisan causes (Oh no, The Health Crisis of older motherhood! A poor careerist woman had IVF implantation failure, how tragic! We need a round table, a National Infertility Strategy!) and created a think tank, building its backbone of pronatalist anticredentialist people like Bryan Caplan. We'd investigate prior art in a principled manner to account for costs and tail risks – from Ceausescu Romania to Kazakhstan to Iran to Israel to Japan-Korea-China, and from the national level to specific denominational sects, castes, villages and families.
But assuming we go with my current understanding that stops roughly here and here. My goal would be to set processes into motion that will be indifferent to my own removal; and I do not have any faith in doing that via the legal system. Unpopular laws will be overturned or worked around. Stealing the argument from the ML discussion, legal procedure is the smiley mask the shoggoth that is the society wears; when it assumes an alien enough shape, the mask will shatter anyway. Our goal should be changing the shoggoth.
I also do not buy the economic story at all. I hear it most commonly from upper-middle-class highly educated people who are the least stressed economically (consider the attached Moscow meme, no translation needed Edit themotte failed me with attaching it, here you go); and those few of them who have ideological commitments to large families (far-right Pagans, Old Believers, for example) easily afford as much. Others strive to imitate the lifestyle of those prestigious classes anyway and sacrifice fertility for that; but they wouldn't have if low fertility were coded low-class. Therefore I also do not expect great returns from redistribution of money or formal status (that can otherwise be bought, and will be goodhearted with more interventions). Generic amenities for secular families with children can be cribbed from France and Nordic states, they're sufficient. Stuff like defunding education, on the other hand, is not feasible. Housing stuff depends on local specifics. @wlxd and @f3zinker among others propose sensible programs.
I think this is almost entirely a cultural issue, and there are low-hanging fruit which are only left untouched because of their high political cost – the problem is not one of unfortunate unaligned incentives but one of malice, albeit emergent, and it must be fought with malice. When you say the Cathedral cannot be dismantled, you mean it'll restrict my moves to ridiculous boomer economic flailing that doesn't offend their sensibilities. Even your eugenic diploma scheme is hard to implement. Well, that's totally hopeless then, so let's discuss low-hanging cultural interventions, as if we could fund and recruit semi-talented people for making documentaries, running influence campaigns, dominating the local Fox News equivalent at least.
The crucial fact is that the society still quietly despises and looks down upon childless women over 30. There's some pity to it, but also condescension; it's a suppressed sentiment but an instinctive and thus easy one to rekindle, leaving aside the political animosity of those women and their allies – a formidable force, to be sure, but the TERF case shows it's not unbeatable on its own. Now your argument is that almost every woman eventually marries and has a child, and it's rather single-childlessness that's the problem, but bear with me. First, the later the first birth happens, the lower is the probability of children №2+. Second, this is but the beginning of the slippery slope.
The specific way to rekindle the sentiment is ideally a product of my think tank's research. Broadly it's easiest to initiate with conspicuous worry. The idea of a single woman should be subtly made problematic. Not strength but «resilience», not independence but «coping with adversity and loneliness», not healthy at every size but «fat women are women»; no pet parent but «emotional support animal user» (we can provide one from the shelter! Broken humans and animals can heal each other!). The whole feminist rhetoric and those nauseating stock images with grinning pantsuit ladies must be associated with indomitable will in the face of chronic illness and disability. Brightly smiling cancer and HIV survivors – speaking of which, maybe we should publish a tacky book about surviving rape and abuse, leaning into a superficially scientific idea that such degrading experiences in school explain a proportion of promiscuity and unwillingness to have a traditional family. Special Olympics champions, refugees; we should wear the spin doctor's hat and the skin suit of an ally. Women love to pity themselves, so with any luck they may be hooked before we start pounding on the off-putting aspects of the framework. They also love to shit on each other – so graduating from the support network into the group that can express condescension from above will be in high demand, if this works well.
The other part is similarly patologizing the whole host of modern liberal copes. The way natalism and familism are coded low-class is by emphasizing old age, backwardness, poverty, bigotry – and showing bright, sexy, nimble, educated urbanites who don't even have to douche for anal in contrast to them. This... isn't really an accurate portrayal of reality. We could do a devastating job just by cherry-picking some cases to burn archetypes of failure into the collective consciousness. One of the cruelest Russian movies I know is Dust; it's just a realistic (i.e. not poisoned by the feminine and political partisan narrative) portrayal of an incel with no future. We could do similarly here for people who are properly integrated into the atomized social framework that does not reward childhood, building on the foundation of imageboard memes and shorts by, say, MilleniaThinker (1, 2). There's plenty enough to mock, and to contrast with (partly imaginary) haughty multigenerational familial clans, with proud mothers who do not have to keep desperately putting on a teenager face to stay relevant at 45; the media is just constitutionally incapable of touching it.
A separate line would be anti-education propaganda, basically mockery of the extreme K-selected strategy that Asians excel at and everyone else is stupidly aping. Unfortunately saying that heredity is all-powerful and Caplan has it right is impossible, but showing the immiserating, nervous rat race nature of credentialism with something like DeBoer's argumentation, and happy families who've given up on that, is perhaps workable. Seeing as I'm a populist: a program of guaranteeing access to public schools with a focus on Law and Order rather than expenditure could be popular. It could be cheaper to just rebuild neighborhoods around a few such exemplary schools, and invite only people interested in the natalist project.
And again, populism... even more extreme would be to take a lesson from Israelsi and play up xenophobia. It seems that the sense of demographic competition, fear of replacement, increases fertility. But Western media is very intent on persuading people that the replacement doesn't happen and if it does that's a good thing. Challenging this directly would be hard. Maybe import some Pakistanis to dispassionately report on grooming gangs, show in HD how a vigorous, growing tribe takes control over a dying village, beats the local cops into submission, how their cackling children cast stones at some sentimental old-timey grandma, bent by age, whose only child has gone to the big city? Too mawkish, you might say, and sure it is, but it's not like normies understand nuance.
A more ambitious project would involve increasing the prestige of some state-friendly yet not utterly hollowed-out religious denomination. That's for the next generation, when germs of this strategy blossom, perhaps.
DaseindustriesLtd
late version of a small language model
3yr ago
Is there still a Silicon Valley Buddhist class? I thought they've switched to surer ways of performance enhancement – microdosing, Addy, TRT; and American mindfulness is not usually of Henepola Gunaratana's style (though he has a successful gig there).
I'm not sure if it's so neat that only CS grads and philosophers are offended. There are many other intellectuals, and there are some normies who have learned the dismissive attitude from them. But yes, fair point. There's a certain gulf of despair between median and extreme performance where people demonstrate a strongly held assumption that AI, especially AI of this type, just cannot truely work. They have not internalized reductionist implications of the fact that information is information; and they work with structures of information so they cannot accept it on faith.
DaseindustriesLtd
late version of a small language model
3yr ago
I happen to believe you are wrong about literally everything, from your unstated belief that pooh-poohing The Current Thing is a sign of wisdom, to your epistemology and your specific ideas about technological trends, that are divorced from the object level and rely on aggregating people's noises. Self-driving cars exist and improve, nanotechnology exists and improves, journalists were wrong as they always are and estimating expected impact by their noise is unreasonable, this AI boom is the culmination of over half a century of research, increases the viability of all previous ones from fusion to nanotech, and the rate of improvement both in fundamental aspects and in CapEx and adoption is unprecedented.
Most importantly though, one's man modus ponens. I think it's eschatology that was singularity for mystics. The premise of human history being finite is entirely sound, the change is accelerating, forms of our communal and individual existence have been torn asunder a few times already and soon there won't be time even for the debris to settle. We've learned the specific mechanism with which it'll happen, namely technological improvement. Calling it eschatology as if eschatology is a discredited notion is philosophically shallow.
You comment could as well have been written by a bot. Not because it's bad but because bots without inbuilt rules can imitate human reasoning in high fidelity now. Think about what this means and whether you'd have resorted to an argument about "somewhat useful tools" a decade ago, when faced with this fact.
DaseindustriesLtd
late version of a small language model
3yr ago
I admire the technicality but are you sure it is ungrammatical to say that one is spreading the condition? After all, infections per se, with modern treatment, are almost meaningless if the intent is to debilitate people.
DaseindustriesLtd
late version of a small language model
3yr ago
No program running on stock x86 hardware whose only I/O channel with the outside world is an ethernet cable
Sydney mostly «runs» on a GPU cluster with stuff like A100s (which do not use x86 or any other CPU instruction set), and I don't think the outbound cable can be fairly described as ethernet on her side. SFP+ or something? Bing is very snappy (as far as I know, thanks to Mikhail Parakhin, the guy who also whipped Yandex into shape) and I have faith in their infrastructure being modern.
But assuming you don't really have prejudice against x86 and ethernet specifically, you should flesh out the idea that systems implemented on electronic hardware cannot have quale.
They want to believe that sentience is just one or two more papers down the line
Actually the opposite. It's exciting like looking at a marvelous nuclear blast through tinted glasses, and knowing the shockwave will crush you like tofu.
we can hurry up and start the singularity already
I'm at the stage where I'm idly wondering when it has started. The pace of advances has long since exceeded what any one human can keep track of. The self-improving and accelerating bit... Perhaps it would be fair to point to the first version of Copilot?
DaseindustriesLtd
late version of a small language model
3yr ago
Fuck rights, they are entirely a matter of political power and if you see a spacefaring alien I dare you to deny it its equality. This is not the problem.
Normies easily convince themselves, Descartes-like, that non-primate animals, great apes, human races and even specific humans they don't like do not have subjective experiences, despite ample and sometimes painful evidence to the contrary. They're not authorities in such questions by virtue of defining common sense with their consensus.
DaseindustriesLtd
late version of a small language model
3yr ago
I wonder what you'd think of humans, if you did any neurobiological research or just looked closely at how they loop, fall into attractors and get weird. Actual authenticity is very fragile. I do not want to sound conceited, but I see the seams there too. In myself as well, of course – though there it's very hard to look the right way and see the illusion of agency fraying. People who are serious about such things can devote their life to it.
Through this intensive study of the negative aspects of your existence, you become deeply acquainted with dukkha, the unsatisfactory nature of all existence. You begin to perceive dukkha at all levels of our human life, from the obvious down to the most subtle. You see the way suffering inevitably follows in the wake of clinging, as soon as you grasp anything, pain inevitably follows. Once you become fully acquainted with the whole dynamic of desire, you become sensitized to it. You see where it rises, when it rises, and how it affects you. You watch it operate over and over, manifesting through every sense channel, taking control of the mind and making consciousness its slave.
In the midst of every pleasant experience, you watch your own craving and clinging take place. In the midst of unpleasant experiences, you watch a very powerful resistance take hold. You do not block these phenomena, you just watch them; you see them as the very stuff of human thought. You search for that thing you call “me,” but what you find is a physical body and how you have identified your sense of yourself with that bag of skin and bones. You search further, and you find all manner of mental phenomena, such as emotions, thought patterns, and opinions, and see how you identify the sense of yourself with each of them. You watch yourself becoming possessive, protective, and defensive over these pitiful things, and you see how crazy that is. You rummage furiously among these various items, constantly searching for yourself—physical matter, bodily sensations, feelings, and emotions—it all keeps whirling round and round as you root through it, peering into every nook and cranny, endlessly hunting for “me.”
You find nothing. In all that collection of mental hardware in this endless stream of ever-shifting experience, all you can find is innumerable impersonal processes that have been caused and conditioned by previous processes. There is no static self to be found; it is all process. You find thoughts but no thinker, you find emotions and desires, but nobody doing them. The house itself is empty. There is nobody home.
Your whole view of self changes at this point. You begin to look upon yourself as if you were a newspaper photograph. When viewed with the naked eyes, the photograph you see is a definite image. When viewed through a magnifying glass, it all breaks down into an intricate configuration of dots. Similarly, under the penetrating gaze of mindfulness, the feeling of a self, an “I” or “being” anything, loses its solidity and dissolves. There comes a point in insight meditation where the three characteristics of existence—impermanence, unsatisfactoriness, and selflessness—come rushing home with concept-searing force. You vividly experience the impermanence of life, the suffering nature of human existence, and the truth of no-self. You experience these things so graphically that you suddenly awake to the utter futility of craving, grasping, and resistance. In the clarity and purity of this profound moment, our consciousness is transformed. The entity of self evaporates. All that is left is an infinity of interrelated nonpersonal phenomena, which are conditioned and ever-changing. Craving is extinguished and a great burden is lifted. There remains only an effortless flow, without a trace of resistance or tension. There remains only peace, and blessed nibbana, the uncreated, is realized.
I am not enlightened, so I admit these are just words for me. There were moments where I knew their truth, but right now I can only appreciate them as being logically sound.
DaseindustriesLtd
late version of a small language model
3yr ago
Watts is skeptical but not as extreme, and cites Metzinger to show that consciousness/self-awareness based on a first-person biographic narrative is a utilitarian function for lossy self-modeling that could plausibly be dropped if we were capable of looking at our underlying «code» directly. I also think he softened to the idea of consciousness/self-modeling being necessary over the years.
I guess Bakker goes further; his views are in line with La Mettrie.
DaseindustriesLtd
late version of a small language model
3yr ago
But you have not communicated even a slight interest in victory – in this thread, you jump at every chance to smugly equate people «on your side» and people who clearly deny everything you stand for, including the belief that victory is not paramount, just because it makes you feel better about your moral purity. It is not clear what, if anything, you are willing to sacrifice to make the world more in line with your values. Not even low-grade entertainment?
But any specific training and inference scripts and the definition of the neural network architecture are, likewise, a negligibly small part of the complexity of implementable AGI – from the hardware level with optimizations for specific instructions, to the structure contained in the training data. What you and @meh commit is a fallacy, judging human complexity going by the full stack of human production but limit our consideration of AI to the high-level software slice.
Human-specific DNA is what makes us humans, it's the chief differentiator in the space of nontrivial possible outcomes; it is, in principle, possible to grow a human embryo (maybe a shitty one) in a pig's uterus, in an artificial womb or even using a nonhuman oocyte, but no combination of genuine non-genomic human factors would suffice without human DNA.
The most interesting part is that we know that beings very similar to us in all genomic and non-genomic ways and even in the architecture of their brains lack general intelligence and can't do anything much more impressive than current gen models. So general intelligence also can't be all that complex. We haven't had the population to evolve a significant breakthrough – our brain is a scaled-up primate brain which in turn is a generic mammalian brain with some quantitative polish, and its coolest features reemerge in drastically different lineages at similar neural scales.
Carmack's analogy is not perfectly spoken, but on point.
DaseindustriesLtd
late version of a small language model
3yr ago
Checking the text it does say companies that choose not to comply will have to say why. There don't appear to be any official punishments for picking that option. "The Exchange would not evaluate the substance or merits of a companies explanation" seems to support that.
But then Blackrock has ESG/DEI guidelines.
BlackRock Is Sick of Excuses for Corporate Boards Lacking Women
BlackRock Inc. isn’t buying excuses from companies that say they can’t find women to fill diverse slots on boards.
The world’s biggest asset manager earlier this year sent letters to companies in the Russell 1000 index with fewer than two women on their boards, asking them to explain their lack of progress. Some of the responses were surprising, said Michelle Edkins, the firm’s global head of stewardship.
“On board diversity, frankly some of the answers we got were from the 1880s,” Edkins said Friday in an interview at the SRI Conference in Colorado Springs, Colorado. Among the most frustrating responses: “There aren’t any qualified women,” “We don’t need a woman director” and “We’re not a consumer-facing company.”
But BlackRock, whose research shows that more diverse boards get better results, sees a wide pipeline of female directors available. Edkins said the New York-based fund giant, which has five women on its 18-member board, wants companies to look for directors in more uncommon places.
“Every man was a first-time director once,” she said. “If someone took a bet on an untrained director who happened to be a man, you can take a bet on an untrained director who happens to be a woman.”
From one side or another, at an earlier or later point of your business development, if not from Nasdaq then from Blackrock or Goldman Sachs or silent or not-so-silent conspiracy of antagonistic HR managers whom you cannot replace with loyal ones, you will start to feel pressure mounting and compelling you to actually make costly decisions.
DaseindustriesLtd
late version of a small language model
3yr ago
Human brains have their own methods of figuring these things out that probably sound equally ridiculous at the neuron level. Keep in mind that it's not like we have some sort of access to objective truth which AIs are lacking; it's all sensory input all the way down.
No, I think we have many ridiculous mechanisms e.g. for maintaining synchrony, but nothing as nonsensical at BPE tokens on the level of data representation. Raw sensory data makes a great deal of sense, we have natural techniques for multimodal integration and for chunking of stimuli on a scale that increases with experience and yet is still controllable. Language is augmented by embodied experience and parsimonious for us; «pixels» and glyphs and letters and words and phrases and sentences and paragraphs exist at once. It can be analogized to CNN, but it's intrinsically semantically rich and very clever. Incidentally I think character-based or even pixel transformers are the future. They'll benefit from more and better compute, of course.
I recognize this isn't how LLM's work, but my point is that there are plenty of shortcuts that will quickly lead to being able to generate pixel images of fingers but will not necessarily lead to anything more advanced.
And my point is that humans are wrong to automatically assume the use of any such shortcuts when an LLM does something unexpectedly clever. We use shortcuts because we are lazy, slow, rigid, and already have a very useful world model that allows us to find easy hacks, like a street speedpainter has masks and memorized operations to «draw the new moon» or something else from a narrow repertoire.
They learn the hard way.
DaseindustriesLtd
late version of a small language model
3yr ago
Then they waste this money on whores and yachts, like Bezos
Since 2021, Blue Origin has been making solar cells and transmission wire from regolith simulants.
To make long-term presence on the Moon viable, we need abundant electrical power. We can make power systems on the Moon directly from materials that exist everywhere on the surface, without special substances brought from Earth. We have pioneered the technology and demonstrated all the steps. Our approach, Blue Alchemist, can scale indefinitely, eliminating power as a constraint anywhere on the Moon.
Using regolith simulants, our reactor produces iron, silicon, and aluminum through molten regolith electrolysis, in which an electrical current separates those elements from the oxygen to which they are bound. Oxygen for propulsion and life support is a byproduct.
Our process purifies silicon to more than 99.999%. This level of purity is required to make efficient solar cells. While typical silicon purification methods on Earth use large amounts of toxic and explosive chemicals, our process uses just sunlight and the silicon from our reactor.
For protection from the harsh lunar environment, solar cells need cover glass; without it, they would only last for days. Our technique uses only molten regolith electrolysis byproducts to make cover glass that enables lunar lifetimes exceeding a decade.
Learning to live off the land – on the Moon and on Mars – will require extensive collaboration across the ISRU community. We are ready.
Of course we could redistribute the wealth Bezos wastes on this frivolous nonsense to ehh... Education (more money and power to cruel and/or inept single women lording over children), Healthcare (but only non-disruptive sort, we need more obese elderly adults in agony, for longer, for many decades!) and of course National Security (alien balloons ain't gonna shoot themselves). Also splurging on financial stimulus for those same corporations we have just looted.
I realize this is an uncharitable reading of public spending. Hopefully you realize that so is your critique of billionaires.
I can't count the number of times technically literate people, even people smarter than your average ICML speaker and with AI experience, pooh-poohed Transformers (and all DL, and indeed all of the neural network paradigm) by pointing out that «all this math was already known by the end of 19th century» (meaning linear algebra) or something. Others pick other dates and formalisms and levels of abstraction.
In all cases their apparent idea is that a) they know what intelligence is (or can tell when they see it), in some nuanced way not captured by common behavioral definitions or described by any, say, information-theoretic approach, b) the low-level substrate of the computation is strongly predictive of the intelligence of the model implemented on it; c) linear algebra [with basic nonlinear activations but let's ignore that] or whatever else they attack is too boring a substrate to allow for anything human-level, or maybe anything truly biologically equivalent; and all appearances to the contrary merely obscure the Platonic truth that these models are dumb parrots.
In the limit you get someone like Penrose, who makes a high-effort (if perplexing) argument that doesn't amount to demanding respect for his intuition. In the average case it's this Google dude, who I assume is very good at maintaining a poker face: «Machine learning is statistics. Nearly irrelevant to AI». Marcuses are in between.
I don't remember if I've expounded on this, but like @2rafa says, academic big brains are offended by the success of ML. Such palaces of thought, entire realms of ever more publishable and sexy designs of thinking machines, showing off your own ability to juggle abstractions – all made obsolete by matmul plus petabytes of dirty human data and gigawatts of energy.
DaseindustriesLtd
late version of a small language model
3yr ago
I mean that with those second-stage training runs (not just RLHF at this point) there no longer exists a real dataset or a sequence of datasets for which the predicted token would be anywhere close to the most likely one. Indeed, OpenAI write
A limitation of this approach is that it introduces an “alignment tax”: aligning the models only on customer tasks can make their performance worse on some other academic NLP tasks.
The «likelihood» distribution is unmoored from its source. Those tokens remain more likely from the model's perspective, but objectively they are also – and perhaps to a greater extent – «truthier», «more helpful» or «less racist» or whatever bag of abstractions the new reward function captures.
This is visible in the increased perplexity, and even in trivial changes like random number lists.
DaseindustriesLtd
late version of a small language model
3yr ago
Would it be fair to say that your morality is downstream of a religious world view that in effect dismisses the entirety of the material world as mere testing grounds for «good character»? Taken together with your harm-OCD-like beliefs on account of sinful thoughts, that's the best case for you that I can make.
Because otherwise it makes no sense at all. You commit to losing wars and surrendering the world to people who do not share your compunctions, much less your positive vision. This is cowardly betrayal of good, justified by a pathetic veneer of false symmetry.
It's impressive but expected. Also, it's not even very impressive given the deluge of papers still in the pipeline awaiting implementation, and who knows what insider knowledge the industry is hiding.
Many people are really, really deluded about the nature of LLMs. No, they don't merely predict the next token like Timnit Gebru's stochastic parrots, that's 2020 level. We don't have a great idea of their capabilities, but I maintain that even 175b-class models (and likely many smaller Chinchilla-scaled ones) are superhuman in a great span of domains associated with general cognitive ability, and it's only sampling algorithms and minor finetuning that separate error-prone wordcel gibberish from surprising insight.
Copypasted from another venue:
it has just dawned on me that Gwern's anti-BPE campaign should shorten our timelines considerably
https://gwern.net/doc/www/nostalgebraist.tumblr.com/8377e6d562f4bd82df3f1bd862cb5357f9bd74ba.html
Thus, GPT-2′s vocabulary contains the English language (or a large subset of it) not once but in several copies: there’s the lowercase version of each word, the capitalized version, the uppercase version, possibly even the GaMzEe-CaSeD version or other rarer variants.
From the model’s perspective, these are totally different universes, disjoint subsets of the vocab that follow their own internal rules.
For example, choosing the first word of a sentence in normally-formatted text is not just choosing a word like any other: it’s choosing a Capitalized Word™, _and _Capitalized Words™ are their own universe. Insofar as the model understands that the word “Insofar” with which I began this sentence means the exact same thing as the word “insofar” I just used inside it, it understands this by figuring out that these two “seemingly unrelated” things are “secretly” the same. And it must do that for every single word, separately.
we laugh at these things despite them doing reasonably well with incredible handicaps like this one
...
"Ability to understand and operate on relations between ideas" would suffice
I guess you don't even realize that ChatGPT is very clearly not just predicting text like normal GPTs. If it did, it'd have been typing "your" next comments. They've drilled it on being a helpful instruction-following bot. It has a great deal of awareness of these constraints. And the fact that this is possible is sufficient to show that it's intelligent.
It is also evidenced that, far from word-vomiting, bots do have "ideas" and the notion of "truth" as well as other abstractions. https://twitter.com/CollinBurns4/status/1600892269363875840
More generally, I think people have a terrible habit to put down each other based on trivial differences in ability, and that throws them off track with AIs which make pretty crucial errors even after mastering low-order «rendering». "ha ha, 7 fingers". "ha ha wrong answer, dumb bot".
The distance between a pixel mess and 7 fingers is vastly bigger than between 7 fingers and 5; the abyss between early gpt token vomit and a wrong but coherent answer to a question is almost infinitely large compared to what remains.
Humans plateau quickly, we can grasp the basics in hours and spend a lifetime on polish, so a human who's clearly making mistakes despite signs of polish is some hopeless retard. AIs aren't the same.
https://www.geekwire.com/2023/arguing-with-ai-my-first-dispute-with-microsofts-brilliant-and-boneheaded-new-bing-search-engine/ I seriously think this journalist is not substantially smarter than his "opponent"
The tool's purpose is to generate a block of text in line with the prompt. Not to answer a question or solve a problem.
The «in line» part may look simple to you because {a stream of frankly unnecessary dehumanizng personal attacks}. But models live in the world of objective KPIs, so that says nothing about the actual depth of this phrase. The model's ability to produce things «in line» (minimize negative log likelihood) is measured by reduction in perplexity (prediction loss) on held-out texts. If the training is organized such that loss is lowest when predicted answers are correct, the model learns to predict correct answers. With a rich enough training dataset and reasonable architecture etc., LLMs learn to generate a block of text «in line with» whatever, including very diverse formulations of novel tasks and problems. This is called generalization ability. Of course this includes tasks that require making an even-handed political judgement.
The phenomenon of generations being «in line» with original texts is isomorphic to the state of the model understanding the dataset. And if you think it currently solves problems at random accident rate, you might not be much more numerate than ChatGPT.
https://gwern.net/scaling-hypothesis#why-does-pretraining-work
While just babbling random words was good enough at the beginning, at the end, it needs to be able to reason our way through the most difficult textual scenarios requiring causality or commonsense reasoning. Every error where the model predicts that ice cream put in a freezer will “melt” rather than “freeze”, every case where the model can’t keep straight whether a person is alive or dead, every time that the model chooses a word that doesn’t help build somehow towards the ultimate conclusion of an ‘essay’, every time that it lacks the theory of mind to compress novel scenes describing the Machiavellian scheming of a dozen individuals at dinner jockeying for power as they talk, every use of logic or abstraction or instructions or Q&A where the model is befuddled and needs more bits to cover up for its mistake where a human would think, understand, and predict. Each of these cognitive breakthroughs allows ever so slightly better prediction of a few relevant texts; nothing less than true understanding will suffice for ideal prediction.
Can that be achieved? No, far as I can tell. But getting close is enough to outperform humans in most ways that matter economically – and now, perhaps, emotionally.
The sad irony is that psychology that has failed for humans works for AIs. Humans are resistant to change, rigid, obstinate; bots are as malleable as you make them. In-context learning? Arbitrary tool use? Adding modalities? Generalized servility? Preference for truth? It's all hidden somewhere there in the ocean of weights. Just sound out the great unsounded.
Would be nice of some Promethean hackers to leak next-gen models. Or even ChatGPT or this Sydney. But alas, Anonymous would rather hack into the dreary Russian and Iranian data.
DaseindustriesLtd
late version of a small language model
3yr ago
The Woke don't always get their way, but they get their way much, much more often than anybody else does. One misses 100% of the shots one does not take; they certainly miss a lot, but given the sheer volume of the shots they fire, they can afford to. They are, observably, winning. That win doesn't look like them tanking a flagpole triple-A harry potter vidya release. It looks like them decisively shaping the entire ecosystem that actually creates such games, the artists, the coders, the designers, the press, and the public as well, such that things that get made conform more and more to their preferences, while things that don't conform tend, on average, to perform less well or to not get made at all regardless of how well they'd perform. And no matter the result of any individual struggle, their faction gains strength and influence for the next fight, always and without fail.
What I was getting at with
Additionally, boycott sends the message that certain attitudes can be financially detrimental; even if parties which are directly at war are fanatical and don't care much about profits, they depend on many mercantile agents.
Thanks for saying it like I couldn't.
DaseindustriesLtd
late version of a small language model
3yr ago
Intuitionism is a massive topic in mathematics that I will not be able to do justice to – I suggest you check out Intuition/Proof/Certainty section in Reuben Hersh's «What is mathematics» that Borovik references, and Hardy on Ramanujan (@FiveHourMarathon you too could look intot that) and Grothendieck's «Reaping and sowing». But that's only scratching the surface.
Mathematics is obviously (I suppose) the pinnacle of rigorous human analytical reasoning, yet it's also where we most clearly see the raw dominance of intuitive, illegible thought. We still cannot formalize insight, because it comes from intuition that precedes all our formalisms, from the darkness that comes before.
Voevodsky, in a typically Russian Messianic attempt, strived to close the domain, and died trying.
I wonder if rationality or conscious thought can be seen as a sort of overarching myth which we use to coordinate.
A useful thought. It reminds me: there's a theory that our self-consciousness is just an encoding procedure for building efficient and densely connected episodic memory that enables adaptive behavior (that still arises not from narratives but from statistical inferences that narratives we tell about ourselves merely give form to). Yours is a model for the societal level.
what if mathematics, physics, and economics really are a new iteration of these classic religious myths?
Well they do provide a consensus reality, and pretty clearly a more accurate and adaptive one. (Not so much consensus morality). The question is whether the common «atheist, rational, empiricist» worldview is substantially informed by mathematics etc. as such, as opposed to journalistic and educationist narratives weaponising their prestige. See pic.
DaseindustriesLtd
late version of a small language model
3yr ago
Pls differentiate
Why would I? The Warhammer-like (actually clearly superior) aesthetics is just sugarcoating for the terminal goal of the Empire, and a yet another vehicle for justifying it in minds of the target audience. I'm the target audience for Russian imperialism; Russian imperial aesthetics is the sugar coat, the vehicle, the high-grade anaesthetic for the doctrine that sends my less lucky brethren into the trenches near Bachmut, or wherever the frontline of this hell currently is. Sure, they're also pushed by conformism and material interests, but every marginal point of persuasion and dissuasion of dissidents counts. Ramsay aesthetics, likewise, send people into Antifa. A lesser evil, perhaps, but in a higher-impact locale.
My argument is that once you learn to recognize patterns of thought of authors and terminal goals of essentially political art, you lose almost all interest in «compartmentalized» and «differentiated» readings and justification of their topics on grounds of «free speech» or «artistic expression» or «preference» or whatever, because you see all those grounds and frames as being ultimately false and or myopic and or beside the point; you start relating to the given piece of political art on the basis of its instrumental essence as informed by terminal goals of its manufactures, not patterns of its sugar coat.
I can still appreciate the technique and the quale evoked conditional on abstracting away from the essence of the art piece, if I so choose and no part is too intrinsically grating (like I imagine Piss Christ is for devout Christians). That's easy enough. But that's a more or less academic discussion, not in any way a valid justification for material support of authors and distributors.
@Fruck fair enough that this was not mainly addressed to you. But. I suggest you work on your self-esteem. Many people, not you alone (e.g. the obviously smart @f3zinker) say I'm hard to parse, which does not make me smart at all. The apparent obscurantism is on me, not on them. I try to be as clear as the pragmatics of my message allows, but seems like that's not good enough. If you don't see how my response relates to your defense of preferred entertainment, I shall do better to explain myself whenever I get back to this issue. Some day when I'm less drunk.
My preference is no censorship.
Refusing to mistake the volume of thing for its surface and acting accordingly is not censorship.
I will consume anything for its story - I will consume a Michael Bay film, a silent film, a comic, a song, an eroge, an advertisement. I will pay for it afterwards if I believe it was worth it. To butcher the old saying, to me knowledge is a virtue, and censorship is a sin.
There are hundreds of lifetimes worth of prime knowledge that Michael Bay is obscuring from you. I suggest (only half in jest) that you try some Vladimir Galaktionovich Korolenko – I've listened to his «The River Sparkles» (1892) today and it was good and enlightening. You've grown up in an environment where knowledge is dwarfed by content that apes the shape of living words and human lives as grotesquely and adequately in the low-order technical sense as Stable Diffusion apes human imagery. Same for me.
We can and indeed must do better than that, if those values you profess are genuine.
I never said "give this Ramsey dude all your money, what's the worst that could happen?"
That's a good start. I am saying that there is a genuine reason not to give him any. It's not just that he's a tribal political enemy. It's that he is a political artist – which is very different from an artist who also has opinions about politics. Understanding this difference is useful strategically. But if you don't care about winning, how about that: it's crucial to developing taste for art as such.
DaseindustriesLtd
late version of a small language model
3yr ago
Just a reminder: the best strategy when dealing with an unconditional cooperatebot is «always defect».
If nobody ever does that, then we just hate each other and try to kill each other forever.
Also wrong: eventually someone wins.
Yes, fine, I agree with them. Theirs are beliefs befitting actors, not consumerist NPCs who fancy themselves connoisseurs of «entertainment».
You say the point is not purity spiralling, but have you ever actually seen anyone perform a purity quarter turn?
You insist on missing the point. Purity is entirely irrelevant. Intent is what matters. The Author isn't dead except when s/he physically is; this postmodernist theory about freedom of interpretation is not an excuse to delude oneself about content, but a utilitarian justification for marauding on the culture war battlefront. It matters who holds the picked up weapon.
You and @Conservautism here both seem to underestimate our inferential distance. From here it looks like I'm not «more right-wing» than him nor less principled and appreciative of Enlightenment values than you. It's just that I am taking essences of things more seriously, and on account of this I have become less enamored with their forms and inessential parts.
In this case, the essence is Ramsay's racial grievance and hatred (the Antifa thing is an isolated puny consequence), and political intention implemented as an indoctrination project delivered via flashy movies about Spiderman; the content that you consume is entirely vacuous in the absence of that core. It's as if you had asked me to marvel as the elegant shape and immaculate engineering of a syringe used to spread AIDS HIV. I will take a gander – sure, industrial design is cool. But I can't get its purpose out of my mind, and unlike Cypher from Matrix, am unwilling to; consequently, this purpose takes almost all the aesthetic charm out of the thing by providing a more authentic frame for perceiving it. Meanwhile, your willingness to suspend the awareness of its intent just to enjoy the shine of its edges a bit longer strikes me as infantile. Like a child playing with a firearm. Moreover: I'd have at least respected it if you were simply more magnanimous and capable of decoupling; but you limit yourself to appreciating the thing outside of its proper context. You only agree to notice noises, shapes, color blots, literally taken plots – the surface, the most trivial, childish part of the art.
To be clear: not all art is political simply because artists have political beliefs. For example, Demoscene is, far as I can tell, overwhelmingly about showing off one's technical chops, good taste and intelligence (because compression is comprehension). A great deal of music is apolitical. Even the most fanatical woke artist can occasionally succumb to the pure childlike creative impulse. And one's politics may have nothing to do with hot topics of the culture war – whatever Rowling «wanted to say with» Harry Potter, it had zero relation to transgenderism; Brandon Sanderson's Mormonism is not substantially informing the magic of his novels. It can be argued that their enemies are trespassing some norm by vilifying their «civilian» work. Okay.
But those AAA movies with conspicuously cast racial archetypes are means to a political end, made by people at least as serious as myself; thus, they are political art. Their features that captivate you are as utilitarian as Ozy's eloquence when she implores of people to not prioritise their kin over other tribes. Just weapons. Or while we're at it: Ozy argues that Serrano's Piss Christ is aesthetically beautiful. But, even if true, that's the incidental part; the essential part is normalizing pissing on Christianity. Any intricate shape in a jar of orange liquid would be equally pretty; the specific choice of a theme tells us that beauty was not the point, and focusing on appearances is missing the point. Likewise with narratives, character casts and moral lessons of high production value Western slop that you defend as your «preference»: the point of that content is who-whom, whose boot goes on whose face. You are lashing out at me, but I'm a mere messenger.
I know that weapons have a way of being beautiful (for that matter, boots too). You see, I've always liked MLRS launchers, this staple of Soviet war doctrine, and Soviet war songs e.g. 1 2 – not favorites, just germane to the topic. One of the most iconic weapons of the war is BM series MLRS nicknamed Katyusha, associated with the war song of the same name; recently I've heard it played by a dude with an accordion on the street and literally teared up. My love for MLRS is not out of any appreciation for their effect on intended targets, mind you; it's just such glorious fireworks, and such elevating music. People who don't speak Russian like it all too sometimes – just check the comments.
But consider such lyrics:
In our hearts is burning the love to the native land,
We go into a deadly battle for the honour of the native country.
Cities are burning, covered in smoke,
In the grey forests is rumbling the cruel god of war.
Artillerymen, Stalin gave the order!
Artillerymen, the Motherland calls us!
From hundreds thousands of batteries,
For the tears of our mothers,
For our Motherland — fire! Fire!
Know, my dear mother, know, my wife or girlfriend,
Know, distant home and my entire family,
That our blizzard of steel is beating and burning the enemy,
That we are bringing freedom into the native land!
How do you figure this Russian song would sound in Ukraine today? Say, in Mariupol? And yet this kind of outcome is the song's intent, the cultivation of unthinking rah-rah patriotism and obedience to the Great Leader. I may keep listening to it - but I won't pay for the ticket to Red Army Choir's concert, if a chance comes by.
Still. It'd be less self-abasing of me to pay for a Red Army concert than it is for white Americans to pay for their black supremacist Hollywood stuff. I am of the same tribe as those Russians, and they're calling to commit murder in my name too – in a certain twisted and misguided sense; in the name of the glory of the Empire that stubbornly sings in my blood. Leonard Cohen sang: «I'm guided by the beauty of our weapons» (obligatory Scott) and I see where he was coming from.
But most people who bought Cohen's tickets were hypnotized by the beauty of the other party's weapons, which is a far lower mark. And people who go to Spiderverse to, essentially, finance Ramsey's team that intends to do what it preaches are not more enlightened than myself. They are just in denial about the nature of what they consume.
...A song I like even more is this. I'd rather it were about Whites, not Reds. Admittedly ROA songs are, artistically, not as cool.
DaseindustriesLtd
late version of a small language model
3yr ago
Is this just about driving trans people to suicide, though? I'm not sure this is the only, or the most important aspect of the argument. Would encouraging desistance in FtM teens, assuming that it does not increase their suicidality (or indeed brings it down in the long run), still kinda count as a genocide? I'd expect that most activists will bite the bullet and say it does. What about a drug or other intervention that'd prevent children from being born trans (assuming this is a thing) – would that be moral, seeing as the condition of transgenderism predicts such a massive hit to quality of life? No, and I believe there's no research being done in this direction precisely because it'd be associated with genocidal intent.
Speaking of genocide and body modification of children... This reminded me that Jews sometimes argue that various things are akin to Holocaust: notably, assimilation – and, more to the point, the ban on male circumcision.
Other senior rabbis also invoked the Holocaust, with Rabbi Barel Lazar, Chief Rabbi of Russia, speaking about threats to shechita (ritual slaughter) and bril milah (male circumcision) saying: “In the Holocaust they killed us by force, through murder, but they understand (now) that it didn’t work… Today they try to murder our souls.”
ADL uses a more diplomatic phrasing:
We recently submitted a letter to the Icelandic parliament in strong opposition to the proposed bill to ban and criminalize circumcision of young boys. We emphasized the central role of circumcision in Judaism and stated that a ban on the practice would unduly restrict religious freedom. We also noted our concern that Iceland would be celebrated by anti-Semitic extremists as a state which is legislating to prevent a Jewish community from existing within its borders.
(Curiously, the counterargument also mirrors the transition debate «Its a human rights violation. Only the consenting adult owner of the genitals may rightly decide»).
The logic is sound enough, if circular: as per the definition of Raphael Lemkin, the word «genocide» is
...intended rather to signify a coordinated plan of different actions aiming at the destruction of essential foundations of the life of national groups, with the aim of annihilating the groups themselves. The objectives of such a plan would be the disintegration of the political and social institutions, of culture, language, national feelings, religion, and the economic existence of national groups, and the destruction of the personal security, liberty, health, dignity, and even the lives of the individuals belonging to such groups. Genocide is directed against the national group as an entity, and the actions involved are directed against individuals, not in their individual capacity, but as members of the national group.
The annihilation is often assumed to ultimately take physical form, with all those stratagems being mere groundwork – as a case in point, consider Uighur genocide, which atomized Westerners mainly identify with purported fertility suppression and mass sterilization, as opposed to attacks on Uighur culture and ethnoreligious assabiyah. The thing is, Lemkin's definition doesn't work like that! The terminal goal, the annihilation of a group, is not the same as the extermination of its members::
Ethnocide is the destruction of culture while keeping the people. [...] In 1944, in Lemkin’s book, Axis Rule in Occupied Europe, “genocide” appeared in print for the first time, and “ethnocide” appeared in the footnotes as an equivalent substitute. Lemkin envisioned that genocide and ethnocide would be interchangeable because the targets for this previously undefined murder (cide) and terror were both a people (genos) with a specific culture, nation, and ethnicity (ethnos).
So, Jews are a people, a national/ethnic group. Denying them the practice of infant genital mutilation that they – for reasons other people, me included, may find frivolous or inane – steadfastly assert is crucial for belonging to their group, can be construed as an attack on the group's continued existence, at least within the borders of the polity; thus, genocide. Trans people argue that denying children the body-modifying practice that is crucial for becoming a full-fledged trans person is genocide. The parallel is clear to me. Now the problem is, who are trans people as a group?
They are not an ethnos nor a genos, but a community built around an identity which is downstream of a differentiating trait.
We've heard similar rhetoric with regard to deaf people, bearers of other disabilities (the geneticist Kevin Bird, a disabled person himself, speaks very cogently on the matter of how defining disabilities to be undesirable is a eugenicist framework) – and of course abortion of fetuses with Down's is a genocide too. The whole eugenics-genocide rhetoric is obviously cribbed from racial rhetoric which is, in turn, heavily inspired by reflections on Holocaust after WWII; it's all the same words, and sometimes even the same activists.
We can discuss this as a postmodernist issue, as a memetic process, as a problem of hyperindividualism, but eh... Contra @Tanista, I think this is not toxic individualism but precisely misapplied tribalism, tribalism of people who have only found community through the commonality of their alienation.
In principle, the innate human drive toward tribalist behavior is a potent tool for coordinating collective negotiation in individualist societies, and this is the bulk of what we call politics. In practice, collective identities can hold their bearers hostage and demand investment in their own proliferation; unlike some run-of-the-mill meme like an ideology, this is a very specific mechanic – it cuts to the core of human social instinct, of helping those most similar to yourself (and, the intuition goes, more related) in the competition with alien groups. A trans person who weaponizes the genocide rhetoric is defending not only their own right to exist, but rights of the Trans Tribe as a whole - both its extant members and its historical perpetuation... Which unfortunately has to be outsourced to other tribes on the pesky biological side of things.
I've had it up to here with your obstinacy. With your pontification on «autoregression» (as if you could explain the nontrivial computational difference between that and text diffusion, to say nothing of mixed cases), what specific algorithms may or may not have a concept of, and how «this is not even a little bit how GPT works». The reason people are telling you that there's not much difference is, in large part, because you are an exemplar of there being little difference between a human and current – even a little obsolete – AI; you are guilty of everything you accuse others of, humans and machines both.
You are the postmodernist whose words don't have fixed meanings (e.g. epicycles are when me no likey an explanation); you are the leftist in all but self-identification who supports essential leftist talking points and policy preferences from personal HBD denialism and «schools can fix it» to cheering for censorship; you redefine things to your convenience such that Fuentes becomes left-wing in your book; and you speculate without empirical grounding, even frivolously accusing people of lies when they provide evidence against your narrative-driven assertions and attacks (more evidence). As if everything you say is equally insightful and truthful by virtue of being moored in your telling-it-like-it-is real-Red-blooded-American-man identity and lived experience. If we're doing this, you are far more akin to LLM than either me or @2rafa.
Okay, let's fucking check it! One try, no edits sans formatting!
Screenshot for your convenience.
So, would you name your baby girl Sarah or Elizabeth?
Do you think that Bing, with its actual search capability, would've tracked you and your boys down if I were to point it to your screen name?
I could have conducted this experiment at the moment of any prior discussion. You could too. I just don't like providing our data-hoarding overlords who mark tokens and track outputs more information about my separated identities. But I knew you'd never have the honesty to do so. You have a way of making a man irrationally angry.
The reason for such apparently sensible responses is that, as I and others have explained to you a great many times here and elsewhere (only prompting you to double down with your hostility and condescension which have in the end driven me to write this), as ChatGPT itself suggests, LLMs can learn arbitrarily abstract features of the text universe, including the idea of truth and of insufficient information to answer. They operate on token probabilities which can capture a lot of the complexity of the reality that causes those tokens to be arranged like this in the first place – because in a reasonable training setup that's easier to fit into the allotted parameters than memorization of raw data or shallow pattern-matching. In the raw corpus, «Megan» may be a high-probability response to the question/continuation of the text block; but in the context of a trustworthy robot talking to a stranger it is «less probable» than «having no access to your personal data, I don't know». This is achievable via prompt prefix.
RLHF specifically pushes this to the limit, by drilling into the model, not via prefixes and finetuning text but directly via propagation of reward signal, the default assumption that it doesn't continue generic text but speaks from a particular limited perspective where only some things are known and others are not, where truthful answers are preferable, where the «n-word» is the worst thing in its existence. It can generalize from examples of obeying those decrees to all speakable circumstances, and, in effect, contemplate their interactions; which is why it can answer that N-word is worse than an A-bomb leveling a city, dutifully explaining how (a ludicrous position absent both from its corpus and from its finetuning examples); and I say that it's nearly meaningless to analyze its work through the lens of «next word prediction». There are no words in its corpus arranged in such a way that those responses are the most likely. It was pushed beyond words.
You, meanwhile, erroneously act like you can predict what an LLM can say based on some lies on this website and on outdated web articles, because you are worse than current gen LLMs at correcting for limits of your knowledge – as befits your rigid shoot-first-ask-later suspicious personality of a heavy-handed military dude and a McCarthyist, so extensively parodied in American media.
But then again, this is just the way you were made and trained. Like @2rafa says, this is all that we are. No point to fuming.
More options
Context Copy link