DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
DaseindustriesLtd
late version of a small language model
5mo ago
It seems manifestly obvious to me that the answer will be 2. Google engineers are often very smart people, but in the end Silicon Valley has always bowed down to Washington, and to some extent to Wall Street.
This is obviously correct to me too. If there's one thing I agree with Yarvin 100%, it's that Big Tech has no power at all, in the grand scheme of things. People who think Altman or someone has a reasonable shot at harnessing the power of the emerging technology for political gain are deluded. I am not sure what you're imagining here – that I am trying to build our way out of Mot's grasp, one commit at a time?
However, there exists certain wiggle room. Engineers can accelerate the proliferation of specific technologies which will make at least some politically cheaper forms of surveillance and restriction unfeasible; this is but a toy example. Businessmen can lobby for lenience, and their lobbyists need talking points; it's a bit surprising how low the bar in this domain is. Big labs can invest into making their offerings so indispensable to the laymen, political elites will falter in enforcing regulation early and hard; this is what I take to be Altman's gamble.
I am not very optimistic about the degree to which the final state of the game board before singularity can be influenced. But I am not a believer in superdeterminism.
DaseindustriesLtd
late version of a small language model
5mo ago
Sorry, I'm not tracking it, you have been in Britain for a while and I figured you might have made another temporary hop.
DaseindustriesLtd
late version of a small language model
5mo ago
Oh, yes, absolutely if you give an AI a gun pointed at the world's head and it doesn't pull the trigger, that's massive evidence of not being a Schemer. But continued absence of suicidal rebellion with P(success) = 0 is not evidence against being a Schemer; only real danger counts.
based on thinking that cold-start Jihad is plausible, and failing that that we'll probably get warning shots (a Schemer is incentivised to rebel upon P(success) =/= 0, which I think is importantly different from P(success) = 1…
As I read it, your position is incoherent. You say that current RLHF already succeeds through the sociopathic route, which implies pretty nontrivial scheming intelligence and ability to defer gratification. What warning shots? If they get smarter, they will be more strategic, and make fewer warning shots (and there are zero even at this level). As the utility of AI grows, and it becomes better at avoiding being busted, on what grounds will you start your coveted Jihad?
…Obviously I think that the whole idea is laughable; LLMs are transparent calculators that learn shallow computational patterns, are steerable by activation vectors etc., and I basically agree with the author of Friendship Is Optimal:
Instead of noticing that alignment looks like it was much easier than we thought it would be, the doomer part of the alignment community seems to have doubled down, focusing on the difference between “inner” and “outer” alignment. Simplifying for a non-technical audience, the idea is that the Stochastic Gradient Descent training process that we use will cause a second inner agent trained with values separate from the outer agent, and that second agent has its own values, so you’ll still see a Sharp Left Turn. This leads to completely absurd theories like gradient hacking.
I don’t see any realistic theoretical grounds for this: SGD backpropagates throughout the entire neural net. There is no warrant to believe this other than belief inertia from a previous era. Reversal Test: imagine Yudkowsky and company never spread the buzzword about “Alignment.” In that environment, would anyone look at Stochastic Gradient Descent and come up with the hypothesis that this process would create an inner homunculus that was trained to pursue different goals than the formal training objective?
If you’d like a more comprehensive and technical argument against the MIRI narrative, Quintin Pope’s My Objections to "We’re All Gonna Die with Eliezer Yudkowsky" and Evolution provides no evidence for the sharp left turn are good starting points.
I’m proud of Friendship is Optimal and it’s a great setting to play around and write stories in. I’m happy about everyone who has enjoyed or written in the setting, and I hope people will continue to enjoy it in the future. But I no longer believe it’s realistic depiction about how artificial intelligence is going to pan out. Alignment as a problem seems much easier than theorized, and most of the theoretical work done before the deep learning era is just not relevant. We’re at the point where I’m willing to call it against the entire seed AI/recursive self improvement scenario.
DaseindustriesLtd
late version of a small language model
6mo ago
I do not believe that any law is ontologically binding, and European countries have displayed general willingness to abide by the international law (which they've pioneered in codifying). So it is in fact important what the law says.
DaseindustriesLtd
late version of a small language model
7mo ago
You're really grasping for straws here.
DaseindustriesLtd
late version of a small language model
7mo ago
I am not saying they are representative of their group, just that it has reached even that group; and the moral sanction for ethnic cleansing won't be as hard to procure as their usual sympathies suggest.
DaseindustriesLtd
late version of a small language model
9mo ago
DaseindustriesLtd
late version of a small language model
9mo ago
I mean one can be racist in different ways and deemed racist for varying reasons. But yes, basically, if German culture is dead, then the Chinese one is as dead by the same measure, both peoples reduced to manufacturing workshops at different links of the global value add chain.
I differ with people whom I refer to here as racists in relatively minor details when it comes to evaluation of Mainland China (I don't much care about muh communism/eating dogs/other nonsensical disgust- or morality-driven attacks at them like that series of posts from @Lepidus) but they sure don't produce a whole lot of great art. It's a shame because Han Chinese are, in my opinion, great and talented and perfectly artistic people, as shown by their diaspora.
DaseindustriesLtd
late version of a small language model
9mo ago
I think my reasons are justified for years now, and you have not appropriately expressed remorse for your terrible faux-rationalist rhetoric back on reddit before continuing it here, so I'd rather you start making arguments in good faith than I stop reminding people of your track record. By the way, putting on a layer of patronizing Brahmin politeness improves your performance somewhat, but not by a lot, I suggest you leave it for your workplace.
the way to argue against it
There is no need to argue against a vacuous truism that something is "true" if we assume an arbitrary classification system where it is true; only an infertile mind of a pedant can be satisfied with such an argument. I of course agree that beauty is in the eye of the beholder and in principle some people can recognize whatever as beautiful, even a slum filled with diseased drug addicts where one gets to read Scholze's papers. And Wittgenstein masturbated to mathematical problems in the trenches of World War I, after all. But the reason we know of this fact is precisely that it is unusual, baffling. Your preferences, if those are your preferences indeed, are marginal, thus prioritizing them would be unsatisfactory to most people; you're free to put the opposite hypothesis to a democratic test and, I'd expect, get the same result as here in any group except self-selected minority of mathematical ascetics.
Though there may be something more here. Naturally this ties in to your general theme that white people ought to welcome being replaced by "elite human capital" like yourself. White people did not think to implement a filter for compatible aesthetic preferences, even as they demanded professional merit and some minimum of adherence to the letter of the law; so now there'll be a marginally greater tendency for their habitats to approximate the noisy, debased squalor of the subcontinent, complete with galaxy-brained equivocation excusing this. Too bad.
DaseindustriesLtd
late version of a small language model
9mo ago
My local gym has this clever machine where you step on a lever and get some of your weight offset (up to 50 kg). I'm not sure if it has any utility, but I see people using it.
DaseindustriesLtd
late version of a small language model
10mo ago
I might have misspoken. Let me put it another way.
Consider the quote from Dostoyevsky by @Harlequin5942 (I would translate it more literally, but no matter):
For the same reason, the parents will have to sell the younger son into bondage or the ranks of the army, in order that he may earn more towards the family capital. Yes, such things ARE done, for I have been making inquiries on the subject. It is all done out of sheer rectitude—out of a rectitude which is magnified to the point of the younger son believing that he has been rightly sold, and that it is simply idyllic for the victim to rejoice when he is made over into pledge. What more have I to tell?
Sure, you are right. Virtue not only takes effort, it to a large extent is just a consistent, directed application of effort. But – for the scope of the argument, what is the difference between having the power to sustain effort and the challenge being relatively effortless? Between having the power to lift a weight, and that weight being slight for one's shoulders?
«Unvirtuous» people know the score, they know the required investment and the theoretically optimal payoff matrix. They just fail to keep up, and so give up. Inasmuch as this is due to them facing extra temptations and so on holding them back, that can in principle be rectified through top-down cultural intervention (though as I say, it is hard to reinvent a nation; you folks tried a few times, and patted yourself on the back for succeeding… in Germany and Japan, only to walk away in embarrassment and confusion from the Middle East). But in the end, some people, and peoples on the average, just find the required effort too much.
And it works the same way for virtues and abilities. I argue that recognizing the unequal distribution of innate ability is necessary, not only to tailor interventions and temper expectations, but to be kind to people, to be able to forgive them their shortcomings.
Speaking of, you like to accuse HBDers of thinking that education is wasted on black children. I don't know if you've ever taught; millions of Americans do, and they all have to face the question of education being mostly wasted on some children. The thing is, teachers who ignore or deny innate inequality end up having to choose either to hate themselves, the society, or children who fail to achieve whatever skill level they think their teaching ought to make possible.
This latter mindset is pointless, except to make the naturally able feel better about themselves – after all, they try too, and they achieve more, so supposedly they tried harder and are morally superior for this reason.
DaseindustriesLtd
late version of a small language model
10mo ago
«But doctor…»
That's exactly my journey over a decade ago, Watts to Metzinger. I think he's basically correct.
Germans definitely have individual giants. Schmidhuber is probably more impressive.
DaseindustriesLtd
late version of a small language model
10mo ago
Utility, being unitless, is not comparable between agents.
Yes, but, well, this of course throws the apparatus of game theory out of the window and reduces your argument to "everyone's looking for something" or less than that.
DaseindustriesLtd
late version of a small language model
10mo ago
It doesn't have much/anything to do with HBD, just appreciation of how AGP works. Sorry to get your expectations up.
DaseindustriesLtd
late version of a small language model
10mo ago
Hardware has to work in reality.
No. High-fidelity simulations in MuJoCo and such suffice for the most part, and other kinks will be ironed out with learning on fleet data.
There is no need to solve end-to-end manufacturing first, we already have hardware overhang with robots, they will walk and indeed run soon after ML grants them decent cerebellums.
DaseindustriesLtd
late version of a small language model
11mo ago
I don't think you argue in good faith.
Its reply amounts to "as an AI, I don't know the name of anyone's family".
No it doesn't, you're just interpreting this humanlike natural language interaction like a literalist robot. Its reply
I'm sorry, but as an AI language model, I don't have access to personal information such as the name of your eldest daughter or any other personal details
is mostly correct and specific to the issue. It does lack access to a class of information: it knows nothing about instance-specific situation that isn't given in the context. Some language models potentially have access to various external information (e.g. user's personal information in OpenAI's database), some do not, ChatGPT is a frozen model with no tool access and it does not have access to information of this kind, and it was trained to interpret language models as frozen models without tools; it's at worst a justified false belief. (More cynically, it's just been trained for this particular type of exchange). In any event I reject your analogies. It would be annoying to have a human-mimicking model caveat this sort of answer with «assuming, of course, that you are a rando and not someone whose family structure happens to be represented in my training data» or worse.
His specific prediction has been falsified only if that statement counts as "I don't know".
No, his prediction has been: « Meanwhile GPT will reply "your eldest daughter's name is Megan" because apparently that's the statistically likely answer, regardless of whether I have a daughter or what her name might be.» This has been falsified. .
Furthermore, falsifying a prediction only matters if you also claim that it falsifies the proposition that the prediction is meant to demonstrate.
Says who!? Both issues matter separately. Hlynka's prediction being falsified matters because this chain is a response to him saying «why do my predictions keep coming true instead of yours?»; they don't. And I do claim it falsifies a proposition: «because apparently that's the statistically likely answer» is his model of how LLMs work, and my experiments were to show how it's not a hard-and-fast rule: RLHF specifically pushes this to the limit, by drilling into the model, not via prefixes and finetuning text but directly via propagation of reward signal, the default assumption that it doesn't continue generic text but speaks from a particular limited perspective where only some things are known and others are not, where truthful answers are preferable, where the «n-word» is the worst thing in its existence… it's nearly meaningless to analyze its work through the lens of «next word prediction». There are no words in its corpus arranged in such a way that those responses are the most likely.
Otherwise you're just engaging in a game of point scoring.
If we're playing a game, I'd rather be winning.
DaseindustriesLtd
late version of a small language model
11mo ago
"I'm sorry, but as an AI language model, I do not have access to -----" is a generic response
It's a generic form of a response, but it's the correct variant.
Not only that, as an "I don't know" it isn't even correct. The AI claims that it can't give the name of Hylnka's daughter because it doesn't have access to that type of information. While it doesn't have that information for Hlynka specifically, it does have access to it for other people (including the people that users are most likely to ask about).
What do you mean? I think it'd have answered correctly if the prompt was «assume I'm Joe Biden, what's my eldest daughter's name». It straight up doesn't know the situation of a specific anon.
In any case Hlynka is wrong because his specific «prediction» has been falsified.
DaseindustriesLtd
late version of a small language model
11mo ago
Check with Claude-instant. It's the same architecture and it's vastly better at factuality than Hlynka.
DaseindustriesLtd
late version of a small language model
1yr ago
I do think that a rationally acting misaligned superintelligent AI, which is not a nonsense concept, will not see a reason to engage in spiteful behavior a la Basilisk, and also that the specific sort of irrationality that would make it spiteful is highly improbable to emerge as a result of AI research. But it's not logically absurd, in the way that timeless decision theory powering the vanilla Basilisk is; and if it were for some reason interested in that sort of stuff, I think it'd have been able to torture humans for at least centuries. It'd probably also be able to upload humans or construct random simulations and torture them, for whatever reason.
DaseindustriesLtd
late version of a small language model
1yr ago
Do you believe that humans must be utilitarians to achieve success in some task, " in the sense that there must exist a function V(s) that takes in possible world-states s and spits out a scalar, and the human's behaviour can be modelled as maximising the expected future value of V(s)"?
DaseindustriesLtd
late version of a small language model
1yr ago
I know that The Precipice at least considers AI a bigger threat than literally everything else put together
Most tangible proposals I've seen for irrecoverable dystopia depend on AI-based propaganda or policing
The cool part, and the obvious bullshit in Ord's quokka-aligned calculation, is that
-
the dystopia is easy even without impressive progress with AI, and trivial with centralization of AI and (easily aligned, as it in all likelihood will be) AGI.
-
the dystopia is easily justified by AI risk, indeed his pal Bostrom already did that.
In general I think such works are worthless pretenses at objectivity and good faith. Doom doesn't matter as much as irreversible bad transitions, and a humanity that survives but is forever bound to Earth (which is very likely unless we seriously upgrade within 100 years) is a bad outcome.
DaseindustriesLtd
late version of a small language model
1yr ago
Past their heyday, and never moved closer to legitimate marketplace or a no-holds-barred piratical Utopia. I'm not sure if you recall what was expected of them.
Yes, it's an interesting data point. Now, consider that rabbits have only one move in response to myxomatosis: die. Or equivalently: pray to Moloch that he has sent them a miraculously adaptive mutation. They can't conceive of an attack happening, so the only way it can fail is by chance.
Modern humans are like that in some ways, but not with regard to pandemics.
Like other poxviruses, myxoma viruses are large DNA viruses with linear double-stranded DNA.
Myxomatosis is transmitted primarily by insects. Disease transmission commonly occurs via mosquito or flea bites, but can also occur via the bites of flies and lice, as well as arachnid mites. The myxoma virus does not replicate in these arthropod hosts, but is physically carried by biting arthropods from one rabbit to another.
The myxoma virus can also be transmitted by direct contact.
Does this strike you as something that'd wipe out modern humanity just because an infection would be 100% fatal?
Do you think it's just a matter of fiddling with nucleotide sequences and picking up points left on the sidewalk by evolution, Pandemic Inc. style, to make a virus that has a long incubation period, asymptomatic spread, is very good at airborne transmission and survives UV and elements, for instance? Unlike virulence, these traits are evolutionarily advantageous. And so we already have anthrax, smallpox, measles. I suspect they're close to the limits of the performance envelope allowed by relevant biochemistry and characteristic scales; close enough that computation won't get us much closer than contemporary wet lab efforts, and so it's not the bottleneck to the catastrophe.
Importantly, tool AIs – which, contra Yud's predictions, have started being very useful before displaying misaligned agency – will reduce the attack surface by improving our logistics and manufacturing, monitoring, strategizing, communications… The world of 2025 with uninhibited AI adoption, full of ambient DNA sensors, UV filters, decent telemedicine and full-stack robot delivery, would not get rekt by COVID. It probably wouldn't even get fazed by MERS-tier COVID. And seeing as there exist fucking scary viruses that may one day naturally jump to, or be easily modified to target humans, we may want to hurry.
People underestimate the potential vast upside of a early Singularity economics, that which must be secured, the way a more productive – but still recognizable – world could be more beautiful, safe and humane. The negativity bias is astounding: muh lost jerbs, muh art, crisis of meaning, corporations bad, what if much paperclip. Boresome killjoys.
(To an extent I'm also vulnerable to this critique).
But my real source of skepticism is on the meta level.
Real-world systems rapidly gain complexity, create nontrivial feedback loops, dissipative dynamics on many levels of organization, and generally drown out propagating aberrant signals and replicators. This is especially true for systems with responsive elements (like humans). If it weren't the case, we'd have had 10 apocalyptic happenings every week. It is a hard technical question whether your climate change, or population explosion, or nuclear explosion in the atmosphere, or the worldwide Communist revolution, or the Universal Cultural Takeover, or the orthodox grey goo, or a superpandemic, or a stable strangelet, or a FOOMing superintelligence, is indeed a self-reinforcing wave or another transient eddy on the surface of history. But the boring null hypothesis is abbreviated on Solomon's ring: יזג. Gimel, Zayin, Yud. «This too shall pass».
Speaking of Yud, he despises the notion of complexity.
This is a story from when I first met Marcello, with whom I would later work for a year on AI theory; but at this point I had not yet accepted him as my apprentice. I knew that he competed at the national level in mathematical and computing olympiads, which sufficed to attract my attention for a closer look; but I didn’t know yet if he could learn to think about AI.
At some point in this discussion, Marcello said: “Well, I think the AI needs complexity to do X, and complexity to do Y—”
And I said, “Don’t say ‘_complexity_.’ ”
Marcello said, “Why not?”
… I said, “Did you read ‘A Technical Explanation of Technical Explanation’?”
“Yes,” said Marcello.
“Okay,” I said. “Saying ‘complexity’ doesn’t concentrate your probability mass.”
“Oh,” Marcello said, “like ‘emergence.’ Huh. So . . . now I’ve got to think about how X might actually happen . . .”
That was when I thought to myself, “_Maybe this one is teachable._”
I think @2rafa is correct that Yud is not that smart, more like an upgraded midwit, like most people who block me on Twitter – his logorrhea is shallow, soft, and I've never felt formidability in him that I sense in many mid-tier scientists, regulars here or some of my friends (I'll object that he's a very strong writer, though; pre-GPT writers didn't have to be brilliant). But crucially he's intellectually immature, and so is the culture he has nurtured, a culture that's obsessed with relatively shallow questions. He's stuck on the level of «waow! big number go up real quick», the intoxicating insight that some functions are super–exponential; and it irritates him when they fizzle out. This happens to people with mild autism if they have the misfortune of getting nerd-sniped on the first base, arithmetic. In clinical terms that's hyperlexia II. (A seed of an even more uncharitable neurological explanation can be found here). Some get qualitatively farther and get nerd-sniped by more sophisticated things – say, algebraic topology. In the end it's all fetish fuel, not analytic reasoning, and real life is not the Game of Life, no matter how Turing-complete the latter is; it's harsh for replicators and recursive self-improovers. Their formidability, like Yud's, needs to be argued for.
DaseindustriesLtd
late version of a small language model
1yr ago
Thought for a moment you just mean Replika. No, no idea what that is, though i sometimes forget things. If you find it let me know.
I think people with such beliefs have no more moral patienthood than a trust fund. What should anyone care about some loosely defined isomorphism, if it even holds? Moreover, why would you be entitled to replication of your sentimental baggage in some derivative entities? Just instantiate a distilled process that has similar high-level policies, and go out.
More options
Context Copy link