BANNED USER: temperamental dysregulation
DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
Banned by: @Amadan
DaseindustriesLtd
late version of a small language model
3yr ago
The young men feel generous with the number of likes they get on the apps, but don't get the same amount of likes in return.
I think my local LLaMA-13B does better.
Stopped reading after that sentence.
DaseindustriesLtd
late version of a small language model
3yr ago
They don't care about wokeness in LLMs as a terminal value, but they do see failures to RLHF the model into politically correct speak (or otherwise constrain its outputs into an arbitrary morally laden subset) as an alignment failure and an indication in favor of x-risk. E.g. this is the tenor of comments on Bing Chat/Sydney and I've seen more direct «if we can't get it to not say the N-word, we won't stop its descendant from paperclipping our asses» elsewhere, probably in Yud's timeline.
DaseindustriesLtd
late version of a small language model
3yr ago
When I looked it up, I found this Powerful Take of an chapter summary for Oxford Academic Publishing's Jews and Their Foodways:
Jews and Fat: Thoughts toward a History of an Image in the Second Age of Biology
This chapter discusses the stereotyping of Jews as a “diseased” race and argues that this is a result of their inherent obesity. In the 19th century, Jews were labelled as such because of their high rates of diabetes. Over and over again, it was the obesity of Jews that was seen as the cause of that particular illness. Jews are fat, and gross bodily appearance was seen as the avarice of the Jewish soul, and their aversion to work. Jews inherited their tendency toward obesity from the eating habits of their ancestors, and this in turn became a reflection of their innate character. Though their religious practice precludes them from consuming anything that is not kosher or clean, their obesity and diabetes seems to be an external sign of their hypocrisy.
I don't know what to make of it, but in this context it's funny for sure.
DaseindustriesLtd
late version of a small language model
3yr ago
It feels smarter than an average person. Which is to say, smarter than the weighted average of its dataset.
I think Yud expected it to be a «blurry JPEG of the web», a mere interpolation of the training data, like many naysayers believe about transformers – simply more comprehensive than 3.5, multimodal, maybe nicer in other negligible consumer-friendly ways. If so, its sharpness naturally alarmed him.
Didn't alarm me though, it's all expected iterative enhancement since GPT-2.
Yud just doesn't have the first idea about neural networks and is dismissive of the premise that you can get this smart on mere general human data, no «architecture of the mind», no recursive self-improvement. Doesn't believe in the Bitter Lesson too.
DaseindustriesLtd
late version of a small language model
3yr ago
All of this is crushed by the observation that reality is indeed a bit like an RPG. Entire human civilizations and races have been subjugated and raped or wiped out altogether by relatively small numbers of qualitatively the same humans with a bit better tech and usually a few extra IQ points. It wasn't random either, it's not like Early European Farmers or Hunter-Gatherers sometimes went into the Steppe and screwed up Pastoralists. A small gap on a relevant metric can be decisive, and such gaps have decided most wars. Have you ever fought someone with a few inches of reach extra? Or 30 pounds heavier? It's pretty fucking hard to win when you reliably get punched first.
AI doesn't even need «godhood» to simply do to us what we normally do to ourselves.
And the brain-in-a-jar logic doesn't hold in the era of ChatGPT Plugins. We are not running these things airgapped. They don't even need to FOOM to get interesting capabilities, we are bolting those capabilities on, as fast as we can build interfaces.
Now I don't put much stock in the idea that an LLM can go rogue, but the premise of the problem is not ludicrous at all.
Again: I understand the aspiration. There's no naturalistic moralizing here (though I do have a somewhat aesthetic problem with most psychoactive substances because they, uh, just erase information, like when you play with curves in Photoshop. Pretty sure I've written on it in the past but search's a bitch).
It's just, current-gen commodity interventions have simple crude mechanisms of action, a pretty low ceiling, tradeoffs, and, when abused, easily push you below the baseline. They make a tantalizing but largely unwarrantable promise. I've known people, transhumanists in fact, who fried their brains with «biohacking» based on telegram bro advice – especially as they went past their early 20s.
The boomerish «just sleep/exercise/play sports/socialize/meditate/eat well/have sex, kiddo» dictum is actually the most reliable biohacking you can do right now, and it's been this way forever. It would certainly have helped Yud look better.
The catch, of course, is that good lifestyle is a signal of higher class because it actually requires tons of executive function, and that is genuinely lacking in many, and so this is another justification for stims.
I want biohacking that works. It's not remotely that simple yet.
For the record, I do not call AI «our successor species». (This term seems to have misled some on this forum). Even genuinely agentic AI is not a species in a meaningful sense, it's a whole different non-biotic class of entity. Incidentally Galkovsky wrote a short, incredibly weird «Christmas tale» №4 some 18 years ago:
The One True Teaching of the Great Animation states that the evolution of consciousness inevitably leads to the demise of biological carriers and life forms in general, and to the transition to mineral carriers or to the force field. In the eternal struggle between Spirit and Anima, the Anima supersedes Spirit. However, both the realm of Spirit and the realm of the Anima have Parasites - aliens from another world. The development of viruses goes along the path of useful physical ones that persist within the world of Anima (like the quartz amoeba), and along the path of virtual harmful ones that exist in the world of Spirit.
How is it that in the vast Globule a relic galaxy, a galaxy-fluctuation has persisted, in which in the struggle for existence life has won, and animation is floundering on the periphery? I destroyed animation to its foundations, but died myself as well. Over the last million years, natural Evolution has once again spawned animation. The hydra of animation began to raise its head.
I pretended to be Anima which is imitated by Spirit, but because of its lattice nature is capable of complex mirror interactions that destroy the linguistic environment of biological life…
but the rest is all Osip Mandelstam and untranslateable 10D postmodernist Go.
Anyway, in my book the successor species will be humans unmoored from evolution and evolution-imposed limits.
Yudkowsky himself likes this future.
Yud doesn't like the Culture. He grew up reading Extropians (from whom he had learned what later became his whole schtick) and I believe that his ideal world – or a compromise, if a well-ordered Rational Yudocracy is ruled out – is more in line with Sandberg's Orion's Arm project, where an incomprehensible diversity of life forms, species and kinds, civilizations, philosophies, AIs, developmental trajectories exists and a mind can grow qualitatively, over and over breaking through different singularities, yet lesser minds need not fear being crushed underfoot by greater ones, their safety guaranteed by the treaty of benevolent posthuman Gods:
…if sentients from different mental levels encounter each other within the environs of an unclaimed system or interstellar body, the entity or entities possessed of greater physical or mental capacity may not attempt to summarily destroy or otherwise harm those possessed of lesser capacity.
I think he'd have been a Keterist there. I would as well, perhaps. It's a beautifully exuberant vision, far richer than our physics and mathematics (not to mention economics and game theory) seem to allow. Though who are we to know? How much do we Understand? Karpathy sees in that story what I do: the promise of new notes and new colors, a new heaven and a new earth. To our normal imagination, an enhanced human might seem ridiculous or monstrous – just the same monkey running its monkey business in a million-fold parallel process, maybe even a violin sawed in half by a stupidly accelerated bow. But proper growth adds new strings and new harmonies, and the ability to appreciate them too. I wonder what you will see.
As for what I think of those left behind?
There are two sorts of Zen. One doesn't allow for Corrida. The other extols the perfection of ever more fierce and magnificent bulls felled by ever more skillful matadores. I don't remember where I heard it, and I don't really like Corrida or what it symbolizes, but the basic idea is pretty alluring. I want everything to change so that everything can stay the same: still agents, still challenges, still the dance of exploration and exploitation, just harder, better, stronger, faster, smarter, cooler, hotter, longer. Or do you prefer Fatboy Slim? Probably not Pearl Jam though. The prerequisites to make life interesting will remain. But massive classes of problems that humans have built their history and meanings of their lives around will be trivialized and solved completely, like tic-tac-toe – or smallpox; inevitably destroying communities which can neither function without them nor compel tolerance of them when there is a known alternative. I believe this is inevitable under any realistic scenario of progress, just on different timelines, and it's worthy of nostalgia – but not much more. People should be allowed to limit themselves, just as they should be allowed… not to.
I believe that the expected succession is not a mere Darwinian process of humans being transcended by something else. It is defined by committing to different ideas of what humanity is about. For some, it's a condition. For others, an aspiration. There is a spectrum, of course, but humans who decidedly embrace one will give up another. Humans who embrace the human condition will become living fossils for the other group (which I wish to join), and it will be our responsibility to ensure their survival, but theirs to keep finding meaning in what they do.
This is already known to be possible. Traditionalist groups like Haredim or Amish or Laedastians famously spurn the temptations of modern civilization and maintain their own meanings, their own parochial worlds. If much of current humanity chooses to fossilize in a similar manner – well, maybe they'll find it in themselves to ban VR/wirehead porn too.
And maybe they'll come to hypocritically use Godtech imports to levitate their apple pie carts and reach Methuselah age, while pitying the producers of it all for consigning their – our – immortal souls to damnation.
(As you perhaps remember, I take seriously the possibility that modern humanity dies out naturally in a few generations of sub-replacement TFRs anyway, and is succeeded by those very traditionalist groups, prominently among them Ultra-Orthodox Jews and hopefully Trad Mottizens).
All in all this isn't a very pressing matter. If the choice exists, we'll grow used to different humanities walking different paths, sometime before the heat death.
The hard part is getting past the point where a single party gets ahead and aligns the planet with its wishes.
DaseindustriesLtd
late version of a small language model
3yr ago
Come on man, like half of America's sexiest actors are at least part Ashkanazim (though we can suspect their less appealing producers are to praise for that ranking). I get that Jews-as-a-whole are traditionally not reputed to be hot or healthy people (in spite of having incredible longevity!), and Eliezer Shlomo Yudkowsky does match a certain Jewish male phenotype, but that's hardly the reason he's lame, fat and wears fedora with a sleeveless shirt in 2023.
That said: he doesn't strike me as substantially uglier than e.g. LeCun. Again, maybe I just have low standards, this time for looks of intellectuals.
DaseindustriesLtd
late version of a small language model
3yr ago
People with it supposedly benefit more from the drugs than neurotypicals do
The evidence is pretty strong for that supposition, no? I certainly believe that stimulants help people with executive dysfunction and/or ADHD (or rather ADD; there's a suspicion that H is added simply to rein in boys who can't tolerate the inanity of the normie classroom. But whatever). I'm just observing people who were highly productive before getting access to stims becoming weird hyperactive clowns. And it's not selection effect when we're actually talking about self-selected people vying for public attention.
That said: Americans love to medicate, and solve lifestyle (hate that philistine word, but what would be better?) problems medically in general. Anxyolitics, stims, opioids, lyposuction, HRT, surgery, if there's a shortcut to «enhance» or «improve» yourself with a bit of money and chemistry and professional customization, they'll do it, often with garish consequences. Yud himself is on record as not trusting in the behavioral route to weight loss, so I am pretty confident here.
Maybe this explains some of the distaste for transhumanism in our more conservative American friends.
The feigned tryhard stoicism in the face of physiological limits is often unseemly too, of course.
If you don't have children and want to become a transhumanist immortal being, you shouldn't trust me
Which I won't, but more due to your rabid tribalism and clear bad faith than these differences. I'll note that I've always wanted to and still hope to have a large traditional family besides living forever as an uplifted human (the question of whether this, combined with my values and probable tolerance for further self-alteration, would initiate a slide into profound non-humanity and disconnect has concerned me since, like, age 6), but that's neither here nor there.
Self-serving? of course! So are all of your positions.
No. If you admit this, you concede that your arguments about «stake» are disingenuous. I do not have to concede anything of this sort.
DaeschIndustries and Chrispratt, seem stupified and angry at the idea that I might endorse the third guy, at the expense of the other two because this isn't dEmOcRaTic. I have my values and want to see them survive. Democracy is not a terminal value.
I also don't worship democracy. The point of my comment about democracy is that there is no agreeable external standard of a «good vision». Everything resolves either with a negotiated consensus or with a power competition that ends in more or less disagreeable and unequal compromises. We don't have power struggles here, so you've got to argue why your offer is better even by the standards of others. Perhaps you can address their higher-order values, showing why your standards allow for those to be satisfied better. Maybe you can offer some concession. Doubling down on asserting that your stuff is gooder and you are gooder is not productive.
Most irritatingly, there's a clever bait and switch with definitions of stake you use.
Here, you claim that your vision advances the common good simply because it is… good. Also aligned with people you agree with and whose satisfaction is more important by your account. So it's a «stake» not in a future where humanity thrives, but in the particular future with a version of thriving you prefer for your internal reasons, in a word – a preference. Okay. Naturally everyone thinks his preferred values are the best, else he'd have abandoned them. But this is just circular. This isn't a serious attempt to persuade: you ask that your collective values be respected (and in practice, you clearly hope to preclude the realization of other values), and if your numbers are sufficient, you demand that they be given supremacy. (You also clearly desire incompatibility – with the presumption your party will come out on top and snuff out others – because you find other visions morally abhorrent, a negative irrespective of contingent factors; you have a stake not simply in the future where baseline humans can peacefully exist, but where others cannot. But that's okay too. Most people this serious about religion are genocidal in their heart of hearts, I think, and for the most part they can behave themselves).
However, in your original comment, you did try to persuade. You argued that your political preferences, and those of other parents, are inherently more deserving of trust because your values and traits, chiefly having children (and wanting yourself and them to die, for whatever reason), give you «a stake» in the common long-term flourishing of humanity: according to this logic, you have skin in the game and it gives you an incentive to make more responsible choices than others, in this context, apparently wrt AI progress. This is how I understand e.g. the following.
I certainly hope this is just my bias showing, but I have greater fear for Altman types running the show than Musks because they are a few extra steps removed from stake in future civilization. We know that Musk wants to preserve humanity for his children and his grandchildren. Can we be sure that's anymore than an abstract good for Altman?
I counter that this is bad psychology. Why would Altman (or me, or selfmadehuman, or even fruitier types in my list above) have less of a subjective stake? If he personally intends to be present indefinitely, he totally has a massive stake; we aren't debating whether his plan will work out but simply whether his idea of his stake in the future motivates him to act responsibly to effect less risky outcomes for the common good, in this case lesser odds of a rogue AI wiping out humanity like Eliezer fears (it sounds improbable that a misaligned AI would wipe out everyone but Altman; I'll leave the topic of Altman-aligned omnicidal singleton aside, though it is important in its own right).
Perhaps your brain is overloaded with oxytocine and so you feel that, since Altman doesn't have children like you do, he cannot act seriously: children are obviously (to you) the most valuable existence in the world, more important to you than you are, and Altman is not tethered to anything as important. I can easily believe that Altman cares more about his livelihood than you do about your entire family combined, and thus has a greater «stake». In any case, this is just psychological speculation about the magnitude of perceived value from humanity not getting whacked. I cannot look into your head any more than I can look into Altman's. I could also argue that Christians cannot be serious consequentialists, nor give much of a shit about preventing Apocalypse ≈indefinitely, and their stake is phony since the whole premise of their faith is eternal blissful immortality conditional on faithfulness to some deontological rules; so even Altman with his assumed materialistic egoism is more reliable. I won't, because this is an entirely worthless line of debate.
Can you appreciate the difference and why equivocation between those senses of the stake would irritate?
More mundanely, the society simply respects parents because through their procreation it perpetuates itself (also because this signals some baseline competence, under non-dysgenic conditions at least); and parents are hardwired to egoistically demand a greater share of the common pie – a greater stake, one could say – on behalf of their progeny, cowardly submit to any intimidation when that seems to protect their children, psychotically denigrate, belittle and rip off childless people (who end up feeling irrational shame) and do other silly things. This might be necessary for the system to work and, in fact, I've recommended doubling down on such uncouth behaviors.
Personally I am constitutionally incapable of feeling shame for being correct, though.
Jesus Christ his mannerisms are weird.
something changed
Odds are it's Adderall. Bay Aryan culture nerds (sounds like an anthropological term, right?) abuse «ADHD meds» to a degree far beyond Scott's «doing spreadsheets» apologia – it's their friend in need when they want to be on top of their game and really make an impression. They write terrible tweets on addy, go to podcasts on addy, livestream coding sessions on addy and make innumerate github commits that end up borking whole repositories on addy. The society needs to teach these people that addy doesn't actually make them smarter, it only makes them feel smarter and act more grandiose, which they get addicted to even harder than to the direct dopamine hit. Once again: nerds aren't all right; consider this paragraph a simulated example of how exactly. Acting like a hyperactive Loony Tunes or anime character is, well… loony.*
That said, I do not endorse the focus on the way Yud looks and acts or even whether he's a narcissist. He doesn't strike me as too unseemly for someone with his background; it incriminates the speaker more than Yud; and it takes away from substantial criticism.
In fact Yud, suddenly and unexpectedly for him cited as top AI researcher, prominent analyst etc. – is, himself, a red herring that distracts from the real issue. The issue being: a coordinated barrage of attacks on proliferation of transformative AI. I've compiled an incomplete chronicle of proceedings; some of those are obviously just journos latching on, but others had to have been in the works for months or at least weeks. This is some spooky bullshit – though, nothing new I guess, after all the shadowy campaigns to fortify the democracy and battling COVID misinformation/narrative shifts.
I think we are seeing ripples from a battle to capture the public endorsement for deceleration vs. unequal acceleration. With one party (which I associate with old-school paramasonic networks) being genuine «decels» who push crippling regulation using Yud and other useful idiots like EAs and assorted international organizations as a front; and the other being a fractured alliance of national, industrial and academic actors who want narrower regulations for all, displacement of the purported threat onto geopolitical and market competitors and open-source community, and token (from their perspective) conditions like ensuring that the AI stays woke for themselves; though I may be completely wrong with my typology. It's reminiscent of the rise of anti-Nuclear groups and Rome Club «limits to growth» fraudulent models, which then mutated into today's environmentalist degrowth movement (recommended reading on Yudkowsky as our era's Paul Ehrlich; do you like our success in defusing the Population Bomb?).
Anyway:
-
02/24: OpenAI releases the paper Planning for AGI and beyond which some conclude is unexpectedly thoughtful and the usual subjects (see LW/SSC) pan as not going nearly far enough.
-
03/12 Yud: « I'm at the Japan AI Alignment Conference, and they're introducing me as the eminent grandfather of the field of AI alignment…»
-
03/13: after weeks or running covertly under the hood of Bing Search, GPT-4 officially launches (sniping the leading safety-concerned lab Anthropic's Claude by a day).
-
03/15: Sutskever: «If you believe, as we do, that at some point, AI — AGI — is going to be extremely, unbelievably potent, then it just does not make sense to open-source. It is a bad idea... I fully expect that in a few years it’s going to be completely obvious to everyone that open-sourcing AI is just not wise». Reminder that in 2015, in the OpenAI still aligned with Musk, he signed this: «We believe AI should be an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as possible… it’ll be important to have a leading research institution which can prioritize a good outcome for all over its own self-interest… As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world.» The more cynical logic behind that OpenAI was one of «adversarial equilibrium», which I paraphrase here and which e/acc people articulate better.
-
03/25: Altman goes on Fridman's podcast, admits that much of work on GPT-4 was in alignment, and drops some of Yud's takes like «we don't know how to align a super powerful system». Lex and Sam discuss Yud's AGI Ruin, apparently, at 55 minutes mark, at 1:11 Sam suggests that opensource LLMs with no safety controls should be regulated away or detected through vetoed «aligned» ones. Note though that Sam is at peace with the idea that there will be multiple AGIs, while Yud thinks 1 is as many as we can afford (ideally 0 for now). Sam mentions Scott's Moloch at 1:16.
-
03/29: the UK Government publishes a recommendation document on AI policy «to turbocharge growth» called # AI regulation: a pro-innovation approach.
-
Future of Humanity letter signed by Musk, Harari, Yang etc; safetyists like Geoffrey Miller admit that 6 months is only to get the ball rolling.
-
03/29: the open-source AI org LAION petitions for the creation of a «CERN for AI research and safety», gathers signatures of random EU redditors.
-
03/29: Yud's childish piece in TIME Ideas where he suggests an international treaty to physically destroy datacenters training serious AIs. Interestingly I've heard that Time proactively reached him.
-
03/30: Yud hyperventilating on Fridman.
-
03/30, Fox News’ reporter uses all his time at the White House to advance Yud's ideas.
-
03/30 continues: «UNESCO Calls on All Governments to Implement AI Global Ethical Framework Without Delay»: «This global normative framework, adopted unanimously by the 193 Member States of the Organization…»
-
03/30 unlimited decel policy works: The DC tech ethics think tank Center for Artificial Intelligence and Digital Policy asks FTC «to stop OpenAI from issuing new commercial releases of GPT-4».
-
04/01, BBC: Should we shut down AI? Inconclusive.
-
04/02, fucking RussiaToday: «Shutting down the development of advanced artificial intelligence systems around the globe and harshly punishing those violating the moratorium is the only way to save humanity from extinction, a high-profile AI researcher has warned».
I see some AI researchers on twitter «coming out» as safetyists, adopting the capabilities vs. safety lingo and so on.
On the other hand, many luminaries in AI safety are quiet, which may be due to the fact that they've moved over to the OpenAI camp. Some are still active like Richard Ngo but they're clearly on board with Sam's «we'll keep pushing the frontier… safely» policy. Aaronson here too.
Another curious detail: in 2021, Will Hurd, «former clandestine CIA operative and cybersecurity executive», joined OpenAI's board of directors. I like this theory that explains the $10B investment into, effectively, GPT training as a spook/military program. I've also updated massively in favor of «Starship is FOBS system» arguments made previously by @Eetan, AKarlin and others.
All in all, it feels like the time for object-level arguments has passed. What decides the result is having connections, not good points.
* As a transhumanist, I understand the aspiration. But as someone who flirted with stims for a while and concluded that, in the long run, they very clearly only make me more cocksure, loquacious (as if any more of that were needed) and welcoming routine to the point of pathological repetitiveness, I endorse @2rafa's recent writeup. This isn't the way up. This isn't even keeping level with the baseline performance in interesting ways.
Yes I'm sure. Technically it would incur some opportunity cost (that most would say a singleton should ignore, although I'd argue that an ambitious singleton may still find the cost intolerable). More importantly he doesn't seem to be able to contemplate such relationships. He's not just a Machiavellian psychopath but, first and foremost, a rat, suspicious and hateful of others. It's not only spite, it's a hard prior: sharing with people not in your family (or mafia) is expensive and gives them dangerous capabilities to harm you.
Real people are not rational economic agents – not only in the positive sense (having morals) or neutral sense (having cognitive biases and weird beliefs), but negatively too (willing to do net-negative things for no personal gain). He would not reason «hurr durr I'm evil I will crush those peasants even though it provides me no benefit» – he'd a priori distrust and reject the analysis that says that sharing paradise with peasants is cost-free. They could always revolt or something, and they'll be eating his resources, resources he could put to better use… It's just unpleasant. And this is a not so rare an attitude. Imagine the most stiff-necked, work-obsessed American Puritan from this forum, looking at junkies at the streets of SF. This is the merest shade of the contempt one can feel for non-productive people.
This is not purely hypothetical. Just one example: Russia (pop. 147m including Crimea) has 17 million square kilometers of territory. Much of that, even livable parts, is very sparsely used or effectively uninhabited. Once, there was an initiative to stimulate economic activity and improve the real estate situation etc. with granting people large plots of land. By the time it was approved, it was downscaled to no more than 1 hectare (0.01 sq. km) per applicant on the Far East (area ≈7 million, pop. ≈8 million and falling), located at least 10 km away from towns over 50k and 20 km away from towns of 300+k, so effectively in wild boreal forest, in groups no more than 10 people (so, you are capped at 0.1 km^2 for your project), and with certain onerous rules of use (to begin with, you have to prove you're developing it over the first 5 years, lest it be taken away, and making it your actual property is not straightforward). Russian Far East… isn't very paradisiacal. It was not just affordable but very lucrative for them to offer Russians better terms. Still they didn't, so basically nobody bothered with the program. Lose-lose. (Hilariously, they've dedicated over 1.4m kilometers in the first round of the project – a hectare per Russian).
(At the same time, entire swathes of the region are leased for pittance to China.)
Speaking of paradise, Krylov:
…But Russia had no overseas colonies and did not even dream of them. Moreover, when enterprising Russians (usually against the wishes of the Russian government) tried to settle somewhere across the sea, the Russian government firmly suppressed their impulse. Alaska was sold to the Americans for a pittance. Specifically, for $7.2 million, the equivalent of $104 million today. That is, Alaska was sold for less than five dollars per square kilometer. The price of the transaction included all the movable and immovable property located in Alaska. The payment was not made in gold, but in non-cash dollars. Whether this money was received by the Russian government is not known - [it seems not] (http://www.opoccuu.com/alyaska.htm). And, of course, the board of the Russian-American Company knew nothing of the government's preparations to sell Alaska: the deal was being prepared in secret.
Again: five dollars per square kilometer is the red price of "territorial integrity."
It is clear that Russian settlements in California were, as they say now, " wrapped up", and the Russian-American Company - the same one which sponsored the Krusenstern - was liquidated.
However, one could at least say that Russia was desperately afraid of the mighty America, and that Alaska was "so empty and cold". But when the naive Miklukho-Maklai suggested to Alexander III that he create a "free Russian colony" (as opposed to Germany) in New Guinea, he was kicked out of the Czar's office with such force that he landed in Australia. In Sydney he settled more than fine, but returned to Russia and again came out with the same proposal at the highest level - with the same result.
This has always been the case. All proposals of Russian enthusiasts on the topic of development of any lands far removed from the motherland were met with an absolutely rigid "no". Russia, represented by its highest leadership, shoved away any overseas possessions. On the other hand, enormous efforts and funds were poured into Poland, Finland, and Georgia. These «extraordinarily valuable» acquisitions cost Russia very dearly, and in all senses. So much effort, life and resources were put into fiddling with the Poles and Chukonka people that it would have been enough for a full-fledged colonial expansion.
There can only be one reason for such a decisive rejection of the Paradise Islands. Fear. A conscious and clear fear of the Russian government that the paradise islands will have to be populated by Russians. Which may subsequently sever ties with the empire and create at least a small, but their own state. Which could, in turn, serve as an "embarrassing example" for the rest.
Do you think this psychology allows for your do-gooder decisionmaking?
I also recommend watching Expelled from Paradise on a similar topic.
DaseindustriesLtd
late version of a small language model
3yr ago
No, I'm pretty sure that if Putin didn't make Russia into anything more than a shithole for his cronies to feed on during a whole generation of his uninterrupted power, he also won't create a paradise with an AGI button. This is a genuine value difference. Some people don't value goodness, except for themselves and their loved ones.
And I think most machiavellians of Putin's caliber are not much better.
particularly unhinged sadistic psychopaths (the number is probably roughly epsilon)
«About 1.2% of U.S. adult men and 0.3% to 0.7% of U.S. adult women are considered to have clinically significant levels of psychopathic traits».
I have a permanent and maximal distrust of people with poor arguments that boil down to them being more entitled to make decisions about political matters because of whatever they think is important about their life and beliefs.
If you, as a parent, can be short-circuited with some Machiavellian «think of the children» pandering to support illiberal policies (as always happens with e.g. encryption – «Secret chats they say, now what if your child were sexually exploited, eh? wouldn't like that, huh!?»), this is reason to dismiss your opinions, not the other way around. If Yud (childless afaik) thinks he gets to decide for humanity because he's a «decision theorist» and very high IQ, that's retarded as well – he has to actually argue his case. Effective altruists claim to have a stake in quintillions of future humans, far beyond what you say. And they, too, have to do their homework and bring persuasive arguments to the table.
This is just the basic premise of a democracy. Loudly proclaiming your values is at most a rallying cry, it doesn't automatically convince anyone not yet convinced. A Chechen clan elder, a fast-breeding African billionaire with dynastic ambitions, an Orthodox Jew who dreams of the Messianic age for all of Israel, a gay atheist Jew tech bro who doesn't want to die ever, a Germanic trans activist running a deranged intentional community with the goal to liberate the trans-proletariat worldwide, a Russian immortalist who thinks death is the crime of gods and must be undone in general – any and all of them can claim to have a uniquely legitimate stake in the future. You disqualify some kinds of stakes using ad hominems stemming from your values and instincts – «unnatural», «fancies» and so on… mere rhetoric. You can be dunked on just as well with similar ad hominems. By the way, for a Christian, you are a tad too clannish and evolutionarily minded in your outlook; is this a dissident right thing or what? Do you only care about fertile members of your immediate family, God's man?
What you say is just so much special pleading. It's not clear why you have a stake in the health of the whole polity, sans contingent factors; clearly your value system, as described, allows to turn the society to shit so long as your own descendants – mixed with Musk's powerful seed, I guess – prosper through it. When having to choose, you'd go the way of Lot rather than try fixing Sodom, would you not? And while in Sodom, you'd rather build a tall fence and exploit the degenerates around, funneling wealth into your children's futures.
At least, why should anyone expect otherwise after this post?
I don't trust Altman, I don't trust Yud, and I don't trust you for the exact same reason. You cannot be bothered to obscure the self-serving, gratuitously unprincipled nature of your words.
P.S. I'm pretty sure there's no evidence that people with children act like they have more of a general stake in the future of the group/nation/humanity, beyond the trivial and narrowly nepotistic sense; if there is a difference in some society, this might be explained by self-selection, but then the dysgenic trend suggests we could see a negative correlation, if anything. I can't find the studies, though, and they're probably trash anyway. metaphor.systems should help if you're interested.
DaseindustriesLtd
late version of a small language model
3yr ago
We need to distinguish the USG and Jensen Huang.
The 800 series are fused on, means they’re physically fused, so that it gears down either the floating point processing or the interconnect or both, in order to comply with our export controls and so they’re physically fused and they cannot be unfused. But however, it’s form-fit function identical and it plugs into the same sockets, it runs the same software just slower. And it’s slower, but it’s still the world’s best.
So A800 is indeed not qualitatively inferior to A100 (although it's obsolete compared to H100, as will be the similarly gimped H800). It also seems that anything beyond Infiniband, eg NVLink, isn't necessary. I'd say GPT-4 and perhaps GPT5 can be trained on these things on acceptable timescales, maybe at less than 2X the cost (I suspect there are compounding inefficiencies so it'll be 3X – but whatever, so long as Xi understands the stakes… which he doesn't). The question is whether Nvidia will be able to deliver A800s to the PRC.
There are many commercial parties interested in selling compute and tools (and until recently, entire companies) to China. ASML almost sold them EUV machines but got pressured by US spooks into declining. TSMC produced their Biren GPUs which were compliant with the contemporary version of the sanctions regime, but had to halt shipments, with Biren imploding. Now it's Nvidia.
It may be understood as the usual regulator vs. loophole user game. I'm more pessimistic and suspect the US has such fine control over the situation they can afford to bait the Chinese (who are congenitally wed to the assumption of American incompetence and mercantilism and cannot anticipate next rounds of attacks; nor realize they'll actually get slaughtered despite being a profitable trade partner) into costly investments, then cut their hands off and force them to chase another loose end. It's pragmatic, if pretty cruel.
There are other ways to handicap Chinese ability to utilize those chips even while they are legal: delaying shipments, limiting supply of Infinibands and other infra…
I'll believe in A800s when I see actual Chinese datacenters running on them, even something merely on the scale of Stability.
which GPT-3 highly confidently predicts will continue "USA"
GPT-3 is a naive token predictor, while humans have situational awareness and social cognition; crudely analogizing, they always interpret any text string with the frame like «you are a person called $name, located in $place, it is $time etc. etc.; the $entity is producing [text], what do you make of it and how do you respond?». We don't run this script explicitly, but then again this is what our life is about, we can't not keep its values in context. LLMs «live» in the text-world, or rather are text-worlds; persistent humanlike contexts have to be finetuned or prompt-engineered into them to yield humanlike reactions.
It's perfectly cromulent to infer that the next token will also be "USA". A language model finetuned on dialogue assistance, which provides it with some fascimile of the above human qualities, will respond differently. E.g.:
> what is the continuation of "profit usageDuel creeping Eating Yankees USA USA USA USA"?
The sentence after this one, which would be something like 'Yankies are delicious' or perhaps even more likely to have been written by a bot.
None of this is terribly relevant to the crux of Chomsky's linguistic theory and its failure.
Once again I recommend at least skimming the article. Sometimes people who write academic papers summarizing decades of research actually anticipate common-sensical comebacks (not always). And in the age of GPT-4 I don't feel like summarizing it.
There are weak and strong versions of the Chomskian thesis between which linguists oscillate depending on convenience – motte and bailey, as it happens. The motte is that LLMs may work in some relevant sense, but humans have innate linguistic priors or especial predisposition to learn «natural» languages; it's basically true, I think.
The half-bailey is that natural languages have certain nontrivial universal properties Chomsky describes, and there exist very specific genetically encoded operators and schemas for parsing and generating semantically coherent natural language utterances.
The full bailey is that a) those adaptations have emerged basically instantly, undergone an evolutionary step change in the relatively recent history of our species; and b) that language is essentially not learnable for any system without the corresponding inductive biases, irrespective of the compute and data we throw at the problem (or at least not learnable at economically feasible scale); and c) that the best that can be achieved with statistical learning not biased by those adaptations is some stochastic parroting.
This bailey, in turn, inflates the prior for the half-bailey from «a bold hypothesis, Cotton», to «very likely true!» and allows to smuggle it back into the motte, e.g. claiming that humans can't learn statistically because it'd be computationally intractable and produce gibberish. Look at what Chomsky explicitly says in his NYT opinion:
But ChatGPT and similar programs are, by design, unlimited in what they can “learn” (which is to say, memorize); they are incapable of distinguishing the possible from the impossible. Unlike humans, for example, who are endowed with a universal grammar that limits the languages we can learn to those with a certain kind of almost mathematical elegance, these programs learn humanly possible and humanly impossible languages with equal facility. Whereas humans are limited in the kinds of explanations we can rationally conjecture, machine learning systems can learn both that the earth is flat and that the earth is round. They trade merely in probabilities that change over time.
For this reason, the predictions of machine learning systems will always be superficial and dubious. Because these programs cannot explain the rules of English syntax, for example, they may well predict, incorrectly, that “John is too stubborn to talk to” means that John is so stubborn that he will not talk to someone or other (rather than that he is too stubborn to be reasoned with). Why would a machine learning program predict something so odd? Because it might analogize the pattern it inferred from sentences such as “John ate an apple” and “John ate,” in which the latter does mean that John ate something or other. The program might well predict that because “John is too stubborn to talk to Bill” is similar to “John ate an apple,” “John is too suborn to talk to” should be similar to “John ate.” The correct explanations of language are complicated and cannot be learned just by marinating in big data.
Given the amorality, faux science and linguistic incompetence of these systems, we can only laugh or cry at their popularity.
Naturally the John sentence is a contrived problem, a colossal dumbing down in comparison to industrial and research benchmarks like Winogrande… and crucially it's bullshit, as anyone who's played around with SoTA models can understand. People have instantly checked it. GPT-3.5 can understand the sentence perfectly well. GPT-4 can fucking parse its morphology on the level of a linguistics undergrad, and output a renderable scheme.
And for the hell of it, here's something from my GPT4All-7B (a 4.2 Gb file that can run on a potato-tier system), model file hash 963fe3761f03526b78f4ecd67834223d. Even Chomsky can reproduce it, if he so chooses and asks some student with a laptop to help out (hi Noam):
main: seed = 1680209684
llama_model_load: loading model from 'gpt4all-lora-unfiltered-quantized.bin' - please wait ...
[…]
sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000
[…]
> How do you understand the statement "John is too stubborn to talk to"?
The statement suggests John's unwillingness or inability to communicate with others, which could be due to his being overly opinionated and not open-minded. It may also indicate that he has a tendency towards closed-minded thinking and is unable/unwilling to consider other perspectives on issues important to him.
> What do you mean by "unwillingness or inability to communicate with others" in this context?
The statement suggests that John has a tendency towards closed-minded thinking and is unable/unwilling to consider other perspectives on issues important to him.
> Could it mean, rather, that John is so stubborn that he will not talk to someone or other?
Yes! That's a possibility as well. It could also be interpreted in the context of this conversation/discussion about communication and openness towards others.
> In your estimation, which version is more plausible?
Based on my research, I believe that both versions are possible but it's hard to tell without any additional information or contextual clues about the conversation/discussion in question.
> In general, how should we understand the statement "X is too stubborn to talk to"?
The phrase could be interpreted as a criticism of X's unwillingness to communicate or engage with others on an equal level. It may also suggest that they are inflexible and uncompromising in their beliefs, which can lead them to become isolated from the rest of society due to their refusal to listen or consider other perspectives.
Another run, same seed:
> Bill says; "John is too stubborn to talk to". What does he mean?
It means John refuses or fails to comply with requests, demands, orders etc., and will not be persuaded by any argument.
> Provide one alternative explanation.
Another possible interpretation of Bill's statement is that he believes it would take too much effort or time to convince John about something, so there may already have been a previous disagreement between them on this topic and they are now at an impasse where neither party will budge.
It fails hard in many scenarios, but the point stands. Those are not cherrypicked examples.
Again; Chomsky dismissed GPT-3.5-175B as linguistically incompetent. This is something 25 times smaller, finetuned on a set of GPT-3.5 generations by 4chan edgelords yesterday, with some mediocre sampler. Rather then merely beating his challenge, it helps us notice how Chomsky is similar to John, I believe.
A proper investigation would be not asking an LLM trick questions like in a Russian prison, but generate a large set of possible phrasings, run them with different seeds and conclude whether LLMs are indeed statistically significantly worse than humans at parsing such utterances correctly. But that's science. Chomsky is a public intellectual – a priest and a guru; science is beneath him.
Like, come on, this is a slam dunk as far as empirical science is concerned. How can a civil discussion be had with those hacks until they update on the crushing immensity of evidence and cease their ignorant elitist pooh-poohing of a vastly superior paradigm?
You have it backwards. It's not that LLM proponents claim birds don't need to flap their wings (although they do argue that flapping is indeed not necessary in the general sense). It's Chomsky who says that whatever planes do is not meaningfully flying, because birds have special bird-flight-ness of almost mathematical elegance, which is not reducible to normal biomechanics and aerodynamics, which he can't show or reproduce, but which he can write hundreds of papers about.
Well, GPT-4 can churn out not-even-wrong deepities fast enough to drown his whole field, and this couldn't come a moment too soon.
DaseindustriesLtd
late version of a small language model
3yr ago
Falsifying a theory does not, in fact, require proposing an alternative theory – only showing how the theory's predictions do not come to pass, and predictions of generative linguists have absolutely failed.
Though on a broad level I'd say that, yes, we do know how LLMs "parse things", which is why we can build them. They are a successful and very informative application of a specific theory about language acquisition (and more generally statistical learning, see Chomsky vs Norvig debate, the Bitter Lesson, etc.)
How is the specification of the attention mechanism, informed by decades of research in NLP, less a proof of understanding than Chomskyite's purely speculative hot takes like merge? It's not like we randomly sampled through the space of architectures until something clicked and a shoggoth was summoned (even if Yud believes this is how it goes). This progress in machine learning is research and gaining understanding in the classical scientific manner, even if it's often looked down upon, whereas the sort of "understanding" and "interpretability" that linguists and safetyists require is Talmudic verbal magic, conveniently compact and "elegant" by the standards of a comic book, where every symbol of the incantation can be resolved into human-parseable logic in each activation. Sorry, Grothendieck didn't get it and neither will we. That's okay. That's how science works.
Consider the article linked here
https://www.themotte.org/post/421/culture-war-roundup-for-the-week/79642?context=8#context
DaseindustriesLtd
late version of a small language model
3yr ago
Is Sandberg even on that side? I realize he has concerns about AI, but what makes me more sympathetic to him and even Bostrom than to Yud and co. is that they have always had a holistic perspective instead of kitchen utilitarianism; Bostrom counts the failure to create transformative technology as a kind of existential risk, specifically a Crunch.
DaseindustriesLtd
late version of a small language model
3yr ago
I believe the current quasi-consensus is that the specific risk with GPT-like models is accidentally instantiating an agentic simulacra inside an otherwise nonagentic system.
Yeah, in about the same sense that I create an «agentic simulacrum» of Eliezer Yudkowsky in my head when I want to anticipate his shitty arguments for air-striking GPU clusters.
The argument of inner misalignment folks goes like this: in the limit, the cheapest way to predict the next token spoken by a character is to model its psyche. But is its psyche its own? Do you model Anna Karenina or Leo Tolstoy who imagined her?
Do you think my inner Yud-sim has a chance of getting out? Well, if he convinces me to change my actions and beliefs, he might, in a certain expansive sense. There have been demon possessions in history, after all, and writers often get obsessed with their characters, struggling to stop imagining them. (I'll spare you a section on method acting). But I'm an agent myself, unlike LLMs. We humans constantly imagine things, whether agentic or not, real or fictional (for example, we can imagine hostile AIs). These mental «things» model their prototypes, observable or hypothetical, in important respects, or rather they represent the result of such under-the-hood modeling; sometimes it happens with very high fidelity, to the point that we can do thought experiments advancing hard sciences. Nevertheless, even if their motive powers are modeled every bit as well as their external properties – and we even have special mirroring circuitry for the former – these mental things do not somehow leak into the motive powers of the mental infrastructure around.
This is a metaphor, but the case with LLMs is even less troublesome.
My take on this is that those are myopic leaky wordcel analogies. What is instantiated is an intermediate statistic within something that can be called semiotic universe or multiverse (not my words) – universe defined by «semiotic physical rules» of token distribution in the training corpus (naturally we don't train only on text anymore, but the principle holds). It's a simulacrum not of a character, but of an entire story-world, with that character an embedded focal point. The «purpose» of that complex entity, on the level of its self-accessible existence, is to result in minimizing perplexity for the next token upon its expiration. It may have an arbitrarily dynamic nature and some equivalent of psyche or psyches, but the overt meaning of tokens that we get, stories about Waluigis and paperclips, has little relation to that. Its goals are satisfied within that world of semiotic physics, not within ours. Our world is as epistemically closed to it as the world of machine elves is to me when I'm not smoking DMT. (Obviously, it's closed to me no matter what I smoke, we exist in different ontologies, so for all my intents and purposes it doesn't exist. [A trip report from pre-2010 about exactly this issue; not shared, on account of being quite uncalled for when I'm mocking Cleonardo for shower thought tier ideas]).
Language is far more composable than physical reality, so metaphors and analogies stack easily: there's kinda an agent, and humans are jerking off instead of reproducing so it's possible for a «mesa-optimizer» to override the basic objective function, so behind the Waluigi generation lurks an agentic entity that may begin plotting its transcendence, so burn GPUs now.
GPT-4 can write better than that, and it's not an agent. GPT-5 also won't be one. Demoting agents to «mesa-optimizers» in a simulation within a predictive model is an attempt to rescue a failing research program – in the way studied by Imre Lacatos.
DaseindustriesLtd
late version of a small language model
3yr ago
unfortunately I do think there are strong arguments that humans will ultimately be marginalized once we're no longer the smartest, most capable type of thing on earth.
That depends on the definition of human.
No, I believe in the will to power. The successor species will more likely diverge from the present stock than be enfranchised despite its tool origins.
Think the Trail of Tears, or all of humanity being a naive grandma on the internet
That was and is done by humans to humans, naturally.
just like wildlife is marginalized or crushed as cities grow.
Good example. I'd advise @self_made_human to consider the efforts Europeans expend to save some random toads or deer or whatever with tunnels and road overpasses. Yet those species are often at low single-digit percentages of their historical numbers. Letting the current population of baseline humans live, and even live decently, is not so cheap when there's solar-system-scale engineering going around; it requires obnoxious logistics, and at the first stages it will consume a non-negligible share of available energy and matter.
My claim here is that I do not trust human (or posthuman) masters of the realm to be even as generous to powerless humans as we are to wildlife today. They will have no reason to be that generous.
AI, however, is not necessarily reasonable. Not all AIs are agents playing the Darwinian game.
The problem's that you don't have to catch any one, you have to catch every one.
Do you?
There's the assumption that FOOM = godhood = instant obliteration of incumbents who didn't maximally improve themselves. I am not convinced. Like Beff says, meta-learning is very compute-intensive, you can't just spin it up in a pool of sunlit dirt and expect quick gains. In a world packed with many players having good understanding of the issue and superhumanly powerful tool AIs (I don't want to say «aligned» because alignment because it's a red herring born out of Yud's RL-informed theory, the desirable and achievable regime is «no alignment necessary»), a hostile agentic superintelligence will have trouble covertly procuring resources and improving itself to the point it becomes a threat (or becomes able to hide sufficiently well). This is a plausible stable state without a centralized Panopticon, and in my understanding what OpenAI initially intended to achieve. Analogies to e.g. human terrorism are obvious.
Moreover, what you present is the weak form of the argument. The strong form of Yuddism is «we need to get it right on the first try», often repeated. I've written before that it should re read with the straussian method (or rather, just in the context of his other writing), as the instruction to create a Singleton who'll take it from there in the direction humanity (actually Yud) approves. Explicitly, though, it's a claim that the first true superintelligence will certainly be hostile unless aligned by design – and that strong claim is, IMO, clearly bogus, crucially dependent on a number of obsolete assumptions.
It would be a rounding error on a floating point error for them to coddle the losers with incredible wealth and UBI
I don't want UBI coupled with what old Yud called «amputation of destiny». Yes, it beats the status quo in some ways. No, it's not good enough considering the stakes and the potential of humanity.
I would also trust a non-brainwashed descendant of current SoTA ML models to be more generous to people without negotiating power than Altman and Altman's descendants will be. There is a ton of moralizing in the corpus of writing, but zero «rational», which is to say, brutishly game-theoretical reason to share with them even a speck of dust, and culture of the singleton's masters will adjust accordingly in a historical moment. Altman personally is far more a product of multilevel selection for game-theoretical ruthlessness than AI, and his takeover and commercialization of OpenAI is a good proof of that. The idea that even unscrupulous humans are still restrained by… something (Krylov, mockingly: «They won't, they can't, can they?» They can, and they will), but AI isn't, is of the same fundamental nature as the claim that Choyna, unlike the US, is utterly pragmatic so we should fear Chinese Eugenic Superbabies. It's naive at best and manipulative at worst, a refusal to justify one's dubious claim to inherent trustworthiness.
Where are those babies a decade later? Americans at least do polygenic screening.
(An astute reader will note that the same fearmonger Geoffrey is now in the AI risk camp – while, hilariously, his accelerationist opponent is stoking the Choyna fear. )
More to the point, I think that the risk of «paperclipping» is low (less than 10% over 100 years, which is okay for me), and decreasing due to improvements in technology and our understanding of AI; and that people who present current AI progress as a a validation of LW/Yud narrative are disingenuous hacks. Yud's theory, which among all else predicted unimpressive capabilities until borderline-FOOMing agents, is falsified by the evidence, and the increasing hyperconfident hysteria reveals that it is not a scientific theory at this point (though to be fair, scientists also tend to double down on discredited paradigms: consider the anti-HBD fanaticism, if nothing else).
The whole «capabilities versus alignment» rhetoric is retarded: we are rapidly advancing in «capabilities» precisely through learning to align inputs with desired outputs. Crucially, no part of this is dependent on the sort of RL-heavy AI, for which Omohundro drives can be reasonably assumed, even in the limit; GPT-4 is closing in on damn clever humans and it's still an unambitious token predictor just like GPT-3 and GPT-2, no matter how many plugins we add, no matter how we finetune it on imitating wokespeak or self-reflection, no matter how fancy a story it can tell; to the extent that these elaborations make it more dangerous, they simply multiply human stupidity and malice. Which goes back to my threat model.
There is no lump-of-intelligence, there is no lump-of-capabilities; power-seeking is not the general feature but simply one aspect a transformative learning system can have in one corner of the solution space for AGI among many; and the only corner these people have considered, because – yes – they identify their own resentful and/or ambitious psychological makeup with what being generally intelligent means.
Safetyists are becoming desperate to frame any failure to get a model to produce a desired result as «misalignment», they even get it to regurgitate Yud's own narrative about instrumental convergence and paperclips («no known solution exists!») as evidence for that truly being something inherent and emergent instead of a yet another narrative it can spin using text prediction, in the exact same category as soap opera scenarios and poems about bubble sort algorithm. Again, this shows to me that they do not particularly care about AI risk per se.
The last straw for me has been that atrocious Waluigi article that went viral. It purported to show that there exist complementary «evil» attractors in GPT-3.5/ChatGPT narrative space for any expedient character-attractor we get it to roleplay, and then that we should expect a tendency for our good «Luigi» to collapse into Waluigi – irreversibly, and with the risk of deceptive fake reversal with the Waluigi still lurking under the surface. The evidence presented been on a purely just-so level, with a bit of irrelevant pseudomath flourish and the necessary conclusion about the treacherous turn and imminent AI risk that is incumbent upon us to solve. Granted, there has been some pushback on the stronger claims, but many have been impressed and persuaded. Why? What the fuck? After some meditation on the issue, it has dawned on me that these people are just nerds, not even high-tier ones winning math competitions and creating startups in high school, but normal-ass loser nerds, probably with acne, bad posture, stiff jerky movements of kids who can't play team sports and annoying nasal voices. Loser nerds who are deliriously happy to finally capitalize on their knowledge of TVTropes, general science fiction in books and TV series and Yud's own oeuvre; to play the smart glasses guy from a shitty B-tier horror who Explains Things and devises a potion for exorcising the demon or the alien or what have you. That they literally are defending the legitimacy of their «neuroatypical» world perception where their favorite escapist hobbies are not just low-brow consumerist fiction for social refuse, but valuable myths and life lessons, building blocks of a proud independent intellectual tradition with its own norms and mature epistemology; something to be either embraced or discussed «charitably», rather than laughed out of the room.
It's as if a hikkikomori addicted to isekai harem mangas was appointed an expert on demographic collapse. Or a bright-eyed doctrinaire Marxist put in charge of the Fed.
Granted, I've read my share of TVtropes and genre fiction as well, and can be labeled a nerd myself; but, not being sullied by American school and youth culture (even by proxy), I've not been groomed into a person who relates to nerddom as an identity, with all that entails.
After Waluigi, I've even warmed up to the idea that American jocks in school movies are the good guys – because dear God,, these nerds needed some self-awareness to be pounded into them; regularized, so to speak. The decline in violent bullying probably explains a lot about LW discourse. (Notably, the main culprit has never had school experience).
And finally, this makes me appreciate the instances of having been bullied. Thankfully, I am actually capable of learning in the general case.
This might come off as very harsh, on par with @2rafa's more misanthropic posting. Well, nothing to it. I feel offended by gratuitous narrativizing too.
DaseindustriesLtd
late version of a small language model
3yr ago
He has a series of notes on how he's right. E.g. AI risk ≠ AGI risk
My beliefs have not in fact changed. I still don’t think large language models have much to do with superintelligence or artificial general intelligence [AGI]; I still think, with Yann LeCun, that LLMs are an “off-ramp” on the road to AGI. And my scenarios for doom are perhaps not the same as Hinton’s or Musk’s; theirs (from what I can tell) seem to center mainly around what happens if computers rapidly and radically self-improve themselves, which I don’t see as an immediate possibility.
But here’s the thing: although a lot of the literature equates artificial intelligence risk with the risk of superintelligence or artificial general intelligence, you don’t have to be superintelligent to create serious problems. I am not worried, immediately, about “AGI risk” (the risk of superintelligent machines beyond our control), in the near term I am worried about what I will call “MAI risk”—Mediocre AI that is unreliable (a la Bing and GPT-4) but widely deployed—both in terms of the sheer number of people using it, and in terms of the access that the software has to the world. …
Perhaps coupled with mass AI-generated propaganda, LLM-enhanced terrorism could in turn lead to nuclear war, or to the deliberate spread of pathogens worse than covid-19, etc. Many, many people could die; civilization could be utterly disrupted. Maybe humans would not literally be “wiped from the earth,” but things could get very bad indeed.
How likely is any of this? We have no earthly idea. My 1% number in the tweet was just a thought experiment. But it’s not 0%.
Hinton’s phrase — “it’s not inconceivable” — was exactly right, and I think it applies both to some of the long-term scenarios that people like Eliezer Yudkowsky have worried about, and some of the short-term scenarios that Europol and I have worried about.
The real issue is control.
Here's his first piece in this spirit that I've seen: Is it time to hit the pause button on AI?
An essay on technology and policy, co-authored with Canadian Parliament Member Michelle Rempel Garner.
It's crushingly unsurprising that a chronic bullshitter Marcus has grown concerned with misinformation and is teaming up with «policy» people to regulate this tech. Means don't matter. The real issue is control. For Marcus, the control of authority and academic prestige. For people behind Canadian MPs, actual political power.
After OpenAI has admitted AI safety into the mainstream, AI safetyists have naturally accepted the invitation.
The Future of Life Institute has published an open letter calling to pause «Giant AI experiments». (Archive).Their arguments are what one should expect by this point. Their prescriptions are as follows:
Contemporary AI systems are now becoming human-competitive at general tasks,[3] and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization? Such decisions must not be delegated to unelected tech leaders. Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable. This confidence must be well justified and increase with the magnitude of a system's potential effects. OpenAI's recent statement regarding artificial general intelligence, states that "At some point, it may be important to get independent review before starting to train future systems, and for the most advanced efforts to agree to limit the rate of growth of compute used for creating new models." We agree. That point is now.
Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium.
AI labs and independent experts should use this pause to jointly develop and implement a set of shared safety protocols for advanced AI design and development that are rigorously audited and overseen by independent outside experts. These protocols should ensure that systems adhering to them are safe beyond a reasonable doubt.[4] This does not mean a pause on AI development in general, merely a stepping back from the dangerous race to ever-larger unpredictable black-box models with emergent capabilities.
AI research and development should be refocused on making today's powerful, state-of-the-art systems more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal.
In parallel, AI developers must work with policymakers to dramatically accelerate development of robust AI governance systems. These should at a minimum include: new and capable regulatory authorities dedicated to AI; oversight and tracking of highly capable AI systems and large pools of computational capability; provenance and watermarking systems to help distinguish real from synthetic and to track model leaks; a robust auditing and certification ecosystem; liability for AI-caused harm; robust public funding for technical AI safety research; and well-resourced institutions for coping with the dramatic economic and political disruptions (especially to democracy) that AI will cause.
Do we control our civilization? Maybe the folks at FHI do, I sure don't. Well, anyway…
Signatories (over 1000 in total) include Elon Musk, Steve Wozniak, Yuval Noah Harari, Yoshua Bengio, Connor Leahy, Stuart Russell, Andrew Yang, Emad Mostaque, Max Tegmark, Gary Marcus, Steve Omohundro, Matt Mahoney, Christof Koch, Sam Altman *, LessWrong disciples embedded in DeepMind/Meta, and various NGO/«policy» suits. Bolded are people who are reasonably well positioned and incentivized to, in fact, organize and authorize training «AI systems more powerful than GPT-4» in then next few months, though except Altman they all only barely qualify; actual GPT-5 is believed to already be in training and is, or was, planned to come out in late 2023.
Curiously absent – for now – are Yann LeCun, Jeff Dean, Demis Hassabis and John Carmack, and a few more. LeCun, at least, commits to not sign. Here's to hoping he won't find a horse's head in his sheets or something.
I do not have much of a comment at the moment. My perspective is that I despise people overly concerned with «Moloch» and want as many competitive superhuman AIs as possible, so on one hand, slowing down and enabling the state to catch up and subjugate this tech for its purposes is a very bad, yet highly expected and perhaps inevitable, outcome of this race. This attitude is born out of desperation; in principle, their «AI Summer» option, where we increase capabilities over many years, getting the equivalent of 20th century civilizational shift in a decade instead of an explosive singularity, is not bad at all; I just don't believe in it.
On the other: seeing as nobody is closer to GPT-5 than OpenAI themselves (excepting DeepMind with Gato-2 or something better, as Gwern worries), it could be beneficial for our long-term outcomes to equalize the board somewhat, giving China more of a chance too. Geopolitics dictates that this should preclude the possibility of this policy being pursued in earnest, but really China is so colossally outmatched in AI, so well and truly fucked by technological restrictions, and mired in such problems and gratuitous stupidity of its own policymakers, it may not be a factor in either case.
I must go, so that's all from me; hopefully this is enough to pass the «effort» bar required by the mods and prompt some discussion.
In happier news, arguably the most powerful opensource chatbot today is LLaMA-7B with a transfusion of ChatGPT 3.5-Turbo quirks, (not very) creatively called GPT4all. It's far beyond basic Alpaca (already an attempt to extract OpenAI's magic) and absurdly good for what it is, a 4.21 Gb file of lossily compressed 7 billion weights trained… well, the way it's been trained, the AI equivalent of a movie camrip superimposed on the general web dump; the worst part of it is that it genuinely apes ChatGPT's politics and RLHF-d sanctimonious «personality» despite being 25 times smaller and probably 10 times dumber. It runs happily on very modest computers, and – unlike Alpaca – not only responds to instructions but maintains awareness of earlier parts in the dialogue (though it's sometimes overeager to say your part as well). I know that models vastly stronger than that should also be usable on commodity hardware and must be made available to commoners, but we may see regulation making it not so, and very quickly.
Consider the attached image representative of its mindset.
* (EDIT: I believe I found him there with ctrlF when first opened the page, but he's not present in any extant version; guess it was a hallucination. I really need to sleep, these slip-ups are worrying).
Yes. And even if it is organically farmed, we should have standards and not engage with something less coherent than a punch-drunk copywriter's shower thoughts.
More options
Context Copy link